Posts

Deployments are a key part of running a web application, but most of the time, they are an afterthought when it comes to process and developer ease-of-use. However, using a number of AWS services, deployments can be made much easier, with most of the process automated. On the Accord ODR project, we use ECS to host our application using Docker containers. Here, I’ll show how GitHub, CodePipeline and ECR (Elastic Container Registry) can be used to create an easy-to-use process that mostly just happens in the background and is very light touch. ...

Scheduled jobs (or Cron jobs) are a standard part of web development. Whether you want to update comment counts every night or check for updates once a week, a scheduled job allows you to “set and forget”. There are a number of methods to implement scheduled jobs. Some rudimentary web apps check the schedule each time someone visits a page. This works as long as there are regular visitors, otherwise the schedule may slip during quiet periods. Better methods include either using the underlying server’s cron implementation (if possible) or possibly using a background worker like Sidekiq that also has scheduling capabilities. ...

Security Groups are a best practice feature of VPCs in AWS that act similar to a firewall. They allow access to various resources such as EC2 instances, load balancers or RDS databases to be controlled to other resources or a set of IP addresses. For example, you may set up an EC2 instance to only be accessible by a load balancer. This means uses cannot bypass the load balancer to access your instance directly. You might set this up as follows: ...

I recently started working on a ticket where I was asked to add a new mail template and the logic behind it. I have not really worked with either Mandrill or Mailchimp much before, so I started doing research into how to do this. Every blog post and Stack Overflow post that I could find is aimed at doing this for the first time, i.e. integrating a Ruby on Rails app to use Mandrill/Mailchimp. After looking at documentation and reading through our codebase I managed to work it out in the end, but I hope that this post will explain to any other developer how to do this off the bat. ...

One of the best ways of working out who is how many people are visiting your site, see patterns of traffic and analyse them over time is to look at your access logs. Most web servers collect some form of access logs, mostly in a standard format such as the Common Log Format. As a CDN, CloudFront can also collect access logs when configured to do so, storing them in an S3 bucket ready for analysis. ...

Remember remember, the 5th November…the Resolver developers took part in another AWS GameDay! Our team of 4 had a wide range of engineering, AWS, and most importantly, GameDay experience. Given that the Resolver team were prize-winners at the last GameDay in July, the pressure was on to keep up our reputation. Like the last time, the premise of the day was that our team works for a fictitious start-up, ‘Unicorn Rentals’. However this time, the theme and focus of the day was on security. Each team would have to complete various tasks to win points, and the teams with the most points win prizes. There were noticeably more teams taking part this time - from just 6 in July to now 33! It was going to be tough to keep our prize-winning reputation. ...

If you have an app that accepts file uploads, then either you’re aware of the potential risks, or you should be. With Accord ODR, we started by limiting the types of files that can be uploaded by users. This is the easy part: disallow executable files or anything else that can be scripted. This includes not only obvious candidates like JavaScript files but also things like WMF images, since these can contain executable code. ...

VCR is a gem that is super helpful if your Ruby application is using an external API. When testing, if an actual API is called, this will result in slow test performance. Although a single test may not seem that slow when run individually, the impact of using many API calls in tests will significantly slow down the time it takes to run tests. This is why VCR is so handy! What it allows you to do is to record a ‘cassette’ of your API request and will save the results and then use the cassette in testing from then on. This means that once set up, the first time that the test is run, VCR uses the actual API and records the actual response in a (pretend!) ‘cassette’, this saved response is then used in place of using the actual API each time the test is run. The benefits of this is that now your tests will run faster, but also that the tests will not be dependent on the external service working. ...

Last time, we mentioned CI as one of things that we try to automate as much as possible. This time, we’ll go through more details about our CI setup and how it relates to our deployment pipeline. Setting up the CodeBuild project As well as being used as part of a CodePipeline, CodeBuild can also be used on its own as a script-runner of sorts. This makes it ideal as somewhere to run automated tests, and this is exactly what we do. ...

At Resolver, we aim to automate as much of the boring technical jobs as possible, and that includes the whole continuous integration (CI) and deployment pipeline. We trigger these from GitHub, so a mere push or merge can kick-off our automated test suite or deploy to our staging environment for more automated and manual testing. Our deployment pipeline also happens to deploy the same artefacts that have been deployed to our staging environment to our production environment. The end goal is to be able to do this automatically, but for that to be viable, we need to have 100% trust in our testing strategy, and as with most organisations, we’re not there yet. Therefore, for the time being, we want to be able to pause the pipeline after the staging deployment, and manually approve it to continue to production. ...