Technical on Resolver Engineering Blog

Hack week 2023: OpenAI for dispute resolution

Fri, 31 Mar 2023 13:45:35 +0100

This post was partially written by and edited using ChatGPT.

Last month the Resolver development team had their first official hack week, which was dedicated to checking out the potential to improve our software with the help of OpenAI.

What is OpenAI?

You’ve probably heard of OpenAI since their ChatGPT program has made headlines frequently in recent months.

According to my assistant editor:

OpenAI is an artificial intelligence research laboratory consisting of a team of experts and researchers in the field of machine learning, natural language processing, robotics, and other related areas. The organization aims to develop advanced artificial intelligence in a safe and beneficial manner to benefit humanity as a whole. They conduct cutting-edge research and also develop various AI-based products and tools such as language models, chatbots, and robotics applications.

ChatGPT

Talking robots! Let's talk about APIs

Fri, 11 Mar 2022 14:45:17 +0000

What is an API?

API stands for Application Programming Interface, let’s break that down:

Interface - Every device we use has some kind of interface. Your TV has a remote with buttons on it, while a light switch has an even more straightforward interface- on or off. We use these interfaces to get the device to do what we want.

Application Programming - The Application Programming bit is easier to understand. An API is how applications talk to each other. All software that you can interact with through code has some form of an API.

AWS S3 UTF-8 Content Disposition

Fri, 18 Feb 2022 16:09:29 +0000

Have you ever tried to make the download filename of an AWS S3 object be in Korean? No…? Being a developer who, until recently, had only written in English, I hadn’t either. However, it turns out using some UTF-8 characters in S3 object request parameters can lead to some interesting errors.

How to Change the Filename of a Download

When downloading a file directly from the browser using a HTML anchor tag, the download=<filename> attribute can be used. However, the attribute will only work for same-origin URLs and the blob: and data: schemes. For example:

Rolling back AWS Elastic Container Service (ECS) Deployments

Wed, 13 Oct 2021 13:07:43 +0100

What is ECS?

AWS Elastic Container Service (ECS) is a fast and popular way to orchestrate containerised applications in AWS’s cloud computing platform. ECS comes with autoscaling baked in and natively runs containers on AWS’s Fargate serverless compute engine. What makes ECS particularly nice to use is that it abstracts away much of the operations management that comes along with rolling your own deployment, either on-site or on cloud-based computing, such as AWS EC2.

Upgrading to Rails 6.0

Wed, 29 Sep 2021 10:00:02 +0100

Introduction

Whenever a Ruby on Rails developer hears the phrase “New Rails update!” they will no doubt experience a mixture of excitement and fear. It’s always very exciting to see the new tools that come with a new Rails version and all the new possibilities that go with it. But at the same time, you know that you are bound to have some hiccups with some legacy code or some old gem that you use on your application. Nevertheless, I decided it was time to upgrade our Rails app.

Creating a Gem in Ruby: a Jewelcrafting guide for Developers

Wed, 25 Aug 2021 13:05:38 +0100

What is a Gem?

Gems are small chunks of code used to perform specific duties without interfering with, or becoming a direct part of our code. The code required for a gem will still be a part of our project’s ecosystem, but we will ultimately remove some overhead, and avoid unnecessary duplication by using a gem instead of coding it ourselves.

Why create a Gem?

Gems are a helpful byproduct created when you either have a piece of self contained code, or else when a developer realizes the code might benefit others. If we want to use that code elsewhere or want to share it, that’s when it’s time to make a gem.

Using Awesomplete for auto-complete inputs

Tue, 10 Aug 2021 17:43:04 +0100

Awesomplete is a front-end autocomplete tool that compares a user’s input to a defined data list, and if it recognises the input will make suggestions for the user to choose from.

We chose to use the Awesomplete tool in the Accord team while building the journey to enable users to create and submit their dispute. Awesomplete provided a neat solution to our problem of how to let the user choose the organisation they are raising a dispute against. The simplest solution would have been a dropdown list of all available organisations. However, as we will be working with ADR providers representing potentially hundreds or thousands of organisations, a dropdown list would not have been a feasible solution. Enter Awesomplete!

Setting up VPC peering

Mon, 29 Mar 2021 08:00:00 +0100

VPCs are a way of partitioning infrastructure in AWS to isolate them from communication with other infrastructure. They operate at the network level similar to VLANs and block all external communication by default.

However, sometimes it is necessary to allow some inter-VPC communication. For example, if you have multiple AWS accounts that need to access some shared service, that will require infrastructure in one VPC to access infrastructure in another VPC.

Cross-region deployments with CodePipeline

Mon, 15 Mar 2021 08:00:00 +0000

Last time, we looked at building a deployment pipeline using CodePipeline.

Given the increasing number of countries that are implementing data residency policies, we now need to be able to deploy a copy of our application in a different region along with its database and connected services at the same time as in the main region (London eu-west-2 in this case).

We’ll look at how we can amend our earlier CodePipeline deployment pipeline to deploy the same application image to ECS in two different regions simultaneously.

Triggering AWS ECS deployments via GitHub, CodePipeline and ECR

Mon, 01 Mar 2021 08:00:00 +0000

Deployments are a key part of running a web application, but most of the time, they are an afterthought when it comes to process and developer ease-of-use.

However, using a number of AWS services, deployments can be made much easier, with most of the process automated.

On the Accord ODR project, we use ECS to host our application using Docker containers. Here, I’ll show how GitHub, CodePipeline and ECR (Elastic Container Registry) can be used to create an easy-to-use process that mostly just happens in the background and is very light touch.

Adding cron jobs with error recovery to AWS using Lambda, Step Functions and EventBridge

Mon, 15 Feb 2021 08:00:00 +0000

Scheduled jobs (or Cron jobs) are a standard part of web development. Whether you want to update comment counts every night or check for updates once a week, a scheduled job allows you to “set and forget”.

There are a number of methods to implement scheduled jobs. Some rudimentary web apps check the schedule each time someone visits a page. This works as long as there are regular visitors, otherwise the schedule may slip during quiet periods. Better methods include either using the underlying server’s cron implementation (if possible) or possibly using a background worker like Sidekiq that also has scheduling capabilities.

Allowing CloudFront to access load balancers through Security Groups

Mon, 01 Feb 2021 08:00:00 +0000

Security Groups are a best practice feature of VPCs in AWS that act similar to a firewall. They allow access to various resources such as EC2 instances, load balancers or RDS databases to be controlled to other resources or a set of IP addresses.

For example, you may set up an EC2 instance to only be accessible by a load balancer. This means uses cannot bypass the load balancer to access your instance directly. You might set this up as follows:

Sending emails using Mandrill/Mailchimp with Ruby on Rails

Mon, 18 Jan 2021 08:00:00 +0000

I recently started working on a ticket where I was asked to add a new mail template and the logic behind it. I have not really worked with either Mandrill or Mailchimp much before, so I started doing research into how to do this. Every blog post and Stack Overflow post that I could find is aimed at doing this for the first time, i.e. integrating a Ruby on Rails app to use Mandrill/Mailchimp. After looking at documentation and reading through our codebase I managed to work it out in the end, but I hope that this post will explain to any other developer how to do this off the bat.

Setting up Athena to analyse CloudFront access logs

Mon, 14 Dec 2020 08:00:00 +0000

One of the best ways of working out who is how many people are visiting your site, see patterns of traffic and analyse them over time is to look at your access logs.

Most web servers collect some form of access logs, mostly in a standard format such as the Common Log Format. As a CDN, CloudFront can also collect access logs when configured to do so, storing them in an S3 bucket ready for analysis.

Virus scanning files in S3 and integrating with Rails

Mon, 16 Nov 2020 08:00:00 +0000

If you have an app that accepts file uploads, then either you’re aware of the potential risks, or you should be.

With Accord ODR, we started by limiting the types of files that can be uploaded by users. This is the easy part: disallow executable files or anything else that can be scripted. This includes not only obvious candidates like JavaScript files but also things like WMF images, since these can contain executable code.

Using VCR in tests

Mon, 02 Nov 2020 08:00:00 +0000

VCR is a gem that is super helpful if your Ruby application is using an external API. When testing, if an actual API is called, this will result in slow test performance. Although a single test may not seem that slow when run individually, the impact of using many API calls in tests will significantly slow down the time it takes to run tests.

This is why VCR is so handy! What it allows you to do is to record a ‘cassette’ of your API request and will save the results and then use the cassette in testing from then on. This means that once set up, the first time that the test is run, VCR uses the actual API and records the actual response in a (pretend!) ‘cassette’, this saved response is then used in place of using the actual API each time the test is run. The benefits of this is that now your tests will run faster, but also that the tests will not be dependent on the external service working.

CI using GitHub and AWS CodeBuild

Mon, 19 Oct 2020 08:00:28 +0100

Last time, we mentioned CI as one of things that we try to automate as much as possible. This time, we’ll go through more details about our CI setup and how it relates to our deployment pipeline.

Setting up the CodeBuild project

As well as being used as part of a CodePipeline, CodeBuild can also be used on its own as a script-runner of sorts. This makes it ideal as somewhere to run automated tests, and this is exactly what we do.

Sending Slack alerts to approve CodePipeline deployments

Mon, 28 Sep 2020 08:00:00 +0100

At Resolver, we aim to automate as much of the boring technical jobs as possible, and that includes the whole continuous integration (CI) and deployment pipeline. We trigger these from GitHub, so a mere push or merge can kick-off our automated test suite or deploy to our staging environment for more automated and manual testing.

Our deployment pipeline also happens to deploy the same artefacts that have been deployed to our staging environment to our production environment. The end goal is to be able to do this automatically, but for that to be viable, we need to have 100% trust in our testing strategy, and as with most organisations, we’re not there yet. Therefore, for the time being, we want to be able to pause the pipeline after the staging deployment, and manually approve it to continue to production.

Running database migrations on deployment for Fargate containers

Mon, 14 Sep 2020 08:00:00 +0100

One of the tenets of modern framework-based web development is the management of your database state in code. Rather like infrastructure-as-code, it means there is an immutable history of the state of the application database at any point in time, which can be re-applied or rolled back as necessary. In addition, the desired state of the database and the application are tied together at any point in time to reduce the chances of divergence.

Finally forcing computers to think like humans

Mon, 31 Aug 2020 08:00:00 +0100

On 17th June 2020 I attended my first AWS summit. New to the industry, it was one of the first tech conferences I’d been to (other than the Women in Development conference earlier this year, back in that beautiful time when people were allowed to meet in groups…) and was certainly the first virtual online conference I’ve been to. I didn’t quite know what to expect, but it ended up being an intense, engaging half-day where I learnt a LOT.

Implementing HTTP Basic Auth for Fargate using Lambda@Edge and CloudFront

Mon, 17 Aug 2020 08:00:00 +0100

Ask any developer the quickest and hackiest way of locking unauthorised users out of your website and they will probably mention HTTP Basic Auth. As an extremely simple method of authentication that is built-in to the HTTP protocol standard and supported by every web browser, it’s the universal, go-to method of setting up a username and password authentication prompt with minimal work.

Simplicity and minimalism of course bring with them compromise, and most things you may associate with authentication systems cannot be achieved easily or at all with HTTP Basic Auth, so you would probably not consider it your primary authentication method for anything but the most basic of needs. But for our needs, it’s the perfect method of stopping casual snooping and accidental usage of a non-live system.

If you only learn 5 Docker Compose commands, it should be these

Mon, 03 Aug 2020 08:00:04 +0100

In my previous blog post, I gave a brief introduction to containerisation, and the differences between Docker and Docker Compose. Here I’m going to share the Docker Compose commands that I use on a daily basis and have found the most essential.

In my opinion, if you only learn 5 Docker Compose commands, they should be:

docker-compose up

Run this command to launch your project.

The up command builds or creates all the services in the docker-compose file and starts a container for each of them.

A beginner’s guide to Docker and containerisation

Mon, 20 Jul 2020 08:00:00 +0100

I joined Resolver in February 2020 and was put into the Accord team, working on a brand new project (they had literally typed the words ‘rails new’ the week before I joined). This meant I would be involved from the ground up, that I would see the app grow and develop from the very start, and that I wouldn’t need to get to grapple with years of legacy code.

As we had decided to build the app with Docker, which I haven’t used before, it also meant I was about to get to grips with it very quickly! This whale of a task initially felt quite daunting, but thanks to the gloriously comprehensive Docker documentation, Linux Academy courses about Orchestration, Containers and Docker, Rob Isenberg’s invaluable Docker for Rails Developers book, and, of course, a lot of Googling, I’m now happy to share my understanding of containers and Docker in my next few blog posts. I hope they can help make the topic a little less confusing!

Granting time-limited access to assets in S3 using CloudFront

Mon, 06 Jul 2020 08:00:00 +0100

There was a time when asset storage was one of the problems facing web app developers - maybe you stored assets on the web server (easy if you only have one, more difficult if you need to sync them up between multiple servers), or maybe you had a shared disk mounted using NFS. In any case, the easier part was granting access to these assets to the right users. Given that assets are normally stored outside the web server’s root directory, it’s easy to write a script that checks user access then serves the asset.

How to redirect an apex domain to www using CloudFront and S3

Mon, 08 Jun 2020 08:00:00 +0100

A common issue for developers and operations staff when it comes to domain names is whether to choose the apex domain (e.g. example.com) or the www subdomain (e.g. www.example.com) as their primary domain, and then how to redirect one to the other.

Choosing to use www over the apex

At Resolver, we use AWS for the majority of our infrastructure, and we generally have staging and production environments for each product or service we build. We use subdomains as namespaces to separate the environments whilst providing similar URLs for each.