On 1st July, a team of Resolver developers made up of both Accord and Consumer members joined forces to compete in the AWS GameDay. The premise of the day is that it is our induction at a brand new start-up ‘Unicorn Rentals’, a fictitious company who dominate the Legendary Animal Rental Market (LARM) by providing temporary use of a unicorn to ever-demanding children.
The developers at Unicorn Rentals have recently moved on to other ventures, leaving a lot of legacy services behind them. It was our job to try to move away from these to ensure faster technology and therefore more unicorns! The strategy throughout the day to earn as many points as possible (and therefore beat the other teams) was to keep our router running, to launch microservices so that other teams can use them and route to as many other teams microservices (especially fast ones!) as possible. Points were lost at any moment for sending messages to our own microservices and for our router not working.
Unfortunately, it was not as simple as getting the above set up and then watching the points roll in! Seemingly every time we appeared to have a steady stream of points being awarded to us, our focus would turn to enabling alarms and bettering infrastructure, yet before we could finish doing so, the points would stop and we needed to turn our attention to finding out what was wrong and how to fix it.
At times it was challenging to remember how everything on AWS works and to then go on to identify what the issue was. Throughout the day, we faced trouble relating to sudden spikes in traffic that needed scaling, containers crashing and bad app deployments! We all gained a sense of how important auto scaling instances are so that services can be maintained even if containers crash. Also spending time to debug through useful logging techniques combined with having good health check notification tools in place were all takeaways from the day.
At the end of the day, it came to light that the organisers of the event had been fiddling with teams settings in order to simulate problem solving in the real world. Luckily, problems do not occur this often or fast at Resolver, yet it was an educational day for all as we were able to find solutions inside a gamified and risk-free environment. We mainly worked together as a team to find solutions, splitting off into pairs or individually where needed to focus on fixing isolated problems. This approach was useful as it lead to a shared understanding yet unfortunately it did not lead us to victory. The teams that came in first and second place shared after the event that they worked largely separated throughout the day – dividing jobs between team members.
Yet, in some respects, victory is ours as it was a great opportunity to work with colleagues who are not usually in our team and to build on various levels of AWS knowledge. Also, we did win a prize for best comeback after a somewhat slow but steady start, so we were all very pleased with that. Hopefully, the unicorn’s put a smile on some virtual children’s faces, too!
