Front-end feels: building our automatic deployment flow

Written on 
June 12, 2018

We’ve just merged and released an automatic deployment flow for our front-end, which has been in the works since January 2017!

It’s been a nail-biting ride of repeat near-launches, but now it’s finally launched we think it deserves some fanfare. SO, we’re popping the hood on exactly how it works and what tools we use.

This long read will hopefully help other developers looking to optimize their processes, as well as anyone interested in the dark underbelly of front-end development. There’s still a lot that can be improved — and we have a ton more ideas we want to test — but for now, we’re taking a nap.

For context, everything relates to the ongoing development of our product, Timely — an automatic time tracking app used across web, desktop and mobile in more than 160 countries.

A bit of background

Why are we so excited about auto-deployment?

With a new automatic flow for deploying front-end code, we can safely deploy and test every single revision of the front-end code in production before rolling it out to Timely users. This idea is nothing new — there are other companies doing similar things — but it’s been a real labor of love. We committed the first lines of code that would eventually become our “deploy everything all the time” automated system way back in January 2017. And now it’s finally here.

Why we decided to build it

In the early days of Timely, the front-end and back-end code lived side-by-side in the same Git repository/project. As new developers joined, we rewrote the entire front-end codebase using a more modern framework and, in the process, split the front-end and the back-end into separate projects. It allowed us to run the project as we saw fit using the tools we wanted and meant we wouldn’t create and back-end “noise” whenever we made changes. Ultimately, we wanted to be able to deploy the two projects separately.

…But we never actually managed to deploy our code separately from the back-end. We ended up in a flow where we would build code for production, copy the artefacts from the build job into the backend repository, and then deploy the entire backend just to get a couple of front-end changes out there. It was a sluggish process, but it worked. Yet the itch remained: we still wanted to be able to test and verify all our code — on production as well as staging — without rolling it out to everyone.

What it doesn’t solve

This automated flow only solves deployments of front-end code. The back-end still deploys the same way as it always has, which means that there’s only one active version available at any given time. I would love for us to manage to make a similar approach work for the backend, but that’s another story!

The NEW Timely development flow

Behold our glorious new front-end architecture (explained in an equally-glorious flow chart):

0 Fe7CEPpRYo8mxjjL

Step 1

During our development process, we work on different branches — usually one per feature or bug. When we’re done developing, we push the code to GitHub, where we then create a pull request. It allows the rest of the team to do code reviews and enables a more transparent way of working, but it also allows us to connect third party services to trigger certain actions whenever a pull request is created or worked on.

Step 2

The third party service we rely on the most here is a service for Continuous Integration called CircleCI. CircleCI is highly configurable, and allows you to set up how you test and build your code the way you want it to be built, and talk to other services that we use. It can be considered as the hub of this deployment flow, and represents one of the more crucial systems; no Circle, no deploys.

Step 3

After the code has gone into Circle, things start to get interesting. Circle has a concept called workflows, which consists of jobs. They allow us to define jobs in our code release process, and decide whether some jobs should be run in parallel and which are inter-dependent. Each job has multiple steps, and some push data to other services we use. The jobs we’re most concerned with here are “deploystaging” and “deployproduction” jobs (far right).

0 i90m3rGDQbdza98E

The “build_production” job builds all our code into a bunch of production-ready, optimized files. When optimized, each file gets a hash appended to it based on its content. The build job also creates a map of manifest pointing files with simple, memorable names for hashed counterparts.

Step 4

The deployment jobs are where the real “magic” happens. Once all the previous jobs in the workflow are green, Circle takes all the files that are created from the “build” job and pushes them to an AWS S3 bucket (file storage server). It also gathers other data related to the change, along with the manifest file, and pushes it to another database called Redis. The data we store there is indexed by the “revision” in Git, which allows us to look it up and serve it to our users. By indexing things based on a stable revision, we’re also able to create a pointer indicating the current version.

Step 5

Once everything is deployed, our back-end loads the data from the “current” revision in the Redis database by default. However, it also allows you to add a “query parameter” to preview another non-current revision. This is how we can do the whole “test every single revision” thing! If you want to preview a change, you just add “?revision={hash}” at the end of the URL in your browser.

And how do we know what hash to use? We built a dashboard/admin panel to view details, and preview or activate our revisions on staging and production:

0 PEfaKaHiV4ryeKcW

It’s a pretty cool little side-project using some pretty new and awesome technology on AWS called Lambda. Lambda allows us to write small, self-contained API’s instead of having to create and maintain a large application. It was quite a fun little exercise to build this thing!

Skip to the end…

Basically we’re pretty stoked. Auto-deployment opens up so many doors for optimizing our front-end. By only delivering the code that users actually need, we can roll out features to select groups of people (like beta users vs gradual rollouts) — and that’s just the start!

Read also

Designed by vikings in Oslo, Norway