familiarcycle

Azure DevOps and the pain of multiple environments

Apr 05 2020

Azure DevOps is a powerful CI/CD platform (that I use daily at work). However, it makes some sacrifices in its aim to be a general-purpose solution for all kinds of software development projects.

This post is about some pains I experience when using Azure DevOps to build and deploy services.

Say you're working on a service. Could be a website, a microservice within a Kubernetes cluster, any such thing.

Typically you don't deploy your changes straight to your Production environment. It's too risky. You’d like to make sure your service starts up and functions correctly when the new bits land on your servers. Additionally, you want to do some integration testing against the external services your code depends on.

Pre-production environments are a useful way to solve these problems. You can deploy your new code to real servers and interact with real peer services, but with no real user traffic,and thus no user impact if something goes wrong.

Azure DevOps makes this lifestyle more painful than it should be.

Here is a (fake) service that deploys to three environments: Test, Int, and Prod.

It's not obvious which commit is live in each environment.

The issue boils down to the fact that this UI, which lives in the "Repos" panel of Azure DevOps, has to be coherent for all types of code: not only services, but also standalone applications and libraries.

The UI must be the lowest common denominator, which is to say it must be show a deployment process in its most abstract form.

I wish I could toggle a switch to have the Repos UI go "Service Mode" and look something like this:

Why is this UI better?

You know exactly which commit is live in each environment. Diffing between two environments is obvious.

The UI is also rollback-proof. If you need to re-deploy an older build in a rollback situation, the environment's tag will shift down accordingly. In the Azure DevOps UI, the rollback is completely invisible.

It's also free of redundant deployment information. Only one commit can be live in an environment at a time. Having a status next to each commit is telling me too much.

More on environment diffing

Diffing between two environments is something I find myself doing daily. Thus my wish that Azure DevOps made it easier to do so at a glance.

To make things concrete: I work on a service with multiple developers, and there are typically a handful of commits each day. Ideally, each individual commit is deployed independently to production. In reality, if the commits are close together (say all in the same morning), only the latest will be deployed.

When I check-in a feature and want to deploy, I need to make sure I'm aware of the commits that will be coming along for the ride and hitting production for the first time, if only to double-check with my teammates that their changes are sufficiently validated in pre-production.

I've already shown what I think is the ideal solution to this problem. However, Azure DevOps does make such diffing possible in a different area of the platform, the Pipelines page.

The "Runs" tab for a pipeline can be a fantastic resource for understanding the deployed state of your service.

The thing is, I'd rather not think about builds at all. If my builds are deterministic and I trust the platform to follow my build scripts correctly, then it's sufficient to just let me know which commit is deployed to each environment.

While Azure DevOps does show the commit message (and ID) for each build, it's never guaranteed that there aren't duplicate or out-of-order builds mixed-in. If I start from the Repo view instead, I don't have to worry about that.

I do think this page and the Pipeline system in general are solid general solutions. I just think that a service-specific interface would make improve the experience greatly.

The closest place Azure DevOps comes to showing exactly what I want is in a pipeline's individual run view:

The view shows a diff between the target environment and the current run. Exactly what I want!

Almost. It seems to take the diff at the creation of the run and never updates it later. Why can't it always be up-to-date?

Conclusion

Azure DevOps is designed for all types of software development projects. Unfortunately, that means the user experience can't be optimized for more specific use-cases, such as deploying services.

The platform could be improved by supporting a service-specific UI for the Repos view, labeling the current commit deployed in each environment. Such a UI would make common tasks such as environment diffing easy to do at a glance.

Since Azure DevOps integrates both the Git repository and the CI/CD pipeline, I think there is opportunity for more code-centric user experiences. As a software developer, that's the most natural way for me to understand the state of things anyway.