In modern software development it’s common to use version control systems for deploying code as well as for keeping track of evolving code. I would like to talk about certain complexities that tend to lurk in this approach.
For our exercise we are going to assume that we have a project that is deployed to three environments: Development, Staging, and Production. Deployment to each of those environments is achieved by pushing something to the following corresponding branches: dev, rc, and master. Every push to any of those branches triggers a build that packages the sources code into binaries and pushes them to the respective environment.
This approach seems straightforward enough. However it introduces complexities invisible at first glance.
Ideally we would also like to have a set of simple rules that you can follow without involving our frontal cortex and only use muscle memory most of the time.
I assume that you are already somewhat familiar with git workflows and branching strategies. I can recommend these resources as a good overview of standard git branching practices. - git workflows man page - branching workflows - a successful git branching model
Sharing Dev Environment
Let’s start with a typical state that you might be in when you decide to start working on a new cool feature. All your branches point to the same commit. Everything that you worked on previously has been merged to rc and master and deployed correspondingly to staging and production environments. Here is what it looks like in git history.
Now, let’s start by adding new changes. You’ve added two commits. You are ready to push your changes to the develop branch in order to test them. Here what your git history looks like.
You attempt pushing to develop and git rejects this operation. Somebody else has already pushed another two commits into the develop branch.
Jane has pushed her changes to the develop branch. She is testing her changes in the development environment.
If you rebase your changes onto Janes changes then you start depending on her changes. You can’t release your changes without bringing her changes along. In order to avoid bringing her changes you would need to rebase your changes back onto the rc branch. It seems inconvenient.
The same thing with cherry pick. Also the rebase/cherry pick workflow can produce conflicts that you don’t really care about until Jane is done with her work.
The most convenient way to approach this problem is to merge your changes into the develop branch.
When you add new changes you commit them to your branch.
When you need to test them in dev environment you just need to merge them in develop branch again.
This approach is very convenient because whenever you decide to merge your changes into the rc branch you can do it without shuffling your changes. Your whole chain of commits stays intact. It’s the same for Jane as well. She only needs to merge her changes into dev. There is very little difference between what you test in dev and what you release to staging.
There is even less difference if you didn’t have any conflicts when you were merging your changes to develop branch. It means other developers’ changes were orthogonal to your changes and didn’t interact with your changes.
Of course there are corner cases when a merge without conflicts doesn’t mean that your changes are not affected by someone else’s changes.
Merging Changes to Staging
When you are done with testing your changes, you can merge your changes to rc.
If there are no other changes in rc then you can either fast-forward rc branch to your latest commit or force a merge commit. I prefer an explicit merge commit because it clearly shows where you did your development and where a complete feature is.
If there are other changes in rc by the time you decide to merge your changes then merge commit is even more favorable. If there are any conflicts then rebasing may require to resolve conflicts in many commits of your branch. Even if you don’t have commits your intermediate commits may not compile or pass tests any more.
Now Jane is the only person using development environment. If she is not interested in your changes and doesn’t care about the other changes in rc branch she can just force push her changes to develop branch and get rid of red merge commits in dev branch. They are not necessary any more.
If she does care about the changes in rc branch or somebody wants to see the released changes in dev environment then rc branch can be merged into the develop branch.
Releasing Hot Fixes
After developers are done implementing a feature they merge their changes to rc branch. The changes are deployed to staging environment and integrated with the other completed features.
The staging environment is where some final tests are done before rolling out the changes to production. It may take some time to ensure that it’s safe to send changes to prod.
During this time an issue in production environment may emerge. Fix for this issue can take precedence over all other features that are currently in staging.
In this case we need to realize that your rc branch is actually two branches that use the same pointer. One branch is the branch that points to the commit that is deployed to staging environment. The other branch is the integration branch for all your changes.
This is the moment when those two branches need to part their ways temporarily.
You need to branch your hot fix commit off of the master branch. You need to create an integration branch that points to the rc branch. And push your hot fix to the rc branch. Here is what git tree should look like after all of these operations.
Now your changes from the former rc branch are not deployed anywhere. You just bookmarked them using the integration branch.
What do we do next? We test our hotfix in staging and merge our hotfix into master. If it’s only a single commit we can safely just fast-forward master branch.
You may wounder how to test your changes in dev environment? Let’s leave it as a homework exercise. Sharing dev environment section gives a pretty good idea how it can be applied to hot fixes.
After the hot fix has been released to master it’s time to bring our integration branch and the rc branch together. We need to merge rc branch into integration branch. Reset the rc branch to the integration branch. And get rid of the integration branch until next time.
Rationale for the Suggested Model
Let’s discuss why the integration branch springs into existence when we decide to do a hot fix. It happens because the integration branch has in fact always been there. Most of the time it points to the same commit as the rc branch. It would be inconvenient to move two pointers together all the time. Instead we only use one pointer - the rc branch. Whenever we need to separate them we introduce the integration branch.
We need these two pointers(branches) because our repository serves two purposes. It keeps our code as well as deploys it. Branches are usually used for marking different stages of code life cycle. Here they are also used for specifying what code is deployed to what environment.
Let’s imagine what it would look like if we had an infinite number of environments available to developers. In this case whenever we need to deploy our code to an environment we pick an environment and specify what commit we want to deploy.
We could use a successful git branching model for working on our code. Each developer would test his code in a completely independent environment. After being merged to master the code would need to be deployed to production environment only.
Since you can have your integration branch deployed in one environment and your hot fix deployed in another environment you don’t need to coordinate how they use rc branch. Because they don’t have to coordinate their deployment to the only available staging environment.
If your code passes tests in any of those environments it’s good to be merged to master.
The situation changes when you have only one staging environment and one development environment. In this case you need to find a way to share them in a coordinated manner. With a little bit of planning git can be efficiently used in such circumstances.