Each year the DevOps Research & Assessment, led by Jez Humble and Nicole Forsgren recently acquired by Google Cloud, produces a report on the state of the art of DevOps entitled “State of DevOps”. This state of the art report is based on more than 31,000 interviews with professionals (all industries, sizes, countries, mainly USA and Europe…). The 2019 edition, released at the beginning of this year, is to date the largest study conducted in this field.
What are the news and surprises of this new edition?
First of all, let’s remember that this study divides the IT organizations of user companies into 4 groups according to their level of performance:
- Elite performer
- High performer
- Medium performer
- Low performer
These groups are defined against the measurement of the 4 DevOps performance indicators detailed later in this article: lead time, deployment frequency, change fail rate, time to restore.
The main lessons learned from this survey can be summarized in 6 points:
- Overall, DevOps practices continue to progress everywhere: the share of High Performers has tripled in 6 years.
- The link between an organization’s economic performance and its ability to deliver software quickly and reliably is undeniable (DevOps). The implementation of DevOps is therefore a profitability factor for companies, giving them a competitive advantage.
- The best DevOps deployment strategies at scale are based on Communities of Practice at all levels of the organization as well as test & learn type Proof of Concept approaches. See the article on Communities of Practice (French): https://blog.operaepartners.fr/2017/04/07/communautes-pratique-lean/
- The use of Cloud Computing continues to be a strong differentiator for Elite and High Performers. Even if this point was already present in previous reports, remember that DORA is now co-authored by Google Cloud teams.
- Productivity gains improve Quality of Life at Work and tend to reduce burn-out. Although it may seem counter-intuitive at first glance, it is also a reality in Lean practice. Employees regain control of their activities, clarify their daily objectives and are empowered to improve their practices. This is a reality in our experience of coaching teams, see the article on this subject (French): https://blog.operaepartners.fr/2017/10/31/il-ny-a-pas-de-lean-et-dagile-sans-respect-des-personnes/.
- There is a “good” process for approving change requests (going into production). Good in the sense that it allows acceleration, maintaining run stability and reducing burn-out.
The obsession with measurement
It can never be said enough, it is essential to measure performance to get out of the world of opinions, to get to the field of more objective facts and to measure the improvements implemented. The lean that inspired DevOps and Agile practices has always proposed to measure performance from the customer’s point of view (and not from the team’s point of view!): product volumes, number of defects, customer satisfaction, inventory, lead time… In this sense, the report recommends to continue to rely on the 4 key DevOps measures:
- Lead time: time elapsed from code commit to production release. Purists will object that the real lead time is the measure from the customer’s request to its delivery. For example, I have measured lead time from a few hours for some teams to several months for others.
- Deployment frequency: the frequency at which software is put into production, from several times a day to several times a year.
- Change Fail: the failure rate when a software is put into production. For example, 15% of releases lead to re-delivery, rollback, incidents…
- Time to restore: the time it takes to fix an incident from the moment it occurs (be careful, this is not the moment when it was reported but the moment when it occurred) until it is finally corrected. Again, times in the range of a few minutes to several weeks are commonly measured.
Finally, in 2019, DORA added a 5th measurement:
- Availability: the rate of availability, already well known by IT teams within the framework of SLAs.
In the 2019 report, it should be noted that once again, performance on the 4 key areas is intrinsically linked. One might think that some organizations will favour one indicator to the detriment of the others. For example, going fast at the expense of quality. Once again, this is a common misconception that proves to be false, the more we improve stability and quality, the faster we go. On this subject, see the excellent post by my colleague Christian Ignace on cognitive bias (bias n°4 – French): https://blog.operaepartners.fr/2019/05/15/mes-heuristiques-face-au-lean-qui-va-gagner/
The report therefore confirms that the High Performers excel on all 4 performance measures at once and the Low Performers underperform on all 4 axes.
Extract from: Accelerate – State of DevOps 2019
In summary, it is striking to note the differences in speed and stability between High Performers and Low Performers: the factors are in the order of 100 to 1000. One can immediately imagine the impact of these figures in terms of time to market for these companies and the impact on their economic performance:
Extract from: Accelerate – State of DevOps 2019
And in your organization, how do you measure the performance and improvement of your software delivery process?
Operae Partners has been a DORA partner for France since 2018 and combines the Lean IT approach with DevOps deployments to improve the entire end-to-end delivery process.
If you would like to discuss this with us, please contact us: firstname.lastname@example.org
Version française de cet article: