Once your organization has embraced and implemented a DevOps approach to software development, the work shouldn’t end there. Your team may have a streamlined DevOps strategy that fosters a culture of continuous delivery, but they still need to avoid falling into the trap of a drifting or coasting mentality.

There are definitive signs that systems are adrift, including:

• Archaeology: Out of date systems and tooling
• Recalcitrance: Last-generation approaches to DevOps (i.e.: VMs instead of containers)
• Fragility: Reluctance to update existing infrastructure/fear of breaking something
• Convolution: Increasing or uncontrolled infrastructure complexity and costs

If any – or all – of these indicators are present, it’s likely time to begin the continuous improvement process. The first step involves developing metrics to track the key DevOps indicators that matter to your business, which will help to determine ROI.

Time-to-Developer Productivity

On-boarding new (or existing) employees to a project is often a lengthy, painful process. No two developer’s machines are alike, and documentation is almost always out of date when it comes time to sign a new team member up. Many real-world teams end up with a fleet of one-off dev environment configurations that are difficult to troubleshoot, update, and collaborate upon. As a result, new team members usually face an uphill battle to productivity.

By using DevOps toolsets, such as Vagrant and Docker, this variability can be greatly reduced or eliminated. Once the team starts treating their environment like any other piece of source code, time-to-productivity plummets. When these practices are properly implemented, it is not uncommon to see developers successfully building, writing, and testing code on day one of a new assignment. In the more traditional approaches we often see amongst our clients, this can take upwards of one to two weeks, or even longer.

Production Defect/Error Rates

By building quality into your DevOps pipeline, you can catch defects before they make it to your deployed software. Utilizing infrastructure as code principles decreases environmental variability and ensures that software running on a developer's laptop acts the same way that it does in production.

Time It Takes to Move Code From Git Commit to Production Deployment

The highest-performing organizations can complete full development cycles in minutes or hours, moving code from a developer commit to production. Legacy workflows can take weeks, months – or in worst-case scenarios – years. By lowering the amount of time it takes to move code from commit to production, organizations greatly reduce their risk of failed deployments by limiting the scope of any one deployment. Additionally, developers become empowered and responsible for ensuring their code works as desired the first time. External dependencies and integration tasks become much simpler, and more features-per-dollar can be delivered.

Time It Takes to Release a New Feature

Business outcomes are largely tied to an organization's ability to innovate and respond to changing market demands. The previous three metrics work in concert to determine how quickly your development team can respond to the needs of the business and deliver the needed features. When new feature work ends up mired in red tape, technical challenges, or poor quality, the needs of the business cannot be satisfied in a timely manner, resulting in missed opportunity. This drops straight to an organization's bottom line.

Customer/End-User Satisfaction

This often overlooked metric is an important high-level indicator of the effectiveness of an organization's overall approach. Low customer satisfaction can be an early indicator of underlying issues, such as inadequate performance, system downtime, poor quality, or incorrect/inadequate feature sets. If the end user is not satisfied with the product, further investigation is typically warranted to identify the root cause.

System Downtime

Downtime impacts your entire software supply chain, from developers all the way to end users. Increasing downtime can be a sign that your infrastructure and/or software is becoming fragile and needs attention. If this metric starts to trend up, it is important to do a root cause analysis to determine the best next steps.

Iterate

Track and feed results back into DevOps planning and execution activities. This is the key takeaway. Tracking the right metrics allows you to try different approaches and know if they are effective or not. In turn, you can know where and when to invest, along with creating a feedback loop to iterate in the right direction. This allows you to build strategies that support a constant strive to drive metrics in the desired direction of increased quality, reduced delivery time, and increased productivity.

Which leads us to the ROI of continuous improvement:

True Agility: Increased ability to innovate, respond to market changes, stay ahead of the competition, stay software nimble and enable the business to deliver.

Retention and Productivity: Increased employee satisfaction and team performance. The best developers will not tolerate an environment that does not support this culture. High performers must be enabled with the right support and tooling to succeed. According to the Society for Human Resource Management, it costs an average of $4,129 to bring on a new hire, with an average recruitment time of 42 days, plus a hard cost to the company of 75% of the salary for that position – all of which could be avoided if employees feel supported and valued.

Performance: Increased system reliability at lower cost. Continuous DevOps improvement allows you to both improve reliability and optimize/maximize the usage of your infrastructure, which will manifest via increased ROI.

Training: Cross-functional collaboration, because building a successful DevOps culture will encourage increased collaboration across your organizational boundaries. Your teams will work to support each other and ensure overall success.

Mitigation: Reduced security risk. Continually improving your DevOps systems and approach allows you to stay nimble enough to respond to (and mitigate) day zero security risks. Lower performing organizations often get stuck in out of date tooling and systems, thus increasing their attack surface. And since a breach costs an average of $3.62-million, this ROI speaks for itself.

Quality: Reduced defect rates. Defects that make their way into production are more costly to fix than ones that are caught earlier on in the development cycle. By implementing the right tooling and knowing how often these defects end up impacting customers, you stand to drive down development costs and improve your end user’s experience.

Awareness: Insight into key business metrics. Knowledge is power. Instrumenting your infrastructure to capture the data your organization depends on allows you to make the strategic investments necessary to drive your business in the right direction