Digital Edition

SYS-CON.TV
Software Quality Metrics For Continuous Delivery | @DevOpsSummit [#DevOps]
Even small changes need to be tracked and their impact on overall software quality must be measured

Software Quality Metrics for Your Continuous Delivery Pipeline | Part I

How often do you deploy new software? Once a month, once a week or every hour? The more often you deploy the smaller your changes will be. That's good! Why? Because smaller changes tend to be less risky since it's easier to keep track of what has really changed. For developers, it's certainly easier to fix something you worked on three days ago than something you wrote last summer. An analogy from a recent conference talk from AutoScout24 is to think about your release like a container ship, and every one of your changes is a container on that ship:

Your next software release en route to meet its iceberg

If all you know is that you have a problem in one of our containers you'd have to unpack and check all of them. That doesn't seem to make sense for a ship, and neither does it for a release. But that's still what happens quite frequently when a deployment fails and all you get is "it didn't work." In contrast, if you were shipping just a couple of containers you would be able to replace your giant, slow-maneuvering vessel with something faster and more agile - and if you're looking for a problem, you'd only have to inspect a handful of containers. While adopting this practice in the shipping industry would be a rather costly approach, this is exactly what continuous delivery allows us to do: Deploy more often, get faster feedback, and fix problems faster.

A great example is Amazon, who shared their success metrics at Velocity:

Some impressive stats from Amazon showing the success of rapid continuous delivery

However - even small changes can have severe impacts. Examples?

  1. Heavy DOM Manipulations through JavaScript: Introduced through a "harmless" new JavaScript library for tracking link clicks
  2. Memory Leaks in Production: Introduced by a not well tested remote logging framework downloaded on GitHub
  3. Performance Impact of Exceptions in Ops: Ops and Dev did not follow the same deployment steps (lack of automation scripts) resulting in thousands of exceptions and maxes out CPU on all app servers

Extending Your Delivery Pipeline
Even small changes need to be tracked and their impact on overall software quality must be measured along the delivery pipeline so that your quality gates can stop even the smallest change from causing a huge issue. The three examples above could have been avoided when automatically looking at the following measures across the delivery pipeline and stopping the delivery when "architectural" regressions are detected:

  • The number of DOM manipulations
  • Memory usage or object churn rate per transaction
  • Number of exceptions, number of database queries or number of log entries.

In a series of blog posts I will introduce you to metrics that you have to measure along your pipeline to act as an additional quality measure mechanism in order to prevent problems listed above. It is important that:

  • Developers get these measurements in the commit stage
  • Automation Engineers need to measure them for the automated unit and integration tests
  • Performance Engineers add them to the load testing reports you do in staging
  • Operations verify how the real application behaves after a new deployment in production

For each metric I introduce, I'll explain why it is important to monitor it, which types of problems can be detected and how Developers, Testers and Operations can monitor these metrics. To ready more on this, click here for the full article.

About Andreas Grabner
Andreas Grabner has been helping companies improve their application performance for 15+ years. He is a regular contributor within Web Performance and DevOps communities and a prolific speaker at user groups and conferences around the world. Reach him at @grabnerandi

In order to post a comment you need to be registered and logged in.

Register | Sign-in

Reader Feedback: Page 1 of 1



ADS BY GOOGLE
Subscribe to the World's Most Powerful Newsletters

ADS BY GOOGLE

CI/CD is conceptually straightforward, yet often technically intricate to implement since it require...
The explosion of new web/cloud/IoT-based applications and the data they generate are transforming ou...
Containers and Kubernetes allow for code portability across on-premise VMs, bare metal, or multiple ...
Enterprises are striving to become digital businesses for differentiated innovation and customer-cen...
Digital Transformation: Preparing Cloud & IoT Security for the Age of Artificial Intelligence. As au...
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't com...
DXWorldEXPO LLC announced today that All in Mobile, a mobile app development company from Poland, wi...
The now mainstream platform changes stemming from the first Internet boom brought many changes but d...
DXWorldEXPO LLC announced today that Ed Featherston has been named the "Tech Chair" of "FinTechEXPO ...
Chris Matthieu is the President & CEO of Computes, inc. He brings 30 years of experience in developm...
Bill Schmarzo, author of "Big Data: Understanding How Data Powers Big Business" and "Big Data MBA: D...
Andi Mann, Chief Technology Advocate at Splunk, is an accomplished digital business executive with e...
In this presentation, you will learn first hand what works and what doesn't while architecting and d...
The Internet of Things is clearly many things: data collection and analytics, wearables, Smart Grids...
To Really Work for Enterprises, MultiCloud Adoption Requires Far Better and Inclusive Cloud Monitori...
We are seeing a major migration of enterprises applications to the cloud. As cloud and business use ...
If your cloud deployment is on AWS with predictable workloads, Reserved Instances (RIs) can provide ...
Disruption, Innovation, Artificial Intelligence and Machine Learning, Leadership and Management hear...
We build IoT infrastructure products - when you have to integrate different devices, different syste...
Consumer-driven contracts are an essential part of a mature microservice testing portfolio enabling ...