Beyond Velocity: Data-Driven Delivery Management for Agile Teams (Part I)
In a volatile and uncertain business environment, delivering software increments with quality and speed is paramount. These objectives are a vital part of agile development, but to move fast, the team needs a thorough understanding of what is being done and, more importantly, if it is on the right track. To do so, it is not enough to ask, “How fast are we going?”. Delivery managers ensure the team progresses efficiently towards goals. They need to dig deeper and use actual data to understand what’s happening behind the scenes. With this understanding, they can identify roadblocks, make informed choices, and ultimately allow the team to build better software and release a faster version.
This article explores data-driven delivery management, a powerful approach that empowers agile teams to unlock valuable insights from their existing development tools. By piecing together those data points, Delivery Managers can get valuable signals that can later be turned into actionable insights.
Data: Your Agile Advantage. Are You Using It?
Every step of your software development process creates valuable signals. These signals are produced by all sorts of activities the team executes. Therefore, a Delivery Manager needs to identify, understand, and integrate it to build knowledge. For getting those signals, it is essential to use the right tools. Here, we present the most common types of tools that can be useful for gathering signals:
Project Management Tools: (Jira, Asana) used for planning, tracking, and reporting on tasks, user stories, and overall project progress. These tools provide a visual overview of workflow, deadlines, and team workload.
Code Checkers (SonarQube, CodeClimate) are used for static code analysis, identifying potential bugs and security vulnerabilities before they cause problems.
Version Control Systems: (Git) Its two significant purposes are maintaining the integrity of the different artifacts and allowing effective collaboration over a shared codebase. The version control system is essential for managing changes and maintaining a history of the project’s evolution.
Deployment Tools: (Jenkins, CircleCI) these are like the pit crew for the software, used to streamline the deployment process. They track how often new versions are released and how quickly issues are fixed.
Testing Tools: ensure the software works properly. These tools are used to maintain a systematic quality assurance strategy. In it, teams create different use cases and register the execution and outcomes of the other test types.
Monitoring Tools: (Datadog, New Relic) These tools monitor software in production parameters, such as performance and overall health, producing a vision of how fast it responds to users and whether there are any errors.
Customer Feedback (Surveys and interviews) allows for gathering the actual outcome: feedback from the users using the software. It can give valuable insights into customer satisfaction by gathering insights into user satisfaction.
Turning Signals into Actionable Insights
Imagine all this information locked away in separate boxes. That could be more helpful! For a Delivery Manager to turn signals into actionable insights, they need to be glued together from all these tools and analyze them to get a complete picture of how the work is progressing. Data integration tools like Swarmia, Minware, or Hevo Data act like bridges, connecting all this data from different tools so you can see the big picture. The Delivery Manager must use the data to answer the right questions and integrate this. Only then are metrics to be defined.
One interesting methodology for determining what metrics are needed is Goal–question – metrics (GQM)1. Following this methodology, here are some questions that should be made and examples of possible metrics to help answer them:
Is the team moving at a good pace?
Metric: Velocity
Why it matters: Velocity measures the amount of work a team completes within a specific timeframe (usually a sprint). It is typically calculated by adding the story points (a relative estimate of effort) of all user stories completed in a sprint. Project management tools like Jira or Asana can track completed user stories and their associated story points.
Are tasks taking too long to complete?
Metric: Lead Time & Cycle Time
Why it matters: Lead Time measures the total time it takes to complete a work item, from when it enters the development process to when it’s ready for deployment. Cycle Time focuses on the time spent actively working on a work item, excluding wait times. Version control systems like Git can track when a work item is started and finished, and you can calculate Lead Time and Cycle Time by analyzing those timestamps.
Are there a lot of bugs in the code?
Metric: Defect Rate
Why it matters: Defect Rate measures the frequency of bugs discovered in the codebase. It’s typically expressed as the number of defects found per unit of work (per 1000 lines of code). Issue tracking tools used for bug reporting can provide data on the number of defects identified. To calculate the defect rate, divide the total defects by the codebase size or the number of user stories completed.
Is most of the code well-tested?
Metric: Code Coverage
Why it matters: Code Coverage measures the percentage of your codebase covered by automated unit tests. Higher code coverage indicates a more thorough testing process and potentially fewer bugs. Testing tools provide code coverage reports indicating which sections of your code are covered by unit tests.
How often are new versions released?
Metric: Deployment Frequency
Why it matters: Deployment Frequency measures how often you release new versions of your software to production. It’s typically expressed as the number oper unit of time (e.g., weekly deployments), as deployment tools like Jenkins or CircleCI track deployment events. Analyze the deployment history to calculate the deployment frequency.
How quickly can we fix problems in production?
Metric: Deployment Lead Time
Why it matters: Deployment Lead Time measures the time it takes to fix a bug or issue in production and deploy a fix to users. It reflects the team’s ability to respond to problems promptly. Track the time from identifying a production issue to deploying a fix. Deployment tools and version control systems can provide timestamps to calculate deployment lead time.
Do we have to roll back releases often?
Metric: Rollback Rate
Why it matters: The Rollback Rate measures the frequency of rollbacks due to bugs or unforeseen issues. A high rollback rate indicates potential problems with the deployment process or code quality. Deployment tools track successful and failed deployments. Analyze deployment history to identify rollbacks and calculate the rollback rate.
Is the software running smoothly?
Metrics: Application Uptime, Average Response Time, Error Rate
Why it matters: These metrics measure the health and performance of the software in production. Application uptime reflects the percentage of time the software is available to users. Average response time indicates how quickly the software responds to user requests. Error rate measures the frequency of errors encountered by users. Monitoring tools like Datadog or New Relic track these metrics continuously. Leverage these tools to analyze application health and performance.
Are the customers or users happy?
Metrics: Customer Satisfaction
Why it matters: Customer satisfaction is a broader measure of how happy the users are with the software. It can be assessed through surveys, interviews, and user reviews. Conduct surveys or interviews with a representative sample of the user base to gauge their satisfaction.
So far, we have seen the limitations of velocity as the sole metric for agile teams. We have also introduced data-driven delivery management as a powerful approach to understanding their work better.
Fast delivery of high-quality software is crucial, but teams need to go beyond measuring velocity to move with agility. They need to understand what is being done and if they are moving in the right direction.
Don’t miss Part II of our series, Beyond Velocity: Data-Driven Delivery Management for Agile Teams, where we’ll transition from theory to practice. We’ll explore real-world scenarios and showcase how agile teams can leverage data to diagnose problems, optimize workflow, and ultimately deliver high-quality software.