In the software engineering models that most organizations adopted in the past few decades, product owners and product managers sat between the business teams and engineering teams. These product teams had many roles: organizing backlogs, prioritizing features, handling communications, and translating business requirements for development teams. But one subtle role that the product teams filled was managing KPIs.

Many companies have adopted AI and vibe-coding tools that enable their business teams to innovate and automate without the organizational overhead of product teams and software engineering teams. While this reduced friction can lead to great outcomes, it can also lead to inefficiencies when decisions on what and how to automate aren’t made effectively. To effectively make these decisions, even vibe coders should measure KPIs.

Organizations that saw effective change through new software releases were often those that were best able to measure and target the efficacy of those changes during the planning and prioritization phases. Many of the projects that I worked on began by implementing measurement tools and refining metrics, developing a data set to analyze, then building and prioritizing a backlog of features to drive progress.

Anyone who has read The Goal by Eliyahu Goldratt should be familiar with the theory of constraints. This theory, originating in manufacturing but applicable to most process driven fields, focuses on managers identifying the bottleneck in their process, then allocating resources to that bottleneck. Improving a step that isn’t the bottleneck has little to no effect on the overall process.

One example of how I applied TOC in my career was a batch process that was running overnight and into morning business hours. Clients expected deliverables in the early morning, and a group of developers and operations teams got together to map the process and identify the constraints. Through this process, we identified several blind spots in our measurements. When we added these measurements into the system, we were quickly able to identify the constraints and develop a plan to improve the process. Additionally, after the automation of a few steps in the process we realized that further development wouldn’t improve delivery since the process now ended before our European team got to the office to start the next step. Further investment would have added no improvement to our KPIs.

Today’s vibe coders should understand and appreciate the lessons learned from the theory of constraints. Begin by building tools to measure enough of your process to find the constraints. From there, identify the areas of your process which will have the greatest effect on the outcomes you are trying to achieve. For example, goals like maximizing throughput, reducing error rates, or reducing costs will frequently force a developer to focus on very different areas of the process.

When anyone can solve the problems that annoy them the most, automation will likely focus on steps that aren’t the bottleneck.

The importance of measurement is emphasized throughout the Agile Congruence Framework, but steps 7 (Determing Value and Priority) and 10 (Remeasuring Outputs) are nearly impossible without effective measurement. Managers of teams with access to vibe coding tools should learn to ask questions like:

  • What KPI are you trying to improve and what is the expected effect?
  • How does the part of the process you are automating affect the overall performance of the system?
  • How can we report the impact of the change you are making?

And of course, continue to ask important congruence questions like

  • What teams will be affected by this change?
  • What training should we develop before rolling out this change?
  • How will this automation affect existing communication patterns?
  • What other formal processes will be affected by this change?

The lesson applies whether you follow the famous line often attributed to Vince Lombardi, “If you don’t keep score, you’re only practicing” or the old management adage “If you can’t measure it, you can’t manage it”.

The practice of measuring first will nearly always lead to better outcomes, and to a faster path to real process improvement. In vibe coding, knowing that you are solving the right problem in the right way will lead to more effective outcomes.