Productivity matter is an awkward problem in software development.
In the traditional concept, production is a process of transforming input into output. Trying to understand developer productivity from this perspective can be contradicted. The input looks simple (“only” developer’s time), the production process is full of blind spots, and the output expectation (sometimes) is exaggerated.
Understanding of input and output is still evolving in software development, as of today, we can conclude that input is all the effort dedicated to the development process (not only coding) and output is not the software but the values it brings (outcome over output).
Yet measuring productivity is still a tough challenge.
The article discusses some productivity assessment approaches in the software industry.
Single Metric
Productivity is broad (contains many aspects) and layered (can be perceived from different points of view). A single Metric only tells limited things about productivity if not nothing (vanity metric).
For example, the line of code feels like a rational metric to measure the coding productivity of developers. But it is not directly related to project progress and people discourage it. The legend tells that, in early 1982, Bill Atkinson removed 2000 lines of code within a week to optimize QuickDraw.
Composite metric (combining several metrics) is not a good idea either. It is even more tricky to implement and biased toward what the number tells (productivity is an abstract concept).
Rather than identify the problem, we can choose the opposite direction, to use metrics to answer the problem instead. The approach is called GQM (Goal-Question-Metric) which defines the goal before choosing the metric.
Forced Rank
Pioneered by GE in the 80s, the Vitality Curve (or Forced Rank) is a popular performance management approach for manufacturing (And yes it is adopted in some tech companies).
The employee is ranked based on individual performance and uses the normal distribution curve (bell curve) to determine who is productive.
The magic number is 20–70–10 where the top 20% is the most productive one, 70% is working adequately, and the last 10% is non-performers (subject to be replaced).
While the approach credited to GE Growth, it also harvests critics such as being too strict, encouraging competition and being a morale destroyer
By Nature, software engineers are knowledge workers who utilize/produce knowledge for day-to-day tasks. They are involved in learning new things, problem-solving, and creative tasks.
Knowledge workers can have new jobs over time, and even the same role in the same team may not work in the same. This situation contrasts with manual labor who usually have regular and standardized tasks.
The Bell Curve Approach should be reconsidered in the case of knowledge workers.
SPACE Framework
SPACE Framework offers holistic dimensions to understanding productivity better and avoiding misconceptions.
DORA Metrics
DevOps Research and Assessment (DORA) advocated 4 DevOps metrics that indicate the team performance.
DevEx Framework
DevEx Framework believes that developer experience is what drives productivity and we should focus on it instead.
Epilogue
So what’s the best approach? It depends on the organization’s situation and condition. Nevertheless, productivity metrics are messages about the value and expectations of the company to their employee.
For further reading, I recommend the book Rethinking Productivity in Software Engineering (2018) which covers various topics of productivity and Measuring Developer Productivity: Real-World Examples (2024) that spill productivity metrics from top companies.
Keys Take Away
- Using a single metric to understand productivity doesn’t make sense.
- Force ranks engineers based on productivity don’t work.
- SPACE Framework tells us that productivity is more than input/output.
- DORA Metrics tells us to be based on data to deliver productivity reports.
- DevEx Framework tells us to be more human-centric on productivity.
