I have a confession to make. I’ve suggested several times that you should focus on outcomes over outputs, but I don’t believe I’ve ever really explained why…
So now it’s time to correct that oversight.
Specifically, I’d like to take a look at the benefits and pitfalls of focusing on outcomes over outputs.
But first, a quick review on what I mean but outcome over output.
What I mean by outcome over output
An outcome is the change in the organization and changes in the behavior of your customers, users and/or stakeholders as a result of your product.
An output is anything that your team delivers. This could include requirements, documentation, processes, software, tests and other things that tend to be measured in order to gauge how your work is proceeding.
The idea of focusing on outcome over output is that you define success in terms of reaching a specific outcome. You then measure progress based on whether you are getting closer to that outcome.
The goal of your team is not to produce outputs; it’s to reach a specific outcome. A successful team seeks to maximize outcome with minimal output. That way, you get as close to the desired change in your organization and your stakeholders’ behavior with the least amount of initial and ongoing work as possible.
The benefits of focusing on outcome over output
There are several reasons I encourage teams and organizations to focus on outcome. Here’s a summary of the three main reasons.
Your definition of success is more meaningful
When you establish a definition of success in terms of an outcome based metric, such as “Increase the number of new and renewing members” you can immediately tie your team’s efforts to something of value for your organization and customers. You know when you’re making a difference.
Compare that to the typical measure of success from an output perspective. When working with an output perspective, you define success based on how many features you deliver, or whether your team met your sprint commitment in terms of points completed. This is a sure fire recipe to turn your organization into a feature factory.
You don’t want to work in a feature factory.
Measuring progress based on outputs delivered doesn’t give you an idea of whether you’re building the right thing. In fact your team is tempted to deliver big easy things because you’ll get more credit. Just because something is big and easy to deliver does not mean that it’s the right thing to deliver.
Measuring progress based on delivering more stuff means that you’re delivering more stuff. Delivering more stuff now means that you have more stuff to maintain later. When you’re measured on how much stuff you deliver and have a choice to solve a problem in a way that requires no new code, or a way that involves adding a whole new feature, you’re going to be tempted to add that new feature.
Your team does not get motivation from hitting their target sprint velocity. This hit home for me a couple of years ago when talking to a team I was coaching. One of the developers came out and said “I get absolutely no motivation from hitting our sprint goal when it’s set as a number of story points. I could care less. Now, when that goal is to help our customer solve a specific problem, that’s a different story. I’ll be more than willing to do what it takes in those situations.”
The lesson? If you work in sprints, set outcome based sprint goals, not output based ones. And ditch the burnup and burndown charts. They don’t tell you if you’re delivering the right things.
You’re encouraged to learn
Focusing on outcomes drives you to learn through feedback. You establish a particular metric that you’re trying to achieve. The best way to know if you’ve reached that metric is to make a hypothesis about some action that will drive progress toward that metric, take that action, and then measure the metric to see the result. Often the action you take is to make a change to your product up to and including a new feature. When you measure the metric, you’re getting feedback. Actionable feedback.
If you hit the target value of the metric you wanted, you can stop work on the current outcome and move to something else. If you made some progress toward your target, you can think about what your next action should be and run a new experiment. If your metric gets worse, you can consider whether you should reverse the action you just took and try something different.
In any event you get information you can act on and you know whether you’re having a meaningful impact.
The only learning you generally do when focused on output is to figure out why you didn’t deliver more output.
You have more flexibility
When you focus on outcomes, your main definition of scope is delivering that outcome, or more precisely hitting the target value of your outcome based metric. Time and budget become constraints within which you have to act in order to reach that outcome.
You have a defined scope, but its definition is high level and offers you latitude to adjust your approach as you learn more about the problem and possible solutions.
When you focus on outputs, you define scope as a list of outputs that you will deliver. You define that list when you’re first starting your initiative – when you are at the point of maximum ignorance about the problem and possible solution.
You identify some outputs that you don’t really need to deliver, and you miss other outputs that you do need. When you discover the missing outputs, it can be a struggle to add them because you get accused of being a scope creep. At the same time you feel pressure to deliver all the items you agreed to, even though you know some are not necessary, and potentially detrimental. If you play your cards right you can often bargain with your stakeholders to remove some unnecessary outputs for others that are necessary. The trick is knowing for sure which outputs those are.
You still should define scope. Just do it in a way that acknowledges you don’t know everything up front and that you will learn along the way.
The pitfalls of focusing on outcome over output
Now that you’ve read the above you’re most likely thinking “why doesn’t everybody do this?” or you’re jumping up from your chair yelling “yeah… but”.
I hear you.
Here are the main pitfalls I see of the focus on outcome over output. To be honest they are mostly arguments that I see coming after the yeah… but.
Focusing on outcomes is hard. Focusing on outputs is (comparatively) easy
It can be difficult to come up with a meaningful outcome based metric that your team can measure and tell when your actions have an impact. You may have trouble isolating the effect of your change on the metric from other factors. Your change may take a while to have full impact, thus rendering the idea of short feedback cycles somewhat moot.
Meanwhile, it’s easy in comparison to count the number of story points you complete or features you deliver and use that as a gauge to measure progress. Although, as a product manager I was talking to about this subject noted, “I have trouble truly knowing what success means when all I do is track features delivered.”
You don’t want to be in the situation of celebrating when you launch and then immediately moving to the next set of features. You’ll end up doing work you don’t need to do, and you’ll never know for sure whether you delivered the right things. You won’t know if you’ve had an impact.
To address this challenge it may help to identify short term leading indicators. Metrics that measure some activity which you believe is an indicator that you’ll eventually get to the outcome you seek.
For example if you’re wanting to improve the number of people who renew, you may assume that the more frequently people use your product the more likely they are to renew and measure usage as a leading indicator.
You can also shorten the length of time to measure your metric and reset your targets. Instead of looking at renewals and new memberships every month, perhaps you explore the change in a week. You’d have to reset the target and baseline values, but at least you’d have a shorter time frame to wait for impacts.
We’re not used to measuring results
I think a big part of it is that people in those organizations are afraid of what they’ll find.
The more you work on something, the more you’re inclined to want it to be successful and the less likely you are to throw it away if it is not. You don’t want to admit something you worked on so long wasn’t the right thing to do. You’d look like a fool.
That depends on your organization’s perspective on learning and its willingness to change direction when something isn’t working. Yes, in some organizations you’d look like a fool for admitting that something you worked on didn’t work. In other organizations you’d look like a fool if you didn’t speak up as soon as you figured it out.
I’d rather work in the latter organization.
The easy way to address this pitfall is to find an organization that has a healthy attitude toward learning and failure. The more difficult way is to try and change the culture of your current organization. If anyone has any suggestions on how to do that, please share them in the comments because I haven’t figured that one out yet.
There’s no time to measure outcomes, we have to move on to the next thing
This is a feature of output based thinking. You set your scope as a list of outputs at the beginning of your effort. You think you won’t be successful unless you deliver all of those outputs. Meanwhile, as you get into things you realize you left stuff out and have extraneous stuff. But you struggle to make substantive changes to the list because you’re afraid of what people will think.
By the time you’re done with that effort, you’re sick of it, and it may have gone on longer than you wanted. The last thing you want to do is work more on that product. The second to last thing you want to do is to take any measurements that will confirm what you fear. Besides, other stuff has piled up.
So the idea of continuing to work on the same thing after a delivery seems rather foreign.
Yes, except if you were to follow an outcome based approach, deliver early, and measure often you’re actually working with (somewhat) objective guidance about whether to keep going or not. You can do the right things and avoid doing the extraneous things because you know if you’re heading in the right direction.
Chances are you may stop working on a particular outcome sooner because you know you’ve hit it, which means you’ll be able to start working on the next problem. And you’ll have produced fewer outputs in the process, meaning fewer things to maintain down the road.
What benefits and pitfalls have you seen?
I’ll admit I’m biased toward an outcomes based approach. So I’m curious. Are there benefits or pitfalls I’ve missed from looking through my outcome colored glasses? If so, please leave your thoughts in the comments.