Within a growth lever that we are focusing on, how do we determine what initiatives to focus on and which to not pay attention to?
One of the easiest ways is to apply a simple scoring method in order to force-rank your roadmap.
ICE stands for impact, confidence, and effort.
It is intentionally designed to be a ratio of Impact over Effort, surrounded by confidence.
This is because you want to be optimizing for maximum impact to your target metric, per amount of effort spent. Confidence underscores Impact and Confidence because with low confidence, the impact could be much lower than expected or the effort could be much higher than expected.
How to put I.C.E. into practice
In many cases, people only think of the impact, and not the required effort, and in even more cases, people don’t consider confidence at all.
Impact is the amount we believe we can improve the target metric.
- We grade impact on a simple 1-5 scale. What is important is that we are grading relative to other items on the list, since we don’t know what the actual impact will be, but that isn’t important for a relative-scoring exercise.
Effort is the number of weeks of a person’s time.
- We grade up by half-weeks with no maximum. Effort estimation isn’t intended to be exact, but again, for relative-ranking, it is only important to be somewhat accurate.
Confidence is derived from the source of the information.
- Confidence can have a score of 50%, 75% or 90% (we’re never 100% confident). A 90% means that we have hard data, strong comparables, or direct expertise which backs up our belief in the impact score. A 75% means that we have indirectly related data, or evidence to support our scoring. 50% means that we could be right or wrong, we don’t have significant data to support or disprove our scoring.
Confidence truly is key
Confidence is the most overlooked of the factors, but is arguably the most important. An easy way to understand confidence is to think about information sources for things like scientific discoveries, news publications and others – your confidence in that information relies on the credibility of the source and how well it is corroborated.
There are a number of idea sources of information, with differing levels of confidence.
Most people are familiar with brainstorming, which is actually the lowest confidence idea source. The equation that we consider is that confidence is made up of a combination of data and relevancy.
We at MasterClass ensure that we have a diverse backlog from a number of different information sources with high confidence.
- We do teardowns of 60 comparable companies cross Entertainment, Education, and Consumer Tech verticals.
- We survey subject matter experts and give them free access to the product.
- We do on-site polls and ask users directly what feature in the backlog would be most beneficial towards their purchase decision.
These are just a few of our most practiced idea sources, but in reality we do them all.
Using I.C.E. to drive focus across initiatives
And so, the equation is the ratio of impact over effort, factored by confidence.
We can then use the value score to sort the backlog from high –> low score.
- Then we take the top ¼ of the backlog and those are things we double down on.
- The next ¼ we continue to evaluate and invest in quick wins.
- The bottom half are things that we determine that we aren’t going to focus on for the quarter.
The techniques we’ve discussed are fairly simple, but have a huge impact on the strength of a backlog – 1.) Applying an impact/effort ratio, 2.) Leveraging idea sources that have high confidence, and 3.) Using a value score to determine what you should invest in, continue evaluating, and not do.
In the next post in this series I’ll discuss core growth levers to evaluate and the basic structure of a Growth Model!
