You can measure hundreds of different metrics, but they will get you nowhere near making good product decisions. By picking a North Star metric, however, you can focus on what helps you build a foundation for data-based decisions that matter. A North Star (or master) metric will very quickly give you the answer to a basic question: is the product evolving according to your expectations, or not?
I’ll describe a simple framework that helps our clients find a master metric for building better products that reflect what users do and what decisions they make inside the app.
Invite stakeholders and at least one technically versed person. Mobile analytics requires experience and the terminology can sometimes overwhelm the stakeholders. Having someone who gets analytics and can translate complex terms into layman’s words is an asset.
Master metric meetings can be held remotely, so you can use Miro (or a similar tool) to organize thoughts.
You start the meeting and set up a goal. In this case, it should be “looking for the master metric that currently matters.”
To better explain the goal, describe what is a good metric and what is a North Star metric if it’s not obvious to all people at the meeting (e.g., not every developer has experience in defining metrics, hence I recommended inviting someone versed in analytics to the meeting).
A good metric lets you measure the usage of a specific feature with direct value to the user.
However, the definition of usage itself can get tricky and deceitful if you don’t understand it correctly. For example, metrics such as the number of sign-ups or the total number of registered users aren’t really going to help you build a better product because they give you neither valuable nor actionable insight.
If you make metrics more granular, say, the number of new users per week or how many times a day/week a user opens your app, you can correlate the results with some other action (e.g., change in design, reduction of the number of steps in a process). In other words, you can get a learning opportunity.
Now the even more granular “percentage of users who do the same thing inside your app X number of times a day/week” is a metric that influences decision-making the most.
With all that in mind, a definition of a North Star metric emerges: every other metric you measure means nothing until this master metric hits a predefined goal.
Now that everyone is on the same page as to the qualitative aspect of metrics, we’re going to focus on bringing to light everything we know about the most important user persona in the current state of the product. Master KPI will refer to this group of people.
Have your meeting attendees work together in a group to find a North Star metric. Set a time limit of ten minutes, for example. Every person works separately, without seeing the results until time runs up.
Use post-it notes in Miro for this step.
When time is up, talk over your results.
When the master metric exercise is complete, all stakeholders should think about the limitations that make a certain metric unsuitable to become a master metric. The constraint can also be that the metric is simply unmeasurable.
Once everyone has learned about the ideas of other team members and business constraints, we repeat the ideation process for the master metric.
The iteration should look like this:
The technical feasibility of a metric also justifies the presence of a technical person — a developer with the knowledge of analytics will be able to tell how difficult is the implementation of that idea or if it’s even possible.
This sifting process should be exhausted to the point of giving you a list of validated metrics.
There’s little room for democracy when voting for a master metric — one designated person has to be the decision-maker. Every stakeholder gets 2 to 3 dots to use for voting on the best master metric. Keep in mind that the number of dots a metric gets is only for the informational purpose of the decision-maker.
It will be up to that person to ultimately make the decision which metric becomes the master metric.
Sample Miro board with the whole process:
A North Star metric is a way to focus on many different measuring options, which is great for successful product development that’s guided by predefined goals. But you have to keep in mind that a North Star metric is valid for a set period of time — once you achieve the predefined metric goal, you should find a new North Star metric to avoid the problem of local optimization. The key with North Star metrics is to find a global optimum. So without working on at least a few North Star metrics, your product might not mature properly.
Having a North Star metric is important, but it doesn’t mean that the one you pick initially should guide your product’s journey forever — there’s a high likelihood that that first North Star metric will change.
Don’t get discouraged by the results with the first North Star metric you define. Choosing that first metric is the most difficult part of the process. The data environment isn’t yet saturated enough for you to make highly accurate decisions — don’t worry about that, just pick the first metric that seems good and iterate to set the baseline for further measurements.
Choosing a North Star metric is only one part of the process — the other is to assume what’s the satisfactory target level of that metric.
You can think about it as a kind of business gambling — you assume that a metric changes by X when you do Y. This approach is the quintessential element of scientific research and the foundation for making hypotheses and experiments.
If you don’t define a target level for your North Star metric, there’s a chance that its role in your product’s success will be blurred (you won’t know if it was the North Star metric that had a direct impact). And in a scenario where your product fails, you’ll try hard to rationalize it.
Thinking about how much you can squeeze out of a given metric before you actually start the implementation takes away the ability to rationalize post-factum.
You can use the Lean Analytics Cycle diagram to find and validate North Star metrics, plus transition from one metric to another:
Again, finding the first North Star metric is the hardest part of the process because you don’t have any points of reference in your data. But pay no mind to that and try to make the best shots possible. This is an iterative process. Once data streams in and you run more experiments, you’ll get better at picking more accurate master metrics.