Published 10 Mar, 2021 6 minutes of reading
Monthly users, app downloads, page views, shares. These numbers are fun to look at––especially when they keep growing––but how much are they telling you? Vanity metrics can be exciting and make you look good, but they’re not necessarily informative in the long run. If our goal is to make better decisions about the products and services we make in order to grow our business, there’s better data we should look at.
In the bustling and ever-changing digital world, our focus should be all about retention. We want our users to find us, but more importantly we want them to engage with our products and stay. Yes, maybe you had 15,000 new subscribers this month, but how many of them were still there after 3 days? After a week? Did your campaign to get the new subscribers have any value for them? Again, it’s all about retention. Which brings us to cohort analysis.
What is cohort analysis?
First, let’s clarify what a cohort is. In behavioral analytics, a cohort is a group of users separated by similar traits or actions (e.g., visiting your site for the first time; downloading an app). Creating cohorts is a method of dividing your users in order to truly understand their engagement and measure the efficiency of your product. In short, a cohort can help you understand how well you’re handling customer retention.
The term cohort dates as far back as the days of the Roman Empire, and began to see more widespread use in demographic literature, usually specifying people by the year in which they were born.
Today, digital product managers can harness the power of analyzing cohorts in a variety of ways to paint a much clearer picture of how their customers engage with the products they build. This allows product managers to design better testing scenarios, iterate product development with clearer roadmaps, and make better decisions for products that people value and care about.
How to segment users into cohorts
There are two main ways in which you can segment your users into cohorts.
Acquisition cohorts
Acquisition cohorts are determined by when users first begin using your product. For example, when they first visit your site.
Behavioral cohorts
Behavioral cohorts are determined by user actions with your product. These actions could be a download, transaction, installation, etc.
Once you have your cohorts, you can then see the percentage of users that come back to your product or site, or perform a certain action over a period of time.
If you see your retention is lower than desired based on your specific goals, you can then analyze and pinpoint where your weak points are, and test changes between cohorts.
Consider the size and source of your cohort
With either acquisition or behavioral cohorts, you’ll want to choose how you measure a cohort by considering the type, size or maturity of your product. In Google Analytics 4 for example, you can measure user acquisition based on day, week, month, or any period of time that’s suitable for you.
Do your users respond better to the same offer in a weekly promotion versus a monthly? How are your users behaving with a particular feature this month compared to the last? What period of time do we need to observe to make a meaningful decision about this aspect of our product? Depending on the scope you choose, cohort analysis allows you to answer questions like this, gain insight, and identify patterns you can act on.
You can also create cohorts according to their sources. If you have an ad running on Google, LinkedIn, or Facebook, you can create cohorts based on these sources and then, for example, figure out which one provides the most user retention. You could get the most initial registrations from Facebook, but maybe the registrations through Google are what provide the quality interactions.
Cohort analysis examples
With the basics out of the way, let’s stop telling, and start showing cohort analysis in action with some examples.
Retention over user & product lifetimes
Let’s look at the users of a transactional website, where they can send and receive payments. The horizontal line in the graph below shows the daily retention of users over time in one specific cohort. The vertical line shows the site’s performance over its lifetime.
You want the percentages along the horizontal line to decrease as slowly as possible as time goes on. Achieving this means achieving a high retention rate. This example shows week by week comparisons, and the scope of your analysis will depend on the goals you set for user engagement.
As we observe the cohorts over time along the vertical line, we want to see not only the total number of users grow, but also see an increase in user retention across cohorts during week 1, 2, etc.
Let’s say the site offers a free trial during the first week. This means that the retention rate on Week 1 would be critical. Understanding the context of what we’re measuring lets our cohort analysis answer simple, valuable questions such as, “Are users willing to pay for this product?”
There are plenty of ways to visualize the data as well. Below, you can see the analysis through a line graph. This is a useful way to view one specific cohort, or compare them.
Another example: cohort analysis can tell us stories
Ok, now let’s shift our focus and see what stories our cohort analysis can tell us. Can you spot two significant moments reflected in this graph?
If we look at the first row, we can see that user retention is very low. In Week 1 it drops to 5.2% and then another staggering drop in Week 2. And pretty much the same pattern for the second cohort (Jan 11 - Jan 17).
But then, something happens. Perhaps this business realized retention was their biggest issue and needed to act fast. Let’s say the transactional site experimented with a reward system to incentivize users to return and keep making transactions. Then, they waited to see how this strategy worked for the next cohort (Jan 18 - Jan 24). The total users increased a little bit, but the retention rate went up. Instead 5.2% from the first cohort, it’s now at 24.2%, and if we look at Week 4 it’s now at 21.9% compared to just a 0.5%.
Now, look at total users from the next cohort (Jan 25 - Jan 31) – wow! 22,000 users compared to 8,000 the week before. The company probably saw that they increased their retention on the site, and knew it was time to then increase their users. Maybe they told their current users they’d receive a credit if they refer their friends. Whatever it was, it worked.
Now, let’s say they had switched their campaigns. Instead of running the campaign with the referral reward program for their users, they first announced the referral program. Maybe users at Week 0 would have increased at the same rate, but then would have dropped during Week 2 and after a month they wouldn’t be using the site anymore. Examples like this show why retention is so important, and how you need to be able to follow the story of your product, and of your retention with cohort analysis.
Cohort analysis and Google Analytics
The cohort analysis report is one of the most underrated reports on Google Analytics, and with the launch of Google Analytics 4, the reporting continues to improve. Prior to GA4, cohort analysis reports could only define cohorts based on acquisition date, but now you can define a cohort based on a myriad of other characteristics (age, gender, language, city, etc.).
Here at Modyo, we’re excited about taking advantage of the new features of Google Analytics 4, so much in fact that we wrote a post about it. Our platform already integrates with GA across a variety of projects. Stay tuned for more information in the future about how you can take advantage of cohort analysis in Google Analytics by signing up for our newsletter.
Use cohort analysis to gain control of your user retention.
Google Analytics and other tools, like Mixpanel or Amplitude, help us at Modyo measure the performance of digital products in many amazing ways. But information overload can easily lead you down a path where information starts to cause a loss of focus.
When it comes to user retention, cohort analysis is a fantastic “shortcut” to cut through the noise and find the signal you’re looking for. It’s a quick path toward understanding how engaging your product is.
Are you working at a company or on a product that could benefit from cohort analysis? It’s just the tip of the iceberg in terms of the tools at your disposal, and we’d love to talk to you about your ideas. If you’re not working on a product per se, but are in search of other solutions as specific as digital onboarding or as ambitious as a complete omnichannel platform, then check those out.
Thanks for reading, and cheers to building better digital products.
Photo by Ryoji Iwata on Unsplash.
