How we Measure Product Market Fit (PMF) at Sutro

7 min readOct 4, 2021

How do you know what to build? Where to prioritize? What direction to steer the ship? Do people like what you’re building?

Rahul Vohra from Superhuman and the PMF Survey

Jarie Bolander (from the Entrepreneur’s Ethos) and I were scratching our collective head(s) as we started to build Sutro in its early days. We had just been acquired by Sani Marc — and we needed an architectural blueprint on what we should market and what features really drive a user to latch onto Sutro.

I wrote a separate blog post in the early days of the beginnings of this experiment. Wow, we’ve come a long way!

Enter Rahul Vohra.

I stumbled across a post from First Round Capital that Rahul wrote on how he measures Product Market Fit at Superhuman. Mind you, I’m a super big fanboy of Rahul and his early work at Rapportive (which was subsequently acquired by LinkedIn).

I’ll quickly distill what the posts says, and how we use it at Sutro.

Rahul was on a quest to find a leading indicator of how to measure Product Market fit at Superhuman, and leaned on the learnings of Sean Ellis, who looked at over 100 startups.

He writes:

The product/market fit definitions I had found were vivid and compelling, but they were lagging indicators — by the time investment bankers are staking out your house, you already have product/market fit. Instead, Ellis had found a leading indicator: just ask users “how would you feel if you could no longer use the product?” and measure the percent who answer “very disappointed.”

Rahul asked four questions of his users at Superhuman:

How would you feel if you could no longer use Superhuman? A) Very disappointed B) Somewhat disappointed C) Not disappointed
What type of people do you think would most benefit from Superhuman?
What is the main benefit you receive from Superhuman?
How can we improve Superhuman for you?

How do we use it at Sutro?

We basically copied the same survey and changed the word Superhuman to Sutro.

The questions now read:

How would you feel if you could no longer use Sutro? A) Very disappointed B) Somewhat disappointed C) Not disappointed
What type of people do you think would most benefit from Sutro?
What is the main benefit you receive from Sutro?
How can we improve Sutro for you?

We needed to figure out when to send the survey, and I leaned on some of the learnings from Dave Bailey.

In a post that Dave wrote back in 2019, there were a series of e-mail templates and when you should send these templates to users. Dave said 15 days after a user signs up, he sends the Product Market Fit survey.

We followed Dave’s recommendation and we send this Typeform 2 weeks after the user purchases a Sutro.

*I should state that there was a bit of an issue with the 2 week timeline, as during the Summer we had a delay in shipping, and ended up irritating people that we were sending these surveys before they had received the product. We are going to change this to trigger 2 weeks after they’ve set up the device.

Here are the questions as of Oct 3, 2021 (incase we change them).

These next two questions we threw in after I took a survey from Readwise.

(As a side note, I would 100% recommend any founder to subscribe to Dave’s newsletter. The newsletter is super thoughtful and really timely).

Analyzing the data

It’s pretty simple to set up, and run an automation through Autopilot, Mailchimp, or Klaviyo.

This is where the fun part begins.

You select for the folks that would be very disappointed, as this is group that you want to target and build the product for. Don’t worry about the somewhat disappointed or not disappointed right now. The current MVP that you have probably is not exciting enough or the market demographic doesn’t match with the demographic you envisioned / should be targeting. As Marc Andressen of A16Z says:

“…the life of any startup can be divided into two parts — before product/market fit and after product/market fit.” He goes on to write: “When you are BPMF, focus obsessively on getting to product/market fit. Do whatever is required to get to product/market fit. Including changing out people, rewriting your product, moving into a different market, telling customers no when you don’t want to, telling customers yes when you don’t want to, raising that fourth round of highly dilutive venture capital — whatever is required.”

You want to target to reach 40% of very disappointed cohort. Build products that get more folks into that territory. Do whatever it takes to get to product/market fit.

Digging into each of these questions and how we use them

How would you feel if you could no longer use Sutro? A) Very disappointed B) Somewhat disappointed C) Not disappointed

This question helps you select the ‘very disappointed’ cohort to analyze the rest of the data around.

What type of people do you think would most benefit from Sutro?

People answering this question basically describe themselves. This helps us with demographic analysis and figuring out which descriptors people most identify with.

This gets circulated to our web, digital, SEO, and email team(s). To help build content with the language that our users are most familiar with (at the end of the day, it’s how they describe themselves).

What is the main benefit you receive from Sutro?

This is what we’re doing right. We need to keep doing this for the cohort that is very disappointed. Because this builds word-of-mouth, this is why people buy the product.

How can we improve Sutro for you?

This is what we need to improve. With a bit of gut, analyzing customer service data, and what the market is saying — we’re able to get a signal on the right way to build the best product. This is what goes into our OKRs and what drives the engineering and product teams.

Looking at the data

Here is our August data.

We’re about 9% away from hitting the 40% product market fit number. A few things to note on here.

On the second question — busy lifestyle and inexperienced pool owners was a little off from what we initially were marketing. We were targeting the pool geeks, and in-fact we should be targeting folks that don’t know much about their pools or water chemistry.

The fourth question, what we need to work on is 90% similar to what we initially thought we should be working on. A simmering of the soup here allows us to see what we can do better and what other things we can make. Which moves to what people would be willing to pay more for, potentially.

I was speaking to Adam Levinter a while ago (the author of the Subscription Boom), and the interesting thing here is we can actually build in higher tier subscription offerings for these ‘pro’ features.

We also do not just look at this in isolation. This is combined with a weekly stand-up that we have with the RevOps team to see if there is any ‘on-the-ground’ feedback that we’re hearing from the sales, digital, and customer experience team(s).

Getting buy-in at Sutro and Sani Marc

Post acquisition by Sani Marc, I have two separate audiences to orient and guide whenever we’re making any changes at Sutro. The first is obviously the internal Sutro organization. This was pretty easy, as I set an objective for 2022 that we need to reach 40% or higher PMF. These were stuck into our OKRs, and we review them bi-weekly (every 2 weeks). It was a little different explaining and educating the Sani Marc folks, as it’s a different framework to an established large company that has been selling a successful product for more than half a century!

Tying this to budgets

I’ll write a separate post on how we’re doing 2022 budgets, a TON of learning there. But for now- you can build a mapping between the features you should be building and the ROI associated with them.

You can make hypotheses against how much more revenue you can generate if you had a particular feature, and how much cost you’d save on returns, CS tickets, RMAs. Not only will you build a stickier product — but you can actually go to your management board with a clear ROI game plan on how you think this will impact your bottom line.

A lot more coming

This is just the tip of the iceberg, and I think I’ve probably jammed a bit too much content into this post. I’ll be writing a lot more about how we’re utilizing PM Fit and steering Sutro to the hockey stick growth come 2022 and onwards.