Case Study: How stack ranking made Animoto rethink the problem they were solving

The discovery phase is the most impactful part of any new project; it sets the direction and locks in the insights that will shape the product that your teams builds. But what happens when, after months of user interviews, you realize that what people told you was their biggest problem isn't actually their top priority at all? Well that's exactly what happened to Micah Rembrandt during the summer of 2021...

Micah is a Senior Product Manager at Animoto, the easy-to-use video creator with over 130,000 paying customers around the world. Even though Animoto is a giant in the video marketing space, Micah has been leading a discovery research effort to understand more about the broader video production process and the pains that people feel throughout it.

Team Animoto on tour ⛵

Micah and his team started off by talking to marketing people about the video production process and quickly found a bunch of technical challenges they faced, many of which centred around video editing. Animoto is a data-driven company though, and as Micah explained...

"Data is more telling than people's opinions. Stack ranking as a methodology is more rigorous than straight-up asking." — Micah Rembrandt, Senior Product Manager at Animoto.

When people hear about stack ranking, they picture a survey question with 5 or 6 options that they drag into order, or a workshop with a bunch of post-it notes ranked in order on a wall. Using these types of stack ranking come with a bunch of limitations though, which is what has prevented Micah from using them in the past.

"In the past, stack ranking has been very hard to analyze — especially with bigger audiences. Giving people a list of 10+ things to rank is overwhelming and leads to unreliable data. Doing it all manually leaves me dealing with the complex math to summarize the results." — Micah Rembrandt, Senior Product Manager at Animoto.

Instead of those traditional ranking methods, Micah decided to try OpinionX — which uses a different method of stack ranking where participants are shown two options at a time and they pick the one that is more important to them. For Micah, the OpinionX approach wasn't just easier for his research participants, it also gave him much more robust comparative data.

As the final step of his initial discovery phase, Micah created a stack ranking survey on OpinionX to validate his findings with hard data. He added the top 12 problem statements to his survey and sent it out to 52 people that fit his target customer profile. Pretty quickly, Micah realized that the results were not coming out as expected...

"All of our other data was pointing to stuff at other points in the journey. We had no data that getting approval was hard — yes people have to do it, but they just send an email or something. When we ran our OpinionX, it came back as the most frustrating part for people. The data really illuminated that it wasn't a technical issue, it was a people issue. It reformatted how we thought about our whole approach; people don't struggle with the tech for giving feedback, they struggle with giving others their feedback." — Micah Rembrandt, Senior Product Manager at Animoto.

^ What stack ranking results look like on OpinionX (it's super simple!)

Micah and his team decided to take this new insight back into user interviews to explore what it meant to people and to figure out why it hadn't appeared more prominently in their previous research.

"Through our follow up research, this problem kept coming up as something that's really painful for people. It's hard to ask people the straight-forward question of what they want, because people don't always know how to necessarily articulate what they want. The underlying needs apart from what people were articulating were what stack ranking helped us to find." — Micah Rembrandt, Senior Product Manager at Animoto.

The most surprising part about Micah's story is that he never intended to include a stack ranking step in his research; in fact, he fully intended on skipping it.

"I wouldn't have done the stack ranking without OpinionX. I have quantitative skills but I'm not a data analyst and my team didn't have access to one for this part of our process. Analyzing manual stack ranking is just too much to take in, in my experience, so we wouldn't have done it given the scope and timing of this project. Who knows where this project would have ended up if we didn't know about OpinionX..." — Micah Rembrandt, Senior Product Manager at Animoto.

Companies love to say that they're data-driven; product analytics give them the numbers they need to optimize conversion, tweak onboarding, and improve retention. But when it comes to what they build next and how they manage their product roadmap, decisions are made using internal assumptions, back-of-napkin prioritization maths, and 'product intuition.'

Join the hundreds of product teams like Micah's that are using OpinionX to make product decisions using real data, by ranking users' priorities during product discovery and roadmap prioritization (like Kristina at Gnosis Safe). Create your own stack ranking survey for free in under 5 minutes today with OpinionX. Get started here.