QCon London 2015–Takeaways from “The Bad Idea Terminator”

It’s very uncharacteristic of me, but I went to a session on the product management track at QCon London – Melissa Perris’ “The Bad Idea Terminator”. Having gone in the room with the expectation of coming out not much wiser, I was pleasantly surprised to find myself in one of the best talks at the conference.

 

Melissa used Firestone and FourSquare as example of the “building trap” whereby ailing companies try to counteract their decline by adding more features without really changing they do things.

We often start off doing things right – we test and iterate on our ideas before we hit the market, and then we end up with something that people want to use. But then we just keep on building without going back to finding those innovative ideas that people love.

The build trap stops us building things that people love because we lose touch with our customers. We stop testing ideas with the market, and confine ourselves in our own bubble and just put our heads down and keep on building.

 

We can fall into the build trap in a number of ways, including:

• pressure from stakeholders to always release new features (Peter Higgs made similar criticisms about modern academia where researchers are pressured to keep publishing papers rather than focusing on finding the next big idea)
• arbitrary deadlines and failure to respond to change – setting deadlines that are too far out and not being flexible enough to adapt to change
• “building is working” mentality – which doesn’t allow time for us to step back and think if we’re building the right things

Building is the easy part.

Figuring out what to build is hard.

 

– Melissa Perri

 

Why don’t we take the time to think before we go and build something? Well, the endowment effect might has something to do with it – as you invest more and more into an idea and it starts to become part of your identity and it becomes hard for you to let go.

In behavioral economics, the endowment effect (also known as divestiture aversion) is the hypothesis that people ascribe more value to things merely because they own them. This is illustrated by the observation that people will tend to pay more to retain something they own than to obtain something owned by someone else—even when there is no cause for attachment, or even if the item was only obtained minutes ago.

 

One of the most important responsibility of a product manager is to say NO to ideas, until we’re able to back it up with tests that prove an idea can work, and I think the same goes to developers.

So how do you become the Bad Idea Terminator, i.e. the person that goes and destroys all the bad ideas so we can focus on the good ones? We can start by identifying some common mistakes we make.

 

Mistake 1 : don’t recognize bias

Product ideas suffer from several types of ideas:

• Causality – we attribute meaning and why things happen to the wrong cause

For example,

We built a mobile app before and it was successful, let’s do another mobile app.

Everyone has a mobile app, so we need one too.

we need to recognize the differences between customers and businesses, what worked under one set of circumstances is not guaranteed to work under another.

• Curse of Knowledge – as experts we cannot put ourselves in the shoes of someone who doesn’t know as much

You should be doing user research and user testing, bring your ideas to the customers and see if that’s what they really want.

• Anchoring – we focus on insignificant data because it’s the first data we see

Whenever someone says something like

All my customers are asking for this!

you should always ask for data, and make people prove what they’re saying is accurate.

 

Mistake 2 : solutions with no problems

When people suggest new ideas, most of the time they come to the table with solutions. Instead, we need to start with the WHY, and focus on the problem that we’re trying to solve.

On the topic of starting with the why, I also find Simon Sinek’s TED talk inspirational, and he also has a book on the same topic.

There are always multiple ways to solve the same problem, and only by focusing on the problem would we be able to decide which solution is the best. Unfortunately, your idea is not always the best idea, and we should be conscious of the Not Invented Here syndrome and our propensity to fall under its influence (even if only at a subconscious level).

After we figure out the problem we still need to align it with our business goals, and decide if it’s a problem we can solve and want to solve.

 

Mistake 3 : building without testing

When we get stuck in the build trap we don’t tend to test our assumption, as we tend to commit to one solution too early. Instead, we should solicit many solutions at first, and get people off the fixation on the one idea.

We also tend to invest too much into the one idea and then have trouble letting go (endowment effect again).

Instead, we should pick out a few ideas that are the most viable and then test them to find the ideas that:

• have the most positive customer feedback, and
• require the smallest investment

using small, data-driven experiments. Techniques such as A/B testing falls right into this (but remember though, A/B testing doesn’t tell the whole story, you probably also want A/A testing to act as blind test group). It could also be as simple as talking to a few customers to get their feedbacks.

There are 3 key experiments you should run:

1. do the customers have this problem?
2. are they interested in solution ideas?
3. are they interested in our solution?

 

Mistake 4 : no success metrics

Another common mistake is to not set success metrics when we go and do experiments, and we also don’t set success metrics when building new features.

Instead, we should set goals early. Doing so early is important because if we set up goals in hindsight then we’ll just change the goals to make the feature look good…

We should be asking questions such as

How much value do we need to capture to make this feature worth building?

We also need to accept that leaning on its own is also a form of success.

 

The risk of continuing with a bad idea is really great, so the earlier we can kill of these bad ideas the lower our risk will be.

And the fast you kill the bad ideas, the more time you will have to devote to the good ones.

Fail fast, so you can succeed faster.

 

– Melissa Perri

and finally,an obligatory picture of terminator of course!

 

I really enjoyed Melissa’s talk, and although I’m a developer, I believe everyone inside an organization has the responsibility to ask questions and help push the organization towards building better products that actually match its customer’s needs.

 

Having read a couple of Dan Ariely’s books in the past year, I find they provide a very insightful backdrop on many of the human irrationalities that underlies/causes us to make the common mistakes that Melissa has identified in her talk.

  

 

Links

Slides for the talk

Simon Sinek – Start with Why TED talk

Start with Why : how great leaders inspire everyone to take action

Predictably Irrational : The Hidden Forces That Shape Our Decisions

The Upside of Irrationality