QCon London 2015–Takeaways from “The Bad Idea Terminator”

It’s very unchar­ac­ter­is­tic of me, but I went to a ses­sion on the prod­uct man­age­ment track at QCon Lon­don Melis­sa Per­ris’ “The Bad Idea Ter­mi­na­tor”. Hav­ing gone in the room with the expec­ta­tion of com­ing out not much wis­er, I was pleas­ant­ly sur­prised to find myself in one of the best talks at the con­fer­ence.

 

Melis­sa used Fire­stone and FourSquare as exam­ple of the “build­ing trap” where­by ail­ing com­pa­nies try to coun­ter­act their decline by adding more fea­tures with­out real­ly chang­ing they do things.

We often start off doing things right – we test and iter­ate on our ideas before we hit the mar­ket, and then we end up with some­thing that peo­ple want to use. But then we just keep on build­ing with­out going back to find­ing those inno­v­a­tive ideas that peo­ple love.

Image

The build trap stops us build­ing things that peo­ple love because we lose touch with our cus­tomers. We stop test­ing ideas with the mar­ket, and con­fine our­selves in our own bub­ble and just put our heads down and keep on build­ing.

 

We can fall into the build trap in a num­ber of ways, includ­ing:

  • pres­sure from stake­hold­ers to always release new fea­tures (Peter Hig­gs made sim­i­lar crit­i­cisms about mod­ern acad­e­mia where researchers are pres­sured to keep pub­lish­ing papers rather than focus­ing on find­ing the next big idea)
  • arbi­trary dead­lines and fail­ure to respond to change – set­ting dead­lines that are too far out and not being flex­i­ble enough to adapt to change
  • build­ing is work­ing” men­tal­i­ty – which doesn’t allow time for us to step back and think if we’re build­ing the right things

Build­ing is the easy part.

Fig­ur­ing out what to build is hard.

 

- Melis­sa Per­ri

 

Why don’t we take the time to think before we go and build some­thing? Well, the endow­ment effect might has some­thing to do with it – as you invest more and more into an idea and it starts to become part of your iden­ti­ty and it becomes hard for you to let go.

In behav­ioral eco­nom­ics, the endow­ment effect (also known as divesti­ture aver­sion) is the hypoth­e­sis that peo­ple ascribe more val­ue to things mere­ly because they own them. This is illus­trat­ed by the obser­va­tion that peo­ple will tend to pay more to retain some­thing they own than to obtain some­thing owned by some­one else—even when there is no cause for attach­ment, or even if the item was only obtained min­utes ago.

 

One of the most impor­tant respon­si­bil­i­ty of a prod­uct man­ag­er is to say NO to ideas, until we’re able to back it up with tests that prove an idea can work, and I think the same goes to devel­op­ers.

So how do you become the Bad Idea Ter­mi­na­tor, i.e. the per­son that goes and destroys all the bad ideas so we can focus on the good ones? We can start by iden­ti­fy­ing some com­mon mis­takes we make.

 

Mistake 1 : don’t recognize bias

Prod­uct ideas suf­fer from sev­er­al types of ideas:

  • Causal­i­ty – we attribute mean­ing and why things hap­pen to the wrong cause

For exam­ple,

We built a mobile app before and it was suc­cess­ful, let’s do anoth­er mobile app.

Every­one has a mobile app, so we need one too.

we need to rec­og­nize the dif­fer­ences between cus­tomers and busi­ness­es, what worked under one set of cir­cum­stances is not guar­an­teed to work under anoth­er.

  • Curse of Knowl­edge – as experts we can­not put our­selves in the shoes of some­one who doesn’t know as much

You should be doing user research and user test­ing, bring your ideas to the cus­tomers and see if that’s what they real­ly want.

  • Anchor­ing – we focus on insignif­i­cant data because it’s the first data we see

When­ev­er some­one says some­thing like

All my cus­tomers are ask­ing for this!

you should always ask for data, and make peo­ple prove what they’re say­ing is accu­rate.

 

Mistake 2 : solutions with no problems

When peo­ple sug­gest new ideas, most of the time they come to the table with solu­tions. Instead, we need to start with the WHY, and focus on the prob­lem that we’re try­ing to solve.

On the top­ic of start­ing with the why, I also find Simon Sinek’s TED talk inspi­ra­tional, and he also has a book on the same top­ic.

image

There are always mul­ti­ple ways to solve the same prob­lem, and only by focus­ing on the prob­lem would we be able to decide which solu­tion is the best. Unfor­tu­nate­ly, your idea is not always the best idea, and we should be con­scious of the Not Invent­ed Here syn­drome and our propen­si­ty to fall under its influ­ence (even if only at a sub­con­scious lev­el).

After we fig­ure out the prob­lem we still need to align it with our busi­ness goals, and decide if it’s a prob­lem we can solve and want to solve.

 

Mistake 3 : building without testing

When we get stuck in the build trap we don’t tend to test our assump­tion, as we tend to com­mit to one solu­tion too ear­ly. Instead, we should solic­it many solu­tions at first, and get peo­ple off the fix­a­tion on the one idea.

We also tend to invest too much into the one idea and then have trou­ble let­ting go (endow­ment effect again).

Instead, we should pick out a few ideas that are the most viable and then test them to find the ideas that:

  • have the most pos­i­tive cus­tomer feed­back, and
  • require the small­est invest­ment

using small, data-dri­ven exper­i­ments. Tech­niques such as A/B test­ing falls right into this (but remem­ber though, A/B test­ing doesn’t tell the whole sto­ry, you prob­a­bly also want A/A test­ing to act as blind test group). It could also be as sim­ple as talk­ing to a few cus­tomers to get their feed­backs.

There are 3 key exper­i­ments you should run:

  1. do the cus­tomers have this prob­lem?
  2. are they inter­est­ed in solu­tion ideas?
  3. are they inter­est­ed in our solu­tion?

 

Mistake 4 : no success metrics

Anoth­er com­mon mis­take is to not set suc­cess met­rics when we go and do exper­i­ments, and we also don’t set suc­cess met­rics when build­ing new fea­tures.

Instead, we should set goals ear­ly. Doing so ear­ly is impor­tant because if we set up goals in hind­sight then we’ll just change the goals to make the fea­ture look good…

We should be ask­ing ques­tions such as

How much val­ue do we need to cap­ture to make this fea­ture worth build­ing?

We also need to accept that lean­ing on its own is also a form of suc­cess.

 

The risk of con­tin­u­ing with a bad idea is real­ly great, so the ear­li­er we can kill of these bad ideas the low­er our risk will be.

And the fast you kill the bad ideas, the more time you will have to devote to the good ones.

Fail fast, so you can suc­ceed faster.

 

- Melis­sa Per­ri

and finally,an oblig­a­tory pic­ture of ter­mi­na­tor of course!

Image

 

I real­ly enjoyed Melissa’s talk, and although I’m a devel­op­er, I believe every­one inside an orga­ni­za­tion has the respon­si­bil­i­ty to ask ques­tions and help push the orga­ni­za­tion towards build­ing bet­ter prod­ucts that actu­al­ly match its customer’s needs.

 

Hav­ing read a cou­ple of Dan Ariely’s books in the past year, I find they pro­vide a very insight­ful back­drop on many of the human irra­tional­i­ties that underlies/causes us to make the com­mon mis­takes that Melis­sa has iden­ti­fied in her talk.

image   image

 

Links

Slides for the talk

Simon Sinek – Start with Why TED talk

Start with Why : how great lead­ers inspire every­one to take action

Pre­dictably Irra­tional : The Hid­den Forces That Shape Our Deci­sions

The Upside of Irra­tional­i­ty