My picks from OSCON keynotes

So OSCON came and went, and whilst I haven’t seen the record­ing for any of the ses­sions, the keynotes (and a bunch of inter­views) are avail­able on YouTube.

Unlike most con­fer­ences, the OSCON keynotes are real­ly short (aver­age 10–15 mins each) and hav­ing watched all the pub­lished keynote ses­sions here are my top picks.

 

Simon Wardley — Situation Normal, Everything Must Change

I’m a big fan of Simon’s work on val­ue chain map­ping, and his OSCON 2014 keynote was one of the most mem­o­rable talks for me last year.

swardley_13

Simon start­ed by point­ing out the lack of sit­u­a­tion­al aware­ness on the part of enter­prise IT. Enter­prise IT lives in a low sit­u­a­tion­al aware­ness envi­ron­ment that relies on back­ward causal­i­ty and ver­bal rea­son­ing (or sto­ry telling), and has no posi­tion or move­ment.

Where­as high-lev­el sit­u­a­tion­al aware­ness envi­ron­ments (e.g. mil­i­tary com­bat) are con­text spe­cif­ic, you have posi­tions and move­ments and usu­al­ly employ some form of visu­al rea­son­ing.

swardley_14

Mil­i­tary actions are dri­ven by your sit­u­a­tion­al aware­ness of the where and why, but in Busi­ness we have a tyran­ny of actions.

swardley_15

and this is where Simon’s val­ue chain map­ping comes in. With maps, you can add posi­tions to your com­po­nents based on the val­ues they pro­vide, as well as move­ment as they evolve from the unchar­tered world (chaot­ic, uncer­tain, unpre­dictable, etc.) to become indus­tri­al­ized.

swardley_16

In terms of method­ol­o­gy, there’s no one size that fits all.

Agil, XP and Scrum are very good on the left side (the unchar­tered world), par­tic­u­lar­ly when you want to reduce the cost of change.

On the right side, as things become indus­tri­al­ized, you want to reduce the cost of devi­a­tion and Six Sig­ma is good.

In the mid­dle where you want to build a prod­uct, lean is par­tic­u­lar­ly strong.

swardley_17

If you take a large scale project, rather than hav­ing a one-size-fits-all method­ol­o­gy, you can employ dif­fer­ent method­olo­gies based on where that com­po­nent is in its evo­lu­tion. For devel­op­ers, this is no dif­fer­ent to the argu­ments for poly­glot pro­gram­ming, or poly­glot per­sis­tence, because no sin­gle lan­guage or data­base is good for all the prob­lems we have to solve.

Why should the way we work be any dif­fer­ent?

swardley_18

By over­lap­ping the val­ue chain maps for dif­fer­ent areas of the busi­ness you can start to iden­ti­fy over­laps with­in the orga­ni­za­tion, and a snip­pet from his most recent post shed some hor­ri­fy­ing light on the amount of dupli­ca­tion that exists:

…To date, the worst exam­ple I know of dupli­ca­tion is one large glob­al com­pa­ny that has 380 cus­tomised ver­sions of the same ERP sys­tem doing exact­ly the same process…

The US air force dis­cov­ered that, as peo­ple came up with new ideas they tend to add fea­tures to that idea and made it bet­ter (and more com­plex); they then added even more fea­tures and made the idea so com­plex it’s com­plete­ly use­less to any­one, and that’s approx­i­mate­ly where they shipped it. (for reg­u­lar read­ers of this blog, you would prob­a­bly have read about this obses­sion of fea­tures many times before)


So Lt. Col. Dan Ward came up with FIST (Fast, Inex­pen­sive, Sim­ple and Tiny), which in his own words:

…FIST stands for Fast, Inex­pen­sive, Sim­ple and Tiny. It’s a term I use to describe a par­tic­u­lar approach to acqui­si­tions and sys­tem devel­op­ment. As you might guess, it involves using a small team of tal­ent­ed peo­ple, a tight bud­get, a short sched­ule and adher­ing to a par­tic­u­lar set of prin­ci­ples and prac­tices…

in oth­er words, small is beau­ti­ful, and it’s a theme that we have seen repeat­ed­ly — be it microser­vices, or Amazon’s two-piz­za teams, etc.

And as you impose con­straints on the teams — tight bud­get, short sched­ule — you encour­age cre­ativ­i­ty and inno­va­tion from the teams (some­thing that Kevlin Hen­ney also talked about at length dur­ing his Joy of Cod­ing clos­ing keynote).

 

How­ev­er, even with small teams, you still have this prob­lem that things need to evolve. For­tu­nate­ly we have a solu­tion to that too, in what is known as the three par­ty sys­tem where you have:

  • pio­neers — who are good at explor­ing the unchar­tered world
  • set­tlers — who are good at tak­ing half-baked ideas and make use­ful prod­ucts for oth­ers
  • town plan­ners — who are good at tak­ing a prod­uct and indus­tri­al­is­ing it into com­mod­i­ty and util­i­ty

swardley_20

Once you have a map, you can also start to play games and antic­i­pate change. Or bet­ter yet, you can manip­u­late the map.

You can accel­er­ate the pace of evo­lu­tion by using open prac­tices — open source, open API, etc. Or you can slow the process down by using patents, or FUD.

The key thing is that, once you have a map, you can see where things are mov­ing and visu­al­ly rea­son about why you should attack one com­po­nent over anoth­er. And that’s how you can turn busi­ness into sit­u­a­tion­al aware­ness first, and actions after.

As things move from prod­uct to com­mod­i­ty, they enable a new gen­er­a­tion of ser­vices to spawn up (won­der), but they also cause death to orga­ni­za­tions stuck behind the iner­tia bar­ri­er (death).swardley_21

This is a pat­tern that Simon calls War, Peace and Won­der, and is iden­ti­fi­able through weak sig­nal detec­tion and see rough­ly when these changes will like­ly hap­pen.

swardley_22

Simon fin­ished this bril­liant ses­sion with three lessons:

  1. if you’re a start up, have no fear for large cor­po­rates because they suck at sit­u­a­tion­al aware­ness;
  2. the future is awe­some, and pio­neers have already moved into the space of open hard­ware and open biol­o­gy;
  3. open source itself is chang­ing, we have new peo­ple com­ing in as new set­tlers

 

I hope you enjoyed Simon’s talk, his blog has much more infor­ma­tion and goes into each of these top­ics in a greater deal of detail. If you fol­low him on Twit­ter (@swardley) he also post reg­u­lar tit­bits of wis­dom, which I have start­ed to col­lect.

 

James Pearce — How Facebook Open Sources at Scale

…We use in pro­duc­tion what we open source, and we open source only what we use in pro­duc­tion…

- James Pearce

Nuff said 

 

Martin Fowler — Making Architecture Matter

I don’t like the term “soft­ware archi­tec­ture” as it sum­mons up these images of some senior per­son in an orga­ni­za­tion who’s set­ting rules and stan­dards on how soft­ware should be writ­ten but hav­ing actu­al­ly writ­ten any soft­ware for maybe 10 or 20 years. These archi­tects, Joel Spol­sky use the term “archi­tec­ture astro­nauts”, often cause a lot of prob­lems in soft­ware projects. So the whole term “archi­tect” and “archi­tec­ture” has a kin­da nasty taste to it.

- Mar­tin Fowler

For me, the key points from this talk are:

  • the notion that archi­tects shouldn’t code is wrong (or, don’t be an ivory tow­er archi­tect!)
  • archi­tec­ture is real­ly the shared under­stand­ing of the system’s design amongst its expert devel­op­ers
    • archi­tec­ture dia­grams are just (often imper­fect) rep­re­sen­ta­tions of this shared under­stand­ing
    • as soft­ware projects grow, what mat­ters the most is for you to ensure a good shared under­stand­ing between peo­ple lead­ing the project
  • archi­tec­ture is also the deci­sions that you wish you could get right ear­ly
    • your con­cern is there­fore the deci­sions that are hard to change, e.g. the pro­gram­ming lan­guage
  • com­bin­ing the two def­i­n­i­tions above, and you can think of archi­tec­ture as the “impor­tant things that I need to always keep in my head whilst I’m work­ing on the sys­tem”
  • when con­front­ed with requests for more fea­tures over qual­i­ty, don’t make the moral argu­ment of crafts­man­ship
    • when it comes to a bat­tle between crafts­man­ship and eco­nom­ics, eco­nom­ics always wins
  • a com­mon fal­la­cy is to think that soft­ware qual­i­ty is some­thing that can be trad­ed off for cost (like you do with cars or cell­phones)
  • soft­ware has both exter­nal (vis­i­ble to users) as well as inter­nal (good mod­u­lar­i­ty, etc.) and archi­tec­ture is about inter­nal qual­i­ty
    • what mat­ters with inter­nal qual­i­ty is the long term pic­ture
    • well main­tained code base gives you a plat­form to build upon and can make future devel­op­ment eas­i­er and faster
    • poor­ly main­tained code base makes it hard­er and hard­er for you to make changes to
    • this is why archi­tec­ture mat­ters!

 

Raffi Krikorian — Hacking Conway’s Law

Conway’s law has been a trendy top­ic at con­fer­ences this past 12 months, and every­one is basi­cal­ly singing the same tune — apply Conway’s law in reverse and orga­nize your com­mu­ni­ca­tion struc­ture to fit the soft­ware you want to build.

 

OSCON is coming to Europe!

At long last, we’ll see a ver­sion of OSCON in Europe this year, on 26th-28th Octo­ber in Ams­ter­dam. Some pret­ty cool tech com­pa­nies will be rep­re­sent­ed there — GitHub, DataS­tax, Google, Thought­Works, Pay­Pal, Heroku and Spo­ti­fy to name a few, and of course, our very own Gamesys 

I will giv­ing a talk on the work I did with Neo4j a while back, which you can read all about in this post.

p.s. Rachel Reese (of Jet) is com­ing over from the US and talk­ing about build­ing reac­tive ser­vice with F#!

 

Links