CraftConf 15–Takeaways from “Beyond Features”

This was one of my favourite talks at the conference and having seen quite a few of his talks Dan North is always a safe bet!

 

Rantifesto about Agile Methodology

The talk started with a Rantifesto about the state of Agile.

When the Agile manifesto was conceived in 2001, it made a set of value statements of things that we care about:

image

However, as of 2015, we as an industry have come to demonstrate with our behaviours a different set of value statements:

  • Processes and tools over  individuals and interactions
  • Comprehensive documentation over  working software
  • Contract negotiation over customer collaboration
  • Following a plan over adapting to change

because

“Culture eats strategy for breakfast”

– Peter Drucker

i.e. you can have the best intentions (manifesto) but unless you change your system of work (methodology) it will not make the tiniest bit of impact.

 

Whilst supporters of the agile methodologies would say that they optimize for quality, feedback, communication, collaboration, etc. But above all, agile methods are optimized for predictability, which was something that was seriously lacking when the agile methodologies came out in the 90s.

Michael Nygard also touched on this in his Archi­tec­ture with­out an End State talk that, in big enterprises there were these cycles of 3 year plans :

  • organization brings in a new CIO who instils a 3-year plan to revolutionize the company’s IT organization
  • the business and market has changed in the meantime but the plan forged ahead
  • and a new CIO comes in and quietly kills off the previous regime’s plan and instils his own vision of a 3-year plan

 

We base Software Engineering on Civil Engineering

After bashing on agile methodologies a few more rounds (and it was fun to watch Winking smile) Dan hypothesized that, perhaps our obsession with predictability stems from the fact that for so long we have based software engineering on civil engineering.

As a young industry, where the majority of us are 1st generation software engineers, we have been making things up as we go and we look to civil engineering for metaphor. I think this highlights two issues that we should address as an industry.

Mentorship programmes

As a whole, an alarmingly small percentage of developers stay in the industry for more than 10 years, in part thanks to a lack of career progression at most companies. Those few brilliant minds, who have been in the industry for decades and with valuable experiences to share, are often locked up inside companies such as Google and Microsoft and are rarely available to the general public.

The internet is a great place to find answers to just about anything you can think of, but it’s also full of noise and bad advice, not to mention a lot of prejudice and judgement. I have found conferences to be a great place to meet and speak to people who are genuinely smart, open-minded, and whose opinions came from a wealth of experience. But, attending conferences requires significant investment of time and finance, and the people you need to speak to about particular problems aren’t always available.

Some form of mentorship programmes would help pass on the knowledge and experience to younger generations and maybe, just maybe, help ease this endless cycle of reinventing the wheel.

Easier access to previous bodies of work

Although there is a vast wealth of previous work for us to learn from, they’re often not easily discoverable or accessible (e.g. hidden behind ACM memberships) and hard to digest (why are technical papers still published in paper format when new mediums are available is baffling).

The sheer number of academic papers that are published every year can also be overwhelming to those of us not steeped in the academic world. To get started, look for a Papers We Love  meetup group in your city and go to their meet ups.

 

In civil engineering, we have a much older, vigorous and disciplined industry where things tend to get done. In civil engineering, you front-load your risk and everyone has a clear line of responsibility in a project:

image

 

Whilst metaphors can be useful to help us make sense of unfamiliar things, but we can then over-extend them.

And since we think software engineering is like civil engineering, so other characteristics of civil engineering that has nothing to do with software suddenly becomes more pervasive too.

Ultimately there are many places – e.g. the need for a quantity surveyor to provide estimates for materials and costs – where the metaphor starts to break down.

Dan hypothesized that the reason why we’re so hung up on features in software is because in our metaphor, civil engineering, the bigger is better.

image

Whilst attacking this fixation on features from a different angle, Melissa Perri described the tendency many organizations have of keep on building features as The Build Trap.

the_build_trap

 

Intuitively we all know that in software engineering, the bigger is not better, in fact, the opposite is true.

So what if civil engineering isn’t an appropriate metaphor for software engineering?

 

Software Surgery

Earlier in the opening keynote Complexity is Outside the Code Dan and Jessica Kerr said that the goal of software engineering should be to sustainably minimize lead time to business impact.

Lead time to someone saying thank you is the only reputation metric that matters.

– Dan North

Lead time is mostly made up of handoffs, so even if your developers and testers are very efficient in their siloed tasks, the handoffs are still going to hurt you. A typical work item spends 90% of its life not having stuff done to it – mostly because it’s waiting in the queue.

Then Dan theorized that, maybe, software is more like surgery instead of civil engineering. Maybe, the way we approach and consume software, is very similar to the way we approach and consume surgery.

No one wants surgery!

Just as no one wants to go under the knife for fun, neither do businesses want software for the sake of having them – they want solutions to problems and it just so happens that in this day and age that solution is often made up of software.

But if I must have a surgery, I want to have the minimum amount of surgery possible (i.e. small is beautiful).

I want the most competent and experienced professions to operate on me. And I want them to use established, proven techniques, and yet, still prepared for the unexpected and ready to use more experimental approaches when required.

But mostly, I rather not have the surgery, I just want to be well!

Features + Discovery + Kaizen

Just as surgery is more than just cutting people up, there’s more to software than features.

In the context of surgery, there is also discovery,  which is about the individual patients and exploring what problems they have by using blood tests and other exploratory steps.

For this to work there is a trust relationship between the patient and the surgeon. Whereby the patient allows himself to be vulnerable and have faith in the surgeon’s knowledge of the domain and professionalism to make him better.

And then there’s Kaizen. In medical science, the use of anaesthetics, washing your hands before surgery, etc. are all about improving the process of surgery and help reduce the mortality rate of patients.

We need to re-engage

Dan used the civil engineering metaphor once more to describe how, by not engaging with the business to find out what problems they’re trying to solve, we have allowed ourselves to be the order takers who simply react to the list of features we’re handed.

Instead, we should aim for a more consultative relationship like the one between patients and surgeons.

We also need to re-engage with the IT management whom has been steeped in the civil engineering metaphor for so long. We need to think about what it means to lead and govern an IT organization in which less is more. We need to help them help us.

And we need to re-engage with ourselves. The craft of surgery is not about cutting people up, it’s about saving lives. The craft of software development is not about programming, it’s about creating business impact through software.

image

Features, Discovery and Kaizen are all first class work, you need to schedule, measure, track and showcase each and have some of each in flight at all times.

 

Links

CraftConf 15 experience report

Phew, what a week, finally back in the UK after a good few days in Budapest for CraftConf. What a beautiful city and what a great conference.

On a personal level it’s been a good trip, caught up with some old friends, and met some new ones. Big thanks to Adam Granicz and the lovely folks of IntelliFactory for showing us a good time and the Budapest night life, it was a blast!

I also caught up with Martin Kleppmann whom I met at dev/winter/2015 and found out about the book he’s been working on – Designing Data-Intensive Applications – which gives you an overview of just about everything you need to be aware of in that space.

The Venue & Food

As a whole, Budapest is a beautiful city with plenty to see, and the views at night is breathtaking. The conference is hosted at one of the prime locations in Budapest – the Várkert Bazar. It was recently renovated (at a great cost as locals kept telling us!) and only open to public since last September.

The food at the conference was one of the best I’ve had at any conference. They even served seafood at lunch, seafood!

The Talks

There were some really good talks at the conference, I had to make some tough choices. Fortunately all the talks were recorded and also streamed live on the conference website thanks to the uStream guys. Based on what I’ve been told, the recordings should be available as early as this weekend.

I took plenty of notes during the sessions I attended (see below) and would write up my key takeaways from each over the next few weeks, so watch this space!

  • Scaling micro-services at Gilt (Adrian Trenaman)
  • Jepsen IV: Hope Springs Eternal (Kyle Kingsbury)
  • Architecture without an end state (Michael Nygard)
  • Concurrency: It’s Harder (and Easier) Than You Think (Paul Butcher)
  • Beyond Features: Rethinking Agile Planning and Tracking (Dan North)
  • The Hidden Dimension of Refactoring (Michael Feathers)
  • Why is an API like a puppy? (Adewale Oshineye)
  • Microservices AntiPatterns (Tammer Saleh)

My personal favourite talk was Kyle Kingsbury’s Jepsen IV: Hope Springs Eternal. Not only did Kyle pit various NoSQL products against their outlandish claims and put them to the knife, he also provided a nice framework for you to thank and reason about the different properties a consistency model can give you. If you enjoyed Kyle’s post Call me maybe : Elasticsearch then you’ll love this talk.

The not-so-great

My overall experience of CraftConf was great, but there are two things which I hope they’ll improve on in the future:

  • lack of functional programming talks, I saw a total of two talks that are FP-related
  • almost all the talks are very high-level and few showed any code

Whilst I appreciate that the conference focuses on craftsmanship, so it’s natural for the talks to stay high-level. In my opinion, exposing attendees to non-mainstream paradigms (FP, logic programming, AOP, stack-oriented) would also serve to improve their craft. After all, what better way to expand your horizon than to learn yourself a new paradigm?

CraftConf vs CodeMesh

In terms of content, CraftConf is a great conference to go and get inspired. But, if you’re looking for a conference where you can find out about emerging technologies and languages (and have your brain hurt after two days) then you should come to London in November for CodeMesh.

If CraftConf is a conference for software engineering and craftsmanship, then CodeMesh is a conference for computer science.