The Goal, Finding Ultra, and The Agile Manifesto, or “What does running 52 miles have to do with writing good code?”

So, in the mental Mulligan Stew that is my brain, I find odd patterns and connections emerging, or re-emerging, often out of whatever happens to be on my Reading List at the time.  This morning was a perfect example of this happening, and (if you can tough it out the three minutes to the end of this post) I think there’s something useful in it, at least if you’re part of the nerd herd (yes, Jeanne, this one’s for you. 🙂 )

I was meeting with a colleague this morning and we were discussing one of the challenges organizations can face moving product/development teams to SCRUM, a flavor of Agile Development.  The topic we were discussing was both the personal bias among some developers for, and the business or upper-management pressures to fall back on, short-term, informal or “hackish” solutions to problems when something just needs to get done and get in production.

A casual reader might even think that this might make sense.  Isn’t Agile after all, supposed to be, well, agile?  Get something out, test it, get feedback, fix it later as needs be?  Kind of all “Lean Startup“-y?

I’m still relatively new to Scrum myself.  I am a CSPO, but this is still my first year leading a Scrum product development initiative, yet I can say already that I believe that this casual read would be wrong.  One of the central tenets of Scrum and Agile is that test-driven development, or, if you prefer to think of it in terms of the Lean Manufacturing process (from which the Agile disciplines were derived), “designing in quality from the get-go”.  In other words, yes, the principles (see the Principles Document accompanying The Agile Manifesto) strive to be responsive, get stuff out the door, and iterate quickly.  However, whatever does go out the door is meant to be fully-tested, production ready and of high quality, even if it is very small in scope.

You can test out a concept car with no power windows, radio or A/C and painted in primer, and people can still love the styling, fuel economy and future vision that’s rough and unfinished.  But if you put out a jalopy that can’t be trusted not to fall apart or crash, you’ll never get another fair shot at those early reviewers.  Rough is ok.  Even incomplete is ok.  Dangerous or unreliable, that’s not ok.

So, what’s wrong with a short-term hack that you know won’t hold up for the long term or under heavy load or whatever the future is, if that hack buys you some time now or gets management off your back?  The problem, in my opinion, with kicking the can down the road is that is so often makes the eventual solution more expensive; sometimes – given the law of unintended consequences – vastly more so.  The actual comment my friend made this morning was along these lines, In this scenario, which happens all the time in the real world, “the team that takes the shortcut ends up saving half the time now, but spending ten times the effort when they’re all done.”

So, they cut today’s cost by 50%, and raise the total cost by 500%.  In some cases, and this is reality unfortunately, the fast fix is a source of praise or recognition, while the long term impact is often buried in later, routine work.  The result  is that an organization can actually encourage the bad behavior that has an eventual 10x cost.  I don’t have a calculator handy, but I’m pretty sure a bad deal.  What really tickled my brain somewhere is what my colleague said next, which was roughly this; “Somehow I think some development teams lose sight of the actual goal.  In their effort to go faster, they end up actually slowing themselves down.”

It was this particular phrasing that caused the asteroid collision of two books in my head.  I just finished “Finding Ultra” by Rich Roll, overweight-middle-aged-lawyer-turned-extreme-endurance-athlete,  [you should click that one – you gotta see the pictures].  Early in the book, Rich describes the first prescription he received from his coach, when he decided (with no real experience whatsoever) that he was going to become an Ultraman.  One of the first rules his coach imposed was that he had to learn and understand where his aerobic/anearobic threshold was, and change his habits to manage his metabolism around this breakpoint.  He was not initially moving at a steady and sustainable pace, a pace that (once he switched to it) initially felt painfully slow.  This change, he was instructed, was necessary because without that change, he would burn out too fast and slow his later progress, or cause physical problems that would interrupt or end a long event.

In other words, until he changed how he approached each element or sub-part of the race, the faster he ran, the slower he finished.

Back in school, I read The Goal by Eli Goldratt.  In this fictional tale, a factory manager (and his Socratic mentor) work to understand and fix the problems in a production plant plagued by delays, high costs and poor outputs.  Everything from his marital life to a scene involving a marching cub scout troop eventually reveal the underlying principles that help solve the problem. (If you’re interested in production operations or business at all, this book remains a quick and relevant read.)  While there are a number of more detailed lessons on Operations Management to be found there, I remember discussing the “big takeaway” with Ricardo Ernst, my ops professor at Georgetown and one of the funniest, smartest and most valuable teachers it has been my honor to study with.  The bullet-point version was this.

  • If you have a guy putting 10 wheels an hour on cars, and you provide the right incentives to make it 11, he will.
  • If you have another guy putting on 14 hoods an hour and you provide the right incentives to make it 16, he will.
  • Do this all down the line, and what you have is a crew of “top performers”, every one of them beating their quotas and earning bonuses… and a factory that’s going to be shut down because everything is going wrong.


The system can’t run any faster than it’s slowest step, plus if you incent only speed, quality will suffer besides.  So what happens? Raw unit throughput is constrained by the slowest part of the process (say, the wheel guy), rework costs balloon (because quality inevitably falls), inventory expense explodes (because of all the half finished cars piling up before the wheel station), and finished-product output craters. All the while, your individual performers are each beating their quotas and earning bonuses, while the business loses its shirt.


What’s the point?  Well, here’s the (possibly?) useful thought I’m hoping came out of the mental Mulligan Stew.  Whether the Goal-with-a-capital-G (hey, there’s a reason he titled the book that way) is cars produced, the finishing time in a 320 mile race, or, where this all started, which is writing good software, when you focus on  local rather than global optima, what you get is counter-productivity.  Maybe that tortoise was on to something…


Nate Silver, Fox News, and the Gutenberg Effect, or “How a World Awash in Data Explains GOP Befuddlement on Nov 7th”

Plenty of ink has already been spilled over, at and about Nate Silver and the 538 Blog this election cycle, and even after the election is over, there are still some folks who both deny his math and/or claim that the problem was Hurricane Sandy, Chris Christie or that the Obama campaign “stole the election” or “suppressed the vote“.

What in the world does any of this have to do with the (somewhat intermittent) “Digital Water” meme I’m supposed to be so focused on and my obsession with how people will, and do, react to a world ever-more awash in data?

What was interesting to me as an analysis guy, and appalling to me as a data head and independent voter,  was watching the comments and criticisms of Silver’s 538 Blog before the election.  The astonishing litany of rationales assembled by Fox et al for why Silver was wrong, and just how wrong he was, defied both advanced statistics of the type in which Silver is an expert and the common sense in which we mere mortals are more versed.  While he admits to being an Obama supporter, he’s first and foremost a statistician and forecaster dedicated to understanding the science of accurate predictions.  Yet there were volumes written on critiques of his methodology, his assumptions, his math skills, and probably far more personal attacks on blogs I don’t read.

Nevertheless, Silver has now shown in two elections in a row and 99 out of 100 states called correctly that a deep understanding of not just polls and statistics, but a respect for math and facts can not be undone by all the denials (google “Karl Rove + election night + meltdown”)  and logical contortions (see “Dick Morris + prediction + landslide”)  that kept the conservative faithful, engaged, entertained and ultimately, completely unprepared for Election Day.

In the inevitable party navel-gazing the follows an election-year blowout, two questions have been haunting the conservative rank-and-file.  The first is the obvious “how could America have voted for this guy again?”  This is basically a partisan and political discussion of little interest to me, at least in this context.

More salient to this discussion are “How did we get it THAT wrong?”  This has mostly been addressed in the press by dissecting the exit polls, and talking changing demographics, Hispanic turnout and the fallout among sensible centrists like me from Republican candidates who don’t believe in eighth grade biology or a planet much older than Hal Holbrook.  (While much ignored nationally compared to Todd Akin, this last one, an unopposed Congressman who believes Earth is 9,000 years old, evolution is a lie created by Satan himself and – most insultingly – who also sits on the House Science Committee, is exactly the kind of story that sends sane moderates like me running into the arms of an otherwise completely beatable incumbent.  God Bless Bobby Jindal and his “we have just GOT to stop saying stupid shit” speech.)

Is that what really happened?  I think there’s more going on here, and my answer is two parts.  The first comes from Silver, not in his blog, but in his book, The Signal and The Noise.  I was listening to it on audio CD in my car this week and had to back it up and listen to it three times.  Silver was speaking about the changes that came after Gutenberg’s invention of the printing press, but the same is even more relevant to the “Digital Water” phenomenon, where the world is awash not only in objective and numerical data but the self-published content of every opinion, theory and form of intellectual quackery imaginable.   He explained what I am calling here the “Gutenberg Effect” as follows:

“Paradoxically, the result of having so much more shared knowledge was increasing isolation…  The instinctual shortcut that we take when we have too much information is to engage with it selectively, picking out the parts we like and ignoring the remainder, making allies with those who have made the same choices, and enemies of the rest.”

Put into the context of the 2012 Election Cycle, I think what went wrong was the intellectual and media isolation that many partisans, but particularly those on the right, increasingly engaged in.  The so-called echo chamber, in which attitudes and platitudes of an openly partisan nature ricochet and amplify through the canyons of Fox News, and Rush Limbaugh’s radio show (or, if you prefer, MSNBC, the Daily Kos and the Rachel Madow Show) increasingly discount or vilify any opinion or person with an alternate view.

Many or even most of the criticisms however, are ideological, personal, unsubstantiated and/or filled with logical fallacies and downright absurdity, but not facts, and not math.  And here is where the world awash in data rears its head in Election 2012.  The Gutenberg Effect that Silver describes appears to have actually caused the Republican Party to drink so much of of its own pre-filtered Kool-Aid that a “shellshocked” Mitt Romney seems to have been telling the truth when he told reporters early on November 6th that his staff hadn’t even written a concession speech.

Despite the fact that (as Silver’s blog highlights) an objective read of the numbers showed Romney would have to essentially run the table on the swing states and catch every break to win, the Romney campaign – and millions of hardworking and genuinely dedicated supporters – quite literally couldn’t believe it when he, conclusively and resoundingly, lost.

If the first thing that happened was this Gutenberg Effect, an ideologically aligned group of people taking stock of data selectively to support their pre-established beliefs, I believe the second was a staggering act of exploitation by the very purveyors of that selectively-chosen information.  Check out the video below starting at 5:01, an exchange between David Frum and Joe Scarborough, two guys I don’t always agree with but who I think generally put “smart”, “factual”, and “conservative” rightly back together in one sentence.

To quote Frum, “…the real locus of the problem is the Republican activist base, and the Republican donor base. They went apocalyptic over the past four years, and that was exploited by a lot of people in the conservative world.  I won’t soon forget the lupine smile that played over the head of one major conservative institution when he told me that ‘our donors think the apocalypse has arrived‘. Republicans have been fleeced, exploited and lied to by a conservative entertainment complex.”

Taken together, I believe these can show both the root cause of the completely dumbfounded Republican reaction on November 7th, and also, I believe, a guide to a much truer understanding of on-the-ground election realities for any national campaign going forward.  A clear-eyed view of the state of the race should start with three things:

1.  Understand the Gutenberg Effect and realize the election-strategy dangers in an intentionally (and ideologically tilted) selective filter when viewing an over-abundance of opinions, polls and data;

2.  Acknowledge that the media makes far more money if they denigrate the opposition and radicalize and rile up the faithful than if they help their chosen team actually win elections; and

3.  Take these facts together and strive for the most objective, fact-based view possible of polls, voters, the economy and the country over the coming election cycle, and make sure you listen to, and account (literally) for the views, numbers and opinions presented by the people who most disagree with you.

While I think the right currently has a larger problem than the left in this area, at least for now (i.e. they are often a party whose candidates lose swing votes like mine when they not only ignore but vilify math, science, and objective, rigorous analysis), the lesson for all sides is, I believe, to separate your opinions from the data.  Stop attacking people like Nate Silver, and perhaps start reading his book instead.

%d bloggers like this: