There’s a truism that gets passed around before Rugby World Cups, when journalists analyse the squads being selected by their national coaches. Here’s an example of it from the Guardian about the Australian squad:
Cheika’s squad hits the key metrics that have mattered most at the World Cup to date –average age and number of Test caps. The 2015 World Cup Wallabies boast a combined 1236 caps, with an average playing age of 27 years. The average team age of the last four World Cup winners was 27 (Australia, 1999), 28 (England, 2003), 27 (South Africa, 2007) and 28 (New Zealand, 2011). Of that group, the All Blacks held the most caps with 709, followed by South Africa with 668, England 638, and the 1999 Wallabies with 622. The trend is clear: experience matters. (Emphasis added.)
This journalist has followed a common line of thinking, which goes like this: these are some characteristics of people who have succeeded in the past; if we copy them then we’ll be just as successful. So you need a squad with an average age of 28 that has the most caps you can get. Sorted.
Here’s how the Hulk responded when he read the Guardian article.
By looking just at the winners, the journalist hasn’t checked himself with a simple, obvious question, which is, “What were the ages and caps of the teams that didn’t win the World Cup? Were they any different from the team that won?” He’s been guilty of survivorship bias – just looking at characteristics of successful people, assuming that those characteristics must be what made them successful, and not checking whether they did anything different from what the unsuccessful people did. He’s made a ton of other logical mistakes, but we’ll concentrate on this howler.
Does Average Age Matter?
We’ll start with how old the winning team needs to be, and the assertion that 28 is the magic number. Here’s the average age of the major nations at the World Cup in 2011, which New Zealand won[1].
26 – Australia, Wales
28 – NZ, Scotland, England, Ireland, S Africa
29 – France
So most teams had an average age of 28, and all but one of those teams didn’t win the World Cup. If I wanted to ram things home with some random analysis, I’d say that teams that didn’t have an average age of 28 did better than those that did. France (average age 29) came second, Australia (average age 26) came third, and Wales (average age 26) came fourth. All the others with average age 28, except NZ and Scotland, lost in the quarter finals. Scotland didn’t qualify from the group. I doubt that Tonga, average age 29, were bemoaning their excessive age as the key reason they didn’t take home the silverware.
Just so we’re not deceived by averages, the age difference between the oldest and youngest player in these teams ranged from 10 years to 15 years. Rather than coaches nurturing a cohort who reach 28 at the World Cup, it seems that players peak in their mid to late 20s but that coaches choose their best players for each position, even if they’re much older or younger. We’ll come back to this issue of having the best players once we’ve looked at whether having lots of caps is important.
Do More Caps Mean More Success?
Here’s an analysis of how caps affected 2011 Rugby World Cup success. I’ve given all the quarter finalists a placing of 6th (halfway between 8th for being the worst quarter finalist and 4th for being the worst semi finalist), and Scotland 10th (just missed out on quarters by coming third in its group).
Can you see the relationship between caps and placing? No? That’s because there isn’t one. This scatter chart has an R-squared (a measure of correlation between two variables) of less than 0.02, which is pretty much the same thing as being random. Even the most hopeful, biased analyst would be embarrassed about claiming a relationship that has an R-squared of anything less than 0.5. If you do a similar analysis using matches won instead of placing, you get the same non-relationship.
The lesson from all this is that you can find yourself down some blind alleys if you just look at snippets from success stories to show you what to do. You need to think about it properly. So put down that millionaire, gold medallist philanthropist’s autobiography, take off your rose tinted glasses, and look beyond the top of the podium to see what actually distinguishes successes from failures.
What Does Seem to Matter?
I’m not going to stop here and leave rugby fans hanging by ending on what doesn’t matter. I don’t pretend to know for sure exactly what creates World Cup success, and I’m not going analyse what I suspect in this article; but here are a few facts that give us some hints:
Since IRB rankings began in 2003, the top ranked team in the world has always won the World Cup (England in 2003, S Africa in 2007, and New Zealand in 2011), i.e. nothing magical happens at the World Cup to stop the best team winning it
Since the World Rugby Player of the Year awards started in 2001, the country with most winners and nominations in the years since the previous World Cup, and including the World Cup year, has always won the World Cup.[2] (S Africa tied with NZ for nominations in the years leading up to 2007, which S Africa won)
The U21 World Rugby Championship was the earliest junior world cup, and began in 2002. The team that won it 2 out of 3 times between 2002 and 2004, in that magical 7-9 years before players hit their late 20s at the 2011 World Cup, was, yes, New Zealand
Home advantage does count, but not as much as being the best team in the world. Southern hemisphere teams are typically ranked higher than Northern, and have won 6 of the 7 World Cups to date. No Northern hemisphere team has won a home World Cup. New Zealand won both times it hosted, and South Africa won the one time it hosted. Of the Southern hemisphere World Cup hosts, only Australia hasn’t won: it lost in the final in extra time against the then top ranked team in the world, England
Finally, it’s sport, so there’s always randomness around the edges that can make a big difference to individual matches: different weather suits different teams, referees have different interpretations, players have good and bad days, players are binned or not binned on marginal calls, game plans come off or bomb, key players get injured, and the officials miss the odd forward pass
What I take from this tiny sample of data is that the best team is made of the best players, and those players are created at least a decade before the World Cup. If you’re there or thereabouts with player quality, then you’ve got an even better chance if you’re playing at home and have a bit of luck. So it strikes me that the most important people in your country’s World Cup attempt are the people organising youth policy and player development, and the commercial team bidding to host it.
Of course, before affirming this as anything more than a supposition, I’d want to analyse it properly.
[1]I’ve used the starting line up from the first game for each team, so I’m not tempted to select matches arbitrarily. I’ve also included only the teams from the original Tri Nations and Five Nations, because I want to have a level playing field when analysing caps later.
[2] In my analysis, I gave winners double the weighting of other nominees, but that doesn’t affect the result
4 years of Kardelen research revealed that rapid and predictable portfolio company growth requires the creation of a Performance Environment. Currently implementing such an environment relies on the implicit knowledge of investors and Chairmen to drive the change, which requires luck and causes delays of months or years.
Kardelen has developed a diagnosis to this luck and delay, steering investment of time and money towards the areas most requiring attention. Kardelen benchmarks and traffic lights the portfolio company versus performance environment best practices, alerting investors and management to areas of weakness and outlining potential actions to address issues.
It takes one day’s conversation with the CEO to benchmark around 150 areas of assessment. We then produce a high level summary of priority concerns, and detail specifics and proposed action steps behind each area that will prompt management to rapidly and reliably create a Performance Environment.
Not many SMEs grow to any material size at all. If you take 100 typical UK companies, 2 of them have more than 20 employees, 1 of them has more than 50.
But small businesses that have private equity investors grow reliably into the rare 1% with more than 50 employees and then beyond. They consistently outperform their non-PE peers in employment growth, earnings growth, productivity, and (lower) default rates. There are plenty of exceptions to this sweeping observation, but that’s the trend and if you look at the evidence it’s a compelling one. Looking from the outside though, it’s a puzzling one. How do investors who have virtually no sector experience, virtually no executive experience, whose main input is to sit as a non-Exec on the Board once a month, who take out chunky fees, and who rarely stay involved with the business for more than a handful of years, make such a big difference?
Kardelen’s research found that one of the key benefits of private equity investment is the prompt to create a performance environment that allows a promising company to grow reliably and beyond its historical constraints. Surprisingly, this labourious task is largely left to chance, reliant on the team’s ability to adopt the performance disciplines, and the investor’s ability to provide and prompt a suitable performance environment and support structure.
We think that investors and investees can establish performance environments much more quickly and reliably than they do today, but only if the investees understand from the outset what a performance environment looks like, and if investees have the skills to implement it instead of working things out as they go along.
We offer two services to make this happen. In our first service, we diagnose the gaps in investees’ environments, and give the senior team training in all relevant areas of weakness so that they can become adept at developing their own performance environment with no lost time and no need for luck. In our second service, we work with the management team with its strategic plan immediately after investment, prompting the team to include all the important business building steps from the outset.
Imagine you’re a sports coach who can only train each of your athletes for 3 days every 6 months, and you have no direct access to any athlete for the rest of the year. Would you train those athletes the same way Phil Jackson trained the Chicago Bulls or the LA Lakers? Or imagine you’re a great schoolteacher, who has just 6 days a year with each class of pupils. Would you use the same approach that even your most inspiring schoolteacher or university professor used?
Sportsmen are usually in competitive action for less than 2 hours a week. The bulk of their time is spent practising, making adjustments, and mastering skills well away from the field of competition. Students are learning full time for months or years, understanding principles and slowly building a foundation before starting a lifetime’s vocations. A typical manager has around 6 days of training each year, and needs to use that training in real life, skillfully, straight away. This is a completely different challenge, and those few training days need to be used really well.
Here are some ways management training can make those precious days of training really count.
We focus only on the few things that help us best improve. None of us can absorb more than a handful of new things in a day, and then integrate them into our way of operating. If we try to do more, then we’re drinking from a fire hose; so we need to be wise about choosing the few critical things we learn and practise
We learn skills that work well, straight away. Elegant principles and beautiful insights are never the final step. Every beautiful insight needs to be turned into an effective approach that Martha can apply skillfully on Monday
We start mastering our new skills well before we leave training. Understanding something is not the same as knowing how to apply it skillfully. We need guided practice, to recognise how to use a new skill well, before we use it in anger
We get prompts to improve our expertise in the 98% of our time that we are not in training. This can mean anything from simple practice reminders to incorporating 3 or 4 approaches into a team playbook
We can learn lots of things from great sports coaches and brilliant teachers, but management training is a very different prospect, and in many ways a much more difficult one. We think we’ve got good management training when we can make the absolute most of a handful of days, to help someone get much better, straight away, at skills that really matter.
Goal setting can be an outstanding way of improving performance if it’s done well. If I were to boil down the years of studies about good goal setting to just two critical things to get absolutely right, those things would be difficulty and feedback:
Goals should be just at the edge of your ability: just difficult enough that if you concentrate hard you’ll hit them, but on an off day you won’t
You get continual, very specific feedback about how you’re doing, which tells you what you need to adjust to get in the performance sweet spot
Let’s consider what happens when we don’t bother about difficulty and feedback. Imagine a tennis lesson to improve your return of serve. How much will you improve if your coach serves 130mph bombs past your ears? Or if he dollies it over the net to you? How will you know how to adjust your return if you can’t tell whether it’s difficult for your opponent to hit back again, or if your coach says nothing?
These two things: task difficulty and feedback, are the heart of Mihaly Csikszentmihalyi’s superbly well researched conditions for getting into a state of peak experience, or “Flow”. Other research, on goal feedback, shows that if you don’t get feedback then you get no benefit at all from goal setting.
Now let’s look at the SMART goals acronym, the popular wisdom on good goal setting. There are different words that each different letter of SMART can stand for, which is a bit of a worry in itself, but let’s take a common definition: Specific, Measurable, Achievable, Relevant, and Time-based. To be fair, these can all be useful characteristics of goals in the right circumstances, and have decent evidence to back them up. So they’re not wrong, but they miss our two critical characteristics of difficulty and feedback. SMART only skirts around goal difficulty. SMART doesn’t even mention feedback, and without feedback you might as well not bother.
SMART goals also miss plenty of other very important characteristics of goals: using learning and process goals in the short term; having performance goals that are entirely within your control; commitment to goals; repositioning goals upwards or downwards when they become too easy or difficult; and taking breaks from goals entirely.
So my advice is to beware snappy acronyms and use what works: set goal difficulty just at the edge of your ability, and get continual feedback to help you adjust and learn. Everything else is important but secondary.
Using hypotheses can help us become clearer thinkers, save us time, and help us communicate succinctly. We can use them wherever we’re not sure and care enough about getting to a good answer. They’re at the heart of scientific problem solving, and have been used by great thinkers since before our boy Aristotle. Wherever I’ve trained the use of hypotheses, people find it a refreshing and mind opening approach. I hope it’s worthy of your attention.
Let me say what I mean by stating a hypothesis. All we are saying is: “This is what I suppose given what I know right now.” That’s all. It’s our first venture at an answer, and acts as a starting point in getting to a good solution. We could be supposing anything from “Miranda will be the best leader for the company” to “evolution explains everything.”
Our hypothesis is a working answer that we hold gently and challenge hard. By stating this working answer, and making it tangible, we give ourselves something concrete to test with thinking and evidence. Our job is to challenge it: “Does it cover things completely? Is it consistent with observations? Does it make sense logically? Is it unequivocal with no room for misunderstanding? Is it simple enough to be obvious? Can I think of any exceptions that hole my beautiful hypothesis below the water line?” As we challenge our working answer with evidence and clear thinking, we expect it to change, just like the detective’s naive first guess in an episode of CSI. If we’re really hungry investigators or expansive thinkers, we’re rarely happy until our first guess has been challenged and changed at least a couple of times.
As we go through this process, our hypothesis slowly solidifies into a thesis; our supposition turns into our position on the matter. In some cases, we might even get to the verifiable truth: “It was Professor Plum whodunit,” or, “This business will be profitable.” Often, we’ll never know the truth but will run with our best thesis: “John will be the best Governor,” or, “profit share is the right incentive scheme.”
Starting by stating what I suppose has a host of advantages over just asking questions or musing distractedly. It forces me to be concrete about what I think, which highlights weaknesses and makes my thinking better. It turns my perspective into one of a humble investigator who welcomes challenge, as opposed to a blustering know-it-all or a vague wonderer. It gives me a focus for my investigative efforts. It enables me at any stage to be explicit about my current position on the matter, being overt about where I’m confident and where I’m unsure. I can communicate my position at any time, so that other people can understand, challenge and contribute.
Using a hypothesis has drawbacks, typically because of using it badly. The biggest is that we get attached to our hypotheses and slip into trying to prove them. We’re all guilty of this, though it’s an even easier trap to fall into if we don’t think by hypothesis, and so don’t welcome self-challenge or new insight.
Hypotheses are essential to the scientific problem solving that guided Newton and the analytical rhetoric that guided Madison and Martin Luther King. Everyone that learns the skill becomes a better thinker. If you use hypotheses, you’ve got exalted company in Aristotle, Cicero, every mechanic or plumber who actually fixes your problem, every great fictional detective and, I suppose, some real ones too.
