England’s Exit from the Rugby World Cup

Damned Lies, England’s Exit from the Rugby World Cup, & Confusing “Should” with “Is”

Warning: this article will not appeal to anyone who doesn’t follow rugby union – anyone reading it who expects any other subject matter will feel ticked off.

Have a go at this calculation, rugby followers:

[Number of English newspapers] x [pundits plus readers] x [days since England’s first match against Fiji]

The answer you get is the number of opinions about why England under-performed at the rugby world cup and became the first former winner to be eliminated at the group stage.

I’ll summarise these several hundred thousand opinions into 3 themes:

1. It’s the players
2. It’s the coaching team
3. It’s the RFU

Let’s have a look at the thinking and some facts, and see how well these theories stack up.

Is it the players?  Well yes, but not in the way you think it was

The “it’s the players” argument goes like this: England has the biggest player population of any nation and so should consistently have the best set of players to choose from, but the elite English players aren’t the best in the world.  Something is going badly wrong.

First, let’s see whether England has better or worse players than other nations.  Here’s the number of nominations for the world player of the year by nation since England won the world cup in 2003.  I’ve gone back to 2004 because there are top players playing in this world cup who were in the list back in 2004.

England doesn’t seem to have produced many of the world’s top players since 2003.  In fact, it had the same number of world player of the year nominees in the single year it won the world cup in 2003, as it has had in total in the 12 years since.

How about England’s player quality in just the Home Nations?  The most solid data we have here for comparison is who was selected for the most recent British & Irish Lions tour, to Australia in 2013.  Looking at the starting XV across the 3 tests, the average number of players by nation was Wales 8, Ireland 4, England 3, and Scotland 0.^[1]  So even in the British Isles, England ranks second bottom.

So England has had fewer top players in recent years than any top nation except Argentina, Italy and Scotland.  How can this be, given England’s massive player base?  Let’s look at that next.

Here’s a chart showing the IRB rating of every nation in the rugby world cup on 9th October 2015, versus the number of registered players in that country.

You can see that once a nation has more than about 30-40,000 registered players (all the countries named in the chart), then there’s pretty much no relationship between player population and success.  If it did, then Japan wouldn’t be celebrating a great world cup.  Instead, its union would be holding a 360 degree inquiry into why Scotland, with half the playing population beat Japan so convincingly and qualified ahead of the Brave Blossoms.  Italy would also be gnashing its teeth each time the smaller Wales player population finishes above it in the 6 nations, which is every time.

I’m not saying player population is irrelevant, just that other things are likely much more relevant once the population is big enough.  This kind of makes sense given that a match day squad has 23 players.

Let’s look at this another way to see if anything about player population is useful.  Rugby is a minority sport in most countries.  Let’s look at how relatively popular it is by country, by looking at the number of rugby players there are per thousand of population, and see if that’s related to performance.  Here’s a chart showing that relationship.^[2]

Now we’re getting somewhere.

It seems that the popularity of rugby union in the country is pretty important, i.e. do the most talented kids choose to play rugby instead of football or rowing or boxing, and do they have equally talented peers to keep them on their toes and prompt them to get better?

Looking at it this way, England mildly underperforms, but you wouldn’t expect it to top the performance rankings given that rugby union in England isn’t all that popular.^[3]

Here’s what I conclude: England hasn’t had the greatest set of players to choose from since 2003, but given that rugby union isn’t that popular in England, you wouldn’t expect it to lead the world except when the stars align and a few once in a generation players come along at the same time.  The fact that England is a highly populated country is less important than the fact that rugby doesn’t draw that much of the top talent.

Is it the Coaching Team?  Doesn’t Look Like it to me

OK, so how about that coaching team?  There have been lots of specific criticisms of the coaching team from playing style to selection and nepotism.  These may be correct or incorrect, but the way I’ll measure the coaching team’s performance is whether they do better or worse than expected given the group of players at their disposal.

Before the World Cup, the England coaching team had trained England to second in the 6 Nations Championship four years in a row.  England has fewer top players than France, Wales and Ireland on our international player of the year measure, and fewer than both Wales and Ireland on our British Lions measure.  England has finished ahead of 2 of those countries every year for the last 4 years.

In the 2015 World Cup, England were beaten by Wales and Australia, the teams that came third and fourth in the previous World Cup with relatively young teams, and who both are ranked higher and have more top class players than England.

So the England coaches did a better job in the 6 Nations than you’d predict given player quality, and did as you should expect in the World Cup if you can take off your red rose tinted spectacles.  There may be better coaching teams available, but England’s coaches seem to be doing, on average, a slightly better job than their international quality 6 Nations peers.  England performing below expectations looks like an issue with the expectations, and these were maybe set back in the day, when England had more world class players and a disproportionate number of British & Irish Lions.

Surely the RFU Must Carry the Can?  I Don’t See How They Do

The “blame the RFU” argument goes something like this: the RFU is the richest union in world rugby, with the biggest playing population; so with that combination of financial and playing resources England should have the best managed and capable team on the planet.  Simple.

We’ve talked about how playing population isn’t important once it’s higher than about 30-40,000.  So that leaves the money – what can the RFU actually do with all that money?  It seems there’s maybe 3 big things

1. Select and recruit a good set of coaches – we’ve covered this and I’d say the coaches are OK
2. Give the elite players the best possible facilities – no one is arguing that these aren’t wonderful
3. Provide grass roots coaching to develop the next 2 generations of talent – now we’re onto something, so let’s talk about this next

I never thought I’d say RFU and “chink of light” in the same sentence but take a look at this information on junior players coming through:

World Junior Player of the Year Nominations by Country

England now matches the best nations in the world for numbers of top young players.

Now have a look at this information on how England’s junior teams are now performing:

England Placings in Junior Championships

England’s junior teams are now often more or less the best in Europe, and in amongst the best in the world.  People who’ve watched England in the World Cup and pundits who have rated England’s players have seen the start of this generation coming through.

I don’t know how much we can credit the RFU for this uplift, how much is clubs and schools, and how much is the nation’s talented 8 to 10 year olds watching England win the World Cup in 2003, and taking up rugby instead of football.  I think much of it is the 2003 factor, and that the relationship between team performance and rugby popularity actually works in both directions; but it seems churlish to give the RFU no credit at all for all the good players coming through.

So on balance it seems that the RFU isn’t doing a terrible job with the things it can actually control.

Who’s the Luckiest Man on the Planet?

I’ve no idea whether the England coaching team will keep their jobs.  Sacking them seems harsh given the job they’ve done without a world-leading, or even Britain-leading, group of players.  But with the talent coming through, the coaching team for the next 2 World Cups is going to have some genuinely good raw materials to work with and may be able to fulfill England’s inflated expectations.

And This Relates to Critical Thinking How?

This is critical thinking blog, and all I’ve done is analyse the ass off a rugby team’s performance.  Here’s the critical thinking lesson I took from looking at this.  When something isn’t going as you think it should, it’s important to put the scatter gun back in the holster and have a colder, harder look at the facts and what matters, because you’ve probably misunderstood something basic, like whether more players equals better players.  There’s often a world of difference between “should” and “is.”  It’s especially important to make this distinction when the subject is as emotionally distressing as a few 80 minute games of rugby.

^[1] There is another source people use to compare players: the European Player of the Year.  However, if you look at how players are chosen in this competition, you see it is really a showcase for the European Champions Cup and is biased strongly to the 30 players in the teams that reach the final.  Since annual selection started, 4 of the 5 European Players of the Year played in the team that won the final, the fifth played for the team that lost in the final.

^[2] The size of the bubbles in the chart indicates the size of the playing population for each country. I’ve omitted the Pacific Islands from this chart because Tonga and Samoa have very small player populations, and Fiji has historically concentrated its top talent on 7-a-side rugby.  These nations also lose disproportionate numbers of their best players to other nations

^[3] France, Australia and South Africa have regional and demographic concentrations of players.  If we looked at registered players in just those regions or demographics, rugby would look much more popular there and their bubbles would move to the right on the chart

A Settled Team is a Winning Team?

A Settled Team is a Winning Team – or is it the Other Way Around?

There’s a received wisdom that says you need a settled team to get good results.  Here’s Will Greenwood talking about the England team in the 2015 Rugby World Cup:

“But the most important thing right now – and I am not inside the England camp so I can only speculate on the mood within it – is stability. I cannot emphasise that enough. Chopping and changing helps no one.”

Google “a settled team is a winning team” and you get pages of similar appeals to find a settled line up, from all sorts of well meaning experts and supporters.

If we accept these philosophers’ statements as true, we’re accepting 2 things:

1. It’s true – teams do better when they have a settled line up
2. We’ve got cause and effect the right way around, i.e. cause=settled team, effect=better results

Let’s test it shall we?

Do Settled Teams Get Better Results?

We can’t do this test by looking at dozens of teams, and seeing whether the ones that have more settled line ups do better, because that doesn’t help us work out cause and effect.  To get to the bottom of whether a settled line up is a winning one, and to understand what causes what, we need to choose one team, and see whether the team performs better when its line up is settled compared to when it makes lots of changes.

For my team, I’ve chosen the Liverpool football team in the 2014-15 Premiership season.  I chose Liverpool because they’re better than average and so should be able to string a good run of games together if the manager can get that settled line up.  They’re also not so good that they never lose, so they should give us a good range of results to analyse.

I’ve looked at Liverpool’s starting team in every Premiership game and counted how many player changes the manager made from the previous game.  The number of player changes ranged from 0 to 4 over the season, with an average of 2.3 player changes from the previous game.  Not all player changes are voluntary because of things like suspensions and injuries, and players coming back from suspension and injury, but that’s all part of the hurly burly of achieving our target of a settled line up.

Now let’s look at the average number of points Liverpool got when they had a more settled line up than average (0, 1 or 2 player changes from the previous game), versus when they a less settled line up (3 or 4 player changes)^[1].

Yes, you read the table correctly.  Liverpool did no better or worse, whether they made a handful of changes, or if they changed a third of the team.  This doesn’t mean having a settled team is irrelevant, but it does mean that other things, say how good the opposition is, are likely much more relevant.  So just having a settled line up doesn’t, even for a better than average team like Liverpool, reliably give you better results.

What if we Got Our Cause and Effect the Wrong Way Around?

It looks like our cause and effect theory doesn’t hang together: the team stability doesn’t give us better results.  But what if the cause and effect worked the other way around?  What if better results caused a more settled team?  Let’s look at the data again, remembering that the manager made on average 2.3 player changes after each match.  Here’s a chart of how many changes the manager made, depending on whether the previous result was a win, draw or loss.

He’s making 50% more changes when he loses.  Given that some changes are enforced by injury or suspension, win lose or draw, then I’d bet he’s making most of his voluntary changes after defeats.

So it seems that good results cause team stability, and bad results cause the manager to tinker a bit more with his line up.  The classic stick while you’re ahead, roll the dice again if you’re behind.

Testing the Theory in A Completely Different Environment

Every good European knows that Britain and France are fond cousins with opposite personalities, so if the theory works in France too, then maybe we’re onto something.   I also want to test things away from the pressure cooker environment of football.  So let’s look at the France rugby team in the 2015 Six Nations competition.

Can you see the pattern?  Let’s look first at how results affect team stability:

• Average number of changes after a victory: 1.5
• Average number of changes after a defeat: 6.5 – Sacre Bleu!

So good results are followed by team stability, bad ones by a petite Revolution.

Let’s now look at how team stability affects results:

• Results after making hardly any player changes: won 1, lost 1
• Results after lots of player changes: won 1, lost 1

So whether Phillipe Saint-Andre made lots of changes, or hardly any changes, turned out to be immaterial.^[2]

So What?

So how do we square all this with our instinct that the teams we all recall as winners are all pretty stable?  The Arsenal invincibles?  The great Barcelona teams?  The world beating Australian and West Indies cricket teams from over the years?  Here’s what I conclude.  Winners make fewer changes, but they don’t win because they make fewer changes, they make fewer changes because they’re winners.

Leaving aside some armchair philosophy about whether causality exists at all, it seems to me that it feels natural to think that people’s actions (changing the team) cause outcomes (bad results) and it’s not that natural to think the other way around, i.e. that outcomes prompt or cause people’s actions.  That natural way of thinking tricks us into getting our cause and effect the wrong way around.  But the direction of cause and effect is obvious when you look at the evidence and think about what’s going on.

Makes me wonder about other cause-effects we’ve been accustomed to hearing about.  Do great or shocking governments cause national economies to thrive or flounder?  Do world class universities produce outstanding graduates?  Do a few corrupt old men create a monopolistic, lucrative global sports governing body? Or does it all work the other way around?

^[1] For non football fans, you get 3 points for a win, 1 for a draw and 0 for a defeat.

^[2] What did affect whether France won or lost? They beat both teams that regularly finish below them in the table, and they lost to all 3 teams that regularly finish above them.