Lessons from cricket's battle with bias

Research indicates that the use of neutral umpires has removed bias, a lesson perhaps for appointing officials in all fields.

The Australian summer of cricket has largely been a controversy free zone but that could quickly change if the International Cricket Council (ICC) decides to move away from using neutral umpires for test matches.

The quality and bias of cricket umpires has long been a sore point among fans. Prior to 1994, both test umpires were allowed to be from the home country. Between 1994 and 2002, it was mandatory for one umpire to be from a neutral country. Since 2002, it has been mandatory for both umpires to be from neutral countries.

The shift in regulations provides an ideal ‘natural experiment’ that can be used to assess umpire bias. That’s the basis of new research by economists Abhinav Sacheti, David Paton and Ian Gregory-Smith, which was recently published in the Journal of the Royal Statistical Society.

Using a sample of 1,000 test matches, between 1986 and 2012, they find clear evidence that umpires from the home country are biased towards the home team. That bias was reduced after 1994 with the move towards one neutral umpire and then eliminated once it became mandatory to have both umpires from a neutral country.

The research focuses on leg before wicket (LBW) decisions, which are easily the most arbitrary decision made by a cricket umpire and the one that can have the largest impact on the outcome of a match.

The raw data shows that obvious bias is not apparent in every country; Australia has traditionally been a great beneficiary of favourable calls, while the West Indies and New Zealand have had no such luck.

But the raw data has some clear limitations since there are a range of factors that might affect LBW decisions. Some teams are simply better than others and may not lose their wicket as often or as easily; that could be relevant for a country such as Australia, which enjoyed an unprecedented period of success throughout most of the sample.

The researchers’ model finds that two home umpires causes a 21 percentage point decline in the number of LBW decisions against the home team per innings -- after controlling for other relevant factors. That is roughly equivalent to one extra LBW decision per innings in favour of the home team, which is surely enough to have a significant influence on the outcome of a match.

The effect of this bias was halved when the sport shifted towards one home and one neutral umpire and virtually eliminated when cricket moved to two neutral umpires.

The authors also discount the long-held belief that the crowd may effect decision-making. They argue that if the bias reflected the influence of the crowd, then we would expect to see more favourable decisions during the first two innings of a match, since crowd sizes are demonstrably higher.

However, the data shows that the bias exhibited is actually greater during the third and fourth innings of a test match. While that discounts the theory that crowd pressure influences umpires, it raises a far more important point: umpires show more bias towards the home team when the match is in the balance.

In the past couple of years there has been some discussion about returning to the home umpire policy. Some commentators have argued that the implementation of the Decision Review System (DRS) reduces the degree to which umpires can influence the outcome.

The research shows that DRS has resulted in fewer LBW decisions, on average, but it still leaves the umpire with considerable discretion. Even with the DRS in place -- and it is worth noting that it still isn’t accepted by India -- it appears clear that the use of neutral umpiring is more important to the integrity of the game.

Related Articles