Eye on Hawkeye

Some sort of event going on in South Africa has set of the talk about the role of technology in sports officiating again. I think Kartikeya at A Cricketing View brings to this issue many of the approaches that are needed.

In particular, he highlights the fact that the aim of a drive to use technology is not about getting decisions right, but getting them “verifiably right”. It's not even about being correct. Questions are only raised because there appears to be, and often is, available evidence beyond what the umpires have traditionally relied on. Being absolutely correct is still impossible, it is only a question of whether there is a decent basis for the claim that a mistake was made.

As he says, the uncertainty is shifted to “hitherto unforeseen areas”, and this can be a problem if the technology is seen as decisive. This is related to the other important point Kartikeya makes - that technology is more than just the technical. It's use affected by the interests of those involved, and how it is understood and portrayed by them. There are a lot of sides to this - the example of the Lampard goal in the soccer raises questions, yes, but technology doesn't really come into it until we start thinking about the efficiencies and other outcomes of using broadcasters compared with those of using two more officials.

While I think these points form a strong basis for discussing technology in sports officiating, I don't agree with all the conclusions reached or implications made. The professional nature of modern umpiring, or at least the particular system run by the ICC, quite conceivably hinders umpires, rather than allowing best performances over an extended period of time. Neither is the need for umpires simply caused by players trying deceive. Asking players to give themselves out LBW might work (sometimes) in the backyard or park, but it is clearly not just a matter of honesty. This is an extreme case where the Law itself was developed assuming indepedent involvement, but it is not the only example.

This example is very relevant,as Kartikeya does focus on LBW, in particular Hawk-eye. He begins by agreeing with what I think is the vital point - we have to move past the broad rhetoric of technology vs human, and look at each method of officiating as a separate case. The fact that technology shifts uncertainty rather than eliminating it is an argument against technophilia, not against individual solutions. Being verifiably right or close to it can be a valid aim, and uncertainty is sometimes (often, for example, in run outs) not “merely” shifted, but reduced. Even a change that is not quantifiable may be arguably more acceptable for some reason. This needs to be hammered out, not pre-judged one way or the other.

So what about LBWs and Hawk-eye? It is quite different to most other examples, in that the answer does not rely simply on observing, but on some level of speculation. The basis of that speculation was changing even before the advent of Hawk-eye, so I don't think it can be blamed for changing the Law. Using a technological system to observe and speculate is not in itself any worse than putting it simply in the hands of an umpire, especially under recent versions. However, traditionally there are a large number of umpires who are unreasonably biased against giving LBWs to one extent or another, and it should also be acknowledged both that Hawk-eye is potentially much harsher on batsmen than an umpire could reasonably be. I don't think “correcting” either of these aspects of human umpiring is a bad change, but it is a change.

The reason for this is that uncertainty, and the benefit thereof, is already acknowledged in our understanding of the rule. Shifting the uncertainty isn't simply shifting the errors. Thankfully, it should be possible to talk sensibly about what the uncertainty is for Hawk-eye, and tests have been done by the MCC. Questions remain, as the reports are light-on. Kartikeya rightly objects to two-part classification of “normal” and “extreme” LBWs, and to the 'average error', suggesting the median rather than the mean. We don't know how broad this test was. I understand that pitch variation and calibration based on play are fairly unimportant, but Soulberry's reference to all the bowling variations is important here. I'd prefer an indication of the distribution of errors for a whole range of deliveries, varying over the distance and other factors and summarised with a confidence interval, rather than any sort of average or claimed absolute maximum.

That is the technical side. There is a protocol for its use in the context of the UDRS. The uncertainty is not ignored at all - while the beneficiary of the doubt in this system is controversial, a half-ball's width (let alone 45mm) is generous enough to make worrying about errors as large as half the stump's width quite ridiculous. On top of that, the Hawk-eye document implies that the protocol also prevents LBW decisions when the batsman comes too far down the pitch, following traditional umpiring rather than the letter of the law. It might fall to further criticism, but I wouldn't say Hawk-eye has been “uncritically adopted”. It certainly hasn't been adopted as though it gives certainty about the trajectory.

Yes, it is convenient for the marketing of Hawk-eye to ignore the uncertainty. It probably does give the wrong impression sometimes, but I don't agree that it would be less persuasive if a range of likely locations were shown. Most of the commentators who leap to absolute statements are of the sort that did that anyway when given much less. Most of us are ok with the idea of an approximation at one level or another. I am probably not the best sample, but I would find an indication of the confidence interval more persuasive on first look. I don't have any problem with a commentator saying “Hawk-eye gives greater than 95% probability of hitting the stumps, so that's a good decision.” In general, I'd love to see an understanding of the uncertainty involved encouraged with that sort of display, although in practise it's not obvious how to show it simply without giving all sorts of other impressions. (Mind you, it's a bit funny to worry about showing about the Hawk-eye uncertainty in an application that doesn't even claim to perfectly display the Hawk-eye results!)

All in all, I think Hawk-eye can be used sensibly. Any suggestion that Hawk-eye gives certainty where an umpire speculates should be corrected, but that correction shouldn't simply rubbish the claim, but replace it with an good idea of how accurately a the Hawk-eye system performs that same speculation. Further testing showing greater errors, or concerns about the reliability or even integrity of those operating the system, may give reason to prefer a human umpire, but not simply the fact that the technology is limited and has been over-rated.