Exploring Duckworth-Lewis: What makes a good model?


Cricket was plagued by a difficult question for 30 years until the Duckworth-Lewis method provided an elegant solution which provides a great case study for would be model-builders.

Cricket has a lot of idiosyncrasies that endear the sport to its legion of fans,but equally confuse a first-time spectator. From field positions with names like ‘Silly Mid Off’ to stopping for ‘Tea’ and variations in format producing matches varying length from half a day to up to five, the game is difficult for a newcomer to access.

Test cricket – the sport’s oldest and purest form – follows a four innings format that can last up to five days; if the four innings aren’t completed – for example, due to weather interruption – the game is considered a draw. This may not seem fair in many circumstances, but that is the rule and game strategy has developed to account for it

As cricket has looked to broaden its interest, shorter formats with a focus on taking the game to conclusion (much better for fans and bettors) have become increasingly important. The Limited Overs format – literally the number of overs each team bats for is fixed at either 20 or 50 – have certainly became more popular, condensing action into a single day.

Cricket’s big question

Limited Overs cricket – in conjunction with the famous British weather – produced a conundrum that lasted 30 years, before being eventually solved by Frank Duckworth – a mathematical scientist working for the Atomic industry – and Tony Lewis – a lecturer in Quantitative Research Methods.

The question was how to calculate a fair target score in the face of interruptions in player due to rain or bad light. This is essentially predicting what the chasing team should be expected to score under equal conditions, which is the kind of question bettors spend their time trying to model in other contexts.

Here is an example of the question that needed solving:

Limited Overs Match – 50 Overs

Team A completes 50 overs, scoring 240 runs;

Team B bats for only 20 overs scoring 96 runs for the loss of 4 wickets

It rains, leaving time for only 10 more Overs.

What is the fair target for Team B to chase with the remaining 10 overs?

Starting with a hypothesis

When you build a model you start with a hypothesis. The initial hypothesis cricket applied – and one which on the face of it appears the most logical – was to apply a simple pro-rata calculation.

Team A scored runs at 240 in their full allocation of 50 overs, so if they had only faced 30 overs (60%) – what Team B is now limited to –  this would suggest a target score of 240*0.6=144.

Testing your hypothesis – a lot

Unfortunately, the pro-rata approach produced skewed results as it is far easier to score 144 runs from 30 overs than 240 from 50 because you are less likely to run out of wickets in the shorter period. This was proven by testing the hypothesis – looking back at the huge history of completed games and seeing if the logic worked. However there was evidence that the pro-rata approach didn’t

Basing your handicapping on a flawed logic is a common problem that experienced bettors will recognise, which is of course why models need testing and constant revision.

With pro-rata discarded a more sophisticated approach was then applied using the runs scored in the 25 most productive Overs in Team A’s Innings and extrapolating the target; but this also led to issues, as in cases where the interruption was relatively brief the target score was unaffected. One of the most famous being in the 1992 World Cup when South Africa required 22 from 13 deliveries, but rain reduced this to 21 runs off a single ball1.

After pouring over historical data, the stroke of genius that came from Duckworth and Lewis was considering wickets and runs together as a measureable resource so that an adjusted fair target – also known as the Par Score – can be reached and adjusted across all circumstances.

Here is the basic Duckworth-Lewis formula:

Team B’s target score:

Team A’s score x Team B’s Resource²

Team B’s resource:

(Starting Resources – Resources Used + Resources Remaining)

We can then apply this formula to the question above.

Team B’s target score:

240 x 66.7% (100% – 61.6% + 28.3%2) = 160.1

As Team B have already scored 96 runs they need to score another 65.1 runs to win.

The D/L approach produces a target that is 17 runs higher than pro-rata, illustrating how our intuition – which immediately suggested the pro-rata solution – can produce quick assessments that are often ill-suited to probabilistic reasoning. It is easy to think you have an effective model, but very hard to actually build (and demonstrate) a robust model.

Minimum data & robustness

One key component of building an effective model is knowing the minimum amount of data that is required to make the output valid. This is true of D/L which requires a minimum number of overs to have been faced to apply. For 50-over matches, each team must face at least 20 overs for the result to be valid. For Twenty20 games each side must face at least five overs.3

Given the huge amount of available dataDuckworth and Lewis were able to backfit and test their model, with each additional limited overs game adding to the database of results and thereby strengthening its accuracy.

Models should also be robust enough to work in different contexts so long as the underlying systems are the same. In the case of cricket this means so long as the same rules apply, D/L should work anywhere in the world – it does. This should be true of any model, though often back-fitting is used to reverse engineer the logic.

Predicting the future

The reason that this whole story is relevant for bettors is that the question Duckworth and Lewis faced is the same challenge that handicappers face – how to predict the future. The question cricket has tried solving was much more complex than was initially appreciated and many still argue that D/L has limitations.