Quality AI-model in 2023

The most frequently asked question to everyone of Cycling Oracle is: "How accurate is this computer-model with its predictions"?  We found an answer for the 2023 season: quite accurate.

CyclingOracle predicts the results of all major cyclingraces for men and women. The model looks at the qualities of riders, the race-profile and the strength of the field of participants. More detail and background information about the AI model can be found in this article. Although looking ahead and looking forward to the next race is CyclingOracle's motivation, we also like to evaluate the quality of the predictions. To what extent is the prediction corresponding with the final result? And which races are more easily predictable by the model?

To assess the quality of the prediction, points are given to riders in the prediction and their performance. If a rider from the top-10 of the prediction finishes in the top-20 of the race, the model receives points. The computer gets more points if a rider finishes higher. In addition, the performance of the predicted winner is more important number 10's result. Predicted winners get a higher multiplication factor. An overview of the points and factors can be found in table 1.

It allows us to compare the scored points with the maximum amount of points available, which results in a percentage. What percentage of the best possible predictions does Cycling Oracle's model get?

One-day races Men Elite

If we look at the top-10 prediction of the 14 largest one-day races for men, the computer gets an average score of 65%. The AI model's prediction therefore achieves an average of 65 percent of the maximum possible points according to the scoring system in these races. This score varies between 36 and 83 percent per race. The best predicted race of 2023 was the E3 Classic. The computer predicted the podium completely correct: Van Aert won, ahead of Van der Poel and Pogacar. Mohoric, Pedersen and Küng were also in prediction top-10 and scored points for the computer by finishing top-20.

Omloop Het Nieuwsblad was the one-day race that the computer predicted worst: a score of 36 percent on the top-10 prediction. The predicted winner Laporte still finished on the podium and number 2 in the prediction Kristoff finished fourth, but otherwise only Pidcock and Pasqualon from the top 10 actually managed to finish in the top-20 of the season-opener.

6 out of 14 predicted winners actually won and 28 of the 42 predicted riders from the top-3 also managed to finish on the podium. In addition to the E3 Classic, the Ronde van Vlaanderen, Paris-Roubaix, Il Lombardia and World Time Trial Championships were also predicted very well by the AI model (score well above 70%). The Flèche Wallonne and Gent-Wevelgem were two races that scored lower (well below 60%).

Tour de France Femmes

In the Tour de France Femmes the computer achieved a reasonably stable form. Stage 5 (won by Bauernfeind from the breakaway) was predicted worst by the computer with a score of 29 percent. On the other hand, the penultimate stage with a finish on the Tourmalet and the final time trial were predicted very well with a score of 84 percent. 9 of the 10 cyclists from the predicted top-10 finished in the first 15 in the daily results. Only Norsgaard did not manage to score any points for the model (41st instead of the predicted 8th place). In stage 7, even all 10 riders from the top-10 prediction finished top-20.

The model correctly predicted the winner of a stage twice, 7 times the predicted winner was on the podium and always (9/9) in the top-5. In three stages (1, 4 and 6) the entire top-3 prediction was correct, except that one escapee was finishing in front off the peloton. It's competley normal the AI model predicted Van Vleuten as the overall winner after her 5 consecutive victories in the classification of the largest stage races. Vollering beated her, resulting in a quality-score of the GC of 68 percent.

Tour de France

The AI model scored well in predicting flat stages and time trials in the 2023 Tour de France (Men). Also mountain stages contested by the strongest GC-riders, get a high score. Those stages where the breakaway fights for victory turn out to be more difficult to predict. Top-10 prediction for stage 16, ITT to Combloux, was the best. The predicted winner (Pogacar) came second, but all podium finishers (Vingegaard, Pogacar and Van Aert) were also ranked top-3 by the AI model. Only one rider from the predicted top-10 finished outside top-20 (Hindley finished 24th). Prediction-quality is therewith 92 percent of maximum. The computer prediction of stages 4 and 11 (both won by Philipsen), 14 (Carlos Rodríguez) and 21 (Jordi Meeus on the Champs Elysees) also scored above 80 percent.

Opposite the accurate predictions in aforementioned stages, the prediction-quality was low in stages 5, 9, 10, 12 and 15. Out of the prediction top-10 of stage 5, only Daniel Felipe Martinez (prediction 4th) finished top-10: 7th in the stage to Laruns won by Hindley. It resulted in a score of 4% of maximum. Stage 10 was contested by the breakaway (Bilbao and Zimmermann), like predicted, but the computer predicted some other riders to win (Van der Poel and Wright) Dat is een schamele 4% van de maximaal mogelijke score. Etappe 10 werd wel voor de vluchters (Bilbao en Zimmermann): 8 percent score thanks to a top-10 result by Alaphilippe.

Finally, if we look at the performance of the GC prediction in the Tour de France, the computer (and CyclingOracle) can be quite satisfied. The predicted numbers 1 and 2 finished in the same order: Vingegaard defeated Pogacar. Numbers 3, 5 and 6 in the prediction unfortunately crashed early in the race (Mas, Carapaz and Bardet) and Adam Yates, as Pogacar's master servant, performed better than expected (3rd instead of 8th). The top-3 prediction scores 83 percent of the maximum and the top-10 prediction prior to the Tour de France achieves a score of 77 percent.

Move on to 2024

In 2024 we will reflect more frequently and extensively on the performance of the AI model by assessing it according to the method used in this blog. Studying the strengths and weaknesses of the computer helps to improve the AI model.

If you have any questions or suggestions related to the model or our work: feel free to connect. Slide into our DM at Instagram, or send us an e-mail at [email protected].