Companies must master AI—and all techniques, whether it is (un)supervised learning or reinforcement learning as is set to revolutionise predictive powers and maximise chances of success in sports and other fields.
ChatGPT has revolutionised the field of conversational artificial intelligence (AI). Beyond its ability to generate effective human-like responses, the breakthrough has been to integrate big data (large language) models with deep reinforcement learning techniques enhanced by human feedback (RLHF).1 That is, reinforcement learning is an AI technique whereby the agents in the model learn from their actions through rewards and penalties in function of good/bad actions—but in the case of Open AI’s Chat GPT and also other high profile AI applications like DeepMind’s Sparrow),2 rewards are specifically provided by a set of human interventions, allowing machines to grasp elements of decision-making distinctly embedded in human experience.
This part of feedback, even if proven to be effective, may however be subject to biases if human interventions tend to cater to certain preferences. Still, the key generative AI technology here is reinforcement learning which might become a critical technique of machine learning. Further, the use cases can be enlarged by adding gamification,3 where the reinforcement will come from the winning rewards for instance in education, training, and many other cases.
Beyond ChatGPT: the value of reinforcement learning in sport analytics
In order to understand the value of reinforcement learning, a case in point is big data sports analytics. Money in sports has become really big, and any possibility to leverage data to better predict a team´s or player´s relative performance to its competitors may guarantee large payoffs.
In the 1980s, Bill Benter became one of the most profitable gamers of all time by leveraging his own built statistical prediction model for horse racing. Twenty years later, using stats in baseball, coach Billy Beane managed to win the series with the Oakland Athletics,4 building a team with cohesive strength, which however was not necessarily seen as the most performing in the sport by the not analytically trained baseball insiders. What AI and replacement learning further bring beyond those first data-predictive successes, is a much broader set of big data and flexible predictive models that ultimately may provide even more powerful insights as to who will be winning a horse race or which team will win a championship.
Take football for example as the number one global sport. The issue with football is that the average number of goals per game is typically low (it was 2.6 for the most recent World Cup in Qatar) as is the percentage of time “creating or conceding a goal” (in the range of 2% of the total time of the game). Faced with this infrequency, it is rather difficult to predict the odds of performance unless based on past performance, or the prevalence of known scorers, for example. But with big data, the best sport AI analysts would now ingest thousands of games, with hundreds of features to refine the predictive power of their analytics. These days, one can collect data on pretty much anything, from the location of players, the chain of their moves, timing of actions, type of plays (defence/offence), features such as shots and passes, the direction of play (lateral, backward, forward), the velocity of actions, and many more. All those actions that contribute the bulk of time of the game beyond scoring, should bring significant insight for better prediction and for maximising the chance of success.
Such reinforcement learning tools are now becoming the reference, being used in ice hockey, rugby, basketball, and American football(gridiron football), for a variety of uses such as player scouting or valuation, or field strategies. An extra decade after Beane’s success story, the director of data research and analytics of Liverpool Football Club convinced the leadership to acquire both Sadio Mane and Mohamed Salah each for less than 40 million pounds. Those two players were instrumental in making the club win the Champions League, and are still today perceived as top players in the UK and European leagues.5 Needless to say, both players are likely now worth a few multiples of their original transfer price.
As an example of insights, consider a model that evaluates each player´s on-ball action based on its probability of creating and conceding a goalscoring opportunity in the context in which the action occurred. This framework6 is much more complete than looking at goals only as it considers all types of technical actions like passes, crosses, dribbles, take-ons, shots, interceptions, and tackles. The framework is also only possible because of data tracking and sophisticated machine learning tools that can assess all complex combinations of actions among different players. Looking at model results for the UK league, for example,7 a ton of new insights can be obtained, such as:
- 10% of Premier League players have negative value, that is, their actions help the competing team. Not good for sure.
- On average, the most valuable player is the keeper, not the central forward.
- Left-position players tend to be slightly more valuable than right players. Not sure why but the wisdom is that left players have higher velocity in sports (tennis is another example).
- Value goes slightly up the more forward a player is (remember that this is not tautological as each action is weighted by its probability of creating/conceding a goal).
- There is a real trade-off between quantity and quality of actions.
- But a group of 20% of players is also able to deviate from this trade-off and boost both the quantity and quality of actions. Needless to say, those are the most interesting players.
- In general, the value of a player varies from 1 to 2, for each position. This is really significant: imagine you have 11 players at the top end of the
value range; you are sure to win. - The most valuable player in the league is a Belgian midfield player (Kevin De Bruyne). He also has value statistics up to 5 times the average value of his competitive peers with the same midfield position.
- Liverpool and Manchester City have the largest pool of most valuable players.
- 95% of on-ball actions do not directly change the score but influence the game indirectly. This deep data underground is where the value of AI reinforcement learning lies.
Reinforcement learning will become mainstream, get ready to use it
Companies must master AI—and all techniques, whether it is (un)supervised learning or reinforcement learning. Besides cases of sports/games and chatbots, here are a few examples that prove its wide applicability. In healthcare, reinforcement learning has been used for lung cancer and epilepsy and the use of erythropoiesis-stimulating agents (ESAs) in patients with chronic kidney disease. In industries, a large set of manufacturing companies are propelling the automation of their factories by using deep reinforcement on robots to learn how to optimise tasks for the best efficacy, speed, and precision. In retail, the personalisation of product promotion is based largely on reinforcement learning algorithms.
We are just at the start of the AI revolution but managers should urgently be aware that new algorithms and techniques such as reinforcement learning are now set for prime time.
This article was originally published on May 4, 2023.
About the Author
Jacques Bughin is CEO of MachaonAdvisory, and a former professor of Management while retired from McKinsey as senior partner and director of the McKinsey Global Institute. He advises Antler and Fortino Capital, two major VC/PE firms, and serves on the board of multiple companies.
References
1 Learning from Human Feedback: Challenges for Real-World Reinforcement Learning in NLP. 2020. Challenges of Real-World RL Workshop at NeurIPS 2020. https://research.google/pubs/pub49732/.
2 Perspectives on the Social Impacts of Reinforcement Learning with Human Feedback. 2023. ArXiv. https://arxiv.org/pdf/2303.02891.pdf.
3 onvergence of Gamification and Machine Learning: A Systematic Literature Review. 2021. Tech Know Learn. https://link.springer.com/article/10.1007/s10758-020-09456-4.
4 The Lessons of Moneyball for Big Data Analysis. DataCenter Knowledge. 2011. https://www.datacenterknowledge.com/archives/2011/09/23/the-lessons-of-moneyball-for-big-data-analysis.
5 Evaluating Soccer Player: from Live Camera to Deep Reinforcement Learning. 2021. ArXiv. https://arxiv.org/abs/2101.05388.
6 Actions Speak Louder Than Goals: Valuing Player Actions in Soccer. 2018. ArXiv. https://arxiv.org/abs/1802.07127.
7 Bringing objectivity and predictability to one of the most diverse and opiniated sports in the world by leveraging data. 2022. Repositório Universidade Nova. https://run.unl.pt/handle/10362/142482.