The Achilles’ Heel Of AI: How A Major Tech Risk To Your Business Could Be One You Haven’t Heard Of - And What You Should Do


In an era increasingly defined by the transformative power of AI, business leaders face a critical challenge: the risk of model collapse. This phenomenon, where AI becomes unstable or ceases to function effectively, is a looming threat with profound implications for our reliance on the critical technology of our age.

Model collapse stems from using AI-generated data when training or refining models rather than relying on information directly generated by human beings or devices other than the AI systems themselves. It comes about when AI models create terabytes of new data, which contain little of the originality, innovation, or variety possessed by the original information used to “train” them. Or when AI models are weaponised to generate misinformation, deep fakes, or “poison” data. A downward spiral can result in progressively degraded output, leading to model collapse. The consequences could be far-reaching, potentially resulting in financial setbacks, reputational damage and job losses.

In my talk, I will dive into this little-known risk, drawing on insights from my research and that of others. I’ll explore how the quality and provenance of data are too often overlooked in business decisions about the implementation and use of AI tools. Yet data plays a pivotal role in determining these systems' reliability, effectiveness - and value to the bottom line. I’ll cite real-world examples, such as the challenges faced by ChatGPT and Stack Overflow, to understand the tangible impact of the phenomenon.

I’ll also talk about potential solutions for mitigating model collapse and outline a roadmap for businesses to foster a strong data infrastructure on which to base their AI strategies.

Ultimately, this talk aims to empower businesses with the knowledge, understanding, and tools to navigate the complexities of this new frontier of technology safely and effectively. We’ll see how, by fostering greater data literacy and committing to the provenance of data, we can ensure that AI remains a valuable business asset rather than a potential liability.


Sir Nigel Shadbolt is Executive Chair of the Open Data Institute, which he co-founded with Sir Tim Berners-Lee, and is one of the UK’s foremost computer scientists.  

He is a leading researcher in artificial intelligence and was one of the originators of the interdisciplinary field of web science. He is Principal of Jesus College Oxford, a Professor of Computer Science at the University of Oxford and a visiting Professor of Artificial Intelligence at the University of Southampton.

In 2009 the Prime Minister appointed him and Sir Tim Berners-Lee as Information Advisors to transform access to Public Sector Information. This work led to the highly acclaimed site that now provides a portal to tens of thousands of datasets. In 2010, he joined the UK government’s Public Sector Transparency Board – overseeing Open Data releases across the public sector.

He is a Fellow of the Royal Society and the Royal Academy of Engineering, and a Fellow and former President of the British Computer Society. He was knighted in 2013 for ‘services to science and engineering’.