(Understanding AI) Context rot: the emerging challenge that could hold back LLM progress

What if attention isn't all you need?

Nov 10, 2025

∙ Paid

The final chapter of this series that Tim has been writing for a year. If SAIL subscribers like this type of content, we may ask Tim to publish the previous episodes as well!
Understanding AI
Why large language models struggle with long contexts
It’s Future of Transformers Week at Understanding AI! All week I’ll be examining the limitations of today’s LLMs and discussing possible solutions…
Read more
a year ago · 113 likes · 9 comments · Timothy B. Lee

Many people believe that…

Keep reading with a 7-day free trial

Subscribe to SAIL Media to keep reading this post and get 7 days of free access to the full post archives.