(Understanding AI) Context rot: the emerging challenge that could hold back LLM progress

What if attention isn't all you need?

Nov 10, 2025

∙ Paid

The final chapter of this series that Tim has been writing for a year. If SAIL subscribers like this type of content, we may ask Tim to publish the previous episodes as well!
Understanding AI
Why large language models struggle with long contexts
It’s Future of Transformers Week at Understanding AI! All week I’ll be examining the limitations of today’s LLMs and discussing possible solutions…
Read more
2 years ago · 113 likes · 9 comments · Timothy B. Lee

Many people believe that…

SAIL Media

(Understanding AI) Context rot: the emerging challenge that could hold back LLM progress

What if attention isn't all you need?

This post is for paid subscribers