2025 · Preprint
Attention is All You Need Until You Need Retention
Yaşlıoğlu, M. M.
arXiv, vol.1, no.1, pp.1-20
Can attention substitute for retention in language models?
The success of transformer architectures is attributed to attention, yet as context windows grow it becomes visible that retention is not the same thing. This preprint discusses the distinction between attention and persistent memory, and its architectural consequences.
Being able to look at something is not the same as remembering it.
AIAttentionRetentionLLMArchitecture
How to cite
Yaşlıoğlu, M. M. (2025). Attention is All You Need Until You Need Retention. arXiv, vol.1, no.1, pp.1-20.
@misc{yaslioglu2025attention,
author = {Yaşlıoğlu, M. M.},
title = {Attention is All You Need Until You Need Retention},
year = {2025},
journal = {arXiv, vol.1, no.1, pp.1-20}
}