News
How sparse attention solves the memory bottleneck in long-context LLMs
1+ mon, 1+ week ago (296+ words) LLMs are getting pulled into longer and messier workflows, handling large inputs and generating longer and longer token sequences. Coding assistants need to keep track of repositories, issue threads, terminal outputs, and earlier edits. Research agents need to carry facts…...
How Databricks’ FlashOptim cuts LLM training memory by 50 percent
4+ week, 18+ hour ago (762+ words) How Databricks" FlashOptim cuts LLM training memory by 50 percent'TechTalks This article is part of our coverage of the latest in AI research. Training large language models is an expensive endeavor, largely due to the massive accelerator memory required for each…...
How Sakana AI’s new technique solves the problems of long-context LLM tasks - TechTalks
1+ mon, 4+ week ago (236+ words) A new technique developed by researchers at Sakana AI, called Context Re-Positioning (RePo), allows Large language models (LLMs) to dynamically re-organize their internal view of their input data to better handle long-context tasks." LLMs process information in a strictly linear…...
Recursive Language Models: A new framework for infinite context in LLMs
2+ mon, 4+ day ago (436+ words) Recursive Language Models (RLMs), a new framework developed by researchers at MIT CSAIL, provide a solution to the limited context window of large language models (LLMs). This approach enables models to process arbitrarily long prompts without incurring massive memory costs…...
How test-time training allows models to ‘learn’ long documents instead of just caching them
2+ mon, 2+ week ago (305+ words) To understand the significance of this approach, it is necessary to look at the current tradeoff between accuracy and efficiency when working on longer contexts. Full-attention transformers are currently the gold standard for accuracy because they are designed to recall…...