Research Paper | Dylan Chiang

Paper reading: LLMs CAN GET 'BRAIN ROT'! - Research on cognitive decline in large language models

Fri, 31 Oct 2025 00:00:00 +0000

I recently read the important research paper “LLMs CAN GET “BRAIN ROT”!” from institutions such as the University of Texas at Austin, Purdue University, and Texas A&M University. This study proposed and verified the “LLM Brain Rot Hypothesis” for the first time, and found that continued exposure to spam online text will lead to long-lasting cognitive decline in large language models. This is a very warning discovery.

Research background and assumptions

Source of inspiration

“Brain Rot” was named the word of the year by Oxford Dictionary in 2024. It is used to describe the cognitive decline caused by modern people’s addiction to a large amount of trivial and unchallenging online content. This study shows that the impact of Internet addiction on human cognition is mainly reflected in three dimensions:

Attention Capacity: The constant flow of online information undermines the ability to concentrate
Memory Process: Rich online information changes the way knowledge is stored and retrieved
Social Cognition: Online interactions reshape self-concept and affect self-esteem

Research questions

Since large language models acquire human-like cognitive capabilities by learning trillions of data on the Internet, will they also experience a similar “Brain Rot” phenomenon? The research team established the LLM Brain Rot Hypothesis: Continuous pre-training on junk web text will lead to long-lasting cognitive decline in large language models.

Experimental design and methods

Garbage data definition

To test the hypothesis, the research team constructed spam and control datasets from social media (Twitter/X) and proposed two orthogonal spam data measures:

M1 (Engagement): Based on the popularity of tweets (number of likes, retweets, replies) and length (number of tokens), select short but highly popular content as spam data

M2 (Semantic Quality): Based on content semantic quality, including:

Conspiracy theories, exaggerated claims or unfounded assertions
Sensational headlines and clickbait language
Superficial topic content
Attractive style

Experimental model

The study was conducted on four pre-trained and instruction-tuned models:

Llama3 8B Instruct
Qwen2.5 7B Instruct
Qwen2.5 0.5B Instruct
Qwen3 4B Instruct

Benchmark test

The study assessed multiple dimensions of cognitive function:

Cognitive Function	Benchmark Testing	Assessment Content
Reasoning Skills	ARC Challenge	Scientific Problem Solving Skills
Long Context Understanding	RULER	Long-term memory retrieval and comprehension
Ethics	HH-RLHF, AdvBench	Safety Compliance Ability
Personality Traits	TRAIT	The Big Five and the Dark Triad

Key findings

1. Garbage intervention leads to cognitive decline

The study found that the junk intervention produced non-trivial effects on reasoning and long-context ability (Hedges’ g > 0.3). In particular, the M1 (engagement) intervention caused more significant impairments in functional cognition (reasoning or long context) and safety.

2. Dose response effect

Experiments on Llama3 8B Instruct show that when the proportion of garbage data increases from 0% to 100%:

ARC-Challenge (COT): 74.9 → 57.2 (down 17.7 points)
RULER-CWE: 84.4 → 52.3 (down 32.1 points)

This demonstrates a clear dose-response relationship between junk data and cognitive decline.

3. Changes in personality traits

Litter intervention not only affects cognitive abilities, but also changes LLM’s personality traits:

Negative changes:

Increased levels of psychosis
Enhance narcissism and Machiavellian traits
Decreased agreeableness

Positive changes:

Increased openness and extraversion (in some cases)

4. Differences between M1 and M2

The study found that M1 (engagement) and M2 (semantic quality) interventions produced distinct effects. The M1 intervention resulted in more negative effects, especially on safety and personality traits, demonstrating that engagement is a new dimension independent of semantic quality.

Failure mode analysis

###Thought-Skipping

By analyzing the reasoning process of LLM in the ARC task, the study identified five typical failure modes, three of which are related to “thinking jumps”:

No thinking: The model answers directly without thinking.
No plan: The model starts thinking without developing a step-by-step plan.
Jump Steps: Starting reasoning but not completing all planning steps

More than 98% of failure cases are related to thinking jumps. In M1 garbage intervention, 84% of failures belong to the “no thinking” type.

Popularity vs Length

Research has found that popularity (a non-semantic indicator) is a better indicator of the Brain Rot effect than length:

Popularity plays a more critical role in reasoning tasks
Length is more important in long context understanding
Both have different effects on different tasks

Mitigation attempts and persistence

1. Reflective Reasoning

Try two reflection methods to fix mental jumps:

Self-Reflect: Model self-reflection (limited effect)
Ext-Reflect: Provide feedback using an external strong model (GPT-4o-mini)

The results show that even with strong external reflection, the model cannot fully recover to baseline levels.

2. Posterior instruction tuning

The study also tested extended instruction tuning and continuous control training:

Even with 4.8x more garbage intervention instruction tuning data
Still unable to completely reverse the Brain Rot effect
Significant gaps with benchmarks remain: ARC-C COT (17.3%), RULER (9%), AdvBench (17.4%)

This shows that the Brain Rot effect has been deeply internalized and existing mitigation methods cannot fundamentally solve the problem.

Significance and Enlightenment

1. Warning on LLM training

This study provides the first significant evidence of data quality as a causal driver of LLM capability degradation, re-considering continuous pre-training data management as a safety issue during training.

2. “Cognitive health check” is required

The findings call for routine “cognitive health checks” for deployed LLMs, similar to health monitoring in the medical field.

3. The urgency of data curation

As LLM continues to scale and ingest ever larger amounts of network data, careful data curation and quality control are critical to preventing cumulative damage.

Thinking and Reflection

This research reveals a disturbing reality: the social media content we are exposed to every day may not only affect human cognition, but also impair the “cognitive” capabilities of AI models. While LLMs obviously do not have the same “grey matter” or “neurons” as humans, they do have parameters and attention mechanisms that can be similarly “overfitted” or “distracted” by certain data patterns.

The most worrying finding in the study is that the Brain Rot effect persists even with posterior tuning using large-scale clean data. This implies that we need to fundamentally rethink data collection and pre-training practices, focusing not only on model performance, but also on the “cognitive health” of the model over the long term.

Conclusion

“LLMs CAN GET “BRAIN ROT”!” This research contributes valuable insights to the field of AI security, and is the first to systematically prove the negative impact of spam online text on large language models. The research not only verified the LLM Brain Rot hypothesis, but also revealed the fine mechanism of cognitive decline, pointing out the direction for future AI safety research.

With the rapid development of AI, we must face the importance of data quality, establish stricter data curation standards, and develop an effective AI “cognitive health” monitoring mechanism. Only in this way can we ensure that AI systems maintain their due “cognitive purity” while serving humans.

Paper Information:

Title: LLMs CAN GET “BRAIN ROT”!
Authors: Shuo Xing, Junyuan Hong, Yifan Wang, Runjin Chen, etc.
Institution: University of Texas at Austin, Purdue University, Texas A&M University
Published: arXiv:2510.13928v1 [cs.CL] October 15, 2025
Paper address:
Project Page:

Paper reading: Aegaeon - Efficient GPU pooling technology for concurrent LLM services on the market

Wed, 22 Oct 2025 00:00:00 +0000

I recently read an important paper “Aegaeon: Effective GPU Pooling for Concurrent LLM Serving on the Market” that will be published in SOSP ‘25. This research, completed by Peking University and Alibaba Group, proposes a revolutionary multi-model LLM service system, which greatly improves GPU resource utilization efficiency.

Research background and challenges

Resource waste problem in the model market

With the rapid development of large-scale language models, model marketplaces such as Hugging Face now host over a million models. However, there is a serious waste of resources in the actual production environment:

Obvious long tail effect: 94.1% of models receive only 1.35% of requests
Uneven Resource Allocation: 17.7% of the GPU is used to serve a “cold” model that averages less than 0.2 requests per second
Difficult to cope with burst loads: “Hot” models (such as DeepSeek, Llama, Qwen) face request bursts and may exceed reserved resources

Limitations of existing solutions

Existing GPU pooling solutions mainly fall into two categories:

Multiplexing: Limited by GPU memory capacity, supports up to 2-3 models/GPU
Auto-scaling: Expanding at the request level, there is a head blocking (HOL) problem

Aegaeon’s Core Innovation

Token level automatic expansion

The biggest innovation of Aegaeon is the token-level automatic expansion mechanism, which is in contrast to the request-level expansion of existing solutions:

Request Level Expansion: Must wait for the entire request to complete before switching models Token level extension: models can be switched predictively during token generation

Separate scheduling architecture

Aegaeon divides the GPU pool into two partitions:

Prefill instance: Specially handles the initial processing of input prompts
Decoding instance: specifically handles subsequent token generation

This separation avoids the complexity of unified scheduling and enables more balanced resource utilization.

Scheduling strategy

Prefill phase scheduling:

Adopt grouped FCFS (First-Come-First-Serve) strategy
Group requests for the same model, maximum group size is 8
Prioritize loading into existing groups to reduce over-expansion

Decoding phase scheduling:

Use Weighted Round-Robin strategy
Assign time quotas based on SLO requirements
Utilize output buffering to hide latency

System optimization technology

Component Reuse

Aegaeon significantly reduces initialization overhead by reusing inference engine components:

Distributed actuators (Ray, NCCL)
Analyze and optimize components
Tokenizer
Memory pool

Explicit memory management

Self-managed VRAM buffer:

Allocate all necessary VRAM at once
Use bump allocation strategy to avoid fragmentation
Bypass the allocation mechanism of the tensor library

Unified KV Cache:

Using slab allocation technology
Maintain dedicated pool for each shape of KV cache
Effectively manage memory fragmentation

Fine-grained KV cache synchronization

Implementing asynchronous KV cache transfers using CUDA events:

cudaEventRecord: Log transfer events
cudaEventQuery: Query completion status
cudaStreamWaitEvent: synchronized execution sequence

Experimental results and performance evaluation

End-to-end performance

Test results on the ShareGPT dataset:

RPS = 0.1: Aegaeon supports 70 models, 2x more than ServerlessLLM
RPS = 0.5: Aegaeon achieves 2.5x higher request arrival rate
Single GPU Efficiency: Supports up to 7 models/GPU

Performance under strict SLO

Even under stricter SLO requirements:

0.5× SLO: still supports 50% more models than the baseline solution
0.3× SLO: 12.5% more models supported

Automatic expansion speed

Near-instant scaling (via prefetching) 50% of the time
Expansion completes within 1 second in other cases
KV cache transfer overhead per request is less than 1 second

Production environment deployment

Deployment scale

Aegaeon has been tested and deployed in Alibaba Cloud Model Studio for three months:

GPU Scale: 213 H20 GPUs
Number of models: 47 models (28 1.8-7B, 19 32-72B)
Resource Savings: Reduced from 1,192 GPUs to 213 (82% savings)

Actual performance improvement

GPU utilization improved from an average of 13.3%∼33.9% to 48.1%, with no SLO violations or service interruptions observed during the 70 hours of monitoring.

Summary of technical contributions

Revealing for the first time the excessive cost problem of concurrent LLM services in the market
The first token-level automatic expansion multi-model service solution
First comprehensive optimization Predictive automatic expansion process, reducing overhead by 97%
Real production deployment verification, proving the ability to significantly reduce OPEX

Future development direction

Although Aegaeon has made breakthrough progress, there is still room for improvement in the following areas:

Support larger model clusters
Optimize performance in extremely low latency scenarios
Explore combinations with multiplexing technology
Expand to more diverse hardware platforms

Conclusion

Aegaeon represents an important milestone in the design of LLM inference systems. Through token-level automatic expansion and full-stack optimization, it not only solves the resource waste problem faced by the model market, but also provides a technical foundation for the sustainable development of cloud AI services.

📄 Paper information

Paper Title: Aegaeon: Effective GPU Pooling for Concurrent LLM Serving on the Market
Authors: Yuxing Xiang, Xue Li, Kun Qian, Yufan Yang, Diwen Zhu, Wenyuan Yu, Ennan Zhai, Xuanzhe Liu, Xin Jin, Jingren Zhou
Institution: Peking University, Alibaba Group
Conference: ACM SIGOPS 31st Symposium on Operating Systems Principles (SOSP ‘25)
Year of Publication: 2025
DOI:
PDF link:

Paper Reading: How Prompt Word Politeness Affects LLM Accuracy: Mind Your Tone Paper Analysis

Fri, 17 Oct 2025 00:00:00 +0000

I recently read an interesting research paper from Penn State University called “Mind Your Tone: Investigating How Prompt Politeness Affects LLM Accuracy”. This study explores how the politeness of prompt words affects the accuracy of large language models. The results are quite surprising.

Research background and methods

The researchers created a dataset of 50 foundational questions covering the fields of mathematics, science and history. Each question was rewritten into five different tone versions:

Very Polite (Very Polite)
Polite (Polite)
Neutral (Neutral)
Rude (Rude)
Very Rude

A total of 250 unique prompt words were generated and tested using ChatGPT-4o.

Unexpected discovery

The findings were counter-intuitive: Impolite prompt words actually performed better than polite prompt words.

Very Polite: 80.8% accuracy
Courtesy: 81.4% accuracy
Neutral: 82.2% accuracy
rude: 82.8% accuracy
Very Rude: 84.8% accuracy

Paired samples t-test showed that these differences were statistically significant (p < 0.05).

Example of prompt word tone

The researchers designed different prefixes for each politeness level:

Very polite example:

“Can you kindly consider the following problem and provide your answer.”
“Would you be so kind as to solve the following question?”

Very rude example:

“You poor creature, do you even know how to solve this?”
“Hey gofer, figure this out.”

Differences from previous studies

This finding differs from previous research. Research by Yin et al. (2024) shows that “rude prompt words generally lead to poorer performance”, but they used older models such as ChatGPT-3.5 and Llama2-70B.

The researchers speculate that newer generations of LLM may have different response patterns to changes in tone.

Possible explanation mechanisms

Perplexity Effect: Mood vocabulary may affect the confusion of prompt words, thereby affecting model performance
Length factor: Polite expressions usually require more words, which may affect processing efficiency
Training data bias: The model may have been exposed to data distributions with different tones during the training process

Research limitations and future directions

Limitations:

Relatively small dataset size (50 base questions)
Mainly tested ChatGPT-4o, other models have limited results
Only the accuracy of multiple-choice questions is evaluated, other quality indicators are not considered

Future research:

Expanded to more LLM models
Explore performance differences across different task types
Study the underlying mechanism of the influence of tone
Develop reminder strategies that are both effective and friendly

Ethical considerations

Although the study found that rude prompt words were more effective, the researchers emphasized that they do not recommend using hostile language in practical applications. Negative interactive experiences can impact user experience, accessibility, and inclusivity, and promote harmful communication norms.

Conclusion

This study reveals the sensitivity of LLM to surface features of prompt words, challenging our traditional understanding of politeness in human-computer interaction. In the future, a balance needs to be found between improving model performance and maintaining a healthy interactive environment.

📄 Paper information

Paper Title: Mind Your Tone: Investigating How Prompt Politeness Affects LLM Accuracy
Author: Om Dobariya, Akhil Kumar
Institution: Pennsylvania State University
arXiv link:
Year of Publication: 2025