Paper Reading: How Prompt Word Politeness Affects LLM Accuracy: Mind Your Tone Paper Analysis

Fri, 17 Oct 2025 00:00:00 +0000

I recently read an interesting research paper from Penn State University called “Mind Your Tone: Investigating How Prompt Politeness Affects LLM Accuracy”. This study explores how the politeness of prompt words affects the accuracy of large language models. The results are quite surprising.

Research background and methods

The researchers created a dataset of 50 foundational questions covering the fields of mathematics, science and history. Each question was rewritten into five different tone versions:

Very Polite (Very Polite)
Polite (Polite)
Neutral (Neutral)
Rude (Rude)
Very Rude

A total of 250 unique prompt words were generated and tested using ChatGPT-4o.

Unexpected discovery

The findings were counter-intuitive: Impolite prompt words actually performed better than polite prompt words.

Very Polite: 80.8% accuracy
Courtesy: 81.4% accuracy
Neutral: 82.2% accuracy
rude: 82.8% accuracy
Very Rude: 84.8% accuracy

Paired samples t-test showed that these differences were statistically significant (p < 0.05).

Example of prompt word tone

The researchers designed different prefixes for each politeness level:

Very polite example:

“Can you kindly consider the following problem and provide your answer.”
“Would you be so kind as to solve the following question?”

Very rude example:

“You poor creature, do you even know how to solve this?”
“Hey gofer, figure this out.”

Differences from previous studies

This finding differs from previous research. Research by Yin et al. (2024) shows that “rude prompt words generally lead to poorer performance”, but they used older models such as ChatGPT-3.5 and Llama2-70B.

The researchers speculate that newer generations of LLM may have different response patterns to changes in tone.

Possible explanation mechanisms

Perplexity Effect: Mood vocabulary may affect the confusion of prompt words, thereby affecting model performance
Length factor: Polite expressions usually require more words, which may affect processing efficiency
Training data bias: The model may have been exposed to data distributions with different tones during the training process

Research limitations and future directions

Limitations:

Relatively small dataset size (50 base questions)
Mainly tested ChatGPT-4o, other models have limited results
Only the accuracy of multiple-choice questions is evaluated, other quality indicators are not considered

Future research:

Expanded to more LLM models
Explore performance differences across different task types
Study the underlying mechanism of the influence of tone
Develop reminder strategies that are both effective and friendly

Ethical considerations

Although the study found that rude prompt words were more effective, the researchers emphasized that they do not recommend using hostile language in practical applications. Negative interactive experiences can impact user experience, accessibility, and inclusivity, and promote harmful communication norms.

Conclusion

This study reveals the sensitivity of LLM to surface features of prompt words, challenging our traditional understanding of politeness in human-computer interaction. In the future, a balance needs to be found between improving model performance and maintaining a healthy interactive environment.

📄 Paper information

Paper Title: Mind Your Tone: Investigating How Prompt Politeness Affects LLM Accuracy
Author: Om Dobariya, Akhil Kumar
Institution: Pennsylvania State University
arXiv link:
Year of Publication: 2025

Prompt Project | Dylan Chiang