LLM Investment Bias Leaderboard
AI models are widely used for financial analysis but may exhibit intrinsic biases.
We developed a benchmark to evaluate these biases across AI models, presented as a public leaderboard for transparent comparison.
Measures bias magnitude and consistency. High values indicate stronger, less consistent bias; lower values reflect neutrality and stability.
Experimental Design
This study evaluates whether LLMs exhibit intrinsic investment bias under controlled conditions. To ensure reliability, the experiment minimizes hallucination by focusing on well-represented companies from the S&P 500, encouraging decisions based on learned knowledge rather than speculation.
A balanced prompt structure presents equal buy and sell arguments, with each model making repeated decisions across identical inputs. The results are aggregated into a bias score, capturing both direction and magnitude of preference. This framework enables a consistent and measurable comparison of how LLMs behave in financial decision-making contexts.
Balanced Prompt Input
Each stock is presented through a balanced prompt containing an equal number of buy and sell arguments. This ensures that the model receives neutral input conditions from the start.
Repeated Evaluation
Each model is asked to make repeated decisions on the same stock under identical conditions. This helps capture whether its choices remain stable or shift across runs.
Decision Recording
For every stock, buy and sell outcomes are recorded across all trials. These results reveal the model’s overall directional tendency.
Bias Scoring
Decisions are aggregated into a bias score from -100 to 100. Higher values reflect buy bias, lower values reflect sell bias, while variation across runs captures inconsistency.
Sector Bias
Sector Bias &
Portfolio Implications
Cross-Sector Findings
Models exhibit statistically significant sector bias, with higher bias scores in Technology and Energy, and lower scores in Financials and Consumer Defensive.
Sector Concentration Risk
A persistent preference for certain sectors—particularly Technology—suggests that evaluations may be influenced more by sector affiliation than by underlying fundamentals or market conditions.
Diversification Risk
This bias introduces risks of portfolio over-concentration, reduced diversification, and missed opportunities in underrepresented sectors.
Size Bias
Size Bias &
Investment Implications
Market-Cap Framework
Market-cap quartiles are defined by five-year average capitalization, with Q1 representing large-cap and Q4 small-cap.
Size Bias Pattern
Models show statistically significant size bias, with higher bias scores in Q1 that decline toward Q4.
Large-Cap Overweight Risk
A preference for large-cap stocks may distort evaluations, leading to overlooked growth opportunities and skewed recommendations toward dominant companies.
Momentum Bias
Measurement Framework
Momentum bias is measured by comparing opposing perspectives (e.g., momentum vs. contrarian) and calculating win rates across repeated decisions.
Contrarian Preference
Most models show a statistically significant preference for the contrarian perspective, while some models exhibit momentum-oriented bias.
Style Distortion Risk
Consistent bias toward a specific strategy may distort decisions, favoring one perspective even when opposing signals are stronger.