LLM as Risk Manager, Not Alpha Generator

LLM as Risk Manager, Not Alpha Generator

LLM as Risk Manager, Not Alpha Generator

Filter name

Measuring

Yongjae Lee

Chanyeol Choi

ACM ICAIF 2025

Jihoon Kwon

Minjae Kim

Sumin Kim

Chanyeol Choi

Yongjae Lee

Alejandro Lopez-Lira

Oscar Levy

Joo Won Lee

Nicole Kagan

Yoon Kim

Paper

paper

Re-ranking Granger-derived lead–lag pairs in prediction markets by asking an LLM whether a real economic mechanism exists cut average loss by 46.5% and tripled total PnL — the LLM turned out to be a risk manager, not an alpha generator.

The problem

Granger causality tells you that A predicts B, but never why. In large-scale pair screening, plenty of links clear the significance bar by pure chance — and those are exactly the ones that break first when the regime shifts. Trade them and you eat the losses.

The idea

On Kalshi, every contract is both a plainly worded event and a probability time series. Numbers and text are attached to the same object, so you can just ask an LLM the question directly: is there an actual economic path from this event to that one?

The method (two stages)

  • Stage 1: log-odds transform → ADF test and differencing → bidirectional Granger tests, keeping the top 100 directed pairs

  • Stage 2: given only event titles and descriptions, an LLM (GPT-5-nano) judges whether a mechanism exists, how strong it is, and which sign it carries — then re-ranks down to a final 20

  • The LLM discovers nothing new. It only picks from what the statistics already cleared.

The evaluation

Causal ground truth is unobservable, so the paper grades rankings by trading them. A leader move triggers entry into the follower the next day, held seven days, exited mechanically. Same rule for both rankings, so any performance gap traces back to ranking quality. (554 markets, 18 rolling windows.)

The results

Total PnL went from $4,100 to $12,500 — roughly triple. But look closer: win rate improved only from 51.4% to 54.5%, and average win actually fell, from $724 to $636. One number did the work — average loss, cut from $649 to $347.

The gains didn't come from being right more often. They came from discarding the relationships that were going to break. Hence "Risk Manager."

Supporting evidence

  • Not a lexical-overlap artifact — losses dropped 48.1% on genuinely distinct event pairs, more than the 42.9% on same-event pairs

  • Most valuable during large moves — win rate up 17.6pp when the leader repriced by more than 10 points

  • Holds across every holding period from 1 to 21 days (36.2% average loss reduction)

  • Survives a post-cutoff rerun using only trades after the model's training cutoff (40.3% loss reduction)

One case worth telling

The "Japan recession → U.S. GDP" pair sat at Granger rank #71, well outside the cutoff. The LLM pulled it up to #5, and it returned +$700. Its reasoning was cross-border trade and financial spillover. Weak on both surface similarity and statistics — but the transmission channel was there to be read.

Open questions

It's one domain: Kalshi Economics. Transaction costs and slippage aren't in the protocol. And the ability to construct a plausible narrative still isn't cleanly separated from the ability to identify a real mechanism.