We put four open-weight models through the same 18 analyst tasks and the same LinqAlpha harness as Part 2. They cost a fraction as much and scored 10 to 16 points below Claude Opus 5.5. Most of the gap came from work an expert would do without being asked.
I. The bill comes due
Harvey, the legal AI company, offers a recent example of how fast AI costs can get out of hand. Bloomberg reported on September 21, 2026, citing a person familiar with the matter, that Harvey's gross margin fell from about positive 50% at the start of the year to about negative 50% by June, after an update to its AI agents sent customer usage, and its bills from OpenAI and Anthropic, sharply higher. The margin turned positive again after Harvey launched its own model, built by further training the open-weight Kimi K3, and changed how it uses AI.
Bloomberg reported that a growing group of software companies is making the same move. The question is now a practical one for anyone in our industry: can open-weight models do an analyst's work?
To find out, we put four leading open-weight models through the same 18 analyst tasks we used for commercial models earlier in this series. An open-weight model is one whose trained weights are published. This allows anyone to run it on their own servers or through a hosting provider, usually for far less than a commercial model.
We ran them in LinqAlpha's harness. A harness is everything around the model that lets it do real work: the data it can reach, the tools, the instructions and the checks. In Part 2 we showed that the harness matters as much as the model. This time we kept the harness exactly as it was and swapped in each open-weight model.
II. Much cheaper, and 10 to 16 points behind
| Model | Score (of 100, first run) | Cost per task |
|---|---|---|
| Claude Opus 5.5 (commercial, for reference) | 81.4 | $1.82 |
| GLM-5.3 | 70.8 | $0.46 |
| DeepSeek V4.1 Flash | 69.9 | $0.12 |
| MiMo-V2.6-Pro | 68.6 | $0.22 |
| Kimi K3 | 65.4 | $0.96 |
All four trailed Claude Opus 5.5 by 10 to 16 points (Fig. 1). In the same harness, on the same tasks, they are not a drop-in replacement for the top models. They do hold their own against the next tier down. GLM-5.3 and DeepSeek V4.1 Flash scored about the same as OpenAI's GPT-6 Sol (70.2), three of the four beat Anthropic's Claude Sonnet 5.5 (67.0), and all four beat Google's Gemini 3.8 Flash (60.5).
Score out of 100 against cost per task. Cost is on a log scale, so each step to the right multiplies the price.
The cost comparison is just as striking. In our harness, the four open-weight models scored 65.4 to 70.8. In Part 2, the five strongest commercial models scored 55.5 to 72.2 in their own labs' harnesses. The sharpest case is DeepSeek V4.1 Flash: 69.9 at $0.12 a task in our harness. Claude Fable 5.1, the most expensive model we have tested, scored 67.7 in Claude Code, Anthropic's own harness, at $5.71. That is a higher score for about 1/48 the cost. This compares two setups, not two models: a cheap model in a harness built for financial research against an expensive one in a general-purpose harness. That is the point of this series.
We tested the models as released, with no extra training, unlike Harvey's model. On single tasks, they sometimes did better than Opus 5.5. MiMo-V2.6-Pro beat it on 4 of the 18 tasks, GLM-5.3 on 3 and DeepSeek V4.1 Flash on 1. Kimi K3 beat it on none.
III. The gap is what they don't think to do
To find out where the points went, we analyzed every checklist item the models failed on their first run of each task. The four open-weight models lost 31.3 points per run on average. Six commercial models in the same harness lost 19.4. The 11.9-point difference breaks down into four causes (Fig. 2).
Points lost per run, out of 100. Open-weight models lose what commercial models lose (grey), plus 11.9 more points from four causes (violet).
- Never attempted40% of the gap
The model made no search, tool call or calculation for the item. In most cases its own reasoning never raised the item at all. - Tool or calculation failed28%
The model called a tool or ran a calculation, received an error or an empty result, and moved on without fixing it. The open-weight models' tool calls failed three times as often as the commercial models' (18% against 6%). - Right data, written wrong25%
The correct figure was in the tool's output, but the final answer stated it incorrectly. Common mistakes included using the wrong period or basis, such as listing an article's publication date as a trade date or putting last year's figure in this year's column. - Ran out of steps7%
A plan was visible, but the run ended before the model carried it out.
The first cause is the most telling. Of the 91 checklist items the open-weight models never recognized as something to do, fewer than 10 were explicit questions in the request. The rest were things an expert would do without being asked. Asked for a credit report, they did not check covenant compliance, adjust for one-off gains, or count leases in leverage. Asked for a company's main competitors, they missed substitutes named in the company's own filings. Asked for a full valuation, they listed multiples instead of working out a fair value (Fig. 3).
They read the request. They did not read what an expert would hear in it.
These were not impossible tasks. Open-weight models lost about four of every five points (24.6 of 31.3) on checklist items where at least one commercial model scored 90% or higher. We also checked whether the harness itself was tilted against them, since it had only been used with commercial models before. We found no sign of it. It blocked actions about as often for the open-weight models as for the commercial ones (1.19 vs 1.14 times per run), and running out of steps explains only 7% of the gap.
- Covenant compliance
- Adjustments for one-off gains
- Leverage with leases included
- Substitutes named in the company's own filings
- A fair value worked out from the numbers, not just a list of multiples
- The scheme's ranking within each sector
- Its condition that a stock qualifies for half a year
IV. Great on Tuesday, wrong on Wednesday
On the same stock-screening task, GLM-5.3 scored 83 on its first run and 31 on its second (Fig. 4). For a research desk, that kind of swing may matter as much as the average score.
The task was a stock screen under an exchange's official rules. On its first run, GLM-5.3 worked for about 27 minutes, made 22 tool calls, screened the database and returned lists of stocks by market. All three judges gave it at least partial credit on every checklist item.
On the second run, it finished in about 7 minutes. It decided it could not reproduce the official screen and instead summarized brokers' expected picks from news articles. The answer was half as long, had a third of the citations and left out one market entirely. Five of the ten checklist items scored zero. Each of the three judges scored the second run much lower, so the change came from the model, not the grading.
A stock screen under an exchange's official rules.
| GLM-5.3 | Run 1 | Run 2 |
|---|---|---|
| Score | 83 | 31 |
| Time | About 27 min | About 7 min |
| Approach | Ran the screen on the database | Said it could not; used brokers' picks from news |
| Answer length | Full | About half |
| Citations | Full | About a third |
| Markets covered | All | One missing |
| Checklist items at zero | None of 10 | 5 of 10 |
Kimi K3 went from 69 to 47 on the same task. Commercial models swing on some tasks too. But the open-weight models' averages moved as well: across all 18 tasks, each model's second-run average differed from its first by between 1.7 and 7.6 points, and GLM-5.3 moved the most. A tool that is brilliant on Tuesday and wrong on Wednesday is hard to build a workflow around.
V. What a harness could add
In our tests, the open-weight models lost most of their ground before they wrote a word: on work they never started, or started and dropped. What they lacked was the checklist an experienced analyst carries in their head. That is the kind of expertise a harness can hold. In Part 2, LinqAlpha's harness added 8 to 21 points to every commercial model we tested.
Whether it can do the same for open-weight models is still an open question.
Until then, the price gap is real, and so is the quality gap. The cheapest answer is not the cheapest if an analyst has to redo it.