Freeze the case set
Record dataset origin, split, taxonomy, model and configuration before comparison.
Evidence / results and limits
On real US complaints, Gyde's model scored above the prompt-only baselines and below Sonnet with retrieved examples on issue accuracy. Frontier models led the synthetic India preview and the small statement test. On synthetic phone-photo statements, running Gyde's model first and sending only failed rows to a larger model was more accurate than the larger model alone, at less than half the cost. Operations tests included failed requests and missed recovery targets.
These results reflect the stated test data and operating conditions. A pilot must test your workload separately.
ONE WORKLOAD / PILOT PATH
01US complaints
Measured, October 2026. The same held-out public complaint set was used for each model. These results do not establish performance on an institution's own cases.
| Test or model | Result | Limit |
|---|---|---|
| Test population | Measured, October 2026: 1,999 real complaints and 82 issue labels. | CFPB product and issue taxonomy. |
| Gyde fine-tuned 8B model | Measured, October 2026: 55.0% issue accuracy; 83.5% product accuracy. | The issue result trails Sonnet with retrieved examples. |
| Claude Sonnet 5.5 with retrieved examples | Measured, October 2026: 56.6% issue accuracy with 5 retrieved examples. | Slightly ahead on issue accuracy. A comparable cost study is still needed. |
| Claude Sonnet 5.5, prompt only | Measured, October 2026: 44.1% issue accuracy. | No retrieved examples. |
| GPT-5.4, prompt only | Measured, October 2026: 38.7% issue accuracy with medium reasoning; 33.9% by default. | No retrieved examples. |
02India complaints
Measured, October 2026. Frontier models lead on this synthetic set. The labels were agreed by AI labellers and were not human-reviewed.
| Test or model | Result | Limit |
|---|---|---|
| Test population | Measured, October 2026: 463 synthetic complaints with AI-agreed labels. | No real-complaint evaluation yet. |
| Gyde fine-tuned 8B model | Measured, October 2026: 96.3% category accuracy. | Behind the frontier models on this set. |
| Claude Sonnet 5.5 | Measured, October 2026: 99.4% category accuracy. | Synthetic, AI-labelled evaluation. |
| GPT-5.4 | Measured, October 2026: 99.1% category accuracy. | Synthetic, AI-labelled evaluation. |
03Statement categorisation
Measured, 22 October 2026. The small volunteer set gives about a ±5-point margin; statement-level income, obligation and borrower outcomes have not been measured.
| Test or model | Result | Limit |
|---|---|---|
| Checked sample | Measured, 22 October 2026: 257 checked lines from 4 volunteers' statements. | The human check started from Claude's draft labels, which may favour Claude. |
| Gyde fine-tuned model | Measured, 22 October 2026: about 80% correct per line. | Exploratory result with about ±5 points of uncertainty. |
| Claude Sonnet | Measured, 22 October 2026: about 93% correct per line. | Exploratory result with about ±5 points of uncertainty. |
04Phone-photo statements
Measured, October to November 2026, on 96 synthetic phone-photo bank statements from 24 layouts not seen in training. Row-level fixes check every row's balance and date, compare two readings and send only the failed rows to a larger model. No real lender photos have been tested. Token use was measured; GPU cost is estimated.
| Route | Result | Limit |
|---|---|---|
| Largest model alone | Measured, October to November 2026: 77.1% of statements fully right; about ₹5,803 per 1,000 pages. | No checks, so all 22 wrong statements were accepted. |
| Largest model with the same checks | Measured, October to November 2026: 88.5% fully right; about ₹6,039 per 1,000 pages. | No small model in the route. |
| Gyde model first, safe mode | Measured, October to November 2026: 78.1% fully right; 0 of 96 wrong results accepted; about ₹5,059 per 1,000 pages. | Statements that fail a check go to the larger model or a person. |
| Gyde model first, row-level fixes | Measured, October to November 2026: 90.6% fully right; 3 of 96 wrong results accepted; about ₹2,686 per 1,000 pages. | The 3 accepted errors are statements the larger model alone also got wrong. |
05Operations
Measured, October 2026. These controlled tests do not establish production availability.
| Test | Result | Limit |
|---|---|---|
| Steady-traffic rehearsal | Measured, 21 October 2026: 92.1% of requests served in an 8-hour rehearsal. | The rehearsal included planned failures and a substantial interruption. |
| Planned host failures | Measured, 21 October 2026: first recovery took 34 minutes; after a fix, the next took 2 minutes 44 seconds. | The improvement from one follow-up test is not a general recovery time. |
| Fresh-host failover | Measured, 21 October 2026: 2 of 5 runs missed the 240-second recovery target. | Controlled fault tests in one geography, not an availability trial. |
| Two-cloud serving run | Measured, reported November 2026: 5,562 requests over 3 hours across two Indian GPU clouds, with no failed requests. | A short run; it does not establish production availability. |
| Response time, India-hosted | Measured, reported November 2026: 267 ms median time to first token; about ₹79 per 1,000 requests. | Open-weight GLM models on dual Blackwell GPUs in India, not the fine-tuned 8B models. |
| Purchase to serving | Measured, 19 October 2026: JarvisLabs 138 s; CloudPe 270 s; Google Cloud 284 s. | These runs do not establish future capacity or recovery time. |
06Method
Record dataset origin, split, taxonomy, model and configuration before comparison.
Include scheduled requests, failures, planned events and injected failures in the denominator.
Show date, hardware, serving conditions, sample size, measured or estimated status and limitations together.
07Economics still under study
Full provider billing, failed starts, standby storage and equal-availability comparisons remain unreconciled. The useful unit is cost per successful, quality-acceptable request.
Estimate the cost of your workload08Questions for your pilot
Test representative, permissioned cases. Count severe errors against your current baseline.
Measure served requests, interruptions and response time from the client. Check where traffic leaves the network and how recovery works.
Keep estimated and billed charges separate, then compare them with the number of quality-acceptable results.
?Buyer questions
No. The controlled tests include failed requests and an extended rehearsal with an outage. They are useful for pilot design, but do not establish a production availability percentile.
No. Cost per successful, quality-acceptable request must include failed starts, standby capacity, idle time and actual billed charges at comparable availability.
No. A representative customer set must be evaluated within the agreed data boundary using the institution's taxonomy, baseline and acceptance rules.
Gyde Inference pilot
Describe the task, data boundary and decision you need to make. We will scope the test and operating conditions together.
Discuss a workload Do not submit sensitive records through the marketing form.