Evidence / results and limits

The test results, including the misses

On real US complaints, Gyde's model scored above the prompt-only baselines and below Sonnet with retrieved examples on issue accuracy. Frontier models led the synthetic India preview and the small statement test. On synthetic phone-photo statements, running Gyde's model first and sending only failed rows to a larger model was more accurate than the larger model alone, at less than half the cost. Operations tests included failed requests and missed recovery targets.

These results reflect the stated test data and operating conditions. A pilot must test your workload separately.

01US complaints

Issue classification on real CFPB complaints.

Measured, October 2026. The same held-out public complaint set was used for each model. These results do not establish performance on an institution's own cases.

Issue classification on real CFPB complaints.
Test or modelResultLimit
Test populationMeasured, October 2026: 1,999 real complaints and 82 issue labels.CFPB product and issue taxonomy.
Gyde fine-tuned 8B modelMeasured, October 2026: 55.0% issue accuracy; 83.5% product accuracy.The issue result trails Sonnet with retrieved examples.
Claude Sonnet 5.5 with retrieved examplesMeasured, October 2026: 56.6% issue accuracy with 5 retrieved examples.Slightly ahead on issue accuracy. A comparable cost study is still needed.
Claude Sonnet 5.5, prompt onlyMeasured, October 2026: 44.1% issue accuracy.No retrieved examples.
GPT-5.4, prompt onlyMeasured, October 2026: 38.7% issue accuracy with medium reasoning; 33.9% by default.No retrieved examples.

Read the US model card

02India complaints

The RBI-category preview has no real-complaint test yet.

Measured, October 2026. Frontier models lead on this synthetic set. The labels were agreed by AI labellers and were not human-reviewed.

The RBI-category preview has no real-complaint test yet.
Test or modelResultLimit
Test populationMeasured, October 2026: 463 synthetic complaints with AI-agreed labels.No real-complaint evaluation yet.
Gyde fine-tuned 8B modelMeasured, October 2026: 96.3% category accuracy.Behind the frontier models on this set.
Claude Sonnet 5.5Measured, October 2026: 99.4% category accuracy.Synthetic, AI-labelled evaluation.
GPT-5.4Measured, October 2026: 99.1% category accuracy.Synthetic, AI-labelled evaluation.

Read the India preview model card

03Statement categorisation

Line accuracy is exploratory and below Sonnet.

Measured, 22 October 2026. The small volunteer set gives about a ±5-point margin; statement-level income, obligation and borrower outcomes have not been measured.

Line accuracy is exploratory and below Sonnet.
Test or modelResultLimit
Checked sampleMeasured, 22 October 2026: 257 checked lines from 4 volunteers' statements.The human check started from Claude's draft labels, which may favour Claude.
Gyde fine-tuned modelMeasured, 22 October 2026: about 80% correct per line.Exploratory result with about ±5 points of uncertainty.
Claude SonnetMeasured, 22 October 2026: about 93% correct per line.Exploratory result with about ±5 points of uncertainty.

04Phone-photo statements

Running the small model first cut cost on synthetic phone photos.

Measured, October to November 2026, on 96 synthetic phone-photo bank statements from 24 layouts not seen in training. Row-level fixes check every row's balance and date, compare two readings and send only the failed rows to a larger model. No real lender photos have been tested. Token use was measured; GPU cost is estimated.

Running the small model first cut cost on synthetic phone photos.
RouteResultLimit
Largest model aloneMeasured, October to November 2026: 77.1% of statements fully right; about ₹5,803 per 1,000 pages.No checks, so all 22 wrong statements were accepted.
Largest model with the same checksMeasured, October to November 2026: 88.5% fully right; about ₹6,039 per 1,000 pages.No small model in the route.
Gyde model first, safe modeMeasured, October to November 2026: 78.1% fully right; 0 of 96 wrong results accepted; about ₹5,059 per 1,000 pages.Statements that fail a check go to the larger model or a person.
Gyde model first, row-level fixesMeasured, October to November 2026: 90.6% fully right; 3 of 96 wrong results accepted; about ₹2,686 per 1,000 pages.The 3 accepted errors are statements the larger model alone also got wrong.

05Operations

Serving and failover tests show missed requests and missed targets.

Measured, October 2026. These controlled tests do not establish production availability.

Serving and failover tests show missed requests and missed targets.
TestResultLimit
Steady-traffic rehearsalMeasured, 21 October 2026: 92.1% of requests served in an 8-hour rehearsal.The rehearsal included planned failures and a substantial interruption.
Planned host failuresMeasured, 21 October 2026: first recovery took 34 minutes; after a fix, the next took 2 minutes 44 seconds.The improvement from one follow-up test is not a general recovery time.
Fresh-host failoverMeasured, 21 October 2026: 2 of 5 runs missed the 240-second recovery target.Controlled fault tests in one geography, not an availability trial.
Two-cloud serving runMeasured, reported November 2026: 5,562 requests over 3 hours across two Indian GPU clouds, with no failed requests.A short run; it does not establish production availability.
Response time, India-hostedMeasured, reported November 2026: 267 ms median time to first token; about ₹79 per 1,000 requests.Open-weight GLM models on dual Blackwell GPUs in India, not the fine-tuned 8B models.
Purchase to servingMeasured, 19 October 2026: JarvisLabs 138 s; CloudPe 270 s; Google Cloud 284 s.These runs do not establish future capacity or recovery time.

06Method

Record cases, requests and failure windows.

01

Freeze the case set

Record dataset origin, split, taxonomy, model and configuration before comparison.

02

Count from the client

Include scheduled requests, failures, planned events and injected failures in the denominator.

03

Pair every result with its boundary

Show date, hardware, serving conditions, sample size, measured or estimated status and limitations together.

07Economics still under study

Delivered cost still needs a pilot.

Full provider billing, failed starts, standby storage and equal-availability comparisons remain unreconciled. The useful unit is cost per successful, quality-acceptable request.

Estimate the cost of your workload

08Questions for your pilot

What should your team test next?

01

Does it handle your cases?

Test representative, permissioned cases. Count severe errors against your current baseline.

02

Can it serve under your rules?

Measure served requests, interruptions and response time from the client. Check where traffic leaves the network and how recovery works.

03

What does an accepted result cost?

Keep estimated and billed charges separate, then compare them with the number of quality-acceptable results.

?Buyer questions

Questions to settle before a pilot.

Has Gyde proved production availability?

No. The controlled tests include failed requests and an extended rehearsal with an outage. They are useful for pilot design, but do not establish a production availability percentile.

Does a spot discount prove lower cost?

No. Cost per successful, quality-acceptable request must include failed starts, standby capacity, idle time and actual billed charges at comparable availability.

Do the model results transfer to our data?

No. A representative customer set must be evaluated within the agreed data boundary using the institution's taxonomy, baseline and acceptance rules.

Gyde Inference pilot

Start with a workload you can measure.

Describe the task, data boundary and decision you need to make. We will scope the test and operating conditions together.

Discuss a workload Do not submit sensitive records through the marketing form.