Freeze the case set
Record dataset origin, split, taxonomy, model and configuration before comparison.
Evidence ledger / reviewed summary
What we have measured, and what we have not
Gyde has run task-specific model evaluations and controlled GPU operations tests. These results describe the tested data, configuration and geography, not a production guarantee. A customer pilot must set its own quality, availability and cost thresholds and measure them on an approved workload.
Evidence reviewed 23 October 2026. Results below are tied to their test populations and operating conditions.
GYDE / INFERENCEILLUSTRATIVE PILOT FLOW
DESIGN PATH / BUYER-SPECIFIC
01WorkloadTask + test cases
02ModelQualified candidate
03PolicyAllowed route
04ServingApproved capacity
05EvidenceOutcome + cost
Decision: proceed, redesign or stop
01US complaints
Issue classification on real CFPB complaints.
Measured, October 2026. The same held-out public complaint set was used for each model. These results do not establish performance on an institution's own cases.
| Test or model | Result | Limit |
|---|---|---|
| Test population | Measured, October 2026: 1,999 real complaints and 82 issue labels. | CFPB product and issue taxonomy. |
| Gyde fine-tuned 8B model | Measured, October 2026: 55.0% issue accuracy; 83.5% product accuracy. | The issue result trails Sonnet with retrieved examples. |
| Claude Sonnet 5.5 with retrieved examples | Measured, October 2026: 56.6% issue accuracy with 5 retrieved examples. | Slightly ahead on issue accuracy, at much higher estimated cost. |
| Claude Sonnet 5.5, prompt only | Measured, October 2026: 44.1% issue accuracy. | No retrieved examples. |
| GPT-5.4, prompt only | Measured, October 2026: 38.7% issue accuracy with medium reasoning; 33.9% by default. | No retrieved examples. |
02India complaints
The RBI-category preview has no real-complaint test yet.
Measured, October 2026. Frontier models lead on this synthetic set. The labels were agreed by AI labellers and were not human-reviewed.
| Test or model | Result | Limit |
|---|---|---|
| Test population | Measured, October 2026: 463 synthetic complaints with AI-agreed labels. | No real-complaint evaluation yet. |
| Gyde fine-tuned 8B model | Measured, October 2026: 96.3% category accuracy. | Behind the frontier models on this set. |
| Claude Sonnet 5.5 | Measured, October 2026: 99.4% category accuracy. | Synthetic, AI-labelled evaluation. |
| GPT-5.4 | Measured, October 2026: 99.1% category accuracy. | Synthetic, AI-labelled evaluation. |
03Statement categorisation
Line accuracy is exploratory and below Sonnet.
Measured, 22 October 2026. The small volunteer set gives about a ±5-point margin; statement-level income, obligation and borrower outcomes have not been measured.
| Test or model | Result | Limit |
|---|---|---|
| Checked sample | Measured, 22 October 2026: 257 checked lines from 4 volunteers' statements. | The human check started from Claude's draft labels, which may favour Claude. |
| Gyde fine-tuned model | Measured, 22 October 2026: about 80% correct per line. | Exploratory result with about ±5 points of uncertainty. |
| Claude Sonnet | Measured, 22 October 2026: about 93% correct per line. | Exploratory result with about ±5 points of uncertainty. |
04Operations
Serving and failover tests show missed requests and missed targets.
Measured, October 2026. These controlled tests do not establish production availability.
| Test | Result | Limit |
|---|---|---|
| Steady-traffic rehearsal | Measured, 21 October 2026: 92.1% of requests served in an 8-hour rehearsal. | The rehearsal included planned failures and a substantial interruption. |
| Planned host failures | Measured, 21 October 2026: first recovery took 34 minutes; after a fix, the next took 2 minutes 44 seconds. | The improvement from one follow-up test is not a general recovery time. |
| Fresh-host failover | Measured, 21 October 2026: 2 of 5 runs missed the 240-second recovery target. | Controlled fault tests in one geography, not an availability trial. |
| Purchase to serving | Measured, 19 October 2026: JarvisLabs 138 s; CloudPe 270 s; Google Cloud 284 s. | These runs do not establish future capacity or recovery time. |
05Method
Record cases, requests and failure windows.
Count from the client
Include scheduled requests, failures, planned events and injected failures in the denominator.
Pair every result with its boundary
Show date, hardware, serving conditions, sample size, measured or estimated status and limitations together.
06Economics still under study
Delivered cost still needs a pilot.
Full provider billing, failed starts, standby storage and equal-availability comparisons remain unreconciled. The useful unit is cost per successful, quality-acceptable request.
07Customer pilot
Your workload needs its own evidence.
01
Quality
Representative, permissioned cases and severe-error counts against a frozen baseline.
02
Operations
Client-side served-request rate, interruption windows, p95 latency, egress and recovery evidence.
03
Cost
Estimated and billed charges kept separate, then reconciled to accepted results.
?Buyer questions
Questions to settle before a pilot.
Answers reflect the current prototype and scoped engagement approach.
Has Gyde proved production availability?
No. The controlled tests include failed requests and an extended rehearsal with an outage. They are useful for pilot design, but do not establish a production availability percentile.
Does a spot discount prove lower cost?
No. Cost per successful, quality-acceptable request must include failed starts, standby capacity, idle time and actual billed charges at comparable availability.
Do the model results transfer to our data?
No. A representative customer set must be evaluated within the agreed data boundary using the institution's taxonomy, baseline and acceptance rules.
Gyde Inference pilot
Start with a workload you can measure.
Describe the task, data boundary and decision you need to make. We will scope the test and operating conditions together.
Discuss a workload Do not submit sensitive records through the marketing form.