Sarvam Prism trades open-ended answers for fast, scored decisions
The 105-billion-parameter model ranks predefined options in one pass. Its latency, confidence and benchmark claims need their stated test conditions.
Sarvam Prism is built to choose among supplied answers, rather than compose a conversation. Released on November 3, the decision model takes context, questions and candidate options, then returns probabilities in a forward pass. That makes its natural territory routing, classification and evaluation—places where a system needs a bounded decision rather than another paragraph.
Consider a support message describing a duplicate charge. A workflow might ask which team should receive it and whether it needs urgent attention. Prism can score those predefined choices together. It cannot discover a missing category simply by writing a new answer, because its output is constrained to the supplied options. Designing the choices remains part of designing the system.
The 44-millisecond result is a test condition
Sarvam reports a 44 ms median response time on one NVIDIA B200. The model card specifies 1,000 requests from a public decision benchmark, sent one at a time after loading, with a 95th-percentile latency of 420 ms. The median is not a promise that every long document or concurrent production request completes in 44 ms.
The release’s accuracy comparison averages three indices: a public decision benchmark, Sarvam’s recreation of an Artificial Analysis index with a substituted knowledge test, and an internal Indian-language benchmark. These are vendor-run evaluations. The recreated index should not be represented as an independent score issued by Artificial Analysis.
Prism’s reported average is 71.8, with some slower models scoring higher. The useful claim is a position in this particular speed–accuracy comparison, not universal superiority over conversational models or every decision workload.
Confidence is useful only when its meaning survives deployment
Sarvam emphasizes calibration: whether predictions made with high confidence are wrong as often as their probabilities imply. In its reported English evaluation, answers at 90% confidence or above had a 2.2% observed error rate against 3.3% expected errors. Those results support examining confidence-based routing, but do not establish the same relationship on a new company’s data.
A threshold can send uncertain cases to a person, yet a badly chosen answer set can still force a confident choice among unsuitable options. Including an abstention category where appropriate and checking errors against real outcomes are therefore consequential design decisions, not cosmetic additions to the prompt.
Open weights do not make it a small local model
Prism is based on Sarvam-105B and released under Apache 2.0. Its documented local requirements include an FP8-capable GPU with about 120 GB of free memory and roughly 106 GB of disk space for weights. That puts the published configuration in server-class hardware territory, despite the low measured response time.
The weights use a custom FP8 layout and the supplied inference code; the model card says they cannot simply be loaded through a general-purpose causal-language-model interface. A hosted API is another route, with its own deployment and data-handling considerations.
Prism also normalizes candidate order before scoring, which removed option-order changes in Sarvam’s reported test. That is a useful engineering detail precisely because it is less mysterious than “better judgment.” The release’s strongest idea is to make a narrow decision explicit and inspectable. Whether the decision is good still depends on the question, the available answers and the evidence used to check it.
