Non-functional requirements for AI systems

Availability and scalability still matter, but an AI system needs eight more targets that a normal service doesn't: quality, latency, cost, safety, freshness, reliability, explainability and determinism. They pull against each other, so you name a number for each and agree with stakeholders which one wins.

Property of the model	Consequence	Requirement it creates
Outputs are probabilistic	Tests can't prove correctness	Quality targets, determinism tolerance
Tokens are generated one at a time	Slow, variable response time	Latency targets for first and last token
Every token is billed	Cost grows with usage and context size	Cost per request or per user
Follows instructions found anywhere in its input	Can be manipulated or leak data	Safety and compliance
Knowledge is frozen at training time	Stale answers	Freshness of retrieved data
Usually an external API	Outages and rate limits you don't control	Reliability with fallbacks
Reasoning is opaque	Hard to trust or audit	Explainability: citations, logs

Property of the model

Consequence

Requirement it creates

Outputs are probabilistic

Tests can't prove correctness

Quality targets, determinism tolerance

Tokens are generated one at a time

Slow, variable response time

Latency targets for first and last token

Every token is billed

Cost grows with usage and context size

Cost per request or per user

Follows instructions found anywhere in its input

Can be manipulated or leak data

Safety and compliance

Knowledge is frozen at training time

Stale answers

Freshness of retrieved data

Usually an external API

Outages and rate limits you don't control

Reliability with fallbacks

Reasoning is opaque

Hard to trust or audit

Explainability: citations, logs

Running example: a support assistant that answers customers from a company knowledge base. All targets are illustrative; the real ones come from your stakeholders.

Quality

≥ 90% correct · < 2% unsupported claims

Why it's AI-specific: outputs are probabilistic, so the only proof is an evaluation.

Measure on a golden set of real questions with known good answers.
Track correctness, faithfulness (is every claim backed by a source) and task success separately.
Gate every prompt or model change on the eval score, like a regression test.

"Quality is a number on a golden set, not a feeling. I'd target 90% correct and under 2% unsupported claims."

saying "accurate answers" without saying who judges and on what data.

Latency

chat: TTFT < 1 s, full < 6 s · autocomplete < 300 ms

Why it's AI-specific: tokens are generated one by one, so total time grows with answer length.

Split it: time to first token (feels responsive) and total time (feels finished).
Streaming fixes the first, not the second. Set both, at p95.
The target depends on the product: chat tolerates seconds, autocomplete doesn't.

"low latency" isn't a requirement. "p95 first token under 1 second" is.

Cost

< ₹2 per conversation · < $0.01 per document

Why it's AI-specific: cost is per token, so it scales with traffic, context size and agent steps.

Set it per unit of value: per conversation, per document, per active user.
Levers: smaller models for easy requests, prompt caching, tighter retrieval, token budgets.

"Input tokens usually dominate, so retrieval size is my main cost lever."

Safety and compliance

no PII leaks · harmful content blocked · data residency

Why it's AI-specific: the model follows instructions hidden in user input or documents, so it can be manipulated.

Make it testable: zero leaks on a red-team set, block rate on a harmful-prompt set.
Cover where data goes: which provider, which region, is it used for training.

"zero PII leakage" alone can't be verified. Pair it with a red-team test that proves it.

Freshness

new docs searchable within 1 hour

Why it's AI-specific: the model's knowledge stops at its training date. Fresh facts must come from retrieval.

The target sets the pipeline: an hour means streaming or frequent re-indexing; a day means a nightly batch.
Deletions count too: a removed policy must stop being cited.

Reliability

graceful fallback if a provider fails

Why it's AI-specific: you depend on an external model API with outages, rate limits and silent version changes.

Fallback chain: second provider or smaller model, then plain search results, then a human.
Timeouts and retries with a limit, and pinned model versions so behaviour doesn't change under you.

Explainability

citations · decision logs

Why it's AI-specific: you can't read the model's reasoning, so trust and audits need evidence.

Users see citations to the source documents.
Auditors see logs: input, retrieved context, model version, output, and any action taken.

Determinism

same input, consistent decision (workflows)

Why it's AI-specific: sampling means the same prompt can give different outputs.

Chat can vary in wording. A workflow that approves or routes must give the same decision.
Tools: low temperature, structured outputs, pinned model versions, and rules in code for the final decision.

assuming temperature 0 makes a model fully deterministic. It reduces variance; it doesn't remove it.

Quality, latency, cost: pick your corner

QUALITY ▲ bigger model, │ more context, more agent │ more steps steps │ │ LATENCY ◄───────┴───────► COST slower answers bigger bill

A bigger model, more context or more agent steps raises quality but adds latency and cost. Every design decision should say which corner it favours. Routing easy requests to a small model and hard ones to a big model is how you get closer to all three.

All eight on one table

Requirement	Example target	Why it's AI-specific
Quality	≥ 90% correct on golden set; < 2% unsupported claims	Outputs are probabilistic; need evals
Latency	TTFT < 1 s, full answer < 6 s (chat); < 300 ms (autocomplete)	Token generation is slow and variable
Cost	< ₹2 per conversation; < $0.01 per document	Token costs scale with usage and context
Safety	Zero PII leakage on red-team set; harmful content blocked	Models can be manipulated
Freshness	New docs searchable within 1 hour	Model knowledge is frozen
Reliability	Graceful fallback if a provider fails	External model dependencies
Explainability	Citations; decision logs	Trust, regulation
Determinism	Same input, consistent decision (workflows)	Sampling variance

The 30-second answer

"Beyond availability and scalability, an AI system needs targets that come from the model being probabilistic, slow and billed per token. I set numbers for quality, measured on a golden set; latency, both time to first token and total; cost per request or per user; safety and compliance, like PII leaks and data residency; freshness of knowledge; reliability with a fallback when a provider fails; explainability through citations and logs; and determinism where a workflow needs consistent decisions. Quality, latency and cost pull against each other, so I rank them with stakeholders and say which corner each design choice favours."