Glossary
This glossary gives short definitions for reference. The chapters explain how each method works, with examples and limitations.
Ablation: An experiment that removes or replaces one component to compare its contribution under otherwise shared conditions. Interactions with other components can affect the result.
Access token: A credential that authorizes access to protected resources under specified permissions. In the MCP flow in Section 9.8, the token must be intended for the receiving server. That server checks its audience and must not pass the token through to another service.
Access-control list (ACL): A list of users or roles allowed to access a resource. The application checks it before protected information enters a model request.
Accuracy: The share of all cases classified correctly. It hides error types when one class dominates.
Action: A model or agent-policy output that proposes the next step, such as a tool call, clarification question, delegated subtask, or final answer.
Agent: A system in which a model selects actions toward a goal from observations, tools, and state, while application code enforces authorization, execution, budgets, and stopping rules.
Agent harness: Runtime software that coordinates a model’s use of approved tools. It carries out the model/tool exchange. Some products also use the term for session support.
Agentic RAG: A retrieval-augmented system in which an agent chooses and iterates retrieval actions from observations, within a bounded recovery loop.
Answer relevance: Whether a response resolves the requested information rather than only discussing the same topic.
Application Programming Interface (API): A defined interface through which application code sends a request to a provider and reads the response.
Application software: The code that forms requests from user input and company data, invokes the model, checks whether its answer is usable, and authorizes any next action.
Approximate nearest neighbor (ANN) search: A vector search method that aims to reduce search work while allowing its results to differ from exact search. It may miss a neighbor that exact search would find.
ARES: A research framework that assesses context relevance, answer faithfulness, and answer relevance with trained judges and a human-labeled evaluation set. See RAG quality measures in RAG quality measures.
Atomic update: A check and change treated as one indivisible operation, so concurrent requests cannot both act on an unchecked old value.
Attention: The transformer operation that lets each token representation combine information from other positions in the sequence.
Authorization: The application decision that a specific identity may perform a specific action on a specific resource under the current policy.
Authorization server: The system that issues access tokens after checking the required credentials and authorization. In the user-delegated flow in Section 9.8, it verifies the user and obtains permission.
Base model: A pre-trained language model that primarily continues text. Alignment training adapts the model to follow instructions and conversational roles.
Benchmark saturation: A condition in which many models score near a benchmark’s maximum, leaving little separation between their scores. It differs from contamination: models can solve the items without having seen them in training.
BERTScore: A text-comparison metric that matches context-dependent token vectors between a candidate and a reference. Similarity to a reference does not establish factual support.
BLEU: A machine-translation metric that combines modified n-gram precision with a brevity penalty.
Blinding: Hiding information, such as a model’s identity, from graders when it could influence their judgment. The graders still receive the task and evidence needed to assess the answer.
BM25: A lexical ranking function that rewards matching query terms, weights rare terms more, and accounts for term frequency and document length.
Bootstrap confidence interval: An interval estimated by resampling evaluation cases with replacement and recomputing a statistic. The percentile method used in Chapter 4 selects quantiles of those resampled statistics. Its intended repeated-evaluation coverage depends on the sampling assumptions and interval method. It is not the probability that a particular candidate improves the product. See Score uncertainty in Score uncertainty.
Byte-pair encoding (BPE): A subword method that starts with small units and repeatedly merges the pair that occurs most often in the training text.
Cache-augmented generation (CAG): Chan et al.’s method that reuses a prepared document prefix’s attention state to answer independent questions without retrieving passages for each question. The documents must fit usable context, and model weights remain fixed. See When retrieval is useful in When retrieval is useful.
Canary release: A partial and time-limited deployment of a change, evaluated before full rollout.
Capture: Identifying candidate information from a conversation and associating it with its source.
Chain-of-thought prompting: Prompting a model to generate intermediate steps as text before its final answer.
Chat model: An instruction-following model used with a role-aware serialization that distinguishes system, user, assistant, and sometimes tool messages.
Chat template: The model-specific rule that converts system, user, and assistant messages into the control tokens and text the model was trained on.
Checkpoint: A saved model version containing a set of parameters and its associated configuration. Loading another checkpoint changes the model used for inference. A workflow checkpoint is a different object: saved execution state for one thread or session. See Session state and recovery in Session state and recovery.
Checkpointer: A component that saves workflow state and restores it for later calls.
Chunk: The source unit placed in a retrieval index and later returned as evidence.
Chunk overlap: Text copied around a split into adjacent chunks so a claim crossing the split can remain together during retrieval.
Citation support: Whether the evidence attached to each claim supports that claim.
Client: In MCP, the host’s protocol component that connects to one server and exchanges messages with it. A host creates a separate client instance for each server it uses.
CodeAct: An agent pattern in which the model’s action is an executable program, run in an isolated sandbox.
Cohen’s kappa: An agreement measure that corrects observed agreement for the agreement expected by chance from each rater’s label rates.
Component evaluation: Assessment of one system stage using inputs and criteria suited to that stage, such as retrieval precision or answer faithfulness with fixed context. See Component contributions and improvements in Component contributions and improvements.
Confidence calibration: Agreement between predicted probabilities and the observed frequency of the event they describe. For a positive-class probability, compare with the frequency of actual positives. For confidence in a chosen answer, compare with answer accuracy. Results apply to the tested model, prompt, score calculation, generation settings, and data.
Confidence score: A number expressing how strongly a system supports a specified event or answer. A useful cutoff can be tested on labeled cases without interpreting the score as a calibrated probability.
Confused deputy: A security failure in which software with legitimate authority is tricked into using that authority for an unapproved caller or purpose. See Permissions and trust for connected tools in Permissions and trust for connected tools.
Confusion matrix: A table that counts each actual class against each predicted class.
Constrained decoding: Restricting next token choices to continuations compatible with permitted syntax. Section 2.4 explains grammar-based constraints and their limits.
Context engineering: Selecting, ordering, labeling, and trimming information supplied to a model, then logging the final request where permitted.
Context relevance: Whether the retrieved passages are useful for the question.
Context selection: Choosing which policy, history, and retrieved data enter a request.
Context window: The model’s token capacity for the serialized request and, for APIs with a shared limit, the generated output.
Conversation summary: A compressed representation of earlier turns. It should preserve decisions, constraints, entities, and unresolved questions while discarding conversational filler.
Corpus: The authorized collection of documents or records available to the retrieval system.
Correctness: Whether an answer matches a trusted result or expert judgment.
Cosine similarity: A similarity measure based on the angle between two vectors. It equals the dot product when both vectors have length one.
Criteria drift: The change in evaluation criteria that occurs as people grade real outputs and refine what they expect.
Cross-encoder: A scoring model that reads the query and one candidate together to score their relevance.
Data contamination: A condition in which benchmark items or close variants entered model training or evaluation-time retrieval, so a high score may not show generalization.
Decoding strategy: The rule that turns next-token probabilities into a selected token, such as taking the most probable token or sampling.
Deep learning: Machine learning based on neural networks with multiple layers.
Direct Preference Optimization (DPO): An alignment method that trains directly on preferred and rejected response pairs, without a separate reward model or reinforcement-learning step.
Document: In common RAG libraries, a text payload paired with metadata. Depending on loader configurations, it represents complete files, single pages, sections, or individual database records.
Dot product: The sum of the products of matching coordinates in two vectors. Section 6.5 uses it to compare a query vector with a passage vector.
Edge: In a workflow graph, a connection specifying which node can run next.
Embedding: A learned vector representation associated with a token or another discrete object. Similarity in embedding space is useful but does not guarantee identical meaning or factual equivalence.
Embedding model: A model that maps a text unit to a fixed-dimensional vector used for similarity search.
Episodic memory: Timestamped records of earlier interactions, actions, and outcomes that an application may retrieve while access and retention rules permit.
Error analysis: Grouping failures by cause to identify the affected system component and the next test or repair.
End-to-end evaluation: Assessment of final outputs from the complete system, with its stages running together. See Component contributions and improvements in Component contributions and improvements.
Euclidean distance: The straight-line distance between two vector endpoints. Smaller values mean closer vectors. Squared Euclidean distance gives the same ranking. See Vector search in Online retrieval.
Evaluation: A repeatable procedure that runs test inputs through a specific system version and scores the results without updating model parameters. Its results can guide later development changes and release decisions.
Evaluation harness: Software that runs evaluation tasks, records trial steps, grades results, and combines scores.
Evaluation run: One execution of an evaluation specification on a system version.
Evaluation specification (eval): A combination of fixed cases, a scoring procedure, a comparison target (scores from a reference system or a minimum or maximum acceptable score), and the product decision it informs.
Evaluation-driven development (EDD): The book’s workflow for treating each proposed change as an experiment with fixed inputs, explicit scoring, uncertainty, and a decision.
Evaluator-optimizer: A bounded loop that generates a candidate, grades it with an external evaluator, and revises it until the criteria pass or a budget expires.
Event log: A log describing an occurrence, such as a rejected tool argument, with the context needed to investigate it. Chapter 13 distinguishes event logs from traces.
Evidence validation: A check that each field in a response follows from the supplied data and application rules.
F1: The harmonic mean of precision and recall.
Faithfulness: Whether an answer’s claims are supported by the context supplied to the model.
Few-shot prompting: Prompting with instructions plus worked examples inside the request.
FLARE: A retrieval method that forms a query from predicted upcoming answer text when it contains low-confidence tokens, then regenerates using the retrieved information.
Foundation model: A model trained on broad data that teams can adapt to many downstream tasks.
Function calling: A model interface that returns a structured proposal for a tool name and arguments. The application validates the proposal and executes only permitted calls.
Generation evaluation: Assessment of a model’s generated answers for correctness, support from submitted context, citation support, and relevance to the question. See Answer metrics in Answer metrics.
Golden set: A versioned collection of evaluation tasks, each with an expected outcome or grading criterion.
Graph schema: A specification of allowed graph record types, relationships, and properties. See Online retrieval in Online retrieval.
Greedy selection: A decoding rule that selects a token with the highest next-token probability at each generation step.
Groundedness: Whether an answer’s claims are supported by the supplied evidence.
Guardrail metric: A measured requirement that a proposed change must satisfy, such as a latency, cost, or unsafe-answer limit. It differs from a safety control that blocks an action.
Handoff: A transfer of a task from one active agent loop to another, with the state needed by the next loop and the conditions for returning control.
Held out: Kept separate from the examples used to train or tune the version under evaluation. In Section 2.3, cases are excluded from prompt demonstrations and prompt tuning. Chapter 4 distinguishes development cases from held-out release cases.
Human-in-the-loop gate: A workflow pause for a person’s decision. Approval for an action is tied to its displayed parameters and relevant current state.
Hierarchical Navigable Small World (HNSW) graph: A layered graph structure used by an approximate nearest neighbor index to find candidate vectors without comparing the query with every vector.
Hybrid retrieval: Retrieval that combines lexical and semantic candidate lists to catch both exact terms and paraphrases.
Idempotency key: An identifier used by a service to recognize repeated submissions of one operation rather than treat each retry as a new operation.
Idempotent operation: An operation whose intended effect is the same whether an identical request is applied once or several times. See Workflow state and transitions in Workflow state and transitions.
In-context learning: Using instructions or examples in the prompt sent for a task to influence the model’s response. Its learned parameters stay fixed.
Index: A data structure that maps query representations to candidate source units. It may be lexical, vector-based, structured, or hybrid.
Inference: The operating mode in which a trained model receives a request and generates output while its parameters stay fixed.
Instruct model: A base model further trained to respond to instructions, usually through demonstrations and preference-based alignment.
Inverted index: A search structure that maps each indexed term to the documents or chunks containing it. Stored term counts and document lengths can support lexical ranking such as BM25.
IRCoT: A retrieval method that alternates retrieval with reasoning steps on multi-step questions, so retrieved information and generated reasoning guide subsequent steps.
Join: A workflow step that waits for required branch results and applies a defined rule for combining them.
JSON-RPC: A JSON message format for requests, responses, and notifications that need no response.
Key-value (KV) cache: Saved numeric attention state for tokens a model has already processed, reused during later generation. It is inference state rather than changed model weights or a stored answer. See When retrieval is useful in When retrieval is useful.
Knowledge graph: Records for entities such as organizations or events together with relationships between records. See Online retrieval in Online retrieval.
Knowledge-augmented generation (KAG): OpenSPG’s approach combining linked graph facts, supporting source passages and an LLM-generated plan of lookups and reasoning operations. See Online retrieval in Online retrieval.
Large language model (LLM): A language model trained at large scale to process and generate text tokens.
LLM-as-a-judge: An evaluation method that uses a language model to score or compare candidate outputs against a rubric.
LLMCompiler: A system that plans function calls as a dependency graph and dispatches ready calls in parallel.
Log-probability: The natural logarithm of a token’s probability: the exponent whose exponential gives that probability. An API may report probabilities before or after generation adjustments, or omit values. Documented placeholder values are not exact log-probabilities.
Logit: The unnormalized score a model computes for one vocabulary token at a generation step. It can be positive or negative.
Long-term memory: Records an application saves and supplies in later requests, such as a user’s preferences or earlier decisions. Saving these records does not update the model’s learned parameters. See Selecting information for each request in Selecting information for each request.
LoRA adapter: The learned matrices from Low-Rank Adaptation, whose product defines updates to selected model weights. Their updates can be applied alongside the original weights or merged into them for inference.
Logical form: A structured instruction specifying an operation, arguments, and references to earlier results. KAG uses these instructions to organize lookups and reasoning. See Online retrieval in Online retrieval.
Lost-in-the-middle behavior: The tendency of some models to use evidence near the start or end of a long context more reliably than equally relevant evidence in the middle.
Low-Rank Adaptation (LoRA): A fine-tuning method that trains small matrices whose product defines updates to selected model weight matrices while the original weights are fixed. See Answering from retrieved evidence in Answering from retrieved evidence.
Machine learning: An approach within AI in which software trains a model from data to make predictions or generate outputs on new inputs.
Macro averaging: The unweighted mean of the per-class scores.
Maximal Marginal Relevance (MMR): A selection method that trades relevance against redundancy with results already selected.
Mean Reciprocal Rank (MRR): The average over questions of one divided by the rank of the first relevant result, with zero when none is retrieved.
Memory system: The database, index, and application logic that manages persistent state across agent sessions.
Metadata: Source identity and attributes stored with a chunk, such as document, page, section, date, company, permission, or content type.
Micro averaging: Summing TP, FP, and FN across classes before applying the classification metric’s formula to those totals.
Mid-response steering: Additional user input received while a response is still in progress. The application or a supporting provider continues from updated conversation state. Earlier output and completed actions remain unchanged unless the application handles them separately. See Request, response, and validation cycle in Request, response, and validation cycle.
Model alignment: The process of making a model’s behavior better match intended instructions, preferences, or safety criteria in specified situations. Chapter 1 focuses on further training that adjusts the model’s parameters toward those behaviors. See Model alignment in Model alignment.
Model Context Protocol (MCP): A client-server protocol enabling an AI host to discover and invoke server-provided tools, resources, and prompt templates.
Model judge: A language model used to grade outputs against a rubric.
Multi-agent system: A system that coordinates two or more agents through explicit communication, routing, termination, and rules for which component may read or change shared state.
N-gram: A run of n neighboring tokens, used by overlap metrics such as BLEU and ROUGE.
Neural network: A model that transforms numeric inputs through connected processing layers, using parameters learned during training.
Node: In a workflow graph, a function that reads state and returns updates.
Normal distribution: A probability distribution with a symmetric, bell-shaped curve around its mean. The exact paired t-test in Section 4.8 assumes this distribution for the per-case differences.
OAuth: An authorization protocol for obtaining limited access to a service through access tokens. In the user-delegated flow in Section 9.8, a user grants access without sharing the user’s password with the client.
Observation: The environment result returned after an action, such as tool output, error, updated state, or user response.
Observed agreement: The fraction of audited cases for which two label sources, such as a human and a model judge, give the same label.
Online controlled experiment: Random assignment of users or sessions to two system versions, followed by a comparison of product outcome metrics. Also called an A/B test.
Open weights: Trained model parameter files made available for use under a license. Their availability alone does not establish access to training data or unrestricted rights to use the model.
Optical Character Recognition (OCR): Conversion of text in page images into searchable text. Extraction checks still need to identify missed or misread content.
Orchestrator-worker pattern: A pattern in which an orchestrator decomposes a request at run time, assigns subtasks to workers, and combines their results.
Output specification: The fields, types, allowed values, evidence requirements, and missing-evidence response that make a model answer usable by the next program step.
p95 latency: The ninety-fifth percentile of measured response times, calculated with a specified quantile method. Ties and finite samples can change how many observations fall above it. See Product requirements and evaluation criteria in Product requirements and evaluation criteria.
Parallelization: A workflow pattern that runs independent subtasks concurrently and joins their results.
Parent-child retrieval: Matching a query against small source units, then expanding a selected unit to its larger parent section for generation.
Parameter: A number learned during training and stored in the model. Parameters, also called weights, stay fixed during ordinary inference. Training updates them, while loading another saved model version, called a checkpoint, replaces the set used for inference.
Pass@k: A code-generation metric for the probability that at least one of k generated attempts passes the tests.
Pass^k: A repeated-trial measure that counts success only when all k attempts for a task succeed.
PKCE: Proof Key for Code Exchange, which binds an authorization code to a secret verifier held by the requesting client.
Plan-and-Act: A method that separates a high-level planner from an executor that carries out environment actions. Its replanning variant revises the plan from observations.
Plan-and-Execute: A planner that creates a full plan before acting and then runs its steps.
Plan-and-Solve: A prompting method that first divides a task into subtasks and then carries out the plan. It does not define a complete production planning architecture.
Planner: The model or software component that proposes steps and their order from a goal and the available state. The application checks the proposed work before execution. See Choosing a planning strategy.
Position bias: A judge’s preference for one answer position in a comparison, regardless of the answers’ content.
Positive class: The outcome a classification task treats as present, such as
requires escalation. It fixes which errors count as false positives and false negatives.Precision: The share of positive predictions that are correct.
Precision@k: The share of the first k retrieval result positions filled by distinct labeled evidence units. Unfilled positions count as misses under this convention. See Retrieval metrics in Retrieval metrics.
Prefix: The token sequence already present when the model computes scores for the next position.
Pretraining: Training on broad text data to learn general patterns before later task-specific tuning.
Procedural memory: The agent’s model parameters and reusable code or instructions that determine how it performs a task.
Product requirement: A verifiable statement of required or forbidden system behavior under stated conditions, traced to a product outcome.
Promotion: The decision to make a captured candidate available as a reusable fact or procedure after checking supporting records, identity, intended use, confidence, expiry, and required authority. Human review is required only where policy calls for it.
Prompt: The input text that describes a task and can include instructions, examples, or both.
Prompt chaining: A workflow pattern that splits a task into steps whose intermediate outputs can be typed, inspected, and retried.
Prompt engineering: Improving how task instructions, examples, and output requirements are written in a model request.
Prompt injection: An attack in which instructions placed in text the application treats as data change model behavior or tool use. It is direct when the user writes them and indirect when they arrive in retrieved or external content.
Provenance: The record of the supporting sources used to validate a stored fact.
Proxy metric: A measurement used in place of the outcome of interest. It can be measured offline or in production. See Section 4.1.
RAG triad: A three-part view of RAG quality: context relevance, faithfulness, and answer relevance. TruLens calls its answer-support dimension groundedness. See RAG quality measures in RAG quality measures.
RAGAS framework: A research framework for automated RAG evaluation. The original paper describes reference-free checks for context relevance, faithfulness, and answer relevance. The
ragaspackage supplies software implementations. See Answer metrics in Answer metrics.ragas: A software library providing metrics for evaluating retrieval-augmented answers and their contexts. Metric results depend on the evaluator and assessment procedure.ReAct: An agent-control pattern that interleaves reasoning, actions, and environment observations so a later decision can use the latest result. Validation, authorization and budgets are additional application controls. See ReAct with explicit budgets.
Reasoning: Using available information through one or more intermediate steps to reach a conclusion. In an LLM system, this may refer to internal model computation, generated intermediate steps, or application-managed stages. See Model alignment in Model alignment.
Reasoning tokens: Tokens a model generates before its final answer that a provider may hide, summarize, or count separately.
Recall: The share of actual positive cases that the system finds.
Recall@k: The share of all labeled evidence units found among the first k retrieval results. It requires a nonempty labeled evidence set. See Retrieval metrics in Retrieval metrics.
Reciprocal rank fusion (RRF): A fusion method that combines ranked lists by summing 1/(k + rank) for each candidate, ignoring raw scores on different scales.
Reducer: A rule for applying an update to a state field, such as appending a trace entry rather than replacing the trace.
Reflection: A revision pattern in which a model critiques its own or another model’s output before another attempt.
Reflexion: A feedback method that generates a verbal reflection from an evaluator’s feedback and the prior attempt, then saves it for the next attempt without updating model weights.
Reinforcement learning from human feedback (RLHF): An alignment setup in which response comparisons train a reward model, and reinforcement learning then makes higher-scoring responses more likely.
Release decision record: A versioned document that links the task definition, evaluation results, component traces, thresholds, blocking failures, and follow-up to a PASS or HOLD decision for specified conditions.
Release gate: The set of score limits and mandatory checks a candidate must pass before release. See Product requirements and evaluation criteria in Product requirements and evaluation criteria.
Request context: The policy, user input, history, retrieved evidence, tool descriptions, and other information assembled for one model request.
Request trace: A record of the policy, history, evidence, tools, token allocations, and other inputs actually sent for one model request.
Requirement traceability: The links from each requirement to its measurement, threshold, evaluation cases, and results.
Reranker: A model or scoring step that examines a query together with each retrieved candidate and reorders the results. Task evaluation determines whether the new order improves retrieval. See Online retrieval in Online retrieval.
Resource server: The server, such as an MCP server, that checks an access token before serving the requested capability.
Retrieval-augmented generation (RAG): Retrieving external evidence at request time and placing it in the model’s context so the answer can be grounded in it.
Retrieval evaluation: Assessment of the ranked source results returned for each question. The metrics in Chapter 7 compare those results with labeled evidence and can be summarized across questions. See Retrieval metrics in Retrieval metrics.
Retrieval hit at k (Hit@k): A retrieval measure that equals one when at least one labeled evidence item appears among the first k results for a question, and zero otherwise. Also called evidence hit at k. See Retrieval metrics in Retrieval metrics.
Retriever: The search component that takes a query and returns candidate passages from the document collection. See Offline ingestion and online answering in Offline ingestion and online answering.
Retrieved evidence: Source records found by searching stored documents or querying a service for the current request.
Reward model: A learned scorer that assigns a score to a candidate response in its prompt context.
ReWOO: A planning method that separates planning from tool observations. Workers fill planned output variables with tool results, and a solver uses the plan and evidence to answer.
ROUGE: A family of n-gram overlap measures developed for summary evaluation.
Routing: A workflow pattern that classifies a request into a small explicit set and sends it to a specialized workflow.
Rubric: Grading criteria and rules for assigning grades to an answer. These rules are separate from the thresholds used to decide whether a system can be released.
Rubric drift: Changes in grading criteria or in how they are applied over time or versions. Ambiguous fixed criteria can produce inconsistent grades without a change over time.
Safety guardrail: A policy or validation step that restricts inputs, outputs, tool use, or state changes. It may use deterministic code, a model, or human review. Authorization for actions with external effects should not depend on model judgment alone.
Sampling: Drawing a token at random according to candidate probabilities. More probable tokens are selected more often across repeated draws.
Sampling weight: A factor that adjusts each evaluation case’s contribution to reflect its frequency in the target traffic rather than the evaluation sample. Classification metrics and error costs explains the disjoint-group calculation.
Schema: A description of the fields, types, and values allowed in a data object.
Security boundary: A division enforced by software controls such as identity, permission, validation, or isolation. Prompt delimiters do not create one.
Self-enhancement bias: A judge’s preference for outputs from its own model family.
Selective retrieval: A history or knowledge policy that searches stored records and includes only items relevant to the current request.
Semantic memory: Curated facts or preferences stored separately from the events from which they were derived.
Semantic search: Retrieval that ranks evidence by embedding similarity instead of exact word overlap.
Service level objective (SLO): A target value or range for a measured service indicator, such as p95 latency or error rate.
Shadow deployment: Running a candidate on copies of live requests while users receive only the current system’s answers.
Slice: A subset of evaluation cases grouped by a relevant property, such as language, customer tier, risk level, query type, document length, or tool requirement.
Sliding window: A history policy that keeps the most recent turns and drops older ones. It is cheap and predictable but can lose earlier commitments.
Softmax: The function that turns logits into probabilities by exponentiating each score and dividing by the sum, so the results add up to one.
Span: A description of one operation, with its start and end times and associated information. Parent and trace identifiers connect related spans. See Chapter 13.
State machine: A process represented by its current state and allowed transitions to another state.
Streaming: Delivery of partial response events while generation continues. A streamed fragment is provisional until the response finishes and the application validates it. See Request, response, and validation cycle in Request, response, and validation cycle.
Structural validation: A check that a response has the required shape, fields, and types.
Structured output: An approach that declares the fields and value types receiving code expects. Supported schema constraints can restrict generation to permitted fields and values.
Subword token: A token that can be a whole common string or a fragment of a rarer word, so the same text can produce different token counts under different tokenizers.
Supervised Fine-Tuning (SFT): Further training on prompts paired with suitable responses, which makes those responses more likely.
Swarm: A multi-agent pattern in which the active worker hands control to the next worker without a central router. It does not require all workers to run at once.
System instruction: Application rules supplied in the interface’s designated instruction role. Their priority relative to user messages depends on the model and interface. These rules describe limits and required handling that persist across requests.
Task DAG: A directed acyclic graph whose nodes are executable tasks and whose edges state that one task depends on another task’s output.
Task specification: A statement of the goal, supplied data, allowed evidence, response fields, and what to return when evidence is missing or conflicting.
Temperature: For positive values, a generation setting that divides logits before softmax. Lower positive values concentrate probability on the highest-scoring tokens. Zero-temperature behavior is a separate API convention.
Token: A piece of text, such as a word, word fragment, punctuation mark, or byte, that a tokenizer maps to an identifier the model processes.
Tokenizer: The component that splits text into tokens and gives each token an identifying number from a fixed vocabulary.
Tombstone: A small record stating that an item was deleted or replaced, without retaining the removed value. Replicas and indexes use it to avoid restoring an old copy. See Correction and deletion in Correction and deletion.
Tool: An operation the software can execute, with a name, description, defined input types, and a result returned to the model or workflow.
Tool allowlist: The set of tools and tool schemas that application policy permits for the current identity and task.
Tool description: The information supplied to a model about one callable function and the arguments it accepts.
Tool specification: A description of a capability’s purpose, inputs, outputs, errors, effects on stored data or other systems, and required permission.
Top-k: A sampling rule that keeps only the k most probable tokens before sampling.
Top-p: A sampling rule that keeps the smallest set of most probable tokens whose total probability reaches a threshold.
Trace: Linked operations belonging to a request, including nested or repeated steps. It shows how work relates across model, retrieval, and tool operations. See Chapter 13.
Training: The operating mode in which software compares predictions with example targets or preferences and updates the model’s parameters.
Trajectory: The ordered record of model decisions, tool calls, observations, and state changes in one trial.
Transformer: A neural-network design in which attention lets each token representation combine information from other positions.
Transition: In a workflow, movement from one state or step to another under a defined rule.
Trial: One attempt at one evaluation task. Because sampling, retrieval, and tools can vary, the same system can produce different answers across trials.
Unigram: A subword tokenizer that scores possible splits of a word and selects a likely one.
User or resource owner: In OAuth-based MCP authorization, the person who grants permission.
User request: The current task or question. It can include user data, but the application must still treat that data as input rather than as a rule that overrides its own policy.
Vector: A list of numbers used to represent a token, text unit, or other object for computation.
Vector database: A system that stores vectors together with source text and metadata so an index can return similar records for a query vector.
Vector search: Comparing a query vector with stored vectors under a chosen similarity or distance measure to find candidate records. Section 6.5 explains how their IDs lead to passage text and metadata.
Veto requirement: A mandatory product requirement whose failure blocks release and produces a HOLD decision. See Product requirements and evaluation criteria in Product requirements and evaluation criteria.
Verbosity bias: A judge’s tendency to favor longer answers because they appear more complete, even when length does not improve answer quality.
Vocabulary: The fixed list of token identifiers a tokenizer and model share. Its entries may represent complete words, word fragments, punctuation, bytes, or other subword units.
Weighted averaging: The sum of per-class scores multiplied by each class’s share of actual cases in the evaluated data.
Wilson score interval: An interval estimating a binomial success probability from a success count and trial count under independent, equal-probability trials.
WordPiece: A subword tokenizer that takes the longest matching vocabulary pieces from left to right.
Workflow: A predefined sequence or graph of model calls, tool calls, validations, and state transitions whose control flow is governed by application code.
Workflow quality: How well the complete process meets the task’s requirements, including answer correctness, allowed actions, and limits on time and cost. See Building and measuring prompt chains in Building and measuring prompt chains.
Workflow state: The data available at a workflow step, such as an order ID, selected policy, draft reply, and status. It can be temporary or saved between calls.
Working memory: Active task state, such as the goal, plan, and intermediate results. The application selects part of this state for the current model request.
Zero-shot prompting: Prompting with instructions but no demonstrations.