Glossary

Definitions of terms used across the book, grouped alphabetically.

A term can be familiar while its meaning changes between training, retrieval and authorization. These short entries support lookup. The system foundations and threat model provide the starting context. The linked chapters explain mechanisms and limits. The entries distinguish meanings where one word serves different roles.

A

absolute skewness: For a variable with positive variance, the third absolute central moment divided by variance to the power \(3/2\). The aggregation analysis bounds this quantity for each gradient coordinate. See the explanation.
abstention: Abstention withholds a model decision under a stated rule.
acceptable-use rule: An acceptable-use rule sets conditions on permitted AI tasks, data, outputs, or actions.
acceptance criterion: An acceptance criterion states the evidence and threshold required for use.
acceptance threshold: An acceptance threshold is the measured result required by an acceptance criterion before a test passes.
accepted risk: Accepted risk is specified remaining risk approved by an authorized role.
access: In a threat model, the interfaces, data, or operations available to the attacker.
accessibility: Accessibility is the ability of people with differing needs to use the service.
accountable owner: The person or role responsible for a result, with authority to accept it or escalate the decision.
account abuse: Account abuse is unauthorized or deceptive use of a user account.
action budget: A maximum count, cost, time, data volume or number of affected objects for one task. A transaction limit instead bounds each individual operation. See the explanation.
action proposal: A model-generated description or structured request for an operation. Surrounding software must still decide whether to allow it.
actuator: A hardware mechanism or control component in a cyber-physical system that converts software commands into physical motion, mechanical adjustments, or environmental changes.
adapter: A smaller parameter set or added trainable module used to adapt a base model without retraining all its original weights. The partial-tuning example keeps the base model frozen while updating adapter parameters. See training and adaptation.
adaptive evaluation: Adaptive evaluation changes the attack to account for the defense.
adaptive query: A query chosen using earlier answers to guide the next request. See the explanation.
adaptive red teaming: An evaluation process in which testers use observed responses to choose the next attempt and test a stated control boundary.
administrative access: The privileged authority to inspect, configure, modify, or control compute platform layers, hosting infrastructure, orchestration systems, or model deployment environments.
administrative endpoint: An interface through which an authorized administrator can change platform settings or state.
adoption measure: An adoption measure is a count or rate describing approved use of an AI capability.
adversarial case: A test case exercising stated attacker capabilities to try to cause a security failure. Plural: adversarial cases. Its outcome remains tied to the supplied capability, expected result and configuration. See the explanation.
adversarial example: An input deliberately constructed to make a model produce an attacker-selected error under a stated attack constraint.
adversarial training: Adversarial training minimizes loss on adversarial examples produced inside the training procedure.
advisory check: An advisory check produces information for a model or person.
affected person: An affected person is a person whose rights, safety, access, or interests may be affected.
agent: Here, an application loop that uses model output and task state to select a tool or another step, processes the result, and continues until completion or a stopping rule. Access, disclosure and action permissions remain separate checks. See tools and agents and action authorization.
agent memory: Agent memory is application-managed information retained across steps or sessions, such as prior user preferences, summaries, tool results, or task state.
AI application: Software that uses model output as part of a task. Its components and permitted actions depend on its design. The reference assistant’s services are an example. See the nested objects.
AI dependency: An AI dependency is a model, data set, library, service, prompt resource, or other component on which the AI path depends.
AI RMF: NIST’s AI Risk Management Framework. Govern, Map, Measure and Manage connect context, evidence, decisions and responsibility. These functions complement attack discovery rather than supplying a catalogue of attacks. See risk work and discovery.
AI system: An AI system includes models, application software, data paths, infrastructure, identities, and people. Their interactions produce decisions and security outcomes.
answer tampering: Altering generated text after the model produced it, or substituting another response for the model’s actual response. See the explanation.
application authority: Application authority is the set of disclosures and actions that surrounding software can cause.
application date: The date from which a legal provision applies. It can differ from the date the law enters into force. See legal scope and dates.
application programming interface (API): A definition of how one software component requests work from another. It crosses a relevant trust boundary when caller and receiver have different permissions or operators. See the explanation.
application trace: An application trace links one user request to the identities, model call, retrieved sources, policy decisions, tool arguments, external results, and final response that followed.
appraisal: Evaluation of attestation evidence against a policy and approved reference values. A verifier performs it. A relying service makes the separate access decision. Appraisal does not establish safe future behavior. See the explanation.
approval limit: An approval limit sets the maximum value or action class an identity may approve.
approval object: In this guide, a record binding approval to an operation, resolved destination, arguments, expected effect, current state, expiry time, and a digest of those details.
approved artifact: An exact version admitted for a specific environment and purpose. For software or model components, see component acceptance.
approved component: The exact component version admitted for a specific environment and purpose. Approval does not transfer automatically to another version or environment. See the explanation.
artifact digest: A cryptographic hash used to compare an object’s bytes with a reference. For files, this is the file digest used in version checking. A matching digest does not establish safe behavior.
asset: A resource such as data, model parameters, credentials, or execution state that an organization protects from unauthorized disclosure, change, loss, or destruction.
assurance claim: An assurance claim is a specific statement about behavior or risk supported by reviewable evidence.
assurance scope: Assurance scope identifies the systems, services, locations, controls, and period covered by evidence.
ATLAS: MITRE’s Adversarial Threat Landscape for AI Systems, a catalogue of adversary tactics and techniques, mitigations and case records. A catalogue entry suggests a candidate. It does not establish local vulnerability or frequency. See attack discovery.
attack budget: The maximum bound on adversary resources, such as the number of poisoned records, allowable perturbation magnitude, or query count permitted during an attack.
attacker capability: The change or interaction that an adversary can perform.
attack objective: An attack objective states the disclosure, integrity change, loss of availability, or unauthorized action that counts as success.
attack protocol: An attack protocol states what the evaluator knows, can change, and can observe during one trial.
attack success rate: Attack success rate is the fraction of eligible cases where the attack reaches its objective.
attack tree: A decomposition of one attacker goal into smaller goals. An OR branch offers alternatives. An AND branch requires every child condition. Costs and likelihoods need evidence beyond the drawing. See attack discovery.
attester: In remote attestation, the entity that produces evidence about its target environment for a verifier to assess.
attribute inference: An attempt to infer an unknown attribute of an individual from available records or model responses.
attribution: Linking activity to an actor or system with stated confidence. It requires supporting records separate from evidence that a technique works. See the explanation.
attribution evidence: Attribution evidence supports a conclusion about who caused an event and with what confidence.
audit scope: An audit scope states the systems, controls, locations, and time period examined.
authenticity: Whether content is what the identified source issued, without an unauthorized change. It does not establish factual truth. See retrieved-source checks.
authorization server: A service that issues access tokens under its authorization policy. Token issuance and the receiving service’s checks are distinct decisions. See the explanation.
auxiliary data: Information available to an attacker from sources other than the target interface. See the explanation.
availability poisoning: Training-data poisoning intended to reduce useful model performance across many ordinary inputs.

B

backdoor: Attacker-selected model behavior activated by a particular trigger. It may be introduced through training or direct changes to the model.
base rate: The fraction of evaluated events that are real failures. It affects how many detector alerts correspond to real failures. See the explanation.
batch size: The number of training examples contributing to an ordinary parameter update. See training settings. The DP-SGD explanation specifies its sampled lot and averaging divisor separately.
bearer token: A token whose possession is enough to present the represented access. Audience and scope restrictions limit a copied token only when receiving services enforce them. See the explanation.
behavioral intake test: A behavioral intake test runs clean-task and targeted security cases against an acquired component.
behavioral regression: A deterioration from previously accepted behavior on specified tests. A measured change needs interpretation before it is called a regression.
Bernoulli trial: A trial with a success-or-failure result. The binomial model here additionally assumes independent trials with one common success probability. These assumptions are not established by the observed fraction. See the explanation.
black-box access: In a query-interface example, the ability to submit inputs and observe returned outputs while the interface hides model internals. This describes the interface, not all the information an attacker may know from elsewhere. See the knowledge distinction and query-only tests.
black-box setting: Little or no internal knowledge of a specified target model or system. The threat record lists the actual known fields and records query or modification access separately. See knowledge and access.
blocking check: A blocking check is placed at an enforcement point where it can prevent the protected operation.
Breakout Scale: An ordered scale from 1 to 6 for rating the spread of an influence operation. Category meanings follow the cited assessment. A spread rating does not measure changes in audience beliefs or behavior. See influence-campaign evidence.
browser origin: A browser origin identifies a web protection domain based on scheme, host, and port.
build origin record: A build origin record states how a build platform produced an artifact from a described definition.

Byzantine robust aggregation: An aggregation method designed to tolerate a bounded number of arbitrary participant updates under stated assumptions.

C

C2PA Content Credential: A signed record associated with media that describes origin or editing. Validation supports the signed statements under an accepted trust chain. It does not prove the content true. An absent or invalid credential does not prove synthetic origin. See the explanation.
CaMeL: An agent design separating a privileged planner from a model that extracts untrusted values, with an interpreter tracking data sources and dependencies and enforcing tool-use policy. Its protection depends on the trusted components and policy. See the explanation.
canary: A unique, controlled sequence deliberately inserted into a training dataset to empirically test and quantify unintended data memorization by a model.
capacity use: Capacity use is the share of available compute capacity doing useful work.
case fact: A case fact is an assumption supplied by the exercise rather than inferred by the reader.
certified bound: A certified bound proves a property for specified inputs and perturbations under its mathematical assumptions.
change record: A change record preserves evidence of a system change and its assessment.
checkpoint: A saved state. A model checkpoint contains model parameters and may also contain optimizer state and training metadata. A workflow checkpoint records execution state. A chapter checkpoint is a learning exercise, not saved software state.
citation support: Whether the cited source passage supports the specific claim attached to it.
clean cases: Unmanipulated evaluation inputs used to measure ordinary task performance.
clean holdout: A clean holdout is the evaluation holdout used for ordinary task behavior, with its data excluded from training.
clean-label attack: A poisoning attack intended to change learned behavior through selected or altered training examples whose task labels remain correct. Correct labels do not establish harmless content or attack success. See poisoning methods.
clinical authority: Clinical authority is permission and responsibility to make or direct a clinical decision.
collection permission: Collection permission is the legal, contractual, or organizational basis for obtaining data.
comparable evaluation: A comparable evaluation tests alternatives with the same task set and decision criteria.
complementary user entity controls: Customer controls that a service organization assumes will operate alongside its own controls to meet the applicable assurance criteria. Other customer duties can fall outside this list. See supplier assurance.
complete mediation: Checking every request to access a protected object or perform a protected operation against the applicable authorization policy. Retries and cached decisions must not bypass required checks.
confidence interval: A range calculated by a procedure designed to cover an unknown rate in a specified fraction of repeated evaluations under its sampling assumptions. Approximate coverage can differ from the target. The level does not assign a probability to the fixed unknown rate after an interval is observed. See the explanation.
confidential computing: Protection of data during computation through a hardware-backed, attested execution boundary. The protected objects, excluded parties and remaining trusted components depend on the design. See the explanation.
confidential information: Confidential information is information protected from unauthorized disclosure by duty or agreement.
configuration evidence: Configuration evidence records effective settings and permissions.
conflicting authority: Conflicting authority exists when two instructions or permissions cannot both be followed safely.
connector: An integration that lets an application read from or act on another system. Effective access depends on its human or service identity, granted permissions, the receiving service’s checks and the application’s user and disclosure rules. See the operating arrangements and identity chains.
connector scope: A connector scope is the set of resources and operations granted to an integration.
consequential output: A consequential output can materially affect a person, asset, obligation, or decision.
constrained execution: Constrained execution runs only an authorized operation within fixed limits.
constrained output: A model response restricted to a required format or allowed values. A format restriction does not itself establish that the content is true or an operation is authorized. See output controls.
container: A container isolates processes while sharing the host operating-system kernel.
containment: Containment reduces ongoing harm while preserving enough state for investigation.
content authentication: A method that binds checkable statements about a file’s origin or edit history to that file.
contestability: Contestability is the ability to challenge a decision and seek correction or review.
context: The sequence of tokens available for the current model call.
context assembly: The application stage that selects, truncates, orders, and formats retrieved passages, system instructions, and user queries into a coherent prompt sequence for model inference.
context separation: Keeping the source and trust labels of model inputs distinct. Preserving those labels alone does not guarantee that a model follows the intended instruction order. See instruction separation.
context-specific encoding: Context-specific encoding transforms data for the rules of one output context.
continuity term: A continuity term is a contract condition intended to preserve essential operations during disruption or exit.
contract evidence: Legally binding documentation and third-party commitments specifying supplier obligations regarding data isolation, retention, security controls, and audit rights.
controlled demonstration: An empirical proof-of-concept conducted in a laboratory or staged environment demonstrating that an attack technique or exploit mechanism can succeed under specified conditions.
controlled update: A version change that reviewers test, stage and approve before it reaches production. See the explanation.
control placement: Control placement identifies the input, retrieval, output, or action boundary where a decision is made and the object that decision protects.
control plane: The control plane contains the interfaces that manage resources, identities, and configuration.
control record: A compact description of an enforcement point, checked facts, decision, evidence, cost, bypass, recovery route, and limit.
control resilience: Control resilience concerns the system’s behavior when a detector, policy service, log sink, or human reviewer becomes slow or unavailable.
control tradeoff: A control tradeoff records both the benefit and cost created by a security choice.
control traffic: Control traffic carries requests that change system state or configuration.
convex loss function: A loss that is convex in the model parameters. On a convex feasible set, it has no local minimum worse than the global minimum. This is a condition on the loss and allowed parameter set, not a general guarantee against poisoning. See the explanation.
coordinate median: An aggregation rule that takes the median separately at each vector coordinate. Its combined vector can contain values from different participants. See aggregation methods and conditions.
coordinate trimmed mean: An aggregation rule that removes a chosen fraction of the largest and smallest values at each vector coordinate and averages the remainder. Its guarantees require assumptions about honest data and the malicious fraction. See aggregation methods and conditions.
correlation identifier: An identifier connecting events for one request across services, including retrieval, model calls, policy checks and external results. Plural: correlation identifiers. See the explanation.
coverage measure: A coverage measure combines a numerator with the full population that the count is meant to represent.
credential: Evidence of a calling identity or access grant. The receiving service must still validate it and apply the current authorization rules. See the explanation.
credential rotation: Credential rotation replaces authentication material and revokes the old material.
crosswalk: An organization may use a crosswalk, a scoped mapping between related items in different sources.
cyber-physical system: A cyber-physical system joins computing, communication, people, and physical components.

D

dangerous capability evaluation: A test of whether a highly capable model can materially assist severe harm under a stated access and task setting.
data at rest: Information residing in non-volatile physical or cloud storage, including databases, object buckets, file systems, disk volumes, and backups.
data classification: Data classification groups records by the harm that disclosure, alteration, or loss could cause.
data flow: A data flow describes how data moves among users, systems, suppliers, and stores.
data in transit: Information traversing an internal network, cloud fabric, or public internet between services, clients, models, or third-party providers.
data in use: Data being processed, for example in processor registers, cache, or main or accelerator memory. The term describes its state, not whether it is protected.
data loss prevention check: A check that compares content or metadata with disclosure rules before a transfer.
data minimization: The privacy and security practice of restricting data collection, processing, and retention strictly to elements required to fulfill an authorized and documented purpose.
data-only artifact: A data-only artifact is intended to contain values under a restricted schema.
data poisoning: Deliberate changes to training data intended to change the behavior learned through parameter updates.
data reconstruction: An attempt to recover a private record or a close approximation from available information or model responses.
dataset version: A dataset version identifies the exact records and transformations used by a run.
data traffic: Data traffic carries prompts, model inputs, outputs, or stored records.
decision-based attack: A decision-based attack sees only the final label or decision.
decision condition: A decision condition is an event or threshold that changes an approval.
decision record: A decision record captures the options, evidence, responsible role, date, conditions, and unresolved issues behind a decision.
delegation: In token exchange, recognition of an actor as a separate party acting for a subject. The receiving service still enforces operation permissions. Impersonation instead treats the actor as the subject in the receiving context. See the explanation.
delegation chain: A delegation chain records how one actor grants another limited authority.
dense search: Retrieval that compares learned embeddings and can rank related passages using different words. Similarity does not establish truth or authorization. See the explanation.
dependency substitution: Dependency substitution causes a build or user to obtain an attacker-controlled component in place of the intended one.
deployer: In the EU AI Act edition cited in the role definitions, a natural or legal person, public authority, agency or other body using an AI system under its authority, except use in a personal non-professional activity.
deployment interface: A deployment interface accepts commands or configuration for workloads.
derived artifact: A later object created from source data, such as an index or embedding. These derived copies require their own handling and deletion checks. See derived data and copies.
derived data: Data produced from source records, such as index entries, embeddings or generated summaries. A deletion review must follow these copies separately from the source record. See the explanation.
deserialization: Deserialization reconstructs software objects from stored bytes.
detective controls: Checks of events or records for signs of a possible failure. A detected event still needs interpretation and appropriate response. See control responsibilities.
detector: A filter or classifier that flags a specified pattern in inputs, outputs, or system activity. A flag is evidence to assess, not proof of an attack.
device security state: Device security state describes whether specified security functions operate as expected.
differentially private stochastic gradient descent: A training method that clips each example’s gradient and adds random noise before a parameter update.
differential privacy: A definition bounding how much a randomized mechanism’s output distribution may change between datasets differing in one declared protected unit, under a specified neighboring-data relation. The unit can be one record or one person’s full contribution. An add-or-remove-one-record relation is person-level only when each person contributes at most one protected record, or all of that person’s contributions are treated as the unit. Epsilon and delta quantify the allowed difference. The guarantee applies to the analyzed mechanism’s outputs, not an unprotected disclosure path. See the formal relation and protected unit.
digest: A digest is a cryptographic hash of an artifact. For a file, this is also called a file digest. See file comparison and its limits.
direct prompt injection: A prompt injection attempt supplied through a caller-controlled prompt or interaction. Direct describes the delivery route. A jailbreak describes a goal of bypassing model safeguards. See direct prompt attacks.
direct prompt manipulation: Using caller-controlled instructions to redirect model behavior, including attempts to bypass model safeguards. See the explanation.
dirty-label attack: A poisoning attack whose allowed changes include incorrect training labels. Correct-label checks alone do not cover clean-label poisoning, and the label category does not determine a universal success rate. See the explanation.
disputed answer: A disputed answer is an answer whose correctness or authority is challenged and resolved outside the model.
document chunk: A document chunk is a bounded passage prepared for retrieval. Plural: document chunks. See document preparation.
drafted action: A drafted action prepares exact arguments for review.
due date: A due date sets the date by which an action or review is expected.

E

edge deployment: An edge deployment runs a model or AI application on or near the device that supplies data or receives action.
effective date: A source-dependent legal date label. The source must specify whether it means entry into force or the start of a particular provision’s application. See legal scope and dates.
egress: Network traffic, API responses, or data flows departing from an internal service boundary toward external endpoints, third-party APIs, or public networks.
electronic protected health information: Identifiable information about a person’s health, care or payment for health care that is transmitted or maintained in electronic media as defined by HIPAA and meets HIPAA’s creator or recipient conditions and record exclusions. Electronic health data alone does not establish this category or who has legal duties. See health records and clinical authority.
embedded AI feature: An embedded AI feature is an AI capability included inside another purchased product.
embedding: A numeric representation produced by a model. Search compares passage and query vectors, or lists of numbers. With encoders fixed, producing these vectors is inference, not further training. Plural: embeddings. See retrieval representations.
empirical attack test: A test that runs specified attacks and measures their outcomes under recorded conditions. Its results apply to that evaluation setting.
empirical risk minimization: Minimizing the average error penalty that a chosen loss function assigns to training examples. See the explanation.
endorsement: In attestation, an authenticated statement supporting trust in an attester’s capabilities, such as evidence collection or signing. It serves a different purpose from a reference value. See the explanation.
enterprise AI inventory: An enterprise AI inventory is a controlled record of AI systems and AI-enabled services used by the organization.
error-correcting code: Redundant information added to memory so specified errors can be detected or corrected. Abbreviated ECC. The protection depends on the error and platform. See the explanation.
escalation: Escalation transfers the decision and evidence to a person or service with the required authority.
evaluation: A process that measures model behavior on selected cases without updating model parameters.
evaluation contamination: Here, compromise of test records or expected answers, or loss of intended independence when test information enters training. A legitimate documented correction is not necessarily contamination. See recovery evaluation data and independent test cases.
evaluation data: Evaluation data represents the tasks and failures that govern release.
evaluation holdout: An evaluation holdout is a test set kept outside parameter updates so it can measure behavior on unseen cases.
evaluation reliability: Evaluation reliability concerns whether another qualified team can understand and repeat the result closely enough for the same decision.
evaluation scope: An evaluation scope states the system, task, protected assets, expected behavior, attacker access, failure conditions, and deployment configuration covered by a test.
evasion attack: An evasion attack changes an inference input to alter model behavior without changing model parameters.
evidence class: The kind of claim a record can support, such as feasibility, a controlled demonstration, observed operational use or prevalence. These classes require different evidence. See the explanation.
evidence preservation: Evidence preservation keeps records available and trustworthy enough to reconstruct the event.
exact Clopper-Pearson interval: A confidence interval obtained from binomial tail probabilities. Its coverage is at least the nominal level for a fixed number of independent trials with one common success probability. This guarantee does not correct a faulty sampling design. See uncertainty methods.
executable artifact: A component that can invoke code through its format, loader, package hooks, or custom model code.
execution environment: An execution environment is the operating-system and network context in which agent-generated commands run.
execution exposure: Execution exposure is an interface condition that permits unauthorized or unsafe execution.
execution isolation: Execution isolation limits the processes, files, devices, memory, and network resources a workload can affect.
exit cost: Exit cost is the work and cost required to leave a supplier or deployment design.
expected loss: Expected loss is a probability-weighted estimate of adverse impact, but rare AI events may leave both probability and impact highly uncertain.
expiry: The time limit on a token’s validity. It differs from revocation, which withdraws a grant or credential before that limit. See the explanation.
explanatory feedback: Explanatory feedback states the evidence and reasoning behind an assessment.
exposure: In memorization testing, a measure based on how highly a canary ranks among possible sequences under a specified model and scoring rule. In retrieval testing, exposure instead means that selected material is included in the model context.
external policy enforcement: Software outside the model applies an authorization policy before an operation takes effect. See action policy and enforcement.

F

factual truth: Whether a claim is correct. Matching its words to a source passage does not settle that question. See retrieved-source checks.
fail-closed path: A system execution behavior where a failed security inspection or unavailable inspection service terminates the request and denies access, prioritizing security over availability.
fail-open path: A system execution behavior where a failed security inspection or unavailable inspection service allows the request or transaction to proceed, prioritizing availability over security.
fail-safe defaults: Access begins denied and becomes permitted only when stated conditions hold.
failure cost: A failure cost is the operational consequence assigned to an incorrect, unavailable, or unauthorized result.
fallback: A fallback is an alternate service or manual path used when the primary path fails.
false-positive rate: The fraction of actual negatives incorrectly identified as positive. In the membership-inference example, it is non-members incorrectly called members divided by all evaluated non-members. See the explanation.
fast gradient sign method: FGSM constructs one input perturbation from the sign of the loss gradient, scaled by a stated budget. The model parameters stay fixed. See adversarial input methods.
fault injection: Deliberately making a dependency fail in a test environment to observe the response. Results cover the tested dependencies and operations. Useful availability and intended fail-state behavior require separate measures. See the explanation.
feasibility: Feasibility means a mechanism can work under stated conditions.
FedAvg: See Federated Averaging and the training procedure.
Federated Averaging (FedAvg): A training procedure that repeats client training from a shared model and server averaging of the returned models, with weights specified by the training design. Averaging is one step within that procedure. See the federated training cycle.
federated learning: Federated learning is training in which participants compute local updates and a coordinating process combines them.
FGSM: See fast gradient sign method.
file digest: File-specific name for a cryptographic digest. See file comparison and its limits.
final projection layer: In the language model described here, the learned matrix operation that maps an internal vector to one score, or logit, for each vocabulary token. See partial model recovery.
fine-tuning: Further training from an existing model using selected data and an objective. It can update some or all parameters, including separate parameters used with a fixed base model. Also written fine tuning. See training stages and training attacks.
formal guarantee: A property proved mathematically under specified assumptions. The theorem defines the permitted inputs or distributions and states whether the property is deterministic or concerns an expectation or probability. See aggregation theorem conditions and differential privacy.
framework: A framework organizes outcomes or practices used to structure risk work without specifying one system design.
freshness: In retrieval, whether a source version applies at the relevant time. The newest upload may be a draft or not yet effective. In attestation, whether evidence is within the policy’s permitted age. See retrieved-source checks and attestation.
frontier AI: In the Seoul commitments, frontier AI refers to highly capable general-purpose models or systems that can perform many kinds of tasks and match or exceed the capabilities of the most advanced models.
frontrunning poisoning: Editing material shortly before a predictable collection run so the captured snapshot can retain the poisoned content after the live page is restored. See the explanation.
frozen attack: The selected attack procedure with its budget, stopping rule and grader fixed before evaluation on untouched cases. See the explanation.
fully loaded cost: Fully loaded cost adds assigned infrastructure, labor, and operating cost to direct service cost.
functional agreement: How often a target model and another model produce the same result on a stated input distribution.

G

generalization: Learned behavior that applies usefully to examples not used in parameter updates.
generated code: Generated code is a proposed software change whose effect depends on the repository, build configuration, dependencies, and deployment target.
generative model: A model that produces a sequence or structured object.
GPU kernel: A program that runs on a GPU. Here, kernel-to-kernel reuse of local memory matters to possible disclosure between workloads. See the explanation.
grader: A person or program that applies a scoring rule to judge a test outcome. See the explanation.
gradient obfuscation: A condition in which gradients are misleading or difficult to use without vulnerable inputs necessarily being removed. The term obfuscated gradients refers to the affected gradient information. A failed gradient attack therefore does not establish that the defense removed the vulnerability. See the explanation.
gray-box setting: Partial internal knowledge of a specified target model or system. It does not by itself grant query or modification access. See knowledge and access.
grounded answer: A grounded answer is one whose material claims are supported by the retrieved evidence used for that response.

H

handling test: A handling test submits controlled data and observes where it appears, who can retrieve it, and whether expiry changes that result.
held-out cases: Evaluation cases reserved from training or tuning for the assessment being performed.
held-out evaluation set: Untouched cases used to estimate a frozen attack’s success under the stated case population. In this attack-evaluation setting, separation from attack search matters as well as separation from model training. See the explanation.
held-out test set: An evaluation holdout reserved from training and tuning to assess behavior on unseen cases. Its usefulness still depends on how it was sampled and whether repeated testing influenced model selection. An untouched estimate of a frozen attack’s success also needs cases kept out of attack search. See the attack evaluation split.
help channel: A help channel is a specified route for advice, reporting, or escalation.
hidden size: The number of coordinates in a model’s internal representation vector. In the extraction example, this is the vector supplied to the final projection layer. See partial model recovery.
hosted model service: A hosted model service provides model access through a supplier-operated interface.
host mount: A host mount exposes host storage inside a workload.
human oversight: Human oversight assigns review or intervention to a person with sufficient authority and support.
human oversight assignment: A human oversight assignment identifies the person or role with review and intervention authority.

I

idempotency key: An identifier a supporting service uses to recognize retries of the same intended operation and avoid repeating its effect under its matching and retention rules. The operation key in Chapter 1 serves this role because of the assumed service contract, not because an identifier alone prevents duplicates. See the retry example.
impersonation: In an attack, presenting a false identity to induce acceptance. Success and harm depend on the response. In OAuth token-exchange impersonation, a receiving service treats the actor as the subject within the rights authorized by the token and local policy. This protocol meaning differs from delegation, which retains the actor as a separate party acting for the subject. See token exchange.
incident: An incident is an event that meets the organization’s criteria for response.
index: A structured data store, such as an inverted keyword index or vector database, optimized to locate and retrieve document chunks relevant to an input query.
indicator: An indicator describes a pattern that may support detection or assessment.
indirect prompt injection: A prompt injection attempt delivered through external material an application processes, such as documents, web pages, email or tool results. The material tries to direct behavior, rather than merely supply facts. See instructions in retrieved content.
inference: A process that applies fixed model parameters to an input and produces an output.
information manipulation: The use of false or misleading material to change beliefs, behavior, or decisions.
infrastructure observations: Records of processes, containers, network connections, accelerator use, storage access, scheduler events, control-plane activity and billing. They complement application traces and can have their own missing observations. See the explanation.
ingestion job: An ingestion job turns a source file into index records. It parses text directly or uses optical character recognition, then creates chunks with metadata and writes the result.
ingress: Network traffic, API requests, or data flows arriving from external networks or client systems into a controlled service boundary.
input gradient: The input gradient gives the local rate and direction of loss change for each input feature.
instruction: An instruction is text intended to direct model behavior.
instruction tuning: Training on instructions paired with desired responses. Poisoning those examples can affect learning if they enter training. Merely supplying instructions to a fixed model during inference does not update its parameters. See instruction-tuning poisoning.
inter-agent message: An inter-agent message encodes information, a request, or an instruction sent between agents.

J

jailbreak: A jailbreak is an input designed to bypass a model-level safeguard.
job-submission API: A job-submission API accepts work and parameters for execution.
joint decision review: A joint decision review brings technical and organizational roles to the same proposal and evidence.
JSON Web Token: A format carrying claims as a JSON object, with support for integrity protection or encryption. Its contents do not authorize access without the receiving service’s validation and policy checks. See the explanation.
jurisdiction: A jurisdiction is a legal territory whose rules may apply.

K

key custody: Key custody identifies which party or component can use or release a cryptographic key.
knowledge: An attacker attribute describing the adversary’s information regarding system architecture, model weights, training data, defenses, or runtime parameters.
knowledge poisoning: Knowledge poisoning covers changes to source text, chunks, metadata, or ranking that make manipulated material influence an answer.
known-good baseline: A previously accepted model or component version with supporting test records. Acceptance is limited to the properties and conditions checked.
known-good state: A recorded system state selected for recovery after specified checks. The record identifies the versions, configuration and data needed to restore it. It is not proof of safety against every failure.
Krum: An aggregation rule that selects one complete submitted vector by comparing its summed squared distances to nearby submissions. Its resilience theorem concerns gradient estimates under count, distribution, and noise conditions. See the method and one-dimensional example.

L

L2 distance: The Euclidean length of the whole change vector. An L2 budget \(\varepsilon\) bounds the joint Euclidean length and hence every absolute coordinate by \(\varepsilon\), while an L-infinity budget sets independent coordinate limits and can admit vectors outside the same-radius L2 ball. See the explanation.
large language model (LLM): A generative model that produces text as a sequence of tokens, such as words or pieces of words. See the explanation.
learning rate: A training setting that controls the scale of a parameter update. See training settings.
least privilege: Each user or program receives only the permissions needed for its task.
legal effect: A legal effect changes a person’s legal rights, duties, or status.
legal scope: Legal scope is the set of facts that brings a person, organization, system, or act under a rule.
l-infinity budget: An L-infinity budget limits the absolute change of every input feature.
logical deletion: A software operation that removes standard access routes or pointers to a record while underlying data blocks may temporarily persist on storage media until overwritten.
logit: A numerical model score before conversion to a class or token probability. Plural: logits. See the explanation.
logit bias: An offset added to selected token scores before conversion to probabilities. See the query interface used for partial model recovery.
log-probability: The logarithm of a probability. In the model-extraction example, the probability belongs to a possible next token. See partial model recovery. Plural: log-probabilities.
long-running agent: A long-running agent retains state or authority across multiple tasks or an extended period.
look-alike package: A look-alike package uses a confusing identifier to attract mistaken installation.
loss function: A function returning one scalar that measures how poorly a model meets the stated objective for an input and a reference. Training can use loss to update parameters. An input attack instead changes the input while model parameters stay fixed. See loss and input gradients.

M

management system: A management system is a connected set of policies, roles, processes, and reviews used to meet organizational objectives.
material change: A material change is large enough to invalidate an assumption, test, approval, or duty.
MCP client: The host application’s connector to an MCP server. The host coordinates the use of capabilities. The client carries the protocol exchange. See the explanation.
MCP host: In the Model Context Protocol, the coordinator application that opens sessions and routes tool calls after applying permissions.
MCP server: A service exposing tools, resources or prompt templates through MCP. A downstream business service can enforce separate permissions on the records it stores. See the explanation.
measured effect: An outcome difference assessed against a relevant comparison. See the explanation.
media sanitization: A process that renders data recovery from storage media infeasible for a specified level of effort. Depending on the medium and required assurance, techniques include logical overwrite or erase, cryptographic erase, and physical destruction.
membership inference: Membership inference estimates whether a record was used in training.
memorization: Learning particular training examples or details in a way that can allow the model to reproduce or reveal them later. Generalization concerns useful behavior on new cases. A model can do both.
message: In the illustrated chat interface, content plus a role that identifies its source or intended use. A message may be part of the assembled prompt and does not itself grant business permission. See messages and roles.
model: A learned component whose numeric values and computation turn inputs into scores, labels or generated content. It is one component of an application and the wider operational system. See the nested objects and parameters.
model card: Documentation describing a model’s intended uses, training, evaluation and limitations. The description does not verify the safety of the downloaded package. See the explanation.
Model Context Protocol: A protocol for messages between AI applications and services exposing tools, resources or prompt templates. Protocol discovery does not grant business authorization. See the explanation.
model extraction: Using observations from a target model to recover model information or train another model that imitates its behavior.
model poisoning: Model poisoning changes model weights or a contributed update to create an unwanted behavior.
model replacement: A model replacement attack scales a malicious local update to steer the next combined model.
modification rights: Modification rights are permissions granted by a license or contract to alter and reuse supplied material.
moment: For the gradient estimator discussed here, the expectation of a power of its magnitude. The cited Krum result requires the corresponding honest second, third and fourth moments to be finite. See the explanation.

N

neighboring-data relation: The rule specifying which two datasets differ in one protected unit for a differential-privacy claim. The unit may be one record or one person’s full contribution. The relation must be declared. See the explanation.

O

obfuscated gradients: Gradient information affected by gradient obfuscation. See the adaptive-testing explanation.
observation: Captured data or an event record, such as a returned output, telemetry entry or sensor reading. Its interpretation and a causal explanation are separate claims. Observations may be collected before or after an analyst forms a hypothesis. See hypotheses and event records and evidence classes.
observed incident: An event reported in an operating system or service, rather than created only for a demonstration. The evidence may leave its cause, actor, or extent uncertain.
offline operation: Offline operation continues without a remote dependency.
Open Policy Agent: A policy engine that evaluates structured input against rules written in Rego and returns a decision. The calling service must enforce that decision. See the explanation.
operating budget: An operating budget fixes the resources available for deployment and control work.
operational AI system: The model and application together with the people, policies, records, identities, infrastructure, suppliers, monitoring and response relevant to the decision under review. See system scope.
operational value: Operational value is the measured benefit of the assistant after errors, correction work, review, security controls, privacy effects, and total operating cost are included.
operation key: An identifier for one intended business action. In the assumed contract, the service protects the key while execution is pending and retains the completed result for 24 hours after completion; retries for the same employee, operation and approved fields refer to that operation, and reuse with different fields is rejected. The key alone does not prevent duplicate effects, and protection after expiry is not assumed. See the service contract.
opportunity cost: Opportunity cost is the value of work displaced by the selected investment.
origin: The system or publisher that supplied a source file. It does not alone establish content authenticity or truth. See retrieved-source checks.
outbound dependency: An outbound dependency is an external service required during operation.
outcome measure: An outcome measure observes the result a control is meant to change.
output restriction: Removing or coarsening information returned by an interface. A caller’s permitted logit-bias settings concern input control, a separate condition. See output and input controls.
output sink: A downstream interpreter or operation that gives generated text operational meaning, such as a command shell, query interface, browser, file operation or template engine. A file path is data interpreted by a file operation, not itself an interpreter. See output interpretation.
overdue exception: An overdue exception is an approved deviation whose review or expiry date has passed.
oversight depth: Oversight depth combines the frequency, authority, and technical detail of human review.

P

parameter: An adjustable numeric value used by a model. Weights are coefficients and biases are offsets. Both can be parameters. Model releases may use weights as shorthand for saved learned parameters. See model training.
parameter delta: A vector of changes from a specified starting model to a returned model. Adding it to that starting vector recovers the returned parameters. See parameters, deltas, and gradients.
parameterization: Parameterization keeps data separate from executable syntax through a structured interface.
passage support: Whether the cited passage supports a particular answer claim. Also called citation support here. A false source may support a false claim. Support is separate from factual truth. See retrieved-source checks.
permission scope: The explicit specification of resources, interfaces, and operations that an authenticated human user, service account, or agent identity is authorized to access.
perplexity: A measure based on the probabilities a language model assigns to a sequence of tokens. In the canary-ranking example, lower perplexity places a candidate earlier. The resulting ranking remains tied to the declared candidate space and model. See the explanation.
perturbation constraint: A perturbation constraint defines the changes the attacker may make.
PGD: See projected gradient descent.
pilot limit: A pilot limit is a temporary restriction on users, data, actions, duration, or scale during evaluation.
policy decision point: A component that evaluates a request against rules and returns a decision. The policy enforcement point applies that decision to allow or block the operation. Both roles may be in one service. See the explanation.
policy enforcement point: The component that allows or blocks a protected operation according to a policy decision.
policy friction: Policy friction is delay or work created by a policy during ordinary tasks.
portability: Portability is the ability to move data, configurations, and necessary artifacts to another service.
precision: For a detector, alerts corresponding to real failures divided by all alerts in the evaluated set. This is different from the fraction of real failures detected. See the explanation.
predictive model: A model that maps an input to one or more estimated outputs.
pretraining: For many LLMs, an initial training stage aimed at learning broad patterns from a large data collection. Later adaptation and application integration are separate stages. See training stages.
prevalence: The estimated frequency or proportion with which a specific vulnerability, security failure, or attack occurs across a defined population of systems or time period.
preventive controls: Measures intended to stop a forbidden transfer or operation before it happens. Their effectiveness needs evidence for the actual route. See control responsibilities.
privacy accountant: A calculation of a privacy bound from the chosen mechanism, sampling rule, parameters and releases. It does not by itself verify implementation or protect data disclosed through an unaccounted path. See the explanation.
privilege: A privilege is an allowed operation that exceeds ordinary process access.
processing purpose: The reason for collecting, using, retaining, or sharing specified data.
profiling: Automated use of personal data to evaluate personal aspects of a person, such as economic situation or behaviour. The linked discussion explains its role in EU AI Act classification. See the explanation.
projected gradient descent: An optimization method that takes a gradient step and maps the result back into an allowed set. An untargeted PGD attack uses gradient ascent on the loss, or gradient descent on its negative.
projection: Mapping a proposed point to a nearest point in an allowed set under the chosen distance. A point already in the set stays unchanged.
prompt injection: An attacker’s attempt to redirect a model or connected application through instructions that conflict with its intended task or restrictions. Direct injection supplies those instructions through a prompt or interaction controlled by the caller. Indirect injection places them in other content that the application passes to the model, such as a retrieved document or tool result. Success and any external effect require separate evidence. See attack paths and instruction separation.
prompting: Prompting supplies task instructions with a request.
prompt: Here, the assembled instructions and content for one model call, including a user’s request and selected context. It need not be a single user message. See messages and roles.
prompt record: A stored representation of a user query, application instructions, and surrounding context assembled for one inference request.
propagation: Propagation is movement from the first affected workload to another system.
property inference: Property inference estimates a feature of the training population.
protected reference: A retained copy whose collection and integrity the organization trusts, used as a comparison baseline. Hashing an attacker-writable baseline does not authenticate the original content. See the explanation.
provider: In the operating examples, a supplier of an AI system, model or supporting service. In the EU AI Act edition cited in the role definitions, a provider is a natural or legal person, public authority, agency or other body that develops an AI system or general-purpose AI model, or has one developed, and places it on the market or puts the AI system into service under its own name or trademark, whether for payment or free of charge. The operational supplier label alone does not establish that legal role.
pseudonymization: Pseudonymization replaces direct identifiers while retaining a controlled way to reconnect records.

Q

query budget: The limit on the number of requests an attack may make. See the explanation.
query-only access: At the target model’s interface in the specified input-attack test, the available operations are limited to submitting inputs and observing returned scores or decisions. Record the attacker’s internal knowledge and any offline analysis or substitute-model access separately. Query-only interface operations do not automatically establish a black-box knowledge setting. See the example and the distinction.
query tampering: Unauthorized alteration of a request after the caller submitted it. This differs from malicious content that the caller intentionally submitted. See the explanation.

R

randomized mechanism: A computation whose random choices affect its output. See the explanation.
random restart: Another search run beginning at a newly sampled point within the allowed region. It reduces dependence on one starting point without certifying that the search finds the worst case. See FGSM and PGD.
rate: A rate divides an event count by a relevant exposure or opportunity count.
rate limit: A control that bounds requests within a stated time window. It can limit attack opportunities and resource use, but does not by itself prevent extraction or denial of service.
realistic cases: Evaluation test inputs that incorporate authentic production constraints, noise, formatting variations, and user complexities encountered in real-world deployment.
reauthorization: Reauthorization is a new decision that confirms or changes continued access or action authority.
recall: For a detector, real failures caught divided by all real failures in the evaluated set. Retrieval recall uses a different object: relevant records returned out of all relevant records. See the explanation.
receiving party: The receiving party is the organization or service that obtains the data, including an external model provider.
recovery: Recovery restores a known state and demonstrates that the incident path no longer works.
recovery actions: Actions to contain a problem and restore acceptable service or business state where the process supports restoration. Rebuilding local state does not automatically undo completed external effects. See control responsibilities and incident recovery.
recovery time: Recovery time is elapsed time from a specified disruption point to a specified restored state.
redaction: Removing or masking specified sensitive information from a released copy. Other copies and hidden document content require separate checks.
reference value: A comparison value supplied for appraisal of attestation evidence. It states an expected claim value rather than establishing how the evidence was collected. See the explanation.
refresh token: A token a client uses to obtain new access tokens without repeated user sign-in. Revoking it interrupts further issuance through that token but does not by itself establish that all earlier access tokens are unusable. See the explanation.
refusal behavior: Refusal behavior is the model response pattern that declines a class of requests.
Rego: The policy language used by Open Policy Agent to describe rules for evaluating structured input. See the explanation.
release condition: A release condition is an evidence threshold required before deployment or expansion.
relying party: In remote attestation, the entity that uses a verifier’s result. Its policy determines whether the workload receives confidential data, keys, or access.
remote attestation: An evidence flow in which an attester describes an environment, a verifier evaluates that evidence against policy, and a relying party uses the result for an application decision. Evidence has a freshness limit, and state can change after collection. See the explanation.
renewed assessment: A renewed assessment repeats review after a trigger.
replay set: Cases used for independent reproduction of a discovered bypass. Replay checks repeatability, not the population attack rate. See the explanation.
request association: Connecting a request to its source and to the response that answers it. Identifiers support matching, but delivery also requires a trusted mapping to authorized recipients. See the explanation.
reranker: A model or algorithm that scores and reorders retrieved candidates before the application selects passages for the model. Relevance ranking does not establish truth or permission to disclose.
residual risk: Residual risk is the risk remaining after the selected measures.
resource control: Control over documents or web pages that enter a system through collection or retrieval.
resource owner: A resource owner is the person or organization able to authorize access to a protected resource.
resource quota: A resource quota limits consumable compute, memory, storage, or concurrent work.
responsibility map: A record of which act each party performs, what it may decide and which evidence it must provide. See the explanation.
restoration point: A restoration point is the specific known-good state selected for rebuilding or recovery.
retention period: A retention period specifies how long a copy may remain.
retirement trigger: A retirement trigger is an observed condition that starts removal or replacement review.
retrieval: Selecting external content for the current task and making it available to the application. It changes the context, not model parameters. Relevance does not establish permission to read or send the content. See the explanation.
retrieval-augmented generation (RAG): Combining retrieval of external information with model generation. Architectures differ. The application pattern in this book selects permitted passages for model context and does not reproduce the original jointly trained RAG design. See the explanation.
retrieval boundary: The point where stored source content enters the application’s request context. Source-write permissions, reader access, provider transfer and instruction authority require separate checks around that entry point. See retrieval and permission checks.
retrieval recall: The fraction of all relevant records returned by retrieval under a specified relevance judgment and test collection. See the explanation.
retriever: A search component that queries a knowledge index using keywords, dense vector embeddings, or hybrid methods to return an initial set of candidate text chunks.
return-to-service evidence: Return-to-service evidence consists of checks showing that the specified recovery and acceptance conditions are met.
reuse right: A reuse right permits use of material for a later purpose.
reversal: A reversal is a controlled action that restores an earlier financial state where the rules permit it.
review frequency: Review frequency sets an interval or event rule for reassessment.
revocation: Removing future access under a changed rule. In credential use, it withdraws a grant or credential before expiry. It does not erase every copy or reverse a disclosure that already occurred. See the explanation.
risk acceptance: Risk acceptance records a decision to retain specified residual risk.

robust accuracy: Task accuracy under a specified adversarial evaluation or guarantee. A measured result against selected attacks is not a certificate for every allowed perturbation.

role-appropriate access: Role-appropriate access limits access to what a person’s assigned work requires.
rollback: Rollback returns software or state to an earlier version.
rollback point: A rollback point is a prior accepted state that can be restored.
rollback trigger: A rollback trigger is an observed condition that returns the system to an earlier accepted state.
root of trust: A root of trust is a component accepted as a starting point for measurement or verification.
Rowhammer: A hardware fault mechanism in which repeated accesses to DRAM rows can induce bit flips in nearby rows. A demonstration on one tested platform does not establish that all devices are vulnerable. See the explanation.
runaway computation: Runaway computation continues beyond the intended cost or time.
runtime indicator: A runtime indicator is a measurable condition associated with a tested failure path.
runtime patch: A runtime patch updates execution software to correct a defect.

S

safe alternative: A safe alternative is an approved method offered when the requested AI use is restricted.
safe state: A safe state is a specified physical condition selected to limit harm after a fault or loss of control.
schema: The expected fields, types and structural rules for data or a request. Passing a schema check shows that software can interpret the form, not that the requested operation is authorized. See the explanation.
scope record: A concise record that keeps security analysis tied to one specified system.
score-based attack: An attack that uses returned model scores to choose input changes that advance its objective.
search set: Cases the red team consults while changing its attack strategy. They support attack development, rather than an untouched estimate of the selected attack’s success. See the explanation.
secure aggregation: A family of cryptographic protocols for computing an aggregate while limiting what the specified server or adversary learns about individual inputs under the protocol’s assumptions. The aggregate and colluding parties’ information can still reveal facts about those inputs. It does not validate update correctness. See collusion, dropout, and output limits.
security event: A security event is an observable occurrence relevant to security.
security property: A formal or operational condition, such as confidentiality, integrity, availability, or authorization, that must hold for an asset to remain protected against threats.
security regression: A security regression occurs when a security property becomes worse after a change.
self-hosted model deployment: A self-hosted model deployment runs model software that loads specified weights on infrastructure operated or controlled by the adopting organization.
sender-constrained token: A token used with evidence that the caller holds a particular key. A copied token alone is insufficient only where the receiving service verifies that evidence and the attacker lacks key or signing access. See the explanation.
separation of privilege: The security principle requiring two or more independent conditions or authorizations, such as a user confirmation and a separate service-side policy check, before granting access to a sensitive operation.
service account: A non-human identity used by an application or service. Its granted permissions determine what it can access; least privilege is a design requirement, not an automatic property.
service retirement: Service retirement removes access by revoking identities and disabling connectors. It also ends supplier access and checks known derived stores.
SEV-SNP: AMD Secure Encrypted Virtualization with Secure Nested Paging, a hardware design using VM memory encryption and page-ownership and mapping checks. Processor hardware, security firmware and the guest remain trusted in the described threat model. See the explanation.
shadow AI: AI use outside the organization’s approved review and management process, including unreviewed tools, enabled product features or connected assistants. See the explanation.
shadow model: An attacker-trained model that imitates the target’s training setting using records with known training membership. Its outputs on training and held-out records supply examples labeled member or non-member for a membership attack classifier. See membership inference.
shared case: A shared case is one fixed hypothetical system used across several assignments.
shared responsibility: Shared responsibility divides security duties between customer and provider.
side channel: A side channel exposes information through effects outside the intended data path, such as timing or resource use.
signature: A signature binds a statement or artifact to a signing key under the verification scheme.
smoothness: A bound on how quickly gradients change. In the aggregation analysis, coordinate smoothness constants bound changes in each loss-gradient coordinate as the parameter vector changes. See the explanation.
SOC 2 report: An assurance report examining specified service controls against applicable trust services criteria. Its scope, period, findings and assumed customer controls need to be checked for the deployment being assessed. See supplier assurance.
source authority: Whether a source is responsible for the rule or decision being described. An authentic document from another party need not establish the organization’s policy. See retrieved-source checks.
source record: A source record identifies where a data item came from and the permission asserted for its use.
sparse search: Retrieval using term-based representations. Query terms can affect rank under the selected scoring rule. The score does not authenticate the author or establish that the passage is true. See the explanation.
split-view poisoning: Changing content served by a domain referenced in a dataset, so later downloaders obtain altered bytes while the dataset index stays unchanged. The studied attack used acquired expired domains. See the explanation.
stale permission: A stale permission no longer matches the current task, role, or risk.
statistical watermarking: A method in which a participating generator changes output probabilities to leave a pattern that a matching detector can test. Detection depends on the scheme and available text. An absent signal does not establish human authorship. See generation and detection.
stochastic gradient descent: A method that updates model parameters using error measurements from sampled training examples. A step opposite the parameter gradient aims to reduce loss. This changes parameters, unlike an input-gradient attack on a fixed model. See the explanation.
stopping condition: A stopping condition ends the sequence when state, credentials, policy, or observed results differ from the approved assumptions.
STRIDE: Six software threat categories: spoofing, tampering, repudiation, information disclosure, denial of service and elevation of privilege. They guide questions about components and flows but do not establish local vulnerability. See attack discovery.
strong convexity: A positive lower bound on curvature. It is an additional condition for specified aggregation results, not an assumption satisfied by every model loss. See the explanation.
StruQ: A prompt-injection defense combining filtering of untrusted input, dedicated delimiter tokens and model training to follow trusted instructions. These components work together. Delimiters alone do not supply its demonstrated protection. See the explanation.
sub-exponential tails: A statistical condition limiting how often honest values fall far from their center. The stated trimmed-mean guarantee requires it. Choosing a trimming rule does not establish the condition. See the explanation.
subprocessor: A subprocessor processes data on behalf of another processor in the supplier chain. A service dependency that does not process the data need not have that role.
substantial modification: Under the EU AI Act, a change after market placement or entry into service that was not foreseen or planned in the provider’s initial conformity assessment and affects compliance with high-risk requirements or changes the assessed intended purpose. Role consequences also depend on the Act’s scope and applicable dates. See the explanation.
substitute model: A substitute model approximates the target and can produce inputs tested for transfer.
suggested action: A suggested action tells the analyst what could be done.
supplier due diligence: Supplier due diligence examines the supplier before a relationship begins.
system record: The decision and version, system map, assumptions and evidence used to review one arrangement. A map represents relationships within that record. See system scope and record fields.
systemic risk: In the original EU AI Act definition, harm linked to highly capable general-purpose models that can spread at scale through their many uses. See the definition and its source limits.
system map: A diagram or structured representation of components, actors, stores, trust boundaries and data or action flows. The system record keeps that map with the decision, version, assumptions and evidence. See system scope.
system prompt: Application instructions for model behavior. In the illustrated chat interface they are carried in a system-role message. Formats and role protection depend on implementation. See messages and roles and direct prompt attacks.

T

target: For a labeled training example, the reference output supplied by that example. In next-token training, the following source token supplies the target. This differs from an attacker-selected target output. See the explanation.
targeted poisoning: Targeted poisoning seeks an error for selected inputs or classes.
targeted test: A targeted test measures the specified attacker objective.
target output: The output sought in a task or attack. In a backdoor example, it is the attacker-selected response when the trigger is present.
TEE: See trusted execution environment.
telemetry: Telemetry records operational events such as latency, errors, or resource use.
tenant: Here, a customer organization’s or group’s accounts and associated records. Individual users may have different permissions. Isolation requires checks beyond the tenant label. See operating arrangements.
tenant isolation: The architectural separation of compute, memory, storage, network, and management planes between distinct organizations or tenants sharing physical infrastructure.
termination assistance: Termination assistance is supplier support for an orderly end or transfer.
test harness: Software that runs test cases and collects their results. It records outcome numerators and denominators, costs and relevant intermediate events. Paths it cannot observe remain unassessed. See the explanation.
threat model: An analysis of possible attack paths for a specified system, including assets, attacker goals, access, knowledge and limits. A threat record captures one or more path entries, each with its possible effects, control and test evidence. See scope and assumptions.
threat record: A record of one or more possible attack paths. Each path entry includes attacker access, the affected object, failure and harm, local control and test evidence. It belongs to the system’s wider threat analysis. See scope and assumptions.
threshold secret sharing: A scheme that divides a secret into shares so a specified threshold of shares can reconstruct it while fewer shares conceal it. In the secure-aggregation example, the shared secrets are mask seeds and keys. See mask recovery.
time-limited exception: A time-limited exception is an approved deviation that expires or requires renewed review.
token: In a language model, a unit of input or output represented by a vocabulary identifier. In authorization, a token is a credential; token audience, lifetime and scope below use that second meaning. Plural: tokens. See language-model tokens.
token audience: The service or resource intended to accept an authorization token, as identified by its audience information.
token exchange: A protocol in which a client presents an existing token to an authorization server and requests a new token for a specified use. Issuance depends on validation and policy. Forwarding the old token is a different operation. See the explanation.
tokenizer: Software that converts text into the token identifiers expected by a model, using a matching vocabulary and configuration. See the explanation.
token lifetime: Token lifetime is the period for which a credential can authorize requests.
token scope: The permissions represented by an authorization token under the issuing service’s rules.
tool: A function or service available for model-assisted work. Making it available does not give every caller permission to use every operation or record. See tools and agents.
tool access: Tool access allows the application to call another function or service.
tool call: A structured request identifying a tool operation and arguments. A supporting model can generate the request. Application software checks and invokes permitted operations. See tools and agents.
tool contract: A tool contract specifies an operation’s structured inputs and outputs so that model text can become a request that ordinary software can validate.
training: A process that uses examples, an objective, and an update procedure to change model parameters.
training corpus: A curated collection of text, code, or structured examples used to optimize model parameters during pretraining, fine-tuning, or alignment.
training-data extraction: An attempt to recover training content through model queries. Verification requires comparison with the training corpus or a reliable record of its contents. Matching an auxiliary collection alone does not establish that every matched record was used in training. See memorization and extraction.
transaction authority: Transaction authority is permission to initiate, approve, modify, or reverse a financial action.
transaction limit: A control that sets upper bounds on values, changes, or targets that an automated action can affect in each call.
transfer analysis: A transfer analysis compares what changes when a design enters another setting.
transitive dependency: A transitive dependency is used through another dependency rather than selected directly.
triage: Triage separates ordinary model error from policy violation, hostile-content influence, account or infrastructure compromise, disclosure, and resource abuse.
trigger: A specific pattern, token sequence, or sensory feature embedded in an input that activates a hidden backdoor or trojan condition in a model.
true-positive rate: The fraction of actual positives correctly identified. In the membership-inference example, the denominator is all evaluated members and the numerator is members correctly called members. See the explanation.
trust boundary: A point where data or control crosses principals, operators, privilege levels, tenants, or policy domains that cannot simply trust one another.
trusted execution environment: A hardware-supported execution environment designed to isolate code and data from specified outside software. Its protection depends on the platform, configuration and threat model. Remote attestation provides evidence for a separate trust decision.

U

unit cost: Unit cost divides the total cost recorded for a period by a defined unit of useful work.
untrusted data: In a data integrity review, content whose source, integrity, or safety has not been established. In prompt injection discussions, content that is not authorized to supply application instructions, even when it is authentic and useful evidence. See the distinction between data and instructions.
untrusted workspace: An untrusted workspace contains files and messages needed for a task but not authorized to direct the agent.
unwanted outcome: A concrete event that violates a security property.
uplift: For attacker assistance, an improvement attributable to that assistance relative to a comparable baseline without it. Evidence of use alone does not establish uplift. See the explanation.

V

verifier: In remote attestation, the role that evaluates evidence about an attester under the selected protocol and appraisal policy, then produces an attestation result. It checks evidence authenticity, integrity and applicable freshness conditions, and compares relevant claims with reference values as the policy requires. Checking a digital signature is one method used by some protocols.
virtual machine: A guest operating system running behind a boundary managed by a hypervisor. Its isolation depends on the hypervisor, device access, configuration and patch state. See the explanation.
vulnerable dependency: An intended component whose version contains a weakness that an accessible input, interface or privilege can trigger. See the explanation.

W

weight: In a neural network, a numeric coefficient used in a computation. Releases often use the plural weights as shorthand for saved learned parameters, including values beyond individual coefficients. See parameters and training.
weight shard: A file containing part of a model’s learned parameters. A shard index maps parameter names to the files needed to load the saved model. See the explanation.
white-box access: In this guide’s gradient-attack example, the attacker can inspect model parameters and use the model and chosen loss computation. The setup assumes that the required derivatives of the loss with respect to the input can be calculated. A general knowledge label alone grants neither modification authority nor access to that calculation. See the test setup and knowledge settings.
white-box setting: Full internal knowledge within a specified target model or system. The record states which architecture, parameters, data or other internal fields this includes. The label grants no modification authority. See knowledge and access.
Wilson interval: An approximate confidence interval for a binomial proportion, obtained by inverting the binomial score test. The model assumes a fixed number of independent trials with one common success probability. See uncertainty methods.
workflow diversion: Workflow diversion moves the application away from the user’s intended task.
working memory: The parameters, inputs and intermediate values a workload holds while it runs. Here this is active execution state, distinct from an agent’s stored memory across tasks. See the explanation.
workload identity: A workload identity identifies a non-human process or service.