Module 6 — Accuracy, Robustness, Cybersecurity and the Quality Management System

Module 6 of the AI Act Provider certification: Art. 15 declared performance, resilience and AI-specific attacks — data poisoning, model poisoning, adversarial examples, model evasion — and the Art. 17 quality management system.

Article 15 is short and consequential: it is where a provider states what its system can do, and commits to it. Article 17 is what makes the rest repeatable rather than heroic.

Art. 15 — the three properties

High-risk systems shall be designed and developed so as to achieve an appropriate level of accuracy, robustness and cybersecurity, and to perform consistently in those respects throughout their lifecycle.

Appropriate is relative to the intended purpose. The Regulation sets no numeric floor, which is the only workable choice across the range of Annex III. What it does instead is force declaration: the levels of accuracy and the relevant accuracy metrics shall be declared in the accompanying instructions for use.

That declaration is load-bearing in three directions at once. It is what your deployers monitor against under Art. 26(4). It is what a market surveillance authority tests against. And it is what makes an unstated performance regression a compliance event rather than a product issue.

State the conditions with the number. An accuracy figure without the population, the input distribution and the operating envelope it was measured on is not a declaration; it is a marketing claim that will be tested under conditions you did not intend.

Robustness

Systems shall be as resilient as possible regarding errors, faults or inconsistencies that may occur within the system or the environment in which it operates, in particular due to interaction with natural persons or other systems. Technical and organisational measures may include backup or fail-safe plans.

Two things worth noting. "Interaction with natural persons" means the messy inputs real users produce, not the clean inputs your test harness produces. And fail-safe is named: a system that degrades into confident nonsense under out-of-distribution input is not robust in the sense the article means, however good its headline accuracy.

Art. 15(4) also addresses systems that continue to learn after being placed on the market or put into service: they shall be developed to eliminate or reduce as far as possible the risk of possibly biased outputs influencing input for future operations (feedback loops), and to ensure such loops are duly addressed with appropriate mitigation measures. This is the same concern Art. 10(2) raises for training data, seen at runtime.

Cybersecurity, and why generic controls are not enough

Systems shall be resilient against attempts by unauthorised third parties to alter their use, outputs or performance by exploiting vulnerabilities.

Art. 15(5) then names the AI-specific attack surface, and this is the paragraph that separates an AI Act security position from an ordinary one. The technical solutions shall be appropriate to the relevant circumstances and the risks, and shall include, as appropriate, measures to prevent, detect, respond to, resolve and control for:

• data poisoning — manipulating the training dataset;
• model poisoning — manipulating pre-trained components used in training;
• model evasion — inputs designed to cause the model to make a mistake;
• confidentiality attacks — extracting information about the model or its training data;
• model flaws — defects in the model itself.

A hardened CI/CD pipeline, SSO and network segmentation address none of these. They are necessary and they are not sufficient, and the file needs to show that the AI-specific surface was assessed separately.

Practically: provenance and integrity controls on training data; verification of third-party pre-trained components and their supply chain; adversarial testing as part of the Art. 9 test plan; rate limiting and query monitoring against extraction; and monitoring for distribution shift that can indicate an attack rather than drift.

Art. 17 — the quality management system

Providers of high-risk systems shall put in place a quality management system ensuring compliance with the Regulation, documented in a systematic and orderly manner in the form of written policies, procedures and instructions. It shall include at least:

• a strategy for regulatory compliance, including compliance with conformity assessment procedures and procedures for the management of modifications;
• techniques, procedures and systematic actions for the design, design control and design verification;
• techniques, procedures and systematic actions for development, quality control and quality assurance;
• examination, test and validation procedures to be carried out before, during and after development, and their frequency;
• technical specifications, including standards, to be applied and, where relevant harmonised standards are not applied in full, the means to ensure compliance;
• systems and procedures for data management, including acquisition, collection, analysis, labelling, storage, filtration, mining, aggregation, retention and any other operation regarding data performed before and for the purpose of placing systems on the market;
• the risk management system of Art. 9;
• the setting-up, implementation and maintenance of a post-market monitoring system per Art. 72;
• procedures related to the reporting of serious incidents per Art. 73;
• the handling of communication with competent authorities, notified bodies, other operators, customers or other interested parties;
• systems and procedures for record keeping of all relevant documentation and information;
• resource management, including security-of-supply related measures;
• an accountability framework setting out the responsibilities of management and other staff.

The list reads as a table of contents for everything in this track, which is the point: Art. 17 is the container that keeps Art. 9 to Art. 15 running when the people who built the system move on.

Art. 17(2) allows implementation proportionate to the size of the provider's organisation, while preserving the degree of rigour and level of protection required. And providers already subject to quality management obligations under sectoral Union law may integrate these elements into those existing systems — the same permission Art. 9(10) gives for risk management.

What a reviewer looks for first

In practice, three checks separate a real QMS from a documented one:

The modification procedure. Art. 17's first element names the management of modifications. Show a change that went through it, and what the assessment concluded about substantial modification.

The test procedures and their frequency. A stated cadence, and evidence it was followed.

The accountability framework. Named roles, with authority, not a RACI diagram nobody uses.

Check yourself

1. We report 94% accuracy. — Under what conditions, on which population, against which metric? Art. 15(3) requires the levels and the metrics declared in the instructions, and the declaration is what you are held to.
2. Our security programme is ISO 27001 certified, so Art. 15(5) is covered. — It is not. Data poisoning, model poisoning, model evasion and confidentiality attacks are AI-specific and need to be assessed and addressed as such.
3. Our model retrains continuously on production data. — Art. 15(4) requires the feedback-loop risk to be eliminated or reduced as far as possible and duly addressed, and Art. 13 requires predetermined changes to be declared at the initial conformity assessment.
4. Do we need a new management system for Art. 17? — No, if an existing one genuinely hosts the listed elements. What is assessed is the presence and documentation of the elements, not the certificate.

Previous: Module 5 — Logging, instructions and oversight by design Next: Module 7 — Conformity, declaration, marking, registration →