What types of AI systems require a conformity assessment under the EU AI Act?

High-risk AI systems require conformity assessments before deployment. These include AI used in critical infrastructure management, employment and worker management, access to essential services, law enforcement, border control, and administration of justice. The first enforcement actions confirm that recruitment and CV screening AI falls in this category regardless of deployment scale.

How does the EU AI Act's adversarial robustness requirement apply in practice?

For AI systems with 'systemic risk' designation (large foundation model providers above 10^25 FLOPs), Article 55 requires adversarial robustness testing including red team exercises before deployment, ongoing production monitoring for adversarial exploitation, and 72-hour incident reporting for detected adversarial attacks. This creates a direct regulatory mandate for AI red teaming.

EU AI Act: First Enforcement Actions and Security Implications

Q: What are the incident reporting obligations for AI providers under the EU AI Act?

Providers and deployers of high-risk AI systems must report serious incidents (defined as incidents causing or likely to cause death, serious harm, or significant disruption to critical infrastructure) to national supervisory authorities within 72 hours, mirroring the urgency of GDPR breach notification timelines.

The European AI Office has issued its first formal enforcement notices under the EU AI Act. Three cases. All instructive.

The notices target organisations operating AI systems assessed as using prohibited practices or deploying high-risk systems without completing conformity assessments. The enforcement actions resolve several previously ambiguous areas of the regulation that compliance teams have been circling for months. What follows covers those notices and the EU AI Act’s published requirements as documented in official regulatory guidance.

What the Regulation Actually Requires

The Act establishes a risk-based classification with obligations that scale with risk level. The categories:

Prohibited practices (banned outright):

Real-time biometric identification in public spaces (with narrow, defined exceptions)
Social scoring systems
Subliminal manipulation techniques
Exploitative targeting of vulnerable groups

High-risk AI systems (conformity assessment required before deployment):

Critical infrastructure management
Employment and worker management systems
Access to essential services
Law enforcement and border control applications
Administration of justice

General-purpose AI (GPAI) models:

All GPAI models above 10²³ FLOPs training compute must comply with transparency and copyright obligations
Models with “systemic risk” designation (above 10²⁵ FLOPs) face additional adversarial robustness testing requirements

The First Three Cases

Case 1: Biometric Categorisation in Retail

A European retail chain operating an AI system that inferred customer demographic categories from CCTV footage (to personalise in-store promotions) received a prohibition notice. The system was assessed as performing “biometric categorisation” under Article 5. Cease operation within 30 days.

The case establishes something important: demographic inference from physical characteristics, even when used for commercial purposes rather than law enforcement, falls within the biometric categorisation prohibition. Any retailer still running similar systems should treat this notice as directly relevant.

Case 2: Recruitment AI Without Conformity Assessment

A staffing agency operating an AI-assisted CV screening tool received notice that the system constituted a high-risk AI application under Annex III and required a conformity assessment that had not been completed. System suspended pending assessment.

This is significant for any organisation using AI in hiring, performance assessment, or workforce management, all classified as high-risk regardless of scale. The “we’re too small for this to apply to us” assumption doesn’t hold.

Case 3: GPAI Provider Transparency Notice

A European provider of a general-purpose AI API received a formal request for documentation demonstrating compliance with Article 53 transparency requirements: training data provenance documentation and copyright compliance procedures. First formal transparency disclosure request under the GPAI provisions.

This one is early in the process. But it signals that the AI Office is actively working through the GPAI provider population rather than waiting for complaints.

What Security Teams Need to Track

Adversarial Robustness as a Compliance Requirement

For organisations operating AI systems with systemic risk designation, Article 55 requires adversarial robustness testing. The AI Office guidance specifies this includes red team exercises before deployment, ongoing monitoring for adversarial exploitation in production, and incident reporting within 72 hours of detected adversarial attacks.

This is a direct regulatory mandate for AI red teaming. If your security programme hasn’t yet included AI systems in scope for adversarial testing, the regulation has now made that a compliance question as well as a security one.

Technical Documentation: The Gap Most Teams Have

High-risk AI systems must maintain technical documentation under Article 11 covering system architecture and training methodology, training and test datasets including characteristics and preprocessing, cybersecurity measures implemented, and logging and monitoring capabilities.

Security teams should verify that this documentation is current and accurately reflects the deployed system. Early supervisory reviews have flagged discrepancies between documentation and actual implementation. Auditors will find those discrepancies faster than you’d expect.

The 72-Hour Clock

The Act establishes incident reporting obligations for providers and deployers of high-risk AI systems. Serious incidents (defined as incidents causing or likely to cause death, serious harm, or significant disruption to critical infrastructure) must be reported to national supervisory authorities within 72 hours. Same urgency as GDPR breach notification.

Organisations without AI-specific incident response procedures should build them now, not during a live incident.

If You’ve Already Implemented NIST AI RMF

The AI Office has published a crosswalk between the EU AI Act and NIST AI RMF 2.0. The GOVERN, MAP, MEASURE, and MANAGE functions map broadly to the Act’s conformity assessment and post-market monitoring requirements. You’re not starting from scratch.

The gap is primarily in the sector-specific and consequence-focused additions: the physical safety implications of AI failures that the RMF’s technology-agnostic framing doesn’t emphasise. That delta is tractable for organisations that have already done serious RMF work.

Full enforcement of all provisions is phased through 2027. But the first enforcement actions confirm the AI Office is actively pursuing non-compliance during the transition period. Waiting for the deadlines to pass before taking action is a reasonable bet only if your systems aren’t already in scope, and based on Cases 1 and 2, scope is being interpreted broadly.