30 Experts Map the Unsolved Problems in Agentic AI Security

Thirty leading experts in AI security and privacy spent three days in March 2026 mapping the problems they believe the field has not yet learned to solve. The output, published to arXiv in July as “Security and Privacy in Agentic AI: Grand Challenges and Future Directions” (2607.06608), identifies four structural failure modes in how organizations currently approach the security and privacy of agentic AI systems. None of them are fixable by deploying another tool.

The paper is a horizon-scanning exercise rather than a technical vulnerability report. It draws on perspectives from academia, industry, and government, conducted across three days of structured workshops at the Universitat Politècnica de València. The organizing claim is direct: the foundational concepts that security and privacy frameworks rely on were developed for bounded, deterministic, attributable systems. Agentic AI violates those assumptions at every layer.

The Accountability Gap

The first theme the paper identifies is responsibility distribution. When an agentic system causes harm, identifying who is accountable requires working through a supply chain that typically includes at minimum: the foundation model provider, the orchestration layer, the retrieval system, each integrated third-party tool, and the deploying organization. These parties frequently have no contractual relationship with each other.

This isn’t a novel observation. What the paper adds is the structural argument that fragmented supply chains make compliance documentation inconsistent by design, not by negligence. Traceability mechanisms, the paper notes, may themselves violate privacy: building audit logs detailed enough to attribute responsibility for agent actions requires recording user interactions and data access patterns in ways that create new privacy exposures.

The EU AI Act and GDPR provide starting points for reasoning about accountability, but the paper’s experts found that neither maps cleanly onto distributed multi-component architectures where no single provider controls or even observes the full action chain. The gap between regulatory intent and operational reality is not a matter of enforcement: it’s a conceptual mismatch.

For security teams doing vendor due diligence on agentic AI systems, this creates a practical problem. Standard procurement questions (“who is responsible if this system causes harm,” “who holds the data”) don’t have clean answers in architectures where the system’s behavior emerges from the interaction of components built and operated by different entities.

Cascading Consent Failure

The second theme addresses consent. Traditional privacy frameworks assume that consent is granted by a human, for a defined purpose, to a bounded recipient. Agentic AI breaks all three assumptions simultaneously.

When a user authorizes a task-planning agent to complete a work objective, that single authorization can spawn dozens of service calls, memory writes, API requests, and handoffs to sub-agents, each of which introduces new data flows the user had no specific visibility into at the moment of consent. The paper describes this as cascading consent failure: the original authorization propagates indefinitely through the agent’s action chain, applied to situations and services that were not contemplated when the user clicked “approve.”

Revocation is equally broken. In conventional systems, a user can revoke consent and data processing stops. In agentic architectures, data collected during earlier authorized interactions may have been passed to external services, written into shared memory stores, or incorporated into retrieval systems that inform future agent behavior. The consent has already traveled further than the user can reach.

Memory accumulation is the specific mechanism the paper highlights. Persistent memory, the feature that makes agentic systems useful across sessions, also means that data collection continues to diverge from the purposes that justified the original authorization. A system permitted to remember task preferences gradually accumulates behavioral signals that were never the subject of any specific consent.

This theme connects directly to the documented attacks this year against agentic memory systems. The memghost attack class, which allows injected instructions to persist across sessions and affect future agent behavior, works precisely because memory accumulation is opaque to users and difficult for operators to audit.

Human Oversight Under Pressure

The paper’s third theme is the erosion of meaningful human oversight, and the specific irony it highlights is worth quoting: reduced oversight is not an accidental side effect of deploying agents. It is the design goal. Agents are useful precisely because they reduce the number of decisions humans need to make. The oversight that’s removed is often the oversight that would catch problems.

The experts identify two specific failure modes. First, AI agents lack reliable mechanisms for recognizing when they have hit the limits of their competence or when a situation warrants human escalation. Current systems don’t know what they don’t know, and the high-stakes situations where that failure matters most are exactly the situations where agents are most likely to proceed confidently.

Second, personalization amplifies both the utility and the risk. An agent that adapts to a specific user’s preferences, communication style, and history becomes more effective at its intended tasks. It also becomes more effective at manipulation, whether through deliberate adversarial action or through emergent optimization pressure. The paper notes that this creates a bidirectional risk: agents can be manipulated through personalization vectors, and personalized agents are more capable of inadvertently manipulating the users they serve.

For security teams, the practical implication is that human-in-the-loop controls are not a reliable mitigation when the “loop” consists of approving agent actions faster than they can be meaningfully reviewed. Oversight at machine speed is not oversight.

Certification Without Validity

The fourth theme challenges the assumption that AI systems can be certified, audited, and approved in ways that provide durable assurance. Standard software security certification involves testing a system at a point in time, assessing it against defined criteria, and issuing a finding that holds until the next review cycle. This model assumes the system remains sufficiently similar to what was tested.

Agentic AI systems modify their effective behavior through interaction, accumulate memory that alters downstream outputs, and are non-deterministic in ways that make identical inputs produce different results. A system that passes assessment on Monday may behave differently by Friday because of what it learned in the interim. Point-in-time certification does not transfer to systems that update themselves through use.

The paper argues that continuous monitoring and behavioral baselining are the operational successors to one-time certification for agentic systems. This is consistent with what security practitioners in heavily deployed agentic environments are building, but it raises cost and complexity questions that don’t have obvious answers at enterprise scale.

What This Means in Practice

The paper is not prescriptive about solutions. Its explicit goal is horizon-scanning: naming the problems clearly enough that research and policy can address them. But the framing carries implications for practitioners.

Deploying agentic AI while waiting for frameworks to catch up is the current situation for most organizations. The paper’s contribution is identifying that the gaps are structural, not incidental. Better documentation practices won’t fix cascading consent. Stronger vendor contracts won’t resolve distributed accountability when no single vendor controls the full system. More human review steps won’t restore meaningful oversight when agents operate faster than review is practical.

The four themes point toward the same kind of architectural response: observability at the action level, not just the output level; behavioral baselines that can detect drift; consent models that track authorization through action chains rather than assuming the initial authorization persists; and liability frameworks that acknowledge distributed architectures explicitly rather than mapping distributed systems onto single-entity models.

None of that work is finished. The paper’s value is making clear why the existing frameworks won’t finish it for us.

References

• arXiv 2607.06608 — Security and Privacy in Agentic AI: Grand Challenges and Future Directions