Researchers from HKUST and NUS propose a tagging-and-filtering defense that asks LLMs to cite which instruction they are answering, cutting prompt injection success rates from over 80% to under 1% without any fine-tuning.
Hugging Face disclosed an intrusion carried out end-to-end by an autonomous AI agent — and the incident exposed a troubling asymmetry: the attacker operated freely while defenders were blocked by safety guardrails on commercial AI models.
Design patterns for a prompt injection and jailbreak detection layer: rule-based filters, semantic classifiers, canary tokens, and output validation for production LLMs.