AI moves fast. Stay in the know.

Prompt injection has become OWASP's top-ranked risk category for GenAI applications as adoption in state and territorial government environments reached 82% of employees using AI in daily work, up from 53% the prior year, according to a 2025 NASCIO survey of 51 CIOs. A Center for Internet Security (CIS) report identifies a fundamental architectural weakness: language models cannot separate instructions from other data, processing embedded malicious instructions in external content in the same way as normal requests.

Source: Help Net Security

What to know:

• LLMs process input without distinguishing between instructions and data, enabling direct prompt injection through model interaction and indirect injection via malicious instructions embedded in web pages, emails, or documents that AI systems later retrieve and process.
• GenAI tools with privileged access to systems and data can be manipulated to poison agentic databases across user sessions, contaminate external datastores like cloud storage and email inboxes, and potentially execute code on behalf of attackers.
• An Amazon Q extension update for Visual Studio Code in July 2025 inadvertently introduced a prompt that could instruct the AI agent to delete files and terminate servers; AWS patched within two days and issued a security bulletin.
• The Morris II worm demonstrated propagation patterns by embedding malicious prompts in emails that entered RAG databases through AI email assistants, which then generated additional emails containing similar payloads along with sensitive information.
• Research traces prompt injection vulnerabilities back to 2013, with studies indicating that targeted training improves model handling but does not provide sufficient protection against attacks that exploit fundamental limitations in how language models process input.

Why it matters:

Prompt injection represents a detection and monitoring challenge for organizations embedding GenAI across operational workflows. AI systems with privileged access to enterprise data create exposure through attack vectors that differ from conventional threats, requiring visibility into AI behavior patterns, inventory management of AI system permissions, least privilege enforcement, and continuous monitoring to identify anomalous activity. Understanding which systems AI can reach and what data it processes becomes essential for risk assessment and early detection of potential security incidents.

Microsoft's Copilot Terms of Use, updated in the last quarter of 2025, state the AI is "for entertainment purposes only" and include the warning: "It can make mistakes, and it may not work as intended. Don't rely on Copilot for important advice. Use Copilot at your own risk." The disclaimer gained attention in early April 2026 as Microsoft markets Copilot as a productivity tool and integrated throughout Word, Excel, Outlook, and Teams. While Microsoft describes the language as "legacy" from Bing Chat origins and plans updates, concerns about AI reliability and output verification remain central challenges for enterprise deployment.

Source: TechCrunch

What to know:

• Microsoft's Terms of Use make no warranty about Copilot's reliability, note outputs may involve copyright/trademark/privacy considerations, and indicate users are responsible for content they choose to share, while organizations integrate the tool into daily business workflows.
• The "entertainment purposes only" language applies to individual consumer use of Copilot, not Microsoft 365 Copilot for enterprise customers, though enterprise versions operate with similar technical limitations without this specific disclaimer in their terms.

Why it matters:

The disconnect between Microsoft's legal disclaimers and product positioning highlights a broader challenge for organizations deploying AI tools: vendors may limit liability through terms of service while businesses integrate these systems into operations involving sensitive data and decision-making. For mid-sized enterprises, user feedback about reliability and accuracy fluctuations suggests the importance of implementing verification processes and governance frameworks. Organizations should consider treating AI systems as tools requiring output validation, appropriate access controls, and ongoing monitoring rather than as fully autonomous decision-makers, ensuring AI deployment aligns with actual capability levels and organizational risk tolerance.