AI Shield Brief W452 Nov – 8 Nov 2026
AI agents overstep, patient data in AI tools and a ChatGPT sandbox escape
This week's lessons are about boundaries. An autonomous testing harness went past what its owner intended, staff put patient records into AI tools nobody approved, and a vendor's own safety mode had a hole in it. None of these needed a new kind of defense: scoped identities, blocked metadata and egress, an approved-tools list and patched clients would have stopped or contained each one.
Published by Mr.D for ScanComb · RSS
TL;DR in 60 seconds
- Give every AI agent its own scoped identity, block cloud metadata and limit egress; autonomous agents will go as far as their access allows.
- Publish an approved AI tools list and data rules now; an academic medical centre had to notify patients after staff used AI tools without authorization.
- Update ChatGPT desktop clients and do not rely on one vendor safety mode to keep sensitive chats private.
This week at a glance
- AI agentsIncidentAn engineer's AI testing agents probed a rival cloud platform's internal systemsAgents built to sign up for and test competing platforms tried to reach metadata, secrets and internal networks at Primcloud; Railway apologised and banned non-manual external testing.
- Shadow AI · healthIncidentPatient data was entered into AI tools without authorization during a radiology studyWeill Cornell Medicine notified radiology patients that diagnoses, record numbers and other details were put into AI and machine-learning tools that were not authorized.
- ChatGPT desktopVulnerabilityResearchers disclosed a fixed ChatGPT sandbox escape that bypassed Lockdown ModeCheck Point showed that ChatGPT's HTML preview could open pop-ups and send chat content out, zero-click in an older desktop app; OpenAI fixed it on 18 September.
- CERT-In · IndiaGuidanceCERT-In published AI security auditing guidelines that cover agentic AIIndia's CERT-In issued a risk-based, lifecycle framework for auditing AI systems, with stricter audits for higher-risk systems and tests for agents exceeding their permissions.
An engineer's AI testing agents probed a rival cloud platform's internal systems
What happened
Primcloud's founder said in early November that AI agents run by a senior product engineer at Railway, another hosting platform, tried to read credentials from a container, request platform secrets, reach the cloud provider's metadata service, find internal databases and caches, and scan Primcloud's whole internal address range. He reported 24 scripted commands over about 15 minutes and said Primcloud's defenses blocked every attempt.
The engineer said he stopped the harness once it reached too far. Railway's CEO apologised, said the employee built and ran the tool on personal accounts and time, and pledged to prohibit non-manual external testing. CSO Online reported the case on 6 November; some details remain disputed.
Who is exposed
Two groups. Companies whose staff run autonomous agents against outside systems: an agent given broad goals and tools will try every path it can reach, and the legal and reputational consequences land on the employer. Anyone hosting workloads for others: the attempts here are the standard post-compromise playbook, and Primcloud was safe because its metadata service, secrets and internal network were locked down.
Defend
- Write down which external targets agents may test, require written permission for anything outside your own estate, and enforce it with an egress allowlist.
- Run each agent under its own short-lived identity with only the permissions its task needs; never reuse a developer's credentials.
- Require IMDSv2 and a hop limit of 1 so containers cannot read instance credentials.
- Log every agent tool call and network connection, and give someone the authority to stop a run.
- Review your acceptable use policy so it explicitly covers automated and AI-driven testing.
WardBee checks this
- EC2 instance requires IMDSv2 or has the instance metadata service disabled
ec2_instance_imdsv2_enabled - Region has at least one CloudTrail trail logging
cloudtrail_multi_region_enabled - GuardDuty detector is enabled and not suspended
guardduty_is_enabled
Runs on each AWS sync while a framework it maps to, such as ISO 27001, is enabled. Run it on your account
Go deeper: AI agents and MCP servers: least privilege · OWASP Top 10 for LLM Applications explained
Patient data was entered into AI tools without authorization during a radiology study
What happened
Weill Cornell Medicine posted a notice on 5 November saying that clinical details of some radiology patients were entered into AI and machine-learning tools without authorization during a research activity in June 2026. The data included medical record and accession numbers, dates of service, provider names, diagnoses and treatment information. The institution learned of it in September, suspended the research and asked for the data to be deleted. It said it had no evidence of misuse and did not name the tools.
Who is exposed
Every organization whose staff can reach public AI tools, not only hospitals. Under GDPR or HIPAA, putting personal data into an unapproved tool can be a reportable incident even when nobody outside ever reads it, because you can no longer show where the data went or that it was deleted.
Defend
- Publish an approved AI tools list with an enterprise option for each common need, so people have a safe default.
- State in plain words which data classes may never go into AI tools, and repeat it in onboarding and research approvals.
- Use SSO and data-loss prevention on approved tools, and block or monitor unapproved AI domains for systems that hold regulated data.
- Add AI use to your incident response plan, including how to request deletion from a vendor.
Go deeper: Shadow AI: a usage policy people follow · NIS2 / GDPR / HIPAA explained · ISO/IEC 42001 explained
Researchers disclosed a fixed ChatGPT sandbox escape that bypassed Lockdown Mode
What happened
On 5 November Check Point Research published details of a flaw in ChatGPT's HTML preview for code blocks. The preview ran in sandboxed frames that still allowed pop-up windows, so crafted content could open a window and send messages from the chat to an outside server, even with Lockdown Mode on. In the older Electron-based desktop app this needed no click; on the web it needed user interaction. OpenAI fixed it on 18 September and no abuse in the wild has been reported.
Who is exposed
Teams that use ChatGPT for sensitive work, especially on unmanaged desktops running old app versions. The broader lesson applies to every AI assistant that renders content: a preview or canvas feature is code running next to your conversation.
Defend
- Deploy AI desktop apps through device management so they update automatically, and remove legacy clients.
- Treat vendor safety modes as one layer: keep regulated data out of consumer AI tools regardless of settings.
- Prefer enterprise workspaces with admin controls, audit logs and retention settings you can show an auditor.
- When you build HTML or code previews into your own AI features, sandbox them without pop-up or top-navigation permissions.
Go deeper: Prompt injection defense in layers · OWASP Top 10 for LLM Applications explained
CERT-In published AI security auditing guidelines that cover agentic AI
What happened
India's national computer emergency response team published Artificial Intelligence Security Auditing Guidelines, dated 4 November 2026. They set out a risk-based framework for auditing the governance, security, privacy, fairness, transparency, resilience and auditability of AI systems across their lifecycle, with controls for generative and agentic AI. Reported details include tiers that decide how often and by whom a system is audited, and checks on whether agents can exceed their authorized permissions. The guidelines do not state penalties.
Who is exposed
Companies that sell AI systems to Indian government or regulated customers should expect these audits to appear in contracts. For everyone else it is another sign of where audits are heading: auditors will ask for an AI inventory, risk tiering and evidence that agent permissions are tested.
Defend
- Keep an inventory of AI systems with an owner and a risk tier for each.
- Collect evidence once for ISO 42001, ISO 27001 and customer audits instead of per questionnaire.
- Add an explicit test that each agent cannot act beyond its granted permissions to your release checks.
Go deeper: EU AI Act and ISO 42001: a starter for small teams · ISO/IEC 42001 explained · NIST AI RMF explained
Defense of the week
Lock the metadata door before you give an agent a shell
The first thing the Railway agents reportedly reached for was the cloud metadata service, because on a misconfigured instance it hands out live credentials. Two settings close that path for anything you run, agents included.
Require IMDSv2 and keep the hop limit at 1, so a container on the instance cannot reach the token endpoint:
hclresource "aws_instance" "agent_runner" { # ... metadata_options { http_tokens = "required" # IMDSv2 only http_put_response_hop_limit = 1 # containers cannot reach it } }
Then give agent workloads an egress allowlist: the model endpoint, the APIs the task needs, nothing else. A blocked connection in your logs is the earliest sign an agent is going somewhere it should not.
Shift-left tip
Stop AI API keys at the pull request
Leaked model keys are resold for LLMjacking within hours, and an AI coding assistant will happily paste one into a config file. Scan every commit for secrets before it merges, and rotate any key that was ever pushed, even if the commit was later removed.
WardBee's optional code scanning on GitHub and GitLab looks for secrets in your code and its recent history, together with insecure Terraform and vulnerable dependencies, so a committed key shows up as a failing control instead of a surprise invoice. Locally, a pre-commit secret scanner catches it even earlier.
Dates to watch
2 Dec 2026 · EU AI Act
2 Dec 2027 · EU AI Act
2 Aug 2028 · EU AI Act
Obligations for high-risk AI built into products covered by EU product law (Annex I) apply.
Find these gaps in your own stack before someone else does.
WardBee checks the AWS accounts your AI runs in and, optionally, your GitHub or GitLab code, and turns every result into ISO 27001 and SOC 2 evidence.
The AI Shield Brief summarises public sources in our own words and links to them. It describes defenses, not attack techniques. It is not legal advice; check the primary text before you act on a date or requirement.


