How a B2B SaaS Team Contained Prompt Injection Across an AI Agent With Live Tool Access
An AI agent with database and API tool access was hijacked via prompt injection in user-supplied data, exfiltrating records. The team rebuilt the agent with layered defences and cut critical incidents to zero.
Security · advanced · 12 min read · 2286 words · By Sandip Das · Published
Composite scenario: built from common industry patterns; the figures are illustrative.
What this covers
- The Situation
- The Problem
- Constraints
- Options Considered
- The Solution
- Implementation
- Results
- What Went Wrong Along the Way
- Lessons Learned
- Apply It Yourself
Key numbers
| Metric | Before | After |
|---|---|---|
| Critical prompt-injection incidents | 1 confirmed (14 exfil queries in 72 hrs) | 0 in 8 weeks post-deploy |
| Mean time to detect anomalous tool use | ~72 hours (manual log review) | 87 seconds (automated alert) |
| Sensitive columns reachable by agent | 12 columns unrestricted | 0 (column-level REVOKE enforced) |
| Rows accessible per agent query | Unlimited | Capped at 50 by OPA policy |
| SOC 2 audit findings on agent controls | Not assessed | 0 findings |
Key components
- Prompt Guard (DistilBERT classifier) — Scans every tool observation before it re-enters the LLM context, blocking strings with injection probability above 0.72.
- OPA Policy Engine — Enforces per-tenant, per-tool access rules — including column deny-lists and row-count caps — as an auditable sidecar before any tool is invoked.
- Tool Router — Centralised dispatch layer that calls OPA for every tool invocation and raises a PermissionError if the policy denies the request.
- Column-level PostgreSQL REVOKE — Removes the agent's database role's SELECT privilege on sensitive columns at the database layer, providing a hard stop independent of application logic.
- Output Filter (Presidio) — Final gate that scans the LLM's raw response for PII and secret patterns before the text is returned to the user.
- OpenSearch Audit Log with Alerting — Streams every tool invocation and orchestrator decision; fires a PagerDuty alert within 90 seconds when anomalous tool-call volume is detected.
The Situation
This is a composite scenario assembled from patterns observed across multiple production AI agent deployments; all organisations, names, and figures are illustrative and do not represent any single real company.
The organisation is a 120-engineer B2B SaaS company selling a revenue-intelligence platform to mid-market sales teams. Their flagship product ingests CRM data, call transcripts, and deal notes, then surfaces recommendations through a chat interface. In late 2024 the product team shipped an AI assistant — internally called Aria — backed by GPT-4o via the OpenAI API, orchestrated with LangChain, and given three live tools: a read/write PostgreSQL query tool, a Salesforce CRM REST tool, and an S3 file-read tool. Aria could answer questions like "Which deals in EMEA are at risk this quarter?" by running real queries against the production database.
The team was eight engineers: three backend, two ML/LLM, one security, one SRE, and a product manager. The stack was Python 3.12, FastAPI, LangChain 0.2, PostgreSQL 15 on RDS, S3, and EKS on AWS. Monthly active users at launch: roughly 4,200 across 310 enterprise accounts.
The rest of this case study — the trade-offs, the scaling numbers and the build prompt — is available to LearnXops members. Create an account or see membership.
Topics: security, ai agents, prompt injection, llmops, zero trust, mlops
Related case studies
- AWS Keys Leaked to GitHub: The First 60 Minutes and the Clean-Up
- How a 120-Engineer SaaS Company Replaced Its Corporate VPN with Zero-Trust Access
- How a B2B SaaS Team Cut Supply-Chain Risk by Enforcing SBOMs, Sigstore Signing, and OPA Admission Policies
- How a B2B SaaS Team Eliminated Plaintext Secrets and Cut Credential Exposure Risk by 94%