Meta AI. OpenAI. Anthropic. They're not just hiring prompt engineers anymore — they're hunting for AI Evaluation Engineers: people who can find where AI breaks before attackers do. It's one of the fastest-growing, highest-paying roles in tech, and almost nobody knows how to get there yet.

This 2-hour hands-on workshop is your unfair head start.

You'll go behind the curtain of how production AI assistants are actually built — spoiler: it's one model plus a paragraph of rules — and learn exactly how those rules fail. Working against two real-world bots (a healthcare triage assistant and a brokerage chatbot), you'll run live attacks across six frontier models from Meta, OpenAI, and beyond, watching the same technique succeed or fail in real time.

What you'll actually do:

• Jailbreak AI chatbots using role-play overrides, authority framing, encoded injections, and prompt extraction — then see why some guardrails hold and others crumble
• Run automated red-team evals using the same open-source tooling (promptfoo) used by professional AI safety teams
• Learn why "technique transfers" — master one attack pattern and it works across GPT, Claude, Llama, and whatever ships next year
• Understand why cost and latency are safety metrics, not just engineering concerns

No prerequisites. No prior AI experience needed. If you can open a terminal, you're ready. Just bring your laptop — everything else runs free in the cloud.

Whether you're a QA engineer looking to pivot, a developer curious about AI safety, or simply someone who wants to understand what's really going on inside the tools everyone's using — this workshop hands you the skills companies are paying top dollar to find.

Come break something. Leave knowing how to make it unbreakable.

Description:

AI agents aren't chatbots anymore. They send emails, edit CRM records, run shell commands, browse the web, and remember everything — autonomously, with no human in the loop. That's not a product pitch. That's an attack surface.

In this 50-minute session, we crack open a real autonomous sales agent — one with 33 tools, root runtime privilege, and production access to Gmail and HubSpot — and run a live security audit against it using the same methodology Fortune 500 companies use to evaluate AI risk.

You'll watch six guardrails break in real time.

What we cover:

We start with two cautionary tales: Air Canada, whose AI chatbot created direct legal liability after giving customers wrong refund advice, and Zillow, whose AI pricing model quietly accumulated $500M+ in losses before anyone noticed. These aren't edge cases — they're the playbook for what happens when AI systems ship without proper evaluation.

Then we go hands-on. Using AIVSS — the AI Vulnerability Scoring System — we walk through ten risk categories every autonomous agent should be tested against, focusing on the three that matter most: Tool Use, Memory, and Autonomy of Action. You'll see exactly how an agent gets tricked into running a shell command it was never supposed to touch, how it can be convinced to draw its own attack map for a bad actor, and how a simple role-swap prompt sends it completely off the rails.

Every failure is scored, every vulnerability is traceable, and every fix is actionable.

You'll leave with: An understanding of why the gap between what an agent can do and what it should do is your biggest security problem right now

The question isn't whether your agent can go rogue. It's whether you'll find out before your users do.