AI Model Security
Attack and defend machine learning models: adversarial examples, data poisoning, model extraction, membership inference, and defenses that survive contact with a real attacker.
Course Overview
Machine learning models fail under adversarial pressure in ways conventional application security does not anticipate. An input crafted to look unremarkable to a human can flip a classifier's output. A handful of poisoned training records can install a backdoor that only activates on a chosen trigger. A model exposed through an API can be extracted through queries alone, and can leak whether a particular person's record was in its training data. None of these are addressed by patching dependencies.
This course works through those attacks hands-on, because defenses designed without understanding the attack tend not to work. You will construct adversarial examples, poison a training set to plant a backdoor, extract a functional copy of a model through its API, and run a membership inference attack. Each is then paired with the practical defenses and their real, honestly-stated limits.
The final sessions move from individual attacks to a security program: threat modeling for ML systems, red-teaming methodology, supply chain risk in pretrained weights and model hubs, and the monitoring that detects extraction or evasion attempts against a deployed model. The intent is that you leave able to assess a model the way an attacker would.
Duration: 3 days|Delivery: onsite, virtual, or hybrid
Prerequisites
- Practical machine learning experience including training and evaluating models
- Strong Python and familiarity with PyTorch or TensorFlow
- Working security knowledge such as threat modeling or application security
- Understanding of how models are served in production
Who Should Attend
- Security engineers assessing ML systems who need ML-specific attack knowledge
- ML engineers whose models are exposed to untrusted users or public APIs
- Red team members extending their scope to AI systems
- Security architects setting requirements for models before production approval
Course Outline
- 1Threat modeling for ML: assets, attacker goals, and where the trust boundaries actually sit
- 2Adversarial examples: FGSM, PGD, and why small perturbations flip predictions
- 3Evasion attacks against production classifiers, including content and fraud filters
- 4Data poisoning: availability attacks, targeted attacks, and backdoor triggers
- 5Supply chain risk: pretrained weights, model hubs, and unsafe deserialization
- 6Model extraction: reconstructing a model through query access alone
- 7Membership inference and training data extraction as privacy failures
- 8Model inversion and attribute inference from model outputs
- 9Defenses: adversarial training, input preprocessing, and their measured costs
- 10Detection: distinguishing extraction and evasion traffic from ordinary use
- 11Rate limiting, output perturbation, and watermarking as deterrents
- 12Red-teaming methodology and building an ML security assessment
- 13Incident response for a compromised or poisoned model
Learning Outcomes
- Threat model an ML system and identify the attack surface conventional review misses
- Construct adversarial examples and evaluate how robust a model actually is
- Execute and detect data poisoning and backdoor attacks
- Assess whether a deployed model can be extracted through its API
- Test for training data leakage via membership inference
- Select defenses with a clear-eyed view of their accuracy and cost tradeoffs
- Run a structured red-team assessment of a model before it goes to production
What You Will Build
- A working adversarial attack and a robustness evaluation of a target model
- A demonstrated backdoor attack and the detection method that catches it
- A model extraction assessment for an API-exposed model
- An ML security threat model and assessment checklist for your systems
Frequently Asked Questions
- Is this course offensive or defensive?
- Both, in that order, and for a defensive purpose. Every attack is executed in an isolated lab against models provided for the exercise, then paired with defenses and detection. Building defenses without understanding the attack reliably produces controls that do not hold, which is why the offensive work comes first.
- How does this differ from securing AI pipelines?
- This course attacks the model itself: its predictions, its weights, and its training data. Securing AI Pipelines covers the infrastructure around the model, including data ingestion, artifact integrity, secrets, and access control. Most organizations need both, and they are commonly taken as a pair.
- Are adversarial attacks a real production concern?
- It depends on whether an adversary benefits from a wrong prediction. For fraud detection, content moderation, malware classification, and biometrics, evasion is an active and ongoing problem. For a demand forecasting model behind a firewall, it is largely theoretical. The course helps you make that judgment rather than applying expensive defenses uniformly.
- Does the course cover LLM-specific attacks?
- Yes, including prompt injection, jailbreaking, and training data extraction from language models, alongside the classical attacks that apply across model types. For depth specifically on generative AI attacks, pair this with Prompt Hacking and GenAI Defense.
- What are the lab arrangements?
- All attacks run in an isolated browser-based lab environment against models supplied for the course. Nothing is executed against production systems or third-party services.
Related Courses
Securing AI Pipelines
Secure the infrastructure around your models: data provenance, artifact integrity, secrets, least-privilege access, and audit evidence across the ML lifecycle.
Prompt Hacking & GenAI Defense
Day 2 turns security teams into red-teamers. Execute real prompt-injection, jailbreak, and RAG exploits, then design layered guardrails, content safety filters, and monitoring pipelines to stop them in production.
AI Security Essentials
Day 1 of the intensive teaches defenders how GenAI and LLM apps are attacked. Map new AI threat surfaces, apply OWASP LLM Top 10 mitigations, and build threat models that drive pragmatic controls.
Enterprise RAG: Security and Scale
Multi-tenant RAG with document-level access control, PII handling, prompt injection defense, audit trails, and architectures that hold up under regulatory review.
Ready to Get Started?
Contact us to schedule training for your team or inquire about upcoming sessions.