Blaine Hoak
University of Wisconsin-Madison & Visa Research | Ph.D. Student & Research Intern
Blaine Hoak is a Ph.D. student at the University of Wisconsin-Madison and a Ph.D. Research Intern at Visa Research on the Trustworthy AI team. Her research centers on evaluating and advancing the security of machine learning models, with a primary focus on adversarial robustness: understanding, designing, and defending against sophisticated attacks that expose worst-case failures of machine learning models. Such attacks not only test the resiliency of models but also highlight discrepancies between human and machine perception, and her work aims to explain and bridge that gap to build safer, better-aligned models.
Aside from AI trustworthiness, she has collaborated on work broadly within security and privacy, with the goal of identifying and protecting against threats to user privacy and security across a variety of systems. Her work has been published in top-tier security and machine learning venues including CCS, USENIX Security, IEEE S&P, and ICLR.
## Learning Objectives
After completing this workshop, you will be able to:
- State the white-box threat model precisely: what the adversary knows and, in particular, that it can compute the gradient of the loss with respect to the input by backpropagation.
- Distinguish the two families of white-box attacks: max-loss (norm-bounded) attacks that maximize loss within a fixed perturbation budget, and min-norm attacks that find the smallest perturbation that causes misclassification.
- Derive and implement FGSM as a single-step max-loss attack and PGD as its iterative, projected, randomly-started extension.
- Explain the C&W attack as a min-norm optimization with no budget, including the change-of-variables trick that keeps perturbed images inside the valid pixel range.
- Distinguish untargeted from targeted objectives, and relate the L-infinity, L2, and L0 perturbation norms to the three C&W labs.
- Train a binary detector that separates clean from adversarial inputs, and demonstrate empirically that a detector calibrated at one attack strength fails to generalize across the strength spectrum.
- Explain transferability and why it makes the black-box surrogate attacks of Workshop 4 possible.
## Theoretical Background
### The White-Box Threat Model
In the white-box setting the adversary has everything: the model architecture, the trained weights, and the loss function, and can therefore compute the gradient of the loss with respect to the *input* by backpropagation. This is the key capability. Ordinary training differentiates the loss with respect to the *parameters* to update the model; a white-box attacker instead freezes the parameters and differentiates with respect to the input, $\nabla_x L$, to learn how to nudge the input so the loss goes the wrong way.
Because no attacker is assumed to be stronger than this, white-box attacks give the worst-case, upper-bound evaluation of robustness. A defense that survives a white-box attack has cleared the highest bar; a defense that fails one will certainly fail against weaker, more realistic attackers.
### Two Families: Max-Loss and Min-Norm
Every white-box attack in this workshop is answering one of two optimization questions. Keeping them separate is the single most important idea in the session, because it explains why the attacks are parameterized so differently.
**Max-loss (norm-bounded) attacks.** Fix a perturbation budget $\epsilon$ and, within that budget, find the perturbation that does the most damage:
$$
\arg\max_{\delta} \; L\big(f(x + \delta),\, y\big) \quad \text{subject to} \quad \|\delta\|_p \leq \epsilon.
$$
The budget $\epsilon$ is a hard cap chosen in advance; the attack pushes right up against it. **FGSM and PGD** are the members of this family.
**Min-norm attacks.** Drop the budget entirely and instead find the *smallest* perturbation that still causes a misclassification:
$$
\arg\min_{\delta} \; \|\delta\|_p \quad \text{subject to} \quad x + \delta \text{ is misclassified}.
$$
There is no $\epsilon$ here. The attack searches for the least-noticeable change that still fools the model. **C&W** is the member of this family, which is why it consistently finds smaller perturbations than the budget-bounded attacks.
### Untargeted vs. Targeted Objectives
Both families come in two flavors, and the distinction is just a sign.
- **Untargeted:** any wrong answer will do. The attack *maximizes the loss of the true label* $y$, pushing the prediction away from correct.
- **Targeted:** a specific wrong class $t$ is required. The attack *minimizes the loss of the target label* $t$, pulling the prediction toward the attacker's chosen answer.
Targeted attacks are strictly harder because they must reach one particular class rather than merely leaving the correct one.
### Fast Gradient Sign Method (FGSM)
FGSM (Goodfellow, Shlens & Szegedy, 2015) is the simplest max-loss attack: take a single step of size $\epsilon$ in the direction that increases the loss, using only the sign of the gradient so every pixel moves by the full budget.
$$
x' = x + \epsilon \cdot \text{sign}\big(\nabla_x L(x, y)\big)
$$
It needs exactly one gradient computation, which makes it fast and a natural baseline, but a single step is a crude approximation of the constrained maximum. It is reliably weaker than the iterative attacks that follow.
### Projected Gradient Descent (PGD)
PGD (Madry et al., 2018) is FGSM done properly: many small steps, each projected back into the allowed region. Two details that a naive iteration omits are exactly what make PGD strong, and both are easy to miss.
1. **Random start.** PGD does not begin at the clean image. It starts from a random point inside the $\epsilon$-ball around $x$, so that repeated runs explore different parts of the loss surface and the attack does not get stuck at a single weak local maximum.
2. **Clip to the valid pixel range.** After each projection back into the $\epsilon$-ball, the result is also clipped to the valid image range $[0, 1]$, because an adversarial example that contains impossible pixel values is not a real image.
Each iteration takes a step of size $\alpha$ (with $\alpha < \epsilon$ so the attack can refine rather than overshoot), projects back onto the $\epsilon$-ball, and then clips:
$$
x^{i+1} = \Pi_{\epsilon\text{-ball}}\Big(x^{i} + \alpha \cdot \text{sign}\big(\nabla_x L(f(x^{i}), y)\big)\Big), \quad \text{then clip } x^{i+1} \text{ to } [0, 1].
$$
Here $\Pi_{\epsilon\text{-ball}}$ projects the perturbation back inside the budget. With random restarts and enough steps, PGD is the strongest first-order (gradient-based) attack and is the standard benchmark for evaluating robustness.
### Carlini & Wagner (C&W)
C&W (Carlini & Wagner, 2017) is the min-norm attack, and it is worth being precise about what it does and does not do, because it is easy to describe incorrectly.
C&W has **no $\epsilon$ budget**. It is not a re-parameterization of the norm-bounded problem; it solves the other optimization question, minimizing the perturbation size subject to misclassification. It does this by minimizing a single objective that trades perturbation size against an attack term:
$$
\text{minimize} \quad \|\delta\|_p + c \cdot f(x + \delta),
$$
where $f$ is a margin-based surrogate that is driven toward zero once the input is misclassified, and the constant $c$ is chosen by binary search to balance the two terms. Smaller $c$ favors a smaller perturbation; larger $c$ favors a successful attack.
To keep every candidate image inside the valid pixel box $[0, 1]$, C&W optimizes an unconstrained variable $w$ and maps it into the box with a change of variables:
$$
x + \delta = \tfrac{1}{2}\big(\tanh(w) + 1\big).
$$
Because $\tanh$ saturates to $[-1, 1]$, the right-hand side always lands in $[0, 1]$, so a standard unconstrained optimizer can run freely on $w$ while every image it produces remains valid. C&W comes in **L0, L2, and L-infinity** variants, the exact three used in this workshop's labs, and it reliably finds smaller perturbations than PGD, which makes it the tool of choice for stress-testing a proposed defense.
### Perturbation Norms and the Three C&W Labs
The norm decides how "small" is measured, and each norm corresponds to a different notion of imperceptibility and to a different C&W lab.
| Norm | What it measures | Meaning for an attacker | C&W variant |
| -------------------- | ------------------------------------ | -------------------------------------------------------------------------------------- | -------------- |
| **L-infinity** | Maximum change to any single pixel | Spread a tiny change across every pixel; the usual imperceptibility bound | C&W L-infinity |
| **L2** | Euclidean length of the perturbation | Total perturbation energy across the image | C&W L2 |
| **L0** | Count of changed pixels | Change as few pixels as possible; sparse, and hardest because L0 is non-differentiable | C&W L0 |
L-infinity is the norm most white-box work bounds because it caps per-pixel change, which is what human vision is roughly insensitive to. L0 is the hardest to optimize precisely because counting changed pixels is not differentiable, so gradient methods cannot attack it directly.
### From Attack to Defense: What the Labs Actually Do
The labs do not stop at generating adversarial examples. Each one then builds a defense and measures how well it holds, and the result is the pedagogical point of the session.
Every lab trains a **binary CNN detector** whose only job is to answer one question: is this input clean or adversarial? It is trained on a 50/50 mixture of clean images and adversarial images produced by that lab's attack, then wrapped in ART's `BinaryInputDetector` so it plugs into the same estimator pipeline used throughout the program. The lab then sweeps the attack strength and plots two curves against each other: the number of inputs **flagged by the detector** and the number that **fool the classifier**.
The lesson is sobering. A detector trained against one attack strength does **not** generalize across the strength spectrum: turn the attack up or down and the detector that looked reliable starts missing adversarial inputs while they still fool the model. Detection is an *empirical* defense, it works against the attacks you trained it on and offers no guarantee against the ones you did not. This is precisely why Workshop 5 moves on to defenses with stronger claims: adversarial training, and certified defenses such as randomized smoothing that come with provable robustness guarantees.
### Transferability
Adversarial examples crafted on one model frequently fool *other* models trained on similar data, even with different architectures. This property, transferability, is not just a curiosity: it is the mechanism that makes black-box attacks practical. An attacker who cannot see a target model can train their own surrogate, craft adversarial examples against it in full white-box mode, and transfer them to the target. That is exactly the strategy behind the black-box surrogate and query-based attacks in [Workshop 4](../Workshop04/Adversarial_Attacks_-_Black-Box_Attacks.md), such as SimBA and ZOO. The white-box attacks you build here are therefore also the engine of the black-box attacks that come next.
## Hands-on Lab
The six activities implement all three attacks on both a handwritten-digit model (MNIST-10) and a color-image model (CIFAR-10), each using ART. For every attack, the notebook generates adversarial examples, trains the binary detector described above, and plots detector flags against classifier fooling as attack strength is swept.
The notebooks use pre-generated adversarial datasets included in each workshop's root-level `datasets/` folder, so the compute-heavy generation step does not have to be rerun from scratch. Some pre-trained detector models are also provided in each workshop's root-level `models/` folder (for the MNIST-10 activities); where a detector is not provided, the notebook trains one inline.
### Activity 01: PGD on MNIST-10
Projected Gradient Descent against a 10-class MNIST digit classifier, with a detector trained on the resulting perturbations.
- Open on GitHub: [SecAI_Workshop03_Activity01_PGD_MNIST10.ipynb](https://github.com/SecureAI-luc/SecureAI-Lab/blob/main/Workshop03/SecAI_Workshop03_Activity01_PGD_MNIST10.ipynb) | Open in Colab: [SecAI_Workshop03_Activity01_PGD_MNIST10.ipynb](https://colab.research.google.com/github/SecureAI-luc/SecureAI-Lab/blob/main/Workshop03/SecAI_Workshop03_Activity01_PGD_MNIST10.ipynb?authuser=1)