Tamer Abuhmed
Sungkyunkwan University | Associate Professor
Dr. Tamer Abuhmed is an associate professor in the Department of Computer Science and Engineering at Sungkyunkwan University since September 2019. He received his Ph.D. degree from the School of Information Technology and Communication at INHA University in August 2012 under the supervision of Professor DaeHun Nyang, where he was a member of the Information Security Research Lab (ISRL). His main research interests are 1) Information Security, including the design and analysis of algorithms and protocols for user authentication, software security, adversarial machine learning, and network security. 2) Computer-aided design and diagnosis methods for biomedical applications.
## Learning Objectives
After completing this workshop, you will be able to:
- Distinguish the three sources of bias, data bias (historical, representation/sampling, and measurement/label bias), algorithmic bias (from the model and its objective), and societal bias (from the deployment context and feedback loops), and identify where each enters the pipeline.
- Define the main group-fairness criteria, demographic (statistical) parity, equalized odds, equal opportunity, predictive parity, and calibration, and contrast them with individual fairness.
- Explain the impossibility result: why, when base rates differ across groups, calibration and equalized odds cannot both hold, so fairness requires choosing the definition that fits the context rather than a single universal metric.
- Measure bias with quantitative fairness metrics, disparate impact ratio, statistical parity difference, and equal opportunity difference, and read what each value implies.
- Place mitigation techniques in the pipeline: pre-processing (reweighing, resampling, disparate-impact remover, optimized preprocessing), in-processing (fairness constraints, adversarial debiasing), and post-processing (threshold adjustment, reject-option classification), and state the accuracy trade-off each carries.
- Describe the canonical fairness-benchmark datasets, Adult/Census Income, COMPAS, and German Credit, and identify the protected attributes each involves.
- Argue why fair AI requires organizational commitment, stakeholder engagement, diverse teams, and audits, not only a technical fix, and connect this to regulation (Workshop 10) and transparency (Workshop 8).
## Theoretical Background
### Where Bias Comes From
"Bias" in AI is not one thing. It helps to separate three sources, because each enters at a different stage and is addressed differently.
**Data bias** originates in the training data itself, before any model is fit.
- *Historical bias* is present when the data faithfully records a world that is already unequal. The data may be accurate and still encode discrimination, because past decisions, past hiring, past sentencing, past lending, were themselves unfair.
- *Representation (sampling) bias* arises when some groups are under-sampled, so the model has too few examples to learn them well. A model trained mostly on one population predicts worst for the populations it saw least.
- *Measurement (label) bias* arises when the recorded features or labels are noisy proxies for what we actually care about, and the noise is not evenly distributed. If arrest is used as a proxy for crime, and policing intensity varies by neighborhood, the label itself is biased.
**Algorithmic bias** is introduced by the model and its objective, even on reasonable data. A model that minimizes average error will happily sacrifice accuracy on a small subgroup to gain a little on the majority, because the objective never asked who bears the errors. Model capacity, feature interactions, and regularization can all create disparate impact that was not obviously present in the data.
**Societal bias** comes from the deployment context and the feedback loops a deployed system creates. A biased predictive-policing model sends more officers to a neighborhood, which produces more recorded incidents there, which confirms the model, a self-reinforcing loop. Bias here is not a static property of a dataset but a dynamic property of a system embedded in the world.
### Defining Fairness
Once bias is on the table, the natural question is what "fair" would even mean. There is no single answer; there is a family of precise definitions, most of them expressed in terms of a protected attribute $A$ (for example sex or race), the true label $Y$, and the model prediction $\hat{Y}$. The definitions below are **group-fairness** criteria: they demand equality of some statistic across groups.
**Demographic (statistical) parity** requires equal positive-prediction rates across groups, independent of the true label.
$$
P(\hat{Y} = 1 \mid A = 0) = P(\hat{Y} = 1 \mid A = 1)
$$
**Equalized odds** requires equal *true positive rates and* equal *false positive rates* across groups, so the model is equally accurate, and equally wrong, for everyone.
$$
P(\hat{Y} = 1 \mid A = a, Y = y) \ \text{equal across } a, \ \text{for } y \in \{0, 1\}
$$
**Equal opportunity** is the weaker, one-sided version: equal true positive rates across groups (equalized odds restricted to $Y = 1$). It asks that among people who truly qualify, the chance of being correctly identified does not depend on group membership.
$$
P(\hat{Y} = 1 \mid A = 0, Y = 1) = P(\hat{Y} = 1 \mid A = 1, Y = 1)
$$
**Predictive parity** requires equal positive predictive value (precision) across groups: when the model predicts positive, it should be right equally often for each group.
$$
P(Y = 1 \mid \hat{Y} = 1, A = 0) = P(Y = 1 \mid \hat{Y} = 1, A = 1)
$$
**Calibration** requires that a predicted score mean the same thing across groups: among everyone assigned probability $s$, the same fraction are truly positive, regardless of group.
Standing apart from all of these is **individual fairness**: the principle that any two individuals who are similar with respect to the task should receive similar predictions. Group fairness constrains averages over groups; individual fairness constrains treatment of specific people, and the two can pull in different directions.
| Definition | Equalizes across groups | Intuition |
|---|---|---|
| Demographic parity | Positive-prediction rate | Equal selection rates |
| Equalized odds | TPR and FPR | Equal benefit and equal harm |
| Equal opportunity | TPR only | Equal chance for the truly qualified |
| Predictive parity | Positive predictive value (precision) | A positive prediction is equally trustworthy |
| Calibration | Meaning of a score | A score means the same for everyone |
| Individual fairness | (per-individual) | Similar people, similar treatment |
### The Impossibility Result
Faced with this menu, the natural instinct is to satisfy all of it. That is provably impossible in general. When the base rate of the outcome differs across groups, a condition that holds in almost every real application, several of these definitions become mutually exclusive. The sharpest form, established independently by Kleinberg, Mullainathan and Raghavan (2016) and by Chouldechova (2017), is that **calibration and equalized odds cannot both hold** once base rates differ: a classifier that is calibrated for both groups must have unequal false positive or false negative rates, and vice versa.
This was not a hypothetical. It was exactly the crux of the ProPublica/COMPAS dispute (below): the tool's vendor argued it was fair because it was calibrated across race; ProPublica argued it was unfair because its false-positive rates differed across race. Both were right about their own metric. They were measuring different, incompatible definitions of fairness.
The practical lesson is decisive: **there is no universal fairness metric.** Fairness is a modeling choice about which definition fits the stakes of the specific context, who is harmed by a false positive versus a false negative, and what the law and the affected community require. Choosing the metric is a value judgment that a technical team cannot make alone.
### Measuring Bias
Definitions become actionable through metrics that put a number on the gap. Three recur throughout the field and in the lab:
- **Disparate impact ratio**, the ratio of the unprivileged group's positive rate to the privileged group's. A value of $1.0$ is parity; the common "four-fifths rule" from US employment law flags a ratio below $0.8$ as evidence of adverse impact.
- **Statistical parity difference**, the same idea as a difference rather than a ratio: the unprivileged positive rate minus the privileged positive rate. $0$ is parity; negative values indicate the unprivileged group is selected less often.
- **Equal opportunity difference**, the difference in true positive rates between groups. $0$ means the truly-qualified are identified equally well regardless of group.
A metric near its fair value is necessary but not sufficient, a model can look fair on one metric and biased on another, which is the impossibility result showing up in practice.
### Mitigating Bias Across the Pipeline
Interventions are grouped by *where in the pipeline* they act, and each stage has a characteristic trade-off with accuracy.
| Stage | What it changes | Example techniques | Trade-off |
|---|---|---|---|
| **Pre-processing** | The training data | Reweighing, resampling, disparate-impact remover, optimized preprocessing | Model-agnostic; may lose information and struggles with correlated proxies |
| **In-processing** | The learning algorithm | Fairness constraints, adversarial debiasing, fair representation learning | Directly optimizes fairness but requires changing the training procedure |
| **Post-processing** | The trained model's outputs | Threshold adjustment, reject-option classification | Works on any trained model but can be less accurate and may treat individuals inconsistently |
The lab operates at the **pre-processing** stage. Pre-processing is attractive because it is model-agnostic: transform the data once, then any downstream classifier inherits the improved fairness. The cost is that editing features and labels can discard useful signal, and correlated proxy features make it hard to remove a protected attribute's influence cleanly, which is why the mitigation is measured, not assumed to work.
### The Datasets: Canonical Fairness Benchmarks
The lab uses datasets that are standards in the fairness literature, each tied to a real high-stakes decision and a real protected attribute.
- **Adult / Census Income** predicts whether a person's income exceeds \$50K from census features. The protected attributes are **sex** and **race**; the well-known finding is that naive models predict high income far more often for men than for women. This is the lab's primary dataset.
- **COMPAS** contains recidivism-risk scores from Broward County, Florida. Its protected attribute is **race**. It is the dataset behind ProPublica's 2016 "Machine Bias" investigation, which found that the tool's false-positive rate, labeling defendants high-risk who did not go on to reoffend, was roughly twice as high for Black defendants as for white defendants. COMPAS is the case study that made the impossibility result concrete.
- **German Credit** predicts credit risk (good vs. bad) from financial and demographic features. Its protected attributes are **sex** and **age**. It is a smaller, classic benchmark for credit-scoring fairness.
### Fairness Is Not Only a Technical Problem
The guest lecture's central argument is that the technical machinery above is necessary but not sufficient. A fairness metric can be satisfied on a test set and still fail in deployment as data drifts, as feedback loops kick in, or as the definition chosen turns out to mismatch the harm that matters. Sustaining fairness is an organizational commitment:
- **Stakeholder engagement**, involving the people affected by a system in defining what fairness means for it, and building channels for those harmed to be heard.
- **Diverse teams**, because a homogeneous team is unlikely to anticipate the harms it does not itself experience.
- **Audits and ongoing monitoring**, treating fairness as a property to be re-verified as systems age, not a box checked once at launch.
This organizational layer is the bridge to the rest of the program. **Workshop 8** takes up transparency, explainability, and interpretability, the tools that let auditors and stakeholders actually see what a model is doing. **Workshop 10** turns to regulation and standards, the external accountability, from anti-discrimination law to AI-specific frameworks, that increasingly requires exactly the fairness assessment practiced here.
## Hands-on Lab
The activity is implemented with **AI Fairness 360 (AIF360)**, IBM's open-source fairness toolkit, over a **scikit-learn** logistic-regression classifier, with **pandas** and **NumPy** for data handling. It loads a canonical fairness-benchmark dataset (the Adult/Census Income data by default, with COMPAS and German Credit selectable in the same notebook), measures bias with AIF360's metrics (statistical parity difference, disparate impact, average odds difference, equal opportunity difference, and the Theil index), applies a pre-processing mitigation, and re-measures the same metrics to see the improvement, and its accuracy cost, directly.
The mitigation is AIF360's **Optimized Preprocessing** (`OptimPreproc`), a pre-processing technique that learns a probabilistic transformation of features and labels to reduce group disparity while limiting how much individual records are distorted. The transformation is learned on the training split only and then applied to held-out data, so the reported fairness gains reflect generalization rather than leakage.
### Activity 01: Pre-processing Bias Mitigation
Measure bias on a fairness-benchmark dataset with AIF360, apply the Optimized Preprocessing pre-processing mitigation, and compare the fairness metrics (and classifier accuracy) before and after the transformation.
- Open on GitHub: [SecAI_Workshop07_Activity01_Preprocessing_biase_metigation.ipynb](https://github.com/SecureAI-luc/SecureAI-Lab/blob/main/Workshop07/SecAI_Workshop07_Activity01_Preprocessing_biase_metigation.ipynb) | Open in Colab: [SecAI_Workshop07_Activity01_Preprocessing_biase_metigation.ipynb](https://colab.research.google.com/github/SecureAI-luc/SecureAI-Lab/blob/main/Workshop07/SecAI_Workshop07_Activity01_Preprocessing_biase_metigation.ipynb?authuser=1)
To run the activity, open its Colab link and use **File → Save a copy in Drive** to get your own editable copy, then execute it there.