Mitigating Dialect Bias in Toxicity Detection

Fairness evaluation and bias mitigation for toxicity classification models

View code View report

Overview

Automated toxicity classifiers can learn to associate African American English (AAE) with toxicity, creating a higher risk that non-toxic posts by Black users are removed or penalized. We measured this disparity, tested multiple mitigation strategies, and examined whether improvements transferred to a second dataset.

Research Question

Do toxicity classification models disproportionately flag African American English (AAE) as toxic compared to Standard American English (SAE), and can this disparity be reduced without significantly harming overall model performance?

My Contribution

This was a group research project. I implemented and evaluated the XGBoost and vector-scaling mitigation approaches, contributed to the adversarial-debiasing experiments, and analyzed subgroup fairness and performance tradeoffs across all methods.

Approach

We used the Davidson hate-speech and offensive-language dataset as the primary toxicity data, paired it with probabilistic dialect labels from TwitterAAE, and used HateXplain to test out-of-domain generalization.

Baselines: XGBoost on BERT embeddings, ToxicBERT, and a fine-tuned BERT classifier
Vector scaling — post-hoc calibration to equalize subgroup score distributions
Reweighting — adjusting training sample weights across dialect subgroups
Adversarial debiasing — an adversarial network trained to remove dialect signal from learned representations

Evaluation

We compared accuracy and F1 with subgroup false-positive rate (FPR), false-negative rate (FNR), and disparate impact. Because the central harm was over-flagging non-toxic AAE text, model selection emphasized reducing the AAE–SAE FPR gap while keeping F1 near the baseline. We then applied the strongest configurations to HateXplain without retuning to test whether the gains survived distribution shift.

Results / Findings

0.233 → 0.011 AAE–SAE false-positive-rate gap

95% Reduction in FPR gap

Method Gap F1

XGBoost baseline 0.233 0.84

Re-weighting 0.06 0.81

Adversarial debiasing 0.03 0.82

Vector scaling 0.011 0.83

AAE–SAE false-positive-rate gap on TwitterAAE, with overall F1 alongside. Vector scaling gave the smallest gap while keeping F1 close to the 0.84 baseline.

What We Found

In-domain, vector scaling produced the strongest balance: it lowered the XGBoost FPR gap from 0.233 to 0.011 while retaining an F1 of 0.83 versus the 0.84 baseline. Reweighting and adversarial debiasing also reduced the gap, but no method dominated every metric. Reducing false positives for AAE often increased false negatives, making the result a fairness–performance tradeoff rather than a single optimal solution.

The improvement did not transfer cleanly to HateXplain. Without retuning, vector scaling's FPR gap rose to 0.18 and F1 fell to 0.64, showing that the in-domain calibration was highly dataset-specific.

Limitations

Dialect was inferred with probabilistic labels rather than self-reported speaker identity, and the experiments covered a limited set of English-language datasets and model families. Group-specific calibration also assumes access to a dialect label at inference time and degraded substantially under distribution shift. Future work should use more representative dialect-labeled data and test interventions designed to remain robust across domains, languages, and intersecting identities.