Generating AUs on Faces Utilizing ChatGPT Image Generation Models to Improve Performance and Fairness in AU Detection

Abstract

This thesis explores whether a gender and ethnicity balanced synthetic dataset of facial images, generated with the ChatGPT-Image-1 API, can improve both performance and subgroup fairness in Facial Action Unit (FAU) detection. First, 40 high-resolution neutral identities evenly split across gender (20 male, 20 female) and across four ethnic groups (Black, Brown, Asian, White) are created. For each identity 25 AU-rich expressions are synthesized, yielding a dataset of 1,040 images (40 neutral + 1,000 expression frames). This data is used to fine-tune a FMAE-IAT AU-detector fine-tuned on BP4D dataset. Two fine-tuning regimes are evaluated: (i) an imbalanced real-world baseline (BP4D), (ii) an imbalanced real-world baseline (BP4D) and a synthetic dataset. F1 and AUC scores are used to measure overall and cross demographic performance, while FID is used to measure the realism of the generated dataset.