Week 4

Topic: Responsible AI : Bias and Fairness

Keynote Speaker: Rongkun Zhu

Time: Aug 12, 20:00 - 21:30

Venue: Room 341, School of Business

Tencent Meeting: #907-2153-6929

Compendium

1. Introduction

Bias in AI systems is a prevalent issue in real-world applications. AI algorithms, particularly machine learning models, are prone to bias because they rely on historical data that often contain biases due to past societal or organizational inequalities.

Fairness in AI refers to creating AI models that are free from unfair bias and ensure equitable treatment across different demographic groups, such as race, gender, and socioeconomic status. Ensuring fairness is crucial for AI applications in sensitive areas like hiring, criminal justice, and lending.

2. Types of Bias

2.1 Historical Bias

This type of bias arises from historical inequalities in the training data. For example, if women were underrepresented in a particular job sector, an AI model trained on that data might favor men for similar job positions.

2.2 Measurement Bias

Measurement bias happens when the features used in the AI model capture biased or partial information. For instance, using zip codes as a feature in a loan approval model might introduce racial bias.

2.3 Aggregation Bias

Aggregation bias results from treating diverse populations as homogeneous groups. A model that works well on the majority population may underperform on minority groups.

3. Fairness Metrics

3.1 Demographic Parity

Demographic parity ensures that the outcome is independent of protected attributes. For instance, in hiring, the proportion of selected candidates from different demographic groups should be similar.

3.2 Equalized Odds

Equalized odds aim to equalize false positive and false negative rates across different demographic groups. For instance, a medical diagnostic model should have similar performance for men and women.

3.3 Fairness through Unawareness

This approach involves removing protected attributes, such as race or gender, from the model. However, it may not always ensure fairness as proxy variables can indirectly introduce bias.

4. Techniques for Fairness in AI

4.1 Pre-processing

In this technique, the data is adjusted before being fed into the model. This includes techniques like reweighting or resampling to reduce bias.

4.2 In-processing

In-processing methods modify the learning algorithm itself. For example, constraints can be added to the model to ensure fairness during training.

4.3 Post-processing

Post-processing methods adjust the model’s output to make it fairer. For instance, the output probabilities of a classifier can be calibrated to reduce bias.

5. Case Study: COMPAS Recidivism Algorithm

The COMPAS recidivism algorithm is a widely known example of bias in AI. This algorithm, used in criminal justice for predicting the likelihood of reoffending, was found to disproportionately flag African-American defendants as high risk compared to white defendants.

6. Conclusion

Bias and fairness are critical concerns in AI systems, especially in high-stakes applications. Various fairness metrics and techniques can be applied to mitigate bias, but ensuring fairness remains a complex challenge that requires careful consideration of the context and the data being used.

Material

Presentation Slides: https://wma17.github.io/24summer/docs/pdfs/Week4_RessponsibleAI.pdf