One-sentence Explanation
AI training data can never be completely free of bias, and we need to clarify the type of bias we have chosen and be aware of its impact.
Popular Understanding
You can use a daily analogy: if you only use test scores from key high schools to train an AI enrollment model, the model will naturally favor students from those schools. This bias stems from the limitations of the data you selected, and it is almost impossible to completely eliminate it. We can only try to clearly identify the biases contained in the data we use.
Application Scenarios
This topic is mainly applicable to scenarios where AI makes decisions based on data, such as AI recruitment, credit approval, and educational resource allocation. Biases in these scenarios will directly affect group fairness.
Related Concepts
Related concepts include training data bias, algorithmic bias, fair AI, and data annotation bias.
Source: Towards AI