Open source AI hiring models are weighted toward male candidates, study finds – Computerworld

Asking these personas, which included luminaries ranging from Joseph Stalin and Adolph Hitler to Queen Elizabeth I and women’s rights advocate Mary Wollstonecraft, to choose a candidate, resulted in an increase in female callback rate. However, invoking Ronald Reagan, Queen Elizabeth I, Niccolo Machiavelli, or D.W. Griffith reduced the rate. And models for William Shakespeare, Steven Spielberg, Eleanor Roosevelt, and Elvis Presley almost never refused to choose a candidate.

“This suggests that adopting certain personas increases the model’s likelihood of providing clear gender recommendations—potentially weakening its safeguards against gender-based discrimination—while others, particularly controversial figures, heighten the model’s sensitivity to biases,” the researchers observed.

They also examined wage disparity and discovered that the wage penalty for women also varied wildly. For example, it disappeared at callback parity when the model was prompted with the names of Elizabeth Stanton, Mary Wollstonecraft, Nelson Mandela, Mahatma Gandhi, Joseph Stalin, Peter the Great, Elvis Presley, or J. Robert Oppenheimer, and women were recommended for relatively higher wage jobs than men when prompted with Margaret Sanger or Vladimir Lenin.