Eticas

High-performance AI without the risks. 

Case Studies

Algorithmic Impact Assessment of the Predictive System for Risk of Homelessness for the Allegheny County

recommender systems

Eticas audited the algorithmic system that predicts homelessness risk in Allegheny County, Pennsylvania, USA. The model took into consideration 964 variables coming from different departments and regulatory bodies of the county (demographic and mental health variables, justice, prison, hospital visits and substance use registers, amongst others).

This audit was focused on determining the algorithmic model accuracy, as well as evaluating different metrics to be able to measure system bias regarding protected groups and sociodemographic discrimination such as age, race, gender, and disability.

The audit revealed:

Overall accuracy of the model

Eticas discovered the predictive model was accurate, except for the 0–17 age group which was heavily underrepresented in the training data used to build the model. For this reason, the County was advised to handle this group separately by a business rule, and not by the model.

Algorithmic discrimination

Throughout the audit, the model was evaluated to determine if the algorithm and the risk score it produced introduced a large disadvantage for potentially already disadvantaged groups, to which Eticas did not find such disadvantage or discrimination for the following reasons: Disability, age, race, and property-based discrimination.

There were no clear indicators of discrimination based on gender and veteran status, but some inconsistencies suggest that these factors’ performance should closely monitored.

Recommendations

Allegheny County Department of Human Services’ Response to Eticas’ report

The county committed to make a series of changes to enhance the accuracy of the system and the protection of its stakeholders, including but not limited to: