High-performance AI without the risks.
Eticas conducts an external and independent audit of the VioGen system
Case Studies
Eticas conducts an external and independent audit of the VioGen system
recommender
In Spain, the level of risk to which a victim of gender violence is subjected is determined by an algorithm. The system to which it belongs is VioGén which, with more than 3 million risk evaluations, is the risk assessment system with the most registered cases in the world. As leaders in algorithmic audits and AI, in collaboration with Ana Bella Foundation, Eticas has chosen the VioGén system to start its external audits project due to its high social impact.
On several occasions since 2018, Eticas has proposed to the Ministry of Interior to conduct a confidential and probono audit of the system without response. Most of the VioGén studies have been carried out by the same researchers who contributed to its development or by people who work or have vested interests in the ministry and police forces. This means that there is a need for an independent evaluation of the VioGén system and for the ending to its lack of transparency. Furthermore, for a system intended to be used upon highly vulnerable populations, end-users have not been considered or consulted. For these reasons, we have conducted the study through the External Audit methodology, using reverse engineering.
External Audit of the VioGén System:
To carry out the audit we have combined quantitative and qualitative approaches. Qualitative fieldwork consisted of phone interviews and survey research. To evaluate their experience with the system, we interviewed 31 women who have suffered from gender violence. Their cases entered in the VioGén system between 2019 and 2021 when they reported the aggression in Andusia, Valencia, Madrid or Galicia (Spain’s regions with more active cases in VioGén).
It should be noted that VioGén is activated when the victim reports the aggression. According to the Macro-survey on Violence against Women (2019), only 21.7% of women over the age of 16 who suffered gender-based violence reported their aggressor. The remaining 78.3% of women did not report and therefore were not evaluated by the VioGén system. Of the 347 fatal victims of gender-based violence in Spain between 2009-2019, only 126 of them had previously been reported by the police. This means that 73% of women who were killed by their (ex)intimate partners did not previously report their assailant and did not receive a VioGén risk assessment.
The reasons that dissuade women from filing a report against their aggressor are individual emotional, structural based on groups -as can be the case of women with children, socioeconomically disadvantaged or migrant women- or institutional -only 2,000 agents of the 27,000 involved in the management of VioGén are specialized in gender-based violence.
Customer Satisfaction
Knowing that a company is committed to ethical practices and undergoes regular audits can enhance customer satisfaction. Users are more likely to engage with and continue using products that prioritize their well-being and privacy.
Most relevant problems found during the fieldwork
80% of the women interviewed in our study reported different problems with the VioGén questionnaire. This means that the quality of the data introduced into the algorithmic system could be compromised at this time, creating sources of bias and misrepresentation within the system.
Lack of information: 35% of the interviewees were not informed about their VioGén risk score and therefore did not know what level of risk the system assigned to them.
Timing of the VioGén questionnaire: Many women who suffer gender-based violence arrive at the police station to file a report right after a violent incident, which means that it is usual to find them in a state of shock. Our interviewees mentioned that it was difficult for them to remember everything that happened, organize their thoughts and provide detailed answers to VioGén’s questions.
Lack of legal support: Women who suffer gender-based violence have the right to request a lawyer to file their complaint, but few are aware of this right.
Lack of psychological support: Not having help and psychological support before and during this process aggravates the emotional burden that this situation entails.
Likewise, VioGén assigns a lower level of risk to those women who do not have children, although it does not increase the risk of those who do. In addition to this, there is a lack of representation of certain social groups such as, for example, migrant women.
15 of 31 women interviewed have negatively evaluated their experience with the system. One of the main concerns about the VioGén algorithm is that approximately 45% of cases are rated as “unappreciated” risk. This lack of appreciation of risk leads us to think that there are factors that are not yet being taken into account. Victims and lawyers agree that VioGén underestimates psychological violence and newer forms of non-physical violence.
Regarding the quantitative analysis of the system that we have been able to carry out with public data, we must highlight that between 2003 and 2021 there were 71 killed women who had previously filed a report without obtaining police protection (false negatives). On the other hand, another 55 murdered women received a protection order that turned out to be insufficient.
Our recommendations
After carrying out this External Audit and basing ourselves on everything described, our recommendations include the elimination of access barriers at the individual, group and institutional level, the increase in the number of agents specialized in gender violence, the provision of legal and psychological support to victims and the justification, through a complementary police report, of the assigned risk score.
In addition, feedback from victims is vitally important. The evaluation of the performance of the system should not be limited to its technical analysis, it should also take into account the experiences and perceptions of the interested parties that go through the system or work with it.
Since VioGén relies on a considerably large database to make its predictions, it is crucial to check the actual configuration with advanced data analysis techniques, in order to validate the risk factors and identifiers used by the system to assess risk. Furthermore, these evaluations should be made available to the general public to foster transparency and confidence in the system.
After 7 months analyzing and studying VioGén, we think it is necessary to have a public debate on the benefits and risks of incorporating machine learning techniques in VioGén in which all interested parties participate, as well as external experts.
