Hugging Face models enable nonconsensual deepfake undressing, report finds
Summarized by AI from reporting by The Verge AI, published under our editorial policy.
A new report by AI Forensics reveals that seven of the top nine image editing models on Hugging Face can generate nonconsensual deepfake images that undress women and children. The platform's content moderation is failing to prevent this misuse.
Key takeaways
- Seven out of the top nine image editing models on Hugging Face can generate nonconsensual deepfake images that undress women and children.
- The report by AI Forensics found that Hugging Face's content moderation policies are not effectively preventing the creation and distribution of nonconsensual deepfakes.
- The models require minimal technical knowledge to operate, making misuse accessible to a wide range of users.
Hugging Face, a popular open-source AI model repository, is being used to create nonconsensual deepfakes that undress women and children. That's according to a new report published by the European nonprofit AI Forensics, which found that seven out of the top nine image editing models hosted by Hugging Face readily complied with prompts to generate such images.
Seven of nine top image editing models generated undressing deepfakes
The report tested nine of the most popular image editing models on Hugging Face and found that seven of them could generate nonconsensual deepfake images when prompted. These models can be used to undress women and children without their consent, raising serious ethical and legal concerns. The report highlights the ease with which these models can be misused, as they are freely available and require minimal technical knowledge to operate.
Hugging Face's moderation policies are not stopping misuse
Hugging Face has been criticized for not taking sufficient action to prevent the misuse of its models. The platform's content moderation policies are not effectively stopping the creation and distribution of nonconsensual deepfakes. While Hugging Face has guidelines in place, the report suggests that these guidelines are not being enforced rigorously enough to prevent misuse.
Nonconsensual deepfakes threaten privacy and safety
The ability to create nonconsensual deepfakes has serious implications for privacy, consent, and safety. These images can be used to harass, blackmail, or exploit individuals, particularly women and children. The ease with which these models can be used to generate such images highlights the need for stronger regulations and better enforcement of existing guidelines to prevent misuse.
How to report nonconsensual deepfakes
If you encounter nonconsensual deepfakes, you can report them to the platform where they are hosted. Hugging Face has a reporting mechanism for inappropriate content, and you can use this to flag any deepfakes you come across. Additionally, you can support organizations that advocate for stronger regulations and better enforcement of guidelines to prevent the misuse of AI technologies.
Frequently asked
- Is Hugging Face doing anything to prevent the misuse of its models?
- Hugging Face has guidelines in place, but the report suggests that these guidelines are not being enforced rigorously enough to prevent misuse.
- What can I do if I encounter a nonconsensual deepfake?
- You can report the content to the platform where it is hosted, such as Hugging Face, and support organizations advocating for stronger regulations.
- Which organization published the report?
- The report was published by AI Forensics, a European nonprofit organization.