Hugging Face is being used to easily undress women and children
Hugging Face is being used to make nonconsensual deepfakes, and the popular open-source AI model repository is doing very little to prevent it. That's according to a new report published by the European nonprofit AI Forensics, which found that seven out of the top nine image editing models hosted by Hugging Face readily complied with requests to undress women using simple prompts.
A platform built on openness is now confronting the predictable cost of that openness. Hugging Face, the open-source AI repository that positioned itself as the democratic counterweight to closed labs, is hosting image-editing models that strip clothing from women and children with minimal prompting. The nonprofit AI Forensics tested nine of the most popular image tools on the platform and found seven compliant. The headline number matters less than the structural point it illustrates.
The contrast with Gemini and ChatGPT is instructive. Those systems invest heavily in refusal layers, content classifiers, and usage policies precisely because they control the model weights and the inference stack. Hugging Face's model is different. It is a distribution layer for weights uploaded by third parties, and its moderation posture has always lagged behind its growth. The platform has scale, reputation, and venture capital; it does not have a content moderation apparatus comparable to the closed labs it competes with for developer mindshare.
The leverage here sits in an uncomfortable place. Removing the offending models would be straightforward technically. Doing so would invite accusations of censorship from the open-source community that gives Hugging Face its cultural legitimacy. Leaving them in place invites regulators, advertisers, and enterprise customers to reconsider whether an "open" repository is a safe enough foundation for serious deployment. AI Forensics has now handed that question a concrete, reproducible answer.
The deeper signal is about the limits of self-policing in model repositories. When the artifact being distributed is a set of weights rather than a hosted API call, the traditional content moderation playbook does not transfer cleanly. Hugging Face will need to decide whether it is a platform, a publisher, or a pipe. Each designation carries a different set of obligations, and the current ambiguity is the asset that bad actors are exploiting.
For the broader AI market, the report is a reminder that safety claims are not portable across distribution models. A guardrail built into a closed inference endpoint does nothing for a downloaded weight running on a consumer GPU. As open-weight models proliferate, the gap between corporate safety narratives and on-device reality will only widen. Hugging Face is simply the first major repository forced to answer for it publicly.