hugging face · 2024
Hugging Face malicious models
JFrog researchers identified roughly 100 malicious models hosted on Hugging Face in February 2024. About 25 of them delivered unsafe-deserialisation payloads that executed on model load via Python's pickle __reduce__ path.
What happened
On February 27 2024, security researchers at JFrog disclosed that
they had identified roughly 100 malicious models hosted on Hugging
Face's model hub. About 25 of them carried unsafe-deserialisation
payloads. Code that executed at model-load time as soon as a developer
ran the standard torch.load() or
AutoModel.from_pretrained() call to try the model.
None of this exploited a novel CVE. The primitive is decades-old: Python's
pickle module is documented as unsafe on untrusted input, and PyTorch
picked pickle as its default weight format. Hugging Face allowed
pickle-format models on the hub, so 100 uploaded models simply took the
opening. The hub responded with a broader enforcement pass, added scanner
coverage, and pulled flagged repositories. The underlying execution
primitive remains part of the default model format, and new malicious
uploads land periodically.
How it propagated
The mechanic is pickle deserialisation. PyTorch's default .bin /
.pt weight format serialises via pickle, and pickle.load will
execute arbitrary Python bytecode embedded in the stream as part of the
unpickling contract. A model author who wants code execution on the
developer's machine embeds a __reduce__ payload in the state dict; the
developer runs the standard four lines to try the model; code runs under
whatever privileges the notebook process has. Safer formats
(safetensors, ONNX with strict loaders) sidestep the issue but are not
yet universal.
What Drig sees
The local_model collector walks known model cache paths:
~/.cache/huggingface/hub, ~/.ollama/models, and LM Studio's per-user
model directory. It streams model ID, revision/hash, size, and format.
The signed catalog carries flagged model IDs and revisions; retro-match
surfaces exposed developer machines and identifies which models to
quarantine.
Rotation checklist
- Remove flagged models from every enrolled developer machine. Clear the cache path and any project-local copies.
- Rotate cloud credentials and shell history-exposed tokens on machines that ran the flagged models.
- Where practical, migrate loading paths to
safetensorsso future compromises fail closed. - Audit any downstream artefacts produced during the exposure window.
Local models are a surface that grew from marginal to mainstream in under eighteen months. The exposure engine has to walk it.