Skip to content
Now taking design partners.Roadmap →

hugging face · 2024

Hugging Face malicious models

JFrog researchers identified roughly 100 malicious models hosted on Hugging Face in February 2024. About 25 of them delivered unsafe-deserialisation payloads that executed on model load via Python's pickle __reduce__ path.

Run this campaignScale: ~100 backdoored models, ~25 unsafe-deserialisation payloads

What happened

On February 27 2024, security researchers at JFrog disclosed that they had identified roughly 100 malicious models hosted on Hugging Face's model hub. About 25 of them carried unsafe-deserialisation payloads. Code that executed at model-load time as soon as a developer ran the standard torch.load() or AutoModel.from_pretrained() call to try the model.

None of this exploited a novel CVE. The primitive is decades-old: Python's pickle module is documented as unsafe on untrusted input, and PyTorch picked pickle as its default weight format. Hugging Face allowed pickle-format models on the hub, so 100 uploaded models simply took the opening. The hub responded with a broader enforcement pass, added scanner coverage, and pulled flagged repositories. The underlying execution primitive remains part of the default model format, and new malicious uploads land periodically.

How it propagated

The mechanic is pickle deserialisation. PyTorch's default .bin / .pt weight format serialises via pickle, and pickle.load will execute arbitrary Python bytecode embedded in the stream as part of the unpickling contract. A model author who wants code execution on the developer's machine embeds a __reduce__ payload in the state dict; the developer runs the standard four lines to try the model; code runs under whatever privileges the notebook process has. Safer formats (safetensors, ONNX with strict loaders) sidestep the issue but are not yet universal.

What Drig sees

The local_model collector walks known model cache paths: ~/.cache/huggingface/hub, ~/.ollama/models, and LM Studio's per-user model directory. It streams model ID, revision/hash, size, and format. The signed catalog carries flagged model IDs and revisions; retro-match surfaces exposed developer machines and identifies which models to quarantine.

Rotation checklist

  • Remove flagged models from every enrolled developer machine. Clear the cache path and any project-local copies.
  • Rotate cloud credentials and shell history-exposed tokens on machines that ran the flagged models.
  • Where practical, migrate loading paths to safetensors so future compromises fail closed.
  • Audit any downstream artefacts produced during the exposure window.

Local models are a surface that grew from marginal to mainstream in under eighteen months. The exposure engine has to walk it.