Skip to content

Model Scanner Detection Categories and Severity Levels

Model scanner defines attacks by technique, providing an estimated severity and the rationality for classifying it with that severity.

Detection Category Estimated Severity Definition Rationality for Severity
Arbitrary Code Execution Critical Adversaries can inject malicious code into a model, which will be executed whenever the hijacked model is loaded into memory. This vulnerability can be used to exfiltrate sensitive data, execute malware (such as spyware or ransomware) on the machine, or run any kind of malicious scripts.
Expand for File Information
Model Format and File Extensions:
- Cloudpickle: .pkl, .pickle
- Dill: .dill
- GGUF: .gguf
- HDF5: .h5, .hdf5
- JobLib: .joblib
- Keras: .keras
- NeMo: .nemo
- Numpy: .npy, .npz
- Pytorch: .pt, .bin, pth, ckpt
- Pickle: .pkl
- R: .rds (plain and compressed)
- Skops: .skops
 Arbitrary code execution attacks are relatively easy to perform and may lead to critical outcomes such as execution of malicious code on an organization's computers.
Expand for More Information
Vulnerable Formats:
- CloudPickle
- Joblib
- Keras
- Nemo
- Pickle
- R
- skops

HiddenLayer Tech Blogs:
- R-bitrary Code Execution
- Security Advisory: 2024-06-skops
- Models are Code
- CWE-502

MITRE ATLAS
Command and Scripting Interpreter
- AML T0050
- AML TA0005
ML Supply Chain Compromise
- AML T0010
- AML TA0004
User Execution
- AML T0011
- AML TA0005

OWASP Top 10:
- ML06
- LLM05
Arbitrary Read Access High Adversaries can craft a malicious model that will exfiltrate sensitive data upon loading.
Expand for File Information
Model Format and File Extensions:
- ONNX: .onnx
 Arbitrary read access attacks are relatively easy to perform and may lead to critical outcomes such as an attacker exfiltrating sensitive data.
Expand for More Information
Vulnerable Formats:
- PMML
- SavedModel

HiddenLayer Tech Blogs:
- Models are Code

MITRE ATLAS
ML Supply Chain Compromise
- AML T0010
- AML TA0004

OWASP Top 10:
- ML06
- LLM05
Decompression Vulnerabilities High Adversaries can exploit vulnerabilities in popular compression formats to cause denial of service or leak sensitive data.
Expand for File Information
Model Format and File Extensions:
- Keras: .keras
- NeMo: .nemo
- Softensors: .safetensors
- Tensorflow: .savedmodel, .tf, .pb
- Zip: .zip
 Decompression vulnerabilities are relatively easy to exploit and may lead to high-impact outcomes such as denial of service, code execution, or data leakage.
Expand for More Information
Vulnerable Formats:
- PyTorch
- Tar
- Zip

MITRE ATLAS
ML Supply Chain Compromise
- AML T0010
- AML TA0004

OWASP Top 10:
- ML06
- LLM05
Denial of Service Medium Adversaries can craft a malicious model, or modify legitimately pre-trained model, in order to disrupt the system the model will be loaded on.
Expand for File Information
Model Format and File Extensions:
- Cloudpickle: .pkl, .pickle
- Dill: .dill
- HDF5: .h5, .hdf5
- JobLib: .joblib
- NeMo: .nemo
- Numpy: .npy, .npz
- Pytorch: .pt, .bin, pth, ckpt
- Pickle: .pkl
 Denial of service attacks are relatively easy to perform and may lead to disruption or degradation of service.
Expand for More Information
Vulnerable Formats:
- All model formats

MITRE ATLAS
ML Supply Chain Compromise
- AML T0010
- AML TA0004

OWASP Top 10:
- ML06
- LLM05
Directory Traversal Medium Adversaries can craft a malicious model, or modify legitimately pre-trained model, in order to gain unauthorised access to sensitive files on the system.
Expand for File Information
Model Format and File Extensions:
- ONNX: .onnx
 Directory traversal attacks are relatively easy to perform and may grant an attacker access to sensitive files on the file system.
Expand for More Information
Vulnerable Formats:
- ONNX

HiddenLayer Tech Blogs:
- ONNX Vulnerability Report

MITRE ATLAS
ML Supply Chain Compromise
- AML T0010
- AML TA0004

OWASP Top 10:
- ML06
- LLM05
Embedded Payloads Low Adversaries can embed malicious payloads (such as backdoors, coin miners, spyware, and ransomware) inside the model’s tensors. Such payloads can be injected in plain text, obfuscated, or embedded using steganography.
Expand for File Information
Model Format and File Extensions:
- HDF5: .h5, .hdf5
- Safetensors: .safetensors
 Malicious payloads can be embedded in ML models relatively easily; this may lead to malware components being distributed on an organization's computers.
Expand for More Information
Vulnerable Formats:
- All model formats

HiddenLayer Tech Blogs:
- Weaponizing Machine Learning Models with Ransomware
- Pickle Files

MITRE ATLAS
- ML Supply Chain Compromise
- AML T0010
- AML TA0004

OWASP Top 10:
- ML06
- LLM05
Graph Payload High Adversaries can inject a computational graph payload, introducing a secret attacker-controlled behavior into a pre-trained model.
Expand for File Information
Model Format and File Extensions:
- ONNX: .onnx
 Model backdooring may be relatively difficult to perform and can lead to critical outcomes such as biased or inaccurate output.
Expand for More Information
HiddenLayer Tech Blogs:
- Shadow Logic

Vulnerable Formats
- All model formats

MITRE ATLAS
Backdoor ML Model: Inject Payload
- AML T0018.001
- AML TA0006

OWASP Top 10:
- ML06
- LLM05
Network Requests High Adversaries can craft a malicious model that will make network requests upon loading.
Expand for File Information
Model Format and File Extensions:
- Cloudpickle: .pkl, .pickle
- Dill: .dill
- HDF5: .h5, .hdf5
- JobLib: .joblib
- NeMo: .nemo
- Numpy: .npy, .npz
- Pytorch: .pt, .bin, pth, ckpt
- Pickle: .pkl
 Network requests are relatively easy to perform and may be used to exfiltrate data, download payloads, or initiate command and control communications.
Expand for More Information
Vulnerable Formats:- CloudPickle
- Joblib
- Keras
- Nemo
- Pickle
- R
- skops

HiddenLayer Tech Blogs:
- Pickle Files

MITRE ATLAS
ML Supply Chain Compromise
- AML T0010
- AML TA0004

OWASP Top 10:
- ML06
- LLM05
Repository Sideloading Medium Adversaries can load code or model artifacts from an unexpected location, bypassing checks performed on the artifacts in the repository. Repository sideloading is an expected behavior allowed by Hugging Face; however, it can be abused to bypass security checks.
Expand for More Information
Vulnerable Formats
- JSON
Suspicious File Format Medium Adversaries can modify data structures and encodings in an attempt to evade detection.
Expand for File Information
Model Format and File Extensions:
- Cloudpickle: .pkl, .pickle
- Dill: .dill
- HDF5: .h5, .hdf5
- JobLib: .joblib
- NeMo: .nemo
- Numpy: .npy, .npz
- Pytorch: .pt, .bin, pth, ckpt
- Pickle: .pkl
 File format tampering is usually indicative of a targeted attack.
Expand for More Information
Vulnerable Formats:
- Pickle
- ProtoBuf

MITRE ATLAS
ML Supply Chain Compromise
- AML T0010
- AML TA0004
Suspicious Functions High The presence of these functions themselves is not inherently malicious, but they can be used in conjunction with other functions to create a malicious model.
Expand for File Information
Model Format and File Extensions:
- Cloudpickle: .pkl, .pickle
- Dill: .dill
- HDF5: .h5, .hdf5
- JobLib: .joblib
- NeMo: .nemo
- Numpy: .npy, .npz
- Pytorch: .pt, .bin, pth, ckpt
- Pickle: .pkl
 Functions can be used in conjunction with other functions to create a malicious model.
Expand for More Information
Vulnerable Formats:
- Pickle

MITRE ATLAS
ML Supply Chain Compromise
- AML T0010
- AML TA0004