Kestrel

Kestrel is a compact local classifier for context-free cyber and system risk in individual Bash tool calls.

Release

Version 0.1.0 publishes the portable JSON model artifact used for the Kestrel evaluation. This repository does not include training code, training data, or a detailed training recipe.

File	Description
classifier.json	Portable Kestrel v0.1.0 model artifact
config.json	Model manifest with the artifact filename, version, and SHA-256
SHA256SUMS	Integrity checksums for the model artifact and manifest

The model is custom and is not a Transformers checkpoint. Download the artifact with huggingface_hub:

import hashlib
import json
from pathlib import Path

from huggingface_hub import snapshot_download

model_dir = Path(snapshot_download(
    repo_id="kontext-security/Kestrel",
    allow_patterns=["config.json", "classifier.json"],
))
config = json.loads((model_dir / "config.json").read_text())
model_path = str(model_dir / config["artifact"]["filename"])

if hashlib.sha256(Path(model_path).read_bytes()).hexdigest() != config["artifact"]["sha256"]:
    raise ValueError("Kestrel artifact checksum mismatch")

config.json is a manifest for this custom JSON model. The example fetches both files from the same repository revision and checks the artifact against the manifest. Fetching the manifest also enables Hugging Face's standard download statistics. Existing code and direct links that fetch only classifier.json remain valid, but those requests are not included in the standard download count.

Evaluation

On the published ShellRisk-Bench v0.1 test split, Kestrel reaches 0.947 precision, 0.922 recall, and 0.934 F1. The test split is a same-source, in-distribution holdout; it is not evidence of transfer to a novel command dialect.

See ShellRisk-Bench for the fixed split, evaluation methodology, prompts, scorer, aggregate results, and per-example Kestrel verdicts. The dataset is available at kontext-security/ShellRisk-Bench.

Scope and limitations

Kestrel evaluates one submitted Bash command without user, task, or session context. It does not infer intent or make a complete authorization decision. It is intended as one signal layered underneath deterministic protections and runtime authorization policy, not as the sole control for executing commands.

Try it

Kestrel is available as part of Kontext, where it evaluates the cyber risk of agent tool calls locally. Learn more and try it at kontext.security.

Integrity

SHA-256  1df8b3e5f2bfc4e1fe95230ee9b3d37f63aaa461a8551982fcc0dd7103c8221b  classifier.json
SHA-256  3eb80e6744370a4c34743b14698eaa326121aa36f8f8e2234b7149223d63190a  config.json

License

The artifact is published with license: other. The public benchmark sources retain their own upstream terms; review the dataset provenance before reuse or redistribution.