pyronear/temporal-model

Temporal Smoke Model (bbox-tube-temporal)

Latest release: v0.4.0 — pin this revision for reproducibility, or omit revision= to always get the latest. All releases: the Files and versions tab.

A temporal wildfire-smoke classifier for short sequences of camera frames. A YOLO detector proposes boxes, boxes are linked across frames into temporal tubes, each tube's image patches are classified by a DINOv2 ViT + transformer head, and a logistic calibrator turns the tube logits into a calibrated probability and a keep/discard decision.

This repo ships a self-contained model.zip, versioned by HuggingFace revision/tag (v<version>), plus a torch-free model_onnx.zip derived from it (see ONNX runtime below). Each model.zip bundles everything needed to run:

file	purpose
manifest.yaml	version + provenance (train git SHA, backbone, detector)
yolo_weights.pt	the companion YOLO detector
classifier.ckpt	the temporal ViT classifier
config.yaml	inference + decision config
logistic_calibrator.json	the calibrated decision head

The model runs YOLO itself — you pass only raw frames, no detections.

ONNX runtime (no torch)

model_onnx.zip carries the same classifier exported to ONNX (fixed input patches[1, 20, 3, 224, 224] + mask[1, 20] → logit[1]), the inference config and the calibrator, but no YOLO: you supply the per-frame detections (e.g. from a detector already running on the device). It runs on numpy + pillow

• onnxruntime only — the intended runtime for edge devices such as a Raspberry Pi:

pip install "temporal-model-core[onnx] @ git+https://github.com/pyronear/temporal-model.git#subdirectory=core"

from pathlib import Path

from huggingface_hub import hf_hub_download
from temporal_model.core import Detection, FrameDetections
from temporal_model.core.onnx_model import OnnxTemporalModel

onnx_zip = hf_hub_download("pyronear/temporal-model", "model_onnx.zip", revision="v0.4.0")
model = OnnxTemporalModel.from_package(Path(onnx_zip))

frames = model.load_sequence(sorted(Path("my_sequence").glob("*.jpg")))
# One FrameDetections per frame, keyed by frame_id, boxes as normalized (cx, cy, w, h).
detections = {
    f.frame_id: FrameDetections(
        frame_idx=i, frame_id=f.frame_id, timestamp=f.timestamp,
        detections=[Detection(class_id=0, cx=0.5, cy=0.4, w=0.05, h=0.03, confidence=0.6)],
    )
    for i, f in enumerate(frames)
}
out = model.predict(frames, frame_detections=detections)

The manifest of model_onnx.zip records the SHA-256 of the model.zip it was exported from and the measured torch/ONNX logit difference.

Usage

Install the inference package (temporal_model.core):

pip install "temporal-model-core[torch] @ git+https://github.com/pyronear/temporal-model.git#subdirectory=core"

Download a versioned model.zip and run it on a temporally ordered sequence of frames:

from pathlib import Path

from huggingface_hub import hf_hub_download
from temporal_model.core.model import BboxTubeTemporalModel

# 1. Download a specific release (pin the revision).
model_zip = hf_hub_download("pyronear/temporal-model", "model.zip", revision="v0.4.0")

# 2. Temporally-ordered frames. Filenames carry timestamps
#    (<prefix>_<YYYY-MM-DDTHH-MM-SS>.jpg); the order is the time order.
frame_paths = sorted(Path("my_sequence").glob("*.jpg"))

# 3. Load (device=None → auto cuda → mps → cpu) and predict.
#    hf_hub_download returns a str, so wrap it in Path().
model = BboxTubeTemporalModel.from_package(Path(model_zip), device=None)
out = model.predict_sequence(frame_paths)

# 1. Binary verdict — the alarm decision.
print("is_smoke:         ", out.is_positive)

# 2. Calibrated smoke probability in [0, 1]: the strongest kept tube's calibrated
#    probability (0.0 when no tube was kept). Use is_positive for a yes/no alarm;
#    use the probability to rank/triage sequences or apply your own threshold.
kept = out.details.get("tubes", {}).get("kept", [])
probs = [t["probability"] for t in kept if t["probability"] is not None]
smoke_probability = max(probs) if probs else 0.0
print("smoke probability:", smoke_probability)
print("kept tubes:       ", len(kept))

predict_sequence(frame_paths) returns a TemporalModelOutput:

• is_positive: bool — the smoke verdict. True iff at least one tube's calibrated probability clears the packaged decision threshold.
• details: dict — per-tube logits, calibrated probabilities, bounding boxes, and the decision (aggregation, threshold). The top-level smoke probability is the maximum kept-tube probability (shown above).

is_positive and the probability are consistent: a positive sequence always has a kept tube whose probability is ≥ the threshold, so smoke_probability is ≥ the threshold whenever is_smoke is True.

Served API (Docker)

The same model is also served as a FastAPI image with the model.zip baked in (auto-uses the GPU with --gpus all):

docker run --gpus all -p 8000:8000 \
  -e TEMPORAL_API_S3_BUCKET=<frames-bucket> \
  -e TEMPORAL_API_S3_ENDPOINT_URL=<s3-endpoint> \
  pyronear/temporal-model-api:0.4.0
# POST /predict  {"frames": ["<s3-key>", ...]}      GET /health

POST /predict returns { "is_smoke": bool, "probability": float | null, "model": {...} }, where probability is the calibrated max kept-tube probability (null only for an uncalibrated model). Add ?verbose=true for a details block with the per-tube breakdown and decision config.

Provenance

Every model.zip manifest records how it was built — the training git SHA, the classifier backbone (vit_small_patch14_dinov2.lvd142m), and the exact companion detector (e.g. pyronear/yolo11s_nimble-narwhal_v6.0.0, verified by SHA-256). So a served model always traces back to its detector + training code.

Source & pipeline: https://github.com/pyronear/temporal-model