Models

Port a model with an AI agent

A ready-to-paste brief for Claude Code, Codex or Cursor that turns a trained model into a PACS-AI container under human review.

The backend ships a machine-readable skill, .claude/skills/pacs-ai-model-mapping/SKILL.md, that Claude Code loads automatically, and AGENTS.md for Codex, Cursor and other agents. Porting becomes a supervised generation-and-review task: the agent drafts the five model-specific files, you review them and the benchmark predictions. The brief below gives the agent everything it needs in one message; fill in the angle-bracket fields for your model.

Before you start

A trained checkpoint in a Hugging Face repository (gated is fine) and a read token in hf_token.txt at the repository root, gitignored.
The training facts the agent cannot guess: dataset_mean / dataset_std, num_videos, stride, resize, the exact head names in the checkpoint, each head task and range, and the SeriesTime selection rule.
A GPU host with Docker, jq, membership in the heartwisehub Docker Hub organization if you will push there, and, if the agent should register the container, a PACS-AI admin account plus a filled scripts/.env.deploy (including FIREBASE_API_KEY).
Two or three benchmark studies with the prediction your training code produced for them, so the port can be verified numerically.

The brief (paste into your agent)

Copy, fill in the <fields>, and paste as the first message to Claude Code, Codex or Cursor opened in a clone of HeartWise-AI/pacs-ai-backend.

You are porting a trained model onto the PACS-AI backend as a new inference container.

Repository: HeartWise-AI/pacs-ai-backend (default branch master). Read AGENTS.md and
.claude/skills/pacs-ai-model-mapping/SKILL.md first and follow them exactly; also read
model-examples/README.md and model-template/README.md for the container contract.

Model to port
- Model name (registered name): <ModelName>
- Task and clinical output: <one sentence: what it predicts, units, thresholds>
- DICOM modality: <XA | US | CT | MRI | DX | CR>   (MRI, not MR, is the value the registry uses)
- Hugging Face repo id and checkpoint filename: <owner/repo>, <best_model_epoch_K.pt>
- Architecture: <CLIP-family VideoEncoder+MultiInstanceLinearProbing | other; describe>
- Normalization from training: dataset_mean=<...>, dataset_std=<...>
- Videos per study (num_videos): <N>; stride: <2>; resize: <224>
- Output heads (name in checkpoint -> task, range or threshold, unit): <head: regression 0-100 %, head2: binary 0.5>
- Series selection: sort by SeriesTime inside logic.py, take the first dicomUploadMax (= num_videos)
- Step-2 metadata (per-DICOM variables the user must tag before inference): <none, so [] | list>
- Output modes to support: <HTML, JSON>
- Benchmark studies and expected predictions: <study 1 -> value, study 2 -> value>

What to produce
1. Create model-examples/<ModelName>/ by copying model-examples/CathEF-CLIP as the template.
   Copy main.py, config.json, requirements.txt, Dockerfile, nginx.conf, supervisord.conf,
   docs/ (the Dockerfile copies it and main.py mounts it), models/*.py and utils/* verbatim.
2. Author the five model-specific files:
   - models/config.json: architecture, ModelStateDict.model_path = the checkpoint filename,
     dataset_mean/std, VideoMILWrapper num_videos/stride/resize from the values above.
   - models/class_mapping.json: one entry per head with head_dim, task (regression clamped to
     min/max, or binary_classification through a sigmoid), name, unit, threshold. Head names
     must match the checkpoint keys mil_model.module.heads.<name>.*.
   - data/model_info.json: modelId, modelName, version, modality, domainName, stage,
     dicomTargetLevel, dicomUploadMin 1, dicomUploadMax = num_videos, supportedDicomModalities,
     supportedDicomTags, supportedOutputModes, approve/reject feedback questionnaires, and
     supportedAdditionalMetadata exactly as specified above ([] means no Step-2 page). Leave
     onboardingModelQuestionnaires out unless I give you the clinical questions for it.
   - logic.py: subclass BasePredictionService; implement load_model, _run_inference,
     _postprocess, _handle_json_output, _handle_html_output; sort DICOMs by SeriesTime and
     truncate to dicomUploadMax inside logic.py; build the boolean video_mask for real videos
     and pass it to the model (the padding-mask contract from PR #242); if Step-2 metadata is
     non-empty, also implement _filter_dicoms_with_metadata (runs only when additionalMetadata
     is supplied).
   - download_model.py: snapshot_download(repo_id="<owner/repo>") with the hf_token; do not
     leave the template repo id.
   Also write data/model_facts.json (Model Facts label) from the information above.
3. Verify before you report:
   - Load the checkpoint into the rebuilt model with 0 missing and 0 unexpected keys.
   - Padding-invariance test: with the real videos fixed, changing the zero-padding content
     must move every logit by exactly 0.0.
   - Reproduce the benchmark predictions locally with the same preprocessing and selection;
     a residual of about 0.01 is the DICOM-versus-mp4 decode path, anything larger means wrong
     selection or wrong mean/std, so fix it.
   - docker build --secret id=hf_token,src=./hf_token.txt -t heartwisehub/<modelname>:<version> .
     then start the container on pacs-net and call GET /inference/model-info and
     POST /inference/predict with one benchmark study.
4. Deliverables: the new directory, a short REPORT.md with the verification numbers
   (key-loading result, padding-invariance delta, benchmark reproduction table, model-info
   response), and the exact deploy command
   (DEFAULT_OUTPUT_MODE=<HTML|JSON> ./scripts/deploy-model.sh model-examples/<ModelName>
   --hf-token-file hf_token.txt). Do not run the deploy script yourself unless I say so.

Rules
- Do not commit weights, hf_token.txt, or any DICOM file. Do not touch other models.
- Ask me before inventing any clinical threshold, unit or normalization value.
- If a check fails, say so and stop; do not paper over it.

What you review when the agent is done

download_model.py points at your repository, not the template model.
models/config.json normalization and num_videos match training; class_mapping.json head names match the checkpoint keys and the report shows 0 missing / 0 unexpected keys.
logic.py sorts by SeriesTime, truncates to dicomUploadMax, and threads the video_mask; the padding-invariance delta in the report is exactly 0.0.
The benchmark reproduction table matches your training numbers within about 0.01.
data/model_info.json has the supportedAdditionalMetadata you asked for and the output modes you will register; remember the registered outputMode is chosen at deploy time.
No weights, token or DICOM files appear in git status.

The same brief works for non-CLIP models; replace the architecture and head instructions with your framework and keep the container contract, the SeriesTime rule and the verification steps. For an agent that can browse, DeepWiki holds an AI-generated, source-linked map of the backend that answers questions about the shared prediction framework and the model build process.