CLAUDE.md for Data Science Repos
Reproducibility rules for notebooks, pipelines, and experiments.
A CLAUDE.md template for data science and ML repos: environment discipline, notebook hygiene, data versioning rules, experiment tracking conventions, and the reproducibility standards that make 'it worked on my machine' impossible.
The template
# CLAUDE.md ## Project [Project] — [one sentence: what's being modeled/analyzed and for whom]. ## Environment ```bash uv sync # exact env from lockfile — NEVER pip install ad hoc uv run jupyter lab # notebooks uv run python -m src.train # pipelines run as modules, not scripts uv run pytest tests/ # tests for src/ code ``` ## Layout - `notebooks/` — exploration only; numbered `01_explore_users.ipynb`. Nothing in notebooks/ is imported by anything - `src/` — real code: features, training, evaluation. Anything used twice moves here from notebooks - `data/` — gitignored. `data/raw` is read-only truth; derived data goes in `data/processed` with the code that made it - `configs/` — experiment configs (yaml); `models/` — gitignored artifacts ## Rules ### Reproducibility - Every random operation gets an explicit seed from config — numpy, torch, sklearn, sampling - Raw data is READ-ONLY: never edit files in data/raw; transformations are code that writes to data/processed - Every derived dataset/model must be reproducible by a single command — if it takes manual steps, it doesn't exist - Record data snapshot/version in experiment config, not in your head ### Notebook hygiene - Restart-and-run-all must pass before a notebook is committed - No secrets, no absolute paths (use the project-root helper), outputs cleared for diffs except final figures - A notebook conclusion someone will rely on gets promoted: logic to src/ with a test, findings to a markdown report ### Experiments - One experiment = one config file = one tracked run (MLflow/W&B) — no untracked "quick tries" that become the production model - Evaluation metrics computed by shared code in src/eval.py, never re-implemented per notebook (metric drift is how teams fool themselves) - Compare models on the SAME frozen test split; the split definition lives in code ### Honesty - Report negative results — a model that doesn't beat the baseline is a finding - Baseline first: any new model must be compared against the dumb baseline (mean, majority class, last value) ## Verification Done means: pipeline runs end-to-end via its command, seeds set, run tracked, metrics from shared eval code, and restart-and-run-all passes on touched notebooks.
How to use
- 1Copy to your repo root and adapt the layout section to your structure.
- 2The 'metric drift' rule (shared eval code) is the highest-value line for teams — enforce it in review too.
- 3Pair with the Honest Data Analysis skill from our skills library for analysis-quality rules.
Examples
Promoting a notebook finding
Input
Agent finds a promising feature in an exploration notebook.
Output
Following the promotion rule: moves the feature computation to src/features.py with a unit test, adds it to the feature config, reruns the tracked experiment against the frozen split with the shared eval — instead of leaving a load-bearing result inside a notebook cell that only runs on one laptop.
Pro tips
- 'If it takes manual steps, it doesn't exist' is worth putting on a wall — it's the whole file in one line.
Frequently asked questions
We use conda/poetry instead of uv — does it matter?+
No — swap the environment block. What matters is the lockfile discipline: the agent never installs packages outside the declared environment.