# SYN-ML-M01 — Manufacturing ML workbench

**SYNTHETIC DEMONSTRATION.** All batch records, measurements, mechanisms, costs and prospective examples are fictional. No employer/customer data, real customer outcomes, production integrations, approvals or confidential information are represented. Models are actually trained offline on the generated history. This is not a claim of real operational effectiveness.

## Problem, scope and responsibility

Prioritize additional dimensional inspection for an upcoming SYN-VS25 valve batch at M1 / SYN-ST-M01 / SYN-MC-01 in SYN-OPS-01. The label is **at least one of 24 diameters strictly outside 24.90–25.10 mm**. Predicted batch probability is not a unit defect rate. The independently generated learning history contains 720 batches and 17,280 piece inspections. It is not the eight-skid demand plan, a throughput claim, supplier inventory, the original 90-observation SPC ledger, or the 32-cut factorial experiment.

The portfolio demonstration covers synthetic data design, prediction cutoff, feature definitions, offline training, model comparisons, evaluation, browser inference and inspectable decisions. Professional records and personal AutoOps AI remain separate. Recommendation authority is limited to inspection support; standard inspection, containment, component disposition and equipment acceptance continue independently.

## Architecture

Next.js static export; React/TypeScript UI; local SVG charts; small committed JSON model artifacts; pure browser inference. There is no remote inference API, external model service, training in the browser or required Python runtime in the deployed Site.

- `scripts/train-manufacturing-ml.py`: NumPy generator, scikit-learn fitting/selection and deterministic artifact export.
- `data/factory/ml/basis.ts`: identity, feature units, editing bounds and cutoff.
- `data/factory/ml/models.json`: fitted medians, missing-indicator feature indices, training scaling statistics, logistic coefficients/intercept, tree node arrays, prevalence and frozen reference policy.
- `data/factory/ml/evaluation.json`: validation/holdout features, labels and scores; upcoming unobserved examples; candidate selection grid and frozen evaluation.
- `lib/factory/ml/inference.ts`: imputation, indicators, standardization, logistic sigmoid, tree traversal, extrapolation/missingness notices and noncausal explanations.
- `lib/factory/ml/evaluation.ts`: confusion counts, precision/recall, Brier, log loss, five fixed calibration bins and threshold/cost exploration.
- `components/factory/MlWorkbench.tsx` and `MlCharts.tsx`: prediction, input editing, results, reliability/threshold/chronology plots and equivalent numeric tables.
- `lib/factory/ml-session.ts`: versioned ML preferences. Factory session schema 5 migrates schemas 1–4, retaining previous work. Tab storage remains optional, with memory fallback; reset ML changes only ML state.

The shared factory session imports only basis/small fitted artifacts. Full history/piece CSVs and evaluation are not dependencies of every quality page. The detailed decision export includes the model, feature vector, probability, policy, assumptions, warnings, frozen results and historical outcome only for an unchanged heldout record. Editing any feature creates a visitor scenario with unknown outcome. Historical evaluation always uses its original inputs.

## Prediction cutoff and feature definitions

All inputs must be recorded **before batch machining begins**:

| Feature | Unit | Meaning |
|---|---|---|
| toolCuts | cuts | Cumulative completed cuts since tool replacement |
| feedMmRev | mm/rev | Commanded feed for the upcoming batch |
| speedMMin | m/min | Commanded cutting speed |
| coolantLMin | L/min | Pre-start flow reading; can be missing |
| coolantTempC | °C | Pre-start temperature reading; can be missing |
| setupOffsetUm | µm | Commanded preset, not a measured first-off result |
| hardnessHB | HB | Incoming material hardness recorded before machining; can be missing |

Final diameters, nonconforming counts, target, rework/NCR/FAT disposition, sequence, split and hidden-condition metadata are excluded. No label-derived aggregate is an input. Missing features are replaced by training medians, with separate missing indicators. Visitor bounds are conceptual input limits, not machine qualification; outside observed training ranges is labeled extrapolation.

## Generator and changed condition

Version `ml-1.0`; NumPy default_rng seed **581037**. Feed is sampled from .12/.16/.20, speed from 120/150/180, flow nominally from 6/8/10 with reading noise. Coolant temperature, material hardness, setup preset, latent batch offset and piece errors vary. Tools are replaced every 6–10 generated batches. Batch sequence is hypothetical chronology without real dates or a production-rate claim.

The fictional generation equations (µm) use `A=(feed-.16)/.04`, `B=(speed-150)/30`, `C=(actualFlow-8)/2`, `wear=toolCuts/240`:

```
mean = 15 + 50*wear + 12*A + 9*B - 12*C
       + .9*(temperature-24) + setupOffset + .12*(hardness-210)
       + 24*(A>.5 and B>.5 and actualFlow<8)
       + 6*Z_batch + 18*changed
sigma = 10 + 4*wear
diameter_mm = round(25 + (mean + sigma*Z_piece)/1000, 5)
```

Batch 625 introduces an unobserved +18 µm offset and flow over-reading of 1.2 L/min: actualFlow becomes recordedFlow minus 1.2. Missingness rises too. Flow/temperature/hardness baseline missing probabilities are .06/.04/.03; each increases .03 after the change. Inputs are made missing after latent measurements generate outcomes. Measurement values and labels are generated before any model selection. The change was included in the defined generator; the model cannot observe its flag.

## Chronological training and decisions

| Split | Batches | Pieces | Positive batches | Missing input cells |
|---|---:|---:|---:|---:|
| Training / 1–432 | 432 | 10,368 | 26 | 69 |
| Validation / 433–576 | 144 | 3,456 | 5 | 14 |
| Final holdout / 577–720 | 144 | 3,456 | 23 | 18 |

All pieces stay in their parent batch. No random split; no holdout or validation preprocessing fit. The same simulated machine/tool dynamics recur across chronological batches, so this is neither an unseen-machine test nor an independent-factory test.

1. Fit `SimpleImputer(strategy="median", add_indicator=True)` on training only.
2. Fit `StandardScaler` on the training imputed matrix only. Used by logistic regression; tree uses the imputed matrix directly.
3. Baseline is constant training prevalence: 26/432 = 6.0185%.
4. Fit L2 logistic candidates C=.1,1,10 (lbfgs, max_iter=3000) and trees with depths 3/4 and minimum leaves 16/32. Trees are justified by the defined nonlinear feature interaction; no guarantee they outperform logistic.
5. Within each family choose minimum validation Brier; then compare against the prevalence baseline. No refit on validation or test.
6. Choose reference threshold on validation over .02–.98 in steps of .02 using flagged×$18 + missed-positive×$180. Lower threshold wins a tie. Model selection and threshold selection share a small validation set with only five positives; selection is uncertain.
7. Evaluate the frozen policy on the holdout once. Post-holdout visitor exploration never changes the benchmark or claims fresh generalization evidence.

Selected logistic C=1.0; tree depth=3, minimum leaf=32. Default model logistic; default threshold **20%**.

## Measurable synthetic results

At the same frozen 20% threshold, 144 holdout batches / 23 positive:

| Model | TP / FP / FN / TN | Precision | Recall | Brier | Log loss | Assumed cost proxy |
|---|---|---:|---:|---:|---:|---:|
| Prevalence | 0 / 0 / 23 / 121 | Undefined | 0% | .1441 | .5010 | $4,140 |
| Logistic | 9 / 1 / 14 / 120 | 90% | 39.1% | .1054 | .4076 | $2,700 |
| Tree | 9 / 5 / 14 / 116 | 64.3% | 39.1% | .1178 | .8379 | $2,772 |

Logistic validation recall was 5/5 with 5 false positives and Brier .0148. Holdout before the change detected 3/3 positives among 48 batches; after the change detected only 6/20 among 96 batches, with 14 missed flags. Condition comparison is descriptive, not a causal estimate or confidence interval. Every error is inspectable.

Precision=TP/(TP+FP), undefined when no batches flagged. Recall=TP/(TP+FN). Brier is mean squared probability error. Log loss uses probabilities clipped to double-precision epsilon. Five fixed bins cover [0,.2), [.2,.4), [.4,.6), [.6,.8), [.8,1]; display mean prediction, observed positive fraction and batch count. Empty bins are undefined, not zero. Small bins and few positives limit calibration conclusions; no calibration model is fitted to the holdout. There is no cross-validation, confidence-interval claim or real-world performance estimate.

Costs are editable assumptions, not savings or ROI. Additional inspection is assumed perfect for flagged batches; no detection credit is given to standard inspection. A false negative means the model missed a positive batch, not that defective product escaped. Fixed overhead, customer consequences, actual disposition and measurement uncertainty are omitted. Threshold charts expose inspection workload and missed flags; adjusting these after seeing outcomes is exploratory.

## Run and reproduce

Python 3.12.14 was used. Install the exact packages in `requirements-ml.txt` (NumPy 2.3.5, SciPy 1.16.3, scikit-learn 1.7.2, joblib 1.6.0, threadpoolctl 3.7.0, cloudpickle 3.1.2). From the source checkout:

```sh
python3 -m venv .venv-ml
. .venv-ml/bin/activate
python -m pip install -r requirements-ml.txt
OPENBLAS_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 OMP_NUM_THREADS=1 python scripts/train-manufacturing-ml.py
pnpm check:ml
pnpm typecheck
pnpm build
```

The standalone pipeline download can be reproduced by placing it in `scripts/` under a new directory and putting `requirements-ml.txt` at that directory's root. It creates `data/factory/ml/` and `public/documents/engineering-lab/ml/`; the full UI source is in the Site project. `.venv-ml/` must stay out of source control. A fixed stack/platform and seed reproduce the recorded artifact hashes; minor cross-platform floating-point differences can occur.

The dataset manifest records SHA256 checksums of batch history, piece data, fitted parameters, predictions and evaluation. `reference-predictions.json` contains independently computed scikit-learn probabilities for all 720 batches. `scripts/verify-ml.ts` checks all 2,160 browser scores to 1e-12 tolerance, independent target derivation, preprocessing, metrics, split/cutoff, validation selection, threshold cost arithmetic, migration and quality independence. Training is not part of the Site build; committed artifacts make deployment deterministic.

## Lessons and limits

The model can prioritize investigation but cannot validate a physical part. Hidden process changes and scarce positive cases break an apparently promising validation result. Linear contributions and tree paths are associations, not root causes. This is a supervised classification demonstration, not an LLM, vision system, live factory integration or production monitoring service.

A real trial would require representative permissioned records, measurement-system validation, reliable pre-start timestamps, station/material diversity, prospective evaluation, drift monitoring, approved inspection plans and accountable quality decisions. Retraining is deliberately absent from this phase. Simulation and broader optimization remain later phases.

Primary method references: [scikit-learn common pitfalls](https://scikit-learn.org/stable/common_pitfalls.html), [probability calibration](https://scikit-learn.org/stable/modules/calibration.html), [LogisticRegression 1.7](https://scikit-learn.org/1.7/modules/generated/sklearn.linear_model.LogisticRegression.html), [DecisionTreeClassifier 1.7](https://scikit-learn.org/1.7/modules/generated/sklearn.tree.DecisionTreeClassifier.html), [SimpleImputer 1.7](https://scikit-learn.org/1.7/modules/generated/sklearn.impute.SimpleImputer.html).