giglez/VERIFICATION.md

# GigLez — Deployable-Artifact Verification

**Purpose:** prove that the *thing you actually ship* — the Docker container running
`src/api/main_simple.py` — serves the **full user journey**, end to end, the way a real
user experiences it. Unit tests and in-process checks passing is **not** the same as the
shipped app working. This doc is self-contained: you can clear context, come back, and run
it top to bottom.

Last grounded against the code on **2026-07-20** (branch `master`).

---

## 0. What "the deployable artifact" actually is

| Layer | What it is | Where |
|-------|-----------|-------|
| Container | `python:3.10-slim`, non-root, torch-free (417 MB) | `Dockerfile` |
| Process | `uvicorn src.api.main_simple:app` on `0.0.0.0:8000` | `docker-compose.yml` |
| Served UI | One SPA page + 6 JS modules | `templates/index.html`, `static/js/{detail-modal,map,upload,search,stats,main}.js` |
| API | FastAPI JSON endpoints (see §3) | `main_simple.py` |
| Data store | JSON file `{captures:[...], upload_counter, last_updated}` on a **named volume** `giglez_data:/app/data` | `data/captures_simple.json` |
| ML model | `models/category_classifier.joblib` (sklearn 1.6.1 / numpy 2.2.6 — pins must match) | baked into image |

**Runtime external dependencies (important):** the UI pulls Leaflet, Leaflet.markercluster,
Leaflet.heat, and Chart.js from **unpkg/jsdelivr CDNs**, and map tiles from the **OSM tile
server**. The artifact is **not self-contained for offline use** — an air-gapped host will
render a broken map. If offline deploy is ever a requirement, vendor these assets locally.

---

## 1. The full user journey (the 9 things to prove)

```
1. BUILD    docker compose up --build → container reports healthy
2. LOAD     GET / returns the SPA; all 6 JS + CSS assets load; no console errors
3. MAP      Leaflet map renders; existing captures show as markers/clusters; stat counters populate
4. UPLOAD   user picks a .sub, sets GPS (or "use my location"), submits → progress → result card with a device match
5. IDENTIFY the uploaded capture gets a device_name + device_category + confidence (parse→match→ML ran)
6. PLOT     the new capture appears on the map at its GPS
7. SEARCH   query by protocol / frequency / geo returns the capture
8. STATS    counters + distributions reflect the new data
9. PERSIST  restart the container → data survives (named volume did its job)
```

Verification is layered so a failure tells you *where* it broke:

- **§2 Layer 0** — artifact builds & boots
- **§3 Layer 1** — API contract (deterministic curl/jq) → proves the backend serves the journey
- **§4 Layer 2** — browser/UI (Playwright) → proves the *actual UI a user touches* works
- **§5 Layer 3** — persistence across restart
- **§6 Layer 4** — regression guardrails (things fixed recently that must stay fixed)

---

## 2. Layer 0 — Build & boot the real artifact

```bash
cd ~/coding/security/giglez

# Build + run exactly what ships
docker compose up --build -d

# Wait for healthy (compose healthcheck hits /health). Give it ~15s.
docker compose ps            # STATUS should show "healthy"
docker inspect --format '{{.State.Health.Status}}' $(docker compose ps -q giglez)
```

**PASS:** container status = `healthy`.
**If it fails:** `docker compose logs giglez` — look for import errors (usually a pin
mismatch: the joblib model needs sklearn 1.6.1 / numpy 2.2.6; `requirements-prod.txt`, not
the stale root `requirements.txt`).

Set a base URL for the rest of the doc:

```bash
BASE=http://localhost:8000
```

> **Caveat when running inside Claude Code's shell:** the `rtk` hook intercepts *piped*
> `curl` and returns a fake compressed body like `{"ok":true,"top":[]}`. Always write the
> response to a file first (`curl -s "$BASE/..." -o /tmp/r.json`) then inspect with
> `jq . /tmp/r.json`. In a normal human terminal this caveat does not apply.

---

## 3. Layer 1 — API contract checks (deterministic)

These prove the backend serves stages 2–8 without a browser. Copy-paste each block; the
expected shape is shown.

### 3.1 Health + API banner + SPA HTML

```bash
curl -s "$BASE/health" -o /tmp/h.json;      jq . /tmp/h.json
# expect: {"status":"healthy","database":"not_connected","mode":"simple"}

curl -s "$BASE/api" -o /tmp/api.json;        jq .status /tmp/api.json
# expect: "operational"

curl -s "$BASE/" -o /tmp/index.html
grep -c 'id="map"' /tmp/index.html          # expect: 1  (SPA shell served)
```

### 3.2 Static assets the page actually loads (stage 2)

```bash
for f in detail-modal map upload search stats main; do
  code=$(curl -s -o /dev/null -w '%{http_code}' "$BASE/static/js/$f.js")
  echo "$f.js -> $code"      # every one must be 200
done
curl -s -o /dev/null -w 'main.css -> %{http_code}\n' "$BASE/static/css/main.css"
```

**PASS:** all `200`. A `404` here = broken page for real users.

### 3.3 Baseline data snapshot (stages 3 & 8)

```bash
curl -s "$BASE/api/v1/stats/summary" -o /tmp/s0.json
jq '{total_captures, unique_devices, data_sources}' /tmp/s0.json
BASELINE=$(jq .total_captures /tmp/s0.json); echo "baseline captures = $BASELINE"

curl -s "$BASE/api/v1/query/captures?limit=1000" -o /tmp/q0.json
jq '.total' /tmp/q0.json                     # should equal $BASELINE
```

### 3.4 Upload → identify → persist (stages 4, 5, 9-write)

A minimal, self-contained RAW `.sub` (no dependency on the big dataset, which is *excluded*
from the image):

```bash
cat > /tmp/sample.sub <<'SUB'
Filetype: Flipper SubGhz RAW File
Version: 1
Frequency: 433920000
Preset: FuriHalSubGhzPresetOok650Async
Protocol: RAW
RAW_Data: 350 -350 350 -700 700 -350 350 -350 700 -700 350 -350 350 -700 700 -350 350 -700 350 -350 700 -350 350 -700 350 -350 350 -700 700 -700 350 -350
SUB

MANIFEST='{"captures":[{"filename":"sample.sub","latitude":47.6062,"longitude":-122.3321,"timestamp":"2026-07-20T12:00:00Z"}]}'

curl -s -X POST "$BASE/api/v1/captures/upload" \
  -F "files=@/tmp/sample.sub" \
  -F "manifest=$MANIFEST" -o /tmp/up.json

jq '{success, total_successful, name:.successful[0].device_name,
     category:.successful[0].device_category,
     conf:.successful[0].match_confidence,
     id:.successful[0].id}' /tmp/up.json
```

**PASS:** `success=true`, `total_successful=1`, and `device_name` / `device_category` /
`id` are non-null (parse → match → ML category all ran). Note the returned `id`.

### 3.5 New capture is queryable & plottable (stages 6 & 8)

```bash
curl -s "$BASE/api/v1/stats/summary" -o /tmp/s1.json
echo "after = $(jq .total_captures /tmp/s1.json)  (expect baseline+1 = $((BASELINE+1)))"

# The capture carries GPS so the map can plot it:
NEWID=$(jq '.successful[0].id' /tmp/up.json)
curl -s "$BASE/api/v1/captures/$NEWID" -o /tmp/det.json
jq '.capture | {id, latitude, longitude, device_category}' /tmp/det.json
```

**PASS:** count = baseline+1; the capture has numeric `latitude`/`longitude`.

### 3.6 Export (bonus)

```bash
curl -s "$BASE/api/v1/export" -o /tmp/export.ndjson
head -1 /tmp/export.ndjson | jq .            # each line is one capture record
```

---

## 4. Layer 2 — Browser / UI checks (the real user journey)

The API can be perfect while the UI is broken (bad JS, CDN blocked, map never inits). This
layer drives a **real headless browser** through the DOM. Node Playwright 1.58.2 is present.

Run the bundled script:

```bash
cd ~/coding/security/giglez
BASE=http://localhost:8000 node scripts/verify_ui.mjs
```

It performs, against real element IDs from `index.html`:

1. `goto /`, collect **console errors** and **failed network requests**.
2. Assert the Leaflet map container `#map` exists and Leaflet initialised (`.leaflet-container`).
3. Read the `#total-captures` / `#unique-devices` counters.
4. Switch to the Upload section (`a[href="#upload"]`), set `#file-input` to `/tmp/sample.sub`,
   fill `#default-lat` / `#default-lon`, click the **Upload** button (`onclick="uploadFiles()"`).
5. Wait for `#upload-results` to become visible and assert it shows a device match.
6. Re-query `/api/v1/query/captures` and assert the total incremented.
7. Save screenshots to `./verify-shots/` at each step.

**PASS criteria (the script prints a final PASS/FAIL and exits non-zero on failure):**
- 0 console errors, 0 failed asset requests
- `.leaflet-container` present (map actually rendered)
- upload results panel shows a device name + category
- capture total incremented by 1

If you'd rather check by eye: open `http://localhost:8000` in a browser, open DevTools
Console (should be clean), confirm the map tiles draw, then walk Upload → Search yourself.

---

## 5. Layer 3 — Persistence across restart (stage 9)

The whole point of the named volume: a redeploy/restart must not lose user data.

```bash
BEFORE=$(curl -s "$BASE/api/v1/stats/summary" | jq .total_captures 2>/dev/null); echo "before=$BEFORE"
docker compose restart giglez
sleep 12
AFTER=$(curl -s "$BASE/api/v1/stats/summary" -o /tmp/sa.json && jq .total_captures /tmp/sa.json)
echo "after=$AFTER"     # must equal before
```

**PASS:** `after == before`. If it resets, the volume mount or `save_captures()` path is
wrong and every restart wipes contributions.

> Harder test (proves the volume, not just the file): `docker compose down` (keeps named
> volumes) then `docker compose up -d` and re-check the count. Use `down -v` only when you
> intend to wipe the store.

---

## 6. Layer 4 — Regression guardrails (recently fixed — must stay fixed)

### 6.1 BinRAW captures produce a category (commit `4396c74`)
BinRAW `.sub` used to yield zero identification. Upload one and confirm a category comes back:

```bash
cat > /tmp/binraw.sub <<'SUB'
Filetype: Flipper SubGhz RAW File
Version: 1
Frequency: 433920000
Preset: FuriHalSubGhzPreset2FSKDev476Async
Protocol: BinRAW
Bit: 48
TE: 400
Bit_RAW: 48
Data_RAW: A9 3C 5F 12 87 D4
SUB
curl -s -X POST "$BASE/api/v1/captures/upload" \
  -F "files=@/tmp/binraw.sub" \
  -F 'manifest={"captures":[{"filename":"binraw.sub","latitude":47.61,"longitude":-122.33,"timestamp":"2026-07-20T12:05:00Z"}]}' \
  -o /tmp/b.json
jq '.successful[0] | {device_category, match_confidence}' /tmp/b.json
```
**PASS:** `device_category` is non-null.

### 6.2 Per-device category shows the device's OWN catalog category (commit `41850b5`)
A named device must not be mislabeled with the ML overall call (e.g. "Acurite … → Fan
Controller"). Inspect the match list from §3.4's upload:

```bash
jq '.successful[0] | {headline_category: .device_category,
     per_device: [.matched_devices[] | {device_name, category}]}' /tmp/up.json
```
**PASS:** each `matched_devices[].category` is that device's real catalog category; the
top-level `.device_category` may differ (it's the ML overall call) — that's expected.

---

## 7. Pass/fail scorecard

| # | Stage | Check | Pass condition |
|---|-------|-------|----------------|
| 0 | Build | §2 | container `healthy` |
| 2 | Load | §3.1–3.2 | SPA HTML + all 6 JS + CSS = 200 |
| 3 | Map data | §3.3 | stats + query return baseline |
| 4/5 | Upload+ID | §3.4 | success, device_name+category+id non-null |
| 6/8 | Plot+Stats | §3.5 | count = baseline+1, GPS present |
| 2–8 | **UI** | §4 | Playwright PASS, 0 console errors, map rendered, upload result shown |
| 9 | Persist | §5 | count unchanged after restart |
| R1 | BinRAW | §6.1 | category non-null |
| R2 | Category | §6.2 | per-device catalog category correct |

All green ⇒ the deployable artifact serves the full user journey and is safe to put in
front of a real user (beta). Remaining true prod gaps are **operational**, not functional:
public host, TLS/reverse proxy, and (optional) vendoring the CDN assets for offline use.

---

## 8. Teardown

```bash
docker compose down            # keep the named volume (data persists)
# docker compose down -v       # ONLY if you want to wipe the capture store
rm -f /tmp/sample.sub /tmp/binraw.sub /tmp/*.json /tmp/index.html /tmp/export.ndjson
```

**Do not** commit any captures created during verification — the JSON store is baked into
the image as seed data. If you uploaded against a local (non-Docker) `main_simple`, strip
your test records from `data/captures_simple.json` before committing (filter by the test
GPS you used).