giglez/VERIFICATION.md

GigLez — Deployable-Artifact Verification

Purpose: prove that the thing you actually ship — the Docker container running src/api/main_simple.py — serves the full user journey, end to end, the way a real user experiences it. Unit tests and in-process checks passing is not the same as the shipped app working. This doc is self-contained: you can clear context, come back, and run it top to bottom.

Last grounded against the code on 2026-07-20 (branch master).

---

0. What "the deployable artifact" actually is

Layer	What it is	Where
Container	python:3.10-slim, non-root, torch-free (417 MB)	Dockerfile
Process	uvicorn src.api.main_simple:app on 0.0.0.0:8000	docker-compose.yml
Served UI	One SPA page + 6 JS modules	templates/index.html, static/js/{detail-modal,map,upload,search,stats,main}.js
API	FastAPI JSON endpoints (see §3)	main_simple.py
Data store	JSON file {captures:[...], upload_counter, last_updated} on a named volume giglez_data:/app/data	data/captures_simple.json
ML model	models/category_classifier.joblib (sklearn 1.6.1 / numpy 2.2.6 — pins must match)	baked into image

Runtime external dependencies (important): the UI pulls Leaflet, Leaflet.markercluster, Leaflet.heat, and Chart.js from unpkg/jsdelivr CDNs, and map tiles from the OSM tile server. The artifact is not self-contained for offline use — an air-gapped host will render a broken map. If offline deploy is ever a requirement, vendor these assets locally.

---

1. The full user journey (the 9 things to prove)

1. BUILD    docker compose up --build → container reports healthy
2. LOAD     GET / returns the SPA; all 6 JS + CSS assets load; no console errors
3. MAP      Leaflet map renders; existing captures show as markers/clusters; stat counters populate
4. UPLOAD   user picks a .sub, sets GPS (or "use my location"), submits → progress → result card with a device match
5. IDENTIFY the uploaded capture gets a device_name + device_category + confidence (parse→match→ML ran)
6. PLOT     the new capture appears on the map at its GPS
7. SEARCH   query by protocol / frequency / geo returns the capture
8. STATS    counters + distributions reflect the new data
9. PERSIST  restart the container → data survives (named volume did its job)

Verification is layered so a failure tells you where it broke:

• §2 Layer 0 — artifact builds & boots
• §3 Layer 1 — API contract (deterministic curl/jq) → proves the backend serves the journey
• §4 Layer 2 — browser/UI (Playwright) → proves the actual UI a user touches works
• §5 Layer 3 — persistence across restart
• §6 Layer 4 — regression guardrails (things fixed recently that must stay fixed)

---

2. Layer 0 — Build & boot the real artifact

cd ~/coding/security/giglez

# Build + run exactly what ships
docker compose up --build -d

# Wait for healthy (compose healthcheck hits /health). Give it ~15s.
docker compose ps            # STATUS should show "healthy"
docker inspect --format '{{.State.Health.Status}}' $(docker compose ps -q giglez)

PASS: container status = healthy. If it fails: docker compose logs giglez — look for import errors (usually a pin mismatch: the joblib model needs sklearn 1.6.1 / numpy 2.2.6; requirements-prod.txt, not the stale root requirements.txt).

Set a base URL for the rest of the doc:

BASE=http://localhost:8000

Caveat when running inside Claude Code's shell: the rtk hook intercepts piped curl and returns a fake compressed body like {"ok":true,"top":[]}. Always write the response to a file first (curl -s "$BASE/..." -o /tmp/r.json) then inspect with jq . /tmp/r.json. In a normal human terminal this caveat does not apply.

---

3. Layer 1 — API contract checks (deterministic)

These prove the backend serves stages 2–8 without a browser. Copy-paste each block; the expected shape is shown.

3.1 Health + API banner + SPA HTML

curl -s "$BASE/health" -o /tmp/h.json;      jq . /tmp/h.json
# expect: {"status":"healthy","database":"not_connected","mode":"simple"}

curl -s "$BASE/api" -o /tmp/api.json;        jq .status /tmp/api.json
# expect: "operational"

curl -s "$BASE/" -o /tmp/index.html
grep -c 'id="map"' /tmp/index.html          # expect: 1  (SPA shell served)

3.2 Static assets the page actually loads (stage 2)

for f in detail-modal map upload search stats main; do
  code=$(curl -s -o /dev/null -w '%{http_code}' "$BASE/static/js/$f.js")
  echo "$f.js -> $code"      # every one must be 200
done
curl -s -o /dev/null -w 'main.css -> %{http_code}\n' "$BASE/static/css/main.css"

PASS: all 200. A 404 here = broken page for real users.

3.3 Baseline data snapshot (stages 3 & 8)

curl -s "$BASE/api/v1/stats/summary" -o /tmp/s0.json
jq '{total_captures, unique_devices, data_sources}' /tmp/s0.json
BASELINE=$(jq .total_captures /tmp/s0.json); echo "baseline captures = $BASELINE"

curl -s "$BASE/api/v1/query/captures?limit=1000" -o /tmp/q0.json
jq '.total' /tmp/q0.json                     # should equal $BASELINE

3.4 Upload → identify → persist (stages 4, 5, 9-write)

A minimal, self-contained RAW .sub (no dependency on the big dataset, which is excluded from the image):

cat > /tmp/sample.sub <<'SUB'
Filetype: Flipper SubGhz RAW File
Version: 1
Frequency: 433920000
Preset: FuriHalSubGhzPresetOok650Async
Protocol: RAW
RAW_Data: 350 -350 350 -700 700 -350 350 -350 700 -700 350 -350 350 -700 700 -350 350 -700 350 -350 700 -350 350 -700 350 -350 350 -700 700 -700 350 -350
SUB

MANIFEST='{"captures":[{"filename":"sample.sub","latitude":47.6062,"longitude":-122.3321,"timestamp":"2026-07-20T12:00:00Z"}]}'

curl -s -X POST "$BASE/api/v1/captures/upload" \
  -F "files=@/tmp/sample.sub" \
  -F "manifest=$MANIFEST" -o /tmp/up.json

jq '{success, total_successful, name:.successful[0].device_name,
     category:.successful[0].device_category,
     conf:.successful[0].match_confidence,
     id:.successful[0].id}' /tmp/up.json

PASS: success=true, total_successful=1, and device_name / device_category / id are non-null (parse → match → ML category all ran). Note the returned id.

3.5 New capture is queryable & plottable (stages 6 & 8)

curl -s "$BASE/api/v1/stats/summary" -o /tmp/s1.json
echo "after = $(jq .total_captures /tmp/s1.json)  (expect baseline+1 = $((BASELINE+1)))"

# The capture carries GPS so the map can plot it:
NEWID=$(jq '.successful[0].id' /tmp/up.json)
curl -s "$BASE/api/v1/captures/$NEWID" -o /tmp/det.json
jq '.capture | {id, latitude, longitude, device_category}' /tmp/det.json

PASS: count = baseline+1; the capture has numeric latitude/longitude.

3.6 Export (bonus)

curl -s "$BASE/api/v1/export" -o /tmp/export.ndjson
head -1 /tmp/export.ndjson | jq .            # each line is one capture record

---

4. Layer 2 — Browser / UI checks (the real user journey)

The API can be perfect while the UI is broken (bad JS, CDN blocked, map never inits). This layer drives a real headless browser through the DOM. Node Playwright 1.58.2 is present.

Run the bundled script:

cd ~/coding/security/giglez
BASE=http://localhost:8000 node scripts/verify_ui.mjs

It performs, against real element IDs from index.html:

1. goto /, collect console errors and failed network requests.
2. Assert the Leaflet map container #map exists and Leaflet initialised (.leaflet-container).
3. Read the #total-captures / #unique-devices counters.
4. Switch to the Upload section (a[href="#upload"]), set #file-input to /tmp/sample.sub, fill #default-lat / #default-lon, click the Upload button (onclick="uploadFiles()").
5. Wait for #upload-results to become visible and assert it shows a device match.
6. Re-query /api/v1/query/captures and assert the total incremented.
7. Save screenshots to ./verify-shots/ at each step.

PASS criteria (the script prints a final PASS/FAIL and exits non-zero on failure):

• 0 console errors, 0 failed asset requests
• .leaflet-container present (map actually rendered)
• upload results panel shows a device name + category
• capture total incremented by 1

If you'd rather check by eye: open http://localhost:8000 in a browser, open DevTools Console (should be clean), confirm the map tiles draw, then walk Upload → Search yourself.

---

5. Layer 3 — Persistence across restart (stage 9)

The whole point of the named volume: a redeploy/restart must not lose user data.

BEFORE=$(curl -s "$BASE/api/v1/stats/summary" | jq .total_captures 2>/dev/null); echo "before=$BEFORE"
docker compose restart giglez
sleep 12
AFTER=$(curl -s "$BASE/api/v1/stats/summary" -o /tmp/sa.json && jq .total_captures /tmp/sa.json)
echo "after=$AFTER"     # must equal before

PASS: after == before. If it resets, the volume mount or save_captures() path is wrong and every restart wipes contributions.

Harder test (proves the volume, not just the file): docker compose down (keeps named volumes) then docker compose up -d and re-check the count. Use down -v only when you intend to wipe the store.

---

6. Layer 4 — Regression guardrails (recently fixed — must stay fixed)

6.1 BinRAW captures produce a category (commit 4396c74)

BinRAW .sub used to yield zero identification. Upload one and confirm a category comes back:

cat > /tmp/binraw.sub <<'SUB'
Filetype: Flipper SubGhz RAW File
Version: 1
Frequency: 433920000
Preset: FuriHalSubGhzPreset2FSKDev476Async
Protocol: BinRAW
Bit: 48
TE: 400
Bit_RAW: 48
Data_RAW: A9 3C 5F 12 87 D4
SUB
curl -s -X POST "$BASE/api/v1/captures/upload" \
  -F "files=@/tmp/binraw.sub" \
  -F 'manifest={"captures":[{"filename":"binraw.sub","latitude":47.61,"longitude":-122.33,"timestamp":"2026-07-20T12:05:00Z"}]}' \
  -o /tmp/b.json
jq '.successful[0] | {device_category, match_confidence}' /tmp/b.json

PASS: device_category is non-null.

6.2 Per-device category shows the device's OWN catalog category (commit 41850b5)

A named device must not be mislabeled with the ML overall call (e.g. "Acurite … → Fan Controller"). Inspect the match list from §3.4's upload:

jq '.successful[0] | {headline_category: .device_category,
     per_device: [.matched_devices[] | {device_name, category}]}' /tmp/up.json

PASS: each matched_devices[].category is that device's real catalog category; the top-level .device_category may differ (it's the ML overall call) — that's expected.

---

7. Pass/fail scorecard

#	Stage	Check	Pass condition
0	Build	§2	container healthy
2	Load	§3.1–3.2	SPA HTML + all 6 JS + CSS = 200
3	Map data	§3.3	stats + query return baseline
4/5	Upload+ID	§3.4	success, device_name+category+id non-null
6/8	Plot+Stats	§3.5	count = baseline+1, GPS present
2–8	UI	§4	Playwright PASS, 0 console errors, map rendered, upload result shown
9	Persist	§5	count unchanged after restart
R1	BinRAW	§6.1	category non-null
R2	Category	§6.2	per-device catalog category correct

All green ⇒ the deployable artifact serves the full user journey and is safe to put in front of a real user (beta). Remaining true prod gaps are operational, not functional: public host, TLS/reverse proxy, and (optional) vendoring the CDN assets for offline use.

---

8. Teardown

docker compose down            # keep the named volume (data persists)
# docker compose down -v       # ONLY if you want to wipe the capture store
rm -f /tmp/sample.sub /tmp/binraw.sub /tmp/*.json /tmp/index.html /tmp/export.ndjson

Do not commit any captures created during verification — the JSON store is baked into the image as seed data. If you uploaded against a local (non-Docker) main_simple, strip your test records from data/captures_simple.json before committing (filter by the test GPS you used).