giglez/docs/TESTING_STRATEGY.md

GigLez Testing Strategy - Phase 1 & 2 Validation

Branch: p1-p2-validation Created: 2026-01-12 Purpose: Comprehensive testing before deployment

---

Testing Pyramid

                    ┌─────────────┐
                    │   Manual    │  ← Termux workflow testing
                    │   Testing   │
                    └─────────────┘
                  ┌───────────────────┐
                  │   Integration     │  ← API endpoint testing
                  │     Tests         │
                  └───────────────────┘
              ┌─────────────────────────┐
              │      Unit Tests         │  ← Component testing
              │                         │
              └─────────────────────────┘

---

Phase 1: Unit Tests

1. GPS Validator Tests

File: tests/unit/test_gps_validator.py

Test Cases:

• ✅ Valid coordinates (NYC, London, Tokyo)
• ✅ Invalid latitude (> 90, < -90)
• ✅ Invalid longitude (> 180, < -180)
• ✅ Null Island detection (0.0, 0.0)
• ✅ Accuracy thresholds (< 50m pass, > 50m fail)
• ✅ High-quality detection (< 10m)
• ✅ Anonymization (precision reduction)
• ✅ Distance calculation (Haversine)

2. .sub File Parser Tests

File: tests/unit/test_sub_parser.py

Test Cases:

• ✅ Valid KEY format file
• ✅ Valid RAW format file
• ✅ Valid BinRAW format file
• ✅ Invalid file format (missing fields)
• ✅ Invalid frequency (out of range)
• ✅ Metadata extraction (protocol, frequency, modulation)
• ✅ Key data parsing (hex to bytes)
• ✅ Raw data parsing (timing array)

3. Storage Backend Tests

File: tests/unit/test_storage.py

Test Cases:

• ✅ LocalStorage: save file
• ✅ LocalStorage: retrieve file
• ✅ LocalStorage: file exists check
• ✅ LocalStorage: delete file
• ✅ LocalStorage: SHA256 path sharding
• ✅ LocalStorage: storage statistics
• ✅ S3Storage: mock save/retrieve (if boto3 available)

4. Database Models Tests

File: tests/unit/test_models.py

Test Cases:

• ✅ User model creation
• ✅ Session model creation
• ✅ Device model creation
• ✅ Capture model creation (with GPS validation)
• ✅ Capture deduplication (file_hash primary key)
• ✅ Relationships (user -> captures, session -> captures)
• ✅ to_dict() serialization

5. Configuration Tests

File: tests/unit/test_settings.py

Test Cases:

• ✅ Development mode detection
• ✅ Production mode detection
• ✅ Environment variable loading
• ✅ Settings validation (production checks)
• ✅ Computed properties (use_redis, use_celery)
• ✅ Database URL generation

---

Phase 2: Integration Tests

1. API Startup Tests

File: tests/integration/test_api_startup.py

Test Cases:

• ✅ Application starts successfully
• ✅ Database connection established
• ✅ PostGIS extension available
• ✅ Storage backend initialized
• ✅ Health endpoint responds
• ✅ OpenAPI docs available (dev mode)

2. Upload Endpoint Tests

File: tests/integration/test_upload.py

Test Cases:

• ✅ Upload single .sub file with manifest
• ✅ Upload multiple .sub files
• ✅ Duplicate file detection (same hash)
• ✅ Invalid manifest JSON (400 error)
• ✅ Missing manifest entry (file skipped)
• ✅ Invalid GPS coordinates (file rejected)
• ✅ Invalid .sub file format (file rejected)
• ✅ Session creation
• ✅ Session reuse (existing UUID)
• ✅ File stored in storage backend
• ✅ Capture record created in database
• ✅ Response format validation

3. Query Endpoint Tests

File: tests/integration/test_query.py

Test Cases:

• ✅ Get capture by file_hash
• ✅ 404 for non-existent capture
• ✅ Device listing (when implemented)
• ✅ Statistics endpoint (when implemented)

4. Storage Integration Tests

File: tests/integration/test_storage_integration.py

Test Cases:

• ✅ Upload → Storage → Retrieve (round trip)
• ✅ Multiple uploads (different files)
• ✅ File sharding verification
• ✅ Storage statistics accuracy

5. Database Integration Tests

File: tests/integration/test_database.py

Test Cases:

• ✅ Create tables (if not exists)
• ✅ Insert capture with GPS
• ✅ PostGIS geometry auto-population
• ✅ Spatial query (within radius)
• ✅ Trigger execution (session stats update)
• ✅ Deduplication (unique file_hash)

---

Phase 3: Manual Testing (Termux)

1. Environment Setup Tests

File: tests/manual/termux_setup.md

Test Cases:

• ✅ PostgreSQL installation
• ✅ Database creation
• ✅ Schema creation (no errors)
• ✅ PostGIS extension enabled
• ✅ Python dependencies installation
• ✅ Settings validation passes
• ✅ Storage directory creation

2. API Server Tests

File: tests/manual/termux_api.md

Test Cases:

• ✅ Server starts on localhost:8000
• ✅ Health check responds
• ✅ OpenAPI docs accessible
• ✅ CORS headers present
• ✅ Request logging works
• ✅ Exception handling (test invalid request)

3. Upload Workflow Tests

File: tests/manual/termux_upload.md

Test Cases:

• ✅ Create test .sub file
• ✅ Create test manifest
• ✅ Upload via curl
• ✅ Verify response (201, success message)
• ✅ Check file in storage directory
• ✅ Query capture from database
• ✅ Verify GPS coordinates
• ✅ Upload duplicate (verify deduplication)

4. Real-World Workflow Tests

File: tests/manual/termux_realworld.md

Test Cases:

• ✅ Capture actual signal with T-Embed (if available)
• ✅ Save .sub file to device
• ✅ Get GPS coordinates (termux-location)
• ✅ Create manifest from GPS data
• ✅ Upload to API
• ✅ Verify in database
• ✅ Query back by hash
• ✅ Verify storage location

---

Test Data & Fixtures

1. Sample .sub Files

Directory: tests/fixtures/sub_files/

Files Needed:

• valid_key_433mhz.sub - Standard KEY format (Princeton)
• valid_raw_315mhz.sub - RAW format with timing data
• valid_binraw_868mhz.sub - BinRAW format
• invalid_missing_freq.sub - Missing frequency field
• invalid_bad_format.sub - Corrupted file

2. Sample Manifests

Directory: tests/fixtures/manifests/

Files Needed:

• valid_single.json - Single capture
• valid_multiple.json - Multiple captures
• valid_session.json - With session UUID
• invalid_missing_gps.json - Missing coordinates
• invalid_bad_json.json - Malformed JSON

3. Database Fixtures

Directory: tests/fixtures/database/

Files Needed:

• sample_devices.sql - Sample device records
• sample_signatures.sql - Sample signature records
• sample_users.sql - Test users

---

Test Execution Plan

Step 1: Unit Tests (Local)

# Install test dependencies
pip install pytest pytest-asyncio pytest-cov

# Run unit tests
pytest tests/unit/ -v

# Run with coverage
pytest tests/unit/ --cov=src --cov-report=html

Expected: All unit tests pass (green)

Step 2: Integration Tests (Local/Termux)

# Setup test database
./scripts/setup_database.sh
psql -U giglez_user -d giglez -h localhost -f scripts/create_schema.sql

# Run integration tests
pytest tests/integration/ -v

# Run all tests
pytest tests/ -v

Expected: All integration tests pass (green)

Step 3: Manual Testing (Termux)

# Start API server
python src/api/main.py

# In another terminal, run manual tests
bash tests/manual/run_manual_tests.sh

Expected: All manual tests succeed

Step 4: Real-World Testing (Termux + T-Embed)

# Follow tests/manual/termux_realworld.md
# Capture actual signals, upload, verify

Expected: Full workflow works end-to-end

---

Success Criteria

Phase 1 (Database & Core)

• ✅ All unit tests pass (GPS, parser, storage, models)
• ✅ Database schema creates without errors
• ✅ PostGIS queries work
• ✅ Models can be instantiated and saved

Phase 2 (API)

• ✅ API server starts successfully
• ✅ Upload endpoint accepts files
• ✅ Deduplication works (duplicate uploads rejected)
• ✅ Files stored correctly
• ✅ Database records created
• ✅ Query endpoints return data

Phase 3 (Termux Workflow)

• ✅ Can install and run on Termux
• ✅ Can upload .sub files via curl
• ✅ Can query captures
• ✅ Storage persists across restarts
• ✅ Database persists across restarts

---

Test Configuration

pytest.ini

[pytest]
testpaths = tests
python_files = test_*.py
python_classes = Test*
python_functions = test_*
addopts =
    -v
    --strict-markers
    --tb=short
markers =
    unit: Unit tests (fast, no external dependencies)
    integration: Integration tests (database, API)
    manual: Manual tests (human verification)
    slow: Slow tests (skip in CI)

conftest.py (Shared Fixtures)

import pytest
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker

@pytest.fixture
def db_session():
    """Test database session"""
    # Create test database session
    pass

@pytest.fixture
def test_storage():
    """Test storage backend"""
    # Create temporary storage
    pass

@pytest.fixture
def test_client():
    """FastAPI test client"""
    from fastapi.testclient import TestClient
    from src.api.main import app
    return TestClient(app)

---

Bug Tracking

Issues Found During Testing

Template for each issue:

## Issue #N: [Brief Description]

**Severity**: Critical / High / Medium / Low
**Component**: GPS Validator / Parser / API / Database
**Found In**: Unit Test / Integration Test / Manual Test

### Reproduction
Steps to reproduce the issue

### Expected Behavior
What should happen

### Actual Behavior
What actually happens

### Fix
How to fix it

### Test Coverage
New test to prevent regression

---

Documentation Updates Needed

Based on test results:

• Update README with verified installation steps
• Document any configuration changes needed
• Add troubleshooting section for common issues
• Update API examples with working curl commands
• Document Termux-specific setup steps

---

Next Steps After Validation

1. ✅ Fix all critical bugs
2. ✅ Add regression tests for fixed bugs
3. ✅ Update documentation based on findings
4. ✅ Merge p1-p2-validation → master
5. ✅ Tag release: v0.1.0-alpha
6. ✅ Begin Phase 2 completion (auth, background tasks)

---

Testing Status: Ready to Execute Branch: p1-p2-validation Next Action: Create test files and fixtures