giglez/TESTING_STRATEGY.md

# GigLez Testing Strategy - Phase 1 & 2 Validation

**Branch**: `p1-p2-validation`
**Created**: 2026-01-12
**Purpose**: Comprehensive testing before deployment

---

## Testing Pyramid

```
                    ┌─────────────┐
                    │   Manual    │  ← Termux workflow testing
                    │   Testing   │
                    └─────────────┘
                  ┌───────────────────┐
                  │   Integration     │  ← API endpoint testing
                  │     Tests         │
                  └───────────────────┘
              ┌─────────────────────────┐
              │      Unit Tests         │  ← Component testing
              │                         │
              └─────────────────────────┘
```

---

## Phase 1: Unit Tests

### 1. GPS Validator Tests
**File**: `tests/unit/test_gps_validator.py`

**Test Cases**:
- ✅ Valid coordinates (NYC, London, Tokyo)
- ✅ Invalid latitude (> 90, < -90)
- ✅ Invalid longitude (> 180, < -180)
- ✅ Null Island detection (0.0, 0.0)
- ✅ Accuracy thresholds (< 50m pass, > 50m fail)
- ✅ High-quality detection (< 10m)
- ✅ Anonymization (precision reduction)
- ✅ Distance calculation (Haversine)

### 2. .sub File Parser Tests
**File**: `tests/unit/test_sub_parser.py`

**Test Cases**:
- ✅ Valid KEY format file
- ✅ Valid RAW format file
- ✅ Valid BinRAW format file
- ✅ Invalid file format (missing fields)
- ✅ Invalid frequency (out of range)
- ✅ Metadata extraction (protocol, frequency, modulation)
- ✅ Key data parsing (hex to bytes)
- ✅ Raw data parsing (timing array)

### 3. Storage Backend Tests
**File**: `tests/unit/test_storage.py`

**Test Cases**:
- ✅ LocalStorage: save file
- ✅ LocalStorage: retrieve file
- ✅ LocalStorage: file exists check
- ✅ LocalStorage: delete file
- ✅ LocalStorage: SHA256 path sharding
- ✅ LocalStorage: storage statistics
- ✅ S3Storage: mock save/retrieve (if boto3 available)

### 4. Database Models Tests
**File**: `tests/unit/test_models.py`

**Test Cases**:
- ✅ User model creation
- ✅ Session model creation
- ✅ Device model creation
- ✅ Capture model creation (with GPS validation)
- ✅ Capture deduplication (file_hash primary key)
- ✅ Relationships (user -> captures, session -> captures)
- ✅ to_dict() serialization

### 5. Configuration Tests
**File**: `tests/unit/test_settings.py`

**Test Cases**:
- ✅ Development mode detection
- ✅ Production mode detection
- ✅ Environment variable loading
- ✅ Settings validation (production checks)
- ✅ Computed properties (use_redis, use_celery)
- ✅ Database URL generation

---

## Phase 2: Integration Tests

### 1. API Startup Tests
**File**: `tests/integration/test_api_startup.py`

**Test Cases**:
- ✅ Application starts successfully
- ✅ Database connection established
- ✅ PostGIS extension available
- ✅ Storage backend initialized
- ✅ Health endpoint responds
- ✅ OpenAPI docs available (dev mode)

### 2. Upload Endpoint Tests
**File**: `tests/integration/test_upload.py`

**Test Cases**:
- ✅ Upload single .sub file with manifest
- ✅ Upload multiple .sub files
- ✅ Duplicate file detection (same hash)
- ✅ Invalid manifest JSON (400 error)
- ✅ Missing manifest entry (file skipped)
- ✅ Invalid GPS coordinates (file rejected)
- ✅ Invalid .sub file format (file rejected)
- ✅ Session creation
- ✅ Session reuse (existing UUID)
- ✅ File stored in storage backend
- ✅ Capture record created in database
- ✅ Response format validation

### 3. Query Endpoint Tests
**File**: `tests/integration/test_query.py`

**Test Cases**:
- ✅ Get capture by file_hash
- ✅ 404 for non-existent capture
- ✅ Device listing (when implemented)
- ✅ Statistics endpoint (when implemented)

### 4. Storage Integration Tests
**File**: `tests/integration/test_storage_integration.py`

**Test Cases**:
- ✅ Upload → Storage → Retrieve (round trip)
- ✅ Multiple uploads (different files)
- ✅ File sharding verification
- ✅ Storage statistics accuracy

### 5. Database Integration Tests
**File**: `tests/integration/test_database.py`

**Test Cases**:
- ✅ Create tables (if not exists)
- ✅ Insert capture with GPS
- ✅ PostGIS geometry auto-population
- ✅ Spatial query (within radius)
- ✅ Trigger execution (session stats update)
- ✅ Deduplication (unique file_hash)

---

## Phase 3: Manual Testing (Termux)

### 1. Environment Setup Tests
**File**: `tests/manual/termux_setup.md`

**Test Cases**:
- ✅ PostgreSQL installation
- ✅ Database creation
- ✅ Schema creation (no errors)
- ✅ PostGIS extension enabled
- ✅ Python dependencies installation
- ✅ Settings validation passes
- ✅ Storage directory creation

### 2. API Server Tests
**File**: `tests/manual/termux_api.md`

**Test Cases**:
- ✅ Server starts on localhost:8000
- ✅ Health check responds
- ✅ OpenAPI docs accessible
- ✅ CORS headers present
- ✅ Request logging works
- ✅ Exception handling (test invalid request)

### 3. Upload Workflow Tests
**File**: `tests/manual/termux_upload.md`

**Test Cases**:
- ✅ Create test .sub file
- ✅ Create test manifest
- ✅ Upload via curl
- ✅ Verify response (201, success message)
- ✅ Check file in storage directory
- ✅ Query capture from database
- ✅ Verify GPS coordinates
- ✅ Upload duplicate (verify deduplication)

### 4. Real-World Workflow Tests
**File**: `tests/manual/termux_realworld.md`

**Test Cases**:
- ✅ Capture actual signal with T-Embed (if available)
- ✅ Save .sub file to device
- ✅ Get GPS coordinates (termux-location)
- ✅ Create manifest from GPS data
- ✅ Upload to API
- ✅ Verify in database
- ✅ Query back by hash
- ✅ Verify storage location

---

## Test Data & Fixtures

### 1. Sample .sub Files
**Directory**: `tests/fixtures/sub_files/`

**Files Needed**:
- `valid_key_433mhz.sub` - Standard KEY format (Princeton)
- `valid_raw_315mhz.sub` - RAW format with timing data
- `valid_binraw_868mhz.sub` - BinRAW format
- `invalid_missing_freq.sub` - Missing frequency field
- `invalid_bad_format.sub` - Corrupted file

### 2. Sample Manifests
**Directory**: `tests/fixtures/manifests/`

**Files Needed**:
- `valid_single.json` - Single capture
- `valid_multiple.json` - Multiple captures
- `valid_session.json` - With session UUID
- `invalid_missing_gps.json` - Missing coordinates
- `invalid_bad_json.json` - Malformed JSON

### 3. Database Fixtures
**Directory**: `tests/fixtures/database/`

**Files Needed**:
- `sample_devices.sql` - Sample device records
- `sample_signatures.sql` - Sample signature records
- `sample_users.sql` - Test users

---

## Test Execution Plan

### Step 1: Unit Tests (Local)
```bash
# Install test dependencies
pip install pytest pytest-asyncio pytest-cov

# Run unit tests
pytest tests/unit/ -v

# Run with coverage
pytest tests/unit/ --cov=src --cov-report=html
```

**Expected**: All unit tests pass (green)

### Step 2: Integration Tests (Local/Termux)
```bash
# Setup test database
./scripts/setup_database.sh
psql -U giglez_user -d giglez -h localhost -f scripts/create_schema.sql

# Run integration tests
pytest tests/integration/ -v

# Run all tests
pytest tests/ -v
```

**Expected**: All integration tests pass (green)

### Step 3: Manual Testing (Termux)
```bash
# Start API server
python src/api/main.py

# In another terminal, run manual tests
bash tests/manual/run_manual_tests.sh
```

**Expected**: All manual tests succeed

### Step 4: Real-World Testing (Termux + T-Embed)
```bash
# Follow tests/manual/termux_realworld.md
# Capture actual signals, upload, verify
```

**Expected**: Full workflow works end-to-end

---

## Success Criteria

### Phase 1 (Database & Core)
- ✅ All unit tests pass (GPS, parser, storage, models)
- ✅ Database schema creates without errors
- ✅ PostGIS queries work
- ✅ Models can be instantiated and saved

### Phase 2 (API)
- ✅ API server starts successfully
- ✅ Upload endpoint accepts files
- ✅ Deduplication works (duplicate uploads rejected)
- ✅ Files stored correctly
- ✅ Database records created
- ✅ Query endpoints return data

### Phase 3 (Termux Workflow)
- ✅ Can install and run on Termux
- ✅ Can upload .sub files via curl
- ✅ Can query captures
- ✅ Storage persists across restarts
- ✅ Database persists across restarts

---

## Test Configuration

### pytest.ini
```ini
[pytest]
testpaths = tests
python_files = test_*.py
python_classes = Test*
python_functions = test_*
addopts =
    -v
    --strict-markers
    --tb=short
markers =
    unit: Unit tests (fast, no external dependencies)
    integration: Integration tests (database, API)
    manual: Manual tests (human verification)
    slow: Slow tests (skip in CI)
```

### conftest.py (Shared Fixtures)
```python
import pytest
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker

@pytest.fixture
def db_session():
    """Test database session"""
    # Create test database session
    pass

@pytest.fixture
def test_storage():
    """Test storage backend"""
    # Create temporary storage
    pass

@pytest.fixture
def test_client():
    """FastAPI test client"""
    from fastapi.testclient import TestClient
    from src.api.main import app
    return TestClient(app)
```

---

## Bug Tracking

### Issues Found During Testing
**Template for each issue**:

```markdown
## Issue #N: [Brief Description]

**Severity**: Critical / High / Medium / Low
**Component**: GPS Validator / Parser / API / Database
**Found In**: Unit Test / Integration Test / Manual Test

### Reproduction
Steps to reproduce the issue

### Expected Behavior
What should happen

### Actual Behavior
What actually happens

### Fix
How to fix it

### Test Coverage
New test to prevent regression
```

---

## Documentation Updates Needed

Based on test results:
- [ ] Update README with verified installation steps
- [ ] Document any configuration changes needed
- [ ] Add troubleshooting section for common issues
- [ ] Update API examples with working curl commands
- [ ] Document Termux-specific setup steps

---

## Next Steps After Validation

1. ✅ Fix all critical bugs
2. ✅ Add regression tests for fixed bugs
3. ✅ Update documentation based on findings
4. ✅ Merge `p1-p2-validation` → `master`
5. ✅ Tag release: `v0.1.0-alpha`
6. ✅ Begin Phase 2 completion (auth, background tasks)

---

**Testing Status**: Ready to Execute
**Branch**: `p1-p2-validation`
**Next Action**: Create test files and fixtures