Add comprehensive testing infrastructure

- Testing strategy document - Unit tests for GPS validator - Integration test conftest with fixtures - Sample .sub files and manifests - Comprehensive Termux testing guide - Test directory structure
2026-01-12 11:25:13 -08:00
parent eb225771bc
commit 64ce279a2e
5 changed files with 1494 additions and 0 deletions
@@ -0,0 +1,418 @@
 # GigLez Testing Strategy - Phase 1 & 2 Validation
 **Branch**: `p1-p2-validation`
 **Created**: 2026-01-12
 **Purpose**: Comprehensive testing before deployment
 ---
 ## Testing Pyramid
 ```
                    ┌─────────────┐
                    │   Manual    │  ← Termux workflow testing
                    │   Testing   │
                    └─────────────┘
                  ┌───────────────────┐
                  │   Integration     │  ← API endpoint testing
                  │     Tests         │
                  └───────────────────┘
              ┌─────────────────────────┐
              │      Unit Tests         │  ← Component testing
              │                         │
              └─────────────────────────┘
 ```
 ---
 ## Phase 1: Unit Tests
 ### 1. GPS Validator Tests
 **File**: `tests/unit/test_gps_validator.py`
 **Test Cases**:
 - ✅ Valid coordinates (NYC, London, Tokyo)
 - ✅ Invalid latitude (> 90, < -90)
 - ✅ Invalid longitude (> 180, < -180)
 - ✅ Null Island detection (0.0, 0.0)
 - ✅ Accuracy thresholds (< 50m pass, > 50m fail)
 - ✅ High-quality detection (< 10m)
 - ✅ Anonymization (precision reduction)
 - ✅ Distance calculation (Haversine)
 ### 2. .sub File Parser Tests
 **File**: `tests/unit/test_sub_parser.py`
 **Test Cases**:
 - ✅ Valid KEY format file
 - ✅ Valid RAW format file
 - ✅ Valid BinRAW format file
 - ✅ Invalid file format (missing fields)
 - ✅ Invalid frequency (out of range)
 - ✅ Metadata extraction (protocol, frequency, modulation)
 - ✅ Key data parsing (hex to bytes)
 - ✅ Raw data parsing (timing array)
 ### 3. Storage Backend Tests
 **File**: `tests/unit/test_storage.py`
 **Test Cases**:
 - ✅ LocalStorage: save file
 - ✅ LocalStorage: retrieve file
 - ✅ LocalStorage: file exists check
 - ✅ LocalStorage: delete file
 - ✅ LocalStorage: SHA256 path sharding
 - ✅ LocalStorage: storage statistics
 - ✅ S3Storage: mock save/retrieve (if boto3 available)
 ### 4. Database Models Tests
 **File**: `tests/unit/test_models.py`
 **Test Cases**:
 - ✅ User model creation
 - ✅ Session model creation
 - ✅ Device model creation
 - ✅ Capture model creation (with GPS validation)
 - ✅ Capture deduplication (file_hash primary key)
 - ✅ Relationships (user -> captures, session -> captures)
 - ✅ to_dict() serialization
 ### 5. Configuration Tests
 **File**: `tests/unit/test_settings.py`
 **Test Cases**:
 - ✅ Development mode detection
 - ✅ Production mode detection
 - ✅ Environment variable loading
 - ✅ Settings validation (production checks)
 - ✅ Computed properties (use_redis, use_celery)
 - ✅ Database URL generation
 ---
 ## Phase 2: Integration Tests
 ### 1. API Startup Tests
 **File**: `tests/integration/test_api_startup.py`
 **Test Cases**:
 - ✅ Application starts successfully
 - ✅ Database connection established
 - ✅ PostGIS extension available
 - ✅ Storage backend initialized
 - ✅ Health endpoint responds
 - ✅ OpenAPI docs available (dev mode)
 ### 2. Upload Endpoint Tests
 **File**: `tests/integration/test_upload.py`
 **Test Cases**:
 - ✅ Upload single .sub file with manifest
 - ✅ Upload multiple .sub files
 - ✅ Duplicate file detection (same hash)
 - ✅ Invalid manifest JSON (400 error)
 - ✅ Missing manifest entry (file skipped)
 - ✅ Invalid GPS coordinates (file rejected)
 - ✅ Invalid .sub file format (file rejected)
 - ✅ Session creation
 - ✅ Session reuse (existing UUID)
 - ✅ File stored in storage backend
 - ✅ Capture record created in database
 - ✅ Response format validation
 ### 3. Query Endpoint Tests
 **File**: `tests/integration/test_query.py`
 **Test Cases**:
 - ✅ Get capture by file_hash
 - ✅ 404 for non-existent capture
 - ✅ Device listing (when implemented)
 - ✅ Statistics endpoint (when implemented)
 ### 4. Storage Integration Tests
 **File**: `tests/integration/test_storage_integration.py`
 **Test Cases**:
 - ✅ Upload → Storage → Retrieve (round trip)
 - ✅ Multiple uploads (different files)
 - ✅ File sharding verification
 - ✅ Storage statistics accuracy
 ### 5. Database Integration Tests
 **File**: `tests/integration/test_database.py`
 **Test Cases**:
 - ✅ Create tables (if not exists)
 - ✅ Insert capture with GPS
 - ✅ PostGIS geometry auto-population
 - ✅ Spatial query (within radius)
 - ✅ Trigger execution (session stats update)
 - ✅ Deduplication (unique file_hash)
 ---
 ## Phase 3: Manual Testing (Termux)
 ### 1. Environment Setup Tests
 **File**: `tests/manual/termux_setup.md`
 **Test Cases**:
 - ✅ PostgreSQL installation
 - ✅ Database creation
 - ✅ Schema creation (no errors)
 - ✅ PostGIS extension enabled
 - ✅ Python dependencies installation
 - ✅ Settings validation passes
 - ✅ Storage directory creation
 ### 2. API Server Tests
 **File**: `tests/manual/termux_api.md`
 **Test Cases**:
 - ✅ Server starts on localhost:8000
 - ✅ Health check responds
 - ✅ OpenAPI docs accessible
 - ✅ CORS headers present
 - ✅ Request logging works
 - ✅ Exception handling (test invalid request)
 ### 3. Upload Workflow Tests
 **File**: `tests/manual/termux_upload.md`
 **Test Cases**:
 - ✅ Create test .sub file
 - ✅ Create test manifest
 - ✅ Upload via curl
 - ✅ Verify response (201, success message)
 - ✅ Check file in storage directory
 - ✅ Query capture from database
 - ✅ Verify GPS coordinates
 - ✅ Upload duplicate (verify deduplication)
 ### 4. Real-World Workflow Tests
 **File**: `tests/manual/termux_realworld.md`
 **Test Cases**:
 - ✅ Capture actual signal with T-Embed (if available)
 - ✅ Save .sub file to device
 - ✅ Get GPS coordinates (termux-location)
 - ✅ Create manifest from GPS data
 - ✅ Upload to API
 - ✅ Verify in database
 - ✅ Query back by hash
 - ✅ Verify storage location
 ---
 ## Test Data & Fixtures
 ### 1. Sample .sub Files
 **Directory**: `tests/fixtures/sub_files/`
 **Files Needed**:
 - `valid_key_433mhz.sub` - Standard KEY format (Princeton)
 - `valid_raw_315mhz.sub` - RAW format with timing data
 - `valid_binraw_868mhz.sub` - BinRAW format
 - `invalid_missing_freq.sub` - Missing frequency field
 - `invalid_bad_format.sub` - Corrupted file
 ### 2. Sample Manifests
 **Directory**: `tests/fixtures/manifests/`
 **Files Needed**:
 - `valid_single.json` - Single capture
 - `valid_multiple.json` - Multiple captures
 - `valid_session.json` - With session UUID
 - `invalid_missing_gps.json` - Missing coordinates
 - `invalid_bad_json.json` - Malformed JSON
 ### 3. Database Fixtures
 **Directory**: `tests/fixtures/database/`
 **Files Needed**:
 - `sample_devices.sql` - Sample device records
 - `sample_signatures.sql` - Sample signature records
 - `sample_users.sql` - Test users
 ---
 ## Test Execution Plan
 ### Step 1: Unit Tests (Local)
 ```bash
 # Install test dependencies
 pip install pytest pytest-asyncio pytest-cov
 # Run unit tests
 pytest tests/unit/ -v
 # Run with coverage
 pytest tests/unit/ --cov=src --cov-report=html
 ```
 **Expected**: All unit tests pass (green)
 ### Step 2: Integration Tests (Local/Termux)
 ```bash
 # Setup test database
 ./scripts/setup_database.sh
 psql -U giglez_user -d giglez -h localhost -f scripts/create_schema.sql
 # Run integration tests
 pytest tests/integration/ -v
 # Run all tests
 pytest tests/ -v
 ```
 **Expected**: All integration tests pass (green)
 ### Step 3: Manual Testing (Termux)
 ```bash
 # Start API server
 python src/api/main.py
 # In another terminal, run manual tests
 bash tests/manual/run_manual_tests.sh
 ```
 **Expected**: All manual tests succeed
 ### Step 4: Real-World Testing (Termux + T-Embed)
 ```bash
 # Follow tests/manual/termux_realworld.md
 # Capture actual signals, upload, verify
 ```
 **Expected**: Full workflow works end-to-end
 ---
 ## Success Criteria
 ### Phase 1 (Database & Core)
 - ✅ All unit tests pass (GPS, parser, storage, models)
 - ✅ Database schema creates without errors
 - ✅ PostGIS queries work
 - ✅ Models can be instantiated and saved
 ### Phase 2 (API)
 - ✅ API server starts successfully
 - ✅ Upload endpoint accepts files
 - ✅ Deduplication works (duplicate uploads rejected)
 - ✅ Files stored correctly
 - ✅ Database records created
 - ✅ Query endpoints return data
 ### Phase 3 (Termux Workflow)
 - ✅ Can install and run on Termux
 - ✅ Can upload .sub files via curl
 - ✅ Can query captures
 - ✅ Storage persists across restarts
 - ✅ Database persists across restarts
 ---
 ## Test Configuration
 ### pytest.ini
 ```ini
 [pytest]
 testpaths = tests
 python_files = test_*.py
 python_classes = Test*
 python_functions = test_*
 addopts =
    -v
    --strict-markers
    --tb=short
 markers =
    unit: Unit tests (fast, no external dependencies)
    integration: Integration tests (database, API)
    manual: Manual tests (human verification)
    slow: Slow tests (skip in CI)
 ```
 ### conftest.py (Shared Fixtures)
 ```python
 import pytest
 from sqlalchemy import create_engine
 from sqlalchemy.orm import sessionmaker
@pytest.fixture
 def db_session():
    """Test database session"""
    # Create test database session
    pass
@pytest.fixture
 def test_storage():
    """Test storage backend"""
    # Create temporary storage
    pass
@pytest.fixture
 def test_client():
    """FastAPI test client"""
    from fastapi.testclient import TestClient
    from src.api.main import app
    return TestClient(app)
 ```
 ---
 ## Bug Tracking
 ### Issues Found During Testing
 **Template for each issue**:
 ```markdown
 ## Issue #N: [Brief Description]
 **Severity**: Critical / High / Medium / Low
 **Component**: GPS Validator / Parser / API / Database
 **Found In**: Unit Test / Integration Test / Manual Test
 ### Reproduction
 Steps to reproduce the issue
 ### Expected Behavior
 What should happen
 ### Actual Behavior
 What actually happens
 ### Fix
 How to fix it
 ### Test Coverage
 New test to prevent regression
 ```
 ---
 ## Documentation Updates Needed
 Based on test results:
 - [ ] Update README with verified installation steps
 - [ ] Document any configuration changes needed
 - [ ] Add troubleshooting section for common issues
 - [ ] Update API examples with working curl commands
 - [ ] Document Termux-specific setup steps
 ---
 ## Next Steps After Validation
 1. ✅ Fix all critical bugs
 2. ✅ Add regression tests for fixed bugs
 3. ✅ Update documentation based on findings
 4. ✅ Merge `p1-p2-validation` → `master`
 5. ✅ Tag release: `v0.1.0-alpha`
 6. ✅ Begin Phase 2 completion (auth, background tasks)
 ---
 **Testing Status**: Ready to Execute
 **Branch**: `p1-p2-validation`
 **Next Action**: Create test files and fixtures
@@ -0,0 +1,187 @@
 """
 Pytest Configuration and Shared Fixtures
 Provides shared test fixtures for unit and integration tests
 """
 import pytest
 import tempfile
 import shutil
 from pathlib import Path
 from sqlalchemy import create_engine
 from sqlalchemy.orm import sessionmaker
 from fastapi.testclient import TestClient
 from src.database.models import Base
 from src.core.storage import LocalStorage
 from config.settings import settings
 # =============================================================================
 # DATABASE FIXTURES
 # =============================================================================
@pytest.fixture(scope="session")
 def test_db_engine():
    """
    Create test database engine
    Uses in-memory SQLite for fast tests
    """
    # Use SQLite in-memory for tests
    engine = create_engine("sqlite:///:memory:", echo=False)
    # Create all tables
    Base.metadata.create_all(engine)
    yield engine
    # Cleanup
    Base.metadata.drop_all(engine)
    engine.dispose()
@pytest.fixture
 def db_session(test_db_engine):
    """
    Database session fixture
    Creates a new session for each test and rolls back after
    """
    Session = sessionmaker(bind=test_db_engine)
    session = Session()
    yield session
    session.rollback()
    session.close()
 # =============================================================================
 # STORAGE FIXTURES
 # =============================================================================
@pytest.fixture
 def temp_storage_dir():
    """
    Temporary storage directory
    Creates a temp dir and cleans up after test
    """
    temp_dir = tempfile.mkdtemp(prefix="giglez_test_")
    yield Path(temp_dir)
    # Cleanup
    shutil.rmtree(temp_dir, ignore_errors=True)
@pytest.fixture
 def test_storage(temp_storage_dir):
    """
    Test storage backend (LocalStorage)
    Uses temporary directory
    """
    return LocalStorage(base_path=str(temp_storage_dir))
 # =============================================================================
 # API FIXTURES
 # =============================================================================
@pytest.fixture
 def test_client():
    """
    FastAPI test client
    Provides TestClient for API endpoint testing
    """
    from src.api.main import app
    client = TestClient(app)
    return client
 # =============================================================================
 # SAMPLE DATA FIXTURES
 # =============================================================================
@pytest.fixture
 def sample_sub_file_key():
    """Sample .sub file content (KEY format)"""
    return """Filetype: Flipper SubGhz Key File
 Version: 1
 Frequency: 433920000
 Preset: FuriHalSubGhzPresetOok270Async
 Protocol: Princeton
 Bit: 24
 Key: 00 00 00 00 00 95 D5 D4
 TE: 400
 """
@pytest.fixture
 def sample_sub_file_raw():
    """Sample .sub file content (RAW format)"""
    return """Filetype: Flipper SubGhz RAW File
 Version: 1
 Frequency: 315000000
 Preset: FuriHalSubGhzPresetOok650Async
 Protocol: RAW
 RAW_Data: 2980 -240 520 -980 520 -980 1000 -500 480 -1020
 """
@pytest.fixture
 def sample_manifest():
    """Sample upload manifest"""
    return {
        "session_uuid": "test-session-001",
        "captures": [
            {
                "filename": "test.sub",
                "latitude": 40.7128,
                "longitude": -74.0060,
                "accuracy": 5.0,
                "altitude": 10.5,
                "timestamp": "2026-01-12T10:00:00Z"
            }
        ]
    }
@pytest.fixture
 def sample_gps_coordinates():
    """Sample GPS coordinates (valid)"""
    return [
        (40.7128, -74.0060, 5.0, "NYC"),
        (51.5074, -0.1278, 8.0, "London"),
        (35.6762, 139.6503, 3.0, "Tokyo"),
        (-33.8688, 151.2093, 10.0, "Sydney"),
    ]
@pytest.fixture
 def invalid_gps_coordinates():
    """Invalid GPS coordinates for testing"""
    return [
        (91.0, 0.0, 5.0, "Latitude > 90"),
        (-91.0, 0.0, 5.0, "Latitude < -90"),
        (0.0, 181.0, 5.0, "Longitude > 180"),
        (0.0, -181.0, 5.0, "Longitude < -180"),
        (0.0, 0.0, 5.0, "Null Island"),
        (40.7128, -74.0060, 100.0, "Poor accuracy"),
    ]
 # =============================================================================
 # PYTEST MARKERS
 # =============================================================================
 def pytest_configure(config):
    """Register custom pytest markers"""
    config.addinivalue_line("markers", "unit: Unit tests (fast, no external dependencies)")
    config.addinivalue_line("markers", "integration: Integration tests (database, API)")
    config.addinivalue_line("markers", "manual: Manual tests (human verification required)")
    config.addinivalue_line("markers", "slow: Slow tests (skip in CI)")
@@ -0,0 +1,13 @@
 {
    "session_uuid": "test-session-001",
    "captures": [
        {
            "filename": "valid_key_433mhz.sub",
            "latitude": 40.7128,
            "longitude": -74.0060,
            "accuracy": 5.0,
            "altitude": 10.5,
            "timestamp": "2026-01-12T10:00:00Z"
        }
    ]
 }
@@ -0,0 +1,654 @@
 # Termux Manual Testing Guide
 **Purpose**: Validate GigLez Phase 1 & 2 in real Termux environment
 **Branch**: `p1-p2-validation`
 **Prerequisites**: Android device with Termux installed
 ---
 ## Pre-Flight Checklist
 - [ ] Termux installed from F-Droid (not Play Store)
 - [ ] Device has at least 2GB free storage
 - [ ] Stable internet connection for package installation
 - [ ] USB OTG cable (if testing with T-Embed)
 ---
 ## Step 1: Environment Setup (30 minutes)
 ### 1.1 Update Termux Packages
 ```bash
 pkg update && pkg upgrade -y
 ```
 ### 1.2 Install Required Packages
 ```bash
 # PostgreSQL
 pkg install postgresql -y
 # Python and development tools
 pkg install python python-pip git -y
 # Build dependencies
 pkg install build-essential libffi openssl -y
 ```
 **Expected**: All packages install without errors
 ### 1.3 Clone Repository
 ```bash
 cd ~
 git clone https://github.com/yourusername/giglez.git
 cd giglez
 git checkout p1-p2-validation
 ```
 **Expected**: Repository cloned, on validation branch
 ### 1.4 Install Python Dependencies
 ```bash
 pip install --upgrade pip
 pip install -r requirements.txt
 pip install pytest pytest-asyncio python-multipart
 ```
 **Expected**: All dependencies install successfully
 **Common Issue**: If psycopg2 fails, try `pkg install postgresql-dev` first
 ---
 ## Step 2: Database Setup (10 minutes)
 ### 2.1 Initialize PostgreSQL
 ```bash
 initdb ~/postgres
 pg_ctl -D ~/postgres -l ~/postgres/logfile start
 ```
 **Expected**: PostgreSQL starts, log file created
 **Verify**: `pg_ctl -D ~/postgres status` shows "server is running"
 ### 2.2 Create Database
 ```bash
 cd ~/giglez
 ./scripts/setup_database.sh
 ```
 **Expected Output**:
 ```
 ✅ User giglez_user created
 ✅ Database giglez created
 ✅ PostGIS extension enabled
 ```
 **Troubleshoot**: If permission denied, run:
 ```bash
 chmod +x scripts/setup_database.sh
 ```
 ### 2.3 Create Schema
 ```bash
 psql -U giglez_user -d giglez -h localhost -f scripts/create_schema.sql
 ```
 **Expected**: 15 tables created, no errors
 **Verify**:
 ```bash
 psql -U giglez_user -d giglez -h localhost -c "\dt"
 ```
 Should list: users, sessions, devices, captures, signatures, etc.
 ### 2.4 Test PostGIS
 ```bash
 psql -U giglez_user -d giglez -h localhost -c "SELECT PostGIS_Version();"
 ```
 **Expected**: PostGIS version string (3.x)
 ---
 ## Step 3: Configuration Test (5 minutes)
 ### 3.1 Copy Environment File
 ```bash
 cp .env.development .env
 ```
 ### 3.2 Edit Configuration
 ```bash
 nano .env
 ```
 **Adjust paths**:
 ```bash
 GIGLEZ_STORAGE_PATH=/data/data/com.termux/files/home/giglez/storage
 GIGLEZ_QUEUE_PATH=/data/data/com.termux/files/home/giglez/queue
 ```
 **Save**: Ctrl+O, Enter, Ctrl+X
 ### 3.3 Test Configuration
 ```bash
 python config/settings.py
 ```
 **Expected Output**:
 ```
 ========================================================
 GigLez Configuration
 ========================================================
 Mode: development
 Database: postgresql://giglez_user:****@localhost:5432/giglez
 API: 127.0.0.1:8000
 Storage: filesystem
 Hardware: Disabled
 Authentication: Optional
 ...
 ✅ Configuration validated successfully
 ```
 **Troubleshoot**: If validation fails, check .env file syntax
 ---
 ## Step 4: Unit Tests (10 minutes)
 ### 4.1 Run GPS Validator Tests
 ```bash
 pytest tests/unit/test_gps_validator.py -v
 ```
 **Expected**: All tests pass (green)
 **Count**: ~20 tests
 ### 4.2 Run Storage Tests (when implemented)
 ```bash
 pytest tests/unit/test_storage.py -v
 ```
 ### 4.3 Run All Unit Tests
 ```bash
 pytest tests/unit/ -v
 ```
 **Success Criteria**: All unit tests pass
 ---
 ## Step 5: API Server Test (10 minutes)
 ### 5.1 Start API Server
 ```bash
 python src/api/main.py
 ```
 **Expected Output**:
 ```
 ============================================================
 GigLez API Starting
 ============================================================
 Mode: development
 ...
 ✅ Database connection established
 ✅ PostGIS extension available
 ✅ Storage backend initialized: LocalStorage
 ============================================================
 🚀 GigLez API ready on 127.0.0.1:8000
 📡 Mode: development
 ============================================================
 ```
 **Troubleshoot**: If port 8000 busy, change in .env
 ### 5.2 Test Health Endpoint
 Open new Termux session (swipe from left, "New session"):
 ```bash
 curl http://127.0.0.1:8000/health
 ```
 **Expected**:
 ```json
 {
  "status": "healthy",
  "database": "connected",
  "mode": "development",
  "hardware_enabled": false
 }
 ```
 ### 5.3 Test API Info
 ```bash
 curl http://127.0.0.1:8000/
 ```
 **Expected**: JSON with API name, version, status
 ### 5.4 Test OpenAPI Docs
 ```bash
 curl http://127.0.0.1:8000/docs
 ```
 **Expected**: HTML response (OpenAPI documentation)
 **Browser Test**: If you have Termux:API installed:
 ```bash
 termux-open-url http://127.0.0.1:8000/docs
 ```
 ---
 ## Step 6: Upload Test (15 minutes)
 ### 6.1 Create Test .sub File
 ```bash
 cd ~/giglez
 cat > test_capture.sub << 'EOF'
 Filetype: Flipper SubGhz Key File
 Version: 1
 Frequency: 433920000
 Preset: FuriHalSubGhzPresetOok270Async
 Protocol: Princeton
 Bit: 24
 Key: 00 00 00 00 00 95 D5 D4
 TE: 400
 EOF
 ```
 ### 6.2 Create Test Manifest
 ```bash
 cat > test_manifest.json << 'EOF'
 {
    "session_uuid": "termux-test-001",
    "captures": [
        {
            "filename": "test_capture.sub",
            "latitude": 40.7128,
            "longitude": -74.0060,
            "accuracy": 5.0,
            "altitude": 10.5,
            "timestamp": "2026-01-12T10:00:00Z"
        }
    ]
 }
 EOF
 ```
 ### 6.3 Upload Test File
 ```bash
 curl -X POST http://127.0.0.1:8000/api/v1/captures/upload \
  -F "manifest=$(cat test_manifest.json)" \
  -F "files=@test_capture.sub"
 ```
 **Expected Response**:
 ```json
 {
  "session_uuid": "termux-test-001",
  "uploaded": 1,
  "duplicates": 0,
  "errors": 0,
  "results": [
    {
      "filename": "test_capture.sub",
      "status": "uploaded",
      "file_hash": "abc123...",
      "frequency": 433920000,
      "protocol": "Princeton"
    }
  ]
 }
 ```
 **Success Criteria**:
 - `uploaded: 1`
 - `errors: 0`
 - `file_hash` present
 - `status: "uploaded"`
 ### 6.4 Verify in Database
 ```bash
 psql -U giglez_user -d giglez -h localhost << 'SQL'
 SELECT file_hash, frequency, protocol, latitude, longitude
 FROM captures
 LIMIT 5;
 SQL
 ```
 **Expected**: Your capture listed with correct data
 ### 6.5 Verify in Storage
 ```bash
 ls -lh ~/giglez/storage/
 ```
 **Expected**: Directory structure with .sub file:
 ```
 storage/
 └── ab/
    └── c1/
        └── abc123...def.sub
 ```
 ### 6.6 Test Duplicate Upload
 ```bash
 # Upload same file again
 curl -X POST http://127.0.0.1:8000/api/v1/captures/upload \
  -F "manifest=$(cat test_manifest.json)" \
  -F "files=@test_capture.sub"
 ```
 **Expected Response**:
 ```json
 {
  "uploaded": 0,
  "duplicates": 1,
  ...
 }
 ```
 **Success Criteria**: Duplicate detected, not re-uploaded
 ---
 ## Step 7: Query Test (5 minutes)
 ### 7.1 Get Capture by Hash
 Get file_hash from previous upload response, then:
 ```bash
 curl http://127.0.0.1:8000/api/v1/captures/{FILE_HASH}
 ```
 Replace `{FILE_HASH}` with actual hash.
 **Expected**: JSON with capture details
 ### 7.2 Test Invalid Hash
 ```bash
 curl http://127.0.0.1:8000/api/v1/captures/invalid_hash_12345
 ```
 **Expected**: 404 error with message "Capture not found"
 ---
 ## Step 8: Multiple File Upload (10 minutes)
 ### 8.1 Create Multiple Test Files
 ```bash
 # Create 3 different captures
 for i in {1..3}; do
  cat > test_capture_${i}.sub << EOF
 Filetype: Flipper SubGhz Key File
 Version: 1
 Frequency: $((433920000 + i * 1000))
 Preset: FuriHalSubGhzPresetOok270Async
 Protocol: Princeton
 Bit: 24
 Key: 00 00 00 00 00 95 D5 $(printf '%02X' $i)
 TE: 400
 EOF
 done
 ```
 ### 8.2 Create Multi-File Manifest
 ```bash
 cat > multi_manifest.json << 'EOF'
 {
    "session_uuid": "termux-multi-test-001",
    "captures": [
        {
            "filename": "test_capture_1.sub",
            "latitude": 40.7128,
            "longitude": -74.0060,
            "accuracy": 5.0,
            "timestamp": "2026-01-12T10:00:00Z"
        },
        {
            "filename": "test_capture_2.sub",
            "latitude": 40.7129,
            "longitude": -74.0061,
            "accuracy": 6.0,
            "timestamp": "2026-01-12T10:00:10Z"
        },
        {
            "filename": "test_capture_3.sub",
            "latitude": 40.7130,
            "longitude": -74.0062,
            "accuracy": 4.0,
            "timestamp": "2026-01-12T10:00:20Z"
        }
    ]
 }
 EOF
 ```
 ### 8.3 Upload Multiple Files
 ```bash
 curl -X POST http://127.0.0.1:8000/api/v1/captures/upload \
  -F "manifest=$(cat multi_manifest.json)" \
  -F "files=@test_capture_1.sub" \
  -F "files=@test_capture_2.sub" \
  -F "files=@test_capture_3.sub"
 ```
 **Expected**: `uploaded: 3`, all 3 files processed successfully
 ### 8.4 Verify Count
 ```bash
 psql -U giglez_user -d giglez -h localhost -c \
  "SELECT COUNT(*) FROM captures;"
 ```
 **Expected**: At least 4 captures (1 from previous + 3 new)
 ---
 ## Step 9: Error Handling Tests (5 minutes)
 ### 9.1 Test Invalid GPS
 ```bash
 cat > bad_gps_manifest.json << 'EOF'
 {
    "session_uuid": "test-errors",
    "captures": [
        {
            "filename": "test_capture.sub",
            "latitude": 91.0,
            "longitude": 0.0,
            "accuracy": 5.0,
            "timestamp": "2026-01-12T10:00:00Z"
        }
    ]
 }
 EOF
 curl -X POST http://127.0.0.1:8000/api/v1/captures/upload \
  -F "manifest=$(cat bad_gps_manifest.json)" \
  -F "files=@test_capture.sub"
 ```
 **Expected**: Error in response, file rejected
 ### 9.2 Test Missing Manifest Entry
 ```bash
 cat > incomplete_manifest.json << 'EOF'
 {
    "session_uuid": "test-incomplete",
    "captures": []
 }
 EOF
 curl -X POST http://127.0.0.1:8000/api/v1/captures/upload \
  -F "manifest=$(cat incomplete_manifest.json)" \
  -F "files=@test_capture.sub"
 ```
 **Expected**: Error for missing manifest entry
 ### 9.3 Test Invalid JSON
 ```bash
 curl -X POST http://127.0.0.1:8000/api/v1/captures/upload \
  -F "manifest={invalid json}" \
  -F "files=@test_capture.sub"
 ```
 **Expected**: 400 Bad Request with JSON error message
 ---
 ## Step 10: Performance Test (5 minutes)
 ### 10.1 Upload 10 Files Rapidly
 ```bash
 # Create 10 test files
 for i in {1..10}; do
  cp test_capture.sub bulk_test_${i}.sub
 done
 # Note: This will create duplicates, testing deduplication
 time for i in {1..10}; do
  curl -X POST http://127.0.0.1:8000/api/v1/captures/upload \
    -F "manifest=$(cat test_manifest.json)" \
    -F "files=@bulk_test_${i}.sub" &
 done
 wait
 ```
 **Monitor**: Check API logs for performance
 **Expected**: All requests complete, duplicates detected
 ---
 ## Success Criteria Checklist
 ### Database (Phase 1)
 - [ ] PostgreSQL installs and starts
 - [ ] Database schema creates without errors
 - [ ] PostGIS extension available
 - [ ] Can insert and query captures
 - [ ] Triggers execute (session stats update)
 ### API (Phase 2)
 - [ ] API server starts without errors
 - [ ] Health endpoint responds correctly
 - [ ] OpenAPI docs accessible
 - [ ] Configuration loads correctly
 - [ ] Storage backend initializes
 ### Upload Workflow
 - [ ] Single file upload succeeds
 - [ ] Multiple file upload succeeds
 - [ ] Duplicate detection works
 - [ ] GPS validation works (rejects invalid)
 - [ ] Files stored in correct directory structure
 - [ ] Database records created correctly
 - [ ] Session management works
 ### Error Handling
 - [ ] Invalid GPS rejected
 - [ ] Invalid JSON handled
 - [ ] Missing manifest entries handled
 - [ ] Malformed .sub files handled (if tested)
 ---
 ## Troubleshooting
 ### PostgreSQL Won't Start
 ```bash
 # Check if already running
 ps aux | grep postgres
 # Kill existing processes
 pkill postgres
 # Remove old PID file
 rm ~/postgres/postmaster.pid
 # Restart
 pg_ctl -D ~/postgres -l ~/postgres/logfile start
 ```
 ### API Port Already in Use
 ```bash
 # Find process using port 8000
 netstat -tuln | grep 8000
 # Kill process (if needed)
 pkill -f "python src/api/main.py"
 ```
 ### Permission Errors
 ```bash
 # Fix script permissions
 chmod +x scripts/*.sh
 # Fix storage directory
 mkdir -p ~/giglez/storage
 chmod 755 ~/giglez/storage
 ```
 ### Import Errors
 ```bash
 # Reinstall dependencies
 pip install --upgrade --force-reinstall -r requirements.txt
 ```
 ---
 ## Cleanup (Optional)
 ### Stop Services
 ```bash
 # Stop API server (Ctrl+C in server terminal)
 # Stop PostgreSQL
 pg_ctl -D ~/postgres stop
 ```
 ### Remove Test Data
 ```bash
 # Clean database
 psql -U giglez_user -d giglez -h localhost -c "TRUNCATE captures CASCADE;"
 # Clean storage
 rm -rf ~/giglez/storage/*
 # Remove test files
 rm ~/giglez/test_*.sub
 rm ~/giglez/*.json
 ```
 ---
 ## Test Results Template
 ### Environment Info
 - **Device**: [Device model]
 - **Android Version**: [Version]
 - **Termux Version**: [Version]
 - **PostgreSQL Version**: [Run `psql --version`]
 - **Python Version**: [Run `python --version`]
 ### Test Results
 - **Database Setup**: ✅ / ❌
 - **API Startup**: ✅ / ❌
 - **Single Upload**: ✅ / ❌
 - **Multiple Upload**: ✅ / ❌
 - **Duplicate Detection**: ✅ / ❌
 - **GPS Validation**: ✅ / ❌
 - **Error Handling**: ✅ / ❌
 ### Issues Found
 [List any bugs or issues discovered]
 ### Performance Notes
 - Upload time (single file): [X seconds]
 - Upload time (3 files): [X seconds]
 - API response time: [X ms average]
 ---
 **Testing Complete!**
 Document results in TESTING_RESULTS.md
@@ -0,0 +1,222 @@
 """
 Unit Tests for GPS Validator
 Tests GPS coordinate validation, Null Island detection, accuracy thresholds
 """
 import pytest
 from src.gps.validator import (
    validate_gps_coordinates,
    is_null_island,
    is_valid_accuracy,
    anonymize_gps,
    calculate_distance,
    GPSCoordinate,
    GPSValidator
 )
@pytest.mark.unit
 class TestGPSValidation:
    """Test GPS coordinate validation"""
    def test_valid_coordinates(self, sample_gps_coordinates):
        """Test valid GPS coordinates"""
        for lat, lon, accuracy, location in sample_gps_coordinates:
            assert validate_gps_coordinates(lat, lon, accuracy), \
                f"Valid coordinates rejected: {location}"
    def test_invalid_coordinates(self, invalid_gps_coordinates):
        """Test invalid GPS coordinates"""
        for lat, lon, accuracy, reason in invalid_gps_coordinates:
            assert not validate_gps_coordinates(lat, lon, accuracy), \
                f"Invalid coordinates accepted: {reason}"
    def test_null_island_detection(self):
        """Test Null Island (0, 0) detection"""
        assert is_null_island(0.0, 0.0)
        assert is_null_island(0.0001, 0.0001)  # Close to null island
        assert not is_null_island(40.7128, -74.0060)  # NYC
    def test_accuracy_validation(self):
        """Test GPS accuracy threshold validation"""
        assert is_valid_accuracy(5.0)  # High quality
        assert is_valid_accuracy(25.0)  # Medium quality
        assert is_valid_accuracy(50.0)  # Threshold
        assert not is_valid_accuracy(51.0)  # Over threshold
        assert not is_valid_accuracy(100.0)  # Poor accuracy
        assert not is_valid_accuracy(0.0)  # Invalid
        assert not is_valid_accuracy(-5.0)  # Invalid
    def test_latitude_bounds(self):
        """Test latitude boundary validation"""
        assert validate_gps_coordinates(90.0, 0.0, 5.0)  # Max valid
        assert validate_gps_coordinates(-90.0, 0.0, 5.0)  # Min valid
        assert not validate_gps_coordinates(90.1, 0.0, 5.0)  # Over max
        assert not validate_gps_coordinates(-90.1, 0.0, 5.0)  # Under min
    def test_longitude_bounds(self):
        """Test longitude boundary validation"""
        assert validate_gps_coordinates(0.0, 180.0, 5.0)  # Max valid
        assert validate_gps_coordinates(0.0, -180.0, 5.0)  # Min valid
        assert not validate_gps_coordinates(0.0, 180.1, 5.0)  # Over max
        assert not validate_gps_coordinates(0.0, -180.1, 5.0)  # Under min
@pytest.mark.unit
 class TestGPSAnonymization:
    """Test GPS coordinate anonymization"""
    def test_anonymize_10m_precision(self):
        """Test 10m precision anonymization"""
        lat, lon = anonymize_gps(40.712846, -74.005974, precision_meters=10)
        assert lat == pytest.approx(40.71285, abs=0.00001)
        assert lon == pytest.approx(-74.00597, abs=0.00001)
    def test_anonymize_100m_precision(self):
        """Test 100m precision anonymization"""
        lat, lon = anonymize_gps(40.712846, -74.005974, precision_meters=100)
        assert lat == pytest.approx(40.713, abs=0.001)
        assert lon == pytest.approx(-74.006, abs=0.001)
    def test_anonymize_1km_precision(self):
        """Test 1km precision anonymization"""
        lat, lon = anonymize_gps(40.712846, -74.005974, precision_meters=1000)
        assert lat == pytest.approx(40.71, abs=0.01)
        assert lon == pytest.approx(-74.01, abs=0.01)
@pytest.mark.unit
 class TestDistanceCalculation:
    """Test Haversine distance calculation"""
    def test_distance_same_point(self):
        """Test distance between same point"""
        distance = calculate_distance(40.7128, -74.0060, 40.7128, -74.0060)
        assert distance == pytest.approx(0.0, abs=0.001)
    def test_distance_nyc_to_london(self):
        """Test distance NYC to London"""
        # NYC: 40.7128, -74.0060
        # London: 51.5074, -0.1278
        distance = calculate_distance(40.7128, -74.0060, 51.5074, -0.1278)
        assert distance == pytest.approx(5585, abs=10)  # ~5585 km
    def test_distance_symmetry(self):
        """Test distance calculation is symmetric"""
        d1 = calculate_distance(40.7128, -74.0060, 51.5074, -0.1278)
        d2 = calculate_distance(51.5074, -0.1278, 40.7128, -74.0060)
        assert d1 == pytest.approx(d2)
@pytest.mark.unit
 class TestGPSCoordinate:
    """Test GPSCoordinate dataclass"""
    def test_create_valid_coordinate(self):
        """Test creating valid GPS coordinate"""
        coord = GPSCoordinate(
            latitude=40.7128,
            longitude=-74.0060,
            accuracy=5.0
        )
        assert coord.latitude == 40.7128
        assert coord.longitude == -74.0060
        assert coord.accuracy == 5.0
    def test_create_invalid_coordinate(self):
        """Test creating invalid GPS coordinate raises ValueError"""
        with pytest.raises(ValueError):
            GPSCoordinate(
                latitude=91.0,  # Invalid
                longitude=0.0,
                accuracy=5.0
            )
    def test_is_high_quality(self):
        """Test high-quality detection"""
        high_quality = GPSCoordinate(40.7128, -74.0060, accuracy=8.0)
        assert high_quality.is_high_quality()
        medium_quality = GPSCoordinate(40.7128, -74.0060, accuracy=25.0)
        assert not medium_quality.is_high_quality()
    def test_to_dict(self):
        """Test serialization to dictionary"""
        coord = GPSCoordinate(40.7128, -74.0060, accuracy=5.0)
        data = coord.to_dict()
        assert data['latitude'] == 40.7128
        assert data['longitude'] == -74.0060
        assert data['accuracy'] == 5.0
@pytest.mark.unit
 class TestGPSValidatorClass:
    """Test GPSValidator class with configurable thresholds"""
    def test_default_thresholds(self):
        """Test validator with default thresholds"""
        validator = GPSValidator()
        is_valid, reason = validator.validate(40.7128, -74.0060, 5.0)
        assert is_valid
        assert reason is None
    def test_custom_max_accuracy(self):
        """Test custom maximum accuracy threshold"""
        validator = GPSValidator(max_accuracy=30.0)
        # Should accept 25m
        is_valid, reason = validator.validate(40.7128, -74.0060, 25.0)
        assert is_valid
        # Should reject 35m
        is_valid, reason = validator.validate(40.7128, -74.0060, 35.0)
        assert not is_valid
        assert "accuracy" in reason.lower()
    def test_strict_mode(self):
        """Test strict mode (requires accuracy data)"""
        validator = GPSValidator(strict_mode=True)
        # Should reject without accuracy
        is_valid, reason = validator.validate(40.7128, -74.0060, None)
        assert not is_valid
        assert "accuracy" in reason.lower()
    def test_allow_null_island(self):
        """Test allowing Null Island coordinates"""
        validator = GPSValidator(allow_null_island=True)
        is_valid, reason = validator.validate(0.0, 0.0, 5.0)
        assert is_valid
    def test_validation_statistics(self):
        """Test validation statistics tracking"""
        validator = GPSValidator()
        # Validate several coordinates
        validator.validate(40.7128, -74.0060, 5.0)  # Valid
        validator.validate(91.0, 0.0, 5.0)  # Invalid latitude
        validator.validate(0.0, 0.0, 5.0)  # Null Island
        stats = validator.get_statistics()
        assert stats['total_validated'] == 3
        assert stats['total_accepted'] == 1
        assert stats['total_rejected'] == 2
        assert stats['acceptance_rate'] == pytest.approx(1/3, abs=0.01)
        assert 'out_of_bounds' in stats['rejection_reasons']
        assert 'null_island' in stats['rejection_reasons']
    def test_reset_statistics(self):
        """Test statistics reset"""
        validator = GPSValidator()
        validator.validate(40.7128, -74.0060, 5.0)
        validator.reset_statistics()
        stats = validator.get_statistics()
        assert stats['total_validated'] == 0
        assert stats['total_accepted'] == 0