Add comprehensive testing infrastructure
- Testing strategy document - Unit tests for GPS validator - Integration test conftest with fixtures - Sample .sub files and manifests - Comprehensive Termux testing guide - Test directory structure
This commit is contained in:
@@ -0,0 +1,418 @@
|
||||
# GigLez Testing Strategy - Phase 1 & 2 Validation
|
||||
|
||||
**Branch**: `p1-p2-validation`
|
||||
**Created**: 2026-01-12
|
||||
**Purpose**: Comprehensive testing before deployment
|
||||
|
||||
---
|
||||
|
||||
## Testing Pyramid
|
||||
|
||||
```
|
||||
┌─────────────┐
|
||||
│ Manual │ ← Termux workflow testing
|
||||
│ Testing │
|
||||
└─────────────┘
|
||||
┌───────────────────┐
|
||||
│ Integration │ ← API endpoint testing
|
||||
│ Tests │
|
||||
└───────────────────┘
|
||||
┌─────────────────────────┐
|
||||
│ Unit Tests │ ← Component testing
|
||||
│ │
|
||||
└─────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Unit Tests
|
||||
|
||||
### 1. GPS Validator Tests
|
||||
**File**: `tests/unit/test_gps_validator.py`
|
||||
|
||||
**Test Cases**:
|
||||
- ✅ Valid coordinates (NYC, London, Tokyo)
|
||||
- ✅ Invalid latitude (> 90, < -90)
|
||||
- ✅ Invalid longitude (> 180, < -180)
|
||||
- ✅ Null Island detection (0.0, 0.0)
|
||||
- ✅ Accuracy thresholds (< 50m pass, > 50m fail)
|
||||
- ✅ High-quality detection (< 10m)
|
||||
- ✅ Anonymization (precision reduction)
|
||||
- ✅ Distance calculation (Haversine)
|
||||
|
||||
### 2. .sub File Parser Tests
|
||||
**File**: `tests/unit/test_sub_parser.py`
|
||||
|
||||
**Test Cases**:
|
||||
- ✅ Valid KEY format file
|
||||
- ✅ Valid RAW format file
|
||||
- ✅ Valid BinRAW format file
|
||||
- ✅ Invalid file format (missing fields)
|
||||
- ✅ Invalid frequency (out of range)
|
||||
- ✅ Metadata extraction (protocol, frequency, modulation)
|
||||
- ✅ Key data parsing (hex to bytes)
|
||||
- ✅ Raw data parsing (timing array)
|
||||
|
||||
### 3. Storage Backend Tests
|
||||
**File**: `tests/unit/test_storage.py`
|
||||
|
||||
**Test Cases**:
|
||||
- ✅ LocalStorage: save file
|
||||
- ✅ LocalStorage: retrieve file
|
||||
- ✅ LocalStorage: file exists check
|
||||
- ✅ LocalStorage: delete file
|
||||
- ✅ LocalStorage: SHA256 path sharding
|
||||
- ✅ LocalStorage: storage statistics
|
||||
- ✅ S3Storage: mock save/retrieve (if boto3 available)
|
||||
|
||||
### 4. Database Models Tests
|
||||
**File**: `tests/unit/test_models.py`
|
||||
|
||||
**Test Cases**:
|
||||
- ✅ User model creation
|
||||
- ✅ Session model creation
|
||||
- ✅ Device model creation
|
||||
- ✅ Capture model creation (with GPS validation)
|
||||
- ✅ Capture deduplication (file_hash primary key)
|
||||
- ✅ Relationships (user -> captures, session -> captures)
|
||||
- ✅ to_dict() serialization
|
||||
|
||||
### 5. Configuration Tests
|
||||
**File**: `tests/unit/test_settings.py`
|
||||
|
||||
**Test Cases**:
|
||||
- ✅ Development mode detection
|
||||
- ✅ Production mode detection
|
||||
- ✅ Environment variable loading
|
||||
- ✅ Settings validation (production checks)
|
||||
- ✅ Computed properties (use_redis, use_celery)
|
||||
- ✅ Database URL generation
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Integration Tests
|
||||
|
||||
### 1. API Startup Tests
|
||||
**File**: `tests/integration/test_api_startup.py`
|
||||
|
||||
**Test Cases**:
|
||||
- ✅ Application starts successfully
|
||||
- ✅ Database connection established
|
||||
- ✅ PostGIS extension available
|
||||
- ✅ Storage backend initialized
|
||||
- ✅ Health endpoint responds
|
||||
- ✅ OpenAPI docs available (dev mode)
|
||||
|
||||
### 2. Upload Endpoint Tests
|
||||
**File**: `tests/integration/test_upload.py`
|
||||
|
||||
**Test Cases**:
|
||||
- ✅ Upload single .sub file with manifest
|
||||
- ✅ Upload multiple .sub files
|
||||
- ✅ Duplicate file detection (same hash)
|
||||
- ✅ Invalid manifest JSON (400 error)
|
||||
- ✅ Missing manifest entry (file skipped)
|
||||
- ✅ Invalid GPS coordinates (file rejected)
|
||||
- ✅ Invalid .sub file format (file rejected)
|
||||
- ✅ Session creation
|
||||
- ✅ Session reuse (existing UUID)
|
||||
- ✅ File stored in storage backend
|
||||
- ✅ Capture record created in database
|
||||
- ✅ Response format validation
|
||||
|
||||
### 3. Query Endpoint Tests
|
||||
**File**: `tests/integration/test_query.py`
|
||||
|
||||
**Test Cases**:
|
||||
- ✅ Get capture by file_hash
|
||||
- ✅ 404 for non-existent capture
|
||||
- ✅ Device listing (when implemented)
|
||||
- ✅ Statistics endpoint (when implemented)
|
||||
|
||||
### 4. Storage Integration Tests
|
||||
**File**: `tests/integration/test_storage_integration.py`
|
||||
|
||||
**Test Cases**:
|
||||
- ✅ Upload → Storage → Retrieve (round trip)
|
||||
- ✅ Multiple uploads (different files)
|
||||
- ✅ File sharding verification
|
||||
- ✅ Storage statistics accuracy
|
||||
|
||||
### 5. Database Integration Tests
|
||||
**File**: `tests/integration/test_database.py`
|
||||
|
||||
**Test Cases**:
|
||||
- ✅ Create tables (if not exists)
|
||||
- ✅ Insert capture with GPS
|
||||
- ✅ PostGIS geometry auto-population
|
||||
- ✅ Spatial query (within radius)
|
||||
- ✅ Trigger execution (session stats update)
|
||||
- ✅ Deduplication (unique file_hash)
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Manual Testing (Termux)
|
||||
|
||||
### 1. Environment Setup Tests
|
||||
**File**: `tests/manual/termux_setup.md`
|
||||
|
||||
**Test Cases**:
|
||||
- ✅ PostgreSQL installation
|
||||
- ✅ Database creation
|
||||
- ✅ Schema creation (no errors)
|
||||
- ✅ PostGIS extension enabled
|
||||
- ✅ Python dependencies installation
|
||||
- ✅ Settings validation passes
|
||||
- ✅ Storage directory creation
|
||||
|
||||
### 2. API Server Tests
|
||||
**File**: `tests/manual/termux_api.md`
|
||||
|
||||
**Test Cases**:
|
||||
- ✅ Server starts on localhost:8000
|
||||
- ✅ Health check responds
|
||||
- ✅ OpenAPI docs accessible
|
||||
- ✅ CORS headers present
|
||||
- ✅ Request logging works
|
||||
- ✅ Exception handling (test invalid request)
|
||||
|
||||
### 3. Upload Workflow Tests
|
||||
**File**: `tests/manual/termux_upload.md`
|
||||
|
||||
**Test Cases**:
|
||||
- ✅ Create test .sub file
|
||||
- ✅ Create test manifest
|
||||
- ✅ Upload via curl
|
||||
- ✅ Verify response (201, success message)
|
||||
- ✅ Check file in storage directory
|
||||
- ✅ Query capture from database
|
||||
- ✅ Verify GPS coordinates
|
||||
- ✅ Upload duplicate (verify deduplication)
|
||||
|
||||
### 4. Real-World Workflow Tests
|
||||
**File**: `tests/manual/termux_realworld.md`
|
||||
|
||||
**Test Cases**:
|
||||
- ✅ Capture actual signal with T-Embed (if available)
|
||||
- ✅ Save .sub file to device
|
||||
- ✅ Get GPS coordinates (termux-location)
|
||||
- ✅ Create manifest from GPS data
|
||||
- ✅ Upload to API
|
||||
- ✅ Verify in database
|
||||
- ✅ Query back by hash
|
||||
- ✅ Verify storage location
|
||||
|
||||
---
|
||||
|
||||
## Test Data & Fixtures
|
||||
|
||||
### 1. Sample .sub Files
|
||||
**Directory**: `tests/fixtures/sub_files/`
|
||||
|
||||
**Files Needed**:
|
||||
- `valid_key_433mhz.sub` - Standard KEY format (Princeton)
|
||||
- `valid_raw_315mhz.sub` - RAW format with timing data
|
||||
- `valid_binraw_868mhz.sub` - BinRAW format
|
||||
- `invalid_missing_freq.sub` - Missing frequency field
|
||||
- `invalid_bad_format.sub` - Corrupted file
|
||||
|
||||
### 2. Sample Manifests
|
||||
**Directory**: `tests/fixtures/manifests/`
|
||||
|
||||
**Files Needed**:
|
||||
- `valid_single.json` - Single capture
|
||||
- `valid_multiple.json` - Multiple captures
|
||||
- `valid_session.json` - With session UUID
|
||||
- `invalid_missing_gps.json` - Missing coordinates
|
||||
- `invalid_bad_json.json` - Malformed JSON
|
||||
|
||||
### 3. Database Fixtures
|
||||
**Directory**: `tests/fixtures/database/`
|
||||
|
||||
**Files Needed**:
|
||||
- `sample_devices.sql` - Sample device records
|
||||
- `sample_signatures.sql` - Sample signature records
|
||||
- `sample_users.sql` - Test users
|
||||
|
||||
---
|
||||
|
||||
## Test Execution Plan
|
||||
|
||||
### Step 1: Unit Tests (Local)
|
||||
```bash
|
||||
# Install test dependencies
|
||||
pip install pytest pytest-asyncio pytest-cov
|
||||
|
||||
# Run unit tests
|
||||
pytest tests/unit/ -v
|
||||
|
||||
# Run with coverage
|
||||
pytest tests/unit/ --cov=src --cov-report=html
|
||||
```
|
||||
|
||||
**Expected**: All unit tests pass (green)
|
||||
|
||||
### Step 2: Integration Tests (Local/Termux)
|
||||
```bash
|
||||
# Setup test database
|
||||
./scripts/setup_database.sh
|
||||
psql -U giglez_user -d giglez -h localhost -f scripts/create_schema.sql
|
||||
|
||||
# Run integration tests
|
||||
pytest tests/integration/ -v
|
||||
|
||||
# Run all tests
|
||||
pytest tests/ -v
|
||||
```
|
||||
|
||||
**Expected**: All integration tests pass (green)
|
||||
|
||||
### Step 3: Manual Testing (Termux)
|
||||
```bash
|
||||
# Start API server
|
||||
python src/api/main.py
|
||||
|
||||
# In another terminal, run manual tests
|
||||
bash tests/manual/run_manual_tests.sh
|
||||
```
|
||||
|
||||
**Expected**: All manual tests succeed
|
||||
|
||||
### Step 4: Real-World Testing (Termux + T-Embed)
|
||||
```bash
|
||||
# Follow tests/manual/termux_realworld.md
|
||||
# Capture actual signals, upload, verify
|
||||
```
|
||||
|
||||
**Expected**: Full workflow works end-to-end
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
### Phase 1 (Database & Core)
|
||||
- ✅ All unit tests pass (GPS, parser, storage, models)
|
||||
- ✅ Database schema creates without errors
|
||||
- ✅ PostGIS queries work
|
||||
- ✅ Models can be instantiated and saved
|
||||
|
||||
### Phase 2 (API)
|
||||
- ✅ API server starts successfully
|
||||
- ✅ Upload endpoint accepts files
|
||||
- ✅ Deduplication works (duplicate uploads rejected)
|
||||
- ✅ Files stored correctly
|
||||
- ✅ Database records created
|
||||
- ✅ Query endpoints return data
|
||||
|
||||
### Phase 3 (Termux Workflow)
|
||||
- ✅ Can install and run on Termux
|
||||
- ✅ Can upload .sub files via curl
|
||||
- ✅ Can query captures
|
||||
- ✅ Storage persists across restarts
|
||||
- ✅ Database persists across restarts
|
||||
|
||||
---
|
||||
|
||||
## Test Configuration
|
||||
|
||||
### pytest.ini
|
||||
```ini
|
||||
[pytest]
|
||||
testpaths = tests
|
||||
python_files = test_*.py
|
||||
python_classes = Test*
|
||||
python_functions = test_*
|
||||
addopts =
|
||||
-v
|
||||
--strict-markers
|
||||
--tb=short
|
||||
markers =
|
||||
unit: Unit tests (fast, no external dependencies)
|
||||
integration: Integration tests (database, API)
|
||||
manual: Manual tests (human verification)
|
||||
slow: Slow tests (skip in CI)
|
||||
```
|
||||
|
||||
### conftest.py (Shared Fixtures)
|
||||
```python
|
||||
import pytest
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
|
||||
@pytest.fixture
|
||||
def db_session():
|
||||
"""Test database session"""
|
||||
# Create test database session
|
||||
pass
|
||||
|
||||
@pytest.fixture
|
||||
def test_storage():
|
||||
"""Test storage backend"""
|
||||
# Create temporary storage
|
||||
pass
|
||||
|
||||
@pytest.fixture
|
||||
def test_client():
|
||||
"""FastAPI test client"""
|
||||
from fastapi.testclient import TestClient
|
||||
from src.api.main import app
|
||||
return TestClient(app)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Bug Tracking
|
||||
|
||||
### Issues Found During Testing
|
||||
**Template for each issue**:
|
||||
|
||||
```markdown
|
||||
## Issue #N: [Brief Description]
|
||||
|
||||
**Severity**: Critical / High / Medium / Low
|
||||
**Component**: GPS Validator / Parser / API / Database
|
||||
**Found In**: Unit Test / Integration Test / Manual Test
|
||||
|
||||
### Reproduction
|
||||
Steps to reproduce the issue
|
||||
|
||||
### Expected Behavior
|
||||
What should happen
|
||||
|
||||
### Actual Behavior
|
||||
What actually happens
|
||||
|
||||
### Fix
|
||||
How to fix it
|
||||
|
||||
### Test Coverage
|
||||
New test to prevent regression
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Documentation Updates Needed
|
||||
|
||||
Based on test results:
|
||||
- [ ] Update README with verified installation steps
|
||||
- [ ] Document any configuration changes needed
|
||||
- [ ] Add troubleshooting section for common issues
|
||||
- [ ] Update API examples with working curl commands
|
||||
- [ ] Document Termux-specific setup steps
|
||||
|
||||
---
|
||||
|
||||
## Next Steps After Validation
|
||||
|
||||
1. ✅ Fix all critical bugs
|
||||
2. ✅ Add regression tests for fixed bugs
|
||||
3. ✅ Update documentation based on findings
|
||||
4. ✅ Merge `p1-p2-validation` → `master`
|
||||
5. ✅ Tag release: `v0.1.0-alpha`
|
||||
6. ✅ Begin Phase 2 completion (auth, background tasks)
|
||||
|
||||
---
|
||||
|
||||
**Testing Status**: Ready to Execute
|
||||
**Branch**: `p1-p2-validation`
|
||||
**Next Action**: Create test files and fixtures
|
||||
Reference in New Issue
Block a user