# GigLez Deployment Considerations: Development vs Production

**Created**: 2026-01-12
**Purpose**: Identify critical differences between local development and production deployment

---

## Executive Summary

GigLez needs to support two distinct deployment scenarios:

1. **Local Development** (Termux on Android)
   - Resource-constrained (mobile device)
   - Single-user access
   - Direct USB serial communication
   - Local file storage
   - Development/testing workflow

2. **Production Server** (Cloud/VPS)
   - Multi-user platform
   - Public API access
   - No hardware connection
   - Object storage for files
   - High availability requirements

**Key Insight**: We need a **modular architecture** that supports both scenarios without code duplication.

---

## Architecture Strategy

### Hybrid Architecture: "Capture" vs "Platform" Mode

```
┌─────────────────────────────────────────────────────────┐
│                    GigLez Application                    │
├─────────────────────────────────────────────────────────┤
│                                                           │
│  ┌────────────────┐         ┌────────────────┐          │
│  │  Capture Mode  │         │ Platform Mode  │          │
│  │   (Termux)     │         │   (Server)     │          │
│  ├────────────────┤         ├────────────────┤          │
│  │ - T-Embed USB  │         │ - No hardware  │          │
│  │ - Local GPS    │         │ - File uploads │          │
│  │ - Auto-capture │         │ - Multi-user   │          │
│  │ - Local API    │         │ - Public API   │          │
│  └────────────────┘         └────────────────┘          │
│           │                          │                    │
│           └──────────┬───────────────┘                    │
│                      │                                    │
│              ┌───────▼────────┐                           │
│              │  Shared Core   │                           │
│              ├────────────────┤                           │
│              │ - Database     │                           │
│              │ - Parser       │                           │
│              │ - Matcher      │                           │
│              │ - GPS Validator│                           │
│              │ - API Endpoints│                           │
│              └────────────────┘                           │
└─────────────────────────────────────────────────────────┘
```

---

## Detailed Comparison

### 1. Hardware Access

#### Development (Termux)
- **T-Embed Connection**: USB serial communication (pyserial)
- **GPS Source**: Android native GPS via SL4A or Termux:API
- **Real-time Capture**: Direct signal scanning and capture
- **Storage**: Local filesystem on Android device

**Implications**:
- Need hardware abstraction layer
- Support offline operation
- Handle USB disconnections
- Batch upload when network available

#### Production (Server)
- **No Hardware**: Accept pre-captured .sub files only
- **GPS Source**: Embedded in upload manifest
- **No Real-time**: Historical data only
- **Storage**: Object storage (S3, MinIO, or filesystem)

**Implications**:
- No serial communication code needed
- Network-dependent operation
- Focus on file processing pipeline
- Horizontal scaling possible

**Solution**:
```python
# Abstract hardware interface
class CaptureSource(ABC):
    @abstractmethod
    def scan_signals(self): pass

    @abstractmethod
    def get_gps(self): pass

class TEmbedSource(CaptureSource):
    """Development: Direct hardware"""
    pass

class UploadSource(CaptureSource):
    """Production: File uploads"""
    pass
```

---

### 2. Database Configuration

#### Development (Termux)
- **PostgreSQL**: Local installation via `pkg install postgresql`
- **Host**: localhost or 127.0.0.1
- **Port**: 5432
- **Connections**: Small pool (5 + 5 overflow)
- **Storage**: Device internal storage or SD card

**Considerations**:
- Limited RAM on mobile devices
- Smaller connection pool
- May need to tune PostgreSQL for mobile
- Backup to SD card or cloud

#### Production (Server)
- **PostgreSQL**: Dedicated server or managed database (RDS, Cloud SQL)
- **Host**: Remote database server
- **Port**: 5432 (or custom)
- **Connections**: Larger pool (20 + 40 overflow)
- **Storage**: SSD/NVMe for performance

**Considerations**:
- Connection pooling via PgBouncer
- Read replicas for scaling
- Automated backups
- Point-in-time recovery

**Solution**: Environment-based configuration
```bash
# Development (.env)
GIGLEZ_DB_HOST=localhost
GIGLEZ_DB_POOL_SIZE=5
GIGLEZ_DB_MAX_OVERFLOW=5

# Production (.env.production)
GIGLEZ_DB_HOST=db.giglez.com
GIGLEZ_DB_POOL_SIZE=20
GIGLEZ_DB_MAX_OVERFLOW=40
```

---

### 3. File Storage

#### Development (Termux)
- **Location**: `/data/data/com.termux/files/home/giglez/storage/`
- **Structure**: Organized by session/date
- **Size**: Limited by device storage (typically < 64GB)
- **Access**: Direct filesystem access

**Structure**:
```
storage/
├── captures/
│   ├── 2026-01-12/
│   │   ├── session_abc123/
│   │   │   ├── capture_001.sub
│   │   │   ├── capture_002.sub
│   │   │   └── manifest.json
```

#### Production (Server)
- **Location**: Object storage (S3, MinIO, GCS) or networked filesystem
- **Structure**: Content-addressed by file hash
- **Size**: Unlimited (cloud storage)
- **Access**: Via storage API or CDN

**Structure**:
```
s3://giglez-captures/
├── ab/
│   ├── c1/
│   │   └── abc123...def456.sub  # First 2+2 chars of hash for sharding
```

**Solution**: Storage abstraction
```python
class StorageBackend(ABC):
    @abstractmethod
    def save_file(self, file_hash: str, content: bytes): pass

    @abstractmethod
    def get_file(self, file_hash: str) -> bytes: pass

class LocalStorage(StorageBackend):
    """Development: Filesystem"""
    pass

class S3Storage(StorageBackend):
    """Production: S3-compatible storage"""
    pass
```

---

### 4. API Access & Security

#### Development (Termux)
- **Access**: Local only (127.0.0.1) or local network
- **Port**: 8000 (or any available)
- **HTTPS**: Not required (localhost)
- **Authentication**: Optional (single user)
- **CORS**: Permissive (development)
- **Rate Limiting**: None needed

**Use Cases**:
- Testing API endpoints
- Web UI development
- Local data management
- Debugging

#### Production (Server)
- **Access**: Public internet
- **Port**: 443 (HTTPS) behind reverse proxy
- **HTTPS**: Required (Let's Encrypt)
- **Authentication**: JWT + API keys mandatory
- **CORS**: Strict origin whitelisting
- **Rate Limiting**: Aggressive (per-user, per-IP)

**Security Requirements**:
- Nginx/Caddy reverse proxy
- SSL/TLS certificates
- API key rotation
- Request validation
- SQL injection prevention
- File upload limits
- DDoS protection

**Solution**: Environment-based security
```python
# config/security.py
if os.getenv("GIGLEZ_MODE") == "production":
    REQUIRE_AUTH = True
    RATE_LIMIT = "10/minute"
    ALLOWED_ORIGINS = ["https://giglez.com"]
else:
    REQUIRE_AUTH = False
    RATE_LIMIT = None
    ALLOWED_ORIGINS = ["*"]
```

---

### 5. Performance & Scaling

#### Development (Termux)
- **Workers**: 1 Uvicorn worker
- **Concurrency**: Minimal (single user)
- **Caching**: In-memory only (LRU)
- **Background Tasks**: Simple queue or synchronous
- **Resource Limits**: RAM < 2GB, CPU 2-4 cores

**Optimization Focus**:
- Memory efficiency
- Minimal background processes
- Battery life considerations
- Offline operation

#### Production (Server)
- **Workers**: Multiple Uvicorn workers (4-8)
- **Concurrency**: High (100+ concurrent users)
- **Caching**: Redis + LRU
- **Background Tasks**: Celery + RabbitMQ/Redis
- **Resource Limits**: RAM 8-32GB, CPU 4-16 cores

**Optimization Focus**:
- Query performance
- Connection pooling
- Horizontal scaling
- CDN for static assets
- Load balancing

**Solution**: Conditional imports
```python
# src/api/main.py
if PRODUCTION_MODE:
    from celery import Celery
    celery_app = Celery('giglez', broker='redis://localhost')
else:
    # Use FastAPI BackgroundTasks for development
    celery_app = None
```

---

### 6. Logging & Monitoring

#### Development (Termux)
- **Logging**: Console output (stdout)
- **Level**: DEBUG
- **Format**: Colorized, verbose
- **Storage**: Optional file logging
- **Monitoring**: Manual observation

**Example**:
```python
logger.add(sys.stdout, level="DEBUG", colorize=True)
```

#### Production (Server)
- **Logging**: Structured JSON to file + syslog
- **Level**: INFO/WARNING
- **Format**: JSON for parsing
- **Storage**: Log rotation + cloud storage
- **Monitoring**: Prometheus + Grafana, error tracking (Sentry)

**Example**:
```python
logger.add(
    "/var/log/giglez/app.log",
    level="INFO",
    format="{time:YYYY-MM-DD HH:mm:ss} | {level} | {message}",
    rotation="100 MB",
    compression="gz"
)
```

**Solution**: Environment-based logging
```python
# config/logging.py
if PRODUCTION_MODE:
    setup_production_logging()
else:
    setup_development_logging()
```

---

### 7. Backup & Disaster Recovery

#### Development (Termux)
- **Database**: Manual pg_dump to SD card
- **Files**: Synced to cloud storage (Google Drive, Dropbox)
- **Frequency**: On-demand
- **Recovery**: Restore from local backups

#### Production (Server)
- **Database**: Automated daily backups with PITR
- **Files**: Object storage replication (S3 versioning)
- **Frequency**: Hourly incremental, daily full
- **Recovery**: Automated restore procedures

**Backup Strategy**:
```bash
# Development
pg_dump giglez > /sdcard/giglez_backup_$(date +%Y%m%d).sql

# Production
pg_dump giglez | gzip | aws s3 cp - s3://giglez-backups/$(date +%Y%m%d).sql.gz
```

---

### 8. Network Considerations

#### Development (Termux)
- **Connectivity**: WiFi or cellular (intermittent)
- **Bandwidth**: Potentially limited (mobile data)
- **Latency**: Variable (mobile networks)
- **Offline Support**: Critical (wardriving in field)

**Implications**:
- Queue uploads for when network available
- Support offline capture and analysis
- Minimize API calls
- Compress uploads

#### Production (Server)
- **Connectivity**: Always online (datacenter)
- **Bandwidth**: High (1Gbps+)
- **Latency**: Low (< 50ms typical)
- **Offline Support**: Not applicable

**Implications**:
- Assume network availability
- No offline mode needed
- Can make external API calls freely
- Stream large responses

**Solution**: Offline queue
```python
# src/capture/uploader.py
class UploadQueue:
    def queue_capture(self, capture_data):
        if network_available():
            upload_immediately(capture_data)
        else:
            save_to_local_queue(capture_data)

    def process_queue(self):
        """Called when network becomes available"""
        for item in get_queued_items():
            try:
                upload_immediately(item)
                mark_as_uploaded(item)
            except NetworkError:
                break  # Stop and try again later
```

---

## Configuration Strategy

### Environment Variables Approach

Use different `.env` files for each environment:

```bash
# .env.development (Termux)
GIGLEZ_MODE=development
GIGLEZ_DB_HOST=localhost
GIGLEZ_DB_POOL_SIZE=5
GIGLEZ_API_HOST=127.0.0.1
GIGLEZ_API_PORT=8000
GIGLEZ_STORAGE_TYPE=filesystem
GIGLEZ_STORAGE_PATH=/data/data/com.termux/files/home/giglez/storage
GIGLEZ_REQUIRE_AUTH=false
GIGLEZ_LOG_LEVEL=DEBUG
GIGLEZ_ENABLE_HARDWARE=true
GIGLEZ_TEMBED_PORT=/dev/ttyUSB0

# .env.production (Server)
GIGLEZ_MODE=production
GIGLEZ_DB_HOST=db.giglez.com
GIGLEZ_DB_POOL_SIZE=20
GIGLEZ_API_HOST=0.0.0.0
GIGLEZ_API_PORT=8000
GIGLEZ_STORAGE_TYPE=s3
GIGLEZ_STORAGE_BUCKET=giglez-captures
GIGLEZ_REQUIRE_AUTH=true
GIGLEZ_LOG_LEVEL=INFO
GIGLEZ_ENABLE_HARDWARE=false
```

### Configuration Loading

```python
# config/settings.py
import os
from enum import Enum

class DeploymentMode(Enum):
    DEVELOPMENT = "development"
    PRODUCTION = "production"

class Settings:
    def __init__(self):
        self.mode = DeploymentMode(os.getenv("GIGLEZ_MODE", "development"))

    @property
    def is_production(self) -> bool:
        return self.mode == DeploymentMode.PRODUCTION

    @property
    def is_development(self) -> bool:
        return self.mode == DeploymentMode.DEVELOPMENT

    @property
    def enable_hardware(self) -> bool:
        """Only enable hardware access in development"""
        return os.getenv("GIGLEZ_ENABLE_HARDWARE", "false").lower() == "true"

    @property
    def storage_backend(self) -> str:
        return os.getenv("GIGLEZ_STORAGE_TYPE", "filesystem")

settings = Settings()
```

---

## Recommended Architecture

### Modular Component Design

```
giglez/
├── src/
│   ├── core/                    # Shared core (both modes)
│   │   ├── database/            # Database models & queries
│   │   ├── parser/              # .sub file parser
│   │   ├── matcher/             # Signature matching
│   │   ├── gps/                 # GPS validation
│   │   └── storage/             # Storage abstraction
│   │
│   ├── capture/                 # Capture mode only (Termux)
│   │   ├── tembed.py            # T-Embed communication
│   │   ├── scanner.py           # Signal scanning
│   │   ├── gps_manager.py       # Android GPS integration
│   │   └── uploader.py          # Offline queue & upload
│   │
│   ├── api/                     # API endpoints (both modes)
│   │   ├── main.py              # FastAPI app
│   │   ├── routes/
│   │   │   ├── captures.py      # Upload endpoints
│   │   │   ├── devices.py       # Device catalog
│   │   │   ├── query.py         # Search endpoints
│   │   │   └── stats.py         # Statistics
│   │   ├── auth.py              # Authentication
│   │   ├── middleware.py        # Rate limiting, CORS
│   │   └── dependencies.py      # Dependency injection
│   │
│   └── web/                     # Web interface (both modes)
│       ├── static/              # CSS, JS, images
│       └── templates/           # HTML templates
│
└── config/
    ├── settings.py              # Environment-based config
    ├── database.py              # Database connection
    ├── storage.py               # Storage backend selection
    └── logging.py               # Logging configuration
```

---

## Implementation Recommendations

### 1. Use Feature Flags

```python
# src/api/main.py
from config.settings import settings

app = FastAPI()

# Conditional hardware endpoints
if settings.enable_hardware:
    @app.post("/api/capture/start")
    async def start_capture():
        # Only available in Termux mode
        pass

# Always available endpoints
@app.post("/api/captures/upload")
async def upload_captures():
    # Available in both modes
    pass
```

### 2. Storage Abstraction

```python
# config/storage.py
from src.core.storage import LocalStorage, S3Storage

def get_storage_backend():
    if settings.storage_backend == "s3":
        return S3Storage(
            bucket=os.getenv("GIGLEZ_STORAGE_BUCKET"),
            access_key=os.getenv("AWS_ACCESS_KEY_ID"),
            secret_key=os.getenv("AWS_SECRET_ACCESS_KEY")
        )
    else:
        return LocalStorage(
            base_path=os.getenv("GIGLEZ_STORAGE_PATH")
        )

storage = get_storage_backend()
```

### 3. Optional Dependencies

```python
# requirements.txt (core)
fastapi==0.109.0
sqlalchemy==2.0.25
psycopg2-binary==2.9.9
...

# requirements-dev.txt (Termux additions)
-r requirements.txt
pyserial==3.5
aioserial==1.3.2

# requirements-prod.txt (Server additions)
-r requirements.txt
boto3==1.34.0  # S3 storage
redis==5.0.1   # Caching
celery==5.3.4  # Background tasks
gunicorn==21.2.0  # WSGI server
```

### 4. Graceful Degradation

```python
# src/api/routes/captures.py
@app.post("/api/captures/upload")
async def upload_captures(files: List[UploadFile]):
    # Core functionality works in both modes
    captures = []

    for file in files:
        # Parse .sub file (works everywhere)
        metadata = parse_sub_file(file)

        # Store file (abstracted)
        storage.save_file(metadata.file_hash, file.content)

        # Save to database (works everywhere)
        capture = Capture(**metadata)
        db.add(capture)

        captures.append(capture)

    db.commit()

    # Background matching (production) or synchronous (development)
    if settings.is_production:
        celery_app.send_task('match_signatures', args=[capture.file_hash])
    else:
        match_signatures_sync(capture.file_hash)

    return {"uploaded": len(captures)}
```

---

## Testing Strategy

### Development Testing (Termux)
- Direct hardware testing
- USB serial communication
- GPS acquisition
- Offline queue processing
- Local file storage

### Production Testing (Server)
- API endpoint testing
- File upload validation
- Multi-user concurrency
- Rate limiting
- Authentication
- S3 storage integration

### Shared Testing
- Database operations
- .sub file parsing
- Signature matching
- GPS validation
- Query performance

**Test Structure**:
```
tests/
├── unit/                    # Unit tests (both modes)
│   ├── test_parser.py
│   ├── test_matcher.py
│   └── test_gps.py
│
├── integration/             # Integration tests
│   ├── test_api.py          # Both modes
│   ├── test_hardware.py     # Development only
│   └── test_storage.py      # Both modes
│
└── e2e/                     # End-to-end tests
    ├── test_capture.py      # Development only
    └── test_upload.py       # Both modes
```

---

## Deployment Checklist

### Development (Termux) Setup
- [ ] Install Termux from F-Droid
- [ ] Install PostgreSQL: `pkg install postgresql`
- [ ] Install Python: `pkg install python`
- [ ] Install dependencies: `pip install -r requirements-dev.txt`
- [ ] Configure USB permissions for T-Embed
- [ ] Set up Termux:API for GPS
- [ ] Create `.env.development`
- [ ] Run database setup: `./scripts/setup_database.sh`
- [ ] Test hardware connection
- [ ] Start API: `python src/api/main.py`

### Production (Server) Setup
- [ ] Provision VPS (2GB+ RAM, 2+ cores)
- [ ] Install PostgreSQL 16 + PostGIS
- [ ] Install Python 3.10+
- [ ] Install Redis (for caching)
- [ ] Install Nginx (reverse proxy)
- [ ] Configure SSL with Let's Encrypt
- [ ] Set up S3/MinIO for file storage
- [ ] Configure firewall (allow 80, 443)
- [ ] Create `.env.production`
- [ ] Run database setup
- [ ] Set up systemd service
- [ ] Configure automated backups
- [ ] Set up monitoring (optional)

---

## Performance Considerations

### Development (Termux) Optimizations
- Smaller connection pool (5 + 5)
- In-memory caching only
- Synchronous processing (no Celery)
- Minimal logging
- Local file storage

### Production (Server) Optimizations
- Larger connection pool (20 + 40)
- Redis caching
- Async background tasks (Celery)
- Structured logging with rotation
- CDN for file delivery
- Read replicas for scaling
- Materialized view refresh (cron)

---

## Cost Considerations

### Development (Termux)
- **Hardware**: LilyGo T-Embed (~$30-50)
- **Android Device**: Existing phone/tablet
- **Infrastructure**: $0 (local)
- **Storage**: Device storage only
- **Total**: ~$30-50 one-time

### Production (Server)
- **VPS**: $10-50/month (2-8GB RAM)
- **Database**: Included or $10-30/month (managed)
- **Storage**: $0.02/GB/month (S3) or included (filesystem)
- **Bandwidth**: Typically included (1-5TB)
- **Domain**: $10-15/year
- **SSL**: $0 (Let's Encrypt)
- **Total**: $10-80/month

---

## Recommended Phase 2 Approach

### Build for Both from the Start

**Strategy**: Implement the API with abstraction layers that work in both environments.

**Implementation Order**:
1. ✅ Core API structure (FastAPI app)
2. ✅ Storage abstraction (filesystem + S3)
3. ✅ Upload endpoint (works in both modes)
4. ✅ Authentication (optional in dev, required in prod)
5. ✅ Background tasks (sync in dev, Celery in prod)
6. ✅ Environment-based configuration

**Key Principle**:
> Write code that works in both environments by default, with conditional imports/features for mode-specific functionality.

---

## Summary: Key Differences

| Aspect | Development (Termux) | Production (Server) |
|--------|---------------------|---------------------|
| **Hardware** | T-Embed USB + GPS | None (file uploads) |
| **Users** | Single user | Multi-user |
| **Database** | Local PostgreSQL (small pool) | Remote PostgreSQL (large pool) |
| **Storage** | Local filesystem | S3/object storage |
| **API Access** | localhost only | Public internet |
| **Security** | Optional auth | Required auth + HTTPS |
| **Background Tasks** | Synchronous | Celery async |
| **Caching** | In-memory LRU | Redis + LRU |
| **Logging** | Console (DEBUG) | File + JSON (INFO) |
| **Monitoring** | Manual | Automated (Prometheus) |
| **Scaling** | N/A | Horizontal |
| **Cost** | $0/month | $10-80/month |

---

## Next Steps

Before implementing Phase 2, we need to:

1. ✅ **Create environment configuration** (`.env.development`, `.env.production`)
2. ✅ **Implement storage abstraction** (`src/core/storage/`)
3. ✅ **Create settings module** (`config/settings.py`)
4. ✅ **Document deployment procedures** (development guide, production guide)
5. ✅ **Begin FastAPI implementation** with environment awareness

**Recommendation**: Start with a **shared API implementation** that works in both modes, then add mode-specific features as needed.

---

**Document Version**: 1.0
**Status**: Ready for Phase 2 Implementation
**Next Action**: Create environment-aware FastAPI application structure