feat: RTL_433 protocol database import - iteration 1/5

- Expanded protocol database from 18 → 299 signatures (16.6x increase) - Imported 281 protocols from RTL_433 open-source database (286 total devices) - Created automated import script: scripts/import_rtl433_protocols.py - Generated rtl433_protocols_imported.py with timing/frequency/modulation data - Updated protocol_database.py to include RTL433_PROTOCOLS - All 26 tests passing Breakdown by category: - Weather: 116 protocols - Sensors: 36 protocols - TPMS: 25 protocols - Security: 23 protocols - Home Automation: 18 protocols - Other: 50+ protocols Frequency coverage: - 433.92 MHz: 248 protocols - 315.00 MHz: 32 protocols - 915.00 MHz: 1 protocol This provides comprehensive coverage of Sub-GHz IoT devices for accurate identification from raw RF captures.
2026-02-14 18:55:55 -08:00
parent 01b06fadc6
commit 9f73595b20
32 changed files with 15365 additions and 50 deletions
@@ -0,0 +1,513 @@
+# GigLez Installation Guide
+
+## Quick Start (Recommended)
+
+### 1. **Using the Safe Install Script**
+
+```bash
+# Navigate to project directory
+cd /home/dell/coding/giglez
+
+# Create virtual environment and install all dependencies
+./scripts/safe_install.sh --create-venv
+source venv/bin/activate
+./scripts/safe_install.sh --all
+```
+
+That's it! The script handles everything automatically.
+
+---
+
+## Manual Installation (Alternative)
+
+### Prerequisites
+
+- **Python 3.9+** (check with `python3 --version`)
+- **PostgreSQL 12+** (for database)
+- **Git** (for version control)
+
+### Step-by-Step
+
+#### 1. **Create Virtual Environment**
+
+```bash
+cd /home/dell/coding/giglez
+
+# Create venv
+python3 -m venv venv
+
+# Activate it
+source venv/bin/activate
+
+# Verify activation (should show venv path)
+which python
+```
+
+#### 2. **Upgrade pip**
+
+```bash
+# IMPORTANT: Always upgrade pip first
+python -m pip install --upgrade pip setuptools wheel
+```
+
+#### 3. **Install Dependencies**
+
+```bash
+# Install all project dependencies
+pip install -r requirements.txt
+
+# Or install individually
+pip install fastapi uvicorn sqlalchemy psycopg2-binary loguru
+```
+
+#### 4. **Install Optional Dependencies**
+
+```bash
+# For development
+pip install pytest black flake8
+
+# For ML features (when ready)
+pip install tensorflow lightgbm scikit-learn
+```
+
+---
+
+## Troubleshooting Common pip Issues
+
+### Issue 1: "pip: command not found"
+
+**Solution:**
+```bash
+# Install pip
+python3 -m ensurepip --upgrade
+
+# Or use system package manager
+sudo apt-get install python3-pip  # Ubuntu/Debian
+sudo yum install python3-pip      # RHEL/CentOS
+```
+
+### Issue 2: "Permission denied" errors
+
+**DON'T use sudo pip!** Use virtual environment instead:
+
+```bash
+# Create venv if you haven't
+python3 -m venv venv
+source venv/bin/activate
+
+# Now install (no sudo needed)
+pip install package_name
+```
+
+### Issue 3: "No module named 'pip'"
+
+**Solution:**
+```bash
+# Reinstall pip
+python3 -m ensurepip --default-pip
+python3 -m pip install --upgrade pip
+```
+
+### Issue 4: psycopg2 compilation errors
+
+**Solution:** Use binary version:
+```bash
+pip install psycopg2-binary
+```
+
+Or install system dependencies first:
+```bash
+sudo apt-get install libpq-dev python3-dev  # Ubuntu/Debian
+sudo yum install postgresql-devel python3-devel  # RHEL/CentOS
+```
+
+### Issue 5: "ERROR: Could not find a version that satisfies the requirement"
+
+**Solution:**
+```bash
+# Update pip
+pip install --upgrade pip
+
+# Try with --upgrade flag
+pip install --upgrade package_name
+
+# Check Python version compatibility
+python --version  # Must be 3.9+
+```
+
+### Issue 6: Dependency conflicts
+
+**Solution:**
+```bash
+# Use our safe install script
+./scripts/safe_install.sh package_name
+
+# Or create fresh environment
+deactivate
+rm -rf venv
+python3 -m venv venv
+source venv/bin/activate
+pip install -r requirements.txt
+```
+
+---
+
+## Safe Install Script Usage
+
+### Basic Commands
+
+```bash
+# Check environment
+./scripts/safe_install.sh --check
+
+# Install single package
+./scripts/safe_install.sh requests
+
+# Install specific version
+./scripts/safe_install.sh requests 2.28.0
+
+# Install from requirements.txt
+./scripts/safe_install.sh --all
+
+# Install from custom requirements file
+./scripts/safe_install.sh -r requirements-dev.txt
+```
+
+### Advanced Features
+
+```bash
+# Create backup before installing
+./scripts/safe_install.sh --backup
+
+# Security scan
+./scripts/safe_install.sh --scan
+
+# Resolve dependencies
+./scripts/safe_install.sh --resolve
+
+# Rollback to previous state
+./scripts/safe_install.sh --rollback .pip_backups/installed_packages_20260115_123456.txt
+```
+
+### What the Script Does
+
+✅ **Automatic Virtual Environment**
+- Detects if venv exists
+- Creates new one if needed
+- Activates automatically
+
+✅ **Dependency Checking**
+- Checks for conflicts before installing
+- Dry-run simulation
+- Asks for confirmation
+
+✅ **Backup & Rollback**
+- Saves package list before changes
+- Can restore previous state
+- Timestamped backups
+
+✅ **Verification**
+- Verifies installation after completion
+- Checks package integrity
+- Reports errors clearly
+
+---
+
+## Database Setup
+
+### 1. **Install PostgreSQL**
+
+```bash
+# Ubuntu/Debian
+sudo apt-get install postgresql postgresql-contrib postgis
+
+# macOS
+brew install postgresql postgis
+
+# Verify installation
+psql --version
+```
+
+### 2. **Create Database**
+
+```bash
+# Switch to postgres user
+sudo -u postgres psql
+
+# In psql prompt:
+CREATE DATABASE giglez;
+CREATE USER giglez_user WITH PASSWORD 'your_secure_password';
+GRANT ALL PRIVILEGES ON DATABASE giglez TO giglez_user;
+
+# Enable PostGIS extension
+\c giglez
+CREATE EXTENSION postgis;
+\q
+```
+
+### 3. **Configure Environment**
+
+Create `.env.development`:
+
+```bash
+cp .env.example .env.development
+
+# Edit with your database credentials
+nano .env.development
+```
+
+```env
+# Database
+DATABASE_URL=postgresql://giglez_user:your_secure_password@localhost:5432/giglez
+
+# Other settings...
+```
+
+### 4. **Run Migrations**
+
+```bash
+# Create tables
+python scripts/create_schema.sql
+
+# Or use Alembic (if configured)
+alembic upgrade head
+```
+
+---
+
+## Optional Components
+
+### RTL_433 (for RF signal decoding)
+
+```bash
+# Ubuntu/Debian
+sudo apt-get install rtl_433
+
+# macOS
+brew install rtl_433
+
+# Build from source
+git clone https://github.com/merbanan/rtl_433.git
+cd rtl_433
+mkdir build && cd build
+cmake ..
+make
+sudo make install
+```
+
+### Security Tools
+
+```bash
+# Install safety for security scanning
+pip install safety
+
+# Run security check
+safety check
+```
+
+### Development Tools
+
+```bash
+# Code formatting
+pip install black isort
+
+# Linting
+pip install flake8 pylint
+
+# Type checking
+pip install mypy
+```
+
+---
+
+## Verifying Installation
+
+### 1. **Check Python Packages**
+
+```bash
+# List installed packages
+pip list
+
+# Check specific package
+pip show fastapi
+
+# Verify all requirements
+pip check
+```
+
+### 2. **Test Database Connection**
+
+```bash
+# Quick test
+python3 << EOF
+from sqlalchemy import create_engine
+engine = create_engine('postgresql://giglez_user:password@localhost:5432/giglez')
+connection = engine.connect()
+print("✓ Database connection successful!")
+connection.close()
+EOF
+```
+
+### 3. **Test API Server**
+
+```bash
+# Start development server
+uvicorn src.api.main:app --reload
+
+# In another terminal, test endpoint
+curl http://localhost:8000/health
+
+# Should return: {"status":"ok"}
+```
+
+### 4. **Run Tests**
+
+```bash
+# Run all tests
+pytest
+
+# With coverage
+pytest --cov=src tests/
+
+# Specific test file
+pytest tests/test_parser.py
+```
+
+---
+
+## Environment Management
+
+### Activating Virtual Environment
+
+```bash
+# Every time you work on the project:
+cd /home/dell/coding/giglez
+source venv/bin/activate
+
+# Verify activation
+which python  # Should show venv path
+```
+
+### Deactivating
+
+```bash
+deactivate
+```
+
+### Updating Dependencies
+
+```bash
+# Update single package
+pip install --upgrade package_name
+
+# Update all packages (careful!)
+pip list --outdated
+pip install --upgrade -r requirements.txt
+
+# Safer: use safe_install script
+./scripts/safe_install.sh --all
+```
+
+### Freezing Dependencies
+
+```bash
+# Save current package versions
+pip freeze > requirements-lock.txt
+
+# Install exact versions later
+pip install -r requirements-lock.txt
+```
+
+---
+
+## Docker Alternative (Optional)
+
+If you prefer Docker:
+
+```bash
+# Build image
+docker build -t giglez:latest .
+
+# Run container
+docker run -p 8000:8000 --env-file .env.development giglez:latest
+
+# With database
+docker-compose up
+```
+
+---
+
+## Next Steps
+
+After installation:
+
+1. ✅ **Configure environment** - Edit `.env.development`
+2. ✅ **Setup database** - Create tables and extensions
+3. ✅ **Import test data** - Run dataset download script
+4. ✅ **Start dev server** - `uvicorn src.api.main:app --reload`
+5. ✅ **Open browser** - Visit `http://localhost:8000/docs`
+
+---
+
+## Getting Help
+
+### Check Logs
+
+```bash
+# Application logs
+tail -f logs/app.log
+
+# Error logs
+tail -f logs/error.log
+
+# Upload logs (during beta)
+tail -f logs/uploads/uploads_$(date +%Y-%m-%d).log
+```
+
+### Common Commands
+
+```bash
+# Check what's using port 8000
+lsof -i :8000
+
+# Kill process on port 8000
+kill $(lsof -t -i:8000)
+
+# Check Python path
+python -c "import sys; print(sys.path)"
+
+# Check installed package location
+python -c "import fastapi; print(fastapi.__file__)"
+```
+
+### Reinstall from Scratch
+
+```bash
+# Deactivate if active
+deactivate
+
+# Remove virtual environment
+rm -rf venv
+
+# Remove pip cache
+rm -rf ~/.cache/pip
+
+# Start fresh
+python3 -m venv venv
+source venv/bin/activate
+pip install --upgrade pip
+pip install -r requirements.txt
+```
+
+---
+
+## Support
+
+- **Documentation:** `/docs` directory
+- **Issues:** GitHub Issues
+- **Email:** support@giglez.com
+
+---
+
+**Last Updated:** 2026-01-16
+**Tested On:** Ubuntu 22.04, macOS 13, Python 3.9-3.11