Trilltechnician/giglez
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
bba30ad2da674a149b8db041010c2fcbf5de6b83
giglez/PROJECT_STATUS.md
T

11 KiB
Raw Blame History

GigLez Project Status

Date: 2025-01-11 Phase: Foundation & Planning Status: Architecture Complete, Ready for Implementation

Completed Tasks

1. Research & Documentation

  • ✅ Researched Flipper Zero Sub-GHz database structure
  • ✅ Documented RTL_433 protocol database (200+ protocols)
  • ✅ Analyzed Universal Radio Hacker signal formats
  • ✅ Identified key signature sources and file formats

2. Project Architecture

  • ✅ Created comprehensive CLAUDE.md with primary directives
  • ✅ Designed modular directory structure
  • ✅ Established clear separation of concerns (capture/gps/database/matcher/api/web)

3. Database Design

  • ✅ Complete PostgreSQL schema with 9+ tables
  • ✅ Geospatial indexing for location queries
  • ✅ Signature matching tables (Flipper, RTL_433, community)
  • ✅ User authentication and community voting system
  • ✅ Materialized views for performance optimization
  • ✅ Trigger functions for automatic statistics updates

4. Signature Database Integration

  • ✅ Documented Flipper Zero .sub file format parsing
  • ✅ RTL_433 JSON output field mapping
  • ✅ Import pipeline design for all signature sources
  • ✅ Signature matching algorithm specification
  • ✅ Community contribution workflow

5. Hardware Integration

  • ✅ T-Embed communication protocol design (JSON over serial)
  • ✅ Command reference (SCAN, CAPTURE, STATUS, etc.)
  • ✅ Event notification system
  • ✅ Python controller implementation (sync & async)
  • ✅ GPS coordination strategy

6. Development Infrastructure

  • ✅ requirements.txt with all dependencies
  • ✅ .gitignore configured for Python/data files
  • ✅ Initial Python module structure
  • ✅ T-Embed controller classes (sync/async)

Current Project Structure

giglez/
├── CLAUDE.md                    # Primary project directives
├── README.md                    # User-facing documentation
├── PROJECT_STATUS.md           # This file
├── requirements.txt            # Python dependencies
├── .gitignore                  # Git exclusions
│
├── docs/                       # Technical documentation
│   ├── database_schema.md      # Complete DB schema
│   ├── signature_databases.md  # Signature import guide
│   └── tembed_setup.md        # Hardware setup & protocol
│
├── src/                        # Source code
│   ├── __init__.py
│   ├── capture/               # T-Embed communication
│   │   ├── __init__.py
│   │   ├── tembed.py         # Controller implementation
│   │   └── scanner.py        # (TODO) Signal scanner
│   ├── gps/                   # GPS integration
│   ├── database/              # Database models & ORM
│   ├── matcher/               # Signature matching engine
│   ├── api/                   # RESTful API
│   └── web/                   # Web interface
│
├── signatures/                # Signature databases
│   ├── flipper/              # Flipper Zero .sub files
│   ├── rtl433/               # RTL_433 protocols
│   ├── urh/                  # URH signal definitions
│   └── community/            # User submissions
│
├── scripts/                   # Utility scripts
├── config/                    # Configuration files
└── tests/                     # Test suite

Key Technical Decisions

1. Hardware Stack

  • Primary Device: LilyGo T-Embed with CC1101 (300-928 MHz)
  • Control Hub: Android Termux environment
  • GPS Source: Android device native GPS
  • Communication: USB Serial (115200 baud, JSON protocol)

2. Database

  • Engine: PostgreSQL with PostGIS for geospatial queries
  • ORM: SQLAlchemy for Python integration
  • Migration: Alembic for schema versioning

3. API Architecture

  • Framework: FastAPI (async, high performance)
  • Server: Uvicorn with uvloop
  • Authentication: JWT tokens, API keys

4. Signature Sources

  1. Flipper Zero: 1000+ device signatures (.sub format)
  2. RTL_433: 200+ protocol definitions (JSON)
  3. URH: Community signal patterns
  4. User Submissions: Photos + verified captures

5. Matching Strategy

  • Exact match: Protocol + Frequency + Timing (100% confidence)
  • Partial match: Protocol + Frequency (80% confidence)
  • Bit pattern matching with masks (90% confidence)
  • Weighted scoring based on source reliability

Next Steps - Implementation Roadmap

Phase 1: Core Infrastructure (Week 1-2)

Priority: HIGH

Database Setup

  • Install PostgreSQL in Termux
  • Create database and user
  • Run schema creation script (from database_schema.md)
  • Set up Alembic migrations
  • Test geospatial queries

T-Embed Integration

  • Flash Bruce firmware to T-Embed
  • Test serial communication from Termux
  • Verify command/response protocol
  • Implement error handling and reconnection
  • Create scanner module (src/capture/scanner.py)

GPS Module

  • Create GPS manager (src/gps/manager.py)
  • Test Android SL4A integration
  • Implement coordinate streaming
  • Add accuracy filtering
  • Create GPS logging

Phase 2: Signature Import (Week 2-3)

Priority: HIGH

Flipper Zero Database

  • Clone flipperzero-firmware repository
  • Extract .sub files from assets
  • Create parser (src/matcher/flipper_parser.py)
  • Import signatures to database
  • Verify protocol coverage

RTL_433 Protocols

  • Clone rtl_433 and rtl_433_tests
  • Extract protocol definitions
  • Create parser (src/matcher/rtl433_parser.py)
  • Map JSON fields to database schema
  • Import test data samples

Import Scripts

  • scripts/import_flipper.py
  • scripts/import_rtl433.py
  • scripts/update_signatures.sh (cron job)

Phase 3: Matching Engine (Week 3-4)

Priority: HIGH

Signature Matcher

  • Create matcher module (src/matcher/engine.py)
  • Implement exact matching
  • Implement partial matching
  • Add bit pattern matching with masks
  • Create confidence scoring algorithm
  • Optimize database queries

Testing

  • Unit tests for matching logic
  • Test against known signatures
  • Benchmark query performance
  • Validate confidence scores

Phase 4: Capture Workflow (Week 4-5)

Priority: HIGH

Capture Coordination

  • Create main capture loop (src/main.py)
  • Integrate T-Embed + GPS + Database
  • Implement event handling
  • Add automatic signature matching
  • Create session management
  • Implement capture deduplication (file hashing)

File Management

  • .sub file storage on SD card
  • Automatic file retrieval
  • Local caching strategy
  • Export to multiple formats

Phase 5: API Development (Week 5-6)

Priority: MEDIUM

RESTful API (src/api/)

  • FastAPI application setup
  • Authentication (JWT + API keys)
  • Endpoints:
    • POST /api/captures (submit capture)
    • GET /api/captures (query by location/time)
    • GET /api/devices (device database)
    • POST /api/identifications (user ID)
    • GET /api/sessions (capture sessions)
    • GET /api/heatmap (geographic density)
  • Rate limiting
  • CORS configuration
  • API documentation (OpenAPI/Swagger)

Phase 6: Web Interface (Week 6-8)

Priority: MEDIUM

Mapping UI (src/web/)

  • Choose mapping library (Leaflet.js/Mapbox)
  • Create heatmap visualization
  • Device marker clustering
  • Click for device details
  • Filter by protocol/frequency/device type
  • Timeline slider for captures

Capture Interface

  • Live capture status display
  • Session controls (start/stop)
  • Statistics dashboard
  • Device identification form
  • Photo upload for evidence

Community Features

  • User registration/login
  • Device identification voting
  • Reputation system
  • Leaderboard

Phase 7: Community System (Week 8-10)

Priority: LOW
  • User authentication system
  • Device submission workflow
  • Photo storage (local/S3)
  • Voting and verification
  • Moderation tools
  • Public API for data access

Phase 8: Optimization & Deployment (Week 10-12)

Priority: LOW
  • Database query optimization
  • Materialized view refresh strategy
  • Caching layer (Redis)
  • Background job queue (Celery)
  • Mobile responsive UI
  • Docker containerization
  • CI/CD pipeline
  • Production deployment guide

Immediate Next Steps (This Week)

1. Database Setup (Day 1)

# Install PostgreSQL in Termux
pkg install postgresql

# Start PostgreSQL
initdb ~/postgres
pg_ctl -D ~/postgres -l logfile start

# Create database
createdb giglez

# Run schema
psql giglez < scripts/schema.sql

2. Create Schema Script (Day 1)

Extract SQL from docs/database_schema.md into executable script.

3. T-Embed Testing (Day 2-3)

  • Flash Bruce firmware
  • Test serial communication
  • Verify JSON protocol
  • Capture test signals

4. GPS Integration (Day 3-4)

  • Install SL4A in Android
  • Test GPS acquisition
  • Stream coordinates to Termux
  • Log GPS data

5. First Capture (Day 5)

  • Integrate all components
  • Capture real signal with GPS
  • Store in database
  • Verify data integrity

Known Challenges

Technical

  1. Battery Life: Continuous GPS + RF scanning drains battery quickly
    • Solution: Implement duty cycling, power management
  2. Serial Reliability: USB serial can disconnect on Android
    • Solution: Auto-reconnect logic, connection monitoring
  3. Database Size: Raw captures can grow large quickly
    • Solution: Compression, selective storage, archival strategy

Hardware

  1. CC1101 Frequency Gaps: Can't cover entire spectrum
    • 300-348, 387-464, 779-928 MHz only
  2. Antenna Tuning: Different frequencies need different antennas
    • Solution: Multi-band antenna or frequency-specific sessions

Community

  1. Verification Quality: User-submitted IDs may be incorrect
    • Solution: Multi-voter verification, reputation system
  2. Privacy: GPS coordinates could reveal home locations
    • Solution: Anonymization options, GPS precision controls

Resources Needed

Development

  • Termux on Android with USB OTG support
  • LilyGo T-Embed CC1101 device
  • SD card (16GB+) for T-Embed
  • Multi-band sub-GHz antenna

Testing

  • Known devices for validation (garage remote, car key fob, etc.)
  • RTL-SDR for signal verification (optional)

Success Metrics

Phase 1 Complete When:

  • ✅ Database schema created and tested
  • ✅ T-Embed communicates reliably with Termux
  • ✅ GPS coordinates stream to application
  • ✅ First signal captured and stored with GPS

MVP Complete When:

  • ✅ 1000+ signatures imported (Flipper + RTL_433)
  • ✅ Automatic device matching works
  • ✅ Web interface shows captures on map
  • ✅ Can capture, identify, and visualize devices end-to-end

Production Ready When:

  • ✅ Community submission system live
  • ✅ API publicly accessible
  • ✅ 10,000+ captures in database
  • ✅ 50+ verified device types
  • ✅ Multi-user support with authentication

Contributing

See CONTRIBUTING.md for development workflow, code standards, and pull request process.

Questions & Discussion

For questions about architecture decisions or implementation approaches, see:

  • Technical discussions: GitHub Issues
  • Implementation help: GitHub Discussions
  • Real-time chat: [Discord] (coming soon)

Last Updated: 2025-01-11 Next Review: After Phase 1 completion

Reference in New Issue View Git Blame Copy Permalink