giglez/POSTGRESQL_SETUP_EXPLANATION.md

PostgreSQL Setup - Why I Cannot Complete It

Current Situation

PostgreSQL Status: ✅ Installed and running

$ systemctl status postgresql
● postgresql.service - PostgreSQL RDBMS
     Active: active (exited) since Mon 2026-01-12 06:49:52 PST; 10h ago

Problem: 🔒 I don't have sudo privileges

---

What Needs to Happen

To set up PostgreSQL for GigLez, we need to:

1. Create Database User

CREATE USER giglez_user WITH PASSWORD 'giglez_dev_password';

2. Create Database

CREATE DATABASE giglez OWNER giglez_user;

3. Enable PostGIS Extension

\c giglez
CREATE EXTENSION postgis;

4. Grant Permissions

GRANT ALL PRIVILEGES ON DATABASE giglez TO giglez_user;

---

Why I Cannot Do This

Problem: Requires sudo Access

All PostgreSQL administrative tasks require:

sudo -u postgres psql

When I try:

$ sudo -u postgres psql
sudo: a password is required

Result: ❌ Cannot execute without your password

---

The Setup Script (Already Created)

File: scripts/quick_db_setup.sh

What it does:

#!/bin/bash
# 1. Check PostgreSQL is running
# 2. Use sudo to connect as postgres user
# 3. Create giglez_user with password
# 4. Create giglez database
# 5. Enable PostGIS extension
# 6. Grant privileges
# 7. Create schema (tables, indexes)

Why I can't run it: Line 28 requires sudo:

sudo -u postgres psql << 'EOF'
  CREATE USER giglez_user ...
EOF

---

What YOU Need to Do

Option 1: Run the Setup Script (Recommended)

One command (requires your password):

./scripts/quick_db_setup.sh

What will happen:

1. Prompt for sudo password
2. Create database user giglez_user
3. Create database giglez
4. Enable PostGIS extension
5. Create all tables (devices, signatures, captures)
6. Create indexes for performance

Time: ~30 seconds

---

Option 2: Manual Setup (If script fails)

Step-by-step commands (you'll be prompted for password):

1. Connect to PostgreSQL

sudo -u postgres psql

2. Create User and Database

-- Create user
CREATE USER giglez_user WITH PASSWORD 'giglez_dev_password';

-- Create database
CREATE DATABASE giglez OWNER giglez_user;

-- Grant privileges
GRANT ALL PRIVILEGES ON DATABASE giglez TO giglez_user;

-- Exit
\q

3. Enable PostGIS

sudo -u postgres psql -d giglez -c "CREATE EXTENSION IF NOT EXISTS postgis;"

4. Create Schema

psql -U giglez_user -d giglez -h localhost << 'EOF'
-- You'll be prompted for password: giglez_dev_password

-- Devices table
CREATE TABLE IF NOT EXISTS devices (
    id SERIAL PRIMARY KEY,
    device_name VARCHAR(200),
    manufacturer VARCHAR(100),
    model VARCHAR(100),
    device_type VARCHAR(50),
    typical_frequency INTEGER,
    protocol VARCHAR(100),
    description TEXT,
    first_seen TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    is_verified BOOLEAN DEFAULT FALSE
);

-- Signatures table
CREATE TABLE IF NOT EXISTS signatures (
    id SERIAL PRIMARY KEY,
    device_id INTEGER REFERENCES devices(id),
    protocol VARCHAR(100),
    frequency INTEGER,
    modulation VARCHAR(50),
    bit_pattern BYTEA,
    bit_mask BYTEA,
    timing_min INTEGER,
    timing_max INTEGER,
    raw_pattern TEXT,
    confidence_threshold FLOAT DEFAULT 0.7,
    source VARCHAR(50),
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

-- Captures table
CREATE TABLE IF NOT EXISTS captures (
    file_hash VARCHAR(64) PRIMARY KEY,
    filename VARCHAR(500),
    frequency INTEGER,
    protocol VARCHAR(100),
    latitude DECIMAL(10, 8),
    longitude DECIMAL(11, 8),
    captured_at TIMESTAMP,
    device_id INTEGER REFERENCES devices(id),
    match_confidence FLOAT,
    uploaded_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

-- Indexes
CREATE INDEX IF NOT EXISTS idx_signatures_frequency ON signatures(frequency);
CREATE INDEX IF NOT EXISTS idx_signatures_device ON signatures(device_id);
CREATE INDEX IF NOT EXISTS idx_captures_frequency ON captures(frequency);

\q
EOF

---

After PostgreSQL Setup

1. Import Signature Data

Import Flipper Zero signatures (85 devices):

# Adapt SQLite script for PostgreSQL
python3 scripts/import_flipper_to_postgres.py

Or use existing SQLite data:

# Convert SQLite to PostgreSQL
sqlite3 giglez.db .dump | psql -U giglez_user -d giglez -h localhost

2. Switch to Full API

Stop simplified server:

# Find process
ps aux | grep main_simple
kill <PID>

Start full API:

python3 src/api/main.py

Verify connection:

curl http://localhost:8000/health
# Should show: "database": "connected"

---

Why PostgreSQL vs SQLite?

Current Situation: SQLite ✅

File: giglez.db (72 KB, 85 devices)

Advantages:

• ✅ No setup required
• ✅ Single file
• ✅ Fast for small datasets
• ✅ Already populated with Flipper signatures

Limitations:

• ❌ No geographic queries (PostGIS)
• ❌ Limited concurrency
• ❌ No spatial indexing
• ❌ Doesn't scale to millions of records

Production Goal: PostgreSQL + PostGIS

Advantages:

• ✅ PostGIS for geographic queries (radius search, bounding box)
• ✅ Spatial indexing (GiST) for performance
• ✅ Scalability (millions of captures)
• ✅ Concurrent writes (multiple users uploading)
• ✅ Advanced queries (complex geo searches)

Example PostGIS query:

-- Find all captures within 10km of coordinates
SELECT * FROM captures
WHERE ST_DWithin(
    ST_MakePoint(longitude, latitude)::geography,
    ST_MakePoint(-74.0060, 40.7128)::geography,
    10000  -- 10km in meters
);

This is impossible in SQLite!

---

Current Workaround

Why We Built main_simple.py

Purpose: Test web interface without database dependency

What works:

• ✅ Web interface (HTML/CSS/JS)
• ✅ Map visualization
• ✅ Navigation
• ✅ API documentation

What doesn't work:

• ❌ File uploads (no backend processing)
• ❌ Device matching (no database)
• ❌ Search (no data)
• ❌ Real statistics (shows zeros)

This is TEMPORARY - meant only for UI/UX testing.

---

Comparison Table

Feature	SQLite (Current)	PostgreSQL (Needed)	Simple Mode (Testing)
Setup	✅ Done	⏳ Needs sudo	✅ None
Signatures	✅ 85 loaded	⏳ Need import	❌ None
Geographic queries	❌ No PostGIS	✅ PostGIS	❌ None
Uploads	⏳ Possible	✅ Full featured	❌ Disabled
Scalability	⚠️ ~10K records	✅ Millions	N/A
Concurrent users	⚠️ Limited	✅ Unlimited	N/A
Web interface	✅ Works	✅ Works	✅ Works

---

Step-by-Step: What You Need to Do

Phase 1: PostgreSQL Setup (5 minutes)

# 1. Run setup script (enter password when prompted)
cd /home/dell/coding/giglez
./scripts/quick_db_setup.sh

# Expected output:
# ✅ PostgreSQL is running
# ✅ User and database created
# ✅ PostGIS enabled
# ✅ Schema created
# ✅ Database setup complete!

Phase 2: Import Signatures (2 minutes)

Option A: From SQLite (quick):

# Export from SQLite
sqlite3 giglez.db ".dump devices signatures" > data.sql

# Import to PostgreSQL
psql -U giglez_user -d giglez -h localhost -f data.sql
# Password: giglez_dev_password

Option B: Re-import from Flipper (fresh):

# Create PostgreSQL version of import script
python3 scripts/import_flipper_to_postgres.py

Phase 3: Start Full API (1 minute)

# Stop simple server
pkill -f main_simple

# Start full API
python3 src/api/main.py

# Test
curl http://localhost:8000/health
# Should show: "database": "connected"

Phase 4: Test Everything (5 minutes)

# Open browser
http://localhost:8000

# 1. Upload a .sub file
# 2. See it on the map
# 3. Search for it
# 4. View statistics

---

Why This Matters

Current State: "Hello World"

• Web interface loads
• UI/UX testable
• But no real functionality

After PostgreSQL: "Production MVP"

• Upload .sub files
• Automatic device identification
• Geographic search
• Interactive map with real data
• Statistics dashboard with real numbers
• Actual Wigle-style platform!

---

Security Notes

Default Password (Development)

Current: giglez_dev_password

WARNING: This is in .env.development - fine for local testing, NOT for production

For production, change to strong password:

# Generate random password
openssl rand -base64 32

# Update .env.production
GIGLEZ_DB_PASSWORD=<strong-random-password>

Connection String

Development:

postgresql://giglez_user:giglez_dev_password@localhost:5432/giglez

Production:

• Use environment variables
• Encrypt connection
• Restrict network access
• Use SSL certificates

---

Troubleshooting

"PostgreSQL is not running"

sudo systemctl start postgresql
sudo systemctl enable postgresql  # Start on boot

"Role 'giglez_user' already exists"

# Drop and recreate
sudo -u postgres psql -c "DROP USER IF EXISTS giglez_user;"
./scripts/quick_db_setup.sh

"Database 'giglez' already exists"

# Drop and recreate
sudo -u postgres psql -c "DROP DATABASE IF EXISTS giglez;"
./scripts/quick_db_setup.sh

"Permission denied"

# Grant all privileges
sudo -u postgres psql -d giglez -c "GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO giglez_user;"

---

Summary

What I Did ✅

1. ✅ Created full API (src/api/main.py)
2. ✅ Created simplified test version (src/api/main_simple.py)
3. ✅ Created setup script (scripts/quick_db_setup.sh)
4. ✅ Documented everything

What I Cannot Do 🔒

1. ❌ Run sudo commands (need your password)
2. ❌ Create PostgreSQL user
3. ❌ Create PostgreSQL database
4. ❌ Enable PostGIS extension

What YOU Need to Do 👤

Single command:

./scripts/quick_db_setup.sh

Then:

python3 src/api/main.py

That's it! 🎉

---

Current Status

Web Interface: ✅ Working (simplified mode)

http://localhost:8000

Database: ⏳ Waiting for your setup

./scripts/quick_db_setup.sh

Next Step: Run the setup script when ready!

---

Created: 2026-01-12 Status: PostgreSQL setup documented and ready Action Required: User needs to run ./scripts/quick_db_setup.sh