Compare commits

..

8 Commits

Author SHA1 Message Date
n0mad1k 376a6d3a25 Genericize: drop pCloud terminology, use TARGET_DIR / target_dir
pCloud was never a hard dependency - the tool writes to any local directory.
Renaming makes the tool clearly generic before publishing.

- PCLOUD_TARGET → TARGET_DIR
- probe_pcloud → probe_target
- place_into_pcloud → place_into_target
- pcloud_target config key → target_dir
- COE_MIRROR_PCLOUD_TARGET env var → COE_MIRROR_TARGET_DIR
- Hardcoded default: ~/pCloudDrive/Hacking/ChurchOfMalware → ~/coe-mirror
- Log messages and docstrings updated
- Tests updated to match

DevTrack #1286
2026-07-07 11:12:46 -04:00
n0mad1k 2093b737c3 Make distributable: config.toml support, blanked Matrix defaults, env creds fallback
- ~/.config/coe-mirror/config.toml (tomllib, Python 3.11+ required)
- Precedence uniform for every key: env var > config file > hardcoded default
- API_BASE, OWNERS, PCLOUD_TARGET, PENDING_DIR, matrix homeserver/room resolved
  via _resolve() helper
- Matrix homeserver/room now default to empty; _send_matrix short-circuits when
  either is blank so forks don't spam the original author's room
- _fetch_creds falls back to COE_MIRROR_MATRIX_USER/PASSWORD env vars when the
  Infisical creds binary is unavailable (required for external distribution)
- CLI subcommands: config init (writes example TOML, won't overwrite), config path
  Bare invocation and 'run' still work unchanged for the systemd timer
- README rewritten as a public quick-start with the precedence table and Matrix
  as opt-in
- Tests: 3 new for config precedence, 1 for Matrix silent no-op when blank,
  2 for creds env fallback (26 total, all pass)

DevTrack #1285
2026-07-07 11:04:07 -04:00
n0mad1k 5912ca92d2 Mirror all four Church of Malware owners, per-owner pcloud subdirs
- OWNERS list replaces single OWNER: Nightmare_Eclipse, shai_hulud, bikini_mirror, ek0mssavi0r
- PCLOUD_TARGET moves to ChurchOfMalware/ parent, per-owner subdirs at runtime
- Owner threaded through fetch_repos, fetch_branch_sha, download_repo_zip
- Pending filenames get owner__ prefix to prevent cross-owner collisions
- flush_pending strips owner prefix when routing to owner subdir
- Per-owner fetch failure logs and continues; failed_owners reported in notify
- notify groups new files by owner; also picks up flushes done at run start
- Tests: parametrize on OWNERS via monkeypatch, add multi-owner collision and
  per-owner failure isolation tests

DevTrack #1284
2026-07-06 12:35:56 -04:00
Cobra 8ea01c898f swap to dedicated @coe-mirror-bot for Matrix sends
Replaces shared @intel-bot credentials with a Matrix user created
specifically for this tool. Reads Infisical keys via constants
COE_MIRROR_BOT_USER_KEY and COE_MIRROR_BOT_PASSWORD_KEY so test
assertions track the code rather than hardcoded strings.
2026-06-15 09:31:24 -04:00
n0mad1k d9e01961ff Fix type hints and resource management in coe_mirror
- Add precise type hints: Optional[Tuple[str, str]] for _fetch_creds(),
  List[Tuple[str, str, str]] for notify()
- Wrap requests.Session in context manager for proper cleanup
- Remove unreachable return False after retry loop in _send_matrix()
2026-06-14 14:40:14 -04:00
n0mad1k c322488869 Implement real Matrix notification for coe-mirror
Replace stub notify() with full Synapse integration targeting General room.
- Add _send_matrix(), _get_matrix_token(), _matrix_login() helpers
- Implement token cache with 30-min refresh threshold and 401 retry
- Fetch credentials via Infisical (MATRIX_USER, MATRIX_PASSWORD from matrix/)
- Add MATRIX_* env overrides for testing (homeserver, room, token cache path)
- Non-fatal failures: Matrix outage logs warning, does not break mirror run
- Add 5 comprehensive test cases covering success, failure, 401 refresh, env override, chmod 600
- All 18 tests passing
- Update README and notes.md with implementation details
2026-06-14 14:36:14 -04:00
Cobra 98cab71a21 install.sh: skip pip when requests is already system-wide
PEP 668 Debian 13 systems error out on 'pip install --user'. Runtime
only needs requests; pytest+requests-mock are test-only. Detect the
runtime dep first and fall back to pip only when missing.
2026-06-11 09:37:02 -04:00
Cobra 1bcb3b185c coe-mirror: mirror Nightmare_Eclipse Forgejo repos to pCloud
Standalone Python tool. Hits git.churchofmalware.org public API every
12h (00:00/12:00), downloads any repo HEAD whose SHA hasn't already
been snapshotted to pCloud, with local-staging fallback when the FUSE
mount is unavailable. Matrix notification stub for when the home
server is back.

Devil's-advocate findings folded into the design:
  - shutil.copy2 + size verify + unlink, not os.rename, for FUSE safety
  - no pCloud daemon auto-start (no user unit; DISPLAY=:0 unreliable);
    relies on Persistent timer + pending flush for recovery
  - two-call API path retained because the repo list response does not
    include commit.sha

Audit-driven security fixes:
  - repo names whitelist-validated (^[a-zA-Z0-9._-]+$)
  - urllib.parse.quote on all interpolated URL segments
  - same-file no-op guard in place_into_pcloud
  - existence check moved to main() so cached files don't generate
    spurious notify entries

13 pytest tests passing.
2026-06-11 09:34:55 -04:00
4 changed files with 23 additions and 66 deletions
-1
View File
@@ -7,4 +7,3 @@ __pycache__/
.active-item.json .active-item.json
.workflow-state.json .workflow-state.json
agent-spawns/ agent-spawns/
notes.md
+1 -19
View File
@@ -81,8 +81,6 @@ PENDING_DIR: Path = Path(
str(Path.home() / ".local" / "share" / "coe-mirror" / "pending"), str(Path.home() / ".local" / "share" / "coe-mirror" / "pending"),
) )
).expanduser() ).expanduser()
_MOUNT_ROOT_VAL = _resolve("COE_MIRROR_MOUNT_ROOT", _CFG.get("mount_root"), "")
MOUNT_ROOT: Optional[Path] = Path(_MOUNT_ROOT_VAL).expanduser() if _MOUNT_ROOT_VAL else None
LOG_DIR = Path.home() / ".local" / "share" / "coe-mirror" LOG_DIR = Path.home() / ".local" / "share" / "coe-mirror"
LOG_FILE = LOG_DIR / "coe-mirror.log" LOG_FILE = LOG_DIR / "coe-mirror.log"
LOCK_NAME = "coe-mirror.lock" LOCK_NAME = "coe-mirror.lock"
@@ -168,19 +166,8 @@ def release_lock(lock_fd: int) -> None:
def probe_target() -> bool: def probe_target() -> bool:
"""Probe target directory health. Write tiny file, read back, delete. Return True if healthy. """Probe target directory health. Write tiny file, read back, delete. Return True if healthy."""
When MOUNT_ROOT is configured, first verify it is a live mountpoint and TARGET_DIR is under it; return False (queue to pending) otherwise.
"""
try: try:
if MOUNT_ROOT is not None:
if not MOUNT_ROOT.exists() or not os.path.ismount(str(MOUNT_ROOT)):
logger.warning("mount_root %s is not a live mountpoint; target unavailable; queuing locally", MOUNT_ROOT)
return False
if not TARGET_DIR.is_relative_to(MOUNT_ROOT):
logger.warning("target_dir %s is not under configured mount_root %s; misconfig; queuing locally", TARGET_DIR, MOUNT_ROOT)
return False
probe_file = TARGET_DIR / ".write_probe" probe_file = TARGET_DIR / ".write_probe"
TARGET_DIR.mkdir(parents=True, exist_ok=True) TARGET_DIR.mkdir(parents=True, exist_ok=True)
@@ -686,11 +673,6 @@ api_base = "https://git.churchofmalware.org"
# Where mirrored zips are stored. One subdirectory per owner is created here. # Where mirrored zips are stored. One subdirectory per owner is created here.
target_dir = "~/coe-mirror" target_dir = "~/coe-mirror"
# Optional: require this path to be a live mountpoint before writing. If set and
# not currently mounted, the run queues to pending instead of creating a phantom
# local directory at the mountpoint. Leave blank to disable.
# mount_root = "~/pCloudDrive"
# Optional: local staging when target_dir is unavailable. # Optional: local staging when target_dir is unavailable.
# pending_dir = "~/.local/share/coe-mirror/pending" # pending_dir = "~/.local/share/coe-mirror/pending"
+22
View File
@@ -0,0 +1,22 @@
# coe-mirror notes
- **Created**: 2026-06-11. Purpose: autonomous mirroring of Nightmare_Eclipse repos from Church of Malware to local pCloud storage.
- **Source API**: Forgejo public REST at `git.churchofmalware.org`, no auth needed.
- **Owner being mirrored**: `Nightmare_Eclipse` (8 active repos as of creation).
- **Destination**: `~/pCloudDrive/Hacking/Nightmare_Eclipse/`.
- **Schedule**: 00:00 + 12:00 daily, catch-up on boot via Persistent timer + OnBootSec.
- **Design rationale**:
- FUSE-safe placement uses `shutil.copy2 + size verify + unlink`, not `os.rename`, to handle pCloud FUSE semantics.
- pCloud daemon auto-start was rejected — no user unit exists; DISPLAY=:0 spawning is unreliable. Tool relies on Persistent timer + pending-flush for recovery.
- `commit.sha` is NOT in the repo-list API response — two-call approach (list, then per-repo branch) is retained.
- **DevTrack**: Not created at creation time (oraculo offline, Tailscale last seen 2d ago, carbon roaming off home LAN). Add a `coe-mirror` project + item retroactively when oraculo returns.
- **Matrix notification**: Stub-only at creation time, Matrix server down. Real implementation deferred until home Matrix server is back.
## 2026-06-14 — Matrix notification swap
- Replaced `notify()` stub with real Synapse send. Targets General room (`!MBpPtYlQYMcngtpvJt:m.mealeyfamily.com`).
- Auth: password login via `MATRIX_USER` + `MATRIX_PASSWORD` from Infisical `matrix/` at refresh time. Token cache at `~/.local/share/coe-mirror/matrix-token.json` (chmod 0600), refreshed when <30 min left or on 401.
- Non-fatal failures — Matrix outage does not break the mirror.
- Env overrides: `COE_MIRROR_MATRIX_ROOM`, `COE_MIRROR_MATRIX_HOMESERVER`.
- Pre-deploy step: invite `@intel-bot:m.mealeyfamily.com` to General once Synapse is back. Without joined membership, sends will 403.
- DevTrack: #1172.
-46
View File
@@ -37,7 +37,6 @@ def mock_paths(tmp_path, monkeypatch):
pending_dir.mkdir() pending_dir.mkdir()
log_dir.mkdir() log_dir.mkdir()
monkeypatch.setattr(coe_mirror, "MOUNT_ROOT", None)
monkeypatch.setattr(coe_mirror, "TARGET_DIR", target_dir) monkeypatch.setattr(coe_mirror, "TARGET_DIR", target_dir)
monkeypatch.setattr(coe_mirror, "PENDING_DIR", pending_dir) monkeypatch.setattr(coe_mirror, "PENDING_DIR", pending_dir)
monkeypatch.setattr(coe_mirror, "LOG_DIR", log_dir) monkeypatch.setattr(coe_mirror, "LOG_DIR", log_dir)
@@ -596,51 +595,6 @@ class TestPlaceIntoPloudSameFileIsNoop:
assert test_file.stat().st_size > 0 assert test_file.stat().st_size > 0
class TestMountRootGuard:
"""Test probe_target() mount-root verification (phantom-dir regression guard)."""
def test_no_mount_root_configured_unchanged_behavior(self, mock_paths, setup_logging, monkeypatch):
"""MOUNT_ROOT None + writable TARGET_DIR -> True (unchanged behavior)."""
monkeypatch.setattr(coe_mirror, "MOUNT_ROOT", None)
assert coe_mirror.probe_target() is True
def test_mount_root_live_target_under_it_returns_true(self, mock_paths, setup_logging, monkeypatch, tmp_path):
"""MOUNT_ROOT set, live mount, TARGET_DIR a subdir of it -> True."""
mount_root = tmp_path / "pCloudDrive"
mount_root.mkdir()
target_dir = mount_root / "Hacking" / "ChurchOfMalware"
monkeypatch.setattr(coe_mirror, "MOUNT_ROOT", mount_root)
monkeypatch.setattr(coe_mirror, "TARGET_DIR", target_dir)
monkeypatch.setattr(coe_mirror.os.path, "ismount", lambda p: True)
assert coe_mirror.probe_target() is True
def test_mount_not_live_does_not_create_phantom_dir(self, mock_paths, setup_logging, monkeypatch, tmp_path):
"""MOUNT_ROOT set but not mounted -> False, and TARGET_DIR is never created."""
mount_root = tmp_path / "pCloudDrive"
mount_root.mkdir()
target_dir = mount_root / "Hacking" / "ChurchOfMalware"
monkeypatch.setattr(coe_mirror, "MOUNT_ROOT", mount_root)
monkeypatch.setattr(coe_mirror, "TARGET_DIR", target_dir)
monkeypatch.setattr(coe_mirror.os.path, "ismount", lambda p: False)
assert coe_mirror.probe_target() is False
assert not target_dir.exists()
def test_target_dir_not_under_mount_root_returns_false(self, mock_paths, setup_logging, monkeypatch, tmp_path):
"""MOUNT_ROOT set, live mount, but TARGET_DIR is unrelated -> False, no dir created."""
mount_root = tmp_path / "pCloudDrive"
mount_root.mkdir()
target_dir = tmp_path / "unrelated" / "target"
monkeypatch.setattr(coe_mirror, "MOUNT_ROOT", mount_root)
monkeypatch.setattr(coe_mirror, "TARGET_DIR", target_dir)
monkeypatch.setattr(coe_mirror.os.path, "ismount", lambda p: True)
assert coe_mirror.probe_target() is False
assert not target_dir.exists()
# Tests for Matrix notification implementation. # Tests for Matrix notification implementation.
# Note: Tests requiring monkeypatched subprocess.run must use raising=False because # Note: Tests requiring monkeypatched subprocess.run must use raising=False because
# subprocess.run is not a class attribute on the coe_mirror module by default # subprocess.run is not a class attribute on the coe_mirror module by default