nightshift/README.md

# NightShift

<p align="center">
  <img src="docs/images/logo.png" width="220">
</p>

Auditable local-first AI coding pipelines.

NightShift is a deterministic pipeline runner for AI-assisted coding work. It reads markdown tasks, builds bounded context, asks configured agents for plans or patches, validates and applies those patches through explicit stages, runs checks, and leaves a human-reviewable artifact trail.

NightShift is not an autonomous software engineer. It is an orchestration layer that treats AI agents as unreliable workers inside bounded, testable, auditable workflows.

## Current Status

NightShift now supports the full local patch workflow:

- `nightshift init`, `validate`, `status`, `run`, `run --task`, `run --all`, and `web`.
- Markdown task parsing with dependencies.
- Command, Ollama, and OpenAI-compatible agent backends.
- Per-agent model settings such as `temperature`.
- Repo lookup tools: scoped `list_files`, `read_file`, and `grep`.
- Planner lookup requests with `files-inspected.md` artifacts.
- `repo_context` stage for `context-pack.md`.
- Project context chart generation at `.nightshift/project-context-chart.md`.
- `code_writer` stage that requires unified diff output.
- `patch_normalizer`, `patch_validator`, and `patch_apply` stages.
- Patch dry-run and apply modes.
- Test/static failure repair loops through existing retry routing.
- Run logs, dashboard log tails, git status artifacts, diffs, stage summaries, and final reports.

The default posture remains local-first and review-first: agents propose; NightShift validates, applies, tests, and records.

## What NightShift Is

NightShift is built for reviewable automation:

- local-first execution
- declarative pipeline stages
- markdown task files
- command-backed and model-backed agent wrappers
- explicit retry limits
- scoped repository lookup
- patch validation before mutation
- command allowlists
- durable markdown/text artifacts
- compact context handoff
- final reports for human review

The goal is to wake up to useful artifacts and a repository state you can inspect.

## What NightShift Is Not

NightShift does not push branches, deploy software, run unbounded task swarms, or grant agents unlimited repository access. Human review remains the final authority.

## Install

Development install:

```bash
pip install -e .
```

You can also run the CLI module directly from a checkout:

```bash
python -m nightshift.cli --help
```

NightShift uses the Python standard library for runtime behavior where practical. PyYAML is used automatically if installed, but starter configs work with the built-in YAML subset parser.

## Getting Started

Start with the [Quickstart](QUICKSTART.md). It uses deterministic fake agents so you can verify lookup, context generation, patch validation, patch apply, tests, and artifacts without installing a model.

After that works, continue with [Tutorial 01: Running NightShift With Real Local Models](examples/tutorial/01-intro.md). It swaps the fake agents for Ollama-backed agents such as `qwen2.5-coder:14b` and walks through dry-run and apply-mode patch generation.

### Quickstart Commands

Validate the included end-to-end patch example:

```bash
python -m nightshift.cli validate --config examples/quickstart-lisp/nightshift.yaml
```

Run the first task against a copy of the example project. The pipeline uses `patch_apply mode: apply`, so running it directly against `examples/quickstart-lisp/` will modify those files.

```bash
cp -r examples/quickstart-lisp /tmp/nightshift-quickstart
python -m nightshift.cli run --config /tmp/nightshift-quickstart/nightshift.yaml --task TASK-001
```

For a new project:

```bash
nightshift init
nightshift validate
nightshift status
nightshift run --task TASK-001
```

Open the read-only artifact dashboard:

```bash
pip install flask
nightshift web
```

## Task File Example

Tasks live in markdown checklist format:

```markdown
# Tasks

- [ ] TASK-001: Add parser support

Description:
Implement parsing for the target language.

Acceptance Criteria:
- Parses numbers
- Parses symbols
- Parses nested lists
- Includes unit tests
```

NightShift parses task id, title, completion state, description, acceptance criteria, dependency bullets, and raw task markdown.

## Pipeline Example

```yaml
pipeline:
  max_task_retries: 2
  continue_on_task_failure: false
  stages:
    - id: plan
      type: agent
      agent: planner
      output: plan.md

    - id: context
      type: repo_context
      output: context-pack.md

    - id: implement
      type: code_writer
      agent: implementer
      output: proposed.patch

    - id: normalize
      type: patch_normalizer
      output: normalized.patch

    - id: validate_patch
      type: patch_validator
      output: patch-validation.md
      max_files: 8
      max_lines: 800
      on_fail: implement

    - id: apply_patch
      type: patch_apply
      mode: apply
      output: patch-apply-output.txt
      on_fail: implement

    - id: test
      type: command
      commands:
        - python -m unittest discover -v
      output: test-output.txt
      on_fail: implement

    - id: review
      type: agent_review
      agent: reviewer
      on_fail: implement
      output: review.md
```

Use `mode: dry_run` for patch applicability checks without modifying files. Use `mode: apply` to write the validated patch to the target project.

## Agent Backends

NightShift supports:

- `backend: command`
- `backend: ollama`
- `backend: openai_compatible`

Example Ollama agent:

```yaml
agents:
  implementer:
    backend: ollama
    model: qwen2.5-coder:14b
    base_url: http://localhost:11434
    temperature: 0.2
    system_prompt: agents/implementer.md
```

The Ollama backend uses the local HTTP API instead of `ollama run`, which keeps exact patch output away from terminal rendering and line wrapping.

Example OpenAI-compatible agent:

```yaml
agents:
  implementer:
    backend: openai_compatible
    model: local-model
    base_url: http://localhost:11434/v1
    api_key_env: OPENAI_API_KEY
    temperature: 0.2
    system_prompt: agents/implementer.md
```

NightShift passes prompt bundles to agents and persists stdout, stderr, exit code, duration, and prompt artifacts. Code writer agents should return unified diffs. On retries, patch artifacts are versioned by attempt, for example `repair-1.patch`, `normalized-1.patch`, and `patch-validation-1.md`.

Review agents should emit:

```yaml
status: pass | fail | retry | escalate
reason: <short explanation>
next_stage: <optional stage id>
context_update: <compact useful note>
```

## Safety Model

NightShift validates paths, commands, and patches before mutation.

Path safety:

- project roots are resolved with `pathlib`
- task and prompt files must stay inside the project root
- artifact paths cannot escape `.nightshift/`
- repo lookup tools are constrained by `safety.scoped_paths`

Command safety:

- command stages must match `allowed_commands`
- forbidden fragments are blocked before allowlist acceptance
- command output and exit codes are recorded
- command stages stop at the first failing or timed-out command

Patch safety:

- code changes are represented as unified diffs
- patches are normalized and validated before apply
- path traversal and forbidden paths are rejected
- scoped paths, max files, and max changed lines are enforced
- `patch_apply` records apply output and git status artifacts

## Artifact Layout

A run creates human-readable artifacts:

```text
.nightshift/
  project-context.md
  project-context-chart.md
  nightshift.log
  runs/
    <run-id>/
      run.log
      run-summary.md
      config.snapshot.yaml
      run-metadata.md
      prompts/
        <agent-id>.md
      tasks/
        TASK-001/
          task.md
          context.md
          files-inspected.md
          context-pack.md
          plan.md
          proposed.patch
          repair-1.patch
          normalized.patch
          normalized-1.patch
          patch-validation.md
          patch-validation-1.md
          applied.patch
          applied-1.patch
          patch-apply-output.txt
          patch-apply-output-1.txt
          test-output.txt
          review.md
          stage-results.md
          context-out.md
          task-completion.md
          diff.patch
          final-notes.md
```

Exact artifact names depend on configured stage `output` values.

## Development

Run tests:

```bash
python -m unittest discover -v
```

Compile-check modules:

```bash
python -m compileall nightshift tests
```

Additional docs:

- [Quickstart](QUICKSTART.md)
- [Tutorial: running real local models](examples/tutorial/01-intro.md)
- [Config reference](docs/config-reference.md)
- [Artifact review workflow](docs/artifact-review.md)
- [Troubleshooting](docs/troubleshooting.md)
- [Quickstart Lisp example](examples/quickstart-lisp/)

## Roadmap

The active roadmap now lives in [docs/design.md](docs/design.md). Completed phase checklists are cleared from that document so it stays focused on the current platform shape and the next important work.