nightshift/README.md

# NightShift

<p align="center">
  <img src="docs/images/logo.png" width="220">
</p>

Auditable local-first AI coding pipelines.

NightShift is a deterministic pipeline runner for AI-assisted coding work. It reads markdown tasks, builds bounded context, asks configured agents for plans or patches, validates and applies those patches through explicit stages, runs checks, and leaves a human-reviewable artifact trail.

NightShift is not an autonomous software engineer. It is an orchestration layer that treats AI agents as unreliable workers inside bounded, testable, auditable workflows.

## Current Status

NightShift now supports the full local patch workflow:

- `nightshift init`, `validate`, `status`, `run`, `run --task`, `run --all`, and `web`.
- Markdown task parsing with dependencies.
- Command, Ollama, and OpenAI-compatible agent backends.
- Per-agent model settings such as `temperature`.
- Repo lookup tools: scoped `list_files`, `read_file`, and `grep`.
- Planner lookup requests with `files-inspected.md` artifacts.
- `repo_context` stage for `context-pack.md`.
- Project context chart generation at `.nightshift/project-context-chart.md`.
- `code_writer` stage that requires unified diff output.
- `patch_normalizer`, `patch_validator`, and `patch_apply` stages.
- Patch dry-run and apply modes.
- Test/static failure repair loops through existing retry routing.
- Run logs, dashboard log tails, git status artifacts, diffs, stage summaries, and final reports.

The default posture remains local-first and review-first: agents propose; NightShift validates, applies, tests, and records.

## What NightShift Is

NightShift is built for reviewable automation:

- local-first execution
- declarative pipeline stages
- markdown task files
- command-backed and model-backed agent wrappers
- explicit retry limits
- scoped repository lookup
- patch validation before mutation
- command allowlists
- durable markdown/text artifacts
- compact context handoff
- final reports for human review

The goal is to wake up to useful artifacts and a repository state you can inspect.

## What NightShift Is Not

NightShift does not push branches, deploy software, run unbounded task swarms, or grant agents unlimited repository access. Human review remains the final authority.

## Install

Development install:

```bash
pip install -e .
```

You can also run the CLI module directly from a checkout:

```bash
python -m nightshift.cli --help
```

NightShift uses the Python standard library for runtime behavior where practical. PyYAML is used automatically if installed, but starter configs work with the built-in YAML subset parser.

## Getting Started

Start with the [Quickstart](QUICKSTART.md). It uses deterministic fake agents so you can verify lookup, context generation, patch validation, patch apply, tests, and artifacts without installing a model.

After that works, continue with [Tutorial 01: Running NightShift With Real Local Models](docs/tutorial/01-intro.md). It swaps the fake agents for Ollama-backed agents such as `qwen2.5-coder:14b` and walks through dry-run and apply-mode patch generation.

### Quickstart Commands

Validate the included end-to-end patch example:

```bash
python -m nightshift.cli validate --config examples/quickstart-lisp/nightshift.yaml
```

Run the first task against a copy of the example project. The pipeline uses `patch_apply mode: apply`, so running it directly against `examples/quickstart-lisp/` will modify those files.

```bash
cp -r examples/quickstart-lisp /tmp/nightshift-quickstart
python -m nightshift.cli run --config /tmp/nightshift-quickstart/nightshift.yaml --task TASK-001
```

For a new project:

```bash
nightshift init
nightshift validate
nightshift status
nightshift run --task TASK-001
```

Open the read-only artifact dashboard:

```bash
pip install flask
nightshift web
```

## Task File Example

Tasks live in markdown checklist format:

```markdown
# Tasks

- [ ] TASK-001: Add parser support

Description:
Implement parsing for the target language.

Acceptance Criteria:
- Parses numbers
- Parses symbols
- Parses nested lists
- Includes unit tests
```

NightShift parses task id, title, completion state, description, acceptance criteria, dependency bullets, and raw task markdown.

## Pipeline Example

```yaml
pipeline:
  max_task_retries: 2
  continue_on_task_failure: false
  stages:
    - id: plan
      type: agent
      agent: planner
      output: plan.md

    - id: context
      type: repo_context
      output: context-pack.md

    - id: implement
      type: code_writer
      agent: implementer
      output: proposed.patch

    - id: normalize
      type: patch_normalizer
      output: normalized.patch

    - id: validate_patch
      type: patch_validator
      output: patch-validation.md
      max_files: 8
      max_lines: 800

    - id: apply_patch
      type: patch_apply
      mode: apply
      output: patch-apply-output.txt

    - id: test
      type: command
      commands:
        - python -m unittest discover -v
      output: test-output.txt
      on_fail: implement

    - id: review
      type: agent_review
      agent: reviewer
      on_fail: implement
      output: review.md
```

Use `mode: dry_run` for patch applicability checks without modifying files. Use `mode: apply` to write the validated patch to the target project.

## Agent Backends

NightShift supports:

- `backend: command`
- `backend: ollama`
- `backend: openai_compatible`

Example Ollama agent:

```yaml
agents:
  implementer:
    backend: ollama
    model: qwen2.5-coder:14b
    temperature: 0.2
    system_prompt: agents/implementer.md
```

Example OpenAI-compatible agent:

```yaml
agents:
  implementer:
    backend: openai_compatible
    model: local-model
    base_url: http://localhost:11434/v1
    api_key_env: OPENAI_API_KEY
    temperature: 0.2
    system_prompt: agents/implementer.md
```

NightShift passes prompt bundles to agents and persists stdout, stderr, exit code, duration, and prompt artifacts. Code writer agents should return unified diffs.

Review agents should emit:

```yaml
status: pass | fail | retry | escalate
reason: <short explanation>
next_stage: <optional stage id>
context_update: <compact useful note>
```

## Safety Model

NightShift validates paths, commands, and patches before mutation.

Path safety:

- project roots are resolved with `pathlib`
- task and prompt files must stay inside the project root
- artifact paths cannot escape `.nightshift/`
- repo lookup tools are constrained by `safety.scoped_paths`

Command safety:

- command stages must match `allowed_commands`
- forbidden fragments are blocked before allowlist acceptance
- command output and exit codes are recorded
- command stages stop at the first failing or timed-out command

Patch safety:

- code changes are represented as unified diffs
- patches are normalized and validated before apply
- path traversal and forbidden paths are rejected
- scoped paths, max files, and max changed lines are enforced
- `patch_apply` records apply output and git status artifacts

## Artifact Layout

A run creates human-readable artifacts:

```text
.nightshift/
  project-context.md
  project-context-chart.md
  nightshift.log
  runs/
    <run-id>/
      run.log
      run-summary.md
      config.snapshot.yaml
      run-metadata.md
      prompts/
        <agent-id>.md
      tasks/
        TASK-001/
          task.md
          context.md
          files-inspected.md
          context-pack.md
          plan.md
          proposed.patch
          normalized.patch
          patch-validation.md
          applied.patch
          patch-apply-output.txt
          test-output.txt
          review.md
          stage-results.md
          context-out.md
          task-completion.md
          diff.patch
          final-notes.md
```

Exact artifact names depend on configured stage `output` values.

## Development

Run tests:

```bash
python -m unittest discover -v
```

Compile-check modules:

```bash
python -m compileall nightshift tests
```

Additional docs:

- [Quickstart](QUICKSTART.md)
- [Tutorial: running real local models](docs/tutorial/01-intro.md)
- [Config reference](docs/config-reference.md)
- [Artifact review workflow](docs/artifact-review.md)
- [Troubleshooting](docs/troubleshooting.md)
- [Quickstart Lisp example](examples/quickstart-lisp/)

## Roadmap

The active roadmap now lives in [docs/design.md](docs/design.md). Completed phase checklists are cleared from that document so it stays focused on the current platform shape and the next important work.