mirror of
https://github.com/khodges42/nightShift.git
synced 2026-06-14 10:08:37 +00:00
K. Hodges
f7fed4535b
- Add `integ-test` to create, set up, validate, and run integration template tasks - Add `integ-report` to summarize latest integration run artifacts - Switch default pastebin template from model fallback to single `qwen3-coder:30b` - Support optional Ollama fields: `num_ctx`, `num_predict`, `seed`, and `stop` - Add `nightshift validate` preflight for task-specific test files - Update pastebin docs, config reference, and ideas tracking - Add tests for integration helpers, task-test validation, config parsing, and template expectations
1.1 KiB
1.1 KiB
Future Ideas
Not to implement until we get successful long running runs.
I am realizing "templates" are abstracted from the user
- I think templates will be a first class citizen, a package for deployments, and a harness for performance tests
- These should live external to nightshift/project_templates as users will likely create their own
- one solution would be to reference two directories when looking up templates, builtin ones will be in nightshift/project_templates or users can define a templates directory in their nightshift config
nightshift config
- store user settings in ~/.nightshift/config.yaml
- things like templates folder (can also live here)
- maybe this is later
A way to easily make A/B tests to benchmark models?
- Right now I can do this manually, for example I want to run the tutorial-pastebin with qwen3.6:27b as the planner and qwen2.5-coder:14b as the coder, and another with qwen3.6:27b as both, etc.
- Maybe there is a way to make it easier to do that, possibly by creating a template that can be controlled by a larger multi-run file?
- This is probably for way later.