mirror of
https://github.com/khodges42/nightShift.git
synced 2026-06-14 18:18:36 +00:00
K. Hodges
f7fed4535b
- Add `integ-test` to create, set up, validate, and run integration template tasks - Add `integ-report` to summarize latest integration run artifacts - Switch default pastebin template from model fallback to single `qwen3-coder:30b` - Support optional Ollama fields: `num_ctx`, `num_predict`, `seed`, and `stop` - Add `nightshift validate` preflight for task-specific test files - Update pastebin docs, config reference, and ideas tracking - Add tests for integration helpers, task-test validation, config parsing, and template expectations
18 lines
1.1 KiB
Markdown
18 lines
1.1 KiB
Markdown
### Future Ideas
|
|
Not to implement until we get successful long running runs.
|
|
|
|
## I am realizing "templates" are abstracted from the user
|
|
* I think templates will be a first class citizen, a package for deployments, and a harness for performance tests
|
|
* These should live external to nightshift/project_templates as users will likely create their own
|
|
* one solution would be to reference two directories when looking up templates, builtin ones will be in nightshift/project_templates or users can define a templates directory in their nightshift config
|
|
|
|
## nightshift config
|
|
* store user settings in ~/.nightshift/config.yaml
|
|
* things like templates folder (can also live here)
|
|
* maybe this is later
|
|
|
|
## A way to easily make A/B tests to benchmark models?
|
|
* Right now I can do this manually, for example I want to run the tutorial-pastebin with qwen3.6:27b as the planner and qwen2.5-coder:14b as the coder, and another with qwen3.6:27b as both, etc.
|
|
* Maybe there is a way to make it easier to do that, possibly by creating a template that can be controlled by a larger multi-run file?
|
|
* This is probably for way later.
|