wiki updated added WIKI_API_CONTRACT and WIKI_CUTOVER_RUNBOOK

updated:
Home
WIKI_MIGRATION_ACCEPTANCE_CRITERIA
WIKI_SETUP
stan44 2026-02-23 20:19:14 -06:00
parent b01e0273ae
commit 6d33cd748f
5 changed files with 219 additions and 162 deletions

14
Home.md

@ -9,12 +9,14 @@ Project Journal is an offline-first journaling system with encrypted monthly vau
Use these pages as your Gitea wiki starter set: Use these pages as your Gitea wiki starter set:
- `Home` (this page) - `Home` (this page)
- `Setup and Installation` (`WIKI_SETUP.md`) - `Setup and Installation` (`WIKI_SETUP.md`)
- `CLI Reference` (`WIKI_CLI.md`) - `CLI Reference` (`WIKI_CLI.md`)
- `Vaults and Data Handling` (`WIKI_VAULTS.md`) - `Vaults and Data Handling` (`WIKI_VAULTS.md`)
- `C# Migration Status` (`WIKI_CSHARP_MIGRATION.md`) - `Migration Acceptance Criteria` (`WIKI_MIGRATION_ACCEPTANCE_CRITERIA.md`)
- `Troubleshooting` (`WIKI_TROUBLESHOOTING.md`) - `API and Sidecar Contract` (`WIKI_API_CONTRACT.md`)
- `Development Workflow` (`WIKI_DEVELOPMENT_WORKFLOW.md`) - `Cutover Runbook (Phase 6)` (`WIKI_CUTOVER_RUNBOOK.md`)
- `Troubleshooting` (`WIKI_TROUBLESHOOTING.md`)
- `Development Workflow` (`WIKI_DEVELOPMENT_WORKFLOW.md`)
## Quick Start (Python Baseline) ## Quick Start (Python Baseline)

97
WIKI_API_CONTRACT.md Normal file

@ -0,0 +1,97 @@
# API and Sidecar Contract
This page documents the canonical transport contract used by the C# backend.
## HTTP Surface (Locked)
- `POST /api/command`
- `GET /health`
- `GET /healthz`
No template/demo endpoints are part of parity mode.
## Envelope
Request (`POST /api/command`):
```json
{
"action": "entries.list",
"id": "optional-guid",
"payload": {},
"correlationId": "optional-client-correlation-id"
}
```
Success response:
```json
{
"ok": true,
"data": {}
}
```
Failure response:
```json
{
"ok": false,
"error": "Missing or invalid payload"
}
```
## Action Map
`fragments.*`
- `fragments.list` -> no payload
- `fragments.get` -> `id` (guid)
- `fragments.create` -> `payload: { type, description, time?, tags? }`
- `fragments.update` -> `id` (guid) + `payload: { type?, description?, time?, tags? }`
- `fragments.delete` -> `id` (guid)
- `fragments.search` -> optional top-level `type`, `tag`
`entries.*`
- `entries.list` -> `payload: { dataDirectory? }`
- `entries.load` -> `payload: { filePath }`
- `entries.save` -> `payload: { fileName, content, dataDirectory? }`
`search.*`
- `search.entries` -> `payload: { dataDirectory, query?, section?, startDate?, endDate?, tags?, types?, checked?, unchecked? }`
`vault.*`
- `vault.initialize` -> `payload: { password, vaultDirectory }`
- `vault.load_all` -> `payload: { password, vaultDirectory, dataDirectory }`
- `vault.save_current_month` -> `payload: { password, vaultDirectory, dataDirectory, nowUtc? }`
- `vault.rebuild_all` -> `payload: { password, vaultDirectory, dataDirectory }`
- `vault.clear_data_directory` -> `payload: { dataDirectory }`
`db.*`
- `db.status` -> `payload: { password, dataDirectory? }`
- `db.initialize_schema` -> `payload: { dataDirectory? }`
- `db.hydrate_workspace` -> `payload: { password, dataDirectory? }`
`ai.*` (bridge only, execution remains Python)
- `ai.health` -> no payload
- `ai.summarize_entry` -> `payload: { content, fileStem? }`
- `ai.summarize_all` -> `payload: { entries }`
- `ai.chat` -> `payload: { prompt }`
- `ai.embed` -> `payload: { content }`
`speech.*` (bridge/orchestration in C#, execution remains Python)
- `speech.devices.list` -> no payload
- `speech.transcribe` -> `payload: { audioBase64|audio_base64 OR text, engine?, whisperModel|whisper_model?, simulateDelayMs|simulate_delay_ms? }`
`config.*`
- `config.get` -> no payload
## DTO Notes (C#)
The C# side uses DTOs as command/service contracts in `journal-master/journal/Journal.Core/Dtos/`, including:
- `EntrySearchRequestDto`, `EntrySearchResultDto`
- `CreateFragmentDto`, `UpdateFragmentDto`, `FragmentDto`
- `SpeechTranscribeRequestDto`, `SpeechTranscribeResultDto`, `SpeechDeviceDto`
- `AiHealthDto`
Primary action routing is implemented in:
- `journal-master/journal/Journal.Core/Entry.cs`

75
WIKI_CUTOVER_RUNBOOK.md Normal file

@ -0,0 +1,75 @@
# Hybrid-to-C# Cutover Runbook (Phase 6)
This runbook defines the staged cutover from Python baseline to C# non-AI backend, while keeping AI/speech execution in Python.
## Architecture Lock
- C# is system-of-record for non-AI backend paths.
- AI inference and speech execution remain Python runtime.
- C# uses bridge/orchestration contracts (`ai.*`, `speech.*`) only.
## Stage 0: Gate Green (Required Before Any Soak/Pilot)
Run:
```powershell
./scripts/migration-gate.ps1
```
Must pass:
- C# builds (Sidecar + API)
- C# smoke tests
- Python vs C# parity harness
- API contract tests
Artifacts:
- `logs/parity_harness_results.json`
- `fixtures/vaults/manifest.json`
## Stage 1: Internal Soak (8h+)
Run hybrid mode and exercise:
- login + vault load/unload
- create/edit/save/reload large entries
- search filters (date/section/tag/type/checkbox)
- speech actions (healthy + failure path)
- app idle periods + repeated tab switching
Acceptance:
- no stuck login loops
- no sidebar/session degradation
- no vault corruption
- no plaintext leftovers after shutdown/cleanup
## Stage 2: Small User Pilot
Enable 1-3 pilot users with representative workflows.
Collect:
- reproducible bug reports (steps + timestamps)
- gate command output after fixes
- any parity mismatches from harness
## Stage 3: Default Cutover
Keep backend default as:
- `JOURNAL_BACKEND_MODE=csharp-hybrid`
Promote once Stage 0-2 remain stable over consecutive runs.
## Rollback Path (Immediate)
Set environment variable and restart:
```powershell
$env:JOURNAL_BACKEND_MODE = "python"
python .\journal\run_desktop.py
```
Rollback must remain validated in smoke and manual startup checks.
## Operational Policy During Cutover
- UI freeze except bug fixes that block usability or stability.
- No feature expansion until gate regressions are clear.
- Every merge affecting backend contract must include updated evidence in `MIGRATION_ACCEPTANCE_CRITERIA.md`.

@ -1,165 +1,35 @@
# C# Backend Parity Acceptance Criteria (Python Twin) # Migration Acceptance Criteria (Source of Truth)
## Goal Authoritative file:
Create a C# backend that is behaviorally equivalent to the current Python backend for core journaling workflows, with zero data-loss regressions and compatible vault handling. - `MIGRATION_ACCEPTANCE_CRITERIA.md`
## Parity Rule This wiki page is a quick pointer and status summary. Update the root file first, then mirror highlights here.
Behavior parity is required. Internal implementation can differ.
## Source of Truth (Current Python) ## Current snapshot (2026-02-24)
- `journal/core/storage.py`
- `journal/core/encryption.py`
- `journal/core/parser.py`
- `journal/core/models.py`
- `journal/core/database.py`
- `journal/cli/main.py`
- `journal/ai/analysis.py`
- `journal/ai/chat.py`
- `journal/core/speech.py`
## Definition of Done (Release Gate) - Architecture lock:
1. All `P0` criteria below are marked `Pass`. - Non-AI backend remains C# system-of-record.
2. Existing Python-created vault files load successfully in C# backend. - AI and speech execution remain Python-side (C# bridge/orchestration only).
3. Side-by-side output comparison on shared fixture corpus shows no functional mismatch for `P0` flows. - Completed parity gates:
4. No plaintext journal data remains after graceful shutdown flow. - `API-002`: HTTP contract lock via `/api/command` + `/health`/`/healthz` with contract tests.
- `SPC-001`: Speech bridge contract implemented (`speech.devices.list`, `speech.transcribe`).
- `OBS-001`: Structured redacted logging envelope and redaction tests.
- Release gate command:
- `./scripts/migration-gate.ps1`
## Shared Fixture Corpus (Required) ## Gate command coverage
1. `fixtures/vaults/`
- Multiple months (`YYYY-MM.vault`) generated by Python app.
- Includes at least one wrong-password test case.
2. `fixtures/entries/`
- Daily, deep, recovery, fragment-heavy entries.
- Includes multiline fragments, tags, checkboxes, unusual spacing.
3. `fixtures/search/`
- Queries for text, section, tag, type, checked, unchecked, date ranges.
4. `fixtures/ai/`
- Stubbed LLM/embedding responses for deterministic comparison.
## Acceptance Matrix `migration-gate.ps1` runs:
Use status values: `Not Started`, `In Progress`, `Blocked`, `Pass`, `Fail`. 1. C# sidecar/api builds
2. C# smoke tests
3. Python-vs-C# parity harness + vault fixture matrix
4. API contract tests
| ID | Priority | Feature | Acceptance Criteria | Status | Owner | Evidence | Artifacts:
| ------- | -------- | --------------------------------- | ---------------------------------------------------------------------------------------------------------------------- | ----------- | ----- | -------- | - `logs/parity_harness_results.json`
| DOM-001 | P0 | Fragment model parity | Fragment has `type`, `description`, `time`, `tags`; rejects empty `type`/`description`; trims values. | Pass | Stan44/Codex | `Journal.Core/Models/Fragment.cs`, `Journal.Core/Services/FragmentService.cs`, smoke test `CreateAsync trims fields` + `UpdateAsync rejects whitespace type` | - `fixtures/vaults/manifest.json`
| DOM-002 | P0 | Journal entry domain parity | Entry model supports `date`, `raw_content`, `sections`, `fragments` equivalent to Python semantics. | Not Started | | |
| PAR-001 | P0 | Date extraction parity | Parser reads `**Date:**` or `Date:`; falls back to filename stem when missing. | Not Started | | |
| PAR-002 | P0 | Section parsing parity | Recognizes canonical section titles and captures section content lines. | Not Started | | |
| PAR-003 | P0 | Checkbox parsing parity | Parses markdown checkboxes (`- [ ]`, `- [x]`) and preserves checked state by checkbox text. | Not Started | | |
| PAR-004 | P0 | Fragment parsing parity | Parses `!TYPE @time #tags` plus multiline description blocks with same boundary behavior as Python regex parser. | Not Started | | |
| MRG-001 | P0 | Merge behavior parity | Saving existing entry merges non-empty section updates and appends only non-duplicate fragments by description. | Not Started | | |
| MRG-002 | P0 | Markdown reconstruction parity | Entry serialization writes canonical section order and fragment block formatting equivalent to Python `to_markdown()`. | Not Started | | |
| VLT-001 | P0 | Vault crypto format compatibility | Uses AES-256-GCM with payload layout compatible with Python (`salt + nonce + tag + ciphertext`). | Not Started | | |
| VLT-002 | P0 | KDF compatibility | Uses PBKDF2-HMAC-SHA256 with matching salt/key sizes and iteration count for Python vault compatibility. | Not Started | | |
| VLT-003 | P0 | Monthly vault naming parity | Vault filename format exactly `YYYY-MM.vault`. | Not Started | | |
| VLT-004 | P0 | Load workflow parity | `load all vaults` clears decrypted workspace first, then decrypts/extracts monthly vaults. | Not Started | | |
| VLT-005 | P0 | Wrong password behavior | Wrong password returns explicit failure and does not corrupt existing vault files. | Not Started | | |
| VLT-006 | P1 | Legacy vault handling | Legacy `_init_vault.vault` handling behavior is preserved or intentionally migrated with backward-compat note. | Not Started | | |
| VLT-007 | P0 | Current-month optimized save | Supports current-month save path equivalent to Python optimization. | Not Started | | |
| VLT-008 | P0 | Full rebuild save | Supports full monthly regroup/rebuild save flow from decrypted `.md` files. | Not Started | | |
| DAT-001 | P0 | Decrypted data cleanup | Graceful shutdown removes decrypted workspace artifacts. | Not Started | | |
| DB-001 | P1 | SQLCipher compatibility | Database key derivation and SQLCipher connection behavior are compatible with Python expectations. | Not Started | | |
| DB-002 | P1 | Schema parity | `entries`, `sections`, `fragments`, `tags`, `fragment_tags` schema exists with compatible constraints. | Not Started | | |
| SCH-001 | P0 | Search parity (content) | Search supports text query over full entry content. | Not Started | | |
| SCH-002 | P0 | Search parity (filters) | Search supports date range, section filter, tag filter, fragment type filter, checked/unchecked filters. | Not Started | | |
| CLI-001 | P0 | Vault CLI parity | CLI supports vault load/save with password prompt semantics matching Python UX. | Not Started | | |
| CLI-002 | P0 | Search CLI parity | CLI options match existing Python capabilities sufficiently for drop-in replacement. | Not Started | | |
| API-001 | P0 | Sidecar protocol stability | Stdin/stdout contract is line-delimited JSON with `{ok,data}` or `{ok:false,error}` and strict action routing. | In Progress | Stan44/Codex | `Journal.Core/Entry.cs`, smoke tests for unknown action / missing payload / missing id handling |
| API-002 | P1 | HTTP API parity | API exposes endpoints equivalent to supported sidecar actions; no template-only endpoints in parity mode. | Not Started | | |
| AI-001 | P1 | LLM summarize parity | Entry/all-entry summarize workflow callable with equivalent behavior and timeout controls. | Not Started | | |
| AI-002 | P2 | Embedding parity | Embedding endpoint integration available with equivalent request/response contract. | Not Started | | |
| AI-003 | P1 | Cloud chat parity | Cloud chat request/response flow available and configurable. | Not Started | | |
| SPC-001 | P2 | Speech parity | Engine-selectable speech flow (whisper/google/sphinx-like modes) retained or intentionally delegated. | Not Started | | |
| CFG-001 | P0 | Config parity | Equivalent configuration keys exist for paths, vault format, AI endpoints, and speech settings. | Not Started | | |
| OBS-001 | P1 | Logging/error parity | Failures return actionable messages without leaking secrets or plaintext journal data. | In Progress | Stan44/Codex | Error envelope behavior implemented in sidecar; structured logging policy pending |
## Mandatory Gate Tests (P0) ## Phase 6 execution
1. `Vault Compatibility`
- Given existing Python vault fixtures, C# load succeeds with correct password.
- Wrong password consistently fails without partial corruption.
2. `Parser + Merge`
- Given fixture entries, C# parser output matches Python parser output for sections/fragments/checkboxes.
- Saving edited entry preserves merge semantics.
3. `Search`
- For shared corpus, C# and Python return the same entry set for all P0 search filters.
4. `Transport`
- Sidecar commands for core actions produce stable JSON success/error envelopes.
5. `Cleanup`
- After graceful shutdown sequence, decrypted workspace is empty.
## Migration Phases (Recommended) - Operational cutover/rollback steps are tracked in:
1. `Phase 1: Fragment Vertical Slice` - `WIKI_CUTOVER_RUNBOOK.md`
- Complete `DOM-001`, `API-001`, persistence beyond in-memory.
2. `Phase 2: Entry/Parser/Merge Twin`
- Complete `DOM-002`, `PAR-*`, `MRG-*`.
3. `Phase 3: Vault/Crypto Twin`
- Complete `VLT-*`, `DAT-001`.
4. `Phase 4: Search + CLI Twin`
- Complete `SCH-*`, `CLI-*`, `CFG-001`.
5. `Phase 5: AI/Speech`
- Complete `AI-*`, `SPC-001`.
6. `Phase 6: Frontend Cutover`
- Tauri frontend switches to C# backend only after all P0 criteria pass.
## Rules for Change Requests
1. Any intentional divergence from Python behavior requires:
- Written rationale.
- New acceptance criterion replacing old one.
- Migration note describing user-visible impact.
2. No removal of a `P0` criterion without both maintainers approving in writing.
## Status Snapshot (2026-02-22, Phase 1.2 Progress)
This section is a proposed status/evidence snapshot based on current code in:
- Python source: `Project_Journal/`
- C# migration: `Project_Journal/journal-master/journal/`
It does not change release gates or acceptance definitions.
### Phase Summary
| Phase | Current State | Evidence |
| ----- | ------------- | -------- |
| Phase 1: Fragment Vertical Slice | In Progress (near-complete) | Fragment model/service/sidecar + persisted repository + smoke tests: `journal-master/journal/Journal.Core/Models/Fragment.cs`, `journal-master/journal/Journal.Core/Entry.cs`, `journal-master/journal/Journal.Core/Repositories/FileFragmentRepository.cs`, `journal-master/journal/Journal.SmokeTests/Program.cs` |
| Phase 2: Entry/Parser/Merge Twin | Not Started | No C# entry/parser/merge domain yet. |
| Phase 3: Vault/Crypto Twin | Not Started | No C# vault/crypto compatibility module yet. |
| Phase 4: Search + CLI Twin | Not Started | C# search remains fragment-scoped only (not full entry parity). |
| Phase 5: AI/Speech | Not Started | No C# AI/speech parity services yet. |
| Phase 6: Frontend Cutover | Not Started | P0 parity not yet achieved. |
### Proposed Acceptance Status Updates
| ID | Proposed Status | Evidence | Notes |
| ------- | --------------- | -------- | ----- |
| DOM-001 | Pass | `journal-master/journal/Journal.Core/Models/Fragment.cs`, `journal-master/journal/Journal.Core/Services/FragmentService.cs`, `journal-master/journal/Journal.SmokeTests/Program.cs` | Fields/validation/trim behavior implemented and smoke-tested. |
| DOM-002 | Not Started | `journal-master/journal/Journal.Core/Entry.cs` | No C# `JournalEntry` parity domain yet. |
| PAR-001 | Not Started | `journal-master/journal/Journal.Core/Entry.cs` | No C# parser/date extraction path yet. |
| PAR-002 | Not Started | `journal-master/journal/Journal.Core/Entry.cs` | No section parser in C#. |
| PAR-003 | Not Started | `journal-master/journal/Journal.Core/Entry.cs` | No checkbox parser in C#. |
| PAR-004 | Not Started | `journal-master/journal/Journal.Core/Entry.cs` | No fragment markdown parser in C#. |
| MRG-001 | Not Started | `journal-master/journal/Journal.Core/Entry.cs` | No entry merge behavior in C#. |
| MRG-002 | Not Started | `journal-master/journal/Journal.Core/Entry.cs` | No markdown reconstruction pipeline in C#. |
| VLT-001 | Not Started | `journal-master/journal/Journal.Core/Entry.cs` | No vault encryption/decryption compatibility module yet. |
| VLT-002 | Not Started | `journal-master/journal/Journal.Core/Entry.cs` | No PBKDF2 compatibility implementation yet. |
| VLT-003 | Not Started | `journal-master/journal/Journal.Core/Entry.cs` | No monthly vault naming/save implementation yet. |
| VLT-004 | Not Started | `journal-master/journal/Journal.Core/Entry.cs` | No load-all-vaults workflow in C#. |
| VLT-005 | Not Started | `journal-master/journal/Journal.Core/Entry.cs` | Wrong-password behavior not implemented in C# vault flow. |
| VLT-006 | Not Started | `journal-master/journal/Journal.Core/Entry.cs` | Legacy vault handling not implemented/documented in C#. |
| VLT-007 | Not Started | `journal-master/journal/Journal.Core/Entry.cs` | Current-month optimized save path not implemented in C#. |
| VLT-008 | Not Started | `journal-master/journal/Journal.Core/Entry.cs` | Full rebuild save flow not implemented in C#. |
| DAT-001 | Not Started | `journal-master/journal/Journal.Core/Entry.cs` | Decrypted workspace cleanup flow not implemented in C#. |
| DB-001 | Not Started | `journal-master/journal/Journal.Core/Entry.cs` | No SQLCipher keying/connection behavior in C#. |
| DB-002 | Not Started | `journal-master/journal/Journal.Core/Entry.cs` | No `entries/sections/fragments/tags/fragment_tags` schema in C#. |
| SCH-001 | Not Started | `journal-master/journal/Journal.Core/Repositories/FileFragmentRepository.cs` | C# search is fragment-centric, not full-entry content search. |
| SCH-002 | Not Started | `journal-master/journal/Journal.Core/Repositories/FileFragmentRepository.cs` | Date/section/checkbox filters not implemented for full entry search. |
| CLI-001 | Not Started | `journal-master/journal/Journal.Sidecar/App.cs` | Sidecar exists; vault CLI parity not implemented. |
| CLI-002 | Not Started | `journal-master/journal/Journal.Sidecar/App.cs` | Search CLI parity not implemented. |
| API-001 | In Progress | `journal-master/journal/Journal.Core/Entry.cs`, `journal-master/journal/Journal.SmokeTests/Program.cs` | JSON envelope routing implemented and tested; fixture-driven P0 gate tests still pending. |
| API-002 | Not Started | `journal-master/journal/Journal.Api/Program.cs` | HTTP API still template endpoint, no journal parity endpoints. |
| AI-001 | Not Started | `journal-master/journal/Journal.Core/Entry.cs` | No summarize workflow/service in C#. |
| AI-002 | Not Started | `journal-master/journal/Journal.Core/Entry.cs` | No embeddings integration in C#. |
| AI-003 | Not Started | `journal-master/journal/Journal.Core/Entry.cs` | No cloud chat flow in C#. |
| SPC-001 | Not Started | `journal-master/journal/Journal.Core/Entry.cs` | No speech module/service in C#. |
| CFG-001 | Not Started | `journal-master/journal/Journal.Core/ServiceCollectionExtensions.cs` | C# config surface is partial and not parity-complete. |
| OBS-001 | In Progress | `journal-master/journal/Journal.Core/Entry.cs` | Actionable error envelope exists; structured logging/secret-scrub policy pending. |
### Snapshot Risks/Blockers
1. Required fixture corpus (`fixtures/vaults`, `fixtures/entries`, `fixtures/search`, `fixtures/ai`) is still missing.
2. P0 gate test harness for side-by-side Python vs C# parity is not yet implemented.
3. Phase 1 remaining item is final API-001 hardening + fixture-backed transport stability evidence.

@ -100,3 +100,16 @@ cd journal-master/journal
./scripts/dotnet-min.ps1 build Journal.Sidecar/Journal.Sidecar.csproj ./scripts/dotnet-min.ps1 build Journal.Sidecar/Journal.Sidecar.csproj
./scripts/dotnet-min.ps1 run --project Journal.SmokeTests/Journal.SmokeTests.csproj ./scripts/dotnet-min.ps1 run --project Journal.SmokeTests/Journal.SmokeTests.csproj
``` ```
Release gate (repo root):
```powershell
cd Project_Journal
./scripts/migration-gate.ps1
```
This command runs:
- C# sidecar/api builds
- C# smoke suite
- Python-vs-C# parity harness + vault fixture matrix
- HTTP API contract tests