journal/README.md

Project Journal

A structured journaling system with encrypted monthly vaults, a Tauri desktop app, a web gateway server, CLI tools, and optional AI-assisted analysis.

Repository Layout

backend/                       Monorepo root
├── Journal.Core/              .NET class library — all business logic and services
├── Journal.AI/                .NET class library — LLM/AI integration (LLamaSharp)
├── Journal.Sidecar/           Console app — stdin/stdout JSON protocol (Tauri sidecar bridge + CLI)
├── Journal.WebGateway/        ASP.NET Core app — HTTP wrapper for browser/web mode
├── Journal.SmokeTests/        Integration tests (~80 tests, no test framework dependency)
├── Journal.App/               SvelteKit + Tauri desktop app
│   ├── src/                   SvelteKit frontend source
│   ├── src-tauri/             Rust Tauri shell (sidecar process manager)
│   └── static/                Static assets
├── Journal.DevTool/           Pre-built SDT orchestrator (sdt.exe) + Python scripts
├── Directory.Build.props      Shared .NET build properties (TFM, nullable, etc.)
├── Directory.Packages.props   Centralized NuGet package versions
├── Journal.slnx               Visual Studio solution (all .NET projects)
├── package.json               npm workspace root (Journal.App)
└── devtool.json               SDT workflow/toolchain configuration

Deployment Modes

The supported primary workflow is the desktop app with the local sidecar. WebGateway is a separate secondary surface for browser/mobile access.

Mode	Host	Frontend
Tauri desktop app	Journal.App (Tauri + Rust)	SvelteKit embedded via Tauri WebView
WebGateway server	Journal.WebGateway (ASP.NET Core)	SvelteKit build served from wwwroot
Sidecar CLI / stdin	Journal.Sidecar (console)	None — raw JSON protocol

All three modes share the same Journal.Core service layer and command protocol.

---

Platform Support

• Windows — first-class (primary development target)
• Linux — first-class
• macOS — best effort

Prerequisites

• .NET 10 SDK
• Node.js + npm (for Journal.App frontend)
• Rust + Cargo (for Journal.Sidecar Tauri desktop build)
• PowerShell 7+ (pwsh) recommended for scripts

---

Quickstart

Option A — Tauri Desktop App

Install dependencies and build:

npm install          # from repo root (workspaces)
dotnet publish Journal.Sidecar/Journal.Sidecar.csproj -c Release -r win-x64 --self-contained -p:PublishSingleFile=true
npm run tauri build -w Journal.App

Tauri auto-detects Journal.Sidecar.exe in the repository. On first launch it walks up from the working directory to find Journal.Sidecar/ and resolves the built executable. If you want the desktop app to use a different root, set it from Settings. That choice applies only to the desktop app.

Option B — WebGateway Server (browser mode)

Build the web UI bundle, then publish the gateway with web assets embedded:

npm run build -w Journal.App
dotnet publish Journal.WebGateway/Journal.WebGateway.csproj -c Release -r win-x64

Set the authoritative journal root explicitly before running a published gateway:

$env:JOURNAL_PROJECT_ROOT = "F:\path\to\journal-root"

Run the gateway:

dotnet run --project Journal.WebGateway

Open http://localhost:5180 in your browser. The gateway serves the SvelteKit build and proxies /api/command calls to Journal.Core.

Before browser login will work, generate a password hash and place it in Journal.WebGateway/appsettings.json under Security:AccessPasswordHash:

dotnet run --project Journal.WebGateway -- hash-password

This prints a PBKDF2-SHA256 hash. Store the hash, not the plaintext password.

Quick health check:

Invoke-RestMethod http://127.0.0.1:5180/api/health

Option C — Sidecar / CLI only

dotnet build

# Run in stdin/stdout protocol mode
dotnet run --project Journal.Sidecar

# CLI subcommands
dotnet run --project Journal.Sidecar -- vault load --password <value>
dotnet run --project Journal.Sidecar -- vault save --password <value>
dotnet run --project Journal.Sidecar -- search "your query" --tag stress --start-date 2026-02-01

---

C# Backend

Projects

Project	Type	Purpose
Journal.Core	Class library	Domain models, services, repositories, DTOs — all business logic
Journal.AI	Class library	LLM/AI integration (LLamaSharp) — references Journal.Core
Journal.Sidecar	Console app	Stdin/stdout JSON protocol + vault/search CLI subcommands
Journal.WebGateway	ASP.NET Core	HTTP API wrapper; serves built SvelteKit UI from wwwroot
Journal.SmokeTests	Console app	~80 integration tests (no xunit/nunit)

Solution File

Journal.slnx  (all .NET projects: Core, AI, Sidecar, WebGateway, SmokeTests)

All .NET projects share build properties via Directory.Build.props and NuGet versions via Directory.Packages.props (central package management).

Architecture

Entry  (thin command dispatcher — shared by all three hosts)
  ├── Fragments/    IFragmentService      → FragmentService      → SQLiteFragmentRepository (SQLCipher)
  ├── Entries/      IEntryFileService     → EntryFileService     → SqliteEntryFileRepository
  │                 IEntrySearchService   → EntrySearchService   (raw content + structured filters)
  │                 JournalParser         (date / section / checkbox / fragment parsing)
  ├── Lists/        IListService          → ListService          → SqliteListRepository
  ├── Todos/        ITodoService          → TodoService          → SqliteTodoRepository
  ├── Vault/        IVaultStorageService  → VaultStorageService  → IVaultCryptoService
  ├── Database/     IJournalDatabaseService (SQLCipher schema/key derivation/hydration)
  │                 IDatabaseSessionService (encrypted connection lifecycle after auth)
  ├── Ai/           IAiService            → LlamaSharpAiService | DisabledAiService
  ├── Speech/       ISpeechBridgeService  → DisabledSpeechBridgeService
  ├── Sidecar/      SidecarCli
  ├── Logging/      CommandLogger, LogRedactor
  └── Config/       IJournalConfigService → JournalConfigService

Services live under Journal.Core/Services/ in domain-specific subdirectories, each with its own namespace (e.g. Journal.Core.Services.Ai).

Build

# Build all .NET projects via the solution
dotnet build

Run Smoke Tests

dotnet run --project Journal.SmokeTests

Dependencies

NuGet package versions are managed centrally in Directory.Packages.props. Project-level .csproj files reference packages without version numbers.

• Journal.Core — Microsoft.Data.Sqlite.Core, SQLitePCLRaw.bundle_e_sqlcipher, Microsoft.Extensions.DependencyInjection.Abstractions
• Journal.AI — LLamaSharp, LLamaSharp.Backend.Cpu, LLamaSharp.Backend.Vulkan + references Journal.Core
• Journal.Sidecar — Microsoft.Extensions.DependencyInjection, NAudio, Whisper.net + references Journal.Core, Journal.AI
• Journal.WebGateway — Microsoft.NET.Sdk.Web + references Journal.Core, Journal.AI
• Journal.SmokeTests — references Journal.Core

Encryption

• Vault: AES-256-GCM with PBKDF2-HMAC-SHA256 key derivation (600k iterations)
• Database: SQLCipher with PBKDF2-derived key
• Fragments and structured data are stored in the encrypted SQLCipher database; auth is required via vault.load_all or db.hydrate_workspace
• DatabaseSessionService holds the encryption password in memory after first auth and closes the connection on vault.clear_data_directory

Environment Variables

Journal backend:

Variable	Default	Description
JOURNAL_PROJECT_ROOT	auto-detected	Override project root path (vault path resolution)
JOURNAL_VAULT_DIR	<root>/journal/vault	Override vault directory path
JOURNAL_DATA_DIR	(empty)	Override decrypted data directory path
JOURNAL_AI_PROVIDER	none	AI provider mode (none, llamasharp)
JOURNAL_GPU_LAYERS	-1 (all)	Number of model layers to offload to GPU (-1 = all, 0 = CPU only)
JOURNAL_LOG_LEVEL	warning	Log verbosity (trace, debug, information, warning, error, critical)
JOURNAL_WEB_DIST	auto	Override web UI dist path for WebGateway

SDT orchestrator:

Variable	Default	Description
SDT_ENV_PROFILE	dev	Active runtime environment profile (dev, ci, release)
SDT_LOG_LEVEL	information	CLI log verbosity (trace through critical)

---

AI / LLM Notes

The Journal.AI project uses LLamaSharp for local LLM inference.

• CPU backend (LLamaSharp.Backend.Cpu) is always installed as a fallback.
• Vulkan backend (LLamaSharp.Backend.Vulkan) provides GPU acceleration for AMD, Intel, and NVIDIA GPUs. LLamaSharp picks the best available backend at runtime.
• All backend packages must share the same version. Currently pinned to 0.25.0 because LLamaSharp.Backend.Vulkan has not yet published a 0.26.0 release. Watch the NuGet page and upgrade all three packages together when a new version ships.
• Known issue: on some machines the Vulkan backend falls back to CPU because the internal vulkaninfo --summary detection times out at 1 second. If you see CPU-only inference despite having a Vulkan-capable GPU, this is likely the cause. The LLamaSharp team has acknowledged the issue (#930).
• Set JOURNAL_GPU_LAYERS=-1 (the default) to offload all model layers to the GPU, or 0 to force CPU-only.

---

Journal.WebGateway

An ASP.NET Core minimal API that wraps Journal.Core for browser use.

Gateway Authentication

• Browser access is protected by a cookie login page at /gateway/login.
• The configured secret is Security:AccessPasswordHash, not a plaintext password.
• Generate a hash with:

dotnet run --project Journal.WebGateway -- hash-password

• Paste the resulting value into Journal.WebGateway/appsettings.json or set it via environment variable Security__AccessPasswordHash.
• The vault password remains separate and is still entered when unlocking the encrypted workspace.
• In published deployments, also set GatewaySettings:RepoRoot or JOURNAL_PROJECT_ROOT. Published WebGateway builds no longer guess the vault root at runtime.
• If the gateway hits a locked workspace, the web UI prompts for the vault password and retries instead of leaving the raw lock error on screen.

Authoritative Root

• Desktop and future mobile-native apps are the authoritative clients.
• Published WebGateway should point at that same authoritative root, but it does not share the desktop process or unlock state.
• In the desktop Settings screen, use Adopt Current Root under Gateway Root to write the current desktop root into the packaged gateway appsettings.json as a one-time alignment step.
• For manual deployments, set GatewaySettings:RepoRoot in output/webgateway/appsettings.json or set JOURNAL_PROJECT_ROOT before launch.

Endpoints

Method	Path	Description
GET	/api/health	Health check
POST	/api/command	Send a JSON command to Entry.HandleCommandAsync
GET	/api/web/status	Reports web dist path and whether UI is available
GET	/api/sidecar/root	Returns current project root (auto-detected or custom)
POST	/api/sidecar/root	Disabled in WebGateway; root is startup-only
GET	/api/runtime/diagnostics	Returns resolved root, vault path, DB path, and gateway path
GET	/*	Serves built SvelteKit UI from wwwroot (SPA fallback)

Web UI Resolution

On startup, Journal.WebGateway resolves the web dist in this order:

1. JOURNAL_WEB_DIST environment variable
2. <AppContext.BaseDirectory>/wwwroot (embedded in published output)
3. Journal.App/build (dev fallback — relative to repo root)

If no dist is found, / returns a JSON status message instead of the UI.

Running WebGateway

dotnet run --project Journal.WebGateway

For published output, configure the root before launch:

$env:JOURNAL_PROJECT_ROOT = "F:\path\to\journal-root"
.\Journal.WebGateway.exe

---

Journal.App (Tauri + SvelteKit)

A Tauri 2 desktop application with a SvelteKit 5 / TypeScript frontend.

Tech Stack

• Frontend: SvelteKit 5, TypeScript, Vite 6
• Tauri shell: Rust (Tauri 2), tokio for async process I/O
• Backend bridge: Journal.Sidecar.exe managed as a long-lived child process

Tauri Sidecar Architecture

The Rust layer (src-tauri/src/lib.rs) manages a persistent Journal.Sidecar.exe child process:

• Sidecar is auto-started on first command and restarted if it dies
• Commands are sent as JSON lines to stdin, responses read from stdout
• JOURNAL_PROJECT_ROOT is set to the resolved repo root before spawning
• On Windows, the process is created with CREATE_NO_WINDOW

Tauri commands exposed to the frontend:

Command	Description
sidecar_command	Forward a CommandEnvelope to Journal.Sidecar and return parsed JSON
get_sidecar_root	Get the current resolved sidecar root path
set_sidecar_root	Override sidecar root path (saves to settings.json, restarts sidecar)
get_runtime_diagnostics	Inspect resolved root, vault/database paths, sidecar path, and gateway path/url
get_ui_settings	Load tag/fragment-type settings from settings.json
set_ui_settings	Persist tag/fragment-type settings
shutdown	Stop the sidecar and exit the app

Sidecar path resolution order:

1. Exact sidecar binary path if the configured root is already the executable
2. <root>/Journal.Sidecar(.exe)
3. Recursive scan of <root>/Journal.Sidecar/
4. Tauri bundled resource path: <resourceDir>/bin/Journal.Sidecar(.exe)

Frontend State

The frontend uses Svelte stores as the source of truth:

Store	State	Purpose
entries.ts	entriesStore	Journal entry list and drafts
fragments.ts	fragmentsStore	Fragment CRUD + parse/serialize helpers
todos.ts	todoListsStore, todosStore	Todo lists and items
lists.ts	listsStore	Generic lists
settings.ts	settingsTags, settingsFragmentTypes	Tag/type configuration

Store-First Rule: components call store helpers for CRUD; they do not embed mutation or parsing logic directly.

Dev Setup

npm install              # from repo root (workspaces)
npm run dev -w Journal.App       # SvelteKit dev server at http://localhost:1420
npm run tauri dev -w Journal.App # Tauri dev mode (opens desktop window)

Publishing

# Web bundle only (for WebGateway)
npm run build -w Journal.App
# Output: Journal.App/build/

# Tauri raw exe (no installer)
npm run tauri build -w Journal.App -- -- --bundles none
# Output: Journal.App/src-tauri/target/release/journalapp.exe

# Tauri with NSIS installer
npm run tauri build -w Journal.App -- -- --bundles nsis

---

Sidecar Protocol

Journal.Sidecar communicates over stdin/stdout using newline-delimited JSON. One JSON object in, one JSON object out.

Command Format

{
  "action": "fragments.create",
  "correlationId": null,
  "id": null,
  "type": null,
  "tag": null,
  "payload": { "type": "!TRIGGER", "description": "stomach drop" }
}

Fields:

• action — Operation to perform (e.g. fragments.list, vault.load_all)
• correlationId — Optional tracing ID (auto-generated if omitted)
• id — Target entity ID (for get/update/delete)
• type / tag — Filter parameters (for fragment search)
• payload — Request body, deserialized per action

Response Format

Success:

{ "ok": true, "data": { "id": "abc-123", "type": "!TRIGGER", "description": "...", "time": "...", "tags": [] } }

Error:

{ "ok": false, "error": "Description is required" }

Available Actions

Action	Description	Key Requirements
fragments.list	List all fragments	—
fragments.get	Get by ID	id
fragments.create	Create fragment	payload (CreateFragmentDto)
fragments.update	Update fragment	id, payload (UpdateFragmentDto)
fragments.delete	Delete fragment	id
fragments.search	Filter by type/tag	type and/or tag
lists.list	List all lists	—
lists.get	Get list by ID	id
lists.create	Create list	payload
lists.update	Update list	id, payload
lists.delete	Delete list	id
todos.list	List all todo lists	—
todos.get	Get todo list by ID	id
todos.create	Create todo list	payload
todos.update	Update todo list	id, payload
todos.delete	Delete todo list	id
todos.items.create	Add todo item	payload
todos.items.update	Update todo item	id, payload
todos.items.delete	Delete todo item	id
entries.list	List persisted entries from SQLCipher store	—
entries.load	Load one entry file	payload.filePath
entries.save	Save/merge entry content	payload.content, optional payload.filePath, payload.mode, payload.fileName
entries.delete	Delete an entry file	payload.filePath
templates.list	List templates from SQLCipher store	—
templates.load	Load a template	payload.filePath
templates.save	Save/create a template	payload.name
templates.delete	Delete a template	payload.filePath
search.entries	Search entries with filters	optional query/section/date/tags/types/checked/unchecked
vault.initialize	Ensure vault directory exists	payload.password, payload.vaultDirectory
vault.load_all	Restore encrypted SQLCipher DB snapshot from vault	payload.password, payload.vaultDirectory
vault.rebuild_all	Persist encrypted SQLCipher DB snapshot to vault	payload.password, payload.vaultDirectory
vault.clear_data_directory	No-op for SQLCipher-first mode (compat command)	—
db.status	DB key/schema compatibility snapshot	payload.password
db.initialize_schema	Initialize SQLCipher schema in the database file	payload.password
db.hydrate_workspace	Bootstrap DB + set session password	payload.password
config.get	Return current config snapshot	—
ai.health	AI provider health status	—
ai.summarize_entry	Summarize one entry	payload.content, optional payload.fileStem
ai.summarize_all	Summarize multiple entries	payload.entries[]
ai.chat	Chat via AI provider bridge	payload.prompt
ai.embed	Generate embedding vector	payload.content
speech.devices.list	List audio input devices	—
speech.transcribe	Transcribe audio (base64) or text	payload.audioBase64 or payload.text

Sidecar CLI Mode

In addition to stdin/stdout protocol, Journal.Sidecar supports direct CLI subcommands:

# Load/decrypt vault snapshot into SQLCipher DB workspace
dotnet run --project Journal.Sidecar -- vault load

# Save (rebuild) vault snapshot from SQLCipher DB
dotnet run --project Journal.Sidecar -- vault save

# Search entries (query + filters)
dotnet run --project Journal.Sidecar -- search "common text" --tag stress --type !TRIGGER --start-date 2026-02-01 --end-date 2026-02-28 --section Summary --checked "med taken"

Password behavior:

• Omit --password → prompts securely in terminal
• Pass --password <value> → non-interactive/automation mode

Optional path overrides:

• --vault-dir <path>
• Env fallback: JOURNAL_VAULT_DIR, JOURNAL_DATABASE_DIR, JOURNAL_APP_DIR

Search CLI flags:

• positional query (optional)
• --tag / -t (repeatable)
• --type / -y (repeatable)
• --start-date / -s (yyyy-MM-dd)
• --end-date / -e (yyyy-MM-dd)
• --section / -sec
• --checked / -chk (repeatable)
• --unchecked / -uchk (repeatable)

---

Publishing

Sidecar (self-contained executable)

dotnet publish Journal.Sidecar/Journal.Sidecar.csproj -c Release -r win-x64 --self-contained -p:PublishSingleFile=true -p:IncludeNativeLibrariesForSelfExtract=true
# Output: Journal.Sidecar/bin/Release/net10.0/win-x64/publish/Journal.Sidecar.exe

To exclude debug symbols: add -p:DebugType=none

For a smaller build that requires .NET 10 on the target machine:

dotnet publish Journal.Sidecar/Journal.Sidecar.csproj -c Release -r win-x64 -p:PublishSingleFile=true

WebGateway (with embedded web UI)

# Step 1: build web assets
npm run build -w Journal.App

# Step 2: publish gateway
dotnet publish Journal.WebGateway/Journal.WebGateway.csproj -c Release -r win-x64

---

DI Registration

ServiceCollectionExtensions.AddFragmentServices() wires everything. Any host calls:

services.AddFragmentServices();
services.AddSingleton<Entry>();

Key registrations:

• IDatabaseSessionService → DatabaseSessionService (singleton)
• IFragmentRepository → SqliteFragmentRepository (singleton, SQLCipher-backed)
• IFragmentService → FragmentService (singleton)
• IEntryFileRepository → SqliteEntryFileRepository (singleton, SQLCipher-backed)
• IEntryFileService → EntryFileService (singleton)
• IListRepository → SqliteListRepository (singleton)
• IListService → ListService (singleton)
• ITodoRepository → SqliteTodoRepository (singleton)
• ITodoService → TodoService (singleton)
• IVaultCryptoService → VaultCryptoService (singleton)
• IVaultStorageService → VaultStorageService (singleton)
• IJournalDatabaseService → JournalDatabaseService (singleton)
• IAiService → LlamaSharpAiService or DisabledAiService (per JOURNAL_AI_PROVIDER)
• ISpeechBridgeService → DisabledSpeechBridgeService
• IJournalConfigService → JournalConfigService (singleton)
• CommandLogger (singleton)
• SidecarCli (singleton)

---

Extending with New Modules

The Command/Entry pattern uses dot-notation actions. To add a module:

1. Create model, DTO, repository, and service in Journal.Core/Services/<Domain>/
2. Register services in ServiceCollectionExtensions.cs
3. Inject the service into Entry.cs and add cases to the switch
4. No changes needed to App.cs, Journal.WebGateway/Program.cs, or the Tauri Rust shell

---

SDT DevTool

Journal.DevTool/ contains the pre-built SDT (Stan's Dev Tools) orchestrator (sdt.exe) and its Python build/automation scripts. Workflows are defined in devtool.json at the repo root. See Journal.DevTool/README.md for full documentation.

Workflows (devtool.json)

ID	Label	Group	Description
build-dotnet	Build .NET Projects	Build	dotnet build — all C# projects in solution
sidecar	Publish Sidecar	Build	Build Journal.Sidecar as self-contained exe → output/
web	Build Web UI	Build	Build SvelteKit bundle → Journal.App/build/
webgateway	Publish WebGateway	Build	Publish ASP.NET host with embedded web UI (depends on web)
tauri	Build Tauri Desktop App	Build	Build desktop exe, no installer (depends on sidecar)
tauri-nsis	Build Tauri + NSIS Installer	Build	Build desktop exe with NSIS installer (depends on sidecar)
all	Full Release Build ✦	Build	Sidecar → Web → WebGateway → Tauri, in dependency order
sync-output	Sync Build Assets to Output	Build	Sweep repo for newest builds and copy to output/
stage-output	Stage Output Bundle	Build	Full publish + stage journalapp.exe into output/
run-gateway-dev	Run WebGateway Server (Dev)	Dev	Start HTTP gateway via dotnet run at http://localhost:5180
run-gateway-prod	Run WebGateway Server (Output)	Dev	Start compiled gateway from output/webgateway
test	Run Smoke Tests	Test	Run all ~80 integration tests in Journal.SmokeTests
gate	Run Migration Gate	Test	Full build + smoke tests + parity check
nuget-export	Export NuGet Cache	Cache	Prime and export .nuget cache to zip for offline use
nuget-import	Import NuGet Cache	Cache	Import cache zip and validate restore
npm-clean	Clean Node Modules	System	Remove Journal.App node_modules

Environment Profiles

SDT supports dev, ci, and release profiles (configured in devtool.json under envProfiles). Select the active profile via SDT_ENV_PROFILE or from the TUI.

Key Scripts (Journal.DevTool/scripts/)

Script	Purpose
build.py	Orchestrated project builds
publish-sidecar.py	Publish Journal.Sidecar single-file exe
publish-app.py	Build web bundle or Tauri desktop app
publish-webgateway.py	Publish Journal.WebGateway with web assets
publish-output.py	Stage full output bundle
run-webgateway.py	Run Journal.WebGateway with controlled env
migration-gate.py	End-to-end build + smoke + parity check gate
pip-min.py	pip wrapper with repo-local cache
dotnet-min.py	dotnet wrapper with resilient NuGet defaults

---

Notes

• Journal content and templates persist in SQLCipher (entry_documents) under the vault DB directory.
• The legacy placeholder file _init_vault.vault is treated as obsolete — the C# backend ignores and removes it during vault load.
• On Windows + Tauri, the sidecar process is spawned with CREATE_NO_WINDOW to suppress the console window.