Project_Journal-Csharp_backend-Python-Frontend/journal-master/journal/REFACTORING_SUMMARY.md

Backend Refactoring Summary

Problem

Entry.cs was a 550-line god class that contained command dispatching, business logic, HTML sanitization, logging infrastructure, and 13 private payload record types. The two Python sidecar services (PythonSidecarAiService and PythonSidecarSpeechService) duplicated ~80 lines of identical process/JSON plumbing. Payload DTOs were hidden as private records inside Entry.cs instead of being in the Dtos/ folder.

What Changed

1. Slimmed Entry.cs to a thin dispatcher (~330 lines)

Removed all business logic, HTML processing, logging implementation, and private record types. Entry now only parses the incoming JSON command, routes to the correct service, and returns the {ok, data} / {ok: false, error} envelope.

2. Extracted HtmlSanitizer (new file)

StripRichHtml and LooksLikeRichHtml moved from Entry.cs to Services/HtmlSanitizer.cs as a static utility class.

3. Extracted CommandLogger (new file)

LogStart, LogSuccess, LogFailure, EmitLog, ShouldLog, and LogLevelRank moved from Entry.cs to Services/CommandLogger.cs. Entry now receives this as a dependency.

4. Extracted IEntryFileService + EntryFileService (new files)

SaveEntry, LoadEntry, ListEntries, and ResolveTargetPath moved out of Entry.cs into a proper service with an interface. This follows the same pattern as IFragmentService / FragmentService.

5. Added IEntryFileRepository + DiskEntryFileRepository (new files)

EntryFileService now delegates all filesystem I/O (read, write, append, list, exists) to an IEntryFileRepository, keeping only business logic (HTML sanitization, parsing, merging) in the service. This mirrors the Fragment module's repository pattern (IFragmentRepository → FragmentService). An in-memory implementation can be swapped in for testing.

6. Extracted PythonSidecarClient (new file)

The duplicated SendAsync, LastJsonLine, and TryKill methods were extracted from both PythonSidecarAiService and PythonSidecarSpeechService into a shared Services/PythonSidecarClient.cs. Both services now delegate to it.

7. Moved payload records to Dtos/CommandDtos.cs (new file)

The 16 private payload/result records that were inside Entry.cs are now in Dtos/CommandDtos.cs. Records used through public interfaces (EntrySavePayload, EntryListItem, EntryLoadResult, EntrySaveResult) are public; the rest remain internal.

8. Moved database result records to Dtos/DatabaseDtos.cs (new file)

JournalDatabaseStatus and JournalDatabaseHydrationResult moved from IJournalDatabaseService.cs to Dtos/DatabaseDtos.cs for consistency with the other DTO files.

9. Moved fragment storage to encrypted SQLCipher database

Standalone fragments were previously stored in an unencrypted JSON file (fragments.json), then briefly in an unencrypted SQLite database. For medical/sensitive use cases, fragments are now stored in the existing encrypted SQLCipher database (journal_cache.db) alongside entries, sections, and tags.

• Updated fragments table schema: entry_id is now nullable, added guid TEXT UNIQUE column. Standalone fragments use guid + entry_id IS NULL; entry-linked fragments use entry_id + guid NULL.
• SqliteFragmentRepository now depends on IDatabaseSessionService instead of managing its own database connection.
• Tags use the shared normalized tags + fragment_tags tables (join via integer IDs).
• Fragment CRUD requires the database to be unlocked first (via vault.load_all or db.hydrate_workspace).

10. Added IDatabaseSessionService + DatabaseSessionService (new files)

A singleton that stores the encryption password in memory after authentication. When vault.load_all or db.hydrate_workspace is called, the session stores the password and lazily opens/caches an encrypted SQLCipher connection. All encrypted database consumers (currently SqliteFragmentRepository) use this shared session.

11. Organized Services directory into domain modules

The flat Services/ directory (28 files) was reorganized into 9 subdirectories with dedicated namespaces:

• Services/Ai/ — IAiService, DisabledAiService, PythonSidecarAiService
• Services/Config/ — IJournalConfigService, JournalConfigService
• Services/Database/ — IJournalDatabaseService, JournalDatabaseService, IDatabaseSessionService, DatabaseSessionService
• Services/Entries/ — IEntryFileService, EntryFileService, IEntrySearchService, EntrySearchService, JournalParser, HtmlSanitizer
• Services/Fragments/ — IFragmentService, FragmentService
• Services/Logging/ — CommandLogger, LogRedactor
• Services/Sidecar/ — PythonSidecarClient, SidecarCli
• Services/Speech/ — ISpeechBridgeService, DisabledSpeechBridgeService, PythonSidecarSpeechService
• Services/Vault/ — IVaultCryptoService, VaultCryptoService, IVaultStorageService, VaultStorageService

Each subdirectory has its own namespace (e.g. Journal.Core.Services.Ai). All consumer files updated with explicit using statements.

Files Created

• Journal.Core/Services/Entries/HtmlSanitizer.cs
• Journal.Core/Services/Logging/CommandLogger.cs
• Journal.Core/Services/Entries/IEntryFileService.cs
• Journal.Core/Services/Entries/EntryFileService.cs
• Journal.Core/Services/Sidecar/PythonSidecarClient.cs
• Journal.Core/Repositories/IEntryFileRepository.cs
• Journal.Core/Repositories/DiskEntryFileRepository.cs
• Journal.Core/Repositories/SqliteFragmentRepository.cs
• Journal.Core/Dtos/CommandDtos.cs
• Journal.Core/Dtos/DatabaseDtos.cs
• Journal.Core/Services/Database/IDatabaseSessionService.cs
• Journal.Core/Services/Database/DatabaseSessionService.cs

Files Modified

• Journal.Core/Entry.cs — slimmed to thin dispatcher, wired IDatabaseSessionService
• Journal.Core/Services/Ai/PythonSidecarAiService.cs — delegates to PythonSidecarClient
• Journal.Core/Services/Speech/PythonSidecarSpeechService.cs — delegates to PythonSidecarClient
• Journal.Core/Services/Database/IJournalDatabaseService.cs — result records moved to Dtos
• Journal.Core/Services/Database/JournalDatabaseService.cs — schema updated (nullable entry_id, guid column)
• Journal.Core/ServiceCollectionExtensions.cs — registers all new services, updated namespaces
• Journal.SmokeTests/Program.cs — updated with new dependencies and encrypted DB persistence test
• Journal.Sidecar/App.cs — updated namespace imports

Verification

• All 4 projects build successfully
• 65/70 smoke tests pass (5 Python sidecar tests fail only when Python is not installed on the machine, which is pre-existing)