# Chunk Engine Progress

## Purpose

This log tracks milestone-relevant chunk-engine progress, blockers, benchmark runs, and real-app comparison data. It is not meant to capture every casual test run.

## Current headline status

- Dense is still the production/reference backend.
- The chunk backend is integrated into the main app and benchmark harness as the main experimental lane.
- Chunk now has page-local storage, active-page scheduling, dirty-chunk rendering, motion-regression coverage, and app-sized benchmark modes.
- High simultaneous move volume is now the primary chunk optimization blocker.
- Further chunk parity work is gated behind active-move instrumentation and optimization.
- Post-instrumentation baselines show chunk still ahead on settled/localized workloads, but behind dense on sustained active-move scenes.
- Live app branch-level counters now show the stable long-press gas case is materially more controlled than before.
- The main remaining single-thread hotspot is now mixed-scene `full-runtime` work across solids, liquids, and gases.
- New live-app stress runs show gas-specific retry/runtime work is harder to spike, while mixed material scenes still pay too much broad runtime-chain cost.

## Open blockers

- Chunk still needs more branch-level metrics to fully explain attempt spikes across all material classes.
- Gas-heavy and continuous-paint scenes still need targeted high-move optimization.
- Mixed scenes still route too many runtime-heavy particles through broad lifecycle / thermal / reaction / pressure work in the same frame.
- Move-attempt cost and stalled-motion cost need clearer measurement and trend tracking.
- Stable gas and overload behavior are improved, but mixed-scene full-runtime cost still needs to come down before parity work resumes.
- Real-app chunk results still need to beat dense consistently, not only in selected scenes.
- Chunk parity remains paused behind the active-move optimization gate.

## Manual app scenarios

### `active_sand_flood`

- Setup: continuously pour a dense falling solid stream until the scene sustains `10k+` simultaneous moves.
- Measure: average FPS, minimum FPS, move attempts, successful moves, stalled movable cells.
- Acceptance: no wake/sleep artifact and move cost scales better than the recorded baseline.

### `mixed_pile`

- Setup: create a layered sand/water pile over a floor and let it settle for 30 seconds.
- Measure: average FPS, minimum FPS, stability of settling.
- Acceptance: no obvious shimmer or frozen partial-settle behavior.

### `active_gas_burst`

- Setup: inject a large gas burst in a confined chamber until gas dominates movement.
- Measure: average FPS, minimum FPS, move attempts, full-runtime step count.
- Acceptance: gas remains visually coherent and does not dominate frame time disproportionately.

### `continuous_mixed_paint`

- Setup: sustain mixed solid/liquid/gas painting for 30 seconds.
- Measure: FPS plus movement/runtime bucket timings.
- Acceptance: no severe collapse when move volume spikes.

### `tool_stress`

- Setup: apply wind/air/gravity tools continuously over a mixed-material scene for 30 seconds.
- Measure: FPS, visible stability, and chunk field activity.
- Acceptance: field-driven motion remains active without repainting.

## Metrics table

Required columns for recorded runs:

- date
- commit or branch label
- backend
- build
- scene name
- world size
- particle count
- avg FPS
- min FPS
- avg step ms
- avg frame/build ms
- loaded chunks
- active chunks
- stepped chunks
- dirty chunks
- field cells
- move attempts
- successful moves
- swap attempts
- stalled movable cells
- movement-only fast-path count
- full-runtime step count
- activation ms
- movement ms
- runtime ms
- field decay ms
- render ms
- notes

## Dated entries

### 2026-03-09

- Replaced the chunk prototype hot path with page-local storage/scheduling scaffolding and removed the old global sparse/LINQ step path.
- Added app-sized benchmark mode and snapshot mode to `Sand.Benchmarks`.
- Extended shared/app frame stats with chunk workload counters and surfaced them in the app debug overlay.
- Reordered the chunk program around an active-move optimization gate before further parity expansion.
- Added move-attempt, stalled-motion, fast-path/full-runtime, and coarse timing buckets to chunk step stats, app overlay, and benchmark output.
- Recorded the first post-instrumentation high-motion baseline below.
- Added row-count guided stepping, narrower border wakes, same-page occupancy fast paths, and limited gas-runtime throttling.
- Result: `continuous_mixed_paint` improved materially and `active_sand_flood` improved versus the prior local pass, but chunk is still behind dense on sustained flood and gas-burst scenes.
- Added branch-level move-attempt counters (`vertical/diagonal/lateral`) plus full-runtime counters by particle kind (`solid/liquid/gas`) to the chunk overlay.
- Live-app chunk gas stress with `ultratanium` now shows the runtime bottleneck clearly: a representative run reported `moves 2357`, `attempts 9161`, `Att v 5467 d 2221 l 1073`, `full 5644`, `s 0 l 0 g 5644`, `run 11.52 ms`, `render 0.52 ms`.
- Interpretation: render is not the primary limiter in the observed gas stress case; the next optimization pass should reduce general `full-runtime gas` exposure and repeated vertical gas probes without particle-specific throttles.
- Follow-up retry-reduction work made the worst spikes harder to trigger in the live app, but later `ultratanium` screenshots still showed large `full-runtime gas` counts once the gas mass interacted with terrain and boundaries.
- Interpretation update: the next generic gas optimization should focus on thermally stable gas near non-gas boundaries, not only open-cloud gas and not particle-id-specific behavior.

### 2026-03-10

- Added app-shell frame instrumentation and overload control so the main app now reports total frame, update, sim-loop, build, upload, draw, and fixed-step counts.
- Result: long-press gas abuse is materially harder to use to collapse the app, because the fixed-step loop no longer tries to chase every missed step forever under overload.
- Recent live-app screenshots now show a clearer split:
  - stable gas runs can remain responsive with `Full 0`, meaning the worst gas-runtime branch is no longer the default limiter
  - mixed scenes still fall back into expensive `full-runtime` work across solids, liquids, and gases
- Updated single-thread focus: stop treating gas-only stress as the only blocker and reduce the mixed-scene runtime chain before planning multithreading.
- Current code work is trimming the chunk full-runtime path so particles only pay for lifecycle, burn, emission, special, phase, reaction, pressure, and thermal domains when they actually use them.
- Latest mixed-scene live tests after runtime trimming showed a much healthier profile at roughly `44.7k` particles:
  - `FPS 23`
  - `moves 4936`, `attempts 5466`, `stalled 218`
  - `app frame 44.07 ms`, `sim 41.55 ms`, `steps 2`
  - `render 1.18 ms`, `draw 0.97 ms`
- Interpretation update: the app is no longer mostly failing from catch-up debt or render/upload cost in this stress case. The remaining hotspot is honest mixed-scene simulation/runtime work.

## Baseline measurements

| date | commit/label | backend | build | scene | world | particles | avg FPS | min FPS | avg step ms | avg frame/build ms | loaded chunks | active chunks | stepped chunks | dirty chunks | field cells | move attempts | successful moves | swap attempts | stalled movable cells | movement-only fast-path count | full-runtime step count | activation ms | movement ms | runtime ms | field decay ms | render ms | notes |
| --- | --- | --- | --- | --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | --- |
| 2026-03-09 | local-working-tree | dense | Release | active_sand_flood | 265x200 | 1920 | 433.46 | n/a | 2.265 | 0.042 | 0 | 0 | 0 | 0 | 0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.000 | 0.000 | 0.000 | 0.000 | 0.042 | dense baseline for sustained falling solids |
| 2026-03-09 | local-working-tree | chunk | Release | active_sand_flood | 265x200 | 2520 | 200.45 | n/a | 4.806 | 0.182 | 21 | 21 | 21 | 21 | 0 | 1330.0 | 1330.0 | 0.0 | 0.0 | 1330.0 | 0.0 | 0.016 | 4.722 | 0.000 | 0.001 | 0.182 | movement-only work dominates; chunk still behind dense here |
| 2026-03-09 | local-working-tree | dense | Release | active_gas_burst | 265x200 | 1566 | 1150.01 | n/a | 0.848 | 0.022 | 0 | 0 | 0 | 0 | 0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.000 | 0.000 | 0.000 | 0.000 | 0.022 | dense baseline for sustained gas injection |
| 2026-03-09 | local-working-tree | chunk | Release | active_gas_burst | 265x200 | 2130 | 290.73 | n/a | 3.406 | 0.034 | 16 | 16 | 10 | 10 | 0 | 1143.8 | 1147.3 | 0.0 | 0.0 | 142.8 | 1009.8 | 0.005 | 0.005 | 3.024 | 0.001 | 0.034 | full-runtime gas path is the dominant blocker |
| 2026-03-09 | local-working-tree | dense | Release | continuous_mixed_paint | 265x200 | 2216 | 697.89 | n/a | 1.411 | 0.022 | 0 | 0 | 0 | 0 | 0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.000 | 0.000 | 0.000 | 0.000 | 0.022 | dense baseline for sustained mixed painting |
| 2026-03-09 | local-working-tree | chunk | Release | continuous_mixed_paint | 265x200 | 2196 | 323.38 | n/a | 3.016 | 0.076 | 23 | 23 | 23 | 23 | 0 | 1320.5 | 991.1 | 0.4 | 0.1 | 386.8 | 388.9 | 0.008 | 0.001 | 2.233 | 0.001 | 0.076 | movement and runtime both contribute under mixed active paint |

## Next actions

- reduce move-heavy cost on `active_sand_flood`, `active_gas_burst`, and `continuous_mixed_paint`
- reduce mixed-scene `full-runtime` cost by narrowing the lifecycle / reaction / pressure / thermal chain per particle
- keep stable-gas wins intact while mixed-scene runtime work is reduced
- keep `ultratanium` as a stress particle, but avoid particle-id-specific throttles or hacks
- continue reducing chunk overhead on gas-heavy and continuous-paint workloads
- use the healthier mixed-scene overlay numbers as the single-thread baseline before any multithread planning
- return to same-page falling-solid movement overhead after the mixed-scene runtime chain is under better control
- add deterministic snapshot review for the named scenes after `120` and `300` steps
- keep further parity work paused until the active-move gate shows repeatable gains