perf: optimize ledger render - lazy baseline + tail file reads - #7
Open
adiled wants to merge 3 commits into
Open
perf: optimize ledger render - lazy baseline + tail file reads#7adiled wants to merge 3 commits into
adiled wants to merge 3 commits into
Conversation
Remove the two biggest performance bottlenecks: 1. Baseline caching — baseline was recomputed every ~1s tick on the entire ledger regardless of whether anything changed. Now it only re-computes when ledger_files_mtime() or newest_record_ts() changes, i.e. only when a new record is appended. 2. Ledger panel tail-read — ledger table no longer clones app.agg.records (which stores all filtered records). Instead it calls load_top_records() which uses safe backward-byte seeking to read only the last N lines from each ledger file, parsing only what is needed for display. New functions in ledger_read.rs: - load_top_records(n) — top-N newest records without full parse - read_last_n_lines(p, n) — safe backward-seek without line corruption - ledger_files_mtime() — max mtime across all ledger files - newest_record_ts() — timestamp of the most recent record App struct gains: - last_baseline_mtime: u64 - last_baseline_load_ts: Option<f64> Zero regression on existing paths — iter_records unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Jab log LLM-heavy workloads chalatay hain to ledger.jsonl bohat bara ho jata hai.
CCFT ki TUI har ~1 second:
app.agg.recordsmeinrecords.clone()+ sort karta haiIs se render stuttering hota hai.
Solution
1. Baseline caching — mtime + newest timestamp track karta hai, baseline sirf tab recompute hota hai jab ledger file actual me badle (nayi line aaye).
2. Ledger tail-read —
load_top_records(n)file ke last N lines ko backward-seek se padhta hai, bina poor file parse kiye. Ledger panel direct file se data leta hai, clone/remove sort.New functions in
ledger_read.rsload_top_records(n)— top-N newest records without full file parseread_last_n_lines(p, n)— safe backward-seekledger_files_mtime()— max mtime across ledger filesnewest_record_ts()— newest record timestampNew fields on
Applast_baseline_mtime: u64last_baseline_load_ts: Option<f64>Regression risk
iter_records()unchanged — pure addition. Existing behavior untouched.