Session cost total silently loses a fixed chunk of spend the first time context compaction succeeds in a process's lifetime

[RyanNieburViewpoint]

Describe the bug

Environment: Copilot CLI v1.0.77, macOS, hostType: github

Scope: This has only been observed and verified in the Copilot CLI. I
have not tested or verified whether this affects other Copilot clients/IDEs
(VS Code extension, JetBrains, web, etc.) — the mechanism described below is
specific to the CLI's local session/event architecture (events.jsonl,
session.compaction_* events, local SQLite usage ledger), which may or may
not be shared with other clients.

Summary: In a long-running, tool-call-heavy session, the exit banner
reported 508 AI Credits. Summing every individual request logged for the
same session in the local usage ledger (assistant_usage_events in the
CLI's local SQLite store) totals 873.81 AI Credits across 158 requests —
a 365.46 AIC undercount. Critically, the local ledger's total is the
one that matches actual billed usage (confirmed against the increase in
account usage shown in the GitHub billing/web UI for this period) — the
exit banner and live in-session indicator are the ones that are wrong, not
the other way around. Continued investigation (resuming the session,
closing it again, and analyzing a second shutdown) pinpointed the exact
mechanism: a fixed, one-time chunk of already-accrued-but-unflushed cost
disappears at the moment of the first successful context-compaction in a
given CLI process's lifetime (whether that's the original session launch
or a later /resume). Before that first successful compaction, tracking is
invisible to us; after it, tracking is essentially perfect (drift-free to
within rounding) until the process ends or is resumed again, at which point
the cycle repeats.

Reproduction / evidence

1. The gap is a fixed dollar amount, not a percentage — confirmed across 8 readings:

Reading	Time	Shutdown/checkpoint reported	True ledger total	Gap (AIC)
checkpoint 1	19:43:31	350.55	716.01	365.46
checkpoint 2	19:45:24	402.08	767.55	365.47
checkpoint 3	19:49:34	482.45	847.91	365.46
checkpoint 4	19:52:36	499.81	865.27	365.46
checkpoint 5	19:53:31	505.10	870.56	365.46
shutdown 1	20:06:26	508.34	873.81	365.47
checkpoint 7 (post-resume)	20:21:00	593.97	1,033.07	439.10
shutdown 2 (post-resume, final)	20:25:58	636.07	1,075.16	439.09

The gap holds exactly flat at 365.46-365.47 AIC across 6 consecutive
readings spanning 22+ minutes and 331.78 AIC of brand-new, correctly-tracked
activity — then jumps once to a new fixed value (439.09-439.10) and holds
flat there too.

2. Both jumps land within 0.5 seconds of a compaction event:

• Jump 1 (+365.46 AIC): the cumulative ledger crosses this threshold at
  19:24:19.7Z — 0.5s before session.compaction_complete fires at
  19:24:20.2Z. This is the session's very first compaction attempt.
• Jump 2 (+73.63 AIC): the cumulative ledger crosses this threshold at
  20:15:45.5Z — 0.5s before the next successful session.compaction_start
  fires at 20:15:46.1Z.

3. The critical detail that makes the pattern click: one compaction attempt failed.
The session went through 4 compaction attempts total, but only 3 succeeded:

#	Start	Result	Context
1	19:22:38	✅ success	1st compaction of the original process launch — LOST 365.46 AIC
2	19:45:06	✅ success	2nd compaction of the same process launch — clean, no loss
3	20:15:04	❌ failed — "error": "Compaction failed: received empty response from model"	1st compaction attempt after /resume — never completed, no state change
4	20:15:46	✅ success	Retry, 30s later — effectively the 1st successful compaction after resume — LOST 73.63 AIC

Excluding the failed attempt, every loss event corresponds exactly to the
first successful compaction of a process's lifetime — the original launch
(session.start at 19:02:57) and the later /resume (confirmed via a
session.resume event at 20:12:44.539Z) each produced exactly one loss, at
their respective first successful compaction, and no loss on any subsequent
compaction in that same process's lifetime.

Root cause (strong hypothesis, needs engineering confirmation)

The in-memory cost/usage accumulator appears to be (re)initialized whenever
the CLI process starts or a session is resumed. It tracks new activity
correctly from that point — but the first time a compaction successfully
completes in that process's lifetime, whatever internal state-reconciliation
or checkpoint-numbering logic compaction performs mishandles the handoff,
silently discarding the cost accrued since process start that hadn't yet
been captured by a periodic flush. Because this reconciliation only
misfires once per process lifetime, later compactions in the same run are
unaffected — but each new process attach (via /resume, or possibly a crash
recovery) resets the clock and makes the next first-successful-compaction
vulnerable again.

Impact

Any session that is long/complex enough to trigger auto-compaction will
under-report real spend by whatever cost had silently accrued before that
first successful compaction — and if the session is resumed later, this can
happen a second time. For very long sessions with multiple resumes, this
loss could compound to a significant, unbounded amount over time. This is
not just a cosmetic display bug: users relying on the live in-session
indicator or the exit-banner total for budgeting will be materially misled
about how much they were actually charged, since the local ledger — not
the banner — is what matches real billed usage.

Suggested fix direction

Audit the compaction success-path for whatever step transitions/reconciles
the cost accumulator (likely tied to the checkpointNumber assignment logic,
since successful compactions carry a checkpointNumber and failed ones do
not). Ensure it merges with the accumulator's current value instead of
overwriting/reinitializing it. Also worth checking whether the periodic
session.usage_checkpoint mechanism only begins firing after the first
compaction in a process lifetime (it appears to start only around then in
this session), which would independently explain why the first compaction
window has no earlier flush to protect it.

Appendix: independently reproduced in a second, unrelated session

While preparing this report (itself running in a separate CLI session), that
session hit its own first auto-compaction. The same signature reproduced
exactly:

• All 12 periodic checkpoints before the first compaction matched the
  authoritative per-request ledger with zero gap.
• The instant session.compaction_complete fired for that session's first
  (and so far only) compaction, the ledger jumped by ~48.7 AIC in a single
  request-processing gap, and the next periodic checkpoint showed a ~30.9
  AIC gap versus the ledger that had not existed moments before.

This is a second, independent data point (different repo, different
conversation, no relationship to the session in the main report above)
showing the loss occurring at the exact same trigger — the first successful
compaction of a process's lifetime — reinforcing that this is a general,
reproducible mechanism rather than something specific to one unusual
session.

Appendix: secondary, distinct issue found during investigation

Separately from the cost-accounting bug above: one conversational turn in
this session has zero rows in the per-request usage ledger despite being a
real turn with a real assistant response (the ledger's turn-index sequence
jumps directly over it). This is data missing at the recording layer itself,
not merely excluded from a summary — a different failure mode from the
cost-accumulator bug above, and possibly worth a separate, smaller bug report
if useful.

Affected version

GitHub Copilot CLI 1.0.77.

Steps to reproduce the behavior

1. Start a Copilot CLI session and work in it long enough (many tool calls /
   agentic turns) to trigger the CLI's automatic context compaction (occurs
   around the ~200k token threshold — watch for a /compact-style event or
   just a long, tool-heavy session).
2. Before that first compaction, periodically note the AI Credits total
   shown live in the CLI (or check session.usage_checkpoint events in that
   session's ~/.copilot/session-state/<session-id>/events.jsonl) — this
   matches actual usage exactly at this stage.
3. Let the first auto-compaction complete successfully (a
   session.compaction_complete event with "success": true in the same
   events.jsonl).
4. Compare the live/reported AI Credits total immediately after that
   compaction against the sum of total_nano_aiu for that session in the
   local usage ledger (~/.copilot/session-store.db, table
   assistant_usage_events, SELECT SUM(total_nano_aiu) FROM assistant_usage_events WHERE session_id = '<id>', divided by 1e9 to get
   AI Credits). A gap appears at this exact moment and does not go away.
5. Continue working and close the session; compare the final exit-banner
   total against the same ledger SUM() query. The exit banner is now a
   fixed number of credits lower than the actual/billed total, matching the
   gap that first appeared at step 3-4.
6. (Optional, to fully confirm the mechanism) If the session is later
   resumed and undergoes another successful compaction, watch whether
   another fixed-size gap appears at that point too — this happened in both
   sessions I tested it in (once per session per process attach: original
   launch and again after resume).

I don't have a single CLI command that reproduces this in isolation (it
depends on the CLI's internal auto-compaction trigger) — this was found and
confirmed via the two data sources above (live/exit-banner totals vs. the
local SQLite ledger), cross-referenced against each session's own
events.jsonl compaction event timestamps.

Expected behavior

The AI Credits total shown live during a session, and the final total shown
in the exit banner, should always match the actual usage that gets billed
to the account (i.e., match the sum of the per-request local usage ledger).
Successfully completing a context-compaction should never cause any
already-accrued cost to be silently dropped from the running/reported total
— compaction should only affect the conversation context sent to the model,
not the accounting of prior usage.

Additional context

• The local per-request usage ledger (assistant_usage_events in
  ~/.copilot/session-store.db) appears to be the authoritative source —
  its totals match the actual increase in account usage shown in the
  billing/usage UI. The live in-session indicator and exit-banner total are
  the values that are wrong.
• The loss is a fixed, one-time amount, not a percentage or an ongoing
  drift — it appears exactly once per successful compaction that is the
  first successful compaction of a given CLI process's lifetime (whether
  that's the original session launch, or after a /resume). Later
  compactions within that same process lifetime do not cause further loss.
• I verified this pattern independently in two separate sessions (different
  repositories, unrelated conversations), with the same signature both
  times: zero gap before the first successful compaction, then a fixed gap
  appearing right at that compaction's timestamp and persisting afterward.
• This has only been observed/tested in the Copilot CLI — I have not
  verified whether it affects other Copilot clients (VS Code extension,
  JetBrains, web, etc.).
• Separately, I also noticed the local usage ledger itself is occasionally
  missing an entire conversational turn's usage rows (a turn with a real
  assistant reply but zero corresponding ledger entries) — this looks like a
  distinct, second data-loss mechanism, unrelated to the compaction-timing
  issue described above, and may be worth investigating separately.
• Happy to share the exact session ID and further supporting detail via the
  private feedback channel if useful for your team to pull internal logs.