eric
738ac26210
Root cause of the production hang reported after the previous fix: refresh_tail unconditionally replaced the chunker's entire `pending` buffer with the fresh read, discarding pending[..complete_len] -- content already confirmed real but not yet drained (anything under the 256KiB emit target). Since a fresh read always starts exactly at confirmed_len (past that already-confirmed prefix), the next rescan() started from complete_len=0 within a buffer that no longer contained the records needed to reprove it, silently regressing confirmed_len. In a small buffer this just caused wasteful oscillation (each poll's full remaining read happened to re-derive the same progress); at production scale, once a poll's window failed to independently re-establish the same high-water mark (e.g. capped by READ_WINDOW, or landing on an unlucky boundary), confirmed_len could regress and then get permanently wedged well behind the real write cursor, manifesting as a "torn/corrupt" bail with confirmed_len frozen at a value far below the real content size, despite the source visibly still growing. Fix: refresh_tail now truncates pending to complete_len (keeping the confirmed-but-undrained prefix intact) before appending the fresh bytes, instead of replacing pending wholesale. Added a regression test (refresh_tail_never_discards_already_confirmed_undrained_content) that fails against the old behavior and passes against the fix -- verified by temporarily reverting the fix and confirming the test catches it. Also add verbose (deliberately noisy for now) tail.rs diagnostics: a log line on every confirmed_len advance, and a throttled "stuck" line (including a byte preview of what's at the confirmed boundary and whether it changed since the last check) whenever there's more to read but nothing validates -- both before and after rename, where previously there was no progress visibility at all before rename was observed. This is what surfaced the bug: real production logs showed confirmed_len permanently frozen at a fixed byte count for 4+ minutes while the source kept growing, which is inconsistent with the "held back by one record" design and pointed straight at a state-management bug rather than a writer-side or filesystem-caching issue.