Skip to content

fix(storage): gate async flush completion on sent offset instead of persisted_size - #16596

Merged
kalragauri merged 3 commits into
googleapis:mainfrom
kalragauri:fix/buffered-upload
Oct 29, 2026
Merged

kalragauri merged 3 commits into
googleapis:mainfrom
kalragauri:fix/buffered-upload

Conversation

@kalragauri

Copy link
Copy Markdown
Contributor

AsyncWriterConnectionImpl::Flush() sends its payload with flush: true, state_lookup: true, and the service answers with a single response carrying persisted_size. On regular (non-appendable) objects, the service only persists 256 KiB multiples before the upload is finalized, so an unaligned flush returns a persisted_size smaller than the bytes sent (0 for the 15,890-byte upload in the StartBufferedUpload sample). Because OnFlush kept reading until persisted_size >= offset_, its next Read() blocked until the stall timeout cancelled the stream, affecting both AsyncWriter::Flush() on unbuffered uploads and the flushes issued by AsyncWriterConnectionBufferedState and AsyncWriterConnectionResumedState. Those two classes also waited for target_offset <= persisted_size before completing Flush() promises and checked !resend_buffer_.empty() in NeedsFlush() and ClearHandlersIfEmpty(), leaving Flush() futures and high-water-mark waiters pending (and looping on empty flushes when idle) whenever an non-persisted tail remained in the buffer.

This PR does the following:

  • OnFlush now completes on the first response carrying persisted_size or resource while still recording that value in persisted_state_.
  • In the buffered and resumed writers, SetFlushed() drops its unused persisted_size parameter and completes pending Flush() promises up to the sent offset (buffer_offset_ + write_offset_), matching the AsyncWriter::Flush() contract that flushed data may not be persisted yet.
  • NeedsFlush() and ClearHandlersIfEmpty() now share HasUnsentDataAtOrAboveLwm() (write_offset_ < resend_buffer_.size() && resend_buffer_.size() >= buffer_size_lwm_), so once all buffered bytes have been sent the writer goes idle and releases high-water-mark waiters instead of re-flushing data the service already received.
  • OnQuery() still trims resend_buffer_ by persisted_size, so any unpersisted bytes are replayed on Resume() as before.

@product-auto-label product-auto-label Bot added the api: storage Issues related to the Cloud Storage API. label Oct 27, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the asynchronous writer connections (writer_connection_buffered and writer_connection_resumed) to correctly handle scenarios where the service's persisted size lags behind the sent bytes, such as on regular GCS buckets that only persist data in 256 KiB multiples. Specifically, Flush() now completes immediately upon receiving the first response carrying a persisted_size rather than waiting for it to match the expected offset, preventing potential hangs. Internal state checks like NeedsFlush and ClearHandlersIfEmpty have been updated to track unsent data using the write offset, and extensive unit and integration tests have been added to verify correct behavior with unaligned uploads. No review comments were provided, so there is no additional feedback to address.

@codecov

codecov Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.80318% with 29 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.37%. Comparing base (829914d) to head (93641c8).

Files with missing lines Patch % Lines
.../internal/async/writer_connection_buffered_test.cc 95.51% 14 Missing ⚠️
...rage/internal/async/writer_connection_impl_test.cc 87.71% 14 Missing ⚠️
...e/internal/async/writer_connection_resumed_test.cc 99.58% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #16596      +/-   ##
==========================================
- Coverage   92.39%   92.37%   -0.03%     
==========================================
  Files        2266     2266              
  Lines      220143   220810     +667     
==========================================
+ Hits       203409   203965     +556     
- Misses      16734    16845     +111     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@kalragauri
kalragauri marked this pull request as ready for review October 27, 2026 12:14
@kalragauri
kalragauri requested review from a team as code owners October 27, 2026 12:14
@kalragauri
kalragauri requested a review from v-pratap October 27, 2026 12:14

@v-pratap v-pratap left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix looks right to me. Completing Flush() at the sent offset matches the AsyncWriter::Flush() docs, and the unsent-data check stops the empty-flush loop. All three writer unit tests pass under ASan too.
One doc gap: the public BufferedUploadHwmOption comment still says flow control is released only at the LWM. That is no longer true when the unpersisted tail is at or above the LWM. Please update it (and the stale buffer_size_hwm_ member comments).

@kalragauri
kalragauri enabled auto-merge (squash) October 28, 2026 08:37
@kalragauri
kalragauri merged commit 608aeef into googleapis:main Oct 29, 2026
71 checks passed

This branch was successfully deployed

1 active deployment
false — 93641c84 Deployed Oct 29, 2026 by kalragauri via Save PR ref #12697
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: storage Issues related to the Cloud Storage API.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Sponsor
SponsoredKunjungi sekarang
Promo