Skip to content

feat(core): Add GoogleCloudStorageOutputStream to the core module and integrate with GcsWriteChannel#291

Open
dheerajsngh wants to merge 41 commits into
GoogleCloudPlatform:mainfrom
dheerajsngh:analytics-write-3
Open

feat(core): Add GoogleCloudStorageOutputStream to the core module and integrate with GcsWriteChannel#291
dheerajsngh wants to merge 41 commits into
GoogleCloudPlatform:mainfrom
dheerajsngh:analytics-write-3

Conversation

@dheerajsngh

@dheerajsngh dheerajsngh commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Type of Change

  • feat: A new feature
  • fix: A bug fix
  • docs: Documentation only changes
  • style: Changes that do not affect the meaning of the code (white-space, formatting, missing semi-colons, etc)
  • refactor: A code change that neither fixes a bug nor adds a feature
  • perf: A code change that improves performance
  • test: Adding missing tests or correcting existing tests
  • chore: Changes to the build process or auxiliary tools and libraries such as documentation generation

Description

What?

  • Added GoogleCloudStorageOutputStream to enable writing to GCS via GcsWriteChannel.
  • Added unit tests covering buffering, stream closure, exception handling, and GCS writes.

Why?

  • Provides a standard Java OutputStream write path that complements the library's existing optimized read path (GoogleCloudStorageInputStream).

Checklist

  • PR title follows Conventional Commits (e.g., feat(core): Add GoogleCloudStorageOutputStream with tests)
  • All files include the Apache License 2.0 header
  • Documentation has been updated to reflect changes

Generated/Assisted by Agent? [Yes]

@dheerajsngh dheerajsngh requested a review from a team as a code owner June 8, 2026 05:59
@codecov

codecov Bot commented Jun 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.87%. Comparing base (e45feaa) to head (0d35daa).

Additional details and impacted files
@@             Coverage Diff              @@
##               main     #291      +/-   ##
============================================
+ Coverage     97.69%   97.87%   +0.18%     
- Complexity      486      502      +16     
============================================
  Files            35       36       +1     
  Lines          1561     1601      +40     
  Branches        140      144       +4     
============================================
+ Hits           1525     1567      +42     
+ Misses           17       16       -1     
+ Partials         19       18       -1     
Files with missing lines Coverage Δ
...yticscore/core/GoogleCloudStorageOutputStream.java 100.00% <100.00%> (ø)

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a new GoogleCloudStorageOutputStream to the core module, enabling a standard Java OutputStream interface for writing data to Google Cloud Storage. It integrates with the existing GcsWriteChannel and provides robust exception handling through a new utility class, ensuring consistent error reporting across the library. These changes facilitate easier integration with data processing frameworks that rely on standard Java I/O streams.

Highlights

  • New OutputStream Implementation: Added GoogleCloudStorageOutputStream to provide a standard Java OutputStream write path for GCS.
  • Write Channel Management: Implemented GcsWriteChannel to manage GCS write sessions, handle byte writing, and finalize uploads.
  • Configuration Options: Introduced GcsWriteOptions to support configurable upload strategies like parallel composite uploads and disk buffering.
  • Exception Handling: Added GcsExceptionUtil to centralize and standardize the translation of GCS storage exceptions into standard Java IO exceptions.
  • Interface Updates: Updated GcsClient and GcsFileSystem interfaces to include the create method for object writing.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces unified writing capabilities to Google Cloud Storage (GCS) by adding GcsWriteOptions, GcsWriteChannel, and GoogleCloudStorageOutputStream to support various upload strategies (such as parallel composite upload and journaling) and proper exception translation. The review feedback highlights two critical improvements: first, ensuring that GoogleCloudStorageOutputStream.write loops until the buffer is fully written to prevent data loss from partial writes; second, delaying setting the closed flag in GcsWriteChannel.close() until after successful execution to allow safe retries upon transient failures or interruptions.

# Conflicts:
#	client/src/main/java/com/google/cloud/gcs/analyticscore/client/GcsClientImpl.java
#	client/src/test/java/com/google/cloud/gcs/analyticscore/client/GcsClientImplTest.java
@dheerajsngh dheerajsngh force-pushed the analytics-write-3 branch 2 times, most recently from 8e0c87b to 83bc9c1 Compare July 6, 2026 10:39
# Conflicts:
#	client/src/main/java/com/google/cloud/gcs/analyticscore/client/ConfigurationUtil.java
#	client/src/main/java/com/google/cloud/gcs/analyticscore/client/GcsClientImpl.java
#	client/src/main/java/com/google/cloud/gcs/analyticscore/client/GcsClientOptions.java
#	client/src/main/java/com/google/cloud/gcs/analyticscore/client/GcsExceptionUtil.java
#	client/src/main/java/com/google/cloud/gcs/analyticscore/client/GcsWriteOptions.java
#	core/src/integrationTest/java/com/google/cloud/gcs/analyticscore/core/TestInputStreamInputFile.java
@dheerajsngh

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces GoogleCloudStorageOutputStream, a unified output stream that wraps a WritableByteChannel to provide standard java.io.OutputStream semantics for writing objects to Google Cloud Storage, along with its corresponding unit tests. The review feedback suggests several improvements: replacing the channel.isOpen() check in close() with a direct call to channel.close() to ensure proper upload finalization, and using Preconditions.checkNotNull instead of checkState for validating method arguments. Additionally, the unit tests should be updated to align with these changes by verifying idempotent close behavior and asserting NullPointerException instead of IllegalStateException.

@dheerajsngh dheerajsngh requested a review from animesh-g July 10, 2026 07:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants