[CELEBORN-2174] Remove partitionSplitEnabled from ReserveSlots and FileInfo

[SteNicholas]

What changes were proposed in this pull request?

Remove partitionSplitEnabled from ReserveSlots and FileInfo. Address comment #3492 (comment).

Why are the changes needed?

Since 0.6.0, Celeborn removed celeborn.client.shuffle.mapPartition.split.enabled to enable shuffle partition split at default for MapPartition. Therefore, partitionSplitEnabled of ReserveSlots and FileInfo is unnecessary because MapPartition enables partition split by default.

Does this PR resolve a correctness bug?

No

Does this PR introduce any user-facing change?

No.

How was this patch tested?

CI.

[xy2953396112]

LGTM~

[Copilot]

Pull request overview

This pull request removes the partitionSplitEnabled parameter from ReserveSlots messages and FileInfo classes as a cleanup effort. Since version 0.6.0, Celeborn enables shuffle partition split by default for MapPartition, making this parameter unnecessary. This change simplifies the codebase by removing configuration that is no longer needed.

Changes:

• Removed partitionSplitEnabled field from FileInfo, DiskFileInfo, and MemoryFileInfo classes
• Removed partitionSplitEnabled parameter from ReserveSlots protocol message and related serialization/deserialization code
• Removed partitionSplitEnabled from PartitionDataWriterContext class
• Updated all test files to use the modified constructors and method signatures

Reviewed changes

Copilot reviewed 25 out of 25 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
common/src/main/java/org/apache/celeborn/common/meta/FileInfo.java	Removed partitionSplitEnabled field, getter, and setter methods
common/src/main/java/org/apache/celeborn/common/meta/DiskFileInfo.java	Removed partitionSplitEnabled parameter from constructors; changed import from Arrays to Collections
common/src/main/java/org/apache/celeborn/common/meta/MemoryFileInfo.java	Removed partitionSplitEnabled parameter from constructors
common/src/main/scala/org/apache/celeborn/common/protocol/message/ControlMessages.scala	Removed partitionSplitEnabled from ReserveSlots message and its serialization/deserialization
common/src/main/scala/org/apache/celeborn/common/util/PbSerDeUtils.scala	Removed partitionSplitEnabled from FileInfo serialization/deserialization; refactored to use pattern matching
worker/src/main/java/org/apache/celeborn/service/deploy/worker/storage/PartitionDataWriterContext.java	Removed partitionSplitEnabled field, getter, constructor parameter, and toString representation
worker/src/main/java/org/apache/celeborn/service/deploy/worker/storage/MapPartitionDataReader.java	Simplified stream release logic to always send BufferStreamEnd
worker/src/main/java/org/apache/celeborn/service/deploy/worker/storage/PartitionFilesSorter.java	Removed partitionSplitEnabled from MemoryFileInfo constructor call
worker/src/main/scala/org/apache/celeborn/service/deploy/worker/Controller.scala	Removed partitionSplitEnabled parameter from reserveSlots methods
worker/src/main/scala/org/apache/celeborn/service/deploy/worker/PushDataHandler.scala	Removed conditional check for partitionSplitEnabled before HARD_SPLIT logic
worker/src/main/scala/org/apache/celeborn/service/deploy/worker/storage/StorageManager.scala	Removed partitionSplitEnabled from createPartitionDataWriter and file creation methods
worker/src/main/scala/org/apache/celeborn/service/deploy/worker/storage/StoragePolicy.scala	Removed partitionSplitEnabled from createMemoryFileInfo and createDiskFile calls
client/src/main/scala/org/apache/celeborn/client/LifecycleManager.scala	Removed partitionSplitEnabled = true from ReserveSlots message creation
common/src/test/scala/org/apache/celeborn/common/util/PbSerDeUtilsTest.scala	Updated test to not set partitionSplitEnabled in protocol buffer
worker/src/test/*	Updated all test files to use modified constructors without partitionSplitEnabled

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[SteNicholas]

Ping @RexXiong.