Skip to content

Conversation

@GabrielCWT
Copy link
Contributor

This PR adds 2 new fields to the sys.servers table which is displayed on the web console. These will be stored on DiscoveryDruidNode

  1. Available Processors
  2. Total Memory (Displayed in binary bytes)

Both fields represent the available resource on the machine. If the service is hosted in a container, it will represent the resource available to the container instead. (Container Aware)

The reason for adding this feature is so that we are able to know the cost of running each of the servers. This is useful when deploying on Kubernetes as we have each server running on an isolated pod.

Interestingly, it seems that from Java 17, the OPERATING_SYSTEM_MX_BEAN is container aware which helps to simplify the implementation. It might be possible to simplify some legacy code which can be found in the cgroups folder i.e. Cpu.java and Memory.java. I have not looked into detail but I feel that it is something worth looking into.

https://developers.redhat.com/articles/2022/04/19/java-17-whats-new-openjdks-container-awareness#recent_changes_in_openjdk_s_container_awareness_code

@GabrielCWT GabrielCWT marked this pull request as ready for review October 9, 2025 09:11
@FrankChen021 FrankChen021 requested a review from Copilot October 13, 2025 02:01
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds CPU and memory information to the web console by exposing these metrics through the sys.servers table. The new fields help understand resource availability for each server, which is particularly useful for Kubernetes deployments where each server runs in an isolated pod.

Key changes:

  • Added available_processors and total_memory fields to the sys.servers table and web console
  • Implemented container-aware resource detection using Java 17's OperatingSystemMXBean
  • Updated web console to display CPU processors and total memory with proper formatting

Reviewed Changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
processing/src/main/java/org/apache/druid/utils/JvmUtils.java Added getTotalMemory() method using OperatingSystemMXBean for container-aware memory detection
server/src/main/java/org/apache/druid/discovery/DiscoveryDruidNode.java Added availableProcessors and totalMemory fields with proper serialization and backwards compatibility
sql/src/main/java/org/apache/druid/sql/calcite/schema/SystemSchema.java Extended sys.servers table schema to include new CPU and memory columns
web-console/src/views/services-view/services-view.tsx Added CPU processors and total memory columns to services view with aggregation support
web-console/src/utils/general.tsx Enhanced formatBytes function to support binary byte formatting
sql/src/test/java/org/apache/druid/sql/calcite/schema/SystemSchemaTest.java Updated tests to include new fields in expected results
server/src/test/java/org/apache/druid/discovery/DiscoveryDruidNodeTest.java Updated serialization tests to include new fields
docs/querying/sql-metadata-tables.md Added documentation for new available_processors and total_memory columns
website/.spelling Added new field names to spelling whitelist

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Copy link
Member

@FrankChen021 FrankChen021 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

generally looks good, left some minor comments

Copy link
Member

@FrankChen021 FrankChen021 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

public class DiscoveryDruidNode
{
private static final Logger LOG = new Logger(DiscoveryDruidNode.class);
private static final Integer UNKNOWN_VALUE = -1;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
private static final Integer UNKNOWN_VALUE = -1;
private static final int UNKNOWN_VALUE = -1;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated

@FrankChen021 FrankChen021 requested a review from kfaraz October 22, 2025 02:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants