Skip to content

proposal: PROM-116: Horizontally scalable Prometheus in Prometheus Organization and Governance - #116

Open
bwplotka wants to merge 9 commits into
mainfrom
bwplotka/scalable-lts-scope
Open

bwplotka wants to merge 9 commits into
mainfrom
bwplotka/scalable-lts-scope

Conversation

@bwplotka

@bwplotka bwplotka commented Nov 20, 2026 •

Copy link
Copy Markdown
Member

Potentially controversial one, but let's see. We always wanted to do this, but now timing might be great 💪🏽

Following up on the November 19, 2026 Prometheus.rs SIG discussion, this proposal brings horizontally scalable, distributed long-term storage requirements into official Prometheus governance for the next-generation Rust and columnar metrics effort.

Key areas for feedback from maintainers and the Steering Committee:

  • Whether the preferred placement should live directly in prometheus/prometheus.rs as composable monolith and distributed modes, or in a separate repository under Prometheus governance as outlined in Alternative 1.
  • Whether the proposed monolith-priority safeguards, including default single-node UX, one-way crate dependencies, CI benchmark budgets, and separate maintainership, are strong enough to protect single-node Prometheus users.

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
@bwplotka
bwplotka force-pushed the bwplotka/scalable-lts-scope branch from 106b612 to 622ec29 Compare November 20, 2026 05:42
@bwplotka bwplotka changed the title proposal: Bringing scalable LTS scope to Prometheus governance proposal: PROM-116: Bringing scalable LTS scope to Prometheus governance Nov 20, 2026
@bwplotka
bwplotka marked this pull request as ready for review November 20, 2026 05:47
Comment thread proposals/0116-scalable-lts-scope.md Resolved
Comment thread proposals/0116-scalable-lts-scope.md Resolved
Comment thread proposals/0116-scalable-lts-scope.md Resolved
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
Comment thread proposals/0116-scalable-lts-scope.md Outdated

## Alternatives

### 1. Separate repository and binary under Prometheus governance

@saswatamcode saswatamcode Nov 20, 2026 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thinking through this more I'd prefer a particular combined version of alternatives 1 and 2.
Whereby,

  • Prometheus does govern scalable LTS scope, but through shared library crates (promql-rs, scrape.rs, sth sth storage.rs/tsdb.rs and friends...think anything that can be treated as reusable library, that is maybe currently hidden away as a go package in prometheus or thanos)
  • prometheus/prometheus.rs composes those shared crates into a single node monolith binary
  • thanos-io/thanos.rs (or corthamir.rs) composes the same as distributed infra binaries. I'd propose shared governance across orgs here even, so all parties can collaborate

This has quite a few benefits,

  • From Prometheus end user pov, they still get the single binary they know and love, maybe even with prometheus-operator support, just way more efficient.
  • From Thanos/Cortex/Mimir end user pov, they can gradually swap out pieces of their infra with new columnar one, maybe querier first, then ingester, then stores, as we build and create the crates for those pieces and integrate into corthamir binary.
  • Since everything is maintained as shared libraries, operators who had already created their own implementations internally, can simply refactor and adopt libraries, and contribute, without having to commit to prometheus/corthamir binary. Particularly relevant for AI era. Essentially prometheus org expands governance scope to maintain building blocks for scalable LTS.

This way we also don't have to part with Prometheus/Thanos branding/seo. Wdyt?

@bwplotka bwplotka Nov 20, 2026 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice idea! I will add as another alternative, let's discuss!

I think we should aim higher and simplify life of everyone in CNCF. We should "move" Cortex and Thanos projects into Prometheus governance effectively. I am pretty sure this is what everyone wanted from day 1 (WDYT @caniszczyk , any advice?)

Moving under a single governance has multiple benefits:

  1. It simplifies the already complex CNCF landscape. I think it's important.
  2. It gives a middle ground to soften any previous misalignments (if there were any) between Cortex and Thanos.
  3. It increases chances Grafana will join one day (we want that IMO)
  4. We don't need to do (it's difficult!) work to setup/change governance in Thanos. I personally won't be able to this work if it's not part of Prometheus.

So I think it's important it's in Prometheus, if Prometheus SC and community allows it.

@bwplotka bwplotka Nov 20, 2026 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As per repo, I don't think it matters much, let's just choose one thing:

A) prometheus/prometheus.rs with multiple crates (and monolith binary)
B) prometheus/prometheus.rs with monolith binary, creates in separate small repositories
C) prometheus/prometheus.rs with monolith binary, prometheus/distro_prometheus.rs for other distributed creates

🤷🏽

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed in 174079e: added the shared-crates plus separate-org distributed binaries option as Alternative 2.

## Bringing scalable LTS scope to Prometheus governance

* **Owners:**
* Bartek Plotka [@bwplotka](https://github.com/bwplotka)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This list is based on the most active people on the SIG meeting yesterday. If you want to co-own, let me know!

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
Comment thread proposals/0116-scalable-lts-scope.md Resolved
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

@metalmatze metalmatze left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for writing this up so quickly after yesterday's SIG meeting, @bwplotka!

One thing I tripped over is that "LTS" is ambiguous. When I first read the title, I thought of Prometheus LTS as in Long Term Support, since we already ship LTS releases. I'm sure others will read it the same way.

Long-term storage also isn't what's new here. You can already run a single Prometheus with 10 years of retention today. What actually expands the scope is horizontal scalability, so I'd suggest we drop "LTS" and frame the proposal around horizontally scalable Prometheus instead.

I also don't see why the title is about governance. From my point of view, this proposal is about bringing a horizontally scalable Prometheus written in Rust to the Prometheus org. Of course the org is under Prometheus governance, but that's an implementation detail. Something like "proposal: PROM-116: Horizontally scalable Prometheus in Rust" would say that more directly.

Comment thread proposals/0116-scalable-lts-scope.md Resolved

@friedrichg friedrichg left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fully in support. Only one minor nit

Comment thread proposals/0116-scalable-lts-scope.md Resolved
Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
… scalable Prometheus in Rust

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
…onflict

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
…ernative

Signed-off-by: Bartek Plotka <bwplotka@gmail.com>
@bwplotka bwplotka changed the title proposal: PROM-116: Bringing scalable LTS scope to Prometheus governance proposal: PROM-116: Horizontally scalable Prometheus in Rust Nov 21, 2026
@bwplotka bwplotka changed the title proposal: PROM-116: Horizontally scalable Prometheus in Rust proposal: PROM-116: Horizontally scalable Prometheus in Prometheus Organization and Governance Nov 21, 2026
@bwplotka

Copy link
Copy Markdown
Member Author

Great comments, changed "LTS" to a new title, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants