Caching the raw response for /series endpoint #8502
Description
Is your proposal related to a problem?
We're having performance issues with Thanos Query Frontend: The thing is, in our organization, we're encountering quite large responses to the /series endpoint. We're talking about JSON files on the order of 2 million lines.
Under these circumstances, Thanos Query Frontend consumes excessive CPU and memory.
Profiling Thanos Query Frontend, we found this:
It seems the issue is the DecodeResponse method, specifically the Unmarshall of this line:
https://github.com/thanos-io/thanos/blob/main/pkg/queryfrontend/labels_codec.go#L236C13-L236C39
Describe the solution you'd like
We thought a possible solution would be to have the option to cache the raw response directly. This would avoid having to unmarshall the response, which seems to be penalizing performance.