Architecture
Big picture
Thanos is one binary run as several roles. Some roles hold metric data and implement the gRPC StoreAPI (pkg/store/storepb/rpc.proto:26); one role, the Querier, has no data of its own and instead fans a query out to all of them and merges the results. The unifying abstraction is that every data source looks the same to the Querier, so sidecars, store gateways, receivers, rulers, and even other Queriers are interchangeable behind one interface.
Components
sidecar
Runs next to a Prometheus server. It uploads completed TSDB blocks to object storage through the shipper (pkg/shipper/shipper.go:344) and exposes Prometheus's recent in-memory data over the StoreAPI. Subcommand registered at cmd/thanos/main.go:56-63.
store gateway
Serves historical blocks that live in object storage back over the StoreAPI. The block reader is the BucketStore (pkg/store/bucket.go:384), which reads blocks from the external thanos-io/objstore library.
query (Querier)
The fan-out and merge engine. It implements PromQL and the StoreAPI but stores nothing itself; it discovers downstream StoreAPI endpoints and merges their series. The implementation is ProxyStore in pkg/store/proxy.go:84.
receive
The push path. It accepts Prometheus remote-write and stores series locally, for sources that cannot be scraped. Package pkg/receive/.
compact
Runs compaction and downsampling over blocks in object storage, keeping long-range queries fast. Packages pkg/compact/ and pkg/compactv2/.
rule, query-frontend
rule evaluates recording and alerting rules and is itself a StoreAPI source. query-frontend (pkg/queryfrontend/) splits and caches queries in front of the Querier.
How a request flows
A PromQL query reaches a Querier, whose StoreAPI entry point is ProxyStore.Series (pkg/store/proxy.go:277).
pkg/store/proxy.go:287matches the request's external label selectors against the configured selector labels; an unrelated request returns early.pkg/store/proxy.go:302-309extracts tenant info from gRPC metadata and re-attaches it to the outgoing context for multi-tenancy.pkg/store/proxy.go:312narrows the candidate stores by time range and matchers; zero matches returns early.pkg/store/proxy.go:319-333rebuilds the downstreamSeriesRequest, folding external labels into the matchers and carrying through shard info andWithoutReplicaLabels.pkg/store/proxy.go:357starts one asynchronous stream per store withnewAsyncRespSet; a failure either emits a warning or aborts, per the partial-response strategy.pkg/store/proxy.go:376loads all streams into a loser tree withNewProxyResponseLoserTree; if dedup is enabled,pkg/store/proxy.go:378wraps it in aResponseDeduplicator.pkg/store/proxy.go:382-397pulls series in global sorted order, honoringr.Limit, and sends each withsrv.Send;pkg/store/proxy.go:400flushes any remaining buffered series.
Key design decisions
- No deduplication in the fan-in. The
NewProxyStoredoc comment states dedup must happen at the highest level, just before PromQL (pkg/store/proxy.go:160-161). The proxy only deduplicates when the top-level option is enabled (pkg/store/proxy.go:377-379), which avoids repeated dedup in stacked, federated query layers. - Loser-tree k-way merge. Sorted streams from many stores are merged with a tournament tree (
pkg/store/proxy_merge.go:197) rather than buffering everything in memory, keeping merge comparisons and memory bounded. - One recursive StoreAPI abstraction. Because the Querier is also a StoreAPI server, federated multi-layer query is a built-in property rather than a special mode. The
rpc.protocomment notes this optimizes resource usage on federated queries (pkg/store/storepb/rpc.proto:35).
Extension points
- StoreAPI (
pkg/store/storepb/rpc.proto:26): any component that implements theSeriesstreaming RPC can be queried by a Querier. - Object storage: pluggable backends (S3, GCS, Azure, Swift, Tencent COS) through the external
thanos-io/objstorelibrary. - The
Clientinterface (pkg/store/proxy.go:52): the common type the Querier binds every downstream component to.