← All topics

Aggregations: the analytics engine

The pivot from 'which documents match?' to 'what can I compute across them?' — and it maps straight onto GROUP BY.

elasticsearch-06-aggregations.md
readonly

Elasticsearch deep-dive · Part 6 of 8. Previous: Why results come back ranked: BM25 · Next: Running at scale: shards, replicas & the write path.

Everything so far has answered one question: which documents match? You wrote queries, they filtered and ranked, they handed back a list of hits. Aggregations answer a completely different question — what can I compute across the documents that matched? Counts, averages, groupings, histograms, distributions.

This is the analytics engine of Elasticsearch. It’s the machinery under every Kibana dashboard and the “A” in the reason people run the ELK stack for logs and metrics. And here’s the good news if you already know SQL: aggregations map almost one-to-one onto the GROUP BY and aggregate functions you write every day. You’re not learning a new idea; you’re learning a new syntax for an idea you already have.

Two families

There are two kinds of aggregation, and keeping them straight is most of the battle.

Metric aggregations compute a single number over a set of documents. These are the direct analogs of SQL’s aggregate functions:

AggregationSQL equivalent
avg, sum, min, maxAVG(), SUM(), MIN(), MAX()
value_countCOUNT(column)
cardinalityCOUNT(DISTINCT column)
statscount/min/max/avg/sum, all in one call

stats is a small gift: one aggregation returns the count, min, max, average, and sum together, so you don’t fire five separate ones to describe a column.

Bucket aggregations group documents into buckets — this is GROUP BY. Each bucket is a set of documents that share something, and every bucket carries a doc_count: how many documents fell into it (your COUNT(*) per group).

AggregationGroups bySQL equivalent
termsone bucket per distinct valueGROUP BY status
date_histogrambuckets by time intervalGROUP BY day / week / month
range / histogrambuckets by numeric rangesGROUP BY a price band

So terms on a status field gives you one bucket for paid, one for pending, one for cancelled, each with its own doc count — exactly what GROUP BY status produces.

The power move: nesting

Metric and bucket aggregations are useful alone, but the real leverage comes from combining them.

The request shape

Aggregations don’t get their own endpoint. They ride on _search, alongside the query. That raises an obvious question: if you only want the grouped numbers, why pay to return documents too?

You don’t. Set "size": 0 to say “I don’t want any hits back — just the aggregation results.” It’s faster (Elasticsearch skips fetching and scoring documents to return) and the answer lands in a separate aggregations block in the response.

Here’s the parallel, side by side. The SQL you’d write:

SELECT category, AVG(price) AS avg_price, COUNT(*) AS n
FROM products
WHERE in_stock = true
GROUP BY category;

And the Elasticsearch request:

GET /products/_search
{
  "size": 0,
  "query": { "term": { "in_stock": true } },
  "aggs": {
    "by_category": {
      "terms": { "field": "category" },
      "aggs": {
        "avg_price": { "avg": { "field": "price" } }
      }
    }
  }
}

Read it against the SQL. The terms aggregation named by_category is your GROUP BY category. The nested avg_price metric is your AVG(price). The query is your WHERE. The response mirrors it:

{
  "aggregations": {
    "by_category": {
      "buckets": [
        { "key": "keyboards", "doc_count": 42, "avg_price": { "value": 58.3 } },
        { "key": "mice",      "doc_count": 31, "avg_price": { "value": 24.9 } }
      ]
    }
  }
}

Each entry in by_category.buckets[] is one group: key is the category, doc_count is your COUNT(*), and avg_price.value is your AVG(price). You didn’t need to select the count — the bucket hands it to you for free.

You can’t aggregate on analyzed text

Now the concept from earlier in this series comes back to collect its debt. Text analysis — the tokenizing, lowercasing, stemming that makes full-text search work — is exactly what makes aggregating on a text field either fail or lie to you.

The fix is the rule from Part 2: aggregate and sort on keyword, numeric, or date fields — never on analyzed text. Those field types build doc_values and store the value whole, exactly one per document, which is what grouping needs.

This is the whole reason dynamic mapping gives a string two fields. When Elasticsearch first sees a string like a product name, it maps it as name (a text field, analyzed for search) and name.keyword (a keyword field, stored whole for aggregating and sorting). So to group by product name, you write:

"terms": { "field": "name.keyword" }

The .keyword sub-field is the one with doc_values and the intact value. Search hits name; analytics hits name.keyword. Same string, two jobs, two field types.

Where this goes next

You’ve now got the load-bearing shape of every aggregation: metrics compute numbers, buckets group documents, nesting gives you a metric per group, size: 0 drops the hits, and you aggregate on keyword/numeric/date fields — never analyzed text. That covers the large majority of real dashboards and reports.

There’s more depth here when you need it, all built on the same nesting idea:

  • Sub-aggregations go deeper — bucket inside bucket inside metric, for multi-level breakdowns.
  • Pipeline aggregations compute over the output of other aggregations — running totals, moving averages, bucket-to-bucket derivatives.
  • cardinality gives you approximate distinct counts (like COUNT(DISTINCT)) cheaply at scale, trading a little accuracy for a lot of memory saved on high-cardinality fields.

But the more pressing question is how any of this runs when your index is spread across many machines. An aggregation like terms has to be computed on each shard and then merged into one answer — and that merge step has consequences for accuracy and cost. That’s Part 7: how Elasticsearch runs across shards and replicas, and what the write path looks like underneath.

If you can say which field type you’d aggregate on for a product-name breakdown and why the text version fails — and why you’d set size: 0 when you only want the numbers — you understand aggregations better than most people copy-pasting dashboard queries.