Articles

Vitess Scatter-Gather Query Cost: Measure and Reduce (2026)

April 6, 2026 9 min read

Vitess scatter-gather queries are the silent assassins of your database performance, often hiding in plain sight until they bring your application to its knees.

Let’s see what happens when a Vitess "scatter-gather" query runs. Imagine you’ve got a sharded table, say users, spread across multiple shards. A query like SELECT * FROM users WHERE user_id IN (100, 500, 1000000) needs to hit all of those shards to collect the full result set. Vitess orchestrates this by sending the query to each shard (the "scatter" part) and then merging the results (the "gather" part). This isn’t inherently bad, but the cost can explode if not managed.

Here’s a typical Vitess topology:

vtgate:
  - hostname: vtgate-01
    port: 15991
vtctld:
  - hostname: vtctld-01
    port: 15999
vttablet:
  - hostname: vttablet-01
    port: 15992
    tablet_type: primary
    cell: zone1
    shard: 0
  - hostname: vttablet-02
    port: 15992
    tablet_type: primary
    cell: zone1
    shard: 1
  - hostname: vttablet-03
    port: 15992
    tablet_type: primary
    cell: zone1
    shard: 2

A query hitting users table, sharded by user_id, might look like this in your application:

# Using a hypothetical Vitess Python client
vitess_client.execute(
    "SELECT * FROM users WHERE user_id IN (100, 500, 1000000)",
    {},
    shards=["zone1-0", "zone1-1", "zone1-2"] # Explicitly specifying shards for demonstration
)

When you run this, vtgate (the query gateway) receives the request. It consults its topology to know which shards contain data for the users table. For a scatter-gather, it will send a separate query to each vttablet responsible for a relevant shard. Each vttablet executes its local part of the query. Once all vttablet instances respond, vtgate aggregates these results and returns them to the client. The "cost" here isn’t just CPU or network; it’s the number of distinct queries executed across your cluster.

The core problem Vitess scatter-gather solves is enabling SQL queries across horizontally partitioned (sharded) data. Without it, you’d need to manually query each shard and aggregate results in your application, which is complex and inefficient. However, the system’s design means that any query not routed to a single shard becomes a scatter-gather. This happens most often with:

  1. WHERE clauses that don’t map to a shard key: If you filter on a column that isn’t part of your sharding scheme, Vitess has no way to know which shard(s) to target. For example, if users is sharded by user_id and you query SELECT * FROM users WHERE city = 'New York', Vitess must scan all shards.
  2. JOINs between sharded tables: If you JOIN two tables that are sharded independently, Vitess might need to scatter-gather one or both sides of the join to find matching rows across all shards.
  3. ORDER BY or GROUP BY on non-indexed or non-shard-key columns: While Vitess can sometimes optimize these, if the sorting or grouping requires data from multiple shards, it can lead to scatter-gather.
  4. Queries hitting metadata tables: Vitess exposes internal metadata tables (like Vitess.vreplication or Vitess.resharding_logs). Queries against these tables often require a cluster-wide scan.
  5. COUNT(*) on sharded tables without a shard key filter: A simple SELECT COUNT(*) FROM users requires hitting every shard and summing the counts.

To measure scatter-gather query cost, you need to look at vtgate’s query log. Enable detailed query logging on vtgate by setting the --query-log-dir flag to a writable directory (e.g., /tmp/vt-querylogs). Then, you can analyze the logs. Look for queries that appear multiple times with different target shard specifications.

Here’s an example log entry snippet you’d find in /tmp/vt-querylogs/vtgate.log:

2023-10-27T10:00:00.123456Z INFO query.querylog(vtgate-01:15991): query="SELECT * FROM users WHERE city = 'New York'" user="app_user" conn_id=1234,1235,1236 uid=12345,12356,12378,12390,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12300,12311,12322,12333,12344,12355,12366,12377,12388,12399,12

More Deep Dives in Vitess

Want structured learning?

Take the full Vitess course →