Vitess’s MoveTables is the tool for surgically migrating tables from one keyspace to another within a running Vitess cluster, without downtime.
Let’s watch it in action. Imagine we have a customer keyspace with a users table, and we want to move this users table to a new, empty new_customer keyspace.
First, we need to create the destination keyspace if it doesn’t exist.
vtctlclient CreateKeyspace new_customer
Now, let’s check our current state. We have a users table in customer.
-- In the customer keyspace
SELECT COUNT(*) FROM users;
-- Output: 1000000
We want to move this users table to new_customer. The MoveTables command orchestrates this. It doesn’t just copy data; it sets up a replication stream from the source to the destination and then performs a cutover.
The command itself is straightforward:
vtctlclient MoveTables customer.users new_customer.users
What happens under the hood? Vitess doesn’t just dump and load. It’s a much more sophisticated process:
-
Schema Copy: Vitess first ensures the schema for
usersis identical in thenew_customerkeyspace. Ifnew_customeris truly empty and the table doesn’t exist, it will create it based on the source schema. -
Data Copy (Resharding/Migration): For large tables, Vitess uses a background data copy mechanism. It reads data from the source table and writes it to the destination table. This happens in parallel, often leveraging multiple
vtworkerprocesses or even dedicated migration processes managed byvtgate. The key is that the source table remains available for reads and writes during this phase. -
Replication Setup: Crucially, Vitess sets up a continuous replication stream from the source table’s
vtgateto the destination’svtgate(or directly to the destination shards). This ensures that any writes happening to theuserstable in thecustomerkeyspace after the data copy begins are captured and applied to thenew_customerkeyspace. This is often achieved by tailing the binlogs from the source MySQL instances. -
Cutover: Once the initial data copy is complete and replication has caught up (meaning the destination is in sync with the source), Vitess performs a near-instantaneous cutover. This involves:
- Briefly pausing writes to the source table (a very short, application-transparent window if
grace_periodis used or if the application can handle a brief read-only state). - Ensuring all pending replication writes are applied to the destination.
- Atomically updating the routing rules in
vtgateso that all future queries forusersnow go to thenew_customerkeyspace. - Resuming writes (if they were paused).
- Briefly pausing writes to the source table (a very short, application-transparent window if
The vtctlclient MoveTables command is the entry point, but Vitess provides several flags and related commands for fine-grained control. For instance, MoveTables implicitly uses Migrate and Materialize workflow types. You can monitor the progress using vtctlclient GetTablets and observing the replication lag or using vtctlclient GetSrvKeyspace to see routing rule changes.
The power of MoveTables lies in its ability to perform this migration without requiring application downtime. The application continues to send requests to the original keyspace, and Vitess transparently redirects the traffic once the migration is complete.
A common misconception is that MoveTables is just a cp command for databases. It’s far more nuanced. It’s a stateful, distributed transaction that involves schema management, parallel data transfer, and real-time replication, all orchestrated by vtgate and vtctl. The command is idempotent; running it again after a successful migration will have no effect.
Once MoveTables is complete, the users table will exist and be served from the new_customer keyspace. The original users table in the customer keyspace will be left in place, but all new traffic will be routed to new_customer. You would then typically use vtctlclient DropTable or vtctlclient DeleteTable to remove the old table once you’re confident in the migration.
The next step after a successful MoveTables is often dealing with the leftover data and ensuring your application configuration correctly reflects the new keyspace for the migrated table.