lundi 1 juin 2015

What is the idiomatic way to perform a migration on a dynamo table

Suppose I have a dynamo table named Person, which has 2 fields, name (string), age (int). Let's assume it has a TB worth of data and experiences a small amount of read throughput, but a ton of write throughput. Now I want to add a new field called Phone (string). What is the best way to go about moving the data from one table to another?

Note: Dynamo doesn't let you rename tables, and fields cannot be null.

Here are the options I think I have:

  • Dump the table to .csv, run a script (overnight probably since it's a TB worth of data) to add a default phone number to this file. (Not ideal, will also lose all new data submitted into old table, unless I bring the service offline to perform the migration (which is not an option in this case)).
  • Use the SCAN api call. (SCAN will read all values, then will consume significant write throughput on the new table to insert all old data into it).

How can I do perform a dynamo migration on a large table w/o significant data loss?




Aucun commentaire:

Enregistrer un commentaire