Live upgrade from Tarantool 1.6 to 1.10

General storage upgrade

Upgrade storage instances by performing the following steps for each replica set:

1. Pick a replica (a read-only instance) from the replica set. Stop this replica and start it again on the target Tarantool version. Wait until it reaches the running status (box.info.status == running).
2. Restart all other read-only instances of the replica set on the target version one by one.
3. Make one of the updated replicas the new master using the applicable instruction from Switching the master.
4. Restart the last instance of the replica set (the former master, now a replica) on the target version.

5. Run box.schema.upgrade() on the new master. This will update the Tarantool system spaces to match the currently installed version of Tarantool. The changes will be propagated to other nodes via the replication mechanism later.
6. Run box.snapshot() on every node in the replica set to make sure that the replicas immediately see the upgraded database state in case of restart.

1.6 storage upgrade specifics

• Replication check: New Tarantool nodes follow 1.6 nodes just fine, but some 1.6 nodes might disconnect from new nodes with an ER_LOADING error. This is not critical, the error goes away when replication on 1.6 is restarted:

  old_repl = box.cfg.replication
  box.cfg{replication = ""}
  box.cfg{replication = old_repl}
  

• Point of no return: When upgrading from Tarantool 1.6, the step 3 (switching the master) is the point of no return. Оnce you complete it, the schema is no longer compatible with the initial version.

• Restarting on the target version (steps 1, 2, and 4): Tarantool 1.10+ fails to recover from 1.6 xlogs, unless box.cfg{force_recovery = true} is set. There is a slight difference between 1.6 and 1.10 xlogs, which makes 1.6 xlogs appear erroneous to 1.10+ instances. In order to work around this, start the instance in force_recovery mode. To do so, add the line force_recovery = true to the file where the instance is initialized – for example, to init.lua.

• Running box.schema.upgrade() (step 5): There was a breaking change between 1.6 and 1.10 – in 1.6, the field type num was an alias to number, and in 1.10, num is converted to unsigned. This means that after box.schema.upgrade() is performed on the master, the user might have some spaces with unsigned fields containing non-unsigned values: double, int, and so on. This will make the snapshot inconsistent, unless an extra action is performed after box.schema.upgrade(). Run this code in the Tarantool console on the new master:

  -- First find all spaces containing unsigned fields with non-unsigned values in them.
  -- Say, we have one such space denoted problematic_space and the problem is in field problematic_field_no.
  a = box.space.problematic_space:format()
  a[problematic_field_no].type = 'number'
  box.space.problematic_space:format(a)
  

• Taking snapshots (step 6): The user might be concerned with snapshot size in 1.10 – it’s drastically smaller than the one created by 1.6 (for example, ~300 Mb vs. 6 Gb in some corner cases). There is nothing to worry about. Tarantool 1.6 didn’t compress snapshots, while Tarantool 1.10 and above does that.