Mysql Innodb Hotcopy in Python
Lately for Spinn3r I’ve been writing programs in Python to automate database operations. This hotcopy script utilizes LVM. Previously we didn’t use LVM and so this script used to do an Innodb Freeze. The simple idea here is to restore a mysql shard replica’s data from another replica…
Look at the code here:
mysql cluster tools @ bitbucket.org
The cool feature this hotcopy script has is an undo/restore mechanism that
kicks in when an exception is caught. Basically I’ve implemented a nullary function queue.
With each state change, I push the reverse (or undo/restore operation) onto the queue. When an exception is caught the rollback() method repeatedly pops and executes the last nullary off the queue until there are none left.
What I’m calling a nullary function is merely a function call wrapped in a closure (although maybe that’s not correct because they say Python doesn’t have true closures) in the form of a lambda with zero arguments.
Python’s excellent exception handling makes this rollback() feature really useful because there are lots of moving parts at work that could break and throw an exception. The hotcopy process causes several state changes on the source replica such as : set single user mode, stop replication (if the source is a slave), take LVM snapshot, mount snapshot etc…
I don’t like LVM snapshots laying around so we can COW forever! Nor would I want a database server to remain in “single user mode”…
Three ways to extend this project:
I’m sure many like Spinn3r have similar infrastructure goals. The above InnoDB modification is one of several Desirable Innodb Features that Spinn3r will probably throw down cash for…
Check it out at bitbucket.org, my Mysql Innodb Hotcopy program.
This is not even a release candidate… but if anyone wants to look at the code… feel free.
Please leave your thoughts and comments.
leave a comment