libvirt users - Nov 2014 - Re: KVM incremental backup using CBT

> > A quick google search found this:
> >
http://callecalle.uach.cl/ovirt-engine/docs/manual/en_US/html/Technical_Gu
> > id e/QEMU_Guest_Agent_Overview.html
> > 
> > Sadly, it didn't spell out the name of where you install hook
scripts
> > into the guest.  But this commit is pretty telling:
> >
https://github.com/qemu/qemu/blob/master/scripts/qemu-guest-agent/fsfreeze
> > -h ook
After a little bit of fiddling i've got this running. Here is what's 
neccessary in case someone wanna try this too.

qemu-guest.-agent has to run with the option: "-F
/etc/qemu/fsfreeze-hook".
This file contains as pointed out by Eric:

https://github.com/qemu/qemu/blob/master/scripts/qemu-guest-agent/fsfreeze-hook

Then you need to have a directory called /etc/qemu/fsfreeze-hook.d. Inside 
this directory this mysql-flush script:

https://github.com/qemu/qemu/blob/master/scripts/qemu-guest-agent/fsfreeze-hook.d/mysql-flush.sh.sample

I had to adjust this script a little for mariadb 10.0.14. But after that 
everything works as expected.

Here the output:

Sun  2 Nov 13:46:14 CET 2014: execute /etc/qemu/fsfreeze-hook.d/mysql-flush 
freeze
+ MYSQL='mysql -uroot -ppassword'
+ FIFO=/tmp/mysql-flush.fifo
+ case "$1" in
+ mkfifo /tmp/mysql-flush.fifo
+ flush_and_wait
+ printf 'FLUSH TABLES WITH READ LOCK \G\n'
+ mysql -uroot -ppassword
+ read
++ echo 'SHOW STATUS LIKE "Key_blocks_not_flushed"'
++ mysql -uroot -ppassword
++ tail -1
++ cut -f 2
+ '[' 0 -gt 0 ']'
++ mktemp /tmp/mysql-flush.XXXXXX
+ INNODB_STATUS=/tmp/mysql-flush.eTdvA4
+ '[' 0 -ne 0 ']'
+ trap 'rm -f /tmp/mysql-flush.eTdvA4' SIGINT
+ :
+ printf 'SHOW ENGINE INNODB STATUS \G'
+ mysql -uroot -ppassword
++ grep 'Log sequence number' /tmp/mysql-flush.eTdvA4 tr -s ' '
++ cut '-d ' -f4
+ LOG_CURRENT=12242169543
++ grep 'Log flushed up to' /tmp/mysql-flush.eTdvA4 tr -s ' '
++ cut '-d ' -f7
+ LOG_FLUSHED=12242169543
+ '[' 12242169543 = 12242169543 ']'
+ break
+ rm -f /tmp/mysql-flush.eTdvA4
Sun  2 Nov 13:46:14 CET 2014: /etc/qemu/fsfreeze-hook.d/mysql-flush finished 
with status=0
Sun  2 Nov 13:46:14 CET 2014: execute /etc/qemu/fsfreeze-hook.d/mysql-flush 
thaw
+ MYSQL='mysql -uroot -ppassword'
+ FIFO=/tmp/mysql-flush.fifo
+ case "$1" in
+ '[' '!' -p /tmp/mysql-flush.fifo ']'
+ echo
+ rm -f /tmp/mysql-flush.fifo
+ printf 'UNLOCK TABLES \G\n'
Sun  2 Nov 13:46:14 CET 2014: /etc/qemu/fsfreeze-hook.d/mysql-flush finished 
with status=0

Just to be sure i activated the general.log for mysql:

141102 13:46:14     6 Connect   root@localhost as anonymous on 
                    6 Query     select @@version_comment limit 1
                    6 Query     FLUSH TABLES WITH READ LOCK
                    7 Connect   root@localhost as anonymous on 
                    7 Query     select @@version_comment limit 1
                    7 Query     SHOW STATUS LIKE
"Key_blocks_not_flushed"
                    7 Quit      
                    8 Connect   root@localhost as anonymous on 
                    8 Query     select @@version_comment limit 1
                    8 Query     SHOW ENGINE INNODB STATUS
                    8 Quit      
                    6 Query     UNLOCK TABLES
                    6 Quit

Nice!

cheers
t.

On 11/02/2014 02:13 PM, Thomas Stein wrote:
> After a little bit of fiddling i've got this running. Here is
what's
> neccessary in case someone wanna try this too.
That's awesome news! Thanks for sharing.

>
https://github.com/qemu/qemu/blob/master/scripts/qemu-guest-agent/fsfreeze-hook.d/mysql-flush.sh.sample
> 
> I had to adjust this script a little for mariadb 10.0.14. But after that 
> everything works as expected.
Do you want to post your version of the script for others to learn from?

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

Am 03.11.14 09:39, schrieb Eric Blake:
> On 11/02/2014 02:13 PM, Thomas Stein wrote:
> 
>> After a little bit of fiddling i've got this running. Here is
what's
>> neccessary in case someone wanna try this too.
> 
> That's awesome news! Thanks for sharing.
> 
> 
>>
https://github.com/qemu/qemu/blob/master/scripts/qemu-guest-agent/fsfreeze-hook.d/mysql-flush.sh.sample
>> 
>> I had to adjust this script a little for mariadb 10.0.14. But after
that
>> everything works as expected.
> 
> Do you want to post your version of the script for others to learn from?
Sure. Here we go:

----
#!/bin/bash -x

# Flush MySQL tables to the disk before the filesystem is freezed.
# At the same time, this keeps a read lock while the filesystem is freezed
# in order to avoid write accesses by the other clients.

MYSQL="mysql -uroot -ppassword"
FIFO=/tmp/mysql-flush.fifo

flush_and_wait() {
    printf "FLUSH TABLES WITH READ LOCK \\G\n"
    read < $FIFO
    printf "UNLOCK TABLES \\G\n"
}

case "$1" in
    freeze)
        mkfifo $FIFO || exit 1
        flush_and_wait | $MYSQL &
        # wait until every block is flushed
        while [ "$(echo 'SHOW STATUS LIKE
"Key_blocks_not_flushed"' |\
                 $MYSQL | tail -1 | cut -f 2)" -gt 0 ]; do
            sleep 1
        done
        # for InnoDB, wait until every log is flushed
        INNODB_STATUS=$(mktemp /tmp/mysql-flush.XXXXXX)
        [ $? -ne 0 ] && exit 2
       trap "rm -f $INNODB_STATUS" SIGINT
        while :; do
            printf "SHOW ENGINE INNODB STATUS \\G" | $MYSQL >
$INNODB_STATUS

            LOG_CURRENT=$(grep 'Log sequence number' $INNODB_STATUS tr
-s ' ' | cut -d' ' -f4)

            LOG_FLUSHED=$(grep 'Log flushed up to' $INNODB_STATUS tr -s
' ' | cut -d' ' -f7)

            [ "$LOG_CURRENT" = "$LOG_FLUSHED" ] &&
break
            sleep 1
        done
        rm -f $INNODB_STATUS
        ;;

    thaw)
        [ ! -p $FIFO ] && exit 1
        echo > $FIFO
        rm -f $FIFO
        ;;
esac
----

cheers
t.