-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, has anyone had ext4 filesystem remounting hours after a live-migration? We''ve been having those ''read only remounts'' always hours after a live-migration. The more load a domU has the faster we get ''read only remount''. We are suspecting something is getting corrupted after a live migration, and under load an error bumps: kernel: [3424760.190005] EXT4-fs error (device xvda3): ext4_mb_generate_buddy: EXT4-fs: group 55: 6102 blocks in bitmap, 6101 in gd kernel: [3424760.190059] Aborting journal on device xvda3-8. kernel: [3424760.191197] EXT4-fs (xvda3): Remounting filesystem read-only kernel: [3424760.204396] EXT4-fs (xvda3): delayed block allocation failed for inode 374665 at logical offset 318 with max blocks 1 with error -30 kernel: [3424760.204447] This should not happen!! Data will be lost kernel: [3424760.204483] EXT4-fs (xvda3): ext4_da_writepages: jbd2_start: 1015 pages, ino 374665; err -30 We have to fsck the volume from dom0 before re-creating domU. Our system runs on : dom0 debian6 2.6.32-5-xen-amd64 xen-hypervisor 4.0.1-2 DomU are: domU paravirtual debian6 2.6.32-5-xen-amd64 3 iscsi volumes act as vxda''s, swap, root and home (400G) 7 vcpus 8 GB RAM Regards, Sergi -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJO1f+8AAoJEN00VGSIizZ0zVIH/0KLyFgLAwJNmtXVbHo0VcsY IicYbcv5CJNkJZ/WDskPK/33HsWH98epWzsRbr33qAibZrKfJig/1HuHs8SwGwGe Xt1ii9+gnhy/oyNSnPbgmDpD6W81iEPkCTLm0fdHmndP9E5dbIWOzCMcIMvFB0RD +2bPKQ3CcKEqnbXcriYzKVdfU2Y4/UbgZJYCQBJizK20A9KbqwAyLve/zRxJ7WP8 Q7+zWXkxefwcVq9rcBfkITvlrLQw9b9kK1KT7+7MrnxkG0ujIFYKVqkBxuIb4C3u kILnz8Gh6vuiKhHy+5DKqEpGmhSsq/VVymLQCNerveluer2TF9QOTisv2kXw58Y=+hQj -----END PGP SIGNATURE-----
Konrad Rzeszutek Wilk
2011-Dec-12 16:37 UTC
Re: ext4_mb_generate_buddy remounting read-only on domU
On Wed, Nov 30, 2011 at 11:04:44AM +0100, Sergi Seira wrote:> -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi, > > has anyone had ext4 filesystem remounting hours after a live-migration? > > We''ve been having those ''read only remounts'' always hours after a live-migration. > The more load a domU has the faster we get ''read only remount''. > We are suspecting something is getting corrupted after a live migration, and under load an error bumps: > > kernel: [3424760.190005] EXT4-fs error (device xvda3): ext4_mb_generate_buddy: EXT4-fs: group 55: 6102 blocks in bitmap, 6101 in gd > kernel: [3424760.190059] Aborting journal on device xvda3-8. > kernel: [3424760.191197] EXT4-fs (xvda3): Remounting filesystem read-only > kernel: [3424760.204396] EXT4-fs (xvda3): delayed block allocation failed for inode 374665 at logical offset 318 with max blocks 1 with error -30 > kernel: [3424760.204447] This should not happen!! Data will be lost > kernel: [3424760.204483] EXT4-fs (xvda3): ext4_da_writepages: jbd2_start: 1015 pages, ino 374665; err -30 > > We have to fsck the volume from dom0 before re-creating domU. > > Our system runs on : > > dom0 debian6 2.6.32-5-xen-amd64This looks familiar, http://xen.1045712.n5.nabble.com/ext4-BUG-in-dom0-Kernel-2-6-32-36-td4773487.html Ah no. There was some fix to blktap for this I think. What exactly is the drivery you are using to drive the root/home/swap of the guest? Is it blkback or blktap?> xen-hypervisor 4.0.1-2 > > DomU are: > > domU paravirtual debian6 2.6.32-5-xen-amd64 > 3 iscsi volumes act as vxda''s, swap, root and home (400G) > 7 vcpus > 8 GB RAM > > Regards, > Sergi > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.11 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iQEcBAEBAgAGBQJO1f+8AAoJEN00VGSIizZ0zVIH/0KLyFgLAwJNmtXVbHo0VcsY > IicYbcv5CJNkJZ/WDskPK/33HsWH98epWzsRbr33qAibZrKfJig/1HuHs8SwGwGe > Xt1ii9+gnhy/oyNSnPbgmDpD6W81iEPkCTLm0fdHmndP9E5dbIWOzCMcIMvFB0RD > +2bPKQ3CcKEqnbXcriYzKVdfU2Y4/UbgZJYCQBJizK20A9KbqwAyLve/zRxJ7WP8 > Q7+zWXkxefwcVq9rcBfkITvlrLQw9b9kK1KT7+7MrnxkG0ujIFYKVqkBxuIb4C3u > kILnz8Gh6vuiKhHy+5DKqEpGmhSsq/VVymLQCNerveluer2TF9QOTisv2kXw58Y> =+hQj > -----END PGP SIGNATURE----- > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel