01.07.2016 10:51, Dmitry Melekhov ?????:> 01.07.2016 09:46, Dmitry Melekhov ?????: >> 01.07.2016 09:23, Lindsay Mathieson ?????: >>> How did you do your downgrade? >> >> downloaded packages from >> https://download.gluster.org/pub/gluster/glusterfs/3.7/3.7.11/CentOS/epel-7.2/x86_64/ >> >> and >> https://download.gluster.org/pub/gluster/glusterfs/3.7/3.7.11/CentOS/epel-7.2/noarch/ >> >> >> and >> yum downgrade * >> from this directory > > OK, forgot to restart libvirtd after this downgrade. > Rebooted one of servers to be completely sure. > > Now I get > > virsh create w8test.xml > error: Failed to create domain from w8test.xml > error: End of file while reading data: Input/output error > > :-( > > looks like some problems with libgfs packages for centos... >OK, in our case we solved this problem by installing 3.7.12 again and restarting libvirtd after that. Now we have all this working. As I said this is centos7. I guess we did not restart libvirtd during upgrade to 3.7.11 too, so I don't know why libvirt doesn't work with 3.7.11 libs... :-(
01.07.2016 11:21, Dmitry Melekhov ?????:> OK, in our case we solved this problem by installing 3.7.12 again and > restarting libvirtd after that. > Now we have all this working. > As I said this is centos7. > > I guess we did not restart libvirtd during upgrade to 3.7.11 too, so I > don't know why libvirt doesn't work > with 3.7.11 libs... :-( >Oops, by "solved" I mean that now we can start VMs :-D But they don't work, they dies: [2016-07-01 06:57:51.836023] I [MSGID: 108019] [afr-transaction.c:1339:afr_post_blocking_inodelk_cbk] 0-pool-replicate-0: Blocking inodelks failed. block I/O error in device 'drive-ide0-0-0': Operation not permitted (1) [2016-07-01 06:57:51.837022] E [MSGID: 114031] [client-rpc-fops.c:1676:client3_3_finodelk_cbk] 0-pool-client-7: remote operation failed [Invalid argument] [2016-07-01 06:57:51.837086] E [MSGID: 114031] [client-rpc-fops.c:1676:client3_3_finodelk_cbk] 0-pool-client-6: remote operation failed [Invalid argument] [2016-07-01 06:57:51.837306] E [MSGID: 114031] [client-rpc-fops.c:1676:client3_3_finodelk_cbk] 0-pool-client-8: remote operation failed [Invalid argument] [2016-07-01 06:57:51.837658] E [MSGID: 114031] [client-rpc-fops.c:1676:client3_3_finodelk_cbk] 0-pool-client-6: remote operation failed [Invalid argument] [2016-07-01 06:57:51.837940] E [MSGID: 114031] [client-rpc-fops.c:1676:client3_3_finodelk_cbk] 0-pool-client-7: remote operation failed [Invalid argument] [2016-07-01 06:57:51.838533] E [MSGID: 114031] [client-rpc-fops.c:1676:client3_3_finodelk_cbk] 0-pool-client-8: remote operation failed [Invalid argument] [2016-07-01 06:57:51.838583] W [MSGID: 108019] [afr-lk-common.c:1082:is_blocking_locks_count_sufficient] 0-pool-replicate-0: Unable to obtain blocking inode lock on even one child for gfid:c45544e7-f449-48cf-bbc1 -cfd9f3ebc401. End on gluster side in brick log: [2016-07-01 07:31:44.013224] E [MSGID: 115053] [server-rpc-fops.c:314:server_finodelk_cbk] 0-pool-server: 3325937: FINODELK 0 (52dfd80c-ef02-4749-a43a-db9573113420) ==> (Invalid argument) [Invalid argument] [2016-07-01 07:31:44.013306] E [MSGID: 115053] [server-rpc-fops.c:314:server_finodelk_cbk] 0-pool-server: 3325938: FINODELK 0 (52dfd80c-ef02-4749-a43a-db9573113420) ==> (Invalid argument) [Invalid argument] [2016-07-01 07:31:44.013410] E [MSGID: 115053] [server-rpc-fops.c:314:server_finodelk_cbk] 0-pool-server: 3325939: FINODELK 0 (52dfd80c-ef02-4749-a43a-db9573113420) ==> (Invalid argument) [Invalid argument] [2016-07-01 07:31:44.013485] E [MSGID: 115053] [server-rpc-fops.c:314:server_finodelk_cbk] 0-pool-server: 3325940: FINODELK 0 (52dfd80c-ef02-4749-a43a-db9573113420) ==> (Invalid argument) [Invalid argument] [2016-07-01 07:31:44.013535] E [MSGID: 115053] [server-rpc-fops.c:314:server_finodelk_cbk] 0-pool-server: 3325941: FINODELK 0 (52dfd80c-ef02-4749-a43a-db9573113420) ==> (Invalid argument) [Invalid argument] [2016-07-01 07:31:44.013628] E [MSGID: 115053] [server-rpc-fops.c:314:server_finodelk_cbk] 0-pool-server: 3325943: FINODELK 0 (52dfd80c-ef02-4749-a43a-db9573113420) ==> (Invalid argument) [Invalid argument] [2016-07-01 07:31:44.013703] E [MSGID: 115053] [server-rpc-fops.c:314:server_finodelk_cbk] 0-pool-server: 3325944: FINODELK 0 (52dfd80c-ef02-4749-a43a-db9573113420) ==> (Invalid argument) [Invalid argument] [2016-07-01 07:31:44.013732] E [MSGID: 115053] [server-rpc-fops.c:314:server_finodelk_cbk] 0-pool-server: 3325945: FINODELK 0 (52dfd80c-ef02-4749-a43a-db9573113420) ==> (Invalid argument) [Invalid argument] [2016-07-01 07:31:44.013734] E [MSGID: 115053] [server-rpc-fops.c:314:server_finodelk_cbk] 0-pool-server: 3325946: FINODELK 0 (52dfd80c-ef02-4749-a43a-db9573113420) ==> (Invalid argument) [Invalid argument] [2016-07-01 07:31:44.013782] E [MSGID: 115053] [server-rpc-fops.c:314:server_finodelk_cbk] 0-pool-server: 3325947: FINODELK 0 (52dfd80c-ef02-4749-a43a-db9573113420) ==> (Invalid argument) [Invalid argument] [2016-07-01 07:31:44.013793] E [MSGID: 115053] [server-rpc-fops.c:314:server_finodelk_cbk] 0-pool-server: 3325948: FINODELK 0 (52dfd80c-ef02-4749-a43a-db9573113420) ==> (Invalid argument) [Invalid argument] [2016-07-01 07:31:44.013895] E [MSGID: 115053] [server-rpc-fops.c:314:server_finodelk_cbk] 0-pool-server: 3325949: FINODELK 0 (52dfd80c-ef02-4749-a43a-db9573113420) ==> (Invalid argument) [Invalid argument] [2016-07-01 07:31:44.013918] E [MSGID: 115053] [server-rpc-fops.c:314:server_finodelk_cbk] 0-pool-server: 3325950: FINODELK 0 (52dfd80c-ef02-4749-a43a-db9573113420) ==> (Invalid argument) [Invalid argument] So, gfapi is broken...