Hi, I was trying to setup glusterfs 3.7.4-2 on CentOS 6.7 on Microsoft Azure. We are fortunate to receive service from Microsoft Bizspark program for startups. Unfortunately, we are forced to run the virtual machines on separate Azure accounts. This in turn causes the VM's not being able to ping each other. Reading the gluster documentation at http://gluster.readthedocs.org/en/latest/Install-Guide/Configure/ it says "For the Gluster to communicate within a cluster either the firewalls have to be turned off or enable communication for each server." In my case this is impossible because the VM's under different Azure accounts are unable to ping each other. This restriction is a design choice Microsoft has made. The needed glusterfs service ports are open between the VM's. What could be a remedy to this problem? Here's detailed information about my setup: (on hostA) # gluster peer status Number of Peers: 1 Hostname: hostB.domain Uuid: 7e468940-97e5-4595-a835-5eeedae07770 State: Peer in Cluster (Connected) (on hostB) # gluster peer status Number of Peers: 1 Hostname: hostA.domain Uuid: 1a5a62a8-4e08-4f06-9f5d-e159a2f7e5c9 State: Peer in Cluster (Connected) (on hostA) # gluster volume create home replica 2 hostA.domain:/data/home/gv0 hostB.domain:/data/home/gv0 volume create: home: failed: Host hostA.domain is not in 'Peer in Cluster' state (/var/log/glusterfs/etc-glusterfs-glusterd.vol.log) [2015-09-22 07:54:36.374674] I [MSGID: 106487] [glusterd-handler.c:1402:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2015-09-22 07:59:08.515867] E [MSGID: 106452] [glusterd-utils.c:5569:glusterd_new_brick_validate] 0-management: Host hostA.domain is not in 'Peer in Cluster' state [2015-09-22 07:59:08.515896] E [MSGID: 106536] [glusterd-volume-ops.c:1273:glusterd_op_stage_create_volume] 0-management: Host hostA.domain is not in 'Peer in Cluster' state [2015-09-22 07:59:08.515908] E [MSGID: 106301] [glusterd-syncop.c:1274:gd_stage_op_phase] 0-management: Staging of operation 'Volume Create' failed on localhost : Host hostA.domain is not in 'Peer in Cluster' state Best regards, Aki -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20150924/6f4cd7a2/attachment.html>
Did you made: gluster peer probe SERVER? Waht is telling you the probe? EDV Daniel M?ller Leitung EDV Tropenklinik Paul-Lechler-Krankenhaus Paul-Lechler-Str. 24 72076 T?bingen Tel.: 07071/206-463, Fax: 07071/206-499 eMail: mueller at tropenklinik.de Internet: www.tropenklinik.de Von: gluster-users-bounces at gluster.org [mailto:gluster-users-bounces at gluster.org] Im Auftrag von Aki Ketolainen Gesendet: Donnerstag, 24. September 2015 11:51 An: gluster-users at gluster.org Betreff: [Gluster-users] glusterfs on Microsoft Azure? Hi, I was trying to setup glusterfs 3.7.4-2 on CentOS 6.7 on Microsoft Azure. We are fortunate to receive service from Microsoft Bizspark program for startups. Unfortunately, we are forced to run the virtual machines on separate Azure accounts. This in turn causes the VM's not being able to ping each other. Reading the gluster documentation at http://gluster.readthedocs.org/en/latest/Install-Guide/Configure/ it says "For the Gluster to communicate within a cluster either the firewalls have to be turned off or enable communication for each server." In my case this is impossible because the VM's under different Azure accounts are unable to ping each other. This restriction is a design choice Microsoft has made. The needed glusterfs service ports are open between the VM's. What could be a remedy to this problem? Here's detailed information about my setup: (on hostA) # gluster peer status Number of Peers: 1 Hostname: hostB.domain Uuid: 7e468940-97e5-4595-a835-5eeedae07770 State: Peer in Cluster (Connected) (on hostB) # gluster peer status Number of Peers: 1 Hostname: hostA.domain Uuid: 1a5a62a8-4e08-4f06-9f5d-e159a2f7e5c9 State: Peer in Cluster (Connected) (on hostA) # gluster volume create home replica 2 hostA.domain:/data/home/gv0 hostB.domain:/data/home/gv0 volume create: home: failed: Host hostA.domain is not in 'Peer in Cluster' state (/var/log/glusterfs/etc-glusterfs-glusterd.vol.log) [2015-09-22 07:54:36.374674] I [MSGID: 106487] [glusterd-handler.c:1402:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2015-09-22 07:59:08.515867] E [MSGID: 106452] [glusterd-utils.c:5569:glusterd_new_brick_validate] 0-management: Host hostA.domain is not in 'Peer in Cluster' state [2015-09-22 07:59:08.515896] E [MSGID: 106536] [glusterd-volume-ops.c:1273:glusterd_op_stage_create_volume] 0-management: Host hostA.domain is not in 'Peer in Cluster' state [2015-09-22 07:59:08.515908] E [MSGID: 106301] [glusterd-syncop.c:1274:gd_stage_op_phase] 0-management: Staging of operation 'Volume Create' failed on localhost : Host hostA.domain is not in 'Peer in Cluster' state Best regards, Aki
Can you configure your /etc/hosts file to include these hostnames along with their IPs? That should solve your problem I guess. -Atin Sent from one plus one On Sep 24, 2015 3:26 PM, "Aki Ketolainen" <ak-su at outlook.com> wrote:> Hi, > > I was trying to setup glusterfs 3.7.4-2 on CentOS 6.7 on Microsoft Azure. > We are fortunate to receive service from Microsoft Bizspark program for > startups. > Unfortunately, we are forced to run the virtual machines on separate Azure > accounts. > This in turn causes the VM's not being able to ping each other. > > Reading the gluster documentation at > http://gluster.readthedocs.org/en/latest/Install-Guide/Configure/ > it says "For the Gluster to communicate within a cluster either the > firewalls have to be turned off or enable communication for each server." > > In my case this is impossible because the VM's under different Azure > accounts are unable to > ping each other. This restriction is a design choice Microsoft has made. > The needed glusterfs > service ports are open between the VM's. > > What could be a remedy to this problem? > > Here's detailed information about my setup: > > (on hostA) > # gluster peer status > Number of Peers: 1 > > Hostname: hostB.domain > Uuid: 7e468940-97e5-4595-a835-5eeedae07770 > State: Peer in Cluster (Connected) > > (on hostB) > # gluster peer status > Number of Peers: 1 > > Hostname: hostA.domain > Uuid: 1a5a62a8-4e08-4f06-9f5d-e159a2f7e5c9 > State: Peer in Cluster (Connected) > > (on hostA) > # gluster volume create home replica 2 hostA.domain:/data/home/gv0 > hostB.domain:/data/home/gv0 > volume create: home: failed: Host hostA.domain is not in 'Peer in Cluster' > state > > (/var/log/glusterfs/etc-glusterfs-glusterd.vol.log) > [2015-09-22 07:54:36.374674] I [MSGID: 106487] > [glusterd-handler.c:1402:__glusterd_handle_cli_list_friends] 0-glusterd: > Received cli list req > [2015-09-22 07:59:08.515867] E [MSGID: 106452] > [glusterd-utils.c:5569:glusterd_new_brick_validate] 0-management: Host > hostA.domain is not in 'Peer in Cluster' state > [2015-09-22 07:59:08.515896] E [MSGID: 106536] > [glusterd-volume-ops.c:1273:glusterd_op_stage_create_volume] 0-management: > Host hostA.domain is not in 'Peer in Cluster' state > [2015-09-22 07:59:08.515908] E [MSGID: 106301] > [glusterd-syncop.c:1274:gd_stage_op_phase] 0-management: Staging of > operation 'Volume Create' failed on localhost : Host hostA.domain is not in > 'Peer in Cluster' state > > Best regards, > > Aki > > > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://www.gluster.org/mailman/listinfo/gluster-users >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20150924/afa20096/attachment.html>
Hi, Replying to the gluster-users list with a solution to this problem. Our VMs are under 4 different Azure subscriptions in separate virtual networks. VMs under these subscriptions are unable to ping each other, thus unable to be a part of a Gluster cluster or be a Gluster client. Mr. Tom Wu has written a Powershell script to connect Azure virtual networks under different Azure subscriptions. https://tombwu.wordpress.com/2015/07/02/build-complex-cross-azure-subscription-windows-lab-by-using-azure-site-to-site-vpn/ This solved my problem completely. The throughput to the Glusterfs client is approx. 100 Mbps with the basic Azure VPN gateway. There is a high performance VPN gateway but it costs about 8? per day. One thing that was missing from Tom's blog post was how the Azure Powershell authentication is done. This needs to be set in Azure Powershell session with Get-AzurePublishSettingsFile and Import-AzurePublishSettingsFile. -- Aki From: ak-su at outlook.com To: gluster-users at gluster.org Subject: glusterfs on Microsoft Azure? Date: Thu, 24 Sep 2015 09:51:18 +0000 Hi, I was trying to setup glusterfs 3.7.4-2 on CentOS 6.7 on Microsoft Azure. We are fortunate to receive service from Microsoft Bizspark program for startups. Unfortunately, we are forced to run the virtual machines on separate Azure accounts. This in turn causes the VM's not being able to ping each other. Reading the gluster documentation at http://gluster.readthedocs.org/en/latest/Install-Guide/Configure/ it says "For the Gluster to communicate within a cluster either the firewalls have to be turned off or enable communication for each server." In my case this is impossible because the VM's under different Azure accounts are unable to ping each other. This restriction is a design choice Microsoft has made. The needed glusterfs service ports are open between the VM's. What could be a remedy to this problem? Here's detailed information about my setup: (on hostA) # gluster peer status Number of Peers: 1 Hostname: hostB.domain Uuid: 7e468940-97e5-4595-a835-5eeedae07770 State: Peer in Cluster (Connected) (on hostB) # gluster peer status Number of Peers: 1 Hostname: hostA.domain Uuid: 1a5a62a8-4e08-4f06-9f5d-e159a2f7e5c9 State: Peer in Cluster (Connected) (on hostA) # gluster volume create home replica 2 hostA.domain:/data/home/gv0 hostB.domain:/data/home/gv0 volume create: home: failed: Host hostA.domain is not in 'Peer in Cluster' state (/var/log/glusterfs/etc-glusterfs-glusterd.vol.log) [2015-09-22 07:54:36.374674] I [MSGID: 106487] [glusterd-handler.c:1402:__glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req [2015-09-22 07:59:08.515867] E [MSGID: 106452] [glusterd-utils.c:5569:glusterd_new_brick_validate] 0-management: Host hostA.domain is not in 'Peer in Cluster' state [2015-09-22 07:59:08.515896] E [MSGID: 106536] [glusterd-volume-ops.c:1273:glusterd_op_stage_create_volume] 0-management: Host hostA.domain is not in 'Peer in Cluster' state [2015-09-22 07:59:08.515908] E [MSGID: 106301] [glusterd-syncop.c:1274:gd_stage_op_phase] 0-management: Staging of operation 'Volume Create' failed on localhost : Host hostA.domain is not in 'Peer in Cluster' state Best regards, Aki -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20151017/23d612b3/attachment.html>