My servers have 2 NICs each. My plan is to connect one NIC each to the LAN and one NIC each to a dedicated SAN. My LAN is segmented into VLANs, so I would hook each LAN NIC into a port tagged for each VLAN I want the guests to be able to access. Then the Management interface would need to be on one of those VLANs but so far as I can tell it doesn''t support that natively. I plan to host VMs that will participate in the same VLAN as the management interface of XCP. I was able to create a vlan network like this: # xe network-create name-label=MGT name-description=Management VLAN then a VLAN interface with this command: # xe vlan-create vlan=100 pif-uuid=<uuid of the physical nic> network- uuid=<uuid of MGT network> Then I used ''xsconsole'' to change the management interface to this new network. It works up to a point. I can manage the host on VLAN 100, and I can spin up a VM and attach it to that same VLAN. Everybody can talk to each other. The problem is when I try to join the servers together in a pool. I get an error stating that the management interface cannot be on a VLAN. Is there a good reason for that restriction? What are my alternatives? Brett Westover
No, sorry. You can try to hack the xcp/xapi underlying configuration, but this require significant understanding of xapi internals (and, of cause, every your change will be your own problem without community support). But outside this you can not use tagged vlans for management interface of xapi. On 18.11.2011 04:38, Brett Westover wrote:> My servers have 2 NICs each. My plan is to connect one NIC each to > the LAN and one NIC each to a dedicated SAN. > > My LAN is segmented into VLANs, so I would hook each LAN NIC into > a port tagged for each VLAN I want the guests to be able to access. > > Then the Management interface would need to be on one of those VLANs > but so far as I can tell it doesn''t support that natively. > > I plan to host VMs that will participate in the same VLAN as the > management interface of XCP. > > I was able to create a vlan network like this: > # xe network-create name-label=MGT name-description=Management VLAN > > then a VLAN interface with this command: > # xe vlan-create vlan=100 pif-uuid=<uuid of the physical nic> network- > uuid=<uuid of MGT network> > > Then I used ''xsconsole'' to change the management interface to this > new network. > > It works up to a point. I can manage the host on VLAN 100, and I can > spin up a VM and attach it to that same VLAN. Everybody can talk to > each other. > > The problem is when I try to join the servers together in a pool. I > get an error stating that the management interface cannot be on a VLAN. > > Is there a good reason for that restriction? What are my alternatives? > > > Brett Westover > > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users
>No, sorry. You can try to hack the xcp/xapi underlying configuration, >but this require significant understanding of xapi internals (and, of >cause, every your change will be your own problem without community >support).>But outside this you can not use tagged vlans for management interface >of xapi.Do you know the reason for this restriction? It seems like a matter of a settings change in openvswitch to allow any interface to accept tagged frames. I must be misunderstanding some requirement of xapi. So what does one do in my situation? Since I only have two interfaces, and one essentially gets eaten up for management only, do I put both the LAN and SAN traffic on the same interface?
I''ve tackled this another way. (assuming you are using XCP ) My XCP vm servers usually have two gigabit ports, each connected to redundant switches. I make all vlan''s accessible via each of these ports via 802.1Q trunk, and use active/passive on the bridge. Then I create interfaces for each of the vlans, that I bridge to my vm''s. But as you''ve seen, the management vlan can not be on a tagged interface. The trick to have the management interface work on this setup is to have your switch port configured with a native non-tagged vlan. A slight security issue, just make sure you restrict your vm''s to only the tagged interfaces. There''s even an example on the manual for this. Hope this helps. -Javier On Thu, Nov 17, 2011 at 8:24 PM, Brett Westover <bwestover@pletter.com>wrote:> >No, sorry. You can try to hack the xcp/xapi underlying configuration, > >but this require significant understanding of xapi internals (and, of > >cause, every your change will be your own problem without community > >support). > > >But outside this you can not use tagged vlans for management interface > >of xapi. > > Do you know the reason for this restriction? It seems like a matter of a > settings change in openvswitch to allow any interface to accept tagged > frames. I must be misunderstanding some requirement of xapi. > > > So what does one do in my situation? Since I only have two interfaces, > and one essentially gets eaten up for management only, do I put both the > LAN and SAN traffic on the same interface? > > > > > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
>The trick to have the management interface work on this setup is to have your >switch port configured with a native non-tagged vlan. A slight security issue, >just make sure you restrict your vm''s to only the tagged interfaces. There''s >even an example on the manual for this. Hope this helps.>-JavierThanks I''ll try this. I am curious about the security issue though. What is it? My management vlan, is the highest security domain in the network. It can reach any lower level security domain, but next to nothing can get INTO the management vlan if it didn''t start there. If I make that VLAN untagged on the switch port that XCP is plugged into, and set the PVID (default vlan) to the same, then XCP can ''natively'' be on that vlan. Then I can also send tagged vlans to that same interface, so I can have VMs using other vlans over the same interface. Is that right? Finally, if I want to make a "management" VM, couldn''t I just tie it to the physical interface, instead of one of my VLANs, and then it would be on the management VLAN as well? Would this work? Is there a security risk involved? Thanks for your help Brett Westover
On Fri, Nov 18, 2011 at 11:04 AM, Brett Westover <bwestover@pletter.com> wrote:> > >The trick to have the management interface work on this setup is to have your > >switch port configured with a native non-tagged vlan. A slight security issue, > >just make sure you restrict your vm''s to only the tagged interfaces. There''s > >even an example on the manual for this. Hope this helps. > > >-Javier > > Thanks I''ll try this. I am curious about the security issue though. What is it? >IMHO, the security risks arise more from misconfiguration. Since you want to make sure none of the non management vm''s can access this higher privilege vlan.> My management vlan, is the highest security domain in the network. It can reach any lower level security domain, but next to nothing can get INTO the management vlan if it didn''t start there. > > If I make that VLAN untagged on the switch port that XCP is plugged into, and set the PVID (default vlan) to the same, then XCP can ''natively'' be on that vlan. Then I can also send tagged vlans to that same interface, so I can have VMs using other vlans over the same interface. Is that right? >Sounds right. My case, I had a bond created, then vif''s that were tagged which I used for the non management vm''s, and the management vm''s went right on the bond. See here: http://docs.vmd.citrix.com/XenServer/5.6.0fp1/1.0/en_gb/reference.html#networking-concepts-vlans In particular, sections: 7.2.4. Creating VLANs 7.2.5. Creating NIC bonds on a standalone host> > Finally, if I want to make a "management" VM, couldn''t I just tie it to the physical interface, instead of one of my VLANs, and then it would be on the management VLAN as well? Would this work? Is there a security risk involved?
We''ve using management+san traffic together. You can use tags for SAN traffic (if your switches support hybrid ports (tagged+untagged traffic), or you can use them even without creating vlan. Both traffic is very sensitive for stranger eye (f.e. all SAN traffic is unencrypted), so hiding it in isolated network is nice idea. Management interface of xapi is somehow not ''for internet use''. F.e. XCP shall not be updated from centos repos (it will break patches of LVM package and may be some other stuff), so putting management interface (ssh and so on) to the internet is bad idea. In short: hide them both in private network. On 18.11.2011 05:24, Brett Westover wrote:>> No, sorry. You can try to hack the xcp/xapi underlying configuration, >> but this require significant understanding of xapi internals (and, of >> cause, every your change will be your own problem without community >> support). >> But outside this you can not use tagged vlans for management interface >> of xapi. > Do you know the reason for this restriction? It seems like a matter of a > settings change in openvswitch to allow any interface to accept tagged > frames. I must be misunderstanding some requirement of xapi. > > > So what does one do in my situation? Since I only have two interfaces, > and one essentially gets eaten up for management only, do I put both the > LAN and SAN traffic on the same interface? > > > > > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users