CentOS virt - Nov 2015 - Install nested ESXi 6.x host under CentOS 7 kvm host

On 11/27/2015 01:32 PM, Patrick Laimbock wrote:
> On 11/26/15 09:47, C. L. Martinez wrote:
>> Hi all,
>>
>>   I am trying to install nested Esxi 6.x under a Centos7 kvm host to
>> use it as test lab for new ESXi versions, but I am doing something
>> wrong because I can't install it.
>>
>>   I have configured kvm and kvm_intel modules with the following
options:
>>
>> options kvm ignore_msrs=1
>> options kvm_intel nested=1 ept=1
>>
>>   and I add the following options in <cpu> xml guest file:
>>
>>    <cpu mode='custom' match='exact'>
>>      <model fallback='allow'>Westmere</model>
>>      <feature policy='require' name='vmx'/>
>>    </cpu>
>>
>> qemu-kvm package is installed from Virtualization SIG:
>>
>> qemu-kvm-common-ev-2.1.2-23.el7_1.8.1.x86_64
>> qemu-kvm-ev-2.1.2-23.el7_1.8.1.x86_64
>>
>>   Am I doing something wrong or is it not possible to accomplish this
>> scenario??
>
> Here are my old notes how to run ESXi6 on KVM on Fedora 21:
>
> # cat /etc/modprobe.d/nested.conf
> options kvm ignore_msrs=1
> options kvm-intel nested=y ept=y
>
>
> Install ESXI6 Hypervisor with the vmxnet3 nic:
>
> # virt-install --connect=qemu:///system -n esxi6 --cpu=host --vcpus=2
> --ram 4096 --os-type=linux
>
--cdrom=/var/lib/libvirt/images/VMware-VMvisor-Installer-6.0.0-2494585.x86_64.iso
> --disk
>
path=/var/lib/libvirt/images/esxi6.qcow2,format=qcow2,bus=ide,size=25,cache=none
> --hvm --network=default,model=vmxnet3 --virt-type=kvm --arch=x86_64
> --graphics spice --accelerate --video qxl --channel spicevmc
>
> or with the e1000 nic:
>
> # virt-install --connect=qemu:///system -n esxi6 --cpu=host --vcpus=2
> --ram 4096 --os-type=linux
>
--cdrom=/var/lib/libvirt/images/VMware-VMvisor-Installer-6.0.0-2494585.x86_64.iso
> --disk
>
path=/var/lib/libvirt/images/esxi6.qcow2,format=qcow2,bus=ide,size=25,cache=none
> --hvm --network=default,model=e1000 --virt-type=kvm --arch=x86_64
> --graphics spice --accelerate --video qxl --channel spicevmc
>
>
> Boot the ESXi VM and in the console interface go to Troubleshooting
> Options and enable SSH and Console access.
>
> SSH to the ESXi VM:
>
> $ ssh -o IdentitiesOnly=yes root@<IP of ESXi6 VM>
>
> # cd /etc/vmware
> # vi ./config and add the following options:
>
> vmx.allowNested = TRUE
> hv.assumeEnabled = TRUE
>
> Add the IP address of the ESXi Hypervisor to /etc/hosts
> 192.168.122.172 esxi.lab.local
>
> Now reboot the ESXi VM to make those options active.
>
>
>
> I was able to install vCenter 6 (VCSA) on ESXi6 with these steps:
>
> Create the template json file which is explained in
>
http://www.vmware.com/files/pdf/products/vsphere/VMware-vsphere-60-vcenter-server-appliance-cmdline-install.pdf
>
>
> Some tips:
>
http://www.vladan.fr/vcenter-appliance-6-0-simplified-scripted-installation/
>
>
>
> $ cat /home/patrick/VMware/patrick_embedded.json
> {
>      "__comments":
>      [
>          "Test template for VCSA deployment on a nested ESXi
Hypervisor
> running on KVM on F21"
>      ],
>
>      "deployment":
>      {
>          "esx.hostname":"esxi.lab.local",
>          "esx.datastore":"datastore1",
>          "esx.username":"root",
>          "esx.password":"password",
>          "deployment.option":"tiny",
>          "deployment.network":"vm",
>          "appliance.name":"vcsa",
>          "appliance.thin.disk.mode":true
>      },
>
>      "vcsa":
>      {
>          "system":
>          {
>              "root.password":"password",
>              "ssh.enable":true
>          },
>
>          "sso":
>          {
>              "password":"Password1!",
>              "domain-name":"lab.local",
>              "site-name":"patrick_lab"
>          }
>
>      }
> }
>
>
> Next mount the VMware-VCSA-all-6.0.0-2562643.iso file
> # cd /run/media/patrick/VMware\ VCSA/vcsa-cli-installer/lin64
>
> And run the command to deploy the VCSA appliance on top of ESXi:
> $ vcsa-deploy -v /home/patrick/VMware/patrick_embedded.json
>
>
> [patrick at lab lin64]$ ./vcsa-deploy -v
> /home/patrick/VMware/patrick_embedded.json
>
> Start vCSA command line installer to deploy vCSA "vcsa", an
embedded node.
>
> Please see /tmp/vcsa-cli-installer-8159mI.log for logging information.
>
> The SSO password meets the installation requirements.
> Opening vCSA image: /run/media/patrick/VMware VCSA/vcsa/vmware-vcsa
> SSL fingerprint
> fb:9a:f8:d2:86:14:c0:17:6d:02:f4:9d:da:e8:31:ee:fb:3b:58:d4 is different
> from host fingerprint
> 51:A6:F1:28:E7:2D:9B:02:7A:97:0A:1A:87:A3:42:A8:4C:42:24:03 for host
> esxi.lab.local as target type.
> Accept update of SSL fingerprint in known host file
> Write 'yes' or 'no'
> yes
> Opening VI target: vi://root at esxi.lab.local:443/
> Deploying to VI: vi://root at esxi.lab.local:443/
>
> Progress: 99%
> Transfer Completed
> Powering on VM: vcsa
>
> Progress: 48%
> Power On Completed
>
> Installing services...
> Progress: 5%. Setting up storage
> Progress: 50%. Installing RPMs
> Progress: 55%. Installed VMware-unixODBC-2.3.1.vmw.2-6.0.0.x86_64.rpm
> ...
> Profit!
>
> Or not because the Firefox VCSA plugin did not work on a Linux (Fedora)
> host which you need to actually use vCenter. If you make the VCSA gui
> work with Firefox on Linux, I would appreciate an update.
>
> HTH,
> Patrick
Many thanks Patrick for your valuable help!!. I've found the problem: 
removing my <cpu> entries and adding "<cpu
mode='host-passthrough'>",
works...

But in my case, I can't assign vmxnet3 as a virtual nic for ESXi, it 
seems is not supported at this time ...

On 11/27/15 14:40, C.L. Martinez wrote:
[snip]
>>
>> or with the e1000 nic:
>>
>> # virt-install --connect=qemu:///system -n esxi6 --cpu=host --vcpus=2
>> --ram 4096 --os-type=linux
>>
--cdrom=/var/lib/libvirt/images/VMware-VMvisor-Installer-6.0.0-2494585.x86_64.iso
>>
>> --disk
>>
path=/var/lib/libvirt/images/esxi6.qcow2,format=qcow2,bus=ide,size=25,cache=none
>>
>> --hvm --network=default,model=e1000 --virt-type=kvm --arch=x86_64
>> --graphics spice --accelerate --video qxl --channel spicevmc
>>
>
> Many thanks Patrick for your valuable help!!. I've found the problem:
> removing my <cpu> entries and adding "<cpu
mode='host-passthrough'>",
> works...
>
> But in my case, I can't assign vmxnet3 as a virtual nic for ESXi, it
> seems is not supported at this time ...
Have you tried installing with the e1000 nic (the second virt-install 
command)?

Cheers,
Patrick

On 11/27/2015 01:46 PM, Patrick Laimbock wrote:
> On 11/27/15 14:40, C.L. Martinez wrote:
> [snip]
>>>
>>> or with the e1000 nic:
>>>
>>> # virt-install --connect=qemu:///system -n esxi6 --cpu=host
--vcpus=2
>>> --ram 4096 --os-type=linux
>>>
--cdrom=/var/lib/libvirt/images/VMware-VMvisor-Installer-6.0.0-2494585.x86_64.iso
>>>
>>>
>>> --disk
>>>
path=/var/lib/libvirt/images/esxi6.qcow2,format=qcow2,bus=ide,size=25,cache=none
>>>
>>>
>>> --hvm --network=default,model=e1000 --virt-type=kvm --arch=x86_64
>>> --graphics spice --accelerate --video qxl --channel spicevmc
>>>
>
>>
>> Many thanks Patrick for your valuable help!!. I've found the
problem:
>> removing my <cpu> entries and adding "<cpu
mode='host-passthrough'>",
>> works...
>>
>> But in my case, I can't assign vmxnet3 as a virtual nic for ESXi,
it
>> seems is not supported at this time ...
>
> Have you tried installing with the e1000 nic (the second virt-install
> command)?
>
> Cheers,
> Patrick
Yes, with e1000, works ok.

On 11/27/15 14:46, Patrick Laimbock wrote:
> On 11/27/15 14:40, C.L. Martinez wrote:
> [snip]
>>>
>>> or with the e1000 nic:
>>>
>>> # virt-install --connect=qemu:///system -n esxi6 --cpu=host
--vcpus=2
>>> --ram 4096 --os-type=linux
>>>
--cdrom=/var/lib/libvirt/images/VMware-VMvisor-Installer-6.0.0-2494585.x86_64.iso
>>>
>>>
>>> --disk
>>>
path=/var/lib/libvirt/images/esxi6.qcow2,format=qcow2,bus=ide,size=25,cache=none
>>>
>>>
>>> --hvm --network=default,model=e1000 --virt-type=kvm --arch=x86_64
>>> --graphics spice --accelerate --video qxl --channel spicevmc
>>>
>
>>
>> Many thanks Patrick for your valuable help!!. I've found the
problem:
>> removing my <cpu> entries and adding "<cpu
mode='host-passthrough'>",
>> works...
>>
>> But in my case, I can't assign vmxnet3 as a virtual nic for ESXi,
it
>> seems is not supported at this time ...
>
> Have you tried installing with the e1000 nic (the second virt-install
> command)?
Forgot to ask. Is the vmxnet3 kernel module loaded? It's present on EL7:

/usr/lib/modules/3.10.0-229.20.1.el7.x86_64/kernel/drivers/net/vmxnet3/vmxnet3.ko

It should show up in:
# lsmod | grep vmxnet3

If it doesn't then do:
# modprobe -v vmxnet3

and try the virt-install command again.

Cheers,
Patrick