libvirt users - Dec 2018 - Re: avoiding PCI bus 8 / using PCI function / virt-install

I’m using virt-install to spin up VMs.  At times I have a need to spin up
VMs which have 100s of interfaces.  I ran into the PCI issue mentioned in
this previous thread based on how virt-install assigns PCI addresses to
interfaces:



https://www.redhat.com/archives/libvirt-users/2018-December/msg00064.html



Using the info mention there I was able to part hand / part sw re-write an
XML where I would remove PCI bus references above the value of 8 and
re-address the per interface PCI info to use the function field (I don’t
need hot pluggable).



But the process I’ve built is brittle.


Wondering what options i might have to better deal with this PCI issue.



Is there a way to tell virt-install, when building the info it passes to
qemu, to use the function field during the PCI assignment process so as to
support many more interfaces before hitting the PCI bus == 8 issue?



If not, is there a way with one of the virt command line tools to create
the XML (with the PCI addresses specified) so that I can process that XML
and re-write the PCI addressing values?  Right now the only way I’ve been
able to get that detailed XML file is to 1) virt-install and let the VM
begin the boot process and then do a 2) virsh dumpxml and then 3) virsh
destroy/undefine that VM, 4) modify the XML and then 5) virsh create
./modified.xml.  Is there a cleaner way to do this?


Thanks

On Wed, 26 Dec 2018 at 16:26, b f31415 <bf31415@gmail.com> wrote:
> If not, is there a way with one of the virt command line tools to create
> the XML (with the PCI addresses specified) so that I can process that XML
> and re-write the PCI addressing values?  Right now the only way I’ve been
> able to get that detailed XML file is to 1) virt-install and let the VM
> begin the boot process and then do a 2) virsh dumpxml and then 3) virsh
> destroy/undefine that VM, 4) modify the XML and then 5) virsh create
> ./modified.xml.  Is there a cleaner way to do this?
>
It won't do everything you want, but check out virt-install --print-step
--dry-run.  This should at least allow you to get at the XML for the
intermediate steps and remove the brittleness of defining, destroying,
undefining the VM.

Cheers,

- Peter

Thanks.  Maybe I'm missing something but when I try virt-install --dry-run
--print-xml it generates this snippet of XML:

    <interface type="bridge">

      <source bridge="ge_0_0_0_77"/>

      <mac address="52:54:00:01:00:4e"/>

      <model type="virtio"/>

    </interface>

but what i need (and get from virsh dumpxml) has this extra PCI line.

    <interface type='bridge'>

      <mac address='52:54:00:01:00:4e'/>

      <source bridge='ge_0_0_0_77'/>

      <target dev='vnet78'/>

      <model type='virtio'/>

      <alias name='net78'/>

     <address type='pci' domain='0x0000' bus='0x01'
slot='0x0a'
function='0x6' />

    </interface>

Did I miss something?  Do you see PCI in the XML when you do the
virt-install --dryrun approach?

Thanks







On Wed, Dec 26, 2018 at 2:57 PM Peter Crowther
<peter.crowther@melandra.com>
wrote:
> On Wed, 26 Dec 2018 at 16:26, b f31415 <bf31415@gmail.com> wrote:
>
>> If not, is there a way with one of the virt command line tools to
create
>> the XML (with the PCI addresses specified) so that I can process that
XML
>> and re-write the PCI addressing values?  Right now the only way I’ve
been
>> able to get that detailed XML file is to 1) virt-install and let the VM
>> begin the boot process and then do a 2) virsh dumpxml and then 3) virsh
>> destroy/undefine that VM, 4) modify the XML and then 5) virsh create
>> ./modified.xml.  Is there a cleaner way to do this?
>>
>
> It won't do everything you want, but check out virt-install
--print-step
> --dry-run.  This should at least allow you to get at the XML for the
> intermediate steps and remove the brittleness of defining, destroying,
> undefining the VM.
>
> Cheers,
>
> - Peter
>