libvirt users - May 2016 - Re: [libvirt] Creating a storage volume for raw format file

On 04/05/16 14:07, Cole Robinson wrote:
> On 05/04/2016 06:54 AM, Paul Carlton wrote:
>> Hi
>>
>> I'm trying to create a volume in an existing storage pool using
python by
>> calling createXML() on the pool object.
>> If the file that is the subject of the new volume exists you get and
error,
>> also if it doesn't exist
>>
>> I worked out that specifying
>>
>> <source>
>>    <path>path to copy of file</path>
>> </source>
>>
>>   works, seemingly by copying the 'source' file to the intended
location of the
>> volume file
>>
>> However I am concerned about the overhead of copying the file, is there
a way
>> to tell it to move the file (i.e. rename)
> Unfortunately there isn't any simple API to 'rename' a storage
volume
>
> - Cole
>
Actually I don't want to rename a storage pool, in my inept attempt a 
brevity I failed to explain myself clearly.

I want to create a raw format file in a directory that is referenced by 
directory storage pool and register it as a storage volume

If I place the file in the directory then try to create a storage volume 
to reference it the operation fails due to the file already existing.
however if I try to create the storage pool then copy the file to the 
directory it fails to create the storage pool because the file does not 
exist.
I figured out that if I create the file in another directory then create 
the storage volume as follows...

     def create_volume(self, vol_name, src_path):
         def create_vol_xml(name, src):
             xml = """<volume type='file'>
                       <name>{name}</name>
                       <allocation
unit='bytes'>{allocation}</allocation>
                       <capacity
unit='bytes'>{capacity}</capacity>
                       <source>
                         <path>{src_path}</path>
                       </source>
                       <target>
                         <permissions>
                           <mode>{mode}</mode>
                           <owner>{uid}</owner>
                           <group>{gid}</group>
                         </permissions>
                       </target>
                      </volume>"""
             stat = os.stat(src)
             qmeu_info = images.qemu_img_info(src)
             return xml.format(**{'name': name,
                                  'capacity': qmeu_info.virtual_size,
                                  'allocation': qmeu_info.disk_size,
                                  'src_path': src,
                                  'format': qmeu_info.file_format,
                                  'mode': oct(stat.st_mode)[-3:],
                                  'uid': stat.st_uid,
                                  'gid': stat.st_gid})

         vol_xml = create_vol_xml(vol_name, src_path)
         self._vols[vol_name] = self._pool.createXML(vol_xml, 0)

Where 'src_path' is the path to a file in another directory it seems to 
copy src_path to my storage pool directory as 'vol_name' and
creates a storage volume that references it.

This is fine except, my concern is what if the file is very large the 
copy operation could take a long time and we'd use twice the
disk space we should do.

Since I could not find any documentation on the <source> elements for a 
directory pool's volume xml I was wondering if there
was a setting that would make it move/rename src_file to the vol_name 
file in my pool directory rather than copy it?


-- 
Paul Carlton
Software Engineer
Cloud Services
Hewlett Packard
BUK03:T242
Longdown Avenue
Stoke Gifford
Bristol BS34 8QZ

Mobile:    +44 (0)7768 994283
Email:    mailto:paul.carlton2@hpe.com
Hewlett-Packard Limited registered Office: Cain Road, Bracknell, Berks RG12 1HN
Registered No: 690597 England.
The contents of this message and any attachments to it are confidential and may
be legally privileged. If you have received this message in error, you should
delete it from your system immediately and advise the sender. To any recipient
of this message within HP, unless otherwise stated you should consider this
message and attachments as "HP CONFIDENTIAL".

On 05/04/2016 09:25 AM, Paul Carlton wrote:
> 
> 
> On 04/05/16 14:07, Cole Robinson wrote:
>> On 05/04/2016 06:54 AM, Paul Carlton wrote:
>>> Hi
>>>
>>> I'm trying to create a volume in an existing storage pool using
python by
>>> calling createXML() on the pool object.
>>> If the file that is the subject of the new volume exists you get
and error,
>>> also if it doesn't exist
>>>
>>> I worked out that specifying
>>>
>>> <source>
>>>    <path>path to copy of file</path>
>>> </source>
>>>
>>>   works, seemingly by copying the 'source' file to the
intended location of
>>> the
>>> volume file
>>>
>>> However I am concerned about the overhead of copying the file, is
there a way
>>> to tell it to move the file (i.e. rename)
>> Unfortunately there isn't any simple API to 'rename' a
storage volume
>>
>> - Cole
>>
> Actually I don't want to rename a storage pool, in my inept attempt a
brevity
> I failed to explain myself clearly.
> 
> I want to create a raw format file in a directory that is referenced by
> directory storage pool and register it as a storage volume
> 
> If I place the file in the directory then try to create a storage volume to
> reference it the operation fails due to the file already existing.
> however if I try to create the storage pool then copy the file to the
> directory it fails to create the storage pool because the file does not
exist.
> I figured out that if I create the file in another directory then create
the
> storage volume as follows...
> 
Not sure if I fully follow, but if you manually create the file in the
directory, outside of libvirt's APIs, just do a pool.refresh() and libvirt
will notice the new volume and it will show up in listVolumes. createXML is
only for telling libvirt to create an entirely new disk image (likely by
invoking qemu-img), not to teach it about an existing image... that's what
pool.refresh() is essentially for

- Cole
>     def create_volume(self, vol_name, src_path):
>         def create_vol_xml(name, src):
>             xml = """<volume type='file'>
>                       <name>{name}</name>
>                       <allocation
unit='bytes'>{allocation}</allocation>
>                       <capacity
unit='bytes'>{capacity}</capacity>
>                       <source>
>                         <path>{src_path}</path>
>                       </source>
>                       <target>
>                         <permissions>
>                           <mode>{mode}</mode>
>                           <owner>{uid}</owner>
>                           <group>{gid}</group>
>                         </permissions>
>                       </target>
>                      </volume>"""
>             stat = os.stat(src)
>             qmeu_info = images.qemu_img_info(src)
>             return xml.format(**{'name': name,
>                                  'capacity':
qmeu_info.virtual_size,
>                                  'allocation': qmeu_info.disk_size,
>                                  'src_path': src,
>                                  'format': qmeu_info.file_format,
>                                  'mode': oct(stat.st_mode)[-3:],
>                                  'uid': stat.st_uid,
>                                  'gid': stat.st_gid})
> 
>         vol_xml = create_vol_xml(vol_name, src_path)
>         self._vols[vol_name] = self._pool.createXML(vol_xml, 0)
> 
> Where 'src_path' is the path to a file in another directory it
seems to copy
> src_path to my storage pool directory as 'vol_name' and
> creates a storage volume that references it.
> 
> This is fine except, my concern is what if the file is very large the copy
> operation could take a long time and we'd use twice the
> disk space we should do.
> 
> Since I could not find any documentation on the <source> elements for
a
> directory pool's volume xml I was wondering if there
> was a setting that would make it move/rename src_file to the vol_name file
in
> my pool directory rather than copy it?
> 
>

On 04/05/16 14:30, Cole Robinson wrote:
> On 05/04/2016 09:25 AM, Paul Carlton wrote:
>>
>> On 04/05/16 14:07, Cole Robinson wrote:
>>> On 05/04/2016 06:54 AM, Paul Carlton wrote:
>>>> Hi
>>>>
>>>> I'm trying to create a volume in an existing storage pool
using python by
>>>> calling createXML() on the pool object.
>>>> If the file that is the subject of the new volume exists you
get and error,
>>>> also if it doesn't exist
>>>>
>>>> I worked out that specifying
>>>>
>>>> <source>
>>>>     <path>path to copy of file</path>
>>>> </source>
>>>>
>>>>    works, seemingly by copying the 'source' file to the
intended location of
>>>> the
>>>> volume file
>>>>
>>>> However I am concerned about the overhead of copying the file,
is there a way
>>>> to tell it to move the file (i.e. rename)
>>> Unfortunately there isn't any simple API to 'rename' a
storage volume
>>>
>>> - Cole
>>>
>> Actually I don't want to rename a storage pool, in my inept attempt
a brevity
>> I failed to explain myself clearly.
>>
>> I want to create a raw format file in a directory that is referenced by
>> directory storage pool and register it as a storage volume
>>
>> If I place the file in the directory then try to create a storage
volume to
>> reference it the operation fails due to the file already existing.
>> however if I try to create the storage pool then copy the file to the
>> directory it fails to create the storage pool because the file does not
exist.
>> I figured out that if I create the file in another directory then
create the
>> storage volume as follows...
>>
> Not sure if I fully follow, but if you manually create the file in the
> directory, outside of libvirt's APIs, just do a pool.refresh() and
libvirt
> will notice the new volume and it will show up in listVolumes. createXML is
> only for telling libvirt to create an entirely new disk image (likely by
> invoking qemu-img), not to teach it about an existing image... that's
what
> pool.refresh() is essentially for
>
> - Cole
>
>>      def create_volume(self, vol_name, src_path):
>>          def create_vol_xml(name, src):
>>              xml = """<volume type='file'>
>>                        <name>{name}</name>
>>                        <allocation
unit='bytes'>{allocation}</allocation>
>>                        <capacity
unit='bytes'>{capacity}</capacity>
>>                        <source>
>>                          <path>{src_path}</path>
>>                        </source>
>>                        <target>
>>                          <permissions>
>>                            <mode>{mode}</mode>
>>                            <owner>{uid}</owner>
>>                            <group>{gid}</group>
>>                          </permissions>
>>                        </target>
>>                       </volume>"""
>>              stat = os.stat(src)
>>              qmeu_info = images.qemu_img_info(src)
>>              return xml.format(**{'name': name,
>>                                   'capacity':
qmeu_info.virtual_size,
>>                                   'allocation':
qmeu_info.disk_size,
>>                                   'src_path': src,
>>                                   'format':
qmeu_info.file_format,
>>                                   'mode':
oct(stat.st_mode)[-3:],
>>                                   'uid': stat.st_uid,
>>                                   'gid': stat.st_gid})
>>
>>          vol_xml = create_vol_xml(vol_name, src_path)
>>          self._vols[vol_name] = self._pool.createXML(vol_xml, 0)
>>
>> Where 'src_path' is the path to a file in another directory it
seems to copy
>> src_path to my storage pool directory as 'vol_name' and
>> creates a storage volume that references it.
>>
>> This is fine except, my concern is what if the file is very large the
copy
>> operation could take a long time and we'd use twice the
>> disk space we should do.
>>
>> Since I could not find any documentation on the <source> elements
for a
>> directory pool's volume xml I was wondering if there
>> was a setting that would make it move/rename src_file to the vol_name
file in
>> my pool directory rather than copy it?
>>
>>
Cole,

Thanks, that is what I will use!

-- 
Paul Carlton
Software Engineer
Cloud Services
Hewlett Packard
BUK03:T242
Longdown Avenue
Stoke Gifford
Bristol BS34 8QZ

Mobile:    +44 (0)7768 994283
Email:    mailto:paul.carlton2@hpe.com
Hewlett-Packard Limited registered Office: Cain Road, Bracknell, Berks RG12 1HN
Registered No: 690597 England.
The contents of this message and any attachments to it are confidential and may
be legally privileged. If you have received this message in error, you should
delete it from your system immediately and advise the sender. To any recipient
of this message within HP, unless otherwise stated you should consider this
message and attachments as "HP CONFIDENTIAL".