libvirt users - Jan 2020 - Re: Does Libvirt's json parser support single quoted string in qmp json string?

Hi Peter Krempa,

Thanks for your quick response, yes, I know since libvirt 5.8, the -blockdev is
used for disk by default.
However we have to maintain the old releases for the production environment for
a while, so I still need use the old way to create the disk.

    >"base" argument must be a string. this means you need double
quotes in
    >JSON ("). You also must properly escape any further JSON. > 
      
    Do you mean the base argument need be formatted like below?
   
\"base\":\"json:{\"encrypt.key-secret\":\"vol-38973xjl.secret\",\"driver\":\"qcow2\",\"file\":{\"driver\":\"file\",\"filename\":\"/pitrix/data/container/vol-38973xjl.img\"}}\"

    Please help me to point out where the examples are located? I did take a
quick look at those test files, but no one seems to be matched with my case.

Thanks,
Peter Luo

On 31/1/20, 6:52 PM, "Peter Krempa" <pkrempa@redhat.com> wrote:

    
    On Fri, Jan 31, 2020 at 18:38:33 +0800, Peter Luo wrote:
    > Hi All, 
    > 
    >  
    > 
    > I’m using qmp command via “virsh qemu-monitor-command” to perform block
related jobs.
    > 
    > And in my case, I execute block-commit to commit the changes in the top
image to the base image which is encrypted.
    > 
    > So in the base parameter, I need a json string which is single quoted
to support the encryption parameters.
    > 
    > However, the virsh is failed to parse the whole json qmp command. 
    
    None of the above should be required with libvirt-5.10 and later when
    using qemu-4.2. Those two versions start using blockdev way of
    specifying images and encrypted images are supported.
    
    Please consider using libvirt-6.0 though as there are few fixes for
    blockdev.
    
    > 
    >  
    > 
    > Attempt 1: Single quote without backflash, failed 
    > 
    > root@host:~# virsh qemu-monitor-command i-9wdfw2x8 
"{\"execute\": \"block-commit\",
\"arguments\": { \"device\":
\"drive-virtio-disk2\", \"job-id\": \"job100\",
\"base\":'json:{\"encrypt.key-secret\":\"vol-38973xjl.secret\",\"driver\":\"qcow2\",\"file\":{\"driver\":\"file\",\"filename\":\"/pitrix/data/container/vol-38973xjl.img\"}}',
\"top\":
\"/pitrix/data/container/vol-38973xjl_ss-2tw7v0mm.img\"}}"
    > 
    > error: internal error: cannot parse json {"execute":
"block-commit", "arguments": { "device":
"drive-virtio-disk2", "job-id": "job100",
"base":'json:{"encrypt.key-secret":"vol-38973xjl.secret","driver":"qcow2","file":{"driver":"file","filename":"/pitrix/data/container/vol-38973xjl.img"}}',
"top":
"/pitrix/data/container/vol-38973xjl_ss-2tw7v0mm.img"}}: lexical
error: invalid char in json text.
    > 
    >           , "job-id": "job100",
"base":'json:{"encrypt.key-secret":"vo
    
    "base" argument must be a string. this means you need double
quotes in
    JSON ("). You also must properly escape any further JSON.
    
    Given that libvirt now supports the operation natively I'll refrain from
    elaborating on how to properly escape JSON to contain JSON strings, but
    there are few examples in the test suite.
    
    But please use the upstream supported way instead.

On Fri, Jan 31, 2020 at 19:14:46 +0800, Peter Luo wrote:
> 
> Hi Peter Krempa,
Hi, please don't top-post on techincal mailing lists.
> Thanks for your quick response, yes, I know since libvirt 5.8, the
-blockdev is used for disk by default.
> However we have to maintain the old releases for the production environment
for a while, so I still need use the old way to create the disk.

I'll again reiterate, that if you encounter any problems when using
'virsh qemu-monitor-command' and other pass-through options, those are
unsupported from upstream point of view.

If stuff breaks you are on your own!

Please re-consider using newer upstream version where we will be
able to assist you if stuff doesn't work as expected. The argument about
"production environment" is moot as long as you are going to use
'virsh qemu-monitor-command'.
>     >"base" argument must be a string. this means you need
double quotes in
>     >JSON ("). You also must properly escape any further JSON. >
>       
>     Do you mean the base argument need be formatted like below?
>    
\"base\":\"json:{\"encrypt.key-secret\":\"vol-38973xjl.secret\",\"driver\":\"qcow2\",\"file\":{\"driver\":\"file\",\"filename\":\"/pitrix/data/container/vol-38973xjl.img\"}}\"
IMPORTANT: One other thing I should have pointed out:

Using 'json:{}' pseudo-uris as an arguments for the block job commands
most probably isn't going to work either way. qemu isn't able to
properly match it against the internal representation and the only way
to fix it is use node-names. [1]
>     Please help me to point out where the examples are located? I did take
a quick look at those test files, but no one seems to be matched with my case.
I misremembered and there aren't any great examples in our code.

At any rate the "base" argument requires a string:

{ "base":"BASESTR" } ...

Since you want to use the JSON pseudo-protocol syntax that qemu uses you
want to use the following as "BASESTR":

json:{"driver":"qcow2","file":"blah"}

That means you have to escape the doulble quotes in the string, but
since it's supposed to be a string itself it _must_ be enclosed in
double quotes.

{
"base":"json:{\"driver\":\"qcow2\",\"file\":\"blah\"}"
}

And all of that obviously need sto be shell-escaped as well, but that
should be simple by using single quote:

'{
"base":"json:{\"driver\":\"qcow2\",\"file\":\"blah\"}"
}'


But as stated above I don't think it will work in qemu even if you
escape it properly. Please just use newer libvirt.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1785939
(commenting in that bug won't help the now accepted solution is to use
blockdev and node-names)

[2] https://www.json.org/json-en.html
> 
> Thanks,
> Peter Luo
> 
> On 31/1/20, 6:52 PM, "Peter Krempa" <pkrempa@redhat.com>
wrote:
> 
>     
>     On Fri, Jan 31, 2020 at 18:38:33 +0800, Peter Luo wrote:
>     > Hi All, 
>     > 
>     >  
>     > 
>     > I’m using qmp command via “virsh qemu-monitor-command” to perform
block related jobs.
>     > 
>     > And in my case, I execute block-commit to commit the changes in
the top image to the base image which is encrypted.
>     > 
>     > So in the base parameter, I need a json string which is single
quoted to support the encryption parameters.
>     > 
>     > However, the virsh is failed to parse the whole json qmp command. 
>     
>     None of the above should be required with libvirt-5.10 and later when
>     using qemu-4.2. Those two versions start using blockdev way of
>     specifying images and encrypted images are supported.
>     
>     Please consider using libvirt-6.0 though as there are few fixes for
>     blockdev.
>     
>     > 
>     >  
>     > 
>     > Attempt 1: Single quote without backflash, failed 
>     > 
>     > root@host:~# virsh qemu-monitor-command i-9wdfw2x8 
"{\"execute\": \"block-commit\",
\"arguments\": { \"device\":
\"drive-virtio-disk2\", \"job-id\": \"job100\",
\"base\":'json:{\"encrypt.key-secret\":\"vol-38973xjl.secret\",\"driver\":\"qcow2\",\"file\":{\"driver\":\"file\",\"filename\":\"/pitrix/data/container/vol-38973xjl.img\"}}',
\"top\":
\"/pitrix/data/container/vol-38973xjl_ss-2tw7v0mm.img\"}}"
>     > 
>     > error: internal error: cannot parse json {"execute":
"block-commit", "arguments": { "device":
"drive-virtio-disk2", "job-id": "job100",
"base":'json:{"encrypt.key-secret":"vol-38973xjl.secret","driver":"qcow2","file":{"driver":"file","filename":"/pitrix/data/container/vol-38973xjl.img"}}',
"top":
"/pitrix/data/container/vol-38973xjl_ss-2tw7v0mm.img"}}: lexical
error: invalid char in json text.
>     > 
>     >           , "job-id": "job100",
"base":'json:{"encrypt.key-secret":"vo
>     
>     "base" argument must be a string. this means you need double
quotes in
>     JSON ("). You also must properly escape any further JSON.
>     
>     Given that libvirt now supports the operation natively I'll refrain
from
>     elaborating on how to properly escape JSON to contain JSON strings, but
>     there are few examples in the test suite.
>     
>     But please use the upstream supported way instead.
>     
>     
>     
>     
> 
> 
>