libvirt users - Jan 2020 - Does Libvirt's json parser support single quoted string in qmp json string?

Hi All, 

 

I’m using qmp command via “virsh qemu-monitor-command” to perform block related
jobs.

And in my case, I execute block-commit to commit the changes in the top image to
the base image which is encrypted.

So in the base parameter, I need a json string which is single quoted to support
the encryption parameters.

However, the virsh is failed to parse the whole json qmp command. 

 

Attempt 1: Single quote without backflash, failed 

root@host:~# virsh qemu-monitor-command i-9wdfw2x8 
"{\"execute\": \"block-commit\",
\"arguments\": { \"device\":
\"drive-virtio-disk2\", \"job-id\": \"job100\",
\"base\":'json:{\"encrypt.key-secret\":\"vol-38973xjl.secret\",\"driver\":\"qcow2\",\"file\":{\"driver\":\"file\",\"filename\":\"/pitrix/data/container/vol-38973xjl.img\"}}',
\"top\":
\"/pitrix/data/container/vol-38973xjl_ss-2tw7v0mm.img\"}}"

error: internal error: cannot parse json {"execute":
"block-commit", "arguments": { "device":
"drive-virtio-disk2", "job-id": "job100",
"base":'json:{"encrypt.key-secret":"vol-38973xjl.secret","driver":"qcow2","file":{"driver":"file","filename":"/pitrix/data/container/vol-38973xjl.img"}}',
"top":
"/pitrix/data/container/vol-38973xjl_ss-2tw7v0mm.img"}}: lexical
error: invalid char in json text.

          , "job-id": "job100",
"base":'json:{"encrypt.key-secret":"vo

                     (right here) ------^

 

Attempt 2: Single quote with backflash, failed

root@host:~# virsh qemu-monitor-command i-9wdfw2x8 
"{\"execute\": \"block-commit\",
\"arguments\": { \"device\":
\"drive-virtio-disk2\", \"job-id\": \"job100\",
\"base\":\'json:{\"encrypt.key-secret\":\"vol-38973xjl.secret\",\"driver\":\"qcow2\",\"file\":{\"driver\":\"file\",\"filename\":\"/pitrix/data/container/vol-38973xjl.img\"}}\',
\"top\":
\"/pitrix/data/container/vol-38973xjl_ss-2tw7v0mm.img\"}}"

error: internal error: cannot parse json {"execute":
"block-commit", "arguments": { "device":
"drive-virtio-disk2", "job-id": "job100",
"base":\'json:{"encrypt.key-secret":"vol-38973xjl.secret","driver":"qcow2","file":{"driver":"file","filename":"/pitrix/data/container/vol-38973xjl.img"}}\',
"top":
"/pitrix/data/container/vol-38973xjl_ss-2tw7v0mm.img"}}: lexical
error: invalid char in json text.

          , "job-id": "job100",
"base":\'json:{"encrypt.key-secret":"v

                     (right here) ------^ 

 

 

My libvirt version is 5.7.0, libyajl is 2.1.0.

 

Thanks,

Peter Luo

On Fri, Jan 31, 2020 at 18:38:33 +0800, Peter Luo wrote:
> Hi All, 
> 
>  
> 
> I’m using qmp command via “virsh qemu-monitor-command” to perform block
related jobs.
> 
> And in my case, I execute block-commit to commit the changes in the top
image to the base image which is encrypted.
> 
> So in the base parameter, I need a json string which is single quoted to
support the encryption parameters.
> 
> However, the virsh is failed to parse the whole json qmp command. 
None of the above should be required with libvirt-5.10 and later when
using qemu-4.2. Those two versions start using blockdev way of
specifying images and encrypted images are supported.

Please consider using libvirt-6.0 though as there are few fixes for
blockdev.
> 
>  
> 
> Attempt 1: Single quote without backflash, failed 
> 
> root@host:~# virsh qemu-monitor-command i-9wdfw2x8 
"{\"execute\": \"block-commit\",
\"arguments\": { \"device\":
\"drive-virtio-disk2\", \"job-id\": \"job100\",
\"base\":'json:{\"encrypt.key-secret\":\"vol-38973xjl.secret\",\"driver\":\"qcow2\",\"file\":{\"driver\":\"file\",\"filename\":\"/pitrix/data/container/vol-38973xjl.img\"}}',
\"top\":
\"/pitrix/data/container/vol-38973xjl_ss-2tw7v0mm.img\"}}"
> 
> error: internal error: cannot parse json {"execute":
"block-commit", "arguments": { "device":
"drive-virtio-disk2", "job-id": "job100",
"base":'json:{"encrypt.key-secret":"vol-38973xjl.secret","driver":"qcow2","file":{"driver":"file","filename":"/pitrix/data/container/vol-38973xjl.img"}}',
"top":
"/pitrix/data/container/vol-38973xjl_ss-2tw7v0mm.img"}}: lexical
error: invalid char in json text.
> 
>           , "job-id": "job100",
"base":'json:{"encrypt.key-secret":"vo
"base" argument must be a string. this means you need double quotes in
JSON ("). You also must properly escape any further JSON.

Given that libvirt now supports the operation natively I'll refrain from
elaborating on how to properly escape JSON to contain JSON strings, but
there are few examples in the test suite.

But please use the upstream supported way instead.

On 1/31/20 4:38 AM, Peter Luo wrote:
> error: internal error: cannot parse json {"execute":
"block-commit", "arguments": { "device":
"drive-virtio-disk2", "job-id": "job100",
"base":'json:{"encrypt.key-secret":"vol-38973xjl.secret","driver":"qcow2","file":{"driver":"file","filename":"/pitrix/data/container/vol-38973xjl.img"}}',
"top":
"/pitrix/data/container/vol-38973xjl_ss-2tw7v0mm.img"}}: lexical
error: invalid char in json text.
> 
>            , "job-id": "job100",
"base":'json:{"encrypt.key-secret":"vo
> 
>                       (right here) ------^
> 
qemu's QMP language has an extension where it accepts 'string' in 
addition to the JSON "string".  But it uses a hand-rolled parser, so
it
can do whatever it wants.

But virsh uses a 3rd-party JSON parser. When using 'virsh 
qemu-monitor-command', your command line argument HAS to be valid JSON, 
or virsh can't parse it; and if virsh can't parse it, libvirt can't
pass
it on to qemu (even if qemu would have parsed it as written).  To write 
JSON inside JSON, you have to do things like:

"base":"json:{\"encrypt.key-secret\":\"vo...
>   
> 
> Attempt 2: Single quote with backflash, failed
> 
> root@host:~# virsh qemu-monitor-command i-9wdfw2x8 
"{\"execute\": \"block-commit\",
\"arguments\": { \"device\":
\"drive-virtio-disk2\", \"job-id\": \"job100\",
\"base\":\'json:{\"encrypt.key-secret\":\"vol-38973xjl.secret\",\"driver
In shell quoting, "\'" and "'" are identical.  The \
never reached
libvirt.  Your choices for proper shell quoting include:

'{"execute":...
"base":"json:{\"encrypt.key-secret\":\"vol...
"{\"execute\":...
\"base\":\"json:{\\\"encrypt.key-secret\\\":\\\"vol...

or even things like:
q='"'
qq='\"'
"{${q}execute${q}:... 
${q}base${q}:${q}json:{${dq}encrypt.key-secret${dq}:${dq}vol...

But as mentioned elsewhere in this thread, embedding nested json via 
qemu-monitor-command is already a sign that you are using unsupported 
means, and where newer libvirt supports what you want to do natively, 
you're better off upgrading to that supported method instead of trying 
to hack around the command line through an unsupported backdoor.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

On Fri, Jan 31, 2020 at 06:44:42AM -0600, Eric Blake
wrote:
> On 1/31/20 4:38 AM, Peter Luo wrote:
> 
> > error: internal error: cannot parse json {"execute":
"block-commit", "arguments": { "device":
"drive-virtio-disk2", "job-id": "job100",
"base":'json:{"encrypt.key-secret":"vol-38973xjl.secret","driver":"qcow2","file":{"driver":"file","filename":"/pitrix/data/container/vol-38973xjl.img"}}',
"top":
"/pitrix/data/container/vol-38973xjl_ss-2tw7v0mm.img"}}: lexical
error: invalid char in json text.
> > 
> >            , "job-id": "job100",
"base":'json:{"encrypt.key-secret":"vo
> > 
> >                       (right here) ------^
> > 
> 
> qemu's QMP language has an extension where it accepts 'string'
in addition
> to the JSON "string".  But it uses a hand-rolled parser, so it
can do
> whatever it wants.
Can we deprecate & remove this extension in QEMU ?

If we're going to call QMP protocol JSON, then IMHO QEMU should follow 
the JSON spec as closely as possible, without extensions.

On the flip side, if we're going to support extensions like single quoting, 
then we should make it clear to applications that this is not really JSON 
and that they need to provide an impl that is 100% matching QEMU's dialect.
This effectively means they need just import a copy of QEMU's code.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|