Ian Campbell
2011-Oct-07 10:27 UTC
[Xen-devel] [PATCH 0 of 4 v2] libxl: support json for pretty printing objects
Now that Anthony''s QMP series is in we can build upon the use of YAJL to add support for pretty printing libxl objects as JSON. Also includes a user in xl (to print disks on dry run) and an associated fix to the check-xl-disk-parse test script. This is the second posting. A subset of the patches have been applied so I have rebased. I have also addressed most of Ian J''s review comments on "libxl: IDL: autogenerate functions to produce JSON from libxl data structures": - wrap python code to 80 columns - fixup quoting (although I had to add another case in \"%s=%#x\" because "%s=%#x" was confusing emacs'' syntax highlighting due to the # being treated as a comment and hiding the closing ". - remove leftover cruft - added libxl__yagl_gen_enum helper - autogenerate *_gen_json prototypes for builtins instead of handcoding them. I didn''t address the cpuid related comments. I switched to using "xl" not "/usr/sbin/xl" in that patch. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Campbell
2011-Oct-07 10:27 UTC
[Xen-devel] [PATCH 1 of 4 v2] libxl: IDL: autogenerate functions to produce JSON from libxl data structures
# HG changeset patch # User Ian Campbell <ian.campbell@citrix.com> # Date 1317981945 -3600 # Node ID 75a0a29cccbfce75ba3087cdaf53adfd1b377a11 # Parent 43b99763284fa639e47dbeb9245f0ebd1d4f2dad libxl: IDL: autogenerate functions to produce JSON from libxl data structures. Two functions are provided. TYPE_gen_json exposes an interface which is compatible with the YAGL generator infrastructure. TYPE_to_string uses this to produce a pretty printed string. The TYPE_gen_json functions are defined in a new header libxl_json.h which is not exposed via libxl.h due to the use of YAGL datatypes to avoid poluting the namespace us libxl users which don''t use the library themselves. If a libxl user is interested in integrating at the YAGL level then it should #include this file itself. Also update testidl to generate a random version of each IDL datastructure and convert it to JSON. Unfortunately this requires a libxl_ctx and therefore the test must be run on a Xen system now. Signed-off-by: Ian Campbell <ian.campbell@citrix.com> diff -r 43b99763284f -r 75a0a29cccbf tools/libxl/Makefile --- a/tools/libxl/Makefile Fri Oct 07 10:10:31 2011 +0100 +++ b/tools/libxl/Makefile Fri Oct 07 11:05:45 2011 +0100 @@ -84,14 +84,17 @@ _libxl_paths.h: genpath libxl_paths.c: _libxl_paths.h libxl.h: _libxl_types.h +libxl_json.h: _libxl_types_json.h libxl_internal.h: _libxl_types_internal.h +libxl_internal_json.h: _libxl_types_internal_json.h $(LIBXL_OBJS) $(LIBXLU_OBJS) $(XL_OBJS): libxl.h $(LIBXL_OBJS): libxl_internal.h -_libxl_type%.h _libxl_type%.c: libxl_type%.idl gentypes.py libxltypes.py - $(PYTHON) gentypes.py libxl_type$*.idl __libxl_type$*.h __libxl_type$*.c +_libxl_type%.h _libxl_type%_json.h _libxl_type%.c: libxl_type%.idl gentypes.py libxltypes.py + $(PYTHON) gentypes.py libxl_type$*.idl __libxl_type$*.h __libxl_type$*_json.h __libxl_type$*.c $(call move-if-changed,__libxl_type$*.h,_libxl_type$*.h) + $(call move-if-changed,__libxl_type$*_json.h,_libxl_type$*_json.h) $(call move-if-changed,__libxl_type$*.c,_libxl_type$*.c) libxenlight.so: libxenlight.so.$(MAJOR) @@ -140,7 +143,7 @@ install: all ln -sf libxlutil.so.$(XLUMAJOR).$(XLUMINOR) $(DESTDIR)$(LIBDIR)/libxlutil.so.$(XLUMAJOR) ln -sf libxlutil.so.$(XLUMAJOR) $(DESTDIR)$(LIBDIR)/libxlutil.so $(INSTALL_DATA) libxlutil.a $(DESTDIR)$(LIBDIR) - $(INSTALL_DATA) libxl.h _libxl_types.h libxl_utils.h libxl_uuid.h $(DESTDIR)$(INCLUDEDIR) + $(INSTALL_DATA) libxl.h libxl_json.h _libxl_types.h _libxl_types_json.h libxl_utils.h libxl_uuid.h $(DESTDIR)$(INCLUDEDIR) $(INSTALL_DATA) bash-completion $(DESTDIR)$(BASH_COMPLETION_DIR)/xl.sh .PHONY: clean diff -r 43b99763284f -r 75a0a29cccbf tools/libxl/gentest.py --- a/tools/libxl/gentest.py Fri Oct 07 10:10:31 2011 +0100 +++ b/tools/libxl/gentest.py Fri Oct 07 11:05:45 2011 +0100 @@ -5,6 +5,7 @@ import re import random import libxltypes + def randomize_char(c): if random.random() < 0.5: return str.lower(c) @@ -15,6 +16,55 @@ def randomize_case(s): r = [randomize_char(c) for c in s] return "".join(r) +def randomize_enum(e): + return random.choice([v.name for v in e.values]) + +handcoded = ["libxl_cpumap", "libxl_key_value_list", + "libxl_cpuid_policy_list", "libxl_file_reference", + "libxl_string_list", "libxl_cpuarray"] + +def gen_rand_init(ty, v, indent = " ", parent = None): + s = "" + if isinstance(ty, libxltypes.Enumeration): + s += "%s = %s;\n" % (ty.pass_arg(v, parent is None), randomize_enum(ty)) + elif isinstance(ty, libxltypes.KeyedUnion): + if parent is None: + raise Exception("KeyedUnion type must have a parent") + s += "switch (%s) {\n" % (parent + ty.keyvar_name) + for f in ty.fields: + (nparent,fexpr) = ty.member(v, f, parent is None) + s += "case %s:\n" % f.enumname + s += gen_rand_init(f.type, fexpr, indent + " ", nparent) + s += " break;\n" + s += "}\n" + elif isinstance(ty, libxltypes.Struct) \ + and (parent is None or ty.json_fn is None): + for f in [f for f in ty.fields if not f.const]: + (nparent,fexpr) = ty.member(v, f, parent is None) + s += gen_rand_init(f.type, fexpr, "", nparent) + elif hasattr(ty, "rand_init") and ty.rand_init is not None: + s += "%s(%s);\n" % (ty.rand_init, + ty.pass_arg(v, isref=parent is None, + passby=libxltypes.PASS_BY_REFERENCE)) + elif ty.typename in ["libxl_uuid", "libxl_mac", "libxl_hwcap"]: + s += "rand_bytes((uint8_t *)%s, sizeof(*%s));\n" % (v,v) + elif ty.typename in ["libxl_domid"] or isinstance(ty, libxltypes.Number): + s += "%s = rand() %% (sizeof(%s)*8);\n" % \ + (ty.pass_arg(v, parent is None), + ty.pass_arg(v, parent is None)) + elif ty.typename in ["bool"]: + s += "%s = rand() %% 2;\n" % v + elif ty.typename in ["char *"]: + s += "%s = rand_str();\n" % v + elif ty.typename in handcoded: + raise Exception("Gen for handcoded %s" % ty.typename) + else: + raise Exception("Cannot randomly init %s" % ty.typename) + + if s != "": + s = indent + s + return s.replace("\n", "\n%s" % indent).rstrip(indent) + if __name__ == ''__main__'': if len(sys.argv) < 3: print >>sys.stderr, "Usage: gentest.py <idl> <implementation>" @@ -23,27 +73,208 @@ if __name__ == ''__main__'': random.seed() idl = sys.argv[1] - (_,types) = libxltypes.parse(idl) + (builtins,types) = libxltypes.parse(idl) impl = sys.argv[2] f = open(impl, "w") f.write(""" #include <stdio.h> -#include \"libxl.h\" +#include <stdlib.h> +#include <string.h> +#include "libxl.h" +#include "libxl_utils.h" + +static char *rand_str(void) +{ + int i, sz = rand() % 32; + char *s = malloc(sz+1); + for (i=0; i<sz; i++) + s[i] = ''a'' + (rand() % 26); + s[i] = ''\\0''; + return s; +} + +static void rand_bytes(uint8_t *p, size_t sz) +{ + int i; + for (i=0; i<sz; i++) + p[i] = rand() % 256; +} + +static void libxl_cpumap_rand_init(libxl_cpumap *cpumap) +{ + int i; + cpumap->size = rand() % 16; + cpumap->map = calloc(cpumap->size, sizeof(*cpumap->map)); + libxl_for_each_cpu(i, *cpumap) { + if (rand() % 2) + libxl_cpumap_set(cpumap, i); + else + libxl_cpumap_reset(cpumap, i); + } +} + +static void libxl_key_value_list_rand_init(libxl_key_value_list *pkvl) +{ + int i, nr_kvp = rand() % 16; + libxl_key_value_list kvl = calloc(nr_kvp+1, 2*sizeof(char *)); + + for (i = 0; i<2*nr_kvp; i += 2) { + kvl[i] = rand_str(); + if (rand() % 8) + kvl[i+1] = rand_str(); + else + kvl[i+1] = NULL; + } + kvl[i] = NULL; + kvl[i+1] = NULL; + *pkvl = kvl; +} + +static void libxl_cpuid_policy_list_rand_init(libxl_cpuid_policy_list *pp) +{ + int i, nr_policies = rand() % 16; + struct { + const char *n; + int w; + } options[] = { + /* A random selection from libxl_cpuid_parse_config */ + {"maxleaf", 32}, + {"family", 8}, + {"model", 8}, + {"stepping", 4}, + {"localapicid", 8}, + {"proccount", 8}, + {"clflush", 8}, + {"brandid", 8}, + {"f16c", 1}, + {"avx", 1}, + {"osxsave", 1}, + {"xsave", 1}, + {"aes", 1}, + {"popcnt", 1}, + {"movbe", 1}, + {"x2apic", 1}, + {"sse4.2", 1}, + {"sse4.1", 1}, + {"dca", 1}, + {"pdcm", 1}, + {"procpkg", 6}, + }; + const int nr_options = sizeof(options)/sizeof(options[0]); + char buf[64]; + libxl_cpuid_policy_list p = NULL; + + for (i = 0; i < nr_policies; i++) { + int opt = rand() % nr_options; + int val = rand() % (1<<options[opt].w); + snprintf(buf, 64, \"%s=%#x\", options[opt].n, val); + libxl_cpuid_parse_config(&p, buf); + } + *pp = p; +} + +static void libxl_file_reference_rand_init(libxl_file_reference *p) +{ + memset(p, 0, sizeof(*p)); + if (rand() % 8) + p->path = rand_str(); +} + +static void libxl_string_list_rand_init(libxl_string_list *p) +{ + int i, nr = rand() % 16; + libxl_string_list l = calloc(nr+1, sizeof(char *)); + + for (i = 0; i<nr; i++) { + l[i] = rand_str(); + } + l[i] = NULL; + *p = l; +} + +static void libxl_cpuarray_rand_init(libxl_cpuarray *p) +{ + int i; + /* Up to 16 VCPUs on 32 PCPUS */ + p->entries = rand() % 16; + p->array = calloc(p->entries, sizeof(*p->array)); + for (i = 0; i < p->entries; i++) { + int r = rand() % 32*1.5; /* 2:1 valid:invalid */ + if (r >= 32) + p->array[i] = LIBXL_CPUARRAY_INVALID_ENTRY; + else + p->array[i] = r; + } +} +""") + for ty in builtins + types: + if ty.typename not in handcoded: + f.write("static void %s_rand_init(%s);\n" % \ + (ty.typename, + ty.make_arg("p", passby=libxltypes.PASS_BY_REFERENCE))) + f.write("static void %s_rand_init(%s)\n" % \ + (ty.typename, + ty.make_arg("p", passby=libxltypes.PASS_BY_REFERENCE))) + f.write("{\n") + f.write(gen_rand_init(ty, "p")) + f.write("}\n") + f.write("\n") + ty.rand_init = "%s_rand_init" % ty.typename + + f.write(""" int main(int argc, char **argv) { """) - for ty in [t for t in types if isinstance(t,libxltypes.Enumeration)]: + for ty in types: f.write(" %s %s_val;\n" % (ty.typename, ty.typename)) - f.write(" int rc;\n") - f.write("\n") + f.write(""" + int rc; + char *s; + xentoollog_logger_stdiostream *logger; + libxl_ctx *ctx; + logger = xtl_createlogger_stdiostream(stderr, XTL_DETAIL, 0); + if (!logger) exit(1); + + if (libxl_ctx_alloc(&ctx, LIBXL_VERSION, (xentoollog_logger*)logger)) { + fprintf(stderr, "cannot init xl context\\n"); + exit(1); + } +""") + f.write(" printf(\"Testing TYPE_to_json()\\n\");\n") + f.write(" printf(\"----------------------\\n\");\n") + f.write(" printf(\"\\n\");\n") + for ty in [t for t in types if t.json_fn is not None]: + arg = ty.typename + "_val" + f.write(" %s_rand_init(%s);\n" % (ty.typename, \ + ty.pass_arg(arg, isref=False, passby=libxltypes.PASS_BY_REFERENCE))) + f.write(" s = %s_to_json(ctx, %s);\n" % \ + (ty.typename, ty.pass_arg(arg, isref=False))) + f.write(" printf(\"%%s: %%s\\n\", \"%s\", s);\n" % ty.typename) + f.write(" if (s == NULL) abort();\n") + f.write(" free(s);\n") + if ty.destructor_fn is not None: + f.write(" %s(&%s_val);\n" % (ty.destructor_fn, ty.typename)) + f.write("\n") + + f.write(" printf(\"Testing Enumerations\\n\");\n") + f.write(" printf(\"--------------------\\n\");\n") + f.write(" printf(\"\\n\");\n") for ty in [t for t in types if isinstance(t,libxltypes.Enumeration)]: f.write(" printf(\"%s -- to string:\\n\");\n" % (ty.typename)) for v in ty.values: - f.write(" printf(\"\\t%s = %%d = \\\"%%s\\\"\\n\", %s, %s_to_string(%s));\n" %\ + f.write(" printf(\"\\t%s = %%d = \\\"%%s\\\"\\n\", " \ + "%s, %s_to_string(%s));\n" % \ + (v.valuename, v.name, ty.typename, v.name)) + f.write("\n") + + f.write(" printf(\"%s -- to JSON:\\n\");\n" % (ty.typename)) + for v in ty.values: + f.write(" printf(\"\\t%s = %%d = %%s\", " \ + "%s, %s_to_json(ctx, %s));\n" %\ (v.valuename, v.name, ty.typename, v.name)) f.write("\n") @@ -54,10 +285,16 @@ int main(int argc, char **argv) f.write(" rc = %s_from_string(\"%s\", &%s_val);\n" %\ (ty.typename, n, ty.typename)) - f.write(" printf(\"\\t%s = \\\"%%s\\\" = %%d (rc %%d)\\n\", \"%s\", %s_val, rc);\n" %\ + f.write(" printf(\"\\t%s = \\\"%%s\\\" = %%d (rc %%d)\\n\", " \ + "\"%s\", %s_val, rc);\n" %\ (v, n, ty.typename)) f.write("\n") - f.write("""return 0; + f.write(""" + + libxl_ctx_free(ctx); + xtl_logger_destroy((xentoollog_logger*)logger); + + return 0; } """) diff -r 43b99763284f -r 75a0a29cccbf tools/libxl/gentypes.py --- a/tools/libxl/gentypes.py Fri Oct 07 10:10:31 2011 +0100 +++ b/tools/libxl/gentypes.py Fri Oct 07 11:05:45 2011 +0100 @@ -29,7 +29,6 @@ def libxl_C_instance_of(ty, instancename def libxl_C_type_define(ty, indent = ""): s = "" - if isinstance(ty, libxltypes.Enumeration): if ty.comment is not None: s += format_comment(0, ty.comment) @@ -76,7 +75,6 @@ def libxl_C_type_define(ty, indent = "") return s.replace("\n", "\n%s" % indent) def libxl_C_type_destroy(ty, v, indent = " ", parent = None): - s = "" if isinstance(ty, libxltypes.KeyedUnion): if parent is None: @@ -100,6 +98,60 @@ def libxl_C_type_destroy(ty, v, indent s = indent + s return s.replace("\n", "\n%s" % indent).rstrip(indent) +def libxl_C_type_gen_json(ty, v, indent = " ", parent = None): + s = "" + if parent is None: + s += "yajl_gen_status s;\n" + if isinstance(ty, libxltypes.Enumeration): + s += "s = libxl__yajl_gen_enum(hand, %s_to_string(%s));\n" % (ty.typename, ty.pass_arg(v, parent is None)) + s += "if (s != yajl_gen_status_ok)\n" + s += " goto out;\n" + elif isinstance(ty, libxltypes.KeyedUnion): + if parent is None: + raise Exception("KeyedUnion type must have a parent") + s += "switch (%s) {\n" % (parent + ty.keyvar_name) + for f in ty.fields: + (nparent,fexpr) = ty.member(v, f, parent is None) + s += "case %s:\n" % f.enumname + s += libxl_C_type_gen_json(f.type, fexpr, indent + " ", nparent) + s += " break;\n" + s += "}\n" + elif isinstance(ty, libxltypes.Struct) and (parent is None or ty.json_fn is None): + s += "s = yajl_gen_map_open(hand);\n" + s += "if (s != yajl_gen_status_ok)\n" + s += " goto out;\n" + for f in [f for f in ty.fields if not f.const]: + (nparent,fexpr) = ty.member(v, f, parent is None) + s += "s = yajl_gen_string(hand, (const unsigned char *)\"%s\", sizeof(\"%s\")-1);\n" % (f.name, f.name) + s += "if (s != yajl_gen_status_ok)\n" + s += " goto out;\n" + s += libxl_C_type_gen_json(f.type, fexpr, "", nparent) + s += "s = yajl_gen_map_close(hand);\n" + s += "if (s != yajl_gen_status_ok)\n" + s += " goto out;\n" + else: + if ty.json_fn is not None: + s += "s = %s(hand, %s);\n" % (ty.json_fn, ty.pass_arg(v, parent is None)) + s += "if (s != yajl_gen_status_ok)\n" + s += " goto out;\n" + + if parent is None: + s += "out:\n" + s += "return s;\n" + + if s != "": + s = indent + s + return s.replace("\n", "\n%s" % indent).rstrip(indent) + +def libxl_C_type_to_json(ty, v, indent = " "): + s = "" + gen = "(libxl__gen_json_callback)&%s_gen_json" % ty.typename + s += "return libxl__object_to_json(ctx, \"%s\", %s, (void *)%s);\n" % (ty.typename, gen, ty.pass_arg(v, passby=libxltypes.PASS_BY_REFERENCE)) + + if s != "": + s = indent + s + return s.replace("\n", "\n%s" % indent).rstrip(indent) + def libxl_C_enum_to_string(ty, e, indent = " "): s = "" s += "switch(%s) {\n" % e @@ -138,13 +190,13 @@ def libxl_C_enum_from_string(ty, str, e, if __name__ == ''__main__'': - if len(sys.argv) != 4: - print >>sys.stderr, "Usage: gentypes.py <idl> <header> <implementation>" + if len(sys.argv) != 5: + print >>sys.stderr, "Usage: gentypes.py <idl> <header> <header-json> <implementation>" sys.exit(1) - (_, idl, header, impl) = sys.argv + (_, idl, header, header_json, impl) = sys.argv - (_,types) = libxltypes.parse(idl) + (builtins,types) = libxltypes.parse(idl) print "outputting libxl type definitions to %s" % header @@ -167,6 +219,8 @@ if __name__ == ''__main__'': f.write(libxl_C_type_define(ty) + ";\n") if ty.destructor_fn is not None: f.write("void %s(%s);\n" % (ty.destructor_fn, ty.make_arg("p"))) + if ty.json_fn is not None: + f.write("char *%s_to_json(libxl_ctx *ctx, %s);\n" % (ty.typename, ty.make_arg("p"))) if isinstance(ty, libxltypes.Enumeration): f.write("const char *%s_to_string(%s);\n" % (ty.typename, ty.make_arg("p"))) f.write("int %s_from_string(const char *s, %s);\n" % (ty.typename, ty.make_arg("e", passby=libxltypes.PASS_BY_REFERENCE))) @@ -176,6 +230,30 @@ if __name__ == ''__main__'': f.write("""#endif /* %s */\n""" % (header_define)) f.close() + print "outputting libxl JSON definitions to %s" % header_json + + f = open(header_json, "w") + + header_json_define = header_json.upper().replace(''.'',''_'') + f.write("""#ifndef %s +#define %s + +/* + * DO NOT EDIT. + * + * This file is autogenerated by + * "%s" + */ + +""" % (header_json_define, header_json_define, " ".join(sys.argv))) + + for ty in [ty for ty in types+builtins if ty.json_fn is not None]: + f.write("yajl_gen_status %s_gen_json(yajl_gen hand, %s);\n" % (ty.typename, ty.make_arg("p", passby=libxltypes.PASS_BY_REFERENCE))) + + f.write("\n") + f.write("""#endif /* %s */\n""" % header_json_define) + f.close() + print "outputting libxl type implementations to %s" % impl f = open(impl, "w") @@ -222,5 +300,17 @@ if __name__ == ''__main__'': f.write("}\n") f.write("\n") + for ty in [t for t in types if t.json_fn is not None]: + f.write("yajl_gen_status %s_gen_json(yajl_gen hand, %s)\n" % (ty.typename, ty.make_arg("p", passby=libxltypes.PASS_BY_REFERENCE))) + f.write("{\n") + f.write(libxl_C_type_gen_json(ty, "p")) + f.write("}\n") + f.write("\n") + + f.write("char *%s_to_json(libxl_ctx *ctx, %s)\n" % (ty.typename, ty.make_arg("p"))) + f.write("{\n") + f.write(libxl_C_type_to_json(ty, "p")) + f.write("}\n") + f.write("\n") f.close() diff -r 43b99763284f -r 75a0a29cccbf tools/libxl/idl.txt --- a/tools/libxl/idl.txt Fri Oct 07 10:10:31 2011 +0100 +++ b/tools/libxl/idl.txt Fri Oct 07 11:05:45 2011 +0100 @@ -49,6 +49,15 @@ Type.autogenerate_destructor: (default: Indicates if the above named Type.destructor_fn should be autogenerated. +Type.json_fn: (default: typename + "_gen_json" or None if type == None) + + The name of the C function which will generate a YAJL data structure + representing this type. + +Type.autogenerate_json: (default: True) + + Indicates if the above named Type.json_fn should be autogenerated. + Other simple type-Classes ------------------------- diff -r 43b99763284f -r 75a0a29cccbf tools/libxl/libxl.h --- a/tools/libxl/libxl.h Fri Oct 07 10:10:31 2011 +0100 +++ b/tools/libxl/libxl.h Fri Oct 07 11:05:45 2011 +0100 @@ -199,10 +199,10 @@ typedef struct { int v; } libxl_enum_string_table; +typedef struct libxl__ctx libxl_ctx; + #include "_libxl_types.h" -typedef struct libxl__ctx libxl_ctx; - const libxl_version_info* libxl_get_version_info(libxl_ctx *ctx); typedef struct { diff -r 43b99763284f -r 75a0a29cccbf tools/libxl/libxl_internal.h --- a/tools/libxl/libxl_internal.h Fri Oct 07 10:10:31 2011 +0100 +++ b/tools/libxl/libxl_internal.h Fri Oct 07 11:05:45 2011 +0100 @@ -35,7 +35,9 @@ #include "flexarray.h" #include "libxl_utils.h" + #include "_libxl_types_internal.h" +#include "libxl_json.h" #define LIBXL_DESTROY_TIMEOUT 10 #define LIBXL_DEVICE_MODEL_START_TIMEOUT 10 @@ -362,6 +364,14 @@ _hidden char *libxl__cpupoolid_to_name(l _hidden int libxl__enum_from_string(const libxl_enum_string_table *t, const char *s, int *e); +_hidden yajl_gen_status libxl__yajl_gen_asciiz(yajl_gen hand, const char *str); + +_hidden yajl_gen_status libxl__string_gen_json(yajl_gen hand, const char *p); + +typedef yajl_gen_status (*libxl__gen_json_callback)(yajl_gen hand, void *); +_hidden char *libxl__object_to_json(libxl_ctx *ctx, const char *type, + libxl__gen_json_callback gen, void *p); + /* holds the CPUID response for a single CPUID leaf * input contains the value of the EAX and ECX register, * and each policy string contains a filter to apply to @@ -447,6 +457,7 @@ _hidden int libxl__qmp_initializations(l #include <yajl/yajl_gen.h> _hidden yajl_gen_status libxl__yajl_gen_asciiz(yajl_gen hand, const char *str); +_hidden yajl_gen_status libxl__yajl_gen_enum(yajl_gen hand, const char *str); typedef enum { JSON_ERROR, diff -r 43b99763284f -r 75a0a29cccbf tools/libxl/libxl_json.c --- a/tools/libxl/libxl_json.c Fri Oct 07 10:10:31 2011 +0100 +++ b/tools/libxl/libxl_json.c Fri Oct 07 11:05:45 2011 +0100 @@ -18,6 +18,7 @@ #include <yajl/yajl_parse.h> #include <yajl/yajl_gen.h> +#include <libxl.h> #include "libxl_internal.h" /* #define DEBUG_ANSWER */ @@ -71,6 +72,216 @@ yajl_gen_status libxl__yajl_gen_asciiz(y return yajl_gen_string(hand, (const unsigned char *)str, strlen(str)); } +yajl_gen_status libxl__yajl_gen_enum(yajl_gen hand, const char *str) +{ + if (str) + return libxl__yajl_gen_asciiz(hand, str); + else + return yajl_gen_null(hand); +} + +/* + * YAJL generators for builtin libxl types. + */ +yajl_gen_status libxl_uuid_gen_json(yajl_gen hand, + libxl_uuid *uuid) +{ + char buf[LIBXL_UUID_FMTLEN+1]; + snprintf(buf, sizeof(buf), LIBXL_UUID_FMT, LIBXL_UUID_BYTES((*uuid))); + return yajl_gen_string(hand, (const unsigned char *)buf, LIBXL_UUID_FMTLEN); +} + +yajl_gen_status libxl_cpumap_gen_json(yajl_gen hand, + libxl_cpumap *cpumap) +{ + yajl_gen_status s; + int i; + + s = yajl_gen_array_open(hand); + if (s != yajl_gen_status_ok) goto out; + + libxl_for_each_cpu(i, *cpumap) { + if (libxl_cpumap_test(cpumap, i)) { + s = yajl_gen_integer(hand, i); + if (s != yajl_gen_status_ok) goto out; + } + } + s = yajl_gen_array_close(hand); +out: + return s; +} + +yajl_gen_status libxl_key_value_list_gen_json(yajl_gen hand, + libxl_key_value_list *pkvl) +{ + libxl_key_value_list kvl = *pkvl; + yajl_gen_status s; + int i; + + s = yajl_gen_map_open(hand); + if (s != yajl_gen_status_ok) goto out; + + if (!kvl) goto empty; + + for (i = 0; kvl[i] != NULL; i += 2) { + s = libxl__yajl_gen_asciiz(hand, kvl[i]); + if (s != yajl_gen_status_ok) goto out; + if (kvl[i + 1]) + s = libxl__yajl_gen_asciiz(hand, kvl[i+1]); + else + s = yajl_gen_null(hand); + if (s != yajl_gen_status_ok) goto out; + } +empty: + s = yajl_gen_map_close(hand); +out: + return s; +} + +yajl_gen_status libxl_cpuid_policy_list_gen_json(yajl_gen hand, + libxl_cpuid_policy_list *pcpuid) +{ + libxl_cpuid_policy_list cpuid = *pcpuid; + yajl_gen_status s; + const char *input_names[2] = { "leaf", "subleaf" }; + const char *policy_names[4] = { "eax", "ebx", "ecx", "edx" }; + int i, j; + + /* + * Aiming for: + * [ + * { ''leaf'': ''val-eax'', + * ''subleaf'': ''val-ecx'', + * ''eax'': ''filter'', + * ''ebx'': ''filter'', + * ''ecx'': ''filter'', + * ''edx'': ''filter'' }, + * { ''leaf'': ''val-eax'', ..., ''eax'': ''filter'', ... }, + * ... etc ... + * ] + */ + + s = yajl_gen_array_open(hand); + if (s != yajl_gen_status_ok) goto out; + + if (cpuid == NULL) goto empty; + + for (i = 0; cpuid[i].input[0] != XEN_CPUID_INPUT_UNUSED; i++) { + s = yajl_gen_map_open(hand); + if (s != yajl_gen_status_ok) goto out; + + for (j = 0; j < 2; j++) { + if (cpuid[i].input[j] != XEN_CPUID_INPUT_UNUSED) { + s = libxl__yajl_gen_asciiz(hand, input_names[j]); + if (s != yajl_gen_status_ok) goto out; + s = yajl_gen_integer(hand, cpuid[i].input[j]); + if (s != yajl_gen_status_ok) goto out; + } + } + + for (j = 0; j < 4; j++) { + if (cpuid[i].policy[j] != NULL) { + s = libxl__yajl_gen_asciiz(hand, policy_names[j]); + if (s != yajl_gen_status_ok) goto out; + s = yajl_gen_string(hand, + (const unsigned char *)cpuid[i].policy[j], 32); + if (s != yajl_gen_status_ok) goto out; + } + } + s = yajl_gen_map_close(hand); + if (s != yajl_gen_status_ok) goto out; + } + +empty: + s = yajl_gen_array_close(hand); +out: + return s; +} + +yajl_gen_status libxl_string_list_gen_json(yajl_gen hand, libxl_string_list *pl) +{ + libxl_string_list l = *pl; + yajl_gen_status s; + int i; + + s = yajl_gen_array_open(hand); + if (s != yajl_gen_status_ok) goto out; + + if (!l) goto empty; + + for (i = 0; l[i] != NULL; i++) { + s = libxl__yajl_gen_asciiz(hand, l[i]); + if (s != yajl_gen_status_ok) goto out; + } +empty: + s = yajl_gen_array_close(hand); +out: + return s; +} + +yajl_gen_status libxl_mac_gen_json(yajl_gen hand, libxl_mac *mac) +{ + char buf[LIBXL_MAC_FMTLEN+1]; + snprintf(buf, sizeof(buf), LIBXL_MAC_FMT, LIBXL_MAC_BYTES((*mac))); + return yajl_gen_string(hand, (const unsigned char *)buf, LIBXL_MAC_FMTLEN); +} + +yajl_gen_status libxl_hwcap_gen_json(yajl_gen hand, + libxl_hwcap *p) +{ + yajl_gen_status s; + int i; + + s = yajl_gen_array_open(hand); + if (s != yajl_gen_status_ok) goto out; + + for(i=0; i<4; i++) { + s = yajl_gen_integer(hand, (*p)[i]); + if (s != yajl_gen_status_ok) goto out; + } + s = yajl_gen_array_close(hand); +out: + return s; +} + +yajl_gen_status libxl_cpuarray_gen_json(yajl_gen hand, + libxl_cpuarray *cpuarray) +{ + yajl_gen_status s; + int i; + + s = yajl_gen_array_open(hand); + if (s != yajl_gen_status_ok) goto out; + + for(i=0; i<cpuarray->entries; i++) { + if (cpuarray->array[i] == LIBXL_CPUARRAY_INVALID_ENTRY) + s = yajl_gen_null(hand); + else + s = yajl_gen_integer(hand, cpuarray->array[i]); + if (s != yajl_gen_status_ok) goto out; + } + s = yajl_gen_array_close(hand); +out: + return s; +} + +yajl_gen_status libxl_file_reference_gen_json(yajl_gen hand, + libxl_file_reference *p) +{ + if (p->path) + return libxl__yajl_gen_asciiz(hand, p->path); + else + return yajl_gen_null(hand); +} + +yajl_gen_status libxl__string_gen_json(yajl_gen hand, + const char *p) +{ + if (p) + return libxl__yajl_gen_asciiz(hand, p); + else + return yajl_gen_null(hand); +} /* * libxl__json_object helper functions @@ -558,3 +769,68 @@ libxl__json_object *libxl__json_parse(li return NULL; } } + +static const char *yajl_gen_status_to_string(yajl_gen_status s) +{ + switch (s) { + case yajl_gen_status_ok: abort(); + case yajl_gen_keys_must_be_strings: + return "keys must be strings"; + case yajl_max_depth_exceeded: + return "max depth exceeded"; + case yajl_gen_in_error_state: + return "in error state"; + case yajl_gen_generation_complete: + return "generation complete"; + case yajl_gen_invalid_number: + return "invalid number"; + case yajl_gen_no_buf: + return "no buffer"; +#if 0 /* This is in the docs but not implemented in the version I am running. */ + case yajl_gen_invalid_string: + return "invalid string"; +#endif + default: + return "unknown error"; + } +} + +char *libxl__object_to_json(libxl_ctx *ctx, const char *type, + libxl__gen_json_callback gen, void *p) +{ + yajl_gen_config conf = { 1, " " }; + const unsigned char *buf; + char *ret = NULL; + unsigned int len = 0; + yajl_gen_status s; + yajl_gen hand; + + hand = yajl_gen_alloc(&conf, NULL); + if (!hand) + return NULL; + + s = gen(hand, p); + if (s != yajl_gen_status_ok) + goto out; + + s = yajl_gen_get_buf(hand, &buf, &len); + if (s != yajl_gen_status_ok) + goto out; + ret = strdup((const char *)buf); + +out: + yajl_gen_free(hand); + + if (s != yajl_gen_status_ok) { + LIBXL__LOG(ctx, LIBXL__LOG_ERROR, + "unable to convert %s to JSON representation. " + "YAJL error code %d: %s", type, + s, yajl_gen_status_to_string(s)); + } else if (!ret) { + LIBXL__LOG(ctx, LIBXL__LOG_ERROR, + "unable to allocate space for to JSON representation of %s", + type); + } + + return ret; +} diff -r 43b99763284f -r 75a0a29cccbf tools/libxl/libxl_json.h --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/tools/libxl/libxl_json.h Fri Oct 07 11:05:45 2011 +0100 @@ -0,0 +1,23 @@ +/* + * Copyright (C) 2011 Citrix Ltd. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published + * by the Free Software Foundation; version 2.1 only. with the special + * exception on linking described in file LICENSE. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + */ + +#ifndef LIBXL_JSON_H +#define LIBXL_JSON_H + +#include <yajl/yajl_gen.h> + +#include <_libxl_types_json.h> +#include <_libxl_types_internal_json.h> + +#endif /* LIBXL_JSON_H */ diff -r 43b99763284f -r 75a0a29cccbf tools/libxl/libxl_types.idl --- a/tools/libxl/libxl_types.idl Fri Oct 07 10:10:31 2011 +0100 +++ b/tools/libxl/libxl_types.idl Fri Oct 07 11:05:45 2011 +0100 @@ -5,7 +5,7 @@ namespace("libxl_") -libxl_domid = Builtin("domid") +libxl_domid = Builtin("domid", json_fn = "yajl_gen_integer", autogenerate_json = False) libxl_uuid = Builtin("uuid", passby=PASS_BY_REFERENCE) libxl_mac = Builtin("mac", passby=PASS_BY_REFERENCE) libxl_cpumap = Builtin("cpumap", destructor_fn="libxl_cpumap_destroy", passby=PASS_BY_REFERENCE) @@ -16,7 +16,7 @@ libxl_string_list = Builtin("string_list libxl_key_value_list = Builtin("key_value_list", destructor_fn="libxl_key_value_list_destroy", passby=PASS_BY_REFERENCE) libxl_file_reference = Builtin("file_reference", destructor_fn="libxl_file_reference_destroy", passby=PASS_BY_REFERENCE) -libxl_hwcap = Builtin("hwcap") +libxl_hwcap = Builtin("hwcap", passby=PASS_BY_REFERENCE) # # Constants / Enumerations diff -r 43b99763284f -r 75a0a29cccbf tools/libxl/libxltypes.py --- a/tools/libxl/libxltypes.py Fri Oct 07 10:10:31 2011 +0100 +++ b/tools/libxl/libxltypes.py Fri Oct 07 11:05:45 2011 +0100 @@ -50,6 +50,13 @@ class Type(object): self.autogenerate_destructor = kwargs.setdefault(''autogenerate_destructor'', True) + if self.typename is not None: + self.json_fn = kwargs.setdefault(''json_fn'', self.typename + "_gen_json") + else: + self.json_fn = kwargs.setdefault(''json_fn'', None) + + self.autogenerate_json = kwargs.setdefault(''autogenerate_json'', True) + def marshal_in(self): return self.dir in [DIR_IN, DIR_BOTH] def marshal_out(self): @@ -83,6 +90,7 @@ class Builtin(Type): def __init__(self, typename, **kwargs): kwargs.setdefault(''destructor_fn'', None) kwargs.setdefault(''autogenerate_destructor'', False) + kwargs.setdefault(''autogenerate_json'', False) Type.__init__(self, typename, **kwargs) class Number(Builtin): @@ -90,6 +98,7 @@ class Number(Builtin): kwargs.setdefault(''namespace'', None) kwargs.setdefault(''destructor_fn'', None) kwargs.setdefault(''signed'', False) + kwargs.setdefault(''json_fn'', "yajl_gen_integer") self.signed = kwargs[''signed''] Builtin.__init__(self, ctype, **kwargs) @@ -163,6 +172,8 @@ class Aggregate(Type): comment = None else: n,t,const,comment = f + if n is None: + raise ValueError self.fields.append(Field(t,n,const=const,comment=comment)) # Returns a tuple (stem, field-expr) @@ -220,7 +231,10 @@ class KeyedUnion(Aggregate): # void = Builtin("void *", namespace = None) -bool = Builtin("bool", namespace = None) +bool = Builtin("bool", namespace = None, + json_fn = "yajl_gen_bool", + autogenerate_json = False) + size_t = Number("size_t", namespace = None) integer = Number("int", namespace = None, signed = True) @@ -230,7 +244,9 @@ uint16 = UInt(16) uint32 = UInt(32) uint64 = UInt(64) -string = Builtin("char *", namespace = None, destructor_fn = "free") +string = Builtin("char *", namespace = None, destructor_fn = "free", + json_fn = "libxl__string_gen_json", + autogenerate_json = False) class OrderedDict(dict): """A dictionary which remembers insertion order. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Campbell
2011-Oct-07 10:27 UTC
[Xen-devel] [PATCH 2 of 4 v2] xl: allow check-xl-disk-parse to run against installed xl as well as build dir
# HG changeset patch # User Ian Campbell <ian.campbell@citrix.com> # Date 1317981947 -3600 # Node ID 347c7631c123af9abdc5d6dbe10ea32cf5804aab # Parent 75a0a29cccbfce75ba3087cdaf53adfd1b377a11 xl: allow check-xl-disk-parse to run against installed xl as well as build dir I can''t run from the current directory since my build box isn''t running Xen so if ./xl doesn''t exist use the installed version on the assumption that I''ve copied the script to a test host. I think running from the build dir needs the blktap2 libraries, so update LD_LIBRARY_PATH as appropriate. Signed-off-by: Ian Campbell <ian.campbell@citrix.com> diff -r 75a0a29cccbf -r 347c7631c123 tools/libxl/check-xl-disk-parse --- a/tools/libxl/check-xl-disk-parse Fri Oct 07 11:05:45 2011 +0100 +++ b/tools/libxl/check-xl-disk-parse Fri Oct 07 11:05:47 2011 +0100 @@ -2,6 +2,13 @@ set -e +if [ -x ./xl ] ; then + export LD_LIBRARY_PATH=.:../libxc:../xenstore:../blktap2/control + XL=./xl +else + XL=xl +fi + fprefix=tmp.check-xl-disk-parse expected () { @@ -14,8 +21,7 @@ one () { expected_rc=$1; shift printf "test case %s...\n" "$*" set +e - LD_LIBRARY_PATH=.:../libxc:../xenstore \ - ./xl -N block-attach 0 "$@" </dev/null >$fprefix.actual 2>/dev/null + ${XL} -N block-attach 0 "$@" </dev/null >$fprefix.actual 2>/dev/null actual_rc=$? diff -u $fprefix.expected $fprefix.actual diff_rc=$? _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Campbell
2011-Oct-07 10:27 UTC
[Xen-devel] [PATCH 3 of 4 v2] xl: use libxl_device_disk_to_json to pretty print disk configuration
# HG changeset patch # User Ian Campbell <ian.campbell@citrix.com> # Date 1317982001 -3600 # Node ID 7dc93b06164542f06958a181c4747770a5552457 # Parent 347c7631c123af9abdc5d6dbe10ea32cf5804aab xl: use libxl_device_disk_to_json to pretty print disk configuration Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Ian Jackson <ian.jackson@eu.citrix.com> diff -r 347c7631c123 -r 7dc93b061645 tools/libxl/check-xl-disk-parse --- a/tools/libxl/check-xl-disk-parse Fri Oct 07 11:05:47 2011 +0100 +++ b/tools/libxl/check-xl-disk-parse Fri Oct 07 11:06:41 2011 +0100 @@ -51,15 +51,18 @@ expected </dev/null one $e foo expected <<END -disk.backend_domid = 0 -disk.pdev_path = /dev/vg/guest-volume -disk.vdev = hda -disk.backend = 0 -disk.format = 4 -disk.script = (null) -disk.removable = 0 -disk.readwrite = 1 -disk.is_cdrom = 0 +disk: { + "backend_domid": 0, + "pdev_path": "/dev/vg/guest-volume", + "vdev": "hda", + "backend": "unknown", + "format": "raw", + "script": null, + "removable": 0, + "readwrite": 1, + "is_cdrom": 0 +} + END one 0 /dev/vg/guest-volume,,hda one 0 /dev/vg/guest-volume,raw,hda,rw @@ -68,15 +71,18 @@ one 0 format=raw vdev=hda access=rw one 0 raw:/dev/vg/guest-volume,hda,w expected <<END -disk.backend_domid = 0 -disk.pdev_path = /root/image.iso -disk.vdev = hdc -disk.backend = 0 -disk.format = 4 -disk.script = (null) -disk.removable = 1 -disk.readwrite = 0 -disk.is_cdrom = 1 +disk: { + "backend_domid": 0, + "pdev_path": "/root/image.iso", + "vdev": "hdc", + "backend": "unknown", + "format": "raw", + "script": null, + "removable": 1, + "readwrite": 0, + "is_cdrom": 1 +} + END one 0 /root/image.iso,,hdc,cdrom one 0 /root/image.iso,,hdc,,cdrom diff -r 347c7631c123 -r 7dc93b061645 tools/libxl/xl_cmdimpl.c --- a/tools/libxl/xl_cmdimpl.c Fri Oct 07 11:05:47 2011 +0100 +++ b/tools/libxl/xl_cmdimpl.c Fri Oct 07 11:06:41 2011 +0100 @@ -4112,17 +4112,9 @@ int main_blockattach(int argc, char **ar disk.backend_domid = be_domid; if (dryrun_only) { - /* fixme: this should be generated from the idl */ - /* fixme: enums (backend, format) should be converted to strings */ - printf("disk.backend_domid = %"PRIx32"\n", disk.backend_domid); - printf("disk.pdev_path = %s\n", disk.pdev_path); - printf("disk.vdev = %s\n", disk.vdev); - printf("disk.backend = %d\n", disk.backend); - printf("disk.format = %d\n", disk.format); - printf("disk.script = %s\n", disk.script); - printf("disk.removable = %d\n", disk.removable); - printf("disk.readwrite = %d\n", disk.readwrite); - printf("disk.is_cdrom = %d\n", disk.is_cdrom); + char *json = libxl_device_disk_to_json(ctx, &disk); + printf("disk: %s\n", json); + free(json); if (ferror(stdout) || fflush(stdout)) { perror("stdout"); exit(-1); } return 0; } _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Campbell
2011-Oct-07 10:27 UTC
[Xen-devel] [PATCH 4 of 4 v2] libxl: add a test case for correct parsing of disk "backendtype" field
# HG changeset patch # User Ian Campbell <ian.campbell@citrix.com> # Date 1317983222 -3600 # Node ID e530b4b35d3872d93013daa4233cc467a89a1bac # Parent 7dc93b06164542f06958a181c4747770a5552457 libxl: add a test case for correct parsing of disk "backendtype" field Signed-off-by: Ian Campbell <ian.campbell@citrix.com> diff -r 7dc93b061645 -r e530b4b35d38 tools/libxl/check-xl-disk-parse --- a/tools/libxl/check-xl-disk-parse Fri Oct 07 11:06:41 2011 +0100 +++ b/tools/libxl/check-xl-disk-parse Fri Oct 07 11:27:02 2011 +0100 @@ -91,4 +91,20 @@ one 0 "format=raw, vdev=hdc, access=ro, one 0 format=raw vdev=hdc access=ro devtype=cdrom target=/root/image.iso one 0 raw:/root/image.iso,hdc:cdrom,ro +expected <<EOF +disk: { + "backend_domid": 0, + "pdev_path": "/dev/vg/guest-volume", + "vdev": "xvdb", + "backend": "phy", + "format": "raw", + "script": null, + "removable": 0, + "readwrite": 1, + "is_cdrom": 0 +} + +EOF +one 0 backendtype=phy,vdev=xvdb,access=w,target=/dev/vg/guest-volume + complete _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Campbell
2011-Oct-13 09:31 UTC
[Xen-devel] Re: [PATCH 0 of 4 v2] libxl: support json for pretty printing objects
On Fri, 2011-10-07 at 11:27 +0100, Ian Campbell wrote:> Now that Anthony''s QMP series is in we can build upon the use of YAJL > to add support for pretty printing libxl objects as JSON. > > Also includes a user in xl (to print disks on dry run) and an > associated fix to the check-xl-disk-parse test script.In my V1 posting of this series I also included "libxl: probe disk backend type in libxl_device_disk_add" but forgot it in the repost, so here is patch 5/4: # HG changeset patch # User Ian Campbell <ian.campbell@citrix.com> # Date 1318498260 -3600 # Node ID 29bd271877873f4a38217de89f8f4f74405496b5 # Parent 98cb420b606bf69e3e3177ebe8b16b529c5ed0ed libxl: probe disk backend type in libxl_device_disk_add Without this "xl block-attach" does not work. On create do_domain_create already catches this. Signed-off-by: Ian Campbell <ian.campbell@citrix.com> diff -r 98cb420b606b -r 29bd27187787 tools/libxl/libxl.c --- a/tools/libxl/libxl.c Thu Oct 13 10:31:00 2011 +0100 +++ b/tools/libxl/libxl.c Thu Oct 13 10:31:00 2011 +0100 @@ -929,6 +929,9 @@ int libxl_device_disk_add(libxl_ctx *ctx rc = libxl__device_disk_set_backend(&gc, disk); if (rc) goto out; + rc = libxl__device_disk_set_backend(&gc, disk); + if (rc) goto out; + front = flexarray_make(16, 1); if (!front) { rc = ERROR_NOMEM; _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Jackson
2011-Oct-17 15:24 UTC
[Xen-devel] Re: [PATCH 0 of 4 v2] libxl: support json for pretty printing objects
Ian Campbell writes ("[Xen-devel] Re: [PATCH 0 of 4 v2] libxl: support json for > In my V1 posting of this series I also included "libxl: probe disk> backend type in libxl_device_disk_add" but forgot it in the repost, so > here is patch 5/4:Thanks, acked and applied all 5. Ian. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel