<anthony.perard@citrix.com>
2011-Jun-17 15:32 UTC
[Xen-devel] [PATCH V4] libxl, Introduce a QMP client
From: Anthony PERARD <anthony.perard@citrix.com> QMP stands for QEMU Monitor Protocol and it is used to query information from QEMU or to control QEMU. This implementation will ask QEMU the list of chardevice and store the path to serial0 in xenstored. So we will be able to use xl console with QEMU upstream. In order to connect to the QMP server, a socket file is created in /var/run/xen/qmp-$(domid). Signed-off-by: Anthony PERARD <anthony.perard@citrix.com> --- Change v3->v4: - In case of a timeout, there is no more infinit loop - include select bit in the qmp_next function, to avoid have it twice in the code. - introduce libxl__qmp_cleanup to remove the socket file. It''s called from libxl_domain_destroy when a dm is present. - libxl__qmp_get_fd is removed as is not yet used. - qmp_connect renamed to qmp_open - I''m not skip anymore the CRLF a the end of a command. So there will always be another turn in the loop to eat them. - fix coding style issue related to struct definitions. - the receive buffer is now include in the qmp_handler struct to avoid an extra alloc. - in struct json_object, rename floating to fp. - introduce yajl_gen_asciiz that take a string nul-terminated and give it to yajl_gen_string. - use "-chardev ... -mon ..." qemu option instead of -qmp "..." Change v2->v3: - Use of a timeout when wait for a reply from the server. - Use of a command list, a list of pair "command" + callback. It''s associated with an enum. - Introduce libxl__qmp_initializations that will ask of all informations need through QMP. (It''s just a rename of libxl__qmp_get_serial_console_path from the previous patch.) Change v1->v2: - Introduction of libxl_run_dir_path(), should maybe be in another patch. - Add a new static function qmp_synchronous_send that wait the answer from the server. - QMP is know use only inside libxl, so only one command is send through the socket and after, the connection is closed. Config.mk | 1 + config/StdGNU.mk | 2 + tools/libxl/Makefile | 4 + tools/libxl/libxl.c | 4 + tools/libxl/libxl.h | 1 + tools/libxl/libxl_create.c | 4 + tools/libxl/libxl_dm.c | 9 + tools/libxl/libxl_paths.c | 4 + tools/libxl/libxl_qmp.c | 974 ++++++++++++++++++++++++++++++++++++++++++++ tools/libxl/libxl_qmp.h | 35 ++ 10 files changed, 1038 insertions(+), 0 deletions(-) create mode 100644 tools/libxl/libxl_qmp.c create mode 100644 tools/libxl/libxl_qmp.h diff --git a/Config.mk b/Config.mk index aa681ae..8b11cd8 100644 --- a/Config.mk +++ b/Config.mk @@ -133,6 +133,7 @@ define buildmakevars2file-closure echo "XEN_CONFIG_DIR=\"$(XEN_CONFIG_DIR)\"" >> $(1).tmp; \ echo "XEN_SCRIPT_DIR=\"$(XEN_SCRIPT_DIR)\"" >> $(1).tmp; \ echo "XEN_LOCK_DIR=\"$(XEN_LOCK_DIR)\"" >> $(1).tmp; \ + echo "XEN_RUN_DIR=\"$(XEN_RUN_DIR)\"" >> $(1).tmp; \ if ! cmp $(1).tmp $(1); then mv -f $(1).tmp $(1); fi endef diff --git a/config/StdGNU.mk b/config/StdGNU.mk index 25aeb4d..68fa226 100644 --- a/config/StdGNU.mk +++ b/config/StdGNU.mk @@ -52,9 +52,11 @@ PRIVATE_BINDIR = $(PRIVATE_PREFIX)/bin ifeq ($(PREFIX),/usr) CONFIG_DIR = /etc XEN_LOCK_DIR = /var/lock +XEN_RUN_DIR = /var/run/xen else CONFIG_DIR = $(PREFIX)/etc XEN_LOCK_DIR = $(PREFIX)/var/lock +XEN_RUN_DIR = $(PREFIX)/var/run/xen endif SYSCONFIG_DIR = $(CONFIG_DIR)/$(CONFIG_LEAF_DIR) diff --git a/tools/libxl/Makefile b/tools/libxl/Makefile index 84ab76f..be5445d 100644 --- a/tools/libxl/Makefile +++ b/tools/libxl/Makefile @@ -32,6 +32,9 @@ endif LIBXL_OBJS-$(CONFIG_X86) += libxl_cpuid.o LIBXL_OBJS-$(CONFIG_IA64) += libxl_nocpuid.o +LIBXL_OBJS-y += libxl_qmp.o +LIBXL_LIBS += -lyajl + LIBXL_OBJS = flexarray.o libxl.o libxl_create.o libxl_dm.o libxl_pci.o \ libxl_dom.o libxl_exec.o libxl_xshelp.o libxl_device.o \ libxl_internal.o libxl_utils.o libxl_uuid.o $(LIBXL_OBJS-y) @@ -115,6 +118,7 @@ install: all $(INSTALL_DIR) $(DESTDIR)$(LIBDIR) $(INSTALL_DIR) $(DESTDIR)$(INCLUDEDIR) $(INSTALL_DIR) $(DESTDIR)$(BASH_COMPLETION_DIR) + $(INSTALL_DIR) $(DESTDIR)$(XEN_RUN_DIR) $(INSTALL_PROG) xl $(DESTDIR)$(SBINDIR) $(INSTALL_PROG) libxenlight.so.$(MAJOR).$(MINOR) $(DESTDIR)$(LIBDIR) ln -sf libxenlight.so.$(MAJOR).$(MINOR) $(DESTDIR)$(LIBDIR)/libxenlight.so.$(MAJOR) diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c index c21cfe7..079aaab 100644 --- a/tools/libxl/libxl.c +++ b/tools/libxl/libxl.c @@ -35,6 +35,7 @@ #include "libxl_utils.h" #include "libxl_internal.h" #include "flexarray.h" +#include "libxl_qmp.h" #define PAGE_TO_MEMKB(pages) ((pages) * 4) #define BACKEND_STRING_SIZE 5 @@ -757,6 +758,9 @@ int libxl_domain_destroy(libxl_ctx *ctx, uint32_t domid, int force) if (dm_present) { if (libxl__destroy_device_model(&gc, domid) < 0) LIBXL__LOG(ctx, LIBXL__LOG_ERROR, "libxl__destroy_device_model failed for %d", domid); + + /* TODO only call this with qemu-version == qemu-xen */ + libxl__qmp_cleanup(&gc, domid); } if (libxl__devices_destroy(&gc, domid, force) < 0) LIBXL__LOG(ctx, LIBXL__LOG_ERROR, "libxl_destroy_devices failed for %d", domid); diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h index b0471c0..c3bbe87 100644 --- a/tools/libxl/libxl.h +++ b/tools/libxl/libxl.h @@ -518,6 +518,7 @@ const char *libxl_xenfirmwaredir_path(void); const char *libxl_xen_config_dir_path(void); const char *libxl_xen_script_dir_path(void); const char *libxl_lock_dir_path(void); +const char *libxl_run_dir_path(void); #endif /* LIBXL_H */ diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c index 91e2414..e818faf 100644 --- a/tools/libxl/libxl_create.c +++ b/tools/libxl/libxl_create.c @@ -30,6 +30,7 @@ #include "libxl_utils.h" #include "libxl_internal.h" #include "flexarray.h" +#include "libxl_qmp.h" void libxl_domain_config_destroy(libxl_domain_config *d_config) { @@ -514,6 +515,9 @@ static int do_domain_create(libxl__gc *gc, libxl_domain_config *d_config, } if (dm_starting) { + if (dm_info->device_model_version == LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN) { + libxl__qmp_initializations(ctx, domid); + } ret = libxl__confirm_device_model_startup(gc, dm_starting); if (ret < 0) { LIBXL__LOG(ctx, LIBXL__LOG_ERROR, diff --git a/tools/libxl/libxl_dm.c b/tools/libxl/libxl_dm.c index 5e80bc8..d528b1b 100644 --- a/tools/libxl/libxl_dm.c +++ b/tools/libxl/libxl_dm.c @@ -245,6 +245,15 @@ static char ** libxl__build_device_model_args_new(libxl__gc *gc, flexarray_vappend(dm_args, dm, "-xen-domid", libxl__sprintf(gc, "%d", info->domid), NULL); + flexarray_append(dm_args, "-chardev"); + flexarray_append(dm_args, + libxl__sprintf(gc, "socket,id=libxl-cmd,path=%s/qmp-%d,server,nowait", + libxl_run_dir_path(), + info->domid)); + + flexarray_append(dm_args, "-mon"); + flexarray_append(dm_args, "chardev=libxl-cmd,mode=control"); + if (info->type == LIBXL_DOMAIN_TYPE_PV) { flexarray_append(dm_args, "-xen-attach"); } diff --git a/tools/libxl/libxl_paths.c b/tools/libxl/libxl_paths.c index 9c2bd06..ec940e3 100644 --- a/tools/libxl/libxl_paths.c +++ b/tools/libxl/libxl_paths.c @@ -64,3 +64,7 @@ const char *libxl_lock_dir_path(void) { return XEN_LOCK_DIR; } +const char *libxl_run_dir_path(void) +{ + return XEN_RUN_DIR; +} diff --git a/tools/libxl/libxl_qmp.c b/tools/libxl/libxl_qmp.c new file mode 100644 index 0000000..40adf33 --- /dev/null +++ b/tools/libxl/libxl_qmp.c @@ -0,0 +1,974 @@ +/* + * Copyright (C) 2011 Citrix Ltd. + * Author Anthony PERARD <anthony.perard@citrix.com> + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published + * by the Free Software Foundation; version 2.1 only. with the special + * exception on linking described in file LICENSE. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + */ + +/* + * This file implement a client for QMP (QEMU Monitor Protocol). For the + * Specification, see in the QEMU repository. + */ + +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <unistd.h> + +#include <sys/un.h> +#include <sys/queue.h> + +#include <yajl/yajl_parse.h> +#include <yajl/yajl_gen.h> + +#include "libxl_internal.h" +#include "flexarray.h" +#include "libxl_qmp.h" + +/* #define DEBUG_ANSWER */ +/* #define DEBUG_RECEIVE */ + +/* + * json_object types + */ + +typedef enum { + JSON_ERROR, + JSON_NULL, + JSON_BOOL, + JSON_INTEGER, + JSON_DOUBLE, + JSON_STRING, + JSON_MAP, + JSON_ARRAY +} node_type_e; + +typedef struct json_object { + node_type_e type; + union { + bool boolean; + long integer; + double fp; + const char *string; + /* List of json_object */ + flexarray_t *array; + /* List of json_map_node */ + flexarray_t *map; + } u; + struct json_object *parent; +} json_object; + +typedef struct { + const char *map_key; + json_object *obj; +} json_map_node; + +/* + * QMP types & constant + */ + +#define QMP_RECEIVE_BUFFER_SIZE 4096 + +typedef int (*qmp_callback_t)(libxl__qmp_handler *qmp, const json_object *tree); + +typedef enum { + QMP_ANY, + QMP_QMP, + QMP_RETURN, + QMP_ERROR, + QMP_EVENT +} message_type_e; + +static struct { + const char *name; + message_type_e type; +} member_name_to_message_type[] = { + { "QMP", QMP_QMP }, + { "return", QMP_RETURN }, + { "error", QMP_ERROR }, + { "event", QMP_EVENT }, + { "", QMP_ANY }, +}; + +typedef struct callback_id_pair { + int id; + qmp_callback_t callback; + SIMPLEQ_ENTRY(callback_id_pair) next; +} callback_id_pair; + +struct libxl__qmp_handler { + struct sockaddr_un addr; + int qmp_fd; + bool connected; + time_t timeout; + /* this will be used by the synchronous send, so we know + * when we can stop and close the socket + */ + int wait_for_id; + + unsigned char buffer[QMP_RECEIVE_BUFFER_SIZE]; + yajl_handle hand; + + json_object *head; + json_object *current; + + libxl_ctx *ctx; + uint32_t domid; + + int last_id_used; + SIMPLEQ_HEAD(callback_list, callback_id_pair) callback_list; +#ifdef DEBUG_ANSWER + yajl_gen g; +#endif +}; + +static int qmp_send(libxl__qmp_handler *qmp, const char *cmd, qmp_callback_t callback); +static int qmp_synchronous_send(libxl__qmp_handler *qmp, const char *cmd, qmp_callback_t callback, int timeout); + +static const int QMP_SOCKET_CONNECT_TIMEOUT = 5; + +/* + * QMP commands + */ + +static int register_serial0_chardev_callback(libxl__qmp_handler *qmp, const json_object *tree); + +typedef struct { + const char *cmd; + qmp_callback_t callback; +} qmp_command_list_t; + +static const qmp_command_list_t qmp_command_list[] = { + [QMP_COMMAND_QUERY_CHARDEV] = { "query-chardev", register_serial0_chardev_callback }, + [QMP_COMMAND_NUMBER] = { NULL, NULL }, +}; + +/* + * json_object functions + */ + +static int json_object_append_to(libxl__qmp_handler *qmp, json_object *obj, json_object *dst) +{ + if (!dst) + return -1; + + switch (dst->type) { + case JSON_MAP: { + json_map_node *last; + + flexarray_get(dst->u.map, dst->u.map->count - 1, (void**)&last); + last->obj = obj; + break; + } + case JSON_ARRAY: + flexarray_append(dst->u.array, obj); + break; + default: + LIBXL__LOG(qmp->ctx, LIBXL__LOG_ERROR, + "error, try append an object to a %i\n", dst->type); + return -1; + } + + obj->parent = dst; + return 0; +} + +static void json_object_free(libxl__qmp_handler *qmp, json_object *obj) +{ + int index = 0; + + if (obj == NULL) + return; + switch (obj->type) { + case JSON_ERROR: + case JSON_NULL: + case JSON_BOOL: + case JSON_INTEGER: + case JSON_DOUBLE: + break; + case JSON_STRING: + free((void*)obj->u.string); + break; + case JSON_MAP: { + json_map_node *node = NULL; + + for (index = 0; index < obj->u.map->count; index++) { + if (flexarray_get(obj->u.map, index, (void**)&node) != 0) + break; + json_object_free(qmp, node->obj); + free((void*)node->map_key); + free(node); + node = NULL; + } + flexarray_free(obj->u.map); + break; + } + case JSON_ARRAY:{ + json_object *node = NULL; + break; + + for (index = 0; index < obj->u.array->count; index++) { + if (flexarray_get(obj->u.array, index, (void**)&node) != 0) + break; + json_object_free(qmp, node); + node = NULL; + } + flexarray_free(obj->u.array); + break; + } + } + free(obj); +} + +static inline const char *json_object_get_string(const json_object *o) +{ + if (o && o->type == JSON_STRING) { + return o->u.string; + } + return NULL; +} + +static const json_object *json_object_map_get(const char *key, const json_object *o) +{ + flexarray_t *maps = NULL; + int index = 0; + + if (o && o->type == JSON_MAP) { + json_map_node *node = NULL; + + maps = o->u.map; + for (index = 0; index < maps->count; index++) { + if (flexarray_get(maps, index, (void**)&node) != 0) + break; + if (strcmp(key, node->map_key) == 0) { + return node->obj; + } + } + } + return NULL; +} + +/* + * JSON callbacks + */ + +static int json_callback_null(void *opaque) +{ + libxl__qmp_handler *qmp = opaque; + json_object *obj; + +#ifdef DEBUG_ANSWER + yajl_gen_null(qmp->g); +#endif + + obj = calloc(1, sizeof (json_object)); + obj->type = JSON_NULL; + json_object_append_to(qmp, obj, qmp->current); + + return 1; +} + +static int json_callback_boolean(void *opaque, int boolean) +{ + libxl__qmp_handler *qmp = opaque; + json_object *obj; + +#ifdef DEBUG_ANSWER + yajl_gen_bool(qmp->g, boolean); +#endif + + obj = calloc(1, sizeof (json_object)); + obj->type = JSON_BOOL; + obj->u.boolean = boolean; + json_object_append_to(qmp, obj, qmp->current); + + return 1; +} + +static int json_callback_integer(void *opaque, long value) +{ + libxl__qmp_handler *qmp = opaque; + json_object *obj; + +#ifdef DEBUG_ANSWER + yajl_gen_integer(qmp->g, value); +#endif + + obj = calloc(1, sizeof (json_object)); + obj->type = JSON_INTEGER; + obj->u.integer = value; + json_object_append_to(qmp, obj, qmp->current); + + return 1; +} + +static int json_callback_double(void *opaque, double value) +{ + libxl__qmp_handler *qmp = opaque; + json_object *obj; + +#ifdef DEBUG_ANSWER + yajl_gen_double(qmp->g, value); +#endif + + obj = calloc(1, sizeof (json_object)); + obj->type = JSON_DOUBLE; + obj->u.fp = value; + json_object_append_to(qmp, obj, qmp->current); + + return 1; +} + +static int json_callback_string(void *opaque, const unsigned char *str, + unsigned int len) +{ + libxl__qmp_handler *qmp = opaque; + char *t = malloc(len + 1); + json_object *obj = NULL; + +#ifdef DEBUG_ANSWER + yajl_gen_string(qmp->g, str, len); +#endif + + strncpy(t, (const char *) str, len); + t[len] = 0; + + obj = calloc(1, sizeof (json_object)); + obj->type = JSON_STRING; + obj->u.string = t; + + json_object_append_to(qmp, obj, qmp->current); + + return 1; +} + +static int json_callback_map_key(void *opaque, const unsigned char *str, + unsigned int len) +{ + libxl__qmp_handler *qmp = opaque; + char *t = malloc(len + 1); + json_object *obj = qmp->current; + +#ifdef DEBUG_ANSWER + yajl_gen_string(qmp->g, str, len); +#endif + + strncpy(t, (const char *) str, len); + t[len] = 0; + + if (obj->type == JSON_MAP) { + json_map_node *node = malloc(sizeof (json_map_node)); + + node->map_key = t; + node->obj = NULL; + + flexarray_append(obj->u.map, node); + } else { + return 0; + } + + return 1; +} + +static int json_callback_start_map(void *opaque) +{ + libxl__qmp_handler *qmp = opaque; + json_object *obj = NULL; + +#ifdef DEBUG_ANSWER + yajl_gen_map_open(qmp->g); +#endif + + obj = calloc(1, sizeof (json_object)); + if (qmp->head == NULL) { + qmp->head = obj; + } + + obj->type = JSON_MAP; + obj->u.map = flexarray_make(1, 1); + + json_object_append_to(qmp, obj, qmp->current); + + obj->parent = qmp->current; + qmp->current = obj; + + return 1; +} + +static int json_callback_end_map(void *opaque) +{ + libxl__qmp_handler *qmp = opaque; + +#ifdef DEBUG_ANSWER + yajl_gen_map_close(qmp->g); +#endif + + if (qmp->current) { + qmp->current = qmp->current->parent; + } else { + LIBXL__LOG(qmp->ctx, LIBXL__LOG_ERROR, "no parents for a json_object"); + return 0; + } + + return 1; +} + +static int json_callback_start_array(void *opaque) +{ + libxl__qmp_handler *qmp = opaque; + json_object *obj = NULL; + +#ifdef DEBUG_ANSWER + yajl_gen_array_open(qmp->g); +#endif + + obj = calloc(1, sizeof (json_object)); + if (qmp->head == NULL) { + qmp->head = obj; + } + obj->type = JSON_ARRAY; + obj->u.array = flexarray_make(1, 1); + json_object_append_to(qmp, obj, qmp->current); + qmp->current = obj; + + return 1; +} + +static int json_callback_end_array(void *opaque) +{ + libxl__qmp_handler *qmp = opaque; + +#ifdef DEBUG_ANSWER + yajl_gen_array_close(qmp->g); +#endif + + if (qmp->current) { + qmp->current = qmp->current->parent; + } else { + LIBXL__LOG(qmp->ctx, LIBXL__LOG_ERROR, "no parents for a json_object"); + return 0; + } + + return 1; +} + +static yajl_callbacks callbacks = { + json_callback_null, + json_callback_boolean, + json_callback_integer, + json_callback_double, + NULL, + json_callback_string, + json_callback_start_map, + json_callback_map_key, + json_callback_end_map, + json_callback_start_array, + json_callback_end_array +}; + +/* + * QMP callbacks functions + */ + +static const char *get_serial0_chardev(libxl__qmp_handler *qmp, const json_object *tree) +{ + const json_object *ret = NULL; + const json_object *obj = NULL; + const json_object *label = NULL; + const char *s = NULL; + flexarray_t *array = NULL; + int index = 0; + + ret = json_object_map_get("return", tree); + + if (!ret || ret->type != JSON_ARRAY) { + return NULL; + } + array = ret->u.array; + for (index = 0; index < array->count; index++) { + if (flexarray_get(array, index, (void**)&obj) != 0) + break; + label = json_object_map_get("label", obj); + s = json_object_get_string(label); + + /* TODO Could replace serial0 by serial and get all serial ttys, if several */ + if (s && strcmp("serial0", s) == 0) { + const json_object *filename = NULL; + filename = json_object_map_get("filename", obj); + return json_object_get_string(filename); + } + }; + + return NULL; +} + +static int register_serial0_chardev_callback(libxl__qmp_handler *qmp, const json_object *tree) +{ + libxl__gc gc = LIBXL_INIT_GC(qmp->ctx); + char *path = NULL; + const char *chardev = NULL; + int ret = 0; + + chardev = get_serial0_chardev(qmp, tree); + if (!(chardev && strncmp("pty:", chardev, 4) == 0)) { + return 1; + } + + path = libxl__xs_get_dompath(&gc, qmp->domid); + path = libxl__sprintf(&gc, "%s/serial/%d/tty", path, 0); + + LIBXL__LOG(qmp->ctx, LIBXL__LOG_DEBUG, + "qmp serial0: %s (%s)", chardev + 4, path); + ret = libxl__xs_write(&gc, XBT_NULL, path, "%s", chardev + 4); + libxl__free_all(&gc); + return ret; +} + +static int qmp_capabilities_callback(libxl__qmp_handler *qmp, const json_object *tree) +{ + int ret = 0; + + qmp->connected = true; + + return ret; +} + +/* + * QMP commands + */ + +static int enable_qmp_capabilities(libxl__qmp_handler *qmp) +{ + return qmp_send(qmp, "qmp_capabilities", qmp_capabilities_callback); +} + +/* + * Helpers + */ + +static message_type_e qmp_response_type(libxl__qmp_handler *qmp, json_object *resp) +{ + if (resp && resp->type == JSON_MAP) { + flexarray_t *maps = NULL; + json_map_node *node = NULL; + int index = 0; + + maps = resp->u.map; + for (index = 0; index < maps->count; index++) { + int i = 0; + if (flexarray_get(maps, index, (void**)&node) != 0) + break; + for (i = 0; member_name_to_message_type[i].type != QMP_ANY; i++) { + if (strcmp(member_name_to_message_type[i].name, node->map_key) == 0) { + return member_name_to_message_type[i].type; + } + } + } + } + + return QMP_ANY; +} + +static callback_id_pair *qmp_get_callback_from_id(libxl__qmp_handler *qmp, json_object *o) +{ + const json_object *id_object = json_object_map_get("id", o); + int id = -1; + callback_id_pair *pp = NULL; + + if (id_object && id_object->type == JSON_INTEGER) { + id = id_object->u.integer; + + SIMPLEQ_FOREACH(pp, &qmp->callback_list, next) { + if (pp->id == id) { + return pp; + } + } + } + return NULL; +} + +static void qmp_handle_error_response(libxl__qmp_handler *qmp, json_object *resp) +{ + callback_id_pair *pp = qmp_get_callback_from_id(qmp, resp); + const json_object *error = json_object_map_get("error", resp); + const char *msg = json_object_get_string(json_object_map_get("desc", error)); + + if (pp) { + if (pp->id == qmp->wait_for_id) { + /* tell that the id have been processed */ + qmp->wait_for_id = 0; + } + SIMPLEQ_REMOVE(&qmp->callback_list, pp, callback_id_pair, next); + free(pp); + } + + LIBXL__LOG(qmp->ctx, LIBXL__LOG_ERROR, + "received an error message from QMP server: %s", + msg); +} + +static int qmp_handle_response(libxl__qmp_handler *qmp, json_object *resp) +{ + message_type_e type = QMP_ANY; + + type = qmp_response_type(qmp, resp); + LIBXL__LOG(qmp->ctx, LIBXL__LOG_DEBUG, "message type: %i", type); + + switch (type) { + case QMP_QMP: + /* On the greeting message from the server, enable qmp capabilities */ + enable_qmp_capabilities(qmp); + break; + case QMP_RETURN: { + callback_id_pair *pp = qmp_get_callback_from_id(qmp, resp); + + if (pp) { + pp->callback(qmp, resp); + if (pp->id == qmp->wait_for_id) { + /* tell that the id have been processed */ + qmp->wait_for_id = 0; + } + SIMPLEQ_REMOVE(&qmp->callback_list, pp, callback_id_pair, next); + free(pp); + } + break; + } + case QMP_ERROR: + qmp_handle_error_response(qmp, resp); + break; + case QMP_EVENT: + break; + case QMP_ANY: + return -1; + } + return 0; +} + +/* + * Handler functions + */ + +/* JSON gen helper */ +static inline yajl_gen_status yajl_gen_asciiz(yajl_gen hand, const char *str) +{ + return yajl_gen_string(hand, (const unsigned char *)str, strlen(str)); +} + +static int qmp_open(libxl__qmp_handler *qmp, const char *qmp_socket_path, + int timeout) +{ + int ret; + int i = 0; + + qmp->qmp_fd = socket(AF_UNIX, SOCK_STREAM | SOCK_NONBLOCK, 0); + if (qmp->qmp_fd < 0) { + return -1; + } + + memset(&qmp->addr, 0, sizeof (&qmp->addr)); + qmp->addr.sun_family = AF_UNIX; + strncpy(qmp->addr.sun_path, qmp_socket_path, sizeof (qmp->addr.sun_path)); + + do { + ret = connect(qmp->qmp_fd, (struct sockaddr *) &qmp->addr, + sizeof (qmp->addr)); + if (ret == 0) + break; + if (errno == ENOENT || errno == ECONNREFUSED) { + /* ENOENT : Socket may not have shown up yet + * ECONNREFUSED : Leftover socket hasn''t been removed yet */ + continue; + } + return -1; + } while ((++i / 5 <= timeout) && (usleep(200000) <= 0)); + + return ret; +} + +static void qmp_close(libxl__qmp_handler *qmp) +{ + callback_id_pair *pp = NULL; + callback_id_pair *tmp = NULL; +#ifdef DEBUG_ANSWER + if (qmp->g) + yajl_gen_free(qmp->g); +#endif + if (qmp->hand) + yajl_free(qmp->hand); + close(qmp->qmp_fd); + SIMPLEQ_FOREACH(pp, &qmp->callback_list, next) { + if (tmp) + free(tmp); + tmp = pp; + } + if (tmp) + free(tmp); +} + +static int qmp_next(libxl__qmp_handler *qmp) +{ + yajl_status status; + ssize_t rd; + ssize_t bytes_parsed = 0; + + /* read the socket */ + while (1) { + fd_set rfds; + int ret = 0; + struct timeval timeout = { + .tv_sec = qmp->timeout, + .tv_usec = 0, + }; + + FD_ZERO(&rfds); + FD_SET(qmp->qmp_fd, &rfds); + + ret = select(qmp->qmp_fd + 1, &rfds, NULL, NULL, &timeout); + if (ret > 0) { + rd = read(qmp->qmp_fd, qmp->buffer, QMP_RECEIVE_BUFFER_SIZE); + if (rd > 0) { + break; + } else if (rd < 0) { + LIBXL__LOG_ERRNO(qmp->ctx, LIBXL__LOG_ERROR, "Socket read error"); + return rd; + } + } else if (ret == 0) { + LIBXL__LOG(qmp->ctx, LIBXL__LOG_ERROR, "timeout"); + return -1; + } else if (ret < 0) { + LIBXL__LOG_ERRNO(qmp->ctx, LIBXL__LOG_ERROR, "Select error"); + return -1; + } + } +#ifdef DEBUG_RECEIVE + qmp->buffer[rd] = 0; + LIBXL__LOG(qmp->ctx, LIBXL__LOG_DEBUG, "received: ''%s''", qmp->buffer); +#endif + + while (bytes_parsed < rd) { +#ifdef DEBUG_ANSWER + if (qmp->g == NULL) { + yajl_gen_config conf = { 1, " " }; + qmp->g = yajl_gen_alloc(&conf, NULL); + } +#endif + /* parse the input */ + if (qmp->hand == NULL) { + /* allow comments */ + yajl_parser_config cfg = { 1, 1 }; + qmp->hand = yajl_alloc(&callbacks, &cfg, NULL, qmp); + } + status = yajl_parse(qmp->hand, qmp->buffer + bytes_parsed, rd - bytes_parsed); + bytes_parsed += yajl_get_bytes_consumed(qmp->hand); + + /* handle the answer */ + if (status != yajl_status_ok && status != yajl_status_insufficient_data) { + unsigned char *str = yajl_get_error(qmp->hand, 1, qmp->buffer, rd); + + LIBXL__LOG(qmp->ctx, LIBXL__LOG_ERROR, "yajl error: %s", str); + yajl_free_error(qmp->hand, str); + } + + if (status == yajl_status_ok) { +#ifdef DEBUG_ANSWER + const unsigned char *buf = NULL; + unsigned int len = 0; + + yajl_gen_get_buf(qmp->g, &buf, &len); + LIBXL__LOG(qmp->ctx, LIBXL__LOG_DEBUG, "response:\n%s", buf); + yajl_gen_free(qmp->g); + qmp->g = NULL; +#endif + + qmp_handle_response(qmp, qmp->head); + + json_object_free(qmp, qmp->head); + qmp->head = NULL; + qmp->current = NULL; + + yajl_free(qmp->hand); + qmp->hand = NULL; + } + } + return 1; +} + +static int qmp_send(libxl__qmp_handler *qmp, const char *cmd, qmp_callback_t callback) +{ + yajl_gen_config conf = { 0, NULL }; + const unsigned char *buf; + unsigned int len = 0; + yajl_gen_status s; + yajl_gen hand; + + hand = yajl_gen_alloc(&conf, NULL); + if (!hand) { + return -1; + } + + yajl_gen_map_open(hand); + yajl_gen_asciiz(hand, "execute"); + yajl_gen_asciiz(hand, cmd); + yajl_gen_asciiz(hand, "id"); + yajl_gen_integer(hand, ++qmp->last_id_used); + yajl_gen_map_close(hand); + + s = yajl_gen_get_buf(hand, &buf, &len); + + if (s) { + LIBXL__LOG(qmp->ctx, LIBXL__LOG_ERROR, + "could not generate a qmp command"); + return -1; + } + + if (callback) { + callback_id_pair *elm = malloc(sizeof (callback_id_pair)); + elm->id = qmp->last_id_used; + elm->callback = callback; + SIMPLEQ_INSERT_TAIL(&qmp->callback_list, elm, next); + } + + LIBXL__LOG(qmp->ctx, LIBXL__LOG_DEBUG, "next qmp command: ''%s''", buf); + + write(qmp->qmp_fd, buf, len); + write(qmp->qmp_fd, "\r\n", 2); + yajl_gen_free(hand); + + return qmp->last_id_used; +} + +static int qmp_synchronous_send(libxl__qmp_handler *qmp, const char *cmd, qmp_callback_t callback, int ask_timeout) +{ + int id = 0; + int ret = 0; + + id = qmp_send(qmp, cmd, callback); + if (id <= 0) { + return -1; + } + qmp->wait_for_id = id; + + while (qmp->wait_for_id == id) { + if ((ret = qmp_next(qmp)) < 0) { + return ret; + } + } + + return 0; +} + +static libxl__qmp_handler *qmp_init_handler(libxl_ctx *ctx, uint32_t domid) +{ + libxl__qmp_handler *qmp = NULL; + + qmp = calloc(1, sizeof (libxl__qmp_handler)); + qmp->ctx = ctx; + qmp->domid = domid; + qmp->timeout = 5; + + SIMPLEQ_INIT(&qmp->callback_list); + + return qmp; +} + +static void qmp_free_handler(libxl__qmp_handler *qmp) +{ + free(qmp); +} + +/* + * API + */ + +libxl__qmp_handler *libxl__qmp_initialize(libxl_ctx *ctx, uint32_t domid) +{ + int ret = 0; + libxl__qmp_handler *qmp = NULL; + char *qmp_socket; + libxl__gc gc = LIBXL_INIT_GC(ctx); + + qmp = qmp_init_handler(ctx, domid); + + qmp_socket = libxl__sprintf(&gc, "%s/qmp-%d", libxl_run_dir_path(), domid); + if ((ret = qmp_open(qmp, qmp_socket, QMP_SOCKET_CONNECT_TIMEOUT)) < 0) { + LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "Connection error"); + qmp_free_handler(qmp); + return NULL; + } + + LIBXL__LOG(qmp->ctx, LIBXL__LOG_DEBUG, "connected to %s", qmp_socket); + + /* Wait for the response to qmp_capabilities */ + while (!qmp->connected) { + if ((ret = qmp_next(qmp)) < 0) { + break; + } + } + + libxl__free_all(&gc); + return qmp; +} + +int libxl__qmp_send_command(libxl__qmp_handler *qmp, libxl__qmp_command_e command) +{ + return qmp_synchronous_send(qmp, qmp_command_list[command].cmd, qmp_command_list[command].callback, 5); +} + +int libxl__qmp_do_next(libxl__qmp_handler *qmp) +{ + int ret; + + if (!qmp) + return -1; + ret = qmp_next(qmp); + if (ret < 0) { + LIBXL__LOG_ERRNO(qmp->ctx, LIBXL__LOG_ERROR, "QMP, read error"); + } + + return ret; +} + +void libxl__qmp_close(libxl__qmp_handler *qmp) +{ + if (!qmp) + return; + qmp_close(qmp); + qmp_free_handler(qmp); +} + +void libxl__qmp_cleanup(libxl__gc *gc, uint32_t domid) +{ + libxl_ctx *ctx = libxl__gc_owner(gc); + char *qmp_socket; + + qmp_socket = libxl__sprintf(gc, "%s/qmp-%d", libxl_run_dir_path(), domid); + if (unlink(qmp_socket) == -1) { + if (errno != ENOENT) { + LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, + "Can not remove QMP socket file %s", qmp_socket); + } + } +} + +int libxl__qmp_initializations(libxl_ctx *ctx, uint32_t domid) +{ + libxl__qmp_handler *qmp = NULL; + int ret = 0; + + qmp = libxl__qmp_initialize(ctx, domid); + if (!qmp) + return -1; + if (qmp->connected) { + ret = libxl__qmp_send_command(qmp, QMP_COMMAND_QUERY_CHARDEV); + } + libxl__qmp_close(qmp); + return ret; +} diff --git a/tools/libxl/libxl_qmp.h b/tools/libxl/libxl_qmp.h new file mode 100644 index 0000000..0e142fa --- /dev/null +++ b/tools/libxl/libxl_qmp.h @@ -0,0 +1,35 @@ +/* + * Copyright (C) 2011 Citrix Ltd. + * Author Anthony PERARD <anthony.perard@citrix.com> + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published + * by the Free Software Foundation; version 2.1 only. with the special + * exception on linking described in file LICENSE. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + */ + +#include "libxl.h" + +typedef struct libxl__qmp_handler libxl__qmp_handler; +typedef enum libxl__qmp_command_e libxl__qmp_command_e; + +/* QMP Command that can be send */ +enum libxl__qmp_command_e { + QMP_COMMAND_QUERY_CHARDEV, + QMP_COMMAND_NUMBER, +}; + + +_hidden libxl__qmp_handler *libxl__qmp_initialize(libxl_ctx *ctx, + uint32_t domid); +_hidden int libxl__qmp_send_command(libxl__qmp_handler *qmp, libxl__qmp_command_e command); +_hidden int libxl__qmp_do_next(libxl__qmp_handler *qmp); +_hidden void libxl__qmp_close(libxl__qmp_handler *qmp); +_hidden void libxl__qmp_cleanup(libxl__gc *gc, uint32_t domid); + +_hidden int libxl__qmp_initializations(libxl_ctx *ctx, uint32_t domid); -- 1.7.2.5 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Jackson
2011-Jun-20 14:50 UTC
Re: [Xen-devel] [PATCH V4] libxl, Introduce a QMP client
anthony.perard@citrix.com writes ("[Xen-devel] [PATCH V4] libxl, Introduce a QMP client"):> QMP stands for QEMU Monitor Protocol and it is used to query information > from QEMU or to control QEMU.Thanks for this; it''s coming along. I have some comments.... Firstly, can you make sure all your lines are less than around 75 columns ? Otherwise they wrap badly on 80-column terminals (when viewing the code, and when replying to emails containing diffs).> + echo "XEN_RUN_DIR=\"$(XEN_RUN_DIR)\"" >> $(1).tmp; \Can we have the introduction of XEN_RUN_DIR in a separate patch ?> diff --git a/tools/libxl/libxl_qmp.h b/tools/libxl/libxl_qmp.h > new file mode 100644 > index 0000000..0e142fa > --- /dev/null > +++ b/tools/libxl/libxl_qmp.hIs this supposed to be an internal-only header file ? If so it''s not clear to me why it shouldn''t all be in libxl_internal.h.> +/* QMP Command that can be send */ > +enum libxl__qmp_command_e { > + QMP_COMMAND_QUERY_CHARDEV, > + QMP_COMMAND_NUMBER, > +};...> +_hidden int libxl__qmp_send_command(libxl__qmp_handler *qmp, libxl__qmp_command_e command);I''m not entirely convinced by this interface. Wouldn''t it be better to have a specific function for each command ? After all commands may have arguments. Something like: _hidden int libxl__qmp_send_query_chardev(libxl__qmp_handler *qmp); It''s not clear from your header file what the intended calling order of these is. I think there should be a comment in the private header file explaining what to do in what order. And if we are keeping the enums:> +typedef struct libxl__qmp_handler libxl__qmp_handler; > +typedef enum libxl__qmp_command_e libxl__qmp_command_e;I don''t think this _e suffix is very helpful. Just call it libxl__qmp_command. And our practice is not normally to declare the tag as well as the typedef for an enum. So I think you want just typedef enum { .... } libxl__qmp_command.> diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c > index c21cfe7..079aaab 100644 > --- a/tools/libxl/libxl.c > +++ b/tools/libxl/libxl.c...> + > + /* TODO only call this with qemu-version == qemu-xen */ > + libxl__qmp_cleanup(&gc, domid);I don''t think we want to add more TODOs. (And if we did, the word we should use is FIXME.) The best solution is to make this function idempotent, in which case calling it is harmless.> + flexarray_append(dm_args, "-chardev"); > + flexarray_append(dm_args, > + libxl__sprintf(gc, "socket,id=libxl-cmd,path=%s/qmp-%d,server,nowait", > + libxl_run_dir_path(), > + info->domid)); > + > + flexarray_append(dm_args, "-mon"); > + flexarray_append(dm_args, "chardev=libxl-cmd,mode=control");Yuck, what a horrible command line interface is presented here by qemu! But this is not your fault :-).> diff --git a/tools/libxl/libxl_qmp.c b/tools/libxl/libxl_qmp.c > new file mode 100644 > index 0000000..40adf33 > --- /dev/null > +++ b/tools/libxl/libxl_qmp.c > @@ -0,0 +1,974 @@...> +typedef enum { > + JSON_ERROR, > + JSON_NULL, > + JSON_BOOL, > + JSON_INTEGER, > + JSON_DOUBLE, > + JSON_STRING, > + JSON_MAP, > + JSON_ARRAY > +} node_type_e; > + > +typedef struct json_object {It''s odd to see this here in our (libxl) code. Is this kind of thing not provided by the json library ?> +typedef enum { > + QMP_ANY, > + QMP_QMP, > + QMP_RETURN, > + QMP_ERROR, > + QMP_EVENT > +} message_type_e; > + > +static struct { > + const char *name; > + message_type_e type; > +} member_name_to_message_type[] = { > + { "QMP", QMP_QMP }, > + { "return", QMP_RETURN }, > + { "error", QMP_ERROR }, > + { "event", QMP_EVENT }, > + { "", QMP_ANY }, > +};If we do need to define these, we should use a libxl idl enum (with Ian Campbell''s string-mapping-generation patch which I will apply shortly.> +/* > + * json_object functions > + */I don''t understand what these are for. They look disturbingly like boilerplate or at least rather verbose plumbing.> +static int json_object_append_to(libxl__qmp_handler *qmp, json_object *obj, json_object *dst)Does our json library not provide a pile of generic facilities for managing json objects ? Why do we need all of this code ?> +static yajl_callbacks callbacks = {Oh wait, I just looked up yajl. Why are we using a SAX-style (callback-based) json parser ? Surely it would be better to use a json library which simply provides an in-memory data structure for each json object. We don''t anticipate dealing with any huge json objects (which is the motivation for the SAX-style approach). I will stop with comments now because I don''t want to get too bogged down in details if we''re going to change the approach. Ian. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Jackson
2011-Jun-20 15:37 UTC
Re: [Xen-devel] [PATCH V4] libxl, Introduce a QMP client
I wrote:> Oh wait, I just looked up yajl. Why are we using a SAX-style > (callback-based) json parser ? > > Surely it would be better to use a json library which simply provides > an in-memory data structure for each json object. We don''t anticipate > dealing with any huge json objects (which is the motivation for the > SAX-style approach).Well, I went and did some searching and I have discovered that: There does not appear to be any widely-used "atomic-style" json library for C, other than the one which comes with glib (which is unsuitable because we don''t want to import glib into our project). YAJL 2 does have a simple "atomic" ("tree" they call it) interface. It''s not in squeeze or afaict recent fedora. YAJL 1 is even lacking in Debian lenny. :-/. So probably what we have is the best approach. But perhaps it might be worth considering whether we should make our interface to json objects the similar to yajl 1''s tree interface ? Ian. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Campbell
2011-Jun-21 13:33 UTC
Re: [Xen-devel] [PATCH V4] libxl, Introduce a QMP client
On Mon, 2011-06-20 at 15:50 +0100, Ian Jackson wrote:> > > +static struct { > > + const char *name; > > + message_type_e type; > > +} member_name_to_message_type[] = { > > + { "QMP", QMP_QMP }, > > + { "return", QMP_RETURN }, > > + { "error", QMP_ERROR }, > > + { "event", QMP_EVENT }, > > + { "", QMP_ANY }, > > +}; > > If we do need to define these, we should use a libxl idl enum (with > Ian Campbell''s string-mapping-generation patch which I will apply > shortly.Just to note that this is enum is only used internally, using the IDL as is would make it public... It''d be relatively easy to allow for internal types in the IDL but I''m not sure if we want to? I think if we did I''d be tempted to add a separate libxl_internal.idl generating (using the same generator) _libxl_types_internal.[ch] etc. Ian. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Jackson
2011-Jun-21 13:40 UTC
Re: [Xen-devel] [PATCH V4] libxl, Introduce a QMP client
Ian Campbell writes ("Re: [Xen-devel] [PATCH V4] libxl, Introduce a QMP client"):> On Mon, 2011-06-20 at 15:50 +0100, Ian Jackson wrote: > > If we do need to define these, we should use a libxl idl enum (with > > Ian Campbell''s string-mapping-generation patch which I will apply > > shortly. > > Just to note that this is enum is only used internally, using the IDL as > is would make it public... It''d be relatively easy to allow for internal > types in the IDL but I''m not sure if we want to? I think if we did I''d > be tempted to add a separate libxl_internal.idl generating (using the > same generator) _libxl_types_internal.[ch] etc.Yes. My point was that we shouldn''t have handwritten tables of enum-vs-strings when we have a generator ... Ian. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Jackson
2011-Jun-21 15:11 UTC
Re: [Xen-devel] [PATCH V4] libxl, Introduce a QMP client
anthony.perard@citrix.com writes ("[Xen-devel] [PATCH V4] libxl, Introduce a QMP client"):> QMP stands for QEMU Monitor Protocol and it is used to query information > from QEMU or to control QEMU.Thanks. You still haven''t fixed the long lines :-/. Can you do so please as it makes the review a lot easier if you do ? Ian. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Jackson
2011-Jun-21 15:12 UTC
Re: [Xen-devel] [PATCH V4] libxl, Introduce a QMP client
I wrote:> anthony.perard@citrix.com writes ("[Xen-devel] [PATCH V4] libxl, Introduce a QMP client"): > > QMP stands for QEMU Monitor Protocol and it is used to query information > > from QEMU or to control QEMU. > > Thanks. You still haven''t fixed the long lines :-/. Can you do so > please as it makes the review a lot easier if you do ?Oh, wait, this wasn''t v5. I said that for v4 already. Sorry. Ian. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Anthony PERARD
2011-Jun-22 13:09 UTC
Re: [Xen-devel] [PATCH V4] libxl, Introduce a QMP client
On Mon, Jun 20, 2011 at 15:50, Ian Jackson <Ian.Jackson@eu.citrix.com> wrote:> anthony.perard@citrix.com writes ("[Xen-devel] [PATCH V4] libxl, Introduce a QMP client"): >> QMP stands for QEMU Monitor Protocol and it is used to query information >> from QEMU or to control QEMU. > > Thanks for this; it''s coming along. I have some comments.... >[...]>> +/* QMP Command that can be send */ >> +enum libxl__qmp_command_e { >> + QMP_COMMAND_QUERY_CHARDEV, >> + QMP_COMMAND_NUMBER, >> +}; > ... >> +_hidden int libxl__qmp_send_command(libxl__qmp_handler *qmp, libxl__qmp_command_e command); > > I''m not entirely convinced by this interface. Wouldn''t it be better > to have a specific function for each command ? After all commands may > have arguments. Something like: > > _hidden int libxl__qmp_send_query_chardev(libxl__qmp_handler *qmp);Indeed, it should be better to have specific functions. Some commands will have arguments. And have a generic function for all of them will maybe not be easy and not clear. -- Anthony PERARD _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel