Matt Wilson
2012-Aug-30 18:59 UTC
[PATCH 0 of 2 v2] improve checking for documentation tools and formatting
This version makes running ./configure a prerequisite for building documentation. If a tool is not detected by ./configure and is not specified by the user, the make variable will be empty. So I''ve switched to using make conditionals instead of which(1). Matt
Matt Wilson
2012-Aug-30 18:59 UTC
[PATCH 1 of 2 v2] tools: check for documentation generation tools at configure time
It is sometimes hard to discover all the optional tools that should be on a system to build all available Xen documentation. By checking for documentation generation tools at ./configure time and displaying a warning, Xen packagers will more easily learn about new optional build dependencies, like markdown, when they are introduced. Changes since v1: * require that ./configure be run before building docs * remove Docs.mk and make Tools.mk the canonical locaiton where docs tools are defined (via ./configure) * fold in checking for markdown_py Signed-off-by: Matt Wilson <msw@amazon.com> diff -r d7e4efa17fb0 -r 512b4e0c49f3 README --- a/README Tue Aug 28 15:35:08 2012 -0700 +++ b/README Thu Aug 30 10:51:00 2012 -0700 @@ -28,8 +28,9 @@ your system. For full documentation, see the Xen User Manual. If this is a pre-built release then you can find the manual at: dist/install/usr/share/doc/xen/pdf/user.pdf -If you have a source release, then ''make -C docs'' will build the -manual at docs/pdf/user.pdf. +If you have a source release and the required documentation generation +tools, then ''./configure; make -C docs'' will build the manual at +docs/pdf/user.pdf. Quick-Start Guide ================@@ -59,7 +60,6 @@ * GNU gettext * 16-bit x86 assembler, loader and compiler (dev86 rpm or bin86 & bcc debs) * ACPI ASL compiler (iasl) - * markdown In addition to the above there are a number of optional build prerequisites. Omitting these will cause the related features to be @@ -67,6 +67,7 @@ * Development install of Ocaml (e.g. ocaml-nox and ocaml-findlib). Required to build ocaml components which includes the alternative ocaml xenstored. + * markdown Second, you need to acquire a suitable kernel for use in domain 0. If possible you should use a kernel provided by your OS distributor. If diff -r d7e4efa17fb0 -r 512b4e0c49f3 config/Tools.mk.in --- a/config/Tools.mk.in Tue Aug 28 15:35:08 2012 -0700 +++ b/config/Tools.mk.in Thu Aug 30 10:51:00 2012 -0700 @@ -22,6 +22,18 @@ LD86 := @LD86@ BCC := @BCC@ IASL := @IASL@ +PS2PDF := @PS2PDF@ +DVIPS := @DVIPS@ +LATEX := @LATEX@ +FIG2DEV := @FIG2DEV@ +LATEX2HTML := @LATEX2HTML@ +DOXYGEN := @DOXYGEN@ +POD2MAN := @POD2MAN@ +POD2HTML := @POD2HTML@ +POD2TEXT := @POD2TEXT@ +DOT := @DOT@ +NEATO := @NEATO@ +MARKDOWN := @MARKDOWN@ # Extra folder for libs/includes PREPEND_INCLUDES := @PREPEND_INCLUDES@ diff -r d7e4efa17fb0 -r 512b4e0c49f3 docs/Docs.mk --- a/docs/Docs.mk Tue Aug 28 15:35:08 2012 -0700 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,12 +0,0 @@ -PS2PDF := ps2pdf -DVIPS := dvips -LATEX := latex -FIG2DEV := fig2dev -LATEX2HTML := latex2html -DOXYGEN := doxygen -POD2MAN := pod2man -POD2HTML := pod2html -POD2TEXT := pod2text -DOT := dot -NEATO := neato -MARKDOWN := markdown diff -r d7e4efa17fb0 -r 512b4e0c49f3 docs/Makefile --- a/docs/Makefile Tue Aug 28 15:35:08 2012 -0700 +++ b/docs/Makefile Thu Aug 30 10:51:00 2012 -0700 @@ -2,7 +2,7 @@ XEN_ROOT=$(CURDIR)/.. include $(XEN_ROOT)/Config.mk -include $(XEN_ROOT)/docs/Docs.mk +-include $(XEN_ROOT)/config/Tools.mk VERSION = xen-unstable @@ -26,10 +26,12 @@ .PHONY: build build: html txt man-pages - @if which $(DOT) 1>/dev/null 2>/dev/null ; then \ - $(MAKE) -C xen-api build ; else \ - echo "Graphviz (dot) not installed; skipping xen-api." ; fi +ifdef DOT + $(MAKE) -C xen-api build rm -f *.aux *.dvi *.bbl *.blg *.glo *.idx *.ilg *.log *.ind *.toc +else + @echo "Graphviz (dot) not installed; skipping xen-api." +endif .PHONY: dev-docs dev-docs: python-dev-docs @@ -42,18 +44,21 @@ .PHONY: python-dev-docs python-dev-docs: - @mkdir -v -p api/tools/python - @set -e ; if which $(DOXYGEN) 1>/dev/null 2>/dev/null; then \ - echo "Running doxygen to generate Python tools APIs ... "; \ - $(DOXYGEN) Doxyfile; \ - $(MAKE) -C api/tools/python/latex ; else \ - echo "Doxygen not installed; skipping python-dev-docs."; fi +ifdef DOXYGEN + @echo "Running doxygen to generate Python tools APIs ... " + mkdir -v -p api/tools/python + $(DOXYGEN) Doxyfile && $(MAKE) -C api/tools/python/latex +else + @echo "Doxygen not installed; skipping python-dev-docs." +endif .PHONY: man-pages man-pages: - @if which $(POD2MAN) 1>/dev/null 2>/dev/null; then \ - $(MAKE) $(DOC_MAN1) $(DOC_MAN5); else \ - echo "pod2man not installed; skipping man-pages."; fi +ifdef POD2MAN + $(MAKE) $(DOC_MAN1) $(DOC_MAN5) +else + @echo "pod2man not installed; skipping man-pages." +endif man1/%.1: man/%.pod.1 Makefile $(INSTALL_DIR) $(@D) @@ -94,30 +99,40 @@ perl -w -- ./gen-html-index -i INDEX html $(DOC_HTML) html/%.html: %.markdown - @$(INSTALL_DIR) $(@D) - @set -e ; if which $(MARKDOWN) 1>/dev/null 2>/dev/null; then \ - echo "Running markdown to generate $*.html ... "; \ + $(INSTALL_DIR) $(@D) +ifdef MARKDOWN + @echo "Running markdown to generate $*.html ... " $(MARKDOWN) $< > $@.tmp ; \ - $(call move-if-changed,$@.tmp,$@) ; else \ - echo "markdown not installed; skipping $*.html."; fi + $(call move-if-changed,$@.tmp,$@) +else + @echo "markdown not installed; skipping $*.html." +endif html/%.txt: %.txt - @$(INSTALL_DIR) $(@D) + $(INSTALL_DIR) $(@D) cp $< $@ html/man/%.1.html: man/%.pod.1 Makefile $(INSTALL_DIR) $(@D) +ifdef POD2HTML $(POD2HTML) --infile=$< --outfile=$@.tmp $(call move-if-changed,$@.tmp,$@) +else + @echo "pod2html not installed; skipping $<." +endif html/man/%.5.html: man/%.pod.5 Makefile $(INSTALL_DIR) $(@D) +ifdef POD2HTML $(POD2HTML) --infile=$< --outfile=$@.tmp $(call move-if-changed,$@.tmp,$@) +else + @echo "pod2html not installed; skipping $<." +endif html/hypercall/index.html: ./xen-headers rm -rf $(@D) - @$(INSTALL_DIR) $(@D) + $(INSTALL_DIR) $(@D) ./xen-headers -O $(@D) \ -T ''arch-x86_64 - Xen public headers'' \ -X arch-ia64 -X arch-x86_32 -X xen-x86_32 -X arch-arm \ @@ -137,11 +152,24 @@ txt/man/%.1.txt: man/%.pod.1 Makefile $(INSTALL_DIR) $(@D) +ifdef POD2TEXT $(POD2TEXT) $< $@.tmp $(call move-if-changed,$@.tmp,$@) +else + @echo "pod2text not installed; skipping $<." +endif txt/man/%.5.txt: man/%.pod.5 Makefile $(INSTALL_DIR) $(@D) +ifdef POD2TEXT $(POD2TEXT) $< $@.tmp $(call move-if-changed,$@.tmp,$@) +else + @echo "pod2text not installed; skipping $<." +endif +ifeq (,$(findstring clean,$(MAKECMDGOALS))) +$(XEN_ROOT)/config/Tools.mk: + $(error You have to run ./configure before building docs) +endif + diff -r d7e4efa17fb0 -r 512b4e0c49f3 docs/xen-api/Makefile --- a/docs/xen-api/Makefile Tue Aug 28 15:35:08 2012 -0700 +++ b/docs/xen-api/Makefile Thu Aug 30 10:51:00 2012 -0700 @@ -2,7 +2,7 @@ XEN_ROOT=$(CURDIR)/../.. include $(XEN_ROOT)/Config.mk -include $(XEN_ROOT)/docs/Docs.mk +-include $(XEN_ROOT)/config/Tools.mk TEX := $(wildcard *.tex) @@ -42,3 +42,8 @@ .PHONY: clean clean: rm -f *.pdf *.ps *.dvi *.aux *.log *.out $(EPSDOT) + +ifeq (,$(findstring clean,$(MAKECMDGOALS))) +$(XEN_ROOT)/config/Tools.mk: + $(error You have to run ./configure before building docs) +endif diff -r d7e4efa17fb0 -r 512b4e0c49f3 tools/configure.ac --- a/tools/configure.ac Tue Aug 28 15:35:08 2012 -0700 +++ b/tools/configure.ac Thu Aug 30 10:51:00 2012 -0700 @@ -34,6 +34,7 @@ m4_include([m4/curses.m4]) m4_include([m4/pthread.m4]) m4_include([m4/ptyfuncs.m4]) +m4_include([m4/docs_tool.m4]) # Enable/disable options AX_ARG_DEFAULT_DISABLE([githttp], [Download GIT repositories via HTTP]) @@ -80,6 +81,18 @@ AC_PATH_PROG([BISON], [bison]) AC_PATH_PROG([FLEX], [flex]) AX_PATH_PROG_OR_FAIL([PERL], [perl]) +AX_DOCS_TOOL_PROG([PS2PDF], [ps2pdf]) +AX_DOCS_TOOL_PROG([DVIPS], [dvips]) +AX_DOCS_TOOL_PROG([LATEX], [latex]) +AX_DOCS_TOOL_PROG([FIG2DEV], [fig2dev]) +AX_DOCS_TOOL_PROG([LATEX2HTML], [latex2html]) +AX_DOCS_TOOL_PROG([DOXYGEN], [doxygen]) +AX_DOCS_TOOL_PROG([POD2MAN], [pod2man]) +AX_DOCS_TOOL_PROG([POD2HTML], [pod2html]) +AX_DOCS_TOOL_PROG([POD2TEXT], [pod2text]) +AX_DOCS_TOOL_PROG([DOT], [dot]) +AX_DOCS_TOOL_PROG([NEATO], [neato]) +AX_DOCS_TOOL_PROGS([MARKDOWN], [markdown markdown_py]) AS_IF([test "x$xapi" = "xy"], [ AX_PATH_PROG_OR_FAIL([CURL], [curl-config]) AX_PATH_PROG_OR_FAIL([XML], [xml2-config]) diff -r d7e4efa17fb0 -r 512b4e0c49f3 tools/m4/docs_tool.m4 --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/tools/m4/docs_tool.m4 Thu Aug 30 10:51:00 2012 -0700 @@ -0,0 +1,17 @@ +AC_DEFUN([AX_DOCS_TOOL_PROG], [ +dnl + AC_ARG_VAR([$1], [Path to $2 tool]) + AC_PATH_PROG([$1], [$2]) + AS_IF([! test -x "$ac_cv_path_$1"], [ + AC_MSG_WARN([$2 is not available so some documentation won''t be built]) + ]) +]) + +AC_DEFUN([AX_DOCS_TOOL_PROGS], [ +dnl + AC_ARG_VAR([$1], [Path to $2 tool]) + AC_PATH_PROGS([$1], [$2]) + AS_IF([! test -x "$ac_cv_path_$1"], [ + AC_MSG_WARN([$2 is not available so some documentation won''t be built]) + ]) +])
Matt Wilson
2012-Aug-30 19:00 UTC
[PATCH 2 of 2 v2] docs: use elinks to format markdown-generated html to text
Markdown, while easy to read and write, isn''t the most consumable format for users reading documentation on a terminal. This patch uses lynx to format markdown produced HTML into text files. Changes since v3: * check for html to text dump tool in ./configure * switch to using elinks * allow command line flags to dump tool to be specified Signed-off-by: Matt Wilson <msw@amazon.com> diff -r 512b4e0c49f3 -r 651347cccff7 config/Tools.mk.in --- a/config/Tools.mk.in Thu Aug 30 10:51:00 2012 -0700 +++ b/config/Tools.mk.in Thu Aug 30 11:56:01 2012 -0700 @@ -34,6 +34,8 @@ DOT := @DOT@ NEATO := @NEATO@ MARKDOWN := @MARKDOWN@ +HTMLDUMP := @HTMLDUMP@ +HTMLDUMPFLAGS := @HTMLDUMPFLAGS@ # Extra folder for libs/includes PREPEND_INCLUDES := @PREPEND_INCLUDES@ diff -r 512b4e0c49f3 -r 651347cccff7 docs/Makefile --- a/docs/Makefile Thu Aug 30 10:51:00 2012 -0700 +++ b/docs/Makefile Thu Aug 30 11:56:01 2012 -0700 @@ -146,9 +146,20 @@ $(call move-if-changed,$@.tmp,$@) txt/%.txt: %.markdown - $(INSTALL_DIR) $(@D) - cp $< $@.tmp + @$(INSTALL_DIR) $(@D) +ifdef MARKDOWN +ifdef HTMLDUMP + @echo "Running markdown to generate $*.txt ... "; \ + $(MARKDOWN) $< | $(HTMLDUMP) $(HTMLDUMPFLAGS) > $@.tmp $(call move-if-changed,$@.tmp,$@) +else + @echo "html dump tool (like elinks) not installed; just copying $<." \; + cp $< $@; +endif +else + @echo "markdown not installed; just copying $<." \; + cp $< $@; +endif txt/man/%.1.txt: man/%.pod.1 Makefile $(INSTALL_DIR) $(@D) diff -r 512b4e0c49f3 -r 651347cccff7 tools/configure.ac --- a/tools/configure.ac Thu Aug 30 10:51:00 2012 -0700 +++ b/tools/configure.ac Thu Aug 30 11:56:01 2012 -0700 @@ -93,6 +93,16 @@ AX_DOCS_TOOL_PROG([DOT], [dot]) AX_DOCS_TOOL_PROG([NEATO], [neato]) AX_DOCS_TOOL_PROGS([MARKDOWN], [markdown markdown_py]) +AC_ARG_VAR([HTMLDUMP], + [Path to html-to-text generation tool (default: elinks)]) +AC_PATH_PROG([HTMLDUMP], [elinks]) +AS_IF([! test -x "$ac_cv_path_HTMLDUMP"], [ + AC_MSG_WARN([$ac_cv_path_HTMLDUMP is not available so text documentation will be unformatted markdown]) +]) +AC_SUBST([HTMLDUMPFLAGS], ["-dump"]) +AC_ARG_VAR([HTMLDUMPFLAGS], [Flags passed to html to text translation tool]) + + AS_IF([test "x$xapi" = "xy"], [ AX_PATH_PROG_OR_FAIL([CURL], [curl-config]) AX_PATH_PROG_OR_FAIL([XML], [xml2-config])
Ian Campbell
2012-Aug-31 09:25 UTC
Re: [PATCH 1 of 2 v2] tools: check for documentation generation tools at configure time
On Thu, 2012-08-30 at 19:59 +0100, Matt Wilson wrote:> It is sometimes hard to discover all the optional tools that should be > on a system to build all available Xen documentation. By checking for > documentation generation tools at ./configure time and displaying a > warning, Xen packagers will more easily learn about new optional build > dependencies, like markdown, when they are introduced. > > Changes since v1: > * require that ./configure be run before building docs > * remove Docs.mk and make Tools.mk the canonical locaiton where > docs tools are defined (via ./configure) > * fold in checking for markdown_py > > Signed-off-by: Matt Wilson <msw@amazon.com> > > diff -r d7e4efa17fb0 -r 512b4e0c49f3 README > --- a/README Tue Aug 28 15:35:08 2012 -0700 > +++ b/README Thu Aug 30 10:51:00 2012 -0700 > @@ -28,8 +28,9 @@ > your system. For full documentation, see the Xen User Manual. If this > is a pre-built release then you can find the manual at: > dist/install/usr/share/doc/xen/pdf/user.pdf > -If you have a source release, then ''make -C docs'' will build the > -manual at docs/pdf/user.pdf. > +If you have a source release and the required documentation generation > +tools, then ''./configure; make -C docs'' will build the manual at > +docs/pdf/user.pdf.This document was removed in 24563:4271634e4c86, looks like we missed this reference. Could you nuke it as you go please?> diff -r d7e4efa17fb0 -r 512b4e0c49f3 tools/m4/docs_tool.m4 > --- /dev/null Thu Jan 01 00:00:00 1970 +0000 > +++ b/tools/m4/docs_tool.m4 Thu Aug 30 10:51:00 2012 -0700 > @@ -0,0 +1,17 @@ > +AC_DEFUN([AX_DOCS_TOOL_PROG], [ > +dnl > + AC_ARG_VAR([$1], [Path to $2 tool]) > + AC_PATH_PROG([$1], [$2]) > + AS_IF([! test -x "$ac_cv_path_$1"], [ > + AC_MSG_WARN([$2 is not available so some documentation won''t be built]) > + ]) > +]) > + > +AC_DEFUN([AX_DOCS_TOOL_PROGS], [ > +dnl > + AC_ARG_VAR([$1], [Path to $2 tool])Does this do something sensible when $2 is a space separated list? Do we need both PROG and PROGS variants? PROG is just a special case of PROGS, isn''t it? Although seeing that AC apparently has both variants I guess there''s a reason for that.> + AC_PATH_PROGS([$1], [$2]) > + AS_IF([! test -x "$ac_cv_path_$1"], [ > + AC_MSG_WARN([$2 is not available so some documentation won''t be built]) > + ]) > +])
Ian Campbell
2012-Aug-31 09:26 UTC
Re: [PATCH 2 of 2 v2] docs: use elinks to format markdown-generated html to text
On Thu, 2012-08-30 at 20:00 +0100, Matt Wilson wrote:> Markdown, while easy to read and write, isn''t the most consumable > format for users reading documentation on a terminal. This patch uses > lynx to format markdown produced HTML into text files.s/lynx/elinks/ in the comment.> diff -r 512b4e0c49f3 -r 651347cccff7 config/Tools.mk.in > --- a/config/Tools.mk.in Thu Aug 30 10:51:00 2012 -0700 > +++ b/config/Tools.mk.in Thu Aug 30 11:56:01 2012 -0700 > @@ -34,6 +34,8 @@ > DOT := @DOT@ > NEATO := @NEATO@ > MARKDOWN := @MARKDOWN@ > +HTMLDUMP := @HTMLDUMP@ > +HTMLDUMPFLAGS := @HTMLDUMPFLAGS@Does HTMLDUMP="elinks -dump" not work? Ian.
Ian Campbell
2012-Aug-31 12:30 UTC
Re: [PATCH 1 of 2 v2] tools: check for documentation generation tools at configure time
On Fri, 2012-08-31 at 10:25 +0100, Ian Campbell wrote:> > +AC_DEFUN([AX_DOCS_TOOL_PROGS], [ > > +dnl > > + AC_ARG_VAR([$1], [Path to $2 tool]) > > Does this do something sensible when $2 is a space separated list?Not especially: $ ./configure --help| grep mark MARKDOWN Path to markdown markdown_py tool Ian.
Ian Campbell
2012-Aug-31 12:30 UTC
Re: [PATCH 1 of 2 v2] tools: check for documentation generation tools at configure time
On Thu, 2012-08-30 at 19:59 +0100, Matt Wilson wrote:> It is sometimes hard to discover all the optional tools that should be > on a system to build all available Xen documentation. By checking for > documentation generation tools at ./configure time and displaying a > warning, Xen packagers will more easily learn about new optional build > dependencies, like markdown, when they are introduced.Having remembered to rerun autogen.sh I can confirm that I get the same set of docs as I did before on my regular build machine. I get no warnings about missing tools so I suppose I''m building all the docs. (note to self, when this patch goes in check that the cronjob running http://xenbits.xen.org/docs/unstable/ still works. reminders appreciated...) Ian.
Matt Wilson
2012-Sep-04 02:12 UTC
Re: [PATCH 2 of 2 v2] docs: use elinks to format markdown-generated html to text
On Fri, Aug 31, 2012 at 10:26:51AM +0100, Ian Campbell wrote:> On Thu, 2012-08-30 at 20:00 +0100, Matt Wilson wrote: > > Markdown, while easy to read and write, isn''t the most consumable > > format for users reading documentation on a terminal. This patch uses > > lynx to format markdown produced HTML into text files. > > s/lynx/elinks/ in the comment. > > > diff -r 512b4e0c49f3 -r 651347cccff7 config/Tools.mk.in > > --- a/config/Tools.mk.in Thu Aug 30 10:51:00 2012 -0700 > > +++ b/config/Tools.mk.in Thu Aug 30 11:56:01 2012 -0700 > > @@ -34,6 +34,8 @@ > > DOT := @DOT@ > > NEATO := @NEATO@ > > MARKDOWN := @MARKDOWN@ > > +HTMLDUMP := @HTMLDUMP@ > > +HTMLDUMPFLAGS := @HTMLDUMPFLAGS@ > > Does HTMLDUMP="elinks -dump" not work?I don''t think so. AC_PATH_PROG() will just want the program name. This follows the CC / CFLAGS model. Matt
Matt Wilson
2012-Sep-04 02:13 UTC
Re: [PATCH 1 of 2 v2] tools: check for documentation generation tools at configure time
On Fri, Aug 31, 2012 at 01:30:00PM +0100, Ian Campbell wrote:> On Fri, 2012-08-31 at 10:25 +0100, Ian Campbell wrote: > > > +AC_DEFUN([AX_DOCS_TOOL_PROGS], [ > > > +dnl > > > + AC_ARG_VAR([$1], [Path to $2 tool]) > > > > Does this do something sensible when $2 is a space separated list? > > Not especially: > $ ./configure --help| grep mark > MARKDOWN Path to markdown markdown_py toolI''ll fix that up. Matt