Matt Wilson
2012-Aug-30 18:59 UTC
[PATCH 0 of 2 v2] improve checking for documentation tools and formatting
This version makes running ./configure a prerequisite for building documentation. If a tool is not detected by ./configure and is not specified by the user, the make variable will be empty. So I''ve switched to using make conditionals instead of which(1). Matt
Matt Wilson
2012-Aug-30 18:59 UTC
[PATCH 1 of 2 v2] tools: check for documentation generation tools at configure time
It is sometimes hard to discover all the optional tools that should be
on a system to build all available Xen documentation. By checking for
documentation generation tools at ./configure time and displaying a
warning, Xen packagers will more easily learn about new optional build
dependencies, like markdown, when they are introduced.
Changes since v1:
* require that ./configure be run before building docs
* remove Docs.mk and make Tools.mk the canonical locaiton where
docs tools are defined (via ./configure)
* fold in checking for markdown_py
Signed-off-by: Matt Wilson <msw@amazon.com>
diff -r d7e4efa17fb0 -r 512b4e0c49f3 README
--- a/README Tue Aug 28 15:35:08 2012 -0700
+++ b/README Thu Aug 30 10:51:00 2012 -0700
@@ -28,8 +28,9 @@
your system. For full documentation, see the Xen User Manual. If this
is a pre-built release then you can find the manual at:
dist/install/usr/share/doc/xen/pdf/user.pdf
-If you have a source release, then ''make -C docs'' will build
the
-manual at docs/pdf/user.pdf.
+If you have a source release and the required documentation generation
+tools, then ''./configure; make -C docs'' will build the manual
at
+docs/pdf/user.pdf.
Quick-Start Guide
================@@ -59,7 +60,6 @@
* GNU gettext
* 16-bit x86 assembler, loader and compiler (dev86 rpm or bin86 & bcc
debs)
* ACPI ASL compiler (iasl)
- * markdown
In addition to the above there are a number of optional build
prerequisites. Omitting these will cause the related features to be
@@ -67,6 +67,7 @@
* Development install of Ocaml (e.g. ocaml-nox and
ocaml-findlib). Required to build ocaml components which
includes the alternative ocaml xenstored.
+ * markdown
Second, you need to acquire a suitable kernel for use in domain 0. If
possible you should use a kernel provided by your OS distributor. If
diff -r d7e4efa17fb0 -r 512b4e0c49f3 config/Tools.mk.in
--- a/config/Tools.mk.in Tue Aug 28 15:35:08 2012 -0700
+++ b/config/Tools.mk.in Thu Aug 30 10:51:00 2012 -0700
@@ -22,6 +22,18 @@
LD86 := @LD86@
BCC := @BCC@
IASL := @IASL@
+PS2PDF := @PS2PDF@
+DVIPS := @DVIPS@
+LATEX := @LATEX@
+FIG2DEV := @FIG2DEV@
+LATEX2HTML := @LATEX2HTML@
+DOXYGEN := @DOXYGEN@
+POD2MAN := @POD2MAN@
+POD2HTML := @POD2HTML@
+POD2TEXT := @POD2TEXT@
+DOT := @DOT@
+NEATO := @NEATO@
+MARKDOWN := @MARKDOWN@
# Extra folder for libs/includes
PREPEND_INCLUDES := @PREPEND_INCLUDES@
diff -r d7e4efa17fb0 -r 512b4e0c49f3 docs/Docs.mk
--- a/docs/Docs.mk Tue Aug 28 15:35:08 2012 -0700
+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
@@ -1,12 +0,0 @@
-PS2PDF := ps2pdf
-DVIPS := dvips
-LATEX := latex
-FIG2DEV := fig2dev
-LATEX2HTML := latex2html
-DOXYGEN := doxygen
-POD2MAN := pod2man
-POD2HTML := pod2html
-POD2TEXT := pod2text
-DOT := dot
-NEATO := neato
-MARKDOWN := markdown
diff -r d7e4efa17fb0 -r 512b4e0c49f3 docs/Makefile
--- a/docs/Makefile Tue Aug 28 15:35:08 2012 -0700
+++ b/docs/Makefile Thu Aug 30 10:51:00 2012 -0700
@@ -2,7 +2,7 @@
XEN_ROOT=$(CURDIR)/..
include $(XEN_ROOT)/Config.mk
-include $(XEN_ROOT)/docs/Docs.mk
+-include $(XEN_ROOT)/config/Tools.mk
VERSION = xen-unstable
@@ -26,10 +26,12 @@
.PHONY: build
build: html txt man-pages
- @if which $(DOT) 1>/dev/null 2>/dev/null ; then \
- $(MAKE) -C xen-api build ; else \
- echo "Graphviz (dot) not installed; skipping xen-api." ; fi
+ifdef DOT
+ $(MAKE) -C xen-api build
rm -f *.aux *.dvi *.bbl *.blg *.glo *.idx *.ilg *.log *.ind *.toc
+else
+ @echo "Graphviz (dot) not installed; skipping xen-api."
+endif
.PHONY: dev-docs
dev-docs: python-dev-docs
@@ -42,18 +44,21 @@
.PHONY: python-dev-docs
python-dev-docs:
- @mkdir -v -p api/tools/python
- @set -e ; if which $(DOXYGEN) 1>/dev/null 2>/dev/null; then \
- echo "Running doxygen to generate Python tools APIs ... "; \
- $(DOXYGEN) Doxyfile; \
- $(MAKE) -C api/tools/python/latex ; else \
- echo "Doxygen not installed; skipping python-dev-docs."; fi
+ifdef DOXYGEN
+ @echo "Running doxygen to generate Python tools APIs ... "
+ mkdir -v -p api/tools/python
+ $(DOXYGEN) Doxyfile && $(MAKE) -C api/tools/python/latex
+else
+ @echo "Doxygen not installed; skipping python-dev-docs."
+endif
.PHONY: man-pages
man-pages:
- @if which $(POD2MAN) 1>/dev/null 2>/dev/null; then \
- $(MAKE) $(DOC_MAN1) $(DOC_MAN5); else \
- echo "pod2man not installed; skipping man-pages."; fi
+ifdef POD2MAN
+ $(MAKE) $(DOC_MAN1) $(DOC_MAN5)
+else
+ @echo "pod2man not installed; skipping man-pages."
+endif
man1/%.1: man/%.pod.1 Makefile
$(INSTALL_DIR) $(@D)
@@ -94,30 +99,40 @@
perl -w -- ./gen-html-index -i INDEX html $(DOC_HTML)
html/%.html: %.markdown
- @$(INSTALL_DIR) $(@D)
- @set -e ; if which $(MARKDOWN) 1>/dev/null 2>/dev/null; then \
- echo "Running markdown to generate $*.html ... "; \
+ $(INSTALL_DIR) $(@D)
+ifdef MARKDOWN
+ @echo "Running markdown to generate $*.html ... "
$(MARKDOWN) $< > $@.tmp ; \
- $(call move-if-changed,$@.tmp,$@) ; else \
- echo "markdown not installed; skipping $*.html."; fi
+ $(call move-if-changed,$@.tmp,$@)
+else
+ @echo "markdown not installed; skipping $*.html."
+endif
html/%.txt: %.txt
- @$(INSTALL_DIR) $(@D)
+ $(INSTALL_DIR) $(@D)
cp $< $@
html/man/%.1.html: man/%.pod.1 Makefile
$(INSTALL_DIR) $(@D)
+ifdef POD2HTML
$(POD2HTML) --infile=$< --outfile=$@.tmp
$(call move-if-changed,$@.tmp,$@)
+else
+ @echo "pod2html not installed; skipping $<."
+endif
html/man/%.5.html: man/%.pod.5 Makefile
$(INSTALL_DIR) $(@D)
+ifdef POD2HTML
$(POD2HTML) --infile=$< --outfile=$@.tmp
$(call move-if-changed,$@.tmp,$@)
+else
+ @echo "pod2html not installed; skipping $<."
+endif
html/hypercall/index.html: ./xen-headers
rm -rf $(@D)
- @$(INSTALL_DIR) $(@D)
+ $(INSTALL_DIR) $(@D)
./xen-headers -O $(@D) \
-T ''arch-x86_64 - Xen public headers'' \
-X arch-ia64 -X arch-x86_32 -X xen-x86_32 -X arch-arm \
@@ -137,11 +152,24 @@
txt/man/%.1.txt: man/%.pod.1 Makefile
$(INSTALL_DIR) $(@D)
+ifdef POD2TEXT
$(POD2TEXT) $< $@.tmp
$(call move-if-changed,$@.tmp,$@)
+else
+ @echo "pod2text not installed; skipping $<."
+endif
txt/man/%.5.txt: man/%.pod.5 Makefile
$(INSTALL_DIR) $(@D)
+ifdef POD2TEXT
$(POD2TEXT) $< $@.tmp
$(call move-if-changed,$@.tmp,$@)
+else
+ @echo "pod2text not installed; skipping $<."
+endif
+ifeq (,$(findstring clean,$(MAKECMDGOALS)))
+$(XEN_ROOT)/config/Tools.mk:
+ $(error You have to run ./configure before building docs)
+endif
+
diff -r d7e4efa17fb0 -r 512b4e0c49f3 docs/xen-api/Makefile
--- a/docs/xen-api/Makefile Tue Aug 28 15:35:08 2012 -0700
+++ b/docs/xen-api/Makefile Thu Aug 30 10:51:00 2012 -0700
@@ -2,7 +2,7 @@
XEN_ROOT=$(CURDIR)/../..
include $(XEN_ROOT)/Config.mk
-include $(XEN_ROOT)/docs/Docs.mk
+-include $(XEN_ROOT)/config/Tools.mk
TEX := $(wildcard *.tex)
@@ -42,3 +42,8 @@
.PHONY: clean
clean:
rm -f *.pdf *.ps *.dvi *.aux *.log *.out $(EPSDOT)
+
+ifeq (,$(findstring clean,$(MAKECMDGOALS)))
+$(XEN_ROOT)/config/Tools.mk:
+ $(error You have to run ./configure before building docs)
+endif
diff -r d7e4efa17fb0 -r 512b4e0c49f3 tools/configure.ac
--- a/tools/configure.ac Tue Aug 28 15:35:08 2012 -0700
+++ b/tools/configure.ac Thu Aug 30 10:51:00 2012 -0700
@@ -34,6 +34,7 @@
m4_include([m4/curses.m4])
m4_include([m4/pthread.m4])
m4_include([m4/ptyfuncs.m4])
+m4_include([m4/docs_tool.m4])
# Enable/disable options
AX_ARG_DEFAULT_DISABLE([githttp], [Download GIT repositories via HTTP])
@@ -80,6 +81,18 @@
AC_PATH_PROG([BISON], [bison])
AC_PATH_PROG([FLEX], [flex])
AX_PATH_PROG_OR_FAIL([PERL], [perl])
+AX_DOCS_TOOL_PROG([PS2PDF], [ps2pdf])
+AX_DOCS_TOOL_PROG([DVIPS], [dvips])
+AX_DOCS_TOOL_PROG([LATEX], [latex])
+AX_DOCS_TOOL_PROG([FIG2DEV], [fig2dev])
+AX_DOCS_TOOL_PROG([LATEX2HTML], [latex2html])
+AX_DOCS_TOOL_PROG([DOXYGEN], [doxygen])
+AX_DOCS_TOOL_PROG([POD2MAN], [pod2man])
+AX_DOCS_TOOL_PROG([POD2HTML], [pod2html])
+AX_DOCS_TOOL_PROG([POD2TEXT], [pod2text])
+AX_DOCS_TOOL_PROG([DOT], [dot])
+AX_DOCS_TOOL_PROG([NEATO], [neato])
+AX_DOCS_TOOL_PROGS([MARKDOWN], [markdown markdown_py])
AS_IF([test "x$xapi" = "xy"], [
AX_PATH_PROG_OR_FAIL([CURL], [curl-config])
AX_PATH_PROG_OR_FAIL([XML], [xml2-config])
diff -r d7e4efa17fb0 -r 512b4e0c49f3 tools/m4/docs_tool.m4
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/tools/m4/docs_tool.m4 Thu Aug 30 10:51:00 2012 -0700
@@ -0,0 +1,17 @@
+AC_DEFUN([AX_DOCS_TOOL_PROG], [
+dnl
+ AC_ARG_VAR([$1], [Path to $2 tool])
+ AC_PATH_PROG([$1], [$2])
+ AS_IF([! test -x "$ac_cv_path_$1"], [
+ AC_MSG_WARN([$2 is not available so some documentation won''t
be built])
+ ])
+])
+
+AC_DEFUN([AX_DOCS_TOOL_PROGS], [
+dnl
+ AC_ARG_VAR([$1], [Path to $2 tool])
+ AC_PATH_PROGS([$1], [$2])
+ AS_IF([! test -x "$ac_cv_path_$1"], [
+ AC_MSG_WARN([$2 is not available so some documentation won''t
be built])
+ ])
+])
Matt Wilson
2012-Aug-30 19:00 UTC
[PATCH 2 of 2 v2] docs: use elinks to format markdown-generated html to text
Markdown, while easy to read and write, isn''t the most consumable
format for users reading documentation on a terminal. This patch uses
lynx to format markdown produced HTML into text files.
Changes since v3:
* check for html to text dump tool in ./configure
* switch to using elinks
* allow command line flags to dump tool to be specified
Signed-off-by: Matt Wilson <msw@amazon.com>
diff -r 512b4e0c49f3 -r 651347cccff7 config/Tools.mk.in
--- a/config/Tools.mk.in Thu Aug 30 10:51:00 2012 -0700
+++ b/config/Tools.mk.in Thu Aug 30 11:56:01 2012 -0700
@@ -34,6 +34,8 @@
DOT := @DOT@
NEATO := @NEATO@
MARKDOWN := @MARKDOWN@
+HTMLDUMP := @HTMLDUMP@
+HTMLDUMPFLAGS := @HTMLDUMPFLAGS@
# Extra folder for libs/includes
PREPEND_INCLUDES := @PREPEND_INCLUDES@
diff -r 512b4e0c49f3 -r 651347cccff7 docs/Makefile
--- a/docs/Makefile Thu Aug 30 10:51:00 2012 -0700
+++ b/docs/Makefile Thu Aug 30 11:56:01 2012 -0700
@@ -146,9 +146,20 @@
$(call move-if-changed,$@.tmp,$@)
txt/%.txt: %.markdown
- $(INSTALL_DIR) $(@D)
- cp $< $@.tmp
+ @$(INSTALL_DIR) $(@D)
+ifdef MARKDOWN
+ifdef HTMLDUMP
+ @echo "Running markdown to generate $*.txt ... "; \
+ $(MARKDOWN) $< | $(HTMLDUMP) $(HTMLDUMPFLAGS) > $@.tmp
$(call move-if-changed,$@.tmp,$@)
+else
+ @echo "html dump tool (like elinks) not installed; just copying
$<." \;
+ cp $< $@;
+endif
+else
+ @echo "markdown not installed; just copying $<." \;
+ cp $< $@;
+endif
txt/man/%.1.txt: man/%.pod.1 Makefile
$(INSTALL_DIR) $(@D)
diff -r 512b4e0c49f3 -r 651347cccff7 tools/configure.ac
--- a/tools/configure.ac Thu Aug 30 10:51:00 2012 -0700
+++ b/tools/configure.ac Thu Aug 30 11:56:01 2012 -0700
@@ -93,6 +93,16 @@
AX_DOCS_TOOL_PROG([DOT], [dot])
AX_DOCS_TOOL_PROG([NEATO], [neato])
AX_DOCS_TOOL_PROGS([MARKDOWN], [markdown markdown_py])
+AC_ARG_VAR([HTMLDUMP],
+ [Path to html-to-text generation tool (default: elinks)])
+AC_PATH_PROG([HTMLDUMP], [elinks])
+AS_IF([! test -x "$ac_cv_path_HTMLDUMP"], [
+ AC_MSG_WARN([$ac_cv_path_HTMLDUMP is not available so text documentation
will be unformatted markdown])
+])
+AC_SUBST([HTMLDUMPFLAGS], ["-dump"])
+AC_ARG_VAR([HTMLDUMPFLAGS], [Flags passed to html to text translation tool])
+
+
AS_IF([test "x$xapi" = "xy"], [
AX_PATH_PROG_OR_FAIL([CURL], [curl-config])
AX_PATH_PROG_OR_FAIL([XML], [xml2-config])
Ian Campbell
2012-Aug-31 09:25 UTC
Re: [PATCH 1 of 2 v2] tools: check for documentation generation tools at configure time
On Thu, 2012-08-30 at 19:59 +0100, Matt Wilson wrote:> It is sometimes hard to discover all the optional tools that should be > on a system to build all available Xen documentation. By checking for > documentation generation tools at ./configure time and displaying a > warning, Xen packagers will more easily learn about new optional build > dependencies, like markdown, when they are introduced. > > Changes since v1: > * require that ./configure be run before building docs > * remove Docs.mk and make Tools.mk the canonical locaiton where > docs tools are defined (via ./configure) > * fold in checking for markdown_py > > Signed-off-by: Matt Wilson <msw@amazon.com> > > diff -r d7e4efa17fb0 -r 512b4e0c49f3 README > --- a/README Tue Aug 28 15:35:08 2012 -0700 > +++ b/README Thu Aug 30 10:51:00 2012 -0700 > @@ -28,8 +28,9 @@ > your system. For full documentation, see the Xen User Manual. If this > is a pre-built release then you can find the manual at: > dist/install/usr/share/doc/xen/pdf/user.pdf > -If you have a source release, then ''make -C docs'' will build the > -manual at docs/pdf/user.pdf. > +If you have a source release and the required documentation generation > +tools, then ''./configure; make -C docs'' will build the manual at > +docs/pdf/user.pdf.This document was removed in 24563:4271634e4c86, looks like we missed this reference. Could you nuke it as you go please?> diff -r d7e4efa17fb0 -r 512b4e0c49f3 tools/m4/docs_tool.m4 > --- /dev/null Thu Jan 01 00:00:00 1970 +0000 > +++ b/tools/m4/docs_tool.m4 Thu Aug 30 10:51:00 2012 -0700 > @@ -0,0 +1,17 @@ > +AC_DEFUN([AX_DOCS_TOOL_PROG], [ > +dnl > + AC_ARG_VAR([$1], [Path to $2 tool]) > + AC_PATH_PROG([$1], [$2]) > + AS_IF([! test -x "$ac_cv_path_$1"], [ > + AC_MSG_WARN([$2 is not available so some documentation won''t be built]) > + ]) > +]) > + > +AC_DEFUN([AX_DOCS_TOOL_PROGS], [ > +dnl > + AC_ARG_VAR([$1], [Path to $2 tool])Does this do something sensible when $2 is a space separated list? Do we need both PROG and PROGS variants? PROG is just a special case of PROGS, isn''t it? Although seeing that AC apparently has both variants I guess there''s a reason for that.> + AC_PATH_PROGS([$1], [$2]) > + AS_IF([! test -x "$ac_cv_path_$1"], [ > + AC_MSG_WARN([$2 is not available so some documentation won''t be built]) > + ]) > +])
Ian Campbell
2012-Aug-31 09:26 UTC
Re: [PATCH 2 of 2 v2] docs: use elinks to format markdown-generated html to text
On Thu, 2012-08-30 at 20:00 +0100, Matt Wilson wrote:> Markdown, while easy to read and write, isn''t the most consumable > format for users reading documentation on a terminal. This patch uses > lynx to format markdown produced HTML into text files.s/lynx/elinks/ in the comment.> diff -r 512b4e0c49f3 -r 651347cccff7 config/Tools.mk.in > --- a/config/Tools.mk.in Thu Aug 30 10:51:00 2012 -0700 > +++ b/config/Tools.mk.in Thu Aug 30 11:56:01 2012 -0700 > @@ -34,6 +34,8 @@ > DOT := @DOT@ > NEATO := @NEATO@ > MARKDOWN := @MARKDOWN@ > +HTMLDUMP := @HTMLDUMP@ > +HTMLDUMPFLAGS := @HTMLDUMPFLAGS@Does HTMLDUMP="elinks -dump" not work? Ian.
Ian Campbell
2012-Aug-31 12:30 UTC
Re: [PATCH 1 of 2 v2] tools: check for documentation generation tools at configure time
On Fri, 2012-08-31 at 10:25 +0100, Ian Campbell wrote:> > +AC_DEFUN([AX_DOCS_TOOL_PROGS], [ > > +dnl > > + AC_ARG_VAR([$1], [Path to $2 tool]) > > Does this do something sensible when $2 is a space separated list?Not especially: $ ./configure --help| grep mark MARKDOWN Path to markdown markdown_py tool Ian.
Ian Campbell
2012-Aug-31 12:30 UTC
Re: [PATCH 1 of 2 v2] tools: check for documentation generation tools at configure time
On Thu, 2012-08-30 at 19:59 +0100, Matt Wilson wrote:> It is sometimes hard to discover all the optional tools that should be > on a system to build all available Xen documentation. By checking for > documentation generation tools at ./configure time and displaying a > warning, Xen packagers will more easily learn about new optional build > dependencies, like markdown, when they are introduced.Having remembered to rerun autogen.sh I can confirm that I get the same set of docs as I did before on my regular build machine. I get no warnings about missing tools so I suppose I''m building all the docs. (note to self, when this patch goes in check that the cronjob running http://xenbits.xen.org/docs/unstable/ still works. reminders appreciated...) Ian.
Matt Wilson
2012-Sep-04 02:12 UTC
Re: [PATCH 2 of 2 v2] docs: use elinks to format markdown-generated html to text
On Fri, Aug 31, 2012 at 10:26:51AM +0100, Ian Campbell wrote:> On Thu, 2012-08-30 at 20:00 +0100, Matt Wilson wrote: > > Markdown, while easy to read and write, isn''t the most consumable > > format for users reading documentation on a terminal. This patch uses > > lynx to format markdown produced HTML into text files. > > s/lynx/elinks/ in the comment. > > > diff -r 512b4e0c49f3 -r 651347cccff7 config/Tools.mk.in > > --- a/config/Tools.mk.in Thu Aug 30 10:51:00 2012 -0700 > > +++ b/config/Tools.mk.in Thu Aug 30 11:56:01 2012 -0700 > > @@ -34,6 +34,8 @@ > > DOT := @DOT@ > > NEATO := @NEATO@ > > MARKDOWN := @MARKDOWN@ > > +HTMLDUMP := @HTMLDUMP@ > > +HTMLDUMPFLAGS := @HTMLDUMPFLAGS@ > > Does HTMLDUMP="elinks -dump" not work?I don''t think so. AC_PATH_PROG() will just want the program name. This follows the CC / CFLAGS model. Matt
Matt Wilson
2012-Sep-04 02:13 UTC
Re: [PATCH 1 of 2 v2] tools: check for documentation generation tools at configure time
On Fri, Aug 31, 2012 at 01:30:00PM +0100, Ian Campbell wrote:> On Fri, 2012-08-31 at 10:25 +0100, Ian Campbell wrote: > > > +AC_DEFUN([AX_DOCS_TOOL_PROGS], [ > > > +dnl > > > + AC_ARG_VAR([$1], [Path to $2 tool]) > > > > Does this do something sensible when $2 is a space separated list? > > Not especially: > $ ./configure --help| grep mark > MARKDOWN Path to markdown markdown_py toolI''ll fix that up. Matt