Matt Wilson
2012-Aug-29 21:01 UTC
[PATCH 0 of 3] improve checking for documentation tools and formatting
This patch series is a follow up to a change that I posed for the last Xen Documentation Day that used lynx to create plain text from markdown files. I''ve added ./configure time checking for all the tools used in the docs/ tree. The user can persistently override one of these tools at ./configure time by setting the appropriate environment variable. The docs/ tree maintains the list of default tools, so running ./configure is not a prerequisite for running "make -C docs". I''ve switched to using elinks instead of lynx for creating text documentation, as it provides better formatting and interface compared to other tools. Of course the user can override the tool and flags if they''d like to use something else. Here''s a sample of ./configure output after this series is applied: checking for ps2pdf... /usr/bin/ps2pdf checking for dvips... no configure: WARNING: dvips is not available so some documentation won''t be built checking for latex... no configure: WARNING: latex is not available so some documentation won''t be built Please let me know if there are other concerns that need to be addressed. Matt
Matt Wilson
2012-Aug-29 21:01 UTC
[PATCH 1 of 3] tools: check for documentation generation tools at configure time
It is sometimes hard to discover all the optional documentation generation tools that should be on a system to build all available Xen documentation. By checking for documentation generation tools at ./configure time and displaying a warning, Xen packagers will more easily learn about new optional build dependencies, like markdown, when they are introduced. Signed-off-by: Matt Wilson <msw@amazon.com> diff -r d7e4efa17fb0 -r 674b694814c8 README --- a/README Tue Aug 28 15:35:08 2012 -0700 +++ b/README Wed Aug 29 11:07:52 2012 -0700 @@ -28,8 +28,10 @@ your system. For full documentation, see the Xen User Manual. If this is a pre-built release then you can find the manual at: dist/install/usr/share/doc/xen/pdf/user.pdf -If you have a source release, then ''make -C docs'' will build the -manual at docs/pdf/user.pdf. +If you have a source release and the required documentation generation +tools, then ''make -C docs'' will build the manual at docs/pdf/user.pdf. +Running ./configure will check for the full suite of documentation +tools and will display a warning if missing tools are detected. Quick-Start Guide ================@@ -59,7 +61,6 @@ * GNU gettext * 16-bit x86 assembler, loader and compiler (dev86 rpm or bin86 & bcc debs) * ACPI ASL compiler (iasl) - * markdown In addition to the above there are a number of optional build prerequisites. Omitting these will cause the related features to be @@ -67,6 +68,7 @@ * Development install of Ocaml (e.g. ocaml-nox and ocaml-findlib). Required to build ocaml components which includes the alternative ocaml xenstored. + * markdown Second, you need to acquire a suitable kernel for use in domain 0. If possible you should use a kernel provided by your OS distributor. If diff -r d7e4efa17fb0 -r 674b694814c8 config/Tools.mk.in --- a/config/Tools.mk.in Tue Aug 28 15:35:08 2012 -0700 +++ b/config/Tools.mk.in Wed Aug 29 11:07:52 2012 -0700 @@ -22,6 +22,17 @@ LD86 := @LD86@ BCC := @BCC@ IASL := @IASL@ +PS2PDF := @PS2PDF@ +DVIPS := @DVIPS@ +LATEX := @LATEX@ +FIG2DEV := @FIG2DEV@ +LATEX2HTML := @LATEX2HTML@ +DOXYGEN := @DOXYGEN@ +POD2MAN := @POD2MAN@ +POD2TEXT := @POD2TEXT@ +DOT := @DOT@ +NEATO := @NEATO@ +MARKDOWN := @MARKDOWN@ # Extra folder for libs/includes PREPEND_INCLUDES := @PREPEND_INCLUDES@ diff -r d7e4efa17fb0 -r 674b694814c8 docs/Makefile --- a/docs/Makefile Tue Aug 28 15:35:08 2012 -0700 +++ b/docs/Makefile Wed Aug 29 11:07:52 2012 -0700 @@ -3,6 +3,11 @@ XEN_ROOT=$(CURDIR)/.. include $(XEN_ROOT)/Config.mk include $(XEN_ROOT)/docs/Docs.mk +# The default documentation tools specified in Docs.mk can be +# persistently overridden by the user via ./configure, but running +# ./configure is not required to build the docs tree. Thus Tools.mk is +# optionally included. +-include $(XEN_ROOT)/config/Tools.mk VERSION = xen-unstable diff -r d7e4efa17fb0 -r 674b694814c8 tools/configure.ac --- a/tools/configure.ac Tue Aug 28 15:35:08 2012 -0700 +++ b/tools/configure.ac Wed Aug 29 11:07:52 2012 -0700 @@ -34,6 +34,7 @@ m4_include([m4/curses.m4]) m4_include([m4/pthread.m4]) m4_include([m4/ptyfuncs.m4]) +m4_include([m4/docs_tool.m4]) # Enable/disable options AX_ARG_DEFAULT_DISABLE([githttp], [Download GIT repositories via HTTP]) @@ -80,6 +81,17 @@ AC_PATH_PROG([BISON], [bison]) AC_PATH_PROG([FLEX], [flex]) AX_PATH_PROG_OR_FAIL([PERL], [perl]) +AX_DOCS_TOOL_PROG([PS2PDF], [ps2pdf]) +AX_DOCS_TOOL_PROG([DVIPS], [dvips]) +AX_DOCS_TOOL_PROG([LATEX], [latex]) +AX_DOCS_TOOL_PROG([FIG2DEV], [fig2dev]) +AX_DOCS_TOOL_PROG([LATEX2HTML], [latex2html]) +AX_DOCS_TOOL_PROG([DOXYGEN], [doxygen]) +AX_DOCS_TOOL_PROG([POD2MAN], [pod2man]) +AX_DOCS_TOOL_PROG([POD2TEXT], [pod2text]) +AX_DOCS_TOOL_PROG([DOT], [dot]) +AX_DOCS_TOOL_PROG([NEATO], [neato]) +AX_DOCS_TOOL_PROG([MARKDOWN], [markdown]) AS_IF([test "x$xapi" = "xy"], [ AX_PATH_PROG_OR_FAIL([CURL], [curl-config]) AX_PATH_PROG_OR_FAIL([XML], [xml2-config]) diff -r d7e4efa17fb0 -r 674b694814c8 tools/m4/docs_tool.m4 --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/tools/m4/docs_tool.m4 Wed Aug 29 11:07:52 2012 -0700 @@ -0,0 +1,8 @@ +AC_DEFUN([AX_DOCS_TOOL_PROG], [ +dnl + AC_ARG_VAR([$1], [Path to $2 tool]) + AC_PATH_PROG([$1], [$2]) + AS_IF([! test -x "$ac_cv_path_$1"], [ + AC_MSG_WARN([$2 is not available so some documentation won''t be built]) + ]) +])
Matt Wilson
2012-Aug-29 21:01 UTC
[PATCH 2 of 3] docs: use elinks to format markdown-generated html to text
Markdown, while easy to read and write, isn''t the most consumable format for users reading documentation on a terminal. This patch uses lynx to format markdown produced HTML into text files. Changes since v3: * check for html to text dump tool in ./configure * switch to using elinks * allow command line flags to dump tool to be specified Signed-off-by: Matt Wilson <msw@amazon.com> diff -r 674b694814c8 -r 9a308e4fdc19 config/Tools.mk.in --- a/config/Tools.mk.in Wed Aug 29 11:07:52 2012 -0700 +++ b/config/Tools.mk.in Wed Aug 29 11:50:44 2012 -0700 @@ -33,6 +33,8 @@ DOT := @DOT@ NEATO := @NEATO@ MARKDOWN := @MARKDOWN@ +HTMLDUMP := @HTMLDUMP@ +HTMLDUMPFLAGS := @HTMLDUMPFLAGS@ # Extra folder for libs/includes PREPEND_INCLUDES := @PREPEND_INCLUDES@ diff -r 674b694814c8 -r 9a308e4fdc19 docs/Docs.mk --- a/docs/Docs.mk Wed Aug 29 11:07:52 2012 -0700 +++ b/docs/Docs.mk Wed Aug 29 11:50:44 2012 -0700 @@ -10,3 +10,5 @@ DOT := dot NEATO := neato MARKDOWN := markdown +HTMLDUMP := elinks +HTMLDUMPFLAGS := -dump diff -r 674b694814c8 -r 9a308e4fdc19 docs/Makefile --- a/docs/Makefile Wed Aug 29 11:07:52 2012 -0700 +++ b/docs/Makefile Wed Aug 29 11:50:44 2012 -0700 @@ -136,9 +136,17 @@ $(call move-if-changed,$@.tmp,$@) txt/%.txt: %.markdown - $(INSTALL_DIR) $(@D) - cp $< $@.tmp - $(call move-if-changed,$@.tmp,$@) + @$(INSTALL_DIR) $(@D) + set -e ; \ + if which $(MARKDOWN) >/dev/null 2>&1 && \ + which $(HTMLDUMP) >/dev/null 2>&1 ; then \ + echo "Running markdown to generate $*.txt ... "; \ + $(MARKDOWN) $< | $(HTMLDUMP) $(HTMLDUMPFLAGS) > $@.tmp ; \ + $(call move-if-changed,$@.tmp,$@) ; \ + else \ + echo "markdown or html dump tool (like links) not installed; just copying $<."; \ + cp $< $@; \ + fi txt/man/%.1.txt: man/%.pod.1 Makefile $(INSTALL_DIR) $(@D) diff -r 674b694814c8 -r 9a308e4fdc19 tools/configure.ac --- a/tools/configure.ac Wed Aug 29 11:07:52 2012 -0700 +++ b/tools/configure.ac Wed Aug 29 11:50:44 2012 -0700 @@ -92,6 +92,16 @@ AX_DOCS_TOOL_PROG([DOT], [dot]) AX_DOCS_TOOL_PROG([NEATO], [neato]) AX_DOCS_TOOL_PROG([MARKDOWN], [markdown]) + +AC_ARG_VAR([HTMLDUMP], + [Path to html-to-text generation tool (default: elinks)]) +AC_PATH_PROG([HTMLDUMP], [elinks]) +AS_IF([! test -x "$ac_cv_path_HTMLDUMP"], [ + AC_MSG_WARN([$ac_cv_path_HTMLDUMP is not available so text documentation will be unformatted markdown]) +]) +AC_SUBST([HTMLDUMPFLAGS], ["-dump"]) +AC_ARG_VAR([HTMLDUMPFLAGS], [Flags passed to html to text translation tool]) + AS_IF([test "x$xapi" = "xy"], [ AX_PATH_PROG_OR_FAIL([CURL], [curl-config]) AX_PATH_PROG_OR_FAIL([XML], [xml2-config])
An alternative Python markdown implementation is available on some systems as markdown_py, so look for that path as well. Signed-off-by: Matt Wilson <msw@amazon.com> diff -r 9a308e4fdc19 -r f5a57d912d9f tools/configure.ac --- a/tools/configure.ac Wed Aug 29 11:50:44 2012 -0700 +++ b/tools/configure.ac Wed Aug 29 11:58:30 2012 -0700 @@ -91,7 +91,7 @@ AX_DOCS_TOOL_PROG([POD2TEXT], [pod2text]) AX_DOCS_TOOL_PROG([DOT], [dot]) AX_DOCS_TOOL_PROG([NEATO], [neato]) -AX_DOCS_TOOL_PROG([MARKDOWN], [markdown]) +AX_DOCS_TOOL_PROGS([MARKDOWN], [markdown markdown_py]) AC_ARG_VAR([HTMLDUMP], [Path to html-to-text generation tool (default: elinks)]) diff -r 9a308e4fdc19 -r f5a57d912d9f tools/m4/docs_tool.m4 --- a/tools/m4/docs_tool.m4 Wed Aug 29 11:50:44 2012 -0700 +++ b/tools/m4/docs_tool.m4 Wed Aug 29 11:58:30 2012 -0700 @@ -6,3 +6,12 @@ AC_MSG_WARN([$2 is not available so some documentation won''t be built]) ]) ]) + +AC_DEFUN([AX_DOCS_TOOL_PROGS], [ +dnl + AC_ARG_VAR([$1], [Path to $2 tool]) + AC_PATH_PROGS([$1], [$2]) + AS_IF([! test -x "$ac_cv_path_$1"], [ + AC_MSG_WARN([$2 is not available so some documentation won''t be built]) + ]) +])
Ian Jackson
2012-Aug-30 15:39 UTC
Re: [PATCH 1 of 3] tools: check for documentation generation tools at configure time
Matt Wilson writes ("[Xen-devel] [PATCH 1 of 3] tools: check for documentation generation tools at configure time"):> It is sometimes hard to discover all the optional documentation > generation tools that should be on a system to build all available Xen > documentation. By checking for documentation generation tools at > ./configure time and displaying a warning, Xen packagers will more > easily learn about new optional build dependencies, like markdown, > when they are introduced.The way you''ve done this seems a bit odd to me:> diff -r d7e4efa17fb0 -r 674b694814c8 config/Tools.mk.in > --- a/config/Tools.mk.in Tue Aug 28 15:35:08 2012 -0700 > +++ b/config/Tools.mk.in Wed Aug 29 11:07:52 2012 -0700 > @@ -22,6 +22,17 @@ > LD86 := @LD86@ > BCC := @BCC@ > IASL := @IASL@ > +PS2PDF := @PS2PDF@ > +DVIPS := @DVIPS@But this leaves settings of PS2PDF := ps2pdf in docs/Docs.mk AFAICT. Surely this should be all plumbed through to the same places ? Or is this part of the fallback mechanism ? It''s not clear. You do say this:> diff -r d7e4efa17fb0 -r 674b694814c8 docs/Makefile > --- a/docs/Makefile Tue Aug 28 15:35:08 2012 -0700 > +++ b/docs/Makefile Wed Aug 29 11:07:52 2012 -0700 > @@ -3,6 +3,11 @@ > XEN_ROOT=$(CURDIR)/.. > include $(XEN_ROOT)/Config.mk > include $(XEN_ROOT)/docs/Docs.mk > +# The default documentation tools specified in Docs.mk can be > +# persistently overridden by the user via ./configure, but running > +# ./configure is not required to build the docs tree. Thus Tools.mk is > +# optionally included. > +-include $(XEN_ROOT)/config/Tools.mkBut I think setting the same thing in two places like this does need to be documented more clearly. In particular if I''m right about the purpose of the two settings, each set needs a comment pointing at the other so that (a) they can be both changed at once when either is changed (b) people don''t get confused when the setting they''re changing has no effect. Ian.
Ian Jackson
2012-Aug-30 15:40 UTC
Re: [PATCH 3 of 3] tools/docs: allow markdown_py to be used
Matt Wilson writes ("[Xen-devel] [PATCH 3 of 3] tools/docs: allow markdown_py to be used"):> An alternative Python markdown implementation is available on some > systems as markdown_py, so look for that path as well.Acked-by: Ian Jackson <ian.jackson@eu.citrix.com> Good for 4.2. Subject of course to the other patches going in. Ian.
Ian Jackson
2012-Aug-30 15:41 UTC
Re: [PATCH 2 of 3] docs: use elinks to format markdown-generated html to text
Matt Wilson writes ("[Xen-devel] [PATCH 2 of 3] docs: use elinks to format markdown-generated html to text"):> Markdown, while easy to read and write, isn''t the most consumable > format for users reading documentation on a terminal. This patch uses > lynx to format markdown produced HTML into text files....> txt/%.txt: %.markdown > - $(INSTALL_DIR) $(@D) > - cp $< $@.tmp > - $(call move-if-changed,$@.tmp,$@) > + @$(INSTALL_DIR) $(@D) > + set -e ; \ > + if which $(MARKDOWN) >/dev/null 2>&1 && \ > + which $(HTMLDUMP) >/dev/null 2>&1 ; then \ > + echo "Running markdown to generate $*.txt ... "; \So now we have two efforts to try to find markdown, one in configure and one here. Keir, would it be OK if we simply declared that you must run configure to "make docs" ? Ian.
Keir Fraser
2012-Aug-30 15:47 UTC
Re: [PATCH 2 of 3] docs: use elinks to format markdown-generated html to text
On 30/08/2012 16:41, "Ian Jackson" <Ian.Jackson@eu.citrix.com> wrote:> Matt Wilson writes ("[Xen-devel] [PATCH 2 of 3] docs: use elinks to format > markdown-generated html to text"): >> Markdown, while easy to read and write, isn''t the most consumable >> format for users reading documentation on a terminal. This patch uses >> lynx to format markdown produced HTML into text files. > ... >> txt/%.txt: %.markdown >> - $(INSTALL_DIR) $(@D) >> - cp $< $@.tmp >> - $(call move-if-changed,$@.tmp,$@) >> + @$(INSTALL_DIR) $(@D) >> + set -e ; \ >> + if which $(MARKDOWN) >/dev/null 2>&1 && \ >> + which $(HTMLDUMP) >/dev/null 2>&1 ; then \ >> + echo "Running markdown to generate $*.txt ... "; \ > > So now we have two efforts to try to find markdown, one in configure > and one here. > > Keir, would it be OK if we simply declared that you must run configure > to "make docs" ?Yes! -- Keir> Ian.
Ian Campbell
2012-Aug-30 15:47 UTC
Re: [PATCH 2 of 3] docs: use elinks to format markdown-generated html to text
On Thu, 2012-08-30 at 16:41 +0100, Ian Jackson wrote:> Matt Wilson writes ("[Xen-devel] [PATCH 2 of 3] docs: use elinks to format markdown-generated html to text"): > > Markdown, while easy to read and write, isn''t the most consumable > > format for users reading documentation on a terminal. This patch uses > > lynx to format markdown produced HTML into text files. > ... > > txt/%.txt: %.markdown > > - $(INSTALL_DIR) $(@D) > > - cp $< $@.tmp > > - $(call move-if-changed,$@.tmp,$@) > > + @$(INSTALL_DIR) $(@D) > > + set -e ; \ > > + if which $(MARKDOWN) >/dev/null 2>&1 && \ > > + which $(HTMLDUMP) >/dev/null 2>&1 ; then \ > > + echo "Running markdown to generate $*.txt ... "; \ > > So now we have two efforts to try to find markdown, one in configure > and one here.If we are going to have this idea that docs works without or without running configure then I think it is fine for the without case to just copy unconditionally and not run markdown.> Keir, would it be OK if we simply declared that you must run configure > to "make docs" ?Also an option. Ian
Matt Wilson
2012-Aug-30 16:41 UTC
Re: [PATCH 2 of 3] docs: use elinks to format markdown-generated html to text
On Thu, Aug 30, 2012 at 04:47:29PM +0100, Keir Fraser wrote:> On 30/08/2012 16:41, "Ian Jackson" <Ian.Jackson@eu.citrix.com> wrote: > > > Matt Wilson writes ("[Xen-devel] [PATCH 2 of 3] docs: use elinks to format > > markdown-generated html to text"): > >> Markdown, while easy to read and write, isn''t the most consumable > >> format for users reading documentation on a terminal. This patch uses > >> lynx to format markdown produced HTML into text files. > > ... > >> txt/%.txt: %.markdown > >> - $(INSTALL_DIR) $(@D) > >> - cp $< $@.tmp > >> - $(call move-if-changed,$@.tmp,$@) > >> + @$(INSTALL_DIR) $(@D) > >> + set -e ; \ > >> + if which $(MARKDOWN) >/dev/null 2>&1 && \ > >> + which $(HTMLDUMP) >/dev/null 2>&1 ; then \ > >> + echo "Running markdown to generate $*.txt ... "; \ > > > > So now we have two efforts to try to find markdown, one in configure > > and one here. > > > > Keir, would it be OK if we simply declared that you must run configure > > to "make docs" ? > > Yes!Great! The confusing bit of having these tools in two places was only to retain the ability to run ''make -C docs'' without running ./configure. I''ll resubmit with this cleaned up. Matt
Matt Wilson
2012-Aug-30 16:54 UTC
Re: [PATCH 2 of 3] docs: use elinks to format markdown-generated html to text
On Thu, Aug 30, 2012 at 04:41:08PM +0100, Ian Jackson wrote:> Matt Wilson writes ("[Xen-devel] [PATCH 2 of 3] docs: use elinks to format markdown-generated html to text"): > > Markdown, while easy to read and write, isn''t the most consumable > > format for users reading documentation on a terminal. This patch uses > > lynx to format markdown produced HTML into text files. > ... > > txt/%.txt: %.markdown > > - $(INSTALL_DIR) $(@D) > > - cp $< $@.tmp > > - $(call move-if-changed,$@.tmp,$@) > > + @$(INSTALL_DIR) $(@D) > > + set -e ; \ > > + if which $(MARKDOWN) >/dev/null 2>&1 && \ > > + which $(HTMLDUMP) >/dev/null 2>&1 ; then \ > > + echo "Running markdown to generate $*.txt ... "; \ > > So now we have two efforts to try to find markdown, one in configure > and one here.For what it''s worth, if ./configure is run and a markdown binary is found in $PATH, $(MARKDOWN) will be the full path detected at ./configure time like /usr/bin/markdown and the "which" bit will be a no-op.> Keir, would it be OK if we simply declared that you must run configure > to "make docs" ? > > Ian.