webgen users - Mar 2007 - Current status of webgen 0.4.3 and 0.5.0 (new plugin system)

Hi everybody!

As the easter holidays have now finally arrived and all the exams are  
over, I will have more time for webgen again!

Summary
======
* Changes included in the upcoming 0.4.3 release
* webgen repository
* Status on 0.5.0
* Detailed information on the new 0.5.0 plugin system



Changes for 0.4.3
-----------------

I have collected all mails/forum entries/ML postings in which bugs  
were reported, features requested, patches submitted, ... However, I  
did not have time to make changes to the repository till now. So,  
here is a small list of changes for the upcoming 0.4.3 release:

* Include plugins by Andrea Censi (a metainfo and a flickr tag plugin)
* Add the parameters from MenuStyle/Vertical to MenuStyle/Horizontal
* Normalize EOL to \n so that page files with any EOL encoding work  
properly
* Change the way webgen locates it''s data directory (first local,  
then /usr/share/webgen - add info log message with used data dir)
* File/TemplateHandler gets its own set of metainfo and will not use  
File/PageHandler metainfo anymore
* Update examples/templates to now use Textile sothat they work out  
of the box
* Plain website style will be removed due to incompatible license
* Add the possibility to add a prefix to webgen tags sothat the look  
like {wg:menu:} or {webgen:relocatable: default.css}

If you have any comments or suggestions regarding the list above:  
please tell me! :)


webgen repository
-----------------

Andrea Censi asked if it''s possible to have repository access for  
webgen. Since I develop webgen mainly on my laptop using subversion I  
needed the possibility to make local check-ins which would not be  
possible with the subversion repository hosted on Rubyforge. However,  
Andrea was so kind to point me to SVK which solves this dilemma.  
Therefore, Rubyforge hosts the main webgen repository now with  
anonymous read access!

You can access via the following URL: svn://rubyforge.org/var/svn/webgen



Development version 0.5.0
-------------------------

Since the initial release of webgen 0.4.0 I started working on the  
next generation release of webgen, 0.5.0. This release will feature,  
you may believe it or not ;-), an entirely new plugin system! This  
may not come as a surprise for those of you who watched webgen  
develop over time since each major release included a new way of  
handling plugins.

Why a new plugins system, again? Well, the current one works fine,  
but has some limitations which the new one addresses:

* all plugins are loaded at the start
* no way for plugins to provide resources
* no way for coherent plugin documentation
* plugins must be derived from certain base classes
* problems with modules and constant name resolution

The new plugin system addresses all of the above issues and provides:

* on-demand loading of plugins the first time they are accessed
* coherent way to specify documentation for a plugin
* the possibility to provide resources
* any class can be a plugin, only prerequisite is that the  
constructor takes no argument
* easy bundling/sharing of plugins

Those who got interested can view the current development version of  
webgen in the repository: svn://rubyforge.org/var/svn/webgen/devel  
and/or read on to get more detailed information.

As plugins are loaded on demand, the webgen CLI command gets faster  
since only the CLI plugins need to be loaded (currently all plugins  
are loaded, including their dependencies).

Currently, the plugin system itself works and I have already ported  
some plugins to test the new infrastructure.

Other things planned for webgen 0.5.0:

* possibility to localize any file in the src directory
* blogging support (yeah, I know, already said that for 0.4.0)

Again: if anyone has comments/suggestions/critics, don''t hesistate to  
tell me!


webgen 0.5.0 plugin system
--------------------------

Prior to webgen 0.5.0 plugins were just Ruby files. This changes in  
0.5.0 with the concept of a plugin bundle.

A plugin bundle is a directory containing plugins and their  
resources. Consider the following:

websitemanager.plugin/
websitemanager.plugin/plugin.yaml
websitemanager.plugin/manager.rb
websitemanager.plugin/util.rb
websitemanager.plugin/resource.yaml
websitemanager.plugin/resources/templates/...

The file plugin.yaml contains information about the plugins in the  
bundle and looks like this:

Support/WebsiteManager:
   about:
     summary: Provides methods for working with website directories.
     author: Thomas Leitner <t_leitner at gmx.at>
   plugin:
     file: manager.rb
     docufile: manager.documentation
   params:
     sample:
       default: output
       desc: A sample parameter.

Support/Utils:
   about:
     summary: Provides utils methods
     author: Thomas Leitner <t_leitner at gmx.at>
   plugin:
     file: util.rb

As you can see, it''s a YAML file containing hashes with information  
about the plugins. The about section should contain informational  
items about the plugin, like the author or a short description. The  
plugin section contains information about the plugin, like the file  
where the plugin is defined, the class name of the plugin and  
dependencies. Last but not least, the params section contains  
information about plugin parameters. Nothing really complicated here!

So, all the information formerly defined using class methods is now  
defined in plugin.yaml. Therefore, all this information is available  
to the plugin system prior to loading any Ruby source file (and its  
dependencies).

A plugin bundle need not have a plugin.yaml file. If no such file  
exists, it is assumed that the bundle does not contain any plugins.

The resource.yaml is also optional and in it, all resources of the  
bundle are defined. A sample resource.yaml looks like this:

resources/templates/*/:
   name: webgen/website/template/$basename

resources/styles/*/*/:
   name: webgen/website/style/$dir1/$basename

Again, a YAML file whose keys are file globs that can be handled by  
Dir.glob. For each file matching a specific glob, the information  
associated with the glob is associated with the resource. Notice:  
some variables in the form $VARNAME, like $basename, can be used. For  
example, each directory under resources/templates/ is matched by the  
first glob and the name for the resource is set to webgen/website/ 
template/$basename where $basename evaluates to File.basename 
( file ). This allows the definition of several resources with just  
one glob.


Each plugin defined in a plugin bundle can define a documentation  
file. This file should be in Markdown markup because this markup is  
easy to read in textual form and it''s easy to generate HTML output. I  
don''t know if Markdown is the right choice for this or if Textile/ 
RDoc is better - any comments on this???


Existing plugins can be easily translated to the new system: just  
define all static information in plugin.yaml and make sure that the  
initialize method of the plugin does not need any parameters!


So, how does the plugin system itself work?

First you need to load all plugin bundles you want to be available to  
the program. After that the plugin system loads a plugin the first  
time it is accessed. Before the plugin itself gets loaded, all its  
dependencies are loaded to make sure everything works fine. If a  
dependency is not available an error is raised. Loading of a plugin  
is done by just evaluating the file for the plugin in a module (every  
plugin file gets evaluated in the same module) and then an instance  
of the plugin class is created and added to the plugin pool. After  
the instance is created a module is mixed into the instance so that  
the plugin has access to the plugin manager and the parameters, like  
in 0.4.x releases. Therefore any class can act as a plugin!

There is nearly nothing webgen specific in the plugin system now and  
it should be possible for other programs to use this plugin system as  
well. My question is: should this be possible? Does anyone need an  
out-of-the-box plugin system?

That''s all for now! Just one more thing: I will have a guide for  
creating plugins in 0.5.0 available when it is released since this  
was one of the most critized points in the 0.4.x releases!


Bye,
   Thomas

> * Include plugins by Andrea Censi (a metainfo and a flickr tag plugin)
What about Maruku''s content converter plugin?
> * blogging support (yeah, I know, already said that for 0.4.0)
I am faithfully waiting!

-- 
Andrea Censi
      "Life is too important to be taken seriously" (Oscar Wilde)
Web: http://www.dis.uniroma1.it/~censi

>> * Include plugins by Andrea Censi (a metainfo and a flickr tag  
>> plugin)
>
> What about Maruku''s content converter plugin?
Yeah, sorry that I missed that when going over the mails: will also  
be included!

On Fri, Mar 30, 2007 at 03:14:59PM +0200, Thomas Leitner
wrote:
> * blogging support (yeah, I know, already said that for 0.4.0)
I don''t know if it will help, but I created a ''news''
tag that I use for
a my keybox site.  Makes it pretty easy to have news release or blog
type content.  You can see its usage in the sidebar of
keybox.rubyforge.org and on the news page

    http://keybox.rubyforge.org/news.html

I''ve happily donated it to webgen.  Its now uploaded to the Feature
Requests tracker and can be found there.

    Feature Request:
http://rubyforge.org/tracker/index.php?func=detail&aid=9707&group_id=296&atid=1210
    Download link:  
http://rubyforge.org/tracker/download.php/296/1210/9707/1709/news.rb

enjoy,

-jeremy

--
======================================================================= Jeremy
Hinegardner                              jeremy at hinegardner.org

> On Fri, Mar 30, 2007 at 03:14:59PM +0200, Thomas Leitner wrote:
>> * blogging support (yeah, I know, already said that for 0.4.0)
>
> I don''t know if it will help, but I created a
''news'' tag that I use
> for
> a my keybox site.  Makes it pretty easy to have news release or blog
> type content.  You can see its usage in the sidebar of
> keybox.rubyforge.org and on the news page
>
>     http://keybox.rubyforge.org/news.html
>
> I''ve happily donated it to webgen.  Its now uploaded to the
Feature
> Requests tracker and can be found there.
>
>     Feature Request: http://rubyforge.org/tracker/index.php? 
> func=detail&aid=9707&group_id=296&atid=1210
>     Download link:   http://rubyforge.org/tracker/download.php/ 
> 296/1210/9707/1709/news.rb
>
> enjoy,
Thanks, Jeremy! Will be included in the next release!

Bye,
   Thomas