Refactoring, Again :)

Added some more notes about how this is working on the documentation. I do think
that the overall structure is clearer now, and easiest to understand. After all,
that's how it should always be !

--HG--
rename : pelican/processors.py => pelican/generators.py
rename : pelican/generators.py => pelican/writers.py

README.rst

@@ -177,6 +177,10 @@ Source code
 You can access the source code via mercurial at http://hg.notmyidea.org/pelican/
 or via git on http://github.com/ametaireau/pelican/
 
+If you feel hackish, have a look to the `pelican's internals explanations
+<http://alexis.notmyidea.org/pelican/internals.html>`_.
+
+
 Feedback !
 ----------
 

THANKS

@@ -8,3 +8,4 @@ bugs or giving ideas. Thanks to them !
 - Jérome Renard
 - Nicolas Martin
 - David Kulak
+- Arnaud Bos

bin/pelican

@@ -1,38 +1,3 @@
 #!/usr/bin/env python
-import argparse
-
-from pelican.generators import Generator
-from pelican.processors import (ArticlesProcessor, PagesProcessor, 
-                                StaticProcessor, PdfProcessor)
-
-parser = argparse.ArgumentParser(description="""A tool to generate a
-static blog, with restructured text input files.""")
-
-parser.add_argument(dest='path',
-    help='Path where to find the content files')
-parser.add_argument('-t', '--theme-path', dest='theme',
-    help='Path where to find the theme templates. If not specified, it will'
-         'use the default one included with pelican.')
-parser.add_argument('-o', '--output', dest='output',
-    help='Where to output the generated files. If not specified, a directory'
-         ' will be created, named "output" in the current path.')
-parser.add_argument('-m', '--markup', default='rst, md', dest='markup',
-    help='the markup language to use. Currently only ReSTreucturedtext is'
-          ' available.')
-parser.add_argument('-s', '--settings', dest='settings',
-    help='the settings of the application. Default to None.')
-
-
-if __name__ == '__main__':
-    args = parser.parse_args()
-    markup = [a.split()[0] for a in args.markup.split(',')]
-
-    generator = Generator(args.settings, args.path, args.theme, 
-                          args.output, markup)
-
-    processors = [ArticlesProcessor, PagesProcessor, StaticProcessor]
-    if generator.settings['PDF_PROCESSOR']:
-        processors.append(PdfProcessor)
-
-    generator.run(processors)
-    print "Enjoy !"
+from pelican import main
+main()

docs/internals.rst

@@ -0,0 +1,81 @@
+Pelican internals
+#################
+
+This section describe how pelican is working internally. As you'll see, it's
+quite simple, but a bit of documentation doesn't hurt :)
+
+Overall structure
+=================
+
+What `pelican` does, is taking a list of files, and processing them, to some
+sort of output. Usually, the files are restructured text and markdown files,
+and the output is a blog, but it can be anything you want.
+
+I've separated the logic in different classes and concepts:
+
+* `writers` are responsible of all the writing process of the
+  files. It's writing .html files, RSS feeds and so on. Since those operations 
+  are commonly used, the object is created once, and then passed to the 
+  generators.
+
+* `readers` are used to read from various formats (Markdown, and Restructured
+  Text for now, but the system is extensible). Given a file, they return
+  metadata (author, tags, category etc) and content (HTML formated)
+
+* `generators` generate the different outputs. For instance, pelican comes with
+  `ArticlesGenerator` and `PageGenerator`, into others. Given
+  a configurations, they can do whatever they want. Most of the time it's
+  generating files from inputs.
+
+* `pelican` also uses `templates`, so it's easy to write you own theme. The
+  syntax is `jinja2`, and, trust me, really easy to learn, so don't hesitate
+  a second.
+
+How to implement a new reader ?
+===============================
+
+There is an awesome markup language you want to add to pelican ?
+Well, the only thing you have to do is to create a class that have a `read`
+method, that is returning an HTML content and some metadata.
+
+Take a look to the Markdown reader::
+
+    class MarkdownReader(object):
+
+        def read(self, filename):
+            """Parse content and metadata of markdown files"""
+            text = open(filename)
+            md = Markdown(extensions = ['meta', 'codehilite'])
+            content = md.convert(text)
+            
+            metadatas = {}
+            for name, value in md.Meta.items():
+                if name in _METADATAS_FIELDS:
+                    meta = _METADATAS_FIELDS[name](value[0])
+                else:
+                    meta = value[0]
+                metadatas[name.lower()] = meta
+            return content, metadatas
+
+Simple isn't it ?
+
+How to implement a new generator ?
+==================================
+
+Generators have basically two important methods. You're not forced to create
+both, only the existing ones will be called.
+
+* `generate_context`, that is called in a first place, for all the generators.
+  Do whatever you have to do, and update the global context if needed. This
+  context is shared between all generators, and will be passed to the
+  templates. For instance, the `PageGenerator` `generate_context` method find
+  all the pages, transform them into objects, and populate the context with
+  them. Be careful to *not* output anything using this context at this stage,
+  as it is likely to change by the effect of others generators.
+
+* `generate_output` is then called. And guess what is it made for ? Oh,
+  generating the output :) That's here that you may want to look at the context
+  and call the methods of the `writer` object, that is passed at the first
+  argument of this function. In the `PageGenerator` example, this method will
+  look at all the pages recorded in the global context, and output a file on
+  the disk (using the writer method `write_file`) for each page encountered.

pelican/__init__.py

@@ -0,0 +1,104 @@
+import argparse
+import os
+
+from pelican.settings import read_settings
+from pelican.utils import clean_output_dir
+from pelican.writers import Writer
+from pelican.generators import (ArticlesGenerator, PagesGenerator, 
+                                StaticGenerator, PdfGenerator)
+
+
+def init_params(settings=None, path=None, theme=None, output_path=None, 
+        markup=None):
+    """Read the settings, and performs some checks on the environment
+    before doing anything else.
+    """
+    if settings is None:
+        settings = {}
+    settings = read_settings(settings)
+    path = path or settings['PATH']
+    if path.endswith('/'):
+        path = path[:-1]
+
+    # define the default settings
+    theme = theme or settings['THEME']
+    output_path = output_path or settings['OUTPUT_PATH']
+    output_path = os.path.realpath(output_path)
+    markup = markup or settings['MARKUP']
+
+    # find the theme in pelican.theme if the given one does not exists
+    if not os.path.exists(theme):
+        theme_path = os.sep.join([os.path.dirname(
+            os.path.abspath(__file__)), "themes/%s" % theme])
+        if os.path.exists(theme_path):
+            theme = theme_path
+        else:
+            raise Exception("Impossible to find the theme %s" % theme)
+
+    if 'SITEURL' not in settings:
+        settings['SITEURL'] = output_path
+
+    # get the list of files to parse
+    if not path:
+        raise Exception('you need to specify a path to search the docs on !')
+
+    return settings, path, theme, output_path, markup
+
+
+def run_generators(generators, settings, path, theme, output_path, markup):
+    """Run the generators and return"""
+
+    context = settings.copy()
+    generators = [p(context, settings, path, theme, output_path, markup) 
+            for p in generators]
+
+    writer = Writer(output_path)
+
+    for p in generators:
+        if hasattr(p, 'generate_context'):
+            p.generate_context()
+
+    # erase the directory if it is not the source
+    if output_path not in os.path.realpath(path):
+        clean_output_dir(output_path)
+
+    for p in generators:
+        if hasattr(p, 'generate_output'):
+            p.generate_output(writer)
+
+
+def run_pelican(settings, path, theme, output_path, markup):
+    """Run pelican with the given parameters"""
+
+    params = init_params(settings, path, theme, output_path, markup)
+    generators = [ArticlesGenerator, PagesGenerator, StaticGenerator]
+    if params[0]['PDF_GENERATOR']:  # param[0] is settings
+        processors.append(PdfGenerator)
+    run_generators(generators, *params)
+
+
+def main():
+    parser = argparse.ArgumentParser(description="""A tool to generate a
+    static blog, with restructured text input files.""")
+
+    parser.add_argument(dest='path',
+        help='Path where to find the content files')
+    parser.add_argument('-t', '--theme-path', dest='theme',
+        help='Path where to find the theme templates. If not specified, it will'
+             'use the default one included with pelican.')
+    parser.add_argument('-o', '--output', dest='output',
+        help='Where to output the generated files. If not specified, a directory'
+             ' will be created, named "output" in the current path.')
+    parser.add_argument('-m', '--markup', default='rst, md', dest='markup',
+        help='the markup language to use. Currently only ReSTreucturedtext is'
+              ' available.')
+    parser.add_argument('-s', '--settings', dest='settings',
+        help='the settings of the application. Default to None.')
+    args = parser.parse_args()
+    markup = [a.split()[0] for a in args.markup.split(',')]
+
+    run_pelican(args.settings, args.path, args.theme, args.output, markup)
+
+
+if __name__ == '__main__':
+    main()

pelican/contents.py

@@ -2,7 +2,7 @@
 
 
 class Page(object):
-    """Represents a page..
+    """Represents a page
     Given a content, and metadatas, create an adequate object.
 
     :param string: the string to parse, containing the original content.