!!![FEATURE] add document iterator

[jaapio]

The newly added iterators do iterate the toc tree to find the order of rendering of documents. This is required to have single page outputs in the correct order. The downside of this new approach is that only documents that are in a toc-tree are rendered.

[jaapio]

@linawolf can you confirm the new behavior is expected when using toc-trees and that we want to have it this way?

[linawolf]

We would also have to render orphans that are not connected to the toctree. It is totally "legal" to have orphans. In the rendering I would attach all orphans to the end of their corresponding directory in alphabetical order if there is more then one. I would exclude them from rendering int the prev-next buttons (and maybe single HTML) however. We could the leave the responsibity of if they should be rendered or not to the renderer and only have something like an "Is orphan" check

In sphinx all orphans must have the :orphan: modifier at the top of the document or they cause a warning. In either case they are rendered and can be linked with :ref: and :doc: links or cannonical URLs

[jaapio]

I have updated the iterator to ensure orphans are rendered, I use a weak map to track the rendered documents, every time we see a document the flag is set to false this has 2 effects, documents that are seen are not rendered a second time, and documents that are not seen will be rendered afterwards.

There is no option yet to render just the non-orphans, but I think this is easy to implement by adding a filter to the iterator that can be configured via the constructor of the iterator.