From: Vladimír Vondruš Date: Sun, 7 Jun 2020 15:50:53 +0000 (+0200) Subject: documentation/doxygen: switch to a config supplied from a Python file. X-Git-Url: https://www.chiark.greenend.org.uk/ucgi/~cjwatson/git?a=commitdiff_plain;h=1c133b1bae1b9072c4567ec2cb77202c6c2283fc;p=blog.git documentation/doxygen: switch to a config supplied from a Python file. Keeping full backwards compatibility with the original Doxyfile names, except for template files which now have to use the new values. --- diff --git a/doc/documentation/doxygen.rst b/doc/documentation/doxygen.rst index 411fcfac..9181f38b 100644 --- a/doc/documentation/doxygen.rst +++ b/doc/documentation/doxygen.rst @@ -167,9 +167,9 @@ In addition to features `shared by all doc generators <{filename}/documentation. documentation practices --- having the output consist of an actual human-written documentation instead of just autogenerated lists. If you *really* want to have them included in the output, you can enable them - using a default-to-off :ini:`M_SHOW_UNDOCUMENTED` option, but there are - some tradeoffs. See `Showing undocumented symbols and files`_ for - more information. + using a default-to-off :py:`SHOW_UNDOCUMENTED` option, but there are some + tradeoffs. See `Showing undocumented symbols and files`_ for more + information. - Table of contents is generated for compound references as well, containing all sections of detailed description together with anchors to member listings @@ -253,10 +253,8 @@ amount of generated content for no added value. `Configuration`_ ================ -The script takes most of the configuration from the ``Doxyfile`` itself, -(ab)using the following builtin options. The used options are similar to the -`Python config <{filename}python.rst#configuration>`_, but with Doxygen-imposed -naming and constraints. +The script takes a part of the configuration from the ``Doxyfile`` itself, +(ab)using the following builtin options: .. class:: m-table m-fullwidth @@ -277,17 +275,6 @@ Variable Description :ini:`HTML_OUTPUT` The output will be written here :ini:`TAGFILES` Used to discover what base URL to prepend to external references -:ini:`HTML_EXTRA_STYLESHEET` List of CSS files to include. Relative paths - are searched relative to the Doxyfile base dir - and to the ``doxygen.py`` script dir as a - fallback. See `Theme selection`_ for more - information. -:ini:`HTML_EXTRA_FILES` List of extra files to copy (for example - additional CSS files that are :css:`@import`\ ed - from the primary one). Relative paths are - searched relative to the Doxyfile base dir and - to the ``doxygen.py`` script dir as a - fallback. :ini:`DOT_FONTNAME` Font name to use for ``@dot`` and ``@dotfile`` commands. To ensure consistent look with the default m.css themes, set it to @@ -308,107 +295,143 @@ Variable Description actually useful. Doxygen default is ``YES``. =============================== =============================================== -In addition, the m.css Doxygen theme recognizes the following extra options: +On top of the above, the script can take additional options in a way consistent +with the `Python documentation generator <{filename}python.rst#configuration>`_. +The recommended and most flexible way is to create a ``conf.py`` file +referencing the original Doxyfile: + +.. code:: py + + DOXYFILE = 'Doxyfile-mcss' + + # additional options from the table below + +and then pass that file to the script, instead of the original Doxyfile: + +.. code:: sh + + ./doxygen.py path/to/your/conf.py .. class:: m-table m-fullwidth =================================== =========================================== Variable Description =================================== =========================================== -:ini:`M_THEME_COLOR` Color for :html:``, - corresponding to the CSS style. If empty, +:py:`MAIN_PROJECT_URL: str` If set and :ini:`PROJECT_BRIEF` is also + set, then :ini:`PROJECT_NAME` in the top + navbar will link to this URL and + :ini:`PROJECT_BRIEF` to the documentation + main page, similarly as `shown here <{filename}/css/page-layout.rst#link-back-to-main-site-from-a-subsite>`_. +:py:`THEME_COLOR: str` Color for :html:``, + corresponding to the CSS style. If not set, no :html:`` tag is rendered. See `Theme selection`_ for more information. -:ini:`M_FAVICON` Favicon URL, used to populate +:py:`FAVICON: str` Favicon URL, used to populate :html:``. If empty, no :html:`` tag is rendered. Relative - paths are searched relative to the Doxyfile + paths are searched relative to the + :abbr:`config file ` base dir and to the ``doxygen.py`` script dir as a fallback. See `Theme selection`_ for more information. -:ini:`M_LINKS_NAVBAR1` Left navbar column links. See - `Navbar links`_ for more information. -:ini:`M_LINKS_NAVBAR2` Right navbar column links. See - `Navbar links`_ for more information. -:ini:`M_MAIN_PROJECT_URL` If set and :ini:`PROJECT_BRIEF` is also - set, then :ini:`PROJECT_NAME` in the top - navbar will link to this URL and - :ini:`PROJECT_BRIEF` to the documentation - main page, similarly as `shown here <{filename}/css/page-layout.rst#link-back-to-main-site-from-a-subsite>`_. -:ini:`M_HTML_HEADER` HTML code to put at the end of the +:py:`STYLESHEETS: List[str]` List of CSS files to include. Relative + paths are searched relative to the + :abbr:`config file ` + base dir and to the ``doxygen.py`` script + dir as a fallback. See `Theme selection`_ + for more information. +:py:`HTML_HEADER: str` HTML code to put at the end of the :html:`` element. Useful for linking arbitrary JavaScript code or, for example, adding :html:`` CSS stylesheets with additional properties and IDs that are otherwise not possible with just :ini:`HTML_EXTRA_STYLESHEET` -:ini:`M_PAGE_HEADER` HTML code to put at the top of every page. +:py:`EXTRA_FILES: List[str]` List of extra files to copy (for example + additional CSS files that are :css:`@import`\ ed + from the primary one). Relative paths are + searched relative to the + :abbr:`config file ` + base dir and to the ``doxygen.py`` script + dir as a fallback. +:py:`LINKS_NAVBAR1: List[Any]` Left navbar column links. See + `Navbar links`_ for more information. +:py:`LINKS_NAVBAR2: List[Any]` Right navbar column links. See + `Navbar links`_ for more information. +:py:`PAGE_HEADER: str` HTML code to put at the top of every page. Useful for example to link to different versions of the same documentation. The ``{filename}`` placeholder is replaced with current file name. -:ini:`M_PAGE_FINE_PRINT` HTML code to put into the footer. If not +:py:`FINE_PRINT: str` HTML code to put into the footer. If not set, a default generic text is used. If empty, no footer is rendered at all. The ``{doxygen_version}`` placeholder is replaced with Doxygen version that generated the input XML files. -:ini:`M_CLASS_TREE_EXPAND_LEVELS` How many levels of the class tree to - expand. ``0`` means only the top-level - symbols are shown. If not set, ``1`` is +:py:`CLASS_INDEX_EXPAND_LEVELS` How many levels of the class tree to + expand. :py:`0` means only the top-level + symbols are shown. If not set, :py:`1` is used. -:ini:`M_FILE_TREE_EXPAND_LEVELS` How many levels of the file tree to expand. - ``0`` means only the top-level dirs/files - are shown. If not set, ``1`` is used. -:ini:`M_EXPAND_INNER_TYPES` Whether to expand inner types (e.g. a class +:py:`CLASS_INDEX_EXPAND_INNER` Whether to expand inner types (e.g. a class inside a class) in the symbol tree. If not - set, ``NO`` is used. -:ini:`M_MATH_CACHE_FILE` File to cache rendered math formulas. If - not set, ``m.math.cache`` file in the - output directory is used. Old cached output - is periodically pruned and new formulas - added to the file. Set it empty to disable - caching. -:ini:`M_SEARCH_DISABLED` Disable search functionality. If this + set, :py:`False` is used. +:py:`FILE_INDEX_EXPAND_LEVELS` How many levels of the file tree to expand. + :py:`0` means only the top-level dirs/files + are shown. If not set, :py:`1` is used. +:py:`SEARCH_DISABLED: bool` Disable search functionality. If this option is set, no search data is compiled and the rendered HTML does not contain any search-related UI or support. If not set, - ``NO`` is used. -:ini:`M_SEARCH_DOWNLOAD_BINARY` Download search data as a binary to save + :py:`False` is used. +:py:`SEARCH_DOWNLOAD_BINARY` Download search data as a binary to save bandwidth and initial processing time. If - not set, ``NO`` is used. See + not set, :py:`False` is used. See `Search options`_ for more information. -:ini:`M_SEARCH_HELP` HTML code to display as help text on empty +:py:`SEARCH_HELP: str` HTML code to display as help text on empty search popup. If not set, a default message is used. Has effect only if - :ini:`M_SEARCH_DISABLED` is not ``YES``. -:ini:`M_SEARCH_BASE_URL` Base URL for OpenSearch-based search engine + :py:`SEARCH_DISABLED` is not :py:`True`. +:py:`SEARCH_BASE_URL: str` Base URL for OpenSearch-based search engine suggestions for web browsers. See `Search options`_ for more information. Has - effect only if :ini:`M_SEARCH_DISABLED` is - not ``YES``. -:ini:`M_SEARCH_EXTERNAL_URL` URL for external search. The ``{query}`` + effect only if :py:`SEARCH_DISABLED` is + not :py:`True`. +:py:`SEARCH_EXTERNAL_URL: str` URL for external search. The ``{query}`` placeholder is replaced with urlencoded search string. If not set, no external search is offered. See `Search options`_ for more information. Has effect only if - :ini:`M_SEARCH_DISABLED` is not ``YES``. -:ini:`M_VERSION_LABELS` Show the ``@since`` annotation as labels + :py:`SEARCH_DISABLED` is not :py:`True`. +:py:`VERSION_LABELS: bool` Show the ``@since`` annotation as labels visible in entry listing and detailed docs. - Defaults to ``NO``, see `Version labels`_ + Defaults to :py:`False`, see `Version labels`_ for more information. -:ini:`M_SHOW_UNDOCUMENTED` Include undocumented symbols, files and +:py:`SHOW_UNDOCUMENTED: bool` Include undocumented symbols, files and directories in the output. If not set, - ``NO`` is used. See `Showing undocumented symbols and files`_ + :py:`False` is used. See `Showing undocumented symbols and files`_ for more information. +:py:`M_MATH_CACHE_FILE` File to cache rendered math formulas. If + not set, ``m.math.cache`` file in the + output directory is used. Old cached output + is periodically pruned and new formulas + added to the file. Set it empty to disable + caching. =================================== =========================================== Note that namespace, directory and page lists are always fully expanded as these are not expected to be excessively large. -.. block-success:: Hiding extra options from Doxygen +.. block-warning:: Legacy configuration through extra Doxyfile options - Doxygen complains on unknown options, so it's possible to add them + Originally, the above options were parsed from the Doxyfile as well, but + the Doxyfile format limited the flexibility quite a lot. These are still + supported for backwards compatibility and map to the above options as shown + below. Integer and string values are parsed as-is, list items are parsed + one item per line with ``\`` for line continuations and boolean values have + to be either ``YES`` or ``NO``. + + Doxygen complains on unknown options, so as a workaround one can add them prefixed with ``##!``. Line continuations are supported too, using ``##!`` ensures that the options also survive Doxyfile upgrades using ``doxygen -u`` (which is not the case when the options would be specified @@ -419,6 +442,38 @@ these are not expected to be excessively large. ##! M_LINKS_NAVBAR1 = pages \ ##! modules + .. class:: m-table m-fullwidth + + =================================== ======================================= + Legacy ``Doxyfile`` variable Corresponding ``conf.py`` variable + =================================== ======================================= + :ini:`HTML_EXTRA_STYLESHEET` :py:`STYLESHEETS` + :ini:`HTML_EXTRA_FILES` :py:`EXTRA_FILE` + :ini:`M_THEME_COLOR` :py:`THEME_COLOR` + :ini:`M_FAVICON` :py:`FAVICON` + :ini:`M_LINKS_NAVBAR1` :py:`LINKS_NAVBAR1`. The syntax is + different in each case, see + `Navbar links`_ for more information. + :ini:`M_LINKS_NAVBAR2` :py:`LINKS_NAVBAR2`. The syntax is + different in each case, see + `Navbar links`_ for more information. + :ini:`M_MAIN_PROJECT_URL` :py:`MAIN_PROJECT_URL` + :ini:`M_HTML_HEADER` :py:`HTML_HEADER` + :ini:`M_PAGE_HEADER` :py:`PAGE_HEADER` + :ini:`M_PAGE_FINE_PRINT` :py:`FINE_PRINT` + :ini:`M_CLASS_TREE_EXPAND_LEVELS` :py:`CLASS_INDEX_EXPAND_LEVELS` + :ini:`M_FILE_TREE_EXPAND_LEVELS` :py:`FILE_INDEX_EXPAND_LEVELS` + :ini:`M_EXPAND_INNER_TYPES` :py:`CLASS_INDEX_EXPAND_INNER` + :ini:`M_MATH_CACHE_FILE` :py:`M_MATH_CACHE_FILE` + :ini:`M_SEARCH_DISABLED` :py:`SEARCH_DISABLED` + :ini:`M_SEARCH_DOWNLOAD_BINARY` :py:`SEARCH_DOWNLOAD_BINARY` + :ini:`M_SEARCH_HELP` :py:`SEARCH_HELP` + :ini:`M_SEARCH_BASE_URL` :py:`SEARCH_BASE_URL` + :ini:`M_SEARCH_EXTERNAL_URL` :py:`SEARCH_EXTERNAL_URL` + :ini:`M_VERSION_LABELS` :py:`VERSION_LABELS` + :ini:`M_SHOW_UNDOCUMENTED` :py:`SHOW_UNDOCUMENTED` + =================================== ======================================= + `Theme selection`_ ------------------ @@ -426,42 +481,46 @@ By default, the `dark m.css theme <{filename}/css/themes.rst#dark>`_ together with documentation-theme-specific additions is used, which corresponds to the following configuration: -.. code:: ini +.. code:: py :class: m-console-wrap - HTML_EXTRA_STYLESHEET = \ - https://fonts.googleapis.com/css?family=Source+Sans+Pro:400,400i,600,600i%7CSource+Code+Pro:400,400i,600 \ - ../css/m-dark+documentation.compiled.css - M_THEME_COLOR = #22272e - M_FAVICON = favicon-dark.png + STYLESHEETS = [ + 'https://fonts.googleapis.com/css?family=Source+Sans+Pro:400,400i,600,600i%7CSource+Code+Pro:400,400i,600', + '../css/m-dark+documentation.compiled.css' + ] + THEME_COLOR = '#22272e' + FAVICON = 'favicon-dark.png' If you have a site already using the ``m-dark.compiled.css`` file, there's another file called ``m-dark.documentation.compiled.css``, which contains just the documentation-theme-specific additions so you can reuse the already cached ``m-dark.compiled.css`` file from your main site: -.. code:: ini +.. code:: py :class: m-console-wrap - HTML_EXTRA_STYLESHEET = \ - https://fonts.googleapis.com/css?family=Source+Sans+Pro:400,400i,600,600i%7CSource+Code+Pro:400,400i,600 \ - ../css/m-dark.compiled.css \ - ../css/m-dark.documentation.compiled.css - M_THEME_COLOR = #22272e + STYLESHEETS = [ + 'https://fonts.googleapis.com/css?family=Source+Sans+Pro:400,400i,600,600i%7CSource+Code+Pro:400,400i,600', + '../css/m-dark.compiled.css', + '../css/m-dark.documentation.compiled.css' + ] + THEME_COLOR = '#22272e' + FAVICON = 'favicon-dark.png' If you prefer the `light m.css theme <{filename}/css/themes.rst#light>`_ instead, use the following configuration (and, similarly, you can use ``m-light.compiled.css`` together with ``m-light.documentation.compiled-css`` in place of ``m-light+documentation.compiled.css``: -.. code:: ini +.. code:: py :class: m-console-wrap - HTML_EXTRA_STYLESHEET = \ - https://fonts.googleapis.com/css?family=Libre+Baskerville:400,400i,700,700i%7CSource+Code+Pro:400,400i,600 \ - ../css/m-light+documentation.compiled.css - M_THEME_COLOR = #cb4b16 - M_FAVICON = favicon-light.png + STYLESHEETS = [ + 'https://fonts.googleapis.com/css?family=Libre+Baskerville:400,400i,700,700i%7CSource+Code+Pro:400,400i,600', + '../css/m-light+documentation.compiled.css' + ] + THEME_COLOR = '#cb4b16' + FAVICON = 'favicon-light.png' See the `CSS files`_ section below for more information about customizing the CSS files. @@ -469,64 +528,129 @@ CSS files. `Navbar links`_ --------------- -The :ini:`M_LINKS_NAVBAR1` and :ini:`M_LINKS_NAVBAR2` options define which -links are shown on the top navbar, split into left and right column on small -screen sizes. These options take a whitespace-separated list of compound IDs -and additionally the special ``pages``, ``modules``, ``namespaces``, -``annotated``, ``files`` IDs. By default the variables are defined like -following: +The :py:`LINKS_NAVBAR1` and :py:`LINKS_NAVBAR2` options define which links are +shown on the top navbar, split into left and right column on small screen +sizes. These options take a list of :py:`(title, path, sub)` tuples --- +``title`` is the link title; ``path`` is either one of the :py:`'pages'`, +:py:`'modules'`, :py:`'namespaces'`, :py:`'annotated'`, :py:`'files'` IDs or a +compound ID; and ``sub`` is an optional submenu, containing :py:`(title, path)` +with ``path`` being interpreted the same way. -.. code:: ini +By default the variables are defined like following --- two items in the left +column, two items in the right columns, with no submenus: - M_LINKS_NAVBAR1 = pages namespaces - M_LINKS_NAVBAR2 = annotated files +.. code:: py + + LINKS_NAVBAR1 = [ + ("Pages", 'pages', []), + ("Namespaces", 'namespaces', []) + ] + 'LINKS_NAVBAR2' = [ + ("Classes", 'annotated', []), + ("Files", 'files', []) + ] .. note-info:: The theme by default assumes that the project is grouping symbols in namespaces. If you use modules (``@addtogroup`` and related commands) and you want to show their index in the navbar, add ``modules`` to one of - the :ini:`M_LINKS_NAVBAR*` options, for example: + the :py:`LINKS_NAVBAR*` options, for example: - .. code:: ini + .. code:: py - M_LINKS_NAVBAR1 = pages modules - M_LINKS_NAVBAR2 = annotated files + LINKS_NAVBAR1 = [ + ("Pages", 'pages', []), + ("Modules", 'modules', []) + ] + LINKS_NAVBAR2 = [ + ("Classes", 'annotated', []), + ("Files", 'files', []) + ] -Titles for the links are taken implicitly. Empty :ini:`M_LINKS_NAVBAR2` will -cause the navigation appear in a single column, setting both empty will cause -the navbar links to not be rendered at all. +If the title is :py:`None`, it's taken implicitly from the page it links to. +Empty :py:`LINKS_NAVBAR2` will cause the navigation appear in a single column, +setting both empty will cause the navbar links to not be rendered at all. A menu item is higlighted if a compound with the same ID is the current page (and similarly for the special ``pages``, ... IDs). -It's possible to specify sub-menu items by enclosing more than one ID in -quotes. The top-level items then have to be specified each on a single line. -Example (note the mangled names, corresponding to filenames of given compounds -generated by Doxygen): +Alternatively, a link can be a plain HTML instead of the first pair of tuple +values, in which case it's put into the navbar as-is. It's not limited to just +a text, but it can contain an image, embedded SVG or anything else. A complex +example including submenus follows --- the first navbar column will have links +to namespaces *Foo*, *Bar* and *Utils* as a sub-items of a top-level +*Namespaces* item and links to two subdirectories as sub-items of the *Files* +item, with title being :py:`None` to have it automatically filled in by +Doxygen. The second column has two top-level items, first linking to the page +index and having a submenu linking to an e-mail address and a ``fine-print`` +page and the second linking to a GitHub project page. Note the mangled names, +corresponding to filenames of given compounds generated by Doxygen. + +.. code:: py + + LINKS_NAVBAR1 = [ + (None, 'namespaces', [ + (None, 'namespaceFoo'), + (None, 'namespaceBar'), + (None, 'namespaceUtils'), + ]), + (None, 'files', [ + (None, 'dir_d3b07384d113edec49eaa6238ad5ff00'), + (None, 'dir_cbd8f7984c654c25512e3d9241ae569f') + ]) + ] + LINKS_NAVBAR2 = [ + ("Pages", 'pages', [ + ("Contact", ), + ("Fine print", 'fine-print') + ]), + ("GitHub", []) + ] + +.. block-warning:: Legacy navbar link specification in the Doxyfile + + For backwards compatibility, the :ini:`M_LINKS_NAVBAR1` and + :ini:`M_LINKS_NAVBAR2` options in the Doxyfile are recognized. Compared to + the Python variant above, the encoding is a bit more complicated --- these + options take a whitespace-separated list of compound IDs and additionally + the special ``pages``, ``modules``, ``namespaces``, ``annotated``, + ``files`` IDs. An equivalent to the above default Python config would be + the following --- titles for the links are always taken implicitly in this + case: -.. code:: ini + .. code:: ini - M_LINKS_NAVBAR1 = \ - "namespaces namespaceFoo namespaceBar namespaceUtils" \ - "files dir_d3b07384d113edec49eaa6238ad5ff00 dir_cbd8f7984c654c25512e3d9241ae569f" + M_LINKS_NAVBAR1 = pages namespaces + M_LINKS_NAVBAR2 = annotated files + + It's possible to specify sub-menu items by enclosing more than one ID in + quotes. The top-level items then have to be specified each on a single + line. Example (note the mangled names, corresponding to filenames of given + compounds generated by Doxygen): -This will put links to namespaces Foo, Bar and Utils as a sub-items of a -top-level *Namespaces* item and links to two subdirectories as sub-items of the -*Files* item. + .. code:: ini -For custom links in the navbar it's possible to use HTML code directly, both -for a top-level item or in a submenu. The item is taken as everything from the -initial :html:``. In the following snippet, -there are two top-level items, first linking to the page index and having a -submenu linking to an e-mail address and a ``fine-print`` page and the second -linking to a GitHub project page: + M_LINKS_NAVBAR1 = \ + "namespaces namespaceFoo namespaceBar namespaceUtils" \ + "files dir_d3b07384d113edec49eaa6238ad5ff00 dir_cbd8f7984c654c25512e3d9241ae569f" -.. code:: ini + This will put links to namespaces Foo, Bar and Utils as a sub-items of a + top-level *Namespaces* item and links to two subdirectories as sub-items of + the *Files* item. + + For custom links in the navbar it's possible to use HTML code directly, + both for a top-level item or in a submenu. The item is taken as everything + from the initial :html:``. In the + following snippet, there are two top-level items, first linking to the page + index and having a submenu linking to an e-mail address and a + ``fine-print`` page and the second linking to a GitHub project page: + + .. code:: ini - M_LINKS_NAVBAR2 = \ - "pages Contact fine-print" \ - "GitHub" + M_LINKS_NAVBAR2 = \ + "pages Contact fine-print" \ + "GitHub" `Search options`_ ----------------- @@ -543,17 +667,17 @@ Base85-encoded representation of the search binary and loading that asynchronously as a plain JavaScript file. This results in the search data being 25% larger, but since this is for serving from a local filesystem, it's not considered a problem. If your docs are accessed through a server (or you -don't need Chrome support), enable the :ini:`M_SEARCH_DOWNLOAD_BINARY` option. +don't need Chrome support), enable the :py:`SEARCH_DOWNLOAD_BINARY` option. The site can provide search engine metadata using the `OpenSearch `_ specification. On supported browsers this means you can add the search field to search engines and search directly from the address bar. To enable search -engine metadata, point :ini:`M_SEARCH_BASE_URL` to base URL of your -documentation, for example: +engine metadata, point :py:`SEARCH_BASE_URL` to base URL of your documentation, +for example: -.. code:: ini +.. code:: py - M_SEARCH_BASE_URL = "https://doc.magnum.graphics/magnum/" + SEARCH_BASE_URL = "https://doc.magnum.graphics/magnum/" In general, even without the above setting, appending ``?q={query}#search`` to the URL will directly open the search popup with results for ``{query}``. @@ -564,16 +688,16 @@ the URL will directly open the search popup with results for ``{query}``. directly in the browser address bar. However that requires a server-side search implementation and is not supported at the moment. -If :ini:`M_SEARCH_EXTERNAL_URL` is specified, full-text search using an -external search engine is offered if nothing is found for given string or if -the user has JavaScript disabled. It's recommended to restrict the search to -a particular domain or add additional keywords to the search query to filter -out irrelevant results. Example, using Google search engine and restricting -the search to a subdomain: +If :py:`SEARCH_EXTERNAL_URL` is specified, full-text search using an external +search engine is offered if nothing is found for given string or if the user +has JavaScript disabled. It's recommended to restrict the search to a +particular domain or add additional keywords to the search query to filter out +irrelevant results. Example, using Google search engine and restricting the +search to a subdomain: -.. code:: ini +.. code:: py - M_SEARCH_EXTERNAL_URL = "https://google.com/search?q=site:doc.magnum.graphics+{query}" + SEARCH_EXTERNAL_URL = "https://google.com/search?q=site:doc.magnum.graphics+{query}" `Showing undocumented symbols and files`_ ----------------------------------------- @@ -586,21 +710,21 @@ shown. In some cases, however, it might be desirable to show undocumented symbols as well --- for example when converting an existing project from vanilla Doxygen to m.css, not all APIs might be documented yet and thus the output would be -incomplete. The :ini:`M_SHOW_UNDOCUMENTED` option unconditionally makes all +incomplete. The :py:`SHOW_UNDOCUMENTED` option unconditionally makes all undocumented symbols, files and directories "appear documented". Note, however, that Doxygen itself doesn't allow to link to undocumented symbols and so even though the undocumented symbols are present in the output, nothing is able to reference them, causing very questionable usability of such approach. A -potential "fix" to this is enabling the :ini:`EXTRACT_ALL` option, but that -exposes all symbols, including private and file-local ones --- which, most +potential "fix" to this is enabling the :ini:`EXTRACT_ALL` Doxyfile option, but +that exposes all symbols, including private and file-local ones --- which, most probably, is *not* what you want. If you have namespaces not documented, Doxygen will by put function docs into file pages --- but it doesn't put them into the XML output, meaning all links to them will lead nowhere and the functions won't appear in search either. -To fix this, enable the :ini:`XML_NS_MEMB_FILE_SCOPE` option as described in -the `Namespace members in file scope`_ section below; if you document all -namespaces this problem will go away as well. +To fix this, enable the :ini:`XML_NS_MEMB_FILE_SCOPE` Doxyfile option as +described in the `Namespace members in file scope`_ section below; if you +document all namespaces this problem will go away as well. `Content`_ ========== @@ -705,7 +829,7 @@ Table of contents for pages is generated only if they specify Doxygen by default doesn't render namespace members for file documentation in its XML output. To match the behavior of stock HTML output, enable the -:ini:`XML_NS_MEMB_FILE_SCOPE` option: +:ini:`XML_NS_MEMB_FILE_SCOPE` Doxyfile option: .. code:: ini @@ -727,10 +851,10 @@ still need to be patched (or worked around). Private virtual functions, if documented, are shown in the output as well, so codebases can properly follow (`Virtuality guidelines by Herb Sutter `_) To avoid also undocumented :cpp:`override`\ s showing in the output, you may -want to disable the :ini:`INHERIT_DOCS` option (which is enabled by default). -Also, please note that while privates are currently unconditionally exported to -the XML output, Doxygen doesn't allow linking to them by default and you have -to enable the :ini:`EXTRACT_PRIV_VIRTUAL` option: +want to disable the :ini:`INHERIT_DOCS` Doxyfile option (which is enabled by +default). Also, please note that while privates are currently unconditionally +exported to the XML output, Doxygen doesn't allow linking to them by default +and you have to enable the :ini:`EXTRACT_PRIV_VIRTUAL` Doxyfile option: .. code:: ini @@ -764,8 +888,8 @@ typedefs, variables and :cpp:`#define`\ s. The rules are: to some class, these also have the :cpp:`#include` shown in case it's different from the class :cpp:`include`. -This feature is enabled by default, disable :ini:`SHOW_INCLUDE_FILES` to hide -all :cpp:`#include`-related information: +This feature is enabled by default, disable :ini:`SHOW_INCLUDE_FILES` in the +Doxyfile to hide all :cpp:`#include`-related information: .. code:: ini @@ -1224,12 +1348,19 @@ label rendering and For example (the ``@m_class`` is the same as described in .. code:: ini - M_VERSION_LABELS = YES ALIASES = \ "m_class{1}=@xmlonly@endxmlonly" \ "m_since{2}=@since @m_class{m-label m-success m-flat} @ref changelog-\1-\2 \"since v\1.\2\"" \ "m_deprecated_since{2}=@since deprecated in v\1.\2 @deprecated" +.. class:: m-noindent + +in the Doxyfile, and the following in ``conf.py``: + +.. code:: py + + VERSION_LABELS = True + With the above configuration, the following markup will render a :label-flat-success:`since v1.3` label leading to a page named ``changelog-1-3`` next to both function entry and detailed docs in the first case, and a @@ -1264,11 +1395,11 @@ next to both function entry and detailed docs in the first case, and a [--search-no-lookahead-barriers] [--search-no-prefix-merging] [--sort-globbed-files] [--debug] - doxyfile + config Arguments: -- ``doxyfile`` --- where the Doxyfile is +- ``config`` --- where the Doxyfile or conf.py is Options: @@ -1405,18 +1536,20 @@ is an example configuration corresponding to the dark theme: .. code:: ini - HTML_EXTRA_STYLESHEET = \ - https://fonts.googleapis.com/css?family=Source+Sans+Pro:400,400i,600,600i%7CSource+Code+Pro:400,400i,600 \ - ../css/m-dark.css \ - ../css/m-documentation.css - HTML_EXTRA_FILES = \ - ../css/m-theme-dark.css \ - ../css/m-grid.css \ - ../css/m-components.css \ - ../css/m-layout.css \ - ../css/pygments-dark.css \ - ../css/pygments-console.css - M_THEME_COLOR = #22272e + STYLESHEETS = [ + 'https://fonts.googleapis.com/css?family=Source+Sans+Pro:400,400i,600,600i%7CSource+Code+Pro:400,400i,600', + '../css/m-dark.css', + '../css/m-documentation.css' + ] + EXTRA_FILES = [ + '../css/m-theme-dark.css', + '../css/m-grid.css', + '../css/m-components.css', + '../css/m-layout.css', + '../css/pygments-dark.css', + '../css/pygments-console.css' + ] + THEME_COLOR = '#22272e' After making desired changes to the source files, it's possible to postprocess them back to the compiled version using the ``postprocess.py`` utility as @@ -1461,19 +1594,20 @@ Filename Use as ``union*.html`` ======================= ======================================================= -Each template is passed a subset of the ``Doxyfile`` configuration values from -the `Configuration`_ table. Most values are provided as-is depending on their -type, so either strings, booleans, or lists of strings. The exceptions are: - -- The :py:`M_LINKS_NAVBAR1` and :py:`M_LINKS_NAVBAR2` are processed to tuples - in a form :py:`(html, title, url, id, sub)` where either :py:`html` is a - full HTML code for the link and :py:`title`, :py:`url` :py:`id` is empty; - or :py:`html` is :py:`None`, :py:`title` and :py:`url` is a link title and - URL and :py:`id` is compound ID (to use for highlighting active menu item). - The last item, :py:`sub` is a list optionally containing sub-menu items. - The sub-menu items are in a similarly formed tuple, +Each template is passed a subset of the ``Doxyfile`` and ``conf.py`` +configuration values from the `Configuration`_ tables. Most values are provided +as-is depending on their type, so either strings, booleans, or lists of +strings. The exceptions are: + +- The :py:`LINKS_NAVBAR1` and :py:`LINKS_NAVBAR2` are processed to tuples in + a form :py:`(html, title, url, id, sub)` where either :py:`html` is a full + HTML code for the link and :py:`title`, :py:`url` :py:`id` is empty; or + :py:`html` is :py:`None`, :py:`title` and :py:`url` is a link title and URL + and :py:`id` is compound ID (to use for highlighting active menu item). The + last item, :py:`sub` is a list optionally containing sub-menu items. The + sub-menu items are in a similarly formed tuple, :py:`(html, title, url, id)`. -- The :py:`M_FAVICON` is converted to a tuple of :py:`(url, type)` where +- The :py:`FAVICON` is converted to a tuple of :py:`(url, type)` where :py:`url` is the favicon URL and :py:`type` is favicon MIME type to populate the ``type`` attribute of :html:``. @@ -1496,7 +1630,7 @@ argument is an absolute URL. It's useful in cases like this: .. code:: html+jinja - {% for css in HTML_EXTRA_STYLESHEET %} + {% for css in STYLESHEETS %} {% endfor %} @@ -2161,10 +2295,10 @@ Filename Use By default it's those five pages, but you can configure any other pages via the ``--index-pages`` option as mentioned in the `Command-line options`_ section. -Each template is passed a subset of the ``Doxyfile`` configuration values from -the above table and in addition the :py:`FILENAME` and :py:`DOXYGEN_VERSION` -variables as above. The navigation tree is provided in an :py:`index` object, -which has the following properties: +Each template is passed a subset of the ``Doxyfile`` and ``conf.py`` +configuration values from the `Configuration`_ tables and in addition the +:py:`FILENAME` and :py:`DOXYGEN_VERSION` variables as above. The navigation +tree is provided in an :py:`index` object, which has the following properties: .. class:: m-table m-fullwidth diff --git a/documentation/doxygen.py b/documentation/doxygen.py index 8ac8354a..92a4886f 100755 --- a/documentation/doxygen.py +++ b/documentation/doxygen.py @@ -31,6 +31,7 @@ import enum import sys import re import html +import inspect import os import glob import mimetypes @@ -41,8 +42,8 @@ import logging from types import SimpleNamespace as Empty from typing import Tuple, Dict, Any, List +from importlib.machinery import SourceFileLoader from jinja2 import Environment, FileSystemLoader - from pygments import highlight from pygments.formatters import HtmlFormatter from pygments.lexers import TextLexer, BashSessionLexer, get_lexer_by_name, find_lexer_class_for_filename @@ -91,6 +92,56 @@ search_type_map = [ (CssClass.DEFAULT, "var") ] +default_config = { + 'DOXYFILE': 'Doxyfile', + + 'THEME_COLOR': '#22272e', + 'FAVICON': 'favicon-dark.png', + 'LINKS_NAVBAR1': [ + ("Pages", 'pages', []), + ("Namespaces", 'namespaces', []) + ], + 'LINKS_NAVBAR2': [ + ("Classes", 'annotated', []), + ("Files", 'files', []) + ], + + 'STYLESHEETS': [ + 'https://fonts.googleapis.com/css?family=Source+Sans+Pro:400,400i,600,600i%7CSource+Code+Pro:400,400i,600', + '../css/m-dark+documentation.compiled.css'], + 'HTML_HEADER': None, + 'EXTRA_FILES': [], + 'PAGE_HEADER': None, + 'FINE_PRINT': '[default]', + + 'CLASS_INDEX_EXPAND_LEVELS': 1, + 'FILE_INDEX_EXPAND_LEVELS': 1, + 'CLASS_INDEX_EXPAND_INNER': False, + + 'M_MATH_CACHE_FILE': 'm.math.cache', + + 'SEARCH_DISABLED': False, + 'SEARCH_DOWNLOAD_BINARY': False, + 'SEARCH_HELP': +"""

Search for symbols, directories, files, pages or +modules. You can omit any prefix from the symbol or file path; adding a +: or / suffix lists all members of given symbol or +directory.

+

Use ↓ +/ ↑ to navigate through the list, +Enter to go. +Tab autocompletes common prefix, you can +copy a link to the result using ⌘ +L while ⌘ +M produces a Markdown link.

+""", + 'SEARCH_BASE_URL': None, + 'SEARCH_EXTERNAL_URL': None, + + 'SHOW_UNDOCUMENTED': False, + 'VERSION_LABELS': False +} + xref_id_rx = re.compile(r"""(.*)_1(_[a-z-]+[0-9]+|@)$""") slugify_nonalnum_rx = re.compile(r"""[^\w\s-]""") slugify_hyphens_rx = re.compile(r"""[-\s]+""") @@ -109,13 +160,14 @@ class StateCompound: self.parent: str = None class State: - def __init__(self): + def __init__(self, config): self.basedir = '' self.compounds: Dict[str, StateCompound] = {} self.includes: Dict[str, str] = {} self.search: List[Any] = [] self.examples: List[Any] = [] - self.doxyfile: Dict[str, str] = {} + self.doxyfile: Dict[str, Any] = {} + self.config: Dict[str, Any] = config self.images: List[str] = [] self.current = '' # current file being processed (for logging) # Current kind of compound being processed. Affects current_include @@ -758,7 +810,7 @@ def parse_desc_internal(state: State, element: ET.Element, immediate_parent: ET. # Content of @since tags is put as-is into entry description / # details, if enabled. - elif i.attrib['kind'] == 'since' and state.doxyfile['M_VERSION_LABELS']: + elif i.attrib['kind'] == 'since' and state.config['VERSION_LABELS']: since = parse_inline_desc(state, i).strip() assert since.startswith('

') and since.endswith('

') out.since = since[3:-4] @@ -1733,7 +1785,7 @@ def parse_enum(state: State, element: ET.Element): value.brief = parse_desc(state, enumvalue.find('briefdescription')) value.description, value_search_keywords, value.deprecated, value.since = parse_enum_value_desc(state, enumvalue) if value.brief or value.description: - if enum.base_url == state.current_compound_url and not state.doxyfile['M_SEARCH_DISABLED']: + if enum.base_url == state.current_compound_url and not state.config['SEARCH_DISABLED']: result = Empty() result.flags = ResultFlag.from_type(ResultFlag.DEPRECATED if value.deprecated else ResultFlag(0), EntryType.ENUM_VALUE) result.url = enum.base_url + '#' + value.id @@ -1745,12 +1797,11 @@ def parse_enum(state: State, element: ET.Element): state.search += [result] # If either brief or description for this value is present, we want - # to show the detailed enum docs. However, in - # case M_SHOW_UNDOCUMENTED is enabled, the values might have just - # a dummy content in order to make them "appear - # documented". Then it doesn't make sense to repeat the same list - # twice. - if not state.doxyfile['M_SHOW_UNDOCUMENTED'] or value.brief != '': + # to show the detailed enum docs. However, in case + # SHOW_UNDOCUMENTED is enabled, the values might have just a dummy + # content in order to make them "appear documented". + # Then it doesn't make sense to repeat the same list twice. + if not state.config['SHOW_UNDOCUMENTED'] or value.brief != '': enum.has_value_details = True enum.values += [value] @@ -1758,7 +1809,7 @@ def parse_enum(state: State, element: ET.Element): if enum.base_url == state.current_compound_url and (enum.description or enum.has_value_details): enum.has_details = True # has_details might already be True from above if enum.brief or enum.has_details or enum.has_value_details: - if enum.base_url == state.current_compound_url and not state.doxyfile['M_SEARCH_DISABLED']: + if enum.base_url == state.current_compound_url and not state.config['SEARCH_DISABLED']: result = Empty() result.flags = ResultFlag.from_type(ResultFlag.DEPRECATED if enum.deprecated else ResultFlag(0), EntryType.ENUM) result.url = enum.base_url + '#' + enum.id @@ -1834,7 +1885,7 @@ def parse_typedef(state: State, element: ET.Element): typedef.has_details = True # has_details might already be True from above if typedef.brief or typedef.has_details: # Avoid duplicates in search - if typedef.base_url == state.current_compound_url and not state.doxyfile['M_SEARCH_DISABLED']: + if typedef.base_url == state.current_compound_url and not state.config['SEARCH_DISABLED']: result = Empty() result.flags = ResultFlag.from_type(ResultFlag.DEPRECATED if typedef.deprecated else ResultFlag(0), EntryType.TYPEDEF) result.url = typedef.base_url + '#' + typedef.id @@ -1984,7 +2035,7 @@ def parse_func(state: State, element: ET.Element): func.has_details = True # has_details might already be True from above if func.brief or func.has_details: # Avoid duplicates in search - if func.base_url == state.current_compound_url and not state.doxyfile['M_SEARCH_DISABLED']: + if func.base_url == state.current_compound_url and not state.config['SEARCH_DISABLED']: result = Empty() result.flags = ResultFlag.from_type((ResultFlag.DEPRECATED if func.deprecated else ResultFlag(0))|(ResultFlag.DELETED if func.is_deleted else ResultFlag(0)), EntryType.FUNC) result.url = func.base_url + '#' + func.id @@ -2020,7 +2071,7 @@ def parse_var(state: State, element: ET.Element): var.has_details = True # has_details might already be True from above if var.brief or var.has_details: # Avoid duplicates in search - if var.base_url == state.current_compound_url and not state.doxyfile['M_SEARCH_DISABLED']: + if var.base_url == state.current_compound_url and not state.config['SEARCH_DISABLED']: result = Empty() result.flags = ResultFlag.from_type(ResultFlag.DEPRECATED if var.deprecated else ResultFlag(0), EntryType.VAR) result.url = var.base_url + '#' + var.id @@ -2060,7 +2111,7 @@ def parse_define(state: State, element: ET.Element): define.has_details = True # has_details might already be True from above if define.brief or define.has_details: # Avoid duplicates in search - if define.base_url == state.current_compound_url and not state.doxyfile['M_SEARCH_DISABLED']: + if define.base_url == state.current_compound_url and not state.config['SEARCH_DISABLED']: result = Empty() result.flags = ResultFlag.from_type(ResultFlag.DEPRECATED if define.deprecated else ResultFlag(0), EntryType.DEFINE) result.url = define.base_url + '#' + define.id @@ -2128,7 +2179,7 @@ def extract_metadata(state: State, xml): # In order to show also undocumented members, go through all empty # s and fill them with a generic text. - if state.doxyfile['M_SHOW_UNDOCUMENTED']: + if state.config['SHOW_UNDOCUMENTED']: _document_all_stuff(compounddef) compound = StateCompound() @@ -2153,7 +2204,7 @@ def extract_metadata(state: State, xml): # @deprecated, treat it as version in which given feature was deprecated compound.deprecated = None compound.since = None - if state.doxyfile['M_VERSION_LABELS']: + if state.config['VERSION_LABELS']: for i in compounddef.find('detaileddescription').findall('.//simplesect'): if i.attrib['kind'] != 'since': continue since = parse_inline_desc(state, i).strip() @@ -2259,64 +2310,21 @@ def postprocess_state(state: State): state.includes['/'.join(include)] = compound.id - # Assign names and URLs to menu items. The link can be either a predefined - # keyword from the below list, a Doxygen symbol, or a HTML code. The - # template then gets a tuple of (HTML code, title, URL) and either puts - # in the HTML code verbatim (if it's not empty) or creates a link from the - # title and URL. - predefined = { - 'pages': (None, "Pages", 'pages.html'), - 'namespaces': (None, "Namespaces", 'namespaces.html'), - 'modules': (None, "Modules", 'modules.html'), - 'annotated': (None, "Classes", 'annotated.html'), - 'files': (None, "Files", 'files.html') - } - def extract_link(link): - # If this is a HTML code, return it verbatim - if link.startswith(' elements. If it looks like a HTML, take everything until - # the closing , otherwise take everything until the next - # whitespace. - links = [] - while i: - if i.startswith('') + 4 - links += [i[0:end]] - i = i[end:].lstrip() - else: - firstAndRest = i.split(None, 1) - if len(firstAndRest): - links += [firstAndRest[0]] - if len(firstAndRest) == 1: - break; - i = firstAndRest[1] - + # Resolve navbar links that are just an ID + def resolve_link(html, title, url, id): + if not html and not title and not url: + found = state.compounds[id] + title, url = found.name, found.url + return html, title, url, id + for var in 'LINKS_NAVBAR1', 'LINKS_NAVBAR2': + links = [] + for html, title, url, id, sub in state.config[var]: + html, title, url, id = resolve_link(html, title, url, id) sublinks = [] - for sublink in links[1:]: - html, title, url = extract_link(sublink) - sublinks += [(html, title, url, sublink)] - html, title, url = extract_link(links[0]) - navbar_links += [(html, title, url, links[0], sublinks)] - - state.doxyfile[var] = navbar_links - - # Guess MIME type of the favicon - if state.doxyfile['M_FAVICON']: - state.doxyfile['M_FAVICON'] = (state.doxyfile['M_FAVICON'], mimetypes.guess_type(state.doxyfile['M_FAVICON'])[0]) + for i in sub: + sublinks += [resolve_link(*i)] + links += [(html, title, url, id, sublinks)] + state.config[var] = links def build_search_data(state: State, merge_subtrees=True, add_lookahead_barriers=True, merge_prefixes=True) -> bytearray: trie = Trie() @@ -2438,7 +2446,7 @@ def parse_xml(state: State, xml: str): # In order to show also undocumented members, go through all empty # s and fill them with a generic text. - if state.doxyfile['M_SHOW_UNDOCUMENTED']: + if state.config['SHOW_UNDOCUMENTED']: _document_all_stuff(compounddef) # Ignoring compounds w/o any description, except for groups, @@ -3152,7 +3160,7 @@ def parse_xml(state: State, xml: str): # Add the compound to search data, if it's documented # TODO: add example sources there? how? - if not state.doxyfile['M_SEARCH_DISABLED'] and not compound.kind == 'example' and (compound.kind == 'group' or compound.brief or compounddef.find('detaileddescription')): + if not state.config['SEARCH_DISABLED'] and not compound.kind == 'example' and (compound.kind == 'group' or compound.brief or compounddef.find('detaileddescription')): if compound.kind == 'namespace': kind = EntryType.NAMESPACE elif compound.kind == 'struct': @@ -3303,7 +3311,7 @@ def parse_index_xml(state: State, xml): return parsed -def parse_doxyfile(state: State, doxyfile, config = None): +def parse_doxyfile(state: State, doxyfile, values = None): state.basedir = os.path.dirname(doxyfile) logging.debug("Parsing configuration from {}".format(doxyfile)) @@ -3313,55 +3321,30 @@ def parse_doxyfile(state: State, doxyfile, config = None): variable_continuation_re = re.compile(r"""^\s*(##!\s*)?(?P[A-Z_]+)\s*\+=\s*(?P['"]?)(?P.*)(?P=quote)\s*(?P\\?)$""") continuation_re = re.compile(r"""^\s*(##!\s*)?(?P['"]?)(?P.*)(?P=quote)\s*(?P\\?)$""") - default_config = { + default_values = { 'PROJECT_NAME': ['My Project'], 'PROJECT_LOGO': [''], 'OUTPUT_DIRECTORY': [''], 'XML_OUTPUT': ['xml'], 'HTML_OUTPUT': ['html'], - 'HTML_EXTRA_STYLESHEET': [ - 'https://fonts.googleapis.com/css?family=Source+Sans+Pro:400,400i,600,600i%7CSource+Code+Pro:400,400i,600', - '../css/m-dark+documentation.compiled.css'], - 'HTML_EXTRA_FILES': [], 'DOT_FONTNAME': ['Helvetica'], 'DOT_FONTSIZE': ['10'], - 'SHOW_INCLUDE_FILES': ['YES'], - - 'M_CLASS_TREE_EXPAND_LEVELS': ['1'], - 'M_FILE_TREE_EXPAND_LEVELS': ['1'], - 'M_EXPAND_INNER_TYPES': ['NO'], - 'M_THEME_COLOR': ['#22272e'], - 'M_FAVICON': ['favicon-dark.png'], - 'M_LINKS_NAVBAR1': ['pages', 'namespaces'], - 'M_LINKS_NAVBAR2': ['annotated', 'files'], - 'M_MATH_CACHE_FILE': ['m.math.cache'], - 'M_PAGE_FINE_PRINT': ['[default]'], - 'M_SEARCH_DISABLED': ['NO'], - 'M_SEARCH_DOWNLOAD_BINARY': ['NO'], - 'M_SEARCH_HELP': [ -"""

Search for symbols, directories, files, pages or -modules. You can omit any prefix from the symbol or file path; adding a -: or / suffix lists all members of given symbol or -directory.

-

Use ↓ -/ ↑ to navigate through the list, -Enter to go. -Tab autocompletes common prefix, you can -copy a link to the result using ⌘ -L while ⌘ -M produces a Markdown link.

-"""], - 'M_SEARCH_BASE_URL': [''], - 'M_SEARCH_EXTERNAL_URL': [''], - 'M_SHOW_UNDOCUMENTED': ['NO'], - 'M_VERSION_LABELS': ['NO'] + 'SHOW_INCLUDE_FILES': ['YES'] } # Defaults so we don't fail with minimal Doxyfiles and also that the # user-provided Doxygen can append to them. They are later converted to # string or kept as a list based on type, so all have to be a list of # strings now. - if not config: config = copy.deepcopy(default_config) + # + # If there are no `values`, it means this is a top-level call (not recursed + # from Doxyfile @INCLUDEs). In that case (and only in that case) we + # finalize the config values (such as expanding FAVICON or LINKS_NAVBAR). + if not values: + finalize = True + values = copy.deepcopy(default_values) + else: + finalize = False def parse_value(var): if var.group('quote') in ['"', '\'']: @@ -3390,7 +3373,7 @@ copy a link to the result using ⌘ if continued_line: var = continuation_re.match(line) value, backslash = parse_value(var) - config[continued_line] += value + values[continued_line] += value if not backslash: continued_line = None continue @@ -3402,10 +3385,10 @@ copy a link to the result using ⌘ # Another file included, parse it if key == '@INCLUDE': - parse_doxyfile(state, os.path.join(os.path.dirname(doxyfile), ' '.join(value)), config) + parse_doxyfile(state, os.path.join(os.path.dirname(doxyfile), ' '.join(value)), values) assert not backslash else: - config[key] = value + values[key] = value if backslash: continued_line = key continue @@ -3414,9 +3397,9 @@ copy a link to the result using ⌘ var = variable_continuation_re.match(line) if var: key = var.group('key') - if not key in config: config[key] = [] + if not key in values: values[key] = [] value, backslash = parse_value(var) - config[key] += value + values[key] += value if backslash: continued_line = key # only because coverage.py can't handle continue @@ -3432,61 +3415,172 @@ copy a link to the result using ⌘ logging.warning("{}: unmatchable line {}".format(doxyfile, line)) # pragma: no cover # Some values are set to empty in the default-generated Doxyfile but they - # shouldn't be empty. Revert them to our defaults. - # TODO: this may behave strange in corner cases where multiple @INCLUDEd - # files set or append to the same thing + # shouldn't be empty. Delete the variable ín that case so it doesn't + # override our defaults. for i in ['HTML_EXTRA_STYLESHEET']: - if i in config and not config[i]: - config[i] = default_config[i] - - # String values that we want - for i in ['PROJECT_NAME', - 'PROJECT_BRIEF', - 'PROJECT_LOGO', - 'OUTPUT_DIRECTORY', - 'HTML_OUTPUT', - 'XML_OUTPUT', - 'DOT_FONTNAME', - 'M_MAIN_PROJECT_URL', - 'M_HTML_HEADER', - 'M_PAGE_HEADER', - 'M_PAGE_FINE_PRINT', - 'M_THEME_COLOR', - 'M_FAVICON', - 'M_MATH_CACHE_FILE', - 'M_SEARCH_HELP', - 'M_SEARCH_EXTERNAL_URL', - 'M_SEARCH_BASE_URL']: - if i in config: state.doxyfile[i] = '\n'.join(config[i]) - - # Int values that we want - for i in ['DOT_FONTSIZE', - 'M_CLASS_TREE_EXPAND_LEVELS', - 'M_FILE_TREE_EXPAND_LEVELS']: - if i in config: state.doxyfile[i] = int(' '.join(config[i])) - - # Boolean values that we want - for i in ['CREATE_SUBDIRS', - 'JAVADOC_AUTOBRIEF', - 'QT_AUTOBRIEF', - 'INTERNAL_DOCS', - 'SHOW_INCLUDE_FILES', - 'M_EXPAND_INNER_TYPES', - 'M_SEARCH_DISABLED', - 'M_SEARCH_DOWNLOAD_BINARY', - 'M_SHOW_UNDOCUMENTED', - 'M_VERSION_LABELS']: - if i in config: state.doxyfile[i] = ' '.join(config[i]) == 'YES' - - # List values that we want. Drop empty lines. - for i in ['TAGFILES', - 'HTML_EXTRA_STYLESHEET', - 'HTML_EXTRA_FILES', - 'M_LINKS_NAVBAR1', - 'M_LINKS_NAVBAR2']: - if i in config: - state.doxyfile[i] = [line for line in config[i] if line] + if i in values and not values[i]: del values[i] + + # Parse recognized Doxyfile values with desired type. The second tuple + # value denotes that the Doxyfile value is an alias to a value in conf.py, + # in which case we'll save it there instead. + for key, alias, type_ in [ + # Order roughly the same as in python.py default_config to keep those + # two consistent + ('PROJECT_NAME', None, str), + ('PROJECT_BRIEF', None, str), + ('PROJECT_LOGO', None, str), + ('M_MAIN_PROJECT_URL', 'MAIN_PROJECT_URL', str), + + ('OUTPUT_DIRECTORY', None, str), + ('HTML_OUTPUT', None, str), + ('XML_OUTPUT', None, str), + + ('DOT_FONTNAME', None, str), + ('DOT_FONTSIZE', None, int), + ('CREATE_SUBDIRS', None, bool), # processing fails below if this is set + ('JAVADOC_AUTOBRIEF', None, bool), + ('QT_AUTOBRIEF', None, bool), + ('INTERNAL_DOCS', None, bool), + ('SHOW_INCLUDE_FILES', None, bool), + ('TAGFILES', None, list), + + ('M_THEME_COLOR', 'THEME_COLOR', str), + ('M_FAVICON', 'FAVICON', str), # plus special handling below + ('HTML_EXTRA_STYLESHEET', 'STYLESHEETS', list), + ('HTML_EXTRA_FILES', 'EXTRA_FILES', list), + # M_LINKS_NAVBAR1 and M_LINKS_NAVBAR2 have special handling below + + ('M_HTML_HEADER', 'HTML_HEADER', str), + ('M_PAGE_HEADER', 'PAGE_HEADER', str), + ('M_PAGE_FINE_PRINT', 'FINE_PRINT', str), + + ('M_CLASS_TREE_EXPAND_LEVELS', 'CLASS_INDEX_EXPAND_LEVELS', int), + ('M_EXPAND_INNER_TYPES', 'CLASS_INDEX_EXPAND_INNER', bool), + ('M_FILE_TREE_EXPAND_LEVELS', 'FILE_INDEX_EXPAND_LEVELS', int), + + ('M_SEARCH_DISABLED', 'SEARCH_DISABLED', bool), + ('M_SEARCH_DOWNLOAD_BINARY', 'SEARCH_DOWNLOAD_BINARY', bool), + ('M_SEARCH_HELP', 'SEARCH_HELP', str), + ('M_SEARCH_BASE_URL', 'SEARCH_BASE_URL', str), + ('M_SEARCH_EXTERNAL_URL', 'SEARCH_EXTERNAL_URL', str), + + ('M_SHOW_UNDOCUMENTED', 'SHOW_UNDOCUMENTED', bool), + ('M_VERSION_LABELS', 'VERSION_LABELS', bool), + + ('M_MATH_CACHE_FILE', 'M_MATH_CACHE_FILE', str), + ]: + if key not in values: continue + + if type_ is str: + value = '\n'.join(values[key]) + elif type_ is int: + value = int(' '.join(values[key])) + elif type_ is bool: + value = ' '.join(values[key]) == 'YES' + elif type_ is list: + value = [line for line in values[key] if line] # Drop empty lines + else: # pragma: no cover + assert False + if alias: + state.config[alias] = value + else: + state.doxyfile[key] = value + + # Process M_LINKS_NAVBAR[12] into either (HTML, sublinks) or + # (title, URL, ID, sublinks), with sublinks being either + # (HTML) or (title, URL, ID). Those are then saved into LINKS_NAVBAR[12] + # and processed further. + predefined = { + 'pages': ("Pages", 'pages.html'), + 'namespaces': ("Namespaces", 'namespaces.html'), + 'modules': ("Modules", 'modules.html'), + 'annotated': ("Classes", 'annotated.html'), + 'files': ("Files", 'files.html') + } + def extract_link(link): + # If this is a HTML code, return it as a one-item tuple + if link.startswith(' elements. If it looks like a HTML, take everything until + # the closing , otherwise take everything until the next + # whitespace. + links = [] + while i: + if i.startswith('') + 4 + links += [i[0:end]] + i = i[end:].lstrip() + else: + firstAndRest = i.split(None, 1) + if len(firstAndRest): + links += [firstAndRest[0]] + if len(firstAndRest) == 1: + break; + i = firstAndRest[1] + + sublinks = [] + for sublink in links[1:]: + sublinks += [extract_link(sublink)] + navbar_links += [extract_link(links[0]) + (sublinks, )] + + state.config[alias] = navbar_links + + # Below we finalize the config values, converting them to formats that are + # easy to understand by the code / templates (but not easy to write from + # the user PoV). If this is not a top-level call (but a recursed one from + # @INCLUDE), we exit, to avoid finalizing everything multiple times. + if not finalize: return + + # Convert the links from either (html, ) or (title, id) to a + # (html, title, url, id) tuple so it's easier to process by the template. + # The url is not known at this point and will be filled later in + # postprocess_state(). + def expand_link(link): + if len(link) == 1: + return (link[0], None, None, None) + else: + assert len(link) == 2 + if link[1] in predefined: + url = predefined[link[1]][1] + if not link[0]: title = predefined[link[1]][0] + else: title = link[0] + else: + title = None + url = None + return (None, title, url, link[1]) + for key in ('LINKS_NAVBAR1', 'LINKS_NAVBAR2'): + links = [] + for i in state.config[key]: + sublinks = [] + for subi in i[-1]: + sublinks += [expand_link(subi)] + links += [expand_link(i[:-1]) + (sublinks, )] + state.config[key] = links + + # Guess MIME type of the favicon. It's supplied explicitly when coming from + # a conf.py + if 'FAVICON' in state.config and state.config['FAVICON']: + state.config['FAVICON'] = (state.config['FAVICON'], mimetypes.guess_type(state.config['FAVICON'])[0]) + + # Fail if this option is set if state.doxyfile.get('CREATE_SUBDIRS', False): logging.fatal("{}: CREATE_SUBDIRS is not supported, sorry. Disable it and try again.".format(doxyfile)) raise NotImplementedError @@ -3504,8 +3598,8 @@ def run(state: State, *, templates=default_templates, wildcard=default_wildcard, # If math rendering cache is not disabled, load the previous version. If # there is no cache, reset the cache to an empty state to avoid # order-dependent issues when testing - math_cache_file = os.path.join(state.basedir, state.doxyfile['OUTPUT_DIRECTORY'], state.doxyfile['M_MATH_CACHE_FILE']) - if state.doxyfile['M_MATH_CACHE_FILE'] and os.path.exists(math_cache_file): + math_cache_file = os.path.join(state.basedir, state.doxyfile['OUTPUT_DIRECTORY'], state.config['M_MATH_CACHE_FILE']) + if state.config['M_MATH_CACHE_FILE'] and os.path.exists(math_cache_file): latex2svgextra.unpickle_cache(math_cache_file) else: latex2svgextra.unpickle_cache(None) @@ -3561,7 +3655,8 @@ def run(state: State, *, templates=default_templates, wildcard=default_wildcard, DOXYGEN_VERSION=parsed.version, FILENAME=file, SEARCHDATA_FORMAT_VERSION=searchdata_format_version, - **state.doxyfile) + # TODO: whitelist only what matters from doxyfile + **state.doxyfile, **state.config) output = os.path.join(html_output, file) with open(output, 'wb') as f: @@ -3580,7 +3675,8 @@ def run(state: State, *, templates=default_templates, wildcard=default_wildcard, DOXYGEN_VERSION=parsed.version, FILENAME=parsed.compound.url, SEARCHDATA_FORMAT_VERSION=searchdata_format_version, - **state.doxyfile) + # TODO: whitelist only what matters from doxyfile + **state.doxyfile, **state.config) output = os.path.join(html_output, parsed.compound.url) with open(output, 'wb') as f: @@ -3607,7 +3703,8 @@ def run(state: State, *, templates=default_templates, wildcard=default_wildcard, DOXYGEN_VERSION=None, FILENAME='index.html', SEARCHDATA_FORMAT_VERSION=searchdata_format_version, - **state.doxyfile) + # TODO: whitelist only what matters from doxyfile + **state.doxyfile, **state.config) output = os.path.join(html_output, 'index.html') with open(output, 'wb') as f: f.write(rendered.encode('utf-8')) @@ -3617,12 +3714,12 @@ def run(state: State, *, templates=default_templates, wildcard=default_wildcard, # also for nested templates :( f.write(b'\n') - if not state.doxyfile['M_SEARCH_DISABLED']: + if not state.config['SEARCH_DISABLED']: logging.debug("building search data for {} symbols".format(len(state.search))) data = build_search_data(state, add_lookahead_barriers=search_add_lookahead_barriers, merge_subtrees=search_merge_subtrees, merge_prefixes=search_merge_prefixes) - if state.doxyfile['M_SEARCH_DOWNLOAD_BINARY']: + if state.config['SEARCH_DOWNLOAD_BINARY']: with open(os.path.join(html_output, searchdata_filename), 'wb') as f: f.write(data) else: @@ -3630,11 +3727,12 @@ def run(state: State, *, templates=default_templates, wildcard=default_wildcard, f.write(base85encode_search_data(data)) # OpenSearch metadata, in case we have the base URL - if state.doxyfile['M_SEARCH_BASE_URL']: + if state.config['SEARCH_BASE_URL']: logging.debug("writing OpenSearch metadata file") template = env.get_template('opensearch.xml') - rendered = template.render(**state.doxyfile) + # TODO: whitelist only what matters from doxyfile + rendered = template.render(**state.doxyfile, **state.config) output = os.path.join(html_output, 'opensearch.xml') with open(output, 'wb') as f: f.write(rendered.encode('utf-8')) @@ -3645,7 +3743,7 @@ def run(state: State, *, templates=default_templates, wildcard=default_wildcard, f.write(b'\n') # Copy all referenced files - for i in state.images + state.doxyfile['HTML_EXTRA_STYLESHEET'] + state.doxyfile['HTML_EXTRA_FILES'] + ([state.doxyfile['PROJECT_LOGO']] if state.doxyfile['PROJECT_LOGO'] else []) + ([state.doxyfile['M_FAVICON'][0]] if state.doxyfile['M_FAVICON'] else []) + ([] if state.doxyfile['M_SEARCH_DISABLED'] else ['search.js']): + for i in state.images + state.config['STYLESHEETS'] + state.config['EXTRA_FILES'] + ([state.doxyfile['PROJECT_LOGO']] if state.doxyfile['PROJECT_LOGO'] else []) + ([state.config['FAVICON'][0]] if state.config['FAVICON'] else []) + ([] if state.config['SEARCH_DISABLED'] else ['search.js']): # Skip absolute URLs if urllib.parse.urlparse(i).netloc: continue @@ -3665,12 +3763,12 @@ def run(state: State, *, templates=default_templates, wildcard=default_wildcard, shutil.copy(i, os.path.join(html_output, os.path.basename(file_out))) # Save updated math cache file - if state.doxyfile['M_MATH_CACHE_FILE']: + if state.config['M_MATH_CACHE_FILE']: latex2svgextra.pickle_cache(math_cache_file) if __name__ == '__main__': # pragma: no cover parser = argparse.ArgumentParser() - parser.add_argument('doxyfile', help="where the Doxyfile is") + parser.add_argument('config', help="where the Doxyfile or conf.py is") parser.add_argument('--templates', help="template directory", default=default_templates) parser.add_argument('--wildcard', help="only process files matching the wildcard", default=default_wildcard) parser.add_argument('--index-pages', nargs='+', help="index page templates", default=default_index_pages) @@ -3687,10 +3785,19 @@ if __name__ == '__main__': # pragma: no cover else: logging.basicConfig(level=logging.INFO) - # Make the Doxyfile path absolute, otherwise everything gets messed up - doxyfile = os.path.abspath(args.doxyfile) + config = copy.deepcopy(default_config) + + if args.config.endswith('.py'): + name, _ = os.path.splitext(os.path.basename(args.config)) + module = SourceFileLoader(name, args.config).load_module() + if module is not None: + config.update((k, v) for k, v in inspect.getmembers(module) if k.isupper()) + doxyfile = os.path.join(os.path.dirname(os.path.abspath(args.config)), config['DOXYFILE']) + else: + # Make the Doxyfile path absolute, otherwise everything gets messed up + doxyfile = os.path.abspath(args.doxyfile) - state = State() + state = State(config) parse_doxyfile(state, doxyfile) # Doxygen is stupid and can't create nested directories, create the input @@ -3698,7 +3805,7 @@ if __name__ == '__main__': # pragma: no cover os.makedirs(state.doxyfile['OUTPUT_DIRECTORY'], exist_ok=True) if not args.no_doxygen: - logging.debug("running Doxygen on {}".format(args.doxyfile)) + logging.debug("running Doxygen on {}".format(doxyfile)) subprocess.run(["doxygen", doxyfile], cwd=os.path.dirname(doxyfile), check=True) run(state, templates=os.path.abspath(args.templates), wildcard=args.wildcard, index_pages=args.index_pages, search_merge_subtrees=not args.search_no_subtree_merging, search_add_lookahead_barriers=not args.search_no_lookahead_barriers, search_merge_prefixes=not args.search_no_prefix_merging) diff --git a/documentation/templates/doxygen/annotated.html b/documentation/templates/doxygen/annotated.html index 338ad881..132f221d 100644 --- a/documentation/templates/doxygen/annotated.html +++ b/documentation/templates/doxygen/annotated.html @@ -6,7 +6,7 @@
    {% for i in index.symbols recursive %} {% if i.children %} -
  • +
  • {{ i.kind }} {{ i.name }}{% if i.deprecated %} {{ i.deprecated }}{% endif %}{% if i.since %} {{ i.since }}{% endif %} {{ i.brief }}
      {{ loop(i.children)|rtrim|indent(4, true) }} diff --git a/documentation/templates/doxygen/base.html b/documentation/templates/doxygen/base.html index 30477a99..f2129764 100644 --- a/documentation/templates/doxygen/base.html +++ b/documentation/templates/doxygen/base.html @@ -3,39 +3,39 @@ {% block title %}{{ PROJECT_NAME }}{% if PROJECT_BRIEF %} {{ PROJECT_BRIEF }}{% endif %}{% endblock %} - {% for css in HTML_EXTRA_STYLESHEET %} + {% for css in STYLESHEETS %} {% endfor %} - {% if M_FAVICON %} - + {% if FAVICON %} + {% endif %} - {% if not M_SEARCH_DISABLED and M_SEARCH_BASE_URL %} + {% if not SEARCH_DISABLED and SEARCH_BASE_URL %} {% endif %} {% block header_links %} {% endblock %} - {% if M_THEME_COLOR %} - + {% if THEME_COLOR %} + {% endif %} - {% if M_HTML_HEADER %} - {{ M_HTML_HEADER|rtrim|indent(2) }} + {% if HTML_HEADER %} + {{ HTML_HEADER|rtrim|indent(2) }} {% endif %}