From: Vladimír Vondruš 
Date: Fri, 30 Aug 2019 16:26:18 +0000 (+0200)
Subject: documentation/python, m.sphinx: support for external enum value docs.
X-Git-Url: https://www.chiark.greenend.org.uk/ucgi/~cjwatson/git?a=commitdiff_plain;h=820350718027f5d998790dc225536cb5d77228e4;p=blog.git
documentation/python, m.sphinx: support for external enum value docs.
---
diff --git a/doc/documentation/python.rst b/doc/documentation/python.rst
index 3b54d6c0..cfba339b 100644
--- a/doc/documentation/python.rst
+++ b/doc/documentation/python.rst
@@ -780,36 +780,38 @@ plugin might or might not use them.
 
 .. class:: m-table
 
-=========================== ===================================================
-Keyword argument            Content
-=========================== ===================================================
-:py:`mcss_settings`         Dict containing all m.css settings
-:py:`jinja_environment`     Jinja2 environment. Useful for adding new filters
-                            etc.
-:py:`module_doc_contents`   Module documentation contents
-:py:`class_doc_contents`    Class documentation contents
-:py:`enum_doc_contents`     Enum documentation contents
-:py:`function_doc_contents` Function documentation contents
-:py:`property_doc_contents` Property documentation contents
-:py:`data_doc_contents`     Data documentation contents
-:py:`hooks_post_crawl`      Hooks to call after the initial name crawl
-:py:`hooks_scope_enter`     Hooks to call on scope enter
-:py:`hooks_scope_exit`      Hooks to call on scope exit
-:py:`hooks_docstring`       Hooks to call when parsing a docstring
-:py:`hooks_pre_page`        Hooks to call before each page gets rendered
-:py:`hooks_post_run`        Hooks to call at the very end of the script run
-=========================== ===================================================
+=============================== ===============================================
+Keyword argument                Content
+=============================== ===============================================
+:py:`mcss_settings`             Dict containing all m.css settings
+:py:`jinja_environment`         Jinja2 environment. Useful for adding new
+                                filters etc.
+:py:`module_doc_contents`       Module documentation contents
+:py:`class_doc_contents`        Class documentation contents
+:py:`enum_doc_contents`         Enum documentation contents
+:py:`enum_value_doc_contents`   Enum documentation contents
+:py:`function_doc_contents`     Function documentation contents
+:py:`property_doc_contents`     Property documentation contents
+:py:`data_doc_contents`         Data documentation contents
+:py:`hooks_post_crawl`          Hooks to call after the initial name crawl
+:py:`hooks_scope_enter`         Hooks to call on scope enter
+:py:`hooks_scope_exit`          Hooks to call on scope exit
+:py:`hooks_docstring`           Hooks to call when parsing a docstring
+:py:`hooks_pre_page`            Hooks to call before each page gets rendered
+:py:`hooks_post_run`            Hooks to call at the very end of the script run
+=============================== ===============================================
 
-The :py:`module_doc_contents`, :py:`class_doc_contents`,
-:py:`function_doc_contents`, :py:`property_doc_contents` and
-:py:`data_doc_contents` variables are :py:`Dict[str, Dict[str, str]]`, where
-the first level is a name and second level are key/value pairs of the actual
-HTML documentation content. Plugins that parse extra documentation inputs (such
-as `m.sphinx`_) are supposed to add to the dict, which is then used to fill the
-actual documentation contents. The following corresponds to the documentation
-source shown in the `External documentation content`_ section below. Note that
-the dict can already have existing entries added from elsewhere, so it's
-important to avoid fully overwriting it:
+The :py:`module_doc_contents`, :py:`class_doc_contents`, :py:`enum_doc_contents`,
+:py:`enum_value_doc_contents`, :py:`function_doc_contents`,
+:py:`property_doc_contents` and :py:`data_doc_contents` variables are
+:py:`Dict[str, Dict[str, str]]`, where the first level is a name and second
+level are key/value pairs of the actual HTML documentation content. Plugins
+that parse extra documentation inputs (such as `m.sphinx`_) are supposed to add
+to the dict, which is then used to fill the actual documentation contents. The
+following corresponds to the documentation source shown in the
+`External documentation content`_ section below. Note that the dict can already
+have existing entries added from elsewhere, so it's important to avoid fully
+overwriting it:
 
 .. code:: py
 
@@ -1160,7 +1162,7 @@ Property                    Description
 :py:`value.id`              Value ID [5]_
 :py:`value.value`           Value value. Set to :py:`None` if no value is
                             available.
-:py:`value.summary`         Value doc summary
+:py:`value.content`         Value documentation, if any
 =========================== ===================================================
 
 `Function properties`_
diff --git a/doc/plugins/sphinx.rst b/doc/plugins/sphinx.rst
index c0cf1606..e5b839db 100644
--- a/doc/plugins/sphinx.rst
+++ b/doc/plugins/sphinx.rst
@@ -253,15 +253,16 @@ doing the following will ensure it can be easily used:
 ========================================================
 
 In the Python doc theme, the :rst:`.. py:module::`, :rst:`.. py:class::`,
-:rst:`.. py:enum::`, :rst:`.. py:function::`, :rst:`.. py:property::` and
-:rst:`.. py:data::` directives provide a way to supply module, class, enum,
-function / method, property and data documentation content.
+:rst:`.. py:enum::`, :rst:`.. py:enumvalue::`, :rst:`.. py:function::`,
+:rst:`.. py:property::` and :rst:`.. py:data::` directives provide a way to
+supply module, class, enum, function / method, property and data documentation
+content.
 
 Directive option is the name to document, directive contents are the actual
-contents; in addition all the directives have the :py:`:summary:` option that
-can override the docstring extracted using inspection. No restrictions are made
-on the contents, it's also possible to make use of any additional plugins in
-the markup. Example:
+contents; in addition all directives except :rst:`.. py:enumvalue::` have an
+:py:`:summary:` option that can override the docstring extracted using
+inspection. No restrictions are made on the contents, it's also possible to
+make use of any additional plugins in the markup. Example:
 
 .. code:: rst
 
@@ -354,11 +355,30 @@ the :rst:`.. py:data::` / :rst:`.. py:property::` directives `described <#data>`
         Provides a key/value storage with :math:`\mathcal{O}(\log{}n)`-complexity
         access.
 
-`Enums`_
---------
+`Enums and enum values`_
+------------------------
 
-Use :rst:`.. py:enum::` for documenting enums. This directive doesn't support
-any additional options besides :rst:`:summary:`.
+Use :rst:`.. py:enum::` for documenting enums. Values can be documented either
+using the :rst:`.. py:enumvalue::` directive, or in case of short descriptions,
+conveniently directly in the :rst:`.. py:enum::` directive via
+:py:`:value :` options. Example:
+
+.. code:: rst
+
+    .. py:enum:: mymodule.MemoryUsage
+        :summary: Specifies memory usage configuration
+        :value LOW: Optimized for low-memory big-storage devices, such as
+            refrigerators.
+        :value HIGH: The memory usage will make you angry.
+
+    .. py:enumvalue:: mymodule.MemoryUsage.DEFAULT
+
+        Default memory usage. Behavior depends on platform:
+
+        -   On low-memory devices such as refrigerators equivalent to :ref:`LOW`.
+        -   On high-end desktop PCs, this is equivalent to :ref:`HIGH`.
+        -   On laptops, this randomly chooses between the two values based
+            Murphy's law. Enjoy the battery life when you need it the least.
 
 `Functions`_
 ------------
diff --git a/documentation/python.py b/documentation/python.py
index cd757a06..09de8234 100755
--- a/documentation/python.py
+++ b/documentation/python.py
@@ -179,6 +179,7 @@ class State:
         self.module_docs: Dict[str, Dict[str, str]] = {}
         self.class_docs: Dict[str, Dict[str, str]] = {}
         self.enum_docs: Dict[str, Dict[str, str]] = {}
+        self.enum_value_docs: Dict[str, Dict[str, str]] = {}
         self.function_docs: Dict[str, Dict[str, str]] = {}
         self.property_docs: Dict[str, Dict[str, str]] = {}
         self.data_docs: Dict[str, Dict[str, str]] = {}
@@ -1127,15 +1128,12 @@ def extract_class_doc(state: State, entry: Empty):
     return out
 
 def extract_enum_doc(state: State, entry: Empty):
-    # Call all scope enter hooks first
-    for hook in state.hooks_pre_scope:
-        hook(type=entry.type, path=entry.path)
-
     out = Empty()
     out.name = entry.path[-1]
     out.id = state.config['ID_FORMATTER'](EntryType.ENUM, entry.path[-1:])
     out.values = []
     out.has_value_details = False
+    out.has_details = False
 
     # The happy case
     if issubclass(entry.object, enum.Enum):
@@ -1144,8 +1142,6 @@ def extract_enum_doc(state: State, entry: Empty):
             docstring = ''
         else:
             docstring = entry.object.__doc__
-        out.summary, out.content = extract_docs(state, state.enum_docs, entry.type, entry.path, docstring)
-        out.has_details = bool(out.content)
 
         out.base = extract_type(entry.object.__base__)
         if out.base: out.base_link = make_name_link(state, entry.path, out.base)
@@ -1158,17 +1154,12 @@ def extract_enum_doc(state: State, entry: Empty):
             value.value = html.escape(repr(i.value))
 
             # Value doc gets by default inherited from the enum, that's useless
+            # This gets further processed below.
             if i.__doc__ == entry.object.__doc__:
-                docstring = ''
+                value.content = ''
             else:
-                docstring = i.__doc__
+                value.content = i.__doc__
 
-            # TODO: external summary for enum values
-            value.summary = extract_docs(state, {}, EntryType.ENUM_VALUE, [], docstring, summary_only=True)
-
-            if value.summary:
-                out.has_details = True
-                out.has_value_details = True
             out.values += [value]
 
     # Pybind11 enums are ... different
@@ -1178,9 +1169,9 @@ def extract_enum_doc(state: State, entry: Empty):
         # Pybind 2.4 puts enum value docs inside the docstring. We don't parse
         # that yet and it adds clutter to the output (especially if the values
         # aren't documented), so cut that away
-        # TODO: implement this
-        out.summary, out.content = extract_docs(state, state.enum_docs, entry.type, entry.path, entry.object.__doc__.partition('\n\n')[0])
-        out.has_details = bool(out.content)
+        # TODO: implement this and populate each value.content
+        docstring = entry.object.__doc__.partition('\n\n')[0]
+
         out.base = None
 
         for name, v in entry.object.__members__.items():
@@ -1188,10 +1179,33 @@ def extract_enum_doc(state: State, entry: Empty):
             value. name = name
             value.id = state.config['ID_FORMATTER'](EntryType.ENUM_VALUE, entry.path[-1:] + [name])
             value.value = int(v)
-            # TODO: external summary for enum values
-            value.summary = ''
+            value.content = ''
             out.values += [value]
 
+    # Call all scope enter before rendering the docs
+    for hook in state.hooks_pre_scope:
+        hook(type=entry.type, path=entry.path)
+
+    out.summary, out.content = extract_docs(state, state.enum_docs, entry.type, entry.path, docstring)
+    if out.content: out.has_details = True
+
+    for value in out.values:
+        # Keeping the same scope for the value docs as for the outer scope.
+        # There's no distinction between summary and content for enum
+        # values so put that together in one. The summary is only produced by
+        # the raw docstring parser, the m.sphinx directives always produce only
+        # the content.
+        summary, value.content = extract_docs(state, state.enum_value_docs, EntryType.ENUM_VALUE, entry.path + [value.name], value.content)
+        if summary:
+            value.content = '{}
\n{}'.format(summary, value.content).rstrip()
+        if value.content:
+            out.has_details = True
+            out.has_value_details = True
+
+    # Call all scope exit hooks after
+    for hook in state.hooks_post_scope:
+        hook(type=entry.type, path=entry.path)
+
     if not state.config['SEARCH_DISABLED']:
         page_url = state.name_map['.'.join(entry.path[:-1])].url
 
@@ -1210,10 +1224,6 @@ def extract_enum_doc(state: State, entry: Empty):
             result.name = value.name
             state.search += [result]
 
-    # Call all scope exit hooks last
-    for hook in state.hooks_post_scope:
-        hook(type=entry.type, path=entry.path)
-
     return out
 
 def extract_function_doc(state: State, parent, entry: Empty) -> List[Any]:
@@ -2242,6 +2252,7 @@ def run(basedir, config, *, templates=default_templates, search_add_lookahead_ba
             module_doc_contents=state.module_docs,
             class_doc_contents=state.class_docs,
             enum_doc_contents=state.enum_docs,
+            enum_value_doc_contents=state.enum_value_docs,
             function_doc_contents=state.function_docs,
             property_doc_contents=state.property_docs,
             data_doc_contents=state.data_docs,
diff --git a/documentation/templates/python/details-enum.html b/documentation/templates/python/details-enum.html
index 79f21cd4..c884ebaf 100644
--- a/documentation/templates/python/details-enum.html
+++ b/documentation/templates/python/details-enum.html
@@ -13,8 +13,8 @@
                 
                   | {{ value.name }} | -                  {% if value.summary %}
- {{ value.summary }}+                  {% if value.content %}
+{{ value.content }}
                   {% endif %} | 
diff --git a/documentation/test_python/content/content.html b/documentation/test_python/content/content.html
index 1f98a3e9..dbbf096a 100644
--- a/documentation/test_python/content/content.html
+++ b/documentation/test_python/content/content.html
@@ -72,7 +72,8 @@ tho.
 add any detailed block.
             
               class EnumWithSummary(enum.Enum): VALUE = 0
-              ANOTHER = 1
+              ANOTHER = 1
+              THIRD = 3
             
             This summary is preserved
           
@@ -151,12 +152,22 @@ add any detailed block.
                 
                   | VALUE | - A value+ Value docs where this is treated as summary.+ And this as detailed docs by the raw docstring parser, but the theme doesn't
+distinguish between them so they get merged together. | 
                 
                   | ANOTHER | + +This value is documented from within the +enum+directive... | 
+                
+                  | THIRD+ | + ... while this comes from the enumvaluedirective. | 
               
diff --git a/documentation/test_python/content/content/__init__.py b/documentation/test_python/content/content/__init__.py
index b95306e2..ace7afb6 100644
--- a/documentation/test_python/content/content/__init__.py
+++ b/documentation/test_python/content/content/__init__.py
@@ -68,8 +68,13 @@ class EnumWithSummary(enum.Enum):
 
     VALUE = 0
     ANOTHER = 1
+    THIRD = 3
 
-EnumWithSummary.VALUE.__doc__ = "A value"
+EnumWithSummary.VALUE.__doc__ = """Value docs where this is treated as summary.
+
+And this as detailed docs by the raw docstring parser, but the theme doesn't
+distinguish between them so they get merged together.
+"""
 
 def foo(a, b):
     """This summary is not shown either"""
diff --git a/documentation/test_python/content/docs.rst b/documentation/test_python/content/docs.rst
index 23696220..7ab0eed2 100644
--- a/documentation/test_python/content/docs.rst
+++ b/documentation/test_python/content/docs.rst
@@ -88,9 +88,15 @@
         add any detailed block.
 
 .. py:enum:: content.EnumWithSummary
+    :value ANOTHER: This value is documented from within the ``enum``
+        directive...
 
     And this is detailed docs added to the docstring summary. :ref:`VALUE`!!
 
+.. py:enumvalue:: content.EnumWithSummary.THIRD
+
+    ... while this comes from the ``enumvalue`` directive.
+
 .. py:function:: content.foo
     :summary: This overwrites the docstring for :ref:`foo()`, but doesn't
         add any detailed block.
diff --git a/documentation/test_python/content_parse_docstrings/content_parse_docstrings.html b/documentation/test_python/content_parse_docstrings/content_parse_docstrings.html
index f37b9eaa..a6e895bf 100644
--- a/documentation/test_python/content_parse_docstrings/content_parse_docstrings.html
+++ b/documentation/test_python/content_parse_docstrings/content_parse_docstrings.html
@@ -50,7 +50,8 @@ even from a summary.
           
           
             - 
-              class Enum(enum.Enum): VALUE = 3
+              class Enum(enum.Enum): VALUE = 3
+              ANOTHER = 4
             
- This enum has a serious docstring. VALUE works from a summary.
@@ -90,12 +91,19 @@ even from a summary.
                   | VALUE | - +Tho enum value docs are unfortunately *not* processed.+ Enum value docstrings are processed as well.+ The ANOTHER value is documented from within the Enum itself.+ | 
+                
+                  | ANOTHER+ | + Values can be documented from a docstring, too. | 
               
             
-And property details as well.
+And enum details as well.
           
         
         
diff --git a/documentation/test_python/content_parse_docstrings/content_parse_docstrings.py b/documentation/test_python/content_parse_docstrings/content_parse_docstrings.py
index 1250a76c..c248a4c2 100644
--- a/documentation/test_python/content_parse_docstrings/content_parse_docstrings.py
+++ b/documentation/test_python/content_parse_docstrings/content_parse_docstrings.py
@@ -20,11 +20,17 @@ class Class:
 class Enum(enum.Enum):
     """This enum has a *serious* docstring. :ref:`VALUE` works from a summary.
 
-    And property **details** as well."""
+    :value ANOTHER: Values can be documented from a docstring, too.
+
+    And enum **details** as well."""
 
     VALUE = 3
+    ANOTHER = 4
+
+Enum.VALUE.__doc__ = """Enum value docstrings are *processed* as well.
 
-Enum.VALUE.__doc__ = "Tho enum value docs are unfortunately *not* processed."
+The :ref:`ANOTHER` value is documented from within the :ref:`Enum` itself.
+"""
 
 def function(a: str, b: int) -> float:
     """This :ref:`function()` has a *serious* docstring.
diff --git a/documentation/test_python/inspect_all_property/inspect_all_property.html b/documentation/test_python/inspect_all_property/inspect_all_property.html
index ae97cb2d..0692c903 100644
--- a/documentation/test_python/inspect_all_property/inspect_all_property.html
+++ b/documentation/test_python/inspect_all_property/inspect_all_property.html
@@ -97,13 +97,13 @@
                 
                   | VALUE | - A value+ A value | 
                 
                   | ANOTHER | - Another value+ Another value | 
                 
diff --git a/documentation/test_python/inspect_string/inspect_string.Foo.html b/documentation/test_python/inspect_string/inspect_string.Foo.html
index 0569310b..d3abbd90 100644
--- a/documentation/test_python/inspect_string/inspect_string.Foo.html
+++ b/documentation/test_python/inspect_string/inspect_string.Foo.html
@@ -145,13 +145,13 @@
                 
                   | VALUE | - A value+ A value | 
                 
                   | ANOTHER | - Another value+ Another value | 
                 
diff --git a/documentation/test_python/inspect_string/inspect_string.html b/documentation/test_python/inspect_string/inspect_string.html
index 3bd5a79f..6eca098a 100644
--- a/documentation/test_python/inspect_string/inspect_string.html
+++ b/documentation/test_python/inspect_string/inspect_string.html
@@ -139,13 +139,13 @@
                 
                   | VALUE | - A value+ A value | 
                 
                   | ANOTHER | - Another value+ Another value | 
                 
diff --git a/documentation/test_python/pybind_enums/docs.rst b/documentation/test_python/pybind_enums/docs.rst
new file mode 100644
index 00000000..13724de6
--- /dev/null
+++ b/documentation/test_python/pybind_enums/docs.rst
@@ -0,0 +1,5 @@
+.. py:enum:: pybind_enums.MyEnum
+    :value First: First value external documentation
+    :value Second: Second value external documentation
+
+    External details.
diff --git a/documentation/test_python/pybind_enums/pybind_enums.cpp b/documentation/test_python/pybind_enums/pybind_enums.cpp
index 1befd5cd..58532001 100644
--- a/documentation/test_python/pybind_enums/pybind_enums.cpp
+++ b/documentation/test_python/pybind_enums/pybind_enums.cpp
@@ -17,7 +17,7 @@ enum SixtyfourBitFlag: std::uint64_t {
 PYBIND11_MODULE(pybind_enums, m) {
     m.doc() = "pybind11 enum parsing";
 
-    py::enum_(m, "MyEnum", "An enum without value docs :(")
+    py::enum_(m, "MyEnum", "An enum with external value docs, at least")
         .value("First", MyEnum::First)
         .value("Second", MyEnum::Second)
         .value("Third", MyEnum::Third)
diff --git a/documentation/test_python/pybind_enums/pybind_enums.html b/documentation/test_python/pybind_enums/pybind_enums.html
index e776b896..a9e25d98 100644
--- a/documentation/test_python/pybind_enums/pybind_enums.html
+++ b/documentation/test_python/pybind_enums/pybind_enums.html
@@ -26,13 +26,13 @@
         
+        
+          Enum documentation
+          
+            
+              class pybind_enums.MyEnum()
+            
+            
An enum with external value docs, at least
+            
+              | Enumerators |  | 
|---|
+              
+                
+                  | First+ | + +First value external documentation+ | 
+                
+                  | Second+ | + +Second value external documentation+ | 
+                
+                  | Third+ | ++ | 
+                
+                  | CONSISTANTE+ | ++ | 
+              
+            
+
External details.
+