chiark / gitweb /
m.code: add a bunch of advanced .. include:: features.
authorVladimír Vondruš <mosra@centrum.cz>
Thu, 14 May 2020 10:50:44 +0000 (12:50 +0200)
committerVladimír Vondruš <mosra@centrum.cz>
Thu, 14 May 2020 12:14:44 +0000 (14:14 +0200)
It was already useful, but this makes it even more. What's done:

-  :start-on: so it's possible to pin on something in the code without
   having to add a comment marker everywhere
-  :end-before: that can be empty, and thus simply stop at the end of
   the block
-  :strip-prefix: for removing excessive indentation as well as adding
   an ability to interpret code comments as reST.

All combined together, one can now embed Sphinx Gallery files (which can
be converted to ipynb files too) *and* still have a total flexibility
over what's done on the content formatting and layouting side, ignoring
boring portions of the source file etc etc.

doc/plugins/math-and-code-selfcontained.py [new file with mode: 0644]
doc/plugins/math-and-code.rst
plugins/m/code.py
plugins/m/test/code/file.py [new file with mode: 0644]
plugins/m/test/code/page.html
plugins/m/test/code/page.rst

diff --git a/doc/plugins/math-and-code-selfcontained.py b/doc/plugins/math-and-code-selfcontained.py
new file mode 100644 (file)
index 0000000..a917978
--- /dev/null
@@ -0,0 +1,24 @@
+"""
+This is a reST markup explaining the following code, compatible with
+`Sphinx Gallery <https://sphinx-gallery.github.io/>`_.
+"""
+
+# You can convert the file to a Jupyter notebook using the
+# sphx_glr_python_to_jupyter.py utility from Sphinx Gallery.
+
+import math
+
+sin = math.sin(0.13587)
+print(sin)
+
+#%%
+# And a sum with itself turns it into two sins, because the following holds:
+#
+# .. math::
+#
+#   2 a = a + a
+#
+
+two_sins = sin + sin
+if two_sins != 2*sin:
+    print("Assumptions broken. Restart the universe.")
index 904c893611b248c5cfd082d2772bbb4d6f2afcb3..6ef1ae45c8b12f668a3e53822e7c2064df3f2a36 100644 (file)
@@ -360,11 +360,13 @@ option to highlight lines; if you want to add additional CSS classes, use the
             return 0;
         }
 
-The builtin `include directive <http://docutils.sourceforge.net/docs/ref/rst/directives.html#include>`_
+The `builtin include directive <http://docutils.sourceforge.net/docs/ref/rst/directives.html#include>`_
 is also patched to use the improved code directive, and:
 
 -   Drops the rarely useful :rst:`:encoding:`, :rst:`:literal:` and
     :rst:`:name:` options
+-   Adds a :rst:`:start-on:` and :rst:`:strip-prefix:` options, and improves
+    :rst:`:end-before:`. See `Advanced file inclusion`_ below.
 
 Simply specify external code snippets filename and set the language using the
 :rst:`:code:` option. All options of the :rst:`.. code::` directive are
@@ -454,6 +456,95 @@ See the `m.components <{filename}/plugins/components.rst#code-math-and-graph-fig
 plugin for details about code figures using the :rst:`.. code-figure::`
 directive.
 
+`Advanced file inclusion`_
+--------------------------
+
+Compared to the `builtin include directive`_, the m.css-patched variant
+additionally provides a :rst:`:strip-prefix:` option that strips a prefix from
+each included line. This can be used for example to remove excessive
+indentation from code blocks. To avoid trailing whitespace, you can wrap the
+value in quotes. Reusing the snippet from above, showing only the code inside
+:cpp:`main()`:
+
+.. code-figure::
+
+    .. code:: rst
+
+        .. include:: snippet.cpp
+            :code: c++
+            :start-line: 3
+            :end-line: 5
+            :strip-prefix: '    '
+
+    .. include:: math-and-code-snippet.cpp
+        :code: c++
+        :start-line: 3
+        :end-line: 5
+        :strip-prefix: '    '
+
+This isn't limited to just whitespace though --- since the :rst:`.. include::`
+directive works for including reStructuredText as well, it can be used to embed
+parts of self-contained python scripts on the page. Consider this file,
+``two-sins.py``:
+
+.. include:: math-and-code-selfcontained.py
+    :code: py
+
+Embedding it on a page, mixed together with other content (and unimportant
+parts omitted), can look like below. The :rst:`:start-on:` option can be used
+to pin to a particular line (instead of skipping it like :rst:`:start-after:`
+does) and an empty :rst:`:end-before:` will include everything until the next
+blank line. Finally, :rst:`:strip-prefix:` strips the leading :py:`#` from the
+comments embedded in Python code:
+
+.. code-figure::
+
+    .. code:: rst
+
+        .. include:: two-sins.py
+            :start-after: """
+            :end-before: """
+
+        .. code-figure::
+
+            .. include:: two-sins.py
+                :start-on: sin =
+                :end-before:
+                :code: py
+
+            0.13545234412104434
+
+        .. include:: two-sins.py
+            :start-on: # And a sum with itself
+            :strip-prefix: '# '
+            :end-before:
+
+        .. include:: two-sins.py
+            :start-on: two_sins
+            :code: py
+
+    .. include:: math-and-code-selfcontained.py
+        :start-after: """
+        :end-before: """
+
+    .. code-figure::
+
+        .. include:: math-and-code-selfcontained.py
+            :start-on: sin =
+            :end-before:
+            :code: py
+
+        0.13545234412104434
+
+    .. include:: math-and-code-selfcontained.py
+        :start-on: # And a sum with itself
+        :strip-prefix: '# '
+        :end-before:
+
+    .. include:: math-and-code-selfcontained.py
+        :start-on: two_sins
+        :code: py
+
 `Filters`_
 ----------
 
index ccc09cd8f0dc38399b759c8b36aa2a9a6747d98c..1f52c1c06730041f3b447957391c38bfe1ba033e 100644 (file)
@@ -127,7 +127,9 @@ class Include(docutils.parsers.rst.directives.misc.Include):
         'start-line': int,
         'end-line': int,
         'start-after': directives.unchanged_required,
-        'end-before': directives.unchanged_required,
+        'start-on': directives.unchanged_required,
+        'end-before': directives.unchanged,
+        'strip-prefix': directives.unchanged,
         'class': directives.class_option,
         'filters': directives.unchanged
     }
@@ -135,8 +137,9 @@ class Include(docutils.parsers.rst.directives.misc.Include):
     def run(self):
         """
         Verbatim copy of docutils.parsers.rst.directives.misc.Include.run()
-        that just calls to our Code instead of builtin CodeBlock, and is
-        without the rarely useful :encoding:, :literal: and :name: options
+        that just calls to our Code instead of builtin CodeBlock, is without
+        the rarely useful :encoding:, :literal: and :name: options and adds
+        support for :start-on:, empty :end-before: and :strip-prefix:.
         """
         source = self.state_machine.input_lines.source(
             self.lineno - self.state_machine.input_offset - 1)
@@ -183,10 +186,24 @@ class Include(docutils.parsers.rst.directives.misc.Include):
                 raise self.severe('Problem with "start-after" option of "%s" '
                                   'directive:\nText not found.' % self.name)
             rawtext = rawtext[after_index + len(after_text):]
+        # Compared to start-after, this includes the matched line
+        on_text = self.options.get('start-on', None)
+        if on_text:
+            on_index = rawtext.find('\n' + on_text)
+            if on_index < 0:
+                raise self.severe('Problem with "start-on" option of "%s" '
+                                  'directive:\nText not found.' % self.name)
+            rawtext = rawtext[on_index:]
+        # Compared to builtin include directive, the end-before can be empty,
+        # in which case it simply matches the first empty line (which is
+        # usually end of the code block)
         before_text = self.options.get('end-before', None)
-        if before_text:
+        if before_text is not None:
             # skip content in rawtext after *and incl.* a matching text
-            before_index = rawtext.find(before_text)
+            if before_text == '':
+                before_index = rawtext.find('\n\n')
+            else:
+                before_index = rawtext.find(before_text)
             if before_index < 0:
                 raise self.severe('Problem with "end-before" option of "%s" '
                                   'directive:\nText not found.' % self.name)
@@ -194,6 +211,21 @@ class Include(docutils.parsers.rst.directives.misc.Include):
 
         include_lines = statemachine.string2lines(rawtext, tab_width,
                                                   convert_whitespace=True)
+
+        # Strip a common prefix from all lines. Useful for example when
+        # including a reST snippet that's embedded in a comment, or cutting
+        # away excessive indentation. Can be wrapped in quotes in order to
+        # avoid trailing whitespace in reST markup.
+        if 'strip-prefix' in self.options and self.options['strip-prefix']:
+            prefix = self.options['strip-prefix']
+            if prefix[0] == prefix[-1] and prefix[0] in ['\'', '"']:
+                prefix = prefix[1:-1]
+            for i, line in enumerate(include_lines):
+                if line.startswith(prefix): include_lines[i] = line[len(prefix):]
+                # Strip the prefix also if the line is just the prefix alone,
+                # with trailing whitespace removed
+                elif line.rstrip() == prefix.rstrip(): include_lines[i] = ''
+
         if 'code' in self.options:
             self.options['source'] = path
             # Don't convert tabs to spaces, if `tab_width` is negative:
diff --git a/plugins/m/test/code/file.py b/plugins/m/test/code/file.py
new file mode 100644 (file)
index 0000000..de621b6
--- /dev/null
@@ -0,0 +1,25 @@
+"""
+`"Sphinx Gallery"-alike self-contained code file`_
+--------------------------------------------------
+"""
+
+# Running sphx_glr_python_to_jupyter.py on this file will produce a ipynb
+# version of the same (modulo bugs, which there's several of).
+
+import math
+
+#%%
+# This is a reST markup explaining the following code, with the initial
+# ``#<space>`` stripped, and on blank lines only the ``#`` stripped:
+#
+## However in this case both leading ``##`` will be kept.
+#
+# The ``math.sin()`` calculates a sin, *of course*, and the initial indentation
+# of it is stripped also:
+#
+
+def foo():
+# [yay-code]
+    sin = math.sin(0.13587)
+    two_sins = sin + sin
+# [/yay-code]
index 54c84c42049987a9552b6ddb5b6e4c303f081357..0af11332b1230211a2ad60f09adb31a08986a080 100644 (file)
@@ -51,6 +51,22 @@ Leading zeros: <span style="color: #0f7403; background-color: #0f7403">&zwnj;▒
 <pre class="m-code">        <span class="n">nope</span><span class="p">();</span>
     <span class="k">return</span> <span class="nb">false</span><span class="p">;</span>
 <span class="p">}</span></pre>
+<section id="advanced-file-inclusion">
+<h2><a href="#advanced-file-inclusion">Advanced file inclusion</a></h2>
+<section id="sphinx-gallery-alike-self-contained-code-file">
+<h3><a href="#sphinx-gallery-alike-self-contained-code-file">&quot;Sphinx Gallery&quot;-alike self-contained code file</a></h3>
+<p>This is a reST markup explaining the following code, with the initial
+<code>#&lt;space&gt;</code> stripped, and on blank lines only the <code>#</code> stripped:</p>
+<p>## However in this case both leading <code>##</code> will be kept.</p>
+<p>The <code>math.sin()</code> calculates a sin, <em>of course</em>, and the initial indentation
+of it is stripped also:</p>
+<pre class="m-code"><span class="n">sin</span> <span class="o">=</span> <span class="n">math</span><span class="o">.</span><span class="n">sin</span><span class="p">(</span><span class="mf">0.13587</span><span class="p">)</span>
+<span class="n">two_sins</span> <span class="o">=</span> <span class="n">sin</span> <span class="o">+</span> <span class="n">sin</span></pre>
+<p>In comparison, here's the default output without <code class="m-code"><span class="nc">:strip-prefix:</span></code>:</p>
+<pre class="m-code">    <span class="n">sin</span> <span class="o">=</span> <span class="n">math</span><span class="o">.</span><span class="n">sin</span><span class="p">(</span><span class="mf">0.13587</span><span class="p">)</span>
+    <span class="n">two_sins</span> <span class="o">=</span> <span class="n">sin</span> <span class="o">+</span> <span class="n">sin</span></pre>
+</section>
+</section>
 <section id="filters">
 <h2><a href="#filters">Filters</a></h2>
 <p>Applied by default, adding typographically correct spaces before and a color
index b548348f8379bf191255d85f6f339710e69b173f..24e2f7d19a7f54640639f9d1483d5b2762cbb4b9 100644 (file)
@@ -7,6 +7,8 @@ m.code
     :language: tex
 .. role:: rst(code)
     :language: rst
+.. role:: py(code)
+    :language: py
 
 .. code:: c++
 
@@ -48,6 +50,33 @@ Don't trim leading spaces in blocks:
         return false;
     }
 
+`Advanced file inclusion`_
+==========================
+
+.. include:: file.py
+    :start-after: """
+    :end-before: """
+
+.. the following tests :start-on:, empty :end-before: and :strip-prefix: also:
+
+.. include:: file.py
+    :start-on: # This is a reST
+    :end-before:
+    :strip-prefix: '# '
+
+.. include:: file.py
+    :start-after: # [yay-code]
+    :end-before: # [/yay-code]
+    :strip-prefix: '    '
+    :code: py
+
+In comparison, here's the default output without :rst:`:strip-prefix:`:
+
+.. include:: file.py
+    :start-after: # [yay-code]
+    :end-before: # [/yay-code]
+    :code: py
+
 `Filters`_
 ==========