m.code: rudimentary support for 24bit console colors.

MVP I need for an upcoming article (and docs).

diff --git a/doc/css/components.rst b/doc/css/components.rst
index f099aeb5aa913a730f82bc368fd4a88a668c7af8..968fd0f6b4fd23279fb15010a1d23d2728b91e64 100644 (file)
--- a/doc/css/components.rst
+++ b/doc/css/components.rst
@@ -898,8 +898,9 @@ Besides code, it's also possible to "highlight" ANSI-colored terminal output.
 For that, m.css provides a custom Pygments lexer that's together with
 `pygments-console.css <{filename}/css.rst>`_ able to detect and highlight the
 basic 4-bit color codes (8 foreground colors in either normal or bright
-version). Download the :gh:`ansilexer.py <mosra/m.css$master/plugins/m/ansilexer.py>`
-file or use it directly from your Git clone of m.css. Example usage:
+version) and a tiny subset of the 24-bit color scheme as well. Download the
+:gh:`ansilexer.py <mosra/m.css$master/plugins/m/ansilexer.py>` file or use it
+directly from your Git clone of m.css. Example usage:
 
 .. code:: sh
 

diff --git a/doc/plugins/math-and-code-console-colors.ansi b/doc/plugins/math-and-code-console-colors.ansi
new file mode 100644 (file)
index 0000000..a3fa91a
--- /dev/null
+++ b/doc/plugins/math-and-code-console-colors.ansi
@@ -0,0 +1,3 @@
+\e[38;2;220;220;220m\e[48;2;220;220;220m██\e[0m \e[38;2;165;201;234m\e[48;2;165;201;234m██\e[0m \e[38;2;59;210;103m\e[48;2;59;210;103m██\e[0m \e[38;2;199;207;47m\e[48;2;199;207;47m██\e[0m \e[38;2;205;52;49m\e[48;2;205;52;49m██\e[0m \e[38;2;47;131;204m\e[48;2;47;131;204m▓▓\e[0m \e[38;2;116;116;116m\e[48;2;116;116;116m▒▒\e[0m
+
+\e[38;2;59;210;103m  \e[0m \e[38;2;59;210;103m  \e[0m \e[38;2;59;210;103m░░\e[0m \e[38;2;59;210;103m▒▒\e[0m \e[38;2;59;210;103m▓▓\e[0m \e[38;2;59;210;103m\e[48;2;59;210;103m██\e[0m

diff --git a/doc/plugins/math-and-code.rst b/doc/plugins/math-and-code.rst
index 2287aa2b008590e05a9d061bf0cbd918896ea323..01497aacacdfc4af2203e784ab694c37b68c726c 100644 (file)
--- a/doc/plugins/math-and-code.rst
+++ b/doc/plugins/math-and-code.rst
@@ -419,6 +419,16 @@ terminal, it's best to have the listings in external files and use
 
     .. include:: math-and-code-console.ansi
         :code: ansi
+        :class: m-nopad
+
+Apart from the basic color set there's also a very rudimentary support for
+24bit colors using the ``\033[{?};2;{r};{g};{b}m`` color sequence --- currently
+either just the foreground (the ``\033[38;2;`` prefix) or foreground
+immediately followed by background color specification (the
+``\033[48;2;`` prefix):
+
+.. include:: math-and-code-console-colors.ansi
+    :code: ansi
 
 `Code figure`_
 --------------

diff --git a/documentation/doxygen.py b/documentation/doxygen.py
index e8d98f82ee807ff873e0769d393f857463a2d361..f610d196db6693722c0a4220b82c4f3021f45e9c 100755 (executable)
--- a/documentation/doxygen.py
+++ b/documentation/doxygen.py
@@ -1428,7 +1428,10 @@ def parse_desc_internal(state: State, element: ET.Element, immediate_parent: ET.
             else:
                 class_ = 'm-code'
 
-            formatter = HtmlFormatter(nowrap=True)
+            if isinstance(lexer, ansilexer.AnsiLexer):
+                formatter = ansilexer.HtmlAnsiFormatter()
+            else:
+                formatter = HtmlFormatter(nowrap=True)
             highlighted = highlight(code, lexer, formatter)
             # Strip whitespace around if inline code, strip only trailing
             # whitespace if a block

diff --git a/documentation/test_doxygen/contents_code_language/ansi.html b/documentation/test_doxygen/contents_code_language/ansi.html
index afc5348b58f4736237e46cf45dd028130e0a9676..4457075d3f6abf38f2633927ebab899cc194cb14 100644 (file)
--- a/documentation/test_doxygen/contents_code_language/ansi.html
+++ b/documentation/test_doxygen/contents_code_language/ansi.html
@@ -22,7 +22,9 @@
         <h1>
           ANSI color escape code highlighting
         </h1>
-<pre class="m-console">!<span class="g g-AnsiBlue">[</span><span class="g g-AnsiBrightWhite">mosra@don-perverzo </span><span class="g g-AnsiWhite">m.css</span><span class="g g-AnsiBlue">]</span><span class="g g-AnsiBrightCyan">$ </span></pre>
+<pre class="m-console">!<span class="g g-AnsiBlue">[</span><span class="g g-AnsiBrightWhite">mosra@don-perverzo </span><span class="g g-AnsiWhite">m.css</span><span class="g g-AnsiBlue">]</span><span class="g g-AnsiBrightCyan">$ </span></pre><pre class="m-console"><span style="color: #dcdcdc; background-color: #dcdcdc">██</span> <span style="color: #a5c9ea; background-color: #a5c9ea">██</span> <span style="color: #3bd267; background-color: #3bd267">██</span> <span style="color: #c7cf2f; background-color: #c7cf2f">██</span> <span style="color: #cd3431; background-color: #cd3431">██</span> <span style="color: #2f83cc; background-color: #2f83cc">▓▓</span> <span style="color: #747474; background-color: #747474">▒▒</span>
+
+<span style="color: #3bd267">  </span> <span style="color: #3bd267">  </span> <span style="color: #3bd267">░░</span> <span style="color: #3bd267">▒▒</span> <span style="color: #3bd267">▓▓</span> <span style="color: #3bd267; background-color: #3bd267">██</span></pre>
       </div>
     </div>
   </div>

diff --git a/documentation/test_doxygen/contents_code_language/console-colors.ansi b/documentation/test_doxygen/contents_code_language/console-colors.ansi
new file mode 120000 (symlink)
index 0000000..f10d6d6
--- /dev/null
+++ b/documentation/test_doxygen/contents_code_language/console-colors.ansi
@@ -0,0 +1 @@
+../../../plugins/m/test/code/console-colors.ansi
\ No newline at end of file

diff --git a/documentation/test_doxygen/contents_code_language/input.dox b/documentation/test_doxygen/contents_code_language/input.dox
index c8550a9743a9d7ef38c47ea8500adcef51b1f195..6a2033fadeeab61788f19935a00dbfd5c85ea1bb 100644 (file)
--- a/documentation/test_doxygen/contents_code_language/input.dox
+++ b/documentation/test_doxygen/contents_code_language/input.dox
@@ -36,6 +36,8 @@ Another paragraph.
 !\e[0;34m[\e[1;37mmosra@don-perverzo \e[0;37mm.css\e[0;34m]\e[1;36m$ \e[0m
 @endcode
 
+@include console-colors.ansi
+
 */
 
 /** @page warnings Code that produces warnings

diff --git a/plugins/ansilexer.py b/plugins/ansilexer.py
index dd11d5ca0972143596f5d89f443bd9d5a0ee646f..d9d12e393644d3c89291e53685c81d1351e3075a 100644 (file)
--- a/plugins/ansilexer.py
+++ b/plugins/ansilexer.py
@@ -22,7 +22,10 @@
 #   DEALINGS IN THE SOFTWARE.
 #
 
+import re
+
 from pygments.lexer import RegexLexer
+from pygments.formatters import HtmlFormatter
 from pygments.token import *
 
 class AnsiLexer(RegexLexer):
@@ -65,8 +68,35 @@ class AnsiLexer(RegexLexer):
 
         yield (match.start(), string_to_tokentype(token), text)
 
+    def callback_fg_color(lexer, match):
+        token = 'Generic.AnsiForegroundColor{:02x}{:02x}{:02x}'.format(
+            int(match.group(1)), int(match.group(2)), int(match.group(3)))
+        yield (match.start, string_to_tokentype(token), match.group(4))
+
+    def callback_fg_bg_color(lexer, match):
+        token = 'Generic.AnsiForegroundBackgroundColor{:02x}{:02x}{:02x}'.format(
+            int(match.group(1)), int(match.group(2)), int(match.group(3)))
+        yield (match.start, string_to_tokentype(token), match.group(4))
+
     tokens = {
         'root': [
             ('[^\x1b]+', Text),
+            ('\x1b\\[38;2;(\\d+);(\\d+);(\\d+)m\x1b\\[48;2;\\d+;\\d+;\\d+m([^\x1b]+)\x1b\\[0m', callback_fg_bg_color),
+            ('\x1b\\[38;2;(\\d+);(\\d+);(\\d+)m([^\x1b]+)\x1b\\[0m', callback_fg_color),
             ('\x1b\\[(\\d+)(;\\d+)?m([^\x1b]*)', callback)]
     }
+
+_ansi_fg_color_re = re.compile('class="g g-AnsiForegroundColor([0-9a-f]{6})"')
+_ansi_fg_bg_color_re = re.compile('class="g g-AnsiForegroundBackgroundColor([0-9a-f]{6})"')
+
+class HtmlAnsiFormatter(HtmlFormatter):
+    def wrap(self, source, outfile):
+        return self._wrap_code(source)
+
+    def _wrap_code(self, source):
+        for i, t in source:
+            if i == 1: # it's a line of formatted code
+                t = _ansi_fg_bg_color_re.sub('style="color: #\\1; background-color: #\\1"', t)
+                t = _ansi_fg_color_re.sub('style="color: #\\1"', t)
+                #t += 'H'
+            yield i, t

diff --git a/plugins/m/code.py b/plugins/m/code.py
index 4290e01b138b8ae9f8e08aa645b4fccb8812289a..4bb427400d2888d8b5a5c80105457e2443862bdd 100644 (file)
--- a/plugins/m/code.py
+++ b/plugins/m/code.py
@@ -59,7 +59,10 @@ def _highlight(code, language, options, is_block):
     else:
         class_ = 'm-code'
 
-    formatter = HtmlFormatter(nowrap=True, **options)
+    if isinstance(lexer, ansilexer.AnsiLexer):
+        formatter = ansilexer.HtmlAnsiFormatter(**options)
+    else:
+        formatter = HtmlFormatter(nowrap=True, **options)
     parsed = highlight(code, lexer, formatter).rstrip()
     if not is_block: parsed.lstrip()
 

diff --git a/plugins/m/test/code/console-colors.ansi b/plugins/m/test/code/console-colors.ansi
new file mode 100644 (file)
index 0000000..ccd327f
--- /dev/null
+++ b/plugins/m/test/code/console-colors.ansi
@@ -0,0 +1,5 @@
+\e[38;2;220;220;220m\e[48;2;220;220;220m██\e[0m \e[38;2;165;201;234m\e[48;2;165;201;234m██\e[0m \e[38;2;59;210;103m\e[48;2;59;210;103m██\e[0m \e[38;2;199;207;47m\e[48;2;199;207;47m██\e[0m \e[38;2;205;52;49m\e[48;2;205;52;49m██\e[0m \e[38;2;47;131;204m\e[48;2;47;131;204m▓▓\e[0m \e[38;2;116;116;116m\e[48;2;116;116;116m▒▒\e[0m
+
+Leading zeros: \e[38;2;15;116;3m\e[48;2;15;116;3m▒▒\e[0m \e[38;2;15;116;3m▒▒\e[0m
+
+\e[38;2;59;210;103m  \e[0m \e[38;2;59;210;103m  \e[0m \e[38;2;59;210;103m░░\e[0m \e[38;2;59;210;103m▒▒\e[0m \e[38;2;59;210;103m▓▓\e[0m \e[38;2;59;210;103m\e[48;2;59;210;103m██\e[0m

diff --git a/plugins/m/test/code/page.html b/plugins/m/test/code/page.html
index 273749678a3c7414c1f9b6a964c9b2637ef9b7b4..7d50e3862c5c45c06d05afcb92cd6257e98810a4 100644 (file)
--- a/plugins/m/test/code/page.html
+++ b/plugins/m/test/code/page.html
@@ -34,6 +34,12 @@ rendered as plain monospace text: <code>code</code>.</p>
 <pre class="m-console">!<span class="g g-AnsiBlue">[</span><span class="g g-AnsiBrightWhite">mosra@don-perverzo </span><span class="g g-AnsiWhite">m.css</span><span class="g g-AnsiBlue">]</span><span class="g g-AnsiBrightCyan">$ </span>ls
 CONTRIBUTING.rst  CREDITS.rst  <span class="g g-AnsiBrightBlue">doc</span>            <span class="g g-AnsiBrightBlue">plugins</span>        README.rst
 COPYING           <span class="g g-AnsiBrightBlue">css</span>          <span class="g g-AnsiBrightBlue">documentation</span>  <span class="g g-AnsiBrightBlue">pelican-theme</span>  <span class="g g-AnsiBrightBlue">site</span></pre>
+<p>Console colors:</p>
+<pre class="m-console"><span style="color: #dcdcdc; background-color: #dcdcdc">██</span> <span style="color: #a5c9ea; background-color: #a5c9ea">██</span> <span style="color: #3bd267; background-color: #3bd267">██</span> <span style="color: #c7cf2f; background-color: #c7cf2f">██</span> <span style="color: #cd3431; background-color: #cd3431">██</span> <span style="color: #2f83cc; background-color: #2f83cc">▓▓</span> <span style="color: #747474; background-color: #747474">▒▒</span>
+
+Leading zeros: <span style="color: #0f7403; background-color: #0f7403">▒▒</span> <span style="color: #0f7403">▒▒</span>
+
+<span style="color: #3bd267">  </span> <span style="color: #3bd267">  </span> <span style="color: #3bd267">░░</span> <span style="color: #3bd267">▒▒</span> <span style="color: #3bd267">▓▓</span> <span style="color: #3bd267; background-color: #3bd267">██</span></pre>
 <pre class="m-code">// this language is not highlighted</pre>
 <p>Properly preserve backslashes: <code class="m-code"><span class="k">\frac</span><span class="nb">{</span>a<span class="nb">}{</span>b<span class="nb">}</span></code></p>
 <p>Don't trim leading spaces in blocks:</p>

diff --git a/plugins/m/test/code/page.rst b/plugins/m/test/code/page.rst
index 0e92fae8bab1759ef2515338e84770d735fe4c0c..e3d92548eaaf9cb79f2a0cf7e2e15bca584fbc88 100644 (file)
--- a/plugins/m/test/code/page.rst
+++ b/plugins/m/test/code/page.rst
@@ -26,6 +26,11 @@ rendered as plain monospace text: :code:`code`.
 .. include:: console.ansi
     :code: ansi
 
+Console colors:
+
+.. include:: console-colors.ansi
+    :code: ansi
+
 .. code:: whatthefuck
 
     // this language is not highlighted