chiark / gitweb /
doxygen: make it possible to search directly from browser URL bar.
authorVladimír Vondruš <mosra@centrum.cz>
Wed, 2 Jan 2019 13:11:30 +0000 (14:11 +0100)
committerVladimír Vondruš <mosra@centrum.cz>
Wed, 2 Jan 2019 21:07:36 +0000 (22:07 +0100)
Adds an ability to specify search query by adding ?q={query}#search
to URL, which can be then bookmarked in the browsers to provide search
directly from the URL bar (or search input field).

On browsers supporting OpenSearch discovery (Firefox, Chrome, probably
Edge too), it also provides metadata that allow "single-click" addition
of the search engine to browser's search engine list. This option is
unfortunately well-hidden in the current versions (Firefox has it in the
three-dots menu, in Chrome you need to go to Settings while being on
this page and then it shows a suggestion, wtf). The OpenSearch metadata
are provided through a new opensearch.xml file and referenced from page
<head>. OpenSearch also supports autocompletion and search directly from
browser address bar, but for that to work I would need to implement a
server-side search functionality. Not yet, since I don't have any
immediate plans to turn my cloud file-serving Apache installation into a
smart and vulnerable attack target.

This also wraps the input in a <form>, so browsers not supporting
OpenSearch discovery (Vivaldi) can still add the search engine by
right-clicking on the input field. This works in Firefox as well.
Also, because of the <form> added, Vivaldi started autocompleting crap
suggestions for me, so I had to explicitly disable that.

15 files changed:
doc/doxygen.rst
doxygen/dox2html5.py
doxygen/search.js
doxygen/templates/base.html
doxygen/templates/opensearch.xml [new file with mode: 0644]
doxygen/test/layout/pages.html
doxygen/test/layout_generated_doxyfile/index.html
doxygen/test/layout_minimal/index.html
doxygen/test/layout_search_binary/index.html
doxygen/test/layout_search_opensearch/Doxyfile [new file with mode: 0644]
doxygen/test/layout_search_opensearch/index.html [new file with mode: 0644]
doxygen/test/layout_search_opensearch/indexpage.xml [new file with mode: 0644]
doxygen/test/layout_search_opensearch/opensearch.xml.html [new file with mode: 0644]
doxygen/test/test_doxyfile.py
doxygen/test/test_layout.py

index b04faee7b1e3496bc70c78093a0890c0ba7b8e6c..78e950cb9830a88b56c3a8657f792833eb67704a 100644 (file)
@@ -378,17 +378,22 @@ Variable                            Description
                                     ``NO`` is used.
 :ini:`M_SEARCH_DOWNLOAD_BINARY`     Download search data as a binary to save
                                     bandwidth and initial processing time. If
-                                    not set, ``NO`` is used. See `Search`_ for
-                                    more information.
+                                    not set, ``NO`` is used. See
+                                    `Search options`_ for more information.
 :ini:`M_SEARCH_HELP`                HTML code to display as help text on empty
                                     search popup. If not set, a default message
                                     is used. Has effect only if
                                     :ini:`M_SEARCH_DISABLED` is not ``YES``.
+:ini:`M_SEARCH_BASE_URL`            Base URL for OpenSearch-based search engine
+                                    suggestions for web browsers. See
+                                    `Search options`_ for more information. Has
+                                    effect only if :ini:`M_SEARCH_DISABLED` is
+                                    not ``YES``.
 :ini:`M_SEARCH_EXTERNAL_URL`        URL for external search. The ``{query}``
                                     placeholder is replaced with urlencoded
                                     search string. If not set, no external
-                                    search is offered. See `Search`_ for more
-                                    information. Has effect only if
+                                    search is offered. See `Search options`_
+                                    for more information. Has effect only if
                                     :ini:`M_SEARCH_DISABLED` is not ``YES``.
 =================================== ===========================================
 
@@ -531,6 +536,25 @@ being 25% larger, but since this is for serving from a local filesystem, it's
 not considered a problem. If your docs are accessed through a server (or you
 don't need Chrome support), enable the :ini:`M_SEARCH_DOWNLOAD_BINARY` option.
 
+The site can provide search engine metadata using the `OpenSearch <http://www.opensearch.org/>`_
+specification. On supported browsers this means you can add the search field to
+search engines and search directly from the address bar. To enable search
+engine metadata, point :ini:`M_SEARCH_BASE_URL` to base URL of your
+documentation, for example:
+
+.. code:: ini
+
+    M_SEARCH_BASE_URL = "https://doc.magnum.graphics/magnum/"
+
+In general, even without the above setting, appending ``?q={query}#search`` to
+the URL will directly open the search popup with results for ``{query}``.
+
+.. note-info::
+
+    OpenSearch also makes it possible to have autocompletion and search results
+    directly in the browser address bar. However that requires a server-side
+    search implementation and is not supported at the moment.
+
 If :ini:`M_SEARCH_EXTERNAL_URL` is specified, full-text search using an
 external search engine is offered if nothing is found for given string or if
 the user has JavaScript disabled. It's recommended to restrict the search to
index 83c43867661aef237e988c2317e9f48d47df6380..6fab057f7c928b59c053a38da7a0f8f302e9930e 100755 (executable)
@@ -42,6 +42,7 @@ import logging
 from enum import Flag
 from types import SimpleNamespace as Empty
 from typing import Tuple, Dict, Any, List
+from urllib.parse import urljoin
 
 from jinja2 import Environment, FileSystemLoader
 
@@ -3229,6 +3230,7 @@ suffix lists all members of given symbol or directory. Navigate through the
 list using <span class="m-label m-dim">&darr;</span> and
 <span class="m-label m-dim">&uarr;</span>, press
 <span class="m-label m-dim">Enter</span> to go."""],
+        'M_SEARCH_BASE_URL': [''],
         'M_SEARCH_EXTERNAL_URL': ['']
     }
 
@@ -3329,7 +3331,8 @@ list using <span class="m-label m-dim">&darr;</span> and
               'M_FAVICON',
               'M_MATH_CACHE_FILE',
               'M_SEARCH_HELP',
-              'M_SEARCH_EXTERNAL_URL']:
+              'M_SEARCH_EXTERNAL_URL',
+              'M_SEARCH_BASE_URL']:
         if i in config: state.doxyfile[i] = '\n'.join(config[i])
 
     # Int values that we want
@@ -3401,6 +3404,7 @@ def run(doxyfile, templates=default_templates, wildcard=default_wildcard, index_
         if urllib.parse.urlparse(path).netloc: return path
         return os.path.basename(path)
     env.filters['basename_or_url'] = basename_or_url
+    env.filters['urljoin'] = urljoin
 
     # Do a pre-pass and gather:
     # - brief descriptions of all classes, namespaces, dirs and files because
@@ -3492,6 +3496,21 @@ def run(doxyfile, templates=default_templates, wildcard=default_wildcard, index_
             with open(os.path.join(html_output, "searchdata.js"), 'wb') as f:
                 f.write(base85encode_search_data(data))
 
+        # OpenSearch metadata, in case we have the base URL
+        if state.doxyfile['M_SEARCH_BASE_URL']:
+            logging.debug("writing OpenSearch metadata file")
+
+            template = env.get_template('opensearch.xml')
+            rendered = template.render(**state.doxyfile)
+            output = os.path.join(html_output, 'opensearch.xml')
+            with open(output, 'wb') as f:
+                f.write(rendered.encode('utf-8'))
+                # Add back a trailing newline so we don't need to bother with
+                # patching test files to include a trailing newline to make Git
+                # happy
+                # TODO could keep_trailing_newline fix this better?
+                f.write(b'\n')
+
     # Copy all referenced files
     for i in state.images + state.doxyfile['HTML_EXTRA_STYLESHEET'] + state.doxyfile['HTML_EXTRA_FILES'] + ([state.doxyfile['M_FAVICON'][0]] if state.doxyfile['M_FAVICON'] else []) + ([] if state.doxyfile['M_SEARCH_DISABLED'] else ['search.js']):
         # Skip absolute URLs
index 0480e185f326d733485d6cd0955a54d17842e655..a796b53097f818368f3ac66ae78406b04267f874 100644 (file)
@@ -609,6 +609,7 @@ if(typeof document !== 'undefined') {
                    and prevent page layout jumps */
                 document.body.style.overflow = 'auto';
                 document.body.style.paddingRight = '0';
+                return false; /* so the form doesn't get sent */
             }
 
         /* Search hidden */
index 7f4a571edd163a0d9042f0abca90571af03e3516..b4ec324b680e9c353e8481648095773649339ef9 100644 (file)
@@ -9,6 +9,9 @@
   {% if M_FAVICON %}
   <link rel="icon" href="{{ M_FAVICON[0]|basename_or_url|e }}" type="{{ M_FAVICON[1] }}" />
   {% endif %}
+  {% if not M_SEARCH_DISABLED and M_SEARCH_BASE_URL %}
+  <link rel="search" type="application/opensearchdescription+xml" href="opensearch.xml" title="Search {{ PROJECT_NAME }} documentation" />
+  {% endif %}
   {% block header_links %}
   {% endblock %}
   <meta name="viewport" content="width=device-width, initial-scale=1.0" />
           <div id="search-symbolcount">&hellip;</div>
         </div>
         <div class="m-dox-search-content">
-          <input type="search" id="search-input" placeholder="Loading &hellip;" disabled="disabled" autofocus="autofocus" spellcheck="false" />
+          <form{% if M_SEARCH_BASE_URL %} action="{{ M_SEARCH_BASE_URL }}#search"{% endif %}>
+            <input type="search" name="q" id="search-input" placeholder="Loading &hellip;" disabled="disabled" autofocus="autofocus" autocomplete="off" spellcheck="false" />
+          </form>
           <noscript class="m-text m-danger m-text-center">Unlike everything else in the docs, the search functionality <em>requires</em> JavaScript.{% if M_SEARCH_EXTERNAL_URL %} Enable it or <a href="{{ M_SEARCH_EXTERNAL_URL|replace('{query}', '') }}">use an external search engine</a>.{% endif %}</noscript>
           <div id="search-help" class="m-text m-dim m-text-center">
             {{ M_SEARCH_HELP|indent(12) }}
diff --git a/doxygen/templates/opensearch.xml b/doxygen/templates/opensearch.xml
new file mode 100644 (file)
index 0000000..d477ab2
--- /dev/null
@@ -0,0 +1,9 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<OpenSearchDescription xmlns="http://a9.com/-/spec/opensearch/1.1/">
+  <ShortName>{{ PROJECT_NAME }}{% if PROJECT_BRIEF %} {{ PROJECT_BRIEF }}{% endif %}</ShortName>
+  <Description>Search {{ PROJECT_NAME }} documentation</Description>
+  {% if M_FAVICON %}
+  <Image type="{{ M_FAVICON[1] }}">{{ M_SEARCH_BASE_URL|urljoin(M_FAVICON[0])|e }}</Image>
+  {% endif %}
+  <Url type="text/html" template="{{ M_SEARCH_BASE_URL }}?q={searchTerms}#search"/>
+</OpenSearchDescription>
index 6462a713f3a929e04c29fbcfdb6903640d0130ce..dfc3efb81238fe30e3f75ae7fce604f7c866e4c9 100644 (file)
@@ -96,7 +96,9 @@
           <div id="search-symbolcount">&hellip;</div>
         </div>
         <div class="m-dox-search-content">
-          <input type="search" id="search-input" placeholder="Loading &hellip;" disabled="disabled" autofocus="autofocus" spellcheck="false" />
+          <form>
+            <input type="search" name="q" id="search-input" placeholder="Loading &hellip;" disabled="disabled" autofocus="autofocus" autocomplete="off" spellcheck="false" />
+          </form>
           <noscript class="m-text m-danger m-text-center">Unlike everything else in the docs, the search functionality <em>requires</em> JavaScript. Enable it or <a href="https://google.com/search?q=site:mcss.mosra.cz+{}">use an external search engine</a>.</noscript>
           <div id="search-help" class="m-text m-dim m-text-center">
             Some <em>help</em>.
index e770f90c7037442050b830967f8910cce6f9386e..4bc45dbea103d09b3a4f397210fae0d5fb51eefb 100644 (file)
@@ -60,7 +60,9 @@
           <div id="search-symbolcount">&hellip;</div>
         </div>
         <div class="m-dox-search-content">
-          <input type="search" id="search-input" placeholder="Loading &hellip;" disabled="disabled" autofocus="autofocus" spellcheck="false" />
+          <form>
+            <input type="search" name="q" id="search-input" placeholder="Loading &hellip;" disabled="disabled" autofocus="autofocus" autocomplete="off" spellcheck="false" />
+          </form>
           <noscript class="m-text m-danger m-text-center">Unlike everything else in the docs, the search functionality <em>requires</em> JavaScript.</noscript>
           <div id="search-help" class="m-text m-dim m-text-center">
             Search for symbols, directories, files, pages or modules. You can omit any
index e770f90c7037442050b830967f8910cce6f9386e..4bc45dbea103d09b3a4f397210fae0d5fb51eefb 100644 (file)
@@ -60,7 +60,9 @@
           <div id="search-symbolcount">&hellip;</div>
         </div>
         <div class="m-dox-search-content">
-          <input type="search" id="search-input" placeholder="Loading &hellip;" disabled="disabled" autofocus="autofocus" spellcheck="false" />
+          <form>
+            <input type="search" name="q" id="search-input" placeholder="Loading &hellip;" disabled="disabled" autofocus="autofocus" autocomplete="off" spellcheck="false" />
+          </form>
           <noscript class="m-text m-danger m-text-center">Unlike everything else in the docs, the search functionality <em>requires</em> JavaScript.</noscript>
           <div id="search-help" class="m-text m-dim m-text-center">
             Search for symbols, directories, files, pages or modules. You can omit any
index 1455459c9dd60309331e167c12671a010bf18433..c67b6207f6f4ed5f8c99c56d6f5831fb6ca56a96 100644 (file)
@@ -54,7 +54,9 @@
           <div id="search-symbolcount">&hellip;</div>
         </div>
         <div class="m-dox-search-content">
-          <input type="search" id="search-input" placeholder="Loading &hellip;" disabled="disabled" autofocus="autofocus" spellcheck="false" />
+          <form>
+            <input type="search" name="q" id="search-input" placeholder="Loading &hellip;" disabled="disabled" autofocus="autofocus" autocomplete="off" spellcheck="false" />
+          </form>
           <noscript class="m-text m-danger m-text-center">Unlike everything else in the docs, the search functionality <em>requires</em> JavaScript.</noscript>
           <div id="search-help" class="m-text m-dim m-text-center">
             Halp.
diff --git a/doxygen/test/layout_search_opensearch/Doxyfile b/doxygen/test/layout_search_opensearch/Doxyfile
new file mode 100644 (file)
index 0000000..cdcf5af
--- /dev/null
@@ -0,0 +1,10 @@
+PROJECT_NAME            = "A project"
+PROJECT_BRIEF           = "is cool"
+XML_OUTPUT              =
+
+##! M_PAGE_FINE_PRINT   =
+##! M_THEME_COLOR       =
+##! M_LINKS_NAVBAR1     =
+##! M_LINKS_NAVBAR2     =
+##! M_SEARCH_BASE_URL   = http://localhost:8000
+##! M_SEARCH_HELP       = "Right-click to add a search engine."
diff --git a/doxygen/test/layout_search_opensearch/index.html b/doxygen/test/layout_search_opensearch/index.html
new file mode 100644 (file)
index 0000000..8b5ac6d
--- /dev/null
@@ -0,0 +1,76 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+  <meta charset="UTF-8" />
+  <title>A project is cool</title>
+  <link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Source+Sans+Pro:400,400i,600,600i%7CSource+Code+Pro:400,400i,600" />
+  <link rel="stylesheet" href="m-dark+doxygen.compiled.css" />
+  <link rel="icon" href="favicon-dark.png" type="image/png" />
+  <link rel="search" type="application/opensearchdescription+xml" href="opensearch.xml" title="Search A project documentation" />
+  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+</head>
+<body>
+<header><nav id="navigation">
+  <div class="m-container">
+    <div class="m-row">
+      <a href="index.html" id="m-navbar-brand" class="m-col-t-8 m-col-m-none m-left-m">A project <span class="m-thin">is cool</span></a>
+      <div class="m-col-t-4 m-hide-m m-text-right m-nopadr">
+        <a href="#search" class="m-dox-search-icon" title="Search" onclick="return showSearch()"><svg style="height: 0.9rem;" viewBox="0 0 16 16">
+          <path d="m6 0c-3.3144 0-6 2.6856-6 6 0 3.3144 2.6856 6 6 6 1.4858 0 2.8463-0.54083 3.8945-1.4355-0.0164 0.33797 0.14734 0.75854 0.5 1.1504l3.2227 3.7891c0.55185 0.6139 1.4517 0.66544 2.002 0.11524 0.55022-0.55022 0.49866-1.4501-0.11524-2.002l-3.7891-3.2246c-0.39184-0.35266-0.81242-0.51469-1.1504-0.5 0.89472-1.0482 1.4355-2.4088 1.4355-3.8945 0-3.3128-2.6856-5.998-6-5.998zm0 1.5625a4.4375 4.4375 0 0 1 4.4375 4.4375 4.4375 4.4375 0 0 1-4.4375 4.4375 4.4375 4.4375 0 0 1-4.4375-4.4375 4.4375 4.4375 0 0 1 4.4375-4.4375z"/>
+        </svg></a>
+        <a id="m-navbar-show" href="#navigation" title="Show navigation"></a>
+        <a id="m-navbar-hide" href="#" title="Hide navigation"></a>
+      </div>
+      <div id="m-navbar-collapse" class="m-col-t-12 m-show-m m-col-m-none m-right-m">
+        <div class="m-row">
+          <ol class="m-col-t-12 m-col-m-none">
+          </ol>
+          <ol class="m-col-t-6 m-col-m-none" start="1">
+            <li class="m-show-m"><a href="#search" class="m-dox-search-icon" title="Search" onclick="return showSearch()"><svg style="height: 0.9rem;" viewBox="0 0 16 16">
+              <path d="m6 0c-3.3144 0-6 2.6856-6 6 0 3.3144 2.6856 6 6 6 1.4858 0 2.8463-0.54083 3.8945-1.4355-0.0164 0.33797 0.14734 0.75854 0.5 1.1504l3.2227 3.7891c0.55185 0.6139 1.4517 0.66544 2.002 0.11524 0.55022-0.55022 0.49866-1.4501-0.11524-2.002l-3.7891-3.2246c-0.39184-0.35266-0.81242-0.51469-1.1504-0.5 0.89472-1.0482 1.4355-2.4088 1.4355-3.8945 0-3.3128-2.6856-5.998-6-5.998zm0 1.5625a4.4375 4.4375 0 0 1 4.4375 4.4375 4.4375 4.4375 0 0 1-4.4375 4.4375 4.4375 4.4375 0 0 1-4.4375-4.4375 4.4375 4.4375 0 0 1 4.4375-4.4375z"/>
+            </svg></a></li>
+          </ol>
+        </div>
+      </div>
+    </div>
+  </div>
+</nav></header>
+<main><article>
+  <div class="m-container m-container-inflatable">
+    <div class="m-row">
+      <div class="m-col-l-10 m-push-l-1">
+        <h1>
+          A project
+        </h1>
+      </div>
+    </div>
+  </div>
+</article></main>
+<div class="m-dox-search" id="search">
+  <a href="#!" onclick="return hideSearch()"></a>
+  <div class="m-container">
+    <div class="m-row">
+      <div class="m-col-m-8 m-push-m-2">
+        <div class="m-dox-search-header m-text m-small">
+          <div><span class="m-label m-default">Tab</span> / <span class="m-label m-default">T</span> to search, <span class="m-label m-default">Esc</span> to close</div>
+          <div id="search-symbolcount">&hellip;</div>
+        </div>
+        <div class="m-dox-search-content">
+          <form action="http://localhost:8000#search">
+            <input type="search" name="q" id="search-input" placeholder="Loading &hellip;" disabled="disabled" autofocus="autofocus" autocomplete="off" spellcheck="false" />
+          </form>
+          <noscript class="m-text m-danger m-text-center">Unlike everything else in the docs, the search functionality <em>requires</em> JavaScript.</noscript>
+          <div id="search-help" class="m-text m-dim m-text-center">
+            Right-click to add a search engine.
+          </div>
+          <div id="search-notfound" class="m-text m-warning m-text-center">Sorry, nothing was found.</div>
+          <ul id="search-results"></ul>
+        </div>
+      </div>
+    </div>
+  </div>
+</div>
+<script src="search.js"></script>
+<script src="searchdata.js" async="async"></script>
+</body>
+</html>
diff --git a/doxygen/test/layout_search_opensearch/indexpage.xml b/doxygen/test/layout_search_opensearch/indexpage.xml
new file mode 100644 (file)
index 0000000..fa3eefa
--- /dev/null
@@ -0,0 +1,11 @@
+<?xml version='1.0' encoding='UTF-8' standalone='no'?>
+<doxygen xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="compound.xsd" version="1.8.14">
+  <compounddef id="indexpage" kind="page">
+    <compoundname>index</compoundname>
+    <title>A project</title>
+    <briefdescription>
+    </briefdescription>
+    <detaileddescription>
+    </detaileddescription>
+  </compounddef>
+</doxygen>
diff --git a/doxygen/test/layout_search_opensearch/opensearch.xml.html b/doxygen/test/layout_search_opensearch/opensearch.xml.html
new file mode 100644 (file)
index 0000000..87d0eb1
--- /dev/null
@@ -0,0 +1,7 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<OpenSearchDescription xmlns="http://a9.com/-/spec/opensearch/1.1/">
+  <ShortName>A project is cool</ShortName>
+  <Description>Search A project documentation</Description>
+  <Image type="image/png">http://localhost:8000/favicon-dark.png</Image>
+  <Url type="text/html" template="http://localhost:8000?q={searchTerms}#search"/>
+</OpenSearchDescription>
index d4e451c6e0ed4f1f721c0ff5444240c02d079250..9948518a3acb3644fd3a609e760c60a7c71dd0ba 100644 (file)
@@ -58,6 +58,7 @@ class Doxyfile(unittest.TestCase):
             'M_PAGE_HEADER': 'this is "quotes" \'apostrophes\'',
             'M_SEARCH_DISABLED': False,
             'M_SEARCH_DOWNLOAD_BINARY': False,
+            'M_SEARCH_BASE_URL': '',
             'M_SEARCH_EXTERNAL_URL': '',
             'M_SEARCH_HELP':
 """Search for symbols, directories, files, pages or modules. You can omit any
index 0acf223d23f8fd5c0e88c77854c0b007b4a2ed25..a7a32b7221adf73ce97be43831e0806443c96ae9 100644 (file)
@@ -91,3 +91,14 @@ class SearchBinary(BaseTestCase):
         self.run_dox2html5(wildcard='indexpage.xml')
         self.assertEqual(*self.actual_expected_contents('index.html'))
         self.assertTrue(os.path.exists(os.path.join(self.path, 'html', 'searchdata.bin')))
+
+class SearchOpenSearch(BaseTestCase):
+    def __init__(self, *args, **kwargs):
+        super().__init__(__file__, 'search_opensearch', *args, **kwargs)
+
+    def test(self):
+        self.run_dox2html5(wildcard='indexpage.xml')
+        self.assertEqual(*self.actual_expected_contents('index.html'))
+        # Renamed with a HTML extension so dox2html5's metadata parser doesn't
+        # pick it up
+        self.assertEqual(*self.actual_expected_contents('opensearch.xml', 'opensearch.xml.html'))