Highlighting code in django templates

17 September 2021

The documentation for GPXZ is powered by django. I have html templates with code snippets wrapped in <pre><code> tags:

<p>Here's an example request:</p>
<pre><code>import requests

API_KEY = "{{ demo_api_key }}"

response = requests.get(
    'https://api.gpxz.io/v1/point?lat=1&lon=2',
    headers={'x-api-key': API_KEY},
)
print(response.json())
</code></pre>

What I want is for those code snippets to be identified and marked up with syntax highligting classes, so they can be styled with css:

<p>Here's an example request:</p>
<pre class="highlight"><code><span></span><span class="kn">import</span> <span class="nn">requests</span>

<span class="n">API_KEY</span> <span class="o">=</span> <span class="s2">"ak_demo_2JbHmJkX5bidX5b7"</span>

<span class="n">response</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span>
    <span class="s1">'https://api.gpxz.io/v0/point?lat=1&amp;lon=2'</span><span class="p">,</span>
    <span class="n">headers</span><span class="o">=</span><span class="p">{</span><span class="s1">'x-api-key'</span><span class="p">:</span> <span class="n">API_KEY</span><span class="p">},</span>
<span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">response</span><span class="o">.</span><span class="n">json</span><span class="p">())</span>
</code></pre>

The easy option: javascript

There are a number of javascript libraries like prism and highlight that will easily solve this problem. You simply add their script and css to your html <head>: the library will detect any code blocks and apply appropriate highlighting.

This is a great approach for many people, but it wasn’t the direction I wanted to go:

The script may be loaded after the code snippets are initially rendered, resulting in a Flash of Unstyled Content.
It means and extra http request and extra computation for every single request, just to calculate something that is known at build time.

Pygments

Instead of using a javascript library I used pygments, a python package that can take a string of code and add the required html classes for highlighting. Pygments uses the same highlighting classes as my jekyll blog, so I didn’t need to add any extra css to get identical highlighting across the blog and documentation.

First I added data-highlight-lang="python" attributes to the <pre> tags I wanted to be highlighted. Next, I created a django middleware that would check the response content for code snippets, extract any with beautifulsoup, and replace them with the highlighted markup:

class HighlightCodeMiddleware:
    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        response = self.get_response(request)


        # Only consider highlighting html.
        if "text/html" not in response.headers["Content-Type"]:
            return response

        # Check there is code to be highlighted.
        if b'data-highlight-lang' not in response.content:
            return response

        # Find code blocks.
        soup = BeautifulSoup(response.content, 'lxml')
        pre_tags = soup.select('pre[data-highlight-lang]')

        for pre in pre_tags:
        
            # Use the language hint to load the appropriate lexer.
            lang = pre.get('data-highlight-lang')
            lexer = pygments.lexers.get_lexer_by_name(lang)
            
            # Do the highlighting.
            formatter = pygments.formatters.HtmlFormatter()
            highlit = pygments.highlight(code.text, lexer, formatter)
            code.clear()
            for c in list(BeautifulSoup(highlit,  'html.parser').select_one('pre').children):
                code.append(c)
            
            # Add highlight to classlist to be targeted by css.
            classlist = pre.get('class', [])
            classlist.append('highlight')
            pre['class'] = list(set(classlist))
            
        # Return the beautifulsoup object back to bytes.
        response.content = str(soup).encode()

        return response

This does add overhead to each request, but I’m caching response content for these documentation pages so the impact is minimal.

Going the python route also gives more control: I was able to add a custom lexer for a language that wasn’t supported by pygments.