btbytes.com

Using Nix Shell to build this website - Part 1

3 min read

Recently, I came across nix.dev, and I read through the new tutorial material on Nix1. Many things clicked for me including the nix syntax basics.

I decided to apply my recent knowledge to build a reproducible environment for developing and building this website. In short, this website uses Python, many python packages, Pandoc etc to convert markdown to html pages. I also use ansible to deploy the website and its various configurations (eg: nginx, algernon).

It is important for me to be able to keep track of all the details so that I can confidently use this as my playground to try different things without leaving the website in a broken state (which I have done many times, and would like to avoid.. because .. Cool URIs don’t change).

First kiss, and broken teeth

As the Sanskrit saying goes - “prathama cuṁbanaṁ daṁta bhagnam”

I tried this very innocuous shell.nix to start with. Nothing major — a stable nixos version (21.05), and a couple basic packages I need to start building this website — Pandoc, and Python3.10:

{ pkgs ? import (fetchTarball "http://nixos.org/channels/nixos-21.05/nixexprs.tar.xz") {} }:

pkgs.mkShell {
  buildInputs = [
    pkgs.which
    pkgs.htop
    pkgs.zlib
    pkgs.pandoc
    pkgs.python310
  ];
}

I ran nix-shell, which read shell.nix, install packages, and provide a shell with these dependencies in the $PATH.

Immediately, I ran into error: Package ‘ghc-8.10.4’ in /nix/store/y9zg15v4amzmffl3vdv7aay8kzczy7qd-source/pkgs/development/compilers/ghc/8.10.4.nix:263 is not supported on ‘aarch64-darwin’, refusing to evaluate.

Ah, yes, the olde ghc installation errors. I added { allowUnsupportedSystem = true; } to ~/.config/nixpkgs/config.nix, and tried again with no luck. I had to proceed with installing pandoc, pandoc-crossref using homebrew, and commented out pkgs.pandoc from the above file.

Python3.10 instllation also failed, and I downgraded pkgs.python310 to pkgs.python3, which ended up installing Python3.8.11.

Not exactly a ringing endorsement so far.

Installing Python packages

I had already captured python package dependencies in a requirements.txt file and I would install them into a virtualenv, and now I’ve to see how I can get these installed using nix-shell instead.

The new shell.nix with python packages looks like this:

{ pkgs ? import (fetchTarball "http://nixos.org/channels/nixos-21.05/nixexprs.tar.xz") {} }:

  let
    python-with-my-packages = pkgs.python9.withPackages (p: with p; [
      wheel
      # pandoc
      jinja2
      markdown
      libsass
      pygments
      dateparser
      # bs4
      # gitpython
      pybtex
      htmlmin
      # extruct
      tinydb
      # pypandoc
      # pytidylib
      pyquery
      # verbalexpressions
      panflute
      # markdown-full-yaml-metadata
      isoweek
      pinboard
      # pinboard-to-sqlite
    ]);
  in
    pkgs.mkShell {
      buildInputs = [
        pkgs.which
        pkgs.htop
        pkgs.zlib
        python-with-my-packages
    ];
    }

Why the commented out package names? I do not know. But some reasons nix-shell does not like them, like this:

error: undefined variable 'bs4'

       at /Users/pgowda/btbytes/shell.nix:13:7:

           12|       dateparser
           13|       bs4
             |       ^
           14|       # gitpython
(use '--show-trace' to show detailed location information)

and with that I’m putting this experiement to rest for the time being.

Footnotes

  1. I won’t go into what Nix, and NixOS are. There are plenty of blog posts and articles that explain why this new method of building reproducible systems is a great leap for Unix Systems Engineering. You can alo see my NixOS page. ↩

Graph View