Russell Parker

How Learning GPG Is Like Learning Git

russell.parker7@gmail.com (Russell Parker) — Tue, 10 Apr 2018

Experience of Learning GPG

I’ve been trying to teach myself more about cryptography and GPG in particular. The resources I’ve encountered so far mainly fall into two categories:

Step-by-step setup and configuration guides
Tutorials on how to accomplish a specific task

Some include a primer on public-key cryptogaphy but they otherwise lack details that would help build a mental model of GPG. One notable exception was Anatomy of a GPG Key which also echoes similar observations about the style of other guides.

GPG is often criticized for its poor usability and perhaps it’s this poor usability that drives people to make instructional resources rather than conceptual ones. I think a better approach to teaching the tool is emphasizing the underlying structure of keys and how various steps affect this structure. Running a series of commands to perform key maintenance would be more intuitive with an understanding of what’s going on at the key packet-level.

This perspective on studying GPG has parallels in how some people learn Git.

Learning Git

There are tons of Git tutorials covering things like cloning repos, staging changes, and pushing commits. There is nothing wrong with starting here but just memorizing these routines can eventually obscure the simplicity of the underlying system.

Mastering Git means touching on blobs and Merkle trees and SHA-1 hashes. Knowing these details provides a more manageable mental map which is important because a big part of learning is creating a mental scaffolding and incorporating auxiliary info and data points.

Chapter 10 of Pro Git opens with the tradeoff between introducing porcelain commands and covering the core design of Git.

…this is where we’ll go over the inner workings and implementation of Git. We found that understanding this information was fundamentally important to appreciating how useful and powerful Git is, but others have argued to us that it can be confusing and unnecessarily complex for beginners.

While it may be “confusing and unnecessarily complex” to begin here, I’d argue that covering these inner workings eventually leads to a more intuitive grasp of how to accomplish tasks with Git rather than just regurgitating a series of commands.

Conclusion

In The Mythical Man-Month: Essays on Software Engineering, Fred Brooks wrote,

Show me your flowcharts and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won’t usually need your flowcharts; they’ll be obvious.

This line of thinking applies to learning tools as well. Ultimately, a crucial part of understanding systems like Git and GPG is understanding their data strctures. We’re always going to need reference material but focusing too much on just procedural steps will prevent a simpler comprehension.

More man Improvements

russell.parker7@gmail.com (Russell Parker) — Mon, 12 Mar 2018

In a previous post I explored using colors with man. Here are a few more adjustments to make browsing manpages nicer.

We can make navigating the sections of a particular page a little bit easier. By default, there is no simple way to do this. Going off of this forum, we can add some searchable text to denote a section heading:

.\" man.local

.\" Add a searchable string before each section header
.rn SH SH-orig
.de SH
.SH-orig \(sc \\$*
..

This file, man.local, defines a groff macro that uses the escape code \(sc to prepend the glyph § to section headings. The result looks like this:

§ NAME
       man - an interface to the on-line reference manuals

This makes it simpler to navigate a man page because we can search for the glyph and go to the next and previous section with n and N, respectively. less keeps a history so repeating a previous search is easy, but we can also create a key-binding that automates this.

# ~/.lesskey

#command
    # remap tab to search for section
    \t forw-search §\r

Note: #command is not a comment but a required directive.

This can be compiled for less with the lesskey command which will bind Tab to searching for the above glyph.

Prompt

The prompt is a ribbon of text displayed at the bottom of a less screen. man has a default prompt which attempts to show the total number of lines and the current position as a percent. Since the formatted manpage is piped into less though the program does not know about the total length. If you reach the bottom of a file though (try the G command), this additional information will be displayed. A hack to jump to the end of a file and back again is:

# ~/.bashrc

export MANPAGER='less +Gg'

This tells less to run the commands G and g when opening a file which scans to the end of the input and then back to the beginning. By doing this, less figures out the total length of the file and can print the total line numbers and current percent in the ribbon. The tradeoff is that this can cause a noticeable performance hit when opening large man pages like bash. This can also be accomplished by setting the LESS environment variable but doing so can interfere with other programs that use less.

The prompt itself can be simplified somewhat as well to get rid of the overbearing “(press h for help or q to quit)” message.

# ~/.bashrc

export MANLESS='\ Manual\ page\ \$MAN_PN\ %lt/%L %pB\%'

Line Length

I find reading manpages in wide terminals more difficult (although it may make skimming easier). Setting a maximum line width for man. MANWIDTH does exactly this but will hard-code the value instead of acting as a maximum. A solution is to put some logic in a BASH_PROMPT function.

# ~/.bashrc

export PROMPT_COMMAND=prompt
prompt() {
    # set line width for man pages with maximum of 80
    if [ $COLUMNS -lt 80 ]; then
        export MANWIDTH=$COLUMNS
    else
        export MANWIDTH=80
    fi
}

Samsung Chromebook 3 as a Developer Laptop

russell.parker7@gmail.com (Russell Parker) — Wed, 07 Mar 2018

Picking a Laptop

After my last machine broke down last summer I needed a new laptop. I’d been using an ancient HP Pavillion g6 which was bulky and had a dismal battery life (less than 30 minutes) that had finally begun failing after years of service.

My requirements in a laptop are pretty low-spec. I have no need for a powerful CPU, huge amounts RAM, a discrete graphics card, or even large storage. My programming workflows don’t involve running VMs or compiling massive codebases so my heaviest workloads are basically streaming Netflix and using JavaScript-heavy web apps. My priorities are:

long battery life
low cost
small & lightweight

It turns out that there aren’t a ton of modern laptops that match this criteria. Chromebooks are one of the few products that strike a balance I was happy with.

After far too much research, I picked up a Samsung Chromebook 3. It has a dual-core Intel Braswell processor, 4GB of RAM, an 11” display, and weighs in at just two and a half pounds. Although prices have bounced higher recently, I saw it selling on Black Friday for $120. It comes with a minuscule amount of storage (mine’s got 16GB of SSD) but has a micro SD card slot to expand with.

GalliumOS

I decided to install GalliumOS as a Linux distro to replace ChromeOS (the OS that Chromebooks ship with). GalliumOS is Ubuntu-based and designed for Chromebooks.

Installing involved a few extra steps before beginning the normal Linux ISO install process:

Enabling Developer Mode on the Chromebook
Removing the write-protect screw¹
Flashing custom firmware

The GalliumOS wiki provides a guide as well as a useful table for hardware support.

Experience review

I’ve really enjoyed using the Chromebook so far. It’s pretty snappy, really light, and the battery life is unreal. The portability is further enhanced because it’s so cheap to replace that I don’t second-guess taking it places where it could stolen or broken. I think it’s ideal for developers who are looking for a basic, no-frills laptop.

There have definitely been some issues. The built-in microphone currently only records static. Switching audio output when plugging in/out headphones took some effort to sort out. I had a recurring problem where audio would occasionally break when suspending under memory load.

Things aren’t perfect but because GalliumOS focuses on supporting ChromeOS devices the community is aggressive about fixing things that could easily fall by the wayside with larger distros. One of the project maintainers writes²:

We have a history of fixing audio (in/out) on ChromeOS devices even when it is not fixed upstream or in other distros, and it is a priority for us

Overall I’m pretty happy with my Samsung Chromebook 3 and I think it’s a great option as a simple Linux machine.

I used this great guide for removing the Samsung Chromebook 3 write-protect screw ↩
Source ↩

Plotting Website Loads with HAR and gnuplot

russell.parker7@gmail.com (Russell Parker) — Mon, 26 Feb 2018

It’s super useful to analyze the performance of websites and there are some great tools out there like Google PageSpeed Insignts and WebPageTest. One shortfall is they do not let you customize the output much, and I would really like to be able to create a nice waterfall diagram. HAR steps in as a format standard for capturing and exporting HTTP network data and opens the door to do-it-yourself diagram plotting.

Generating a HAR archive

Exporting to HAR format is built-in for Chrome. As explained here, from the Network tab under Developer tools you can simply right-click and export the current HTTP network data as HAR.

Firefox needs an extension to handle exporting to HAR; NetExport was recommended but the project looks abandoned now and isn’t available as a Firefox Add-On.

Plotting with gnuplot

Among the collection of projects related to HAR is har2gnuplot. It’s a handy tool that creates a waterfall plot in gnuplot and saves the result as a PNG image.

To get the project up and running on Ubuntu:

$ git clone git@github.com:fh-salzburg/har2gnuplot.git
$ sudo apt install -y nodejs gnuplot
$ npm install underscore module cli

Now, after copying a .har file into the project directory and running make, a PNG and .gnu file will be generated. After some tweaks with the axes and legend, I got:

Aside from the missing URIs, this image would be perfect to drop into a document or website to visualize resource load times.

Maybe if I have some time I’ll work on a har2gnuplot clone in Python with haralyzer and gnuplot-py to add URI labels and learn more about gnuplot.

Adding Colors to man

russell.parker7@gmail.com (Russell Parker) — Fri, 23 Feb 2018

Color configuration

A somewhat well-known configuration for man is enabling colors. This StackExchange answer provides a great intro to the technologies behind colorizing manpages while this one serves as a more practical guide to getting things working.

The gist is that you can add something like the following to your .bashrc¹ file:

# ~/.bashrc

export MANROFFOPT='-c'

export LESS_TERMCAP_mb=$(tput bold; tput setaf 2)
export LESS_TERMCAP_md=$(tput bold; tput setaf 6)
export LESS_TERMCAP_me=$(tput sgr0)
export LESS_TERMCAP_so=$(tput bold; tput setaf 3; tput setab 4)
export LESS_TERMCAP_se=$(tput rmso; tput sgr0)
export LESS_TERMCAP_us=$(tput smul; tput bold; tput setaf 7)
export LESS_TERMCAP_ue=$(tput rmul; tput sgr0)
export LESS_TERMCAP_mr=$(tput rev)
export LESS_TERMCAP_mh=$(tput dim)

Resulting in:

MAN(1)                        Manual pager utils                        MAN(1)

NAME
       man - an interface to the on-line reference manuals

SYNOPSIS
       man  [-C file]  [-d]  [-D]  [--warnings[=warnings]]  [-R encoding] [-L
       locale] [-m system[,...]] [-M path] [-S list]  [-e extension]  [-i|-I]
       [--regex|--wildcard]   [--names-only]  [-a]  [-u]  [--no-subpages]  [-P
       pager] [-r prompt] [-7] [-E encoding] [--no-hyphenation] [--no-justifi‐
       cation]  [-p string]  [-t]  [-T[device]]  [-H[browser]] [-X[dpi]] [-Z]
       [[section] page ...] ...
       man -k [apropos options] regexp ...
       man -K [-w|-W] [-S list] [-i|-I] [--regex] [section] term ...
       man -f [whatis options] page ...
       man -l [-C file] [-d] [-D] [--warnings[=warnings]]  [-R encoding]  [-L
       locale]  [-P pager]  [-r prompt]  [-7] [-E encoding] [-p string] [-t]
       [-T[device]] [-H[browser]] [-X[dpi]] [-Z] file ...
       man -w|-W [-C file] [-d] [-D] page ...
       man -c [-C file] [-d] [-D] page ...
       man [-?V]

DESCRIPTION
 Manual page man(1) line 1 (press h for help or q to quit)

How does this work?

To understand how the above environment variables work it helps to review what steps normally happen when viewing a manpage:

man renders a page from a (likely compressed) nroff or troff/groff document and pipes the result it into the pager program, usually less
If the piped text indicates formatting that needs to be performed then less has to figure out how to accomplish this for terminal output
less uses the (deprecated) termcap database to look up how to achieve effects like underline and bold. In reality it ends up using termcap’s successor, the terminfo database, which maintains support for the termcap interface. This gives back an escape string which corresponds to the specified effect for your particular terminal
Using these nifty escape strings, less finally displays the manpage to the user

Manpages use formatting like bold (for sections) and underlines (for things like file names and arguments). These should already work out of the box when using the man command but will not change the text color.

MAN(1)                        Manual pager utils                        MAN(1)

NAME
       man - an interface to the on-line reference manuals

SYNOPSIS
       man  [-C file]  [-d]  [-D]  [--warnings[=warnings]]  [-R encoding] [-L
       locale] [-m system[,...]] [-M path] [-S list]  [-e extension]  [-i|-I]
       [--regex|--wildcard]   [--names-only]  [-a]  [-u]  [--no-subpages]  [-P
       pager] [-r prompt] [-7] [-E encoding] [--no-hyphenation] [--no-justifi-
       cation]  [-p string]  [-t]  [-T[device]]  [-H[browser]] [-X[dpi]] [-Z]
       [[section] page ...] ...
       man -k [apropos options] regexp ...
       man -K [-w|-W] [-S list] [-i|-I] [--regex] [section] term ...
       man -f [whatis options] page ...
       man -l [-C file] [-d] [-D] [--warnings[=warnings]]  [-R encoding]  [-L
       locale]  [-P pager]  [-r prompt]  [-7] [-E encoding] [-p string] [-t]
       [-T[device]] [-H[browser]] [-X[dpi]] [-Z] file ...
       man -w|-W [-C file] [-d] [-D] page ...
       man -c [-C file] [-d] [-D] page ...
       man [-?V]

DESCRIPTION
 Manual page man(1) line 1 (press h for help or q to quit)

For an idea of what capabilities less queries from the termcap database, a few of them (along with their terminfo database equivalents) are:

terminfo capname	termcap name	description
bold	md	turn on bold
blink	mb	turn on blinking
sgr0	me	turn off all attributes
smso	so	begin standout mode
rmso	se	exit standout mode
rev	mr	turn on reverse mode
dim	mh	turn on half-bright mode
rmul	ue	exit underline mode
smul	us	begin underline mode

Overwriting Escape Strings

Depending on your system, by the time the manpage reaches less it may already be formatted with escape strings which means that less does not have to do any work to figure out how to display it. The grotty manpage explains this behavior as well as the -c option which seems to apply to most nroff and troff/groff rendering programs:

By default, grotty emits SGR escape sequences (from ISO 6429, also called ANSI color escapes) to change text attributes (bold, italic, colors).

and

Use the -c switch to revert to the old behaviour, printing a bold character c with the sequence `c BACKSPACE c’ and an italic character c by the sequence `_ BACKSPACE c’.

So the manpage rendering program assumes that our terminal interprets a few basic ANSI color escape strings and makes use of them. This is a pretty safe assumption because ANSI codes were standardized a long time ago and modern terminal emulators conform to at least a subset of it. But what if we want to force less to figure out the escape codes on its own? We can use the -c option to force the old behavior and we’ll have man pass this option along when it invokes the renderer by setting the MANROFFOPT environment variable.

A behavior that we ignored earlier is that less actually looks for specific environment variables before proceeding to query termcap. These follow the format LESS_TERMCAP_ suffixed with a termcap name and manually define the escape strings to use. By using these we can essentially lie to less about how to achieve effects like bold so that bolding text also adds colors to it.

Setting Specific Effects

The tput command abstracts over different terminal types to output terminal-dependent escape sequences². This makes the configuration easier to maintain and will still work in the off-chance that your terminal does not use the normal ANSI escape sequences.

Most of the capnames used in the example are pretty self-explanatory. setaf and setab set the foreground and background color, respectively, using the color codes as defined in terminfo:

Color	Code
black	0
red	1
green	2
yellow	3
blue	4
magenta	5
cyan	6
white	7

Really this kind of environment variable belongs in the relatively-unknown lesskey file which has the capability of setting environment variables specifically for less. Unfortunately there is a bug in versions of less below 530 explained well here ↩
These are horrible strings to write and look like “^[[33m”. The Bash Hackers Wiki has a great page on the topic. Try it out in the form tput setaf 2 | cat -v ↩

Bash Autocomplete for tmux

russell.parker7@gmail.com (Russell Parker) — Fri, 16 Feb 2018

I’ve recently become a big fan of tmux and have put far too much time into customizing my config file. When working in the terminal it helps me compartmentalize work and lets me come back a project right where I left off with all of my sessions and windows named to keep things organized. These features are useful when working locally but are a godsend when SSH’d into a remote machine because tmux can keep sessions alive over multiple SSH logins, allowing me to detach from a session and attach to the same one days later.

The one big pain point so far has been tab-completion. Finding and reattaching to a session often looks like this:

$ tmux list-sessions
session-1: 3 windows (created Fri Feb 16 16:37:48 2018) [80x23]
session-2: 4 windows (created Thu Feb  1 15:42:16 2018) [80x23]
session-3: 4 windows (created Tue Feb 13 16:09:11 2018) [80x64]
$ tmux attach-session -t session-1

Bash will tab-complete tmux but not the tmux command attach-session or the specific session name. Having to type out all of this out can get annoying and with completion we could skip tmux list-sessions entirely because hitting Tab should list all of the running sessions for us.

The tmux README recommends using the wonderful tmux-bash-completion script for completions. It can be sourced from .bashrc but this approach is way too easy. We need to keep our home directory clean from non-standard files.

The bash-completion package ships with Ubuntu and provides a ton of completions but unfortunately does not support tmux out of the box (if it did we would’t be addressing this in the first place). One solution is just to save the recommended script to the bash-completion’s completionsdir (which is /usr/share/bash-completion/completions/ on my system). This works fine if you have root privileges but is not portable across systems where you only have basic user privileges, which is the case for me on my university’s Linux servers. After running sudo once on them (and quickly hitting CTRL+C once I realized what I had done), I learned that the admins really do not appreciate students trying to execute commands as root.

A simple work-around is saving the tmux-bash-completion script as ~/.bash_completion which the bash-completion script will parse automatically. Other custom tab-completion scripts can live here as well and it allows you to easily manage the script alongside additional home directory dotfiles with GNU Stow. Perfect!

Download with:

$ curl https://raw.githubusercontent.com/imomaliev/tmux-bash-completion/master/completions/tmux > ~/.bash_completion