<< home
10 minute read

Yak-shaving the software build process

As usual, I am suffocating under a stack of yaks. Come with me on the journey of this particular yak stack, written in faux-present tense so it feels like you’re watching it happen:

Dissecting wayland

I have an academic interest in doing things with windowing on Linux at a lower-level than what windowing toolkits provide. It’s really hard to grasp what the roles of all the pieces are of a graphical Linux environment.

Installing software

First, a little background: My preferred at-home dev box is an installation of KDE Manjaro, which is a flavor of Arch. This wasn’t a frivolous decision, it is a result of intense familiarity of plain Arch in a server administration context. It’s not without its warts, as you will see.

In order to play around with a desktop environment, I want a sandbox to play in. I try to pull VirtualBox off the shelf:

Welcome to fish, the friendly interactive shell
Type help for instructions on how to use fish
~ % sudo pacman -S virtualbox
[sudo] password for alexg:
resolving dependencies...
:: There are 9 providers available for VIRTUALBOX-HOST-MODULES:
:: Repository extra
  1) linux419-virtualbox-host-modules
  2) linux510-virtualbox-host-modules
  3) linux515-virtualbox-host-modules
  4) linux518-virtualbox-host-modules
  5) linux519-virtualbox-host-modules
  6) linux54-virtualbox-host-modules
:: Repository community
  7) linux518-rt-virtualbox-host-modules
  8) linux519-rt-virtualbox-host-modules
  9) virtualbox-host-dkms

Enter a number (default=1): _

Immediately I am puzzled by this. Which do I pick? I can divine that I am supposed to pick the option that matches my kernel version. Does anyone have their kernel version memorized?

I go to figure out which kernel I am on in the settings, already pretty annoyed because pacman already knows my kernel version and ought to be able to act on that info. Then I find out that my kernel wasn’t even on the supported list.

Updating software

99% of the time this is painless, and I generally like the rolling release nature of Arch because it avoids the headache of a “distupgrade”, and it keeps you running the latest software.

So I am not that worried, this is just a nudge to run sudo pacman -Syu, then I can reboot –

:: installing fmt (9.1.0-1) breaks dependency 'libfmt.so=8-64'
    required by aseprite

I begin to hear the sound of static, and shake my head. But not to worry just yet, I didn’t have AUR in the mix. Silly me, I should’ve just run yay -Syu, but my patience is already wearing a bit thin. I allow the compilation to merrily chug away only to be greeted again by:

:: installing fmt (9.1.0-1) breaks dependency 'libfmt.so=8-64'
    required by aseprite

The static gets louder, the room spins, and I begin to question my existence. Clearly this was not the path I was meant to go down. I have not been eaten by a grue, alas there are no steps forward.

I uninstall aseprite to get it out of my way, but I have fully lost sight of the top of the yak stack at this point. I become transfixed by all the possibilities of better system package management, based on the knowledge that it could be better (Are traditional package managers still useful?). In particular, there’s no reason why there couldn’t be two versions of a library on my computer.

Okay, I have to fix this.

So I pivot to an existing pet project, aspen-pkgbuild, a project originally created just to automate the process outlined by Linux From Scratch. The project has its name because it has evolved into a project that builds packages for yet-to-be-created OS called Aspen. The project had gone stale since last working on it since it was quickly outgrowing its architecture (it was an off-the-cuff exploratory project). So it’s time to dust off the repo by moving the entire project to old/.

This goes pretty swimmingly (and had its own healthy yak stack to boot). I had a breakthrough idea of using docker build exclusively, which beats the hell out of trying to run bash in a container and trying to feed commands to it. I was also depending pretty heavily on rake for dependency management.

I spend 2 weeks down this path until I really start to stress the limits of rake, which was created to conceptually match make of unix fame. Both of these tools primarily use timestamps to track if dependencies should be updated, which is a pretty fragile mechanism. Also both tools require manually listing dependencies, which is frustratingly hard to get right.

I almost re-shelve this project but then I saw “Stop Writing Dead Programs” by Jack Rusher at Strange Loop. And I got inspired.

“You have to write everything 3 times before you even get to something worth continuing.”

I want to fix the cognitive slowdown I was experiencing with rake by replacing it wholesale. I feel pretty confident about this move because rake’s core functionality clocks in at only 100 lines. I go to GitLab and make a new repository, and I get excited by the sight of a blank slate. It’s a real good sign my first instinct is to write a detailed readme. Then I get to work sketching out some code:

~/Pro/aspen-build_system (main|✔) % exe/abuild
doing some work
doing some work
^C⏎

It’s ALIVE!

I am probably going to be on this yak for awhile, I am actually having fun here. I hope follow up this post at a future date, and link the follow up here at the bottom. I don’t have any kind of RSS set up so maybe check back in 3 months? No guarantees though, you aren’t the boss of me :) EDIT Jun 2023: slim chance of a follow-up because this project concept is too generalized, and I am likely to make something more specific to my use case (which is building things within docker).

If you found this article interesting or had any questions/critiques, please reach out to blog@a.lexg.dev and include a link to the article. In particular, please let me know if you reference this article elsewhere, so I can add a link to you.