cmd | mutt -s "$subject" "$email"

Noting something across devices (almost all of them are connected to my email).

echo "$x $y $z" | mutt -s "Reminder" "$email"

Sending a morning reminder of active tasks (running as a cron job).

task active | mutt -s "Tasks" "$email"

In most of these examples, classic and modern UNIX utilities are used to stitch together different tools. A foundation model is then applied to tackle problems that go beyond well-defined solutions. Finally, UNIX utilities are used again to constrain, refine, or reshape the model’s output, transforming it into something useful.

Creating playlists

Consider a scenario where you have downloaded a number of songs and want to organize them into playlists. Manually selecting tracks so that they smoothly transition from one to the other can be time-consuming and requires intimacy with one’s music collection. However, by using a model that understands music and sound to translate each song into a point in space, and then interpolating between these points, it is possible to automatically create coherent playlists. This recipe takes advantage of the llm¹ utility and some accompanying plugins,² but the technique can be implemented using analogous tools.

A model that understands audio can be used to embed our music collection ($MC) into a high-dimensional space, where similar songs are placed closer together. With the llm-clap plugin, we can generate embeddings for our collection using the CLAP model with llm embed-multi -m clap songs --files $MC '*' Now, each one of our songs and its corresponding embedding are saved in a local embeddings.db database, which we can query. Then, the llm-interpolate plugin returns interpolated points between a starting and ending point (song), creating between them a path (playlist). For example, this one-liner generates a 5-song .m3u playlist between PacifyHer.wav and redrum.wav:

llm interpolate songs "PacifyHer" "redrum" -n 5 | jq .[] > playlist.m3u

Taking notes

It can be tricky to take notes while watching videos of talks. A model that can generate summaries of the video content can be used to generate notes, which can be reviewed and expanded upon later. This can also help rapidly expand one’s set of notes. The following two-liner uses the llm utility to generate a summary of a video transcript downloaded using yt-dlp and finally pipes the output to create a new note object using zk.³

yt-dlp --no-download --write-subs --output "$OUT" "$URL"
cat "$OUT" | llm -s "Create notes" | zk new -i

Generating reports

It is common practice for people working together to have monthly, weekly, or even daily meetings where all members give a short update on what they have been working on. These reports can be frustrating as they demand the right level of abstraction—neither too detailed for team members lacking context nor too broad to allow meaningful feedback. Forgetting the specifics of your recent work adds to the challenge.

Digital todo-list tools like taskwarrior⁴ can be leveraged to generate these reports by smartly querying them and piping their output into an LLM. The following pipeline (1) queries taskwarrior for all of last week’s completed tasks, (2) exports them in json format, (3) uses jq⁵ to extract the .description attribute from each one, and (4) provides the completed task list to an LLM asking it to generate the report.

task status:completed end.after:today-7d export |
jq '.[] | .description' |
llm -s 'Generate a report based on these tasks.'

Renaming pictures

Consider the scenario where you have a large collection of pictures saved. If these pictures are taken by you, or downloaded from the internet, chances are the image files have vague or useless names.

$ ls 
1672714705640839.png 1689964585834142.png 2.jpg

The laborious process of renaming each one can be automated by leveraging a model with image-understanding capabilities. For this recipe, one can use ollama,⁶ a very usable LLM model fetching and inference tool that works great out-of-box with the moondream vision model, which is small enough to allow for quick inference on a modern laptop. The following pipeline finds every .jpg file in the current directory and asks the model to provide a title for it based on the image contents, some light formatting at the end makes whatever the model outputs into a plausible filename.

find . -name "*.jpg" | 
xargs -I{} ollama run moondream "Title for this: {}" |
tr ' ' '_' | sed 's/$/\.jpg/'

The (slightly truncated) output filenames are (zoom-in to confirm): A_green_dragon_with_wings_and_a_tail.jpg , A_painting_of_a_serene_landscape.jpg , urns_of_stone_red_car_in_foreground.jpg .

Conclusion

This article serves as a starting inspiration point for the community to start using these technologies for fun and profit. Please reach out with thoughts and ideas.

This article is part of Paged Out! magazine, issue #6. You can get a free PDF of the magazine on the Paged Out! website.

The gh CLI allows you to list all issues in a repository as JSON. This can include attributes like title, url, and labels. You can then use jq to transform this output into a line based format, extract whatever attributes you need, and finally add them as tasks with task add.

The gh issue ls --json title,url outputs something like this:

[
  {
    "title": "An Issue",
    "url": "https://github.com/user/repo/issues/1"
  },
  {
    "title": "Another Issue",
    "url": "https://github.com/user/repo/issues/2"
    }
]

Then, jq -c '.[]' will flatten the array and output an issue per line:

{"title":"An Issue",     "url":"https://github.com/user/repo/issues/1"}
{"title":"Another Issue","url":"https://github.com/user/repo/issues/2"}

The final script looks like this:

while read -r issue; do
  title=$(jq -r '.title' <<< "$issue")
  url=$(jq -r '.url' <<< "$issue")
  task add "$title" url:"$url"
done < <(gh issue ls --json title,url | jq -c '.[]')

Here, I’ve added url as a user-defined-attribute (UDA). I can then inspect a task with task info <task-id> to see the URL.

Example output (truncated):

$ task
ID Age   Description 
 1 1min  An Issue
 2 1min  Another Issue

$ task info 1
Name          Value
ID            1
Description   An Issue
Status        Pending
URL           https://github.com/user/repo/issues/1

You can also turn issue labels into task tags by adding labels to the gh issue invocation, like gh issue ls --json title,url,labels.

The MCI Protocol

I wrote a parser for the MCI protocol, an old Windows media API. I was given quite a bit of documentation regarding MCI, and the fact that this protocol even existed was very interesting.

MCI stands for “Media Control Interface” and is a high-level API for controlling media devices. It is now mostly unused in favor of more modern protocols like DirectShow (part of DirectX) and Universal Windows Platform. The idea is that device drivers would implement API and would thus be controllable through high-level commands like play file.wav from 0 to 100.

The general command structure is:

command <device> [parameters ...]

It’s actually quite a nice and simple way of controlling your devices.

IBM filed a patent in 2002 for the mechanism which parses MCI commands. The entire document was quite intriguing, since they are explaining quite a simple program that takes a command string and turns it into a data structure² (basically just a parser). They even describe the tokenization procedure and that tokens will be saved in a linked list (as opposed to an array), probably to fascilitate for low memory devices. I wonder if someone implementing the MCI protocol would have to pay IBM for it.

The Command Table

An interesting part of their implementation is how there is no specific command list/specification. Rather, a command table will have to be provided which is parsed and each command’s arguments are determined from it. This way, someone can add new commands to the protocol by just modifying this table. In our code, we keep the table as a structure in-memory. Here is part of the table that corresponds to MCI’s open commmand:

static CmdTableRow table[] = {
  {"open"            ,MCI_OPEN      ,0          ,MCI_COMMAND_HEAD },
  {""                ,MCI_INTEGER   ,0          ,MCI_RETURN }      ,
  {"notify"          ,0x00000001L   ,-1         ,MCI_FLAG }        ,
  {"wait"            ,0x00000002L   ,-1         ,MCI_FLAG }        ,
  {"type"            ,0x00002000L   ,-1         ,MCI_STRING }      ,
  {"element"         ,0x00000200L   ,-1         ,MCI_STRING }      ,
  {"alias"           ,0x00000400L   ,-1         ,MCI_STRING }      ,
  {"shareable"       ,0x00000100L   ,-1         ,MCI_FLAG }        ,
  {""                ,0x00000000L   ,-1         ,MCI_END_COMMAND } ,
  // ...

For my implementation, I studied how Wine implemented an MCI parser. Their code is a little esoteric, thanks to variable names like dwParam2 and S_MciCmdTable[uTbl].lpTable. It’s not entirely the Wine project’s fault though, as they are following old windows coding style conventions.

Studying Wine’s code gave me some insight into what “compatibility layer” means. If a program had code which executed MCI commands using the Win32 API, Wine goes through and actually parses them at a high level, “translating” them into calls to Linux sound/video APIs.

This is exactly what we did in ScummVM, parsing the MCI commands and running the corresponding calls to ScummVM’s sound/video APIs. You can find ScummVM’s MCI parser implementation here. We decided to steer away from the architecture provided in the afformentioned patent, providing a more readable API by saving each command’s arguments in a hash table rather than a byte array.