TIL

• …

Excerpts from Julia Evans: TIL by Julia Evans:

TILs

RSS

Some small things I’ve learned:

`**` works for globbing in the shell Dec 13 2024

I just learned that globbing in fish is WAY more powerful than I realized, like:

• instead of find . -name '*.md' I can just do ls **.md
• instead of find . -type d -name ‘*blah*’, I can just ls -d **blah*/

A couple of notes:

• apparently you can do this in zsh or bash (with bash’s extended globbing) too, though it might be **/*.md instead of **.md
• as usual globs don’t include dotfiles so it’s not exactly the same as running find

Learned this from this page from 2004 on shells which mentioned offhand that “under zsh, [globbing] is so powerful, it almost makes find redundant.”

Some programming languages buffer stdout and some don’t Nov 27 2024

Today I did some buffering tests in 10 languages where I wrote the equivalent of this program and then ran the program with ./program | grep hello

while True:
    print("hello")
    sleep(1)

• In 4 of the languages (Python, Ruby, C, Perl) nothing at at all gets printed (because they buffer the output when stdout isn’t a TTY, the buffer size is probably 4k or something)
• in the others (Go, C++, JS, Rust, Java, Lua, bash) you see “hello” printed out once per second. PHP is in this category too.

I knew there was some variation here I didn’t have a list of which languages did which and now I do. (If you want to not buffer the output you can flush stdout)

Here’s the code I ran to test this

Copying to your clipboard over SSH in vim with OSC52 Nov 21 2024

I have my local vim set up to integrate with my system clipboard (so that p pastes from my system clipboard etc), but I thought it was impossible to do the same thing over SSH. It turns out that actually it is possible with an ANSI escape sequence called OSC52! A few links:

• It’s a core Neovim feature: the neovim OSC 52 docs
• a vim-osc52 plugin
• a vim github issue about adding OSC 52 support

I haven’t tried this yet but it seems very cool

Dmitry Mazin also told me you can create this script on a remote host, call it pbcopy, and piping into it will copy to your clipboard! I tested it and it works.

#!/bin/bash
printf "\033]52;c;%s\007" "$(base64 | tr -d '\n')"

Every signal that the TTY sends processes Nov 12 2024

Someone pointed me to The TTY demystified on Mastodon and I appreciated the list of every signal that the kernel’s terminal driver sends processes (SIGWINCH, SIGPIPE, SIGCHLD, SIGINT, SIGHUP, SIGTSTP, SIGQUIT). I thought SIGWINCH and SIGHUP were sent by the terminal emulator but they aren’t.

Two kinds of terminal modes: termios and ANSI Nov 11 2024

I learned recently that the terminal has two different kinds of modes that can be set:

1. “termios” modes, which

• are managed by the operating system
• you can view with stty -a
• are set with the ioctl system call

2. “ANSI” modes (not sure if this is the right name), which

• are managed by the terminal emulator
• you can’t normally view, except in GhostTTY’s inspector
• are set by sending ANSI escape codes (like "\033[?25l" for “make cursor invisible”)

litecli: a fancier SQLite REPL Nov 7 2024

Just discovered litecli, it feels like a big improvement over the default SQLite REPL. (autocompletion! support for multiline queries! syntax highlighting!)

Apparently there’s also mycli and pgcli by the same author for MySQL and Postgres.

Programs can make your cursor disappear Nov 6 2024

Yesterday a program made my terminal’s cursor disappear! I was really surprised because programs so rarely mess up my terminal these days.

Normally my prompt looks like this:

But after running echo -e "\033[?25l" (the escape code for “make cursor invisible”), the cursor is gone, like this.

I just dealt with it by opening a new terminal tab, but you can also run reset or echo -e "\033[?25h". I don’t really like using reset because it sleeps for 1 second and I find it kind of annoying, I wish it ran instantly.

Finding when a bug was fixed with git bisect Nov 6 2024

I don’t have much reason to use git bisect usually, but today I noticed that a bug in an open source project had been fixed and I wanted to figure out when.

The main gotcha was that git bisect usually assumes that you’re trying to figure out when a bug was introduced (not fixed), so I had to reverse everything.

step 1: Write a script test.sh that fails if the bug is fixed (so I had to reverse the exit codes):

pip3 install .
if unbuffer sqlite-utils insert test.db test --csv test.csv | grep -o '100.......' | grep 25h
then
    # this means the bug is fixed, so fail (reverse the output)
    exit 1
else
    exit 0
fi

step 2: run git bisect

git bisect start
# we have to mark the current state as "bad" even though it's technically the
# old state that was "bad"
git bisect bad main
git bisect good 0d45ee11
git bisect run test.sh

It worked perfectly and after 10 seconds gave me the exact commit that had fixed the bug.

Python inline dependencies Oct 23 2024

Learned via Jacob Kaplan-Moss today that now Python has a way to specify dependencies inline in scripts?! For example I can create this script:

# /// script
# requires-python = ">=3.9"
# dependencies = [
#   "requests<3",
# ]
# ///
import requests
requests.get("http://example.com")

and then run it with uv run blah.py. I haven’t totally understood what’s going on with uv yet but it seems interesting. The syntax is documented here, some notes on uv from @simonw here

Setting secrets with the Github CLI is great Oct 23 2024

I’ve never used the GitHub gh CLI at all really but I learned recently that you can use it to set secrets to use them in GitHub Actions. SO much easier than setting them from the web interface.

gh secret set FLY_API_TOKEN

Also gh auth token is a great way to get a token to authenticate to the GitHub API, like this:

export GITHUB_TOKEN=$(gh auth token)
curl -H "Authorization: Bearer $GITHUB_TOKEN" \
        https://api.github.com/repos/gohugoio/hugo/releases/tags/v0.136.0 | jq -r '.body'

a Go middleware to recover from panics Oct 23 2024

I’ve been reading Let’s Go by Alex Edwards and one of my favourite tips from the book is to write a HTTP middleware to recover from panics, something like this:

func (app *application) recoverPanic(next http.Handler) http.Handler {
  return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
    defer func() {
      if err := recover(); err != nil {
        w.Header().Set("Connection", "close")
        fmt.Println(err)
        debug.PrintStack() // from "runtime/debug"
        app.serverError(w, fmt.Errorf("%s", err))
      }
    }()
    next.ServeHTTP(w, r)
  })
}

Previously my webserver would just crash if there was a panic (not even returning an error!) and I think this is better.

Restic for backing up SQLite dbs Oct 22 2024

One of the missing pieces about databases for me has been what to use for backups. I recently decided to go with restic and it seems to be working okay so far.

My backup script looks somethnig like this. It pings healthchecks.io when it’s done.

sqlite3 /data/requests.sqlite "VACUUM INTO '/tmp/requests.sqlite'"
gzip /tmp/requests.sqlite

restic -r s3://s3.amazonaws.com/MY_BUCKET/ backup /tmp/requests.sqlite.gz 
restic -r s3://s3.amazonaws.com/MY_BUCKET/ forget -l 7 -H 12 -d 2 -w 2 -m 2 -y 2
restic -r s3://s3.amazonaws.com/MY_BUCKET/ prune
curl https://hc-ping.com/ID

dbmate for SQL migrations with SQLite Oct 22 2024

I’ve been making a web application where I haven’t ironed out my database schema yet so I wanted a migration tool to tweak it as I change my mind.

Someone on Mastodon recommended dbmate recently and it’s been working well. Here’s how I use it:

1. Run dbmate new my_migration
2. Edit the migration file to
3. Run dbmate up to apply it

Then I can run dbmate up in production too. I’ve only been doing “up” migrations, no “down” migrations.

fs_usage can trace filesystem events on Mac OS Oct 21 2024

The thing I miss from Linux while using a Mac is strace. Recently I found out aboutfs_usage that does part of what strace does and I was SO happy.

(I’ve also tried dtruss/dtrace etc and a couple of other things which I have failed to get to work, I’m not willing to turn off SIP)

For example here’s me looking at what config files fish opens:

(for some reason I can’t grep fs_usage’s output directly, not sure why)

how to use semantic HTML Oct 17 2024

I’ve been reading advice to use semantic HTML for years but never really found an explanation that clicked. This article by Heydon Pickering really helped me understand how to do it in practice, specifically this “owl selector” tip for how to make everything evenly spaced:

.stack > * + * {
  margin-top: 1rem;
}

I also found his website’s CSS a really interesting read, especially how most of the selectors are for base HTML tags and classes but there’s occasionally something like this:

.margin-top\:0 {
  margin-top: 0;
}

screenshots in Markdown in vim Oct 4 2024

I put a lot of screenshots in my blog posts and it’s always a little annoying to take the screenshot, put it in the right folder, and insert the <img> tag.

So I finally wrote the world’s smallest vim plugin which lets me just run :Screenshot NAME and it’ll save the image from my clipboard to the right place and write the <img> tag. It uses pngpaste.

diffdiff: a great diff tool Sep 27 2024

diffdiff.net is a really nice side by side online diff tool, by Joe Kaptur. I’ve actually known about it for a long time but I’m always forgetting what it’s called.

Here’s what it looks like diffing some tcpdump output – used this to see that the MSS in two SYN packets didn’t match.

unbuffer Aug 27 2024

Sometimes I run some_program | grep blah, but libc buffers the output because the program isn’t writing to a terminal (in libc, writing to a terminal is line-buffered but writing to a pipe/file isn’t). So the grep doesn’t work. It’s annoying and I’ve never known what to do about it.

I finally found a solution to this thanks to folks on Mastodon: unbuffer from the expect package. unbuffer program will open a new TTY, set the program’s stdout to that tty, and then write to the pipe itself and flush the output.

You can also use unbuffer to trick the program into thinking it’s writing to a TTY for some other reason.