FEED Validator

for Atom and RSS and KML

Congratulations!

This is a valid Atom 1.0 feed.

Recommendations

This feed is valid, but interoperability with the widest range of feed readers could be improved by implementing the following recommendations.

line 9, column 70: Self reference doesn't match document location [help]

 <link rel='self' href='https://www.tbray.org/ongoing/ongoing.atom' />
                                                                      ^

line 61, column 4: p should not contain aria-label attribute [help]

    <svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1 ...
    ^

line 61, column 4: p should not contain role attribute [help]

    <svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1 ...
    ^

line 67, column 4: a should not contain clipPath tag [help]
```
    <clipPath id="r">
    ^
```
line 70, column 4: a should not contain clip-path attribute [help]
```
    <g clip-path="url(#r)">
    ^
```

line 75, column 4: a should not contain text-rendering attribute [help]

    <g fill="#fff" text-anchor="middle" font-family="Verdana,Geneva,DejaVu S ...
    ^

line 76, column 6: g should not contain aria-hidden attribute (2 occurrences) [help]

      <text aria-hidden="true" x="115" y="150" fill="#010101" fill-opacity=" ...
      ^

line 76, column 6: g should not contain textLength attribute (4 occurrences) [help]

      <text aria-hidden="true" x="115" y="150" fill="#010101" fill-opacity=" ...
      ^

Source: http://www.tbray.org/ongoing/ongoing.atom

<?xml version='1.0' encoding='UTF-8'?>
<feed xmlns='http://www.w3.org/2005/Atom'
xmlns:thr='http://purl.org/syndication/thread/1.0'
xml:lang='en-us'>
<title>ongoing by Tim Bray</title>
<link rel='hub' href='http://pubsubhubbub.appspot.com/' />
<id>https://www.tbray.org/ongoing/</id>
<link href='https://www.tbray.org/ongoing/' />
<link rel='self' href='https://www.tbray.org/ongoing/ongoing.atom' />
<link rel='replies' thr:count='101' href='https://www.tbray.org/ongoing/comments.atom' />
<logo>rsslogo.jpg</logo>
<icon>/favicon.ico</icon>
<updated>2024-09-06T14:44:04-07:00</updated>
<author><name>Tim Bray</name></author>
<subtitle>ongoing fragmented essay by Tim Bray</subtitle>
<rights>All content written by Tim Bray and photos by Tim Bray Copyright Tim Bray, some rights reserved, see /ongoing/misc/Copyright</rights>
<generator uri='/misc/Colophon'>Generated from XML source code using Perl, Expat, Emacs, Mysql, Ruby, Java, and ImageMagick. Industrial-strength technology, baby.</generator>
<entry>
<title>0 dependencies!</title>
<link href='https://www.tbray.org/ongoing/When/202x/2024/09/04/0dependencies' />
<link rel='replies' thr:count='7' type='application/xhtml+xml' href='/ongoing/When/202x/2024/09/04/0dependencies#comments' />
<id>https://www.tbray.org/ongoing/When/202x/2024/09/04/0dependencies</id>
<published>2024-09-04T12:00:00-07:00</published>
<updated>2024-09-05T19:22:39-07:00</updated>
<category scheme='https://www.tbray.org/ongoing/What/' term='Technology/Software' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Technology' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Software' />
<summary type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>Here’s a tiny little done-in-a-couple-hours project consisting of a single static Web page and a cute little badge you can slap on your GitHub project</div></summary>
<content type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>
Here’s a tiny little done-in-a-couple-hours project consisting of a single static Web page and a cute little badge you
can slap on your GitHub project.
<img src="https://www.tbray.org/ongoing/When/202x/2024/09/04/0dep.png" alt="0 dependencies!" />
The Web site is at
<a href="https://0dependencies.dev">0dependencies.dev</a>. The badge is visible on my current open-source projects, for example
check out
<a href="https://github.com/timbray/topfew">Topfew</a> (you have to scroll down a bit).
<h2 id='p-3'>Zero, you say?</h2>
In recent months I keep seeing these eruptions of geek angst about the fulminating masses of dependencies squirming
under the surface of just about any software anyone uses for anything. The most recent, and what precipitated this, was Mike
Perham’s
<a href="https://www.mikeperham.com/2016/02/09/kill-your-dependencies/">Kill Your Dependencies</a>.
It’s not just that dependencies are a fertile field for CVEs
(*cough* xz *cough*) and tech debt, they’re also an enemy of predictable performance.
Also, they’re unavoidable. When you take a dependency, often you’re standing on the shoulders of giants. (Unfortunately,
sometimes you’re standing in the shoes of clowns.) Software is accretive and it’s a good thing that that’s OK because it’s also
inevitable.
In particular, don’t write your own crypto, etc. Because in software, as in life, you’re gonna have to take some
dependencies. But… how about we take less? And how about, sometimes
we strive for zero?
<h2 id='p-4'>The lower you go</h2>
… the closer you are to zero. So, suppose you’re writing library code. Consider these
criteria:
<ul>
<li>It’s low-level, might be useful to a lot of apps aimed at entirely different goals.</li>
<li>Good performance is important. Actually, let me revise that: predictably good performance is
important.</li>
<li>Security is important.</li>
</ul>
If you touch all three of these bases, I respectfully suggest that you try to earn this badge:
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="110" height="20" role="img" aria-label="⓿: dependencies!">
<a target="_blank" xlink:href="https://0dependencies.dev">
<linearGradient id="s" x2="0" y2="100%">
<stop offset="0" stop-color="#bbb" stop-opacity=".1"/>
<stop offset="1" stop-opacity=".1"/>
</linearGradient>
<clipPath id="r">
<rect width="110" height="20" rx="3" fill="#fff"/>
</clipPath>
<g clip-path="url(#r)">
<rect width="21" height="20" fill="#555"/>
<rect x="21" width="89" height="20" fill="#d1331b"/>
<rect width="110" height="20" fill="url(#s)"/>
</g>
<g fill="#fff" text-anchor="middle" font-family="Verdana,Geneva,DejaVu Sans,sans-serif" text-rendering="geometricPrecision" font-size="110">
<text aria-hidden="true" x="115" y="150" fill="#010101" fill-opacity=".3" transform="scale(.1)" textLength="110">⓿</text>
<text x="115" y="140" transform="scale(.1)" fill="#fff" textLength="110">⓿</text>
<text aria-hidden="true" x="645" y="150" fill="#010101" fill-opacity=".3" transform="scale(.1)" textLength="790">dependencies!</text>
<text x="645" y="140" transform="scale(.1)" fill="#fff" textLength="790">dependencies!</text>
</g>
</a>
</svg> (By the way, it’s cool that I can toss a chunk of SVG into my HTML and it Just Works. And, you can click on it.)
<h2 id='p-5'>How to?</h2>
First, whatever programming language you’re in, try to stay within the bounds of what comes with the language. In Go, where I
live these days, that means your <code>go.sum</code> file is empty. Good for you!
Second, be aggressive. For example, Go’s JSON support is known to be kind of slow and memory-hungry. That’s OK
because there are better open-source options. For
<a href="https://github.com/timbray/quamina">Quamina</a>, I rejected the alternatives and wrote
<a href="https://github.com/timbray/quamina/blob/main/flatten_json.go">my own JSON parser</a> for the hot code path.
Which, to be honest, is a pretty low bar: JSON’s grammar could be
<a href="https://en.wikipedia.org/wiki/Rice_writing">inscribed on a grain of rice</a>, or you can rely on Doug Crockford’s
<a href="https://www.json.org/json-en.html">JSON.org</a>.
So, get your dependencies to zero and display the badge proudly. Or if you can’t, think about each of your
dependencies. Does each of them add enough value, compared to you writing the code yourself? In particular, taking a dependency
on a huge general-purpose library for one small simple function is an antipattern.
<h2 id='p-6'>What are you going to do, Tim?</h2>
I’m not trying to start a movement or anything. I just made a badge, a one-page website, and a blog post.
If I were fanatically dedicated, <code>0dependencies.dev</code> would be database-backed with a React front-end and multiple
Kubernetes pods, to track bearers of the badge. Uh, no.
But, I’ll keep my eyes open. And if any particularly visible projects that you know about want to claim the badge, let me know
and maybe I’ll start a 0dependency hall of fame.
</div></content></entry>
<entry>
<title>Q Numbers Redux</title>
<link href='https://www.tbray.org/ongoing/When/202x/2024/08/28/Q-Numbers-2' />
<link rel='replies' thr:count='0' type='application/xhtml+xml' href='/ongoing/When/202x/2024/08/28/Q-Numbers-2#comments' />
<id>https://www.tbray.org/ongoing/When/202x/2024/08/28/Q-Numbers-2</id>
<published>2024-08-28T12:00:00-07:00</published>
<updated>2024-09-01T08:26:43-07:00</updated>
<category scheme='https://www.tbray.org/ongoing/What/' term='Technology/Quamina Diary' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Technology' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Quamina Diary' />
<summary type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>Back in July I wrote about <a href='/ongoing/When/202x/2024/07/09/Q-Numbers'>Q numbers</a>, which make it possible to compare numeric values using a finite automaton. It represented a subset of numbers as 14-hex-digit strings. In a remarkable instance of BDD (Blog-Driven Development, obviously) Arne Hormann and Axel Wagner figured out a way to represent all 64-bit floats in at most ten bytes of UTF-8 and often fewer. This feels nearly miraculous to me; read on for heroic bit-twiddling</div></summary>
<content type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>
Back in July I wrote about
<a href="/ongoing/When/202x/2024/07/09/Q-Numbers">Q numbers</a>, which make it possible to compare numeric values using a finite
automaton. It represented a subset of numbers as 14-hex-digit strings. In a
remarkable instance of BDD (Blog-Driven Development, obviously) Arne Hormann and Axel Wagner figured out a way to represent
all 64-bit floats in at most ten bytes of UTF-8 and often fewer. This feels nearly miraculous to
me; read on for heroic bit-twiddling.
<h2 id='p-1'>Numbits</h2>
Arne Hormann worked out how to rearrange the sign, exponent and mantissa that make up a float’s 64 bits into a
big-endian integer that you probably couldn’t do math with but you can compare for equality and ordering. Turn that into sixteen
hex digits and you’ve got automaton fuel which covers all the floats at the cost of being a little bigger.
If you want to admire Arne’s awesome bit-twiddling skills, look at
<a href="https://github.com/timbray/quamina/blob/main/numbits.go">numbits.go</a>. He explained to me how it works in some chat
that I can’t find, and to be honest I can’t quite look at this and remember the explanation.
<pre><code> u := math.Float64bits(f)
// transform without branching
// if high bit is 0, xor with sign bit 1 << 63,
// else negate (xor with ^0)
mask := (u>>63)*^uint64(0) | (1 << 63)
return numbits(u ^ mask)</code></pre>
[Update: Arne wrote it up! See
<a href="/ongoing/When/202x/2024/08/31/QNum-Redux-Explained">Q Numbers Redux Explained</a>.]
Even when I was puzzled, I wasn’t worried because the unit tests are good; it
works.
Arne called these “numbits” and wrote a nice complete API for them, although Quamina just needs <code>.fromFloat64()</code>
and <code>.toUTF8()</code>.
I and Arne both thought he’d invented this, but then he discovered that the same trick was being used in the DB2 on-disk data
format years and years ago. Still, damn clever, and I’ve urged him to make numbits into a standalone library.
<h2 id='p-2'>We want less!</h2>
We care about size; Among other things, the time an automaton takes to match a value is linear (sometimes worse) in its
length.
So the growth from 14 to 16 bytes made us unhappy. But, no problemo! Axel Wagner pointed out that if you use base-128, you
can squeeze those 64 bits into ten usable bytes of UTF-8. So now we’re shorter than the previous iteration of Q numbers
while handling all the float64 values…
But wait, there’s more! Arne noticed that for purposes of equality and comparison, trailing zeroes (<code>0x0</code>, not
<code>‘0’</code>) in those 10-byte strings
are entirely insignificant and can just be discarded. The final digit only has 1/128 chance of being zero, so maybe no big
deal. But it turns out that you do get dramatic trailing-0 sequences in positive integers, especially small ones,
which in my experience are the kinds of numbers you most often want to match. Here’s a chart of the length of the lengths the of
numbits-based Q numbers for the integers zero through 100,000 inclusive.
<table>
<tr align="center"><th>Length</th><th>Count</th></tr>
<tr align="center"><td>1</td><td class="count">1</td></tr>
<tr align="center"><td>2</td><td>1</td></tr>
<tr align="center"><td>3</td><td>115</td></tr>
<tr align="center"><td>4</td><td>7590</td></tr>
<tr align="center"><td>5</td><td>92294</td></tr>
</table>
They’re all shorter than 5 until you get to 1,000.
Unfortunately, none of my benchmarks prove any performance increase because they focus on corner cases and extreme numbers;
the benefits here are to the world’s most boring numbers, namely small non-negative integers.
Here I am, well past retirement age, still getting my jollies from open-source bit-banging. I hope other people manage to
preserve their professional passions into later life.
</div></content></entry>
<entry>
<title>Q Numbers Redux Explained</title>
<link href='https://www.tbray.org/ongoing/When/202x/2024/08/31/QNum-Redux-Explained' />
<link rel='replies' thr:count='0' type='application/xhtml+xml' href='/ongoing/When/202x/2024/08/31/QNum-Redux-Explained#comments' />
<id>https://www.tbray.org/ongoing/When/202x/2024/08/31/QNum-Redux-Explained</id>
<published>2024-08-31T12:00:00-07:00</published>
<updated>2024-09-01T08:18:25-07:00</updated>
<category scheme='https://www.tbray.org/ongoing/What/' term='Technology/Quamina Diary' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Technology' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Quamina Diary' />
<summary type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>[The first guest post in some years. Welcome, Arne Hormann!]</div></summary>
<content type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>
[The first guest post in some years. Welcome, Arne Hormann!]
Hi there, I'm
<a href="https://github.com/arnehormann">Arne</a>. In July, I stumbled on a
<a href="https://lobste.rs/s/6whqcn/q_numbers">lobste.rs link</a> (thanks, Carlana) that led me to
<a href="/ongoing/When/202x/2024/07/09/Q-Numbers">Tim’s article on Q Numbers</a>.
As I'm regularly working with both floating point numbers and Go, the post made me curious — I was
sure I could improve the compression scheme. I posted comments. Tim liked my idea and challenged me to create a PR.
I did the encoding but felt overwhelmed with the integration. Tim took over and merged it. And since
<a href="https://github.com/timbray/quamina/releases/tag/v1.4.0">v1.4.0 released in August
28th</a>, it's in Quamina.
In Tim’s
<a href="/ongoing/When/202x/2024/08/28/Q-Numbers-2">post about that change</a>, he wrote “Arne explained to me how it works in some
chat that I can’t find, and to be honest I can’t quite look at this and remember the explanation”. Well, you're reading it right
now.
<h2 id='p-1'>Float64</h2>
Let's first talk about the data we are dealing with here. Quamina operates on JSON. While no JSON specification limits
numbers to
<a href="https://en.wikipedia.org/wiki/Double-precision_floating-point_format">IEEE 754 floating point</a> (Go calls
them <code>float64</code>),
<a href="https://datatracker.ietf.org/doc/html/rfc8259#section-6">RFC 8259 recommends</a> treating them as such.
They look like this:
<ul>
<li>1 bit sign; 0 is positive, 1 is negative.</li>
<li>11 bits exponent; It uses a bias of <code>1023</code> (<code>(1<<10)-1</code>), and all exponent bits set means Infinity or Not a Number
(<code>NaN</code>)</li>
<li>1 mantissa high bit (implicit, never stored: <code>0</code> if all exponent bits are <code>0</code>, otherwise <code>1</code>). This is required so
there's exactly one way to store each number.</li>
<li>52 explicit mantissa bits. The 53 mantissa bits are the binary digits of the number itself.</li>
</ul>
Both <code>0</code> and <code>-0</code> exist and are equal but are represented differently. We have to normalize <code>-0</code> to <code>0</code> for comparability,
but according to Tim’s tests, <code>-0</code> cannot occur in Quamina because JSON decoding silently converts <code>-0</code> to <code>0</code>.
With an exponent with all bits set, Infinity has a mantissa of <code>0</code>. Any other mantissa is NaN. But both sign values are
used. There are a lot of different NaNs; <code>1<<53 - 2</code> different ones!
In JSON, Infinity and NaN are not representable as numbers, so we don't have to concern ourselves with them.
Finally, keep in mind that these are binary numbers, not decimals. Decimal <code>0.1</code> cannot be accurately encoded. But <code>0.5</code>
can. And all integers up to <code>1<<53</code>, too.
<h2 id='p-2'>Adding numbers to Quamina</h2>
Quamina operates on UTF-8 strings and compares them byte by byte. To add numbers to it, they have to be bytewise-comparable
and all bytes have to be valid in UTF-8.
Given these constraints, let's consider the problem of comparability, first.
<h2 id='p-3'>Sortable bits</h2>
We can use <code>math.Float64bits()</code> to convert a <code>float64</code> into its individual bits (stored as <code>uint64</code>).
Positive numbers are already perfectly sorted. But they are smaller than negative numbers. To fix that, we always flip the
sign bit so it's <code>1</code> for positive and <code>0</code> for negative values.
Negative values are sorted exactly the wrong way around. To totally reverse their order, we have to flip all exponent and
mantissa bits in addition to the sign bits.
A simple implementation would look like:
<pre><code>
func numbitsFromFloat64(f float64) numbits {
u := math.Float64bits(f)
if f < 0 {
return numbits(^u)
}
return numbits(u) | (1 << 63)
}
</code></pre>
Now let's look at the actual code:
<pre><code>
func numbitsFromFloat64(f float64) numbits {
u := math.Float64bits(f)
mask := (u>>63)*^uint64(0) | (1 << 63)
return numbits(u ^ mask)
}</code></pre>
The <code>mask</code> line can be a bit of a headscratcher...
<ol>
<li><code>u>>63</code> moves the sign bit of the number to the lowest bit</li>
<li>the result will be <code>0</code> for positive values and <code>1</code> for negative values</li>
<li>that's multiplied with <code>^0</code> - all 64 bits set to true</li>
<li>we now have <code>0</code> for positive numbers and all bits set for negative numbers</li>
<li>regardless, always set the sign bit with <code>| (1 << 63)</code></li>
</ol>
By xoring <code>mask</code> to the original bits, we get our transformation for lexically ordered numbers.
The code is a bit hard to parse because it avoids branches for performance reasons.
<h2 id='p-4'>Sortable bytes</h2>
If we were to use the uint64 now, it would work out great. But it has to be split into its individual bytes. And on little
endian systems - most of the frequently used ones - the byte order will be wrong and has to be reversed. That's done by storing
the bytes in big endian encoding.
<h2 id='p-5'>UTF-8 bytes</h2>
Our next problem is the required UTF-8 encoding. Axel suggested to only use the lower 7 bits. That means we need up to 10 byte instead of 8 (<code>7*10 = 70</code> fits <code>8*8 = 64</code>).
<h2 id='p-6'>Compress</h2>
The final insight was that trailing <code>0</code> bytes do not influence the comparison at all and can be dropped. Which compresses
positive power-of-two numbers and integers even further. Not negative values, though - the required bit inversion messes it
up.
And that's all on this topic concerning Quamina — even if there's so, so much more about floats.
</div></content></entry>
<entry>
<title>Mozart Requiem</title>
<link href='https://www.tbray.org/ongoing/When/202x/2024/08/25/Mozart-Requiem' />
<link rel='replies' thr:count='4' type='application/xhtml+xml' href='/ongoing/When/202x/2024/08/25/Mozart-Requiem#comments' />
<id>https://www.tbray.org/ongoing/When/202x/2024/08/25/Mozart-Requiem</id>
<published>2024-08-25T12:00:00-07:00</published>
<updated>2024-08-25T16:37:08-07:00</updated>
<category scheme='https://www.tbray.org/ongoing/What/' term='Arts/Music' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Arts' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Music' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Technology/Audio' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Technology' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Audio' />
<summary type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>Vancouver has many choirs, with differing proficiency levels and repertoire choices. Most gather fall-to-spring and take the summer off. Thus, <a href='https://www.facebook.com/SummerChor/'>Summerchor</a>, which aggregates a couple of hundred singers from many choirs to tackle one of the Really Big choral pieces each August. This year it was the <a href='https://en.wikipedia.org/wiki/Requiem_(Mozart)'>Mozart Requiem</a>. Mozart died while writing this work and there are many “completions” by other composers. Consider just <a href='https://en.wikipedia.org/wiki/Modern_completions_of_Mozart%27s_Requiem'>the Modern completions</a>; this performance was of <a href='https://en.wikipedia.org/wiki/Modern_completions_of_Mozart%27s_Requiem'>Robert Levin</a>’s</div></summary>
<content type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>
Vancouver has many choirs, with differing proficiency levels and repertoire
choices. Most gather fall-to-spring and take the summer off. Thus,
<a href="https://www.facebook.com/SummerChor/">Summerchor</a>, which aggregates a couple of hundred singers from many choirs to
tackle one of the Really Big choral pieces each August. This year it was the
<a href="https://en.wikipedia.org/wiki/Requiem_(Mozart)">Mozart Requiem</a>. Mozart died while writing this work and there are
many “completions” by other composers. Consider just
<a href="https://en.wikipedia.org/wiki/Modern_completions_of_Mozart%27s_Requiem">the Modern completions</a>; this performance
was of
<a href="https://en.wikipedia.org/wiki/Modern_completions_of_Mozart%27s_Requiem">Robert Levin</a>’s.
<img src="https://www.tbray.org/ongoing/When/202x/2024/08/25/PXL_20240825_020725834.png" alt="Mozart’s Requiem performed in St. Andrews-Wesley" />
<div class='caption'>Summerchor performs Mozart’s <cite>Requiem</cite> in 
<a href="https://standrewswesley.com">St.Andrews-Wesley United Church</a>, August 24, 2024.</div>
The 200 singers were assisted by four soloists, a piano, a trombone, the church’s excellent pipe organ, and finally, of
course, by the towering arched space.
The combined power of Mozart’s music, the force of massed voices, and the loveliness of the great room yielded a
entirely overwhelming torrent of beauty. I felt like my whole body was being squeezed.
Obviously, in an hour-long work, some parts are stronger than others. For me, the opening Requiem Aeternum and
Kyrie hit hard, then the absolutely wonderful ascending line opening the Domine Jesu totally crushed me.
But for every one of the three-thousand-plus seconds, I was left in no doubt that I was experiencing about as much beauty as
a human being can.
<h2 id='p-1'>God?</h2>
We were in a house of worship. The words we were listening to were liturgical, excerpted from Scripture. What, then, of the
fact that
<a href="/ongoing/When/202x/2022/08/24/On-Faith">I am actively hostile</a> to religion? Yeah, I freely acknowledge that all
this beauty I’m soaking up is founded on faith. Other people’s faith, of other times. The proportion of people who profess that
(or any) faith is monotonically declining, maybe not everywhere, but certainly where I live.
Should I feel sad about that?
Not really; The fact that architects and musicians worked for the Church is related to the fact that the Church was willing to
pay. Musicians can manage without God, generally, as long as they’re getting paid.
<h2 id='p-2'>The sound</h2>
We’ve been going out to concerts quite a lot recently so of course I’ve been
<a href="/ongoing/What/Arts/Music/">writing about it</a>, and usually discussing the sound quality too.
The sound at the <cite>Requiem</cite> was beyond awesome. If you look at the picture above you can see there’s a soundboard
and a guy sitting at it, but I’m pretty sure the only boost was on the piano, which had to compete with 200 singers and the
organ. So, this was the usual classical-music scenario: If you want dynamic range, or to hear soloists, or to blend parts, you
do that with musical skill and human throats and fingers and a whole lot of practice. There’s no knob to twirl.
I mean,
<a href="/ongoing/When/202x/2024/05/26/TTB">I love well-executed electric sound</a>, but large-scale classical, done well, is on
a whole other level.
Above, I mentioned the rising line at the opening of the <cite>Domine Jesu</cite>; the pulses, and the space between them,
rose up in the endless vertical space as they rose up the scale, and yet were clearly clipped at start and end, because you
don’t get that much reverb
when the pews are full of soft human flesh and the roof is made of old wood, no matter how big the church is. I just don’t have
words for how wonderful it sounded.
<h2 id='p-3'>Classical?</h2>
Obviously, this is what is conventionally called “classical” music. But I’m getting a little less comfortable with that term,
especially the connotation that it’s “music for old people” (even though I am one of those). Because so is
<a href="/ongoing/When/202x/2024/05/26/TTB">rootsy rock music</a> and
<a href="/ongoing/When/202x/2024/08/11/Countrywomen">bluegrass and Americana</a> and
<a href="/ongoing/When/202x/2024/06/23/Lounge-Penguin">GoGo Pengin</a> and
<a href="/ongoing/When/202x/2023/10/17/Rock-Tech">Guns N’ Roses</a>.
</div></content></entry>
<entry>
<title>2024 Pollscrolling</title>
<link href='https://www.tbray.org/ongoing/When/202x/2024/08/20/2024-Election-Junky' />
<link rel='replies' thr:count='2' type='application/xhtml+xml' href='/ongoing/When/202x/2024/08/20/2024-Election-Junky#comments' />
<id>https://www.tbray.org/ongoing/When/202x/2024/08/20/2024-Election-Junky</id>
<published>2024-08-20T12:00:00-07:00</published>
<updated>2024-08-20T17:07:32-07:00</updated>
<category scheme='https://www.tbray.org/ongoing/What/' term='The World/Politics' />
<category scheme='https://www.tbray.org/ongoing/What/' term='The World' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Politics' />
<summary type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>The 2024 US election has, in the last few weeks, become the most interesting one I can recall. I’m pretty old, so that’s a strong statement; I can recall a lot of US elections. The Internet makes it way too easy to obsess over a story that’s this big and has this many people sharing opinions. Here is my opinion, not on who’s winning, but on how, with only a very moderate expenditure of time and money, you can be as well-informed as anybody in the world as to how it’s going</div></summary>
<content type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>
The 2024 US election has, in the last few weeks, become the most interesting one I can recall. I’m pretty old, so that’s a
strong statement; I can recall
a lot of US elections. The Internet makes it way too easy to obsess over a story that’s this big and has this many
people sharing opinions. Here is my opinion, not on who’s winning, but on how, with only a very moderate expenditure of time and
money, you can be as well-informed as anybody in the world as to how it’s going.
Disclosures: I’m not American, got no vote on this one. Am left of the US Democratic party, but at this point they represent
an infinitely better option for America than does anything connected to Donald Trump. Thus, the following remarks should be
assumed to have a strong Harris/Walz bias.
I claim that using the following sites should, in 5-15 minutes a day, give you an understanding of the state of the
race that is very close to that of the big-name prognosticators writing in the big-name publications.
<h2 id='p-1'>Polls generally</h2>
They’ve been wrong a lot recently. The industry is still trying to complete the transition off landlines, which few people
have any more; those who still do are an entirely unrepresentative sample. I’ve seen a few smart people guessing that pollers
may be getting better, this cycle. We’ll see, won’t we?
<h2 id='p-2'>Prognosticators generally</h2>
They suck. The arm-waving, gut-feel bullshit, and undisclosed bias is disgusting. In the pages of the really big
properties like the NYT and WaPo, the opinion columnists are mostly partisan hacks looking for reasons to explain why their
favored party is doing just fine and the other is failing. I’ve pretty well given up reading them.
OK, now let’s get to our sources, in no particular order.
<img src="https://www.tbray.org/ongoing/When/202x/2024/08/20/TPM.png" alt="Talking Points Memo" />
<h2 id='p-3'>Talking Points Memo</h2>
<a href="https://talkingpointsmemo.com">TPM</a>
is the home of Josh Marshall, the ur-blogger on US Politics, and still as good as anyone.
He’s hired a bunch of other clear and clear-eyed writers, who obsess about elections all day every
day. They have a strong and acknowledged pro-Democratic and anti-Trump bias, but in my opinion don’t let it
clutter their analyses. If you’re reading the polling sites
that I’m recommending below, <cite>TPM</cite> will have smart interpretive pieces about what they might mean.
A lot of their stuff is free, but the best isn’t. Unfortunately, they only offer annual subscriptions at US$70; I’ve
subscribed for many years. It depends on how fierce the election monkey on your back is, but in my
personal experience, there is no other site that offers more depth and clarity on US politics. Maybe worth your money for one
year, this year.
<img src="https://www.tbray.org/ongoing/When/202x/2024/08/20/Silver.png" alt="Silver Bulletin" />
<h2 id='p-4'>Nate Silver</h2>
The former <cite>538</cite> guy is now at the
<a href="https://www.natesilver.net">Silver Bulletin</a>. Nate is not everyone’s cup of tea, but he retained the IP rights to
the 538 election model, which is updated on a daily basis. He admits to being moderately pro-Harris this time around but argues
convincingly that he doesn’t let this cloud his analysis, which I generally find pretty cogent.
You probably need a little bit of statistical literacy to fully appreciate this stuff.
As of today, August 20, 2024, his probability-of-win numbers are Harris 53.6%, Trump 45.7%.
The <cite>Silver Bulletin</cite> has some free stuff, but the best parts, including the model updates, are paywalled. The price is
currently $14/month, but on September 1st they’re going up to $20, just until the election is over. Because a lot of people like
me signed up with every expectation of unsubscribing in three months.
If you happen to care about professional sports there’s lots of brain candy there too.
<img src="https://www.tbray.org/ongoing/When/202x/2024/08/20/538.png" alt="538 Poll Report" />
<h2 id='p-5'>538</h2>
Their
<a href="https://projects.fivethirtyeight.com/polls/">Latest Polls</a> page is now part of ABC news but seems to retain many of
the virtues that made 538 the flavor-of-the-month a few years ago.
The crucial thing, if you’re visiting once a day, is to find the “Sort by Date” widget and click on “Added”, which brings the
most recent stuff that you probably haven’t seen yet to the top.
As I write this, their national polling average has Harris 46.6%, Trump 43.8%. This is quite different from Silver’s
“probability of winning”. 538’s main virtue is that they get the polls up about as fast as anyone else. It’s free.
<img src="https://www.tbray.org/ongoing/When/202x/2024/08/20/RCP.png" alt="RealClearPolitics" />
<h2 id='p-6'>RCP</h2>
I kind of hate to mention <a href="https://www.realclearpolitics.com">RealClearPolitics</a> because it is at least in part a
hive of filthy MAGA-friendly screamers. Their front page is all links, MAGA-dominated but including a
sprinkling of analysis from more even-handed or openly-progressive sources. Anyhow, the problem with that stuff isn’t the bias,
it’s the fact that they’re just a bunch of low-value prognosticators. I wouldn’t waste much time on the front page, I’d start by
clicking on “Polls”, near the top left.
Their aggregation of poll results will contain about what you’ll see at 538 (sometimes important polls get to one place
first, sometimes the other). But RCP offers other useful resources. There is the
<a href="https://www.realclearpolitics.com/rcp-pollster-scorecard/">RCP Pollster Scorecard</a>, which offers data on the
accuracy and bias of most of the pollers whose results they report. Since some of those pollers are super extra biased, this can
be a useful sanity check.
What I really like is the
<a href="https://www.realclearpolling.com/maps/president/2024/no-toss-up/electoral-college">Electoral College Map</a>, which as
I write is predicting a Trump victory, 287-251 in EC votes. You can click on each state and see the polls they used to compute
their prediction for that state.
I think their MAGA bias shows in the predictions, but that’s OK, because there’s a “Create Your Own Map” link, where
you can disagree with them and explore each side’s path to victory or defeat. Looking at today’s map, my conclusion is that if
Harris can flip Pennsylvania from red to blue she probably wins, and if she can bring along either or both of Arizona and North
Carolina, Trump is roadkill.
<img src="https://www.tbray.org/ongoing/When/202x/2024/08/20/CNN.png" alt="CNN" />
<h2 id='p-7'>CNN</h2>
No, really. It’s
<a href="https://www.cnn.com">cheesy and overhyped</a> but feels to me like it’s speaking to a pretty big constituency that I
don’t know anybody from, there’s a bit of zeitgeist in the flow. To my eye it leans a little more Dem than GOP (that’s a
surprise) and is not actually terrible.
<h2 id='p-8'>When?</h2>
My advice is to wait until at least mid-afternoon, when the polls of the day have been published and ingested, then put in
your pollscrolling time. Won’t take too long and you’ll know what the allegedly-smart people know.
</div></content></entry>
<entry>
<title>Basic Infrastructure</title>
<link href='https://www.tbray.org/ongoing/When/202x/2024/08/13/Basic-Infrastructure' />
<link rel='replies' thr:count='5' type='application/xhtml+xml' href='/ongoing/When/202x/2024/08/13/Basic-Infrastructure#comments' />
<id>https://www.tbray.org/ongoing/When/202x/2024/08/13/Basic-Infrastructure</id>
<published>2024-08-13T12:00:00-07:00</published>
<updated>2024-08-13T14:44:33-07:00</updated>
<category scheme='https://www.tbray.org/ongoing/What/' term='Technology/Cloud' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Technology' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Cloud' />
<summary type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>Recently, I was looking at the infrastructure bills for our <a href='https://cosocial.ca'>CoSocial</a> co-op member-owned Mastodon instance, mostly Digital Ocean and a bit of AWS. They seemed too high for what we’re getting. Which makes me think about the kind of infrastructure that a decentralized social network needs, and how to get it</div></summary>
<content type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>
Recently, I was looking at the infrastructure bills for our
<a href="https://cosocial.ca">CoSocial</a> co-op member-owned Mastodon instance, mostly Digital Ocean and a bit of AWS. They seemed
too high for what we’re getting. Which makes me think about the kind of infrastructure that a decentralized social network
needs, and how to get it.
I worked at AWS for 5½ years and part of my job was explaining why public-cloud infrastructure is a good idea. I had no
trouble doing that because, for the people who are using it, it was (and is) a good idea. The public cloud offers a quality of
service, measured by performance, security, and durability, that most customers couldn’t build by themselves.
One way to put it is this: If you experience problems in those areas, they are much more likely to be problems in your software
than in the cloud infrastructure.
Of course, providing this level of service costs billions in capex and salaries for thousands of expensive senior
engineers. So you can expect your monthly cloud-services bill to be substantial.
<h2 id='p-1'>But what if…</h2>
What if you don’t need that quality of service? What if an hour of downtime now and then was an irritant but not an
existential problem? What if you were OK with occasionally needing to restore data from backup? What if everything on your
server was public data and not interesting to bad actors?
Put another way, what if you were running a small-to-medium Fediverse instance?
If it goes offline occasionally, nobody’s life is damaged much. And, while I grant that this is not well-understood, at this
point in time everything on Fedi should be considered public, and I don’t think that’ll change even when we get end-to-end
encryption because that data of course isn’t plain text. Here is what you care about:
<ol>
<li>Members’ posts don’t get permanently lost.</li>
<li>You don’t want bad people hijacking your members’ accounts and posting damaging stuff.</li>
<li>You don’t want to provision and monitor a relational database.</li>
</ol>
<h2 id='p-2'>“Basic”?</h2>
So, what I want for decentralized social media is computers and storage “in the cloud”, as in I don’t want to have to visit them
physically. But I don’t need them to be very fast or to be any more reliable than modern server and disk hardware generally are. I do
need some sort of effective backup/restore facility, and I want good solid modern authentication.
And, of course, I want this to be a whole lot cheaper than the “enterprise”-facing public cloud. Because I’m not an
enterprise.
(I think I still need a CDN. But that’s OK because they’re commoditized and competitive these days.)
I know this is achievable. What I don’t know is who might want to offer this kind of infrastructure. I think some of it is
already out there, but you have to be pretty savvy about knowing who the vendors are and their product ranges and strengths and
weaknesses.
Maybe we don’t need any new products, just a new product category, so people like me know which products to look at.
How about “Basic Infrastructure”?
</div></content></entry>
<entry>
<title>Countrywomen</title>
<link href='https://www.tbray.org/ongoing/When/202x/2024/08/11/Countrywomen' />
<link rel='replies' thr:count='0' type='application/xhtml+xml' href='/ongoing/When/202x/2024/08/11/Countrywomen#comments' />
<id>https://www.tbray.org/ongoing/When/202x/2024/08/11/Countrywomen</id>
<published>2024-08-11T12:00:00-07:00</published>
<updated>2024-08-11T12:00:00-07:00</updated>
<category scheme='https://www.tbray.org/ongoing/What/' term='Arts/Music' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Arts' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Music' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Technology/Audio' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Technology' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Audio' />
<summary type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>In the last couple of weeks I’ve been at shows by <a href='https://www.mollytuttlemusic.com'>Molly Tuttle</a> and <a href='https://www.sierraferrellmusic.com'>Sierra Ferrell</a> (I recommend clicking both those links just for the front-page portraits). Herewith thoughts on the genres, performances, and sound quality</div></summary>
<content type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>
In the last couple of weeks I’ve been at shows by
<a href="https://www.mollytuttlemusic.com">Molly Tuttle</a> and
<a href="https://www.sierraferrellmusic.com">Sierra Ferrell</a> (I recommend clicking both those
links just for the front-page portraits). Herewith thoughts on the genres, performances, and sound quality.
Tuttle is post-bluegrass and Ferrell is, um, well, Wikipedia says “folk, bluegrass, gypsy jazz, and Latin styles” which, OK, I
guess, but it doesn’t mention pure old-fashioned country, her strongest flavor. These days, “Americana” is used to
describe both these artists. The notion that Americana implies “by white people” is just wrong, check out Rhiannon Giddens’
<a href="https://www.youtube.com/watch?v=RiP8Tfa8bB8">charming video</a> on the origin of the banjo. (Ms Giddens is a goddess;
if you don’t know about her, check her out.)
Both bands (for brevity, just Molly and Sierra) feature mandolin, fiddle, stand-up bass, and acoustic guitar. Molly adds
banjo, Sierra drums and
occasional electric guitar. Both offer flashy instrumental displays; Molly adds
big group meltdowns, veering into jam-band territory. Both women sing divinely and the bands regularly contribute lovely
multi-part harmonies.
I think that Americana just now is one of the most interesting living musical directions. These artists are young, are
standing on firm foundations, and are pushing into new territory. Judging by the crowds these days I’m not alone, so for
those who agree, I’ll offer a few words on each performance.
<h2 id='p-1'>Molly Tuttle and Golden Highway at the Hollywood Theatre</h2>
Interestingly, this is the same venue where I first saw Sierra, back in March of 2022. It’s intimate and nice-looking and has
decent sound.
The crowd was pretty grey-haired; the previous week we’d taken in an Early Music Vancouver concert dedicated to
<a href="https://en.wikipedia.org/wiki/Giovanni_Gabrieli">Gabrieli</a> (1557-1612) and the age demographic wasn’t that
different, except for Molly’s fans wear jeans and leather and, frequently, hippie accoutrements.
It dawns on me that bluegrass is in some respects a “classical” genre; It has lots of rules
and formalisms and an absolute insistence on virtuosic skill.
She played
<a href="https://www.setlist.fm/setlist/molly-tuttle-and-golden-highway/2024/hollywood-theatre-vancouver-bc-canada-3355fc45.html">a
generous selection of favorites</a> (<cite>El Dorado</cite>, <cite>Dooley’s Farm</cite>, <cite>Crooked Tree</cite>) and
exceptionally tasty covers (<cite>Dire Wolf</cite>, <cite>She’s a Rainbow</cite>). The band was awesomely tight and Molly was
in fine form.
<img src="https://www.tbray.org/ongoing/When/202x/2024/08/11/PXL_20240802_053730514.png" alt="Molly Tuttle and Golden Highway" />
In most pictures of Molly she has hair, but during her concerts she usually tells the story of how as a young child she had
Alopecia universalis (total all-body hair loss) and, particularly if the concert venue is warm, whips off her wig. At this show
she talked about how, on behalf of a support organization, she’d visited a little
Vancouver girl with Alopecia, and how sad she was that the kid couldn’t come to the show since the Hollywood is also a bar. It
was touching; good on her.
Molly, a fine singer and songwriter, is also a virtuoso bluegrass guitar flat-picker and her band are all right up
there, so the playing on balance was probably a little finer than Sierra’s posse offered. And as I mentioned, they do the
occasional jam-band rave-up, which I really enjoyed.
But their sound guy needs to be fired. I was at the show alone and thus found a corner to prop myself up that happened
to be right
behind this bozo’s desk. He had a couple of devices that I didn’t recognize, with plenty of sliders, physical and on-screen, and
he was hard at work from end to end “enhancing” the sound. He threw oceans of echo on Molly’s voice then yanked it out,
injected big rumble on song climaxes,
brightened up the banjo and mandolin so they sounded like someone driving nails into metal, and slammed the balance back and
forth to create fake stereo when licks were being traded. This sort of worked when they were doing the extended-jam thing,
but damaged every song that relied on sonic truth or subtlety, which was most of them. Feaugh. Concert sound people
should get out of the fucking way and reproduce what the musicians are playing. I guess Molly must like this or she wouldn’t
have hired him? I wish she could come out and hear what it sounds like though.
Anyhow, it’s a good band that plays good songs with astonishing skill. If you’re open to this kind of music you’d enjoy their
show.
The last encore was <cite>Helpless</cite>. I’m
not 100% sure that Molly knew what she was in for. Every grey-haired Canadian knows that tune and every word of its lyrics. So
as soon as she was three words in, the whole audience was booming along heartily, having a fine time. Quite a few grizzled
cheeks were wet with tears, but I thought Molly looked a little taken aback. She went with it, and it was lovely.
<h2 id='p-2'>Sierra Ferrell at the Orpheum</h2>
<a href="https://en.wikipedia.org/wiki/Orpheum_(Vancouver)">This hall</a> is one of Vancouver’s two big venues where the
symphony plays, operas are presented, and so on. It opened in 1927 and the decor is lavish, tastefully over-the-top, but ignore
the execrable ceiling art.
<img src="https://www.tbray.org/ongoing/When/202x/2024/08/11/PXL_20240811_043529266.png" alt="Sierra Ferrell" />
<div class='caption'>Sierra Ferrell, singing <cite>Whispering Waltz</cite>.</div>
On this picture, my usually-trusty Pixel 7 failed me. The focus is unacceptably bad but I’m
running it anyhow to share Sierra’s outfit, which is as always fabulous. She kicked up her heels once or twice, revealing big
tall Barbie-pink boots under that dress.
The audience had plenty of greybeards but on balance was way younger than Molly’s, with a high proportion of women dressed
to the nines in Western-wear finery and some of the prettiest dresses I’ve seen in years. It was really a lot of fun just
to look around and enjoy the shapes that Sierra’s influence takes.
Sierra is a wonderful singer but those songs, wow, I’m sure some of them will be loved
and shared long after I and she are in the grave.
<a href="https://www.setlist.fm/setlist/sierra-ferrell/2024/the-orpheum-vancouver-bc-canada-5355c709.html">Her set</a> didn’t
leave out any of the favorites. There were a few covers, notably <cite>Me and Bobby McGee</cite>, which was heartbreaking
and then rousing. Before starting Sierra acknowledged her debt to Janis Joplin, whom I never saw, but I felt Janis there in spirit.
Everybody is going to have a few favorites among her songs. The three-song sequence, <cite>Lighthouse</cite>, <cite>The
Sea</cite>, and <cite>Far Away Across the Sea</cite>, was so beautiful it left me feeling emptied. They turned <cite>Far
Away</cite> into a rocker with a bit of extended jamming and it was just wonderful.
But the thing about a Sierra Ferrell show isn’t just the songs or the singing or the playing, it’s
her million watts of charisma, and the connection with the crowd. People kept bringing her floral garlands and, after “Garden”,
someone ran up to the stage with a little potted plant. There are some people who, when they get up on the stage, you just
can’t take your eyes off them, and she’s one of those. I’m pretty confident that if she keeps holding it together and writing
those songs, she’s headed for Dolly Parton territory in terms of fame and fortune.
Any complaints? Yes, this was the first stop on a new tour and the sound was initially pretty rough, but they got it fixed up
so that’s forgivable.
There’s still a problem: When Sierra
leans into a really big note she overloads whatever mike they’re using; not sure what the cure is for that.
Another gripe: Sierra used to have a part of the set where the band gathered around an old-school radio mike with
acoustic instruments and played in a very traditional style. I think she shouldn’t leave that out.
Finally, one more problem: Vancouver loves Sierra just a little too much. Every little vocal flourish, every cool little
instrumental break, every one of those got a huge roar of approval from the crowd, which, fine, but some of those songs take the
level way down and then back up again in a very artful way, and I wished the crowd would shut up, let Sierra drive, and clap at the
end of the song.
<h2 id='p-3'>Americana</h2>
Like I said, this is where some of the most interesting living artists are digging in and doing great work. Highly recommended.
</div></content></entry>
<entry>
<title>Invisible Attackers</title>
<link href='https://www.tbray.org/ongoing/When/202x/2024/07/30/Invisible-Attackers' />
<link rel='replies' thr:count='5' type='application/xhtml+xml' href='/ongoing/When/202x/2024/07/30/Invisible-Attackers#comments' />
<id>https://www.tbray.org/ongoing/When/202x/2024/07/30/Invisible-Attackers</id>
<published>2024-07-30T12:00:00-07:00</published>
<updated>2024-08-03T08:10:15-07:00</updated>
<category scheme='https://www.tbray.org/ongoing/What/' term='The World/Social Media' />
<category scheme='https://www.tbray.org/ongoing/What/' term='The World' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Social Media' />
<summary type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>In the last few days we’ve had an outburst of painful, intelligent, useful conversation about racism and abuse in the world of Mastodon and the Fediverse. I certainly learned things I hadn’t known, and I’m going to walk you through the recent drama and toss in ideas on how to improve safety</div></summary>
<content type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>
In the last few days we’ve had an outburst of painful, intelligent, useful conversation about racism and abuse in the world
of Mastodon and the Fediverse. I certainly learned things I hadn’t known, and I’m going to
walk you through the recent drama and toss in ideas on how to improve safety.
For me, the story started back in early 2023 when
<a href="https://dair-community.social/@timnitGebru">Timnit Gebru</a> (the person fired by Google for questioning the
LLM-is-great orthodoxy, co-author of <cite>Stochastic Parrots</cite>) shouted loudly and eloquently that her arrival on Mastodon
was greeted by a volley of racist abuse. This shocked a lot of hyperoverprivileged people like me who don’t experience that
stuff. As the months went on after that, my perception was that the Mastodon community had pulled up its moderation socks and
things were getting better.
<h2 id='p-3'>July 2024</h2>
Then, just this week,
<a href="https://dair-community.social/@KimCrayton1">Kim Crayton</a> issued a passionate invitation to the “White Dudes for Kamala
Harris” event, followed immediately by examples of the racist trolling that she saw in response. With a content warning that this is
not pretty stuff, here are two of her posts:
<a href="https://dair-community.social/@KimCrayton1/112872020308883967">1</a>,
<a href="https://dair-community.social/@KimCrayton1/112871919476547203">2</a>.
Let me quote Ms Crayton:
<div style="border-left: 1px solid #040;">
<blockquote>The racist attacks you’ve witnessed directed at me since Friday, particularly by instances with 1 or 2
individuals, SHOULD cause you to ask “why?” and here’s the part “good white folx” often miss…these attacks are about YOU…these
attacks are INTENDED to keep me from putting a mirror in your faces and showing you that YOU TOO are harmed by white supremacy
and anti-Blackness…these attacks are no different than banning books…they’re INTENDED to keep you IGNORANT about the fact that
you’re COMPLICIT</blockquote>
</div>
She quite appropriately shouted at the community generally and the
Mastodon developers specifically. Her voice was reinforced by many others, some of whom sharpened the criticism by calling the
Mastodon team whiteness-afflicted at best and racist at worst.
People asked a lot of questions and we learned a few things. First of all, It turns out that some attackers came from
instances that are known to be toxic and should long-since have been defederated by Ms Crayton’s. Defederation is the
Fediverse’s nuclear weapon, our best tool for keeping even the sloppiest admins on their toes. To the extent our tools work at
all, they’re useless if they’re not applied.
But on the other hand it’s cheap and fast to spin up a single-user Mastodon instance that won’t get defederated until the
slime-thrower has thrown slime.
<h2 id='p-2'>Invisibility</h2>
What I’ve only now come to understand is that Mastodon helps griefers hide. Suppose you’re on instance A and looking at a post
from instance B, which has a comment from an account on instance C. Whether or not you can see that comment… is complicated. But lots
of times, you can’t. Let me excerpt a couple of remarks from someone who wishes to remain anonymous.
<div style="border-left: 1px solid #040;">
<blockquote>Thinking about how mastodon works in the context of all the poc i follow who
complain constantly about racist harassment and how often i look at their mentions and how I’ve literally never seen an
example of the abuse they’re experiencing despite actively looking for it.
It must be maddening to have lots of people saying horrible things to you while nobody who’d be willing to defend you can
see anyone doing anything to you.
But also it really does breed suspicion in allies. I believe it when people say they’re being harassed, but when I’m
looking for evidence of it on two separate instances and not ever seeing it? I have to step hard on the part of me that’s like
… really?</blockquote>
</div>
<h2 id='p-4'>Take-away 1</h2>
This is a problem that the Masto/Fedi community can’t ignore. We can honestly say that up till now, we didn’t realize how
serious it was. Now we know.
<h2 id='p-5'>Take-away 2</h2>
Let’s try to cut the Mastodon developers some slack.
Here’s a quote from one, in a private chat:
<div style="border-left: 1px solid #040;">
<blockquote>I must admit that my mentions today are making me rethink my involvement in Mastodon
I am burning myself out for this project for a long time, not getting much in return, and now I am a racist because
I dont fix racism.</blockquote>
</div>
I think it is entirely reasonable to disagree with the team, which is tiny and underfunded, on their
development priorities. Especially after these last few days, it looks like a lot of people — me, for
sure — failed to dive deep into the narrated experience of
racist abuse. In the team’s defense, they’re getting yelled at all the time by many people, all of whom have strong
opinions about their feature that needs to ship right now!
<h2 id='p-6'>Conversations</h2>
One of the Black Fedi voices that most influences me is Mekka Okereke, who
<a href="https://hachyderm.io/@mekkaokereke/112860037633719411">weighed in intelligently</a>, from which this, on the subject
of Ms Crayton:
<div style="border-left: 1px solid #040;">
<blockquote><ul>
<li>She should not have to experience this</li>
<li>It should be easier for admins at DAIR, and across the whole Fediverse, to prevent this</li>
</ul></blockquote>
</div>
Mekka has set up a meeting with the Mastodon team and says Ms Crayton will be coming along. I hope that turns out to be
useful. 
<h2 id='p-7'>More good input</h2>
Let’s start with Marco Rogers, also known as
<a href="https://social.polotek.net/@polotek">@polotek@social.polotek.net</a>. I followed Marco for ages on Twitter, not always
agreeing with his strong opinions on Web/Cloud technology, but always enjoying them. He’s been on Mastodon in recent
months and, as usual, offers
<a href="https://polotek.net/posts/a-different-vision-for-a-healthy-fediverse/">long-form opinions</a> that are worth reading.
He waded into the furore around our abuse problem, starting
<a href="https://social.polotek.net/@polotek/112876826937381377">here</a>, from which a few highlights.
<div style="border-left: 1px solid #040;">
<blockquote>I see a lot of the drama that is happening between people of
color on the platform and the mastodon dev team. I feel like I need to help.
 If people of color still find ourselves dependent on a small team of white devs to get what we want, that is a failure of
the principles of the fediverse.
 I want to know how I can find and support people that are aligned with my values. I want to enable those people to work on a
platform that I can use. And we don't need permission from the mastodon team to do so. They're not in charge.</blockquote>
</div>
Mekka, previously mentioned,
<a href="https://hachyderm.io/@mekkaokereke/112884976001207952">re-entered the fray</a>:
<div style="border-left: 1px solid #040;">
<blockquote>If you run a Mastodon instance, and you don't block at least the minimum list of known terrible instances, and
you have Black users, it's just a matter of time before your users face a hate brigade.
That's the only reason these awful instances exist. That's all they do.
Telling users "Just move to a better server!" is supremely unhelpful. It doesn't help the mods, and it doesn't help the users.
It needs to be easier. It's currently too hard to block them and keep up with the new ones.</blockquote>
</div>
And more;
<a href=" https://infosec.exchange/@jerry/112887084476486558">this</a> is from Jerry Bell, one of the longest-lasting Fediverse
builders (and I think the only person I’m quoting here
who doesn’t present as Black). These are short excerpts from a long and excellent piece.
<div style="border-left: 1px solid #040;">
<blockquote>I am writing this because I'm tired of watching the cycle repeat itself, I'm tired of watching good people get
harassed, and I'm tired of the same trove of responses that inevitably follows. 
… About this time, the sea lions show up in replies to the victim, accusing them of embracing the victim role, trying to
cause racial drama, and so on.
A major factor in your experience on the fediverse has to do with the instance you sign up to. Despite what the folks on
/r/mastodon will tell you, you won't get the same experience on every instance.</blockquote>
</div>
<h2 id='p-8'>What next?</h2>
I don’t know. But I feel a buzz of energy, and smart people getting their teeth into the meat of the problem.
Now I have
thoughts to offer about moving forward.
<h2 id='p-9'>Who are the enemy?</h2>
They fall into two baskets: Professional and amateur. I think the current Mastodon attackers are mostly amateurs. These are
lonely Nazis, incels, channers, your basic scummy online assholes. Their organization is loose at best (“He’s pointing at
her, so I will too”), and they’re typically not well-funded nor are they deep technical experts.
Then there are the pros, people doing this as their day job. I suspect most of those are working for nation states, and
yes, we all know which nation states those probably are. They have sophisticated automation to help them launch armies of bots.
Here are some suggestions about potential fight-backs, mostly aimed at amateurs.
<h2 id='p-10'>Countermeasure: Money</h2>
There’s this nonprofit called
<a href="https://about.iftas.org">IFTAS</a> which is working on tools and support structures for moderation. How about they
start offering a curated allowlist of servers that it’s safe to federate with? How do you get on that list? Pay $50 to IFTAS,
which will add you to the watchlist, and also to a service scanning your members’ posts for abusive stuff during your
first month or so of operation.
Cue the howls of outrage saying “Many oppressed people can’t afford $50, you’re discriminating against the victims!”
I suppose, but they can still get online at any of the (many) free-to-use instances. I think it’s totally reasonable to throw a
$50 roadblock in the process of setting up a server.
In this world, what happens? Joe Incel sets up an instance at ownthelibs.nazi or wherever, pays his $50, and starts throwing
slime. This gets reported and pretty soon, he’s defederated. Sure, he can do it again. But how many times is this
basement-dweller willing to spend $50, leaving a paper trail each time just in case he says something that’s illegal to say in the
jurisdiction where he lives? Not that many, I think?
<h2 id='p-11'>Countermeasure: Steal from Pleroma</h2>
It turns out Mastodon isn’t the only Fediverse software. One of the competitors is
<a href="https://pleroma.social">Pleroma</a>. Unfortunately, it seems to be the server of choice for our attackers, because it’s
easy and cheap to set up. Having said that, its moderation facilities are generally regarded as superior to Mastodon’s, notably a
subsystem called
<a href="https://docs.pleroma.social/backend/configuration/mrf/">Message Rewrite Facility (MRF)</a> which I haven’t been near
but is frequently brought up as something that would be useful.
<h2 id='p-12'>Countermeasure: Make reporting better</h2>
I report abusive posts sometimes, and, as a moderator for
<a href="https://cosocial.ca">CoSocial.ca</a> I see reports too. I think the “Report post” interface on many clients is weak,
asking you unnecessary questions.
And when I get a report, it seems like half the time none of the abusive material is attached, and it takes me multiple
clicks to look at the reported account’s feed, which feels like a pretty essential step.
Here’s how I’d like reporting to work.
<ol>
<li>There’s a single button labeled “Report this post”. When you click it, a popup says “Reported, thanks” and you’re
done. Maybe it could query whether you want to block the user or instance, but it’s super important that the process be
lightweight.</li>
<li>The software should pull together a report package including the reported post’s text and graphics. (Not just the URLs,
because the attackers like to cover their tracks.) Also the attacker’s profile page. No report should ever be filed without
evidence.</li>
</ol>
<h2 id='p-14'>Countermeasure: Rules of thumb</h2>
Lauren offered this: Suppose a reply or mention comes in for someone on Instance A from someone on Instance B. Suppose
Instance A could check whether anyone else on A follows anyone on B. If not, reject the incoming message. This would have to be
a per-user not global setting, and I see it as a placeholder for a whole class of heuristics that could usefully get in the
attackers’ way.
<h2 id='p-13'>Wish us luck</h2>
Obviously I’m not claiming that any of these ideas are the magic bullet that’s going to slay the online-abuse monster. But we
do need ideas to work with, because it’s not a monster that we can afford to ignore.
I care intensely about this, because I think decentralization is an essential ingredient of online conversation, and online
conversation is valuable, and if we can’t make it safe we won’t have it.
</div></content></entry>
<entry>
<title>Union of Finite Automata</title>
<link href='https://www.tbray.org/ongoing/When/202x/2024/07/28/Union-of-Finite-Automata' />
<link rel='replies' thr:count='0' type='application/xhtml+xml' href='/ongoing/When/202x/2024/07/28/Union-of-Finite-Automata#comments' />
<id>https://www.tbray.org/ongoing/When/202x/2024/07/28/Union-of-Finite-Automata</id>
<published>2024-07-28T12:00:00-07:00</published>
<updated>2024-07-29T13:38:55-07:00</updated>
<category scheme='https://www.tbray.org/ongoing/What/' term='Technology/Quamina Diary' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Technology' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Quamina Diary' />
<summary type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>In building Quamina, I needed to compute the union of two finite automata (FAs). I remembered from some university course 100 years ago that this was possible in theory, so I went looking for the algorithm, but was left unhappy. The descriptions I found tended to be hyper-academic, loaded with mathematical notation that I found unhelpful, and didn’t describe an approach that I thought a reasonable programmer would reasonably take. The purpose of this ongoing entry is to present a programmer-friendly description of the problem and of the algorithm I adopted, with the hope that some future developer, facing the same problem, will have a more satisfying search experience</div></summary>
<content type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>
In building Quamina, I needed to compute the union of two finite automata (FAs). I remembered from some university course 100 years
ago that this was possible in theory, so I went looking for the algorithm, but was left unhappy.
The descriptions I found tended to be hyper-academic, loaded with mathematical notation that I found unhelpful, and
didn’t describe an approach that I thought a reasonable programmer would reasonably take. The purpose of this <span
class='o'>ongoing entry is to present a
programmer-friendly description of the problem and of the algorithm I adopted, with the hope that some future developer, facing
the same problem, will have a more satisfying search experience.
There is very little math in this discussion (a few subscripts), and no
circles-and-arrows pictures. But it does have working Go code.
<h2 id='p-1'>Finite automata?</h2>
I’m not going to rehash the theory of FAs (often called state machines). In practice the purpose of an FA is to match (or
fail to match) some
input against some pattern. What the software does when the input matches the pattern (or doesn’t) isn’t relevant to
our discussion today.
Usually the inputs are strings and the patterns are regular expressions or equivalent. In practice, you compile a pattern
into an FA, and then you go through the input, character by character, trying to traverse the FA to find out whether it matches
the input.
An FA has a bunch of states, and for each state
there can be a list of input symbols that lead to transitions to other states. What exactly I mean by “input symbol” turns out to be
interesting and affects your choice of algorithm, but let’s ignore that for now.
The following statements apply:
<ol>
<li>One state is designated as the “start state” because, well, that’s where you start.</li>
<li>Some states are called “final”, and reaching them means you’ve matched one or more patterns. In Quamina’s FAs, each
state has an extra field (usually empty) saying “if you got here you matched P*, yay!”, where P* is a list of labels for the
(possibly more than one) patterns you matched.</li>
<li>It is possible that you’re in a state and for some particular input, you transition to more than one other state. If
this is true, your FA is nondeterministic, abbreviated NFA.</li>
<li>It is possible that a state can have one or more “epsilon transitions”, ones that you can just take any time, not requiring
any particular input. (I wrote about this in
<a href="/ongoing/When/202x/2024/06/17/Epsilon-Love">Epsilon Love</a>.)
Once again, if this is true, you’ve got an NFA. If neither this statement nor the previous are true, it’s
a deterministic finite automaton, DFA.</li>
</ol>
The discussion here works for NFAs, but lots of interesting problems can be solved with DFAs, which are simpler and faster,
and this algorithm works there too.
<h2 id='p-2'>Union?</h2>
If I have <code>FA1</code> that matches “foo” and <code>FA2</code> that matches “bar”, then their union,
<code>FA1 ∪ FA2</code>, matches both “foo”
and “bar”. In practice Quamina often computes the union of a large number of FAs, but it does so a pair at a time, so we’re
only going to worry about the union of two FAs.
<h2 id='p-3'>The academic approach</h2>
There are plenty of Web pages and YouTubes covering this. Most of them are full of Greek characters and math symbols. They go
like this:
<ol>
<li>You have two FAs, call them <code>A</code> and <code>B</code>. <code>A</code> has states <code>A1</code>, …
<code>AmaxA</code>, <code>B</code> has <code>B1</code>, … <code>BmaxB</code></li>
<li>The union contains all the states in <code>A</code>, all the states in <code>B</code>, and the “product” of
<code>A</code> and <code>B</code>, which is to say states you could call <code>A1B1</code>,
<code>A1B2</code>, <code>A2B1</code>, <code>A2B2</code>,
… <code>AmaxABmaxB</code>.</li>
<li>For each state <code>AXBY</code>, you work out its transitions by looking at the transitions of
the two states being combined. For some input symbol, if <code>AX</code> has a transition to
<code>AXX</code> but <code>BY</code> has no transition, then the combined state just has the A
transition. The reverse for an input where <code>BY</code> has a transition but <code>AX</code>
doesn’t. And if <code>AX</code> transitions to <code>AXX</code> and <code>BY</code> transitions to
<code>BYY</code>, then the transition is to <code>AXXBYY</code>.</li>
<li>Now you’ll have a lot of states, and it usually turns out that many of them aren’t reachable. But there are plenty of
algorithms to filter those out. You’re done, you’ve computed the union and <code>A1B1</code> is its
start state!</li>
</ol>
<h2 id='p-4'>Programmer-think</h2>
If you’re like me, the idea of computing all the states, then throwing out the unreachable ones, feels wrong. So here’s
what I suggest, and has worked well in practice for Quamina:
<ol>
<li>First, merge <code>A1</code> and <code>B1</code> to make your new start state
<code>A1B1</code>. Here’s how:</li>
<li>If an input symbol causes no transitions in either <code>A1</code> or <code>B1</code>, it also
doesn’t cause any in <code>A1B1</code>.</li>
<li>If an input symbol causes a transition in <code>A1</code> to <code>AX</code> but no transition
in <code>B1</code>, then you adopt <code>AX</code> into the union, and any other <code>A</code> states
it points to, and any they point to, and so on.</li>
<li>And of course if <code>B1</code> has a transition to <code>BY</code> but
<code>A1</code> doesn’t transition, you flip it the other way, adopting <code>BY</code> and its
descendents.</li>
<li>And if <code>A1</code> transitions to <code>AX</code> and <code>B1</code> transitions
to <code>BY</code>, then you adopt a new state <code>AXBY</code>, which you compute
recursively the way you just did for <code>A1B1</code>. So you’ll never compute anything that’s not
reachable.</li>
</ol>
I could stop there. I think that’s enough for a competent developers to get the idea? But it turns out there
are a few details, some of them interesting. So, let’s dig in.
<h2 id='p-5'>“Input symbol”?</h2>
The academic discussion of FAs is very abstract on this subject, which is fair enough, because when you’re talking about how
to build, or traverse, or compute the union of FAs, the algorithm doesn’t depend very much on what the symbols actually are. But
when you’re writing code, it turns out to matter a lot.
In practice, I’ve done a lot of work with FAs over the years, and I’ve only ever seen four things used as input symbols
to drive them. They are:
<ul>
<li>Unicode “characters” represented by code points, integers in the range 0…1,114,111 inclusive.</li>
<li>UTF-8 bytes, which have values in the range 0…244 inclusive.</li>
<li>UTF-16 values, unsigned 16-bit integers. I’ve only ever seen this used in Java programs because that’s what
its native <code>char</code> type is. You probably don’t want to do this.</li>
<li>Enum values, small integers with names, which tend to come in small collections.</li>
</ul>
As I said, this is all I’ve seen, but 100% of the FAs that I’ve seen automatically generated and subject to
set-arithmetic operations like Union are based on UTF-8. And that’s what Quamina uses, so that’s what I’m going to use in the
rest of this discussion.
<h2 id='p-7'>Code starts here</h2>
This comes from Quamina’s
<a href="https://github.com/timbray/quamina/blob/main/nfa.go">nfa.go</a>. We’re going to look at the function
<code>mergeFAStates</code>, which implements the merge-two-states logic described above.
Lesson: This process can lead to a lot of wasteful work. Particularly if either or both of the states transition on ranges of
values like <code>0…9</code> or <code>a…z</code>. So we only want to do the work merging any pair of states once, and we want
there only to be one merged value. Thus we start with a straightforward memo-ization.
<div class="tbc"><pre>
func mergeFAStates(state1, state2 *faState, keyMemo map[faStepKey]*faState) *faState {
 // try to memo-ize
 mKey := faStepKey{state1, state2}
 combined, ok := keyMemo[mKey]
 if ok {
 return combined
 }
</pre>
Now some housekeeping. Remember, I noted above that any state might contain a signal saying that arriving here
means you’ve matched pattern(s). This is called <code>fieldTransitions</code>, and the merged state obviously has to
match all the things that either of the merged states match. Of course, in the vast majority of cases neither merged state
matched anything and so this is a no-op.
<pre>
 fieldTransitions := append(state1.fieldTransitions, state2.fieldTransitions...)</pre>
Since our memo-ization attempt came up empty, we have to allocate an empty structure for the new merged state, and add it to the
memo-izer.
<pre>
combined = &faState{table: newSmallTable(), fieldTransitions: fieldTransitions}
 keyMemo[mKey] = combined</pre>
Here’s where it gets interesting. The algorithm talks about looking at the inputs that cause transitions in the states we’re
merging. How do you find them? Well, in the case where you’re transitioning on UTF-8 bytes, since there are only 244 values,
why not do the simplest thing that could possibly work and just check each byte value?
Every Quamina state contains a table that encodes the byte transitions, which operates like the Go construct
<code>map[byte]state</code>. Those tables are implemented in
<a href="/ongoing/When/202x/2022/06/25/Small-Tables">a compact data structure optimized for fast traversal</a>. But for doing this
kind of work, it’s easy to “unpack” them into a fixed-sized table; in Go, <code>[244]state</code>. Let’s do that for the states
we’re merging and for the new table we’re building.
<pre> u1 := unpackTable(state1.table)
 u2 := unpackTable(state2.table)
 var uComb unpackedTable</pre>
<code>uComb</code> is where we’ll fill in the merged transitions.
Now we’ll run through all the possible input values; <code>i</code> is the byte value, <code>next1</code> and <code>next2</code>
are the transitions on that value. In practice, <code>next1</code> and <code>next2</code> are going to be null most of the time.
<pre> for i, next1 := range u1 {
 next2 := u2[i]</pre>
Here’s where we start building up the new transitions in the unpacked array <code>uComb</code>.
For many values of <code>i</code>, you can avoid actually merging the states to create a new one. If the transition is the
same in both input FAs, or if either of them are null, or if the transitions for this value of <code>i</code> are the same as
for the last value. This is all about avoiding unnecessary work and the <code>switch</code>/<code>case</code> structure is the
result of a bunch of profiling and optimization.
<pre> switch {
 case next1 == next2: // no need to merge
 uComb[i] = next1
 case next2 == nil: // u1 must be non-nil
 uComb[i] = next1
 case next1 == nil: // u2 must be non-nil
 uComb[i] = next2
case i > 0 && next1 == u1[i-1] && next2 == u2[i-1]: // dupe of previous step - happens a lot
 uComb[i] = uComb[i-1]</pre>
If none of these work, we haven’t been able to avoid merging the two states. We do that by a recursive call to invoke
all the logic we just discussed.
There is a complication. The automaton might be nondeterministic, which means that there might be more than one transition
for some byte value. So the data structure actually behaves like <code>map[byte]*faNext</code>, where <code>faNext</code> is a
wrapper for a list of states you can transition to.
So here we’ve got a nested loop to recurse for each possible combination of transitioned-to states that can occur on this
byte value. In a high proportion of cases the FA is deterministic, so there’s only one state from each FA being merged and this
nested loop collapses to a single recursive call.
<pre> default: // have to recurse & merge
 var comboNext []*faState
 for _, nextStep1 := range next1.states {
 for _, nextStep2 := range next2.states {
comboNext = append(comboNext, mergeFAStates(nextStep1, nextStep2, keyMemo))
 }
 }
 uComb[i] = &faNext{states: comboNext}
 }
 }</pre>
We’ve filled up the unpacked state-transition table, so we’re almost done. First, we have to compress it into its
optimized-for-traversal form.
<pre> combined.table.pack(&uComb)</pre>
Remember, if the FA is nondeterministic, each state can have “epsilon” transitions which you can follow any time without
requiring any particular input. The merged state needs to contain all the epsilon transitions from each input state.
<pre> combined.table.epsilon = append(state1.table.epsilon, state2.table.epsilon...)
 return combined
}</pre>
</div>
And, we’re done. I mean, we are once all those recursive calls have finished crawling through the states being merged.
<h2 id='p-8'>Is that efficient?</h2>
As I said above, this is an example of a “simplest thing that could possibly work” design. Both the recursion and the
unpack/pack sequence are kind of code smells, suggesting that this could be a pool of performance quicksand.
But apparently not. I ran a benchmark where I added 4,000 patterns synthesized from the Wordle
word-list; each of them looked like this:
<code>{"allis": { "biggy": [ "ceils", "daisy", "elpee", "fumet", "junta", … </code> (195 more).
This produced a huge deterministic FA with about 4.4 million states, with the addition of these hideous worst-case
patterns running at 500/second. Good enough for rock ’n’ roll.
How about nondeterministic FAs? I went back to that Wordle source and, for each of its 12,959 words, added a pattern with a
random wildcard; here are three of them:
<code>{"x": [ {"shellstyle": "f*ouls" } ] } 
{"x": [ {"shellstyle": "pa*sta" } ] } 
{"x": [ {"shellstyle": "utter*" } ] }</code>
This produced an NFA with 46K states, the addition process ran at 70K patterns/second.

Sometimes the simplest thing that could possibly work, works.
</div></content></entry>
<entry>
<title>Terse Directions</title>
<link href='https://www.tbray.org/ongoing/When/202x/2024/07/19/Short-Directions' />
<link rel='replies' thr:count='9' type='application/xhtml+xml' href='/ongoing/When/202x/2024/07/19/Short-Directions#comments' />
<id>https://www.tbray.org/ongoing/When/202x/2024/07/19/Short-Directions</id>
<published>2024-07-19T12:00:00-07:00</published>
<updated>2024-07-19T15:40:22-07:00</updated>
<category scheme='https://www.tbray.org/ongoing/What/' term='Technology/Software' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Technology' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Software' />
<summary type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>This post describes a service I want from my online-map provider. I’d use it all the time. Summary: When I’m navigating an area I already know about, don’t give me turn-by-turn, just give me a short list of the streets to take</div></summary>
<content type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>
This post describes a service I want from my online-map provider.
I’d use it all the time. Summary: When I’m navigating an area I already know about, don’t give me turn-by-turn, just
give me a short list of the streets to take.
I’ve been living in Vancouver for decades and, driving or cycling, know how to get almost anywhere. It helps that, like most
North American cities, we have a fairly regular north-south-east-west grid.
These days, when I’m going any distance by car, I get directions from Google Maps because it knows where the traffic is bad, and
the traffic is usually bad somewhere. But it gives me way more directions than I need. Let’s look at a concrete example.
<img src="https://www.tbray.org/ongoing/When/202x/2024/07/19/vancouver-streets.png" alt="Part of central and east Vancouver" />
<div class='caption'>To follow the narrative, you’ll probably have to click to expand, and it’s pretty big. It may be easier
just to open
<a href="https://www.openstreetmap.org/#map=14/49.2698/-123.0918">the map</a> in another window. Map credit:
OpenStreetMap.</div>
My place is near the bottom edge of this map off to the west, between Cambie and Main streets. I occasionally attend a meetup
at New Brighton Park, which is the little splodge of green at the very top right corner of the map, across the highway
from Hastings Racecourse. It’s on McGill street.
To get there, I have to go quite a distance both east and north. Candidates for north/south travel include
Main, Clark, Nanaimo, and Renfrew streets. Candidates for east/west include 12th, Broadway, 1st, and Hastings.
Right now, Google Maps insists on turn-by turn, with three warnings for each turn. It’s dumb and annoying and interrupts
whatever music or show I’m listening to.
What I want is to get in the car and say “Short directions to New Brighton Park” and have it say “Take Main to 12th to Nanaimo to
1st to Renfrew to McGill.” Then when I’m driving, I’d get one vocal warning a block out from each turn,
like “Next left on Nanaimo” or some such.
Of course, when I’m navigating in a strange place, I’d want the traditional turn-by-turn. Don’t know about you, but the bulk
of my navigation is in territory I know and mostly about avoiding traffic.
The OpenStreetMap data is public and good. Traffic data is… a problem. But for anyone who has it, you can have me for a
customer. Just learn to be terse.
</div></content></entry>
<entry>
<title>2009 Ranger</title>
<link href='https://www.tbray.org/ongoing/When/202x/2024/07/12/2009-Ranger' />
<link rel='replies' thr:count='8' type='application/xhtml+xml' href='/ongoing/When/202x/2024/07/12/2009-Ranger#comments' />
<id>https://www.tbray.org/ongoing/When/202x/2024/07/12/2009-Ranger</id>
<published>2024-07-12T12:00:00-07:00</published>
<updated>2024-07-17T10:43:44-07:00</updated>
<category scheme='https://www.tbray.org/ongoing/What/' term='The World/Products/Automobiles' />
<category scheme='https://www.tbray.org/ongoing/What/' term='The World' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Products' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Automobiles' />
<summary type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>This week we’re vacationing at the family cabin on <a href='https://en.wikipedia.org/wiki/Keats_Island_(British_Columbia)'>an island</a>; the nearest town is <a href='https://en.wikipedia.org/wiki/Gibsons'>Gibsons</a>. Mid-week, we hit town to pick up groceries and hardware. Unfortunately, it’s a really demanding walk from the waterfront to the mall, particularly with a load to carry, and there’s little public transit. Fortunately, there’s <a href='https://www.coastcarco-op.ca'>Coast Car Co-op</a>, a competent and friendly little five-car outfit. We booked a couple of hours and the closest vehicle was a 2009 <a href='https://en.wikipedia.org/wiki/Ford_Ranger'>Ford Ranger</a>, described as a “compact pickup” or “minitruck”. It made me think</div></summary>
<content type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>
This week we’re vacationing at the family cabin on
<a href="https://en.wikipedia.org/wiki/Keats_Island_(British_Columbia)">an island</a>;
the nearest town is
<a href="https://en.wikipedia.org/wiki/Gibsons">Gibsons</a>.
Mid-week, we hit
town to pick up groceries and hardware. Unfortunately, it’s a really demanding walk from the waterfront to the
mall, particularly with a load to carry, and there’s little public transit. Fortunately, there’s
<a href="https://www.coastcarco-op.ca">Coast Car Co-op</a>, a competent and friendly little five-car outfit. We booked a couple
of hours and the closest vehicle was a 2009
<a href="https://en.wikipedia.org/wiki/Ford_Ranger">Ford Ranger</a>, described as a “compact pickup” or “minitruck”. It made
me think.
<img src="https://www.tbray.org/ongoing/When/202x/2024/07/12/white-ranger.png" alt="2009 Ford Ranger" />
<h2 id='p-1'>Think back fifteen years</h2>
I got in the Ranger and tried to adjust the seat, but that was as far back as it went. It didn’t go up or down. There were no
cameras to help me back up. There was nowhere to plug my phone in. It had a gearshift on the steering column that moved a little red
needle in a PRNDL tucked under the speedometer. There was no storage except for the truck bed. It wasn’t very fast. The radio
was just a radio. It was smaller than almost anything on the road. I had to manipulate a a physical “key” thing to make it go.
To open and close the window, you have to turn a crank.
I bet there were hardly any CPUs under the hood.
And, it was… perfectly OK.
The rear-view mirrors were big and showed me what I needed. It was dead easy to park, I could see
all four of its corners. There was enough space in the back to carry all our stuff with plenty room to spare. You wouldn’t want
to drive fast in a small tourist town with lots of steep hills, blind corners, and distracted pedestrians.
It wasn’t tracking my trips and selling the info. The seats were comfy enough.
<h2 id='p-2'>Car companies: Dare to do less</h2>
I couldn’t possibly walk away from our time in the Ranger without thinking about the absolutely insane amounts of money
and resources and carbon loading we could save by building smaller, simpler, cheaper, dumber, automobiles.
</div></content></entry>
<entry>
<title>Q Numbers</title>
<link href='https://www.tbray.org/ongoing/When/202x/2024/07/09/Q-Numbers' />
<link rel='replies' thr:count='4' type='application/xhtml+xml' href='/ongoing/When/202x/2024/07/09/Q-Numbers#comments' />
<id>https://www.tbray.org/ongoing/When/202x/2024/07/09/Q-Numbers</id>
<published>2024-07-09T12:00:00-07:00</published>
<updated>2024-07-09T12:00:00-07:00</updated>
<category scheme='https://www.tbray.org/ongoing/What/' term='Technology/Quamina Diary' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Technology' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Quamina Diary' />
<summary type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>This ongoing fragment describes how to match and compare numbers using a finite automaton, which involves transforming them into strings with the right lexical properties. My hope is that there are at least twelve people in the world who are interested in the intersection of numeric representation and finite automata. [Note: This whole piece, except for the description of the problem, has been obsoleted by <a href='/ongoing/When/202x/2024/08/28/Q-Numbers-2'>Q Numbers Redux</a>.]</div></summary>
<content type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>
This ongoing fragment describes how to match and compare numbers using a finite automaton, which involves
transforming them into strings with the right lexical properties. My hope is that there are at least twelve people in the world who
are interested in the intersection of numeric representation and finite automata. 
[Note: This whole piece, except for the description of the problem, has been obsoleted by
<a href="/ongoing/When/202x/2024/08/28/Q-Numbers-2">Q Numbers Redux</a>.]
<h2 id='p-2'>Background</h2>
(Feel free to skip this part if you already know about Quamina.)
This is yet another entry in the
<a href="/ongoing/What/Technology/Quamina%20Diary/">Quamina Diary</a> series of blog posts. Quamina is a Go-language library
that allows you to compile a bunch of “Patterns” together and, when presented with “events”, i.e. JSON data blobs, informs you
which (if any) of the Patterns match each event, at a speed which is high (often millions/second) and only weakly related to the
number of Patterns in any Quamina instance.
Quamina was inspired by
<a href="https://github.com/aws/event-ruler">AWS Event Ruler</a> (“Ruler” for short), a package I helped develop while at AWS
that has since been open-sourced. (Thanks, AWS!) By “based on” I mean “does a subset of the same things compatibly, with a design that
is quite different, in interesting ways”. Quamina is also a fruitful source of software geekery for me to write about
here, which I enjoy.
<h2 id='p-1'>The problem</h2>
Suppose you want to match records for boxes whose height is 20cm. A sample of such a record, with most fields
removed:
<pre><code>{
"box": {
"dimensions": {
"width": 100,
"height": 20,</code></pre>
(Much omitted.)
A Quamina Pattern designed to match those records would look like this:
<pre><code>{
"box": {
"dimensions": {
"height": [ 20 ]
}
}
}</code></pre>
All good so far. But what if, due to some upstream computer program or inventory admin, a
message showed up like so?
<pre><code>{
"box": {
"dimensions": {
"width": 100.0,
"height": 20.0,
</code></pre>
Up until my last PR landed, Quamina didn’t know that “20” and “20.0” and “2.0e1” were the same quantity; it knew how to
compare strings to other strings and that was all. Which was unsatisfactory. And a problem which had been solved years ago
(partly by me) in Ruler.
<h2 id='p-10'>Question</h2>
Pause a moment and ask yourself: How would you write a finite automaton which would Do The Right Thing with numbers?
I’m not going to claim that the way Ruler and Quamina do it is optimal, but it’s worked pretty well for years and processed
trillions of events with no breakages I know of.
Our answer: normalize the numbers into fixed-sized strings whose lexical ordering is that of the numbers they represent.
Code first:
<div class="tbc"><pre>
func qNumFromFloat(f float64) (qNumber, error) {
 if f < -FiveBillion || f > FiveBillion {
 return nil, errors.New("value must be between -5e9 and +5e9 inclusive")
 }
 value := uint64(TenE6 * (FiveBillion + f))
 return toHexStringSkippingFirstByte(value), nil
}
</pre></div>
<h2 id='p-3'>Constraints</h2>
Quamina requires, for numeric matching to work properly, that:
<ol>
<li>The numbers be between -/+5×109, inclusive.</li>
<li>The numbers have no more than five digits to the right of the decimal point.</li>
</ol>
You’ll notice that the code above enforces the first condition but not the second. We’ll get to that.
<h2 id='p-4'>Effects</h2>
So, what that code is doing is:
<ol>
<li>Adding 5.0e9 to the number so it’s in the range 0 … 10.0e9.</li>
<li>Multiplying by 106 to push the five-digit fractional part to the left of the decimal point, preserving its
sixth digit (if any) so the rounding in the next step works.</li>
<li>Converting it from a <code>float64</code> into a <code>uint64</code>.</li>
<li>Turning that into a big-endian 14-byte hex string.</li>
</ol>
So any number that meets the constraints above is represented as 14 hex digits whose lexical order is consistent with the
underlying numbers. “20”, “20.0” and “2.0e1” are all “11C3793911AD00”. Which means that this Pattern will do what reasonable
people expect:
<pre><code>{"box": { "dimensions": { "height": [ 20 ] } } }</code></pre>
<h2 id='p-5'>More formally</h2>
There are 1015 numbers that meet the constraints described above. This process maps them into hex strings. The
first three and their mappings are: 
<pre><code>-5,000,000,000, -4,999,999,999.99999, -4,999,999,999.99998</code>
<code>00000000000000, 00000000000009, 00000000000014</code></pre>
And the last three: 
<pre><code>4,999,999,999.99998, 4,999,999,999.99999, 5,000,000,000</code>
<code> 2386F26FC0FFEC, 2386F26FC0FFF6, 2386F26FC10000</code></pre>
<h2 id='p-6'>Less formally</h2>
This includes “most” numbers that are used in practice, including prices, occurrence counts, size measurements, and so
on.
Examples of numbers that do not meet these criteria include AWS account numbers, some telephone
numbers, and cryptographic keys/signatures. For those, Quamina just preserves the digits, whatever they may be, and in fact,
this also usually ends up doing what people expect.
<h2 id='p-11'>Could we do better?</h2>
I think so. To start with, hex digits are an inefficient way to represent bits; there are
<a href="https://en.wikipedia.org/wiki/Binary-to-text_encoding#Encoding_standards">many other options</a>.
The current hex
approach hasn’t changed since a very early version of Ruler because it’s never been a pain point.
Speaking of Ruler, they recently landed a PR that lets them have 6 fractional digits as opposed to Quamina’s 5, simply by
using decimal rather than binary arithmetic. It’s fast, too! This was made easier by the fact that Java has BigDecimal built in,
while Go doesn’t. There are good open-source options out there, but I am extremely reluctant to accept dependencies in a package as
low-level as Quamina. I don’t think matching more one more fractional digit justifies the cost of a dependency.
<h2 id='p-8'>“Q numbers?”</h2>
In Ruler, the data type is <code>ComparableNumber</code>. In Quamina I toyed with <code>comparableNumber</code> and
<code>canonicalNumber</code> but neither is quite right and both produced excessively long and ugly variable and function
names. So I decided to call them “Q numbers”, where Q is for Quamina. The code became noticeably more readable.
While the set of Rationals is called “Q”, the only other significant use of the phrase “Q number” is some
<a href="https://en.wikipedia.org/wiki/Q_(number_format)">weird old thing</a> out of Texas Instruments.
<h2 id='p-7'>Practicalities</h2>
The code to enforce the constraints and do the conversion isn’t that cheap. When I first naively dropped it in, I saw a
nasty performance regression. Code optimization helped, but
I realized then that it’s really important not to convert an incoming field that happens to be
a JSON number into a Q number unless you know, first, that it meets the constraints and second, that the finite automaton we’re
trying to match has a Pattern with a numerical match that also met the Q number constraints.
The one moderately clever stroke here relies on the fact that Quamina has its own JSON parser,
<a href="/ongoing/When/202x/2022/06/10/Quamina-Optimizing#p-5">because Reasons</a>. The parser obviously has to step its way
through numbers, and it’s easy enough there to notice where the syntax characters like “.” and “e” are and cheaply figure out if
the decimal is the right size.
<h2 id='p-9'>Conclusion</h2>
Quamina now knows numbers. It’s a little slower to match a 14-digit Q than a string like “20”, but finite automata are fast
and anyhow, being right matters.
</div></content></entry>
<entry>
<title>Lounge Penguin</title>
<link href='https://www.tbray.org/ongoing/When/202x/2024/06/23/Lounge-Penguin' />
<link rel='replies' thr:count='2' type='application/xhtml+xml' href='/ongoing/When/202x/2024/06/23/Lounge-Penguin#comments' />
<id>https://www.tbray.org/ongoing/When/202x/2024/06/23/Lounge-Penguin</id>
<published>2024-06-23T12:00:00-07:00</published>
<updated>2024-06-23T12:00:00-07:00</updated>
<category scheme='https://www.tbray.org/ongoing/What/' term='Arts/Music' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Arts' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Music' />
<category scheme='https://www.tbray.org/ongoing/What/' term='The World' />
<category scheme='https://www.tbray.org/ongoing/What/' term='The World' />
<summary type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>Lounge, as in a jazz club. Penguin, as in GoGo Pengin, a piano/bass/drums trio. We caught their show at Jazz Alley in Seattle last week. Maybe you should go hit a jazz lounge sometime</div></summary>
<content type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>
Lounge, as in a jazz club. Penguin, as in GoGo Pengin, a piano/bass/drums trio.
We caught their show at Jazz Alley in Seattle last week.
Maybe you should go hit a jazz lounge sometime.
<h2 id='p-1'>What happened was</h2>
<a href="/ongoing/When/200x/2006/06/11/Now-We-Are-Four">My daughter turned eighteen</a> and graduated high school. She had heard
that
<a href="https://carseatheadrest.com/">Car Seat Headrest</a> was playing Seattle’s Woodland Park Zoo, and could tickets and a
road trip (me buying and driving) be her present? Seemed reasonable, and she found a friend to take along. I wouldn’t mind seeing the
Headrests (decent indie rock stuff) but her party, her friend. I noticed that GoGo Penguin was playing Seattle’s Jazz Alley, and
Lauren was agreeable to coming along for the ride and the show.
I only know about GoGo Penguin because YouTube Music drops them into my default stream
now and then. I’d thought “sounds good, maybe a little abstract”, couldn’t have named a song, but hey.
<h2 id='p-2'>The “Jazz Club” concept</h2>
You’ve seen it in a million old movies, and the
<a href="https://en.wikipedia.org/wiki/Vic_Fontaine">Vic Fontaine</a> episodes of
<a href="https://en.wikipedia.org/wiki/Star_Trek:_Deep_Space_Nine">ST:DS9</a>.
The lights are low, the audience is sitting at tables with little lamps on them, the band’s on a thrust stage among the tables, there’s
expected to be a soft background of clinking glasses and conversation. Some people are focusing in tight on the music, others are
socializing at a respectfully low volume.
Of course, usually a gunfight breaks out or an alien materializes on stage… no
wait, that’s just on-screen not real-life.
All jazz clubs serve alcohol — fancy cocktails, natch — and many will
sell you dinner too.
<a href="https://www.jazzalley.com/www-home/">Dimitriou’s Jazz Alley</a> in Seattle is a fine example.
<img src="https://www.tbray.org/ongoing/When/202x/2024/06/23/PXL_20240621_032418935.png" alt="GoGo Penguin at Demetriou’s Jazz Alley in Seattle" />
<div class='caption'>GoGo Penguin at Jazz Alley; June 20th, 2024. 
Our table was in the balcony.</div>
We had a decent if conventional Pacific-Northwest dinner (crab and halibut), with a good bottle of local white.
They’ve got things set up so most people have finished eating by the time the music starts. The seats were comfy. The decor
was pleasing. The service was impeccable. I felt very grown-up.
<h2 id='p-3'>GoGo Penguin</h2>
They’re three youngish guys from Manchester.
<a href="https://gogopenguin.co.uk">Their Web site</a> says they’re an “emotive, cinematic break-beat trio”. OK
then. Piano/bass/drums is the canonical minimal jazz ensemble. Only they’re not minimal and it’s not jazz. I guess if you
redefined “jazz” as complex rhythmically-sophisticated music featuring virtuoso soloing skills, well yeah. Damn, those guys can
play. But their music is heavily composed, not a lot of opportunities for anyone to stretch out and ride the groove.
And it ain’t got that swing; can it still mean a thing?
I guess so, because I enjoyed myself. There wasn’t a microsecond that was boring, plus the
arrangements were super intelligent and kept surprising me.
But most of all, the bass. Nick Blacka hit me harder than any bassist since I saw (and
<a href="/ongoing/When/200x/2004/10/30/SlyAndRobbie">blogged</a>!) Robbie Shakespeare of
Sly and Robbie in 2004.
It’s really something special. It may be a stand-up acoustic bass, but it’s wired up so he can dominate the band’s sound when
he reaches back for it (which he does neither too little nor too much). Plus the instrument’s acoustic texture roars out
entirely unmarred, you can feel those strings and wood in your gut. He moves between bowing and plucking and banging and
you hardly even notice because it’s always the right thing.
I don’t wanna diss Chris Illingsworth on piano or Jon Scott on drums; both of them made me catch my breath. But it’s Blacka’s
bass explosions that I took home with me.
<h2 id='p-5'>That swing?</h2>
These days my musical obsessions are Americana (i.e. bluegrass with pretensions) and old blues. The first of which also
features instrumental complexity and virtuosity. And, if I’m being honest, both offer a whole lot more soul than Penguins.
I respect what they’re doing. I’ll go see them again. But I wish they’d get the hell out from behind those
diamond-bright razor-sharp arrangements and just get down sometimes.
<h2 id='p-6'>Next?</h2>
Lauren and I had real fun and left feeling a bit guilty that we’ve been ignoring Vancouver’s own jazz clubs. Not that I’m
going to stop going to metal or post-punk or baroque concerts. But jazz clubs are a good grown-up option.
</div></content></entry>
<entry>
<title>Epsilon Love</title>
<link href='https://www.tbray.org/ongoing/When/202x/2024/06/17/Epsilon-Love' />
<link rel='replies' thr:count='2' type='application/xhtml+xml' href='/ongoing/When/202x/2024/06/17/Epsilon-Love#comments' />
<id>https://www.tbray.org/ongoing/When/202x/2024/06/17/Epsilon-Love</id>
<published>2024-06-17T12:00:00-07:00</published>
<updated>2024-06-20T21:57:52-07:00</updated>
<category scheme='https://www.tbray.org/ongoing/What/' term='Technology/Quamina Diary' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Technology' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Quamina Diary' />
<summary type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'><a href='https://github.com/timbray/quamina'>Quamina</a> was for a time my favorite among all my software contributions. But then it stalled after I shipped 1.0 in January of 2023. First of all, I got busy with the <a href='/ongoing/When/202x/2023/07/24/Workin-for-the-Man'>expert witness for Uncle Sam</a> gig and second, there was a horrible problem in there that I couldn’t fix. Except for now I have! And I haven’t done much codeblogging recently. So, here are notes on nondeterministic finite automata, epsilon transitions, Ken Thompson, Golang generics, and prettyprinting. If some subset of those things interests you, you’ll probably like this</div></summary>
<content type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>
<a href="https://github.com/timbray/quamina">Quamina</a> was for a time my favorite among all my software contributions.
But then it stalled after I shipped 1.0 in January of 2023. First of all, I got busy with the
<a href="/ongoing/When/202x/2023/07/24/Workin-for-the-Man">expert witness for Uncle Sam</a> gig and second, there was a horrible
problem in there that I couldn’t fix. Except for now I have! And I haven’t done much codeblogging recently.
So, here are notes on nondeterministic finite automata, epsilon
transitions, Ken Thompson, Golang generics, and
prettyprinting. If some subset of those things interests you, you’ll probably like this.
(Warning: if you’ve already had your hands on the theory and practice of finite automata,
this may all be old hat.)
[Update: This is kind of embarrassing. It looks like what this post refers to as an “epsilon” is not the same epsilon that
features in the theory of finite automata. I mean, it still works well for where I’m using it, but I obviously need dig in
harder and deeper.]
<h2 id='p-2'>Sidebar: What’s a Quamina?</h2>
I don’t think there’s much to be gained by duplicating Quamina’s
<a href="https://github.com/timbray/quamina/blob/main/README.md">README</a> but in brief: “A
fast pattern-matching library in Go with a large and growing pattern vocabulary and no dependencies outside Go’s standard
libraries.” If you want much, much more, this
<a href="/ongoing/What/Technology/Quamina%20Diary/">Quamina Diary</a> blog series has it.
<h2 id='p-1'>The problem</h2>
Combining too many patterns with wild-cards in them caused Quamina 1.0’s data structures to explode in size with a
growth rate not far off the terrifying <code>O(2N)</code>, which meant that once you’d added much more than 20
patterns you couldn’t add any more, because the add-pattern code’s runtime was <code>O(2N)</code> too.
Those structures are state machines generally, “nondeterministic finite automata” (NFA’s) in particular.
Which offer good solutions to many software problems,
but when they get to be any size at all, are really hard to fit into a human mind. So when I was looking at Quamina’s
unreasonably-big automata and trying to figure out how they got that way, my brain was screaming
“Stop the pain!”
<h2 id='p-3'>Lesson: Prettyprint!</h2>
At the point I stalled on Quamina, I’d started a refactor based on the theory that the NFAs were huge because
of a failure to deduplicate state transitions. But the code I’d written based on that theory was utterly broken; it failed
simple unit tests and I couldn’t see why.
During the months when I was ignoring the problem, I privately despaired because I wasn’t sure I could ever crack it, and I
couldn’t stomach more struggling with ad-hoc <code>Printf</code> and debugger output. So I decided to generate
human-readable renditions of my automata. Given that, if I still couldn’t figure out
what was going on, I’d have to admit I wasn’t smart enough for this shit and walk away from the problem.
Which turned out to be a good call. Generating an information-dense but readable display was hard, and I decided
to be ruthless about getting the spaces and punctuation in the right
places. Because I didn’t want to walk away.
Back in the day, we used to call this “prettyprinting”.
It worked! First of all, my prettyprinter showed me that the automata emitted based on my deduplication
theory were just wrong, and what was wrong about them, and I found that code and fixed it.
Bad news: My deduplication theory was also just wrong.
Good news: My prettyprinter provided unavoidable proof of the wrongness
and made me go back to first principles.
And I just landed a PR that cleanly removed the state explosion.
<h2 id='p-8'>Free advice</h2>
I’ll show off the prettyprinter output below where I dig into the state-explosion fix. But for the
moment, a recommendation: If you have a data structure that’s not Working As Intended and is hard to grok, go hide for a couple
of days and write yourself a prettyprinter. Prettyprinting is an intelligence amplifier.
Your Future Self will thank you heartily.
<h2 id='p-4'>“Back to first principles”?</h2>
The single best write-up on NFA and regex
basics that I’ve ever encountered is
<a href="https://swtch.com/~rsc/regexp/regexp1.html">Regular Expression Matching Can Be Simple And Fast (but is slow in Java,
Perl, PHP, Python, Ruby, ...)</a> by
<a href="https://swtch.com/~rsc/">Russ Cox</a>. It’s a discussion of, and reflection on, the regular expression library
constructed by <a href="https://en.wikipedia.org/wiki/Ken_Thompson">Ken Thompson</a> in the mid-Sixties, before he got mixed up
in Unix.
What’s annoying is that I had read this before I started wiring NFAs into Quamina, but ignored most of its important lessons
due to a combination of not understanding them and thinking that my existing code could do what Cox described. A couple of
weeks ago I went back and read it again, and it all made perfect sense and showed me the way forward. So I guess the lesson is that
if you’re not Ken Thompson, you’re going to have trouble understanding what he did until you’ve tried and failed yourself?

So, major thanks to Ken for this (and Unix and other things too) and to Russ for the write-up.
<h2 id='p-5'>Epsilon transitions</h2>
These are the magic bullet that make NFA’s work. Quamina didn’t have them, now it does. There are other bits and
pieces but that’s the core of the thing.
I think the easiest way to explain is by showing you an NFA as displayed by Quamina’s new prettyprinter. It matches the
regular expression <code>"x.*9"</code> — note that the
<code>"</code> delimiters are part of the pattern:
<pre style="font-family: monospace; font-weight: bold; font-size:110%;line-height:150%;"> 758[START HERE] '"' → 910[on "]
910[on "] 'x' → 821[gS]
821[gS] ε → 821[gS] / '9' → 551[gX on 9]
551[gX on 9] '"' → 937[on "]
937[on "] 'ℵ' → 820[last step]
820[last step] [1 transition(s)]</pre>
<ul>
<li>There’s an API to attach labels to states as you build automata, which as a side-effect gives each a random 3-digit
number too. This is done in a way that can be turned into a no-op at production time.</li>
<li>758: The start state; the only character that does anything is the opening <code>"</code> delimiter which
transitions to state 910.</li>
<li>910: You get here when you see the <code>"</code> and the only exit is if you see an <code>x</code>, which
moves to 821.</li>
<li>821: This state is the “glob” <code>*</code> operator. <code>gS</code> in its label stands for “glob spin”. It
has an "epsilon" (<code>ε</code>) transition to itself. In Computer-Science theory, they claim that the epsilon transition
can occur at any time, spontaneously, la-di-da. In programming practice, you take an epsilon transition for every input
character. 821 also has an ordinary transition on <code>9</code> to state 551.
This possibility of having multiple transitions out of a state on the same input symbol,
and the existence of epsilon transitions, are the defining characteristics that make NFAs “nondeterministic”.</li>
<li>551: Its label includes <code>gX</code> for “glob exit”. The only transition is on the closing <code>"</code>
delimiter, to 937.</li>
<li>937 has only one transition, on <code>ℵ</code> (stands for the reserved value Quamina inserts to signal
the end of input) to 820.</li>
<li>820 doesn’t do anything, but the <code>[1 transition(s)]</code> label means that if you reach here you’ve
matched this field’s value and can transition to working on the next field.</li>
</ul>
Now I’m going to display the prettyprint again so you can look at it as you read the next paragraph.
<pre style="font-family: monospace; font-weight: bold; font-size:110%;line-height:150%;"> 758[START HERE] '"' → 910[on "]
910[on "] 'x' → 821[gS]
821[gS] ε → 821[gS] / '9' → 551[gX on 9]
551[gX on 9] '"' → 937[on "]
937[on "] 'ℵ' → 820[last step]
820[last step] [1 transition(s)]</pre>
A little thought shows how the epsilon-transition magic works. Suppose the input string is <code>"xyz909"</code>. The code
will match the leading <code>"</code> then <code>x</code> and hit state 821. When it sees <code>y</code> and <code>z</code>, the
only thing that
happens is that the epsilon transition loops back to 821 every time. When it hits the first <code>9</code>, it’ll advance to
551 but than stall out because the following character is <code>0</code> which doesn’t match the only path forward through
<code>"</code>. But the epsilon transition keeps looping and when the second <code>9</code> comes along it’ll proceed
smoothly through 551, 937, and 820, signaling a match. Yay!
So now, I have a fuzz test which adds a pattern for each of about thirteen thousand 5-letter words, with one <code>*</code>
embedded in each at a random offset, including the leading and trailing positions. The add-pattern code hardly slows down at
all. The matching code slows down a lot,
to below 10,000/second, in stark contrast to most Quamina instances, which can achieve millions of matches/second.
I’m sort of OK with this trade-off; after all, it’s matching 10K-plus patterns! I’m going to work on optimizing it, but I
have to accept that the math, as in finite-automata theory, might be against
me. But almost certainly there are some optimizations to be had. There are possibilities suggested by Cox’s description
of Thompson’s methods. And the search for paths forward will likely be good blog fodder. Yay!
<h2 id='p-6'>Ken again</h2>
When I re-read Russ Cox’s piece, I was looking at the pictures and narrative, mostly ignoring the C code. When everything
was working, I went back and was irrationally thrilled that my bottom-level function for one state traversal had the same name
as Ken Thompson’s: <code>step()</code>.
Also, when you process an NFA, you can be in multiple states at once; see the <code>"xyz909"</code> example above. When
you’re in multiple states and you process an input symbol, you might end up in zero, one, or many new states.
Russ writes, of Ken Thompson’s code, “To avoid allocating on every iteration of the loop, <code>match</code> uses two
preallocated lists <code>l1</code> and
<code>l2</code> as <code>clist</code> and <code>nlist</code>, swapping the two after each step.”
Me too! Only mine are called <code>currentStates</code> and <code>nextStates</code> because it’s 2024.
And thereby hangs a blog or maybe more than one. Because traversing the NFA is at Quamina’s white-hot center. You really
REALLY don’t want to be allocating memory in that code path. Which should be straightforward. But it’s not, for interesting
reasons that raise optimization problems I’m just starting to think about, but you’ll probably hear all about it when I do.
<h2 id='p-7'>Un-generic</h2>
In the process of moving Quamina from DFAs to mixed DFA/NFA to pure-NFA I adopted and then abandoned Go’s generics.
They hate me. Or I’m not smart enough. Or something. I
<a href="/ongoing/When/202x/2022/05/14/Golang-Generics">wrote about the experience</a> back in 2022 and while that piece ended
inconclusively, I am personally much happier with generics-free Go code. Maybe they make other people happy.
<h2 id='p-9'>Hard to understand</h2>
And then finally, there’s this one function I wrote in June 2022, doesn’t matter what it does. It has a a comment at the top that begins:
“Spookeh. The idea is that…” and goes on for a long paragraph which, well, I can’t understand. Then I look at the code and think
“that can’t work.” I keep thinking of sequences that should send it off the rails and write the unit tests and they fail to fail,
and I use the prettyprinter and the NFA it generates is ruthlessly correct. I go back and look at it every few days and end up
shaking my head. This is making me grumpy.
But after all, I did write, in a
<a href="/ongoing/When/202x/2022/05/19/Quamina-Matchers#p-7">previous Quamina Diary episode</a>: “The observation that computer
programmers can build executable abstractions that work but they then have trouble understanding is not new and not
surprising. Lots of our code is smarter than we are.”
But I’ll figure it out. And it’s nice to have interesting computer-programming stuff to blog about.
<!--
computer memory is organized in sequences of numbers
memory thrashing
-->
</div></content></entry>
<entry>
<title>Wikipedia Pain</title>
<link href='https://www.tbray.org/ongoing/When/202x/2024/06/15/Wikipedia-Pain' />
<link rel='replies' thr:count='7' type='application/xhtml+xml' href='/ongoing/When/202x/2024/06/15/Wikipedia-Pain#comments' />
<id>https://www.tbray.org/ongoing/When/202x/2024/06/15/Wikipedia-Pain</id>
<published>2024-06-15T12:00:00-07:00</published>
<updated>2024-06-16T09:37:06-07:00</updated>
<category scheme='https://www.tbray.org/ongoing/What/' term='Technology/Publishing/Reference' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Technology' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Publishing' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Reference' />
<summary type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>There are voices — some loud and well-respected — who argue that Wikipedia is deeply flawed, a hellscape of psychotic editors and contempt for expertise. I mostly disagree, but those voices deserve, at least, to be heard</div></summary>
<content type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>
There are voices — some loud and well-respected — who argue that
Wikipedia is deeply flawed, a hellscape of psychotic editors and contempt for expertise. I mostly disagree, but
those voices deserve, at least, to be heard.
[Note: There’s a companion blog post,
<a href="Edit-War">Sex Edit War!</a>, about my own experience in a Wikipedia Edit War. (I won! It was fun!)
I hope it’ll make some of this narrative more concrete.]
<h2 id='p-1'>Background</h2>
If you look at this post’s
<a href="/ongoing/What/Technology/Publishing/Reference/">Reference Publishing</a> topic, you’ll see a lot of
Wikipedia-related material. I was one of its early defenders against the early-days waves of attackers who compared it to a
<a href="/ongoing/When/200x/2004/11/16/McHenry">public toilet</a> and its editors to the Khmer Rouge.
I should also disclose that, over the years, I have made some 2,300 Wikipedia edits,
<a href="https://xtools.wmcloud.org/pages/en.wikipedia.org/TimBray">created seven articles</a>, and (what makes me happiest)
<a href="https://glamtools.toolforge.org/glamorous.php?doit=1&category=Photographs_by_Tim_Bray&use_globalusage=1&depth=3&show_details=1&projects%5Bwikipedia%5D=1&projects%5Bwikimedia%5D=1&projects%5Bwikisource%5D=1&projects%5Bwikibooks%5D=1&projects%5Bwikiquote%5D=1&projects%5Bwiktionary%5D=1&projects%5Bwikinews%5D=1&projects%5Bwikivoyage%5D=1&projects%5Bwikispecies%5D=1&projects%5Bmediawiki%5D=1&projects%5Bwikidata%5D=1&projects%5Bwikiversity%5D=1">contributed
49 images</a> which have been used, in aggregate, 228 times.
I say all this to acknowledge that I am probably predisposed to defend Wikipedia.
<h2 id='p-2'>What happened was…</h2>
Somebody spoke up on the Fediverse, saying “I wonder if reporters know that Wikipedia hallucinates too??”
I’m not giving that a link, since they followed up with a post asserting that ChatGPT is better than Wikipedia. Life’s too short
for that.
Anyhow,
<a href="https://phanpy.social/#/cosocial.ca/s/112581434393938844">I replied</a> “The difference is, errors in Wikipedia tend to get
systematically fixed. Sometimes it takes more work than it should, but the vast majority of articles are moving in the right
direction a vast majority of the time.” Much discussion ensued; follow the threads.
Shortly thereafter, the redoubtable JWZ
<a href="https://phanpy.social/#/cosocial.ca/s/112595065934183929">complained</a> about an edit to his page and
<a href="https://phanpy.social/#/cosocial.ca/s/112596132485364087">I spoke up</a> noting that the edit had been reversed, as bad
edits (in my experience) usually are. That conversation branched out vigorously, dozens of contributions. Feel free to trawl through
the Fediverse threads, but you don’t have to, I’ll summarize.
<h2 id='p-3'>Gripe: Bad editors</h2>
This kept coming back.
<ul>
<li><a href="https://phanpy.social/#/cosocial.ca/s/112596190950148931">Jamie Zawinski: I just find the culture of the editors
intolerable.</a></li>
<li><a href="https://phanpy.social/#/cosocial.ca/s/112603672166068677">Ruben Schade: I didn’t want to draw the ire of those horrible
admins.</a></li>
<li><a href="https://phanpy.social/#/cosocial.ca/s/112597984955005878">Dave Slusher: The editor culture demonstrably
contains decision makers who ain't that bright and are pretty lazy.</a></li>
</ul>
I dunno. I don’t want to gaslight those people; if that’s the experience they had, that’s the experience they had. My own
experience is different: The editors I’ve interacted with have generally been friendly and supportive, and often exceptionally
skilled at digging up quality citations. But I think that these reports are something Wikipedia should worry about.
<h2 id='p-4'>Gripe: Disrespect of expertise</h2>
By number and volume of complaints, this was the #1 issue that came up in those threads:
<ul>
<li><a href="https://phanpy.social/#/cosocial.ca/s/112596279133214378">Brian Dear: Pantheon published my book in
2017. For years I tried to add to the paltry Wikipedia entries for everything from PLATO, plasma panels, to early MUDs,
instant msging, but no: Wikipedia editors deleted it all (“not authoritative”). Screw ‘em.</a></li>
<li> <a href="https://phanpy.social/#/cosocial.ca/s/112596450752955593">Dan O’Neill: An editor argued with me about the
history of my own company.</a></li>
<li> <a href="https://phanpy.social/#/cosocial.ca/s/112597271195266197">Zawinski: Wikipedia specifically rejects domain experts
until someone who knows nothing cites them elsewhere.</a></li>
<li><a href="https://phanpy.social/#/cosocial.ca/s/112599383637496615">Alex Rosenberg: So many mistakes in early pages about PS3
that I tried to get corrected as a primary source, but some random kiddie blogger's guess as to what an acronym stood for was
more valuable to the editors.</a></li>
</ul>
I generally disagree with these takes. Wikipedia not only respects but requires expert support for its
content. However, it uses a very specific definition of “expert”: Someone who can get their assertions published in one or more
<a href="https://en.wikipedia.org/wiki/Wikipedia:Reliable_sources">Reliable Sources</a>.
I think that if you’re about to
have an opinion about Wikipedia and expertise and citations, you should give that Reliable-Sources article a careful read first.
Here’s why: It is at the white-hot center of any conversation about what Wikipedia should and should not say. Since Wikipedia is
commonly the
top result for a Web search, and since a couple of generations of students have been taught to consult but not cite it,
the article is central to what literate people consider to be true.
Let’s consider the complaints above. Mr Dear literally Wrote the Book. But, I dunno. I went and looked at the PLATO article
and subjects linked to it, and, well, it looks good to me? It cites Mr Dear’s book but just once. Maybe the editors didn’t think
Mr Dear’s book was very good? Maybe Dear says controversial things that you wouldn’t want to publish without independent
evidence? The picture is inconclusive.
As for Mr O’Neill’s complaint, no sympathy. Given the social structure of capitalism, the employees and leadership of a
company are the last people who should be considered Reliable Sources on that company. Particularly on anything that’s
remotely controversial.
Mr Zawinski is upset that the person who chooses citations from Reliable Sources “knows nothing”, which I take to be an
abbreviation for “is not a subject-matter expert”. There’s some truth here.
When it comes to bald statements of fact, you
don’t need to be an expert; If more than one quality magazine or academic journal says that the company was
incorporated in 1989, you don’t need to know anything about the company or its products to allow “founded in 1989” into an article.
On the other hand, I think we can all agree that people who make significant changes on articles
concerning complex subjects should know the turf. My impression is that, for academic subjects, that condition is
generally met.
Mr Rosenberg, once again, is upset that his personal expertise about the PS3 is being disregarded in favor of
material sourced from a gamer blog. I’d have to know the details, but the best possible outcome would be Mr Rosenberg
establishing his expertise by publishing his narrative in a Reliable Source.
<h2 id='p-8'>Bad Pattern</h2>
There’s a pattern I’ve seen a few times where a person sees something in Wikipedia in an area where they think they’re
knowledgeable and think it’s wrong and decide “I’ll just fix that.” Then their edits get bounced because they don’t include
citations. Even though they’re an “expert”. Then that person stomps away fuming publicly that Wikipedia is crap. That’s
unfortunate, and maybe Wikipedia should change its tag-line from “anyone can edit” to “anyone who’s willing to provide citations
can edit.”
<h2 id='p-6'>Implications</h2>
This policy concerning expertise has some consequences:
<ol>
<li>The decision on who is and isn’t an expert is by and large outsourced to the editorial staff of Reliable
Sources.</li>
<li>There are ferocious debates among editors about which sources are Reliable
and which are not, in the context of some specific article. Which is perfectly appropriate and necessary. For example, last time I
checked, Fox News is considered
entirely Reliable on the finer points of NFL football, but not at all on US politics.</li>
<li>There are many things which people know to be true but aren’t in Wikipedia and likely never will be, because no
Reliable Source has ever discussed the matter. For example, I created the
<a href="https://en.wikipedia.org/wiki/East_Van_Cross">East Van Cross</a> article, and subsequently learned the story
of the cross’s origin. I found it entirely convincing but it was from an guy I met at a friend’s party who was a student at the high
school where and when the graphic was first dreamed up. I looked around but found no Reliable Sources saying anything on the
subject. I doubt it’ll ever be in Wikipedia.</li>
</ol>
What do you think of those trade-offs? I think they’re pretty well OK.
The notion that anyone should be allowed to add uncited
assertions to Wikipedia because they think they’re an expert strikes me as simultaneously ridiculous and dangerous.
<h2 id='p-5'>Real problems</h2>
Obviously, Wikipedia isn’t perfect. There are two problems in particular that bother me all the time, one small, one big.
Small first: The editor culture is a thicket of acronyms and it’s hard to keep them straight. I have considered, in some future
not-too-fierce editorial debate, saying “Wait, WP:Potrezebie says you can’t say that!” Then see if anyone calls me on it.
The big problem: The community of editors is heavily male-dominated, and there have repeatedly been credible accusations of misogyny.
I have direct experience: I created the article for
<a href="https://en.wikipedia.org/wiki/Sarah_Smarsh">Sarah Smarsh</a>, because we read her excellent book
<a href="https://en.wikipedia.org/wiki/Heartland_(Smarsh_book)">Heartland</a> in my book club, then I was shocked to find no
entry. Despite the existence of that mainstream-published and well-reviewed book, and the fact that she had published in <cite>The
Guardian</cite> and
the <cite>Columbia Journalism Review</cite>, some other editor decreed that that was insufficient notability.
At the time, I
reacted by gradually accumulating more and more citations and updating the draft. Eventually she published another book and the
argument was over. These days, in that situation I would raise holy hell and escalate the obstruction up the Wikipedia
stack.
To Wikipedia’s credit, its leadership knows about this problem and gives the appearance of trying to improve it. I don’t know
the details of what they’re trying and whether they’re moving the needle at all. But it’s clearly still a problem.
<h2 id='p-9'>Once again…</h2>
I stand by what I said in December 2004: Wikipedia dwarfs its critics.
</div></content></entry>
<entry>
<title>Sex Edit War!</title>
<link href='https://www.tbray.org/ongoing/When/202x/2024/06/15/Edit-War' />
<link rel='replies' thr:count='2' type='application/xhtml+xml' href='/ongoing/When/202x/2024/06/15/Edit-War#comments' />
<id>https://www.tbray.org/ongoing/When/202x/2024/06/15/Edit-War</id>
<published>2024-06-15T12:00:00-07:00</published>
<updated>2024-06-16T09:21:40-07:00</updated>
<category scheme='https://www.tbray.org/ongoing/What/' term='Technology/Publishing/Reference' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Technology' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Publishing' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Reference' />
<summary type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>In January 2010 I drove twenty-five minutes across Vancouver to the University of British Columbia’s main library, with the goal of crushing an opponent in a Wikipedia edit war. The battleground was the entry on <a href='https://en.wikipedia.org/wiki/T._E._Lawrence'>T.E. Lawrence</a> (better known as Lawrence of Arabia). I won that war. As a consequence, I consider myself the world’s leading living expert on Lawrence’s sexuality</div></summary>
<content type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>
In January 2010 I drove twenty-five minutes across Vancouver to the University of British Columbia’s main library, with the goal
of crushing an opponent in a Wikipedia edit war. The battleground was the entry on
<a href="https://en.wikipedia.org/wiki/T._E._Lawrence">T.E. Lawrence</a> (better known as Lawrence of
Arabia). I won that war. As a consequence, I consider myself the world’s leading living expert on Lawrence’s sexuality. 
[Note: This is posted alongside
<a href="/ongoing/When/202x/2024/06/15/Wikipedia-Pain">Wikipedia Pain</a>, which is about the issues of truth and expertise in
Wikipedia editing, in an effort to share what the process feels like from the inside.]
Why Lawrence, anyhow? My
<a href="https://www.textuality.com/BillBray/">Dad</a>, an Alberta farm boy, became a Professor of Agriculture, and spent most
of his career in the Third World, much of it in Lebanon and Jordan. As a result, I spent my youth there, with plentiful
opportunities for touristing all over the Middle East, including many of the spots that appear in Lawrence’s monumental war
memoir <cite>Seven Pillars of Wisdom</cite>.
I ran across <cite>Seven Pillars</cite> in college and devoured it, from time to time thinking “I’ve been there!” While it’s full of
camel charges, train-bombings, and other Ripping Yarns, it’s a difficult book, not a light read at all. But I enjoyed it and was
left wondering who this guy was. So in the course of time I read Lawrence’s other works, some biographies (there are many) and
especially, the collected letters. 
Lawrence was an avid correspondent, sending letters almost like we do emails, multiple times most days. I suspect that a
whole lot of the Lawrence biographers got the idea by reading the letters and like me thinking “who is this guy?” You might want
to do a little Lawrence reading.
Conducting archeology on my blog reveals that I apparently noticed Wikipedia in 2003 and had started contributing to the
Lawrence article by 2004; in that year I also
<a href="/ongoing/When/200x/2004/08/31/Wikipedia">wrote</a> “Maybe the Wikipedia is a short-lived fad, maybe it’ll get better, maybe
it’ll get worse, but I was surprised that nobody pointed this out: The Wikipedia is beautiful. It’s an unexpected and
unexplainable triumph of collective creativity and of order over entropy. I hope it lasts a long time, and those who criticize
it Just Don’t Get It.”
At that time popular opinions of The Encyclopedia That Anyone Can Edit ranged from a headshaking blow-off of the idea’s obvious
craziness to active fear and hostility. British technology journalist Andrew Orlowski once referred to Wikipedians as
“Khmer Rouge in daipers” (sic). I became a partisan, wading into the ring against figures as eminent as Bob McHenry, former Editor
of the Britannica, who compared Wikipedia to a public toilet: “you can’t be sure who was there before you.” I enjoyed rolling out my
rhetorical and polemical cannon and firing back. From
<a href="/ongoing/When/200x/2004/12/06/Trustipedia">December 2004</a>: “One thing is sure: the Wikipedia
dwarfs its critics.”
It must be said that back then, the critics had a point. Those of us who waded in early often found entries about
major subjects of history or culture which were a stinking mess. Lawrence was one such; a farrago of conspiracy theories and
thinly-sourced fantasies.
<h2 id='p-1'>Sex!</h2>
In particular the section about Lawrence’s sexuality, a subject much discussed by his biographers and occasionally in the popular
press. The amount of time I’ve put into making this fact-based would probably be regarded as ridiculous by most sane people. [Would
they be wrong? -Ed.] [Pretty sure. -T.]
I have plenty of by-and-about-Lawrence books on my shelves and had read more or less every published letter, which I thought
gave me a fair claim to knowing him better as a person than your average Wikipedia editor. By dint of dogged incremental
citation-backed edits, I was making good progress by 2009 at introducing order to the chaos.
<h2 id='p-3'>Edit!</h2>
Editing Wikipedia involves regular, often intense, disputes about what should be said. These take place on the “Talk” page
that is associated with each article. For a contentious entry, such as Lawrence’s had become, the Talk page can become huge,
much larger than the entry itself.
In these disputes, the criteria that matter are “notability” and “verifiability”. To be included, a subject must be notable,
i.e. worth mentioning. When is something notable? If, and only if, there are mentions of the subject in multiple credible
mainstream sources. Further, any assertion must be verifiable, i.e. there is evidence to establish that the claims in the
material are correct. Both criteria are addressed by providing citations from
<a href="https://en.wikipedia.org/wiki/Wikipedia:Reliable_sources">Reliable Sources</a>.
On the subject of verifiability, Wikipedia says to the world: Any material that is not verifiable will eventually be
removed. That tenet gives a warm glow to those of us who live on the Internet and care a lot about truth and untruth.
The subject at hand was homosexuality. First, had Lawrence been gay? Second, what was his attitude toward gay people?
Remember, this is a man who died in 1935; in his lifetime, homosexuality was publicly much disapproved-of and in fact
specifically forbidden by law.
I thought I had the facts on my side. Whatever Lawrence’s orientation, there was no evidence of consensual intimacy with
anyone of any gender, and he repeatedly and explicitly denied, in private correspondence, any experience of sex.
On the other hand, his writing includes multiple warm, approving remarks about male/male sexual relationships. So I thought
the case for “celibate and tolerant” was pretty well open and shut.
<h2 id='p-2'>War!</h2>
But then I found I had an adversary.
“Factuarius” – the handle of another active Wikipedia editor – came to fight. For reasons opaque to me, Factuarius was
pretty convinced that Lawrence had been gay and/or disapproved of homosexuality. He was able to assemble citations where people
had alleged relationships between Lawrence and one or another male person, but this was well-plowed ground; biographers had
found an absence of evidence for the relationships and reasonably convincing reasons to doubt their having happened.
Factuarius decided that Lawrence’s having disapproved of homosexuality was the hill he was going to die on. He triumphantly
produced two citations that supported his position, declared victory, and told me to stand down.
The first was “Khondakar Golam Mowla, 2008 p. 258”. The book is <cite>The Judgment Against
Imperialism, Fascism and Racism Against Caliphate and Islam: Volume 1</cite>. You can buy it from Amazon for $36.49 as I write this. It
turns out it is self-published at “AuthorHouse” and that its Foreword denounces, among other things, “Ataturk, a secret
Jew”. The tone generally follows from there. I pointed out to Factuarius that I could go to AuthorHouse and generate a book
claiming Lawrence was from Mars.
That left him hotly defending his last reference, a Lawrence letter cited in “Homosexuality and Orientalism: Edward
Carpenter's journey to the east, P. K. Bakshi, <cite>Prose Studies</cite>, Volume 13, Issue 1 May 1990, pages 151-177,
Routledge”. Seeing no
alternative, I made that drive over to the nearest big University research library.
It took a while to track down <cite>Prose Studies</cite>, whose dusty and clearly-unvisited volumes occupy quite a few
shelf-feet. It was founded in 1977 and the Internet tells me it’s still publishing. I really don’t know what this journal is for
or what effect on
the world, if any, its existence is designed to achieve. [Arrogant, much? -Ed.] [Trying to be polite. -T.]
Sure enough, the article about
<a href="https://en.wikipedia.org/wiki/Edward_Carpenter">Edward Carpenter</a> was there in the May 1990 volume. I read it. I
photographed (badly, with a
2010 phone-cam) the title and index pages to prove that I had done so. The article mentioned Lawrence twice, suggesting in an
off-handed way that he was an example of English fascination with homosexuality and “the Orient”. But there was nothing there
that looked like Factuarius’ citation.
<h2 id='p-4'>Victory!</h2>
I was left happy for multiple reasons. It is a wonderful thing that research libraries exist and preserve academic journals
for their own sake, whether or not any human will ever consult their pages. It was pretty cool playing scholarly sleuth in the
quiet passages of the library. Best of all, Factuarius retired silently from the fray.
Which was actually a pretty minor scuffle by Wikipedia standards. There is a hilarious page entitled
<a href="https://en.wikipedia.org/wiki/Wikipedia:Lamest_edit_wars">Wikipedia:Lamest edit wars</a>, which I recommend
just for fun. It even categorizes them. The first-appearing category is
“Ethnic and national feuds”, featuring the titanic struggles over the ancestries of Frédéric Chopin and Freddie
Mercury. So far, none of these has metamorphosed into a real actual nation-against-nation shooting war, but I’m not saying it
couldn’t happen.
Eventually I took the trouble of collecting every citable fact about Lawrence’s sexuality that I could find in all the known
published resources – online search in the Gutenberg Project and various other sources helped. I published them in a blog piece
entitled
<a href="/ongoing/When/200x/2009/12/20/Sexuality-of-T-E-Lawrence">Sex and T.E. Lawrence</a>, which has been useful in subsequent
(much less dramatic) editing disagreements.
Finally, I gave a talk at a social-media conference sometime in the 2000s entitled <cite>Editing Wikipedia</cite> in which I
had great fun relating this episode, and I think the audience did too. In particular, reading out spicy passages
illustrating Lawrence’s
real kink – there’s strong evidence that he was a masochist. For example, in later life, he paid to have himself whipped
“severely enough to produce a seminal emission”.
The effect, at the end of all this was that material that was not verifiable – an assertion about a historically-notable
person’s viewpoint on a particular issue – was, as it should be, removed from Wikipedia. 
Also, pursuing the truth can be its own reward.
</div></content></entry>
<entry>
<title>Parable of the Sofa</title>
<link href='https://www.tbray.org/ongoing/When/202x/2024/06/01/Parable-of-the-Sofa' />
<link rel='replies' thr:count='8' type='application/xhtml+xml' href='/ongoing/When/202x/2024/06/01/Parable-of-the-Sofa#comments' />
<id>https://www.tbray.org/ongoing/When/202x/2024/06/01/Parable-of-the-Sofa</id>
<published>2024-06-01T12:00:00-07:00</published>
<updated>2024-06-02T08:47:35-07:00</updated>
<category scheme='https://www.tbray.org/ongoing/What/' term='The World/Economics' />
<category scheme='https://www.tbray.org/ongoing/What/' term='The World' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Economics' />
<summary type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>When Lauren was pregnant with a child who’s now turning 25, we purchased a comfy dark-brown leather sofa which fits our living room nicely. What with kids and relatives and employees and cats and Standards Comittees and friends and book clubs and socials, the butt-support cushions had, a quarter century later, worn out. So we had them replaced, at a fair price, by a small local business. Which is something that modern capitalism is trying to make impossible</div></summary>
<content type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>
When Lauren was pregnant with a child who’s now turning 25, we purchased a comfy dark-brown
leather sofa which fits our living room nicely.
What with kids and relatives and
employees and cats and Standards Comittees and friends and book clubs and socials, the butt-support cushions had, a quarter
century later, worn out. So we had them replaced, at a fair price, by a small local business. Which is something that modern
capitalism is trying to make impossible.
<img src="https://www.tbray.org/ongoing/When/202x/2024/06/01/worn.png" alt="Worn leather with a phone for scale" class="inline" />
I’ll be honest; when we realized how ratty the sofa was getting, my first thought was “crap, gonna have to buy a sofa”. But
Lauren said “No, because new sofas are junk. Also, Luxcious.”
I’ll get to Luxcious in a bit, but it turns out that new sofas, by and large, really are. Why would that be? Well, check out
<a href="https://www.dwell.com/article/dtc-sofa-crisis-32304b9e">Why Are (Most) Sofas So Bad?</a> in <cite>Dwell</cite>
magazine which has a weirdly-intermittent paywall, here’s <a href="https://archive.is/deGMX">another version</a>.
From early in the piece: “Sofas made in the past 15 years or so are absolute garbage, constructed of
sawdust compressed and bonded with cheap glue, simple brackets in place of proper joinery, substandard spring design, flimsy
foam, and a lot of staples.” It’s excellent, well-written, and will take you some surprising places.
But the subtext is
drearily familiar. Globalization: Check. Cheap-labor arbitrage: Check. Tax engineering: Check. High profits:
Check. Flat-packing: Check. Late Capitalism: Check check fucking check.
But, quality furniture is expensive to make, and
should be, but doesn’t wear out fast, thus deserves extended maintenance.
<h2 id='p-1'>Luxcious</h2>
Its
<a href="https://luxciousupholstery.ca">Web site</a> (“Breathe new life into old furniture”) is way prettier than its location,
in an old and extremely miscellaneous high-traffic zone: auto-body shops, hipster lounges, self-storage, beauty supplies…
<img src="https://www.tbray.org/ongoing/When/202x/2024/06/01/PXL_20240528_195945287.png" alt="Luxcious on Main Street" />
They’re family-run and idiosyncratic. You have to know how to find the sketchy rear parking lot and walk in the back
door. But they’re friendly and competent. Here’s the new leather they bought for the cushions.
<img src="https://www.tbray.org/ongoing/When/202x/2024/06/01/PXL_20240522_202346949.png" alt="One cow’s worth of leather" />
And here’s the sofa with the re-covered cushions in place.
<img src="https://www.tbray.org/ongoing/When/202x/2024/06/01/PXL_20240528_204400696.png" alt="Sofa with refinished cushions" />
Yes, from this angle, the new cushions make the sofa’s back look shabby, but it’s not as obvious to the naked eye and after a
decade or so we’ll never notice it.
The whole job cost us $1100 Canadian.
Given that the sofa cost three-thousand-plus 1999 dollars and new leather sofas of the “not flat-packed sawdust and
glue” variety quickly get into five figures, the choice was a no-brainer.
<h2 id='p-2'>“Lifestyle”</h2>
This kind of transaction is exactly what modern capitalism is trying to stamp out.
A single-location family-owned business that provides a living for a few people? With no plans to load up on debt or
other financial
engineering? Or for growth into unicorn status? No GenAI dimension? No marketing or public-relations people?
In conversation with venture capitalists, you hear the phrase “lifestyle business”, meaning one that is doing nicely
and rewarding the people who run it and which isn’t planning for unbounded growth. The words “lifestyle business” are always, of
course, uttered in a voice dripping with contempt. Luxcious is a lifestyle business.
It seems blindingly obvious that an economy with a higher proportion of lifestyle businesses is going to be more resilient,
more humane, and immensely more pleasant than the one that the Leaders Of Industry are trying to build.
How would we get there from here? I’m not smart enough to figure out what the regulatory regime is that would ban most of what
private-equity does and tilt the playing field in favor of resilient lifestyle businesses.
But I’d sure vote for a political party that convinced me it was trying to achieve that.
</div></content></entry>
<entry>
<title>Tedeschi Trucks</title>
<link href='https://www.tbray.org/ongoing/When/202x/2024/05/26/TTB' />
<link rel='replies' thr:count='5' type='application/xhtml+xml' href='/ongoing/When/202x/2024/05/26/TTB#comments' />
<id>https://www.tbray.org/ongoing/When/202x/2024/05/26/TTB</id>
<published>2024-05-26T12:00:00-07:00</published>
<updated>2024-05-26T12:00:00-07:00</updated>
<category scheme='https://www.tbray.org/ongoing/What/' term='Arts/Music' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Arts' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Music' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Technology/Audio' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Technology' />
<category scheme='https://www.tbray.org/ongoing/What/' term='Audio' />
<summary type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>Saturday night we went a concert by the <a href='https://www.tedeschitrucksband.com'>Tedeschi Trucks Band</a> (TTB). It was excellent and this is partly a review, but mostly a challenge to the community of touring musicians: “Why aren’t your production values as good as TTB’s?”</div></summary>
<content type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>
Saturday night we went a concert by the
<a href="https://www.tedeschitrucksband.com">Tedeschi Trucks Band</a> (TTB). It was excellent and this is partly a review,
but mostly a challenge to the community of touring musicians: “Why aren’t your production
values as good as TTB’s?”
<h2 id='p-1'>Just the Facts</h2>
TTB lives squarely in the middle of the Southern Rock genre, as invented by the Allman Brothers in 1970 or so. Derek Trucks
is the nephew of the Allmans’ original drummer Butch Trucks and performed in a later iteration of that band.
Susan Tedeschi had a successful career as a touring and recording blueswoman. Then she and Derek got married and merged their
acts.
<img src="https://www.tbray.org/ongoing/When/202x/2024/05/26/PXL_20240526_041330763.png" alt="Tedeschi Trucks Band in concert in Vancouver in 2024" />
It’s a twelve-piece band: Susan and Derek on guitar, three backup vocalists, three horns,
keyboards, bass, and two drummers (one white, one black, per the Southern-Rock canon). The music is blues and soul, wandering into
rock. Some of the songs are their own, others genre chestnuts (<cite>Statesboro Blues</cite>, <cite>High Time We
Went</cite>). They played a three-hour show, but with not that many songs, because every tune features extended
instrumental sections. All twelve members got a chance to shine, Derek had a break on every song, and Susan on quite a few.
<h2 id='p-2'>What was great</h2>
Only a couple of the songs weren’t memorable; they write well and cover only the best chestnuts. The musicianship was
stellar, with electric guitar front and center. Derek is fluid and effortless, with beautiful tone; Susan solos less but
actually plays more interesting stuff. Susan’s the lead voice but four other members are singers, they all got a featured
spot and were all pretty great. Susan doesn’t have the vocal range or the shriek, but she had the most soul.
What was best, though — out into “fucking awesome” territory — was what
classical musicians call “ensemble” and I guess I’d call “band musicianship”. The songs’ arrangements are just razor-sharp,
full of shifts and and breaks and little moments of drama and grace, intros and outros and bridges. The players were effortlessly
locked onto the center of the rhythm, “so tight they were loose” as the saying goes. The amount of practicing this takes must be
epic.
Which was brilliantly supported by the sound people. Every instrument and voice was distinct and clear, and the dynamic
range was maybe the best I’ve ever heard from an electric-guitar-based band. Every moment was multilayered and you could hear
all the layers.
You could tell (well, if you know something about
concert sound, you could) that, at the soundboard, they were intervening judiciously, for example cranking the horns (or backup
singers) and fading the guitars when that’s what the song needed.
It was an audience that was fun to be part of, enthusiastically cheering all the solos and regularly leaping to their feet
when a song hit the big up-curve. Especially impressive given that plenty of the crowd was old enough to have been there for the
birth of Southern Rock.
On top of which, the lighting was subtle and dramatic and tasteful, and only once in the whole three-hour show did they
hurt my brain by obnoxiously flashing brilliant lights in my eyes.
Thus my challenge:
<h2 id='p-3'>To every touring band: Be like TTB!</h2>
Seriously; My time on earth covers most of the history of live electric-music performance, plus I’m an audiophile, and for most of
my life, most of the sound has been shitty. But in the last few years I’ve
regularly heard sound that was better than acceptable, and occasionally dazzlingly-good. But TTB is the most impressive
combination I’ve heard of big ensemble, plenty of electric guitar, and sparkling sound.
There is sort of an excuse: Rock, historically, has been carefully engineered to sound good on car radios; specifically
the kind of car radios owned by impecunious youth. Dynamic range and layering are not features of this landscape.
Anyhow, my exposure to TTB, prior to this, has been mostly YouTube, and I’ve enjoyed them, but I dunno, now that I’ve heard
the real thing, I suspect the online version will feel thin.
If TTB can do it, any band can. But plenty still don’t. That’s increasingly just not acceptable. I wonder if things will
start to get generally better? Because I’m pretty sure the musicians care.
<h2 id='p-4'>Other observations</h2>
Running a 12-piece operation must be freaking expensive. I would love to hear the details of the economics. Saturday
night they filled a 2600-seat hall with an average ticket price around C$120. So that’s over C$300K gross. The hall
<a href="https://vancouvercivictheatres.com/media/9106/commercial-rental-rates-2024-25.pdf">costs C$21K</a> and then there’s
Ticketmaster’s cut, which if the claims of the
<a href="https://www.cbc.ca/news/entertainment/doj-sues-live-nation-1.7212432">recent DOJ litigation</a> are to be believed,
would be egregious.
I wonder how a TTB song gets built? In particular, who does the arrangements? Whoever it is, I’m a fan.
Lauren and I were masked (N95) and looking across the audience as far as we could see revealed one other masked person. I
dunno, 2600 people in an enclosed space. Call me crazy, but… no, call them crazy. I’m serious.
Unusually, there were huge line-ups for the men’s washrooms, almost none for the women’s. The lady in the row behind us
cackled and said “boomer prostates.”
</div></content></entry>
<entry>
<title>The Colors of Racism</title>
<link href='https://www.tbray.org/ongoing/When/202x/2024/05/17/Colors-of-Racism' />
<link rel='replies' thr:count='2' type='application/xhtml+xml' href='/ongoing/When/202x/2024/05/17/Colors-of-Racism#comments' />
<id>https://www.tbray.org/ongoing/When/202x/2024/05/17/Colors-of-Racism</id>
<published>2024-05-17T12:00:00-07:00</published>
<updated>2024-05-25T14:36:42-07:00</updated>
<category scheme='https://www.tbray.org/ongoing/What/' term='The World' />
<category scheme='https://www.tbray.org/ongoing/What/' term='The World' />
<summary type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>Recently, somewhat by accident, I stumbled into reading a couple of monstrously racist texts, and I’m going to need to update the Wikipedia entry for a famous British author. But I learned a few things along the way that I want to share</div></summary>
<content type='xhtml'><div xmlns='http://www.w3.org/1999/xhtml'>
Recently, somewhat by accident, I stumbled into reading a couple of monstrously racist texts, and I’m going to need to update
the Wikipedia entry for a famous British author. But I learned
a few things along the way that I want to share.
<h2 id='p-5'>Disclosure</h2>
I try to be antiracist, but I don’t think I’m particularly good at it. I sometimes have bigoted feelings but try hard to
recognize and not act on them. I’m convinced that humans are naturally tribal and antiracist work will continue to be required
for the foreseeable future.
<h2 id='p-1'>The Author</h2>
<a href="https://en.wikipedia.org/wiki/Anthony_Trollope">Anthony Trollope</a> (1815-1882) wrote 47 novels. I generally like
them and we own a whole shelf-full. They are funny and tender and cynical; his characters love and marry and go into
business and get elected to Parliament and are corrupt and engage in furious professional conflict. Those characters are,
without exception, “gentle”, by which I mean members of the British ruling class.
<img src="https://www.tbray.org/ongoing/When/202x/2024/05/17/Trollope.png" alt="Anthony Trollope in 1864" />
<div class='caption'>Anthony Trollope in 1864.</div>
When I was traveling around the world a lot, regularly crossing major oceans before the era of in-air Internet, Trollope was
a regular companion; his books tend to be big and thick and extremely readable. Want to get started?
<cite>Barchester Towers</cite>, about a bitter feud among the clergymen of an English country town, is one of the funniest books
ever written; also there’s an
<a href="https://en.wikipedia.org/wiki/The_Barchester_Chronicles">excellent BBC adaptation</a>, with Alan Rickman
deliciously smarmy as the horrid Mr Slope.
<h2 id='p-2'>What happened was…</h2>
I’m on a publishing-oriented mailing list and someone wrote “I stumbled on the fact that Trollope
wrote a book that describes race relations in the British West Indies” and someone wrote back “It’s a travelogue not a novel,
it’s called <cite>The West Indies and the Spanish Main</cite>, and be careful, that race-relations stuff may not be pleasant to
read.” On a whim, I summoned up the book from our excellent public-library system and, oh my goodness gracious, that “not
pleasant” was understating it.
<h2 id='p-3'>The book</h2>
Trollope earned his living, while he was establishing his literary career, as an official of the British Post Office, rising to
a high level in the organization and not leaving it until he was almost 50.
In 1859, he was sent to
reorganize the Post Office arrangements in the West Indies and the “Spanish Main”, the latter meaning southern Central America and
northern South America. The expedition lasted several months and yielded this book.
In his autobiography, Trollope wrote that he thought it “the best book which has come from my pen.” I think history
would disagree. It’s on the Internet Archive, but I’m not linking to explicit racism.
So why am I going to write about it?! Because now, 165 years after this book, racism and its consequences remain a
central focus of our cultural struggles. Understanding the forces we combat is kind of important.
Also, I recently researched and wrote about the
<a href="/ongoing/When/202x/2022/06/12/Demerara-Rebellion">Demerara Rebellion</a> (of the enslaved against their oppressors, in
1823) so I have more context on Trollope’s observations than most.
<h2 id='p-4'>Background</h2>
Trollope’s tone is grumpy but good-humored. In the places he visits, he is
generally contemptuous of the hotels, the food, the weather, and the local government.
The main narrative starts in Jamaica. By way of background,
slavery had been abolished in 1833, just 25 years before. Many of the sugar plantations that occupied most of
Jamaica had collapsed. Thus this:
<blockquote>
By far the greater portion of the island is covered with wild wood and jungle… Through this, on an occasional favourable
spot, and very frequently on the roadsides, one see the gardens or provision-grounds of the negroes…
These provision-grounds are very picturesque. They are not filled, as a peasant’s garden in England or in Ireland is
filled, with potatoes and cabbages, or other vegetables similarly uninteresting in their growth; but contain cocoa-trees,
breadfruit-trees, oranges, mangoes, limes, plantains, jack frout, sour-sop, avocado pears, and a score of others, all of which
are luxuriant trees, some of considerable size, and all of them of great beauty… In addition to this, they always have the
yam, which is with the negro somewhat as the potato is with the Irishman; only that the Irishman has nothing else, whereas the
negro generally has either fish or meat, and has also a score of other fruits beside the yam.
</blockquote>
We wouldn’t use that word any more to describe Black people, but it was thought courteous in Trollope’s day. He does
deploy the N-word, albeit rarely, and clarifying that it was normally seen, even back then, as an insult.
<h2 id='p-6'>The bad stuff</h2>
It comes on fast. In the Jamaica chapter, the first few subheadings are: “Introduction”, “Town”, “Country”,
“Black Men”, “Coloured Men”, and “White Men”. That “Black Men” chapter begins with six or so pages of pure racist dogma
about the supposed shortcomings of Black people. I will not enumerate them, and obviously none
stand up to the cold light of scientific inquiry.
But then it gets a little weird. Trollope notes that “The first desire of a man in a state of a civilization is for property…
Without a desire for property, man could make no progress.” And he is harsh in his criticism of the Black population for
declining to work long shifts on the sugar plantations in hopes of building up some capital and getting ahead.
And yet Trollope is forced to acknowledge that his position is weak. He describes an episode of a Black laborer
knocking off work early and being abused by an overseer, saying he’ll starve. The laborer replies “No massa; no
starve now; God send plenty yam.” Trollope muses “And who can blame the black man? He is free to work or free to let it alone.”
It is amusingly obvious that this is causing him extreme cognitive dissonance.
And he seems shockingly oblivious to issues of labor economics. On another occasion it is a group of young women who are
declining the hot nasty work in the cane fields:
<blockquote>
On the morning of my visit they were lying with their hoes beside them…
The planter was with me, and they instantly attacked him. “No, massa; we no workey; money no nuff,” said one. “Four bits
no pay! no pay at all!” said another. “Five bits, massa, and we gin morrow ’arly.” It is hardly necessary to say that the
gentleman refused to bargain with them… “But will they not look elsewhere for other work?” I asked. “Of course they will,” he
said; “… but others cannot pay better than I do.”
</blockquote>
(A “bit” was one eighth of a dollar; I can remember my grandfather referring to a quarter, i.e. a 25¢ coin,
as “two bits”.)
They’re demanding a 20% raise and, as is very common today, the employer deems that impossible.
Trollope contrasts the situation in Barbados, where there is no spare land and thus no “provision grounds” and the working
class (in this case, all-Black) is forced to labor diligently for their daily bread; and is confident that this is better.
He
also visits Cuba, where slavery is still legal, and visits a plantation with an enslaved workforce: “During the crop time … from
November till May, the negroes sleep during six hours out of the twenty-four, have two for their meals, and work for sixteen! No
difference is made on Sunday.” Trollope’s biggest concern was that the enslaved received no religious instruction nor
opportunities to worship.
Trollope regularly also has to wrestle with the tension that arises when he meets an accomplished or wise or influential Black
person. For example, upon arriving in
<a href="https://en.wikipedia.org/wiki/New_Amsterdam,_Guyana">New Amsterdam</a> (in Demerara):
<blockquote>
At ten o’clock I found myself at
the hotel, and pronounce it to be, without hesitation, the best inn, not only in that colony, but in any of these Western
colonies belonging to Great Britain. It is kept by a negro, one Mr. Paris Brittain, of whom I was informed that he was once a
slave… he is merely the exception which proves the rule.
</blockquote>
Here are two more samples of Trollope twisting himself in knots over what seems to him an economic mystery.
<blockquote>
But if the unfortunate labourers could be made to work, say four days a week, and on an average eight hours a day, would
not that in itself be an advantage ? In our happy England, men are not slaves ; but the competition of the labour market
forces upon them long days of continual labour. In our own country, ten hours of toil, repeated six days a week, for the
majority of us will barely produce the necessaries of life. It is quite right that we should love the negroes ; but I cannot
understand that we ought to love them better than ourselves.
</blockquote>
<blockquote>
The complaint generally resolves itself to this, that free labour in Jamaica cannot be commanded; that it cannot
be had always, and up to a certain given quantity at a certain moment ; that labour is scarce, and therefore high priced, and
that labour being high priced, a negro can live on half a day's wages, and will not therefore work the whole day — will not
always work any part of the day at all, seeing that his yams, his breadfruit, and his plantains are ready to his
hands. 
</blockquote>
In what sense is England “happy”? Granted, it’s obvious from the point of view of the “gentle” ruling class, none of whom are
doing manual labour sixty hours per week.
That aside, the question he raises still stands, two centuries later: Why should anyone work harder than they need to, when
the benefits of that work go to someone else?
<h2 id='p-7'>“Coloured”</h2>
There’s lots more of this, but it’s worth saying that while Trollope was racist against Blacks, he was, oddly, not a white
supremacist. He considers the all-white colonial ruling class to be pretty useless, no better than the Blacks he sneers
at, and proclaims that the future belongs to the “coloured” (i.e. mixed-race) people. He backs this up with some weird “Race
Science” that I won’t go into.
<h2 id='p-8'>Unforgivable</h2>
Trollope’s one episode of pure venom is directed at the already-dying-out Indigenous people of the region, pointing out
with approval that one of the island territories had simply deported that whole population, and suggesting that “we get rid
of them altogether.” This seems not to be based on race but on the observation that they “more than once endeavoured to turn
out their British masters”. Colonialism is right behind racism in the line-up of European bad behaviors.
It may also be relevant that he apparently did not meet a single Indigenous West-Indian person.
<h2 id='p-9'>Meta-Trollope</h2>
I finished reading <cite>The West Indies and the Spanish Main</cite> because Trollope’s portrayals of what he saw were so
vivid and I couldn’t help being interested.
I had read Trollope’s autobiography and some more
bits and pieces about him, and had encountered not a word to the effect that whatever his virtues and accomplishments, he was
shockingly racist. So I checked a couple of biographies out of the local library and yep, hardly a mention. One author noted that
<cite>The West Indies and the Spanish Main</cite> was out of tune with today’s opinions, but there was no serious discussion of
the issue. Wikipedia had nothing, and still doesn’t as I write this, but I plan to fix that.
I dug a little harder here and there around the Internet and turned up nothing about anti-Black racism, but a cluster of
pieces addressing antisemitism; see
<a href="https://slantbooks.org/close-reading/essays/troubled-by-trollope/">Troubled by Trollope?</a> and
<a href="https://www.tabletmag.com/sections/arts-letters/articles/anthony-trollope-bicentennial">Why Anthony Trollope Is the
Most Jewish of the Great English Novelists</a>.
There are a few Jews in Trollope’s novels, ranging from wholly-admirable heroes (and heroines) to revolting
villains. So you might think he comes off reasonably well, were it not for casual splashes of antisemitic tropes; the usual crap
I’m not going to repeat here.
In case it’s not obvious, Trollope’s writings and opinions were strikingly self-inconsistent,
often within the course of a few pages. Well, and so is racism itself.
At that point in history there was an entire absence of intersectionalist discourse about racism being, you
know, intrinsically bad, and there were many who engaged in it enthusiastically and sincerely while remaining in polite
society.
Trollope’s racism is undeniable, but then he (once again, inconsistently) sounds non-racist in theory. (However, he was
gloomy about the attitudes of the white population.) Check this out:
<blockquote>
It seems to us natural that white men should hold ascendency over those who are black or coloured. Although we have
emancipated our slaves, and done so much to abolish slavery elsewhere, nevertheless we regard the negro as born to be a
servant.
We do not realize it to ourselves that it is his right to share with us the high places of the world, and that it should be an
affair of individual merit whether we wait on his beck or he on ours. We have never yet brought ourselves so to think, and
probably never shall.
</blockquote>
That text feels remarkably modern to me. I am a little more optimistic than he is in his closing four words; some white
people work hard at antiracism. But for a lot of white people, his take remains depressingly accurate.
<h2 id='p-10'>Degrees of racism?</h2>
I suspect that, if Trollope were with us today, his writings would probably be conventionally antiracist.
His opinions were solidly in his era’s mainstream and I suspect he would find himself in ours, because he was really a pretty
conventional and actually kind of boring person.
With the single exception of those two sentences about the Indigenous people, he seems to exhibit no
particular emotional bias against any ethnic group.
Why, you might wonder, do I mention this? Therein lies a tale. In his autobiography, when he discusses <cite>The West Indies
and the Spanish Main</cite>, he notes that it received a favorable review in <cite>The Times</cite> of London. I thought I’d
like, for the sake of context, to read that. (Thanks to
<a href="https://cosocial.ca/@wdenton">William Denton</a> for retrieving the page images.)
I certainly didn’t enjoy reading <cite>The West Indies</cite> (unsigned) from early 1860 in <cite>The
Times</cite>.
It fills most of a broadsheet page, dozens of column-inches one after the other oozing vitriolic hate of
Black people. I’m not going to even try to describe it any further; I felt literally nauseated in reading and didn’t make it
through to the end.
I suspect that if that <cite>Times</cite> writer were with us today, he’d be an unreconstructed alt-right dog-whistler, a good ole
boy in a MAGA hat.
Reading this crap made me feel a little less angry about Trollope, who generally liked
people. Here’s what I think I learned: Racism comes in multiple flavors.
There are some people (like Trollope) who are intersectionally bigoted in a sort of unthinking and
incurious way, but not that emotionally bound to it. These are the people that need to hear the antiracist message, loudly and
clearly, over and over. Because they might listen and learn.
Then there are the others. In 1860, that <cite>Times</cite> reviewer. Today, the slave-state GOP MAGAs, the Israeli
settler movement, Modi’s Hindutva hoodlums. They genuinely hate The Other, down in their bellies. It’s how they define
themselves. Talking to them is useless. They have to be defeated and removed from positions of power and influence. Then,
thankfully, they can be ignored. Because listening to them is useless too.
</div></content></entry>
</feed>

If you would like to create a banner that links to this page (i.e. this validation result), do the following:

Download the "valid Atom 1.0" banner.
Upload the image to your own server. (This step is important. Please do not link directly to the image on this server.)
Add this HTML to your page (change the image src attribute if necessary):

<a href="http://www.feedvalidator.org/check.cgi?url=http%3A//www.tbray.org/ongoing/ongoing.atom"><img src="valid-atom.png" alt="[Valid Atom 1.0]" title="Validate my Atom 1.0 feed" /></a>

If you would like to create a text link instead, here is the URL you can use:

http://www.feedvalidator.org/check.cgi?url=http%3A//www.tbray.org/ongoing/ongoing.atom

Home · About · News · Docs · Terms