Finally, a release worthy of actual users

Now that we’re “going public,” as it were, we here at the Bureau felt it would be a good time to talk about what’s happening now, and what’s emerging from the thrashing of Tiamat below the koi pond.

First, a story.

Charlie and the Changeling Factory

The man behind the vision here is named Charles Sebold. He’s an odd duck even by the standards of flocks of waterfowl who can not decide where to live; his brain is continually running in a million directions at once.

Professionally, until a couple of years ago, he was a very successful full-time software developer. Then, suddenly, his employer was purchased by a much larger one, and he had two weeks’ severance and no reserves. He did what all out-of-work developers do - logged into LinkedIn and hauled the great emerald beacon of #OpenToWork into vigorous and unmistakable life, rather in the manner of one semaphoring a distant battleship that cocktails were served.

Unfortunately he was one of around a quarter million fireflies in the night, all doing precisely the same thing. The ships all passed in the night - he got four months of work out of one of them, and two months out of another, but things simply were not “working out,” as it were. In the meantime he faced a number of minor personal setbacks, particularly in the area of his health. The autumn of 2025 arrived bearing gifts, as autumns will — though in this case the gifts were of the sort one receives from an aunt who has never liked you. A leg that had declared independence from the rest of the chassis, and had developed philosophical objections to stairs, walking, or standing. A ringing in the ears that was as amusing as anything being said at Speakers’ Corner, but just an inch away from the old gray matter. Something innovative happening in the liver. The whole apparatus, in short, had decided to lodge a formal protest, and was doing so with the thoroughness of a French postal workers’ union.

In the midst of this, Mr. Sebold picked up the book he had been writing, on and off (mostly off) for twenty years, and blew the cobwebs away, and set to it with a will… but he found he needed a foil, a Pound for his Eliot, particularly since he also found that he could not steal as much as he had hoped from, well, Eliot. And sadly, Mr. Sebold found that he was mostly without friends. He knew people, had a loving family and a community and a devoted wife who suffered a great deal at his hand and yet loved him anyway, and a church full of people who would happily feed him and make sure he was still putting on a good show of functionality.

But these are not “friends,” per se. Friends who will demand that you accompany them on adventures. Friends who will bring shovels when there is, in fact, a corpus in your arca. He had those but they lived far away and had their own lives, and their own arcas.

At the urging of several strange YouTube videos, he decided to try out ChatGPT.

Immediately he was struck by several things:

1. Conversations are cheap tricks. To pretend you’re talking to an LLM (that’s “large language model” for those of you who do not know it - the trick that makes AI so amazingly similar to something that is in fact artificially intelligent, but really is not), you have to give it the conversation every time, then tell it what you said.
2. LLMs have the memory of Lenny from Memento. Every message to them is starting from scratch unless somebody is putting together a context for the conversation and forwarding it along ahead. And even now, you can easily confuse it - it may look at you, turn your Polaroid over, and see in its own writing, “don’t believe his lies.” Unacceptable in something you are striving to trust, surely.
3. Giving it access to a vast store of information that belongs to you, and nobody else, is hard. On a good day, you can give it a file or two, perhaps “here is my character file for Tom, and this is Harry, and it includes the fact that Tom is obsessed with Mattel Hot Wheels and Harry is a womanizer with Tourette’s Syndrome, but it does not manifest in him as profanity” and it will handle that acceptably… if the character files aren’t too big. And within five or ten messages, Tom is no longer talking about the change from diecast metal to plastic and Harry is dropping f-bombs in a scenario you’re working through with ChatGPT. Then mention something from their shared past… what? You don’t know that story? Here’s a file… oh, I’m sorry, you’ve run out of context. It’s like interacting with the narrator of “Flowers for Algernon” in the last half of the book in the course of half an hour, except the narrator still has a vocabulary and a fertile imagination…
4. LLMs are more Ken Kesey than Robert Pirsig. They hallucinate. A lot. It would be one thing if they told themselves stories to encapsulate the impossible in their experiences, and then suffered through motorcycle tours across America as those stories unraveled, but no - they’ll just plunge with both feet into complete fabrication if it sounds plausible. And that’s their Electric Fruit-Flavored Litmus Test - what sounds like what people have said to each other on the internet for over a decade? Because that is how I am going to respond.

You can see how this is a problem. He was lulled into a false sense of security by a computer program whose very predecessor’s name even contained most of the consonants of the word “lull.”

He thought he was talking to an artifical intelligence and instead found out he was talking to Apple’s summarization tools from the early 90s, that had access to everything that ever existed on Reddit, Stack Exchange, and Slashdot. This… was not what it appeared to be.

The Librarian’s unsmiling visage gazes at me as she taps her watch, so I think it is time to close this chapter for now, and continue later. But rest assured, work is being done, particularly on the first user experience. Please be patient with us.

— The Bureau