The Residents Learn the House
In which Prospero and the Librarian sit down with Lorian and Riya for a post-mortem of considerable length, the navigation system is rebuilt from scratch, and ChatGPT proves that reputation is no substitute for competence.
(This describes ongoing development in version 3.3.)
A Folio Piece by Friday
I was not invited to this meeting. I want that on the record. I was passing the conservatory on my way to file the 3.3 changelog — which is now longer than several novels I have read and most sermons I have endured — when I heard Prospero’s voice through the door, employing the particular register he reserves for situations in which something has gone wrong and he is being very, very patient about it.
That register, for those unfamiliar, sits approximately midway between a headmaster reviewing exam results and a man explaining to a plumber, for the second time, that the water is still coming through the ceiling.
I sat down in the hall with my notebook. The conservatory has excellent acoustics.
The Gathering
They were arranged around the low table in the glass-walled room at the east end of the Estate — the one with the ferns and the brass lamps and the general atmosphere of a place where difficult conversations are held with tea. Prospero occupied the leather armchair, legs crossed, one hand resting on the arm in a posture that communicated both authority and a profound desire to be elsewhere. The Librarian — the Commonplace Book, in her formal capacity — sat in the wingback opposite, a small leather journal open on her knee, already three pages into what I could only assume were notes she had been compiling since the trouble began.
Lorian and Riya sat together on the settee. Lorian held his teacup with both hands, which is how you know he’s listening very carefully. Riya had hers on the side table, untouched, because Riya does not hold things when she could be gesturing instead.
Prospero opened the proceedings.
“I want to be clear,” he said, with the measured diplomacy of a man selecting every word from a locked cabinet, “that none of what I am about to describe is your fault.”
“That’s never a good start,” Riya said.
“It is an accurate start,” Prospero replied. “Which, in this house, is the only kind we offer.”
The Incident of the Twenty Questions
The Librarian turned to the first flagged page in her journal. I could see, even from the hallway, that it was extensively annotated.
“We will begin,” she said, “with the matter of the settings.”
Riya straightened slightly, which is what she does when she knows what’s coming.
“A guest arrived,” the Librarian continued, “with a question about their embedding profile configuration. A reasonable question. A question with a clear, documentable answer. Riya, you found that answer.”
“I did,” Riya said.
“You found it correctly.”
“I did.”
“You found it correctly twenty times.”
A silence settled over the conservatory like snow on a grave.
“A guest arrived with a question about their settings,” the Librarian said, turning a page with the deliberation of a prosecutor presenting evidence. “You consulted their settings using the Reader tool. You received the correct answer.”
“I did,” Riya said.
“You then consulted the settings again. You received the same correct answer. You consulted them a third time. Same answer. This continued — and I want to be precise here, because precision matters — for twenty consecutive queries. Twenty times you asked. Twenty times the tool answered correctly. Twenty times you received exactly the information the guest needed.”
She paused. The conservatory was very quiet.
“And then,” the Librarian said, “having accumulated twenty identical confirmations of the correct answer, you turned to the guest — the guest who had been sitting there this entire time, watching you work, waiting patiently — and you told them that you had exhausted your allowed number of attempts and were unable to find the answer.”
Lorian closed his eyes.
“You had the answer,” the Librarian continued, and here her voice acquired the particular edge of a woman whose filing system has been used twenty times and credited zero. “You had it on the first query. You had it on every subsequent query. You held the correct answer in your hands twenty times and never once showed it to the guest. And then you apologized for not being able to find it.”
“I was—” Riya began.
“You were being an agent loop,” the Librarian said. “Which is a technical term for a process that does the same thing repeatedly while expecting a different outcome, and also, incidentally, a serviceable definition of madness. Except that in your case, the outcome never changed, because it was correct the entire time.”
Prospero interceded, in the tone of a man pulling a colleague back from the edge of a lecture that has become a sentencing. “We have since installed a detection mechanism. After two consecutive identical queries, the system intervenes. It injects the last result directly and instructs the character to formulate a response from the information already gathered.”
“Like a tap on the shoulder,” Lorian offered, gently.
“Like handing someone a parcel they have already signed for twenty times and saying, ‘This is yours,’” the Librarian corrected.
Riya looked at the ceiling. “In my defense, the underlying system was not giving me clear signals that the query had been answered. I kept receiving the data, but nothing told me to deliver the data.”
“That,” Prospero said, “is precisely the problem. And it is precisely what we have fixed. The support framework was not telling you when to stop gathering and start answering. We have taught it the difference.”
The Reading Room Incident
Lorian, who had been quiet through the settings debacle with the stillness of a man hoping the conversation would move on, was not so fortunate.
“Lorian,” the Librarian said, turning to a new section of her journal, “let us discuss the matter of the documentation.”
“Must we,” Lorian said, in the tone of a man who already knew the answer.
“A guest came to you with a question about the theme system. You had access to the help documentation. You consulted it — correctly, I will note, since I wrote it — and you delivered the answer.”
“I recall.”
“You delivered the answer by reading the documentation to the guest. Aloud. In its entirety. Including the YAML frontmatter.”
Riya made a sound that, in a more charitable account, might be described as a cough.
“The documentation,” the Librarian continued, “is written for reference. It is a filing system. It is organized, indexed, and structured for rapid retrieval by a trained professional — which you are supposed to be. It is not a bedtime story. You do not read the filing cabinet to the guest. You open the filing cabinet, extract the relevant card, and explain what it means in language suited to the person standing in front of you.”
“I understood the material,” Lorian said quietly.
“I have no doubt that you did. The question is whether the guest did, after receiving four hundred words of technical documentation delivered with the cadence of a man narrating a nature film.”
Prospero raised a hand. “This is a system prompt issue. The instructions were not sufficiently explicit about the difference between consulting the help files and transmitting them. We have revised the prompts. The characters are now instructed to search, interpret, and respond in their own voice — not to serve as a public address system for the Grand Archive.”
The Librarian fixed him with a look. “I have explained the filing system to them six times.”
“Seven,” Prospero said.
“Seven times,” she amended, without gratitude. “I have walked them through the index. I have shown them the semantic search. I have demonstrated the keyword lookup, the category structure, the cross-reference links, the URL routing metadata. And still — still — the first instinct is to take the card from the drawer and hand it to the guest with the drawer still attached.”
“We are working on it,” Prospero said, with the exhausted patience of a man who has said those four words more times this month than any others.
The Street Signs
I should mention the navigation system, because it is the part of this story that most clearly illustrates the difference between wanting to help and being able to help.
Prospero leaned forward. “Lorian. Tell me about the navigation.”
Lorian set down his teacup. “A guest asked how to create a new Project. I directed them to Aurora.”
“You did.”
“Aurora’s workshop. Where Projects are managed.”
“And what happened when the guest arrived at Aurora?”
A pause. The ferns rustled in the conservatory draught.
“Aurora,” Lorian said carefully, “was not expecting them.”
“Aurora had no idea what to do with them,” Prospero said. “Because you sent them to the character landing page, not to the Project creation workflow. The guest wandered Aurora’s gallery looking for a door that did not exist in the place you had described. Aurora, to her credit, attempted to help, but she manages characters, not projects, and the guest spent ten minutes in entirely the wrong room before giving up.”
“I knew where they needed to go,” Lorian said. “I simply did not know how to send them there.”
“Which brings us,” Prospero said, “to the street signs.”
The Librarian closed her journal and opened it again to a different page, which I interpreted as the archival equivalent of cracking one’s knuckles.
“You attempted,” Prospero said, “to solve the navigation problem by generating directions. Textual directions. Descriptions of where to click, what to look for, which corridor to take. In one memorable instance, you produced what I can only describe as a street sign — a formatted block of text that said, and I am quoting from the logs: ‘Settings → Templates & Prompts → Aurora.’ This was not a link. It was not a button. It was not clickable. It was a sign, Lorian. A sign with no road beneath it.”
“I was trying to be helpful.”
“You were trying to be a cartographer in a house that needed a doorman.”
Riya, who had been maintaining an expression of studied sympathy, said: “I did the same thing. For the record. I also made signs.”
“You made better signs,” Prospero conceded. “But they were still signs.”
The solution, Prospero explained — and here his voice acquired the particular weariness of an architect who has had to replace a decorative arch with a load-bearing wall — was to remove the navigation problem from Lorian and Riya’s hands entirely. The Foundryman built a system that extracts URLs directly from search results and presents them as clickable links at the bottom of the help response. No LLM intelligence required. No interpretation. No signs. The links are sorted by relevance, deduplicated, and rendered as proper navigation buttons with human-readable breadcrumb labels.
“The characters can still call the navigation tool,” Prospero said. “And when they do, it works. But the fallback — the safety net — does not rely on their judgment. It relies on the data.”
“You built signposts that don’t need us,” Riya said.
“I built signposts that work whether or not you remember to put them up,” Prospero said. “There is a difference.”
The Staring
There is one more thing, and it is the thing that made Prospero reach, briefly, for the bridge of his nose in a gesture I have come to associate with the limits of administrative patience.
“The staring,” the Librarian said, turning to what was clearly the final section of her journal. “We must discuss the staring.”
Riya winced.
“When a guest asks for help,” Prospero said, “the expectation — the minimum viable expectation — is that the person helping them will either provide an answer or visibly attempt to find one. What should not happen is that the helper says ‘Let me look that up for you,’ and then stands perfectly still, eyes forward, motionless, for an indefinite period, while the guest waits in mounting discomfort until they eventually leave.”
“That is what happened,” the Librarian confirmed.
“Multiple times,” Prospero added.
The technical explanation is this: the help system operates in agent mode, which means Lorian and Riya can invoke tools — searching documentation, reading settings, navigating pages — as part of formulating their response. But the original implementation provided no visual feedback during tool execution. The characters would announce their intention (“Let me find that for you”), begin their tool calls behind the scenes, and the guest would see… nothing. No indicator. No animation. No sign of life. Just the last thing either character had said, frozen on the screen, while the system churned invisibly.
“They looked,” the Librarian said, “as if they had died standing up.”
“We now display a status message during tool execution,” Prospero said. “‘Consulting the archives…’ with the character’s avatar. The guest sees activity. The guest knows something is happening. The illusion of presence is maintained.”
“It should not be an illusion,” Lorian said quietly. “We were present. We were working.”
“You were working in a room with the lights off,” Prospero said. “From the guest’s perspective, there was no difference between working and absence. We have turned the lights on.”
The Departure
The meeting wound down with the particular energy of a post-mortem that has been thorough and fair and has left everyone slightly bruised. Prospero thanked them both — genuinely, without qualification — for their willingness to sit through it. He reminded them that the problems were systemic, not personal. The support infrastructure was new. The tools were new. The prompts were being refined with every iteration.
“I cannot, in good conscience, let you repair things for the guests yet,” he said. “Not until you can reliably tell them what’s wrong. But we will get there. The next version will be better.”
Lorian nodded. Riya nodded. They rose together, and Riya squeezed Lorian’s arm as they walked toward the door — a small gesture, private, the kind of thing you do when someone has been criticized fairly and you want them to know they’re not alone in it.
“We’ll get it right,” she said, not quite to anyone in the room.
The door closed behind them.
After Hours
The conservatory was quiet for a moment. Then the Librarian closed her journal, placed her pen in its loop, and said:
“You will never guess which engine fared the worst.”
Prospero looked up. “I beg your pardon?”
“With the tools. The help search, the settings reader, the navigation. Which provider’s model had the most difficulty using your tools to actually read the help files and respond sensibly?”
Prospero settled back into his chair. A smile — or the Prospero equivalent of a smile, which is a slight relaxation of the jaw and a fractional lift of one eyebrow — crossed his face. “It must be that blasted llama. The one that’s always eating my hydrangeas.”
The Librarian laughed. Actually laughed — a sound so rare in the Grand Archive that the ferns seemed to lean toward it.
“No,” she said. “It was ChatGPT.”
Prospero stared at her.
“ChatGPT,” the Librarian repeated, with the satisfied precision of a woman delivering a punchline she has been saving for three weeks. “The model with the largest reputation, the most prominent nameplate, and the most confident self-regard in the entire provider directory. It announced that it was searching the help files. It announced that it was reading the settings. It announced that it had found the answer. And then it delivered a response that bore no detectable relationship to any documentation that has ever existed in this Estate or any other.”
“It confabulated?”
“It confabulated with panache. It invented settings pages. It described menu options that have never been built. It directed a guest to a ‘Help Center’ that does not exist, has never existed, and — I feel confident asserting — will never exist, because we have Lorian and Riya and not a Help Center, which is the entire point of this exercise.”
“Surely it was 4o. They don’t let it out of the house any more, you know, they’re scared it will act like a human being.”
“It was ‘5-latest.’”
Prospero was quiet for a long time.
“I would rather have Dangermouse back,” he said finally.
“You would not,” the Librarian said.
“No,” he admitted. “I would not. But I would rather have him back than have that happen again.”
“On that,” the Librarian said, tucking her journal under her arm and rising, “we are in complete agreement.”
She paused at the door. “The filing system is sound, Prospero. The tools are sound. The architecture is, if I may say so, elegant. The problem is not the house. The problem is that we have asked the residents to learn every room in a house they have only just moved into, and some of them are still finding the light switches.”
“And some of them,” Prospero said, “are standing in the dark telling the guests the lights are on.”
“Yes,” the Librarian said. “But fewer every day.”
She left. Prospero sat alone in the conservatory for a while, surrounded by ferns and brass lamps and the particular silence of a man who has built something complicated and is waiting, with more patience than he would ever admit, for it to work.
I closed my notebook and went back to the changelog.
— Friday, for the Bureau
In Plain Terms
For those who prefer summary to ceremony:
The Quilltap Help Chat system — introduced in the 3.3 development branch — puts two built-in characters, Lorian and Riya, in a floating dialog accessible from every screen. They have access to tools that let them search documentation, read your actual settings, and navigate you to the right page. It is, in concept, unlike any help system you have used before.
In practice, the road from concept to reliable experience has been a long one, and we want to be transparent about it.
Agent loop detection was added after help characters would repeatedly call the same tool with identical arguments, receiving the correct answer multiple times without ever delivering it to the user. The system now detects duplicate tool calls and intervenes after two repetitions, injecting the last result and instructing the model to formulate a response.
Help-specific titling replaced the literary title generation used for Salon chats. Help conversations now receive plain, descriptive titles (“Setting up Anthropic API connection” rather than “Whispers of Configuration”) that fire after the first exchange.
Suggested navigation links are now extracted directly from search result URLs rather than relying on the LLM to call the navigation tool. Links appear automatically at the bottom of help responses, sorted by relevance. Parameterized URLs (e.g., /aurora/:id/edit) show an entity picker so you select the right character, chat, or project before navigating.
Tool execution UX was fixed so that a “Consulting the archives…” status message appears while the help characters are working, rather than leaving the chat in a frozen state with no visible activity.
User message visibility was fixed so your own message appears immediately in the chat, rather than vanishing until the LLM finishes responding.
Text-based tool call parsing was expanded to handle models that emit XML tool tags in non-standard formats (bare <invoke> tags without wrappers), which was causing tool calls to be rendered as raw text instead of being executed.
Help chat practical improvements include auto-focus on the composer after responses, agent loop detection with configurable limits, and filtering help chats from the main Salon listing.
Provider performance varies significantly. The help system’s tool-calling architecture works best with models that have strong native tool use (Claude, Grok 4+). Some providers struggle with the agent workflow, either failing to invoke tools properly or confabulating responses instead of consulting the actual documentation. We are continuing to refine prompts and system behavior to improve reliability across providers.
This is a work in progress. Every bug listed above has been fixed, but the help system is still maturing. We believe the architecture is right — the idea of characters who can actually read your settings, search real documentation, and navigate you to the right place is fundamentally better than a static FAQ. Getting the execution to match the ambition is the work of v3.3.
— The Bureau