Dys

I have used large language models for about a year, which is long enough to file a report.

The report is mixed. Short stories got easier to finish. Novels got worse. 

The novel I am writing is a “systems noir” satire of institutional entropy, and I did not ask the model to imitate Pynchon, Heller or Vonnegut. Imitation is the easy test, and the model can pass it for a page. What I did was harder. I tried to identify what those authors do that make  them who they are, abstract those qualities into instructions, and see whether the model could use them as principles of composition: The system is the protagonist, although nobody in it would recognize the description, and certainly nobody would put it on an organizational chart. The characters have their own purposes, grudges, appetites, theories, loyalties, and occasionally the mistaken impression that they are in charge. They move through departments, private arrangements, historical accidents, technical protocols, rumors, military procedures, obsolete technologies, and the occasional piece of unexplained equipment, each carrying a partial account of what is happening. Their accounts overlap, contradict one another, and sometimes turn out to be describing entirely different parts of the same event. Meanwhile, the machinery keeps running. A minor procedural adjustment in one office becomes a constitutional crisis three jurisdictions away; a forgotten experiment acquires a budget, a security classification, and eventually its own mythology. Nobody is quite stupid enough to explain the disaster, and nobody is sufficiently informed to prevent it. That’s where the comedy lives: not in the exposure of fools, but in the spectacle of intelligent people doing their jobs while the consequences escape into the surrounding ecosystem. The lore comes along with the action, the way weather comes along with geography. It gathers in conversations, technical specifications, rumors, footnotes, institutional folklore, and the accumulated wreckage of previous explanations.

These were not restrictions designed to make the novel obscure. They were ways of keeping its world open, allowing one event to acquire several histories, one character to become a minor figure in somebody else’s story, and a seemingly incidental detail to turn up later with an entirely different set of consequences. The test was whether the model could compose from those principles rather than reproduce their recognizable surface. It could manage individual effects: the digression, the bureaucratic exchange, the historical excursus, the elaborate piece of institutional lore. But as the novel accumulated, it kept trying to put everything back in its proper place. The digression acquired a purpose, the coincidence became a clue, the competing accounts converged on an answer, and the machinery began to look suspiciously like an obstacle constructed for the protagonist to overcome. 

Words are cheap now, and so is revision. Coherence is not. The difficulty did not vanish. It migrated from composition to control, the way a factory fire migrates to the next building.

A short story is a manageable object. It fits in one head, and you can watch the whole machine run. A novel is a different class of artifact. It accumulates characters, subplots, and revisions faster than any single reading can hold. The industry sells the cheap part and shrugs about the expensive part.

At first I took this for a problem of speed. It is not. Professional novelists have always written fast, and the best of them turn out thousands of words a day and still know every page. Their pace is capped by their decisions, so their memory keeps up. The ones who work at industrial volume, with outlines and collaborators, keep the book in a document they control. Either way, someone holds the architecture. The model removes the cap. Output runs at the speed of generation, and my understanding runs at the speed of my annotation. The problem is the gap between the two, and it is a problem of memory, and the memory problem is an authorship problem. When I write a sentence myself, I remember it, and the reason is not quality. Writing is how I make decisions. The detours and awkward formulations stay with me because I paid for them. An LLM hands me a passage after I prompt it with a clever turn of phrase or a contradiction that  I would have been glad to write, and I accumulate it and annotate it without having made any of the grammatical choices behind it. Annotating is not composing. Repeat this enough and I own a manuscript I can navigate but do not understand.

The word processor started this. It made revision free, and the manuscript went liquid. The LLM extends the same economy from revision to composition itself. Nobody sent a memo.

Then comes the other half of the problem: the machine cannot hold the book either. Past about thirty pages, roughly where the manuscript outgrows a single reading’s working memory, the failures stop being typos and become structural. The model smooths toward the passage in front of it, and the book is not in front of it. The failures sort into three families, three to a family.

The model repeats.

  • Structural repetition. Identical paragraph shapes recur dozens of times, because the local context rewards a shape that works.
  • Verbatim recycling. The same phrases surface in unrelated scenes, as default rather than motif.
  • Plot stagnation. The same revelation is staged again and again. The story moves but does not advance.

The model converges.

  • Voice homogenization. Distinct characters begin to speak and think alike, and every voice drifts toward the model’s generic register.
  • Character drift. Traits do not merely fade. They get reassigned. The acerbic man becomes merely cynical. The model is not forgetting in the human sense. It is substituting the statistically common option for the distinctive one, and the same pressure erodes a character’s entire psychological profile.
  • Register drift. My rule is that the comedy must produce vertigo, not contempt. It decays into clearer, funnier, easier contempt, which the model reads as an improvement because the local passage improves.

The model loses the architecture.

  • Structural inconsistency. Characters who must never meet are merged because the merger is locally plausible. A chapter title promises a corpse that will not arrive for three episodes. When the outline drops out of view, the theme slides too, and the prose stays locally fluent and globally off-course. Each fix makes sense where it is made and breaks something elsewhere.
  • Lost in the middle. Even when the character sheet sits inside the context window, attention favors the beginning and the end. By chapter ten the founding details are in the dead zone.
  • Saturation. The window holds more than the manuscript: prompts, instructions, character sheets, conversation history, the model’s own replies. By the time the book is long enough to need memory, the memory is already crowded out.

What this looks like in DYS

The system-as-protagonist decays into Veep. The book runs on one law: the comedy must produce vertigo, not contempt. The target isn’t stupidity, vanity, or hypocrisy, though all three can appear. It’s the individual caught inside an absurdity so elaborate, so internally consistent, and so consequential that laughter gradually becomes a form of disorientation. The president, the general, the bureaucrat are not fools waiting to be exposed. They are intelligent people trying to make sense of a situation in which every decision has a rationale, every procedure has a purpose, and the combined result is something nobody intended and nobody can quite control. The comedy comes from the collision between human intelligence and systems that have acquired a logic of their own.

This is hardly a new literary ambition. For roughly a century, some of the most inventive fiction has been interested in precisely this territory: the individual caught in systems too elaborate to comprehend, institutions whose internal logic produces consequences beyond anyone’s intentions, and the comic distance between the explanations people give for events and the reality those events create. From Kafka’s bureaucratic labyrinths to Heller, Pynchon, Vonnegut, Strugatsy brother or Lem. the novel has repeatedly found ways to make the machinery of modern life its subject without reducing the people caught inside it to caricatures. This is a tradition of literary fiction, not a genre exercise in absurdity. Its comedy depends on the reader gradually discovering that what first appeared ridiculous is also coherent, consequential, and perhaps impossible to escape.

This is where the model begins to drift. Left alone for a few thousand words, it turns the general into a buffoon, the memo into a zinger, and institutional paranoia into personal vanity. It makes the people ridiculous and loses the machinery that makes their predicament absurd. The joke becomes easier to recognize, but the world becomes smaller. Heller’s bureaucratic trap, Pynchon’s vast machinery of history, technology, paranoia, and institutional power, Vonnegut’s human vulnerability in the face of catastrophe treated as ordinary business, Lem’s philosophical collisions between human reason and realities it cannot master, the Strugatsky brothers’ encounters with systems, histories, and forms of intelligence that exceed the limits of human understanding: these are not interchangeable styles or a set of comic mannerisms. They share a deeper operation. They make the reader laugh at a situation, then slowly realize the situation is much larger than the joke. The absurdity is not a decorative layer placed over reality; it is what reality looks like when its underlying logic becomes visible. A political comedy satirizes the people running the show. This novel is interested in what happens to people when the show acquires a logic of its own.

The model keeps the first because it has seen it a thousand times, and that is the one thing the book cannot be.

Rhizomatic architecture collapses into the hero’s journey. The architecture is multi-POV, orbiting a shared phenomenon, with no one able to see the whole. Neither the audience-surrogates nor the institutional figures achieve comprehension, and the comedy collapses if any of them does. The model repairs this. Dys, my delivery worker, gets a mentor, a call, a threshold, a return, and a moment where she finally understands. This one is nastier because I borrowed the monomyth on purpose: Dys is the novel’s Luke Skywalker, Hicks is Han, Vera is Leia, and Act Three is mapped onto Stross’s Singularity Sky. The skeleton is deliberate scaffolding under a Pynchonian surface. The model keeps the scaffolding and drops the building.

Deadpan dialogue slides into The Office cringe. Whitcomb and Devereaux begin in the deadpan register of Mandrake and Muffley in Dr. Strangelove, comic bureaucrats who darken, over the length of a correspondence, into moral actors. That arc depends on deadpan. The humor sits in procedure delivered straight, and the reader supplies the horror. The model supplies it instead. Characters wink. A line lands a beat too long. Somebody notices how absurd things are and says so. This is cringe, and cringe is contempt wearing a nervous smile: the author standing slightly outside the scene, laughing at the people in it. That is exactly the failure I ruled out. Even the Dali Lama, whose rule is that he never explains, starts explaining.

Professional agency drifts into Regency courtship. Vera is a mid-level bureaucrat, the estranged daughter of the general, working the problem from the civilian side. The register I want for her and Hicks is Hawks, two professionals trading fire as equals. The model drifts toward the drawing room. Her competence becomes sensibility and her agency becomes a courtship plot, and the interesting question, what she knows and what it costs her, becomes whether she will be understood. (Austen’s own heroines are sharper than the genre that bears her name. What the model reaches for is the pastiche, which is more common in the corpus than the novels.)

The model knows the references. What it cannot reliably preserve is the reason I am using them against their usual purpose.

Four slides: register, structure, dialogue, character. They look unrelated, and they have one cause. Veep, the hero’s journey, The Office, and the Regency heroine are four of the most heavily imitated shapes in English. In my experience, at every moment of drift the statistically common option beats the one I specified, and none of the four looks like an error. Each reads as a clearer, funnier, more familiar draft. The model’s default is not simply to make things up. It is to make things more familiar. And these are not four failures of taste. They are failures to preserve differences that organize the book: the difference between a system and its representatives, between a structure and its scaffolding, between deadpan and commentary, between agency and recognition. I catch them only because the rules are written down and I know what they protect.

This is why the failures are informative. A failed pastiche shows only that the model is a poor mimic. A failed principle shows something worse. The model can state the principle, accept it, apply it for a few thousand words, and then lose it. That is the difference between knowing a rule and holding a commitment, and it is the difference the Bateson section takes up below.

The deeper reason is structural. Ask the model about Ulysses and it will lay out the Homeric schema and the shifting styles of the chapters. Ask it about Gravity’s Rainbow and it will explain the rocket and the refusal of closure. It knows these books the way a critic does. But writing inside a form is a different skill from describing it. Every register carries its own organization of meaning, and the dominant one in the stories the model has absorbed is the tree: a trunk, a protagonist, a want, branches of subplot that all feed one climax.

 The books whose principles I am trying to write from are rhizomatic. Pynchon’s plots connect any point to any other and have no center. Burroughs cut the line of plot itself. A model asked for the likeliest next passage grows a trunk, because from the inside a rhizome looks like a mistake: a loose thread, a missing stake, a subplot that goes nowhere. It does not merely prefer the tree. It reads the rhizome as damage to be repaired. This is my account of what happens in this writing process, not a claim about everything in the model’s training.

The tree is one loss among five, and the five are not variations on “nonlinear.

I wasn’t asking the model to imitate writers. I was trying to extract the compositional techniques that made their books operate according to different principles, translate those techniques into instructions, and see whether the model could use them in my own novel. The difficulty was that the model kept bringing each principle back to the same narrative structure: the hero’s journey.

Take dramatic irony and dialectical dialogue. I wanted characters arguing from competing institutional interests, each with a defensible account of events, without one of them being appointed to explain what was really happening. The reader should see contradictions the characters don’t recognize. I instructed the model to preserve those competing accounts and let the disagreement stand. But it kept turning the exchange into a revelation: one character sees through the deception, another is exposed, and the protagonist learns something. Political argument becomes a stage in the hero’s education.

I wanted discontinuous narration to work through collision rather than explanation. A phrase, an image, a piece of institutional language could migrate between scenes, acquiring a different meaning in each context. A historical digression might eventually intersect with a present-day event, but it didn’t have to justify itself by advancing the plot. I told the model not to turn recurring material into clues or fragments into a puzzle. It did so anyway. 

The phrase became a clue, the digression became a subplot, and the fragments became evidence in a mystery the protagonist was supposed to solve. Discontinuity was converted into a delayed form of continuity.

The same thing happened with the larger architecture. I wanted the novel to expand through secondary characters, technical histories, bureaucratic procedures, competing explanations, and events whose significance might not become apparent until much later. A minor character could open an entirely different history; several explanations could remain in circulation without converging on a master conspiracy. The model kept turning this network into a quest. Every branch acquired a relationship to the protagonist’s objective, every pattern pointed toward a hidden explanation, and every apparent detour became another step toward the central revelation. The rhizome was rebuilt as a tree.

Character presented a different version of the same problem. I wanted people to be legible through their professional vocabularies, social positions, habits, possessions, gestures, and misunderstandings. Their personalities were inseparable from the institutions and hierarchies in which they operated. But the model kept isolating the individual from that social machinery. Status became personal conflict, social observation became characterization, and the protagonist’s development became the organizing principle. The world was reduced to a backdrop for character arcs.

Finally, I wanted extraordinary events delivered in the language of ordinary business, disturbing information treated as routine, and institutional language allowed to reveal more than its speakers understood. The reader should recognize the implications without a character stepping forward to explain them. The model kept turning estrangement into mystery and mystery into revelation. Someone had to discover what was really happening, understand its significance, and confront it. The ambient dread became a problem with a solution.

These were five different ways of organizing a novel: through competing interpretations rather than moral instruction, collision rather than continuity, proliferation rather than hierarchy, social relations rather than isolated psychology, and implication rather than explanation. The model could reproduce examples of all five. But it kept subordinating them to the same underlying structure: a protagonist, a quest, a series of discoveries, individual transformation, and resolution. It wasn’t simply forgetting my instructions. It was interpreting them through a narrative logic that kept changing what they were for.

The model obeys each instruction in the passage where it appears and betrays the principle that connects them. It writes the digression, then makes the digression serve the quest. It writes the ambiguous scene, then explains it in the next chapter. It creates a character who does not understand the system, then hands her the insight the plot needs. The model does not delete the technique. It domesticates it, putting it back to work for the architecture it was meant to resist. That is my account of what happens in my drafts, not a measurement of models in general.

The model has learned the statistical shapes of literary achievement. It has not reliably learned the intelligence that produces them. It knows what a novel is supposed to look like. The writers we admire are the ones who discovered what else a novel could do.

And the third act becomes the Death Star.

Everything above is drift, a slow slide that can be caught page by page. The ending is different. It is a collapse. The earlier failures are local. The ending is where they converge. Act Three of DYS is built on a refusal: there is no outside, no one comprehends the whole, and the system is not defeated. It continues to work correctly. The book closes on an indicator improving and a briefing already being written. That is the opposite of a climax, and it is the point.

The model cannot leave it alone. Given the last act, it resolves toward the most-told ending in modern fiction. It does not merely invent a Death Star. It invents the narrative conditions that make one necessary. It is the same domestication, applied to the whole book at once. Three things have to exist for that ending to work, and my book denies all three.

It needs a discrete enemy. My antagonist is a system, rational and distributed and without a center. The model hands it a center: a weapon, a deadline, a single point of failure, a trench run in all but name. Operation Epic Fury stops being a doctrinal catastrophe that has already begun and becomes a device to be stopped.

It needs a decisive act. In the book, no character achieves full comprehension, because the comedy collapses if one does. The model gives Dys the moment of understanding and the shot. The chosen one chooses, and it works.

It needs catharsis. The reader is supposed to close the book with vertigo, in a world where the indicator is up and nothing was won. The model supplies a victory, because an unresolved ending reads to it as a defect to repair.

This is not a failure of memory. I remind the model constantly, from the first page on, and every prompt says where we are: Act One, Act Two, Act Three. The reminders hold. What triggers the Death Star is the instruction itself. “Act Three” is a request for the most-told shape in fiction, and the model supplies it whatever I have said about the system, because the label calls the convention up.

I am not asking for no ending. Experimental books still end. Naked Lunch closes on a frame Burroughs built by moving one chapter to the last position, and then it stops. That is an ending that withholds. The model’s Act Three is the opposite: confrontation, decisive act, catharsis. The problem is not that the book has a last movement. It is the model’s idea of what a last movement is.

Every earlier drift had to fight my instructions one passage at a time. At the climax, the whole weight of the form arrives at once, and the model is no longer drifting toward the common shape. It has been drawn into it.

Nine failures, three causes: a machine that prefers the common shape, the easy register, and the nearby page.

There is a tenth failure, and it is the only one on the list that is mine. I do not approve passages, exactly. I accumulate them and annotate them: material arrives, and I mark what is wrong, what must be kept, what contradicts what. Annotation is real work, and it is not composition. It is judgment applied after the fact to text whose choices were made elsewhere. The pile grows faster than my notes can digest it, and the book I hold in my head becomes the book of my annotations, not the book on the page. The flaw is hard to defend because I kept it rather than made it.

So I have been going back to the old manuscripts, the literal paper ones, and finding them close to flawless. Some of this is selection bias. They are what survived the boxes and the moves. But the effect remains, and it is odd. What I respond to is a perfection in the imperfections.  The flaws are not virtues. They are load-bearing, and they record a mind at work on its material.

This is where I am souring on the model as editor. It is excellent at recognizable faults. It can smooth a transition, cut a repetition, tighten a line. It cannot tell a deviation from a pattern apart from a deviation that is the point. An awkward sentence may be bad, or it may carry a rhythm the elegant sentence would kill. A structural inconsistency may be an oversight, or the logic of the story. A character may be unexplained because I have not solved him, or because his opacity is why he works. Three cases, and the machine treats all three as typos.

The control problem and the taste problem turn out to be the same problem. I can only defend an imperfection I chose. Bad prose has never been scarce. The danger is a system that cannot see the architecture, editing a local oddity that a distant chapter depends on. Eliminating an imperfection is editing. Eliminating the reason a work is distinctive is vandalism performed in the name of improvement.

Why can’t the vendors simply fix this? Because it is not a bug. It is a property of the machine, and it has three roots.

Attention dilutes. Every token competes for attention, and narrative importance does not decide the competition. As the manuscript grows, the founding details, the acerbic trait and the register rule, get a smaller share of the model’s attention than the page in front of it. Attention weights are normalized, so as the context grows, each token’s average share shrinks. The cost of attending to everything grows with the square of the length, so the obvious fix is expensive.

Generation is local. The model produces the likeliest next passage given what it is attending to. Where the distinctive trait is under-weighted, the statistically common one wins. That is the slide from acerbic to cynical. Models are also mostly trained and tested on passages far shorter than a novel, so fluency in the paragraph is a stronger skill than consistency across the book.

There is no memory outside the window. During ordinary generation, the model keeps no accumulating memory of the book. It has no state of its own. Anything it “remembers” must be re-read, and the window is shared with instructions, history, and the model’s own replies. The context window is not a memory. It is a fleeting, diluted attention field.

My thirty pages is not a private number. A Cornell group tested seven frontier models on whole novels and found that none kept a stable understanding past about 64,000 tokens, roughly 180 pages, even though the windows on offer run to millions. A 500-page novel runs to something like 150,000 to 200,000 tokens, two to three times that. That was a test of reading, with the book given the whole window to itself. Writing is the harder job, and it happens in a crowded window, so it is no surprise that my wall sits at a fraction of theirs. The advertised window was never the whole limit. America may win the long-context race the way private equity wins a hospital: the window grows, the benchmark scores climb, and the book still falls apart at page thirty.

The models are improving, but I would not bet the manuscript on any given release having outgrown this. And the machine cannot audit itself, because a passage that has drifted feels perfectly fine from the inside. Nobody is checking the checker.

Code shows the same failures with a safety net. A compiler catches drift that a novel cannot, so programmers get the benefit of the machine without the silence. But the net catches only what it can express. Tests verify only the constraints somebody thought to encode, and a refactor that smooths an ugly workaround passes every test and fails in production at three in the morning. The ugliness was carrying knowledge. Verification fixes none of the tenth failure, because it checks the work and cannot tell you who chose it.

Suppose the thirty-page wall falls. The tenth failure remains, because no model can make the choices for me and also leave me knowing the book.

Gregory Bateson gives me a lens for what is going wrong, though he wrote nothing about machines. He defined information as a difference that makes a difference. Acerbic versus cynical is a small difference that carries the whole character. A model drifting toward the statistically common option averages differences away, and in his sense that is literal information loss. The distinctive trait is the information. The generic one is noise that reads as fluency.

His frames fail the same way. “Vertigo, not contempt” is not a line in the story. It is a message about how to read the story, like the signal that says this is play. Models attend to content, so the frame decays first, and once it is gone the same jokes are read literally and the comedy curdles into contempt. The character sheet and the architecture are frames too, which is why they erode before any sentence does.

Why isn’t this a mind, then? By Bateson’s standard a mind is a circuit in which differences are processed and the circuit learns, at more than one level. The model fails on three counts. Its weights are frozen, so everything it knows about my book lives in the window, and nothing in it changes between page one and page forty. A writer is changed by a year with a novel. It holds no commitments: for me “vertigo, not contempt” is a standing obligation, and for the model it is one more piece of text competing for attention. And its forgetting is indifferent. A person’s memory compresses by significance, so the acerbic trait survives because it matters. The model’s dilution is silent. These are architectural gaps, not proofs of what models can never be. But for now the machine has the fluency of a mind and the circuitry of a tool, and the writer is the missing circuit.

The standard remedy is a longer prompt. I have tried it. To keep the model from wobbling I need a page of instructions, and the page has to carry three things: the character’s traits, stated precisely enough that “acerbic” cannot decay into “cynical”; the register rule that forbids the slide into contempt; and the architecture, meaning who must never meet, what must not yet be revealed, and which chapter owes the reader a corpse. Then the page has to be re-fed at every step, because the model does not keep it. Even re-fed, it fails at the ending, because the ending is what the instruction summons.

The page was the optimistic version. In practice it was a thousand prompts, each carrying [character histories], relationships, structural requirements, tonal distinctions, prohibitions, corrections, and reminders about things the model had already been told and had forgotten. Every prompt was an attempt to restore some part of the book’s architecture. Ted Chiang has argued that a ten-thousand-word story takes something on the order of ten thousand choices, nearly one for every word. I wasn’t claiming to have reached that number, but I was getting uncomfortably close to its scale. A thousand prompts, each containing several decisions, and I wasn’t close to a workable draft. I was approaching ten thousand small decisions just to keep the machine pointed in the right direction, and by Chiang’s arithmetic that is the budget for one short story, spent on a novel I did not yet have.

Hence my countermeasure, which is paradoxical: I started using Obsidian. It lets me keep the book in small, granular pieces, because past a certain length the supposedly efficient workflow turns back into manual work anyway. The one-page prompt and the vault are the same document, a filing system for decisions, and a plain filing system I control is easier to maintain than a prompt I must defend against a collaborator who forgets it but still wants to make me the next hottest YA author. I can separate the chapters, preserve the annotations, track the changes, and keep the reasons for making them somewhere other than the conversation in which they were made. I can see the book as a collection of decisions rather than a continuous stream of prose. Nothing particularly sophisticated is happening. I am restoring the relationship between a writer and his material, which the sophisticated workflow had begun to obscure. The most advanced text generator I own sent me back to the plainest tool I could find.

Nobody is holding the book. The model forgets it. It can reproduce the passage in front of it, summarize the chapter, and reconstruct the argument, but it does not carry the accumulated weight of the decisions that produced them. The writer annotated where he might have chosen. Those annotations are now the closest thing the book has to a memory. The editor is the model, but the model has no independent stake in the editorial outcome. It can identify a weakness without understanding why the weakness might be necessary, or recommend a cleaner solution without recognizing what the untidy version was doing. It can improve the sentence while damaging the book. And because it can explain the improvement so fluently, the damage may arrive disguised as a clarification.

A fast writer has an advantage here, though not necessarily a literary one: he remembers the sequence of decisions that brought a passage into existence. He remembers the alternatives, the awkward construction retained for a reason, the paragraph moved three times because its function wasn’t apparent until later. The sentence has a history, even if that history is no longer consciously accessible. The model’s sentences have no such history for me. They arrive already formed, detached from the decisions that would have produced them. I can accept one, reject another, request ten more, and mistake the resulting abundance for progress. The problem isn’t generating alternatives. It’s maintaining a basis for choosing among them, particularly when the choice has consequences several chapters away. Each new prompt can reopen a decision, and each revision can quietly invalidate something the model no longer has in view.

This is where the experiment broke down. The technology was capable of working at close range: generating passages, revising sentences, extending scenes, imitating a register. But the book existed at a different scale. Its problems were neither local nor global but somewhere in between, across the distance separating a paragraph from a chapter, a chapter from the architecture of the whole. The model could work on the immediate material while losing track of the dependencies that made it part of a larger construction. It could remember the premise and lose the decisions. It could follow the latest instruction while undoing the accumulated logic of earlier ones. The longer the collaboration continued, the more time I spent auditing its continuity, restoring context, and checking whether a perfectly plausible intervention had broken something elsewhere. The supposed efficiency began generating its own maintenance costs.

The peculiar result was that I had to go manual. Not because manual work revealed some superior human capacity, or because the writer had rediscovered the virtues of doing everything himself, but because the technology’s middle-distance limitations made the alternative increasingly impractical. Beyond a certain point, I was maintaining the model’s working memory, reconciling its successive versions, and repairing the consequences of its revisions. The tool designed to reduce the administrative burden of writing had acquired an administrative burden of its own. Obsidian was not an upgrade in intelligence or a philosophical correction. It was a workaround for a system that could produce more material than it could reliably keep track of.

We have made it easier to get words onto the page, but have not solved the problem of keeping those words in relation to one another. The cost of production has fallen; the cost of integration has not. In some circumstances it has risen, because the volume of material and the speed of revision increase the number of things that must be checked. The model can be remarkably competent at the point of intervention and unreliable across the distance between interventions. That is a particularly awkward limitation for a technology being sold as a collaborator in extended work. It doesn’t have to write something obviously bad to undermine the project. It only has to make a series of locally reasonable changes whose cumulative effect nobody is tracking. By the time the damage becomes visible, it may be difficult to locate the decision that introduced it. The things we will lose first are the ones we have not yet built the vocabulary to defend.


Posted

in

by

Tags:

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *