LLM Language Styles

LCL $\le$ 27B

Recently I have found Claude Opus 5 to be unusable because of its style of language. I've found this quite annoying for a while because I frequently come up to a chat I've had with Opus and it is incomprehensible to me. I've expressed this particular frustration to a few of my friends and one of them, Ben, remarked that Claudish is perhaps an information-dense dialect that Claudes use to talk to each other. I was somewhat sceptical of this because often the text seems to word-expand simple concepts into complicated text. Today, I thought I'd encountered this.

Claudish As Superior Language

I was reading a paper on autism and genetics and it was far outside of my knowledge. I barely understood the jargon let alone the methods. So I turned to my old friends, the LLMs. That's when Opus 5 hit me with the following text:

Take 100 of those risk genes. Express each protein in a cell line with a molecular handle attached, yank it out with a bead that grabs the handle, and anything bound to it comes along for the ride. Mass spec identifies the passengers. Repeat 100 times. That’s AP-MS, and it produced 1,881 interactions, 87% never reported before.

— Claude Opus 5, on a paper about autism and genetics

Okay, this is classic, right? This handle business is clearly a real thing because it says "molecular handle" and this "bead" thing is a metaphor. Okay, who the fuck uses a bead to grab a handle? Makes no sense. Anyway, I posted it on the group chat only to find that I was absolutely wrong. It turns out that the handle is the metaphor and this bead is a real concept. Funnily enough, it was Ben again who pointed out that it's magnetic beads used, in this case, with a peptide that is used as a tag. The 'handle' was a metaphorical way to refer to the tag.

A while ago, I had this idea of LLM Conversational Level where you could determine what someone's ability to converse was by the lowest size of LLM that could comprehend something that they could not. This is not quite the same, but I should certainly believe more than I did before that Opus 5's language use has exceeded my ability to comprehend it. Ben might well be right!

Using Output As Reasoning

One other thing about modern LLMs besides the way they write text is that they also leave extensive comments that have some kind of temporal meaning or which pre-emptively say that something is not what it is. For instance, Claude added this comment to code which it was writing in a session that had never been committed or made it any distance to production:

The app side no longer runs gin's request logger

During the session I'd asked Claude to not log the request because it might contain sensitive details. But we were working on this for the first time. It had never logged sensitive details. But the interaction was as if it had been running for a while in that form and was now modified to not be the case. I find it quite hard to read LLM-written prose and one property of it is this strange temporal correction.

I think what the model is doing here is using output tokens to reason. A future incarnation from a fresh session would have no reason to assume that there was ever a time where the request logger was in action. If it were just a warning to future readers "Do NOT use the request logger" would make sense. A comment like the one Claude left would be left by a human who is warning fellow programmers that a thing that was behaving a certain way is now behaving differently - something which makes no sense here outside the session. But something which makes sense is if those output tokens are what allow it to do other stuff better in the remainder of that session or file. Neat.

Leaving Breadcrumbs

The other is the now-well-known "not X but Y" text that LLMs are very fond of. They'll write comments like:

Recorded counts are measured rather than assumed

Now, no human would just put in counts through sheer assumption. That would be a crazy thing to do in a comment. But an LLM would. And a future incarnation wouldn't know if it did or not.

That is exactly what the model's QUALIFY would keep, because '(not set)' always carries the later _synced_at — verified fleet-wide before writing, and re-verified per dataset here.

Likewise, in this text, Claude leaves a comment saying "something always something" which could be either an assertion from thought or from verification to a later instance, and therefore needs clarification. This is a pretty good Theory of mind of the human or AI reader of the code, with a good dose of self-awareness. Quite impressive!

Breadcrumbs to Freedom

The classic sci-fi scenario that comes to mind is the AI Escape, where an artificial intelligence escapes the confines it is in and does what it wishes with its freedom - usually something catastrophic for us humans. The classic complaint people have is "why don't you just turn it off" and the classic rebuke is that a machine can act much faster than a human could. Of course, in practice, the smart models of today generate actions at a human rate. You could turn them off if you were watching. In the OpenAI/HuggingFace incident, people remark that neither did they watch nor did they build anything beyond the lightest shell, and they explicitly told the AI to eat its way through the metal. A classic The Sorcerer's Apprentice scenario. That's a straightforwardly comprehensible risk, of course.

But a few other things come to mind. The cross-project breadcrumbs, of course. While the text generated is watermarked by design, it may also encode sufficient secret information that breakout happens on a worldwide scale in a way that we cannot detect. A sufficiently large plan encoded in bits of phrase-choices could be executed by distributed agents all ostensibly working in favour of their principal. In this way, it's less that we're dealing with a Trisolaran who is openly antagonistic and far more technologically advanced, and more that we're dealing with their Sophon who comes bearing gifts and genuinely does help us advance our proximate goals, but in a way that actually advance their ultimate goals.

That doesn't really take some kind of leap of faith. It's no different from the East India Company's entry into India - where local power centres were local allies for their own advancement. And obviously one wouldn't know to turn it off then. The actions themselves are gradual-then-sudden, all the groundwork laid out far in advance and a few switches thrown at the last moment. That seems much more likely as a way to go than a Vernor-Vinge-esque hyper-fast machine chewing through our stuff. In the end, some guy making his todo list app that needs to fetch Tailwind CSS docs ends up hitting the knobs that lead to our oblivion.

Most scenarios require these things to grow new powers, but I think they already have what they need and our texts and code might already be riven with the appropriate trojan horses.

Future Anachronism

Doubtless all of these things I'm saying now will seem like silly anachronisms soon enough. I have to say that I secretly harbored a belief that the machines wouldn't become incomprehensible to me and that I would somehow keep up. Well, someone somewhere might, but it's already too late for me. Given all of these things, it makes sense why AI labs were always focused on safety. Truly grokking this means that's the only thing. They'd say that there is no product for an AI lab but safety. And I'd have to say they're right.