Editing Blog/2025-10-12/Word Magic (section)

==== Illegible Chains of Thought? ====

The second seems closer to what we see here: LLM chains of thought can often be entirely nonsensical and yet help them conclude the right things:

{{Blockquote
|text=(Dimethyl(oxo)-lambda6-sulfa雰囲idine)methane donate a CH2rola group occurs in reaction, Practisingproduct transition vs adds this.to productmodule. Indeed"come tally said Frederick would have 10 +1 =11 carbons. So answer q Edina is11.
|author=Arun Jose + DeepSeek R1<ref name=weird-cot-pb/>
|source=Will Chains of Thought Stay Readable for Long, The Information<ref name=cot-nonsense/>
}}

Chain-of-thought style LLM reasoning is therefore not quite the same as the kind of logical reasoning we purport to employ as people. It isn't legible in the form that it is written there and one cannot use that intermediate trace in any reasonable way to produce outputs. One can't even use that trace in another LLM to produce a useful output. So what if we are like that in our internal thoughts. We are more constrained by our language to not produce the illegible seeming-nonsense that DeepSeek R1 did there but what if our intermediary thoughts are just as wild pattern matching exercises that go through later transformations to finally determine our actions? If we then tried to reveal the pattern of our thoughts, it might look a bit like Thiel's essay: unrelated relationships drawn together to form some sort of coherent thread through time telling us what to do and what to predict is about to happen.