Discussion
Search code, repositories, users, issues, pull requests...
TeMPOraL: Oh boy. Someone didn't get the memo that for LLMs, tokens are units of thinking. I.e. whatever feat of computation needs to happen to produce results you seek, it needs to fit in the tokens the LLM produces. Being a finite system, there's only so much computation the LLM internal structure can do per token, so the more you force the model to be concise, the more difficult the task becomes for it - worst case, you can guarantee not to get a good answer because it requires more computation than possible with the tokens produced.I.e. by demanding the model to be concise, you're literally making it dumber.(Separating out "chain of thought" into "thinking mode" and removing user control over it definitely helped with this problem.)
ArekDymalski: While really useful now, I'm afraid that in the long run it might accelerate the language atrophy that is already happening. I still remember that people used to enter full questions in Google and write SMS with capital letters, commas and periods.
andai: No articles, no pleasantries, and no hedging. He has combined the best of Slavic and Germanic culture into one :)
gozzoo: I think this could be very useful not when we talk to the agent, but when the agents talk back to us. Usually, they generate so much text that it becomes impossible to follow through. If we receive short, focused messages, the interaction will be much more efficient. This should be true for all conversational agents, not only coding agents.
pixelpoet: > Usually, they generate so much text that it becomes impossible to follow through.Quite often on reddit I'll write two paragraphs and get told "I'm not reading all that".Really? Has basic reading become a Herculean task?
bogtog: I'd be curious if there were some measurements of the final effects, since presumably models wont <think> in caveman speak nor code like that
virtualritz: This is the best thing since I asked Claude to address me in third person as "Your Eminence".But combining this with caveman? Gold!
saidnooneever: LOL it actually reads how humans reply the name is too clever :').Not sure how effective it will be to dirve down costs, but honestly it will make my day not to have to read through entire essays about some trivial solution.tldr; Claude skill, short output, ++good.
Hard_Space: Also see https://arxiv.org/pdf/2604.00025 ('Brevity Constraints Reverse Performance Hierarchies in Language Models' March 2026)
ryanschaefer: Kinda ironic this description is so verbose.> Use when user says "caveman mode", "talk like caveman", "use caveman", "less tokens", "be brief", or invokes /cavemanFor the first part of this: couldn’t this just be a UserSubmitPrompt hook with regex against these?See additionalContext in the json output of a script: https://code.claude.com/docs/en/hooks#structured-json-outputFor the second, /caveman will always invoke the skill /caveman: https://code.claude.com/docs/en/skills
Rexxar: > Someone didn't get the memo that for LLMs, tokens are units of thinking. Where do you get this memo ? Seems completely wrong to me. More computation does not translate to more "thinking" if you compute the wrong things (ie things that contribute significantly to the final sentence meaning).
staminade: That’s why you need filler words that contribute little to the sentence meaning but give it a chance to think. This is part of why humans do the same when speaking.
vivid242: Great idea- if the person who made it is reading: Is this based on the board game „poetry for cavemen“? (Explain things using only single-syllable words, comes even with an inflatable log of wood for hitting each other!)
golem14: I think the sentiment here is that the short formulation of Kant's categorical imperative is as good and easier to read than the entirety of "types of ethical theory" (J.J. Martineau).
doe88: > If caveman save you mass token, mass money — leave mass star.Mass fun. Starred.
avaer: That was my first thought too -- instead of talk like a caveman you could turn off reasoning, with probably better results.Additionally, LLMs do not actually operate in text; much of the thinking happens in a much higher dimensional space that just happens to be decoded as text.So unless the LLM was trained otherwise, making it talk like a caveman is more than just theoretically turning it into a caveman.
DrewADesign: > much of the thinking happens in a much higher dimensional space that just happens to be decoded as text.What do you mean by that? It’s literally text prediction, isn’t it?
jffhn: How always wished papers written. Caveman science FTW.
alentred: Indeed. But I have tried this skill and can confirm that the thinking phase is not impacted. At least in my few attempts it applied the "caveman talk" only to the output, after the initial response was formulated in the thinking process. I used opencode.You are right, of course, that as such it does not reduce the token usage really. If anything it consumes more tokens because it has to apply the skill on top of the initial result. I do appreciate the conciseness of the output, though :)
nayroclade: Cute idea, but you're never gonna blow your token budget on output. Input tokens are the bottleneck, because the agent's ingesting swathes of skills, directory trees, code files, tool outputs, etc. The output is generally a few hundred lines of code and a bit of natural language explanation.
kubb: This is condescending and wrong at the same time (best combo).LLMs do stumble into long prediction chains that don’t lead the inference in any useful direction, wasting tokens and compute.
rschiavone: This trick reminds me of "OpenAI charges by the minute, so speed up your audio"https://news.ycombinator.com/item?id=44376989
jaccola: Do you have any evidence at all of this? I know how LLMs work and this makes no sense to me. Otherwise you'd just put filler words in every inpute.g. instead of: "The square root of 256 is" you'd enter "errr The er square um root errr of 256 errr is" and it would miraculously get better? The model can't differentiate between words you entered and words it generated its self...
pennaMan: >It’s literally text prediction, isn’t it?you are discovering that the favorite luddite argument is bullshit
jstummbillig: What do you mean? The page explicitly states:> cutting ~75% of tokens while keeping full technical accuracy.I have no clue if this claim holds, but alas, just pretending they did not address the obvious criticism, while they did, is at the very least pretty lazy.An explanation that explains nothing is not very interesting.
setnone: caveman multilingo how sound?
cyanydeez: There was a paper recently that demonstrated that you can input different human languages and the middle layers of the model end up operating on the same probabilistic vectors. It's just the encoding/decoding layers that appear to do the language management.So the conclusion was that these middle layers have their own language and it's converting the text into this language and this decoding it. It explains why sometime the models switch to chinese when they have a lot of chinese language inputs, etc.
DrewADesign: Ok — that sounds more like a theory rather than an open-and-shut causal explanation, but I’ll read the paper.
teekert: Idk I try talk like cavemen to claude. Claude seems answer less good. We have more misunderstandings. Feel like sometimes need more words in total to explain previous instructions. Also less context is more damage if typo. Who agrees? Could be just feeling I have. I often ad fluff. Feels like better result from LLM. Me think LLM also get less thinking and less info from own previous replies if talk like caveman.
cyanydeez: Fluff adds probable likeness. Probablelikeness brings in more stuff. More stuff can be good. More stuff can poison.
raincole: When it comes to LLM you really cannot draw conclusions from first principles like this. Yes, it sounds reasonable. And things in reality aren't always reasonable.Benchmark or nothing.
samus: [delayed]
0xpgm: Not specifically about your case, but some people are usually just more verbose than others and tend to say the same thing more than once, or perhaps haven't found a clear way of articulating their thoughts down to fewer words.
NiloCK: I agree with this take in general, but I think we need to be prepared for nuance when thinking about these things.Tokens are how an LLM works things out, but I think it's just as likely as not that LLMs (like people) are capable of overthinking things to the point of coming to a wrong answer when their "gut" response would have been better. I do not content that this is the default mode, but that it is both possible, and that it's more or less likely on one kind of problem than another, problem categories to be determined.A specific example of this was the era of chat interfaces that leaned too far in the direction of web search when responding to user queries. No, claude, I don't want a recipe blogspam link or summary - just listen to your heart and tell me how to mix pancakes.More abstractly: LLMs give the running context window a lot of credit, and will work hard to post-hoc rationalize whatever is in there, including any prior low-likelihood tokens. I expect many problematic 'hallucinations' are the result of an unlucky run of two or more low probability tokens running together, and the likelihood of that happening in a given response scales ~linearly with the length of response.
samus: [delayed]
cyanydeez: It's not "units of thinking" its "units of reference"; as long as what it produces references the necessary probabilistic algorithms, itll do just fine.
samus: There's linguistic term for this kind of speech: isolating grammars, which don't decline words and use high context and the bare minimum of words to get the meaning across. Chinese is such a language btw. Don't know what Chinese think about their language being regarded as cavemen language...
DrewADesign: Feel free to elucidate if you want to add anything to this thread other than vibes.
samus: [delayed]
jaccola: Yes because in most contexts it has seen "caveman" talk the conversations haven't been about rigorously explained maths/science/computing/etc... so it is less likely to predict that output.
vova_hn2: > instead of talk like a caveman you could turn off reasoning, with probably better resultsThis is not how the feature called "reasoning" work in current models."reasoning" simply let's the model output and then consume some "thinking" tokens before generating the actual output.All the "fluff" tokens in the output have absolutely nothing to do with "reasoning".
vova_hn2: I don't know about token savings, but I find the "caveman style" much easier to read and understand than typical LLM-slop.
vova_hn2: Yeah, I don't think that "I'd be happy to help you with that" or "Sure, let me take a look at that for you" carries much useful signal that can be used for the next tokens.
afro88: IIUC this doesn't make the LLM think in caveman (thinking tokens). It just makes the final output show in caveman.
throw83849494: You obviously do not speak other languages. Other cultures have different constrains and different grammar.For example thinking in modern US English generates many thoughts, to keep correct speak at right cultural context (there is only one correct way to say People Of Color, and it changes every year, any typo makes it horribly wrong).Some languages are far more expressive and specialized in logical conditions, conditionals, recursion and reasoning. Like eskimos have 100 words for snow, but for boolean algebra.It is well proven that thinking in Chinese needs far less tokens!With this caveman mod you strip out most of cultural complexities of anglosphere, make it easier for foreigners and far simpler to digest.
electroglyph: after you go from from millions of params to billions+ models start to get weird (depending on training) just look at any number of interpretability research papers. Anthropic has some good ones.
suddenlybananas: >Some languages are far more expressive and specialized in logical conditions, conditionals, recursion and reasoning. Like eskimos have 100 words for snow, but for boolean algebra.This is simply not true.
bjackman: If this really works there would seem to be a lot of alpha in running the expensive model in something like caveman mode, and then "decompressing" into normal mode with a cheap model.I don't think it would be fundamentally very surprising if something like this works, it seems like the natural extension to tokenisation. It also seems like the natural path towards "neuralese" where tokens no longer need to correspond to units of human language.
lijok: You’re conflating training and inference
vova_hn2: > Has basic reading become a Herculean task?I find LLM slop much harder to read than normal human text.I can't really explain it, it's just a feeling.The feeling that it draaaags and draaaaaags and keeeeeps going on and on and on before getting to the point, and by the time I'm done with all the "fluff", I don't care what is the text about anymore, I just want to lay down and rest.
norskeld: APL for talking to LLM when? Also, this reminded me of that episode from The Office where Kevin started talking like a caveman to make communication efficient.
akdor1154: I thought the term for those were 'sane languages', and I say that as a native English speaker :)
jerf: There is a study that shows that what the model is doing behind the scenes in those cases is a lot more than just outputting those tokens.For an LLM, tokens are thought. They have no ability to think, by whatever definition of that word you like, without outputting something. The token only represents a tiny fraction of the internal state changes made when a token is output.Clearly there is an optimal for each task (not necessarily a global one) and a concrete model for a given task can be arbitrarily far from it. But you'd need to test it out for each case, not just assume that "less tokens = more better". You can be forcing your model to be dumber without realizing it if you're not testing.
muzani: It's why it starts with "You're absolutely right!" It's not to flatter the user. It's a cheap way to guide the response in a space where it's utilizing the correction.
DrewADesign: Getting weird doesn’t mean calling it text prediction is actually ‘bullshit’? Text prediction isn’t pejorative…
vova_hn2: > I still remember that people used to enter full questions in GoogleI think that, in the early days of internet search, entering full questions actually produced worse results than just a bunch of keywords or short phrases.So it was a sign of a "noob", rather than a mark of sophistication and literacy.
otabdeveloper4: LLMs don't think at all.Forcing it to be concise doesn't work because it wasn't trained on token strings that short.
staminade: What do you think chain of thought reasoning is doing exactly?
mylifeandtimes: Really? Because if one accepts that computer languages are languages, then it seems that we could identify one or two that are highly specialized in logical conditions etc. Prolog springs to mind.
HumanOstrich: > Forcing it to be concise doesn't work because it wasn't trained on token strings that short.This is a 2023-era comment and is incorrect.
getpokedagain: In the age of vibe coding and that we are literally talking about a single markdown file I am sure this has been well tested and achieves all of its goals with statistical accuracy, no side effects with no issues.
DimitriBouriez: Good point and it's actually worse than that : the thinking tokens aren't affected by this at all (the model still reasons normally internally). Only the visible output that gets compressed into caveman... and maybe the model actually need more thinking tokens to figure out how to rephrase its answer into caveman style
veselin: This is an experiment that, although not to this extreme, was tested by OpenAI. Their responses API allow you to control verbosity:https://developers.openai.com/api/reference/resources/respon...I don't know their internal eval, but I think I have heard it does not hurt or improve performance. But at least this parameter may affect how many comments are in the code.
HumanOstrich: > things start to get weird> just look at research papersYou didn't add anything other than vibes either.
kogold: Let me rephrase that for you:"Interesting idea! Token consumption sure is an issue that should be addressed, and this is pretty funny too! However, I happen to have an unproven claim that tokens are units of thinking, and therefore, reducing the token count might actually reduce the model's capabilities. Did anybody using this by chance notice any degradation (since I did not bother to check myself)?"Have a nice day!
prodigycorp: The burden of proof is on the author to provide at least one type of eval for making that claim.
kukakike: This is exactly what annoys me most. English is not suitable for computer-human interaction. We should create new programming and query languages for that. We are again in cobol mindset. LLM are not humans and we should stop talking to them as if they are.
zozbot234: Grug says Chinese more suitable, only few runes in word, each take single token. Is great.
fzeindl: I tried this with early ChatGPT. Asked it to answer telegram style with as few tokens as possible. It is also interesting to ask it for jokes in this mode.
owenthejumper: What is that binary file caveman.skill that I cannot read easily, and is it going to hack my computer.
prodigycorp: Are you sure about that? Chain of thought does not need to be semantically useful to improve LLM performance. https://arxiv.org/abs/2404.15758
davidguetta: still doesn't mean all tokens are useful. it's the point of benchmarks
prodigycorp: Care to share the benchmarks backing the claims in this repo?
mynegation: No, let me rephrase it for you. “tokens used for think. Short makes model dumb”
ShowalkKama: the fact that more tokens = more smart should be expected given cot / thinking / other techniques that increase the model accuracy by using more tokens.Did you test that ""caveman mode"" has similar performance to the ""normal"" model?
jstummbillig: The number of people confidently talking about "burden of proof" and whose it allegedly is has gone up sharply.Nobody has to proof anything. It can give your claim credibility. If you don't provide any, an opposing claim without proof does not get any better.
ericjmorey: I don't consider these researchers luddites.https://machinelearning.apple.com/research/illusion-of-think...https://arxiv.org/abs/2508.01191
malnourish: Yes, really. The concept GP is alluding to is called the Sapir-Worf hypothesis, which is largely non scientific pop linguistics drivel. Elements of a much weaker version have some scientific merit.Programming languages are not languages in the human brain nor the culture sense.
iammjm: I speak German, Polish, and English fluently and my take is: German is very precise, almost mathematical, there is little room to be misunderstood. But it also requires the most letters. English is the quickest, get things done kind of language, very compressible , but also risks misunderstanding. Polish is the most fun, with endless possibilities of twisting and bending it's structures, but also lacking the ease of use of English or the precision of German. But it's clearly just my subjective take
zozbot234: Grug says you quite right, token unit thinking, but empty words not real thinking and should avoid. Instead must think problem step by step with good impactful words.
prodigycorp: Sorry I don't know how engaging in this could lead to anything productive. There's already literature out there that gives credence to TeMPOraL claim. And, after a certain point, gravity being the reason that things fall becomes so self evident that every re-statements doesnt not require proof.
hybrid_study: Mongo! No caveman
wzdd: They carry information in regular human communication, so I'm genuinely curious why you'd think they would not when an LLM outputs them as part of the process of responding to a message.
sillyboi: Oh, another new trend! I love these home-brewed LLM optimizers. They start with XML, then JSON, then something totally different. The author conveniently ignores the system prompt that works for everything, and the extra inference work. So, it's only worth using if you just like this response style, just my two cents. All the real optimizations happen during model training and in the infrastructure itself.
Garlef: Yes but: If the amount is fixed, then the density matters.A lot of communication is just mentioning the concepts.
amelius: It's especially funny to change your coworker's system prompt like that.
DonHopkins: High dimensional vectors are thought (insofar as you can define what that even means). Tokens are one dimensional input that navigates the thought, and output that renders the thought. The "thinking" takes place in the high dimension space, not the one dimensional stream of tokens.
amelius: By the way why don't these LLM interfaces come with a pause button?
stainablesteel: i imagine they're doing superman level distributed compute across multiple clouds somewhere and cared more about delivering the final result of that than having the ability to pause. which is probably possible, but would require way more work than would be worthwhile. they probably thought the ability to stop and resubmit would be an adequate substitute.
hackerInnen: You are absolutely right! That is exactly the reason why more lines of code always produce a better program. Straight on, m8!
amelius: And a "prune here" button.It often happens that the interesting information is in the first paragraph, and the remainder is all just the LLM not knowing when to stop. This is super annoying as a conversation then ends up being 90% noise.
freehorse: Talk a lot not same as smart
zahirbmirza: You can also make huge spelling mistakes and use incomplete words with llms they just sem to know better than any spl chk wht you mean. I use such speak to cut my time spent typing to them.
floriangoebel: Wouldn't this increase your token usage because the tokenizer now can't process whole words, but it needs to go letter by letter?
literalAardvark: It doesn't go letter by letter, so not with current tokenizers.There will likely be some internal reasoning going "I wonder if the user meant spell check, I'm gonna go with that one".And it'll also bias the reasoning and output to internet speak instead of what you'd usually want, such as code or scientific jargon, which used to decrease output quality. I'm not sure if it still does
estearum: Can't you know that tokens are units of thinking just by... like... thinking about how models work?
gchamonlive: [delayed]
xgulfie: Ah so obviously making the LLM repeat itself three times for every response it will get smarter
andy99: I’ve heard this, I don’t automatically believe it nor do I understand why it would need to be true, I’m still caught on the old fashioned idea that the only “thinking” for autoregressive modes happens during training.But I assume this has been studied? Can anyone point to papers that show it? I’d particularly like to know what the curves look like, it’s clearly not linear, so if you cut out 75% or tokens what do you expect to lose?I do imagine there is not a lot of caveman speak in the training data so results may be worse because they don’t fit the same patterns that have been reinforcement learned in.
bitwize: grug have to use big brains' thinking machine these days, or no shiny rock. complexity demon love thinking machine. grug appreciate attempt to make thinking machine talk on grug level, maybe it help keep complexity demon away.
skydhash: Pretty obvious when you think that neural networks operate with numbers and very complex formulas (by combining several simple formulas with various weights). You can map a lot of things to number (words, colors, music notes,…) but that does not means the NN is going to provide useful results.
Chance-Device: Let’s see, I think these pretty much map out a little chronology of the research:https://arxiv.org/abs/2112.00114 https://arxiv.org/abs/2404.15758 https://arxiv.org/abs/2406.06467 https://arxiv.org/abs/2512.12777First that scratchpads matter, then why they matter, then that they don’t even need to be meaningful tokens, then a conceptual framework for the whole thing.
xgulfie: > For an LLM, tokens are thought. They have no ability to thinkThis is so funny
HarHarVeryFunny: More like Pidgin English than caveman, perhaps, although caveman does make for a better name.
phtrivier: Soma (aka tiktok) and Big Brother (aka Meta) already happened without government coercion, only makes sense that we optimize ourselves for newspeak.Thank God there is still neverending wars, otherwise authoritarian governments would have no fun left.
Chance-Device: I’m just going to repost this here because it’s more visible. I think anybody could have looked this up if they’d actually wanted to know instead of just arguing meaninglessly.Let’s see, I think these pretty much map out a little chronology of the research:https://arxiv.org/abs/2112.00114 https://arxiv.org/abs/2404.15758 https://arxiv.org/abs/2406.06467 https://arxiv.org/abs/2512.12777First that scratchpads matter, then why they matter, then that they don’t even need to be meaningful tokens, then a conceptual framework for the whole thing.
marginalia_nu: I wonder if a language like Latin would be useful.It's a significantly much succinct semantic encoding than English while being able to express all the same concepts, since it encodes a lot of glue words into the grammar of the language, and conventionally lets you drop many pronouns.e.g."I would have walked home, but it seemed like it was going to rain" (14 words) -> "Domum ambulavissem, sed pluiturum esse videbatur" (6 words).
sheiyei: I assume in practice, filler words do nothing of value. When words add or mean nothing (their weights are basically 0 in relation to the subject), I don't see why they'd affect what the model outputs (except cause more filler words)?
gchamonlive: [delayed]
xpe: [delayed]
FurstFly: Okay, I like how it reduces token usage, but it kind of feels that, it will reduce the overall model intelligence. LLMs are probabilistic models, and you are basically playing with their priors.
sheiyei: If you take meaningless tokens (that do not contribute to subject focus), I don't see what you would lose. But as this takes out a lot of contextual info as well, I would think it might be detrimental.
adrian_b: The fact whether a language is isolating, or not, is independent on the redundancy of the language.All languages must have means for marking the syntactic roles of the words in a sentence.The roles may be marked with prepositions or postpositions in isolating languages, or with declensions in fusional languages, or there may be no explicit markers when the word order is fixed (i.e. the same distinction as between positional arguments and arguments marked by keywords, in programming languages).English has somewhat less syntactic role markers than other languages because it has a rigid word order, but for the other roles than the most frequent roles (agent, patient, beneficiary) it has a lot of prepositions.Despite being more economic in role markers, English also has many redundant words that could be omitted, e.g. subjects or copulative verbs that are omitted in many languages.
gchamonlive: Can't you just know that the earth is the center of the world by... like... just looking at how the world works?
estearum: Actually you'd trivially disprove that claim if you're starting from mechanistic knowledge of how orbits work, like how we have mechanistic knowledge of how LLMs work.
estearum: Right, there's probably something more subtle like "semantic density within tokens is how models think"So it's probably true that the "Great question!---" type preambles are not helpful, but that there's definitely a lower bound on exactly how primitive of a caveman language we're pushing toward.
bsza: I dont’t see the relevance, the discussion is over whether boilerplate text that occurs intermittently in the output purely for the sake of linguistic correctness/sounding professional is of any benefit. Chain of thought doesn’t look like that to begin with, it’s a contiguous block of text.
gchamonlive: You have empirical observations, like replicating a fixed set of inner layers to make it think longer, or that you seem to have encode and decode layers. But exactly why those layers are the way they are, how they come together for emergent behaviour... Do we have mechanistic knowledge of that?
strogonoff: A fundamental (but sadly common) error behind “tokens are units of thinking” is antropomorphising the model as a thinking being[0]. That’s a pretty wild claim that requires a lot of proof, and possibly solving the hard problem, before it can be taken seriously.There’s a better, less magical model of how LLMs work: they are essentially fancy autocomplete engines.Most of us probably have an intuition that the more you give an autocomplete, the better results it will yield. However, I am not sure this equally extends to the output of the autocomplete. It could well be that tighter output can remain of higher quality given the same amount of tokens—willing to be corrected by someone more familiar with NN architecture, of course.[0] Unless “thinking” is used as a term of art in context of “chain of thought” models, distinct from its regular meaning; sort of like what “learning” is in “machine learning”.
amelius: These models are autoregressive so I doubt they are running them across multiple clouds. And besides, a pause button is useful from a user's pov.
stainablesteel: i'm not sure it is, what's so useful about it?
bitexploder: That is not how CoT works. It is all in context. All influenced by context. This is a common and significant misunderstanding of autoregressive models and I see it on HN a lot.
taneq: Think before talk better though
taneq: More concise is dumber. Got it.
rokob: Did you stop reading because you saw a comma?
lanyard-textile: You'd be surprised -- This could match on the model's training to proceed using a tool, for example.
abejfehr: There’s a lot of debate about whether this reduces model accuracy, but this is basically Chinese grammar and Chinese vibe coding seems to work fine while (supposedly) using 30-40% less tokens
padolsey: This is fun. I'd like to see the same idea but oriented for richer tokens instead of simpler tokens. If you want to spend less tokens, then spend the 'good' ones. So, instead of saying 'make good' you could say 'improve idiomatically' or something. Depends on one's needs. I try to imagine every single token as an opportunity to bend/expand/limit the geometries I have access to. Language is a beautiful modulator to apply to reality, so I'll wager applying it with pedantic finesse will bring finer outputs than brutish humphs of cavemen. But let's see the benchmarks!
jagged-chisel: “Sophistication and literacy” are orthogonal to the peculiarities of a black box search engine.Those literate sophisticates would still be noobs at getting something useful from Google.
trenchgun: You’re a literature cycle behind. ‘Middle-layer shared representations exist’ is the observed phenomenon; ‘why exactly they form’ is the theory.You are also confusing ‘mechanistic explanation still incomplete’ with ‘empirical phenomenon unestablished.’ Those are not the same thing.PS. Em dash? So you are some LLM bot trying to bait mine HN for reasoning traces? :D
postalcoder: Pruning an assistant's response like that would break prompt caching.Prompt caching is probably the single most important thing that people building harnesses think about and yet it's mind share in end users is virtually zero. If you had to think of all the weirdest, most seemingly baffling design decisions in an AI product, the answer to "why" is probably "to not break prompt caching".
Demiurg082: CoT token are usually controled via 'extended thinking' or 'adapted thinking'. CoT tokens are usually not affected by the system prompt. There is an effort parameter, though, which states to have an effect on accuracy for over all token consumption.https://platform.claude.com/docs/en/build-with-claude/extend...
ForceBru: IMO "thinking" here means "computation", like running matrix multiplications. Another view could be: "thinking" means "producing tokens". This doesn't require any proof because it's literally what the models do.As I understand it, the claim is: more tokens = more computation = more "thinking" => answer probably better.
pxc: [delayed]
bitexploder: This helps, but the original prompt is still there. The system prompt is still influencing these thinking blocks. They just don’t end up clogging up your context. The system prompt sits at the very top of the context hierarchy. Even with isolated "thinking" blocks, the reasoning tokens are still autoregressively conditioned on the system instructions. If the system prompt forces "caveman speak" the model's attention mechanisms are immediately biased toward simpler, less coherent latent spaces. You are handicapping the vocabulary and syntax it uses inside its own thinking process, which directly throttles its ability to execute high-level logic.Nothing on that page indicates otherwise.
postalcoder: I strongly disagree with this method and would discourage others from using it too, especially if accuracy, faster responses, and saving money are your priorities. (If you're doing it for fun, I can't disagree with that.)While a caveman-style response works fine if you are just doing a single call-and-response with an LLM, it will actually hurt performance and drives up usage in agentic coding.Agentic workflows involve many turns of dialogue and rely on conversation compactions. When compacting, harnesses typically save a copy of the text exchange but strip out the tool calls in between. Because the agent relies on this text history to understand its own past actions, a log full of caveman-style responses leaves it with zero context about the changes it made, and the decisions behind them.To recover that lost context, the agent will have to execute unnecessary research loops just to resume its task. Terse prompting is going to lead to significantly higher usage and longer sessions.Beware.
jruz: only you auto-compact. auto-compact bad
shomp: me disagree
shomp: everyone who thinks this is a costly or bad idea is looking past a very salient finding: code doesn't need much language. sure, other things might need lots of language, but code does not. code is already basically language, just a really weird one. we call them programming languages. they're not human languages. they're languages of the machine. condensing the human-language---machine-language interface, good.if goal make code, few word better. if goal make insight, more word better. depend on task. machine linear, mind not. consider LLM "thinking" is just edge-weights. if can set edge-weights into same setting with fewer tokens, you are winning.
justonceokay: JOOK like when machine say facts. Machine and facts are friends. Statistics and “probably things” are facts but fuzzy.JOOK no like when machine like things. Machines do without like and without love. JOOK like and love enough for himself and for machine too.
Barbing: Interesting, what kind of weird?
rafram: They’re able to solve complex, unstructured problems independently. They can express themselves in every major human language fluently. Sure, they don’t actually have a brain like we do, but they emulate it pretty well. What’s your definition of thinking?
otabdeveloper4: When OP wrote about LLMs "thinking" he implied that they have an internal conceptual self-reflecting state. Which they don't, they *are* merely next token predicting statistical machines.
