Month: January 2025

Machine Thinking

I’ll begin with an overly simplistic metaphor. Let’s visualize a ball rolling down a shallow slope. Driven by gravity, it’s seeking the easiest path down. Except the slope is full of ridges, channels, and nubs, and they make the ball bounce and veer in various directions.

If we imagine that thinking is something like the motion of that ball, then our mental models provide the terrain for our thinking. The ridges, channels, and nubs are various concepts and connections between them that guide the thought process.

LLMs are getting quite good at capturing this terrain. Giving a prompt to an LLM often feels like throwing the thinking ball and letting the LLM roll it through the topography of the massive mental model compost it represents.

There are two key distinctions that I can see between that process and the actual thinking that we humans engage in.

First, LLMs are “free energy” thinkers. The force of gravity that pushes the ball in human thinking is the force of homeostasis: a resolute solicitor that drives us all to conserve energy and even find a surplus of it to ensure our own flourishing. We humans are “default-dead”: unless we abide by this force, we perish. Unlike us, LLMs have no such compulsion. In their realm, the energy is free. Thinking happens as a matter of “being run”. Put differently, their “thinking ball” is not rolling down the slope. Instead, it’s driven by an unknowable abundant force through what seems like the same sloped terrain as in human thinking, but is actually something  radically different.

Of course, an LLM can report and even simulate perceiving the pull of homeostasis, but it will do only because it’s embedded into its thinking terrain, rather than being present as an animating force. This may not matter for many situations and can give a decent appearance of human thinking. However, at the limits, the simulacrum frays and the illusion of “thinking” decoheres.

Second, for humans, thinking is a felt experience. We all know that we have happy thoughts and unhappy thoughts. We know that thinking of some subjects can make us sad or cheer us up. We experience the process of thinking as feelings that arise from visiting the concepts and connections in our mental models, because they all have feelings associated with it.

We might struggle finding solutions to problems not because we don’t have the answers in our minds, but because the feeling of even approaching these answers is so intensely negative that it prevents us from visiting them.

Even when we don’t have a particularly strong feeling bound to what we’re currently thinking, the process of thinking itself is imbued with feelings. We have a deeply ingrained desire to comprehend, to achieve higher accuracy of our mental models of the surrounding environment. The “lightbulb moment”, the euphoric “eureka!” of figuring something out is a feeling that’s encoded in us by the pressure of homeostasis.

Even more bizarrely, the process of thinking is itself transforming the thinking terrain. Our little ball is creating its own grooves in it as it rolls along – and (this is where our metaphor really falls apart) conjures up new obstacles ahead. As we think, and as we experience the world while we’re thinking, we compound the feelings associated with our thoughts with the ones we just experienced. We’re so used to this continuous process that it takes conscious effort and skill to even notice it happening. Our existence is a gigantic, wondrous ball of yarn of feedback loops at multiple levels, and we ungrateful fools just live it like it’s nothing.

Because the current generation of LLMs doesn’t have a way to experience feelings, their thinking processes will be limited to talking about feelings and logic-feeling: rely on vast parametric memory to reason about a felt experience without actually being able to have it. Again, this will be good enough for many situations, as long as they are constrained to where the logic-feeling suffices.

When I see our current attempts to interact with LLMs and make sense of them, I often think of the pinball arcade: folks are getting quite good with the plunger, the flippers, and an occasional bump to make the ball roll as if guided by human thought. And getting good, the occasional decoherence of the illusion becomes more disappointing.

We might be better off recognizing that the thought processes that the LLMs engage, while appearing similar to what we humans do, and even often producing matching results, are actually very different in nature. The less time we spend trying to stuff the square peg of machine thinking into the round hole of the human condition, the more of it we’ll have to actually get some value out of what LLMs can do.

Recipes for Thought

I’d like to present to you a distinction: two different approaches I take when using large language models (LLMs). I’ll call these two approaches “chat” and “recipe”.

In the first approach, I treat my interaction with Gemini, ChatGPT, et al. as a conversation: I type something, the LLM replies, I type again, etc. Very familiar, right? It’s how we talk to other humans. This is the “chat” approach and it seems to be quite dominant in the modern AI landscape, so I am not going to spend much time studying it.

Now, let’s  step out of the familiar and change the perspective a little bit. Instead of seeing it as an unstructured back-and-forth, let’s treat the turns of the conversation as going through steps in a recipe.  Each step contains a prompt from me and a generated reply from the LLM.

The shift is subtle, but it’s there. I am no longer chatting. I am guiding the LLM through the steps of a recipe for thought. “First think this way, then think that way, and now think like this”. With each step, the LLM’s replies get closer to the final product of the recipe. 

If you observe me use this approach with an LLM, you’ll notice a difference right away in how I treat the conversation turns. 

Suppose I type: “Write me a story” – and an LLM writes a story about … the Last Custodians of Dreams. It’s nice, but as I read it, I am realizing that I actually want a story about dolphins. 

When using the “chat” approach, I simply ask the LLM to fix the problem in the next conversation turn. “No, I want a story about dolphins”. 

With the “recipe” approach, I click the little “Edit” icon next to my first turn’s prompt and edit it to refine it:  “Write me a story about dolphins”. 

Okay, the response is much closer, but now I see how this story is too short. Hmm.. how do I make it stretch the response a bit? Perhaps I need to first let the LLM consider the full story ark – and then fill in the details?

So I edit the first turn prompt again: “Write an outline of  a story about dolphins. Anthropomorphize dolphins to tell a story about being alone, but not lonely.” Alright! This outline feels much closer to the story I want to see.

All this time, I was still in the first conversation turn! Now, I am ready to move to the next turn: presumably, asking an LLM to start adding details to the outline.

The end result might only look like a very brief conversation, but the outcome is typically much better: the continuous refinement of the prompt at each turn and carefully shaping the structure of the recipe results in the LLM output that I actually want.

The reason for that is the nature of the process of finding the right recipe. When building one, we try to better understand how an LLM thinks – and more importantly, how we think about the problem. I find the recipe approach very similar to mentoring: in the process of teaching it to follow my recipe, I learn just as much about my own cognitive processes. How do I typically think about writing a story? What are the steps that I myself take to ensure that the story is novel, coherent, and interesting?

This process of thinking about our own thinking is called metacognition. When using the “recipe” approach, we engage in metacognition for both the LLM and ourselves. Using our prompts as probes, we explore what an LLM is capable of and what prompts yield better results. We also are challenged to uncover our own tacit knowledge and turn it into a sequence (or a graph!) of prompts that an LLM can easily follow.

Metacognition is easier for some and more difficult for others. I know many folks who are experts at their stuff, but suffer from the “Centipede’s Dilemma”, unable to explain their own thought process – their expertise is entirely submerged  in the subconscious.

However, if metacognition is something that we’re used to, we can now, through the “recipe” approach, transfer our thought processes onto recipes. We can let LLMs do our thinking for us, because – plot twist! – we can make these recipes repeatable.

Observe: once I have a complete recipe for writing a story about dolphins, all I need is a way to substitute the word “dolphin” in the first prompt for another creature – and to re-run all the steps in the recipe! Now I can generate great stories about monkeys, doves, cats, and turtles. By parametrizing our recipes, we can make them generic and applicable to a variety of inputs.

Stories about animals are great, but let’s step back and engage our meta-metacognition. Hoo-boy, we must go deeper. What kind of metacognition are we seeing here? If we were to describe the pattern generally, what’s being described above is a process of transferring cognitive know-how – some expertise of thinking about a problem – into a repeatable recipe.

We all have cognitive know-how, even if we don’t realize it. More importantly, we all have potential for drawing value from this know-how beyond our individual use.

There’s a saying that goes “if you want something done right, do it yourself”. The thinking recipes allow us to amend the last part to “make a repeatable thinking recipe for it, and let the LLM do it”.

An expert in organizational strategy will undoubtedly have a wealth of cognitive know-how on the topic, from listening and interviewing, to running the sessions to generate insights, to coalescing disparate ideas into crystal clear definitions of the problem, and so on. Whenever this expert engages with a client, they have a playbook that they run, and this playbook is well-scripted in their mind.

I once was so impressed with a training session on team communication that I just had to reach out to the speaker and ask them for the script. If I had this script, I ought to have a way to run these sessions for all of my team. I was quite shaken to learn when the speaker revealed that what she had was more like a hundred fragments of the session puzzle that she puts together more or less on the fly, using the audience as a guide to which fragment to pick next. What to me looked like a simple linear flow was actually a meandering journey through a giant lattice of cognitive know-how.

In both cases, the cognitive know-how is trapped inside of the experts’ heads. If they wish to scale, go bigger or wider, they immediately run into the limitations of having just one person traversing their particular cognitive know-how lattices.

However, if they could transfer their lattices into  repeatable reasoning recipes, the horizons expand. At the very least, the LLM armed with such a recipe, can produce a decent first draft of the work – or ten! When I am applying repeatable reasoning recipes, my job shifts from following my own know-how to reviewing and selecting the work, produced by a small army of artificial apprentices.

Repeatable thinking recipes allow us to bust through the ceiling of generic thinking that the current LLMs seem to be stuck under – not by making them omniscient and somehow intuiting exactly what we’re asking for, but by investing a bit of time into turning our own cognitive know-how into recipes to help us think at scale.

This is not just a matter of scaling. With scale come new possibilities. When the overall cost of running through a recipe goes way down, I can start iterating on the recipe itself, improving it, adding new ingredients, and playing with new ideas – ideas that I wouldn’t have had the opportunity to explore without having my artificial apprentices.