Recipes for Thoughtfive days beforeI’d like to present to you a distinction: two different approaches I…Machine Thinking

Cravings and Aversions2021Though immediate effects of model flattening are already pretty dramatic, its largest…I’ll begin with an overly simplistic metaphor. Let’s visualize a ball rolling down a shallow slope. Driven by gravity, it’s seeking the easiest path down. Except the slope is full of ridges, channels, and nubs, and they make the ball bounce and veer in various directions.

Mental models2022I use the term “mental models” a lot, and so I figured…Intention and shared mental model space2021Hamilton Helmer pointed out this amazing connection between intention and shared mental…If we imagine that thinking is something like the motion of that ball, then our mental models provide the terrain for our thinking. The ridges, channels, and nubs are various concepts and connections between them that guide the thought process.

AI Developer Experience Asymptotes2023One of the key flaws in confusing what LLMs do with what…How we engage with LLMs2023When we engage with an LLM with an Opportunist Mind, we are…AI Baklava2024So we come up with ways to break software into smaller chunks…LLMs are getting quite good at capturing this terrain. Giving a prompt to an LLM often feels like throwing the thinking ball and letting the LLM roll it through the topography of the massive mental model compost it represents.

There are two key distinctions that I can see between that process and the actual thinking that we humans engage in.

Reinventing Organizations Redux2024This one is a bit out there, if only to connect some…Change2022So far, I carefully avoided the topic of change, presenting my problem-solving…First, LLMs are “free energy” thinkers. The force of gravity that pushes the ball in human thinking is the force of homeostasis: a resolute solicitor that drives us all to conserve energy and even find a surplus of it to ensure our own flourishing. We humans are “default-dead”: unless we abide by this force, we perish. Unlike us, LLMs have no such compulsion. In their realm, the energy is free. Thinking happens as a matter of “being run”. Put differently, their “thinking ball” is not rolling down the slope. Instead, it’s driven by an unknowable abundant force through what seems like the same sloped terrain as in human thinking, but is actually something radically different.

Model flattening2021If you know me, you were probably expecting me to inevitably stir…Of course, an LLM can report and even simulate perceiving the pull of homeostasis, but it will do only because it’s embedded into its thinking terrain, rather than being present as an animating force. This may not matter for many situations and can give a decent appearance of human thinking. However, at the limits, the simulacrum frays and the illusion of “thinking” decoheres.

The Relationship Polarity2021I’ve been feeling a little stuck in my progress with the Four…Second, for humans, thinking is a felt experience. We all know that we have happy thoughts and unhappy thoughts. We know that thinking of some subjects can make us sad or cheer us up. We experience the process of thinking as feelings that arise from visiting the concepts and connections in our mental models, because they all have feelings associated with it.

The third option2024The deeper insight here is that we are surrounded by polarities. Behind…We might struggle finding solutions to problems not because we don’t have the answers in our minds, but because the feeling of even approaching these answers is so intensely negative that it prevents us from visiting them.

Awareness of interoception2022Recently, I have been fascinated by the wonderful and mysterious part of…Even when we don’t have a particularly strong feeling bound to what we’re currently thinking, the process of thinking itself is imbued with feelings. We have a deeply ingrained desire to comprehend, to achieve higher accuracy of our mental models of the surrounding environment. The “lightbulb moment”, the euphoric “eureka!” of figuring something out is a feeling that’s encoded in us by the pressure of homeostasis.

Touching infinity2022As we explore the process of understanding, it may not be immediately…Even more bizarrely, the process of thinking is itself transforming the thinking terrain. Our little ball is creating its own grooves in it as it rolls along – and (this is where our metaphor really falls apart) conjures up new obstacles ahead. As we think, and as we experience the world while we’re thinking, we compound the feelings associated with our thoughts with the ones we just experienced. We’re so used to this continuous process that it takes conscious effort and skill to even notice it happening. Our existence is a gigantic, wondrous ball of yarn of feedback loops at multiple levels, and we ungrateful fools just live it like it’s nothing.

Reasoning boxes2023This story begins with the introduction of metacognition to large language models…Because the current generation of LLMs doesn’t have a way to experience feelings, their thinking processes will be limited to talking about feelings and logic-feeling: rely on vast parametric memory to reason about a felt experience without actually being able to have it. Again, this will be good enough for many situations, as long as they are constrained to where the logic-feeling suffices.

Zones of LLM predictability2023What do judgments about predictability of the LLM output tell us about…Composing graphs with Breadboard2023If you even occasionally scan these chronicles of my learnings, you will…When I see our current attempts to interact with LLMs and make sense of them, I often think of the pinball arcade: folks are getting quite good with the plunger, the flippers, and an occasional bump to make the ball roll as if guided by human thought. And getting good, the occasional decoherence of the illusion becomes more disappointing.

Limits of applying AI2023I’ve been thinking about the boundaries of what’s possible when applying large…The engine and the car2023This is already happening with LLM-based products. Very few current offerings that…Breadboard and OpalAugustJust over two years ago, I landed the first commit in a…We might be better off recognizing that the thought processes that the LLMs engage, while appearing similar to what we humans do, and even often producing matching results, are actually very different in nature. The less time we spend trying to stuff the square peg of machine thinking into the round hole of the human condition, the more of it we’ll have to actually get some value out of what LLMs can do.