Zones of LLM predictability

As you may know, large language models (LLMs) are smack dab in the middle of my tangle of interests presently, so you can bet I spend a lot of time talking with my friends and colleagues about them. One lens that seems to have resulted in fruitful conversations is the one related to predictability of output.

In this lens, we look at the LLM’s output as something that we can predict based on the input – and the reaction we might have on the outcomes. If we imagine a spectrum where the results are entirely unpredictable at one extreme, and can be predicted with utter certainty at the other – then we have a space to play in.

For a simple example, let’s suppose we’re asking two different LLMs to complete the sentence “roses are red, violets are …”. If one LLM just returns a bunch of random characters, while the other consistently and persistently says “blue”, we kind of know where we’d place these models on the spectrum. The random character one goes closer to an unpredictable extreme and the insistent blue one goes closer to the perfectly predictable end.

For ease of navigating our newly created space, let’s break it down into four zones: chaotic, weird, prosaic, and mechanistic. 

🌫️ Chaotic

In the chaotic zone dwell the LLMs that basically produce white noise. They aren’t really models, but random character sequence generators.  By the way, I asked Midjourney illustrate white noise, and it gave me this visage:

(It’s beautiful, Midge, but not what I asked for)

This zone is only here to bookend the very extreme of the spectrum. Suffice to say that we humans tend to only use white noise as means to an end, mostly judging it as useless on its own.

🐲 Weird

The adjacent zone is where the model outputs something that is weird and bizarre, yet strangely recognizable and sometimes even almost right. Remember the whole “hands” thing in the early generative imagery journey? That’s what I am talking about.

(“A normal human hand with five fingers” – whoopsie!)

This zone is where LLMs are at their creative best. Sure, they can’t count fingers, and yes, some – many! – outcomes are creepy and disturbing, but they also produce predictions that are just outside of the norms, while still retaining some traits that keep them outside of the chaotic zone. And that stirs creativity and inspiration in those who observe these outcomes. This is the zone where a model is more of a muse – odd and mysterious, and not very serious. Yet, when paired with a creative mind of a human, it can help produce astounding things.

📈 Prosaic

The prosaic zone is where an LLM produces mostly the results we expect. It might add a bit of flourish in bursts of creativity and insert an occasional (very safe) dad joke, but for the most part, that’s the zone that I also sometimes call the “LLM application zone”. If you ever spend time getting your retrieval-augmented generation to give accurate responses, or only return code results that can actually run – you’ve lived in this zone.

(“a happy software engineer working, stock photo” – oh yes, please! More cliche!)

My own explorations are mostly in this zone. The asymptotes I outlined earlier this year are still in place, and holding. If anything, time has shown that these asymptotes are firmer than I initially expected.

⚙️ Mechanistic

Another bookend of the spectrum is the mechanistic zone. At this point, LLM output is so constrained and deterministic that we become uncertain if using an LLM is even necessary: we might be better off just writing “old school” software that does the job.

The mechanistic zone is roughly the failure case for the current “AI” excitement. Should the next AI winter come, we’ll likely see most of the use cases shift toward this zone: the LLM either constrained, significantly scaled down in size, or entirely ripped out, replaced with code.

💬 A conversation guide

Now that we have the zones marked in the space, we can have conversations about them. Here are some interesting starter questions that generated insights for me and my colleagues:

  • How wide (or narrow) is each zone? For example, I know a few skeptics that don’t even believe that the Prosaic zone exists. For them, its width is zero.
  • How much value will be generated in each band? For instance, the Prosaic zone  is where most of the current attention seems to be. Questions like “Can we make LLMs be useful at an industrial scale? How much value can LLMs produce?” seem to be on everyone’s mind.
  • How will the value generated look for each band? What type of value comes out of the Weird zone? What about the Prosaic zone?
  • What kind of advancements – technological or societal – would it take to change the proportions of the zones?

For more adventurous travelers, here are more questions that push the boundaries of the lens:

  • What does “predictable”even  mean? If I know English, but don’t have the cultural background to recognize the “Roses are Red” ditty, I might find the “blue” perplexing as a completion. Violets are kind of purplish, actually.
  • What do judgments about predictability of the LLM output tell us about the observer? What can we tell about their expectations, their sense of self, and how they relate to an LLM?
  • What is it that LLMs capture that makes their output predictable? What’s the nature of that information and what might we discern about it?

As you can tell, I am pretty intrigued by the new questions that large language models surface to us. If you’re interested in this subject as well, I hope this lens will be useful to you.

Doers, Thinkers, and Magicians

I’ve been reflecting on my experiences of working with developers and developer ecosystems, and I realized that there’s a really interesting twist on the typical “early adopter” story that’s been hiding in the back of my mind.

Let’s suppose that you and I are spinning up a new developer experience project. We have a fledgling developer surface that we’re rapidly shaping and growing, and trying to to get it right by making contact with its intended audience.

The very first of these developers are commonly called early adopters, which originated from Everett Roger’s book Diffusion of Innovations. It is my experience that these early adopters can be further broken into three subgroups: doers, thinkers, and magicians – and the presence of all three is required for the developer surface to successfully navigate toward broad adoption and bring forth our hopes and dreams for it. 

More than that, the mix of these subgroups heavily influences the arc that the project will follow, the pattern into which the developer ecosystem around the developer surface will settle into – should it succeed.

To explore this notion, let’s zoom in on each subgroup.

💪 Doers

The doer early adopters are typically the most populous sub-group. They are very easy to identify: they do stuff with our developer surface, making things with it, poking at it here and there.

Doers bring energy and create the sense of a bustling community emerging around technology or products, powered by technology. They are eager, excited, typically with some tinkering time to spare. They are boisterous, peppering technology or product builders with questions and suggestions. Most of their questions and feedback of a very practical nature: they just want to make our thing do their bidding.

Doers often don’t have enough technical skills to just start doing what they want – not just because our developer surface is new, but because there might be gaps in their understanding of the surrounding technologies. As such, they need patient and consistent investment of hand-holding, be that tutorials, hackathons, or individual support.

If our project has doers in the early adopter mix, we have a key ingredient. We have the potential energy that can be transferred into forward progress. This subgroup of early adopters provides valuable insights on the usability of the technology or product, and their contagious enthusiasm attracts new customers.

If our project doesn’t have doers, we might as well not have a project. The absence of doers in the early adopter mix is a warning sign that we might have come up with something that is deeply uninteresting, incomprehensible, or otherwise impossible to access by doers.

🧠 Thinkers

The thinker early adopters usually come in much smaller numbers than doers. In some ways, thinkers can be seen as a subset of doers, with a key distinction: they actually spend time imagining possibilities and exploring the possibilities of the technology they are studying. They might be playing with the developer surface themselves, but they could also just be observing doers and identifying interesting potentialities in the churning soup of ideation that the doers produce.

One of my first encounters with thinkers was back in the early 2000s, when blogs.msdn.com was introduced as part of Microsoft Developer Network. I was a fairly new doer-inclined developer myself, and I was fascinated by the blog posts from Dare Obasanjo or Nikhil Kothari on the then-nascent .NET framework. They moved from the pragmatic “here’s how you do <blah>” to open-ended cross-blog conversations about second order effects and implications of technology they were using, as well as introduced various completely new ideas into how it might be used. For me, whole new frontiers opened up and connections were made between concepts that I viewed as entirely unrelated – all the while making me ever more energized about the technology.

This is the role of the thinkers: they hold our developer surface in their hands lightly, turning this way and that, and applying intellect and curiosity to consider its potential.

When our project has thinker early adopters, we have acquired a source of more durable energy. While doers do introduce the initial energy, their explorations, being mostly pragmatic and practical, often peter out and lose steam without the influx of new ideas. Thinkers are the ones who introduce these new ideas, and reinvigorate the excitement and enthusiasm.

Not having thinkers as early adopters means that the project is in danger of getting stuck in a premature local maxima. When the doers uncover all the obvious use cases, these might not be the ones that propel our developer surface toward our intended destination – and we’ll have to contend with being stuck in what we view as “mediocre success” or just breaking down our camp and admitting defeat. It is the thinkers who help move doers move beyond the initial local maximas into adjacent areas that are more likely to hold the value we’re looking for.

✨ Magicians

The final ingredient in the early adopter mix are the magicians. The magician early adopters are even more rare. Even having one is an incredible stroke of fortune, and something that we are obliged to cherish.

The magicians are both doers and thinkers, but they have this weird knack for building amazing things that blow your mind. I wish I knew how that works. In the past, I attempted to grow magicians out of thinkers and/or doers, but there doesn’t seem to be a path from here to there.

The magicians are usually experienced and seasoned developers. They grasp the idea behind our developer surface in seconds, and intuitively see the landscape of opportunities. Then, they reach for the simplest path toward the opportunity that appears most valuable – and for some unfathomable reason – they are usually right. They connect bits and pieces of our stuff into something that suddenly looks solid and – this is a common effect – blindingly obvious. “What the hell?! How?! … Oh… Why didn’t I think of this before?!” is a common reaction to a magican’s artifact.

When I worked in Chrome Web Platform, I was very lucky to have a handful of these magicians around me. For some reason, the Chrome team’s DevRel contingent was rife with them. In a more recent memory, Simon Willison’s work on llm has the same magician quality.

The presence of magicians significantly strengthens our project’s chances of broad adoption. Like thinkers, the early adopter magicians uplevel the current understanding of what’s possible – but they do it in an explosive, revolutionary way. “Now <bar> is possible, here’s some code” – and everyone freaks out, dropping all the previous work, being able to not just imagine the potential of the next frontier, but actually try it. This explosive amount of energy that the magicians inject into a project can catapult it way beyond our initial intentions. We just need to be patient, hang on to our dear lives while the starship of a new idea streaks forward, and be ready to explore the crazy new planet it will land on.

Not every developer surface gets magician early adopters. A project can still be moderately successful even in the dearth of magicians. A very common side effect of that is that the developer community grows large and appears vibrant, but the outcomes it produces tend to be on the lower side of our expectations. Low-magician developer ecosystems tend to have very thin long tails, with only a few well-settled participants forming a narrow head.

🧪 The right mix

A reasonable question might be: what is the right mix for our project? Disappointingly, there is no satisfying answer. Developer-oriented projects, at least in my experience, all tend to follow roughly the same shape of proportions: lots of doers, a few thinkers, a couple – if any – of magicians. It is usually the presence of magicians and thinkers that significantly improves the chances of our project going somewhere good. So, if I could offer any advice to budding developer experience makers, it would be this: seek out the thinkers and the magicians. They are the key to passing through the early adoption stage.

Placing and wiring nodes in Breadboard

This one is also a bit on the more technical side. It’s also reflective of where most of my thinking is these days. If you enjoy geeking out on syntaxes and grammars of opinionated Javascript APIs, this will be a fun adventure – and an invitation.

In this essay, I’ll describe the general approach I took in designing the Breadboard library API and the reasoning behind it. All of this is still in flux, just barely meeting the contact with reality.

One of key things I wanted to accomplish with this project is the ability to express graphs in code. To make this work, I really wanted the syntax to feel light and easy, and take as few characters as possible, while still being easy to grasp. I also wanted for the API to feel playful and not too stuffy.

There are four key beats to the overall story of working with the API:

1️⃣ Creating a board and adding kits to it
2️⃣ Placing nodes on the board
3️⃣ Wiring nodes
4️⃣ Running and debugging the board.

Throughout the development cycle, makers will likely spend most of their time in steps 2️⃣ and 3️⃣, and then lean on step 4️⃣ to make the board act according to their intention. To get there with minimal suffering, it seemed important to ensure that placing nodes and wiring them results in code that is still readable and understandable when running the board and debugging it.

This turned out to be a formidable challenge. Unlike trees, directed graphs – and particularly directed graphs with cycles – aren’t as easy for us humans to comprehend. This appears to be particularly true when graphs are described in the sequential medium of code. 

I myself ended up quickly reaching for a way to visualize the boards I was writing. I suspect that most API consumers will want that, too – at least at the beginning. As I started developing more knack for writing graphs in code, I became less reliant on visualizations.

To represent graphs visually, I chose Mermaid, a diagramming and charting library. The choice was easy, because it’s a library that is built into Github Markdown, enabling easy documentation of graphs. I am sure there are better ways to represent graphs visually, but I followed my own “one miracle at a time” principle and went with a tool that’s already widely available.

🎛️ Placing nodes on the board

The syntax for placing nodes of the board is largely inspired by D3: the act of placement is a function call. As an example, every Board instance has a node called `input`. Placing the `input` node on the board is a matter of calling `input()` function on that instance:

import { Board } from “@google-labs/breadboard”;

// create new Board instance
const board = new Board();
// place a node of type `input` on the board.
board.input();

After this call, the board contains an input node.

You can get a reference to it:

const input = board.input();

And then use that reference elsewhere in your code. You can place multiple inputs on the board:

const input1 = board.input();
const input2 = board.input();

Similarly, when adding a new kit to the board, each kit instance has a set of functions that can be called to place nodes of various types on the board to which the kit was added:

import { Starter } from “@google-labs/llm-starter”;

// Add new kit to the existing board
const kit = board.addKit(Starter);

// place the `generateText` node on the board.
// for more information about this node type, see:
// https://github.com/google/labs-prototypes/tree/main/seeds/llm-starter#the-generatetext-node
kit.generateText();

Hopefully, this approach will be fairly familiar and uncontroversial to folks who use JS libraries in their work. Now, onto the more hairy (wire-ey?) bits.

🧵 Wiring nodes

To wire nodes, I went with a somewhat unconventional approach. I struggled with a few ideas here, and ended up with a syntax that definitely looks weird, at least at first.

Here’s a brief outline of the crux of the problem.  In Breadboard, a  wire connects two nodes. Every node has inputs and outputs. For example, the `generateText` node that calls the PaLM API `generateText` method accepts several input properties, like the API key and the text of the prompt, and produces outputs, like the generated text.

So, to make a connection between two nodes meaningful, we need to somehow capture four parameters:

➡️  The tail, or node from which the wire originates.
⬅️ The head, or the the node toward which the wire is directed.
🗣️ The from property, or the output of the tail node from which the wire connects 
👂 The to property, or the input of the head node to which the wire connects

To make this more concrete, let’s code up a very simple board:

import { Board } from "@google-labs/breadboard";

// create a new board
const board = new Board();
// place input node on the board
const tail = board.input();
// place output node on the board
const head = board.output();

Suppose that next, we would like to connect property named “say” in `tail` to property named “hear” in `head`. To do this,  I went with  the following syntax:

// Wires `tail` node’s output named `say` to `head` node’s output named `hear`.
tail.wire(“say->hear”, head);

Note that the actual wire is expressed as a string of text.  This is a bit unorthodox, but it provides a nice symmetry: the code literally looks like the diagram above. First, there’s the outgoing node, then the wire, and finally the incoming node.

This syntax also easily affords fluent interface programming, where I can keep wiring nodes in the same long statement. For example, here’s how the LLM-powered calculator pattern from the post about AI patterns looks like when written with Breadboard library:

math.input({ $id: "math-question" }).wire(
  "text->question",
  kit
    .promptTemplate(
      "Translate the math problem below into a JavaScript function named" +
      "`compute` that can be executed to provide the answer to the" +
      "problem\nMath Problem: {{question}}\nSolution:",
      { $id: "math-function" }
    )
    .wire(
      "prompt->text",
      kit
        .generateText({ $id: "math-function-completion" })
        .wire(
          "completion->code",
          kit
            .runJavascript("compute->", { $id: "compute" })
            .wire("result->text", math.output({ $id: "print" }))
        )
        .wire("<-PALM_KEY", kit.secrets(["PALM_KEY"]))
    )
);

Based on early feedback, there’s barely a middle ground of reactions to this choice of syntax. People either love it and find it super-cute and descriptive (“See?! It literally looks like a graph!”) or they hate it and never want to use it again (“What are all these strings? And why is that arrow pointing backward?!”) Maybe such contrast of opinions is a good thing?

However, aside from differences in taste,  the biggest downside of this approach is that the wire is  expressed as a string: there are plenty of opportunities to make mistakes between these double-quotes. Especially in a strongly-typed land of TypeScript, this feels like a loss of fidelity – a black hole in the otherwise tight system. I have already found myself frustrated by a simple misspelling in the wire string, and it seems like a real problem.

I played briefly with TypeScript template literal types, and even built a prototype that can show syntax errors when the nodes are miswired. However, I keep wondering – maybe there’s an even better way to do that?

So here’s an invitation: if coming up with a well-crafted TypeScript/Javascript API is something that you’re excited about, please come join our little Discord and help us Breadboard folks find an even better way to capture graphs in code. We would love your help and appreciate your wisdom.

Traits of a maker mindset

The whole concept of makers has been on my and my colleagues’ mind a lot, especially with their current rise in prominence with generative AI.  

A fun question popped out in one of the conversations: “How might one tell a maker from a non-maker?” The idea of a “non-maker” immediately felt a bit ridiculous. Since we’ve already established that “maker” is a mindset, it’s pretty clear that shifting out of that mindset is going to land us in a “non-maker” territory. But when we are in the maker mindset, what are the traits and characteristics that might stand out?

After talking to a few folks about this, a rough list started to emerge. It’s still not quite right, but I thought I would share it early for your perusal.

So far, I have four traits that seem to resonate whenever I talk about a “maker mindset” to others. These traits are option-seeking, craving creative friction, zagging, and optimism.

🔢 Option-seeking

When in the maker mindset, we tend to seek more options. If offered a single way to solve a problem, no matter how simple and elegant, a maker in me will perceive it with skepticism. There’s something about optionality and preserving the agency to choose these options that is highly important to a maker mindset. The more knobs, the merrier. The more choices, the more exciting. This is plainly in conflict with the mainstream theories of user experience for the common consumers. When I am in a non-maker mindset, I want a simple single solution that gets the job done. When wearing a maker hat, I will steer away from it.

This might be one of the reasons why open source projects and modular solutions are attractive to makers. Being able to pick and choose whichever pieces I want and combine them in whatever way I want – and have an option to change mind – are a big part of the whole maker experience.

💪 Craving creative friction

Very related, a maker mindset frowns on well-solved problems. We will rarely find makers tinkering with obvious or fully understood problem spaces. For makers,  things have to be difficult and challenging to be attractive. Too much polish is a bit of a letdown – it means that someone already solved all the fun problems. Maker mindset cherishes creative friction – the presence of a challenge in the process of making is what stirs creative juices.

This is why makers don’t mind messing with stuff that isn’t yet fully baked. One of my colleagues put it as “makers are in it for the problems, not solutions.” This is a bit too blunt, but I can’t disagree with the sentiment. For makers, it’s about the journey, though the tantalizing promise of a destination definitely helps.

🔀 Zagging

Makers love to zag when everyone zigs. When in the maker mindset, we tend to look for opportunities that are odd-shaped compared to what everyone else is seeking.

Makers rejoice when it looks like they’re doing something weird.  Being outside of the curve means that there’s a chance we’re ahead of it. Makers wholeheartedly take this chance. Even if zagging doesn’t pay off, the thrill of exploring the wilderness is a powerful force that animates makers.

When everyone is making an AI-powered chatbot, makers are playing with meta-reasoning and autonomous agents. When everyone finally catches onto the agents, makers move on to something else.

☀️ Optimism

This one I am least certain about. It definitely rings true, but I don’t know if the word “optimism” captures the gist. When in the maker mindset, we are driven by a belief that our actions will lead to some outsized outcomes. Somehow, somewhere, we will hit that exponential curve, and things will truly get out of control. There’s a sense of “it’s definitely not working now, but just wait and see” that is like fresh air for makers.

Many makers are techno-optimists, who – often implicitly – believe that technology will solve all problems and do more good than evil over the long run. After all, making something often means creating new technology – be it physical, organizational, or social. And definitely, most makers believe that making something is better than not making it. Making is art, and all maker’s art has purpose, animated by often completely unfounded confidence in better outcomes.

📐 Designing for makers

Despite this list being so unkempt, we can start gleaning some interesting insights about designing user experiences that attract makers.

Makers flip the script on the conventional wisdom of delivering polished, simple experiences to users. Steve Krug’s “Don’t make me think” turns into “Give me an interesting puzzle!” and sometimes into “Ooh, this mess of a product looks perfect for my project”. For makers, rough edges signal exciting possibilities. I am still learning what this all means, but it’s starting to feel that product design for makers is dramatically different from design for users not in the maker mindset.

If such a difference does indeed exist, it’s interesting to consider how a product might be perceived by the same person, but from different mindsets. And perhaps even more granularly: some tools I want to have lots of knobs and options and rough/unexplored edges, and some of them I just want to work, even when I am in the maker mindset.

It seems overwhelming – and likely foolhardy – to establish a precise taxonomy here. The only recipe I know is to have the maker’s intuition. To design for makers, one has to have accumulated a lot of experience of being a maker. There doesn’t seem to be any way around that.

The engine and the car

The whole large language model space is brand new, and there are lots of folks trying to make sense of it. If you’re one of those folks, here’s an analogy that might come handy.

Any gasoline-powered car has an engine. This engine is typically something we refer to as a “V8” or “an inline 4” or sometimes even a “Wankel Rotary Engine”. Engines are super-cool. There are many engine geeks out there – so many that they warrant a video game written for them.

However, engines aren’t cars. Cars are much more than their engines. Though engines are definitely at the heart of every engine, cars have many additional systems around them: fuel, electrical, steering, etc. Not to mention safety features to protect the passengers and the driver, and a whole set of comforts that we enjoy in a modern car. Pressing a button to roll down a window is not something that is done by the engine, but it’s definitely part of the whole car experience.

When we talk about this generation of AI systems, we typically talk about large language models (LLMs). In our analogies, LLMs are like engines. They are amazing! They are able to generate text by making inferences from the massive parametric memory accrued through training over a massive corpus of information.

However, they aren’t cars. One of the most common mistakes that I see being made is confusing engines (LLMs) with cars (LLM-based products). This is so common that even people who work on those products sometimes miss the distinction.

When I talk to the users of the PaLM API, I see this confusion show up frequently in this manner: developers want to reproduce results from the LLM-based products like Bard or ChatGPT . When they try to get the same results from the API, they are disappointed that they don’t match. Factuality is lacking, API can’t go to the internet and fetch an article, etc. 

In doing so, they confuse the engine with the car: the API, which offers access to the model, is not the same as the products built with it. With an LLM API, we have a big-block V8. To make it go down the road, we still need to build the car around it.

 To build on this analogy, we live in the early age of cars: the engines still figure prominently in the appearance and daily experience of a vehicle. We still have to turn the crank to start the car, oil the engine frequently, and be savvy enough to fix minor problems that will definitely arise.

As our cars become more refined, the engines get relegated into a well-insulated compartment. Users of cars rarely see them or operate on them directly.

This is already happening with LLM-based products. Very few current offerings that you might encounter in public use are LLMs that are directly exposed to the user.

So, when you use a chat-based system, please be aware that this is a car, not the engine. It’s a tangle of various AI patterns that are carefully orchestrated to work as one coherent product. There is likely a reasoning pattern at the front, which relies on an LLM to understand the question and find the right tool to answer it. There is likely a growing collection of such tools – each an AI pattern in itself. There are likely some bits for making sure the results are factual, grounded in sources, and safe.

As the LLM products become more refined, the actual value niches for LLMs become more and more recognizable. Instead of thinking of one large LLM that does everything, we might be seeing specialization: LLMs that are purpose-designed for reasoning, narration, classification, code completion, etc. Each might not be super-interesting in itself, but make a lot of sense in the overall car of an LLM-based product.

Perhaps unsurprisingly, the next generation of cars might not even have the same kind of engine. While the window control buttons and the steering systems remain the same, the lofty gasoline engines are being replaced with electric motors that fit into a fraction of space. The car experience remains more or less the same (aside from the annoying/exhilarating engine noise), but the source of locomotion changes entirely.

It is possible that something like this will happen with LLMs and LLM-based products as well. The new open space that was created by LLMs will be reshaped – perhaps multiple times! – as we discover how the actual products are used. 

Steady winds, doldrums, and hurricanes

It just so happened that this year, many of my friends and colleagues ended up looking for new opportunities, and in our conversations, I ended up shaping this metaphor. As most metaphors, it’s not perfect, but hopefully, will stir some new insights for you.

We kept trying to describe the energy within organizations and the animating forces that move them. These forces can make our lives inside these organizations a delight – or a complete and utter misery. It seemed like a good idea to understand how these forces might influence us and find ways to sense these forces early. Preferably, even before committing to join a new team.

The idea of presenting these forces as winds seemed rather generative. If we look at the innovation S-curve, we can spot three different kinds: steady, doldrums, and hurricanes. They don’t exactly match the stages I outlined back in the original article. Instead, these winds follow the angle of the S-curve slope.

⛵The steady winds

Steady winds are consistent. We can feel them going in one direction and they change infrequently. Apparently sailors love them, because they provide a predictable way to navigate. Even if it’s not a tailwind, a steady wind can be harnessed through tacking.

Similarly, organizations that are in the upslope of their development tend to have a relatively consistent animating force that feels like a steady wind. Usually, there’s some big idea, some intention, and a group of highly-motivated individuals who set the direction of this wind.

We can feel it as soon as we step into an organization. It usually appears as the ambition of the  charismatic leader/founder, who knows exactly what they want and is doing everything they can to make it possible. More rarely, it might also appear as a set of ideals that depict some future state of the world – and this team has the fire (and funding) to bring it forth.

Steady winds aren’t always great. Sometimes, a steady wind’s direction is simply incompatible with where we want to go. It might trigger aversion in us, or be in discord with our own principles. The leader might be charismatic, yet have character quirks we deem appalling. The big idea might indeed be big, but no matter how much we try to suspend disbelief, we keep finding it laughable.

At the same time, steady winds bring clarity. They give us a very good idea of what this team is about and where they are going. These folks are going someplace. It’s on us to choose to go there with them.

When considering a new team and sensing a steady wind that moves it, ask yourself: is this wind aligned with what I myself want to do? Does it stir fire in my belly? At the very least, can I tack into this wind in a way that moves me where I want to go? And of course: am I at the place where I want to go on an adventure?

Because joining steady-wind teams definitely brings adventure. It might be glorious and awesome, or it might be like the Donner party, with all the fixin’s of freezing to death, scurvy, and/or dysentery. Only time will tell.

If the wind is favorable and adventure is what you seek, such a team might be a good fit.

⛳ The doldrums

Prior to the invention of motors, doldrums were a terrifying thing for sailors. Doldrums meant that to go anywhere, we have to break out our oars and turn our own sweat into motion. There is no wind to help us go anywhere.

Organizations tend to experience doldrums at the top of the S-curve. Once the niche is fully explored and the product or service is optimized to fit it exactly, it is really not clear where to go next. All successful products end up experiencing this. We can see this as fewer interesting changes in them, and a deluge of incremental improvements that may sound exciting, but don’t actually add up to anything like the stuff the organizations used to produce at the upslope.

To get anything done in this organization requires some form of consensus. There are usually processes. Approvals. Reviews. Endless, exhausting discussions. When in doldrums, there’s a prevailing sense of powerlessness, often accompanied by a weird combination of comfort and toil. Everything is hard, but at least it’s exactly the same as yesterday.

Leaders who used to produce the steady wind at the upslope typically leave when they encounter the doldrums. We won/lost. Why stay? Instead, they are replaced by sailors. These leaders concentrate more on preserving what was accumulated so far. Risk is frowned upon. 

It’s not like nothing gets done in organizations stuck in doldrums. There’s always activity, and an appearance of movement. To create this appearance, there’s a syndrome of chronic bigness: every new initiative is bigger than the previous one, ever more bombastically described and painted in brighter colors. Underneath is the same dull surface of still water.

Doldrums aren’t necessarily a red flag for joining. If what you’re looking for is the steady stillness of boring, yet never-ending work, that might just be the place. Large bureaucracies like government agencies and corporate giants have large organizational swaths that live in the doldrums – and necessarily so. Not everything needs to be an adventure. Sometimes, the slow and steady beat of the oars is the only thing that keeps the grand ship inching forward.

However, if you’re seeking something to fill your sails, please keep walking. Committing to a doldrums team will suck the soul out of you and is not worth it.

🌀 The hurricane

The final part of our story is hurricanes. Sailors caught in storms just hang on to their life, trying to survive and keep the ship afloat.

Similarly, organizations find themselves in turbulent waters. This typically happens on the downslope of the innovation S-curve, when the quiet ride through the doldrums is eventually replaced by contact with reality.

In the hurricane, there’s lots of wind. It’s blowing in all directions. To continue our metaphor, the wind is the animating force that is usually created by organization’s leaders and their intentions. In the hurricane, this intention is chaotic and unpredictable. And it’s usually reactive,  spurred by some external threat.

The downslope of the S-curve isn’t fun. The collective anxiety of leaders who got used to the doldrums creates a vicious cycle, exacerbating the situation further. The overall direction is unclear, but not for the lack of effort. There’s lots of movement, and lots of force, all going in circles.

On very, very rare occasions, a new leader emerges and manages to set the steady wind, bringing the team out of chaos. I have seen it happen, but haven’t experienced it myself. 

Unless you’re a total glutton for punishment or have a severe savior complex itch, it is difficult to recommend joining an organization in the hurricane. The trouble is, it’s often hard to tell. It is in nobody’s interest to reveal the true state of disorder to the candidates. So the hurricane-embattled team might appear as either doldrums or steady winds, depending on who you ask.

One of my colleagues recommended this approach: find someone on the inside. Someone who might still be there or left recently. Ask them candidly: “is this a sh*t show?” Watch their reaction and prod a bit. Look for stories that sound like aimless grasping for straws and high anxiety among the team’s leaders. Those are the telltale signs of the hurricane.

Diving into unpredictability

My previous essay on the topic of unpredictability generated a few insightful comments from my colleagues and friends. One of them led to this vignette.

It is very tempting to imagine that some people are just generally less susceptible to the discomfort of unpredictability than others. It might even feel like coming up with a way to gauge one’s ability to thrive in unpredictable situations would be a useful tool.

My intuition is that this stance needs a bit more nuance. As humans, we all abhor unpredictability. We rarely actually “thrive” in it, at least over the long run. The metaphor that comes to mind is diving.

Some people are great divers. They can spend a significant amount of time under water. They can go deep and explore the parts of the seabed inaccessible to anyone  else. At the same time, nobody would claim that great divers can actually live in the depths of the sea. We all need to come up for air.

In this metaphor, unpredictability is water. If we stay in it for too long, we drown. I see the desire for predictability – or homeostasis – as a gravity-like force that animates all of us. It isn’t something we can completely detach from – though stoics and buddhists try. Just like air that we need to breathe, predictability is something that is essential for nourishing our minds. Our minds are predictive systems. Unpredictability is anti-mind.

Great divers – those who can endure unpredictability better than others – are those who invest generously into techniques and strategies that enable them to stay in the deep longer and even enjoy it. However, prolonged exposure to it will still take the toll, and the need to come up for air will always win over.

Diving into predictability is hard work. Just like with any good diver, if they are making it look effortless, we can bet that a lot of effort was put in before. And just like with any good diver, the “true pirates” who appear as thriving in unpredictability are nearly always those with the decades of practice, with all the blood, sweat, tears, and scars such a practice entails. One of the foundational elements of this practice is finding a way back to the fresh air of a predictable environment.

Sometimes you gotta hit the wall

I’ve probably written about this a few years back, but I still find this mantra useful and worth repeating. It applies to the situations where we’re stuck but we don’t know that we’re stuck – not yet.

When we’re in this state, we have a sense that we’re still moving forward, and we’re making all the right moves. We get upset when our friends or colleagues cautiously share with us that we might be spinning our wheels. Yeah, there’s some loss of traction, but if we just keep going, we will figure this thing out. Just one more push.

Particularly for technologists and other admirers of modernist thinking, the likelihood of becoming stuck in this way somewhere along our careers is pretty high. The idea that if we know what we’re doing and we’re doing everything right, then things should work out according to our plans – it’s just so damn seductive.

We can last quite a bit of time in this purgatory of delusion. There are just so many options to choose from. It’s the environment around us that is all wrong. Someone is actively conspiring against us. There are some indicators that show clearly that we’re still moving forth as planned. The more clever and quick-thinking we are, the more likely we are to come up with a story that keeps us stuck.

Inevitably, there’s a moment when it all comes apart. We finally hit the wall. We’re in shock,  feeling injured by the cruel reality and betrayed by it. But – it is only when we hit that wall do we get the chance for self-reflection. There’s an opportunity, when the shell of self-delusion is cracked, to actually gain some clarity. We might remember our colleagues’ gentle hints and worried faces, the early signs of stuckness we’ve chosen to ignore, and the now-obviously illustory stories we’ve told ourselves.

Should we experience it, this moment is a significant milestone. It allows us to create a little space between reality and the stories we tell ourselves. It allows us to hold our stories as objects instead of being subject to them. Experienced once, it’s a perspective that can be temporarily lost, but never fully forgotten. Next time the allure of modernism tempts us, we might still feel the pull – but think twice about answering the call. Once we’ve hit that wall, we’ve learned that “knowing what we’re doing” and “doing everything right” are just stories we tell ourselves, and they have little to nothing to do with reality.

The somewhat sad part is that this lesson can not be taught. No amount of explanation or teaching will bring one closer to the precious insight without the painful experiential part. This particular bit of wisdom can only be gained by face planting into the unyielding, uncaring reality at full speed. Sometimes you just gotta hit the wall.

Pace layers of predictability

I’ve been talking about pace layering a lot, within the dandelions and elephants story and in other pieces. Here’s yet another way to apply this framing.

One thing that stood out to me was this notion of a common force that animates their formation. Here’s my guess: this force is the desire for predictability.

For example, in organizations, there is usually a larger group — often a dominant majority — of folks who seek a fairly predictable environment. They usually feel rather uncomfortable about the uncertainty of a problem space, and you can hear this discomfort in phrases like “just tell me what to do” or “we need clear timelines and deliverables”. There is nothing wrong with this stance. I find myself in this group at times, especially when I am in fallback. Another way to think of it is that we find ourselves here when our predictability footprint expands. This group tends to seek “one true shared mental model” and usually reacts with pain to disconfirming evidence, forming the slowest-moving predictability pace layer.

To manage that pain, there’s a much smaller group of those who are able to face unpredictability a bit more. These folks have a slightly higher tolerance for ambiguity and can experience polarities not as a series of harrowing swings back and forth, but rather as one nuanced system. They can play with multiple conflicting mental models and see disconfirming evidence as input, rather than pain inflicted. This ability is not without cost, and usually requires continuous support and scaffolding.

This smaller group forms a much thinner and faster-moving predictability pace layer on top of the slower layer. When an organization employs such folks effectively, they become instrumental in gently evolving the “one true shared mental model” of the larger group in a direction that is long-term productive for this organization. This is the stance that I myself enjoy the most and feel that resonant sense of aligning with Purpose.

Sometimes a team is lucky enough to have true pirates: a handful of people whose predictability footprint is so small that they are able to go where even the most ambiguity-tolerant people dare not touch. Disconfirming evidence is necessary sustenance to them. These folks can examine the load-bearing beliefs in ways that would terrify or at least profoundly upset most members of the organization. They can do deep ontological dives and come back unfazed, with insights that have the power to shape the future of the organization.

When employed effectively, these are the peeps that establish foundational frameworks within which the gentle evolution of the organization occurs. This is the fastest-moving predictability layer where I aspire to be, even though most of the time, I pop up to that layer only momentarily.

Of course, this is not some sort of normative layout of layers that every organization must possess. My guess is that each organization has their own layer configurations. 

What’s important about them is that they are typically mostly separate and insular from each other – and for good reason. Exposing folks at the lowest layer to the more intense pace of the higher layer will rapidly trigger allergic response. And giving them a sense of the top layer’s pace will seem like Lovecraftian horror. Boundaries tend to arise to prevent such shenanigans.

What’s most curious is that these pacing layers rarely respect hierarchies and organizational structures. There could be leaders of large teams who crave predictability. There could be random junior team members who are surprisingly great at diving into the abyss of uncertainty and bringing back useful insights. The ability to tolerate unpredictability changes with one’s circumstances, and hierarchies tend to strive for permanence.

As a result, the insulation between layers tends to develop in haphazard and unproductive ways. Within predictability-craving organizations, those who are comfortable with uncertainty are deemed “troublemakers” and shunned or ignored. Conversely, folks who desire more predictability are labeled as “unimaginative” in places where experimentation and exploration are valued. Instead of recognizing mutual significance and collaborating, teams at different predictability pace layers resent each other’s differences.

In practice, this means that harnessing the full potential of this pace layering and using it to the advantage of the organization is very uncommon. I keep wondering what a team that is designed around predictability pace layers – rather than in spite of them – would even look like.

Given that overall levels of unpredictability around us seem to be ever-increasing, this might be an important challenge to take on. Perhaps you, my readers, will find an opportunity here.

AI Patterns and Breadboard

In my last post, I kept talking about AI patterns, but kept it a bit vague. I thought it might be useful to share a couple of examples to describe what I mean by “AI patterns” a bit more clearly. Once again, put your technical hats on.

🗺️ How to read AI pattern diagrams

As part of practicing the “build a thing to build the thing” principle, we implemented quite a few of AI patterns in Breadboard. I will use the diagrams we generate from the boards (thank you Mermaid.js!) to illustrate the patterns. Here’s a quick guide on how to read the diagrams – and as a happy coincidence, a brief overview of Breadboard concepts.

 🔵 The inquisitively blue parallelogram represents the “input” node. This is where the user’s input is requested by the pattern. Because most patterns ask for input first, it’s a good place to start when tracing the flow of the graph.

🟢 The cheerfully green hexagon is the “output” node, which provides the output to the user of the pattern. For many patterns, that’s the end point, the journey’s destination, while for a few – just a brief stopover.

🟡 The curiously yellow boxes are all nodes that do interesting stuff. For example, “generateText” node invokes the LLM, while “promptTemplate” combines a template and various bits of text into a prompt that’s suitable for the LLM. Most of the time, you can guess what the function does by looking at its name.

🔴 The protectively red box with rounded corners is the “secrets” node, which has access to the user’s sensitive data. For most (all?) patterns, it is used to retrieve and pass the API Key to unlock the ability to invoke the large language model.

🍷 The variously-shaped wine-colored boxes are utility nodes: they are mostly here to serve other nodes by supplying important data and making it possible for graphs to be composable and useful. We’ll be mostly ignoring them here – but I will very likely be back to sing their song in the future.

Most nodes will have a two-line label. The first line is the type of the node and a second is its unique identifier. Because there can be multiple nodes of the same type, we need an id to distinguish between them.

 Just like with the literal breadboards,  nodes are connected with wires. Wires are represented by lines with arrows. The direction of the arrow on each wire represents the flow of information. So, when the graph shows this:

… it means that the information flows from the “promptTemplate” node to the “generateText” node.

Each wire is labeled. All labels have the same consistent “out->in” format. A good way to think of it is that every node may have multiple inputs and outputs. The wires connect these inputs and outputs.

In the example above, the output named “prompt” of the “promptTemplate” node is wired to the input named “text” of the “generateText” node. Most of the time, it’s not difficult to infer the purpose of the wire. Like, the wire above flows the prompt produced by the “promptTemplate” node as input of the “generateText” node. If you are curious about all the ins and outs of nodes (pun intended!), check out this guide on Github.

Some wires will have a circle at the end of them, rather than an arrow. These are constant wires. There’s a lot more to them, but for now, a good way to think of them is that they are here to specify constant values. Like in the diagram below, the “template” utility node supplies a constant “template” input to the “promptTemplate” node. 

With this quick Breadboard refresher out of the way, we’re ready to dig into the actual patterns. To keep this post from becoming a book, I’ll give you only three examples.

 🧮 The Calculator pattern

Let’s start with the widely used Calculator pattern (you can also see it here in on Github, implemented in Breadboard):

The structure of this pattern is very simple: user input goes into the “promptTemplate” node, which produces a prompt that goes into the “generateText” node, the output of which is fed to “runJavascript” node, and the result is returned as output.

As it often happens with AI patterns, the magic is in the contents of the prompt template. In this pattern, the LLM is used to find solutions to mathematical problems in a very clever way. 

As you may have heard, LLMs aren’t so great at math. So instead of approaching the problem head-on, we lean onto LLM’s strength: we convert a math problem into a language problem.

In the Calculator pattern, we ask the LLM to do what it does best: generate text. We ask it to write code that solves a math problem, rather than try to find the answer to the question. Here’s a prompt to do that:

Translate the math problem below into a JavaScript function named `compute` that can be executed to provide the answer to the problem.
Math Problem: {{question}}
Solution:

Because writing code is a language problem, LLMs are pretty good at it. So, with a high degree of consistency, the output of the LLM will be a function that, when run, produces the right answer. Leave computation to the old-style computers. Let LLMs write code that will be computed.

For instance, when we replace the {{question}} placeholder with:

What is the square root
of the perimeter of a circle w
ith a diameter of 5?

The LLM will happily produce this function:

function compute() {
  const diameter = 5;
  const radius = diameter / 2;
  const perimeter = 2 * Math.PI * radius;
  return Math.sqrt(perimeter);
}

Which, when executed, will give us the correct answer of `3.963327297606011`. If you ask any conversational agent today a math question and it surprises you with an accurate answer, chances are that some variant of the Calculator pattern is being employed.

📰 The Summarizer pattern

Another common pattern builds on the LLM’s strength of narrating information, even when presented with bits of random content. I experimented with this ability early this year, and here’s an implementation of the pattern in Breadboard (also here on Github):

When we look at the structure above, we can see that user input splits into two paths.

The first route is circuitous. It takes us through the “urlTemplate” node that creates a valid URL (it’s a Google News RSS feed with the topic as the query), which is then fed to the “fetch” node. The “fetch” node grabs the contents of this URL, and sends it to the “xmlToJson” and “jsonata” nodes that munge RSS into a list of headlines.

The second and the first route meet up at the “promptTemplate” node, where they predictable move to the “generateText” node and, finally, the result is presented to the user.

The concept is fairly straightforward: give the LLM a topic and a few sentences, and request a summary. If – as is the case in the graph above – we are summarizing news headlines, a prompt will look something like this:

Use the news headlines below to write a few sentences to summarize the latest news on this topic:
##Topic:{{topic}}
## Headlines{{headlines}}
## Summary:

In this prompt, we have two placeholders: the {{topic}}, which is where the subject of summarization will go, and the {{headlines}}, where we will plug in the various headlines from a news source (Google News).

The key distinction between just asking an LLM a question and using this pattern is that we’re not relying on LLM’s parametric memory to contain the answer. We’re not asking it to find the answer for us. Instead, we are only employing its narrative-making abilities, supplying the raw information in the prompt.

So, if I for example put “breadboards” into the {{topic}} placeholder, and the following list of headlines from Google News (just the first first 20 for this particular board) into the {{headlines}} placeholder:

Thermochromic Treatment Keeps Solderless Breadboards Smokeless - Hackaday
Jumper Wires For Electronic Components - IndiaTimes
10 hostess hacks to make your food look better than it is - Colorado Springs Gazette
Gabriel's Cyberdeck Red V2 Packs in a LattePanda Delta 3, Analog Discovery 2, HackRF One, and More - Hackster.io
How to Measure Voltage on a Breadboard - MUO - MakeUseOf
The Ultimate Breadboard Platform? - Hackster.io
Building Circuits Flexibly - Hackaday
Lewiston Art Festival: A 'dinosaur' of woodwork - Niagara Gazette
Podcast 220: Transparent Ice, Fake Aliens, And Bendy Breadboards ... - Hackaday
Flexboard: a flexible breadboard for flexible and rapid prototyping of ... - Tech Explorist
Derek Fogt | Communities | pinecountynews.com - pinecitymn.com
MARNI JAMESON: Compensate for humdrum food with stylish ... - Sarasota Herald-Tribune
Build HMI screens with MicroLayout for your Meadow Apps - Hackster.io
Tidy Breadboard Uses Banana Bread - Hackaday
Old 6809 Computer Lives Again On Breadboards - Hackaday
My Favorite Things: Hardware Hacking and Reverse Engineering - Security Boulevard
Luna in Cocoa Beach offers serves thoughtful modern Italian food - Florida Today
Teaching Method Increases Students' Interest in Programming and ... - University of Arkansas Newswire
From A 6502 Breadboard Computer To Lode Runner And Beyond - Hackaday
How to Breadboard Electronics Projects with Raspberry Pi Pico - Tom's Hardware

… we will get this output from an LLM:

The latest news on breadboards include a new thermochromic treatment
that keeps solderless breadboards smokeless, a flexible breadboard for
flexible and rapid prototyping, and a new method for teaching students
programming and electronics.

For the quality of the junk we fed it, it ain’t half bad!

The Summarizer pattern has a much more popular cousin named Retrieval-augmented Generation (RAG). RAG is all the rage these days, and everyone wants to have one. If we peek under the covers, we’ll recognize the Summarizer pattern combined with another neat LLM capability of semantic embeddings into the Voltron of patterns.

🔁 The ReAct pattern

I would be remiss not to bring up ReAct when talking about AI patterns. This pattern ushered the new mini-era of LLM applications, a breakthrough that redefined what LLMs can do.

The ReAct pattern is different from the ones mentioned earlier, because it is cyclical: rather than asking an LLM once, it may do so several times, repeating until the problem is solved.

ReAct introduces this really interesting idea that we can induce chain-of-thought reasoning capabilities in LLMs if we structure our interaction with them in a certain way. In this chain of thought, the LLM interacts with the outside environment, suggesting actions to take and then reason about the outcomes of these actions.

I’ve talked about LLM-based reasoning a few times before, so this concept shouldn’t be entirely novel to my readers.

In ReAct, the key trick is in establishing a predictable beat of reasoning within the prompt:

1️⃣ First comes the Question – the question that the user asks
2️⃣ Then, comes the Thought – the opportunity for an LLM to reason about what to do next
3️⃣ After Thought is Action – LLM’s suggested action to take
4️⃣ Finally, the Observation – the outcome of the action, supplied by the tool 

Steps 2️⃣,  3️⃣,  and 4️⃣ keep repeating until the answer is found. 

The LLM is only allowed to pipe in on steps 2️⃣ and 3️⃣: that is, it can only produce the “Thought” and “Action” parts of the overall sequence.

Step 1️⃣ is provided by the user, and the observation in step 4️⃣ is supplied as the outcome of whatever action the LLM suggested to take.

As the steps repeat, all of these steps are being added to the overall prompt, allowing the LLM to see the history of the interaction and reason about it. In this way, and unlike in the Calculator and Summarizer patterns, the ReAct pattern simulates memory: with each invocation, the LLM can see how it acted in the past.

It’s a pretty convoluted contraption, as you can see it from the Breadboard implementation (here on Github):

However, the presence of memory and a repeating cadence of steps guides LLMs to allow it to answer much more complex and interesting questions. The answers are no longer produced in one go. Instead, LLMs walk toward the answer in multiple iterations.

Let’s see if we can make it a bit more concrete by breaking down an actual transcript of a ReAct pattern solving a problem.

First, we supply the question:

1️⃣ Question: what is the square root of the distance between earth and moon in kilometers?

In response, the LLM the produces the following Thought and Action steps:

1️⃣ Question: what is the square root of the distance between earth and moon in kilometers?
2️⃣ Thought: I need to find the distance between earth and moon
3️⃣ Action: search
   Action Input: distance between earth and moon

The LLM is asking to search for the distance between the Earth and the Moon. Notice that it is not trying to make up the answer. It is merely suggesting the next action to take. Taking control over from the LLM, the ReAct pattern invokes search and produces the following Observation:

1️⃣ Question: what is the square root of the distance between earth and moon in kilometers?
2️⃣ Thought: I need to find the distance between earth and moon
3️⃣ Action: search
   Action Input: distance between earth and moon
4️⃣ Observation: 384,400 km

With the observation in hand, the ReAct pattern hands control back to the LLM, and get this response:

1️⃣ Question: what is the square root of the distance between earth and moon in kilometers?
2️⃣ Thought: I need to find the distance between earth and moon
3️⃣ Action: search
   Action Input: distance between earth and moon
4️⃣ Observation: 384,400 km
2️⃣ Thought: I need to square root this distance
3️⃣ Action: math
   Action Input: sqrt(384400)

Great! Now the LLM wants us to do math. Fine. The ReAct machinery rattles and burrs, and produces the next Observation:

1️⃣ Question: what is the square root of the distance between earth and moon in kilometers?
2️⃣ Thought: I need to find the distance between earth and moon
3️⃣ Action: search
   Action Input: distance between earth and moon
4️⃣ Observation: 384,400 km
2️⃣ Thought: I need to square root this distance
3️⃣ Action: math
   Action Input: sqrt(384400)
4️⃣ Observation: 620

The ReAct pattern yields control back to the LLM, and it triumphantly declares victory:

1️⃣ Question: what is the square root of the distance between earth and moon in kilometers?
2️⃣ Thought: I need to find the distance between earth and moon
3️⃣ Action: search
   Action Input: distance between earth and moon
4️⃣ Observation: 384,400 km
2️⃣ Thought: I need to square root this distance
3️⃣ Action: math
   Action Input: sqrt(384400)
4️⃣ Observation: 620
2️⃣ Thought: I now know the final answer
   Final Answer: 620

Great job, model. You did it.

The ReAct pattern also introduces, almost as an afterthought, the concept of tools. Since the LLM is asked to suggest an action, it seems useful to specify the kinds of tools the LLM has at its disposal.

In the transcript above, the “search” and “math” tools were used. For other kinds of problems, there might be a need for other kinds of tools. 

This is where the most valuable aspect of the ReAct pattern resides: if we can specify our own tools, we can make LLMs do useful things. For example, I could hand it a “calendar” tool, an “email” tool, and a list of my friends and ask it to schedule a lunch for us. Or I could turn it into a menu-ordering system, where it would rely on menu-understanding tools to take customer orders.

The pattern stays the same,  but the tools change. With the ReAct pattern, we can build actual helpful agents. If you’ve been watching the LLM space, you have no doubt noticed a lot of activity around this notion.

🍞Patterns with Breadboard

These are just a few examples of interesting patterns that emerged in the generative AI space in the last few months. Honestly, the pace of pattern discovery has been nuts. So far, I see no signs of it slowing down. What I’d like to do with Breadboard is to help make these patterns more legible to everyone – so that more people can play with them, create their own, and explore this fascinating space together.

My intuition is that when we lower the barrier to entry to this process of discovery and make it easy to tell whether the new pattern is good or not, we have a better chance of exploring the space more thoroughly and realizing the full potential of large language models.