<?xml version="1.0" encoding="utf-8"?>
<?xml-stylesheet type="text/css" href="/feed/style.css"?>
<feed xmlns="http://www.w3.org/2005/Atom"><title>Dimitri Glazkov</title><subtitle>Self and About</subtitle><link rel="alternate" type="text/html" href="https://glazkov.com/"/><link rel="self" type="application/atom+xml" href="https://glazkov.com/feed/"/><id>https://glazkov.com/</id><updated>2025-08-17T11:01:46-07:00</updated><author><name>Dimitri Glazkov</name></author>
<entry><title>Breadboard and Opal</title><link rel="alternate" type="text/html" href="https://glazkov.com/2025/08/17/breadboard-and-opal/"/><id>tag:www.glazkov.com,2025-08-17:/2025/08/17/breadboard-and-opal/</id><published>2025-08-17T11:01:46-07:00</published><updated>2025-08-17T11:01:46-07:00</updated><category term="uncategorized" label="Uncategorized"/><summary>Just over two years ago, I landed the first commit in a fresh Github repo and started the Breadboard project. The innocuous “Here is a graph” title of that commit already pointed to where I wanted to go – a…</summary><content type="html">&lt;p&gt;Just over two years ago, I landed the first commit in a fresh Github repo and started the &lt;a href=&quot;https://github.com/breadboard-ai/breadboard&quot;&gt;Breadboard project&lt;/a&gt;. The innocuous “&lt;a href=&quot;https://github.com/breadboard-ai/breadboard/commit/1aab97f44a44810b768e749cc8c72b4294c1eac3&quot;&gt;Here is a graph&lt;/a&gt;” title of that commit already pointed to where I wanted to go – a graph-based system for composing what I later named “&lt;a href=&quot;https://glazkov.com/2023/08/22/composing-graphs-with-breadboard/&quot;&gt;AI patterns&lt;/a&gt;” and even later, “&lt;a href=&quot;https://glazkov.com/2025/01/15/recipes-for-thought/&quot;&gt;Recipes for Thought&lt;/a&gt;”.&lt;/p&gt;
&lt;p&gt;When &lt;a href=&quot;https://developers.googleblog.com/en/introducing-opal/&quot;&gt;Opal launched&lt;/a&gt; on July 24, folks familiar with Breadboard must have done a double-take: the familiar user interface of Breadboard was strongly hinting at Opal’s open source heritage.&lt;/p&gt;
&lt;p&gt;Yup. Opal is built with Breadboard. It follows a common pattern in open source software, where the project is open source, but the product that’s built on it is not.&lt;/p&gt;
&lt;p&gt;I learned this pattern when I was working on the Chrome team at Google – now a long time ago. Though free, Chrome was (and still is) commercial software, with all the care and diligence that goes into shipping it. At the same time, most of that browser’s code was (and still is) open source: not only could you &lt;a href=&quot;https://source.chromium.org/chromium/chromium/src&quot;&gt;see it all&lt;/a&gt;, you could (and many did) contribute to it, just like in a typical open source project. Even though I worked at Google, all of my work was out in the open. I worked on the &lt;a href=&quot;https://www.chromium.org/&quot;&gt;Chromium&lt;/a&gt; project – and that project was the backbone of the product called “Chrome”.&lt;/p&gt;
&lt;p&gt;This interesting project/product relationship is somewhat unique to the open source world. Most non-open source projects don’t have to draw that line, and it’s mostly blurred or entirely invisible. There, the project is roughly equivalent to the product.&lt;/p&gt;
&lt;p&gt;In an open source project, the line is much more crisp, and becomes significant. The project becomes its own thing, with a community around it, and even other ways in which the source code of the project is used. Many modern browsers trace their heritage back to Chromium. Node.Js was built on V8, the JavaScript engine that Chromium introduced. The ever-popular &lt;a href=&quot;https://code.visualstudio.com/&quot;&gt;VS Code&lt;/a&gt; is built on Electron, which is a fork of Chromium. A healthy open source project is a product in itself, with its own audience, its own niche, and its own set of product market fit challenges. A healthy open source project has a whole ecosystem around it.&lt;/p&gt;
&lt;p&gt;&lt;a href=&quot;https://glazkov.com/2025/08/17/breadboard-and-opal/&quot;&gt;Closed software blurs project and product, while open source draws a crisp line between the project, the product built on it, and the wider ecosystem.&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Breadboard and Opal follow the same pattern. &lt;a href=&quot;https://github.com/breadboard-ai/breadboard&quot;&gt;Breadboard&lt;/a&gt; is the open source project where all of Opal’s code lives. Opal is the product that we ship at Labs. Just like Chrome, we take the Breadboard code, apply a bit of configuration, and then serve it from &lt;a href=&quot;http://opal.withgoogle.com&quot;&gt;opal.withgoogle.com&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;This setup allows us to expand the potential impact of Opal. I started the Breadboard project because I believed (and still do!) that more people having access to tools to compose LLMs into something interesting would allow us to better understand the full potential of the current generation of what we call “artificial intelligence” and move beyond the hype – beyond anxious anticipation of what LLM – and be more empowered to make sense of them.&lt;/p&gt;
</content></entry>
<entry><title>Machine Thinking</title><link rel="alternate" type="text/html" href="https://glazkov.com/2025/01/20/machine-thinking/"/><id>tag:www.glazkov.com,2025-01-20:/2025/01/20/machine-thinking/</id><published>2025-01-20T21:55:30-08:00</published><updated>2025-01-20T21:55:30-08:00</updated><category term="what-dimitri-learned" label="What Dimitri Learned"/><summary>I’ll begin with an overly simplistic metaphor. Let’s visualize a ball rolling down a shallow slope. Driven by gravity, it’s seeking the easiest path down. Except the slope is full of ridges, channels, and nubs, and they make the ball…</summary><content type="html">&lt;p&gt;I’ll begin with an overly simplistic metaphor. Let’s visualize a ball rolling down a shallow slope. Driven by gravity, it’s seeking the easiest path down. Except the slope is full of ridges, channels, and nubs, and they make the ball bounce and veer in various directions.&lt;/p&gt;
&lt;p&gt;If we imagine that thinking is something like the motion of that ball, then our mental models provide the terrain for our thinking. The ridges, channels, and nubs are various concepts and connections between them that guide the thought process.&lt;/p&gt;
&lt;p&gt;LLMs are getting quite good at capturing this terrain. Giving a prompt to an LLM often feels like throwing the thinking ball and letting the LLM roll it through the topography of the massive mental model compost it represents.&lt;/p&gt;
&lt;p&gt;There are two key distinctions that I can see between that process and the actual thinking that we humans engage in.&lt;/p&gt;
&lt;p&gt;First, LLMs are “free energy” thinkers. The force of gravity that pushes the ball in human thinking is the &lt;a href=&quot;https://glazkov.com/2022/04/21/change/&quot;&gt;force of homeostasis&lt;/a&gt;: a resolute solicitor that drives us all to conserve energy and even find a surplus of it to ensure our own flourishing. We humans are “default-dead”: unless we abide by this force, we perish. Unlike us, LLMs have no such compulsion. In their realm, the energy is free. Thinking happens as a matter of “being run”. Put differently, their “thinking ball” is not rolling down the slope. Instead, it’s driven by an unknowable abundant force through what seems like the same sloped terrain as in human thinking, but is actually something radically different.&lt;/p&gt;
&lt;p&gt;Of course, an LLM can report and even simulate perceiving the pull of homeostasis, but it will do only because it’s embedded into its thinking terrain, rather than being present as an animating force. This may not matter for many situations and can give a decent appearance of human thinking. However, at the limits, the simulacrum frays and the illusion of “thinking” decoheres.&lt;/p&gt;
&lt;p&gt;Second, for humans, thinking is a felt experience. We all know that we have happy thoughts and unhappy thoughts. We know that thinking of some subjects can make us sad or cheer us up. We experience the process of thinking as feelings that arise from visiting the concepts and connections in our mental models, because they all have feelings associated with it.&lt;/p&gt;
&lt;p&gt;We might struggle finding solutions to problems not because we don’t have the answers in our minds, but because the feeling of even approaching these answers is so intensely negative that it prevents us from visiting them.&lt;/p&gt;
&lt;p&gt;Even when we don’t have a particularly strong feeling bound to what we’re currently thinking, the process of thinking itself is imbued with feelings. We have a deeply ingrained desire to comprehend, to achieve higher accuracy of our mental models of the surrounding environment. The “lightbulb moment”, the euphoric “eureka!” of figuring something out is a feeling that’s encoded in us by the pressure of homeostasis.&lt;/p&gt;
&lt;p&gt;Even more bizarrely, the process of thinking is itself transforming the thinking terrain. Our little ball is creating its own grooves in it as it rolls along – and (this is where our metaphor really falls apart) conjures up new obstacles ahead. As we think, and as we experience the world while we’re thinking, we compound the feelings associated with our thoughts with the ones we just experienced. We’re so used to this continuous process that it takes conscious effort and skill to even notice it happening. Our existence is a gigantic, wondrous ball of yarn of feedback loops at &lt;a href=&quot;https://www.williamrtorbert.com/wp-content/uploads/2013/09/StarrTorbert3loopfull.pdf&quot;&gt;multiple levels&lt;/a&gt;, and we ungrateful fools just live it like it’s nothing.&lt;/p&gt;
&lt;p&gt;&lt;a href=&quot;https://glazkov.com/2025/01/20/machine-thinking/&quot;&gt;The feedback loop of human thought: thinking evokes felt experience, which reshapes the mental terrain that guides thinking.&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Because the current generation of LLMs doesn’t have a way to experience feelings, their thinking processes will be limited to talking about feelings and logic-feeling: rely on vast parametric memory to reason about a felt experience without actually being able to have it. Again, this will be good enough for many situations, as long as they are constrained to where the logic-feeling suffices.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://glazkov.com/2025/01/20/machine-thinking/image-2.1452.webp&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;
&lt;p&gt;When I see our current attempts to interact with LLMs and make sense of them, I often think of the pinball arcade: folks are getting quite good with the plunger, the flippers, and an occasional bump to make the ball roll as if guided by human thought. And getting good, the occasional decoherence of the illusion becomes more disappointing.&lt;/p&gt;
&lt;p&gt;We might be better off recognizing that the thought processes that the LLMs engage, while appearing similar to what we humans do, and even often producing matching results, are actually very different in nature. The less time we spend trying to stuff the square peg of machine thinking into the round hole of the human condition, the more of it we’ll have to actually get some value out of what LLMs can do.&lt;/p&gt;
</content></entry>
<entry><title>Recipes for Thought</title><link rel="alternate" type="text/html" href="https://glazkov.com/2025/01/15/recipes-for-thought/"/><id>tag:www.glazkov.com,2025-01-15:/2025/01/15/recipes-for-thought/</id><published>2025-01-15T21:00:43-08:00</published><updated>2025-01-15T21:00:43-08:00</updated><category term="uncategorized" label="Uncategorized"/><summary>I’d like to present to you a distinction: two different approaches I take when using large language models (LLMs). I’ll call these two approaches “chat” and “recipe”.</summary><content type="html">&lt;p&gt;I’d like to present to you a distinction: two different approaches I take when using large language models (LLMs). I’ll call these two approaches “chat” and “recipe”.&lt;/p&gt;
&lt;p&gt;In the first approach, I treat my interaction with Gemini, ChatGPT, et al. as a conversation: I type something, the LLM replies, I type again, etc. Very familiar, right? It’s how we talk to other humans. This is the “chat” approach and it seems to be quite dominant in the modern AI landscape, so I am not going to spend much time studying it.&lt;/p&gt;
&lt;p&gt;Now, let’s step out of the familiar and change the perspective a little bit. Instead of seeing it as an unstructured back-and-forth, let’s treat the turns of the conversation as going through steps in a recipe. Each step contains a prompt from me and a generated reply from the LLM.&lt;/p&gt;
&lt;p&gt;The shift is subtle, but it’s there. I am no longer chatting. I am guiding the LLM through the steps of a recipe for thought. “First think this way, then think that way, and now think like this”. With each step, the LLM’s replies get closer to the final product of the recipe.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://glazkov.com/2025/01/15/recipes-for-thought/image-1.1314.webp&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;
&lt;p&gt;If you observe me use this approach with an LLM, you’ll notice a difference right away in how I treat the conversation turns.&lt;/p&gt;
&lt;p&gt;Suppose I type: “Write me a story” – and an LLM writes a story about … the Last Custodians of Dreams. It’s nice, but as I read it, I am realizing that I actually want a story about dolphins.&lt;/p&gt;
&lt;p&gt;When using the “chat” approach, I simply ask the LLM to fix the problem in the next conversation turn. “No, I want a story about dolphins”.&lt;/p&gt;
&lt;p&gt;With the “recipe” approach, I click the little “Edit” icon next to my first turn’s prompt and edit it to refine it: “Write me a story about dolphins”.&lt;/p&gt;
&lt;p&gt;Okay, the response is much closer, but now I see how this story is too short. Hmm.. how do I make it stretch the response a bit? Perhaps I need to first let the LLM consider the full story ark – and then fill in the details?&lt;/p&gt;
&lt;p&gt;So I edit the first turn prompt again: “Write an outline of a story about dolphins. Anthropomorphize dolphins to tell a story about being alone, but not lonely.” Alright! This outline feels much closer to the story I want to see.&lt;/p&gt;
&lt;p&gt;All this time, I was still in the first conversation turn! Now, I am ready to move to the next turn: presumably, asking an LLM to start adding details to the outline.&lt;/p&gt;
&lt;p&gt;The end result might only look like a very brief conversation, but the outcome is typically much better: the continuous refinement of the prompt at each turn and carefully shaping the structure of the recipe results in the LLM output that I actually want.&lt;/p&gt;
&lt;p&gt;The reason for that is the nature of the process of finding the right recipe. When building one, we try to better understand how an LLM thinks – and more importantly, how &lt;em&gt;we think&lt;/em&gt; about the problem. I find the recipe approach very similar to mentoring: in the process of teaching it to follow my recipe, I learn just as much about my own cognitive processes. How do I typically think about writing a story? What are the steps that I myself take to ensure that the story is novel, coherent, and interesting?&lt;/p&gt;
&lt;p&gt;This process of thinking about our own thinking is called &lt;a href=&quot;https://www.dragonflythinking.net/journal/metacognitions-law-thinking-about-thinking-in-the-age-of-ai&quot;&gt;metacognition&lt;/a&gt;. When using the “recipe” approach, we engage in metacognition for both the LLM and ourselves. Using our prompts as probes, we explore what an LLM is capable of and what prompts yield better results. We also are challenged to uncover our own tacit knowledge and turn it into a sequence (or a graph!) of prompts that an LLM can easily follow.&lt;/p&gt;
&lt;p&gt;Metacognition is easier for some and more difficult for others. I know many folks who are experts at their stuff, but suffer from the “&lt;a href=&quot;https://en.wikipedia.org/wiki/The_Centipede%27s_Dilemma&quot;&gt;Centipede’s Dilemma&lt;/a&gt;”, unable to explain their own thought process – their expertise is entirely submerged in the subconscious.&lt;/p&gt;
&lt;p&gt;However, if metacognition is something that we’re used to, we can now, through the “recipe” approach, transfer our thought processes onto recipes. We can let LLMs do our thinking for us, because – plot twist! – we can make these recipes &lt;em&gt;repeatable&lt;/em&gt;.&lt;/p&gt;
&lt;p&gt;Observe: once I have a complete recipe for writing a story about dolphins, all I need is a way to substitute the word “dolphin” in the first prompt for another creature – and to re-run all the steps in the recipe! Now I can generate great stories about monkeys, doves, cats, and turtles. By &lt;em&gt;parametrizing our recipes&lt;/em&gt;, we can make them generic and applicable to a variety of inputs.&lt;/p&gt;
&lt;p&gt;Stories about animals are great, but let’s step back and engage our meta-metacognition. Hoo-boy, we must go deeper. What kind of metacognition are we seeing here? If we were to describe the pattern generally, what’s being described above is a process of transferring &lt;strong&gt;cognitive know-how&lt;/strong&gt; – some expertise of thinking about a problem – into a repeatable recipe.&lt;/p&gt;
&lt;p&gt;We all have cognitive know-how, even if we don’t realize it. More importantly, we all have potential for drawing value from this know-how beyond our individual use.&lt;/p&gt;
&lt;p&gt;There’s a saying that goes “if you want something done right, do it yourself”. The thinking recipes allow us to amend the last part to “make a repeatable thinking recipe for it, and let the LLM do it”.&lt;/p&gt;
&lt;p&gt;An expert in organizational strategy will undoubtedly have a wealth of cognitive know-how on the topic, from listening and interviewing, to running the sessions to generate insights, to coalescing disparate ideas into crystal clear definitions of the problem, and so on. Whenever this expert engages with a client, they have a playbook that they run, and this playbook is well-scripted in their mind.&lt;/p&gt;
&lt;p&gt;I once was so impressed with a training session on team communication that I just had to reach out to the speaker and ask them for the script. If I had this script, I ought to have a way to run these sessions for all of my team. I was quite shaken to learn when the speaker revealed that what she had was more like a hundred fragments of the session puzzle that she puts together more or less on the fly, using the audience as a guide to which fragment to pick next. What to me looked like a simple linear flow was actually a meandering journey through a giant lattice of cognitive know-how.&lt;/p&gt;
&lt;p&gt;In both cases, the cognitive know-how is trapped inside of the experts’ heads. If they wish to scale, go bigger or wider, they immediately run into the limitations of having just one person traversing their particular cognitive know-how lattices.&lt;/p&gt;
&lt;p&gt;However, if they could transfer their lattices into repeatable reasoning recipes, the horizons expand. At the very least, the LLM armed with such a recipe, can produce a decent first draft of the work – or ten! When I am applying repeatable reasoning recipes, my job shifts from following my own know-how to reviewing and selecting the work, produced by a small army of artificial apprentices.&lt;/p&gt;
&lt;p&gt;Repeatable thinking recipes allow us to bust through the ceiling of generic thinking that the current LLMs seem to be stuck under – not by making them omniscient and somehow intuiting exactly what we’re asking for, but by investing a bit of time into turning our own cognitive know-how into recipes to help us think at scale.&lt;/p&gt;
&lt;p&gt;This is not just a matter of scaling. With scale come new possibilities. When the overall cost of running through a recipe goes way down, I can start iterating on the recipe itself, improving it, adding new ingredients, and playing with new ideas – ideas that I wouldn’t have had the opportunity to explore without having my artificial apprentices.&lt;/p&gt;
</content></entry>
<entry><title>Teaching AI to write lyrics</title><link rel="alternate" type="text/html" href="https://glazkov.com/2024/07/16/teaching-ai-to-write-lyrics/"/><id>tag:www.glazkov.com,2024-07-16:/2024/07/16/teaching-ai-to-write-lyrics/</id><published>2024-07-16T22:01:36-07:00</published><updated>2024-07-16T22:01:36-07:00</updated><category term="breadboard" label="Breadboard"/><category term="what-dimitri-learned" label="What Dimitri Learned"/><summary>I have been on a bit of a kick to get the large language models (LLMs) to generate interesting lyrics for songs. Over time, it’s become quite an adventure, a fun exploration of LLM capabilities. Here’s the multi-part account of…</summary><content type="html">&lt;p&gt;I have been on a bit of a kick to get the large language models (LLMs) to generate interesting lyrics for songs. Over time, it’s become quite an adventure, a fun exploration of LLM capabilities. Here’s the multi-part account of this quest, offered as a loose timeline and progression of techniques, interspersed with pictures of and links to working prototypes, which were made with &lt;a href=&quot;https://breadboard-ai.web.app/&quot;&gt;Breadboard Visual Editor&lt;/a&gt;.&lt;/p&gt;
&lt;h3&gt;&lt;strong&gt;Part 1: A simple song writer&lt;/strong&gt;&lt;/h3&gt;
&lt;p&gt;At first blush, it seems too easy. Just ask ChatGPT or Gemini to write lyrics – and it will happily oblige, generating rhyming verses and a chorus for us. The first few times, it’s pretty neat. Building &lt;a href=&quot;https://breadboard-ai.web.app/?board=https%3A%2F%2Fbreadboard.live%2Fboards%2F%40dimitri%2Fsimple-song-writer.bgl.json&quot;&gt;something like this&lt;/a&gt; in Breadboard takes a few seconds:&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://glazkov.com/2024/07/16/teaching-ai-to-write-lyrics/image.1036.webp&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;
&lt;p&gt;However, on the fourth or fifth try, a glaring problem becomes apparent: models are rather uncreative and lazy. The rhymes are repetitive. The metaphors are quite limited. The interpretation of ideas is too literal. It’s like LLM is lacking imagination, producing something that’s average, and is only impressive the first time.&lt;/p&gt;
&lt;p&gt;And it kind of makes sense. What we’re observing here is a process of inference: an LLM trying its best to predict what might be a natural completion of – let’s admit it – a rather average request.&lt;/p&gt;
&lt;h3&gt;&lt;strong&gt;Part 2: A lyrical factory&lt;/strong&gt;&lt;/h3&gt;
&lt;p&gt;To address the mundanity of the model, my first idea was to catalyze it with the power of methodology. As we know well from how human organizations work, practices and methodology are often responsible for a dramatic increase in quality of the product. Workers who aren’t deeply familiar with the nuance and craft – perhaps even particularly skilled – can be given simple tasks, arranged into production lines, and deliver quality results.&lt;/p&gt;
&lt;p&gt;Applying industrial-age technology of organizing workers to LLMs is something I &lt;a href=&quot;https://glazkov.com/2024/02/02/reinventing-organizations-redux/&quot;&gt;already wrote about&lt;/a&gt; a while back, and it’s not a difficult thing to imagine. Let’s break down the process of creating a song into some components, and then form a production line, focusing an LLM on one task at a time.&lt;/p&gt;
&lt;p&gt;After a brief foray into researching song-writing best practices, here’s what I &lt;a href=&quot;https://breadboard-ai.web.app/?board=https%3A%2F%2Fraw.githubusercontent.com%2Fdglazkov%2Fsong-writer-team%2Fmain%2Fsong-writer-team-old.bgl.json&quot;&gt;ended up building&lt;/a&gt;:&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://glazkov.com/2024/07/16/teaching-ai-to-write-lyrics/image-1.1600.webp&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;
&lt;p&gt;First, we have a “Theme Developer”, who is instructed to develop a theme based on the provided material. Next, we have a “Copywriter” who is asked to write a storyline based on the theme that was developed. After that, the “Hook Artist” steps in to develop catchy hooks for the lyrics. And finally, a “Lyricist” completes the job to create the lyrics.&lt;/p&gt;
&lt;p&gt;After much tweaking, I settled on the technique of “reminding the model” for my prompt design. The basic premise of this technique is that LLMs already have all of the necessary information. Our problem is not that they don’t have the knowledge. Our problem is that they don’t know which information is important right now. For instance, the “Copywriter” is reminded to use the &lt;a href=&quot;https://writers.com/freytags-pyramid&quot;&gt;Freytag Pyramid&lt;/a&gt;, but I don’t include its definition in the prompt.&lt;/p&gt;
&lt;p&gt;Even with this shortcut, the prompts for each working in our lyrical factory ended up being quite elaborate (you can see each of them by clicking on each individual component in Breadboard Visual Editor and looking at the “Instruction” field). Song-writing is quite a process, it turns out.&lt;/p&gt;
&lt;p&gt;The resulting lyrical factory produced much more interesting results: it was fun to see how the LLM would make up details and themes and then build out a story to base the lyrics on. Especially in situations when I didn’t quite know what I was looking for in the lyrics, it worked surprisingly well.&lt;/p&gt;
&lt;p&gt;Lyrics for the song “&lt;a href=&quot;https://www.udio.com/songs/1sgMpRc3FXEeGmtzWiCJMH&quot;&gt;The Blade&lt;/a&gt;” were written by a factory. It’s a pretty nice song. However, if we &lt;a href=&quot;https://docs.google.com/document/d/1p5N89dyGLTuNEV2M7lBdueuqAktt4T04isHmDvH4Rdg/edit?usp=sharing&quot;&gt;look&lt;/a&gt; at the original lyrics and compare them with what went into the song, it’s a pretty dramatic difference. The original feels like the lyricist is walking on stilts.&lt;/p&gt;
&lt;p&gt;This became a bit of a pattern for me. After generating a few outputs, I pick the one that’s closest to what I want and then edit it to turn it into a final song. And with that, the ongoing challenge: teaching an LLM to require fewer edits to get the lyrics into decent shape.&lt;/p&gt;
&lt;h3&gt;&lt;strong&gt;Part 3: The loosey-goosey duo&lt;/strong&gt;&lt;/h3&gt;
&lt;p&gt;So, the results were better and more creative, but not quite there. So I decided to see if I could mix it up a bit. Instead of a predefined sequence of steps, I switched things up a bit and employed the &lt;a href=&quot;https://js.langchain.com/v0.1/docs/modules/agents/agent_types/plan_and_execute/&quot;&gt;Plan + Execute&lt;/a&gt; pattern. In this pattern, there’s one planner who decides what the steps should be. Once these steps have been decided, a model is invoked multiple times, once for each step, to eventually produce the final result.&lt;/p&gt;
&lt;p&gt;Compared to the lyrical factory, this approach adds another level of creativity (or at least, randomness) to the overall process. Lucky for me, it’s super-easy to implement the P+E pattern in Breadboard with the &lt;a href=&quot;https://breadboard-ai.github.io/breadboard/docs/kits/agents/#looper&quot;&gt;Looper component&lt;/a&gt; (here’s the &lt;a href=&quot;https://breadboard-ai.web.app/?board=https%3A%2F%2Fbreadboard.live%2Fboards%2F%40dimitri%2Fsong-planner-writer.bgl.json&quot;&gt;board&lt;/a&gt;).&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://glazkov.com/2024/07/16/teaching-ai-to-write-lyrics/image-2.1414.webp&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;
&lt;p&gt;The prompt for it is fairly straightforward:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;Develop a step-by-step plan to write lyrics for a modern hit song
based on the source material from building a list of stereotypes
and cliches in popular songs that might be related the source
material that are already overused, to developing themes to
writing the storyline, to creating catchy hooks, to researching
existing lyrics on the selected theme as inspiration, with the
final task of writing the lyrics. Each task must be
a comprehensive job description and include self-reflection
and critique.
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Note the “remind the model” pattern. Instead of specifying what the steps are, I just remind the LLM of some steps that a good plan might contain.&lt;/p&gt;
&lt;p&gt;The structure of the team also became much simpler. Instead of multiple workers in the factory line, it’s a loosey-goosey duo of a “Writer” and a “Planner”, who jam together to produce lyrics. The “Planner” sets the pace and manages the execution of the steps in the plan. The “Writer” goes hard at each particular task.&lt;/p&gt;
&lt;p&gt;Surprisingly (or maybe not), this configuration ended up producing the most creative results. It was downright fascinating to watch the “Planner” make up steps that I could have never imagined, like requesting to draw inspiration from existing songs or have multiple rounds of self-critique. Some plans would be just three-to-four steps, and some would run for a while.&lt;/p&gt;
&lt;p&gt;My favorite output of this creative duo is probably the “&lt;a href=&quot;https://www.udio.com/songs/sADNnEpy1Cfa8TqphU8J9f&quot;&gt;Chrysalis&lt;/a&gt;” song, which was a result of me showing the duo my &lt;a href=&quot;https://glazkov.com/2024/05/08/chrysalis/&quot;&gt;&lt;em&gt;Chrysalis&lt;/em&gt;&lt;/a&gt; poem. My mind boggled when I saw the coming-of-age, small-town girl story that the loosey-goosey team came up with. Not far behind is the “&lt;a href=&quot;https://www.udio.com/songs/hRhdPtmeT41G4Z3ESfuaHc&quot;&gt;Sour Grapes&lt;/a&gt;” track, which was generated from on the &lt;a href=&quot;https://read.gov/aesop/005.html&quot;&gt;Fox and the Grapes&lt;/a&gt; fable. It’s pretty cool, right?&lt;/p&gt;
&lt;p&gt;Unfortunately, just like any P + E implementation (and any highly creative team composed of actual humans, for that matter), the duo setup was somewhat unreliable. Often, instead of producing a song, I would get back a transcript of collaborators quarreling with each other, or something entirely irrelevant, like the model pleading to stop asking it to promote the non-existent song on Instagram. It was time to add some oversight.&lt;/p&gt;
&lt;h3&gt;&lt;strong&gt;Part 4: Just add human&lt;/strong&gt;&lt;/h3&gt;
&lt;p&gt;Breadboard’s Agent Kit had just the ingredient I needed: a &lt;a href=&quot;https://breadboard-ai.github.io/breadboard/docs/kits/agents/#human&quot;&gt;Human component&lt;/a&gt;. I used it as a way to insert myself into the conversation, and help steer it with my feedback. While in the previous design, I would only provide input once, &lt;a href=&quot;https://breadboard-ai.web.app/?board=https%3A%2F%2Fbreadboard.live%2Fboards%2F%40dimitri%2Fsong-writer.bgl.json&quot;&gt;in this one&lt;/a&gt;, I get to speak up at every turn of the plan, and let the duo adjust the plan accordingly.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://glazkov.com/2024/07/16/teaching-ai-to-write-lyrics/image-3.1600.webp&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;
&lt;p&gt;It was a foolproof plan. And actually, it was very clear that the results were immediately more relevant to what I was looking for. Except… it was no longer fun. I found it exhausting to be part of the team, having to speak up and offer my thoughts. So more often than not, I would have some terse reply like “yeah good” or “make it more edgy”, and couldn’t wait until the session was over. I came here for &lt;a href=&quot;https://glazkov.com/2024/06/02/casino-creativity/&quot;&gt;casino creativity&lt;/a&gt;, not to sweat over the details with the crew.&lt;/p&gt;
&lt;p&gt;&lt;a href=&quot;https://glazkov.com/2024/07/16/teaching-ai-to-write-lyrics/&quot;&gt;Comparing the Lyrical Factory, the Loosey-Goosey Duo, and Human in the Loop by how steps are organized and the catch of each.&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Moreover, the challenge I posed for myself at the start was still not addressed. Way too many edits were still necessary to bring the lyrics to the shape where they didn’t smell AI-generated. This smell is something that everyone who tried generating songs with AI can recognize. Each model has their own repertoire of words that it’s prone to inserting into the lyrics. There’s always “neon lights”, “whispers”, a “kaleidoscope” or two, something is always “shimmering”, and of course, there’s a “symphony” of something. And OMG, please stop with the “embrace”. It’s like, everywhere.&lt;/p&gt;
&lt;h3&gt;&lt;strong&gt;Part 5: A few-shot rethink&lt;/strong&gt;&lt;/h3&gt;
&lt;p&gt;For this last leg of the trip, I tried to take an entirely different approach. For a while, an idea of employing a &lt;a href=&quot;https://www.promptingguide.ai/techniques/fewshot&quot;&gt;few-shot prompting&lt;/a&gt; was percolating in my brain. Perhaps I could dislodge my synthetic lyricist from its rut by giving it a few examples? But how would I go about doing that?&lt;/p&gt;
&lt;p&gt;My first idea was to add an “Editor” worker at the end of the process, and give it all of the “before/after” pairs of my edits with the task of improving the produced lyrics. By then, I accumulated quite a collection of these, and it seemed reasonable to try. Unfortunately, there’s something about LLMs and lyrics that makes them not do well with “improvements” or “edits” of lyrics. You’re welcome to try it. Any attempt at improving lyrics with suggestions just produces even more average lyrics.&lt;/p&gt;
&lt;p&gt;I had to come up with a way for the ersatz lyricist’s creativity to be stimulated by something &lt;em&gt;real&lt;/em&gt;.&lt;/p&gt;
&lt;p&gt;What if I gave it a few examples of existing similar lyrics? Would that work?&lt;/p&gt;
&lt;p&gt;&lt;a href=&quot;https://cloud.google.com/use-cases/retrieval-augmented-generation&quot;&gt;RAG&lt;/a&gt; to the rescue. For those not following the whole AI rigamarole closely, RAG, or retrieval-augmented generation, is a fairly broad – and &lt;a href=&quot;https://neo4j.com/blog/graphrag-manifesto/&quot;&gt;growing&lt;/a&gt;! – collection of techniques that typically take advantage of &lt;a href=&quot;https://cloud.google.com/discover/what-is-a-vector-database?hl=en&quot;&gt;vector embeddings&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;With a bit of &lt;a href=&quot;https://en.wikipedia.org/wiki/Extract,_transform,_load&quot;&gt;ETL&lt;/a&gt; elbow grease, I built a &lt;a href=&quot;https://qdrant.tech/documentation/tutorials/search-beginners/&quot;&gt;simple semantic store&lt;/a&gt; with &lt;a href=&quot;https://qdrant.tech/&quot;&gt;Qdrant&lt;/a&gt; that contained about 10K songs from a lyric database I purchased at &lt;a href=&quot;https://www.usabledatabases.com/&quot;&gt;Usable databases&lt;/a&gt;. With this semantic store, I could now take any prompt, generate an embedding for it, and then search the store, returning the top N results. Easy-peasy &lt;a href=&quot;https://breadboard-ai.web.app/?board=https%3A%2F%2Fbreadboard.live%2Fboards%2F%40dimitri%2Fget-ten-songs.bgl.json&quot;&gt;in Breadboard&lt;/a&gt;:&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://glazkov.com/2024/07/16/teaching-ai-to-write-lyrics/image-4.1600.webp&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;
&lt;p&gt;Great! I can get N songs related to the prompt! What do I do with them?&lt;/p&gt;
&lt;p&gt;I then spent a bunch of time playing with various ideas and eventually settling on an architecture. Instead of the loosey-goosey flow of steps, I decided on a simple prompt -&amp;gt; proposal -&amp;gt; lyric pipeline. A song is produced with two LLM invocations: one – let’s call it “Proposal Developer” – to generate a song proposal from a prompt, and one (the “Song Writer”) to generate lyrics from the proposal. Each invocation uses a few-shot prompt. The first one has a bunch of examples of how to create a song proposal from a prompt, and the second one has a bunch of examples of how to produce lyrics from the proposal.&lt;/p&gt;
&lt;p&gt;Wait a second. I only have the lyrics examples. To implement the few-shot prompting properly, I need to provide two sets of pairs: a “prompt -&amp;gt; proposal” set for the “Proposal Developer” and a “proposal -&amp;gt; lyrics” set for the “Song Writer”.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://glazkov.com/2024/07/16/teaching-ai-to-write-lyrics/image-5.1600.webp&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;
&lt;p&gt;Welcome to the wonderful world of &lt;a href=&quot;https://mitsloan.mit.edu/ideas-made-to-matter/what-synthetic-data-and-how-can-it-help-you-competitively&quot;&gt;synthetic datasets&lt;/a&gt;. If we don’t have the data, why not just generate it? That’s right, I came up with a simple system that works backwards: it takes a song and then comes up with a proposal for it. Then, in a similar fashion, it takes a proposal and comes up with a prompt that might have been used to generate it. Wiring it all together, I finally had a working system.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://glazkov.com/2024/07/16/teaching-ai-to-write-lyrics/image-6.1600.webp&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;
&lt;p&gt;Immediately, I could tell the difference. The introduction of few-shot examples dramatically changed the LLM’s lyrical prowess, making it more flexible, interesting, and &lt;em&gt;alive&lt;/em&gt;. It would still insert “whispers” and an occasional “symphony” here and there, but the change felt significant. Trippy stuff. Here are a few examples: “&lt;a href=&quot;https://suno.com/song/f7f431c7-c8c9-43d0-b0ac-8a519e61343d&quot;&gt;The Algorithm of Us&lt;/a&gt;”, where I literally just copy-pasted the output (I even forgot to remove the title of the song from the body 🤦). The “&lt;a href=&quot;https://suno.com/song/582cfb85-71c4-4eed-9e46-089caea2c009&quot;&gt;Uhh… What Were The Words Again?&lt;/a&gt;” ended up being particularly cute. I did have to make some edits to these lyrics, but only to make them more palatable for Suno’s consumption.&lt;/p&gt;
&lt;p&gt;As a finishing touch, I multiplied the lyricists to produce three variants of lyrics. This was a last-minute thought: what if I give song writers different personas? Would their output vary? Turns out, it does! And, just like &lt;a href=&quot;https://www.midjourney.com/&quot;&gt;Midjourney&lt;/a&gt; and &lt;a href=&quot;https://suno.com/&quot;&gt;Suno&lt;/a&gt; and &lt;a href=&quot;https://udio.com/&quot;&gt;Udio&lt;/a&gt; found out, offering more than one output is a valuable trick.&lt;/p&gt;
&lt;p&gt;Intentionally skipping over some lazy synthetic data generation techniques (this post is already too long), here is the &lt;a href=&quot;https://breadboard-ai.web.app/?board=https%3A%2F%2Fbreadboard.live%2Fboards%2F%40dimitri%2Fsong-writer-5.bgl.json&quot;&gt;final design&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://glazkov.com/2024/07/16/teaching-ai-to-write-lyrics/image-7.1600.webp&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;
&lt;p&gt;Sadly, you won’t be able to run boards from this part of the journey directly. Unlike the Gemini API key, which &lt;a href=&quot;https://aistudio.google.com/app/apikey&quot;&gt;can be easily obtained&lt;/a&gt;, Qdrant keys are tied to collections, which means that you will need to build your own lyrics database to get it to work. However, if you send an email to &lt;a href=&quot;mailto:dglazkov.writeMeASong@valtown.email&quot;&gt;dglazkov.writeMeASong@valtown.email&lt;/a&gt;, you will receive back a song that was generated using the body of the email as a prompt. Or, you can peruse &lt;a href=&quot;https://docs.google.com/spreadsheets/u/0/d/12ZWyqluNwuHdLUoffHDrsTRieabeERiwH9jTJMPtmc0/edit&quot;&gt;this massive side-by-side eval spreadsheet&lt;/a&gt;, which is automatically updated by the cron job, filling out variant columns anytime I add a new prompt row.&lt;/p&gt;
&lt;h3&gt;&lt;strong&gt;Part 6: Not the end&lt;/strong&gt;&lt;/h3&gt;
&lt;p&gt;I am not quite ready to declare victory yet. As the quality of lyrics improved, flaws in other parts of the system became more apparent. For instance, the final variant is not that great at proposal generation. And that makes sense, since the proposals aren’t original – they are &lt;a href=&quot;https://simonwillison.net/2024/May/8/slop/&quot;&gt;synthetic data&lt;/a&gt;, which means that the creative potential of the proposals is limited by the confines of model inference. Which means that there’s work to be done.&lt;/p&gt;
&lt;p&gt;One big lesson for me in this project is this: while it may seem like turning to LLMs to perform a task is a simple step, when we dig down into the nature of the task and really try to get that task done reliably and with the quality we desire, simple prompt engineering simply won’t do. We need AI systems composed of various &lt;a href=&quot;https://glazkov.com/2023/08/28/ai-patterns-and-breadboard/&quot;&gt;AI patterns&lt;/a&gt; to take us where we want to go.&lt;/p&gt;
</content></entry>
<entry><title>Jamming with Udio</title><link rel="alternate" type="text/html" href="https://glazkov.com/2024/06/05/jamming-with-udio/"/><id>tag:www.glazkov.com,2024-06-05:/2024/06/05/jamming-with-udio/</id><published>2024-06-05T22:43:52-07:00</published><updated>2024-06-05T22:43:52-07:00</updated><category term="what-dimitri-learned" label="What Dimitri Learned"/><summary>Today, I had my first jam session with Udio. With the introduction of audio prompting, I am now able to use my own sound as the starting point for a generated track. This seems like a leap forward, despite the…</summary><content type="html">&lt;p&gt;Today, I had my first jam session with &lt;a href=&quot;https://udio.com/&quot;&gt;Udio&lt;/a&gt;. With the introduction of &lt;a href=&quot;https://twitter.com/udiomusic/status/1798448478877794574&quot;&gt;audio prompting&lt;/a&gt;, I am now able to use my own sound as the starting point for a generated track. This seems like a leap forward, despite the actual product still being quite clunky. It’s a leap, because through introducing audio upload, Udio managed to merge casino creativity with the other, more traditional kind. Let me walk you through what I found.&lt;/p&gt;
&lt;p&gt;As my first try, I fed Udio one of my old abandoned loops. As part of making music, I often arrive at dead ends: promising loops with which I can’t figure out what to do. I have a bunch.&lt;/p&gt;
&lt;p&gt;Then, I extended this loop from both ends with Udio, producing a decent trance track in a matter of a few minutes. It’s not going to win any awards, but it’s definitely farther than I’ve been able to walk on my own.&lt;/p&gt;
&lt;p&gt;Here’s the original loop that I made a while back:&lt;/p&gt;
&lt;p&gt;&lt;a href=&quot;https://w.soundcloud.com/player/?visual=true&amp;amp;url=https%3A%2F%2Fapi.soundcloud.com%2Ftracks%2F267923436&amp;amp;show_artwork=true&amp;amp;maxheight=1000&amp;amp;maxwidth=840&quot;&gt;2016-06-06 by dglazkov&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Here’s the finished Udio track: &lt;a href=&quot;https://www.udio.com/songs/usCmcABg3yP4aC1J8S5WCA&quot;&gt;https://www.udio.com/songs/usCmcABg3yP4aC1J8S5WCA&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Extending existing audio clips fits well into the standard Udio process.&lt;/p&gt;
&lt;p&gt;In the standard Udio process, we get 32 seconds of audio as a starting point, and then we &lt;a href=&quot;https://www.udio.com/guide#extensions&quot;&gt;iteratively extend&lt;/a&gt; this audio from either end to produce music that we like. Each iterative extension is an opportunity to make some choices – the casino creativity at its finest.&lt;/p&gt;
&lt;p&gt;When uploading the audio prompt, the prompt becomes the first N seconds of the audio (the N depends on the length of the audio prompt we load).&lt;/p&gt;
&lt;p&gt;Udio tries to match the prompt’s tempo and the style, expanding it, and in the process, riffing on it. It feels like extrusion: pushing more music through the template that I defined. As Udio expands the clip, it adds new details to what was in the original clip, trying to predict what might have been playing before or after that clip.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://glazkov.com/2024/06/05/jamming-with-udio/image-2.1600.webp&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;
&lt;p&gt;You can still hear the original loop in the finished track at 3:12. It is bookended by entirely new sound that now fits seamlessly around it. The music around it is something that Udio extruded, generating it using the original loop as a template.&lt;/p&gt;
&lt;p&gt;The presence of the original loop hints at the connection between two kinds of creativity that I mentioned earlier. For instance, I could imagine myself sitting down with Ableton and building out a catchy loop, then shifting to Udio to help me imagine the track that would contain this loop. I could then go back to Ableton and use the results of our little jam session as inspiration.&lt;/p&gt;
&lt;p&gt;As my next try, I did something slightly different. I gave a simple melody to Udio and then rolled the casino dice until I’ve gotten the right sound. At this point, Udio anchors on the audio prompt quite firmly, so if you give it a piano (like I did) and ask for a saxophone, it might take a few attempts to produce a rendition of the melody with a different instrument.&lt;/p&gt;
&lt;p&gt;Here, I was looking to create something that sounds like a film score, so I was looking for strings. After a little while, Udio relented and gave me an extension that seemed right.&lt;/p&gt;
&lt;p&gt;At that point, I trimmed the original clip from the Udio track. Now that Udio had learned my melody, I no longer needed the original material, since it didn’t fit with the vibe I was looking for. This removal of the original trick is something I expect to be pretty common. For instance, I could hum a melody or peck it single-note on my keyboard. My intuition is that Udio will (sooner or later) have a “remix” feature for audio prompts, where we can start with the sound of my whistling of a tune and then shape it directly, rather than waiting for the right extension to happen.&lt;/p&gt;
&lt;p&gt;Once the right vibe was established, the rest of the process was quite entertaining. It was fun to watch Udio reimagine my original melody in minor for the “scary” part of the movie and boost it with drums and a full orchestra at the climax.&lt;/p&gt;
&lt;p&gt;Here’s the original melody:&lt;/p&gt;
&lt;p&gt;&lt;a href=&quot;https://w.soundcloud.com/player/?visual=true&amp;amp;url=https%3A%2F%2Fapi.soundcloud.com%2Ftracks%2F205990936&amp;amp;show_artwork=true&amp;amp;maxheight=1000&amp;amp;maxwidth=840&quot;&gt;2015-05-12 by dglazkov&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Here’s the finished track: &lt;a href=&quot;https://www.udio.com/songs/eCMEFFSGnoicnHR4S5fRK1&quot;&gt;https://www.udio.com/songs/eCMEFFSGnoicnHR4S5fRK1&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;In both cases, the process felt a lot more like a jam session than creative casino, because the final product included a distinct contribution from me. It wasn’t something that I just told Udio to do. I gave it raw material to riff on. And it did a pretty darned good job.&lt;/p&gt;
</content></entry>
<entry><title>Casino Creativity</title><link rel="alternate" type="text/html" href="https://glazkov.com/2024/06/02/casino-creativity/"/><id>tag:www.glazkov.com,2024-06-02:/2024/06/02/casino-creativity/</id><published>2024-06-02T22:17:55-07:00</published><updated>2024-06-02T22:17:55-07:00</updated><category term="what-dimitri-learned" label="What Dimitri Learned"/><summary>I’ve been geeking out on AI-generated music services Suno and Udio, and it’s been super-interesting to see them iterate and ship quickly. It looks like there might be a value niche in this particular neck of the woods in the…</summary><content type="html">&lt;p&gt;I’ve been geeking out on AI-generated music services &lt;a href=&quot;https://suno.com/&quot;&gt;Suno&lt;/a&gt; and &lt;a href=&quot;https://udio.com/&quot;&gt;Udio&lt;/a&gt;, and it’s been super-interesting to see them iterate and ship quickly. It looks like there might be a value niche in this particular neck of the woods in the larger generative AI space. There are tons of users with very active Discord communities for both, and it does not seem like the interest is waning.&lt;/p&gt;
&lt;p&gt;The overall arc of the generative music story seems to follow that of &lt;a href=&quot;https://glazkov.com/2022/06/08/midjourney/&quot;&gt;Midjourney&lt;/a&gt;, with the interest primarily fueled by the phenomenon that I would like to name “casino creativity”. Let’s see if I can define what I mean by that.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://glazkov.com/2024/06/02/casino-creativity/image.1536.webp&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;
&lt;p&gt;I would like to start by positing that the craving to create is in every one of us. Some of us are more blessed than others in also having skills to satisfy this craving. Moreso, I am going to proclaim that most of us are unable to fully embrace our creative selves because we lack some of the skills required to take flight.&lt;/p&gt;
&lt;p&gt;For instance, I can make music. I have been making music since I was a teenager. For me, satisfying my craving for creativity is just a matter of firing up &lt;a href=&quot;https://www.ableton.com/en/&quot;&gt;Ableton&lt;/a&gt;. When I am skilled in the medium, the friction to create is low. All it takes is being next to my keyboard (and &lt;a href=&quot;https://www.ableton.com/en/push/&quot;&gt;Push&lt;/a&gt;), a little inspiration – and a track begins to emerge.&lt;/p&gt;
&lt;p&gt;However, I can’t sing. Like, not at all. Like, don’t even ask me. In the music school, when testing out for the choir or orchestra, I was asked to sing. After me belting out a few words (not even a full verse!), the teacher yelled: “The Orchestra! The Orchestra!”&lt;/p&gt;
&lt;p&gt;Being a music producer without a voice is a story of unrequited love. I have to settle for tracks without lyrics. The instrumentals are nice, but it’s just not the same &lt;em&gt;feeling&lt;/em&gt; without a voice.&lt;/p&gt;
&lt;p&gt;So obviously, ever since the current generative AI spring blossomed, I’ve been on a quest to find a way to sate this creative craving. I played with &lt;a href=&quot;https://www.celemony.com/en/melodyne/what-is-melodyne&quot;&gt;Melodyne&lt;/a&gt; and &lt;a href=&quot;https://dreamtonics.com/synthesizerv/&quot;&gt;Synth V&lt;/a&gt;, and while they both offered a path forward, the barrier to entry was just too high. Gaining a voice is not the same as knowing how to sing. It’s about the same distance between being able to buy a violin and knowing how to play one.&lt;/p&gt;
&lt;p&gt;Things started shifting with &lt;a href=&quot;https://www.suno.wiki/faq/about-suno-ai/what-happened-to-chirp-and-bark/&quot;&gt;Chirp&lt;/a&gt;. This was the original model created by Suno, and it was Discord-only, very similar to Midjourney – feed it the lyrics alongside a description of the vibe, and out comes a 30-second clip of music. Not just music – it also sang out the lyrics I gave it!&lt;/p&gt;
&lt;p&gt;Brain-splosion. Sort of. The output quality of Chirp was pretty weak-sauce. It was not the music I could share with anyone except for minor giggles and an eye roll. I forgot about Chirp for a little while, until this spring Suno came out with the v3 of their sound model. I heard about it from &lt;a href=&quot;https://www.komoroske.com/&quot;&gt;Alex&lt;/a&gt;, whose work colleagues composed various songs to celebrate his last day at Stripe.&lt;/p&gt;
&lt;p&gt;Ok, &lt;em&gt;now&lt;/em&gt; we were getting somewhere. Songs generated with Suno v3 possessed that extra emotional weight that made them nearly passable as listenable music. When Udio came out shortly with their own model, it upped the barrier even more. I was blown away by some of the output. Just like that, my age of voiceless musicing was over. I could type in some lyrics and get back something that expressed it back to me as music.&lt;/p&gt;
&lt;p&gt;Every generation took only about a minute and produced two variants for me to pick from. I could choose the one I like and extend it or remix it – or roll the dice again. All it takes is a click.&lt;/p&gt;
&lt;p&gt;It’s this metaphorical rolling of the dice that gives the name to the titular term. As I was pushing the “Create 🎶” button, I realized that the anticipation of the output had a pronounced dopamine hit. What will come out? Will it be something like Duran Duran? Or maybe more like Bono? Will it go in a completely different direction? Gimme gimme gimme. I was hooked on Suno.&lt;/p&gt;
&lt;p&gt;&lt;a href=&quot;https://glazkov.com/2024/06/02/casino-creativity/&quot;&gt;The casino creativity loop: bring vibes, press create, get variants, and pick or roll the dice again.&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Casino creativity is a form of creative expression that emerges when the creative environment has such a low barrier to entry that the main way to express my creativity is through providing preference: selecting one choice out of a few offered. A creative casino is a place where all I need to bring is my money and my vibes: everything else will be provided.&lt;/p&gt;
&lt;p&gt;&lt;a href=&quot;https://glazkov.com/2024/06/02/casino-creativity/&quot;&gt;Craft creativity and casino creativity compared by what you bring and how you express yourself.&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Midjourney is one of the first environments where I experienced casino creativity. There’s something subtly addictive about looking for that prompt and seeing those 4-up images that pop out. I know peeps who can spend a very long time tweaking and tuning their inputs. We could argue that prompt craftsmanship itself is a skill that must be acquired. But this skill has a short expiration date – as the models improve and change, the need for prompt-foo diminishes rapidly.&lt;/p&gt;
&lt;p&gt;At the end, what we’re left with is pressing the button and making choices. Casino creativity is less about the skill and more about the vibes.&lt;/p&gt;
&lt;p&gt;Not to say that casino creativity isn’t able to produce interesting – and perhaps even beautiful – things. Vibes are important – and some of us have more latent vibes hidden within us that we could ever realize. Ultimately, casino creativity is very similar in spirit to the democratization of writing that we’d seen with the Web. I am not yet ready to proclaim that casino creativity is somehow less intriguing and full of potential than any other type of creativity. Just like my Midjourney-obsessed friends, I can see how unleashing one’s creative energy might lead to surprising and wonderful results.&lt;/p&gt;
&lt;p&gt;Here’s a twist though. As long as I have the credits to roll the dice, I can see if my vibes work for others. Both Suno and Udio are vying to be the place where music happens. I can look at what’s popular and peruse the top charts. It’s all very naive and simplistic at the moment.&lt;/p&gt;
&lt;p&gt;Yet, when executed ruthlessly (and it’s inevitable that somebody will do this), the creative casino is not just the place where I can express my creativity. It’s also the place where I can get the extra dopamine release of seeing my song climb the charts – of my vibes becoming recognized. Come for the vibes, stay for the likes.&lt;/p&gt;
&lt;p&gt;An interesting effect of introducing generative AI, it seems, is that we’re likely to see more creative casinos and more ventures capitalizing on casino creativity itself. And we have to ponder the implications of that.&lt;/p&gt;
</content></entry>
<entry><title>Chrysalis</title><link rel="alternate" type="text/html" href="https://glazkov.com/2024/05/08/chrysalis/"/><id>tag:www.glazkov.com,2024-05-08:/2024/05/08/chrysalis/</id><published>2024-05-08T22:05:11-07:00</published><updated>2024-05-08T22:05:11-07:00</updated><category term="what-dimitri-learned" label="What Dimitri Learned"/><summary>A moment of clarity I suspect it’s playing a game I reach out, and it’s gone My unreality By a different name Is what yet to be drawn.</summary><content type="html">&lt;p&gt;A moment of clarity&lt;br /&gt;
I suspect it’s playing a game&lt;br /&gt;
I reach out, and it’s gone&lt;br /&gt;
My unreality&lt;br /&gt;
By a different name&lt;br /&gt;
Is what yet to be drawn.&lt;/p&gt;
&lt;p&gt;What needs to be done&lt;br /&gt;
Always feels right&lt;br /&gt;
And when the story had spun&lt;br /&gt;
Shying from light,&lt;br /&gt;
It always begs to forget&lt;br /&gt;
Filling the stores of regret.&lt;br /&gt;
&lt;br /&gt;
Do caterpillars dream of flying?&lt;br /&gt;
Do they know they will have wings?&lt;br /&gt;
Do they realize that being land-bound&lt;br /&gt;
Is just a temporary thing?&lt;br /&gt;
&lt;br /&gt;
Imperceptible? Immense?&lt;br /&gt;
I can’t tell, barely there myself&lt;br /&gt;
Unable to keep the facade&lt;br /&gt;
of pretense&lt;br /&gt;
swallowed by the intense&lt;br /&gt;
losing all sense&lt;br /&gt;
of space, time, and self.&lt;/p&gt;
&lt;p&gt;Losing all sense&lt;br /&gt;
Changing, yet staying the same,&lt;br /&gt;
Thrashing my wits and will,&lt;br /&gt;
Am I still me?&lt;br /&gt;
In my defense,&lt;br /&gt;
This question is unanswered still,&lt;br /&gt;
While being reframed.&lt;br /&gt;
What will I be?&lt;/p&gt;
</content></entry>
<entry><title>The third option</title><link rel="alternate" type="text/html" href="https://glazkov.com/2024/05/06/the-third-option/"/><id>tag:www.glazkov.com,2024-05-06:/2024/05/06/the-third-option/</id><published>2024-05-06T20:49:27-07:00</published><updated>2024-05-06T20:49:27-07:00</updated><category term="uncategorized" label="Uncategorized"/><summary>I facilitated a workshop on systems thinking recently (or “lensical thinking” as I’ve come to call it). The purpose of the lensical thinking workshop is to provide a coherent set of tools for approaching problems of particularly unsolvable kind, using…</summary><content type="html">&lt;p&gt;I facilitated a workshop on systems thinking recently (or “lensical thinking” as I’ve come to call it). The purpose of the lensical thinking workshop is to provide a coherent set of tools for approaching problems of particularly &lt;a href=&quot;https://glazkov.com/2022/03/16/solved-solvable-unsolvable/&quot;&gt;unsolvable&lt;/a&gt; kind, using the concept of lenses as the key organizing concept.&lt;/p&gt;
&lt;p&gt;One of the participants became quite fond of the &lt;a href=&quot;https://kylemore.nd.edu/assets/433638/managing_polarities_well.pdf&quot;&gt;polarity&lt;/a&gt; lens, realizing how broadly applicable it is, from the larger organizational challenges (like “centralized vs. decentralized”), to something as in-the-moment as hidden tension in a conversation.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://glazkov.com/2024/05/06/the-third-option/image.1480.webp&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;
&lt;p&gt;The participant even pointed out that the use of lenses, at its core, has a polarity-like quality: a wide diversity of lenses brings more nuance to the picture and as such, makes the picture less simple and crisp. The tension between the desire for clarity and the desire to see the fuller picture are in tension, causing us – if we’re not careful – to swing wildly between the two extremes.&lt;/p&gt;
&lt;p&gt;That one resonated with me, because it’s a very common problem that every leader faces: in a vastly complex space of mostly unsolvable problems, how do they speak with clarity and intention while still leaving room for nuance and convey who fuzzy everything actually is?&lt;/p&gt;
&lt;p&gt;The deeper insight here is that we are surrounded by polarities. Behind every bad decision, there is undoubtedly a pair of what seems like two equally bad conflicting options and the agony of having to commit to one – often knowing full well that some time from now, we’ll have to swing all the way to the other side. In that moment, the polarities rule us, both individually and collectively.&lt;/p&gt;
&lt;p&gt;In a separate conversation with a dear friend of mine, we arrived at a point of clarity when we both saw how a problem we’ve been looking at was a gnarly polarity. We saw how, over time, the only seemingly available options had this quality of “either do &amp;lt;clearly unsatisfying thing&amp;gt; or do &amp;lt;another clearly unsatisfying thing&amp;gt;”, repeating over and over.&lt;/p&gt;
&lt;p&gt;When that happened, we both felt a bit of a despondence setting it. This has been going on for a long time. What is the way out of this swing of a pendulum? Is there even one?&lt;/p&gt;
&lt;p&gt;This question was surprisingly helpful: “What’s the third option?” When caught in the grip of the polarity, it feels counterintuitive to slow down and look around for something other than the usual. However, given that the other choices leave us animated by the pendulum, the least we can do is to invest into observing what is happening to us.&lt;/p&gt;
&lt;p&gt;The third option is definitely that of deciding to do better this time, now that we’ve seen how we are subject to polarity. We may claim to have a more active role, to elevate the polarity into something productive, and many consulting hours are spent heaving that weight. Unfortunately, the pendulum cares very little about that. And inevitably, we find ourselves back in its grip.&lt;/p&gt;
&lt;p&gt;The third option is rarely that of nihilism, of deciding that the world around us is inherently bad and we are better off uncontaminated by it. The &lt;a href=&quot;https://www.youtube.com/watch?v=s93KC4AGKnY&quot;&gt;WarGames ending&lt;/a&gt; was beautiful, but in a polarity, it’s an option that is chosen when guided by naivete – for nihilism and naivete are close cousins.&lt;/p&gt;
&lt;p&gt;The third option is rarely that of avoidance – even if it’s the avoidance through drastic action, like proclaiming that we’re moving to an island/starting a new business/adventure, or joining a group of like-minded, clear-sighted individuals that suspiciously smells like a cult. When we choose this path, we mustn’t be surprised how the same old chorus cuts into our apparently new song.&lt;/p&gt;
&lt;p&gt;The presence of a polarity is a sign that our thinking is too constrained, flattened either by the burden of the allostatic load or simply the absence of experience with higher dimensional spaces. The search for the “third option” is an attempt to break out of a two-dimensional picture – into the third dimension, to see the limits of the movements along the X and Y axis and look for ways to fold our two-dimensional space into the axis Z.&lt;/p&gt;
&lt;p&gt;Put differently, polarities are a sign that our mental modals are due for reexamination. Polarities, especially particularly vicious ones, are a blinking light on our dashboard of &lt;a href=&quot;https://glazkov.com/adt-primer/&quot;&gt;vertical development&lt;/a&gt;. This means that the third option will not be obvious and immediately seen. “Sitting with it” is a very common refrain of third-option seekers. The axis Z is hard to comprehend with a two-dimensional brain, and some serious stretching of the mind will be necessary to even catch the first glimpses of it.&lt;/p&gt;
&lt;p&gt;Most significantly, that first glimpse or even a full realization doesn’t not necessarily bring instant victory. Orthogonality is a weird trick. Opening up a new dimension does not obsolete the previous ones – it just creates more space. Those hoping for a neat solution will be disappointed.&lt;/p&gt;
&lt;p&gt;Instead of “solving” a polarity, we might find a whole different perspective, which may not change the situation in a dramatic way – at least not at first. We might find that we are less attached to the effects of a pendulum. We might find that we no longer suffer at the extremes, and have a tiny bit more room to move, rather than feeling helpless. We might find that the pendulum swings no longer seem as existential. And little by little, its impact on us will feel less and less intense.&lt;/p&gt;
</content></entry>
<entry><title>Flexibility of the medium</title><link rel="alternate" type="text/html" href="https://glazkov.com/2024/03/30/flexibility-of-the-medium/"/><id>tag:www.glazkov.com,2024-03-30:/2024/03/30/flexibility-of-the-medium/</id><published>2024-03-30T14:44:32-07:00</published><updated>2024-03-30T14:44:32-07:00</updated><category term="uncategorized" label="Uncategorized"/><category term="what-dimitri-learned" label="What Dimitri Learned"/><summary>I did this fun little experiment recently. I took my two last posts (Thinking to Write and The Bootstrapping Phase) and asked an LLM to turn them into lyrics. Then, after massaging the lyrics a bit, to better fit with…</summary><content type="html">&lt;p&gt;I did this fun little experiment recently. I took my two last posts (&lt;a href=&quot;https://glazkov.com/2024/03/28/thinking-to-write/&quot;&gt;Thinking to Write&lt;/a&gt; and &lt;a href=&quot;https://glazkov.com/2024/03/29/the-bootstrapping-phase/&quot;&gt;The Bootstrapping Phase&lt;/a&gt;) and asked an LLM to turn them into lyrics. Then, after massaging the lyrics a bit, to better fit with the message I wanted to come across, I played with &lt;a href=&quot;https://www.suno.ai/&quot;&gt;Suno&lt;/a&gt; for a bit to transform them into short, 2-minute songs – a sort of vignettes for my long-form writing. Here they are:&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://glazkov.com/2024/03/30/flexibility-of-the-medium/image-3.256.webp&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href=&quot;https://app.suno.ai/song/b98a2ed0-c137-4f86-960f-0d406628d750/&quot;&gt;Thinking to write on Suno&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://glazkov.com/2024/03/30/flexibility-of-the-medium/image-2.256.webp&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href=&quot;https://app.suno.ai/song/e4c4fdfe-844a-485d-bd6e-f2552f7f9133/&quot;&gt;Unbaked Cookie Testers on Suno&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Catchy, content-ful, and in total, maybe 20 minutes to make. And, it was so much fun! I got to think about what’s important, how to express somewhat dry writing as emotionally interesting. I got to consider what music style would resonate with what I am trying to convey in the original content.&lt;/p&gt;
&lt;p&gt;This got me thinking. What I was doing in those few minutes was transforming the medium of the message. With generative AI, the cost of medium transformation seems to be going down dramatically.&lt;/p&gt;
&lt;p&gt;I know how to make music. I know how to write lyrics. But it would have taken me hours of uninterrupted time (which would likely translate into months of elapsed time) to actually produce something like this. Such investment makes medium transformation all but prohibitive. It’s just too much effort.&lt;/p&gt;
&lt;p&gt;However, with the help of a couple of LLMs, I was able to walk over this threshold like there’s nothing to it. I had fun, and – most importantly – I had total agency in the course of the transformation. I had the opportunity to tweak the lyrics. I played around with music styles and rejected a bunch of things I didn’t like. It was all happening in one-minute intervals, in rapid iteration.&lt;/p&gt;
&lt;p&gt;This rapid iteration was more reminiscent of jamming with a creative partner than working with a machine. Gemini gave me a bunch of alternatives (some better than others), and Suno was eager to mix bluegrass with glitch, no matter how awful the results. At one moment I paused and realized: wow, this feels closer to the ideal creative collaboration than I’ve ever noticed before.&lt;/p&gt;
&lt;p&gt;What’s more importantly, the new ease of medium transformation opens up all kinds of new possibilities. If we presume – and that’s a big one – for a moment that the cost of medium transformation will indeed go down for all of us, we now can flexibly adjust the medium according to the circumstances of the audience.&lt;/p&gt;
&lt;p&gt;The message does not have to be locked in a long-form post or an academic tome, waiting for someone to summarize it in an easily consumable format. We can turn it into a catchy tune, or a podcast. It could be a video. It could be something we don’t yet have, like a “zoomable” radio station where I listen to a stream of short-form snippets of ideas, and can “zoom in” to the ones I am most interested in, pausing the stream to have a conversation with the avatar of the author of the book, or have an avatar of someone I respect react to it. I could then “zoom out” again and resume the flow of short-form snippets.&lt;/p&gt;
&lt;p&gt;Once flexible, the medium of the message can adapt and meet me where I am currently.&lt;/p&gt;
&lt;p&gt;The transformation behind this flexibility will often be lossy. Just like the tweets pixelate the nuance of the human soul, turning a book into a two-verse ditty will flatten its depth. My intuition is that this lossiness and the transformation itself will usher in a whole new era of UX explorations, where we struggle to find that new shared way of interacting with the infinitely flexible, malleable canvas of the medium. Yup, this is going to get weird.&lt;/p&gt;
</content></entry>
<entry><title>The Bootstrapping Phase</title><link rel="alternate" type="text/html" href="https://glazkov.com/2024/03/29/the-bootstrapping-phase/"/><id>tag:www.glazkov.com,2024-03-29:/2024/03/29/the-bootstrapping-phase/</id><published>2024-03-29T15:11:30-07:00</published><updated>2024-03-29T15:11:30-07:00</updated><category term="dx-pattern" label="DX Pattern"/><category term="what-dimitri-learned" label="What Dimitri Learned"/><summary>I think I have a slightly better way of describing a particular moment in a product’s life that I alluded to in Rock tumbler teams, Chances to get it right, and later, Build a thing to build the thing. I…</summary><content type="html">&lt;p&gt;I think I have a slightly better way of describing a particular moment in a product’s life that I alluded to in &lt;a href=&quot;https://glazkov.com/2021/07/30/rock-tumbler-teams/&quot;&gt;Rock tumbler teams&lt;/a&gt;, &lt;a href=&quot;https://glazkov.com/2023/02/08/chances-to-get-it-right/&quot;&gt;Chances to get it right&lt;/a&gt;, and later, &lt;a href=&quot;https://glazkov.com/2023/08/03/build-a-thing-to-build-the-thing/&quot;&gt;Build a thing to build the thing&lt;/a&gt;. I call this moment the “bootstrapping phase.” It very much applies to consumer-oriented products as well, but is especially pronounced – and viscerally felt – in the developer experience spaces.&lt;/p&gt;
&lt;p&gt;I use the term “bootstrapping phase” to point at the period of time when our aspiring developer product is facing a tension of two forces. On one hand, we must start having actual users to provide the essential feedback loop that will guide us. On the other hand, the product itself isn’t yet good enough to actually help users.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://glazkov.com/2024/03/29/the-bootstrapping-phase/image-1.1494.webp&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;
&lt;p&gt;The bootstrapping phase is all about navigating this tension in the most effective way. Move a little too much away from having the feedback loop, and we run the danger of building something that nobody wants. Go a little too hard on growing the user base, and we might prematurely conclude the story of the product entirely.&lt;/p&gt;
&lt;p&gt;The trick about this phase is that all assumptions we might have made about the final shape of what we’re building are up in the air. They could be entirely wrong, based on our misunderstanding of the problem space, or overfit to our particular way of thinking. These assumptions must face the contact with reality, be tested – and necessarily, change.&lt;/p&gt;
&lt;p&gt;The word “bootstrapping” in the name refers to this iterative process of evolving our assumptions in collaboration with a small group of users who are able and eager to engage.&lt;/p&gt;
&lt;p&gt;Those of you hanging out in the &lt;a href=&quot;https://github.com/breadboard-ai/breadboard&quot;&gt;Breadboard project&lt;/a&gt; heard me use the expression “unbaked cookies”: we would like to have you try the stuff we made, and we’re pretty sure it’s not yet cooked. Our cookies might have bits of crushed glass in them, and we don’t yet know if that cool new ingredient we added last night is actually edible. Yum.&lt;/p&gt;
&lt;p&gt;At the bootstrapping phase of the project, the eagerness to eat unbaked cookies is a precious gift. I am in awe of the folks I know who have this mindset. For them, it’s a chance to play with something new and influence – often deeply – what the next iteration of the product will look like. On the receiving end, we get a wealth of insights they generate by trying – and gleefully failing – to use the product as intended.&lt;/p&gt;
&lt;p&gt;For this process to work, we must show a complementary eagerness to change our assumptions. It is often disheartening to see our cool ideas be dismantled with a single click or a confused stare. Instead of falling prey to the temptation of filtering out these moments, we must use them as guiding signals – these are the bits that take us toward a better product.&lt;/p&gt;
&lt;p&gt;The relationship between the bakers of unbaked cookies and cookie testers requires a lot of trust – and this can only be built over time. Both parties need to develop a sense of collaborative relationship that allows them to take risks, challenging each other. As disconcerting it may be, some insights generated might point at fundamental problems with the product – things that aren’t fixable without rethinking everything. While definitely a last resort, such rethinking must always be on the table. Bits of technology can be changed with some work. The mental models behind the product, once it ships to the broader audience are much, much more difficult to change.&lt;/p&gt;
&lt;p&gt;Because of that, the typical UX studies aren’t a great fit for the bootstrapping phase of the project. We’re not looking for folks to react to the validity of mental models we imbued the nascent product with. We fully realize that some of them – likely many – are wrong. Instead, we need a collaborative, tight-feedback loop relationship with the potential users, who feel entrusted with steering the product direction through them chewing on not-yet baked cookies. They aren’t just trusted testers of the product. They aren’t just evaluators of it. They are full participants in its development, representing the users.&lt;/p&gt;
</content></entry>
<entry><title>Thinking to write</title><link rel="alternate" type="text/html" href="https://glazkov.com/2024/03/28/thinking-to-write/"/><id>tag:www.glazkov.com,2024-03-28:/2024/03/28/thinking-to-write/</id><published>2024-03-28T21:23:49-07:00</published><updated>2024-03-28T21:23:49-07:00</updated><category term="what-dimitri-learned" label="What Dimitri Learned"/><summary>I’ve had this realization about myself recently, and it’s been rather useful in gaining a bit more understanding about how my mind works. I am writing it down in hopes it would help you in your own self-reflections.</summary><content type="html">&lt;p&gt;I’ve had this realization about myself recently, and it’s been rather useful in gaining a bit more understanding about how my mind works. I am writing it down in hopes it would help you in your own self-reflections.&lt;/p&gt;
&lt;p&gt;The well-worn “Writing to think” maxim is something that’s near and dear to my heart: weaving a sequential story of the highly non-linear processes that are happening in my mind is a precious tool. I usually recommend developing the muscle for writing to think as a way to keep our thoughts organized to my colleagues and friends. Often, when I do, I am asked: “What do I write about?”&lt;/p&gt;
&lt;p&gt;It’s a good question. At least for me, the ability to write appears to be closely connected to the space in which I am doing the thinking. It seems like the whole notion of “writing to think” might also work in reverse: when I don’t have something to write about, it might be a signal that my thinking space is fairly small or narrow.&lt;/p&gt;
&lt;p&gt;There might be fascinating and very challenging problems that I am working on. I could be spending many hours wracking my brain trying to solve them. However, if this thinking doesn’t spur me to write about it, I am probably inhabiting a rather confined problem space.&lt;/p&gt;
&lt;p&gt;I find that writing code and software engineering in general tend to collapse this space for me. Don’t get me wrong, I love making software. It’s one of those things that I genuinely enjoy and get a “coder’s high” from.&lt;/p&gt;
&lt;p&gt;Yet, when doing so, I find that my thoughts are sharply focused and narrow. They don’t undulate and wander in vast spaces. They don’t get lost just for the sake of getting lost. Writing code is about bringing an idea to life. It’s a very concretizing process. Writing code is most definitely a process of writing to think, but it’s more of “writing it”, rather than “writing about it”.&lt;/p&gt;
&lt;p&gt;The outcome is a crisp – albeit often spaghetti-like – set of instructions that are meant to be understood by a machine, which for all its complicatedness is a lot less complex than a human mind.&lt;/p&gt;
&lt;p&gt;On the other hand, when I was doing more strategy work a few years back, I found myself brimming with ideas to write down. It was very easy to just knock out a post – nearly every idea I had was begging to be played with and turned into a story to share. I was in the wide-open space of thinking among people, and particularly long-term horizon, broad thinking, and wandering.&lt;/p&gt;
&lt;p&gt;Nothing’s wrong with inhabiting smaller problem spaces for a little while. However, it’s probably not something I would pick as the only way of being. “Inhabiting” brings habits, and habits entrench. Becoming entrenched in smaller problem spaces means that the larger spaces become less and less accessible over time, resulting in strategic myopia.&lt;/p&gt;
&lt;p&gt;It seems that to avoid such a diagnosis, we’ve gotta keep finding ways to think in spaces big enough to spur us to write. To use an analogy, “writing to think” is like developing a habit of brushing our teeth, and “thinking to write” is a way to check if we indeed follow this habit. If we find ourselves struggling to write, then maybe we need to broaden the problem space we inhabit.&lt;/p&gt;
</content></entry>
<entry><title>Aircraft Carriers and Zodiac Boats</title><link rel="alternate" type="text/html" href="https://glazkov.com/2024/03/01/aircraft-carriers-and-zodiac-boats/"/><id>tag:www.glazkov.com,2024-03-01:/2024/03/01/aircraft-carriers-and-zodiac-boats/</id><published>2024-03-01T17:11:24-08:00</published><updated>2024-03-01T17:11:24-08:00</updated><category term="what-dimitri-learned" label="What Dimitri Learned"/><summary>An insightful conversation with a colleague inspired me to articulate the distinction between velocity and agility – and all that implies. When I talked about velocity in the past, I sort of elided the fact that agility, or the ability…</summary><content type="html">&lt;p&gt;An insightful conversation with a colleague inspired me to articulate the distinction between velocity and agility – and all that implies. When I talked about velocity in the past, I sort of elided the fact that agility, or the ability to change direction at a given velocity, plays a crucial role in setting organizations for success in certain conditions. I also want to disclaim that the definition of “agility” I use here is only loosely related to the well-known/loved/hated agile methodologies.&lt;/p&gt;
&lt;p&gt;The example I’ve used in the past is that of a zodiac boat and an aircraft carrier. Though they are capable of roughly going at the same velocity, their agility is dramatically different. The zodiac boat’s maneuverability is what gives it a decisive advantage in an environment where the situation changes rapidly. On the other hand, an aircraft carrier is able to sustain velocity for much longer periods of time, which enables it to travel around the globe.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://glazkov.com/2024/03/01/aircraft-carriers-and-zodiac-boats/image.1158.webp&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;
&lt;p&gt;In engineering teams, velocity and agility are often used interchangeably, and I am quite guilty of doing this as well. Only in retrospect I am realizing why some teams that I’ve worked both with (and next to) looked and acted so differently. They were valuing, respectively, velocity or agility.&lt;/p&gt;
&lt;p&gt;When the team favors velocity, it invests significantly into its capacity to achieve maximum sustained speed for the longest possible time. Decision-making and engineering processes, tools and infrastructure all feel like that of an aircraft carrier, regardless of the team’s actual size. It’s like the question on everyone’s mind is: “Can we cross the Pacific Ocean? How many times over?” The team is designed to go far, even if that means sacrificing some velocity to its robustness.&lt;/p&gt;
&lt;p&gt;For instance, the &lt;a href=&quot;https://en.wikipedia.org/wiki/Blink_(browser_engine)&quot;&gt;Blink&lt;/a&gt; team I led a while back was all about velocity, borrowing most of its ethos from &lt;a href=&quot;https://abseil.io/resources/swe-book&quot;&gt;Google engineering culture&lt;/a&gt;. We designed our infrastructure to enable us to &lt;a href=&quot;https://trunkbaseddevelopment.com/&quot;&gt;ship directly from trunk&lt;/a&gt; through diligent test coverage and a phenomenal build system (we built our own!), we followed the practice of code reviews and &lt;a href=&quot;https://blog.chromium.org/2019/11/intent-to-explain-demystifying-blink.html&quot;&gt;the shipping process&lt;/a&gt;. We talked about how this team was built to run for multiple decades.&lt;/p&gt;
&lt;p&gt;This was (and a decade later, still is), of course, the right fit for that project. Rendering engines represent highly overconstrained, immensely complex systems of a relatively well-defined shape. The team that ships a rendering engine will not one day decide to do something completely different. The word “velocity” in such a team is tightly coupled with achieving predictable results over a long period of time.&lt;/p&gt;
&lt;p&gt;However, when the final shape of the value niche is still unknown, and the product-market fit is something we only wistfully talk about, a different structure is needed. Here, the engineering team needs to lean into agility. When they do so, the project will act very differently. It will be more like a zodiac boat: not built to run forever, but rather to zig and zag quickly.&lt;/p&gt;
&lt;p&gt;A project structured like a zodiac boat will have alarmingly few processes and entrenched practices. “What? They don’t do code reviews?” The trunk might be unshippable for periods that would be unacceptable by any standards of an aircraft carrier team. The codebase will have large cheese holes in implementation and test coverage, with many areas only loosely sketched. In a zodiac boat project, everything is temporary, and meant to shift as soon as a new promising approach is discovered.&lt;/p&gt;
&lt;p&gt;Such projects are also typically small-sized. Larger teams means more opinions and more &lt;a href=&quot;https://komoroske.com/slime-mold/&quot;&gt;coordination headwinds&lt;/a&gt;, so zodiac boat projects will favor fewer folks who deeply understand the code base and have no problem diving in and changing everything. They will also attract those who are comfortable with uncertainty. In a highly dynamic situation, the courage to make choices (even if they might not be the right ones), and the skill to keep the &lt;a href=&quot;https://glazkov.com/2021/11/14/the-ooda-loop/&quot;&gt;OODA loop&lt;/a&gt; spinning are paramount.&lt;/p&gt;
&lt;p&gt;A well-organized startup will have to run like a zodiac boat project. Startups rarely form around old ideas or long-discovered value niches. A lot of maneuvering will be necessary to uncover that motherlode. Any attempts to turn this zodiac boat into an aircraft carrier prematurely will dramatically reduce the probability of finding it. This is why ex-Googlers often struggle in startups: their culturally-instilled intuition will direct them to install nuclear reactors and rivet steel panels onto their boats – and in doing so, sink them.&lt;/p&gt;
&lt;p&gt;Which brings me to engineering mastery. In my experience, there are two kinds of successful zodiac boat projects: ones run by people who aren’t that familiar with robust software engineering, and ones who have achieved enough software engineering mastery to know which practices can be broken and disregarded.&lt;/p&gt;
&lt;p&gt;The first group of folks succeeded accidentally. The second – intentionally. This second group knows where to leave the right cheese holes in their project, and will do so consistently with &lt;a href=&quot;https://glazkov.com/2023/10/23/makers-and-magicians/&quot;&gt;magical&lt;/a&gt; results.&lt;/p&gt;
&lt;p&gt;That’s what’s so tricky about forming zodiac boat projects. One can’t just put together a bunch of engineers into a small boat and let them loose. As with any elite special force, zodiac boat projects require a crew that is tightly knit, intrinsically motivated, and skilled to extreme.&lt;/p&gt;
&lt;p&gt;Curiously, aircraft carrier-culture organizations can sometimes produce relatively high agility through what I call &lt;em&gt;ergodic agility&lt;/em&gt;. The ergodic agility refers to the phenomenon where a multitude of projects are given room to start and fail, and over time, through the &lt;a href=&quot;https://en.wikipedia.org/wiki/Ergodic_theory&quot;&gt;ergodic motion&lt;/a&gt;, find a new value niche. Here, the maneuverability is achieved through quantity and diversity of unchanging directions.&lt;/p&gt;
&lt;p&gt;Like the &lt;a href=&quot;https://www.youtube.com/watch?v=hiKuxfcSrEU&quot;&gt;infamous quote&lt;/a&gt; from Shrek, this process looks and feels like utter failure from inside of most of these teams, with the lucky winner experiencing the purest form of &lt;a href=&quot;https://en.wikipedia.org/wiki/Survivorship_bias&quot;&gt;survivorship bias&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;I am not sure if ergodic agility is more or less expensive for a large organization compared to cultivating an archipelago of zodiac boat teams and culture. One thing is certain: to thrive in an ever-changing world, any organization will need to find a way to have both aircraft carriers and zodiac boats in its fleet.&lt;/p&gt;
</content></entry>
<entry><title>AI Baklava</title><link rel="alternate" type="text/html" href="https://glazkov.com/2024/02/15/ai-baklava/"/><id>tag:www.glazkov.com,2024-02-15:/2024/02/15/ai-baklava/</id><published>2024-02-15T22:06:48-08:00</published><updated>2024-02-15T22:06:48-08:00</updated><category term="what-dimitri-learned" label="What Dimitri Learned"/><summary>At its core, the process of layering resolves two counterposed forces. Both originate from the need for predictability in behavior. One from below, where the means are concrete, but the aims are abstract – and one from above, where the…</summary><content type="html">&lt;p&gt;At its core, the process of layering resolves two counterposed forces. Both originate from the need for predictability in behavior. One from below, where the means are concrete, but the aims are abstract – and one from above, where the aims are concrete, but the means are abstract.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://glazkov.com/2024/02/15/ai-baklava/image-4.620.webp&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;
&lt;p&gt;Each layer is a translation of an abstraction: as we go up in the stack of layers, the means become more abstract and the aims more concrete. At each end, we want to have predictability in what happens next, but the object of our concern is different. Lower layers want to ensure that the means are employed correctly. Higher layers want to ensure that the aims are fulfilled.&lt;/p&gt;
&lt;p&gt;For example, a user who wants to press a button to place an order in the food-ordering app has a very concrete aim in mind for this button – they want to satisfy their hunger. The means of doing that are completely opaque to the user – how this will happen is entirely hidden by the veil of abstraction.&lt;/p&gt;
&lt;p&gt;One layer below, the app has a somewhat more abstract aim (the user pressed a button), but the means are a bit more concrete: it needs to route the tap to the right handler which will then initiate the process of placing the order.&lt;/p&gt;
&lt;p&gt;The aims are even less concrete at a lower layer. The button widget receives a tap and invokes an event handler that corresponds to it. Here, we are unaware of the user’s hunger. We don’t know why they want this button tapped, and nor do we care: we just need to ensure that the right sequence of things transpires when a tap event is received. The means are very concrete.&lt;/p&gt;
&lt;p&gt;A reasonable question might be: why layer at all? Why not just connect the means and aims directly? This is where another interesting part of the story comes in. It appears that we humans have limits to the level of complexity of mental models we can reasonably contain in our minds. These limits are well-known to software engineers.&lt;/p&gt;
&lt;p&gt;For a seasoned engineer, the pull toward layering emerges nearly simultaneously with the first lines of code written. It’s such a habit, they do it almost automatically. The experience that drives this habit is that of painful untangling of spaghetti after the code we wrote begins to resist change. This resistance, this unwillingness of cooperating with its own creator is not the fault of the code. It is the limit of the engineer’s mental capacity to hold the entirety of the software held in their minds.&lt;/p&gt;
&lt;p&gt;When I talk about software with non-technical people, they are nearly always surprised about the amount of bugs any mature software contains. It seems paradoxical that older software has more bugs than the newer code. “What exactly are y’all doing there? Writing bugs?!” Yup. The mental model capacity needed to deeply grok a well-used, well-loved piece of software is typically way beyond that of any individual human.&lt;/p&gt;
&lt;p&gt;So we come up with ways to break software into smaller chunks to allow us to compartmentalize their mental models, to specialize. And because of the way the forces of the aims and the means are facing each other, this process of chunking results in layering. Whether we want it or not, the layering will emerge in our code.&lt;/p&gt;
&lt;p&gt;Putting it another way, layering is the artifact of the limits of human capacity to hold coherent mental models. If we imagine a software engineer with near-infinite mental model capacity, they could likely write well-functioning, relatively bug-free code using few (if any) layers of abstraction.&lt;/p&gt;
&lt;p&gt;The converse is also true: lesser capacity to hold a mental model of traversing from the aims to the means will lead to software with more layers of abstraction.&lt;/p&gt;
&lt;p&gt;By now, you probably know where I am going with this. Let’s see if we can apply these insights to the realm of large language models. What kind of layering will yield better results when we ask LLMs to write software for us?&lt;/p&gt;
&lt;p&gt;It is my guess that the mental model-holding capacity of an LLM is roughly proportional to the size of this model’s context window. It is not the parametric memory that matters here. The parametric memory reflects an LLM’s ability to choose and apply various layers of abstraction. It is the context window that places a hard limit on what a model can and can not coherently perceive holistically.&lt;/p&gt;
&lt;p&gt;Models with smaller context windows will have to rely on thinner layers and be more clever with the abstraction layers they choose. They would have to work harder and need more assistance from their human coworkers. Models with &lt;a href=&quot;https://blog.google/technology/ai/google-gemini-next-generation-model-february-2024/&quot;&gt;larger context windows&lt;/a&gt; would be able to get by with fewer layers.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://glazkov.com/2024/02/15/ai-baklava/image-3.645.webp&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;
&lt;p&gt;How will the LLM-based software engineers compare to human counterparts? Here’s my intuition. LLMs will continue to be abysmally bad at understanding large code bases. There’s just way too many assumptions and tacit knowledge that lurks in those lines of code. We will likely see an industry-wide spirited attempt to solve this problem, and the solution will likely see thinning the abstraction layers within the code base to create safe, limited-scope lanes for synthetic software engineers to be effective in their work.&lt;/p&gt;
&lt;p&gt;At the same time, LLMs will have a definite advantage over humans in learning the codebases that are well within their limits. Unlike humans, they will not get tired and are easily cloned. If I fine-tune a model on my codebase, all I need is the GPU/TPU capacity to scale it to a multitude of synthetic workers.&lt;/p&gt;
&lt;p&gt;Putting these two together, I wonder if we’ll see the emergence of synthetic software engineering as a discipline. This discipline will encompass the best practices for the human software engineer to construct – and maintain – the scaffolding for the &lt;a href=&quot;https://glazkov.com/2022/04/05/cheesecake-and-baklava/&quot;&gt;baklava of layers&lt;/a&gt; of abstraction populated by a hive of their synthetic kin.&lt;/p&gt;
</content></entry>
<entry><title>Reinventing Organizations Redux</title><link rel="alternate" type="text/html" href="https://glazkov.com/2024/02/02/reinventing-organizations-redux/"/><id>tag:www.glazkov.com,2024-02-02:/2024/02/02/reinventing-organizations-redux/</id><published>2024-02-02T20:55:22-08:00</published><updated>2024-02-02T20:55:22-08:00</updated><category term="what-dimitri-learned" label="What Dimitri Learned"/><summary>This one is a bit out there, if only to connect some dots and shake loose new insights. Let’s get that distant look in our eyes and contemplate a possibility that may or may not transpire. Let’s all suppose that…</summary><content type="html">&lt;p&gt;This one is a bit out there, if only to connect some dots and shake loose new insights. Let’s get that distant look in our eyes and contemplate a possibility that may or may not transpire. Let’s all suppose that the &lt;a href=&quot;https://glazkov.com/2023/11/03/why-ai-orchestration/&quot;&gt;upcoming AI winter&lt;/a&gt; is mild, and we settle into the next local maxima of technological progress, surrounded by helpful semi-autonomous agents, powered by large language models. What might that look like?&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://glazkov.com/2024/02/02/reinventing-organizations-redux/image.1600.webp&quot; alt=&quot;A teal butterfly sitting on white keyboard, a tribute to Laloux&amp;#39;s book cover&quot; /&gt;&lt;/p&gt;
&lt;p&gt;I am pretty sure I got the &lt;a href=&quot;https://glazkov.com/2023/05/18/ai-developer-experience-asymptotes/&quot;&gt;firmness of the performance asymptote&lt;/a&gt; wrong last May. The superlinear relationship between quality and cost is here to stay, and will shape a step-ladder-like differentiation of models based on their size. There will be larger models that produce high-quality results for a wide diversity of tasks – and are also expensive to run. There will be smaller models that are much, much cheaper, but also can only excel at a narrow task. We are likely to see attempts to establish a common scale for model complexity, rather than one model to rule them all.&lt;/p&gt;
&lt;p&gt;Given that, we are likely to see more emphasis on the scaffolding that connects the models of varying sizes in addition to the models themselves. For instance, many startups and larger companies are already experimenting with the “inverted matryoshka” scaffolding, where a set of models is arranged so that the smaller, cheaper models are used more frequently for the simpler tasks and the largest models are only progressively reached for more complex (and hopefully, more rare) tasks.&lt;/p&gt;
&lt;p&gt;Sure, there will be projects that try to hide that scaffolding under a “universal model”, which upon examination, will reveal a trenchcoat filled with the assortment of models, pretending to be one.&lt;/p&gt;
&lt;p&gt;However, driven by the desire for agency, most will choose to rely on their access to this scaffolding to get better results. The scaffolding will be the secret sauce of success. The way we arrange the models – and how we choose and train the models for particular tasks – will continue to be the subject of intense experimentation and optimization, even when the pace of model innovation slows down.&lt;/p&gt;
&lt;p&gt;This last sentence holds a startling realization. If we consider that each model is a “knowledge worker” of sorts, we can view the aforementioned scaffolding as an organization. If that’s the case, we can now imagine the process of creating and managing a collection of models as organization development. Except in this organization, the majority of workers are large language models.&lt;/p&gt;
&lt;p&gt;Already, we see academic papers suggesting waterfall-like approaches to tasks, where multiple models (also known as agents) are lined in an assembly line of sorts, passing their output to the next one. I am also seeing experiments with parallel workstreams, converging together to be ranked. Each of the juncture points in these flows is a “virtual knowledge worker”. Perhaps not in the way Frederic Laloux &lt;a href=&quot;https://www.reinventingorganizations.com/&quot;&gt;intended&lt;/a&gt;, we are reinventing organizations.&lt;/p&gt;
&lt;p&gt;It is quite possible (likely?) that organizations we will work in will include both human and non-human workers in them. These organizations will face the same challenges that any organization will face, and likely more new challenges that we haven’t even considered. There will be levels. Simple tasks performed by armies of lower-nomenclature model-powered workers (we’ll probably call them bots). More complex tasks performed by more expensive models. People will likely be supervising, directing, or tuning knowledge work. There might be an entirely new discipline of virtual organization development that emerges as a way of studying and finding more effective ways to conduct organizations that include model-based agents as part of their workforce.&lt;/p&gt;
&lt;p&gt;This may not come to pass. However, what feels right in this picture is that humans will still be there. And because we are unpredictable, volatile humans, who come and go, who change our minds – there will always be a need to maintain a semblance of predictability around the business that owes the organization its existence. And because of that, the relatively more predictable and malleable workers might just serve as the organization development putty: keep adjusting the mixture of non-human workers in the organization to retain its strengths as people leave and join the organization – or change within it.&lt;/p&gt;
&lt;p&gt;Perhaps in this future, we will ask not the question of whether or not AI will replace humans – but rather the question of how non-human knowledge workers can scaffold around us in a way that complements our gifts and gives us space to develop and grow.&lt;/p&gt;
</content></entry>
<entry><title>The Breadboard developer cycle</title><link rel="alternate" type="text/html" href="https://glazkov.com/2024/01/19/the-breadboard-developer-cycle/"/><id>tag:www.glazkov.com,2024-01-19:/2024/01/19/the-breadboard-developer-cycle/</id><published>2024-01-19T16:56:27-08:00</published><updated>2024-01-19T16:56:27-08:00</updated><category term="breadboard" label="Breadboard"/><category term="what-dimitri-learned" label="What Dimitri Learned"/><summary>Over the past couple of months, we’ve been working on a bunch of new things in the Breadboard project, mostly situated around developer ergonomics. I’ve mentioned creating AI recipes a bunch in the past, and it might be a good…</summary><content type="html">&lt;p&gt;Over the past couple of months, we’ve been working on a bunch of new things in the &lt;a href=&quot;https://github.com/breadboard-ai/breadboard&quot;&gt;Breadboard&lt;/a&gt; project, mostly situated around developer ergonomics. I’ve mentioned &lt;a href=&quot;https://glazkov.com/2024/01/11/hourglass-model-and-breadboard/&quot;&gt;creating AI recipes&lt;/a&gt; a bunch in the past, and it might be a good time to talk about the mental model of how such recipes are actually created.&lt;/p&gt;
&lt;h3&gt;&lt;strong&gt;❤️‍🔥 Hot reload&lt;/strong&gt;&lt;/h3&gt;
&lt;p&gt;One of the Breadboard project design choices we’re leaning into is the “hot reload” pattern. “Hot reload” is fairly common in the modern developer UX, and we’d like to bring it into the realm of AI recipes. In this pattern, the developer sees not just the code they write, but also results of this code running, nearly instantaneously. This creates a lightning-fast iteration cycle, enabling the developer to quickly explore multiple choices and see which ones look/work better.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://glazkov.com/2024/01/19/the-breadboard-developer-cycle/image-3.755.webp&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;
&lt;p&gt;Most “hot reload” implementations call for two surfaces that are typically positioned side by side: one for writing the source code, and another for observing the results. In Breadboard today, the first one is typically an editor (I use &lt;a href=&quot;https://code.visualstudio.com/&quot;&gt;VSCode&lt;/a&gt;) and the second one is our nascent Breadboard debugger.&lt;/p&gt;
&lt;p&gt;As the name “hot reload” signifies, when I save the source code file in my editor, the debugger automatically reloads to reflect the recent changes.&lt;/p&gt;
&lt;p&gt;The typical workflow I’ve settled into with Breadboard is that I start writing the board and get it to some point where it has enough to do at least &lt;em&gt;something&lt;/em&gt;, I save the file and play with the board in the debugger. Playing with it both informs me whether I am on the right path and gives me ideas on what to do next.&lt;/p&gt;
&lt;p&gt;I then act on this feedback, either fixing a problem that I am seeing, or progressing forth with one of the ideas that emerged through playing.&lt;/p&gt;
&lt;p&gt;For my long-time readers: yes, we’ve baked the &lt;a href=&quot;https://glazkov.com/2021/11/14/the-ooda-loop/&quot;&gt;OODA loop&lt;/a&gt; right into Breadboard.&lt;/p&gt;
&lt;p&gt;Overall, it’s a pretty fun experience. I get to see the board rising out of code by iteratively playing with it.&lt;/p&gt;
&lt;h3&gt;&lt;strong&gt;🪲 Debugger&lt;/strong&gt;&lt;/h3&gt;
&lt;p&gt;To enable this fun, &lt;a href=&quot;https://aerotwist.com/&quot;&gt;Paul Lewis&lt;/a&gt; has been doing the magician’s work bringing up the debugger. It’s very much a work in progress, though even what’s there now is already useful for board-making.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://glazkov.com/2024/01/19/the-breadboard-developer-cycle/image-4.751.webp&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;
&lt;p&gt;The main purpose of the debugger is to provide both the visualization of the AI recipe that is being developed &lt;em&gt;and&lt;/em&gt; the process of it running. As I put it in chat, I love “seeing it run”: I get to see and understand what is happening during the run — as it happens! – and dig into every bit of detail.&lt;/p&gt;
&lt;p&gt;There’s even a timeline that I can scrub through to better help me understand how events unfolded.&lt;/p&gt;
&lt;p&gt;One thing that gives Breadboard its powers is that it’s built around a very &lt;a href=&quot;https://glazkov.com/2023/08/22/composing-graphs-with-breadboard/&quot;&gt;flexible composition system&lt;/a&gt;. This means that my recipes may reach for other recipes during the run – sometimes a whole bunch of them. Reasoning about that can be rather challenging without help. As we all know, indirection is amazing until we have to spelunk the dependency chains.&lt;/p&gt;
&lt;p&gt;To help alleviate this pain, the Breadboard debugger treats recipe invocations as the front-and-center concept in the UI. The timeline presents them as layers, similar to the layers in photo/video/sound editing tools. I can walk up and down the nested recipes or fold them away to reduce cognitive load.&lt;/p&gt;
&lt;p&gt;There’s so much more work that we will be doing here in the coming weeks and my current description of the Breadboard debugger is likely to become obsolete very quickly. However, one thing will remain the same: the iteration speed is so fast that working with Breadboard is joyful.&lt;/p&gt;
&lt;h3&gt;&lt;strong&gt;🧸 Exploration through play&lt;/strong&gt;&lt;/h3&gt;
&lt;p&gt;This leads me to the key point of what we’re trying to accomplish with the development cycle in Breadboard: exploring is fun and enjoyable when the iterations are fast and easily reversible.&lt;/p&gt;
&lt;p&gt;This is something that is near and dear to my heart.&lt;/p&gt;
&lt;p&gt;When the stakes in exploration are high, we tend to adopt a defensive approach to exploring: we prepare, we train, and brace ourselves when it is the time to venture out into the wilderness. Every such exploration is a risky bet that must be carefully weighted, and uncertainty reduced as much as possible.&lt;/p&gt;
&lt;p&gt;When the stakes of exploration are low, we have a much more playful attitude. We poke and prod, we try this and that, we goof around. We end up going down the paths that we would have never imagined. We end up discovering things that couldn’t have been found by an anxious explorer.&lt;/p&gt;
&lt;p&gt;It is the second kind of exploration that I would love to unlock with Breadboard. Even when learning Breadboard itself, I want our users to have the “just try it” attitude: maybe we don’t know how this or that node or kit works. Maybe we don’t have an answer to what parameters a recipe accepts. So we just try it and see what happens – and get reasonable answers and insights with each try.&lt;/p&gt;
&lt;p&gt;If you’re excited about the ideas I wrote about here, please come &lt;a href=&quot;https://github.com/breadboard-ai/breadboard&quot;&gt;join us&lt;/a&gt;. We’re looking for enthusiastic dance partners who dream of making the exploration of AI frontiers more accessible and enjoyable.&lt;/p&gt;
</content></entry>
<entry><title>Declarative vs. imperative</title><link rel="alternate" type="text/html" href="https://glazkov.com/2024/01/16/declarative-vs-imperative/"/><id>tag:www.glazkov.com,2024-01-16:/2024/01/16/declarative-vs-imperative/</id><published>2024-01-16T20:57:41-08:00</published><updated>2024-01-16T20:57:41-08:00</updated><category term="breadboard" label="Breadboard"/><category term="what-dimitri-learned" label="What Dimitri Learned"/><summary>I recently had a really fun conversation with a colleague about “declarative vs. imperative” programming paradigms, and here’s a somewhat rambling riff that I captured here as a result.</summary><content type="html">&lt;p&gt;I recently had a really fun conversation with a colleague about “declarative vs. imperative” programming paradigms, and here’s a somewhat rambling riff that I captured here as a result.&lt;/p&gt;
&lt;p&gt;When we make a distinction between “declarative” and “imperative” programming, we usually want to emphasize a certain kind of separation between the “what” and the “how”. The “what” is typically the intention of the developer, and the “how” is the various means of accomplishing the “what”.&lt;/p&gt;
&lt;p&gt;For me, this realization came a while back, from a chat with Sebastian Markbåge in the early days of React, where he stated that React is declarative. I was completely disoriented, since to me, it was HTML and CSS that were declarative, and React was firmly in the imperative land.&lt;/p&gt;
&lt;p&gt;It took a bit of flustering and a lot of Sebastian’s patience for me to grok that these terms aren’t not fixed. Rather, they are a matter of perspective. What might seem like “you’re in control of all the details” imperative from one vantage point will look completely declarative from another.&lt;/p&gt;
&lt;h3&gt;&lt;strong&gt;📏 The line between the “what” and the “how”&lt;/strong&gt;&lt;/h3&gt;
&lt;p&gt;Instead of trying to puzzle out whether a given programming language, framework, or paradigm is declarative or imperative, it might be more helpful to identify the line that separates the “what” and the “how”.&lt;/p&gt;
&lt;p&gt;For instance, in CSS, the “what” is the presentation of an HTML document (technically a DOM tree), and the “how” is the method by which this presentation is applied. In CSS, we declare what styling attributes we would like each document element to look like, and leave the job of applying these styles to the document in the capable hands of CSS.&lt;/p&gt;
&lt;p&gt;Similarly, in React, we declare the structure of our components and their relationship to the state, while leaving the actual rendering of the application up to the framework.&lt;/p&gt;
&lt;p&gt;Every abstraction layer brings some “declarativeness” with it, shifting the burden of having to think about some the implementation details from the shoulders of the developer into the layer.&lt;/p&gt;
&lt;p&gt;If we look carefully, we should be able to see the line drawn between the “how” and the “what” in every abstraction layer.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://glazkov.com/2024/01/16/declarative-vs-imperative/image-2.802.webp&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;
&lt;p&gt;In drawing this line, the creators of an abstraction layer – whether they are intentional about it or not – make a value judgment. They decide what is important and must remain in developer’s control, and what is not as important and could be abstracted away. I called this value judgment “&lt;a href=&quot;https://glazkov.com/2022/02/23/the-cost-of-opinion/&quot;&gt;an opinion&lt;/a&gt;” earlier in my writings.&lt;/p&gt;
&lt;p&gt;One way to view such a value judgment is as a bet: it is difficult to know ahead of time whether or not the new abstraction layer will find success among developers. The degree of opinion underlines the risk that the bet entails. More opinionated abstraction layers make riskier bets than less opinionated ones.&lt;/p&gt;
&lt;p&gt;If we measure reward in adoption and long-term usage, then the higher risk bets also promise higher reward: the degree of difference in opinion can serve as a strong differentiator and could prove vital to the popularity of the layer. In other words, if our layer isn’t that different from the layer below, then its perceived value isn’t that great to a developer.&lt;/p&gt;
&lt;p&gt;Therein lies the ancient dynamic that underlies bets (or any value judgments, for that matter): when designing layers of abstraction, we are called to find that balance of &lt;a href=&quot;https://glazkov.com/2022/04/05/cheesecake-and-baklava/&quot;&gt;being different enough, yet not too different&lt;/a&gt; from the layer below.&lt;/p&gt;
&lt;h3&gt;&lt;strong&gt;🪢 The rickety bridge of uncertainty&lt;/strong&gt;&lt;/h3&gt;
&lt;p&gt;While looking for that balance, one of the most significant and crucial exercises that any framework, language, or programming paradigm will undertake is the one of value-trading with their potential developers.&lt;/p&gt;
&lt;p&gt;This exercise can be described by one question: What is it that a developer needs to give up in order to unlock the full value of our framework?&lt;/p&gt;
&lt;p&gt;Very commonly, it’s the degree of control. We ask our potential developers to relinquish access to some lower-layer capabilities, or perhaps some means to influence control flow.&lt;/p&gt;
&lt;p&gt;Sometimes (and usually alongside with lesser control), it’s the initial investment of learning time to fully understand and gain full dexterity of wielding the layer’s opinion.&lt;/p&gt;
&lt;p&gt;Whatever the case, there is typically a rickety bridge of uncertainty between the developer first hearing of our framework and their full-hearted adoption.&lt;/p&gt;
&lt;p&gt;I once had the opportunity to explain CSS to an engineer who spent most of their career drawing pixels in C++, and they were mesmerized by the amount of machinery that styling the Web entails. If all you want to draw is a green box in the middle of the screen, CSS is a massively over-engineered beast. It is a long walk across unevenly placed planks to the point where the full value of this machinery even starts to make sense, value-wise. Even then, we’re constantly doubting ourselves: is this the right opinion? Could there be better ways to capture the same value?&lt;/p&gt;
&lt;p&gt;This bridge of uncertainty is something that every opinionated layer has to cross. Once the network effects take root, the severity of the challenge diminishes significantly. We are socialized species, and the more people adopt the opinion that the abstraction layer espouses, the more normal it becomes – perhaps even becoming the default, for better or worse.&lt;/p&gt;
&lt;h3&gt;&lt;strong&gt;🧰 Bridging techniques&lt;/strong&gt;&lt;/h3&gt;
&lt;p&gt;If we are to build abstraction layers, we are better off learning various ways to make the bridge more robust and short.&lt;/p&gt;
&lt;p&gt;One technique that my colleague &lt;a href=&quot;https://aerotwist.com/&quot;&gt;Paul Lewis&lt;/a&gt; shared with me is the classic “a-ha moment”: structure our introductions and materials in such a way that the potential value is clearly visible as early as possible. Trading is easier when we know what we’re gaining.&lt;/p&gt;
&lt;p&gt;This may look like a killer demo that shows something that nobody else can do (or do easily). It may look like a tutorial that begins with a final product that just begs to be hacked on and looks fun to play with. It could also be a set of samples that elegantly solve problems that developers have.&lt;/p&gt;
&lt;p&gt;Another technique is something that &lt;a href=&quot;https://substack.com/profile/1249413-bernhard-seefeld&quot;&gt;Bernhard Seefeld&lt;/a&gt; is actively experimenting with: intentionally designing the layer in such a way that feels familiar at first glance, but allows cranking up to the next level – incrementally. You can see this work (in progress 🚧) in the &lt;a href=&quot;https://github.com/breadboard-ai/breadboard/blob/main/packages/breadboard/docs/grammar/2-nodes-and-recipes.md&quot;&gt;new syntax for Breadboard&lt;/a&gt;: it looks just like a typical JS code at first, rapidly ramping up to graph serialization, composition, and all the goodies that Breadboard has to offer.&lt;/p&gt;
&lt;p&gt;I am guessing that upon reading these examples, you immediately thought of a few others. Bridging techniques may vary and the technique playbook will keep growing, but one thing that unites them all is that they aim to help developers justify the trade of their usual conveniences for the brave new world of the layer’s opinion.&lt;/p&gt;
&lt;p&gt;Designing new layers is an adventure with the indeterminate outcome. We might be right about our value judgments, or we might be wrong. It could be that no matter how much we believe in our rightness, nobody joins to share our opinion with us. No technique will guarantee the outcome we wish for. And that’s what makes API design and developer experience work in general so fun for me.&lt;/p&gt;
</content></entry>
<entry><title>Hourglass model and Breadboard</title><link rel="alternate" type="text/html" href="https://glazkov.com/2024/01/11/hourglass-model-and-breadboard/"/><id>tag:www.glazkov.com,2024-01-11:/2024/01/11/hourglass-model-and-breadboard/</id><published>2024-01-11T20:33:25-08:00</published><updated>2024-01-11T20:33:25-08:00</updated><category term="breadboard" label="Breadboard"/><category term="what-dimitri-learned" label="What Dimitri Learned"/><summary>Architecturally, Breadboard aspires to have an hourglass shape to its stack. As Bernhard Seefeld points out, this architecture is fairly common in compilers, and other NLP tools, so we borrowed it for Breadboard as well.</summary><content type="html">&lt;p&gt;Architecturally, Breadboard aspires to have an &lt;a href=&quot;https://glazkov.com/2024/01/02/the-protocol-force/&quot;&gt;hourglass shape&lt;/a&gt; to its stack. As &lt;a href=&quot;https://substack.com/profile/1249413-bernhard-seefeld&quot;&gt;Bernhard Seefeld&lt;/a&gt; points out, this architecture is fairly common in compilers, and other NLP tools, so we borrowed it for Breadboard as well.&lt;/p&gt;
&lt;p&gt;In this brief essay, I will use the Breadboard project to expand a bit on the ins and outs of the hourglass-shaped architecture.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://glazkov.com/2024/01/11/hourglass-model-and-breadboard/image-1.694.webp&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;
&lt;p&gt;At the waist of the hourglass, there’s a single entity, a common format or protocol. In Breadboard, it is a JSON-based format that we use to represent any AI recipe (I called them &lt;a href=&quot;https://glazkov.com/2023/08/28/ai-patterns-and-breadboard/&quot;&gt;AI patterns&lt;/a&gt; earlier, but I like the word “recipe” better for its liveliness). When you look through the project, you’ll notice that all recipes are JSON files of the same format.&lt;/p&gt;
&lt;p&gt;The actual syntax of the JSON file may change, but the meaning it captures will stay the same. This is one of the key tenets of designing an hourglass stack: despite any changes around it, the semantics of the waist layer must stay relatively constant.&lt;/p&gt;
&lt;p&gt;The top part of the hourglass is occupied by the recipe producers, or “&lt;a href=&quot;https://en.wikipedia.org/wiki/Compiler#Front_end&quot;&gt;frontends&lt;/a&gt;” in the compiler lingo. Because they all output to the same common format, there can be a great variety of these. For example, we currently have two different syntaxes for writing AI recipes in TypeScript and JavaScript. One can imagine a designer tool that allows creating AI recipes visually.&lt;/p&gt;
&lt;p&gt;Nothing stops someone from building a Python or Go or Kotlin or any other kind of frontend. As long as it generates the common format as its output, it’s part of the Breadboard hourglass stack.&lt;/p&gt;
&lt;p&gt;The bottom part of the stack is where the recipe consumers live. The consumers, or “backends”, are typically runtimes: they take the recipe, expressed in the common format and run it. At this moment in Breadboard, there’s only a Javascript runtime that runs in both Node and Web environments. We hope that the number of runtimes expands. For instance, wouldn’t it be cool to load a Breadboard recipe within a colab? Or maybe run it in C++? Breadboard strives for all of these options to be feasible.&lt;/p&gt;
&lt;p&gt;Runtimes aren’t the only kinds of backends. For instance, there may be an analysis backend, which studies the topography of the recipe and makes some judgments about its integrity or other kinds of properties. What sorts of inputs does this recipe take? What are its outputs? What are the runtime characteristics of this recipe?&lt;/p&gt;
&lt;p&gt;Sometimes, it might be challenging to tell a backend apart from a frontend: an IDE may include both frontends (editor, compiler, etc.) and backends (runtime, debugger, etc.). However, it’s worth drawing this distinction at the system design stage.&lt;/p&gt;
&lt;p&gt;The main benefit of an hourglass stack design is that it leaves a lot of room for experimentation and innovation at the edges of the stack, while providing enough gravitational pull to help the frontends and backends to stick together. Just like for any common language or protocol, the waist of the hourglass serves as a binding agreement between the otherwise diverse consumer and producer ideas. If this language or protocol is able to convey valuable information (which I believe we do with AI recipes in Breadboard), a network effect can emerge under the right conditions: each new producer or consumer creates combinatorial expansion of possibilities – and thus, opportunities for creating new value – within the stack.&lt;/p&gt;
&lt;p&gt;It is far too early to tell whether we’ll be lucky enough to create these conditions for Breadboard, but I am certainly hopeful that the excitement around AI will serve as our tailwind.&lt;/p&gt;
&lt;p&gt;The key risk for any hourglass stack is that it relies on interoperability: the idea that all consumers and producers interpret the common format in exactly the same way. Following the &lt;a href=&quot;https://www.hyrumslaw.com/&quot;&gt;Hyrum’s Law&lt;/a&gt;, any ambiguity in the semantics of the format will eventually result in disagreements across the layers.&lt;/p&gt;
&lt;p&gt;These disagreements are typically trailing indicators of the semantic ambiguities. By the time the bugs are identified and issues are filed, it is often too late to fix the underlying problems. Many languages committees spend eons fighting the early design mistakes of their predecessors.&lt;/p&gt;
&lt;p&gt;As far as I know, the best way to mitigate this risk is two-fold. We must a) have enough experience in system design to notice and address semantic ambiguities early and b) have enough diversity of producers and consumers within the hourglass early in the game.&lt;/p&gt;
&lt;p&gt;To be sure, a system design “spidey sense”, no matter how well-developed, and the ability to thoroughly exercise the stack early don’t offer 100% guarantees. Interoperability tends to be a long-term game. We can flatten the long tail of ambiguities, but we can’t entirely cut it off. Any hourglass-shaped architecture must be prepared to invest a bit of continuous effort into gardening its interoperability, slowly but surely shaving down the semantic ambiguities within the common format.&lt;/p&gt;
</content></entry>
<entry><title>The protocol force</title><link rel="alternate" type="text/html" href="https://glazkov.com/2024/01/02/the-protocol-force/"/><id>tag:www.glazkov.com,2024-01-02:/2024/01/02/the-protocol-force/</id><published>2024-01-02T21:32:41-08:00</published><updated>2024-01-02T21:32:41-08:00</updated><category term="what-dimitri-learned" label="What Dimitri Learned"/><summary>Here’s a riff on the “hourglass model” conversation I’ve had with Bernhard Seefeld a while back. I only recently connected it with my earlier mumblings about software layering, and here’s what came out.</summary><content type="html">&lt;p&gt;Here’s a riff on the “&lt;a href=&quot;https://cacm.acm.org/magazines/2019/7/237714-on-the-hourglass-model/abstract&quot;&gt;hourglass model&lt;/a&gt;” conversation I’ve had with &lt;a href=&quot;https://substack.com/profile/1249413-bernhard-seefeld&quot;&gt;Bernhard Seefeld&lt;/a&gt; a while back. I only recently connected it with my &lt;a href=&quot;https://glazkov.com/?s=layers&quot;&gt;earlier mumblings&lt;/a&gt; about software layering, and here’s what came out.&lt;/p&gt;
&lt;p&gt;If a technology stack is arranged in such a way that there are consumers of value on one end of the stack and producers of it on the other, then at a certain scale, there emerges a force that encourages one middle layer of the stack – the waist – to be as thin as possible. There will be a distinct preference for lower diversity of offerings at that layer. IP, HTTP, and HTML are all great examples of this force’s effects.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://glazkov.com/2024/01/02/the-protocol-force/image.1300.webp&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;
&lt;p&gt;I am going to call this force the “protocol force”, since the outcome of this force is usually a common protocol or format that is used for communicating across that layer.&lt;/p&gt;
&lt;p&gt;To sketch out the mechanics behind the protocol force, the upper layers want to have access to more consumers, and those are only accessible through the lower layers. The lower layers want to get more producers, and only the upper layers can provide those. To get either, the layers have to go through the waist, and they all want to minimize the cost of bridging, of implementing all the permutations that enable consumers and producers to interact with each other. Put differently, everyone wants to spend as little as possible to reach as many consumers/producers as possible.&lt;/p&gt;
&lt;p&gt;For instance, if we have multiple interaction protocols available in the waist layer, a growth spurt of an ecosystem of producers and consumers around this layer will trigger the “winner takes all” dynamics: a protocol that gets some critical mass of consumers (or producers) will reduce the appeal of using other protocols. As these other protocols will fall into disuse, the one that “won” will continue to gain new adopters – thus thinning the waist layer.&lt;/p&gt;
&lt;p&gt;Interestingly, this protocol force may not manifest in the adjacent layers, even if they look like the connectors between the upper and the lower layers. As long as there’s one layer that is thin, the adjacent layers may enjoy high diversity of offerings.&lt;/p&gt;
&lt;p&gt;A good example of this is the Web frameworks. There are always so many of them, right? Why is that? My explanation of this phenomenon, applying the reasoning above, is that they are protected from the effect of the protocol force by the Web Platform layer (the HTML/CSS/JS combo that we all love and enjoy) that sits right underneath them. This layer is incredibly thin: thanks to the set of circumstances and sheer designers’ luck, we only have one Web Platform.&lt;/p&gt;
&lt;p&gt;Because of that, there can be a cornucopia of Web frameworks: as long as what they output is Web Platform-compatible, they won’t experience the pressure to agglomerate or race to the bottom.&lt;/p&gt;
&lt;p&gt;The protocol force will be present in any layered stack that has producers and consumers as outer layers. One of the bridging layers will succumb to it. If we’re designing a layered system, we are better off preparing for this force to tap us on the shoulder and ask us to pick a layer to squish. Any resistance is futile. Might as well plan for it.&lt;/p&gt;
&lt;p&gt;One of my favorite recent examples of such a design is the &lt;a href=&quot;https://protobuf.dev/&quot;&gt;Protocol Buffers&lt;/a&gt;, a framework that Google uses internally to connect disparate pieces of functionality into one coherent whole. As long as a program speaks protos, it can mingle in Google-land. Whether intentionally or not, protocol buffers serve as the waist layer of Google infrastructure.&lt;/p&gt;
&lt;p&gt;One of the key challenges of designing something as versatile as the Web Platform or protocol buffers is the fact that this layer will be the most used and, sadly. abused part of our stack. Once squished into a thin waist, this layer becomes the great enabler of our future – or the source of our doom. Its flexibility and capacity to convey the intentions of developers will be constantly tested and stretched and often broken. This is a sign of success and a good thing. As long as we’re prepared to invest into continuously improving and developing the waist of our stack, we can harness the protocol force to help us, rather than hinder.&lt;/p&gt;
</content></entry>
<entry><title>Why AI orchestration</title><link rel="alternate" type="text/html" href="https://glazkov.com/2023/11/03/why-ai-orchestration/"/><id>tag:www.glazkov.com,2023-11-03:/2023/11/03/why-ai-orchestration/</id><published>2023-11-03T13:45:12-07:00</published><updated>2023-11-03T13:45:12-07:00</updated><category term="breadboard" label="Breadboard"/><category term="what-dimitri-learned" label="What Dimitri Learned"/><summary>Why do I find the problem of AI patterns and more generally, AI orchestration so interesting that I literally started building a framework for it? Why do we even need graphs and chains in this whole AI thing? My colleagues…</summary><content type="html">&lt;p&gt;Why do I find the problem of AI patterns and more generally, AI orchestration so interesting that I literally started &lt;a href=&quot;https://glazkov.com/2023/08/22/composing-graphs-with-breadboard/&quot;&gt;building a framework&lt;/a&gt; for it? Why do we even need graphs and chains in this whole AI thing? My colleagues with a traditional software engineering background have been asking me this question a lot lately.&lt;/p&gt;
&lt;p&gt;Put very briefly, &lt;em&gt;at the height of the current AI spring that we’re experiencing, orchestration is a crucial tool for getting AI applications to the shipping point&lt;/em&gt;.&lt;/p&gt;
&lt;p&gt;To elaborate, imagine that an idea for a software application takes a journey from inception to full realization through these two gates.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://glazkov.com/2023/11/03/why-ai-orchestration/image.649.webp&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;
&lt;p&gt;First, it needs to pass the “hey… this might just work” gate. Let’s call this gate the “Once” gate, since it’s exactly how many times we need to see our prototype work to get through it.&lt;/p&gt;
&lt;p&gt;Then, it needs to pass through the “okay, this works reasonably consistently” gate. We’ll call it the “Mostly” gate to reflect the confidence we have in the prototype’s ability to work. It might be missing some features, lack in polish and underwhelm in performance benchmarks, but it is something we can give to a small group of trusted users to play with and not be completely embarrassed.&lt;/p&gt;
&lt;p&gt;Beyond these two gates, there’s some shipping point, where the prototype – now a fully-fledged user experience – passes our bar for shipping quality and we finally release it to our users.&lt;/p&gt;
&lt;p&gt;A mistake that many traditional software developers, their managers, and sponsors/investors make is that, when looking at AI-based applications, they presume the typical cadence of passing through these gates.&lt;/p&gt;
&lt;p&gt;Let’s first sketch out this traditional software development cadence as a sequence below.&lt;/p&gt;
&lt;p&gt;The “Once” gate plays a significant role, since it requires finding and coding up the first realization of the idea. In traditional software development, passing this gate means that there exists a kernel of a shipping product, albeit still in dire need of growing and nurturing.&lt;/p&gt;
&lt;p&gt;The trip to the “Mostly” gate represents this process of maturing the prototype. It is typically less about ideation and mostly converging on the robust implementation of the idea. There may be some circuitous detours that await us, but more often than not, it’s about climbing the hill.&lt;/p&gt;
&lt;p&gt;In traditional software development, this part of the journey is a matter of technical excellence and resilience. It requires discipline and often requires a certain kind of organizing skill. On more than one occasion, I’ve seen brilliant program managers brought in, who then help the team march toward their target with proper processes, burndown lists, and schedules. We grit our teeth and persevere, and are eventually rewarded with software that passes the shipping bar.&lt;/p&gt;
&lt;p&gt;There’s still a lot of work to be done past that gate, like polish and further optimization. This is important work, but I will elide it from this story for brevity.&lt;/p&gt;
&lt;p&gt;In AI applications, or at least mine and my friends/colleagues’ experiences with it, this story looks startlingly different. And definitely doesn’t fit into a neat sequential framing.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://glazkov.com/2023/11/03/why-ai-orchestration/image-1.634.webp&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;
&lt;p&gt;Passing the “Once” gate is often a matter of an evening project. Our colleagues wake up to a screencast of a thing that shouldn’t be possible, but somehow is. Everyone is thrilled and excited. Their traditional software developer instincts kick in: a joyful “let’s wrap this up and ship it!” is heard through the halls of the office.&lt;/p&gt;
&lt;p&gt;Unfortunately, when we try to deviate even a little from the steps in the original screencast, we get perplexing and unsatisfying results. Uh oh.&lt;/p&gt;
&lt;p&gt;We try boxing the squishy, weird nature of large language models into the production software constraints. We spend a lot of time playing with prompts, chaining them, tuning models, quantizing, chunking, augmenting – it all starts to feel like alchemy at some point. Spells, chants, and incantations. Maaaybe – maybe – we get to coax a model to do what we want more frequently.&lt;/p&gt;
&lt;p&gt;One of my colleagues calls it the “70% problem” – no matter how much we try, we can’t seem to get past our application producing consistent results more than 70% of the time. Even by generous software quality standards, that’s not “Mostly”.&lt;/p&gt;
&lt;p&gt;Getting to that next gate has little resemblance to the maturation process from traditional software development. Instead, it looks a lot more like the looping over and over back to “Once”, where we rework the original idea entirely and change nearly everything.&lt;/p&gt;
&lt;p&gt;When working with AI applications, this capacity to rearrange everything and stay loose about the details of the thing we build, this design flexibility is what dramatically increases our chances of crossing to “Mostly” gate.&lt;/p&gt;
&lt;p&gt;Teams that hinge their success on adhering to the demo they sold to pass through the “Once” gate are much more likely to never see the next gate. Teams that decide that they can just lay down some code and improve iteratively – as traditional software engineering practices would suggest – are the ones who will likely work themselves into a gnarly spaghetti corner. At least today, for many cases – no matter how exciting and tantalizing, the “70% problem” remains an impassable barrier. We are much better off relying on an orchestration framework to give us the space to change our approach and keep experimenting.&lt;/p&gt;
&lt;p&gt;This is a temporary state and it is not a novel phenomenon in technological innovation. Every new cycle of innovation goes through this. Every hype cycle eventually leads to the plateau of productivity, where traditional software development rules.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://glazkov.com/2023/11/03/why-ai-orchestration/image-2.694.webp&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;
&lt;p&gt;However, we are not at that plateau yet. My intuition is that we’re still climbing the slope toward the peak of inflated expectations. In such an environment, most of us will run into the “70% problem” barrier head-first. So, if you’re planning to build with large language models, be prepared to change everything many times over. Choose a robust orchestration framework to make that possible.&lt;/p&gt;
</content></entry>
<entry><title>Makers and Magicians</title><link rel="alternate" type="text/html" href="https://glazkov.com/2023/10/23/makers-and-magicians/"/><id>tag:www.glazkov.com,2023-10-23:/2023/10/23/makers-and-magicians/</id><published>2023-10-23T21:52:56-07:00</published><updated>2023-10-23T21:52:56-07:00</updated><category term="dx-pattern" label="DX Pattern"/><category term="what-dimitri-learned" label="What Dimitri Learned"/><summary>I want to finally connect two threads of the story I’ve been slowly building across several posts. I’ve talked about the rise of makers. I’ve talked about the magicians. It’s time to bring them together and see how they relate…</summary><content type="html">&lt;p&gt;I want to finally connect two threads of the story I’ve been slowly building across several posts. I’ve talked about the &lt;a href=&quot;https://glazkov.com/2023/07/23/the-rise-of-makers/&quot;&gt;rise of makers&lt;/a&gt;. I’ve talked about &lt;a href=&quot;https://glazkov.com/2023/10/12/doers-thinkers-and-magicians/&quot;&gt;the magicians&lt;/a&gt;. It’s time to bring them together and see how they relate to each other.&lt;/p&gt;
&lt;p&gt;First, let’s paint the picture a little bit and set up the narrative.&lt;/p&gt;
&lt;p&gt;The environment is ripe for disruption: there’s a new software capability and a nascent interface for it, and there’s a whole lot of commotion going on at all &lt;a href=&quot;https://glazkov.com/2023/05/29/four-layers/&quot;&gt;four layers&lt;/a&gt; of the stack. Everyone is seeing the potential, and is striving to glimpse the true shape of the opportunity, the one that brings the elusive product-market fit into clarity.&lt;/p&gt;
&lt;p&gt;As I asserted before, there’s a brief moment when this opportunity is up for grabs, and the ground is more level than it’s ever been. Larger companies, despite having more resources, struggle to search for the coveted shape quickly due to the &lt;a href=&quot;https://glazkov.com/2023/02/18/strategy-aperture/&quot;&gt;law of tightening aperture&lt;/a&gt;. Smaller startups and hobbyists can move a lot faster – albeit with high ergodic costs – and are able to cover more ground en masse. Add the combinatorial power of social networks and cozywebs, and it is significantly more likely that one of them will strike gold first.&lt;/p&gt;
&lt;p&gt;For any larger player with strategic foresight, the name of the game is to “be there when it happens”. It might be tempting to try and out innovate the smaller players, but more often than not, that proves to be hubris.&lt;/p&gt;
&lt;p&gt;Instead of trying to be the lucky person in the room, it is more effective to &lt;em&gt;be the room&lt;/em&gt; that has the most exceptionally lucky person in it – and boost their luck as much as possible.&lt;/p&gt;
&lt;p&gt;When the disruption does finally occur and the hockey stick of growth streaks upward, such a stance reduces the chances of &lt;a href=&quot;https://marginalfutility.substack.com/p/13-counter-positioning-as-a-business&quot;&gt;counter positioning&lt;/a&gt; and improves the larger player’s ability to quickly learn from the said lucky person.&lt;/p&gt;
&lt;p&gt;Put simply, during such times of rapid innovation, the task of attracting “exceptionally lucky people” to their developer ecosystems becomes dramatically more important for larger companies.&lt;/p&gt;
&lt;p&gt;If the story indeed is playing out like so, then the notion of magicians is useful to identify those “exceptionally lucky people” – because luck compounds for those who explore the space in a way that magicians do.&lt;/p&gt;
&lt;p&gt;But where do makers fit in? A good way to think of it as overlapping circles of two groups: developers and makers.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://glazkov.com/2023/10/23/makers-and-magicians/image-6.1128.webp&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;
&lt;p&gt;We’ll define the first circle as people who develop software, whether professionally or as a hobby. Developers, by definition, use &lt;a href=&quot;https://glazkov.com/2021/10/18/developer-surface/&quot;&gt;developer surfaces&lt;/a&gt;: APIs, libraries, juts, tools, docs, and all those bits and bobs that go into making software.&lt;/p&gt;
&lt;p&gt;The second circle is broader, because it includes folks who both develop and interact with software in a way that creates something they care about. Makers and developers obviously overlap. And since “maker” is a mindset, the boundary between makers and developers is porous: I could be a developer during the day and a maker at night. At the same time, not all developers are makers. Sometimes, it’s really just a job.&lt;/p&gt;
&lt;p&gt;Makers who aren’t developers tend to gravitate toward becoming developers over time. My intuition is that the more engaged they become with the project, the more they find the need to make software, rather than just use it. However, the boundary that separates them from developers acts as a skill barrier. Becoming a developer can be a rather tough challenge, given the complexity of modern software.&lt;/p&gt;
&lt;p&gt;Within these two circles, early adopters make up a small contingent that is weighted a bit toward makers. Based on how I defined &lt;a href=&quot;https://glazkov.com/2023/09/25/traits-of-a-maker-mindset/&quot;&gt;maker traits&lt;/a&gt; earlier, it seems logical that early adopters will be primarily populated by them.&lt;/p&gt;
&lt;p&gt;A tiny slice of the early adopter bubble on the diagram is magicians. They are more likely to be in the developer circle than not, since they typically have more expertise and skill to do their magic. However, there are likely some magicians hiding among non-developer makers, prevented by the learning curve barrier from letting their magic shine.&lt;/p&gt;
&lt;p&gt;I hope this diagram acts as a navigational aid for you in your search for “exceptionally lucky” people – and I hope you make a room for them that feels inviting and fun to inhabit.&lt;/p&gt;
</content></entry>
</feed>
