The expectation gradient

The conflation of “what is” and “what should be” is not the only way in which our intentions impact our prediction error rate. Another source is the intentions that we’re unaware of. To better understand what happens, we are going on another side adventure. And yes, we might even get to cast trigonometry spells again. But first, let’s talk about expectation gradients.

If we view prediction error rate as a measure of the accuracy of our predictions after the fact, expectation gradient is our forecasting metric. An easy way to grok it is to visualize ourselves standing on a trail and looking ahead, trying to guess the gradient of the incline. Is there a hill up ahead, or is it nice and flat? Or perhaps a wall that we can’t scale? The gradient of the path ahead foretells us of the effort we’ll need to put into moving forward.

In a similar vein, the expectation gradient reflects our sense of the difference between our models of “what is” and “what should be.” It is our estimate of the steering effort: how much energy we will need to invest to turn “what is” into “what should be.” A gentle slope of the gradient reflects low estimated effort, and as the estimate grows, the slope becomes steeper. If I find myself in a forest, facing a hungry tiger, I am experiencing a very steep gradient. Sitting in a comfortable chair while sipping eggnog (it is that time of the season!) contentedly and writing, however — that’s the definition of a gentle gradient slope for me.

With our trig hat on, we can picture the expectation gradient as the angle of a triangle. The adjacent side is the distance between “what is” and “what should be” (or a fraction thereof), and the opposite side is the measure of the required energy that we need to muster to steer the environment from “what is” to “what should be.”

The opposite-adjacent relationship to the angle is a tangent. When we deal with tangents, we face impossibilities. There is an asymptote, built into that little arrangement. The wavy tangent line starts slow, but then zooms into the sky, never ever quite fulfilling the promise of meeting required output.

I quite like this framing, because it feels pretty intuitive. The curve practically begs to be broken down into three distinct sections: the section before the kink where we’re reasonably certain that we can achieve our goal, the middle section where we are are uncertain of the outcome, and the asymptote – the section in which we’re pretty certain that our goal is unachievable.

Looking at “dancing with delusion” from the previous piece through the lens of expectation gradient, it’s all about convincing the team that the road ahead is mostly out of the third section, stretching the “uncertain” a bit longer.