Call Me Gearhead

So, I am playing with Google Gears and I’ve got to tell ya, Gears are brilliant.
And the
Why? Because Gears fulfill the ultimate fantasy of any Web developer in a very
radical way: they cut into the browser (IE and Firefox, currently), deep and
wide, introducing themselves inconspicuously as a DOM facility
(google.gears.factory – hah, clever fellers, aren’t they?). And they bring…
well, I think they bring a bit more clarity into this whole murky future Web
thing. Oh noes, did Dimitri finally cross the line from a grouchy nerd to the
full-blown pundit and started predicting the future? Well, let’s hope not. To
the very least, color me intrigued.

If you’re not familiar with Gears, this blog is probably not the place to learn
it, but basically, it breaks down to three things: local storage
management, isolation/threading model, and cross-origin data and resource
access. Yeah, see? It’s not even the same breakdown as
their site
. But that’s why it’s here on my blog: I can skim over the details
and get to the good stuff. And good stuff it is. I am surprised that the
reaction to Gears is so muted, because they attack the status quo of Web
application (heck, any Web site) development with a deft balestra that
rivals pretty much anything I’ve seen on the market since, well, since
XMLHttpRequest (only took 5 years to «discover» that one, eh?). Here are just a
couple of exciting possibilities that come to mind after playing with the
0.2 code.

Client-side Composition

Up until now, whether we are eager to admit it or not, DHTML applications (and I
am intentionally classifying
out of the picture here) had this distinct flavor of a dumb terminal
— it’s catchy. Besides, I still have a 3812 in my basement. Makes
excellent ballast). The server had to pretty much “print” down the wire the
entire snapshot of a page in a tasty soup of HTML. And that’s on every request.

Google pushed the envelope on client-side composition with GMail and some people
took notice, but by and large, server-side composition reigns supreme. Come to a
page on the Web, and for each ounce of content there is a bucket of context:
navigation, branding elements, context-sensitive link lists, spotlights,
testimonials, you name it — all repeating from page to page, racking up
bandwidth, using up server cycles needed for complex frameworks to sift, sort,
transform and align resources into the darned HTML snapshot. And that’s on
every request
, my teary-eyed readers. I would join you in your sorrow for
all this wasted energy, but I must finish this song… er, post.

See, with server-side composition, it is up to the server to determine the
context of the requested page and generate markup that puts those links and
other context elements together with the content of the page. Let’s isolate this
effort into a separate functional component, and call it the context
engine. To summarize, content engine retrieves content of the page, context
engine evaluates content and mashes it with whatever seems relevant, and hands
it off to be served to the user agent. Funny fact: most modern content
management systems are in fact context management systems. Content engine
is dumb and simple (fetch a page, duh). Context engine is complex, nontrivial,
and a gargantuan resource hog.

With Gears, you can finally have proper client-side storage (not the
cookie-based monster), and thus you can have proper client-side state. And thus
you can implement proper client-side composition. Which means that aside from
the obvious “offlining” of frequently-used, but rarely-changed assets
(static pages, scripts, images, stylesheets) using
you can actually move the context engine to the client-side… Hey buddy, an
example wouldn’t hurt, m’kay?

Let’s suppose that on your site, most pages have a sidebar, displaying the list
of upcoming events, relevant to this page. With server-side context page, each
page arrives as the blob of markup, generated by the context engine. With
client-side context engine, the page only contains:

  • content and optionally, a list of keywords (tags) that describe the
    meaning of content
  • URL to the event feed

Upon loading of the page, the client-side context engine kicks in:

  1. It checks to see if local event repository exists, and if it doesn’t,
    creates one by fetching all events from the provided event feed URL.
  2. It evaluates content/tags against local content repository of events and
    adds relevant event to the sidebar
  3. Makes “get added/updated since” requests to the event feed and updates
    content repository, as well as the sidebar
  4. Removes older events from the repository

When this user goes to the next page, the server doesn’t do anything but serve
content. For the duration of the session (or perhaps some pre-defined interval),
the client-side context engine provides relevant events from its local
repository. The happy, unburdened server sends sloppy kisses to Gears. No
wonder, because now, the only task that requires any meaningful computation is
serving the list of events added or changed since specified date or revision
, this one’s for you.

Decoupling context engine from the server not only makes server’s work easier.
It also makes context engine server-independent. Who says that I should only
pull events from this one server? Why not pull them from that popular news
outlet, online local events site, or Google calendar?
here we come!

Even cooler, the client-side context engine is far better suited to keep the
track of user browsing habits, generating personal taxonomy or tag cloud, and
taking it into the account when evaluating relevancy. Am I the only one who gets
goosebumps thinking about the opportunities?

Worker as Service

While playing with Gears’
bits, I realized that cross-origin
introduce a better, more modular and more secure way of building public
JavaScript APIs. In this new release of Gears (developer-only, for now), the
worker can be loaded from a URL, and this URL does not have to originate on the
same server as the document, in which the worker is created. In essence, you can
load and run a script from another server in a completely isolated context,
and you can exchange messages with this script. And this script
can make HttpRequest calls back to that server. It only takes a small
logical step to see that this script can expose the public API of a Web
application, located on that server, via WorkerPool messaging.

Let’s pretend that I have a Web site that wants to use Google Spreadsheet as
table, listing some goods for sale. Here’s how I would connect to the API using
JavaScript (wildly pseudocoding):

	var wp = google.gears.factory.create("beta.workerpool", "1.1");
	// create message
	var getRows = new Message("getRows");
	// ... perhaps create more messages, with parameters or not
	// set up message handler
	wp.onmessage = function(text, id) {
		var message = Message.fromJSONString(text);
		// API initiates communication
		if (message.command == "ready") {
		// ... more message processing
	// finally, kick-start the whole thing by loading the API
	// URL is fictional, of course
	var api = wp.createWorkerFromUrl("");

Yeah, I am skipping lots of details, but I hope the concept is obvious: the
Spreadsheet API handler is loaded as a worker, and the page can then use this
API by exchanging a documented set of messages. No need to knit
double-frames or server-side proxies. It just works.

Because the worker is isolated, we can set up more secure authentication and
increase authentication granularity by accepting only certain messages,
depending on the identity. Also, worker runs as a separate thread, which means
we can do other things while the data is cooking. If it were up to me, I’d be
staying up all night converting all Google API endpoints to this model and
developing a good message exchange protocol. Though I might be getting too old
to stay up all night without dire consequences.

Worker as Module

A reverse of worker-as-service model is worker as module. In this case,
the application accepts cross-origin worker registration via UI, allowing them
to participate via message exchange. For example, Google Reader could
allow users to add plug-ins by allowing the users to enter the URL of the API.
The URL could be an HTML document with some simple markup, referencing the
JavaScript file, containing the plug-in. Easy-peasy. And beautiful.

Thinking Outside of The <object> Box

Finally, let’s pause for a second to ponder the way Gears is implemented.
Instead of building their own
inside of an object node, Gears hook up directly in DOM, without creating a
runtime and certainly not a new
format. They organically extend HTML DOM space and DHTML
developer’s horizon. What’s more, the Gearites
to making (and
) a good effort of helping bring this extension into the new HTML spec. And I
like this thinking.

Margin Marks UI Concept


Margin marks is a user interface concept that aims to expose microformats on a web page in a way that’s intuitive, useful, and positionally relevant, yet has minimal interaction with the page presentation. This concept can be also extended to emphasize other, typically invisible aspects of the content, such as fragment identifiers, classes and even to letting the users add their own marks. You can go ahead and just look at the pictures if you don’t feel like reading.


I’ve been following the thought process of microformats UI in Firefox 3 as documented in Operator‘s
functionality, Alex‘s blog, and uf-discuss list. It’s been exciting to think about the power of microformats and its consumption potential being built into the browser, and as such the decisions about the user interface exposing this power are certainly quite heavy in weight. The greatest problem as it appeared to me was exposing content, marked up with microformats in a way that does not interfere with the page presentation, while at the same time providing comfortable and immediately useful experience for the users. Mike’s current experiment, the Operator, has cool ideas and lots of configurable options, but it still left me wanting something more. Primarily, my holy grail was positional relevance of the consumer user interface to the actual marked-up content. Looking at pages like and my own blog comments, I realized that a page with a lot of microformatted content practically begs for positional correlation between the Operator’s
action drop-downs and the page itself. That’s how the margin marks came along.

General Concept

The margin bar is a vertical pane that is shown on one side of the browser window. Whether it’s on the left or on the right may be configurable by the user. The contents of the margin bar are vertically attached to the page, so
that when the page contents are scrolled, the margin bar contents are scrolled as well. Visually, it’s an extra margin to the page that is controlled by the browser, not page presentation (hence, the margin bar). The margin bar can be visible or hidden, as desired by the user. Naturally, open should be the default state.

The margin bar is narrow, with minimal impact on the width of the browser window. The information provided is hint-like, abbreviated down to icons and perhaps numeric indicators. Visually, it’s a set of glyphs, each positioned alongside the start of relevant content fragment. These glyphs are margin marks. Margin mark identifies vertical position of a content fragment in the margin bar. The mark can be visually presented as an arrow or any other sort of pointer with an icon on it.

Grouping Marks

In situations, when there are more than one marks occupying the same space, the marks are combined into one mark, visually identifying multiple items, together with the number of combined marks. The icon, associated with the top-most mark is displayed.

Mark Actions

Each mark may have one or more actions, associated with it, with one action designated as default. Configuring the actions is part of the browser preferences UI. It is possible that the action may have an icon associated with
it. For instance, if the action is to add event to Microsoft Outlook calendar, the Outlook icon is displayed in the mark, rather than a generic address card. However, this may introduce more confusion, given the diversity of platforms and applications that may be potentially invoked by the users.

Mouse Navigation

When the user hovers the mouse over the mark, the details window is revealed. Moving the mouse off the mark closes the details window. Clicking on the mark invokes the default action. Visually, default action is placed at the top of the details window, so hovering and clicking are intuitively connected: the user does not need to make any further mouse movements to invoke the default action. Hovering the mouse over a group opens the group: the marks in the group are lined up in the bar vertically, allowing the user to explore the marks within the group. Admittedly, this is not very elegant. Perhaps you could come up with a better idea.

Keyboard Navigation

Margin bar participates in the browser chrome tab cycle, preferably placed immediately before the page. Also, there should be a keyboard shortcut to bring keyboard focus into the margin bar. Once the bar acquires keyboard focus, the top-most mark gains it automatically. Then, the following keyboard events are recognized (this list is just a suggestion and food for thought):

  • Down Arrow — move to next mark
  • Shift-Down Arrow — move to next mark within the group. If at the end of the group, move to next mark
  • Up Arrow — move to previous mark
  • Shift-Up Arrow — move to previous mark within the group. If at the beginning of the group, move to previous mark
  • Space — scroll the page down and jump to the first mark in the visible span of the page
  • Enter — invoke mark/note action
  • Tab — go to the browser window
  • Shift-Tab — go to the previous item in the tab cycle

Aural Presentation

Ideally, when used with a browser that is equipped with voice-reading software,
such as JAWS, the user interaction should occur as follows:

  • When the margin bar gains focus, the reader announces: X marks on the page. Mark One. Type: Address Card. Name: Rulon Oboev… and continues reading the mark contents
  • Using arrows, the user can move between the marks. Upon each move, the reader announces the sequential number of the mark, it’s type and contents.
  • After reading contents, the reader announces each action as a link.
  • In addition to standard actions, the “Go to content on page” action is added after the default action.

Microformats Marks

Whenever microformat markup is encountered on page, a mark is placed on the bar at the current vertical position of the starting element of the markup fragment. Should the position change as a result of DOM operation or changing geometry of the page, the mark changes the position accordingly. This may be difficult to implement, so an acceptable solution would be to detect detachment (position change) and somehow change the appearance of the mark to no longer “point” to a place in content. Each mark contains a distinctive icon of the corresponding microformat (address card icon for an hCard, calendar icon for hCalendar event, etc.).

When hovered over the microformat data is presented as a complete note, perhaps using a metaphor, relevant to the specific microformat. For example, the hCard could be rendered as a Rolodex card, and an hAtom entry would be probably best presented as a yellow-pad note, a common visual hint of blog post.

Other Types of Marks

One can also easily extrapolate the use of the margin mark to other types of page metadata. For instance, a mark with a feed icon may be placed whenever a feed is encountered on the page. Usually, these would be at the top, but should there be an a element with the type attribute of application/rss+xml, the mark would be placed accordingly there, too.

Also, the marks could be used to provide a UI to unobtrusively identify HTML elements with an id attribute (HTML fragments). Other uses may include tracking a set of user-specified elements, attribute values, or content (mark everything containing “microformats” on the page).

User Marks

It would be really interesting to offer the users to add their own marks to the page, perhaps by clicking (or right-clicking) on the bar, as a way to annotate the page. As the users add a new mark, they can fill in the fields in the provided dialog box. Typically, this would be a simple note (an hAtom entry), but one can envision adding reminders (an hCalendar event), contact information (an hCard), perhaps re-purposing non-microformatted content from the page), or other types of content. After the mark is added, it is persisted within the browser.

Persistently and reliably identifying is a potential challenge of user mark implementation. Since it is not known when or how the content of the page will change upon next visit, a visual equivalent of um.. somewhere around here may be applied: if the browser can not identify the precise location of the user mark, an extra hint (a question mark, maybe, or a spatial glow/spread to signify uncertainty in position) is added to the mark. When this hint is present, the point line is not displayed.

Other Random Thoughts

Taking one step further brings us to the ability of the browser to communicate with the server when new user marks are added or deleted. Using some simple detection scheme, a browser could recognize that the page accepts mark updates and send newly added marks to the server transparently. An existing blog comment API with some positional extensions could be used or a new protocol could be proposed. I’ll let you figure out what would be best here.

When the margin mark is hovered over or has focus, an additional visual hint could be introduced: a point on the page where the relevant content begins and a horizontal line, connecting it with the mark, like a laser pointer. This could really address the issues of positional relevance.

When the page has more microformatted content beyond the current scroll view, a teaser hint could be shown at the bottom or top of the margin bar (an arrow of some sort?) to indicate that there’s more crunchy markup above or bellow the currenty visible portion of the page.

The margin bar could also have an expanded state, in which it shows details along with the marks. I originally had this in the concept, but I instinctively felt it makes the whole thing too complicated.

Inspiration, Disclaimer, and Licensing

This concept is inspired by the entire super-awesome premise of microformats and the great people around them, by the Alex Faaborg‘s
post on Firefox 3 microformat UI concepts, Mike
‘s ground-breaking Operator extension, and quite obviously, Jack Slocum‘s blog comment system.

I am not a browser developer and honestly do not know how much effort would it take to implement something like this. I did take a brief stroll in a Mozilla trunk and soon realized that one cannot evaluate implementation feasibility by just taking a brief stroll through the code of a browser. I am positive this can be done completely in Javascript, and thus assume that the feasibility is pretty high.

Should anyone find this concept, in full or in parts useful, inspiring, and/or worthy of implementation, I release it as public domain. I think that it would be awfully splendid of you to mention my name, even if somewhere deep in the comments of your shiny new toy. Or maybe bake me a low-carb cake. Or a MacBook Pro. But I won’t insist.

The Horrific Markup of Live Spaces and Possible Explanation of Dare not Getting Microformats

It’s a Friday surprise! After reading this post by Dare Obasanjo,
I dutifully followed the links in the article. Upon stumbling on his Live Spaces friends page, I instinctively hit Ctrl+U to peek at the source code. Ow! Ow! My eyes! My tired bespectacled eyes!

Dear readers (yes, all 2 of you — Privet, Mom!), let’s all hold hands and stand in a circle. Let’s promise ourselves to never look at that pitiful, congealed elephant-man, malignant-growth of a code ever again. It’s just better that way. And you, Spaces developers… Well, shame on you. You should know better than dumping this crap on the Web. No wonder poor Dare can’t get screen-scraping out of his mind: he probably hasn’t even seen good semantic markup, much less realized its benefits. Have you?

Oh, and Dare, bad call on the friends.get example. You should probably use fql.query to get something more useful than a list of friend UIDs for any sort of social network portability. But I’ll leave you alone with your point of view on what can and cannot be an API.

Framework for Tomorrow’s Web

I am trying something new here. Instead of blabbering and shaking wrinkled fists like a cranky old man, I’ll turn my lemon into something with a bit more kinetic oomph. Like a two-by-four. Kidding! Kidding…

If you’re like me, you’re fascinated with Web frameworks. You’ve dabbled in all of them, finding likes and quirks, but ultimately walking away wanting more. If you’re like me, you’ve kept a mental wishlist of what tomorrow’s Web framework should be like. Here’s mine, in no particular order:

  • Framework for Tomorrow’s Web (FTW for short) Speaks Web natively. No extra work is needed by a developer to take advantage of the full HTTP spec potential. GETs are idempotent. POSTs are free of abuse. Etags are meaningful. Did I mention they just work?
  • FTW is intentionally and unapologetically stateless. On the other end of a request, there is a representation of a resource whose change is strictly a function of time. Personalization and other shnazzy Web 2.Poo junk are accomplished by client-side composition.
  • FTW is architected outside of a single server boundaries. Parallelism and massive scalability are part of the deal. Spreading your application across several nodes does not require redesign or even changing a line of code. Perhaps even, the framework does it automatically, expanding or contracting its physical machine span as a reaction to traffic.
  • Just like we grew out mallocs and frees, with FTW, we no longer worry about caching. Caching is effective and completely behind the scenes. FTW developer does not know anything about dependencies, re-syncing, or sliding expiration scales. Caching just happens. No tinkering is necessary.
  • FTW respects the markup. If asked to produce xHTML, it outputs minimalist semantic markup that is free of presentational dribble. It may actually prevent the developer from tossing crap onto the representation.
  • In FTW, you develop using a a strongly, dynamically typed language. Ok, this one is not really a tomorrow’s feature by most accounts, but we .NET folks are still calling them futures.
  • The framework consists of two integral parts: the server-side and the client-side. The client-side part is treated with the same respect as the server-side. They’re equals, except one manages content, and the other manages presentation. In this respect, FTW is not a hodge-podge of multiple technologies that spans several disparate projects.
  • Layers of abstraction are drawn along, not across the separation of concern. In fact, FTW manages separation of concerns. It is developed to support best practices, unified into an intuitive, thorough methodology. You know, the one that doesn’t begin with ‘spose you want to build a Web form.
  • Finally, the FTW‘s architects realize that Web is most naturally modeled as a graph. Instead of messing with tables and rows, the developer creates representations and establishes relationships between them, never departing from hypermedia to another the data abstraction. Dare I predict the end of reign for tabular data, my SQL homeboys? Meh. We’ll see.

I could probably list more items, but these seem to capture the spirit. Speaking of spirits, it’s Friday night. Enough writing. Thank you for reading. Y’all come back now, ya hear?

Javascript Makes Me Giddy

I must be a simple man, ‘cuz things like this in Javascript make me slap my sides, and go Hot damn!

		function Context(base) {
			if (base) {
				var c = function(base) {
					this.base = base;
				c.prototype = base;
				return new c(base);
			return new function() {
				// standard context functions
				this.get = function() {};
				// ...

		// then
		var a = Context();
		// do things with "a"
		var b = Context(a);
		// "b" inherits "a" as context:
		// * changes to "a" make it to "b"
		// * changes to "b" don't make it to "a"
		var c = Context(b);
		// you get the picture ...

I don’t know what’s more exciting here, the “no-much-to-it-ness” or the “bang-for-buck-ness”, but every time I get to work with Javascript, I am excited about the actual process of crafting code.

ECMAScript 4 on the Server

Let’s imagine there is a viable, publicly available ECMAScript4 (that’s Javascript 2.0, y’all) implementation that runs on the server (is there, by the way?), that can be used instead of Ruby, Python, PHP, C# etc. Wouldn’t it be great?

My first reaction is yes!. Actually more like you betcha, my goatee-chinned, mojito-drinking, flip-flop-wearing blog-buddy (that’s how I imagine you, anyway), as there would be one less language to learn for the developer.

But then, I am wondering if the developers would be more prone to get their wires crossed, switching from client to server context inside of the same language?

What do you think? Is less more in this particular case? Is having a separate language for the server side the right thing to do?

Send your answers with a self-addressed and stamped envelope. Or you can just use comments. Or your blog. Or Twitter. Or Facebook. Or Pownce. Whatever. Sheesh, this social networking thing is a full-time job.

Client-side Performance Tip

I wanted to mention this for a while, but only now found a bit of time. This is very important.

When a page loads, it first processes all child elements in the head element of the page. Until all child elements are processed, no rendering of the page will occur. So, first important bit of information is: the page won’t start showing anything until all script and link elements in its head are requested and loaded.

Conforming to RFC2616, all user agents (including Firefox, IE, Opera, and Safari) will only make 2 HTTP requests at a time to load all external assets for the page. Which means that this is not a parallel operation, and regardless of how fast the server is, the visible performance of the page will suffer if there are lots of scripts and links declared in the head element of the page. So, the performance tip is: at all costs, minimize the number of items in the head element of the page.