Where's the Web in "Real Time Web"?

The new hotness online for the technoscenti and early adopter crowd is "Real Time". The recent release of the new beta interface for FriendFeed has intensified the conversation regarding the consumption of information "as it happens". Its most breathless proponents regard it as the evolution of news and the web as a whole.

The "Real Time Web" (RTW), at least in concept, is something of a hybrid between live broadcast technology (one to many) and the ability to custom tailor and blend a number of broadcast sources to get just the information that you want. As soon as new information is available in the channel, you can consume it immediately in real time.

When asked for examples, many people point to Twitterand FriendFeedas the poster children of the RTW. The 'subscription' to individual's status updates has led to a lot of interesting phenomena, such as Twitter breaking many major news stories much more rapidly than traditional news outlets.

But do they really represent the real time web? Clearly they are a form of near real time communication, but it is primarily updates of individuals only within a specific domain. Look closely at the frenetic flow of content that can appear on the new friendfeed beta and you'll see that the "real time" portion is people messaging, sharing and conversing. The data that is presented in real time is FriendFeed's user data, the data specifically controlled by FriendFeed itself (the same is true for Twitter).

I'm not trying to diminish the value in these real time communications, they have tremendous value in their own right. What's exciting though, is that they also present a glimpse of what could be possible with rapid availability of information. In my opinion, what we are missing is the "web" part of the RTW. Twitter and FriendFeed are just the first step because they do not represent a true consolidation of online information in a real time channel.

Why make this distinction? Technologically, presenting information in real time to the user from within a controlled environment (within the walled gardens of a single site) is a significantly easier task than trying to move towards a real time presentation of the web as a whole. It is this latter concept of the RTW that interests me.

Already it seems that these two examples of real time are competing with each other on data availability. Twitter updates take a long time to appear in the friendfeed stream. This has been a loud complaint about FriendFeed, but it points to the problem of having real time cross site and application boundaries. It is not in Twitter or FriendFeed's interests to make their data available in real time outside their sites.

The web is inherently distributed. This is one of its primary strengths. This has allowed the web to grow and scale in amazing and unpredictable ways. Synchronizing distributed systems is notoriously difficult though.

Although many have called for an open source, federated, version of Twitter, I think this lack of a true RTW is the primary stumbling block to its creation and deployment. Some federated synchronization mechanism needs to exist to allow islands of communication to update each other in near real time.

How do we get to the "Web" in "Real Time Web"?

I've only been partly thinking about this, so this post is partly to hear people's research and ideas in this area. There are technologies that have been created that had similar objectives but that fell short for various reasons.

Ping networks, XMPP, update streams, FriendFeed's SUP protocol, all try to improve update efficiencies but all seem to suffer from specific shortcomings. Whether it's single points of failure, spam problems, or difficulty in implementation, there doesn't seem to exist a real time updating silver bullet. I'd be interested to hear people's experience with these technologies.