The latest issue to arise on the Atom syntax mail list could almost have been inspired by my entry two days ago: Should Atom handle today's problems or tomorrow's? The question is whether to create a group within or connected to the Atom community (or whatever standards organization Atom goes with) to keep an eye on big-picture issues that may need to be addressed at some point. Doing so would certainly get my vote.
I'd vote for such a group to be tasked with keeping track of issues, prioritizing which should be addressed in what order, and keeping groups working on concrete items apprised of potential problems that the decisions they are considering might create down the road. But this group should not get into the business of creating specifications--that would only distract from its purpose. If work needs to begin on new specifications, this group should recommend the creation of groups to do that.
Getting back to the title of this entry, a specific issue that has arisen in the discussion is notifications. The web (and much of the internet) today is set up so that a client requests a resource and a server responds by giving it the resource, or a client posting a new resource, or a client posting an update, etc.: the client initiates the interaction. With the rise of blogging, the web is becoming far more dynamic--content is updated far more often than it used to be. To keep on top of all the latest updates, client software polls servers that have been subscribed to at regular intervals to check for updates, and notifies the user when an update is found. This happens not only with feed readers--some web browsers can be configured to poll for changes to bookmarked pages and notify the user of updates.
All of this polling, which, after the user has subscribed to a service, is done without further user interaction. As more people subscribe to more feeds, web pages, etc., the amount of traffic running across the internet will increase dramatically unless methods are devised to minimize the bandwidth needed to keep everyone up-to-date. This may require changes to some of the core technologies that make the internet work.
One thing I didn't mention in my original entry that's worth mentioning is that there are existing technological solutions that can help mitigate bandwidth problems: If-Modified-Since, ETag, caching proxies, etc. The solution may be simply to push fo...Notification techniques like RSS 2.0's cloud are attempts to reduce the need for constant polling. When a client subscribes to a feed, if it has a cloud interface, it can request that it be notified whenever the feed is updated. Then, rather than polling for updates, it just sits back and waits to be told that an update has been made.
One problem area with notification has to do with firewalls and NAT. A connection running through a firewall or NAT server generally has to be initiated by the client (or to be more precise, from behind the firewall or NAT server). Unless some special configuration is done, the only way the client can receive a notification is if they initiate a connection, and the connection is help open until the notification comes. The number of connections that a particular server can hold open at once is limited, and holding them open requires a certain amount of network traffic, so at some point, such a solution will not scale well.
Since NAT servers, and to a great extent firewalls, operate at the TCP/IP level rather than the HTTP (or FTP or HTTPS or SMTP...) level, solutions not requiring persistent connections would either have to be handled at the TCP/IP level, or NAT servers and firewalls would need additional intelligence about the content of TCP/IP packets to allow them to take an active role in routing notifications to their clients.
Changing TCP/IP itself would be a much more drastic step. In the near term at least, developing a notification protocol that would work with all kinds of services and building intelligence about it into firewalls is probably a better solution. But in the long term, perhaps a bigger change is called for. I don't know enough about the underlying technologies to presume to be able to make the decision. Hopefully a group of people who do have that knowledge will form somewhere and come up with a good solution. Since the growth of Atom will directly influence the need for such a solution, the Atom community might be a good place to start such a group.
May 19th, 2004 at 12:36 pm
One thing I didn't mention in my original entry that's worth mentioning is that there are existing technological solutions that can help mitigate bandwidth problems: If-Modified-Since, ETag, caching proxies, etc. The solution may be simply to push for broader use of these.
Another point worth mentioning is that unless methods are developed for delivering only the part of a feed that has been udpated, notification would actually increase bandwidth usage on sites where the feed is updated more frequently than the average polling interval.
That said, the ideas currently stewing my mind involve a combination of notifications, delivery of updated items only, and smart caching proxies.