Syndication vs. email – bandwidth requirements
by ZetaGecko | Add Your Comments | Atom/RSS, Email
With RSS and similar technologies in their current state, publishers run the risk of using significantly more bandwidth than with an email newsletter. Why? Let's say you have a list with 20000 subscribers. If you email them all a 2K newsletter once a day, that's 40MB/day of bandwidth. If you have an RSS feed with the last 10 2K stories in it, and the same number of subscribers load it once an hour 4 hours a day, that's 1.6GB/day, if they don't use If-Modified-Since or ETag, and even if they do, it's 400MB/day plus a lot of brief connections when they check for updates. These numbers aren't precise since they don't account for differences like headers and such, but you get the idea.
So how do we stave off the problem? A few possibilities:
1) Build more infrastructure so that we can use more bandwidth for a similar price. I'm sure this will happen over time, but will it be soon enough?
2) Limit people to less frequent downloads, and enforce the limitation. Some sort of access frequency limitation technology is probably a good idea for any reasonably high-bandwidth feed. Slashdot, for example, stops giving you their feed if you load more than once per hour (that's the stated limit. I'm sure they've built a little flexibility into it).
3) You could force people to use tools that do conditional gets using If-Modified-Since and /or ETag headers. You'd probably cut a lot of your readership out if you did this today, but if you've got an important feed, this might help light a fire under developers to implement these bandwidth-saving features.
4) Don't put too much in your feed. Read my post from yesterday: "Bandwidth Kills - Blog Responsibly".
5) Have multiple version of your feed with different amounts of data in them, and try to get people to subscribe to the smallest one that meets their needs.
6) Use a syndication format with an API that supports loading only the most recently updated items in a feed. I'm not sure whether Atom supports this yet, though I believe they will eventually if not now. Info Bite will support this too. Perhaps an extension could be written for RSS to enable doing this too.
I'm all for replacing email with syndication for many uses, but there are challenges yet to be met before syndication is an ideal solution for everyone.