Sunday, August 17, 2008

No LeftyBlog Left Out

All right, all right. In response to a couple of folk who have asked, "What's up with LeftyBlogs, and your blog and all the Massachusetts blogs?"

There are some explanations for:
  • why some blog posts don't appear for hours or days on LB
  • why some blogs' posts appear quickly and others very late
  • why California's and some other states' posts pop up immediately
  • why all of Massachusetts' blog posts bring up the rear
One example chums asked about was a Friday post here (3:10 p.m. Pacific Time). That happened to be timely, shortly after the announcement by MassResistance's Brian Camenker that he'd filed to start a ballot initiative signature drive to repeal the repeal of the 1913 marriage law set. A couple of other blogs found out about it on Saturday and posted then, including regular read of mine MassResistance Watch. It showed on LB stamped 7:12 a.m. Pacific Sunday — 40 hours later.

It would have been nice if the fellow bloggers had been aware of mine earlier and been able to post or research their posts on Friday instead of seeing it in the Globe on Saturday morning. I doubt anyone rended their garments, but I hear the Net is about timeliness.

Other places that pick up my feeds had the post up a few minutes after I published. UUpdates was one.

The short of it is that LB does a great job (for free). I'd like timely links, but I appreciate what I get anyway. A citation on UniversalHub now gives me many more hits, but I am sure my target audience is more among LeftyBlogs' readers.

Another angle of that is we know they can be hit-or-miss there. A partial answer is to get the feeds from your favorites directly or check them manually. I keep a healthy blog roll here as much for my own convenience as yours.

Some cues that you may need to check your favorites include:
  1. No updates at the Massachusetts feed for hours or a day
  2. Large sets of posts coming together with the same time stamp
  3. A post from a favorite you read a day or more ago suddenly appearing on LB
Regardless, I went to the man at LB, Kari Chisholm, the president of Mandate Media, our LB benefactor. He answered much of the complex question, including:
The basic problem is that some feeds, especially blogger and feedburner, tend to take quite a bit longer to load. Not a huge amount, like 4-5 seconds, compared to a tenth or a half a second.

But we're reading ~2000 blogs. Right now, we're reading them every 20 minutes -- but that's only possible if we kick the slower blogs into a slower rotation than everybody else. (Keep in mind that there's only 3600 seconds in an hour; 1200 seconds in 20 minutes.)

Here's how it works. It reads your blog. If it takes longer than 8 seconds, then it goes into the "slowpokes" queue. Every time LeftyBlogs updates, it reads all the non-slowpoke blogs, and then 10% of the slowpokes blogs. (And if you're no longer slow, then it rewards you and moves you back to the faster queue.)

The folks who are chronically in the slowpokes queue typically update 2-3 times a day. Obviously, the gap between your post and our update is entirely based on when you post.

And then on top of all that... it's true that the internet is full of weird little outages all the time. Not a big deal for us humans (sometimes you gotta reload a web page to see it), but when that happens to LeftyBlogs, you drop into the slowpokes queue WITHOUT having been updated - which means that once in a while, it could be 12, 24, 36 hours before you're updated.

So... I'm working on it. We're trying to figure out more efficient ways to read the feeds, including perhaps reading them in parallel, so that the slow ones don't slow everybody else down.
None of that addresses why even Blogger/Blogspot posts from the west coast appear quickly and the Massachusetts ones tag along much later. We can jest that electrons have to travel all that much farther to Oregon where LB is. Instead, I suspect that we dump into different servers with different capabilities.

Regardless, LB is a fabulous free service, one I use daily. Until they tweak and test though, don't depend solely on it to serve Massachusetts feeds. If you are politically hungry, do your own hunting as well.

Tags: , , ,

No comments: