Scaling, Twitter, some thoughts

Suddenly everyone is a scaling expert. I've worked in some pretty hard-core engineering environments dealing with these kinds of issues and the truth is scaling is hard. I find it amusing how everyone seems to have the solution for Twitter's problems. Blog posts, comments like "just use PHP + ZEND dood", or anything less than a true bottleneck analysis of the system is not going to provide a solution. There's a reason why people pay big money for database performance consultants, scaling experts and their ilk.

Why is everyone talking about scaling? Because it's a topic that gives us nerds "stiffys". This post by Ted Dziuba cracks me up. Quick warning: he's the guy who used to write "uncov" so yeah, it's "colorful". It's funny but also somewhat painful. His post hits close to home with our experience with Grazr. We put a lot of effort into a good scaling architecture but we've not gotten as many users (yet) as we'd like. I even gave a talk at the MySQL conference where this was one of the main points (overemphasis on scaling).

We've been working on a new Twitter-based project at Grazr, and after doing some simple calculations I was surprised at the low volume of data/updates they're really dealing with (yes I understand their topological issues). It's surprising that after a year they haven't solved their problems, there's nothing inherently unique to what Twitter does. Even if their initial system was brain-dead, after a year (and no shortage of cash) I would have thought they would have found a path, no matter how painful, out of their current situation. I think their real problem is that, if the blog reports are to be believed, they've not hired people to focus on this issue. I have to repeat scaling is hard so if you want to solve the problem you need people to focus on it. A technology switch is not going to get you there.

I feel for the twitter guys, I do. They're getting publicly whipped, but it's in the A-list uber-nerd space. The loudest complaints also seem to come from people that aren't really looking to change services. There are calls for twitter replacements, but a quick search will find several examples already. I attended a session at Boston BarCamp3 that was on this same topic as well. We even flirted with the idea here at Grazr, to build a twitter like thing. Why build a twitter replacement when there are lots of these things out there already (e.g. Pownce, Jaiku)? Why don't people switch? Do you think you can out-Twitter Twitter?

The truth is that Twitter is not compelling because of its technology. People, especially technology people, often don't understand user inertia, user investment in a service, and user experience as something separate from the technology. Would it be better if Twitter wasn't down so often? Sure. Do I think it's going to kill them? I think it would take a level of service outage much higher than what people are complaining about now.

Update: The twitter guys have responded. The gist, they're hiring more people to work on the problem. It's the classic "replace the engine while the car is running" problem but one I'm sure they'll fix.