Are Bots On Twitter Hurting Overall System Performance? - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
Infrastructure // PC & Servers
Commentary
10/29/2008
06:05 PM
Allen Stern
Allen Stern
Commentary
50%
50%

Are Bots On Twitter Hurting Overall System Performance?

If you use the "real-time short messaging service" Twitter, you know that over the last year there have been many performance issues. So many, in fact, that users have created a name for the outages, called the "fail whale." I started to poke around and began to wonder if bots running on the Twitter network could be causing some of the system performance issues on the service.

If you use the "real-time short messaging service" Twitter, you know that over the last year there have been many performance issues. So many, in fact, that users have created a name for the outages, called the "fail whale." I started to poke around and began to wonder if bots running on the Twitter network could be causing some of the system performance issues on the service.A bot basically performs a function or functions. In the case of Twitter, most of the bots either post links to content from across the Web or provide a reply service for certain phases.

To get an idea of just how many bots there are on the Twitter service, over the past couple of weeks I analyzed blog posts from more than 10 popular tech blogs. While this won't account for all of the reply service bots, it did provide good insight into the massive number of link-posting bots. For the purposes of this post, the bots listed below come from one post on the tech blog TechCrunch. Here's a list of the bots that reposted a link to the post on TechCrunch along with the number of subscribers (i.e., followers) to each one:

  • top_post - 104
  • tc2tw - 25
  • techupdates - 171
  • RSS_TechCrunch - 350
  • techupdate - 31
  • TechFeed - 207
  • twittfeed - 116
  • RossRSS - 70
  • SocialMedia411 - 51
  • TechGlance - 449
  • top_blog - 112
  • nuws - 132
  • readmashcrunch - 96
  • wiredgnome - 43
  • techcrunchtweet - 47
  • balduaashish - 28
  • tinycomb - 31
  • readburner - 348
  • normman - 89
  • crazyengineers - 25
  • TwitLinksRSS - 392
  • seomasterlist - 207
  • simplynews - 56
  • tweetmeme - 643
  • memebot - 48
  • foobot - 42
  • readburnerRSS - 112
  • techgurus - 225
  • linksgoogle - 300
The bots listed above only represent one post from one tech blog. If you start to multiply this out across blogs and posts, you can see where this might present some level of performance issues for Twitter. Are these bots actually needed? How many are actually being used anymore? With so few subscribers on the bots listed above, why not offer one feed bot and reduce the server load overall?

What's really interesting is that in the case of TechCrunch, its Twitter account has the full feed in the account, so none of these bots are even necessary for TechCrunch, although some of the bots listed also scrape feeds from other tech sites as well.

As Twitter works on implementing a revenue model, I can only imagine that bot traffic could be the first hit.

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
News
Can Cloud Revolutionize Business and Software Architecture?
Joao-Pierre S. Ruth, Senior Writer,  1/15/2021
Slideshows
10 IT Trends to Watch for in 2021
Cynthia Harvey, Freelance Journalist, InformationWeek,  12/22/2020
News
How CDOs Can Build Insight-Driven Organizations
Jessica Davis, Senior Editor, Enterprise Apps,  1/15/2021
White Papers
Register for InformationWeek Newsletters
Video
Current Issue
2021 Top Enterprise IT Trends
We've identified the key trends that are poised to impact the IT landscape in 2021. Find out why they're important and how they will affect you.
Slideshows
Flash Poll