Remember the Python crawler NotSleepy built to suck up all your internets and find your affiliate IDs? Well we kept massaging the code and finally slapped that thing down on a fat pipe. WOW. The stats are rocking now. How about double time!

Latest Stats:
35.6 URLs per second
3.073 Million URLs per day!

Whats most promising is that the new fat pipe is still the bottleneck which means that if anybody really wants to party, all we need to do is lay down some greenbacks and a OC-12 will show us mass terabyte pleasure.


Share it:
These icons link to social bookmarking sites where readers can share and discover new web pages.

  • Digg
  • del.icio.us
  • DZone
  • Ma.gnolia
  • Netscape
  • Reddit
  • StumbleUpon
  • YahooMyWeb