Python

Jan 07

Crazy Python Crawler

Someone emailed me doubting my crawler could operate at the speeds I posted last week so here is a video I took this morning. I should have waited a few minutes after launching it before starting the video as it really starts cranking once all the threads get rocking ...

Continue reading Crazy Python Crawler in >>

Jan 03

Big SEO’s with Crawlers: Lets See Your Stats

Code, Crawlers, Python 8 Comments »

OK I'm just ecstatic with my new crawler, I think nobody but Google has one better than me, and I'm ready for a good old fashion show-and-tell. Multi-threaded programming is a bear to deal with and I've written several crawlers in different languages. For years I've been plagued with ...

Continue reading Big SEO's with Crawlers: Lets See Your Stats in >>