What's the best crawler?

Gsitecrawler is fine but there was some problems about big websites.

A1 Sitemap generator great , but you cant pause crawler only stop and finish. Sometimes needs to pause cause u cant stay online for a week.

xml-sitemaps script ; I tried and nearly my apache will blowed up ! It doesn't know where to stop