Deep Crawl v Screaming Frog

As the original post was over 2 years old, I thought it was time to update this article as so much has changed in the meantime. You will now see the latest info on both the  products

Which is best? Deep Crawl or Screaming Frog?

I’ve used both pieces of software and I love them both, although there are also other options out there.

What is screaming frog SEO Spider

Screaming frog is a desktop application you can download to crawl your website to find technical issues. Screaming frog works by sending a bot to your website to crawl it and reports back certain technical reports. It’s how you use these reports which means you can use the screaming frog tool to boost your SEO efforts.

So whats the differences between Screaming Frog and DeepCrawl.

Screaming Frog

There is a free version of Screaming Frog, but at £149 per year, the paid version is worth the investment.

Pro’s

• No limit on the number of pages it can crawl. The only limitations are from the machine it’s running on.
• The ability to check for broken external links.
• Ability to crawl JavaScript websites
• Can easily crawl a list of URL’s – great for when you’re doing Server Log Analysis
• Unlimited number of sites you can crawl
• Ability to identify large images

Con’s

• No schedule crawling (can be done but not easily)
• Runs on your own server/machine (recently launched the ability to run off an SSD drive – but I haven’t tested this)
• Historically comparison – while it can be done it excel, its a lot of manual work and the ability to write sumifs and vloopup statements. Its all manual.

DeepCrawl

Deepcrawl is a web-based application that has servers to host the bots for crawling. It’s a paid service of around £50 per month. It sounds a lot, but it means you don’t need to worry about slowing down or crashing your machine, etc. Everything is taken care of for you. For more information on Deepcrawl visit here.

Pro’s

• Schedule crawling.
• Change reports (from a previous crawl to the current crawl).
• Web-based (no dependence on your machine).
• Links to Google Analytics to give you even more data.

Con’s

• Doesn’t check to see if external links are active (as a standard feature, you need to check the box, see image below. use 1 credit per link checked)
• Limited crawl budget (need to increase monthly plan).
• Limited to the number of sites you can crawl. The basic plan only allows you to set up 5 projects a month and each project crawls one different website.

For more information on Deepcrawl visit here.

Overall, I do love both products and regularly use both products – it depends on the task I am wanting to do.

If you are unsure which is best for you – here is a video showing the key differences and where each one is best.

Summary Table

* Limited by computer resources

** Active projects are those which have a crawl run in the current billing period.

There are two other packages available from Deepcrawl. A Medium level plan and a bespoke plan.

Andy Halliday

I am the Managing Director of Coreter Media and have been in Digital Marketing since 2009. Initially in-house working for some of the UK’s biggest brands, but now I run my own agency helping small businesses grow.