Screaming Frog SEO Spider is a software to crawl websites’ data and analyze sites’ SEO issues, and it offers Windows, macOS, and Linux operating versions. Screaming Frog SEO Spider offers a free trial version that allows you to crawl 500 URLs. The article shows how to use Screaming Frog SEO Spider with YiLu Proxy on Windows PC.

Download Screaming Frog SEO Spider tool at https://www.screamingfrog.co.uk/seo-spider.

YiLu Proxy dynamic IP is suitable for web scraping, so I directly extract YiLu Proxy dynamic IP information here.

A.Select rotating residential IP or datacenter IP

B.Select “Change IP per request”

C.Click “Refresh(HTTP/Socks5)” button

D.Right click an IP and click “copy IP and port” or directly copy IP address and port.

Select “Use Proxy Server”.

Enter proxy IP address, and port in Screaming Frog SEO Spider proxy configuration.

Click “OK and Restart” to make the changes take effect.

Click “Start”.

Then a pop-up window will appear.

[caption id="attachment_3376" align="alignnone" width="297"] screaming frog enter proxy ip username password[/caption]

You can see the crawling process on the top right, and click “Pause” at any time.

Here are some web pages that have been crawled.

Here I export web pages’ titles.

Click Pages Title - Export.

Then enter a name, select a file type, and the save directory(desktop).

Click “Save”.

Then you can see the file appears on the computer desktop.

Double-click and open it.

Here are these crawled web pages’ titles.