Rent-A-Crawler

15 Aug 2007

A few months ago I drastically changed how the urls on my site were built. I moved to using the ASP.NET 2.0 virtual path provider to make more friendly urls. Because I changed the "directory" structure, havoc was wreaked upon my unsuspecting relative urls. I had to go through my posts and look for bad links. I figured I might had missed some.

So I decided to write a crawler, to crawl my site and look for bad urls. And sure enough...I did miss some. Just in case you want to use it yourself, I have it available for download, along with some notes on how you implement a crawler in C#.

Enjoy!

Filed Under: .NET Pylus

Comments: 0

Comments

Something To Say?

Name: (optional)
Email: (optional, and will not be displayed)
Website: (optional)
Comment:

You can use html tags like <b>, <a>, and <p>. Actually, you can use just about anything. If you want to see what it will look like, click the preview comment button below.