In this project you will develop a multi-threaded program for crawling the Web. Each web page encountered will be processed in some way that is dependent on its content. The software will offer options for constraining the search; for example, to just one web site.

To do this project you must be familiar with Java and programming with threads.