Question ( I MADE A IMAGE OF A TEST )
DESIGN A WEB CRAWLER (class diagram +pseudo code).
A web crawler starts with a list of URLs to visit , As the crawler visits it identifies all the hyperlink in the page and adds them to the list OF URLs to visit and then recursively visit them