Need a crawler to do the following
1. Start from a seed URL and follow links found
2. Extract Title, URL and E-mail address from visited site if certain kewords are found on the site. I will define the keywords to look for.
3. Send a mail to the Email address found
4. Store the data (Title, URL and Email) for future use.
I should be able to start and stop the crawler and I need to ensure that I don't send duplicate mails to the same place.
I am also open to suggestions.
As handsfree as possible. no encryption please,
Thank you