I need a program that scans hundreds of webpages every 30 seconds looking for newly added keywords. This will help to alert me to current news for my stock trading.
## Deliverables
I want to be alerted to changes in financial websites to help my stock trading. I want the program to scan 500-1000 sites searching for new keywords every 30 seconds. Each site will have between 20-25 keywords
The following are three characteristics I would like the program to have.
1. There should be like a table that lists the alerts as they come up. The alert should show that either there's a change or a keyword for the alert, and show the time of the alert. It will only alert 1 time for each new change or keyword. When the program is turned off and then on again, it resumes search for keywords based on the last scan when the program was running. Sometimes the keyword is more than one word. For example if the keyword is "John Smith" the program will only alert when it sees those 2 words in sequence, not simply if it sees the word "Smith". I should be able to include up to 20 keywords for each web page.
2. For this page: <[login to view URL]>
Things will be different and this may require a lot more resources. First the search will be for keywords: 8-K or SC 13D (I may add more later). If the program ever notices those keywords on this site, then it will click on the first link to the right labeled "html". Once it reaches that page, it will click on the first html link file. From there, it searches for other key words.
3. The program should follow an email address. It alerts me everytime a new email appears. This address will be used for sites that send stock newsletters with reports.
4. Be able to scan pages that require a login and password.