Web spider / crawler

I need a custom spider which craws specific sites I specify and pull out the html/text page and collect all text and match/inserts it against tables I have in my database. So the collected information can be used from a search page on my site.

The application should be able :

1. to run as a service under windows 2000

2. be scheduled to run on certain times like hour,daily

3. be customizable by a ini file where instructions can be dropped so the spider behaves different for specific sites

4. as fast as possible without causing to much overhead on site being traversed.

Respond only if you have experience with this material and can show me samples/demo.

## Deliverables

1) Complete and fully-functional working program(s) in executable form as well as complete source code of all work done.

2) Deliverables must be in ready-to-run condition, as follows (depending on the nature of the deliverables):

a) For web sites or other server-side deliverables intended to only ever exist in one place in the Buyer's environment--Deliverables must be installed by the Seller in ready-to-run condition in the Buyer's environment.

b) For all others including desktop software or software the buyer intends to distribute: A software installation package that will install the software in ready-to-run condition on the platform(s) specified in this bid request.

3) All deliverables will be considered "work made for hire" under U.S. Copyright law. Buyer will receive exclusive and complete copyrights to all work purchased. (No GPL, GNU, 3rd party components, etc. unless all copyright ramifications are explained AND AGREED TO by the buyer on the site per the coder's Seller Legal Agreement).

## Platform

Windows 2000.

SQL 7/2000

Mérnöki munka Internetes marketing MySQL PHP Szoftverarchitektúra Szoftvertesztelés SQL Webhosting Weboldal menedzsment Weboldal tesztelés

Projektazonosító: #3488798

A projektről