Hi,
This is a project to scrap websites and to load results in a MySQL database.
This database will then be used by our content managers to write short descriptions of results and route users to the original website.
The data consists in interesting activities to be performed in a given city (restaurants, bars, visits, highlights...with addresses, descriptions, pictures...).
The developer will have to:
- Develop a web scraper scrapping 3 different web sites (these sites will be provided once the freelancer selected)
- Load HTML pages into a collection [login to view URL] class (Object Oriented programming, class to be developed)
- Load the Activity objects into a MySQL database via an [login to view URL] object
- When loading the content in the database, the code will have to check if a similar activity name already exists in the database for this city.
If the activitiy already exists, from the same website, the existing entry is updated
If the activity already exists, from another website, a new entry is created by it is linked to the existing one
If no matching name is found, a new entry is created.
- IMPORTANT note: the scraper MUST respect the [login to view URL] policy of the website. If the [login to view URL] of the provide websites prevent from doing this project, we will provide you alternative websites
- Pictures will be downloaded into a local folder (the URL to this folder will be loaded in the database)
- Build a simple HTML page to test the crawler. The HTML page will allow the tester to do the following things:
1. Select on of the 3 web sites
2. Manually enter the name of a city
3. The page will build the corresponding URL and start scarping it and loading in the database
- The source code will have to be commented
Technologies:
- Programming language: PHP 5.5.3
No PHP framework is authorized on this project, only standard PHP function and packages have to be used.
Programming must be object oriented.
PDO must be used for data access layer.
- Database : MySQL 5.5
- The deliverables will be tested using a MAMP apache and mysql installation on a Mac OS X environment
The developer will be provided with:
- a graphical datamodel
- an SQL file to create the corresponding data structure
- the URL of the 3 web sites to scrap
The project will consist in 3 steps, which will divide the payment:
- Delivery of the scraper + loading in the database for the 1st website : 50%
- Delivery of the scraper + loading in the database for the 2nd website: 25%
- Delivery of the scraper + loading in the database for the 3rd website: 25%
Good luck!
I fully understood your job description. I'm really interested and ready to work as soon as possible.
I hope you consider me as part of your staff. I'm a fast-learner, speed typist, flexible and dedicated.
I am looking forward to your good response.
More about Me: I am a Mechanical Engineer by Profession. I have just completed my graduation from MIT Pune University. I am well versed with all the MS Office tools. I Have a typing seed of 60WPM. I am dedicated towards my work. Providing a quality output towards end is my ultimate aim. I am a new bird in this sky of freelancing but i have come up with strong wings of dedication and smart work to fly high and stay long in this sky.
I use internet on my laptop 8 hours a day. Do Consider.
Thank You.
I am MBA holder , I have efficiency in Microsoft Office 2010.I am a talented and experienced Data Entry Operator. I have good experience with Data Entry and I have been working as a Data Entry Operator for many years. I am a hard worker and enjoy working at all critical conditions. If hired by you I will be responsible to complete the task within the stipulated time frame and with utmost confidence. Outsource me for your project and I promise to bring in the best output.
Hello! Good day! I am Hannah Andrea Tallo. I believe that I am fit for this job and able to meet your standards. I am hardworking, reliable and persistent when it comes to my work. I hope I'll be a given a chance. Thank you! God bless!
I have done B. Technology in Computer Science Also I work on many regular expressions for scrapping data.
I am having more then 5 years experience working with php scripts and mysql queries, I am also very much expert in writing different complex regular expressions and scrapping data.I have done maximum projects of scrapping data in my company.
I am reliable and I do value punctuality as I promised to finished the project before the due date. I am also a hardworking person as I can even work for almost 24 hours a day if the job requires to be finished right away. I have my own laptop and internet connection which will help me finish the project quickly.
regards
imitiaz