Scrapy project

We require a freelance expert on web scraping, able to automate data collection on several websites (about 50) Our technical constraint is that the successful candidate should use Python (2.7) and the open-source library Scrapy. Our targets are mainly two types of websites : wine critics' websites and wine e-commerce websites. For each type, we will define precisely which field we would like to retrieve as structured data. E.g.: { ‘price’: 2054.83, (float) ‘name’: ‘Petrus 2010’, (string) ‘currency’: ‘EUR’, (ISO 4217 currency code) […] } The candidate should define the url traversal required to crawl each website along with any parsing methods necessary to extract each of the fields indicated in the specification. They will also be required to design the data pipelines that transform parsed data into the desired types. E.g.: '10 €’ —> {‘price’: 10, ‘currency’, ‘EUR’} The expected piece of software should be a standard Scrapy project (zipped), containing one or several « spiders » (one per website). Each delivered spider is expected to crawl according to the above description using the standard Scrapy command `scrapy crawl {spider}`.