FULL TIME Python scrapper web crawler web applications scrapy selenium

$750-1500 USD

Lezárt

Kiadva ekkor:

majdnem 8 évvel ezelőtt

$750-1500 USD

Teljesítéskor fizetve

I am looking for expert , experience python scraper developer with tons of experience in scraping .. You will be creating script to scrap millions of data , on regular basis .. this will be web based script .. data will be saved in some kind of db ... Previous experience with amazon , walmart, costco, ebay etc scraping is big plus. I am not looking for command line or desktop based program. This will be web based program that run on linux AWS or some cloud server. You should know following advanced techniques to solve scraping issues 1. Able to run multiple scrap / threads in parallel 2. ABle solve ip blocking issue by proxy IP rotation logic 3. Capcha solver 4. Selenium browser automation to login to certain account and do some steps Here are some idea 1. Logic to accept scraping / browser automation request 2. decode request into scraping / browser request 3. Queue / fifo in case of too many scraping request 4. ip proxy handling logic for scraping request 4. automatically trigger some scraping on daily / timely basis 5. check scraping status, % complete , estimate , check output response/ 6. accept request only from cetrain ip .. and ip based request limit 7. Creating API for accepting request and getting data On average I am looking to pay $50-$100 per scrap / automation website script. And we have 50+ websites that needs to be scraped. Looking for long time partner.. Once I start making revenue from this, you will also own part of company and generate extra revenue ... In application please answer following 1. write 'warriors' before application 2. write your previous scraping experience. What websites and how much data. Any experience with amazon ? 3. Have you ever has issue with ip blocking ? how did you handle it ? If you used proxy rotation, from which website did you get proxies. 4. Any experience with selenium or browser automation ? 5. Any experience with creating api for scraped data ? 6. Are you company or individual. If individual , full time freelancer or you have some other full time job ? 7. Send me example of previous / complex scrap / browser automation projects.

Projektazonosító: 10478597

A projektről

15 ajánlat

Távolról teljesíthető projekt

Aktiválva: 8 évvel ezelőtt

Szeretne pénzt keresni?

E-mail cím

A Freelancer oldalán történő árajánlatadás előnyei

Határozzon meg költségvetést és időkeretet

Kapja meg fizetését a munkáért

Vázolja ajánlatát

Ingyen regisztrálhat és adhat árajánlatot munkákra

15 szabadúszó adott átlagosan $1 325 USD összegű árajánlatot erre a munkára

@phpXpertbd

Dear Sir, I'm very much delighted to let you know that i did data scraping with PHP-cURL, PhantomJS, Node.js, Selenium from many sites. I just scraped the data from web site and then wrote the data in mysql database or excel or csv or xml file. I worked on many similar projects, I have big experience in data mining projects. I have written hundreds of web scrapers which scrape millions of pages each day. I'm ready to fulfill your requirement. I can finish this task in short time, with the best quality. I can assure 100% accuracy. Please give me the opportunity to do the work. With Kind Regards, Debdulal Roy Proshanta

$750 USD 20 napon belül

4,9

(41 értékelés)

6,3

@mmadi

Dear Client, Greetings from Flowgica technologies, I have experience with these skills. We do have similar experience therefore I am looking forward to discuss and move ahead. please check our freelancer portfolio at https://www.freelancer.com/u/mmadi.html?page=portfolio I am ready to work with you,kindly waiting for your response. Thanks & Regards, Mmadi

$1 200 USD 19 napon belül

5,0

(5 értékelés)

5,7

@baddesigner

I am arif from Pakistan, having 11 years of experience in software house as a Project Manager and in the IT-Programming . My areas of expertise are: 1- PHP 2-Express 3-Bootstrap 4- AJAX 5- J Query 6- JavaScript 7- iPhone 8- Mac OS 9- Database Design 10-Cocoa 11-Android 12- Symfony PHP 13- Python 14- Laravel 15-CodeIgniter 16-MySQL 17-RESTful 18-Software Architecture 19- CRM 20- HTML5 21-CSS3/LESS/SASS 22- MariaDB 23-WordPress 24-Angular JS 25-Node js Why choose me? I- I only Take the project which I am sure I can do it within time frame and budget. II- Fast turnaround time . III- 24/7 available to help my clients

$1 184 USD 20 napon belül

5,0

(35 értékelés)

5,1

@techminds4

Dear Prospect Hiring Manager. Thank you for giving me a chance to bid on your project. i am a serious bidder here and i have already worked on a similar project before and can deliver as u have mentioned I have checked your requirements.i have right skills to work on this assignment my award = superb result = happy client. In a good partnership, good results happen. Good cooking makes good eating!BWe consider our client as our partner. can u provide your email or sky-pe etc for further discussion about the project. I am ready to discuss with you with best Regards

$1 250 USD 20 napon belül

0,0

(0 értékelés)

0,0

@Denisakov

warriors 2. I had small experience with amazon: scraping the start reviewers for books. But it wasn't very complex. i have recently designed a scraping tool for TaoBao - chinese wholesaler. 30 brands, each brand between 2000 and 70000 products. Before that did the system for project data scraping for 3 different websites 10000 projects each with 15 data points to scrape. Also did a system for shipping industry with vessel tracking data scraping for 50 ships per day. 3. IP blocking is the constant problem. Public sites usually don't bother with it but commercial just block you flat out, especially the once like Amazon, who sells access through API. I have tried to use IP rotation, but didn't have good experience with that - too flimsy and unreliable, too much maintenance. I use a bit of masking techniques with browser and device type, it can full the server for a while but also not reliable. The best solution I found so far is Google Apps. All requests go through bigG and no one blocks bigG. 4. Heard about it, but didn't have the need so far. 5. No haven't done that. But we will figure it out. 6. Individual, don't have any other job. Could dedicate upto 5 hours a day to your project. 7. That is a funny request, any employer requests complete confidentiality with this things, so No I can't send you any examples of serious scrapping jobs :) My proposal is to use Google Apps for the whole system. It will definitely be the most reliable and the cheapest solution!

$2 222 USD 30 napon belül

0,0

(0 értékelés)

0,0

@Shopify

I want to discuss this project with you further, let me know the best suitable time for you to schedule the meeting, Feel free to message me at any time, i used to be online 14 hrs in a day on this website so probably you will get a quick response from my end.

$1 546 USD 40 napon belül