Fastest way to find redirects a web URL goes through

$30-250 USD

Lezárt

Kiadva ekkor:

körülbelül 9 évvel ezelőtt

$30-250 USD

Teljesítéskor fizetve

I'm currently using CURL and PHP. The task is simple, database of around 1,000,000 urls per week. We need to visit each one and find every redirect it goes through. I'm currently doing this with no problems in PHP however it is VERY slow. That's why I'm looking for the fastest way to do this. Impress me and I'll have more work for you. This is a small task but if you prove yourself I'll have a lot more work for you. Passion is a must! - The software should have enable/disable option for redirects via meta-refresh/javascript location moves. Usually redirect done on header level however.

Software Architecture

Projektazonosító: 7487971

A projektről

9 ajánlat

Távolról teljesíthető projekt

Aktiválva: 9 évvel ezelőtt

Szeretne pénzt keresni?

E-mail cím

A Freelancer oldalán történő árajánlatadás előnyei

Határozzon meg költségvetést és időkeretet

Kapja meg fizetését a munkáért

Vázolja ajánlatát

Ingyen regisztrálhat és adhat árajánlatot munkákra

9 szabadúszó adott átlagosan $138 USD összegű árajánlatot erre a munkára

@chirgeo

Hi. First of all, are this urls from a different domains? I would suggest of course to use a multithreaded approach. Plus, I consider that the best for such things is to use python. So, I can deliver a script that would start n(configured) number of threads to crawl the urls and to detect redirects. I guess there can be cases then multiple level redirects are present? Let me know the details and we can continue.

$100 USD 1 napon belül

4,9

(51 értékelés)

6,3

@hoesoftware

Dear Sir, Greetings from velrajan!! I will create multithreaded apps for you.. The tool will be like this.. 1. Option to browser and load the list of urls from txt file. 2. You can set number of parallel thread you want to execute to increase the speed. 3. Click a button 4. The tool will generate a CSV file like this.. original url, redirected url. Please clarify this point.. "The software should have enable/disable option for redirects via meta-refresh/javascript location moves" I know we can redirect url from meta-refresj and javascript.. don't you want those redirected url? Looking forward to work with you.. Velrajan T.

$210 USD 3 napon belül

5,0

(73 értékelés)

6,1

@GinjuVladLucian

HEllo, If you want, i can do it in java, calling 1000000 urls using multi threading environment. I can't launch all of them at once due to the fact that i don't have a lot of RAMs on my machine. But this can be managed, and a threshold can be established. I guess 10.000 URL in parallel. Regards, VLG

$30 USD 0 napon belül

5,0

(11 értékelés)

3,5

@gnouv

Hello, Did you try with cURL multiple threads? Cheers,

$50 USD 3 napon belül

5,0

(6 értékelés)

2,9

@EngMah

Hi PHP is so slow when compared with C++, but if you do not use many threads, you won't get a better speed that deserves writing new code. I recommend using a thread pool. For example 20 threads are working together and each thread works on a set of URLs (let us say 100 url). By using threads, you will get a very good speed. I can build this using threads and asynchronous programming in Qt, so you get the best speed ever. When we use asynchronous programming, the threads won't consume CPU cycles while they are waiting for the reply to be received. Also in C++ we get faster parsing for response to detect redirects. We will also give enough time to test the program, so I added more days to the project. Thanks in advance

$222 USD 25 napon belül