Develope small scraper / crawler

$10-30 USD

Concluído

Publicado

há mais de 8 anos

$10-30 USD

Pago na entrega

I am looking for a guy who can realize the following project: In Google, the following search string is inserted: "[search keyword] site:*/impressum" What then shall be done is the following: - The urls of all findings are extracted - The urls shall be saved in a DB - The urls are cut that way that just the domain name with the tld (with http://www. in front) is saved in the DB Example: - finding is [login to view URL] - saved in DB: [login to view URL] (if the domain is already in DB, it is not saved twice) - I will provide a list of proxies to be used for that project - The project is done when a test with a certain keyword (32,900 matches) was successful

MySQL

PHP

Python

ID do Projeto: 8990966

Sobre o projeto

9 propostas

Projeto remoto

Ativo há 8 anos

Quer ganhar algum dinheiro?

Endereço de e-mail

Benefícios de ofertar no Freelancer

Defina seu orçamento e seu prazo

Seja pago pelo seu trabalho

Descreva sua proposta

É grátis para se inscrever e fazer ofertas em trabalhos

Concedido a:

@czrain

I'm an experienced Python developer with 10+ software engineering experience. Worked for a couple years at Google, amongst others. For this, I would probably go with Selenium browser automation, for one very particular reason: Google tends to not like web automation using eg. crawlers and puts script obstacles in the way, whereas a real (or emulated browser) will do just fine. Otherwise, this seems like a decently simple task, the only consideration is that google gives you results gradually so browsing through all the pages will take a while. (Especially if there's 32900 matches!) Also, acceptance criteria is weird, you're probably aware that the number of results google gives you is an estimate and there isn't an actual 32900 individual links.

$50 USD em 1 dia

4,9

(4 avaliações)

3,0

9 freelancers estão ofertando em média $81 USD for esse trabalho

@sknadeem123

hi i am ready to provide this type application for easy to use in which technology you wish to use ? Java or Web apps ?etc ?

$111 USD em 4 dias

5,0

(17 avaliações)

4,5

@polaris0621

AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

$157 USD em 1 dia

5,0

(8 avaliações)

4,5

@deescorp

I already have a product like that. the application is build on C# and takes input of search query in files format. Then search the google,bing, yahoo and save the results in database of mysql. I can give you the product so that you can use it on your own. Let me know and i will modify that according to your needs. Thanks

$55 USD em 3 dias

4,9

(10 avaliações)

3,9

@owenzhu

expirenced scraper, has crawled itunes app store, twitter, sina weibo, etc, good at database as well

$55 USD em 3 dias

5,0

(8 avaliações)

3,8

@marketiplace

I got 7+years work experience in Data Collection,Bulk Email Campaign,Excel VBA and Internet Research in IT companies here.I can do create script and scrap datas from site Directory and yellowpages using C++,Python and Perl coding as per your requirements in excel with multiple ip rotations.I have dealt with US,UK and Australia companies President,Directors and Managers for web design and development projects successfully and I have Good Communication with writing skills.I am well versed in Internet,MS Office Applications and Phone Etiquette manners with latest Technologies.I can accept your payment terms.

$55 USD em 1 dia