1. Build a scraper using Python to scrap 13 similar looking real estate websites
2. Clean data using regex etc. (number, text etc. formats)
3. Store the data in our MySQL database in our Contabo VPS cloud (Linux)
4. Setup VPS cloud database and server.
5. Schedule crawler to scrape data every day
6. Write a code to automatically to update the database (sometimes the data is updated, edited or deleted on the source website from where the data is scraped, so these changes should be reflected in our MySQL database after every crawl)
7. Do not have duplicate data in the database
8. Intelligently rotate IP to avoid getting banned
9. Run the crawler and the database from the VPS
10. Provide detailed explanation on how to use the scrapper and database in the VPS.
Deadline: 6 days
More information in the docx.
43 freelancers estão ofertando em média $519 para esse trabalho
I can provide you 13 Scrapy spiders (one per site). Each spider will visit website, crawl it and save information into MySQL database just like you want. I can complete it in less than 5 days for just $750.
Scraper for real estate websites I've been developing such systems for more than 6 years, and I can handle it professionally, do contact me to discuss further. thanks
Hello sir, I have completed web/data scraping jobs for many times. I am interested in your project as well. I would like to discuss details via pm. I am looking forward to hear from you soon. Best Regards,