Develop and implement web crawlers in Python

What we need; Development, implementation & support of the Website and the Crawlers what we ask; fully commented code, open source solutions, documentation & support. General consideration; We want the software/site in English. A. Develop and implement 5 web crawlers to run continuously as a batch job in background updating the mysql db and tracking information changes from sites List; [login to view URL] [login to view URL] [login to view URL] [login to view URL] [login to view URL] The scraper should retrieve ALL the information about the listing and store it directly on our DB. The script should also be able to detect duplicate autos and avoid them. We prefer Python (Scrapy) scripts for the crawlers but this could be implemented in some other language. In the ideal scenario the script should never stop running. A second script for each crawler should be also developed and implemented in order to clean the DB and remove un-published listings . You will also have to create a [login to view URL] where a site admin may be able to see the scrapper progress, restart it, see logs, stop, etc B. We will provide the DB structure for you to work with, but any optimization suggestions are welcome on this matter; Please see [login to view URL]