APPLICATION
The application will crawl and scrape products and prices from some eCommerce sites.
MY WORK
I have developed (in the Java Programming Language) a Generic Web-Scraper Tool - called OpenMana Web Information Miner (OmanaWIM) - that can be configured to scrape any information from any website.
It can do log-in, process JavaScript / AJAX call results, chase multi-level links, post search-forms and handle pagination; can accept / process response in XML; can download images and files; is multi-threaded in a configurable way; can use proxies; supports user-specifiable filters; scraped info can be delivered in JSON or XML / posted to database or Excel/CSV.
THERE WILL BE NO NEED TO WRITE SITE-SPECIFIC-CODE. CAN WORK FOR FUTURE NEEDS ALSO. When page / navigation structure on the web site changes, no need to write new code - just tweak the configuration.
This tool can also straightaway work with sites exposing HTTP-protocol-based APIs / web-services.
Uses open-source libraries: 1. Selenium WebDriver, 2. HtmlUnit & 3. The Castor Framework.
License Fee for Tool: $250.
ME
1. Based in Bengaluru, know Kannada very well.
2. Full-time freelancer, with 15+ years of rich experience in software development.
3. Have expertise in:
A. web application architecture,
B. design and model development(OOAD and UML) including design patterns,
C. Core & Enterprise Java,
D. database development,
E. XML-Schema
4. Cambridge University Certified in Business English.