You will (or already have)
1) write a distributed crawler system running multiple crawlers with a command an control
2) eventually crawl over 120M pages of a social network (Linkedin) and defeat potential crawling roadblocks
3) deliver results in a text comma delineated file(s) based on our data format
You will get:
-Massive cloud computing accounts. There will be almost no limits.
-LI Accounts
-help from our Engineering team.
You will get paid along the way... so don't worry about that. And this will be fun.
Hey,
We would like to work on this project, as we see this as something challenging and new. We are really good at doing such things!! Check PM for details.
regards
redAtom Team