I currently manage a large news and entertainment site on Joomla, over time its becoming more and more work for a single person to update due to mass amount of gossip, news and music videos being released.
I have decided its best to automate a large section of the site, since most news and gossip is usually from the same news providers. The only difference being that title and first paragraph usually change. I have seen a number of components on Joomla and Wordpress which have the possibility to scrape sites either via rss or url and obtain the full page's output. The problem with most of them is they dont all work well, scrape well or dont reword the content. I am looking to move the site to Wordpress or keep it on Joomla but that will be a separate project. If we can get the functionality right the migration wont be too much of an issue.
I see two options as to how we achieve what I would like;
1) You use a existing extension on WP/Joomla and customise it to achieve the desirable listed below
2) You build a independent php based site that scrapes and directly inserts into the database for Joomla / WP the content
Desirable Functionality
- Scrape Article Title, Text and Post Image
- Auto Rewrite Title, Text using Google or other free service (option to disable)
- Option to include source url at bottom of article
- Scrape on Schedule and on Demand
- When scraping on demand option to select which articles to scrape from results
- Limit number of items
- Scrape multiple sites and file in respective categories
- When scraping set author on article to predefined for scrape, ie Bollywood News Scrapes would have a set name and Technology News would use a different author
- Remove any link backs to original site and any link backs mid article
- Copy Meta Tags etc and modify, Auto Tag new Content
- Videos Scraping (optional) - Scrape a specific websites music videos section, scrape the music videos title, youtube link and insert into seperate database and automate posting
Standards
I expect some level of standards from the coder, therefore Honesty is very important, if you are modifying a existing component or know one that does the job and you need to train the template then you must advise this in sources and also document the modifications otherwise I expect the code to be fully commented. Simply if i ever need to update the code or a bug exists another developer should be able to know what was done.
Payment Expectations
Payment will be made in full on completion of the project, simply we test on a server i can provide or you provide, it does what I want, payment is made in full and you provide the full source code and files.
Hi sir,
I am scraping expert, I have did too many similar projects, please check my feedback then you will know.
Can you tell me more details? then I will provide demo data for you.
Thanks,
Kimi
Hi,
My name is Huy. I have worked with Joomla for 4 years. As you can see in my profile, I have completed many Joomla projects. Let me help you a hand.
Dear Sir,
I have read Project Description to understand the overall business domain & your need. May I ask which payment gateway you want to use? May I suggest that we schedule one to one technical meeting to discuss everything, as it is important to make sure that we are both on the same page in terms of requirement understanding, after that we can finalize everything to kick started. So, please click on the chat button to schedule a meeting time, after that I can revise our bid.
Look forward to hearing from you.
Regards
Netizens
Hi,
let me tell you what you will get if you choose to hire me. I am a professional writer. I love writing. I have been writing for all my life and this has now become my passion as well as my profession. I provide extremely customized writing. My writing will suit your need just as you want it to be. I will add a human touch to the writing so that it is not mechanized at all.
I have an excellent track record in the limited time that I have worked on the website. I will provide you with unlimited revisions and make sure that you receive only the best content!
I am a experienced developer of Web Design skilled in smart designer skilled Web(PHP, WordPress, Magento, HTML5, CSS3 and etc) Adobe Photoshop and etc.
I can start your project immediately, and try to sync timezone with clients so you needn't feel timezone differences as well.