Aggregator crawler trabalhos

Tenho um site com Crawler/scraper engine, (Basicamente ele captura imagens de um site e posta no meu,salvando no banco e ja postando, no momento meu site faz isso de alguns sites, preciso adicionar mais 4 sites para copiar as imagens) Meu sistema usa framework laravel.

Estou procurando um desenvolvedor de web crawler para um site de comparação de preços para um nicho.

O que você precisa? Experiência em Ruby on Rails, banco de dados Mysql, interface com API's (lomadee, zanox, etc) e conhecimento básico em Crawler. O que você precisa desenvolver? Um site básico de marketplace que irá exibir: - as ofertas e cupons disponíveis; - gerenciamento dos cupons; - exibir nossa melhor oferta para o produto escolhido; - a lista de lojas parceiras. Para informações detalhadas, entre em contato conosco! ;)

Caros, Precisava de algum programador que entenda bem de crawling. Precisamos pegar vários números de PDFs dos Diários Oficiais de vários Estados brasileiros. Depois disso, precisamos construir uma interface muito simples e sem necessidade de design/usabilidade para poder fazer um seach na base de dados e receber os resultados.

Precisamos de um crawler em Java ou Ruby. Brief anexo. 1- Você pode usar Java ou Ruby para este projeto. 2- Nós queremos consumir parte do conteúdo via Web. A UI não importa muito neste momento. Wireframe navegável:  3- Reports e user management não são necessários, apenas a disponibilização de um logs. Ao mandar uma resposta, esperamos que você (1) diga se pretende usar Java ou Ruby, (2) estime a quantidade de horas que você levará para completar o projeto, (3) estabeleça em quantos dias você espera terminar o projeto e (4) qual será o seu preço. Exemplo da resposta que buscamos: "Vou usar Java, esse projeto vai levar 30 horas e posso te entregar tudo em...

as discussed via email

Por favor, Cadastre-se ou Faça Login para ver os detalhes.

Por favor, Cadastre-se ou Faça Login para ver os detalhes.

Preciso categorizar uma lista de cerca de 10 mil sites por cidade e por estado. As informações encontram-se disponíveis no serviço abaixo, mas sem um webservice / API, com limitação de 3...categorizar uma lista de cerca de 10 mil sites por cidade e por estado. As informações encontram-se disponíveis no serviço abaixo, mas sem um webservice / API, com limitação de 300 consultas a cada 24 horas por IP, e captcha. Caso exista alguma outra solução que não usando o serviço acima, estamos abertos. A ideia é que seja criado uma crawler e que esse crawler já nos retorne com as informações que precisamos. Precisamos de um arquivo de r...

A partir da informação de uma cidade e um termo de procura, criar uma lista com nome, bairro, endereço, website, email, latitude e longitude dos estabelecimentos constantes nessa cidade.

A partir de uma lista de endereços preciso que o robot busque no goglemaps suas coordenadas

Processo: Definir no php proxy ou formas de não identificar o crawler e ser banido o IP. Mesmo sendo url´s distintas, temos urls utilizando o mesmo servidor, que pode identificar e banir o crawler. 1) Lê arquivo .txt com url da loja virtual: a) Http code do site. Se o site estiver online segue para ler as informações. Se o site não estiver gravar no mysql queo site está off. 2) Extrai as seguintes informações da Url Se possível identificar alguns pontos:  HTTP/1.1 200 OK Server: nginx X-Powered-By: ASP.NET Trazer essas informações acima e todas as printadas. Por que a importância dessas informações? A intenção é saber o servidor que ele u...

preciso de programa para fazer web crawler no site da regisro.br. Preciso que a sessaso seja congelada para com apenas 1 unico ip eu consiga fazer varias consultas.

preciso de programa para fazer web crawler no site da regisro.br. Preciso que a sessaso seja congelada para com apenas 1 unico ip eu consiga fazer varias consultas.

preciso de programa para fazer web crawler no site da regisro.br. Preciso que a sessaso seja congelada para com apenas 1 unico ip eu consiga fazer varias consultas.

...no seguinte app android: 1 - Característica: Opção de curtir igual Do Facebook. - UM BOTÃO DE CURTIR EM CADA OFERTA (O USUARIO PODERÁ CURTIR CADA OFERTA APENAS UM AVEZ E DESCUTIR CASO PRECISE) - MOSTRAR ABAIXO DO BOTÃO A QUANTDIDADE DE USUARIOS QUE JÁ CURTIRAM A DETEMINADA OFERTA EXEMPLO: [BOTÃO CURTIR] 10 PESSOAS JÁ CURTIRAM 2 - Característica: -CRIAR UMA ACTIVITY (TELA) NO APP COM OS SEGUINTES RESULTADOS (LISTA AS OFERTAS QUE AQUELE DETEMINADO USUARIO JÁ CURTIU) - TOCAR EM UMA DETERMINADA OFERTA E IR NA ACTIVITY (TELA) DA OFERTA CORESPONDENTE. OUTRAS INFORMAÇÕES: Api do projeto Goceng ( ) está em PHP e o banco de dados em MYSQL bem documentado

Estamos procurando pessoas capacitadas para desenvolver um sistema que possa buscar vídeos de outros sites (XXX) de preferência o profissional com conhecimento a linguagem Python. Gostaríamos de desenvolver esse sistema, com capacidade de manter essas informações em um website que será definido domínio para armazenar esses dados capturados através do (web crawler). Aguardo Propostas!

Pegar informações em outro site e gravar em um banco de dados.

busca produtos de varias lojas e linca em uma unica pagina

Criar um arquivo .xml. É um "sitemap", segundo os padrões usados pelos sites de pesquisa, como o Google. A diferença é que, ao invés de usar um script que faça um crawler (o que é muito demorado e consome muito do servidor), ele pega os dados direto do MySql e monta a URL. É preciso ler duas tabelas de um BD e listar um dos campos dessas tabelas com a URL (ver txt anexo), usando UTF-8. Esse campo possui caracteres acentuados que já estão convertidos. E testar para ver se funciona !

Eu preciso de um crawler, para puxar as informações de vendas desse site e colocar todas dentro de outro site.

Part 2 Crawler - Ebay

I'm in need of a web crawler that can extract specific data from a large online ads website. The extracted data should then be neatly exported to a Google Sheet. Key requirements: - The crawler should not be intrusive to the host website. - I want the crawler to crawl current and expired ads based on links present in the site map, as historic data is also of importance to me - You should be able to provide a way for me to run this crawler myself at any time - ideally on a monthly basis. I haven't yet decided on the precise fields to be extracted. I'm looking for someone with the skills to recommend suitable fields that could be beneficial for my project. It's important that the extracted data is organized and easily readable once exporte...

I'm looking for a skilled developer to build a salon appointment aggregator platform. This innovative tool should streamline the booking process for salons, beauty parlors, and stylists. Key Features Required: - Calendar Integration: This platform must allow each salon to manage their calendar and synchronize it with customer bookings. - Customer Reviews and Ratings: A system for customers to provide feedback on their experience is also required. It should enable future patrons to make informed choices about the services they engage. - SMS Notifications: The platform should notify customers of their bookings via SMS. Client Data Collection: In addition to standard contact information, the system should also be capable of capturing client preferences, including: - Phone ...

I'm in need of a talented professional who can help me with a custom web crawler project. Key Requirements: - **Web Crawler Customization:** The ideal candidate will need to adapt an existing web crawler code to include specific features I require. This includes the ability to extract and filter data from multiple websites. - **Data Management:** I expect the total scale of the data to be relatively small, so the freelancer should have experience handling data sets of this size. - **Data Analysis:** After the information is extracted, I intend to use it primarily for data analysis. The freelancer should have a good understanding of how to structure data for this purpose. Ideal Skills & Experience: - Proven experience in customizing web crawlers and managing...

We are seeking a skilled developer or development team to create a comprehensive web crawler system. This system should scrape, structure, and update information from web pages, handle large projects, and be accessible via a powerful API. The system should offer user-friendly features for both individuals and businesses and include flexible user management for both customers and employees. Project Requirements 1. Web Crawling and Data Extraction -Develop a system capable of scraping entire websites and extracting data from each page. -The system should divide the scraped information into sections to facilitate easy reading and understanding. -For e-commerce sites, the system should identify and structure the following information: --Product category (including breadcrumbs for hiera...

I'm in need of a skilled Android developer to create a bespoke news aggregator app specifically focusing on commodity news. The app should have the following features and functionalities: - Aggregation: The app should aggregate news from reputable sources for commodities such as Oil, Gold, and Silver. - Real-time Updates: The news content should be updated hourly, ensuring users have access to the most recent information. - User-friendly Interface: A clean, intuitive interface that ensures easy navigation and a pleasant user experience. - AdMob Integration: The app should support AdMob for monetization purposes. Ideal skills and experience for this project include: - Proficiency in Android app development, especially with a focus on news aggregation. - Experience with integra...

...building designs based on the input of the given land. Additionally, the app should be able to propose interior designs with the available room input. The ideal freelancer for this project will have: - Extensive experience in AI and machine learning - Strong background in mobile and application development - Exceptional design skills Due to the lack of platform preference mentioned initially, the aggregator should be built to support all major platforms: iOS, Android, and Web. Though the specifics for building design and interior design were not provided, the app should at a minimum, generate: For Building Design Based on Land Input: - Floor plans - Exterior design concepts - Considerations for any structural changes For Interior Design Based on Room Input: - Color scheme...

I am looking to drive traffic to my website, , from guest posts and news aggregators. The main goal behind this initiative is to boost the overall revenue and web traffic of the website. Website URL: Key requirements: - The focus of this project is on increasing website traffic and ultimately improving its revenue. - Experience with guest posting and utilizing news aggregators such as Alltop and others is highly beneficial. - An understanding of SEO, both in the context of guest posting and news aggregators, would be an added advantage. - A proven track record of driving revenue through web traffic is a major plus. - Weekly and monthly work report is mandatory. It's important that the freelancer applying for this project has a strategic approach towards driving traffic and unders...

I'm in need of an experienced API developer(s) to assist with integrating our platform with multiple external systems in USA. Plaid API aggregates with multiple finance systems (banks mainly) and their investment aggregation didn't work for me. Since Plaid API didn't work for me I am thinking we will have our own APIs to integrate to any investment banks/ brokers like E*Trade, TD Ameritrade (now part of Schwab(, Vanguard, Fidelity, Robinhood and many more. See Following APIs are needed at high level: - Minimum 1) Get Holdings/ Positions (stocks, bonds, mutual funds, options, ETFs, etc.). We will pass institute name, account # or login/password, 2) Get Transactions (Dividend Paid, Bough, Interest Charged, Fees, Sold, Bought positions etc.) for a given account/ institu...

...news articles but also provides analytical insights into trending topics and sentiment analysis of news content. Objectives: Main Goal: Develop a web application that personalizes news content for users based on their reading habits and preferences. Secondary Goals: Implement data visualization features to display trends and sentiment analysis of news articles. Minimum Requirements: Content Aggregator: Develop a system that pulls news data from various APIs or web scraping. User Profile Management: Create functionality for users to register, login, and specify their interests and preferences. Recommendation Engine: Implement a machine learning model to analyze user behavior and curate news content accordingly. Optional Enhancements: Sentiment Analysis: Integrate a sentiment ana...

I am looking for a Python experienced person capable of creating a web scrapper that can extract text content from Aliexpress and then categorize these extracted data into a Google Sheet in a specific format. Crawler in python format. Thanks. Please provide a sample

I require a unique search engine designed that can effortlessly comb through several car websites (, , , , , , , Facebook Market Place) and display the results in one convenient place. Key functionalities should include: - Advanced filtering options - Price range filtering - Location-based search Ideal expertise would entail experience in advanced web scraping and designing search algorithms, along with a deep understanding of diverse automobile databases and SEO. The candidate should be capable of verifying and testing the accuracy of the collected data as well. Efficiency in coding with an ability to manage site speed and response time is crucial, especially considering the breadth of services it will be searching. The final product should be a user-friendly platform that effectivel...

I'm seeking a data aggregator for a unique AI training project. Key Tasks: - Gathering text content from official website/source of the particular product/entity. - Focusing specifically on product descriptions, specifications, price and other attributes. Ideal Skills: - Basic searching skills - Able to follow given format of data collection - MS Excel or Google Sheets This project requires precision, and the information gathered will contribute to an AI's learning process. Accuracy in aggregation is, therefore, key.

I'm looking for a skilled developer who can create a coupon code aggregator website similar to retailmenot.com. The key feature of this project is the integration of a web crawler that will search and gather coupon codes from various online sources. Key Requirements: - Development of a coupon code aggregator website - Integration of a web crawler that will search for coupon codes on a daily basis - A search bar where users can enter websites like they are browsing normally and coupon codes will be applied similar to how microsoft edge or Honey does it - The website should have the functionality to automatically apply coupon codes to a shopping cart - Ability for users to sign up and receive daily coupon codes and deals - Ability for the website admin to b...

I'm in need of a skilled web developer to create a news story aggregator website similar to www.drudgereport.com. The key feature of this project is the ability to monetize the site through advertising. Key Requirements: - The website should serve as a news story aggregator, displaying multiple links to various news stories. - The ability to sell ad space on the website is crucial. - The ad space should be allocated to both Banner Ads and Sponsored Content. - Easy backend for the website owner to update links to new stories from other websites. Design & Branding: - While I'm open to suggestions for design and branding, the website should maintain a professional and clean look to attract both readers and advertisers. I want someone who has built a news websi...

I'm seeking a talented app developer to create a cross-platform news app, available on both IOS and Android. Key Requirements: - I envision the app being a source of news content, so I need someone with experience in creating news feed or aggregator type applications. - It should be user-friendly, intuitive to navigate and have a strong focus on an engaging user experience. I don't have a specific layout or design in mind. I'm open to your suggestions. This is a great opportunity to showcase your creativity and design skills. Please include in your proposal: - Relevant experience - Estimated timeline for the project - A rough outline of your plan for the app's design and functionality - Any suggestions you have for the app Looking forward to seeing your pro...

i need a backend preferably in nodejs. one endpoint. that submit some birh date data. then the backend should request the html from anotherwebsite, fill its form click on submit get the data and send them

I'm starting an online insurance aggregator that caters to both B2B and B2C markets. I am seeking an experienced branding expert to create a compelling and memorable brand name and logo that aligns with my company's vision and values. - Objectives: - Develop a unique and catchy brand name suitable for the insurance industry. - Design a professional and visually appealing logo reflecting our innovative and trustworthy service. - Ideal Candidate: - Proven experience in brand development and logo design, preferably within the insurance or fintech sectors. - A strong portfolio showcasing previous work. - Ability to translate company values and services into a brand identity. - Excellent communication skills to ensure the brand name and logo align with our vision...

I am seeking a data aggregator to formulate inclusive and accurate contact lists for my upcoming marketing campaigns. The information required in each list is a blend of phone numbers, email addresses, and the names of company owners. I'll be providing SIC codes as an accurate reference channel for the industries we'll be focusing on. Any freelancer applying for this project should exhibit a well-versed understanding of data aggregation, web scraping, and should be detail-oriented. A knowledge or background in marketing would also be beneficial. Please note information accuracy is paramount, as these lists will directly impact the effectiveness of our campaigns. Your tasks will include: - Gathering the aforementioned contact details - Aligning data with SIC codes provi...

I am in need of a skilled developer to create a web-based news aggregator. This aggregator will specifically focus on news from Uganda and the broader East African region. It has to be user-friendly, intuitive, and visually appealing. Key Requirements: - The aggregator should exclusively prioritize text-based news articles. With featured images No video or audio content is to be included. - The platform should support the English language, as it is the primary medium for news in the region. -The platform shall pick news from sites I authorize to avoid fake news The platform should allow reading of articles (1st 3 paragraphs) before referring user to read fulll article at the parent site from where the article originates. The platform should give use ability to read f...

Please read this carefully. I don't have time to consider and then explain the task to people who did not read it to the end. I'm looking for a skilled developer to create a very simple backend service that aggregates notifications from multiple sources, primarily Gmail APIs and a webhook, an estimate of 5-6 sources in total. The system will maintain a rolling window of the last 12 hours of data, displaying these notifications in a real-time feed accessible via a basic web interface. This will only be used by by internal team of four people. I would like to work with an independent developer who I can communicate with directly over whatsapp. I do not want to work with someone who will outsource this to someone else or who has a team working under one account. Project Requi...

I'm looking for an experienced crawler specialist to help me crawl databases, and specifically collect certain datasets. Key Project Requirements: - Crawler Type: The focus for this project will be on databases, not websites or APIs. - Objective: The primary goal is to extract specific datasets from these databases. - Data Format: The extracted data should be delivered in CSV format. Ideal Skills and Experience: - Proficiency in database crawling and data extraction is a must. - Previous experience in collecting specific datasets from databases is highly preferred. - Familiarity with working with CSV data format. Please share relevant experience when bidding.

I am looking to create a medical news aggregator website, integrating AI plugins for enhanced user experience. The platform should: - Primarily focus on consolidating medical news and research articles. - Categorize content automatically for seamless navigation. - Utilize intelligent search functionality to deliver precise results. - Offer personalized recommendations based on user interactions. Our target audience is broad, encompassing medical professionals, the general public, and patients along with their caregivers. The ideal candidate for this project should have proven experience in website development and AI integration, with an understanding of medical terminology being a plus. They should be capable of designing a user-friendly interface that aligns with the needs of o...

...should offer users the ability to preview books before making a purchase. This preview should include a full chapter of the book. - Search functionality should cover key categories: by title, genre, and author. - The app should also be designed with a user-friendly interface and smooth navigation. Ideal Candidate: - Proven experience in mobile app development, especially with e-commerce or aggregator platforms. - Strong knowledge of both iOS and Android development. - Familiarity with integrating e-commerce features like preview options and search filters. - UI/UX design expertise to create a visually appealing and user-friendly interface. - Excellent communication skills and the ability to work collaboratively to achieve project goals. If you're confident in your abili...

I'm seeking a meticulous and experienced professional in Octoparse to construct a responsive web crawling template. This crawler will be solely focused on an eCommerce platform, aiming to capture extensive product information. Key Responsibilities: - Capture specifics like the URL, category, and SKU for every product. Ideal Candidate: - Knowledgeable in Octoparse. - Proven experience in data mining and web crawling. - Detailed understanding of eCommerce platforms and product specifications. - Ability to maintain precision across multiple product categories and large datasets. I look forward to seeing your proposals and discussing this project further with you.

I'm looking for an experienced web crawler developer to build a tool for me. This tool needs to be able to extract text content and links from various websites. I need data from the whale watch section of The full crypto address within the links should be associated with their name counter part in a spreadsheet or organized file. I have included a highlted picture demostrating the fields I need. They should be done for the full 6 pages. Requirements: - Experience: Please detail your experience in web crawling and provide examples of previous work. - Delivery: The extracted data should be delivered in a spreadsheet file. It's crucial for the freelancer to have in-depth knowledge of web scraping and data extraction. The tool should be efficient and robust, capable

I am seeking an expert to build a web crawler specifically suited for extracting a broad range of data from social media platforms. The unique objective of this project is to efficiently and accurately amass textual content, images, links and more from social media sources. Specifically, the project will revolve around: - Gathering user posts and related content - Documenting user interactions, encompassing likes, comments, and shares - Extracting comprehensive details from user profile Ideal candidates for this project will have: - Proven experience in Data Extraction and Web Crawling - Knowhow of Social Media platforms and their structure - Proficiency in programming languages used for web crawling - Excellent attention to detail and a commitment to accuracy. If ...

I am seeking an expert capable of creating a highly detailed and intricate graphic line art rendition of a lattice boom crawler crane with a luffing jib (two independently controlled booms). This rendition must strictly capture the exterior features of the equipment ( there's a lot). Your focus should also be on all the components that make up the crane. I am open to starting with a more basic line art (outline) and seeing what we need to add. This would be the first design for a construction apparel line You need to have a meticulous eye for detail and at least have a portfolio that showcases any automotive art.

I am looking to create a sports news aggregation website. The website will be primarily focused on sports news and will be used as the basis for a radio broadcast. Key Features: - The primary function of this site is to aggregate sports news from a range of different sources. These sources may include major sports news websites, blogs, newspapers and more. This content will form the basis of our daily radio show. - The site should also have a feature where the latest sports news can be read out loud. This is a key feature as the site will be used for our radio broadcasts. - The website should include a section that is dedicated to sports video highlights. This section will be a key engagement tool for our audience. Ideal Skills and Experience: - Experience in creating news aggregation we...