Codez un logiciel

General:

Hello, I am developing a real time price comparison website. So basically, this how it's working: A user is coming on the website to get a valuation of the product, he defines its product thanks to a form and submit its request. This is starting multiple instance of scrapers that go to some external websites (5), define the product in filling forms and finally grab valuation of the product. At the end, data about product and valuation are saved in a DB.

What you have to do:

_ Develop the "scraping system" that goes & grabs valuations of the product on different websites (5). No website integration or design, just back-end devt.

What it's provided:

- A Rails application with an access to a bitbucket repo.

- In this rails application, you have the form to identify the product

Technologies:

_ Environment is a RoR application

_ Use Phantom JS as a browser and Poltergeist as its driver.

_ Use Nokogiri as a parser

_ Use Capybara as a DSL

_ Use Redis & Sidekiq for running multiple thread in the background and handling queue

_ If issues are encountered with phantomJS & Poltergeist, it's okay to use a firefox instance with selenium webdriver

Mandatory:

All the data collection must be performed live, meaning I don't want to scrape anything before a user come to my site and ask for a comparison.

This is an easy project for someone who has done this before. If you have examples of previous work, this will go in your favour so please reference them. Additionally if you have advice on the a better architecture/solution I am open to ideas.

I am okay to pay 400-500 $US to do this.

Data Mining MySQL Ruby on Rails Software Architecture Web Scraping

Project ID: #8680279

About the project