Scrape ads, modify them and send them back to the original server

Hi there,

we need a script on our server that fetches ads from other sites, makes them static html elements (an image or text with a link) and sends them back to the users of these sites asynchronously.

You will have to use a headless browser / webkit like phantomJS to do so. It has to be very fast and be able to handle millions of queries. The environment is amazon ec2.

We have access to both our server and to the source code of the website that is beeing queried.

Let's call our Server A and the other site B.

Example of one query:

1) Some human on this planet opens B in his/her browser

2) The PHP or JS code embedded in B sends a signal to A that it should start and provides the exactl URL

3) A uses a headless browser to load the url provided. It does not just scrape the source code, it renders the page (meaning JS will be executed)

4) A downloads the ads in the page as static text or static images

5) A now sends the ads back to the B site where they're asynchronously added to the page the user has already opened

Through this process, a dynamic (javascript) ad becomes a static html element.

Notes:

-- we need to decide if we need caching based on performance data. If we need it, it will be very simple, skip steps 1-4 and just send the cached ads

-- there will be only 2 types of ads, so the "ad-detection" will be simple

-- I'm a junior coder and can assist you to the best of my ability.

Looking forwards to your bid!

AJAX JavaScript Node.js PHP Software Architecture

Project ID: #8464376

About the project