Python Scrapy Expert

Web scraping is now an essential element in the race to collect ever-changing data in real time and speed is of the essence! In this task we would like to test your ability to write code that will enable us to harvest specific information from a website in a structured manner, as quickly as possible. Here we go! Plane tickets for the same route (frustratingly) fluctuate hour by hour, meaning a flight booked in the morning could cost you twice as much if booked a few hours later. For the budget-conscious traveller, this can mean the difference between going or staying! Using the website Trip Advisor, we would like you to write software code to help us identify the lowest price for a return, direct economy flight from London to New York, departing on 01/05/19 and returning on 10/05/19. It should search for flights with all airlines offering the London – New York route. The scraper should run for a total or 12 hours and provide an update each hour, saved into CSV. DETAILS You must use programming language Python and Scrapy framework to complete this task YOUR DELIVERABLES For this small project, we expect: An actual small scraper in the form of software code To receive your code within 48 hours Your sample CSV output from running the system How quickly you are able to complete this trial task will greatly influence our assessment. We need our freelancer to react quickly to our needs so will be looking for this from the out start. SELECTION PROCESS We will: Review the paid submissions and choose the winning candidate Open a per-hour job for the initial web scraping project Assess opportunity for long-term work as our partner. Please accept this initial challenge only if you feel comfortable working with Python and Scrapy and have experience in web scraping. This is where we will have a lot of future work to do! This could and hopefully will eventually over time turn into a long-term commitment on a recurring/retainer model.