Find Jobs
Hire Freelancers

Asynchronous Web Scraping and Data Aggregator

$250-750 CAD

Closed
Posted almost 4 years ago

$250-750 CAD

Paid on delivery
OVERVIEW The objective of this project is to aggregate data from various websites related to real estate listings and property info. The results should be aggregated into a single CSV which can be easily viewed in Excel. REQUIREMENTS 1. The program should be configurable to limit results based on price, property type, and city including its surrounding area 2. Individual websites should have ability to be enabled/disabled in the configuration settings 3. Datapoints which are scraped must be normalized and/or converted to the same units of measurement so they can be easily and meaningfully sorted in Excel 4. Furthermore, the program must be able to send asynchronous requests to the websites to speed up execution time 5. All code must be thoroughly commented and easy to understand for future modifications and debugging 6. Code must also be robust to continue working even if minor website layout changes occur 7. Program must also produce meaningful logs to debug and troubleshoot should problems arise in the future 8. Upon completion of the project, an instructional video (screen recording with captions and/or audio narration) and/or detailed instruction manual should be provided to demonstrate the key features and functionality of how the program works as well as the code’s structure Please see attached document for detailed project description including datapoints and sources to scrape. Please answer the following questions when submitting a proposal: 1.) Do you have any questions about the job description? 2.) Do you have suggestions to make this project run successfully? How do you plan on improving the execution speed when scrapping thousands of listings from several websites? 3.) What past project or job have you had that is most like this one and why? 4.) Which of the required job skills do you feel you are strongest at? 5.) What programming language, libraries, and frameworks do you plan on using for this project and why do you choose them VS available alternatives?
Project ID: 25853299

About the project

10 proposals
Remote project
Active 4 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
10 freelancers are bidding on average $753 CAD for this job
User Avatar
Hello. As a web scraping and data mining expert by python and node.js, selenium web driver, I am glad to place the bid on your project. I have experienced LinkedIn profile scraping, amazon products scraping and ticket pricing tracking and so on. I want to discuss more via chat. Regards. Vladimir
$1,000 CAD in 5 days
4.8 (33 reviews)
5.9
5.9
User Avatar
Hello there, I have gone through your requirement and understood. I have few things to ask but let me answer first what you asked in description.. 1.) Do you have any questions about the job description? Yes.. Are you looking for any kind of userform where user to select price range, property type and city selection as a input for listed websites..? what do you mean log? 2.) Do you have suggestions to make this project run successfully? How do you plan on improving the execution speed when scrapping thousands of listings from several websites? Yes, as per my 11 years of experience I know ways where we can improve performance or run scripts simultaneously. 3.) What past project or job have you had that is most like this one and why? I have scraped data from retailer, real estate, yellowpages (for multiple business) websites and lots more 4.) Which of the required job skills do you feel you are strongest at? web scraping and web automations 5.) What programming language, libraries, and frameworks do you plan on using for this project and why do you choose them VS available alternatives? Python, Scrapy, selenium (but will try to avoid selenium as it slow down scraping performance) NOTE : approx 10 websites with 40+ data fields to bring under one umbrella and more analysis, user interface... this will impact 10 days timelines Thanks and Regards Sunil
$850 CAD in 10 days
5.0 (7 reviews)
4.5
4.5
User Avatar
hi i am a fullstack web and application developer. i have about 9 years of experience and most of my activity in developing extensiosn and templates has been for cms.
$300 CAD in 12 days
4.9 (2 reviews)
2.5
2.5
User Avatar
Hi, Hope you are doing good. I have gone through your requirement and I do have skill set you are looking for. I am certified advanced RPA and QA Automation Professional. I have 2+ year of experience with Automation anywhere and successfully delivered 6 bot which are live up and running in production. I also have 10+ year of expertise in Test Automation framewrok designing with Selenium+ java , UFT + VB script and Katalon Studio. I do have very good exposure working with Web,Rest and Soap service, and also desktop based application automation. I also have hands on BDD-Cucumber,Elastic Search,Excel macros, GIT, Gradle,CI CD tools Jenkins,Cloud testing with saucelabs. We can connect and discuss more about your requirements and come up with best solutions. looking forward to working with you. Regards, Anklesh Singh
$677 CAD in 7 days
5.0 (2 reviews)
2.7
2.7
User Avatar
Hi, Your posting is very interesting. I have 8 years of experience in web development field. I have worked on several similar projects scraping email address and phone number and specific data from websites with python scrapy, selenium, BS4, pandas and can deliver quality result to tight deadlines. Looking forward to connecting to discuss. Thanks.
$500 CAD in 7 days
5.0 (3 reviews)
2.0
2.0
User Avatar
Hi I can answer the question. 1) I have some question. - Do you want to GUI scraping program? - Do you want to scrap all of that websites mentioned on attached file at the same time? - What is the meaning of meaningful logs mentioned 7th of REQUIREMENTS lists? 2)Yes i have , I have rich experience with web scraping and now i am developing Real estate websites. So i know about that more exactly. To improve excution speed i will use multi thread. 3)I have postal code scraping project, factory scraping project, and so on. 4)it is selenium, i have rich experience with selenium library. 5)I will do it with python, python is the most easiest programming language to develop scraping language. because that language provides so many scraping libs such as scrapy, urllib, selenium, beautifulsoup, lxml, and so on. I will wait your reply. Best regards.
$750 CAD in 8 days
5.0 (1 review)
0.5
0.5

About the client

Flag of CANADA
Edmonton, Canada
0.0
0
Payment method verified
Member since Feb 18, 2018

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.