Find Jobs
Hire Freelancers

Write Python script to import 1.8M rows of CSV data

$10-30 USD

Completed
Posted over 4 years ago

$10-30 USD

Paid on delivery
I have the following table in a MySQL 2nd Gen 5.7 InnoDB : CREATE TABLE [login to view URL] ( id INT(11) NOT NULL AUTO_INCREMENT, name VARCHAR(256) DEFAULT NULL, source VARCHAR(64) DEFAULT NULL, price INT(11) NOT NULL, year INT(11) NOT NULL, make VARCHAR(64) NOT NULL DEFAULT '', model VARCHAR(64) NOT NULL DEFAULT '', mileage INT(11) NOT NULL, transmission VARCHAR(64) DEFAULT NULL, num_cylinders INT(11) DEFAULT NULL, drive_type VARCHAR(64) DEFAULT NULL, body_type VARCHAR(64) DEFAULT NULL, fuel_type VARCHAR(64) DEFAULT 'Other', title_type VARCHAR(64) DEFAULT NULL, vin VARCHAR(64) DEFAULT NULL, trim VARCHAR(64) DEFAULT NULL, color VARCHAR(128) DEFAULT NULL, location VARCHAR(256) DEFAULT NULL, source_id VARCHAR(24) DEFAULT '', url VARCHAR(512) DEFAULT '', date_listed DATETIME DEFAULT NULL, date_found DATETIME DEFAULT NULL, date_updated DATETIME DEFAULT NULL, num_doors INT(11) DEFAULT NULL, date_analyzed DATETIME DEFAULT NULL, seller_type VARCHAR(15) DEFAULT NULL, details LONGTEXT DEFAULT NULL, PRIMARY KEY (id) ) I have a CSV file of 1.8 million rows that I need to have inserted into this database. I need a Python script to do this for me. The CSV has the following columns: name, year, make, model, trim, mileage, price, transmission, num_cylinders, drive_type, fuel_type, body_type, color, location, url, sp_url, date_posted, date_found, vin, title_type, source The sp_url column from the CSV should not be imported. All other columns from the CSV need to be imported. The database columns num_doors, date_analyzed, seller_type, details, and source_id should all import NULL. Some of the fields have double quotes " in them. These need to be removed. Some of the fields are empty / blank. These need to be imported as NULL. I could not attach the CSV file here because it is too large (700 MB).
Project ID: 20699368

About the project

10 proposals
Remote project
Active 5 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
Awarded to:
User Avatar
I can provide you a simple Python script that will import your CSV into MySQL database. All I need to start is a sample CSV from you.
$30 USD in 1 day
4.9 (536 reviews)
7.7
7.7
10 freelancers are bidding on average $32 USD for this job
User Avatar
Nice to meet you. Very interested in your project. I have good talents and skills of python and i can import data from csv. We can share more details of work via chat. I will try to my best for work. Fast speed, Best quality, Delivering on time is my business style. Thanks. Fang G.
$50 USD in 1 day
4.9 (139 reviews)
7.1
7.1
User Avatar
Hello How are you I have checked your description I am a python expert and that is not a problem for me to edit python script with 1.8M rows data . It is the best proper for me to work on this project , so i am sure I can complete this in time i have full time and I can start to work immediately Please contact me and do let us discuss about your project Thanks for your posting Xu Han
$25 USD in 1 day
4.9 (91 reviews)
6.6
6.6
User Avatar
Hi, Nice to meet you. With 6+ years of experience in Web development, I developed a lot of big and small websites using HTML & PHP(Laravel, Codeigniter, Yii) & Python(Django, Flask) & JS(Angular, Node, React, Vue) & CSS(Bootstrap) frameworks & DB(MySQL, Mongo, Oracle) & CMS platforms(WordPress, Drupal, Joomla) & Ecommerce platforms(Shopify, Magento, Wix) & CRM & APIs. You can check my skills and experiences in my profile. https://www.freelancer.com/u/pandaios Currently, My timezone is very flexible for this project. so I can complete this on time as you want. If you hire me, I will do my best and you will get satisfied with me. Hope to discuss more details via chat. Best regards.
$20 USD in 7 days
5.0 (8 reviews)
4.0
4.0
User Avatar
i have experience in working with database and csv in python of 10mn rows, I can achieve thos task easily.
$40 USD in 3 days
0.0 (0 reviews)
0.0
0.0
User Avatar
Hi, I have a good typing speed and can perform your task with ease. I have gone through your details already and can perform your task on priority basis. please share details once.
$20 USD in 7 days
0.0 (0 reviews)
0.0
0.0
User Avatar
Hi, I can process the file with a python3 script, with or without sql framework like pewee or sqlalchemy.
$35 USD in 2 days
0.0 (0 reviews)
0.0
0.0
User Avatar
Hi.. This is pretty straight forward. Probably the most time consuming thing will be ensuring that the db module (and other pip installs) are easy to install on your system (with what version of python?) prior to running the script.. and documenting command line arguments so that you can add your database server, username and password when you run it.
$20 USD in 1 day
0.0 (0 reviews)
0.0
0.0
User Avatar
Hi, I already made it. Do you need any kind of validation in the CSV fields? My other question is if you will send the CSV and do you expect just the script or the database to be imported? Let me know.
$30 USD in 7 days
0.0 (0 reviews)
0.0
0.0

About the client

Flag of UNITED STATES
Riverton, United States
5.0
3
Payment method verified
Member since Jul 2, 2019

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.