Import 10 million records 2GB .csv (5 parts) every hour to MySQL

Import .csv files from here to MySQL database:

[url removed, login to view]

Must be fast, easy to run, cheap, complete the process in under 5mins. It updates the CSVs' every hour and makes note of the data that changes.

What kind of server/mysql setup is needed on AWS? Let me know your estimations so I can judge how suitable you are for the job.

Skills: Big Data Sales, Excel, MySQL, PHP, Software Architecture

See more: desktop software sort csv file million records, mysql csv import escape, import mysql csv perl, linux shell script mysql csv import, software write mq4, software write chip epson, useful software write book, import text mysql csv, unable add records mysql csv, software write web specs, perl mysql csv import, import csv access update records vbnet, mysql csv export import, vbnet import csv access existing records, free software write user guide, software write edid, free software write company profile, mysql csv import joomla, software write websites idea, mysql million records

About the Employer:
( 32 reviews ) London, United Kingdom

Project ID: #11724502

24 freelancers are bidding on average $165 for this job


I am talented PHP , mySQL ,javescript ,AJAX, WORDpress developer. I have very reasonable rate .I do WORDPRESS PHP , MySQL programming more than 5 years and within this period I had been working as slave [login to view URL], I More

$250 USD in 3 days
(225 Reviews)

A [login to view URL] should would for basic importing and everything . Would need to discuss how you'd be querying the data to get a more accurate guess on what EC2 server would be needed or if you'd need dedicated MySQL. I can More

$250 USD in 3 days
(132 Reviews)

Hello, I am a skilled unix and network engineer with 5 years of experience in unix based systems, virtualization like proxmox,esxi,xen,virsh, iptv,zimbra etc. I like to automate system working with additional shell scr More

$111 USD in 1 day
(75 Reviews)

Hello! I am ready to help you with your task! Do you need an unattended script to do this import job? Please do not hesitate to contact me via messenger!

$250 USD in 20 days
(19 Reviews)

Hello, I have extensive experience with MySQL and importing large datasets on set interval basis. From your information in the project, you mentioned that when the csv's are updated, there are notes of the data tha More

$250 USD in 5 days
(12 Reviews)

Hi, I read carefully your project, and I wanna do it for you. If you wanna work together, send me a message to discuss details. I am an experienced Web Developer and Web Designer, and have worked with wide variety of s More

$150 USD in 11 days
(48 Reviews)
$155 USD in 3 days
(44 Reviews)

there could be easier ways to do this.

$111 USD in 5 days
(1 Review)

hi, i can help you with your project. According to the site, notifies the date of the last update, so I do not see that you need to run a process every hour, but I can do. send me more information about via freelancer More

$333 USD in 5 days
(14 Reviews)

Recent work [login to view URL] (A local service Website) Greetings from InfoRidge Technology!! Dear Employer, Maybe our cover letter is not much powerful as like others but our skills and expertise are overwhelming to o More

$54 USD in 19 days
(14 Reviews)

Hi I have been in this industry for 5 years and such jobs are my daily practice.I can assure you that if you work with me once, you will always work with me for these kind of projects.

$250 USD in 3 days
(5 Reviews)
$155 USD in 10 days
(6 Reviews)

Hello, Thanks for reading my propusal. First of all let me tell you that loading such amount of data won't be fast. Since you need to do it in steps to prevent problems in DB. Secondly I would suggest you to r More

$100 USD in 20 days
(3 Reviews)

Hi There, We can setup aws RDS instance which is highly available and the best performance database for mysql. Initially we would setup micro instance db if the queries are taking time to process then we can upgrade i More

$60 USD in 2 days
(6 Reviews)

I can build this script using Python. I need only credentials of your database (IP, username, password, DB name) and may be an access to your AWS. Because I am not a client of AWS, I can suggest two possible cases of u More

$155 USD in 3 days
(1 Review)

I am an expert of full-stack web development and have similar experience with your requirement. although I am new to here freelancer and earn achievements for future so i will bid reasonable price. My planning is b More

$35 USD in 3 days
(0 Reviews)

I would recommend to use: an rds with mysql or aurora as database with 20.000 provisioned IOps ebs volume (r3 instance family or I family depending on the needs of computing power of the queries). To import the file ca More

$250 USD in 3 days
(0 Reviews)

I will deliver a satisfactory job that will impress you. I crave for a chance to prove to you my worth and abilities.

$100 USD in 2 days
(0 Reviews)

I have the experience in work load of large files with management in record time. Also I have experience handling government databases with extensive confidentiality.

$111 USD in 2 days
(0 Reviews)

Hello. Assuming data integrity is good (fields consistency and no junks) this could be done fast. Standard MySQL setup is sufficient (we are not talking about data access and processing here, right?) as long as the AWS More

$155 USD in 3 days
(0 Reviews)