Find Jobs
Hire Freelancers

PostgreSQL Data Transformation in AWS

$15-25 USD / hour

Closed
Posted 3 months ago

$15-25 USD / hour

Build a glue etl using pyspark to transfer data from mysql to postgres. facing challenges in column mappings between the 2 sources, the target database has datatypes enums and text arrays. should solve the erros in column mappings Should have prior experience ingesting data into postgres enum datatype
Project ID: 37730037

About the project

54 proposals
Remote project
Active 2 mos ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
54 freelancers are bidding on average $22 USD/hour for this job
User Avatar
Hi there,I'm biddin on your project "PostgreSQL Data Transformation in AWS" I have read your project description and i'm an expert in Python and machine learning therefore i can do this project for you perfectly.I still have a few questions. please leave a message on my chat so we can discuss the budget and deadline of the project. Thanks. .. .
$50 USD in 284 days
4.9 (30 reviews)
7.0
7.0
User Avatar
Dear Pushyanthkumar M.,   We carefully studied the description of your project and we can confirm that we understand your needs and are also interested in your project. Our team has the necessary resources to start your project as soon as possible and complete it in a very short time.   We are 25 years in this business and our technical specialists have strong experience in Python, SQL, Amazon Web Services, PostgreSQL, PySpark and other technologies relevant to your project.   Please, review our profile https://www.freelancer.com/u/tangramua where you can find detailed information about our company, our portfolio, and the client's recent reviews.   Please contact us via Freelancer Chat to discuss your project in details or let's appoint time for Zoom meeting to meet and discuss your project in details.    Best regards, Sales department Tangram Canada Inc.   
$25 USD in 5 days
4.3 (58 reviews)
7.5
7.5
User Avatar
Hello there, which version of spark you're running, I faced the similar one with data type because but within timestamp with timezone. Waiting for your response if interested. Thank you!!!
$30 USD in 40 days
5.0 (56 reviews)
6.3
6.3
User Avatar
Hi, I'm a Python Developer on AWS. I have worked on such ETL Glue jobs in the past and I can help you with the error. Please do reach out to me. Thank You
$25 USD in 40 days
5.0 (44 reviews)
5.3
5.3
User Avatar
I will show you my recent projects related to PostgreSQL Data Transformation in AWS then we will move forward. So it's surety for you to get perfect solutions from my side. Also, if you want demo-type things or initial work for your project, then I will show you, and after that, we will finalize our project deal and payment milestones. I am from India, GMT +5:30, and I am available from 8:00 a.m. to 11:00 p.m. We have 16+ years of experience in software development. We have developed over 600 projects and research papers in the fields of machine learning, artificial intelligence, image processing (GIS), network, and SEO-based web and mobile apps. We have successfully completed the projects ChatGPT, Deep Learning, Computer Vision, Natural Language Processing (NLP), Encryption Decryption, Face Detection, UML Diagram, OCR, Big Data, Data Mining, Data Analysis, Statistics, Trading, Text, Image, Multiclass Classification Using Azure ML, Tensorflow, R Programming, OpenCV, Matlab, Hadoop, Artificial Intelligence Program Using PROLOG, Robotics Software, TCP-UDP Networking Project, Cloud Computing, etc. Note: The project has QA, testing, and comments in the code, so it's easy to understand the flow of the project.
$25 USD in 40 days
4.8 (7 reviews)
4.7
4.7
User Avatar
hELLO, gREAT TO MEET YOU. i AM A SENIOR WITH EXTENSIVE EXPERIENCE IN pOSTGREsql DATA TRANSFORMATION IN aws USING pYTHON AND sql. i HAVE SUCCESSFULLY COMPLETED NUMEROUS PROJECTS INVOLVING aMAZON wEB sERVICES AND HAVE A THOROUGH UNDERSTANDING OF ITS CAPABILITIES. wITH MY SKILLS, i CAN QUICKLY TROUBLESHOOT COLUMN MAPPING ISSUES AND CREATE A SEAMLESS etl PROCESS USING gLUE AND pYSPARK. bY UTILIZING MY EXPERTISE, i CAN ENSURE A SMOOTH AND EFFICIENT TRANSFER OF DATA FROM mYsql TO pOSTGREsql. sOME POSSIBLE SOLUTIONS FOR THIS JOB COULD INCLUDE: - uTILIZING aws gLUE'S DYNAMIC FRAME FEATURE TO HANDLE VARIABLE COLUMN MAPPINGS - wRITING CUSTOM sql QUERIES TO ACCURATELY MAP COLUMNS BETWEEN DIFFERENT DATABASES - uSING pYTHON FUNCTIONS TO AUTOMATE DATA TRANSFORMATIONS AND REDUCE MANUAL WORK i AM CONFIDENT IN MY ABILITIES AND WOULD BE THRILLED TO JOIN YOUR TEAM. tHANK YOU FOR CONSIDERING ME. cHEERS.
$25 USD in 40 days
5.0 (1 review)
3.7
3.7
User Avatar
Hi, Thank you for the opportunity to propose data transformation solution using PySpark ETL. As an experienced data engineer skilled in heterogeneous data integration, I believe I can help address the column mapping challenges between MySQL and Postgres for this project. Some relevant qualifications: - Extensive history designing and implementing ETL pipelines to load structured data between disparate systems like RDBMS, Kubernetes, Data Warehouses etc. - Hands on expertise with PySpark SQL for complex data wrangling, transformations, and data type conversions between source and target schemas. - Developed robust error handling strategies to ensure fail-safe data loads despite mismatches in column structures, data types etc. - Knowledge of PostgreSQL arrays, enums would help resolve target schema requirements involving such data types. An initial ETL job would then: 1) Extract data from MySQL 2) Apply transformations for column restructuring, data type conversions 3) Load data into Postgres while resolving errors Subsequent jobs would implement quality checks and handle exception cases. Documentation and unit tests ensure reliability. Please feel free to explore github for relevant code samples. I'm confident my skills can help deliver a production-ready ETL solution to resolve the data integration challenges. Thank you for considering my expertise. Regards, Matt
$20 USD in 40 days
5.0 (6 reviews)
3.7
3.7
User Avatar
Hi, I am a professional web developer and I can do this project PostgreSQL Data Transformation in AWS, I have 5 years of experience in web development. I have done many projects like this. I can do this job for you. I can start right now. Please contact me. Thanks
$15 USD in 2 days
5.0 (1 review)
3.8
3.8
User Avatar
Hello I’ve carefully checked your requirements and really interested in PostgreSQL Data Transformation in AWS. I am confident for your project because I have skills of Amazon Web Services, SQL, PySpark, PostgreSQL and Python. I’m ready to discuss your project and start immediately. Looking forward to hearing you back and discussing all details. Sincerely, Yehor.
$20 USD in 24 days
5.0 (1 review)
3.0
3.0
User Avatar
Hello. Client As an experienced developer who's earned his stripes in the challenging world of AWS, your data transformation project sounds like the perfect fit for my skills and expertise. Throughout my career, I've flexed my Python muscles to develop robust solutions with AWS, and I'm well-versed in SQL, ensuring a seamless migration of your MySQL data to PostgreSQL. Not only have I tackled complex ETLs with Glue and PySpark in the past, but I've also encountered similar challenges with column mappings between diverse data sources. This gives me a sharp eye for identifying and rectifying possibles errors that might arise while migrating your data and puts me in a unique position to solve any mishaps that may occur. If you choose me, I am able to start work immediately and always keep in touch with you and share the project progress. Looking forward to discussing with you more. Best Regards. Daniel
$20 USD in 40 days
5.0 (3 reviews)
3.0
3.0
User Avatar
⭐⭐⭐======Hi Pushyanthkumar M. =====⭐⭐⭐ I am excited to submit my proposal for your project. I have developed a strong set of skills that make me confident in my ability to deliver high-quality work to your project. My approach to any project is to first gain a deep understanding of the client's needs and requirements. I will work closely with you to ensure that I understand your project goals and objectives, and that I am able to deliver results that meet or exceed your expectations. In terms of technical skills, I have extensive experience in SQL, Python, Amazon Web Services, PySpark and PostgreSQL. I'm able to work independently, be detail-oriented, have strong problem-solving ability and excellent communication skills. Overall, I am confident that I can provide the best result beyond your requirements, and help you achieve your project goals. Looking forward to the opportunity to work with you and contribute to the development of the new website. Thank you.
$50 USD in 37 days
5.0 (1 review)
2.6
2.6
User Avatar
Hello, Client. I carefully read your job posting and it caught my attention immediately. I am an experienced full-stack developer who has extensive experiences. This is my main skill and I can finish your project in a short time with High Quality Result and Code. I am able to work full time for your project and will provide you the best product. Best Regards, Igor.
$20 USD in 40 days
5.0 (1 review)
2.4
2.4
User Avatar
I will assist you perfectly with the requirements, let me know when we can connect to discuss this further and get started working on it ASAP. Looking forward to working with you
$20 USD in 40 days
5.0 (2 reviews)
2.3
2.3
User Avatar
Hi there, I bring over 5 years of extensive experience in Python, SQL, AWS, PostgreSQL, and PySpark, specializing in ETL processes. I have a proven track record of successfully transferring data between MySQL and PostgreSQL, adept at resolving challenges in column mappings. My expertise includes handling PostgreSQL enum datatypes and text arrays, ensuring seamless data transformation. I propose a robust Glue ETL solution utilizing PySpark to facilitate smooth data migration while addressing any errors in column mappings effectively. With a meticulous approach and comprehensive understanding of both source and target databases, I am well-equipped to deliver optimal results within your specified requirements. Best Regards, Junaid.K
$25 USD in 40 days
5.0 (2 reviews)
2.4
2.4
User Avatar
I’m Vaibhav a Certified Cloud Solution Expert & Consultant at NexAI Tech as an Advanced AWS Partner. I provide full-stack cloud implementation, migration and managed services to future-proof your IT infrastructure. With extensive experience in AWS, Azure & GCP I am well equipped to efficiently handle this task. My expertise spans advisory, deployment, security, data, networking and innovation on AWS Cloud. Whether it's a troubleshooting complex task issue to debug or to streamline workflows through effective DevOps practices we at NexAI are capable of completing it within the time limit providing fruitful end results. Your task needs someone who is skilled and experienced. This is why I invite you to have a chat where I can assist you with this task. Let’s get started with a quick chat by answering your questions and discussing this project in detail. Looking forward to hearing from you. Best Regards, Vaibhav, Cloud Solution Expert & Consultant, NexAI Tech | Support for Innovation & Growth | India | USA
$20 USD in 40 days
4.9 (3 reviews)
1.5
1.5
User Avatar
I can hep you in migrating data to PostgreSQL databases using AWS DMS, either from another PostgreSQL database or from one of the other supported databases. #Can you share the AWS server details on which we have to work ?...I am ready to help you with this right now… My skills ? Hands-on experience of the Linux server with great troubleshooting and optimization skill. Working experience on Apache, MySQL, PHP, NodeJS, PHP-FPM, ModSecurity, Fail2Ban setup operations. Exposure on AWS services eg. EC2, VPC, RDS, CloudWatch, S3, etc. Good knowledge of WHM, cPanel, Plesk control panel. SSL setup for Apache, Tomcat, and Nginx. Database setup and troubleshooting knowledge - e.g. MySQL, PostgreSQL, MongoDB. I look forward to hearing from you.
$15 USD in 40 days
5.0 (2 reviews)
0.9
0.9
User Avatar
Greetings, I am AWS Data Engineer and I all the required skill set for your requirement. I will provide 100% refund if you are not happy with solution.
$20 USD in 40 days
5.0 (1 review)
1.0
1.0
User Avatar
Hello! I am very confident about your job and I can finish it ASAP. Let's start now. thanks Gene.
$20 USD in 40 days
1.0 (1 review)
1.6
1.6
User Avatar
Greetings, Having carefully reviewed your project description, I am confident in my ability to execute this project to perfection. I possess a broad spectrum of skills, knowledge, and experience in this specific field, making me the ideal candidate to handle your project. My proficiency includesPySpark, SQL, PostgreSQL, Python and Amazon Web Services, which positions me as the best choice for the successful completion of your project. While I am well-prepared to begin, I have a few clarifying questions. Kindly drop me a message in the chat so that we can engage in a discussion regarding the project's budget and deadline. Thank you, and I look forward to the opportunity to collaborate on your project.
$15 USD in 27 days
0.0 (0 reviews)
0.0
0.0
User Avatar
I am aPostgreSQL, PySpark, Python, Amazon Web Services and SQL Deep Learning Specialist with extensive experience in PyTorch and network filter pruning. I specialize in feature map contribution and can deliver high-quality code within your tight deadline. I am serious about meeting your requirements and ensuring the success of your project. Let's discuss further details and costs.
$50 USD in 39 days
0.0 (0 reviews)
0.0
0.0

About the client

Flag of UNITED STATES
Saint Paul, United States
4.9
8
Payment method verified
Member since Jun 7, 2019

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.