Redshift jobs

...Spark, is also crucial. In addition to these core skills, we require expertise in AWS cloud services, particularly AWS Glue and Amazon Kinesis. Experience with AWS Glue will be vital for ETL operations and data integration tasks, while familiarity with Amazon Kinesis is important for real-time data processing applications. Furthermore, the candidate should have a solid understanding of Amazon Redshift, our data warehousing solution, to manage and analyze large datasets efficiently. The Senior Data Engineer will be responsible for designing and implementing scalable data pipelines, ensuring data quality and integrity, and optimizing data processing workflows. The role involves close collaboration with data scientists, analysts, and other stakeholders to support data-driven decis...

I have AWS RDS MySQL database that is of 1TB (1000GB) size. I'm looking for freelancer to migrate it to AWS Redshift. I have ~10 MySQL queries that I want to be re-written for Redshift. I will share all the details. Start your proposal with "Redshift" keyword to be considered.

I am looking for a freelancer to create a document that compares the performance and cost of Redshift serverless and Redshift provisioned. The purpose of this document is to help me make a decision between the two options. I want the comparision in terms of query time, pricing, warm up time and compare with Redshift provisioned and redshift serverless. Specific dimensions that I would like to compare include speed, cost, and scalability. The target audience for this document is technical experts. Ideal skills and experience for this project include: - Strong knowledge of Redshift serverless and Redshift provisioned - Experience in conducting performance and cost comparisons - Ability to present complex technical information in a clear and ...

I am looking for a freelancer who can complete some somewhat simple SQL queries for data analysis. The ideal candidate should have experience in MySQL and be comfortable working with intermediate queries involving JOINS and GROUP BY. This project requires the following skills and experience: - Strong proficiency in SQL - Experience with MySQL or Redshift SQL - Ability to write intermediate queries involving JOINS and GROUP BY If you have the necessary skills and experience, please submit your proposal.

...Timings : 12pm - 9pm IST M - F AWS Data Engineer Requirements • Collaborate with business analysts to understand and gather requirements for existing or new ETL pipelines. • Connect with stakeholders daily to discuss project progress and updates. • Work within an Agile process to deliver projects in a timely and efficient manner. • Have worked extensively on Redshift and understands performance tuning techniques and management of Redshift data workloads. • Design and develop Airflow DAGs to schedule and manage ETL workflows. • Implement best practices for data engineering, including data modeling, data warehousing, and data pipeline architecture. • Monitor and troubleshoot ETL pipelines to ensure smooth operation. • Custom Python f...

I am looking for a freelancer who can help me with a project that involves reading data from a Redshift serverless DB using AWS Glue. The project requirements are as follows: Output format preference: - The extracted data should be in JSON format. Data transformation and cleaning: - No significant cleaning or transformation is required. - I just need the raw data extracted from the Redshift serverless DB. Ideal skills and experience: - Experience working with Redshift serverless DB and AWS Glue. - Experience in python and typescript If you have the necessary skills and experience, please submit your proposal outlining your approach to this project.

I am a professional with advanced knowledge and experience in Redshift. I am interested in furthering my understanding of the subject by delving deeper into the topics of data warehousing and query optimization. I am seeking individual training to best fit my needs.

...following AWS services: EC2, S3, Redshift, and Glue. The main objective of this project is to efficiently store, analyze, and migrate data using AWS services. I expect the project to be completed within 1-3 months. Responsibilities: The chosen candidate will be responsible for the following tasks: - Utilize EC2 for scalable computing power and efficient data processing. - Utilize S3 for secure and reliable data storage. - Utilize Redshift for high-performance data warehousing and analytics. - Utilize Glue for ETL (Extract, Transform, Load) processes to efficiently transform and migrate data. Requirements: The ideal candidate for this project should have the following skills and experience: - Strong expertise in AWS services, specifically EC2, S3, Redshift, and G...

...Observatory Data Repository etc; You are free to use any visual tool that is free for research purposes. Some suggestions include Tableau Public, Google Data Studio,Microsoft Power BI, etc. You may also consider the following visual tools for data extraction, transformation, and loading: you can use any of the following : , Talend Open Studio,MarkLogic,Oracle Data Integrator,Amazon RedShift I will need assistance with data extraction, transforming ,loading and perform exploratory data analysis to understand patterns and trends in the data and Create meaningful visualizations to represent your findings. My desired end result is an Excel spreadsheet or some other file for the data presentation . If you are able to execute this task effectively and timely, I would be very eager to

Hello, I need someone who can provide a support for my aws glue job project. Its mainly involve in: -creating glue job script, -configuring connection with redshift, pgAdmin db, s3 -creating triggers and crawlers -transforming and enriching data -python knowledge is required -testing and debugging experience - creating glue workflow is a plus please reach out if you are interested in providing support. Regards Sanch Adane

I need a Java developer who can create a lambda function to read and write data to a Redshift serverless DB. The purpose of this operation is data backup. Requirements: - Experience with Java and lambda functions - Experience with Redshift serverless DB - Ability to prioritize data types or tables (if specified) - Knowledge of strict security and compliance protocols

Hi there! I am looking for a talented 3D modeller to provide a rough quote for creating a decent quality 3D model of an agricultural machine for use in animations. The reference image attached is the only one I have at the moment but I'm in the process of getting more imagery, dimensions and hopefully diagrams of the object. It doesn't need to be super-...quote for creating a decent quality 3D model of an agricultural machine for use in animations. The reference image attached is the only one I have at the moment but I'm in the process of getting more imagery, dimensions and hopefully diagrams of the object. It doesn't need to be super-photorealistic but should have some basic materials applied that I'll be able to work with in Cinema 4D 2023 and Redshift. L...

Looking for Backend Developer in python who have a Good knowledge in Python, pycharm,Developing Django Application , Deployment and AWS services such as S3, Redshift, Dynamodb, SQL, MySQL etc

Project Title: Expertise in Python (pycharm), AWS s3, redshift, dynamodb, lambdas, Git, Jenkins, and Aurora I am looking for a skilled Python developer with experience in AWS services and Git version control. The ideal candidate should be familiar with the following: AWS Services: - AWS S3 - Redshift - DynamoDB - Lambda - Aurora Git and Jenkins: - Proficient in using Git for version control - Knowledge of Jenkins for continuous integration and deployment The candidate should also be good with Unit testing in Python. If you have the required skills and experience in Python, AWS services, and Git, please apply for this project.

Time : 9am to 1 pm Total duration : 20hr Total 5 days of training Toc/syllabus attached

I am looking for an experienced AWS data engineer who can assist me with Serverless Redshift and PySpark. I do not need help with setting up a system of automation, but I may require assistance with running analytics on the data. The ideal candidate should have experience with the following: - Serverless Redshift - PySpark Skills and experience required for this project: - Strong knowledge of AWS services, particularly Serverless Redshift and PySpark - Experience in data engineering and analytics - Familiarity with S3, Lambda, Boto3, and step functions would be a plus - Ability to work independently and efficiently - Excellent problem-solving and communication skills Working time = 8:30 PM EST to 10:30 PM EST (6 AM IST to 8 AM IST) Duration = 3 to 6 months

We need AWS Data Engineer (Redshift, Talend is must) to work for 2 hours a day remotely. We will give 25-28k per month

Task 1: Create an Infrastructure as Code (in Cloud Formation Python) to spin-up an S3 bucket, 2 EC2 instances, 1 RDS. Also create an auto-scaling capability in when the EC2 CPU usage goes more than 80%. Please demo the code and deploy it in AWS to show it works. Taks 2: Question 1: Create a project in GIT in GIT CI, upload a sample code from Task 1, fork the code and add AWS Redshift service to the code, run the pipeline, merge it back to the main branch. Task 3: Describe with an architecture diagram on how we can migrate an Weather Forecast application installed in EC2 to AWS Faregate in details step by step guide.

Role & Responsibilities: ● Work with cloud engineers and customers to solve for ...developing Big Data solutions (migration, storage, processing) ● Experience in SQL and Query optimisation ● Ability to clearly communicate technical roadmap, challenges and mitigation ● Experience building and supporting large-scale systems in a production environment Technology Stack: ● Cloud Platforms – AWS ● Mandatory – High programming skill in Python and Pyspark, Hands-on experience with the AWS Redshift ● Nice to have - Experience in Bigdata Technologies such as Hive, Spark, Lambda, AWS Cloud with focus on AWS Glue to create ETL pipelines ● Design Dimensional Models for Data Warehouse and Data Marts ● Knowledge of version control systems, build automation and CI/CD tools and fram...

Looking to have ometria rest api set up to connect into redshift database

I am looking for an experienced AWS Redshift expert to help me with data modeling. The ideal candidate should have experience with query optimization and be able to troubleshoot performance issues. This is an ongoing gig, and will be paid as we go. I need initial handholding. Person with good communication skills in English is preferred. Someone that can get onto a phone call for 15 mins sessions is preferred. Occasionally I may have some questions about AWS Redshift implementation.

Looking for a tutor who can help me with Redshift, Oracle, and NoSQL. Specifically, I need the most help with Redshift. My current level of expertise in Redshift, Oracle, and NoSQL is beginner. I require tutoring sessions three times a week or more. Ideal skills and experience for the job include: - Strong understanding of Redshift, Oracle, and NoSQL and experience working with it - In-depth knowledge of Redshift, Oracle, and NoSQL and the ability to explain concepts to a beginner - Familiarity with NoSQL databases - Excellent communication skills and the ability to explain complex concepts in a simple and understandable way - Availability to have tutoring sessions three times a week or more If you have the necessary skills and experience to help me wi...

I am looking for a freelancer with experience in Redshift SQL Query Development to assist me with a Data Analysis project. The ideal candidate should have the following skills and experience: Skills: - Proficient in Redshift SQL Query Development - Experience in Data Analysis Experience: - Previous experience in developing intermediate level queries - Ability to work with complex data sets The project involves: - Developing an intermediate level Redshift SQL query for data analysis - Working with a specific data set provided by the client - Providing insights and recommendations based on the analysis If you have the necessary skills and experience, please apply for this project.

We need a engineer to migrate Teradata to AWS RedShift. Please reach us in case you know both Teradata and RedShift. Thanks.

I'm looking for a freelancer to help with a project that includes database migration. The database I'm working with is a Relational database (MySQL) that needs to be migrated to Redshift. Successful applicants should include evidence of relevant past work in their application. In addition to the database migration, the project will also require extensive SQL coding, use of cloud platforms such as AWS and Redshift, and experience with advanced SQL. Candidates must also have experience with Data Warehousing and an Analytics Platform – the specific platform will depend on the nature of the project. As such, I'm seeking an experienced developer, ideally with a background in database migration who can demonstrate expertise in the tools and skills required. D...

My project is to install an array of VFX software on a Mac OSX system. This includes a number of independent license installers for Houdini, Renderman, Redshift, and Arnold. I need assistance with the setup and installation for these indie licenses on this operating system. I already have the licenses so just need the right kind of help to install the software correctly. This is a small, specific project that requires knowledge of the software, operating system, and ability to manage software installation in order to complete.

AWS Data Engineer with min of 6 to 9 years of experience (JD1) · Collaborate with business analysts to understand and gather requirements for existing or new ETL pip...AWS Data Engineer with min of 5 to 7 years of experience (JD 2) · Experience with AWS ( Glue, Lambda, Appflow, DyanmoDB,Athena, Step function,S3) · Experience with relational SQL and NoSQL databases like Mysql, Postgres, Mongodb and Cassandra. · Experience with data pipeline tools like Airflow, etc. · Experience with AWS cloud services like: EC2, S3, EMR, RDS, Redshift, BigQuery · Experience with stream-processing systems like: Storm, Spark-Streaming, Flink etc. · Experience with object-oriented/object funct...

Looking for Data Engineers having atleast 3 years of experience in end to ETL/ELT, data transformation...platforms can also apply Only candidates who can work in US timezone (EST/CST) apply AWS services such as Glue, Lambda, Athena, S3, SNS, Kinesis, Data-Pipelines, Pyspark, etc. Kafka/Kafka Connect, Spark, Flink or AWS Kinesis Apache Nifi Dataflow Kubernetes AWS Data Pipeline Snowflake GCP tools - GCS, GKE, BigQuery, Cloud SQL, Cloud Connector Golang Airflow Typescript Data: DBT, Fivetran, Redshift, PostgreSQL Infra: GitHub, Bazel, Docker Azure Data Factory Azure Databricks DAX MDX Terraform Visualization tools knowledge - Data Studio, Amplitude, Tableau Languages - Scala, Python, SQL Azure Synapse Mapping Data Flow Interested candidates can whatsapp on +1-81...

...EMR, Hadoop, and AWS services and Pyspark · Proficiency with Data Processing: HDFS, Hive, Spark, Python. · Strong analytic skills related to working with structured, semi structured and unstructured datasets. · Expertise in AWS cloud native services. · Good knowledge of any RDBMS/NoSQL database with strong SQL writing skills · Experience on Datawarehouse tools like Redshift · Experience in Deployment and migration of various workloads to Cloud services from traditional infrastructure or other Clouds. · Strong analytical and problem-solving capability · Excellent verbal and written communications skills · Ability to collaborate effectively across global teams &middo...

Need to understand something on AWS Redshift and hopefully create a small query/report from data we have in AWS Redshift

Hi, I have a simple 3D Geometry .FBX file that I made in Blender that I would like to have a Growth Simulation done in Houdini and rendered in Redshift. The geometry is very similar and nothing heavy, its just basically a decorated torus. You don't need to render it, I can do that (unless it's best if you do it) We can talk about this and how the animation would go! The .fbx file has all the colours and materials already. 240 frames. I can share with you everything if you are interested! Thank you!

Need someone who has good experience in spark,redshift,s3 and aws glue

Data Migration from RDBMS to AWS S3 and Redshift 1. Creating a framework that converts scripting languages like PLSQL, BTEQ etc to Python and PySpark to use Databricks as a compute. 2. A framework that converts the existing RDBMS scripts to Python or PySpark to readily use in AWS databricks compute. Need someone who had done this before or part of this. Should be able to give some used cases on how they implemented it. Main RDBMS being used is Teradata and BTEQ scripting.

Having expertise in AWS CLOUD • Designing and deploying dynamically scalable, available, fault-tolerant, and re...Cloud Watch, Lambda,Quick Sight,Red Shift. • Experience in Automation the AWS resources deployment using IAC (Terraform) • Code writing skills (python)for serverless Lambda • Experience in deploying Open VPN Cloud setup for Security • Monitoring infrastructure health, security using sass application (prowler ,cloud spoit ) • Designed the Dashboard using QuickSight using direct query with RDS & REDSHIFT • Selecting appropriate Cloud services to design and deploy an application based on given requirements • Implementing cost-control strategies • Understanding of application lifecycle management • Understanding in t...

Having expertise in AWS CLOUD • Designing and deploying dynamically scalable, available, fault-tolerant, and re...Cloud Watch, Lambda,Quick Sight,Red Shift. • Experience in Automation the AWS resources deployment using IAC (Terraform) • Code writing skills (python)for serverless Lambda • Experience in deploying Open VPN Cloud setup for Security • Monitoring infrastructure health, security using sass application (prowler ,cloud spoit ) • Designed the Dashboard using QuickSight using direct query with RDS & REDSHIFT • Selecting appropriate Cloud services to design and deploy an application based on given requirements • Implementing cost-control strategies • Understanding of application lifecycle management • Understanding in t...

...creating updates and maintaining the ETL jobs with same technology stack. If you are interested in technological innovations and are still looking for new sources of knowledge, we would like to welcome you on board. Check below what we offer and what we expect. Our requirements: 1. At least 2 to 3 years of relevant experience as Big Data Engineer, Understanding of MongoDB(NoSql database) and redshift database 2. Min 2 years of relevant hands-on Application Development experience into Scala with Spark framework Experience in building modern and scalable REST–based microservices using Scala with Spark framework. 3. Expertise with functional programming using Scala Experience in implementing RESTful web services in Scala, Experience into No SQL/ SQL databases. 4. Should have ...

We are looking for support on Data Engineer (AWS glue, Athena, Redshift, Python and Snowflake). We will give 23-25k per month

...to upload some project for learning purpose to AWS 1- Like create CI/ CD pipelines jekins etc 2- add some security features 3 - how to secure servers with multple staff logins 4 - Teach me how to create EC2 instances and other related concepts with practical 3 - S3 buckets and their policies - cloud formations - beanstalk - cloudfront - kinesis, SQS,SNS - Amazon dynamoDB, and other - Aurora - Redshift and other database practical - clooud watch,cloud trail 4 -Some microservices 5 - Docker containers 6 - and VPC concepts with practical so i i can build my confidence and learn faster as well - And few other services 7 - As i am mostly concentrating on python dont have much time to spend on AWS so with someone help i can make this process faster. Any idea how much would you charge ...

...stack that includes custom web crawlers hosted in AWS EC2 and S3, publishing applications in Snowflake and Redshift, and processing applications in AWS Redshift, AWS Glue, and Snowflake/Snowpipe. We use Sigma for data visualization because it is very easy to develop in, integrates extremely well with Snowflake, and can handle very large datasets with high performance. The application this role will build and run will need to track operations across this entire stack, including monitoring and alerting on operations parameters as well as data continuity at the field level. This position requires a combination of process management and development skills. Strong experience with both Redshift and Snowflake are required, as is experience building python applications...

Outcome expected: Build UI which should allow to select redshift schema (UI might have additional restrictions which schemas can be selected )which will be copied to S3 bucket in other environment. (here we have 2 redshift databases which are in two different env. (for Ex: A1 & A2). which doesn't have direct access to each other. So we should have to copy the schemas from A1 Redshift to A1 s3 bucket and A1 s3 bucket to A2 s3 bucket and then A2 s3 bucket to A2 redshift databases. By click of button we would to be able to initiate copy operation Every operation invocation must create audit record containing who performed operation when it happened, complete details of copy source and approval comments. Unload and Copy operation progress should be vie...

Data modeling for lending business. Loading data from multiple systems into AWS S3 buckets. Finally the data has to be loaded into Amazon Redshift

Hi, We are team of 13 developers, and we are expanding. We are looking for Machine Learning Engineer with 3+ years of experience. Your main role and responsibility is to build an algorithm from scratch or modify existing algorithm for our SaaS Product. This backend work is not a common backend API development. It has complex flow and process to make i...Python and common machine learning frameworks - Has a good mathematical and theoretical understanding of machine learning fundamentals - Has significant experience building and deploying machine learning applications at scale - Has a solid understanding of computer science fundamentals like algorithms You are good at: - Python - Machine Learning - Big data and ETL Pipeline (AWS Redshift) - AWS for Machine Learning ...

Can you create a Azure Data Factory pipeline which reads a parquet file from Blob Storage and writes into Redshift or Synapse or Snowflake Use Azure Databricks for basic Transformation. Blob Storage --> Azure Databricks - > Redshift

Create DaaS using structured data residing on Redshift. DaaS is a collection of template based reports with filters offered in different combinations to several subscription levels.

Need a technical author who has experience in writing on topics like AWS Azure GCP DigitalOcean Heroku Alibaba Linux Unix Windows Server (Active Directory) MySQL PostgreSQL SQL Server Oracle MongoDB Apache Cassandra Couchbase Neo4J DynamoDB Amazon Redshift Azure Synapse Google BigQuery Snowflake SQL Data Modelling ETL tools (Informatica, SSIS, Talend, Azure Data Factory, etc.) Data Pipelines Hadoop framework services (e.g. HDFS, Sqoop, Pig, Hive, Impala, Hbase, Flume, Zookeeper, etc.) Spark (EMR, Databricks etc.) Tableau PowerBI Artificial Intelligence Machine Learning Natural Language Processing Python C++ C# Java Ruby Golang Node.js JavaScript .NET Swift Android Shell scripting Powershell HTML5 AngularJS ReactJS VueJS Django Flask Git CI/CD (Jenkins, Bamboo, TeamCity, Octopus Depl...

--ROLE-- The AWS DevOps Engineer will be working closely with the founders of a startup to design and create an AWS cloud infrastructur...can come into our London office early on in the project to meet the team, that would be a bonus. However, we are also open to fully remote working for the right candidate. --RESPONSIBILITIES-- • Designing and implementing cloud infrastructure • Implementing the CI/CD pipeline preferably with GitHub • Security and performance • Networking --EXPERIENCE REQUIRED-- • AWS resources (RDS, DynamoDB, Redshift, Lambda, API Gateway, Event Bridge, EC2) • Big data infrastructure • Infrastructure as code with Terraform --DESIRABLE EXPERIENCE-- • Data lake and data warehouse --THE COMPANY-- Early stage startup driving...

i created this project (a project builds an **ELT pipeline** that extracts data from **S3**, stages them in **Redshift**, and transforms data into a set of **dimensional tables** for Sparkify analytics team to continue finding insights in what songs their users are listening to) it is very simple and it is all ready and done. but i have one issues: Unable to run Please address the issue noted below. The script, results in the below error: "screen shot". the project is attached

Create a Data pipeline using Airflow ( S3-->Data Bricks--->Redshift ). Composer

Can you create a sample Data Pipeline using Apache Airflow. Source: S3 Target: Redshift

Help needed with updating AWS cloudformation template (yaml) related to RedShift, secrets, glue, lambda, etc.