Find Jobs
Hire Freelancers

Spark Scla

$2-8 USD / hour

Closed
Posted almost 3 years ago

$2-8 USD / hour

Expertise in designing and deployment of Hadoop Cluster and different analytical tools including Pig, Hive, HBase, Sqoop, Kafka Spark with Cloudera distribution. Working on a live 20 nodes Hadoop cluster running on CDH4.4. Working with highly unstructured and semi structured data of 40 TB in size (120 TB with replication factor of 3) Managing external tables in Hive for optimized performance. Very good understanding of Partitions and Bucketing in Hive Developed Spark scripts using Scala as per the requirement using Spark 1.5 framework. Using Spark API’s over Cloudera Hadoop Yarn to perform analytics on data used for Hive stored at HDFS. Developed Scala Scripts, UDFs using both Data frames/SQL and RDD in Spark for data aggregation, queries and writing data back onto HDFS. Exploring Spark to improve the performance and optimization of the existing algorithms in Hadoop using Spark context, Spark data frames, pair RDDs, double RDDs and Yarn. Developed Spark code and Spark-SQL/Streaming for faster testing and processing of data. Experience in deploying data from various sources into HDFS and facilitating report building on top of it as per the business requirement. Performed transformations, cleaning, standardization and filtering of data using Spark Scala/Python and loaded the final required data to HDFS. Load the data into Spark immutable RDDs and perform in-memory computation to generate quick and better response. Analyzing how the data been processed by Informatica can be effectively processed using Spark and its API’s.
Project ID: 30636255

About the project

6 proposals
Remote project
Active 3 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
6 freelancers are bidding on average $13 USD/hour for this job
User Avatar
Hi, I am an experienced Data Engineer with a solid background in Spark. I have worked on many Big Data projects with Spark, Scala, Python, Cassandra, Snowflake, AWS ,... Let's have a call for more details about the project. Regards
$8 USD in 30 days
5.0 (2 reviews)
2.9
2.9
User Avatar
Hi, I have 6 years of experience and my entire work experience is on these technologies. I think we can discuss more about this
$15 USD in 40 days
5.0 (1 review)
0.8
0.8
User Avatar
Having totally 10 years of IT experience which includes 6+ years in java and 3 years in big data. I have great experience in data engineering projects, hadoop, Hbase, Spark, Scala, Hive, Impala, Pig, and Big data projects using python. I am great fit for this role because as a professional Data Science Specialist, my goals are to help you by using my expertise. I am professional Big Data Expert at Geeky Bug. I have 3 years’ experience in this field. I have solved dozens of Big Data and Hadoop issues. I always consider task as a new Challenge to me and give my 100% to solve this. I can help you with more enhancement and many other effective suggestions. If we were to get started, here’s how I would approach this project: • Chat with you to discuss more requirements and main points. • Prioritize the above to get as more effective suggestions to make your work easier. • I will provide you efficient solution as much as possible.
$5 USD in 40 days
0.0 (0 reviews)
0.0
0.0
User Avatar
3 years of experience in Big Data Domain. Having experience in HDFS, Hive, Spark, Scala, Pyspark, Sqoop. Working in Capgemini.
$5 USD in 40 days
0.0 (0 reviews)
0.0
0.0
User Avatar
As I am A meticulous and goal-driven Hadoop& Spark Developer with 2+ years of experience as an Hadoop Developer in Bigdata Solutions team. Adept at leveraging Hadoop and Spark for processing Large insights. Proven track record of reducing processing time by 50%. Multi-functional team-player .A thorough professional with a proactive attitude, capable of thinking in & out of the box and deftness in imparting product and process training.
$5 USD in 40 days
0.0 (0 reviews)
0.0
0.0

About the client

Flag of UNITED STATES
Queens, United States
0.0
0
Member since Aug 20, 2020

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.