Hadoop Jobs

- **Core Architecture:** Spring Cloud + Kafka + Hadoop + Python Automation，This project requires a certain level of technical expertise.

I have an existing analytics initiative that now needs a dedicated Redshift-based warehouse. The core objective is to design and implement a robust schema in Amazon Redshift, then ingest data coming from three different sources—our operational SQL databases, a set of RESTful APIs, and periodic flat-file drops in CSV or JSON. Here is what I’m aiming for: • A well-structured Redshift warehouse (star or snowflake schema, whichever is most appropriate) built to scale and documented clearly. • Reliable, automated ingestion pipelines for each source type. For SQL we currently use PostgreSQL and MySQL; for APIs the payloads are mostly JSON; the flat files live in S3. • Transformations that standardise data types, handle slowly changing dimensions, and enforce dat...

I need a propensity-modelling software package that plugs directly into our CRM system, website analytics, and sales database, unifying those streams into a clean, continuously updated dataset. On top of that data layer, the build must train, evaluate, and deploy the best-performing predictive models—whether regression, decision-tree, neural-network, or any other technique that proves superior—then surface the results through a lightweight web interface and an API our teams can call in real time. Key deliverables • Automated ETL jobs and data-quality checks for the three sources mentioned above • Modular training pipeline with experiment tracking, lift/ROC reporting, and feature-importance visuals • Scoring service exposed via REST (or GraphQL) endpoints plu...

Key Responsibilities: • Design, develop, and deploy AI/ML solutions end-to-end • Lead AI architecture and solution design for enterprise applications • Build and optimize machine learning and deep learning models • Deploy and monitor models in production environments • Collaborate with cross-functional teams including product and engineering • Mentor junior AI engineers and contribute to technical leadership • Conduct research and implement state-of-the-art AI techniques • Ensure data quality, security, and model performance optimization Required Skills & Qualifications: • 10+ years of experience in AI/ML or Software Engineering roles • Strong proficiency in Python and data processing libraries (NumPy, Pandas) • Hands-on experienc...

Project Description My current production workload runs entirely on AWS EC2, and while usage grows, monthly costs are increasing and application response times are degrading. I’m looking for an experienced AWS / DevOps engineer to perform a data-driven infrastructure audit and recommend optimizations. The engagement will begin with a deep-dive assessment of the existing environment, followed by a cost- and performance-balanced architecture proposal. All findings must be supported with measurable evidence, not assumptions. Phase 1 — Infrastructure Audit The audit should cover: EC2 CPU, memory, and disk I/O utilization Database behavior and performance (MySQL) Network flow and latency analysis CloudWatch metrics and logs review Identification of bottlenecks, ...

**Operating System:** CentOS 7.x (7.9 recommended) **Core Architecture:** Spring Cloud + Kafka + Hadoop + Python Automation **Core Package Name:** ``