Data Engineer
Hunter's Hub Incorporated
• Implement request for ingestion, creation, and preparation of data sources
• Develop and execute jobs to import data periodically/ (near) real-time from an external source
• Setup a streaming data source to ingest data into the platform
• Delivers data sourcing approach and data sets for analysis, with activities including data staging, ETL, data quality, and archiving
• Design a solution architecture on both On-premises and Cloud platforms to meet business, technical and user requirements
• Profile source data and validate fit-for-purpose
• Works with Delivery lead and Solution Architect to agree pragmatic means of data provision to support use cases
• Understands and documents end user usage models and requirements
• Bachelor’s degree in maths, statistics computer science, information management, finance or economics
• At least 3 years’ experience integrating data into analytical platforms using patterns like API, files, XML, json, flatfiles, Hadoop file formats, and Cloud file formats.
• Experience in ingestion technologies (e.g. sqoop, nifi, flume), processing technologies (Spark/Scala) and storage (e.g. HDFS, HBase, Hive) are essential
• Experience in designing and building data pipelines using Cloud platform solutions and native tools.
• Experience in Python, JVM-compatible languages, use of CICD tools like Jenkins, Bitbucket, Nexus, Sonarqube
• Experience in data profiling, source-target mappings, ETL development, SQL optimisation, testing and implementation.
• Expertise in streaming frameworks (Kafka/Spark Streaming/Storm) essential
• Experience managing structured and unstructured data types
• Experience in requirements engineering, solution architecture, design, and development / deployment
• Experience in creating big data or analytics IT solution
• Track record of implementing databases and data access middleware and high-volume batch and (near) real-time processing
- ...Job Description: Design, develop, and maintain data integration and ETL processes using IBM DataStage based on business and technical... ...Qualifications: At least 3–5 years of experience in Data Engineering, ETL Development, or Data Integration. Strong hands-on...
- ...This position designs data products and data pipelines that are resilient to change, modular, flexible, scalable, reusable, and cost effective. Utilizes software engineering principles to deploy and maintain fully automated data transformation pipelines that combine a...
- ...your skills will evolve at the frontier of innovation and AI, ensuring continuous growth and development. We are looking for a Data Engineer with skills in cloud platforms (GCP/AWS/Azure), SQL and a programming language (python ideally). We look for people who...
- ...Data Technical Lead plays a pivotal role within our evolving Data organization, leading the solution design, development, and strategic... ...using AWS core data services, driving innovation in Data Engineering, Governance, Integration, and Virtualization. Oversee all technical...
- ...As a Data Platform Engineer, you will engage in the design and implementation of data platform solutions. Your typical day will involve collaborating with various teams to ensure seamless integration of data systems, contributing to the overall architecture, and addressing...
- • Implement request for ingestion, creation, and preparation of data sources • Develop and execute jobs to import data periodically/ (near) real-time from an external source • Setup a streaming data source to ingest data into the platform • Delivers data sourcing approach...
- ...Job Overview We are looking for a highly skilled Senior Data Engineer to design, build, and maintain scalable data pipelines and architectures. You will play a key role in transforming raw data into actionable insights, supporting data-driven decision-making across...
- ...The Data Architect strategically leads the creation and maintenance of an organization's master data blueprint. This role defines data structure, governance, and integration across systems, ensuring effective use for business goals and stakeholder needs. RESPONSIBILITIES...
- ...applying best practices and the spark configuration balancing. Conversion and migration of jobs from one platform to another big data platform. Integration of a job from its ingested source transforming it into various targets like databases, extracts files, and...
- ...Requirements: # Experience: Proven experience as a Data Engineer or similar role with a focus on data pipelines, cloud infrastructure, and reporting. # Technical Skills: Strong understanding of data engineering concepts, tools, and technologies (e.g., SQL, Python...
- ...Thinking Machines is a technology consultancy building AI & data platforms to solve high-impact problems for our clients. Our vision... ...Data science ain’t data science without data. As an Engineering Consultant (Data Engineering), you will collect and consolidate...
- ...Hiring: Azure Data Engineer Location: Quezon City, Metro Manila Work Setup: Hybrid (2 WFH and 3 Onsite) Shift: Night Shift Employment Type: Full-Time Qualifications: * 5+ years of Data Engineering experience * Strong experience with Azure Data Factory...
- ...We’re looking for passionate and experienced Data Engineering Specialists who thrive on solving complex problems and unlocking the full potential of data for our customers. If you have a strong technical background in AI, machine learning, and cloud-based analytics platforms...
- ...immediate team and across multiple teams. Develop and optimize data pipelines to ensure efficient data flow and processing.... ...Familiarity with cloud platforms and services related to data engineering. Proficient in data warehousing concepts and practices....
170000 Php
...passport holder) Work Setup: Hybrid: 3 days onsite (BGC), 2 days remote The Qlik Sense Specialist will be part of the Data Engineering team, responsible for designing, building, and maintaining interactive dashboards and analytical solutions that empower business...- ...Work closely with the delivery teams, Google, and customer engineering teams to: Understand and use repeatable and consistent technology... ...or fixes as needed to resolve any issues relating to their data platform. Investigate issues in and maintain data models to...
- ...jti.com Professional Area: Information Technology / Data & Analytics Role: Permanent Professional Level:... ...Reporting to: D&IT Data Team Manager Data Engineer Manager As a Data Engineer Manager , you will design,...
- ...access controls, including role-based access control (RBAC), secure data sharing, and data governance policies to protect sensitive data... ...data integrity and availability. • Collaborate with data engineers to integrate Snowflake with external data sources, data lakes,...
- ...Result Areas) ~ Work with business subject matter experts and data developers to create data testing strategies ~ Develop automated... ...timeliness, integrity, and accuracy ~ Collaborate with data engineers, application developers, business owners, business subject...
- ...possibilities for the communities we live in. Our Team The Data and Analytics team are a progressive group of diverse and... ...ecosystem. About the Role As a Senior Data & Analytics Consultant/Engineer, you will lead the technical delivery of large-scale data...
775000 $ per day
Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality... ...and deploy data across systems. Summary: As a Data Engineer, you will design, develop, and maintain data solutions that facilitate...- ...Data Engineer (Azure Databricks & Spark) Work Setup: Onsite – Taguig, Philippines Industry: Multinational Company (MNC) Employment Type: Permanent Only for local candidates in the Philippines Arbeit Group is currently looking for an experienced Data Engineer...
- ...Join us and build an exceptional experience for yourself, and a better working world for all. EY-GDS Consulting – Senior Data Engineer The opportunity We’re looking for a Cloud Senior Data Engineer to join the EY GDS Data Engineering Team. This is a fantastic...
- ...Job Title: Data Platform Engineer The tasks/responsibilities that the role performs to address requirements in Key Result Areas: ● Implement and manage tools and applications that enhance productivity and user experience on enterprise data platforms such as data...
- ...Key Responsibilities ● Develop and maintain data ingestion pipelines in Azure Databricks. ● Perform data transformation and preparation... ...Qualifications ● 4–5 years of relevant experience in data engineering, cloud data platforms, or similar roles. ● Strong hands-on...
775000 $ per day
Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality... ...Position Overview We are seeking a highly skilled Senior Data Engineer to design, build, and optimize our next-generation data platform...775000 $ per day
Assists with the data platform blueprint and design, encompassing the relevant data platform components. Collaborates with the Integration... ...and data models. #LI-PH Summary: As a Data Platform Engineer, you will spend your day supporting the development and design...- ...We are looking for a skilled Data Engineer to design, build, and manage our data infrastructure. You will be responsible for constructing robust data pipelines that collect, process, and store data from various sources, enabling data-driven decision-making for our business...
- ...About the Role We are seeking a Data Engineer – Snowflake (Associate) to join a managed services team supporting enterprise data platforms. In this role, you will develop, optimize, and maintain scalable Snowflake-based data solutions while ensuring platform reliability...
- ...Environmental Conditions Office Job Description Summarized Purpose: We are offering an opportunity for a Mid-Level Data Engineer to design, build, test, tune, and support production data pipelines using PySpark, Python, advanced SQL, AWS data services,...Remote job
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Engineer. Be the first to apply!
