Senior Data Engineer

We are looking for Senior Data Engineer to join our innovative COMBATAI program

The program aim is to produce world-class capabilities for structured & un-structured data, AI lifecycle management that will introduce a new way to empower value from data & AI

We're looking for a skilled Data Engineer to join our team. You'll work with bid data lakehouse & warehouse, focusing on Data pipeline automation methods. Your role involves technical leading of data structure for contextual pipelines, and SQL knowledge 

is a plus

 

:Key Responsibilities

Build smart logic data pipelines for processing data, manipulating, and deriving insights from data

Collaborate with data stewards to define and build metadata schemes

Implement data FAAS and workflows for example: metadata generator, metadata auto-extraction, validation procedures according to schemes and taxonomy, format conversions and more

Develop tools for audit, observability, and compliance analytics

Date feature Analyzing raw data and organizing it for Interpreting trends and patterns

Conducting complex data analysis and reporting on results

Preparing data for prescriptive and predictive modeling

Exploring ways to enhance data quality and reliability, data fusion from different sources

Developing analytical tools, algorithms and programs

Collaborating with data scientists and architects on several projects

 

 

:Requirements

BSc in computer science or Data science

Five years of hands-on experience as lead big data engineer

Experience in developed and deployed Big Data & AI solutions with tools and frameworks such as Apache Nifi, Kafka, Spark, Graph DB, NoSQL DB, Airflow, microservices, S3 object storage

Coding: Python, Bash

 

:Advantages

Experience with concepts of managing metadata & Taxonomy beside big data objects

Experience with public cloud Big Data & AI services

Familiar with machine learning concepts