Senior Data Engineer, Data Fabric

תאור משרה

We are looking for

A Senior Data Engineer to take technical leadership in building and evolving the data fabric within our Military AI program
You will design and implement scalable data architectures, structure contextual data pipelines, and automate processes across big data environments. This is a unique opportunity to lead the technical design and innovation of advanced data infrastructure driving intelligence and operational decision-making

Main Responsibilities

Build smart logic data pipelines for processing, manipulating, and deriving insights from data
Collaborate with data stewards to define and build metadata schemes
Implement Data Functions as a Service (FAAS) and workflows including metadata generation, auto extraction, validation procedures, and format conversions
Develop tools for audit, observability, and compliance analytics
Analyze raw data, organize, and prepare data for prescriptive and predictive modeling
Conduct complex data analysis and report results
Enhance data quality, reliability, and manage data fusion from multiple sources
Develop analytical tools, algorithms, and data programs
Provide technical leadership of data structuring for contextual pipelines

Requirements

M.Sc. in Computer Science or Data Science
5+ years of hands-on experience as a lead Big Data Engineer – must
Proven ability to develop and deploy Big Data & AI solutions using Apache Nifi, Kafka, Spark, Graph DB, NoSQL DB, and Airflow
Experience with Microservices architecture and S3 object storage – must
Strong coding skills in Python and Bash – must
Experience with metadata management and taxonomy for big data structures – advantage
Experience with public cloud Big Data & AI services – advantage
Familiarity with machine learning concepts – advantage
Excellent technical skills
Independent hands-on developer with a growth mindset
Team player, open-minded, and eager to learn