Senior Data Engineer

Responsibilities

• Work closely with Subject Matter Experts (SMEs) to design and develop the ontology (data model) and data pipelines.
• Design, implement, and maintain robust data pipelines and ETL processes.
• Develop and optimize cloud-based data storage and processing solutions.
• Collaborate with data analysts and data scientists to understand data requirements and deliver high-quality data solutions.
• Ensure data integrity and security across all data workflows and storage solutions.
• Monitor and troubleshoot data pipelines, addressing any issues promptly to ensure the smooth flow of data.
• End to End data pipeline development – responsible for using Python and Pyspark to perform ETL and derive new datasets necessary for business applications.

• Bachelor's Degree in Computer Science, Engineering, or related field or related experience.
• 5+ years of experience as a data engineer.
• Hands‑on experience with building data pipelines.
• Strong SQL skills and experience working with large datasets and complex data structures.
• Proficient in Python, and Pyspark.
• Experience in Typescript (preferred) or Java script.
• Excellent problem‑solving and analytical skills with a strong attention to detail.

• Experience with Palantir Foundry application development and data visualization tools.
• Knowledge with commercial visualization tools such as Tableau or Power BI.
• Experience in Typescript (preferred) or Java script.
• Databases – familiarity with common relational database models and proprietary instantiations, such as SAP, Salesforce, etc.
• Git – knowledge of version control / collaboration workflows and best practices.

Agile – familiarity with agile and iterative working methodology and rapid user feedback gathering concepts.

UX design – knowledge of best practices and applications.

Skills:

pipelines, python, etl, data, cloud, design.