Data Engineer-Onshore

The Senior Data Engineer plays a key role in advancing the company’s AI strategy by designing, developing, and optimizing scalable data platforms. This role focuses on data engineering as part of the AI team, contributing to technical development, data architecture, pipeline optimization, and actionable intelligence. The ideal candidate will have expert‑level experience with Databricks, strong software engineering capabilities, and a proven ability to build robust, secure, and scalable data systems.

The Senior Data Engineer will work closely with cross‑functional teams to deliver high‑quality data solutions, optimize production pipelines, and support AI and machine learning capabilities.

• Design, develop, refine, and implement high‑quality data architecture solutions for large‑scale, complex data platforms.
• Establish best practices for writing modular, reusable, maintainable, and scalable data engineering code.
• Develop and maintain greenfield data engineering projects and production‑level data pipelines.
• Design data‑driven solutions to address complex cybersecurity challenges.
• Architect end‑to‑end data platforms by evaluating technical approaches, trade‑offs, risks, and performance requirements.
• Work hands‑on with engineers to implement data architecture and pipeline solutions.
• Collaborate with Dev Ops and MLOps teams to monitor and optimize data pipelines and machine learning workflows.
• Improve model and pipeline performance through monitoring, troubleshooting, and tuning to meet latency and cost requirements.
• Evaluate third‑party AI tools against internally developed solutions to maximize business value and return on investment.
• Establish data governance, data quality, and security standards to ensure the ethical and compliant use of sensitive information in AI models.
• Develop and implement incremental development strategies and production release plans aligned with the long‑term product vision.
• Serve as the subject matter expert in data engineering for the AI team.
• Ensure data pipelines are reliable, scalable, secure, and optimized for production workloads.

• Bachelor’s degree in Engineering, Computer Science, or a related technical field.
• Advanced degree in Engineering, Computer Science, Machine Learning, or Artificial Intelligence is preferred.
• Strong experience working as an individual contributor with a hands‑on engineering mindset and a focus on delivery.
• At least 4 years of experience working with data platform solutions across data and analytics throughout the software development life cycle (SDLC).
• At least 2 years of experience designing and mapping requirements into Databricks data pipeline capabilities on Google Cloud Platform (GCP).
• Hands‑on experience with Databricks Lakehouse architecture, including Databricks Datalake and Data Catalog capabilities.
• Experience designing Databricks Medallion Architecture, including data structure, data access, and data quality decisions.
• Strong experience with batch, mini‑batch, and real‑time data processing, including trade‑offs and best practices.
• Hands‑on experience with ELT/ETL processes, data pipelines, and orchestration tools.
• Experience processing large volumes of unlabeled data with complex schemas.
• Strong understanding of data modeling techniques and modern data platform trends, including AI applications in data engineering.
• Proficiency in Python, Pandas, PySpark, and related data processing frameworks.
• Experience with AI and machine learning tools such as Tensor Flow, PyTorch, Keras, Lang Chain, Lang Smith, and Infrastructure as Code (IaC).
• Experience working with structured and unstructured data formats, including Parquet, PostgreSQL, DuckDB, JSON, and PDF.
• Experience with infrastructure sizing and estimation…