Senior Databricks Engi

Job Title: Senior Databricks Engineer
Location: Onsite – Sugarland, Tx.
Employment Type: Full-Time

We are seeking an experienced Senior Databricks Engineer to design, develop, optimize, and manage large-scale data pipelines and analytics solutions using Databricks, Spark, and modern cloud platforms (Azure/AWS/GCP). The ideal candidate has deep expertise in distributed data processing, Delta Lake architecture, and end-to-end data engineering best practices. You will work with cross-functional teams—including product, analytics, and delivery—to deliver scalable data solutions for enterprise clients.

• Design, develop, and optimize Spark/Databricks ETL/ELT pipelines at scale.
• Implement efficient Delta Lake architectures (bronze/silver/gold layers).
• Write high-performance PySpark/SQL code following best practices.
• Tune clusters, jobs, and workflows for performance and cost efficiency.

• Develop data models and schemas to support analytics, reporting, and ML workloads.
• Implement medallion architecture following industry and Databricks standards.
• Design scalable and secure storage patterns using cloud-native services.

• Work with cloud platforms (Azure preferred, AWS or GCP acceptable).
• Implement CI/CD pipelines for Databricks using Git, Azure Dev Ops, or Git Hub Actions.
• Integrate Databricks with third-party systems, ADF, Event Hub, Kafka, APIs, etc.
• Configure workflows in Databricks (Jobs, Workflows, Unity Catalog).

• Implement data validation, observability, and monitoring frameworks.
• Apply best practices in data governance, lineage, and cataloging.
• Enforce security guidelines using RBAC, Unity Catalog, and encryption policies.

• Lead technical solutioning for client projects and POCs.
• Provide mentorship to junior engineers.
• Collaborate closely with business analysts, product managers, and stakeholders.
• Prepare architectural documentation, standards, and reusable accelerators.

• Investigate data issues, performance bottlenecks, and operational failures.
• Drive continuous improvement across data engineering processes.
• Ensure reliable and timely data delivery for downstream applications.

• Bachelor’s or master’s in computer science, Data Engineering, Information Systems, or related field.
• 6+ years of experience in data engineering.
• 3+ years of hands‑on experience with Databricks and Spark.
• Strong experience in Azure Databricks preferred; AWS or GCP equivalent.

• Expertise in Databricks, Apache Spark, Delta Lake, PySpark, SQL.
• Experience with Azure Data Factory, ADLS, Synapse, Azure Dev Ops.
• Understanding of data warehousing, ETL/ELT, data lakes, medallion architecture.
• Experience with CI/CD, Git branching strategies, and MLOps/Data Ops practices.
• Familiarity with Kafka/Event Hub, Databricks Autoloader, or real-time streaming a plus.
• Knowledge of ML workflows in Databricks is a strong advantage.

• Strong problem‑solving and analytical abilities.
• Excellent communication and stakeholder management.
• Ability to lead technical discussions with clients and internal team.

• Competitive salary and benefits.
• Opportunity to work on modern cloud and data architectures.
• Professional development and certification support.
• Unlimited opportunities to learn on our multiple Training Platforms.
• Certifications reimbursement.
• Flexibility.
• Opportunity to work on multiple technologies.

Address: Biz-Metric Partners Inc, 1650 Highway 6, Suite 130, Sugar Land, TX 77478

Email:

Career related enquiries: ta