Databricks Practice Lead​/Engineering Manager

Position Summary

Scicom Infrastructure Services is seeking an experienced Databricks Practice Lead / Engineering Manager to provide hands‑on technical leadership while managing a team of data engineers, architects, and consultants supporting complex enterprise and government programs.

This role requires a senior Databricks expert who can design and oversee modern data platforms, establish technical standards, guide delivery teams, and remain actively involved in architecture, troubleshooting, code reviews, and client‑facing solution development. The successful candidate will balance deep technical expertise with strong people leadership, delivery management, and stakeholder communication skills.

• Serve as the organization’s subject‑matter expert for the Databricks Lakehouse Platform.
• Design scalable, secure, and highly available data architectures using Databricks, Apache Spark, Delta Lake, and cloud‑native technologies.
• Lead the implementation of batch, streaming, ETL, ELT, analytics, machine‑learning, and AI‑enabled data solutions.
• Define architectural standards for medallion architectures, data modeling, ingestion, transformation, orchestration, and data consumption.
• Establish governance frameworks using Unity Catalog, including data lineage, access controls, auditing, metadata management, and secure data sharing.
• Guide Databricks workspace design, cluster configuration, serverless computing, workload isolation, performance tuning, and cost optimization.
• Oversee integration between Databricks and cloud platforms such as Microsoft Azure, AWS, or Google Cloud.
• Develop or review solutions involving PySpark, Spark SQL, Python, Delta Live Tables, Structured Streaming, Auto Loader, MLflow, and Databricks Workflows.
• Lead platform migrations and modernization efforts from legacy databases, data warehouses, Hadoop environments, and traditional ETL platforms.
• Establish development standards for source control, automated testing, CI/CD, infrastructure as code, monitoring, and production support.
• Conduct architecture reviews, code reviews, technical assessments, and root‑cause analyses.
• Evaluate emerging Databricks capabilities and recommend appropriate adoption strategies.

• Manage, mentor, and develop a team of Databricks engineers, data engineers, architects, and technical consultants.
• Assign resources and responsibilities based on project needs, employee strengths, availability, and technical complexity.
• Establish measurable goals, performance expectations, development plans, and technical competency standards.
• Conduct regular one‑on‑one meetings, performance reviews, coaching sessions, and technical development activities.
• Support recruiting, interviewing, candidate evaluation, onboarding, and workforce planning.
• Identify technical or performance gaps and coordinate training, mentoring, or corrective action as appropriate.
• Promote collaboration, accountability, documentation, knowledge sharing, and continuous improvement.
• Develop reusable accelerators, reference architectures, templates, and delivery playbooks.
• Build and maintain a strong Databricks practice capable of supporting multiple concurrent client engagements.

• Provide delivery oversight for Databricks and data‑engineering projects from planning through implementation and operational support.
• Translate business, functional, security, and contractual requirements into technical plans and deliverables.
• Develop project estimates, staffing plans, delivery schedules, milestones, and risk‑mitigation strategies.
• Monitor project scope, schedule, quality, budget, resource utilization, dependencies, and technical risks.
• Ensure deliverables meet client requirements, internal quality standards, security controls, and contractual commitments.
• Coordinate work across engineering, cloud, cybersecurity, data governance, analytics, project‑management, and client teams.
• Track delivery metrics and provide clear status reports to internal leadership, clients, and program stakeholders.
• Lead technical escalations and ensure issues are resolved promptly and appropriately…