Data Platform Engineer

Position

Description:

Senior Cloud Systems Engineer

Start Date – End Date: 02/02/2026 – 12/31/2026

Note:

Period of performance may extend beyond 12/31/2026.

• Hands‑on experience administering Databricks (workspace administration, clusters/compute policies, jobs, SQL warehouses, repos, runtime management) and expertise using Databricks CLI.
• Strong Unity Catalog administration: metastores; catalogs/schemas; grants; service principals; external locations; storage credentials; governed storage access.
• Identity & Access Management proficiency: SSO concepts, SCIM provisioning, group‑based RBAC, service principals, least‑privilege patterns.
• Security fundamentals: secrets management, secure connectivity, audit logging, access monitoring and evidence‑ready operations.
• Cloud platform expertise (AWS): IAM roles/policies, object storage security patterns, networking basics (VPC concepts), logging/monitoring integration.
• Automation skills: scripting and/or IaC using Terraform/CLI/REST APIs for repeatable configuration and environment promotion.
• Experience implementing data governance controls (classification/tagging, lineage/metadata integrations) in partnership with governance teams.
• CI/CD practices for jobs/notebooks/config promotion across SDLC environments.
• Understanding of lakehouse concepts (e.g., Delta, table lifecycle management, separation of storage/compute).
• SQL proficiency and data engineering fundamentals for troubleshooting query performance issues, understanding ETL/ELT workflow patterns, and debugging data pipeline failures; basic Python/Scala familiarity for notebook/code issue diagnosis.
• Experience with compliance and regulatory frameworks (FedRAMP, HIPAA, SOC2, or similar) including implementation of data residency requirements, retention policies, and audit‑ready evidence collection.
• Hands‑on experience with AWS security and networking services including Private Link, Secrets Manager/Systems Manager integration, Cloud Watch/Cloud Trail integration, S3 bucket policies, cross‑account access patterns, and KMS encryption key management.
• Experience administering Databricks serverless compute, Workspace Git integrations (Git Lab), Databricks Asset Bundles (DABs) for deployment automation, and modern workspace features supporting Dev Ops workflows.
• SLA/SLO management and stakeholder communication skills; ability to define platform service levels, produce operational reports, translate technical issues to business stakeholders, and manage vendor relationships (Databricks account teams).

• Bachelor’s degree in a related field or equivalent practical experience.
• 7+ years in cloud/data platform administration and operations, including 4+ years supporting Databricks or similar platforms.
• Databricks Platform Administrator/Databricks AWS Platform Architect.
• Databricks Certified Data Engineer Associate/Professional.
• AWS Certified Solutions Architect Associate or Professional.

• Administer Databricks account and work spaces across SDLC environments; standardize configuration, naming, and operational patterns.
• Configure and maintain clusters/compute, job compute, SQL warehouses, runtime versions, libraries, repos, and workspace settings.
• Implement platform monitoring/alerting, operational dashboards, and health checks; maintain runbooks and operational procedures.
• Provide Tier 2/3 operational support: troubleshoot incidents, perform root‑cause analysis, and drive remediation and preventive actions.
• Manage change control for upgrades, feature rollouts, configuration changes, and integration changes; document impacts and rollback plans.
• Enforce least privilege across platform resources (work spaces, jobs, clusters, SQL warehouses, repos, secrets) using role/group‑based access patterns.
• Configure and manage secrets and secure credential handling (secret scopes / key management integrations) for platform and data connectivity.
• Enable and maintain audit logging and access/event visibility; support security reviews and evidence requests.
• Administer Unity Catalog governance: metastores, catalogs/schemas/tables, ownership, grants, and environment/domain…