Remote Databricks Infrastructure Engineer Job Terrytown Louisiana USA,IT/Tech

Piper Companies is seeking a Databricks Infrastructure Engineer to support the design, development, and administration of enterprise cloud data infrastructure supporting forecasting, analytics, and machine learning initiatives. The Databricks Infrastructure Engineer will be responsible for managing Databricks environments, implementing infrastructure automation, optimizing data pipelines, and ensuring secure, scalable, and reliable cloud platform operations.

Responsibilities of the Databricks Infrastructure Engineer include:

Design, configure, administer, and maintain Databricks platform infrastructure supporting enterprise data engineering, analytics, and forecasting workloads
Manage Databricks work spaces, clusters, SQL warehouses, jobs, workflows, Unity Catalog, and access controls
Develop and maintain Infrastructure as Code (IaC) solutions using Terraform and cloud automation technologies
Implement CI/CD pipelines for Databricks notebooks, workflows, data pipelines, and platform deployments
Support cloud-based data ingestion, transformation, feature engineering, and machine learning workloads
Monitor platform performance, pipeline reliability, compute utilization, and cloud infrastructure costs
Optimize Databricks clusters, workload scheduling, storage configurations, and resource allocation
Implement security controls, including identity and access management, network security, secrets management, and data governance
Troubleshoot infrastructure issues, pipeline failures, and performance bottlenecks across Databricks environments
Collaborate with Data Engineers, Data Scientists, Dev Ops Engineers, and Enterprise Architecture teams to support scalable data solutions
Support incident response, root‑cause analysis, disaster recovery, and business continuity initiatives
Develop technical documentation, operational procedures, and platform engineering best practices

Qualifications for the Databricks Infrastructure Engineer include:

Bachelor's degree in Computer Science, Information Technology, Engineering, or a related discipline, or equivalent professional experience
3+ years of experience in cloud data platform engineering, infrastructure administration, or Databricks platform management
Strong hands‑on experience administering Databricks environments, including clusters, jobs, workflows, SQL warehouses, and workspace configurations
Experience with cloud platforms such as Microsoft Azure, AWS, or Google Cloud Platform
Experience implementing Infrastructure as Code using Terraform or similar technologies
Hands‑on experience with CI/CD pipelines, Git‑based source control, and automated deployment workflows
Proficiency with Python, SQL, and scripting for infrastructure automation and platform management
Understanding of data engineering concepts, including ETL/ELT pipelines, batch and streaming data processing, Delta Lake, and data quality
Experience with cloud networking, identity and access management, security controls, monitoring, and logging
Strong troubleshooting, technical documentation, communication, and cross‑functional collaboration skills

Preferred Qualifications:

Experience with Databricks Unity Catalog, MLflow, Delta Live Tables, Databricks Asset Bundles, and model serving
Experience supporting forecasting, machine learning, supply chain, demand planning, or enterprise analytics environments
Databricks Certified Data Engineer, Databricks Certified Platform Administrator, or relevant cloud infrastructure certifications
Experience with cloud cost optimization, Fin Ops practices, and Databricks compute optimization
Familiarity with SAP, ERP systems, supply chain data, inventory management, or sales forecasting integrations
Experience with Azure Dev Ops, Git Hub Actions, and automated Databricks…