Lead Databricks Developer​/Big Data Engineer

Location: Austin, TX

Salary: $55.00 USD Hourly – $57.00 USD Hourly

Position: Lead Databricks Developer/ Dig Data Engineer

Location: Fully Remote but must be located in Austin, TX

Duration: 6 month

• 8 years of experience in ETL/ELT workflows, CI/CD automation, and cross-functional collaboration.
• Strong expertise in data modeling, data lakes (Azure Data Lake Storage), and data warehouses
  .
• Proven ability to implement data validation, governance, lineage, and cataloging
  .
• Hands-on experience with data security measures including encryption, access controls, and auditing.
• Proficiency in Python, R, and SQL for data manipulation and analysis.
• Solid experience with the Azure cloud platform
  , Dev Ops practices, and version control systems.
• Background in Agile environments
  , with strong troubleshooting and debugging skills.
• 5 years of experience designing pipelines and optimizing Spark jobs on Databricks.
• Deep understanding of Apache Spark architecture, RDDs, Data Frames, and Spark SQL
  .
• Hands-on experience with Databricks notebooks, clusters, jobs, and Delta Lake
  .

• Knowledge of ML libraries such as MLflow, Scikit-learn, or Tensor Flow.
• Databricks Certified Associate Developer for Apache Spark
  .
• Azure Data Engineer Associate certification
  .

Thanks & Regards.

Prakash Dwivedi

E: I