Sr Data Engineer Python & Databricks
Job in
Edison, Middlesex County, New Jersey, 08818, USA
Listed on 2026-07-23
Listing for:
Phaxis LLC
Full Time
position Listed on 2026-07-23
Job specializations:
-
IT/Tech
Data Engineering, Cloud Computing: Infrastructure & Operations, Data Science Manager
Job Description & How to Apply Below
Salary is 140k to 170k + bonus
This role focuses on building efficient data pipelines, implementing best practices in data engineering, and ensuring system reliability and performance. The Data Engineer works closely with data scientists, analysts, and software engineers to deliver optimized data solutions for both batch and real-time processing.
Key Responsibilities- Design, develop, and maintain end-to-end data pipelines and ETL/ELT workflows using Python, SQL, and modern orchestration tools (e.g., Airflow, dbt).
- Architect and optimize data storage solutions, including data lakes and data warehouses, using Databricks, Delta Lake, and cloud-native services.
- Build scalable data processing solutions leveraging Databricks notebooks, jobs, and clusters for both batch and streaming data workloads.
- Develop and manage Databricks workflows using Spark (PySpark, SQL, or Scala) to transform, cleanse, and aggregate large datasets.
- Design, develop, and implement complex data integrations across Databricks, cloud platforms (AWS, Azure, or GCP), enterprise applications, APIs, and modern data ecosystems to enable reliable, scalable data exchange and processing.
- Implement data quality checks, schema validation, and monitoring to ensure data accuracy and reliability.
- Optimize Databricks cluster configurations and job performance to minimize cost and maximize throughput.
- Collaborate with Dev Ops teams to automate deployments, CI/CD pipelines, and infrastructure-as-code (IaC) for data systems.
- Bachelor's or Master's degree in Computer Science, Data Engineering, or a related technical field.
- Advanced proficiency in Python and SQL for data manipulation, transformation, and automation.
- Deep experience with Databricks, including Spark optimization, Delta Lake management, job orchestration, and workspace administration.
- Strong understanding of distributed data processing, partitioning strategies, and performance tuning in Databricks and Spark.
- Demonstrated experience designing and implementing enterprise-scale data integrations across Databricks, cloud platforms (AWS, Azure, or GCP), enterprise applications, APIs, and modern data ecosystems.
- Hands-on experience with cloud platforms (AWS, Azure, or GCP) and their data ecosystems (e.g., S3, ADLS, Big Query, Snowflake).
To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
Search for further Jobs Here:
×