Data Engineer & Architect

Job Description

Job Title
:
Data Engineer & Architect

This position is an on-site assignment (preferred).

We are seeking a talented and experienced Data Engineer & Architect with operations support experience and data architecture capabilities. The ideal candidate will have a passion for data, strong design thinking, and a desire to help shape scalable cloud‑based data solutions. This role will support production data operations, develop new features and pipelines, and contribute to the design of enterprise data architecture across AWS and Databricks.

Wage Type
:
Salaried

To perform this job successfully, an individual must be able to perform each of the essential duties satisfactorily. Reasonable accommodation may be made to enable individuals with disabilities to perform the essential functions.

• Be the go-to person for production support. Troubleshoot issues, identify bottlenecks, and ensure smooth data operations.
• Work closely with our partners and internal staff to address any data-related concerns promptly.
• Keep a watchful eye on system performance and proactively address any anomalies.

• Design, develop, and maintain scalable cloud-based data architectures on AWS and Databricks.
• Implement efficient ETL/ELT processes using Databricks notebooks, Delta Lake, and Apache Spark.
• Collaborate with cross-functional teams to understand data requirements and ensure data solutions meet business needs.
• Utilize AWS services such as S3, Glue, Redshift, and EMR, and integrate with Databricks Work spaces for unified data processing.
• Ensure data security, integrity, and compliance with industry standards and regulations, including role-based access control (RBAC) in Databricks.
• Troubleshoot and optimize existing ETL processes for performance and reliability using Databricks performance tuning techniques.
• Stay updated on emerging cloud technologies and recommend improvements to enhance data engineering capabilities, including Databricks Lakehouse architecture.
• Collaborate with data scientists and analysts to provide accessible and high-quality data for analytics and reporting via Databricks SQL and Unity Catalog.

• Participate in designing enterprise‑level data models (conceptual, logical, physical) to support Lakehouse and warehouse structures.
• Define and maintain data standards, naming conventions, and architectural patterns for ingestion, transformation, storage, and governance.
• Evaluate system architecture and make recommendations to support scalability, performance, and cost optimization across AWS and Databricks.
• Assist in defining canonical data entities and integration patterns used across the organization.
• Contribute to solution architecture for data pipelines, APIs, and event‑driven data flows.
• Support the creation of technical documentation including data dictionaries, lineage diagrams, and architectural diagrams.
• Partner with security and governance stakeholders to ensure adherence to data stewardship, quality, and compliance principles.
• Provide architectural input during project planning, backlog grooming, and cross‑team design discussions.
• Carry out responsibilities with professionalism, respect for others, in accordance with the organization's policies and applicable laws
• Any other duties as assigned.

• Reliable production support of the cloud‑based data ecosystem through troubleshooting, debugging, analysis, and code deployments, including Databricks job orchestration and monitoring.
• Support for data warehouse and Lakehouse maintenance, including patches, upgrades, and workspace configuration.
• Clear communications around data initiatives, planning, and problem resolution, including Databricks workspace collaboration.
• Architectural documentation, including data flow diagrams, detailed solution designs, and model definitions.

Reports to:

Head of Integrations & Data Engineering

Supervises: N/A

• Bachelor's degree in Computer Science, Information Technology, or a related field.

• Proven experience as a Cloud Data Engineer with expertise in AWS and…