Data Engineer - Senior Consultant

Job Family

Data Engineering & Architecture Consulting

Up to 10%

Ability to Obtain Public Trust

• Develop and maintain scalable data ingestion, transformation, and curation pipelines using Databricks (Delta Lake, Delta Live Tables, Auto Loader) and AWS services, supporting consistent delivery of analytics-ready datasets across the Data Platform.
• Implement standardized batch and near real-time data pipelines that integrate legacy systems and cloud-native data sources, contributing to enterprise-wide data access, reuse, and platform consistency.
• Support full data pipeline lifecycle activities, including intake, requirements analysis, source profiling, and technical implementation, aligning development work with defined intake processes, SLAs, and governance.
• Build production-ready data pipelines that meet defined technical and documentation standards, including participation in validation, testing, and release processes to enable reliable and compliant production deployments.
• Apply data quality checks, validation rules, and observability practices to improve pipeline reliability, support monitoring, and contribute to platform stability and operational performance targets.
• Integrate pipelines with AWS services (e.g., S3, streaming frameworks, APIs) and enterprise data tools to support secure, scalable data movement and interoperability across the ecosystem.
• Contribute to governed data engineering practices by implementing metadata capture, lineage tracking, and supporting access control patterns aligned to enterprise data governance standards.
• Support analytics and reporting use cases by preparing curated datasets and enabling consumption through SQL-based access, dashboards, and enterprise BI tooling.

• Bachelor’s degree is required
• Minimum FOUR (4) years of experience in data engineering, with hands‑on development of data pipelines in cloud or distributed data environments.
• Strong proficiency in Python, PySpark, and SQL for building and maintaining scalable ETL/ELT pipelines.
• Experience working with Databricks and Delta Lake to support ingestion, transformation, and curated data layer development.
• Working knowledge of AWS data services (e.g., S3, IAM, VPC) and integration patterns supporting secure and scalable data architectures.
• Experience implementing data quality checks, monitoring, and basic performance optimization techniques for pipeline efficiency and reliability.
• Familiarity with data governance concepts, including metadata, lineage, and access control frameworks in regulated environments.
• Experience working within Agile delivery environments and contributing to CI/CD‑enabled development workflows.

• Experience supporting enterprise data platforms or federal data modernization initiatives, particularly in highly regulated environments.
• Exposure to streaming technologies such as Kafka, Kinesis, or Event Bridge for near real‑time data processing.
• Familiarity with Databricks Unity Catalog and governance capabilities (RBAC, data masking, auditing).
• Experience using metadata/catalog tools such as Informatica EDC or similar platforms.
• Understanding of Dev Sec Ops  practices, including automated testing, deployment, and environment promotion across dev/test/prod.
• Exposure to performance optimization techniques (e.g., partitioning, Z‑ordering, clustering) for large‑scale data processing workloads.

Guidehouse offers a comprehensive, total rewards package that includes competitive compensation and a flexible benefits package that reflects our commitment to creating a diverse and supportive workplace.

• Medical, Rx, Dental & Vision Insurance
• Personal and Family Sick Time & Company Paid Holidays
• Position may be eligible for a discretionary variable incentive bonus
• Parental Leave and Adoption Assistance
• 401(k) Retirement Plan
• Basic Life & Supplemental Life
• Health Savings Account, Dental/Vision & Dependent Care Flexible Spending Accounts
• Short-Term & Long-Term Disability
• Student Loan Pay Down
• Tuition Reimbursement, Personal Development & Learning Opportunities
• Skills Development & Certifications
• Employee Referral…