Data Engineer

Position Summary

Amerit Fleet Solutions seeks an experienced Data Engineer to design, build, and maintain a robust data lake infrastructure that will serve as the foundation for enterprise analytics, reporting, and AI/ML initiatives. This role is critical to establishing data governance, quality, and accessibility across the organization. The successful candidate will work cross-functionally with IT, Analytics, Operations, and AI teams to ensure data integrity, compliance, and optimal performance of our data infrastructure.

Salary: $150k - $170k per annum

• Comprehensive health insurance
• 401(k) matching
• professional development budget
• flexible work arrangements

• Data Lake Architecture & Design:
  Design and implement a scalable, cloud-based data lake architecture (AWS/Azure/GCP) that ingests, stores, and manages petabyte-scale data from fleet management systems, maintenance records, vendor systems, and operational databases. Establish data zones (raw, curated, analytics) with appropriate access controls and retention policies.
• Data Integration & ETL/ELT Pipelines:
  Build and maintain automated data pipelines that extract, transform, and load data from multiple sources (work order systems, telematics platforms, financial systems, CRM) into the data lake. Ensure real-time and batch processing capabilities with minimal latency. Document all transformations and business logic.
• Data Quality & Integrity Management:
  Establish and implement comprehensive data quality frameworks including validation rules, anomaly detection, and reconciliation processes. Monitor data accuracy, completeness, and consistency. Create data quality dashboards and alerts to identify and remediate data issues before they impact downstream analytics. Maintain detailed audit trails for all data changes.
• Data Governance & Compliance:
  Develop and enforce data governance policies including data cataloging, metadata management, lineage tracking, and PII/sensitive data protection. Ensure compliance with data privacy regulations (GDPR, CCPA, etc.). Establish data access controls, role-based permissions, and audit logging. Maintain data dictionary and documentation standards.
• Performance Optimization & Monitoring:
  Monitor data lake performance, query execution times, and storage utilization. Optimize data structures, indexing, and partitioning strategies to ensure sub-second query response times. Implement automated scaling policies and cost optimization initiatives. Provide recommendations for infrastructure improvements.
• Data Security & Disaster Recovery:
  Implement encryption, secure data access protocols, and disaster recovery/business continuity plans. Establish backup, replication, and recovery procedures with defined RPO/RTO targets. Conduct security audits and vulnerability assessments. Maintain compliance documentation for SOC 2 and other security standards.
• Documentation & Knowledge Transfer:
  Create comprehensive technical documentation for data lake architecture, data flows, transformation logic, and operational procedures. Develop runbooks for common operations and troubleshooting. Provide training to analysts, data scientists, and other teams on data access, usage best practices, and available datasets.
• Cross-Functional Collaboration:
  
  Partner with business units to understand data requirements and use cases. Collaborate with AI/ML teams on model training data pipelines. Work with analytics teams to optimize queries and reporting. Support data strategy discussions and roadmap planning.

• Advanced SQL and relational database design (PostgreSQL, MySQL, or SQL Server)
• Cloud data platforms (AWS S3, Glue, Redshift, Azure Data Lake Gen2, Synapse, Databricks, Microsoft Fabric)
• ETL/ELT tools (Apache Airflow, dbt, Talend, or cloud-native alternatives)
• Data warehousing concepts and dimensional modeling (star schema, slowly changing dimensions)
• Programming languages:
  Python or Scala for data pipeline development
• Data quality frameworks and tools (Great Expectations, Talend, or similar)
• Version control (Git) and…