AWS Data Engineer

Responsibilities

• Data Pipeline Expertise:
  Capable of implementing a diverse range of data solutions with significant autonomy, demonstrating the ability to independently build and maintain complex ETL (Extract, Transform, Load) pipelines. Possesses advanced skills in pipeline complexity management and orchestration, with a proven track record of developing high-performance data products accessible via APIs for critical tier 1 trading applications. Demonstrates expertise in archiving data products to data lakes for comprehensive analytics purposes.
• Data Quality and Advanced Analytics:
  Strong proficiency in Data Quality, Data Validation, and Anomaly Detection techniques. Experience with AI/ML and Large Language Model (LLM) approaches to data analysis and validation. Ability to implement sophisticated data integrity and verification processes.

• Programming and Cloud Technologies:
  Expert-level skills in Python, SQL, AWS Cloud Services (Cloud Storage, Lambda, ECS, Glue, Data Quality, HDMI, Redshift, Machine Learning, Open Search, RDS).
• API and Resilience Technologies:
  Advanced skills in building resilient API architectures, including microservices design patterns, API gateway configuration, circuit breaker implementations, fault tolerance and error handling mechanisms, and expertise in API development and management tools (Swagger/OpenAPI, Postman, GraphQL, RESTful API design principles).
• Resilience Engineering Techniques:
  Distributed system design, load balancing, failover strategies, horizontal and vertical scaling approaches, containerization and orchestration (Docker, Kubernetes, Helm charts).
• Monitoring and Observability:
  Prometheus, Grafana, ELK Stack (Elasticsearch, Logstash, Kibana), distributed tracing with Jaeger or Zipkin.
• Additional Technical
  
  Competencies:
  
  Proficient in Apache Flink, advanced Java programming skills, proven ability to create and optimize complex data products, and demonstrated subject‑matter expertise in data engineering principles.

• Professional Attributes:
  Capability to work independently through complex data challenges, with a strong preference for candidates with prior experience in the investment data domain and related technologies.

• Deep understanding of data engineering best practices.
• Ability to design scalable, efficient, and resilient data solutions.
• Proven track record of solving intricate data infrastructure problems with robust API architectures.