Data Engineer

Lead Data Engineer | 5+ Years Experience | Scalable Batch & Streaming Pipelines | Cloud Architecture (AWS, Databricks, Kubernetes)

Job Details:

• Salary: $180,000 - $250,000 per year

We are a fast‑growing, tech‑driven firm operating in a fast‑paced market environment where data is central to our competitive edge. Our technology organization values high‑performance engineering, hybrid‑cloud scalability, and rigorous data quality. We foster a collaborative, Agile culture where engineers enjoy deep technical ownership, mentor peers, and directly impact mission‑critical operations.

• Full ownership of the architecture, pushing our data platform beyond current tech limits to support rapid business growth.

• Work within a growing Data Engineering division supporting the strategic role of data at the firm.
• Drive the ongoing design and expansion of our data platform across a wide variety of data sources, supporting an array of streaming, operational, and research workflows.
• Work closely with Trading, Quant, Technology, and Business Operations teams throughout the firm to identify how data is produced and consumed, helping to define and deliver high‑impact projects.
• Build and deploy batch and streaming pipelines to collect and transform our rapidly growing Big Data set within our hybrid cloud architecture utilizing Kubernetes/EKS, Kafka/MSK, and Databricks/Spark.
• Mentor junior engineers in software and data engineering best practices.
• Produce clean, well‑tested, and documented code with a clear design to support mission‑critical applications.
• Build automated data validation test suites that ensure that data is processed and published in accordance with well‑defined Service Level Agreements (SLAs) pertaining to data quality, data availability, and data correctness.
• Challenge the status quo and help push our organization forward as we grow beyond the limits of our current tech stack.

• BS/MS/PhD in Computer Science, Engineering, Physics, Math, or equivalent technical field.
• 5+ years of professional experience developing software applications.
• Java/Scala mandatory;
  Python experience is significant.
• Must possess excellent communication, analytical, and problem‑solving skills.
• Must be responsive to changing market conditions with mandatory self‑mandate self‑mandate self‑mandate self‑mandate Self mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand mand…