Software Engineer, Data Engineering

Location: Charlotte, NC Salary: $69.00 USD Hourly - $74.00 USD Hourly Description:
Data Engineer / Data Platform Engineer (Python, PySpark)

Location: Remote/Hybrid (TBD; non-NYC preferred based on team guidance)
Employment Type: Contingent Resource Assignment
About the Role

The Financial Crimes Technology organization is expanding its in-house data engineering and platform development capabilities to reduce reliance on legacy vendor solutions. As a Data Engineer, you will design and build scalable data pipelines and foundational data platforms that support critical financial crimes functions-including AML, investigations, sanctions, fraud, and KYC. You will work with large datasets, improve data quality, and enable analytics, reporting, and downstream applications.

This role involves solving complex engineering problems, collaborating with cross-functional teams, and applying deep expertise in data processing, controls, and compliance-focused environments.
Responsibilities

• Design, build, and maintain batch and/or streaming data pipelines for financial crimes use cases.
• Develop scalable data transformations using Python and Py Spark , optimizing performance for large distributed datasets.
• Apply strong knowledge of Apache Spark internals (executors, partitions, shuffles, joins, caching) to tune workloads.
• Collaborate with engineering, analytics, and business partners to translate requirements into data models, mappings, and curated datasets.
• Support ingestion of data from multiple internal systems (transactional, case management, reference data, etc.).
• Implement data quality checks, reconciliation frameworks, lineage tracking, and controls to ensure auditability and reliability.
• Contribute to modernization efforts, including migration from legacy tooling to in-house platforms.
• Create high-quality technical documentation, including runbooks, pipeline logic, and architecture references.
• Participate in Agile ceremonies and deliver work aligned to sprint goals using Jira.

Minimum Qualifications

• 5+ years of experience in data engineering, ETL development, or data platform engineering, or equivalent experience through work, training, military service, or education.
• Expertise in:
  • Python
  • PySpark / Apache Spark
  • Advanced SQL
• Experience working with large-scale datasets and performance optimization.
• Strong understanding of data modeling, metadata, lineage, and governance concepts.
• Experience in regulated environments with strong control and audit requirements.
• Excellent communication skills and ability to work with technical and non-technical partners.

Preferred Qualifications

• Experience running PySpark on Google Cloud Platform (Google Cloud Platform) (e.g., Dataproc, Big Query, GCS).
• Hands-on experience with cloud-native big data platforms.
• Knowledge of data governance, security best practices, and compliance requirements.
• Experience with CI/CD for data pipelines, including Git-based workflows.
• Familiarity with orchestration tools such as Airflow.
• Experience with streaming technologies (e.g., Kafka).
• Exposure to lakehouse/warehouse technologies such as Databricks, Snowflake, or Big Query.
• Experience implementing encryption, access controls, data masking, and handling PII.

Domain Experience (Highly Valued)

• Experience in financial crimes or risk disciplines: AML, transaction monitoring, investigations, sanctions screening, fraud, or KYC.
• Understanding of regulatory expectations and strong documentation standards.

By providing your phone number, you consent to: (1) receive automated text messages and calls from the Judge Group, Inc. and its affiliates (collectively "Judge") to such phone number regarding job opportunities, your job application, and for other related purposes. Message & data rates apply and message frequency may vary. Consistent with Judge's Privacy Policy, information obtained from your consent will not be shared with third parties for marketing/promotional purposes.

Reply STOP to opt out of receiving telephone calls and text messages from Judge and HELP for help.
Contact:
This job and many more are available through The Judge Group. Please apply with us today!