Software Engineer

Location: Charlotte, NC
Salary: $69.00 USD Hourly - $74.00 USD Hourly
Description:
Lead Software Engineer - Spark Flow Framework (Contract)

Location: Charlotte, NC
Role Type: Contingent / Contract
About the Role

We are looking for a Lead Software Engineer to guide the technical evolution of Spark Flow, an enterprise-grade data processing framework built on Apache Spark. In this role, you will design clean, composable abstractions, deliver new framework capabilities, and elevate the overall developer experience. You will also architect and deploy AI-enabled features and drive the integration of Spark Flow into the Unity control plane.

This role requires strategic thinking, strong cross-functional collaboration, and the ability to solve complex engineering challenges at scale.
What You'll Do

• Lead the architectural direction of the Spark Flow framework with a focus on modularity, composability, and clear abstraction layers.
• Design and develop new framework features that expand supported use cases and platform capabilities.
• Balance rapid feature delivery with high code quality, ensuring strong performance, backward compatibility, and adherence to governance requirements.
• Improve developer experience by simplifying APIs, configuration patterns, extension points, and onboarding workflows.
• Design and implement AI/ML-driven capabilities that enhance developer productivity and framework operability.
• Integrate Spark Flow as a first-class, managed component within the Unity control plane.
• Partner closely with data, platform, and infrastructure teams to support consistent adoption across environments.
• Provide technical leadership through architecture reviews, code reviews, and mentorship.

Minimum Qualifications

• 5+ years of software engineering experience, or equivalent background through a combination of professional, consulting, military, or educational experience.
• Strong hands-on experience with Apache Spark using Scala and/or Java (Python a plus).
• Proven success designing and building reusable frameworks or libraries.
• Expertise in API and abstraction design, extensibility patterns, and scalable systems architecture.
• Experience applying AI/ML or LLM-based techniques to developer tooling or data frameworks.
• Background working in large-scale, enterprise data environments.

Preferred Qualifications

• Experience integrating frameworks into control planes or internal developer platforms.
• Knowledge of Spark SQL-based architectures and enterprise data governance patterns.
• Experience modernizing or replacing legacy ETL systems.
• Additional skills in SQL, shell scripting, and distributed data processing.

Success in This Role Looks Like

• Delivering clean, scalable abstractions that make Spark Flow easier to use and extend.
• Driving AI-enabled innovation that improves developer workflows and operational efficiency.
• Ensuring Spark Flow integrates seamlessly into the Unity control plane and broader platform ecosystem.
• Acting as a technical leader who elevates code quality, design rigor, and engineering excellence across teams.

By providing your phone number, you consent to: (1) receive automated text messages and calls from the Judge Group, Inc. and its affiliates (collectively "Judge") to such phone number regarding job opportunities, your job application, and for other related purposes. Message & data rates apply and message frequency may vary. Consistent with Judge's Privacy Policy, information obtained from your consent will not be shared with third parties for marketing/promotional purposes.

Reply STOP to opt out of receiving telephone calls and text messages from Judge and HELP for help.

Contact:

This job and many more are available through The Judge Group. Please apply with us today!