Data Engineer; Python

We are looking for a talented Data Engineer with expertise in Python data processing. The ideal candidate will have a strong background in Python API development, parallel data processing, and distributed systems design. You will be responsible for building and maintaining systems that handle large-scale data processing tasks, ensuring high performance and scalability.

• Python API Development
  • Develop and maintain RESTful APIs using Python web frameworks such as FastAPI or Django.
  • Collaborate with front-end developers to integrate user-facing elements with server-side logic.
• Parallel Data Processing
  • Utilize Pandas, Num Py, and other libraries to process large datasets efficiently.
  • Implement multithreading, multiprocessing, and asynchronous programming techniques.
  • Optimize data processing pipelines to handle millions of rows with minimal latency.
• Distributed Systems Design
  • Design and implement distributed systems with a focus on scalability and reliability.
  • Understand and apply core concepts such as load balancing and task queues.
  • Use Docker to containerize applications and manage dependencies.
  • (Preferred) Experience with Kubernetes for container orchestration.
• Technical Communication
  • Clearly articulate complex technical concepts to team members and stakeholders.
  • Document system designs, processes, and code effectively.
  • Collaborate with cross-functional teams to align on project goals and deliverables.

• Experience in Python Web Frameworks
  • Proficiency with FastAPI, Django, or similar frameworks.
  • O C# coding
  • Understanding of RESTful API principles and best practices.
• Docker Knowledge
  • Ability to create and manage Docker Files.
  • Experience with containerization for deployment and development workflows.
• Systems Design Understanding
  • Basic knowledge of load balancing, task queues, and distributed system concepts.
  • Ability to design systems that are scalable and maintainable.
• Concurrent and Parallel Computing Skills
  • Proficiency in multithreading and multiprocessing without relying solely on external libraries or frameworks.
  • Familiarity with asynchronous programming, particularly async
    
    IO in Python.
• Communication Skills
  • Excellent technical communication abilities.
  • Experience collaborating in team environments and conveying complex ideas clearly.

• Education
  • BS or MS in Computer Science
• Advanced Data Processing Tools
  • Experience with Polars, PySpark, or similar tools.
  • Handling of large-scale data processing tasks efficiently.
• Distributed Computing Experience
  • Hands-on experience with distributed architectures in Docker.
  • Familiarity with concepts like task queuing, Map Reduce, and saga patterns.
• Kubernetes Experience
  • Knowledge of container orchestration using Kubernetes.
  • Experience deploying and managing applications in a Kubernetes cluster.
• Problem-Solving at Scale
  • Demonstrated ability to solve complex problems using parallel or distributed computing.
  • Innovative thinking beyond single-threaded processes.

Compensation: $50.00 - $55.00 per hour

Location: Onsite in Houston, TX.