Python Data Engineer

Job Description:
We are looking for a talented Data Engineer with expertise in Python data processing. The ideal candidate will have a strong background in Python API development, parallel data processing, and distributed systems design. You will be responsible for building and maintaining systems that handle large-scale data processing tasks, ensuring high performance and scalability.

Key Responsibilities:

Python API Development:
o Develop and maintain RESTful APIs using Python web frameworks such as FastAPI or Django.
o Collaborate with front-end developers to integrate user-facing elements with server-side logic.


Parallel Data Processing:
o Utilize Pandas, Num Py, and other libraries to process large datasets efficiently.
o Implement multithreading, multiprocessing, and asynchronous programming techniques.
o Optimize data processing pipelines to handle millions of rows with minimal latency.


Distributed Systems Design:
o Design and implement distributed systems with a focus on scalability and reliability.
o Understand and apply core concepts such as load balancing and task queues.
o Use Docker to containerize applications and manage dependencies.
o (Preferred)

Experience with Kubernetes for container orchestration.


Technical Communication:
o Clearly articulate complex technical concepts to team members and stakeholders.
o Document system designs, processes, and code effectively.
o Collaborate with cross-functional teams to align on project goals and deliverables.

Must-Have

Qualifications:


Experience in Python Web Frameworks:
o Proficiency with FastAPI, Django, or similar frameworks.
O C# coding
o Understanding of RESTful API principles and best practices.


Docker Knowledge:
o Ability to create and manage Docker Files.
o

Experience with containerization for deployment and development workflows.


Systems Design Understanding:
o Basic knowledge of load balancing, task queues, and distributed system concepts.
o Ability to design systems that are scalable and maintainable.


Concurrent and Parallel Computing Skills:
o Proficiency in multithreading and multiprocessing without relying solely on external libraries or frameworks.
o Familiarity with asynchronous programming, particularly async

IO in Python.


Communication Skills:
o Excellent technical communication abilities.
o Experience collaborating in team environments and conveying complex ideas clearly.

Preferred Qualifications:

Education:
o BS or MS in Computer Science


Advanced Data Processing Tools:
o

Experience with Polars, PySpark, or similar tools.
o Handling of large-scale data processing tasks efficiently.


Distributed Computing Experience:
o Hands-on experience with distributed architectures in Docker.
o Familiarity with concepts like task queuing, Map Reduce, and saga patterns.


Kubernetes Experience:
o Knowledge of container orchestration using Kubernetes.
o Experience deploying and managing applications in a Kubernetes cluster.


Problem-Solving at Scale:
o Demonstrated ability to solve complex problems using parallel or distributed computing.
o Innovative thinking beyond single-threaded processes.

Compensation: $45.00 - $50.00 per hour

Apply with Indeed

First Name *

Last Name *

Email *

Phone *

I want to get text messages for this job from ENVISN INCORPORATED.

Are you comfortable to work onsite in Houston, TX *

Can you work on 1099 or W2 or willing to transfer your visa and work on W2? *

How many years of experience do you have in Data Engineering *

How many years of hands on experience do you have with Python *