Senior Data Engineer – NiFi & Microservices

Purple Drive

Senior Data Engineer – NiFi & Microservices Share Contractual Phoenix, AZ

Overview:

Design and Implementation:
Creating and implementing data flow solutions using NiFi, including defining data sources, destinations, and transformation processes.

Ingestion and Transformation:
Managing data ingestion from various sources (e.g., files, databases, APIs) and transforming data into desired formats for downstream consumption.

Routing and Enrichment:
Configuring NiFi to route data to appropriate destinations and enriching data with additional information (e.g., timestamps, location).

Scalability and Performance:
Ensuring data flows are scalable and performant to handle varying data volumes and velocity.

NiFi Cluster Management:
Maintaining and improving NiFi clusters, including configuration, optimization, and troubleshooting.

Microservices Integration:
Developing and integrating microservices to support data conditioning, format validation, and transformation processes.

Troubleshooting and Issue Resolution:
Identifying and resolving issues related to data flows, NiFi configurations, and data integration processes

Technical

Skills:

Strong NiFi Expertise:
Deep understanding of NiFi architecture, processors, controllers, and data flow management capabilities.

Data Integration and ETL:
Experience with data integration, ETL processes, and data modeling concepts.

Programming

Languages:

Proficiency in Java, Python, or other relevant scripting languages.

Database and SQL:
Familiarity with databases, SQL, and data manipulation techniques.

Cloud Technologies:
Experience with cloud platforms (e.g., AWS, Azure, GCP) and their integration with NiFi. Big Data Technologies:
Knowledge of big data technologies like Kafka.