Data Platform Engineer

At Peregrine, we’re always seeking Specialist Talent that have the ideal mix of skills, experience, and attitude, to place with our vast array of clients. From Business Analysts in large government organisations to Software Developers in the private sector – we are always in search of the best talent to place, now.

At Peregrine, we find the best talent for our clients. As a permanent employee of Peregrine, with access to all our standard benefits, you will be deployed across our portfolio of clients as a specialist consultant, working on a wide array of complex projects across multiple industries.

As a Data Platform Engineer in a highly regulated environment, you will be responsible for designing, building, and maintaining secure and scalable data infrastructure that supports both cloud and on‑premises platforms. You will play a key role in ensuring that all data systems comply with industry regulations and security standards while enabling efficient access for analytics and operational teams.

A strong command of Apache NiFi is essential for this role. You will be expected to design, implement, and maintain data flows using NiFi, ensuring accurate, efficient, and secure data ingestion, transformation, and delivery. You should be adept at identifying and resolving issues within NiFi flows, managing performance bottlenecks, and implementing robust error‑handling strategies.

You’ll work closely with cross‑functional teams including data architects, compliance officers, and cybersecurity specialists to integrate data from various systems such as databases, APIs, and cloud platforms. Your work will directly support batch processing, real‑time streaming, and event‑driven data pipelines across a variety of use cases.

We’re looking for candidates with over 3 years of relevant experience in data engineering, platform engineering, or a related field, with demonstrated hands‑on expertise in NiFi and data pipeline design in regulated environments.

• Design, develop, and maintain robust and secure data pipelines using NiFi and related big data technologies.
• Troubleshoot and optimize NiFi flows, including performance tuning, error resolution, and flow control.
• Integrate NiFi with cloud platforms, databases (SQL & No
  
  SQL), APIs, and third‑party systems.
• Ensure compliance with regulatory and security requirements across data storage, transfer, and access layers.
• Support data migration efforts and implement disaster recovery protocols.
• Continuously monitor data infrastructure performance and recommend improvements.
• Collaborate with cross‑functional teams to align data platform capabilities with business needs and compliance requirements.
• Maintain documentation of data flows and processes, ensuring knowledge sharing and operational transparency.

You will have the following skills or proven experience:

• Deep understanding of core NiFi concepts:
  Flow Files, Processors, Controller Services, Schedulers, Web UI.
• Experience designing and optimizing data flows for batch, real‑time streaming, and event‑driven architectures.
• Ability to identify and resolve flow issues, optimize performance, and implement error‑handling strategies.
• Optional scripting skills for creating custom NiFi processors.

• Proficiency in Java and SQL.
• Experience with C# and Scala is a plus.
• Experience with ETL tools and big data platforms.
• Knowledge of data modelling, replication, and query optimization.
• Hands‑on experience with SQL and No
  
  SQL databases is desirable.
• Familiarity with data warehousing solutions (e.g., Snowflake, Redshift, Big Query) would be beneficial.

• Comfortable operating in hybrid environments (cloud and on‑prem).
• Experience integrating diverse data sources and systems.
• Understanding of secure data transfer protocols and disaster recovery planning.

• Experience working with CI/CD pipelines to automate deployment of NiFi flows and data infrastructure components.
• Familiarity with tools such as Git, Jenkins, Azure Dev Ops, or similar.

• Comfortable working in Agile/Scrum environments, participating in…