Opensource Data Platform Solution Engineer

Open Source Data Platform Solution Engineer

Whitehall Resources are currently looking for a Open Source Data Platform Solution Engineer in South Yorkshire for an initial 12-month contract.

Cybersecurity Sciences & Analytics (CSA) is a ~$50m and ~150 headcount function, bringing together expertise across cybersecurity, data science, AI, software and cloud engineering, governance, and programme management to:

• Deliver cyber analytics capabilities that measurably improve cyber prevention, control effectiveness, and the resilience of operations across the cyber lifecycle
• Engineer, automate, and continuously enhance cyber detection and response capabilities to enable effective, resilient, and scalable security operations
• Provide trusted Cyber risk, control, and regulatory insights that inform executive decision‑making and investment prioritization
• Ensure cyber products, data, and bank‑wide AI use cases comply with cyber risk appetite, regulatory requirements, and data standards, through effective data and AI governance
• Evolve Cyber analytics platforms, products, and engineering practices through a controlled maturity roadmap, to improve platform and operational resilience, efficiency, and sustainability without increasing risk
• Define and maintain architecture artefacts, including HLDs/LLDs, reference architectures, patterns, ADRs (Architecture Decision Records), and reusable templates.
• Contribute to the design and evolution of a petabyte‑scale data platform built on open‑source technologies.
• Evaluate technologies and drive standardisation: assess OSS components and vendors, run PoCs, document trade‑offs, and promote reusable patterns to reduce fragmentation.
• Own the end‑to‑end solution design for custom‑built applications that support the Bank’s cybersecurity ambition.
• Provide architecture and design input to software engineering and platform teams to support implementation and delivery.
• Partner with central architecture and governance teams to ensure solutions align with the Bank’s controls, policies, standards, and guardrails.
• Help ensure solutions remain compliant, operationally supportable, and within agreed cost boundaries.

• Strong team player, able to collaborate effectively with colleagues across geographies, time zones, and cultures.
• Excellent written and verbal communication skills, with the ability to explain complex technical topics clearly.
• Able to influence without authority; confident facilitating design workshops and governance forums.
• Strong prioritisation and pragmatic decision‑making, balancing target architecture with delivery realities.
• Comfortable designing solutions end‑to‑end, while also able to roll up sleeves and contribute hands‑on (including coding where needed).

• Demonstrable experience designing, deploying, and operating big data platforms at scale.
• Hands‑on experience with OSS data ecosystems (e.g., Spark, Trino, Airflow, Apache Superset, and similar technologies).
• Knowledge of cloud platforms and traditional on‑premise hosting; hands‑on experience with Azure, AWS and/or GCP is beneficial.
• Proven security architecture expertise, including:
• Threat modelling and secure‑by‑design practices
• IAM (RBAC/ABAC), privileged access concepts, and service‑to‑service authentication
• Encryption in transit/at rest, key management, secrets management
• Secure logging/telemetry, auditability, and evidence‑driven control assurance
• Secure SDLC, vulnerability management, and dependency/supply‑chain risk controls
• Proven experience delivering end‑to‑end solution designs, from requirements through to production operations and support.

• Strong team player, able to collaborate effectively with colleagues across geographies, time zones, and cultures.
• Excellent written and verbal communication skills, with the ability to explain complex technical topics clearly.
• Able to influence without authority; confident facilitating design workshops and governance forums.
• Strong prioritisation and pragmatic decision‑making, balancing target architecture with delivery realities.
• Comfortable designing solutions end‑to‑end, while also able to roll up sleeves and contribute hands‑on (including coding where needed).