Senior Software Engineer of Data Platform and Services

Senior Software Engineer of Data Platform and Services

Berlin, Germany

Location:

Berlin, Germany Remote Status:
Fully Remote

Live Person (NASDAQ: LPSN) is a leader in trusted enterprise conversational AI and digital transformation. The world's leading brands use our award-winning Conversational Cloud platform to connect with millions of consumers. We power nearly a billion conversational interactions every month, providing uniquely rich data analytics and safety tools to unlock the power of conversational AI for better business outcomes. Fast Company named Live Person the #1 Most Innovative AI Company in the world.

We are looking for a Senior Software Engineer to join our globally distributed Data Platform team in Europe. This is a high-impact individual contributor role focused on driving the migration of our core data pipelines to modern, scalable platforms. You will take technical ownership of migrating legacy Hadoop/Spark/Impala systems to a modern Databricks-centric stack. This role is ideal for a hands-on engineer who is passionate about modernizing data stacks and influencing platform architecture in a large-scale, global environment.

• Data Pipeline Migration:
  Own and drive the migration of legacy data pipelines (Hadoop, Spark, Impala) to modern platforms, with a primary focus on Databricks. This includes porting complex SQL, re-architecting ETL scripts, and migrating connections and secrets.
• Accelerated Migration through "Vibe Coding": A significant portion of the migration will be executed through "vibe coding"—leveraging state-of-the-art AI coding assistants (such as Claude and Codex) to automate and accelerate the conversion process. Rather than writing everything from scratch, you will orchestrate and guide AI models to port complex SQL queries, translate legacy scripts, untangle dependencies, and quickly generate clean, modern code while maintaining rigorous oversight on system architecture and logic verification.
• Develop and Modernize:
  Design, build, and maintain scalable Airflow DAGs and Databricks Jobs for large-scale data pipelines. Ensure all migrated and new systems are resilient, performant, and cost-efficient.
• Ensure Data Integrity:
  Implement and execute robust validation strategies to ensure data parity and quality between legacy and modern systems throughout the migration process.
• Drive Operational Excellence:
  Enhance our data platform's reliability by contributing to SLOs/SLAs, observability, and incident response practices. Participate in a scalable on-call rotation for production support.
• Data
  
  Experience:
  
  Possess a strong understanding of contact center analytics and the data used to measure and improve customer interactions, including data aggregation, conversation analytics, and business reporting. They should embrace data-driven design and build systems with observability, measurement, and continuous improvement.

• 6–8 years of software engineering experience with a strong background in distributed systems and large-scale data platforms, and a track record of owning complex systems end-to-end.
• Platform Migrations:
  Driving or heavily contributing to legacy-to-modern platform migrations (e.g., Hadoop/Impala to Databricks) in production environments.

• Databricks Ecosystem:
  Jobs, Delta Lake, cluster configuration/optimization, and porting complex SQL from Impala/Hive to Spark SQL.
• Orchestration & Scripting:
  Python for ETL scripting and deep expertise with Databricks for designing, debugging, and rewriting production DAGs.
• Cloud Infrastructure:
  Cloud-native data storage (specifically Google Cloud Storage) and interfacing with Kubernetes (GKE) microservice architectures.

Modern Data Platforms:

• Deep understanding of distributed data processing and real-time vs. batch trade-offs.
• Strong grasp of legacy big data frameworks (Hadoop, Map Reduce) to effectively untangle and migrate them.
• Familiarity with data integration patterns (connectors, webhooks, event-driven systems).

Cloud & Platform Engineering:

• Experience managing and integrating with cloud-native data infrastructure (GCP preferred).
• Experienc…