Platform Staff Engineer

• The Staff Software Engineer (Platform) leads large-scale projects and is a key contributor to their group’s success
• They are involved in long-term strategic planning, mentor senior engineers, and are influential in technical decision-making across multiple teams or a department
• They focus on implementation and system design, as well as maintaining and evolving DISCO’s distributed data platform
• Technical Leadership and Mentorship:
  Provides technical leadership in project teams, guiding engineers, and contributing to technical decision-making. Sets technical direction for large projects or teams, driving innovation, and solving organization-wide problems
• Performance Optimization:
  Focuses on designing systems for high performance, scalability, and reliability
• Ownership:
  Proactively identifies areas of products and systems that currently have problems or will develop problems as well as makes plans to address those problems
• Stakeholder Engagement:
  Works with stakeholders, including product managers, designers, developers, and executives, to align architectural designs with business needs and customer expectations
• Software Development:
  Designs, develops, and maintains sophisticated, scalable, high-performance data software solutions, ensuring they are efficient, scalable, and meet business
• Equirements with architectural excellence for our big data platform at petabyte scale. Responsible for the boundary identification, system design and grooming for one or more sub-systems
• Code Quality and Optimization:
  Upholds and enhances code quality through rigorous review processes, optimization techniques, and adherence to best practices
• Technical Problem Solving:
  Addresses intricate technical issues through innovative problem-solving strategies
• Collaboration and Communication:
  Works effectively within the engineering team and cross-functional teams, collaborating with other engineers, product managers, and stakeholders, contributing to project planning and execution
• Continuous Learning and Adaptation:
  Stays updated with the latest technology trends and advancements, continually enhancing skills and knowledge, bringing innovative ideas to the team
• Technical Documentation:
  Creates and maintains comprehensive technical documentation, ensuring clarity and accessibility of information
• Data Platform Development:
  Builds and maintains scalable and robust data platforms to support analytics, machine learning, and business intelligence
• Data Integration and ETL:
  Develops and optimizes ETL (Extract, Transform, Load) processes and data integration pipelines. Ensures efficient data flow across various sources and systems
• Advanced File Processing:
  Develops extensible file processing pipelines that offer flexible, self-service artifact generation. Researches and evaluates file processing software based on fidelity, reliability, and performance criteria. Models file processing outcomes within an observability framework to provide insights for engineering teams and business leaders. Develops engineering systems that facilitate rapid and automatic evaluation of file processors
• Data Management:
  Crafts and orchestrates scaleout data pipelines that optimize serverless and containerized compute to balance cost, latency, and duration for big-data solutions. Contributes to algorithmically intensive data engines operating on streaming, large, or multi-tenant datasets. Troubleshoots systems, identify root causes, and rectify bugs

• Start-up culture
• Open PTO
• Growth opportunities
• Medical, dental, and vision, 401(k)

Experience leveraging observability systems (logging, metrics, traces) to monitor and debug production subsystems and generate business-level metrics

Demonstrated expertise in designing, implementing, and maintaining (through operational observability) high-availability, high-performance, distributed data processing systems

Experience with Large-Scale Systems:
Involvement in developing, monitoring, and maintaining large-scale, high-availability software systems

Experience with designing, modifying, and operating multi-tenant systems

Familiarity with Agile development methodologies and practices
10+ years of relevant experience in backend engineering, with a substantial focus on big data platforms

Experience with gRPC and Protocol Buffers for efficient, language-agnostic service-to-service communication

Experience selecting and extending 3rd party components (commercial or open source) that provide operational leverage

Experience using feature or release toggles as a code branching strategy

Experience designing and developing from a Security Perspective Deep expertise with both relational and non-relational data stores:
Elasticsearch, Cassandra, MySQL, and PostgreSQLDemonstrated experience in leading or playing key roles in significant software projects, showing an ability to work effectively both independently and as part of a team

Experience with CI/CD pipelines and tools (like Jenkins, Git Lab CI, or CircleCI) for automated testing and deployment of…