Data Infrastructure Engineer; Hadoop

Position: Data Infrastructure Engineer (Hadoop)
Location: New York

Data Infrastructure Engineer (Hadoop)
New York City, NYHybrid Schedule (M/F remote, T/W/TH in-office)
At Magnite, we cultivate an environment of continuous growth and collaboration. Our work impacts what millions of people read, watch, and buy, and we’re looking for people to help us tackle that responsibility with creativity and focus.  Magnite (NASDAQ: MGNI) is the world’s largest independent sell-side advertising platform. Publishers use our technology to monetize their content across all screens and formats including CTV / streaming, online video, display, and audio.

Our tech fuels billions of transactions per day!

About this team:

We are seeking a Data Platform Engineer (Hadoop) to join our Infrastructure Engineering team. This role is responsible for designing, building, maintaining, and modernizing our enterprise big data platforms that support large-scale data processing and analytics across the organization. The ideal candidate has extensive experience with distributed data platforms such as Kafka, Hadoop/MapR, ultra-fast No

SQL databases (Redis, DynamoDB, Aerospike), and Linux-based infrastructure. This individual will work closely with Infrastructure, SRE, Database and Engineering teams to ensure our platforms are scalable, highly available, secure and performant. This is a hands-on engineering role requiring strong operational experience supporting mission-critical production environments while driving platform modernization initiatives. In this role you will:  Cluster & Infrastructure Administration:

Hadoop Management:
Provision, configure, monitor and maintain robust distributed Hadoop clusters (e.g. MapR, Cloudera, Hortonworks)
Linux Systems Engineering:
Manage, optimize, and troubleshoot enterprise Linux servers (Red Hat, CentOS, Rocky Linux) hosting the data platform.

Performance Tuning:
Troubleshooting, root cause analysis for production environments.

High Availability:
Maintain cluster backup, disaster recovery, production support and on-call rotations

Cross-Functional Dev Ops & SRE

Collaboration:

Infrastructure as Code (Iac)
CI/CD Integration Data Ecosystem & Pipeline Management:

Kafka Event Streaming:
Deploy, configure, tune and scale multi-node Kafka clusters, topics, partitions and consumer groups.

No

SQL Database Management:
Administer, schema design, and performance-tune No

SQL databases (such as HBase, Aerospike, Cassandra, or MongoDB)
Ecosystem Integration:
Support and integrate core Hadoop ecosystem tools including Hive, Spark, and HDFS.We’re looking for someone with: 4+ years of experience administering distributed Big Data Platforms.

Messaging Systems:
Proven experience deploying and managing production grade Kafka infrastructure.

No

SQL Databases:
Practical experience administering at least on major No

SQL databaseScripting:
Proficiency with shell scripting (Bash, Python, or similar).

Experience with relational databases such as PostgreSQL or MySWLLinux Administration:
Experience in Linux operating systems, including storage management, networking, security and performance troubleshooting.

Experience with monitoring platforms such as Grafana, Prometheus, or similar

Preferred Qualifications:

Experience with Kubernetes and containerized workloads

Experience with Ceph or other distributed storage platforms

Experience with Infrastructure as Code tools such as Terraform or Spacelift

Experience with configuration management tools such as Puppet or Ansible

Experience with cloud platforms (AWS)
In New York, the base salary range for this role is $170,000 - $180,000..Actual compensation will be determined based on location, experience, skills, and internal equity.

This role is also eligible for:

Annual performance-based bonus

Equity (NASDAQ: MGNI)
Comprehensive benefits package

During the…