Big Data Engineer Rockville, MD

About the Role

We are looking for a highly skilled Big Data Engineer to design, develop and optimize large-scale data processing systems in Rockville, MD. This is a hybrid role, and the source notes two open positions.

You will work with cross-functional teams to build data pipelines, implement data integration solutions and keep big data platforms fast, scalable and reliable. Deep experience with distributed systems, cloud platforms and technologies such as Hadoop and Spark is central.

Key Responsibilities

• Design, develop and maintain large-scale data processing pipelines using Hadoop, Spark, Python and Scala.
• Implement data ingestion, storage, transformation and analysis solutions that are scalable and reliable.
• Optimize existing pipelines for performance, scalability and reliability.
• Work with teams to turn business requirements into technical solutions.
• Run unit, integration and system tests to protect data accuracy and pipeline strength.
• Write and maintain automated unit, integration and end-to-end tests.
• Monitor and troubleshoot pipelines in production.
• Support data scientists and analysts.
• Stay current with emerging big data technologies.

Required

Skills and Qualifications

• Big data engineering with Hadoop, Hive and Trino, including experience with petabytes of data and distributed systems.
• Apache Spark development and performance tuning, including Spark internals and troubleshooting at petabyte scale.
• Advanced SQL, including window functions, multi-table joins and query optimization.
• AWS cloud engineering, including S3, EMR, Glue and Athena, and scaling clusters.
• Python or Scala programming.
• AI and LLM tooling, including Git Hub Copilot and prompt engineering.
• A Bachelor's degree in Computer Science, Information Systems or a related field with at least 5 years of related experience, or equivalent training or work experience.
• Technical expertise in object-oriented and database concepts and experience delivering enterprise-quality solutions in an iterative or Agile environment.

Preferred Skills

The listing names these secondary skills as nice to have:

• CI/CD
• Java
• Systems troubleshooting
• ETL
• Automated testing

A Master's degree is also listed as preferred.

Tools and Technologies

• Hadoop, Hive and Trino
• Apache Spark
• AWS: S3, EMR, Glue and Athena
• Python or Scala, and SQL

Work Environment and Collaboration

The listing describes a hybrid role in Rockville, MD, working with cross-functional teams, data scientists and analysts.

Why This Big Data Engineer Role Stands Out

Editorial note: this section is general guidance for readers. It is not part of the confirmed job requirements.

The listing stresses experience with petabytes of data and distributed systems. It centers on Hadoop, Hive, Trino and Spark, backed by AWS services such as S3, EMR, Glue and Athena.

It also lists AI and LLM tooling, including Git Hub Copilot and prompt engineering, among its top skills.

Who Will Thrive Here

The listing asks for Spark performance tuning, advanced SQL and automated unit, integration and end-to-end testing.

The source notes that two positions are open.

Professional Expectations

Editorial note: this section is general guidance for readers. It is not part of the confirmed job requirements.

The listing asks for testing, production monitoring and troubleshooting, and performance optimization.

Frequently Asked Questions
1. What does a Big Data Engineer do?

You build, optimize and monitor large-scale data pipelines.

2. Which skills are essential?

Hadoop, Spark, AWS, advanced SQL and Python or Scala.

3. How much experience is required?

At least 5 years of related experience, or equivalent training or work experience.

4. Is a degree required?

A Bachelor's degree in a related field, or equivalent experience.

5. Which tools will I use?

Hadoop, Hive, Trino, Spark and AWS services such as S3, EMR, Glue and Athena.

6. Where is this job located?

Rockville, MD.

7. Is the role onsite, hybrid or remote?

Hybrid.

8. What type of employment is this?

The listing does not state the employment type. Please ask when you apply.

9. What does a typical day look like?

The listed duties include building and optimizing pipelines, testing and monitoring production pipelines. This is a general picture based on the listed duties, not a confirmed daily schedule.

10. What communication skills matter most?

The listing mentions collaboration with cross-functional teams, data scientists and analysts.

11. Which concepts should I understand well?

Spark internals, distributed systems and petabyte-scale performance tuning.

12. Are there…