Data Engineer

Join a hybrid Data Engineer opportunity focused on building and improving modern big data solutions for ingestion, integration, and transformation. This role supports impactful data initiatives and offers flexibility to work from an East Coast office or on-site in Knoxville, TN when needed.
Responsibilities

• Design and implement data engineering components that support client requirements and technical solution goals.
• Build and maintain batch and real-time data ingestion, integration, and transformation workflows.
• Work with large-scale data processing and storage technologies to support reliable, efficient data pipelines.
• Contribute to analytics, search, and aggregation capabilities across data platforms.
• Participate in technical design discussions and help ensure the overall health and performance of the solution.
• Support cloud-based data platform services and optimize data workflows for performance and scalability.
• Collaborate with project teams in a hybrid work environment and attend onsite interviews or equipment pickup when required.

Skills

• At least 4 years of hands-on experience with big data technologies.
• Experience with Hadoop ecosystem tools such as HDFS, Kafka, Spark, Spark Streaming, Hive, NiFi, Airflow, Flink, Storm, Oozie, or similar platforms.
• Strong programming experience in Java, Python, or Scala, with Java preferred.
• Working knowledge of No
  
  SQL and MPP data platforms such as HBase, MongoDB, Cassandra, AWS Redshift, Azure SQLDW, or Google Cloud Platform Big Query.
• Experience with AWS data platform services.
• Ability to translate business needs into technical design and implementation.
• Strong problem-solving skills and the ability to work independently on design discussions and solution delivery.

Preferred Skills

• Experience with traditional ETL tools such as Informatica or Talend.
• Familiarity with database technologies such as Oracle, MySQL, SQL Server, or Postgres.
• Knowledge of data governance practices, including security, lineage, and catalog tools like Collibra or Alation.
• Exposure to distributed messaging tools such as ActiveMQ, RabbitMQ, or Solace.
• Experience with search/indexing and microservices architectures.
• Background in performance tuning and data pipeline optimization.
• Cloud data certifications or other relevant big data credentials.

Horizontal is committed to building an inclusive workplace where diverse perspectives are valued, respected, and encouraged. We welcome candidates from all backgrounds to apply and believe that equity and belonging strengthen our teams and the solutions we create.

By applying for this position, you acknowledge and agree that Horizontal Talent may contact you regarding your application using automated technology, including phone calls, SMS/text messages, or email, which may be delivered by our virtual AI recruiter, Alex.