Lead Data Engineer; Big Data Specialist

Our client, a cutting-edge data analytics firm, is seeking a visionary Lead Data Engineer with specialized expertise in Big Data technologies to join their distributed team. This pivotal role involves architecting, building, and maintaining scalable data pipelines and infrastructure that enable advanced analytics and machine learning initiatives. You will lead a team of talented data engineers, drive innovation in data processing, and ensure the reliability and efficiency of our client's data ecosystem.

This is a fully remote position, offering the ultimate flexibility to contribute to critical projects from anywhere in the world. If you are passionate about harnessing the power of Big Data and have a proven track record of building robust data solutions, this is an exceptional opportunity to shape the future of data engineering.

• Lead the design, development, and optimization of large-scale batch and real-time data pipelines.
• Build and manage robust, scalable, and fault-tolerant data infrastructure using Big Data technologies (e.g., Spark, Hadoop ecosystem, Kafka).
• Develop and maintain data warehousing solutions and data lakes.
• Implement ETL/ELT processes to ingest, transform, and load data from various sources.
• Ensure data quality, integrity, and security across all data systems.
• Collaborate with data scientists, analysts, and software engineers to understand data needs and deliver effective solutions.
• Mentor and guide junior data engineers, fostering best practices and technical excellence.
• Automate data processes and monitor system performance for optimal efficiency.
• Stay current with emerging Big Data technologies and best practices.

• Master's or Bachelor's degree in Computer Science, Engineering, or a related quantitative field.
• 5+ years of experience in data engineering, with a strong focus on Big Data technologies.
• Proven experience designing and implementing data pipelines using frameworks like Apache Spark.
• In-depth knowledge of distributed systems and Big Data technologies (Hadoop, Hive, HDFS, Kafka).
• Proficiency in programming languages such as Python, Scala, or Java.
• Strong SQL skills and experience with various database technologies (relational and No
  
  SQL).
• Experience with cloud data platforms (AWS, Azure, GCP) is highly desirable.
• Familiarity with data warehousing concepts and data modeling techniques.
• Excellent leadership, problem-solving, and communication skills.
• Ability to work effectively in a remote, collaborative team environment.

• Highly competitive salary and performance-based bonuses.
• Comprehensive health, dental, and vision insurance.
• Generous stock options or equity grants.
• Full remote work flexibility, allowing you to work from anywhere.
• Opportunities for continuous learning and professional development.
• Access to the latest Big Data tools and technologies.
• Collaborative and innovative team culture.
• Regular virtual team events and professional networking opportunities.