Senior Data Engineer

Job Summary :
You will be responsible for designing, building, and maintaining our data infrastructure. This role involves working closely with data scientists, developers, and other stakeholders to ensure data availability, quality, and reliability.

Key Responsibilities :
Design, develop, and maintain scalable data pipelines to collect, process, and store large volumes of data from various sources.
Implement ETL (Extract, Transform, Load) processes to transform raw data into structured formats.
Integrate data from multiple sources, including databases, APIs, third-party services, and data warehouses.
Ensure data consistency and integrity across different data stores.
Design and manage databases, data warehouses, and data lakes to support data storage and retrieval needs.
Optimize database performance and ensure data security.
Ability to write complex queries dealing with complex joins across numerous tables spanning multiple databases for efficient data pulls.
Implement data validation and quality checks to ensure data accuracy and reliability.
Monitor data pipelines and perform troubleshooting and debugging as needed.
Work closely with data scientists, developers and business stakeholders to understand data requirements and deliver solutions.
Communicate effectively with technical and non-technical team members.
Document data pipelines, workflows, and processes.
Establish and promote best practices for data engineering, including coding standards, version control, and testing.

Qualifications :
Bachelor’s degree in computer science, Engineering, Information Technology, or a related field. A master’s degree is a plus.
4+ years of experience in data engineering, software development, or a related field.

Experience with data warehousing solutions such as Amazon Redshift, Google Big Query, Snowflake, Azure Synapse or similar.
Familiarity with big data technologies (e.g., Hadoop, Spark, Kafka).
Proficiency in programming languages such as Python, Java, or Scala.

Experience with SQL and relational databases (e.g., Postgre

SQL, MySQL, SQL Server).
Strong knowledge of ETL tools and frameworks (e.g., Apache NiFi, Apache Airflow, Talend, Informatica).
Understanding of data modeling and database design principles.
Excellent problem-solving and analytical skills.
Strong communication and teamwork abilities.
Ability to manage multiple tasks and prioritize effectively.

Preferred Qualifications :
Consulting experience with named companies is a plus.

Experience with No

SQL databases (e.g., Mongo

DB, Cassandra).
Knowledge of data visualization tools (e.g., Tableau, Power BI).