Data engineer

Description:

-

• Bachelor's degree in the areas of Computer Science, Engineering, Information Systems, Business, or equivalent field of study required
• 7+ years of experience in working with data solutions.
• 3+ years of experience coding in Python, or Scala or similar scripting language.
• 3+ years of experience in developing data pipelines in AWS Cloud Platform (preferred), Azure, or Snowflake at scale.
• 2+ years Experience in designing and implementing data ingestion with real-time data streaming tools like Kafka, Kinesis or any similar tools. SAP/Salesforce or other cloud integrations are preferred.
• 3+ years experience working with MPP databases such as Snowflake (Preferred), Redshift or similar MPP databases.
• 2+ years experience working with Serverless ETL processes (Lambda, AWS Glue, Matillion or similar)
• 1+ years experience with big data technologies like EMR, Hadoop, Spark, Cassandra, Mongo
  
  DB or other open source big data tools.
• Knowledge of professional software engineering best practices for the full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations.
• Experience designing, documenting, and defending designs for key components in large distributed computing systems
• Demonstrated ability to learn new technologies quickly and independently
• Demonstrated ability to achieve stretch goals in a very innovative and fast paced environment
• Ability to handle multiple competing priorities in a fast-paced environment
• Excellent verbal and written communication skills, especially in technical communications
• Strong interpersonal skills and a desire to work collaboratively

Experience participating in an Agile software development team, e.g. SCRUM

• Responsible for the building, deployment, and maintenance of critical scalable Data Pipelines to assemble large, complex sets of data that meet non-functional and functional business requirements
• Work closely with SMEs, Data Modeler, Architects, Analysts and other team members on requirements to build scalable real time/near real time/batch data solutions.
• Contributes design, code, configurations, and documentation for components that manage data ingestion, real time streaming, batch processing, data extraction, transformation, and loading into Data Lake/Cloud Data Warehouse/MPP (Snowflake/Redshift/similar Technologies ) .
• Owns one or more key components of the infrastructure and works to continually improve it, identifying gaps and improving the platform's quality, robustness, maintainability, and speed.
• Cross-trains other team members on technologies being developed, while also continuously learning new technologies from other team members.
• Interacts with technical teams across and ensures that solutions meet customer requirements in terms of functionality, performance, availability, scalability, and reliability.
• Performs development, QA, and dev-ops roles as needed to ensure total end to end responsibility of solutions.
• Keep up with current trends in big data and Analytics , evaluate tools and pace yourself for innovation.

Mentor Junior engineers ,create necessary documentation and Run-books while still being able to deliver on goals