Sr Data Engineer-ETL

Job Title:

Sr. Data Engineer-ETL Duration:
Long-term Main Skill

• Over 10+ years of experience in the Software Development Industry.
• We need data Engineering exp - building ETLs using Spark and SQL, real‑time and batch pipelines using Kafka/firehose, experience with building pipelines with Databricks/Snowflake, experience with ingesting multiple data formats like JSON/Parquet/Delta etc.

• You have a BS or MS in Computer Science or a similar relevant field.
• You work well in a collaborative, team‑based environment.
• You are an experienced engineer with 3+ years of experience.
• You have a passion for big data structures.
• You possess strong organizational and analytical skills related to working with structured and unstructured data operations.
• You have experience implementing and maintaining high‑performance/high‑availability data structures.
• You are most comfortable operating അന്ത within cloud‑based ecosystems.
• You enjoy leading projects and mentoring other team members.

• Over 10 years of experience in the Software Development Industry.
• Experience or knowledge of relational SQL and No
  
  SQL databases.
• High proficiency in Python, PySpark, SQL and/or Scala.
• Experience in designing and implementing ETL processes.
• Experience in managing data pipelines for analytics and operational use.
• Strong understanding of in‑memory processing and data formats (Avro, Parquet, JSON etc.).
• Experience or knowledge of AWS cloud services: EC2, MSK, S3, RDS, SNS, SQS.
• Experience or knowledge of stream‑processing systems: i.e., Storm, Spark‑Structured‑Streaming, Kafka consumers.
• Experience or knowledge of data pipeline and workflow management tools: i.e., Apache Airflow, AWS Data Pipeline.
• Experience or knowledge of big data tools: i.e., Hadoop, Spark, Kafka.
• Experience or knowledge of software engineering tools/practices: i.e., Git Hub, VS Code, CI/CD.
• Experience or knowledge in data observability and monitoring.
• Hands‑on experience in designing verrassation data schema life‑cycles.
• Bonus – Experience in tools like Databricks, Snowflake and Thought Spot.

Mid‑Senior level

Contract

Analyst

Entertainment Providers

Referrals increase your chances of interviewing at Info Vision Inc. by 2x.