Big Data Architect; AWS

Overview

Soft Serve is a global digital solutions company with headquarters in Austin, Texas, founded in 1993. Our associates work on 2,000+ projects with clients across North America, EMEA, APAC, and LATAM. We are about people who create bold things, make a difference, have fun, and love their work.

Big Data and Analytics is the Center of Excellence's data consulting and engineering branch. Hundreds of data engineers and architects nowadays build data and analytics end-to-end solutions, from strategy to technical design and proof of concepts to full-scale implementation. We have customers in the healthcare, finance, manufacturing, retail, and energy domains.

We hold top-level partnership statuses with all the major cloud providers and collaborate with many technology partners like AWS, GCP, Microsoft, Databricks, Snowflake, Confluent, and others.

Please note that this is a hybrid role, with a requirement to attend the office at least three times per week.

IF YOU ARE

• Experienced with both batch and streaming data processing
• Ready to lead a team of data engineers
• Skilled in developing scalable data solutions using AWS services, building efficient data pipelines
• Familiar with Python, Scala, or Java (preferably Python) and SQL for data manipulation and querying
• Knowledgeable in big data technologies such as Apache Spark, Databricks, or Flink for advanced data processing
• Familiar with orchestration tools like Apache Airflow or Managed Apache Airflow (MWAA) for scheduling workflows
• Experienced with data streaming platforms like Apache Kafka, Amazon Managed Streaming for Apache Kafka (MSK), or Kinesis for real-time data processing
• Familiar with Kafka, streaming concepts, Avro file format, SQL, Git Hub, and Snowflake
• Adept at utilizing data warehouses such as Amazon Redshift or Snowflake for storage and analytics
• Proficient in translating customer requirements into development tasks, estimating timelines, and guiding teams toward project completion

AND YOU WANT TO

• Own source-to-target mappings, drive discovery of new data sources, coordinate with business stakeholders on data ingestion
• Assist the Data Engineering Team with clear implementation scope definitions
• Participate in architecture decision-making and scale the data platform by planning new data ingestion pipelines and maintaining the data model
• Drive requirements gathering with business and tech stakeholders and assist with data modeling for systems using both Snowflake and Kafka
• Create and maintain documentation for the data schema and model
• Work in a team of data-focused engineers dedicated to continuous learning, improvement, and knowledge-sharing
• Be involved in the full project lifecycle, from initial design and proof of concepts (PoCs) to minimum viable product (MVP) development and full-scale implementation
• Investigate new technologies, build internal prototypes, and share knowledge with the Soft Serve Big Data Community

• Work with a cutting-edge technology stack, including services from major cloud providers that are at the forefront of innovation and may not yet be generally available
• Engage with customers from diverse backgrounds, ranging from large global corporations to emerging crypto startups preparing to launch their first product
• Address different business and technology challenges, engage in impactful projects, use top-notch technologies, and drive multiple initiatives as a part of the Center of Excellence
• Support your technical and personal growth — we have a dedicated career plan for all roles in our company
• Investigate new technologies, build internal prototypes, and share knowledge with the Soft Serve data community
• Upskill with full access to Udemy learning courses
• Pass professional certifications, encouraged and covered by the company

This posting includes an anticipated salary range that Soft Serve expects to offer for this position. The range provided reflects the base salary only. Final compensation within this range will be determined based on a number of factors, including, but not limited to: geographic location, relevant experience, education, certifications, skills, budget, and market conditions.

The anticipated salary range for this role is 93,730 – 173,040 USD annually, in line with our internal compensation framework and budget allocation for the role. Most candidates are offered a salary within this disclosed range. If the role is eligible for a variable pay plan, this would be considered separately and may provide additional earnings beyond the base salary range reflected here.

Please note, the information provided in this posting is a general summary and may not include all compensation elements or benefits associated with the position.

All qualified applicants will receive consideration for employment without regard to race, color, religion, age, sex, national origin, disability, sexual orientation, gender identity/expression, or protected veteran status. Soft Serve is an Equal Opportunity Employer.