Senior Data Engineer

About Us

At Cast & Crew, we’ve empowered creativity and supported the global entertainment industry for decades. Together with our family of brands - Backstage, CAPS, Checks & Balances, Final Draft, Media Services, Sargent-Disc, and The TEAM Companies – we operate as a combined entertainment technology and services provider offering industry standard screenwriting accounting software, digital payroll products, data & reporting, and a host of creative tools.

The industry continues to move faster than ever, and the need for our expertise, our technology, and our people has never been greater. We are a production’s best ally every step of the way.#One Cast One Crew

At Cast & Crew, engineers own the systems they build and operate. Our platforms support mission-critical workflows across the entertainment industry, where reliability, scalability, and accountability matter. We hire engineers who take ownership, solve problems end-to-end, automate wherever possible, and continuously raise the bar for engineering excellence.

Cast & Crew is seeking an experienced Senior Data Engineer to design, build, and operate scalable data platforms that power mission-critical applications, analytics, reporting, and AI-driven solutions across the entertainment production ecosystem.

This role focuses on building reliable, cloud-native data pipelines that ingest, transform, secure, and serve operational data for internal engineering teams, business intelligence, and customer-facing applications. You will work across structured and semi-structured data, enabling modern analytics, AI workloads, and real-time event processing.

You will collaborate closely with software engineers, platform engineers, product teams, and data consumers while leveraging AI-assisted development tools to accelerate delivery and improve engineering productivity.

• Bachelor’s degree in Computer Science, Software Engineering, Information Systems, or related field

• 5+ years of experience building production data platforms and pipelines

• Strong Python development experience

• Strong SQL development and query optimization skills

• Experience designing and maintaining ETL/ELT pipelines

• Experience working with relational databases (PostgreSQL, MySQL, Aurora)

• Experience working with No
  
  SQL databases (DynamoDB, MongoDB, document databases)

• Experience building data lakes and cloud-native data architectures

• Experience with Snowflake, Big Query, Redshift, or similar cloud data warehouses

• Experience with AWS cloud services including S3, Lambda, Event Bridge, DynamoDB, ECS/EKS, and IAM

• Experience building event-driven and streaming data solutions

• Experience working with Kafka, SNS/SQS, Kinesis, or similar messaging technologies

• Experience with data orchestration tools such as Airflow, Prefect, or Dagster

• Experience with Git and modern CI/CD pipelines (Azure Dev Ops, Git Hub Actions, or similar)

• Experience building highly available, scalable production systems

• Strong understanding of data modeling, partitioning, indexing, and performance optimization

• Experience monitoring and operating production data platforms

• Experience with Apache Spark or distributed processing frameworks

• Experience with Debezium, CDC, or change data capture architectures

• Experience with Iceberg, Delta Lake, or modern lakehouse technologies

• Experience with Open Search or Elasticsearch

• Experience supporting AI and machine learning data platforms

• Experience building feature stores or vector databases

• Experience with document processing (OCR, Textract, PDF extraction)

• Experience in media, payroll, financial systems, or enterprise SaaS platforms

• Experience using AI-assisted development tools such as Claude Code or Git Hub Copilot

• Enterprise-scale data ingestion and transformation pipelines

• Cloud-native data platforms supporting analytics and operational workloads

• Real-time event-driven data processing solutions

• Data models supporting reporting, AI, and customer-facing applications

• Secure, scalable integrations between enterprise systems

• Automated monitoring, testing, and deployment pipelines for data infrastructure

• Highly reliable and scalable production data pipelines

• High-quality, trusted data powering operational and analytical systems

• Low-latency, automated data movement across enterprise platforms

• Engineering best practices including automation, observability, and testing

• Strong collaboration with software engineering and platform teams

• Continuous improvement through…