Parallel Software Engineer Engineering Mountain View, CA​/Hyderabad, IN​/Remote

Parallel Software Engineer

Mountain View, CA / Hyderabad, IN / Remote

Data Pelago is at the forefront of revolutionizing data processing for traditional analytics and cutting-edge GenAI preprocessing. We are building an innovative data processing engine that is transforming how Apache Spark, Apache Flink, Ray and others operate on diverse, large-scale data. Ourteam of engineers drive and adopt advances in hardware-accelerated computing, parallel processing of large-scale data, query optimization, distributed systems, compilers, machine learning, andcloud-native computing.

We are looking for specialists to join our engineering team and shape thefuture of accelerated data processing.

As a Parallel Software Engineer, you will be a key individual contributor in developing advanced parallel software that unlocks the full potential of diverse hardware accelerators including GPUs and SIMD-capable CPUs. You will enhance functional breadth, performance, scale, and reliability of data processing operators that are integral to our data processing engine. This is a unique opportunity to make a significant impact on a category-defining product and work with a talented team of engineers.

• Architect:
  Influence the architecture of how our data processing engine harnesses the par-allelism in GPUs and SIMD-capable CPUs efficiently in processing diverse large-scale data.
• Design:
  Lead design of functional and performance enhancements to operators and functions that are accelerated by our engine.
• Core Development:
  Individually design, implement, test, optimize, and maintain parallel implementations of operators and functions on diverse acceleration hardware.
• Innovation and Differentiation:
  Analyze advances in accelerated computing hardware, programming models, and related tools and ensure our engine extends technology and product leadership.
• Collaboration:
  
  Partner effectively with the execution engine engineering team in integrating parallel software components with the overall engine.
• Continuous Improvement:
  Foster best practices in design and code reviews, testing, CI/CD,and issue resolution to maintain highest product quality, security, efficiency, & productivity.

• Bachelor's degree in Computer Science or a related field with 5+ years of relevant experience OR a Master's degree in Computer Science or a related field with 3+ years of relevant experience.
• 3+ years of deep technical experience in developing production applications that process large scale data using SIMD-extensions of CPUs (e.g., AVX), GPU programming models (e.g., CUDA, ROCm), or equivalent accelerated computing framework.
• Demonstrated experience working with software libraries, development tools, and profiling tools specific to parallel and accelerated computing.
• Demonstrated experience in troubleshooting and resolving functional and performance anomalies in both pre- and post-production scenarios.
• Strong knowledge of computer architecture.
• Exceptional programming skills in C, C++.
• Extensive development experience in Linux environments.
• Strong analytical and problem-solving skills with a passion for performance optimization.

We value face-to-face collaboration, but recognize that talent can be found anywhere. Our engineering team works at our headquarters in Mountain View, CA, at our India office in Hyderabad, and atremote locations.

• Technology Leadership:
  Shape the architecture and development of core parallel data processing functions of our engine.
• Cutting-Edge Innovation:
  Work on challenging problems at the forefront of accelerated computing and data processing.
• Significant Impact:
  Your contributions will directly impact the performance and scalability of our mission-critical platform.
• Growth:
  Expand your technical expertise and scope of responsibilities working with other talented engineers and with a growing product.
• Competitive compensation, stock options, comprehensive benefits package, leadership development opportunities.