Kernel Driver Software Engineer

Requirements

• Proficiency in C/C++
• Strong understanding of kernel-mode driver development and debugging
• Deep understanding of operating system internals (Linux preferred)
• Experience with hardware/software interfacing and device drivers
• Experience with memory management and synchronization in kernel environments
• Strong understanding of PCIe and other hardware interfaces
• Experience with device virtualization technologies, including SR-IOV and VFIO
• Strong understanding of kernel memory mapping, page table configuration, and IOMMU
• Familiarity with hardware-software co-design principles
• Proven ability to analyze complex technical problems and provide effective solutions
• Excellent communication and collaboration 1 skills
• Experience with version control systems (e.g., Git)
• Experience with debugging tools (e.g., gdb, kgdb)
• (Desirable) Candidates with experience in developing and debugging kernel-mode drivers for GPU or other accelerator devices
• (Desirable) Candidates with a strong understanding of hardware/software interactions
• (Desirable) Candidates with experience in optimizing driver performance for demanding workloads
• (Desirable) Candidates with experience in ML workloads
• (Desirable) Candidates who have debugged complex hardware and software interactions, especially in virtualized environments
• (Desirable) Candidates with experience in implementing and optimizing SR-IOV and VFIO
• (Desirable) Candidates with in-depth knowledge of kernel memory mapping, page tables, and IOMMU
• (Desirable) Candidates with experience in hardware-software co-design projects
• (Desirable) Experience with GPU driver development
• (Desirable) Experience with CUDA, OpenCL, or other GPU programming models
• (Desirable) Experience with performance profiling and benchmarking tools (perf, VTune)
• (Desirable) Knowledge of hardware virtualization techniques, including para-virtualization
• (Desirable) Experience with CI/CD pipelines
• (Desirable) Experience with Rust
• (Desirable) Experience with ML frameworks like Tensorflow or Pytorch
• (Desirable) Experience with data center orchestration technologies (Kubernetes, Docker)

• Kernel-Mode Driver Development:
  Design, develop, and maintain kernel-mode drivers ensuring high reliability, informative debug, and optimal performance
• Performance Optimization:
  Analyze and optimize driver performance for demanding AI workloads, focusing on minimizing latency and maximizing throughput
• Hardware Integration and Co-Design:
  Collaborate closely with hardware engineers throughout the ASIC design process.
• Virtualization Support:
  Implement driver support for device virtualization technologies, including SR-IOV, VFIO, and para-virtualization
• Memory Management:
  Implement efficient memory management strategies considering kernel memory mapping, page tables configuration, NUMA awareness for device data caching, and IOMMU configuration
• Security:
  Build kernel drivers fundamentally designed to support and maintain security across host processes, physical memory spaces, and device attestation
• Debugging and Troubleshooting:
  Diagnose and resolve complex driver-related issues, using common kernel debugging tools and techniques (ftrace, dmesg, etc.) to identify and fix bugs
• Synchronization and Concurrency:
  Design and implement synchronization mechanisms to handle concurrent access to multiple accelerators
• System Validation and Testing:
  Develop and execute comprehensive test plans to validate driver functionality, stability, and performance in manufacturing and in general production environments
• Collaboration and Troubleshooting:
  Collaborate with software and hardware teams to diagnose and resolve complex system-level issues
• Develop and optimize kernel-mode drivers for new ML accelerators
• Implement and optimize memory management, including kernel memory mapping and IOMMU configurations, for high-bandwidth data transfers
• Debug and resolve complex driver-related issues impacting ML workload performance
• Develop performance benchmarks and profiling tools to analyze driver performance
• Integrate driver support for advanced features like hardware virtualization and security, including SR-IOV and VFIO
• Optimizing PCIe communication between the host and PCIe devices, using advanced equipment like PCIe analyzers
• Implement and debug power management features for PCIe devices
• Integrating ML accelerators into containerized and virtualized environments
• Implementing and optimizing para-virtualization techniques for PCIe devices
• Configure and optimize page tables for efficient memory access from the ML accelerator
• Participate in hardware-software co-design reviews across teams to optimize performance and power efficiency