Robotics Data Pipeline Engineer – Multimodal Data

Robotics Data Pipeline Engineer – Multimodal Data

As a Data Pipeline Engineer, you will architect and scale the data infrastructure that feeds our foundation models. Your primary mission is to extract, augment, and align human dexterous manipulation data from massive complex, multi-sensor and egocentric video datasets. Crucially, you will build advanced post-processing algorithms to perform deep force analysis and infer hidden states from raw data—such as processing direct force-torque outputs to quantify grasp dynamics, estimating contact forces from visual cues, extrapolating heavily occluded hand positions, or deriving 3D geometry from 2D frames.

You will use spatial, temporal, and cross-modal data augmentation to multiply the value of every minute of data our teleoperation team collects.

What You Will Be Doing

• Architect end-to-end ingestion pipelines that take raw, unstructured recordings—egocentric video, teleoperation sessions, third-party open datasets—and produce indexed, queryable, training-ready datasets. This includes temporal segmentation of long recordings into action clips, metadata and scene-graph extraction, embedding-based retrieval, and language annotation workflows.
• Design cross-modal validation systems that verify video, proprioception, force/haptic signals, and language annotations agree with each other—e.g., reprojecting robot state into the image plane to confirm video–state consistency, and VLM-assisted checks that instructions match observed behavior.
• Orchestrate hand-tracking, segmentation, depth estimation, 3D reconstruction, and pose-tracking modules; retargeting human demonstrations into robot trajectories; and running simulation-in-the-loop validation (kinematic feasibility, physics replay, motion-consistency filtering) so synthesized data is physically grounded, not just visually plausible.
• Implement robust data augmentation strategies (spatial transformations, temporal scaling, synthetic viewpoints, and sensor noise injection) to expand expert trajectories and improve the robustness of our learning models.
• Unified state–action representations across differing embodiments, coordinate frames, rotation conventions, gripper/hand parameterizations, and sampling rates—with per-dimension validity masking and per-source normalization so that adding a new robot or sensor is a configuration change, not a rewrite.
• Build the tooling that lets researchers query, visualize, and audit datasets (clip browsers, trajectory viewers, annotation review UIs), and turn model-failure analyses into new curation rules and targeted re-collection requests.

What We Are Looking For

• M.S., or Ph.D. in Computer Science, Data Engineering, Machine Learning, Robotics, Mechanical Engineering, or a related field.
• Deep expertise in Python and extensive experience with PyTorch, specifically in handling custom data loaders for multimodal datasets.
• Experience analyzing and processing complex time-series data from force-torque (F/T) sensors, load cells, or tactile arrays, ensuring pristine alignment with visual frames.
• Mastery of video processing pipelines and libraries (OpenCV, FFmpeg, Decord) and managing the I/O bottlenecks of terabyte-scale video datasets.
• Solid working knowledge of 3D geometry and robotics data: coordinate frames and transforms, rotation representations, camera intrinsics/extrinsics, forward/inverse kinematics, URDF—enough to build automated checks that catch geometric inconsistencies in the data.
• Proven ability to implement programmatic and generative data augmentation techniques for computer vision and time-series data.

Bonus Skills

• Experience with NVIDIA's robotic software stack (Open X-Embodiment, DROID, Agi Bot World, Ego Dex, or similar).
• Familiarity with the modern perception toolbox as a user: segmentation (SAM-family), monocular depth, hand/body pose estimation (MANO/SMPL), 6-DoF object pose tracking, point tracking—you don't need to train these models, but you should be comfortable composing and evaluating them in a pipeline.
• Familiarity with distributed data processing systems (Ray, Apache Spark) for cluster computing.
• Background in generating or utilizing synthetic robotic data via simulation (Omniverse, Mu Jo Co ).
• Experience integrating spatial awareness or tactile data representations (e.g., Fourier encoding) into visual pipelines.

Persona AI is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, age, disability, veteran status, or any other characteristic protected by applicable federal, state, or local law.