Doctoral Student in Neuromotor Interfaces Dexterous Robot Teleoperation; multimodal egocentric vision + EMG

The Sensing, Interaction & Perception Lab invites applications for a fully funded PhD position at ETH Zürich at the intersection of robotics, wearable sensing, signal processing, machine learning, and human-computer interaction.

The goal of this PhD is to develop neuromotor interfaces for dexterous robot teleoperation. The primary sensing modality will be surface electromyography (sEMG) measured at the wrist or forearm, with the aim of decoding subtle hand and finger activity, continuous movement, and motor intent in real time.

A secondary component of the project will investigate egocentric vision as a complementary sensing modality. While EMG provides information about human motor intent, egocentric cameras can provide context about the surrounding scene, manipulated objects, hand-object interactions, and task state. Combining these modalities creates opportunities for interfaces that understand both what the user intends to do and what is happening in the environment
.

The PhD will focus on the computational and sensing methods required to make such systems robust, generalizable, and effective in real interactive settings, including multimodal learning and reasoning across neuromotor and egocentric visual signals
, while exploring applications in Mixed Reality and other interactive systems.

Note:

This is not a position in biomedical engineering and there is no health focus.

Wearable EMG provides a direct and unobtrusive way to sense muscle activity underlying hand and finger movements. This creates opportunities for interfaces that can recognize subtle actions, continuously estimate movement, and infer motor intent before or without large observable movements.

Making such interfaces work reliably outside controlled settings remains a substantial research problem. EMG varies across users and recording sessions and is sensitive to electrode placement, contact conditions, movement, and other sources of noise. This motivates research in signal processing, temporal machine learning, representation learning, adaptation, and real-time inference
.

A second challenge is that neuromotor signals alone provide limited information about what the user is interacting with and why a particular movement is occurring. The project will therefore combine EMG with egocentric vision to capture objects, hands, contacts, affordances, and task state. This creates a multimodal research problem: learning representations that combine neuromotor and visual information and developing methods for reasoning about human intent, hand-object interaction, and the state of an ongoing manipulation task
.

For robotic teleoperation, these multimodal signals can support systems that jointly reason about what action the user intends, which object or target the action refers to, and how that intent should be translated to a robot with a different embodiment. Relevant problems include dexterous manipulation, shared autonomy, multimodal intent inference, and reasoning over sequences of human and robot actions.

• Develop signal-processing and machine-learning methods for multichannel EMG, including discrete and continuous decoding of hand and finger activity
• Develop egocentric computer-vision methods for understanding hands, objects, hand-object interactions, contacts, affordances, and task state
• Develop multimodal learning and reasoning methods that combine EMG with egocentric vision to infer motor intent and interaction context
• Investigate reasoning over objects, actions, interaction sequences, and task state to resolve ambiguous motor signals and anticipate intended actions
• Apply methods to dexterous robot teleoperation
  , including mapping human motor intent to robot manipulators and dexterous hands

optionally:

• Prototype wearable sensing systems
  , including electrode configurations, signal acquisition, embedded processing, and hardware and software integration
• Develop and evaluate applications in robotics, Mixed Reality
  , and other interactive systems

and as in each PhD

• Disseminate your findings in publications at top-tier venues and open-source research results and tools where appropriate
• Present research findings at…