Healthcare and Human-Machine Interaction

In systems for therapy and rehabilitation, the person is part of the captured scene. Posture and movement are the decisive information here. The same methods a vehicle uses to perceive its environment also recognise the poses and movements of people.

Recognition without anyone wearing sensors

Image-based analysis needs no markers and no devices on the body. This preserves freedom of movement and lowers the barrier to entry, especially for people with impairments. A stereo camera forms the core of our system for three-dimensional person detection. Learning-based methods recognise body parts such as the head, arms, hands, legs and torso; tracking over time keeps the recognition stable. Large gestures are captured as well as very fine movements.

From recognition to feedback

Recognition alone helps no one. Its value arises when the system responds immediately. In the therapy device MotionComposer, which we develop with our partner MotionComposer GmbH, movement becomes music and sound in real time. The feedback playfully motivates people to move, for all age groups and abilities.

To achieve this, sensors, analysis and output must be designed as one complete system. Delays that would go unnoticed in an analysis at a desk destroy the experience here.

More than the camera

Not every movement can be captured well with a camera. That is why we develop systems that include further sensors:

• a sensor chair whose pressure sensors capture posture and weight shift,
• sensor shoes,
• an interactive therapy table for motor and cognitive exercises of the kind used in occupational therapy, physical therapy and rehabilitation.

The data from several sensors is merged so that the system reliably recognises what the person is doing at the moment. This is sensor fusion, applied to people.

Interaction outside of therapy

The recognition of people, posture and gestures is also needed wherever people and machines share a space: in production, when automated guided vehicles and robots must reliably detect people, or when a machine can be controlled with gestures.

Collaboration

We develop the technology; the therapeutic assessment lies with the specialists. We maintain regional contacts with therapy facilities and clinics.