Projects

Body Multi-Tracker

An app to track from one to four bodies, and to send different kinds of data.

Body Multi-Tracker is an app to track from one to four bodies, and to send different kinds of data. A webcam watches the performers. I click the points that matter, a hand, a fingertip, the head, a knee, and the app streams that movement.

The data can go out as MIDI to Ableton Live, or to any synthesizer that accepts MIDI. The same points can go out as OSC, Open Sound Control, for other software. In Controllers mode, each point sends two control changes, X and Y. In Notes mode, height plays a pitch and speed sets the loudness.

It tracks 46 points per person, including the head, the torso, the limbs, and all five fingertips. Nothing is uploaded. The tracking happens on the computer in front of you.

Notes mode: the hand and all five fingers are tracked, and the stage shows the notes of the scale from C3 to C5.
Notes mode. The right hand and its five fingertips are tracked. Moving a point up and down plays the notes of the pitch.
Controllers mode: the nose is selected and sends X on CC 22 and Y on CC 23.
Controllers mode. The nose is selected. Its X and Y go out as MIDI control changes, CC 22 and CC 23.