Home
Artists
Search Recent Random
Posts
Search DMs Popular Hash Lookup Tags Random
Importer
Import FAQ
Register
Partychan Telegram ThePornDude
Home Artists Posts Import Register

Face, Hand, Pose Tracking & More with MediaPipe GPU Plugin for TouchDesigner (Patreon)

Published:
2023-09-09 00:18:36
Imported:
2023-11

Content

Hey! In this tutorial, we'll go over the new free open-sourced MediaPipe plugin for TouchDesigner that's GPU accelerated and works on Mac and PC with no instillation. The plugin supports face, hand, pose and object tracking with multi-person face detection, object detection, and image segmentation / background removal.


Link to the GitHub to download the Plugin: https://github.com/torinmb/mediapipe-touchdesigner


Dom Scott's website: https://art.domscott.ca/home

Link to Torin's Patreon: https://patreon.com/tblankensmith

Link to the body tracking alternative: https://www.youtube.com/watch?v=83StND-y4fY


Link to OBS for using video with MediaPipe on Mac: https://obsproject.com/

Link to SpoutCam for using video on PC: https://github.com/leadedge/SpoutCam/releases


Files

Face, Hand, Pose Tracking & More with TouchDesigner MediaPipe GPU Plugin

Hey! In this tutorial, we'll go over the new free open-sourced MediaPipe plugin for TouchDesigner that's GPU accelerated and works on Mac and PC with no instillation. The plugin supports face, hand, pose and object tracking with multi-person face detection, object detection, and image segmentation / background removal. Link to the GitHub to download the Plugin: https://github.com/torinmb/mediapipe-touchdesigner Dom Scott's website: https://art.domscott.ca/home Link to Torin's Patreon: https://patreon.com/tblankensmith Link to the body tracking alternative: https://www.youtube.com/watch?v=83StND-y4fY Link to OBS for using video with MediaPipe on Mac: https://obsproject.com/ Link to SpoutCam for using video on PC: https://github.com/leadedge/SpoutCam/releases 0:00 Intro and Overview 1:44 Project Setup 5:43 Face Landmark Detection 9:37 Hand Tracking + Gesture Recognition 13:49 Pose Tracking 16:37 Object Tracking 18:05 Multi Person Face Detection 19:21 Image Classification 20:56 Image Segmentation / Background Removal 23:58 On Mac Pipe Video into MediaPipe with OBS 26:26 On PC Pipe Video into MediaPipe with SpoutCam 28:30 Outro

Comments

Andrea Bucci

Great! But I had the first issue!! https://github.com/torinmb/mediapipe-touchdesigner/issues/35#issue-1906853027

Anonymous

This might be a silly question, but I am wondering if this works with depth cameras such as Kinect V2. The reason I am asking is that the installation for trying to build is in the dark. If I use a regular webcam, the visual information won't register.

tblankensmith
>>120605646

Hey! I haven't tested it out personally, but in theory it should still work with a black and white texture that you get from an infrared camera

Mercedes Invernizzi Oviedo

Hi Torin. Im new here! I from Argentina, buenos aires. i neet some help to this tutorial. i want to know if you can explain us how take the data of mediapipe to create some visuals with our hands or face, around Interactive Segmentation and Image Embedding. do you have that parte of the video to practice?

tblankensmith
>>129361405

Hey! This is a quick example showing how to work with the hand tracking data: https://www.youtube.com/watch?v=NwqLnNyfOb4

Alec Johnson

Thanks Tori! This has been a lifesaver for me. I bought the Orbbec Gemini 2 camera thinking I could do skeleton tracking in TD on Mac M1, and when that did not work, thought I'd bought the wrong depth camera. I've got everything up and running now thanks to your Mediapipe .toe.