MOTION CAPTURE TECHNOLOGIES DISA Seminar Jakub Valcik24. 3. 2015 Outline  Human Motion and Digitalization  Motion Capture Devices  Optical  Ranging Sensor  Inertial and Magnetic  Comparison Human Motion  Musculature  Skeleton  Weight  Injuries  Movement habits  Pregnancy  State of mind  Shoes  Clothes  Type of surface  Slope of surface  Wind  Gravity  Environment Internal factors External Factors  Brain + Skeleton + Muscles Motion Capture  Sequence of individual frames 𝑚 = (𝑓𝑘) 𝑘=1 𝑛  Captured information  Static body configuration  Position in space  Orientation of body  Silhouette vs. Skeleton Appearance Based Approaches  Eadweard Muybridge, 1878 Boys Playing Leapfrog, printed 1887 Appearance Base Approaches  Originally only regular video cameras, CCTV  Silhouette oriented 𝑓𝑘 ≔ 𝐵 𝑘 𝑥, 𝑦 ∈ {0, 1} Silhouette extraction is a common bottleneck Model Based Approaches  Additional abstraction – 2D, 3D  Stick figure, volumetric model, …  Skeleton  Joint (or end-effector), Bone  Undirected acyclic graph 𝐽 – tree  Joint ~ Vertex, Bone ~ Edge  Static configuration ~ Pose 𝑝 ∈ ℝ3×|𝑉 𝐽 |  𝑚 = (𝑓𝑘) 𝑘=1 𝑛 = (𝑝 𝑘) 𝑘=1 𝑛 Motion Capture Devices  Optical  Markerless  Invasive  Inertial  Magnetic  Mechanical  Radio frequency Only model based approaches Both, appearance and model based approaches Optical Markerless MoCap Devices  3D scene reconstruction  Multiple views  Depth sensors  RGB stereoscopic cameras  Multiple synchronized cameras  Additional sensors  Silhouette extraction  Depth sensing  IR camera, ranging sensor Ranging Sensor - Triangulation  Laser beam + IR camera  Projection grid – ‘structured light’  Known variables  Emitter-camera distance  Dot distribution in grid  Depth ~ dot translation  Grid resolution ↑↓ Object distance  PrimeSense (now Apple)  Project Natal => Kinect Kinect v1 IR Structured Light Kinect v1 – Reference Points Kinect v1 – Point Shift (1) Kinect v1 – Point Shift (2) Ranging Sensor – Time of Flight  Light speed 𝑐 ≐ 300 000 𝑘𝑚 𝑠 = 30 𝑐𝑚 𝑛𝑠  Principles  RF modulated – phase shift  Swiss Ranger 4000  PMD CamCube 3.0  Canesta Vision (now MS)  Range gated  Zcam by 3DV (now MS)  TriDiCam  Direct ToF – 3D flash LIDAR  Advanced Scientific Concepts, Inc. Ranging Sensor Comparison Depth Sensor Maximal Range Resolution Field of View [°] Repeatability [mm](1 Sigma) Kinect v1 10m 640x480 57.8 x 43.3 7.6@2m, 27.5@4m Swiss Ranger 4000 8m 176x144 43 x 34 / 69 x 56 4/6 PMD CamCube 3.0 7m 200x200 40 x 40 3@4m Camera Types Comparison  Interference comparison Camera Type Complex Background Heat Source Other Camera Clothes RGB (B/W) ○○ ●● ●● ○○ IR ●○ ○○ ●● ●● Ranging ●● ●○ ○○ ●○ Optical Markerless MoCap Devices  Stereoscopic video cameras  Sony Playstation Eye  Video camera + ranging sensor – triangulation  MS Kinect v1, Asus Xtion live, Structure Sensor, PrimeSense Carmine 1.08  Video camera + ranging sensor – ToF  MS Kinect v2  360° video cameras  Organic Motion PrimeSens Carmine Asus Xtion Live Structured Sensor Price $300 Fun fact #1: Body part estimation based on ML, Learning phase take 24,000 CPU hours Microsoft Kinect v1 Microsoft Kinect v2 Organic Motion Openstage2 Starting price $40,000 Invasive Optical MoCap Devices  Active vs. Passive  Multiple RGB & IR cameras  Precise, fast  Price, markers, additional electric source (active)  Problems  Marker swapping MoCap Suite Vicon MX Starting price $100,000 ? Inertial & Magnetic MoCap Devices  Acceleration, magnetic flux  No global position nor orientation  Pose initialization  Accumulation of error  Gravity, wiring, reinforced concrete MVN XSens Feature MVN Awinda MVN Link Trackers 17 Wireless 17 Wired Latency 30ms 20ms Wireless range 20m/50m 50m/150m Output rate 60Hz 240Hz Starting price from $12,000 Other MoCap Devices  Mechanical system  Exoskeleton directly measures joint angle rotations  Radio Frequency Positioning  RADAR working on high frequencies >50GHz  Inaccurate, large areas (hundreds of meters2) MoCap Devices Comparison MoCap Software Comparison Joint Tracking Error Kinect v1 Cosgun, A., Bünger, M., & Christensen, H. I. (2013). Accuracy Analysis of Skeleton Trackers for Safety in HRI (p. 1). Percentage of Tracked Frames Kinect v1 Cosgun, A., Bünger, M., & Christensen, H. I. (2013). Accuracy Analysis of Skeleton Trackers for Safety in HRI (p. 1). Thank you for your attention Resources  http://nongenre.blogspot.cz/2010/12/how-kinect- senses-depth.html  http://www.freepatentsonline.com/20100118123.pdf  http://users.dickinson.edu/~jmac/selected- talks/kinect.pdf  http://www.nimbocg.com.br/wp- content/uploads/2013/02/mocap11.jpg  Other sources cited in thesis