Note: Tools from the capture studio (MoveBox) can be imported to project 2 if there is a need to record the motions of the participant or user for later use. To animate Microsoft RocketBox avatars with predicted 3D poses, the toolbox first extracted the joints data from pose parameters, computed the transformation between SMPL and RocketBox skeleton structures, and then mapped to the corresponding joint in the Microsoft RocketBox avatar skeleton. The output of the model is a sequence of pose and shape parameters in the SMPL body model format ( ). NOTE: the VIBE repo has a research only license and you should check if your project can use it ( ) We utilized a deep-learning based approach open sourced as VIBE ( ), which trains a temporal model to predict the parameters of the SMPL body model for each frame while a motion discriminator tries to distinguish between real and regressed sequences. 3) MoveBox Offline Video Trackingīody models extracted by an external tool for 3D multi-person human pose estimation from RGB videos. Our demo project also includes finger tracking for the Oculus Quest. The hand tracking is used for an Inverse Kinematics (IK) solver (included) that reconstructs a possible elbow position. The hand and head positions and rotations of the user are transferred to a Microsoft Rocketbox Avatar of your choice. Using an off the shelf HMD we recover realtime motions to create avatar embodiment. Our main contribution is both in the creation of this open source tool as well as the integration of MoveBox with Kinect V2 and Azure Kinect, as well as the interfacing with the Microsoft Rocketbox avatars. Additional features of the Captire Studio include recording, playback and looping animations, as well as basic audio lip sync, blinking and resizing of avatars. Our toolbox enables real-time animation of the user's avatar by converting the transformations between systems that have different joints and hierarchies. We created a Motion Capture (MoCap) studio using a single depth sensor, such as Azure Kinect or Windows Kinect V2. IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR) Paper 1) MoveBox Capture Studio Reilly, Tabitha C Peck, Andrea Stevenson Won, Anthony Steed and Eyal Ofek (2020) " MoveBox: Democratizing MoCap for the Microsoft Rocketbox Avatar Library". Mar Gonzalez-Franco, Zelia Egan, Matthew Peachey, Angus Antley, Tanmay Randhavane, Payod Panda, Yaying Zhang, Cheng Yao Wang, Derek F. Watch the presentation of the tool at IEEE AIVR: Each form of avatar animation is presented as a separate project: a capture studio, an IK and Hand tracking tool for realtime use, and an offline video tracking plugin. MoveBox is a toolbox to animate the Microsoft Rocketbox avatars ( ) using motion captured in three different ways.
0 Comments
Leave a Reply. |