11.3. Detect pauses with MediaPipe

MediaPipe is an AI solution for live and streaming media published by Google that can be used free of charge. This application uses the Pose function.

AI can automatically detect poses from images transferred by a webcam, etc., and apply them to VRoid/VRM.

https://google.github.io/mediapipe/

Warning

Because the IK positions of MediaPipe and this application do not match perfectly, pose detection is not 100%. Please note.

This function cannot be used except for VRoid/VRM.

  1. Click MediaPipe on the 3D Models tab.

../_images/posing_a.png

  1. A confirmation message will be displayed. If there is no problem, press the OK button to proceed.

../_images/posing_b.png

Note

Permission to use the camera is required for the first time, so please allow it before proceeding.

  1. Depending on the specs of your PC, the rear camera image will be displayed while loading.

../_images/posing_c.png

Note

If the app freezes in this state, please quit all of this app once and start it again.

../_images/posing_d.png

11.3.1. save the pose

This is the normal procedure for obtaining a pose.

  1. Click the blue SHUTTER button to the right of the center of the window.

../_images/posing_e.png

Warning

The timer function has been discontinued in version 2.19.0.

If you set the number of seconds for the timer to 1 or more, the timer will start after you press the button, and after that number of seconds the pose will be taken automatically.

  1. Confirm that the pose has been reflected on the main screen of this app.

../_images/posing_f.png

Note

Depending on the position, height, depth, etc. of the person or object being photographed, the pose actually reflected may differ from the assumption.


11.3.2. Always detect poses

This is a function added from ver 2.0. The pose recognized by the webcam is always reflected on the currently selected VRM.

  1. Click the AUTO button at the bottom center of the window.

../_images/posing_e2.png

  1. Pressing the AUTO button again will stop the pause from being reflected.

Note

Continues to reflect the recognized pose at intervals of about 1 second. If the webcam is no longer recognized for some reason, the reflection of the pose will also stop. In that case, please end this application once and start it again.

11.3.3. Switch video source

You can switch if you have multiple webcams connected or if you want to detect poses from photos.

  1. Select the desired one from the video source combo box in the tool window.

imgsource:

Or click the browse button

11.3.4. Save pose with timer

Warning

This feature was discontinued in version 2.19.0.

With the timer function, the camera image can be automatically shot after the specified time and the pose can be detected.

  1. Specify the number of seconds in “Save timer” in the tool window.

../_images/posing_i.png

Can be specified between 1 and 10 seconds

  1. Press the Save Pose button.

../_images/posing_j.png

A loading animation is displayed on the upper left, and after the specified number of seconds, a sound effect is emitted and the display disappears. (It turns red every second)


The pose will be saved according to the normal procedure, so please check and reflect it on the main screen.


11.3.5. Other functions

Warning

This feature was discontinued in version 2.19.0.

../_images/posing_k.png

Selfie Mode:

Invert the image.

Video source:

Specify an image with the camera or browse button.

Min Detection Confidence:

Accuracy of detection

Min Tracking Confidence:

Tracking accuracy

Check the MediaPipe site for other options.

Close Menu

  1. Click img_menubtn on the top right of the menu.

  2. The menu will disappear and you will see the detection preview in the background.

../_images/posing_l.png