3D Model Rotated by 90° on Android Build

[Harish-Narayan]

Description

Expected Behaviour:
The model should retain the same orientation on Android as seen in the Editor.

Actual Behavior:
The model is visibly rotated by ~90° on Android.

When running in the Unity Editor, the 3D model positioning and rotation both look correct. However, in the Android build (APK), the positioning still appears fine, but the model itself is rotated by roughly 90 degrees. I set the Unity's default orientation as Portrait and even enforced it via code.
Seems like,
• Unity Editor: videoRotationAngle is 0.
• Android Build: videoRotationAngle is 270.
which means, should I handle the rotation of my 3d object separately?
If yes, is there a workaround other than just applying a Quaternion.Euler(0,0,-angle) rotation, or is there a more robust workaround??
Would like to know if I am missing some other blind spots that needs to be accounted for?
Looking for a best-practice solution for handling rotation discrepancies, any guidance on how to address this would be greatly appreciated!

[Harish-Narayan]

@homuler

[homuler]

Since the code that reproduces the issue hasn't been shared, I can't say for certain, but perhaps you're passing the input image to MediaPipe as it is? In other words, it might be that you're passing the rotated image (videoRotationAngle != 0) to MediaPipe.

If that's the case, you should rotate the input image correctly before passing it to MediaPipe.

Please refer to the sample app code to know how it handles this situation.

MediaPipeUnityPlugin/Assets/MediaPipeUnity/Samples/Scenes/Face Detection/FaceDetectorRunner.cs

Lines 63 to 65 in fc2e7b5

	var transformationOptions = imageSource.GetTransformationOptions();
	var flipHorizontally = transformationOptions.flipHorizontally;
	var flipVertically = transformationOptions.flipVertically;

MediaPipeUnityPlugin/Assets/MediaPipeUnity/Samples/Scenes/Face Detection/FaceDetectorRunner.cs

Lines 107 to 109 in fc2e7b5

	textureFrame.ReadTextureOnCPU(imageSource.GetCurrentTexture(), flipHorizontally, flipVertically);
	image = textureFrame.BuildCPUImage();
	textureFrame.Release();

If you're not using TextureFrame and you need to rotate the image by yourself, please note that unlike Unity, MediaPipe uses the Image Coordinate System ((0, 0) is at the top left) internally.

[Harish-Narayan]

Thank you for your response!

I’m just getting started with the plugin and currently working on building a jewelry try-on app. For now, I’m directly using the inferred landmarks from the PoseLandmarkerResultAnnotationController.cs in the sample scene and using the same annotation layer from the sample app. Specifically, I’m working with _currentTarget for positioning:

MediaPipeUnityPlugin/Packages/com.github.homuler.mediapipe/Runtime/Scripts/Unity/Annotation/PoseLandmarkerResultAnnotationController.cs

Line 18 in fc2e7b5

private PoseLandmarkerResult _currentTarget;

For placement, I’m using the following snippet:

_necklaceInstance.transform.localPosition = Vector3.SmoothDamp(
    _necklaceInstance.transform.localPosition,
    targetPosition,
    ref _necklaceVelocity,
    positionSmoothTime
);

Quaternion modelOffset = Quaternion.Euler(0f, 0f, -90f);

_necklaceInstance.transform.rotation = Quaternion.Slerp(
    _necklaceInstance.transform.rotation,
    modelOffset,
    step);

The targetPosition is derived from the shoulder landmarks of _currentTarget. Everything seems to be working as expected in the Unity Editor. However, in the Android build, while the position remains correct, the 3D object appears to be rotated by approximately 90/270 degrees. I tried manually offsetting the model but even that doesn't seem to work.

I tried adding this debug statement:
Debug.Log("Rotation angle: " + webCamTexture.videoRotationAngle);

Got Rotation0 in editor, but Rotation270 in app build.

MediaPipeUnityPlugin/Assets/MediaPipeUnity/Samples/Common/Scripts/ImageSource/WebCamSource.cs

Lines 196 to 198 in fc2e7b5

	InitializeWebCamTexture();
	webCamTexture.Play();
	yield return WaitForWebCamTexture();

I would really appreciate your guidance on how to correctly handle this rotation discrepancy. Are there any best practices or recommended approaches to ensure consistency across platforms? Thanks in advance for your help!

[homuler]

When implementing custom logic, I don't think you should simply reuse the sample scene as it is.
For example, scripts attached to GameObjects may manipulate the rotation, but it can be difficult to fully grasp all such influences.

So please create a fresh scene and test whether the same issue occurs there.
If the issue reproduces, it is likely a general problem related to handling device rotation, independent of the plugin.

For now, I’m directly using the inferred landmarks from the PoseLandmarkerResultAnnotationController.cs in the sample scene

Since the AnnotationController is just receiving the results, I think it's easier to read the results directly.

MediaPipeUnityPlugin/Assets/MediaPipeUnity/Samples/Scenes/Pose Landmark Detection/PoseLandmarkerRunner.cs

Lines 162 to 165 in fc2e7b5

	private void OnPoseLandmarkDetectionOutput(PoseLandmarkerResult result, Image image, long timestamp)
	{
	_poseLandmarkerResultAnnotationController.DrawLater(result);
	DisposeAllMasks(result);

[Harish-Narayan]

Hi @homuler,

Thanks again for all your responses. I’m currently working on replicating the same issue in a fresh scene to confirm if it’s indeed related to handling device rotation. I share your suspicion—Mediapipe itself appears to display annotations and face meshes correctly across different devices and orientations, so rotation handling seems a likely culprit.

I’ll let you know once I’ve completed the integration in the new scene. In the meantime, if you know of any example scripts in the plugin that specifically manage device rotation, I’d really appreciate any pointers.

Thanks once more for your assistance so far!