Workflow_FrameSource_CameraDevice¶

Attention

This page is outdated, updates will be online soon…

Demonstrate how to use the Camera.

Demonstrate how to open/close camera
Demonstrate how to switch camera
Demonstrate how to get camera image texture
Demonstrate how to control camera display
Demonstrate how to horizontally flip camera image

How to Use¶

Mark 1: Display system status and operation hint.
Mark 2: Whether open the camera.
Mark 3: Whether display camera image.
Mark 4: Whether capture camera image and display on the cube.
Mark 5: Whether flip the camera image rendering.
Mark 6: Switch camera using device index.

How It Works¶

Camera on/off¶

Camera device can be turned on or off using VideoCameraDevice.enabled.

public void EnableCamera(bool enable)
{
    videoCamera.enabled = enable;
}

Camera image display on/off¶

Camera image display can be turned on or off using CameraImageRenderer.enabled. It will not turn the device on or off, so the AR will still running when you turn off the display. This is usually used when you are displaying on eyewear.

public void ShowCameraImage(bool show)
{
    cameraRenderer.enabled = show;
}

Get camera image texture¶

Use CameraImageRenderer.RequestTargetTexture to register handler to get camera image texture. And use CameraImageRenderer.DropTargetTexture to release resources after use.

public void Capture(bool on)
{
    if (on)
    {
        cameraRenderer.RequestTargetTexture(targetTextureEventHandler);
    }
    else
    {
        cameraRenderer.DropTargetTexture(targetTextureEventHandler);
    }
}

Use the texture directly in the handler.

Action<Camera, RenderTexture> targetTextureEventHandler = (camera, texture) =>
{
    if (texture)
    {
        CubeRenderer.material.mainTexture = texture;
    }
    else
    {
        ...
    }
};

Unity is not able to hide the differences between platforms when using the RenderTexture created from the plugin, so you need to ensure that the differences of render texture coordinates are correctly handled for different rendering APIs when using this texture in render pipeline.

The sample code shows how rendering API will affect the texture, please notice texture is flipped when using Metal, so that the display stays the same. It is one way to eliminate the difference, but usually you need to do it in the shader like the Unity document suggests.

if (SystemInfo.graphicsDeviceType == UnityEngine.Rendering.GraphicsDeviceType.Metal)
{
    CubeRenderer.transform.localScale = new Vector3(-1, -(float)Screen.height / Screen.width, 1);
}
else
{
    CubeRenderer.transform.localScale = new Vector3(1, (float)Screen.height / Screen.width, 1);
}

This sample use the camera image only for display, so it uses the texture. But if you want to access image data from CPU instead of GPU, it is better to get byte array from OutputFrame.inputFrame().image().buffer() and handle different pixel types in an ARSession.FrameChange event handler for better performance.

Switch camera¶

Set VideoCameraDeviceCameraOpenMethod and VideoCameraDeviceCameraIndex or VideoCameraDeviceCameraType before open.

videoCamera.CameraOpenMethod = VideoCameraDevice.CameraDeviceOpenMethod.DeviceIndex;
videoCamera.CameraIndex = index;
videoCamera.Close();
videoCamera.Open();

Flip camera image horizontally¶

This is usually used when you want a mirror display. Use ARSession.HorizontalFlipNormal and ARSession.HorizontalFlipFront to do this. Front camera display is flipped by default.

if (isFront)
{
    arSession.HorizontalFlipFront = flip ? ARSession.ARHorizontalFlipMode.World : ARSession.ARHorizontalFlipMode.None;
}
else
{
    arSession.HorizontalFlipNormal = flip ? ARSession.ARHorizontalFlipMode.World : ARSession.ARHorizontalFlipMode.None;
}