Camera_CustomCamera

Demonstrate how to create an EasyAR app with a non-system-API-based camera.

You can create a customized CameraDevice reference this sample. The sample uses system API as a sample and don’t support Android USB camera out-of-the-box. For Android USB camera, libuvc integration is needed. You can reference this .

Notice: don’t use this sample for Windows USB camera, as they have built-in support in EasyAR.

Notice: Using video stream as frame input is also supported, you can reference API documents to modify this sample.

How to Use

How It Works

Use a custom FrameSource

To use a customized camera device in the scene, replace the VideoCamera with a customized FrameSource. Put it under the ARSession node, the session will use it automatically.

Inherit CameraSource

This sample inherit the CameraSource to implement camera features. Override Open/Close/OnEnable/OnDisable to make the camera run.

If you want to use non-camera frame source, you can inherit FrameSource instead of CameraSource.

public override bool HasSpatialInformation
{
    get { return false; }
}

protected override void OnEnable()
{
    base.OnEnable();
#if !UNITY_EDITOR && UNITY_ANDROID
    if (externalCamera != null)
        externalCamera.Call<bool>("start", cameraCallback);
#elif !UNITY_EDITOR && UNITY_IOS
    if (externalCamera != null)
        externalCamera.start(cameraCallback);
#endif
}

protected override void OnDisable()
{
    base.OnDisable();
#if !UNITY_EDITOR && UNITY_ANDROID
    if (externalCamera != null)
        externalCamera.Call<bool>("stop");
#elif !UNITY_EDITOR && UNITY_IOS
    if (externalCamera != null)
        externalCamera.stop();
#endif
}

public override void Open()
{
}

public override void Close()
{
}

Native camera implementation

This sample use native camera source code to provide camera features. The source code for different platforms are in different places.

Android implementation is in the CustomCameraAndroid Gradle project, and its output is pre-compiled into custom-camera.jar in the Plugins folder.

iOS implementation is in the MyCamera files in the Plugins folder.

Implement camera device using native code provides the most flexibility to use system camera. You can change the source code to make camera feature work which is not integrated in EasyAR VideoCameraDevice.

Expose native camera interface into C#

A few code are used to wrap native interfaces accessible in C#, which directly calls Java or C interface.

private AndroidJavaObject externalCamera;
private CameraCallback cameraCallback;

private class CameraCallback : AndroidJavaProxy
{
}

private ExternalCamera externalCamera;
private Action<IntPtr, int> cameraCallback;

private class ExternalCamera : IDisposable
{
}

Handle InputFrame in camera callback

Both Android and iOS implementation in this sample expose the camera callback to C#. In the camera callback, InputFrameSink.handle is used to handle InputFrame created from the image raw data.

private void HandleSink(Buffer imageBuffer, PixelFormat format, Vector2 imageSize, int orientation, int cameraType, double timestamp)
{
    using (var cameraParams = CameraParameters.createWithDefaultIntrinsics(new Vec2I((int)imageSize.x, (int)imageSize.y), (CameraDeviceType)cameraType, orientation))
    using (var image = new Image(imageBuffer, format, (int)imageSize.x, (int)imageSize.y))
    using (var frame = InputFrame.createWithImageAndCameraParametersAndTemporal(image, cameraParams, timestamp))
    {
        if (sink != null)
            sink.handle(frame);
    }
    imageBuffer.Dispose();
}