NrealFrameSource Class¶

Description¶

A custom frame source which connects Nreal output to EasyAR input in the scene, providing Nreal support using custom camera feature of EasyAR Sense.

This frame source is one type of motion tracking device, and will output motion data in a ARSession .

Nreal SDK For Unity is required to use this frame source, you need to setup Nreal SDK For Unity according to official documents.

NrealFrameSource.CameraRigCandidate is required for availability check, they will be automatically picked from scene objects if not setup.

MonoBehaviour Messages¶

private void OnEnable()

private void OnDisable()

protected virtual void Awake()

protected virtual void OnApplicationPause(bool pause)

protected virtual void OnDestroy()

Fields¶

EnableColorInput¶

C#: public bool EnableColorInput

If color image is used as frame input. Color image is usefull when recording a colored eif file, but not necessary for all EasyAR algorithms.

Properties¶

CameraRigCandidate¶

C#: public NRKernal.NRHMDPoseTracker CameraRigCandidate { get; set; }

Nreal CameraRig.

IsHMD¶

C#: protected override bool IsHMD { get; }

Provide only when creating a new frame source. It will only be accessed during ARSession.Assemble .

If the frame source represent head mounted devices. If true, diagnostics will be displayed on a 3D board in from of camera.

Some frame filter may work different on devices.

Camera¶

C#: protected override Camera Camera { get; }

Provide only when creating a new frame source. It will only be accessed during ARSession.Assemble .

On desktop or mobile phones, it represents the Camera in the virtual world in reflection of real world camera device, its projection matrix and transform will be set to reflect the real world camera, and it is controlled by EasyAR. On head mounted devices, it is only used to display some diagnostics message in front of your eyes, it is not used for camera image rendering, it is not controled by EasyAR.

Display¶

C#: protected override IDisplay Display { get; }

Provide only when creating a new frame source. It will only be accessed during ARSession.Assemble .

Provide display system information. You can use Display.DefaultSystemDisplay or Display.DefaultHMDDisplay for default display.

IsAvailable¶

C#: protected override Optional<bool> IsAvailable { get; }

Provide only when creating a new frame source. It will only be accessed during ARSession.Assemble .

If the frame source is available.

If the value equals null, FrameSource.CheckAvailability will be called and the value will be accessed after Coroutine finish.

OriginType¶

C#: protected override ExternalDeviceFrameSource.DeviceOriginType OriginType { get; }

Device origin type.

OriginCandidate¶

C#: public MonoBehaviour OriginCandidate { get; set; }

ExternalDeviceFrameSource.Origin candidate. It will be automatically get from the scene or generate if not set.

AvailableCenterMode¶

C#: internal protected override IReadOnlyList< ARSession.ARCenterMode > AvailableCenterMode { }

Provide only when creating a new frame source. It will only be accessed during ARSession.Assemble .

All available center modes.

IsCameraUnderControl¶

C#: internal protected override bool IsCameraUnderControl { }

Provide only when creating a new frame source. It will only be accessed during ARSession.Assemble .

When the value is ture, the session will update camera transform depending on the center mode and render camera images.

When creating a HMD extension, it should be false. You should have full control of the 3D camera in the scene. You should handle camera rendering, especially in VST.

Origin¶

C#: protected virtual GameObject Origin { get; }

Device origin. You need to define your own origin when ExternalDeviceFrameSource.OriginType is ExternalDeviceFrameSource.DeviceOriginType.Custom . It is not required to re-define origin in other cases.

ReceivedFrameCount¶

C#: public int ReceivedFrameCount { get; }

Received frame count. Usually used for debug. There are usually hardware issues if this value stop to increase, and re-plug may help when using some devices like Nreal.

Methods¶

OnSessionStart¶

C#: protected override void OnSessionStart( ARSession session)

Provide only when creating a new frame source. It will only be accessed during ARSession.StartSession .

Handle session start if the frame source was assembled into ARSession.Assembly . It is designed for lazy initialization and you can do AR specific initialization work in the method.

OnSessionStop¶

C#: protected override void OnSessionStop()

Provide only when creating a new frame source. It will be accessed during ARSession.StopSession or other session stop/break procedure.

Handle session stop if the frame source was assembled into ARSession.Assembly . You can use this method to destroy and resources created in ARSession.StartSession and during the session running and resotre inner state. It is ensured to be called before session destroy. If the frame source was destroyed before session, it will not be called and session will break.

HandleRenderFrameData¶

C#: protected void HandleRenderFrameData(double timestamp, Pose pose, MotionTrackingStatus trackingStatus)

Input render frame data.

Make sure to call every frame after device data ready, do not skip.These data should match the one driving current Unity rendering camera in the same frame.

HandleCameraFrameData¶

C#: protected bool HandleCameraFrameData(double timestamp, Image image, CameraParameters cameraParameters, ExternalDeviceFrameSource.Extrinsics extrinsics, Pose deviceToOriginTransform, MotionTrackingStatus trackingStatus)

Input camera frame data.

Suggest to input 30 or 60 fps data. Acceptable minimum frame rate = 2, but response time of some algorithms will be affected. It can be called in any thread if all your APIs are thread safe. These data should match the one when camera sensor exposure. It is recommended to input the color data into EasyAR Sense as long as it can be obtained, which is helpful for the effect of EasyAR Mega.To get a better performance, you can design the whole data chain to let raw YUV data passthrough directly from shared memory and pass data pointer directly into EasyAR Sense.Be careful with data ownership.

TryAcquireBuffer¶

C#: protected Optional< Buffer > TryAcquireBuffer(int size)

Try acquire buffer from buffer pool.

CheckAvailability¶

C#: private virtual System.Collections.IEnumerator CheckAvailability()

Provide only when creating a new frame source. It will only be accessed during ARSession.Assemble .

Coroutine to check frame source availability when FrameSource.IsAvailable equals null.