ImageTracking_CloudRecognition

Demonstrate how to use cloud recognition.

• Demonstrate how to create and use offline cache

• Demonstrate how to stop web request when tracking

Configurations

Server access information is required when using cloud recognition, which can be get from CRS page in EasyAR develop center. There are two ways to set this information in Unity.

One is global configuration, all scenes using global cloud recognition configuration will use this value. Select <EasyAR -> Sense -> Configuration> in Unity menu and enter the information in Inspector get from develop center.

Another is local configuration in the scene, it is only valid to the current scene.

How to Use

Mark 1: Display system status and operation hint.

Mark 2: Stop/start cloud recognition.

Mark 3: Clear the targets in use and all offline caches.

How It Works

Trigger cloud recognition

From EasyAR Sense 4.1, all CloudRecognizer requests are triggered by user code. The SDK no longer internally trigger send and receive data with the server, which means you are free to control the frequency of request now.

CloudRecognizer.Resolve(iFrame, (result) => {});

There limitation of lowest request interval is 300ms.

Trigger cloud recognition periodically

It is suggested to trigger cloud recognition in ARSession.FrameChange event handler, which is the event of a frame image data change. You can get the InputFrame from this event.

public void StartAutoResolve(float resolveRate)
{
    ...
    Session.FrameChange += AutoResolve;
    ...
}

private void AutoResolve(OutputFrame oframe, Matrix4x4 displayCompensation)
{
    ...
    using (var iFrame = oframe.inputFrame())
    {
        CloudRecognizer.Resolve(iFrame, (result) => {});
    }
}

In this sample, the resolve will not happen when a target is being tracked or previous resolve did not finish, or the time interval has not reached a preset value (1s).

private void AutoResolve(OutputFrame oframe, Matrix4x4 displayCompensation)
{
    var time = Time.time;
    if (isTracking || resolveInfo.Running || time - resolveInfo.ResolveTime < autoResolveRate)
    {
        return;
    }

    resolveInfo.ResolveTime = time;
    resolveInfo.Running = true;
    ...
}

Cloud recognition result

Cloud recognition is a standalone feature of EasyAR, it can be used without image tracking. If you do not need to track the image after recognition, you can always use the resolve result directly.

CloudRecognizer.Resolve(iFrame, (result) =>
{
    ...
    resolveInfo.CloudStatus = result.getStatus();
    ...
    var target = result.getTarget();
    ...
});

Track target from cloud

If you want to track the image recognized by server, you need to use the target information from result.

The target is cloned in the sample because it will be referenced in the ImageTargetController, so a reference to the internal object should be kept.

CloudRecognizer.Resolve(iFrame, (result) =>
{
    ...
    var target = result.getTarget();
    if (target.OnSome)
    {
        using (var targetValue = target.Value)
        {
            ...
            LoadCloudTarget(targetValue.Clone());
        }
    }
});

Create a GameObject with ImageTargetController and set the ImageTarget from resolve callback handler to ImageTargetController.TargetSource so that the controller can be initialized with ImageTargetController.DataSource.Target.

private void LoadCloudTarget(ImageTarget target)
{
    ...
    var go = new GameObject(uid);
    targetObjs.Add(go);
    var targetController = go.AddComponent<ImageTargetController>();
    targetController.SourceType = ImageTargetController.DataSource.Target;
    targetController.TargetSource = target;
    LoadTargetIntoTracker(targetController);

    targetController.TargetLoad += (loadedTarget, result) =>
    {
        ...
        AddCubeOnTarget(targetController);
    };

    ...
}

There is no need to load the same target more than once and it will fail.

if (!loadedCloudTargetUids.Contains(targetValue.uid()))
{
    LoadCloudTarget(targetValue.Clone());
}

Use offline cache

It is usually a good practice to save recognized target in storage so that it can be loaded next time. It will help recognition the next time the application starts to reduce response time, or even work when the device is offline.

You can save the target into .etd files using ImageTarget.save.

target.save(OfflineCachePath + "/" + target.uid() + ".etd")

Then you can create target with .etd file in the next run using ImageTargetController.DataSource.TargetDataFile type input.

private void LoadOfflineTarget(string file)
{
    var go = new GameObject(Path.GetFileNameWithoutExtension(file) + "-offline");
    targetObjs.Add(go);
    var targetController = go.AddComponent<ImageTargetController>();
    targetController.SourceType = ImageTargetController.DataSource.TargetDataFile;
    targetController.TargetDataFileSource.PathType = PathType.Absolute;
    targetController.TargetDataFileSource.Path = file;
    LoadTargetIntoTracker(targetController);

    targetController.TargetLoad += (loadedTarget, result) =>
    {
        ...
        loadedCloudTargetUids.Add(loadedTarget.uid());
        cachedTargetCount++;
        AddCubeOnTarget(targetController);
    };
}