ImageTracking_Video¶
Demonstrate image tracking feature and video display on top of image.
Demonstrate how to load video with Unity VideoPlayer and play video on target
Demonstrate how to load video with Unity VideoPlayer and play transparent video with custom shader
Demonstrate how to change the video aspect ratio (Unity VideoPlayer feature) to fit image target
How to Use¶
How It Works¶
Play video¶
In this sample, we use the Unity Video Player to display video on top of target image.
Please notice that all scripts in this sample including VideoPlayerAgent are sample code which only write for this sample. Any requirements about the video player would be a general Unity project development problem.
VideoPlayerAgent is not required in your own project, you can use the Unity VideoPlayer in your own way. The sample simply sets VideoPlayer.source and VideoPlayer.url and the video will play.
player.source = VideoSource.Url;
player.url = FileUtil.PathToUrl(path);
It is usually more convenient to put video in resources, we use video file in StreamingAssets in the samples only to keep compatibility, because some versions of Unity will have strange behaviors if video in resources.
There is also another workaround for Unity bug, which is also discussed in this thread. The workaround will use a web video instead of local file which may cause the video play slow. If you are using Unity versions with this issue fixed, you can safely remove the related code in the sample.
Control video play by target events¶
In this sample, video will start to play when target is being tracked and pause when lost. Target events are used to do this.
private void Start()
{
...
controller.TargetFound += () =>
{
...
found = true;
StatusChanged();
};
controller.TargetLost += () =>
{
found = false;
StatusChanged();
};
}
private void StatusChanged()
{
...
if (found)
{
...
player.Play();
}
else
{
...
player.Pause();
}
}
The video mesh is hidden when video data is not ready or the Video Player not prepareCompleted, so that the displayed mesh will always have data.
It is also hidden when target lost to hide video when not playing.
private void Start()
{
...
player.prepareCompleted += (source) =>
{
prepared = true;
StatusChanged();
};
}
private void StatusChanged()
{
if (!ready)
{
meshRenderer.enabled = false;
return;
}
if (found)
{
meshRenderer.enabled = prepared;
}
else
{
meshRenderer.enabled = false;
}
}
Transform scale is set to make the video fit the target region, so that the video will not overflow the image.
transform.localScale = new Vector3(1, 1 / controller.Target.aspectRatio(), 1);
Ways to fit video to the image¶
How the video fit the image (fit inside or stretch or others) can be controlled by VideoPlayer.aspectRatio. This is a Unity VideoPlayer feature.
Transparent video of side-by-side format¶
Transparent video here is a special type of video, it use a side-by-side format usually used by 3D video, its left side is the colored video, and right side is used as alpha channel for the left side.
The video itself has no special things, it is encoded by normal video encoder and multiplexed using normal container. To create such videos, you just need to double your video width and make the right side the alpha value you want for each pixel of the left part using any tool that can achieve this.
The only difference of using this video compared to other videos is the material.
The shader recalculates the pixel value to mix left and right side together.
fixed4 frag(v2f i) : SV_Target
{
fixed4 color_a = tex2D(_MainTex, float2(i.uv.x / 2 + 0.5, i.uv.y));
fixed4 color_rgb = tex2D(_MainTex, float2(i.uv.x / 2, i.uv.y));
return fixed4(color_rgb.rgb, color_a.r);
}
The output in Unity will be like this, except the black part is transparent.
