I'll try to find a new expression. (Patreon)

Hello everyone.

Lately I have been experimenting with an interesting technology called Barracuda in Unity. Unity's Barracuda itself has been around since 2020, but I feel like I've been tinkering with it lately.

What is Barracuda?

Barracuda is a library for inference using neural networks in Unity. Simply put, it is a feature that makes machine learning data available in Unity.

I tried Waifu2xBarracuda.

First,  I tried Waifu2xBarracuda, which uses AI to convert low-resolution images to high-resolution.

It looks good!

It looks like it would be good to apply it to still images, since the processing load is too large for a quirky video. For example, it would be useful to increase the resolution of screenshots and background images in Shoost. (I haven't decided if I'll implement it yet...)

I tried depth estimation from images.

We also tried other techniques to do depth estimation from images.

We can now measure the Depth value from the image and add depth information to the image.

What this allows us to do, for example, is to add lighting or even depth of field.

↑Point light placement.

↑I added depth of field.

Since it is difficult to determine detailed shapes, it is not possible to simply create a three-dimensional representation, but it may be useful for a small improvement in expression.

These are two things I have been wanting to try for a while, so I am glad to have the verification! I have a feeling that something could be done with them if used well.

So lately I have been experimenting with different technologies and exploring new possibilities.

You can try this technology from GitHub here.

Waifu2x 

https://github.com/keijiro/Waifu2xBarracuda 

monocular-depth-unity

https://github.com/GeorgeAdamon/monocular-depth-unity