Home
Artists
Search Recent Random
Posts
Search DMs Popular Hash Lookup Tags Random
Importer
Import FAQ
Register
Partychan Telegram ThePornDude
Home Artists Posts Import Register

WIP: more AI stuff updates & Alive marker (Patreon)

Published:
2023-03-13 19:49:04
Imported:
2023-03

Content

OFFLINE CHATBOT AI
I added support to make the chatbot AI work 100% offline too. In the previous demo I was using an online bot running on google colab & the ElevenLabs API for AI text-to-speech. Both of those are free and awesome but they have some monthly limits after which you have to pay for them. I don't like relying on online services as they might change, go down, etc. and render the whole thing unplayable overnight. Privacy is of course a big concern too

In this quick video I was using a local classic TTS voice. It's possible to make it work at some point also with local AI TTS voices which sound more natural. I was also running locally KoboldAI with a very very basic bot, there are much smarter AI models available. I also didn't adjust the vam lipsync settings so that could be much better too.

SENTIMENT ANALYSIS
I added sentiment analysis to the responses of the AI to get how the AI is 'feeling' about what it says. It interprets the words used and calculates a score for positive, neutral and negative sentiments and a general score of the message. In the video the red-yellow-green bar reflects that general score.

The idea is to use this with Alive to alter the expressions while the character is talking. It's a pretty basic thing, a sad-happy morph essentially, but paired with all the procedural animations Alive is doing it should make things noticeably more immersive once implemented.

ALIVE MARKER SYSTEM

VAM is awesome for making custom scenes. The process however is very tedious and moving & spawning things takes a lot of work.

To help with that I added a marker system to Alive. The way it works is like a 3d cursor. You can bring it close to you by pressing a button or just move it around. When adding atoms now to the scene through the UI they will be added to the marker position & rotation. It's hidden by default and shows when some menus are active like the Aeternum UI scene atoms menu.

For now it only works on the default height but I want to add height detection similar to the person teleport system too, to allow spawning & moving things on top of other things. Eventually this will help get rid of the person teleport system and become a simple & unified way of giving 3d commands and interacting with the scene like: move atom there, clone atom there, walk there, crawl there, sit there (if a chair is nearby),  etc.

Files

WIP: vam + local text-to-speech + local AI + speech sentiment analysis

WIP experimental stuff, virtamate AI https://www.patreon.com/SPQR_aeternum MUSIC: https://soundcloud.com/arnydmusic/trust https://soundcloud.com/arnydmusic/vestige Songs licensed under a Creative Commons License http://creativecommons.org/licenses/by/3.0/

Comments

Vitlam

wow this is promising ! good job.

Anonymous

Thanks for the update on internal tech too. It helps to see what others are trying. I'm designing with azure tts and chatgpt for convo. But I truly share your thought about offline mode. I cant even bring myself to talk to my computer, let alone have it sent all over the place to be thought about and turned into wavs. Really do appreciate the work on the AI. Keep sharing your thoughts failures and innovations. With all the people that were refugeed from Replika when they went PG, some sort of vam solution could bring in tons of new people.

w

Looks Great keep up the good work

HelloFox

Hello. I got your virtual doll plugin, but it seems to be broken. Everytime i activate it, the hip is gonna move away from my vr hand. Like an offset. But its not the move distance from the vr tracker moving. Is there a solution or does some plug in break the system?

SPQRAeternum
>>104342828

In vr you have to select a tracker in the VD Sticky dropdown. You also have to hit Recalibrate when the position is set. When you hit that it's like setting a point A (in the 3d space). As you move around the tracker that will be point B, and the distance from A to B is translated into 0%-100% of the animation. You can also enable the VD Line to see that line exactly. If that doesn't work just let me know

HelloFox

I recalibrated it yes, but the hip does have an offset off the controller and it cant be pushed back. I unchecked some other plugins, but that doesnt work.

SPQRAeternum
>>104343244

I did a quick test and doesn't look broken for me. I think you might be misusing it by trying to make the hip stick to the controller, that's not what that plugin is meant to do. If you want the hip to stay 1:1 in sync with the controller, you can do that with vanilla vam (you don't need plugins), you just have to possess the hip controller with the vr hand, hold it or link it to the vr hand and there will be no offsets

SPQRAeternum
>>104343244

Or maybe it's a position thing? Do you mean that on certain body positions of the model the hip's default position goes to something weird? I'm not sure I understand what you mean by that offset

HelloFox

Yes, the actual position of the hip changes. If i activate the vr doll. Is the plugin kompatibel with other plugins? I had usedbit with earlier versions, but since i updated it. I cant use it in my scene anymore.

SPQRAeternum
>>104346218

Are you sure you're not confusing it for my AEdoll plugin? maybe you were looking for that one and added VirtualDoll instead. VD is meant to modify & animate the hip controller, that's all it does basically

HelloFox
>>104346218

Mh, whats aedoll? I mean that plugin which was integrated in alive, but in newer versions its not a part from alive anymore. Used the standalone virtual doll plugin instead. Tried it again, but cant find a solution. I think its an incompatibility with some plug in, since the update. Cant figure it out.

SPQRAeternum
>>104346218

you can still use older plugins if you think that's the issue, just uncheck "Only show latest" in the select plugin popup and they'll appear. I don't understand what problem you're having, you'll have to make a video or explain the problem and I'll try to help. I can't help with "doesn't work, plugin is broken, can't figure it out, can't find solution" etc, it could be a million things

HelloFox
>>104346218

I can`t upload images/Videos here. The Hip Control switches back to root position, if i enable the vd plugin. I tested it in a new scene without plugins. Maybe i do something wrong, but i can`t move the hip joint in to another position. if i enable vd then it switches back to the initial position. Is there a discord channel? Thank you for your replies. also if i am change the position if the plugin is disabled. If i enable the plugin it goes back to root.

SPQRAeternum
>>104346218

That's just how it works. It animates the position of the hip/head controller and uses the chest as reference. I think you tried the initial version that was in Alive v20. That version didn't have the chest controller link later added in v25, and it allowed more flexible poses. You can still use those versions if you want to. But without that link to the chest fix you'll have tons of other problems unless you keep the pose very static. It's a very complicated problem and that's a small feature/plugin with 1 review on the hub bare in mind, it's not gonna work with all scenarios. The way to use it I think is to activate it and then customize the pose around it and move the hip by moving the chest, like it's a torso controller that contains the upper body

HelloFox
>>104346218

ah ok, now i got it. tried it in desktop mode and saw it uses the chest as a reference. tried to move the pose/hip with moving the chest. I will try it later in VR. But yeah, seems to be not that flexible. Thank you for your replies.