WIP: chat multi-language support (Patreon)

Published:

2023-07-27 22:43:46

Imported:

2023-08

Content

A quick test with the support for multiple languages to the chat feature of Alive. Speech recognition seemed pretty decent. The local assistant component didn't work properly, likely due to the character encoding, for the part with "echo <japanese words>" later in the video.

I was running locally a 7B AI model that doesn't know japanese. It would work better of course with a model that actually knows the language. The point was just to test the speech-to-text functionality and being able to send messages to the AI in a language other than English.

The VAM model was running on the new Gaze v3 system which is WIP, so the body movement is a bit different than v50 and earlier and still needs some extra touches.

Files

WIP: AI chat multi language speech support test

a quick first test after adding foreign languages speech recognition support for AI chat in ALIVE built with Virt-A-Mate echo [japanese words] didn't work but too busy to cut it out. That's a local assistant command that's outside the AI. It was supposed to make the AI reply and read what's after the echo. But failed for japanese here, likely at identifying the characters, tbd would work of course better with an AI model that actually knows japanese. Here i was just testing speech to text. Model was llama2 7B running locally. AI voice from ElevenLabs