It Finally Happened! Server Crash Update (Patreon)

Hi all!

It finally happened, the EU Edge server crashed during 6 events running on VRCDN last Sunday morning at around 5:30AM GMT. Luckily I was online and saw it as it happened and was able to quickly resolve the issue but I thought it would be best to write out exactly what happened and what I'm doing to prevent this from happening in the future.

The specific issue was actually out of VRCDN's control as it was with the route between the US based ingest server and the EU based edge server. The route started experiencing high packet loss for just enough time for the edge server to become disconnected. Streaming at sub 1s latency is very network sensitive so if packets don't come in on time they usually get dropped....and if too many get dropped it's a disconnect.

As some of you may already know I am currently in the process of a complete network rewrite as the current running build is still very much a proof of concept, because of this there are a lot of missing features that you might not even consider....such as automatically re-establishing connections between the ingest and edge servers in the event that the connection is dropped. Between Sunday and Monday I implemented an auto-reconnect feature into the VRCDN development environment so I could begin testing and have it deployed to the network before this weekend. Testing so far is going well and it looks like everything in on schedule.

I would like to sincerely apologise for this disruption during so many events but I would also like to remind everyone that this project is still very much a work in progress and I cannot guarantee 100% stability at this time. But thanks to ALL of your EXTREMELY generous contributions to the network I am certain one day I will be able to guarantee stability across the entire network.

Thank you for your patience and I'd also like to remind everyone that the best place to be is in the VRCDN Discord to directly talk to me, to discuss ideas and get assistance on the network and in-world players. http://discord.gg/yDQYhZc

Thank you for believing in me!

-Jazzy