[Unity Project Download] Piano Note Video Renderer - (RushE 3) (Patreon)

This download contains all the scripts and files I used to create the RushE 3 music visualizer. [Prerequisite: Unity + CSharp Experience Recommended!] If you are confident in Unity and looking to tweak this project for your needs, then this download is perfect for you! I've made progress on a lot of useful assets for making videos in Unity with ray tracing. I'm most proud of getting the path tracing accumulation to work with a dynamically adjustable sample rate while recording.

Unity Version: 2023.2.0b17

Tutorial:

1. Navigate to the main scene (Assets/Scenes/MainScene.unity). Next, open the config file in Assets/StreamingAssets/config.json and set your desired values for framerate and output file paths, etc. Some of these config values are deprecated. The recording uses screenshots now, so setting the resolution width and height won't change anything (set the resolution in the game view of the unity editor instead), and the PathToFfmpegBinary is not necessary either for the current recording method. Drag the .mid file you want into the Assets/StreamingAssets/MIDI folder. The name of the file must be set in the config, but not the path.

2. The only gameObjects necessary for you to interact with are under the "Scripts" parent object in the hierarchy. I'll briefly cover the purposes of each.

3. First lets take a look at the GameManager script. The most important value here you can adjust is the frame number. The frame number corresponds to the frame in the video. It sets the Playhead Time based on the FPS you choose for your rendering. While in play mode, you can slide the frame number to scrub through your MIDI. You can also adjust the physics backup frequency. While recording this video, I ran into occasional crashes due to the ridiculous amount of notes and memory usage of the RUSH E 3 MIDI (would not recommend using this crazy of a MIDI). After a crash, I wouldn't be able to restore the physics simulation, so I'd have to start all the way over. To prevent losing progress like this, I create a backup of the rigidbody positions every "Physics Backup Frame Freq", allowing the simulation to be restored and the recording to continue from before the crash. Use the PhysicsSim gameobject to restore old physics simulations. Match the "TestFrame" with the frame number of the physics backup json file, then click the "Load Button".

4. Under the MidiSpawner GameObject, make sure that "Enable E Spawning" is checked in the inspector. I'm assuming most people downloading this are interested in using a different MIDI, and spawning something else besides an E after each key press. You can swap out the E Prefab with a different gameobject to do that here. Although there is a dependancy on the EBody script, so you'll need to tweak that and add it to whatever prefab you want to spawn in.

5. To start recording, go to the VideoRendererV2 gameObject and check the "Recording" boolean.

6. The simulation won't advance until it is un-paused. You can toggle the paused vs un-paused with the space bar.

7. While recording, you can press the up and down arrow keys to adjust the number of path tracing samples to accumulate before each screenshot (I'm particularly proud of this feature).

That covers the basics! Let me know if you have any questions and I'm happy to help.