Dev log #33 - Refactoring progress and design changes (Patreon)

There is something about music software that doesn't feel right to me. Maybe you agree and that's why you are here. It's hard to explain this feeling of wrongness when I'm such a terrible communicator so I have to rely on this combination of vague handwaving and broken analogies.

It feels natural to draw comparisons to graphical art software since on the surface the problems of digital graphics and digital audio seem pretty similar. You are using a computer interface to generate a sequence of numbers at the end of the day. I think graphical art programs feel pretty good because the user interface maps so closely to the way brains think about visuals.

When you open a digital painting program you are presented with a blank white canvas. You can close your eyes and imagine putting a bit of blue in the top right corner of the image, open your eyes, select blue, put it in the top right corner of the canvas. Maybe the shape of it doesn't match what you had in your head so you select the appropriate tool and smudge it around a bit,  add a bit here and erase a bit there. You can leave little ideas off to the side and come back to them, scribble them out or paint over them.

3D modelling software used to be absolutely horrible for creating high-poly, organic looking models but eventually the idea of digital sculpting came about (I think ZBrush was the first to popularise this), where the software presents you with a digital lump of clay that you can chisel into and paint directly onto.

I think over the years these software developers have managed to create a strong connection between user interface and whatever it is that is going on in the user's head. In graphics this almost comes for free since a computer screen is already a visual image.

Audio is more of a problem. There is something a lot more vague and magical about sound which is why this problem is so hard to identify and so hard to define. The UI concepts that we are made to interact with are always these vague, abstract representations. Usually they are not even representing the sounds themselves but the components, devices and signal chains that eventually go towards producing the sounds.

I have no idea how to solve this problem but part of the reason I started this project in the first place is this nagging feeling that nobody in the audio industry has even identified it as a problem at all. Or perhaps they did but the problem is so hard to communicate that it's impossible to coordinate a team towards addressing it.

The software coming out of the audio industry lately is really fantastic and is being worked on by teams of people who are obviously far smarter and more talented than me so I don't want to seem like I am being disparaging of that. But it seems like everyone's more or less content with the way that music composition software fundamentally works and is happy to just keep adding cool bells and whistles on top of it.

The past month or so has been a bit rocky development wise. I have been refactoring large parts of Blockhead's internals to future-proof things and prepare for certain features and this process has forced me to think more carefully about what these features actually are. As mentioned in previous posts, a while ago I decided every block was going to be running on a sort of mini modular rack, with the sampler integrated closely with the effects and oscillators.

Modular effect systems are a very powerful concept but the more I worked on trying to shoehorn this idea into Blockhead the less sense it made. Eventually I took a step back and realised the only reason I even considered the whole modular rack idea is because I've seen it done in other music software, not because I thought about what would feel right.

The way the effects currently work in Blockhead where they are sort of glued to the back of each sample block already feels pretty bad and the modular thing would exacerbate that even further. I am trying to create a system where it feels like there is more of a mental connection between the UI elements on screen and the sounds being produced, and where it is easy to zoom in and make these discrete little micro-edits, nudge things around and throw things away.

I realised there is a far more "Blockheadian" thing I can do for the effects which fits better with the existing design. I have discussed this a little bit on Discord and it's a pretty unconventional but I think it will feel pretty good and should be much easier to implement than the modular rack idea.

The idea is that effects will themselves be blocks that can be dragged around on the workspace. You would be able to expand them to edit the parameters and envelopes just like you can with sample blocks now.

The signal chain behind the scenes would route audio downwards through the lanes of a track, so the order of the lanes will have syntactic significance (rather than just being visual as they are now). The built-in effects will be lightweight entities that can be copied around and chopped up very quickly. They only exist for the time range that they cover on the workspace (with a short dry/wet crossfade at the starts and ends to avoid clicking).

I think there might be a technical challenge to contend with here when it comes to VST support because VSTs are more heavyweight entities and I don't know if the protocol was really designed with this kind of thing in mind. But I will see how it works when I get to it.

This also fits quite nicely with an older idea I had of "manipulator blocks" which would let you add these special blocks to the workspace that manipulate some parameter or setting of an adjacent block (these could also be modulators like LFOs or envelopes).

In this design there would no longer be a difference between block effects and track effects as I originally planned. A track effect could be achieved by just adding a big effect block to the bottom of the track (I would add some interface to have an effect block optionally infinitely expand to fill the track). Later on it may still make sense to have a separate "track effect" concept.

I feel a bit better having decided on this new system as the previous modular rack idea was causing me a lot of mental stress. This new design will certainly keep changing and I will likely encounter problems that I haven't thought of yet but I am glad I have a clear path forward now.

A brief summary of my progress on the current tasks:

1. The big refactoring: This is probably about 75% done now. The drag and drop system is almost entirely c++ now and feels much less weird and buggy than it does in v0.12.1. Almost everything to do with manipulating workspace/project items is now implemented in c++ and there have been a lot of optimizations and scalability improvements. A lot of interface elements still need to be rebuilt to hook into this new system.

2. Modularizing the sample playback engines: This is still in progress and the API is still in flux but I think I've more or less nailed down how it will work.

3. This new effects system: Not started yet, but I am now engineering everything else with it in mind. There is a possibility I might end up releasing an intermediate "no effects" build before moving on to re-implementing the effects in this new system.