Refactorings and Rewrites (Patreon)

It's been a busy past few weeks, and I haven't had time to update y'all on what's been going on, but I finally have some breathing room.

2.0 was released recently and we've already had several point releases after that fixing bugs and the like. More recently, a new developer has started providing UI updates (thanks  Khamûl), and their work can be seen in the new filters and infos in the modlist gallery. Erri has been hard at work on the website and has made several improvements that will pay off once the build server gets in a better state.

The Wabbajack Server, also known as "The Cache Server" or "The Build Server". Is a rather beefy box I maintain that performs several operations:

* It caches Nexus requests so you can make thousands of requests for Nexus metadata without blowing through your API budget (2500 a day after that 100 per hour).
* It stores VFS info so during compilation your copy of WJ can download information about a huge BSA or 7z without having to analyze the file. It's often faster to download a 400KB metadata file than to scan and hash 2GBs of a BSA
* It allows .meta information to be looked up by hash. If WJ doesn't know where a .7z came from the WJ server may have that information. Note: the server doesn't serve up that .7z it serves up information on what servers (Nexus, etc.) have that .7z
* It downloads every WJ curated list and makes sure that any of the archives in the list aren't broken.
* several other such things...

This server has been patched and half-upgraded countless times as I've often been more concerned with the main app, and getting new features out, than general code clarity or simplicity. Well this week that all came crashing down. I hit an issue where the server code would randomly die, and as far as I could tell it wasn't code I wrote that was the source of the failure. So after debugging it for several days, and dealing with a family emergency in the middle of it all, I finally decided to "do the right thing" and rewrite the code from scratch.

"From Scratch" is a bit less dramatic than it sounds. In general a "refactoring" is when you make major changes to the code, moving stuff around, fixing bugs, etc. However a "rewrite" like this normally means throwing the existing project out, starting a new code project, and then selectively copying back in features and code from the discarded project. Thanks to Git(hub) nothing in the code base is every truly "deleted" so I threw out the old project, and then began to pull in the parts of the server code section by section. A major advantage with this approach is that it allows the person doing the rewrite to evaluate each function in the code to think how it could be improved, or simplified. Often the first time a function is written the programmer only has a partial understand of the problem space. So looping back and rewriting the code once the issues are better understood results in better code.

After only a few days the server is back up and running, and thus far completely without major issues. The Nexus caching and CDN interaction functions are back up and running and since I've taken the time to "do it right" this time, the resulting code is simpler, often faster, and much cleaner.

So that's all why you haven't heard much from me lately, things are going great, we're continuing to see linear growth in the WJ community and the discord is over to 18,600 members. Thank you all for your support both in the discord and here on Patreon. The funds here on this page have been very helpful in getting the server hardware needed to maintain this project, as well as the on-going costs of CDN hosting, developer tools, etc.