Wabbajack 3.0 - Development Update #3 (Patreon)

I finally sat down this past week and figured out CEF (Browser) integration with the new UI toolkit (Avalonia) in Wabbajack 3.0. UI development in general is always a pain and a bit of a time sink, so it's nice that some of he hardest parts of the UI setup are now coded and in pace. The framework for creating "scenes" or modes of the UI works, we can load up a browser window, login to several sites, and switch between UI screens. I'll come back to that code in a week or so when the rest of the app is a bit more fleshed out.

I also spent time coding up a new texture resizer that works way better than the DirectX based resizer in 2.5, as that code was Windows only. The new code isn't quite as fast, but results in much better image matching, is cross platform, and won't crash with strange errors as much.

After the UI work I swapped over to working on the list validation code. I'd sill like to get the list validator running on 3.0 so we can have some of this code in-place and working with Wabbajack 2.5 while coding continues on 3.0. Having one branch of code disconnected with the stable branch creates problems over time, since it's hard to want to fix something in 2.5 when we'll just need to port the changes into 3.0, essentially writing the fix twice. The validator works well enough, but uses way too many system resources, at one point trying to make over 200 web requests at one time, so the limiter needs a bit of reworking. But I did get several more downloaders coded.

This brings me back to some thoughts about the "task" or "CPUs" model of Wabbajack up to this point. Up until now, Wabbajack has always had this view of the world that it has a certain amount of tasks to accomplish, and tries to not overwhelm a computer by only doing a few of them at once. However this is an imperfect model, let's look a bit at why.

* CPU access should be limited by number of high computation tasks being run. Don't extract 200 files at once, this will overwhelm the CPU and cause it to stutter while the user is trying to use it

* Internet bandwidth - we want to use as much bandwidth as possible, but at some point adding more connections just slows everything down, it's better to have 10 connections running at 1x speed, than 20 connections running at 0.5x speed. Also it would be nice for users to be able to throttle network bandwidth if they want to play nice with other people on their network

* Network connections - some home routers try a round-robin approach to traffic balancing. If we allocate 200 connections, your WJ download will run fast, but other people on your network will only get 1/200th of the total bandwidth of your Internet. So that's another reason to limit the number of requests

* Disk bandwidth - same sort of thing as the Internet, a SSD can only push about 500MB/sec. No reason to go much faster. The number of tasks doesn't matter here as much as the amount of bandwidth being used.

* Disk tasks - HDDs prefer to only do one task at once as they only have one motor to move the read/write head. We can't really design for that, but we should be able to limit the tasks to two as much as possible.  

As you can see, some of these limits are based on a count: number of connections/tasks. While others are metered (bandwidth). In the past Wabbajack has lumped all these together, but I don't think that's a good approach. So this is one of my next tasks (pun not intended): figure out a way to limit the different resource usages and allow users to configure each. I'd also love to find a way to show realtime feedback on all these values, so once a second or so the UI could update with "Downloading: 100MB/sec, Hashing: 60MB/sec, Disk Usage: 60MB/sec". But I think that's probably the easy part of this problem.

Thank you for your support, development is going well, I wake up each morning energized, looking forward to the hours I can spend on coding these improvements. We're still several weeks out from a beta, but we're getting there.