Website Survey Results (by Erri120) (Patreon)

Hello users, my name is erri120 and I'm one of the developers of Wabbajack, creator of the Lotus Modlist, and the only person working on the Website. Before we go into the survey, here is a quick history of the website:

I pitched the idea of a simple GitHub Pages website for Wabbajack in October 2019 and began working on it. As some might have guessed if they looked that the Website or certain iterations of the Wabbajack UI itself: I'm not a UI person. In fact I didn't even know much about web dev at all only that JavaScript and SPA with React or other frameworks are very popular.

The first iteration of the Website went live nearly 2 years ago and was written in JavaScript with React. It had a front page, a gallery, a Modlist specific page and a page for documentation. Back in the day most Modlist authors used Google Docs for their README which broke the renderer since it only accepted Markdown READMEs. Nowadays everyone is using GitHub Repos and Markdown READMEs following the Lotus template after I  ̶f̶o̶r̶c̶e̶d̶ convinced them to switch.

After some time I came to absolutely hate working with JavaScript and Redux so for the 2nd iteration we swapped to mobx. This didn't last long and I found myself rewriting the entire thing for the 3rd time in TypeScript. The 3rd iteration went live in April/May 2020 and has been live ever since. Most changes were minor UI changes or additional features like the Archive Search or Manifest Viewer features.

Now it is September 2021 and Wabbajack 3.0 is in development. A lot of things have changed, mostly my experience and knowledge but also the technology. The push of WebAssembly (WASM) in recent years led to the creation of Blazor, a framework for creating websites using .NET. Blazor is a godsent for our project since we can use the same technology (.NET) in the Wabbajack application, build server and website. The 4th iteration of the website is currently still in development and the reason for this survey.

Now onto the survey:

We don't use analytics on the website so we had to make due with a simple survey. Nonetheless we got a total of 636 responses so lets get right into data:

The first question was just to make sure we are all on the same page and everyone participating knows we have a website. Still to my surprise 3.5% (22) did not know we have one.

The first real question was not about the website itself but how you found it. In hindsight I should have added more sources like YouTube since "Other" was the source for 30.2% (192) users. Nearly 50% said they used a Search Engine to find the site which is interesting because we have like no SEO. To my surprise the site was the first link in a Google and DuckDuckGo search so that is nice.

The next questions deal with the first experience a user has on the site. A majority of users, 81.1% (516), already knew about Wabbajack so they likely just wanted to go to the Gallery, Status page or use the download button.

An almost perfect 75 to 25 split with 74.5% (474) saying they joined the Wabbajack Discord via the website. This confirms how important the Discord button the front page is. Remember that we don't have any analytics so we don't know how many people actually clicked that button but this does suggest that it is very important for the growth of our community.

This is the most interesting question so far, since it is vital for the first user experience. Ideally the front page should briefly explain what Wabbajack is, what it does and what it doesn't do. Most designers say you have ~7 secs to hook the user before they loose interest. I'm very surprised at the 71.7% (456) who said the front page does a very good to excellent job at explaining what Wabbajack is. The front page was a pain point since the first iteration and the current live version is still very shit. My guess is people don't have high expectations for a website of a fully open source community made tool in a niche gaming scene.

When developing the website I added features that I thought would be nice to have or only useful to Modlist authors, like the Status Dashboard. The results of this question paint a different picture: Aside from the two obvious winners with the most users it does surprise me how many people use features like the Status Dashboard or the detailed status page of a Modlist. I think some people thought "Detailed Status of a Modlist" is the Modlist specific page you reach via the Gallery that shows the README of a Modlist. However the detailed status page is the page containing a list of all failing and succeeding archives that you can access via the Status Dashboard. Anyways, the Manifest Viewer is the most underused feature which I can totally understand and another questions further deals with the Archive Search.

Since I'm currently working on the 4th iteration, I thought it would be a good idea to see what you think needs changing. I have decided to create 3 diagrams using the results. One is a simple column chart and the other combines the "Requires massive changes" and "Needs a complete rework" columns into one. The result is that the Archive Search needs the most attention (69 votes) followed by the Front Page (49). The Gallery (37) and Detailed Status Info (34) need the least changing according to the survey.

These results are similar to what I think needs changing. The Gallery has always been the main point of the website so it has received the most attention so far. I already talked about how shit the front page is but I'm still surprised that the Archive Search popped up again. In the previous question we looked at the usage of certain pages and the Archive Search was surprisingly used very often. This result might also be very biased since the Archive Search is currently broken...

This is one question where I should have put more thought into the answers since lots of people wrote "I don't use it" instead of skipping the question... Nonetheless we have 532 responses and most users apparently compare Modlists. This is very surprising to see since loading multiple Modlists has always been very hacky and slow. The Archive Search was initially an experiment I did with comparing Modlists being even more experimental. This will definitely get more attention in the next version.

Simple question with expected results: Most users visit the site on a monthly basis. Nothing special here so let's move on.

The last few questions are all "How happy are you with..." or "How good is..." so could directly rate certain aspects of the site starting with the overall design. Like I said at the beginning, I'm not a UI person I'm a programmer who mostly works on libraries and applications so seeing that 80.7% (505) rated the design 7+ makes me very happy and excited because the new design is already 100x better.

Most people are very happy with the performance of the website which is good since I actively worked on that. Some parts like the Archive Search where tons of items need to be rendered at once are still slow but that will be fixed in the new version.

"Do you require accessibility features when browsing the web?" + "How good are the accessibility features of the website?"

I was surprised to see how much information and guidelines exist to make the web more accessible. Everything from aria labels to colour contrast and fonts is now important when dealing with accessibility. These 2 questions are for people who require these features which are apparently 6.2% (38). I don't require these features so I can only go off based on what my developer tools tell me. It's nice to see that most rating 5+ which I expected since I haven't done that much except adding alt information to images. I should have included a note that this question is for people who require accessibility features so here is a chart using the results of only those people:

This is overall a good mid-range which I'm fine with. I might look into adding more accessibility features once the new version is live but that might take a while.

I already talked about how the website doesn't really have any SEO but you still gave it a 10/10. The main problem is that the website is a SPA (Single-page application) hosted as a static site on GitHub Pages. This is a problem because most crawlers can't deal with SPAs and only analyse the static HTML file returned by the server which only contains the text "Loading..." and a script for loading the application.

There are some ways around it, mainly using a Server to pre-render the content however we are using GitHub Pages so we can't do that. Another option is to pre-render the contents during build time and use that as a static option. The loading script will then replace the static site with the live thing once it's loaded so you would get the best out of both worlds though I'm still researching how to implement this correctly with Blazor.

"Any ideas or feature requests for the website?"

This question got 168 responses, here are some of them:

"Manifest viewer could do with human readable names, rather than names of zip files"

Without going into much detail: the website uses the status report of a Modlist for the Archive Search and Manifest Viewer. This status report contains information about every Archive that Wabbajack will download. The problem is that every Archive has a file name but not a mod name. For example: Wabbajack can download from popular file hosters like MEGA or GDrive. In the status report those Archives don't have a mod name, they only have a file name and the link to the download page.

"something like news maybe"

I created some spicy tech in the new iteration so we could put Patreon posts directly on the Website.

"Being as critically-minded as I can be, I can only really think of maybe a quick list somewhere on the website that shows a rundown of "We currently support these games, and this is how many mod packs we have available for them currently", but overall, its pretty great, keep up the good work!"

We will add more stats to the front page like overall downloads and installs.

"Either have links to modlists in the gallery open a new tab or save filters between navigating pages. It's a mild but frequent annoyance having to constantly reselect that I want to browse only Skyrim SE lists whenever navigating back to the gallery."

Highly requested feature, never thought of that probably because working with states in React was a pain.

"where is lotus"

In your heart.

"Browsing mods from archive could have some attributes for ordering like by foliage/weapons/armor or by the separator names or something "simple" like that."

I briefly explained what the Archive Search above and there are multiple responses that indicate some misinterpretation of the feature. I think adding a description to the archive search explaining what it does will alleviate the confusion.

"Adding a accessibility button like going to the top of the page on right side bottom would help."

Noted, though it will likely be a pain to get this to work correctly.

"Basically, let me understand the point of the modlist better from the website so I can decide if I want to invest the time and bandwidth to download it."

I'm not sure I can do much about this on the website. The gallery serves as a place of discovery with a quick description giving you some info and if you want more you can click on the View button to see the README. Other responses also talked about this so it might be that Modlist READMEs should include more information.

"Filtering between tags while looking through the gallery"

This has also been a frequent request. I'm not sure how to go about this since the current search will simply checks if any Modlist titles contain the search text. I don't think adding another search text to the gallery is a good idea so maybe an advanced search where you can select the tags you want? Not sure, will have to brainstorm more on this.

"Thank for your not collecting analytics"

It does make things a bit more trickier since we have no idea what people actually access and use on the site. Before this survey I thought only Modlist Authors and helpers on discord use the Status Dashboard but apparently way more people do.

"Better tags. It feels like a lot of the mod lists use the same tags"

We don't have an actual tag system in place. Modlist authors can use whatever they want as a tag which is why it's weird sometimes.

"Featured/New Modlists. Better sorting of modlists in Gallery, ie by name, creator, downloads, rating"

This goes in hand with what others also suggest: some sort of news/update section. I'm not sure how to do this since the website gets a list of all Modlists and there are no time stamps so we might have to add that. As for the Gallery improvements: we, Wabbajack Staff, Developers and Modlist Authors, are against adding Modlist specific ratings and download counts. I won't provide an in-depth explanation but the main reason is that those 2 factors should not influence your choices. Searching by creator does seem like a good idea though.

"Easier way to compare. Examples would be a table with attributes and check marks for each mod to show if if it had it."

I'm guessing this is about the Archive Search and loading in multiple Modlists. Like said before, the Archive Search does not know about mods but archives (thus the name) and some Modlists have 100 Archives and others have 7000 Archives. It is possible however it is very difficult and not something I want to design.

CONCLUSION

This survey has been very informative and provided good insight into the userbase. The next version of the website and Wabbajack itself is currently still in development but is already very sexy and fast. Everything around Wabbajack is open-source and you can help develop the application and website on GitHub: https://github.com/wabbajack-tools/wabbajack-tools.github.io. You can check the status of the rework for the website in the mega issue here: https://github.com/wabbajack-tools/wabbajack-tools.github.io/issues/356. The data of the survey is available on Google Spreadsheets: https://docs.google.com/spreadsheets/d/1R1R9cIRPagA-OYg2-I_wi82v8Iv9f3lLwcI8cRdVcEM/edit?usp=sharing