New GPU features and how they work (Patreon)

Hi everyone,

if you followed my previous activities, you have probably seen that i was always very interested in cool features. Things like Savestates, FastForward or graphical improvements. But why?

Recreating some old system is one part of the work, but doing some design myself is the other part that i cannot give up. It's just so much fun when i understand a system in a way that i'm able to improve it optionally and do that with a design i have to come up with myself, rather than recreating some existing design.

If you are not interested in these parts of the core, please forgive me. It's just my way to play with the core.

Today we will look at the rendering path that got two optional features recently:

- Bilinear Filtering

- 24Bit color

Both features take place in the pixel rendering module of the PSX core. 

Let's look at an overview:

Rendering starts in the different geometry units, which are the line renderer, the rectangle renderer and the polygon(triangle, quads) renderer.

Each of these modules can generate pixel data consisting of position(x,y), color and texture information.

With this information, the final color and position in the framebuffer is determined, but it still has to be applied.

First, the color from the texture is looked up from the Texture Cache and, depending on the color mode, another lookup for palette color information is required.

Then the texture color and the generated color from the geometry unit are blended together(multiplied) and clipped. At this point dithering is also applied, because the color depth will be reduced afterwards.

In another optional step, the final color can be combined with old color information from the framebuffer, allowing for transparency effects.

You can see the color width of each step in the image by the numbers:

- texture data is always 5 bit per color (15 bit total)

- color from geometry is always 8 bit per color (24 bit total)

- after blending it's also 8 bit per color

- stored in VRAM are only 5 bit per color (15 bit + 1 alpha bit)

If the PSX can generate 8 Bit per color in the GPU, why does it only store 16 Bit color in VRAM? It could do full 24 Bit color just fine?

The problem is the memory bandwidth and size. 24 Bit Color with the same framebuffer size would have required at least 1.5Mbyte of RAM instead of 1 Mbyte and at least 50% more bandwidth.

The good thing for us with MiSTer is the bandwidth and size of our VRAM storage: The DDR3 memory that is used for VRAM has plenty of space left and the additional bandwidth is also still available.

So what does the "Render 24 Bit" feature do now?  It's relativly simple: we store 32bit per pixel in the VRAM.

The original 16 Bit are stored in the original VRAM, while another second "shadow" VRAM is placed somewhere else in DDR3. Each pixel write now write two times to DDR3: for the first 15+1bit of color and also for the remaining 9 bit which would have been thrown away otherwise.

When the data is later send to the screen or HDMI scaler, both VRAM areas must be read again and the information combined. That's it already. The complicated part is here mostly the sharing of DDR3 ressource, which is now used for VRAM, second VRAM , but also SPU RAM and Memory cards at the same time. Oh and by the scaler and the Linux of course. What a little wonder this memory is :) 

The main purpose is, that with 24Bit rendering the dithering can be turned off. The result is very nice:

You can see the crystal clear sun effect in Colin 2.0 Rally.

The Bilinear Filtering part on the other hand needs a bigger change.

Bilinear Filtering requires 4 texture pixels to render 1 pixel of e.g. a triangle.

If a pixel is to be rendered, the color must be picked from the texture. Typically the position in the texture is much more accurate defined than in full numbers. In PSX we have 12 bit more accuracy than that, allowing for 4096 substeps.

In the example the information is picked where the white cross is. It's in the black area, so the pixel will be black.

With texture filtering, the color will be also determined by the adjacent texture pixels. The amount of intensitiy each of these 4 contribute to the final color comes from the position of the sample point. In this example:

As the sampling point(x) is in the upper left corner, the black intensity will be much stronger than the red and green intensity, with blue being the weakest.

It's only some basic math required to calculate the filtered texture color, but having 4 texture lookups per pixel is not something the PSX was made for as the Texture Cache can only provide 1 pixel per cycle.

How can we solve this? Brute force:

We can quadruple the Texture Cache and the Palette Cache!

This sounds insane, but the amount of Texture Cache in the PSX is minimal with 2KByte only. 

We have around 200 KByte of internal BlockRAM free in the PSX core, so spending another ~16 Kbyte or so for texture caches and palette caches isn't really a big deal.

How does it look like?

If we take this example from Croc, you can see that textures which are scaled up to big size profit a lot:

You don't see large bricks of pixels, but instead the whole structure of the texture is much smoother.

You also get antialiasing effects inside the textures. The dark area in the middle of the left image has a sawtooth effect, while on the right side there is a straight border without zigzag.

Does is work well everywhere? No.

What happened here?

2D elements will get blurry when filtered and cannot profit at all.

A console and game engine that know about filtering can of course turn it off for such elements, but as the PSX never had this feature, how should the game know it?

Instead, we can try to guess if an element is 3D or 2D and depending on that filter it or not.

The easiest would be if the developers would just use 2D draw commands for 2D elements, because, well, that's what they are made for. Also, they render much faster.

But if developers didn't do it we can still find out: if a polygon is spanning up a accurate rectangle and the texture scaling inside is 1:1, it's very likely a 2D element.

So i added such a 2D detection and it indeed works: most menu and HUD elements are detected and not filtered. Just like in Crash Team Racing:

Much better!

All new features are available as testversion now:

https://github.com/MiSTer-unstable-nightlies/PSX_MiSTer/releases/download/unstable-builds/PSX_unstable_20221012_122517.rbf

The next official release will follow soon.

I will go back to work on the accuracy tests and fixes next, but i had to take off some days to have some fun.

Hopefully you will also have fun with the new features!