Howdy, Stranger!
It looks like you're new here. If you want to get involved, click one of these buttons!
Quick Links
The trouble with DLSS and related algorithms
The short version is that there is no free lunch. There is no substitute for having enough samples to produce high quality images at high frame rates. But if you can't generate as many samples as you want as fast as you want, then there are trade-offs as to what penalty you take for that.
I made a thread on a related topic a little shy of two years ago. That's still accurate and relevant to the topic at hand, so you may wish to brush up on it if you'd like to know more about this topic. The old thread is here:
https://forums.mmorpg.com/discussion/491977/the-truth-about-high-resolutions-dlss-and-variable-rate-shading
I'm not going to rehash the information in that old thread, however. This thread has some new stuff to say. Today, I want to talk about the trade-offs inherent in using DLSS, FSR, and Nvidia's new frame generation, and important ways that they're different from each other. Such novel methods of generating frames from samples create new options, but it's good to understand the trade-offs. They don't just magically give you better image quality or higher frame rates for free.
Let's back up a bit and explain some terms. The traditional way to do 3D graphics is to have one sample per pixel per frame. You do a bunch of computations in order to conclude that this pixel of this frame should be this particular color. You do analogous computations for each other pixel in a frame, and that's how you render a frame.
But it doesn't necessarily have to be one sample per pixel. You can generate a sample at any arbitrary location on the screen, not just the center of a pixel. You can have multiple samples per pixel and then average their colors to get the color that will be displayed on a monitor. That's called super sample anti-aliasing, or SSAA. You can also have multiple pixels per sample, such as having one sample at the center of each 2x2 grid of pixels and using the color chosen for all four of the pixels. Doing that in the traditional way is called upscaling.
Fundamentally, a sample is one set of computations that say, in this particular frame, the color at this particular point on the screen should be such and such. Generating the samples is most of the work of rendering a frame. For a simple method of one sample per pixel, mapping samples to pixels is trivial. Some recent methods such as DLSS, FSR, and Intel's new XeSS use more complicated methods to map samples to pixels, with each sample affecting multiple pixels (sometimes spread across multiple frames) and each pixel relying on multiple samples.
For simplicity, let's suppose that a video card can generate a fixed number of samples per second. This isn't actually true, for a variety of reasons. For starters, it will vary a lot by game, and also by the graphical settings that you use within a particular game. Let's suppose that we're talking about some fixed game and some fixed settings that you want, with various options turned on or off, scaled up or down, or whatever is relevant.
The assumption of a fixed number of samples per second is also untrue because most stages of the graphics pipeline don't actually depend on the monitor resolution chosen. Rather, the video card computes various information about triangles or quads in screen space without knowing the resolution, then determines which pixels (or more properly, sample locations) each triangle covers at rasterization. Only after this does it do separate computations for each sample. Still, as you can see from many benchmark results, performance scales strongly with monitor resolution, so a large fraction of the workload usually comes after rasterization, and thus scales linearly with the number of samples generated.
Another reason why the fixed number of samples per second is untrue is because more complex methods of generating frames from them carry additional computational overhead. If the GPU is spending 90% of its time generating samples and 10% of its time assembling them into frames to display on your monitor, then it's naturally going to generate fewer samples than if it can spend 100% of its time generating them. This effect makes all of the newer methods such as DLSS worse than they would otherwise be, but it's not a large effect.
Even so, let's suppose that you can generate some fixed number of samples per second for the sake of simplicity. For concreteness, let's suppose that it's 248,832,000. That's enough to render a game at 3840x2160 at 30 frames per second, or 2560x1440 at 67.5 frames per second, or 1920x1080 at 120 frames per second. Even using traditional rendering, you get a strong trade-off between monitor resolution and frame rate.
Being able to generate more samples per second makes everything better. If you double the number of samples per second that you can generate, then you can double your frame rate at a given resolution. Alternatively, you could maintain the same frame rate at a higher resolution. Similarly, fewer samples per second makes everything worse.
The idea of FSR is basically to say, let's not generate all of the samples for a frame. Let's only generate half of them, and then use some fancy algorithm to create the full frame. Because you're only generating half as many samples per frame, you double your frame rate. However, because you only have half as many samples available, you get worse image quality. It doesn't have to be exactly half, and could be any other ratio. But the general rule is that the more samples that you have, the better the image quality that you can get.
DLSS takes a fundamentally different approach. It says, each time we generate a frame, let's only generate half of the samples. That allows us to double the frame rate. However, rather than only using half as many samples to generate a frame, let's use the samples that we generate for one frame and also the samples that we generated for the previous frame. Now we have the full number of samples, so we can get higher image quality.
There is, of course, the drawback that the samples from the previous frame will be wrong because things have moved. They improve on this by attaching a motion vector to each sample. That is, if a particular object is moving up and left on the screen at a rate of 0.04 pixels per millisecond, and 20 milliseconds have passed since the last frame time, then the sample for the new frame will be the old sample color with the position moved up and left by 0.8 pixels. If everything is moving at constant speed on your screen and not accelerating at all, then this works great.
DLSS doesn't say that you have to take exactly half of the samples or use samples from exactly two consecutive frames. As with FSR, you can adjust the ratio of samples to pixels, with more samples inevitably resulting in higher image quality. You can also adjust the number of frames that you consider, and give less weight to older frames.
Because DLSS is able to recycle samples from previous frames, it can get a lot more samples. When it works well, that will give you much higher image quality than FSR. Provided that everything is moving at constant speed and not changing color or brightness, with enough samples and enough frames, DLSS image quality can even rival that of SSAA. Thus, used properly, DLSS will tend to result in better image quality than FSR.
I made a thread on a related topic a little shy of two years ago. That's still accurate and relevant to the topic at hand, so you may wish to brush up on it if you'd like to know more about this topic. The old thread is here:
https://forums.mmorpg.com/discussion/491977/the-truth-about-high-resolutions-dlss-and-variable-rate-shading
I'm not going to rehash the information in that old thread, however. This thread has some new stuff to say. Today, I want to talk about the trade-offs inherent in using DLSS, FSR, and Nvidia's new frame generation, and important ways that they're different from each other. Such novel methods of generating frames from samples create new options, but it's good to understand the trade-offs. They don't just magically give you better image quality or higher frame rates for free.
Let's back up a bit and explain some terms. The traditional way to do 3D graphics is to have one sample per pixel per frame. You do a bunch of computations in order to conclude that this pixel of this frame should be this particular color. You do analogous computations for each other pixel in a frame, and that's how you render a frame.
But it doesn't necessarily have to be one sample per pixel. You can generate a sample at any arbitrary location on the screen, not just the center of a pixel. You can have multiple samples per pixel and then average their colors to get the color that will be displayed on a monitor. That's called super sample anti-aliasing, or SSAA. You can also have multiple pixels per sample, such as having one sample at the center of each 2x2 grid of pixels and using the color chosen for all four of the pixels. Doing that in the traditional way is called upscaling.
Fundamentally, a sample is one set of computations that say, in this particular frame, the color at this particular point on the screen should be such and such. Generating the samples is most of the work of rendering a frame. For a simple method of one sample per pixel, mapping samples to pixels is trivial. Some recent methods such as DLSS, FSR, and Intel's new XeSS use more complicated methods to map samples to pixels, with each sample affecting multiple pixels (sometimes spread across multiple frames) and each pixel relying on multiple samples.
For simplicity, let's suppose that a video card can generate a fixed number of samples per second. This isn't actually true, for a variety of reasons. For starters, it will vary a lot by game, and also by the graphical settings that you use within a particular game. Let's suppose that we're talking about some fixed game and some fixed settings that you want, with various options turned on or off, scaled up or down, or whatever is relevant.
The assumption of a fixed number of samples per second is also untrue because most stages of the graphics pipeline don't actually depend on the monitor resolution chosen. Rather, the video card computes various information about triangles or quads in screen space without knowing the resolution, then determines which pixels (or more properly, sample locations) each triangle covers at rasterization. Only after this does it do separate computations for each sample. Still, as you can see from many benchmark results, performance scales strongly with monitor resolution, so a large fraction of the workload usually comes after rasterization, and thus scales linearly with the number of samples generated.
Another reason why the fixed number of samples per second is untrue is because more complex methods of generating frames from them carry additional computational overhead. If the GPU is spending 90% of its time generating samples and 10% of its time assembling them into frames to display on your monitor, then it's naturally going to generate fewer samples than if it can spend 100% of its time generating them. This effect makes all of the newer methods such as DLSS worse than they would otherwise be, but it's not a large effect.
Even so, let's suppose that you can generate some fixed number of samples per second for the sake of simplicity. For concreteness, let's suppose that it's 248,832,000. That's enough to render a game at 3840x2160 at 30 frames per second, or 2560x1440 at 67.5 frames per second, or 1920x1080 at 120 frames per second. Even using traditional rendering, you get a strong trade-off between monitor resolution and frame rate.
Being able to generate more samples per second makes everything better. If you double the number of samples per second that you can generate, then you can double your frame rate at a given resolution. Alternatively, you could maintain the same frame rate at a higher resolution. Similarly, fewer samples per second makes everything worse.
The idea of FSR is basically to say, let's not generate all of the samples for a frame. Let's only generate half of them, and then use some fancy algorithm to create the full frame. Because you're only generating half as many samples per frame, you double your frame rate. However, because you only have half as many samples available, you get worse image quality. It doesn't have to be exactly half, and could be any other ratio. But the general rule is that the more samples that you have, the better the image quality that you can get.
DLSS takes a fundamentally different approach. It says, each time we generate a frame, let's only generate half of the samples. That allows us to double the frame rate. However, rather than only using half as many samples to generate a frame, let's use the samples that we generate for one frame and also the samples that we generated for the previous frame. Now we have the full number of samples, so we can get higher image quality.
There is, of course, the drawback that the samples from the previous frame will be wrong because things have moved. They improve on this by attaching a motion vector to each sample. That is, if a particular object is moving up and left on the screen at a rate of 0.04 pixels per millisecond, and 20 milliseconds have passed since the last frame time, then the sample for the new frame will be the old sample color with the position moved up and left by 0.8 pixels. If everything is moving at constant speed on your screen and not accelerating at all, then this works great.
DLSS doesn't say that you have to take exactly half of the samples or use samples from exactly two consecutive frames. As with FSR, you can adjust the ratio of samples to pixels, with more samples inevitably resulting in higher image quality. You can also adjust the number of frames that you consider, and give less weight to older frames.
Because DLSS is able to recycle samples from previous frames, it can get a lot more samples. When it works well, that will give you much higher image quality than FSR. Provided that everything is moving at constant speed and not changing color or brightness, with enough samples and enough frames, DLSS image quality can even rival that of SSAA. Thus, used properly, DLSS will tend to result in better image quality than FSR.
Comments
So DLSS 2 or earlier can't increase your FPS when you're CPU bound, but DLSS 3 can.