Video Encoder Quality Comparison

A look into software vs Nvidia hardware video encoding performance in 2026.

The focus is on encoding gameplay videos both for the web and for local storage, where maximizing quality at low bitrates is most important. Fast movement of games and 60 fps makes lower bitrates even more challenging.

Test Clips

Assassin's Creed Black Flag Resynced

4K 60 fps HDR 17 s

Lots of motion and small details in vegetation and grass.
Halo: Campaign Evolved

4K 60 fps HDR 24 s

Less detailed environment but with constant camera movement.
EA Sports WRC

4K 60 fps SDR 14 s

Fast paced but predictable camera move, and motion blur helps to hide smaller details.

All source videos were encoded at 100 Mbps AV1.

Measured Quality

Reference: —
Span: —
Model: —

VMAF

Hover the chart to read values between measured points. Click a curve in the key to hide it, or a codec heading to toggle the whole group.

Visual Quality

Compare cropped still frames from the encoded videos:

Run

Bitrate

Observations

Software Vs Hardware Encoding

AV1

Even with the new UHQ tuning and improved 9th gen NVENC encoders in Blackwell GPUs, software AV1 encoding remains undisputedly best quality especially at lower bitrates. When comparing libsvtav1 preset 2 against av1_nvenc p7 uhq, the software encoded result tops every metric and visual comparison.

The story changes when using faster presets such as the default libsvtav1 preset 8. In such case NVENC is ahead in nearly all metrics. So as long as you are willing to tolerate 2-3 times slower encode of mid range libsvtav1 presets, or up to 10 times slower encode of highest quality presets such as preset 2, software encode still has clear lead. But when using faster presets, you might as well use NVENC.

H265

With older codecs the difference gets closer. Both VMAF and SSIM metrics favor libx265 slow preset but hevc_nvenc is slightly ahead in PSNR. In visual comparison NVENC video looks less noisy but still preserves all the important detail. The default libx265 medium preset is clearly the weakest of the three.

H264

For H264 differences are small and hard to define. At lower bitrates libx264 veryslow preset produces sharper details, but because most of the image is just compression artifacts it looks very noisy in motion. Another thing that the quality metrics don't show is that even at moderate bitrates h264_nvenc tends to produce a fixed looking artifact pattern on smoother surfaces such as the sky. In such scenes both software encoded variants look better even in cases where they are less detailed.

Encoder Settings

Here is a breakdown of the used encoder options and versions. All encodes used ffmpeg 9.0.

Also main focus being web encode, all outputs used -movflags +faststart to have all video metadata at the beginning of file and GOP interval at 1 second -g 60. Latter also ensures that when still frames are captured for visual comparison it can be ensured that previous I-frame is fixed distance away from the captured frame.

Software Encoding

AV1

AV1 tests were encoded with SVT-AV1 4.2.0 in CRF mode. CRF option values range from 0 to 63 where lower values produce higher quality / higher bitrates. You can see list of available preset options and their exact effects at SVT-AV1 FAQ page.

ffmpeg -i input.mp4 -c:v libsvtav1 -crf {crf} -preset {preset} -fps_mode passthrough -g 60 -movflags +faststart output.mp4

Note that it maybe preferable to also use -svtav1-params tune=0 which uses VQ decision path to prioritize details subjectively more visible to humans, but may also introduce artifacts in other parts of image. Default tune is more faithful to PSNR which may use bits on details that don't matter as much. It may also be worth using film-grain/film-grain-denoise options when encoding video with lots of film grain or noise.

H265

H265/HEVC tests were encoded with x265 4.3 in CRF mode. You can find list of preset options, CRF range and other settings in ffmpeg x265 documentation.

ffmpeg -i input.mp4 -c:v libx265 -crf {crf} -preset {preset} -tag:v hvc1 -fps_mode passthrough -g 60 -movflags +faststart output.mp4

H264

H264 tests were encoded with x264 v0.165 in CRF mode. You can find list of preset options and settings in ffmpeg x264 documentation.

ffmpeg -i input.mp4 -c:v libx264 -crf {crf} -preset {preset} -fps_mode passthrough -g 60 -movflags +faststart output.mp4

Hardware Encoding

Only Nvidia encoder was tested as I don't currently have any RDNA 4 devices with the Video Core Next 5.0 encoder for fair comparison. Same with Intel. Apple's VideoToolbox doesn't currently support AV1 and is generally not as good at lower bitrates.

All tests were done using Blackwell (GeForce RTX 50) that have NVENC 9th gen encoders, 610.88 driver and FFmpeg build that used NVIDIA Video Codec SDK 13.1. All encodes use VBR-CQ rate control mode for best quality per bitrate.

Hardware video encode quality highly depends on the hardware itself. Even using identical settings and profiles and same SDK versions a newer generation encoder engines on the GPU may produce massively different results. For example RTX 50 series using same settings usually produces 5% lower bitrate for same quality than RTX 40 series encoder.

AV1

UHQ variant was encoded with:

ffmpeg -hwaccel cuda -hwaccel_output_format cuda -i input.mp4 -an -c:v av1_nvenc -preset p7 -tune uhq -multipass fullres -rc vbr -cq {CQ} -b:v 0 -maxrate 100M -fps_mode passthrough -g 60 -movflags +faststart output.mp4

Notes on used settings:

-hwaccel_output_format cuda allows video frames stay within GPU memory which improves performance a bit. But note this may cause conflicts with filters without additional configuration.
-preset p7 is currently the highest quality preset available in NVIDIA Video SDK.
-tune uhq is currently the highest quality tuning available. This gives huge bump in quality per bitrate.
-multipass fullres allows encoder to run two internal passes per frame. This in my testing gives measurable bump to the quality on all metrics. But note that this can make output non-deterministic, meaning different runs at exact settings can produce slightly different results.
-rc vbr -cq {CQ} activates VBR-CQ variable rate, constant quality mode, which is closest match to CRF behavior and in my test produces clearly best results.
-b:v 0 allows variable bitrate mode. It has to be lower than maxrate, other than that it is ignored by FFmpeg in VBR-CQ mode so there is no point setting it to other than 0.
-maxrate 100M is required for the VBR-CQ to behave consistently. Without a maxrate set, the FFmpeg implementation behaves unpredictably at lower CQ values and will suddenly jump to extremely high bitrate when CQ stops limiting the bitrate. With explicit maxrate setting the bitrate will converge towards it in a smooth curve when CQ doesn't limit the bitrate.

Settings not used, but often included in examples:

Lots of examples online specify -rc-lookahead or -tf_level, but these are automatically enabled with UHQ tune.
-bufsize is also often used in NVENC examples, but it doesn't affect CQ mode.
-spatial-aq and -aq-strength didn't do anything in my tests, so I assume UHQ tune has that handled automatically.
-b_ref_mode each and -bf 4/5 option didn't seem to have meaningful effect for the AV1 encoder. Adjusting this in some cases made it very slightly better, but in other cases very slightly worse.

HQ variant was encoded with fairly basic configuration so it better works as reference for older generation GPUs or for not so optimal NVENC configurations:

ffmpeg -hwaccel cuda -hwaccel_output_format cuda -i input.mp4 -an -c:v av1_nvenc -preset p7 -tune hq -rc vbr -cq {CQ} -b:v 0 -maxrate 100M -fps_mode passthrough -g 60 -movflags +faststart output.mp4

H265

For H265 I also added -b_ref_mode each and -bf 4 which enables bit more extensive B-frame reference lookup. In my testing -bf 4 worked best in various tests so I settled on it. This gave solid bump in quality per bitrate in all metrics on top of the other settings.

ffmpeg -hwaccel cuda -hwaccel_output_format cuda -i input.mp4 -an -c:v hevc_nvenc -preset p7 -tune uhq -multipass fullres -b_ref_mode each -bf 4 -rc vbr -cq {CQ} -b:v 0 -maxrate 100M -tag:v hvc1 -fps_mode passthrough -g 60 -movflags +faststart output.mp4

H264

H264 doesn't support UHQ tune and many of the other options caused unexpected drop in quality.

However I added -spatial-aq 1 -aq-strength 4 because h264_nvenc is very prone to have temporal compression artifacts in smooth/flat surfaces such as the sky. Using spatial-aq mitigates this by allocating more bits to smoother surfaces. However I used a very low strength (4 out of 15) to avoid other parts of the image declining too much.

ffmpeg -hwaccel cuda -hwaccel_output_format cuda -i input.mp4 -an -c:v h264_nvenc -preset p7 -tune hq -multipass fullres -spatial-aq 1 -aq-strength 4 -rc vbr -cq {CQ} -b:v 0 -maxrate 100M -fps_mode passthrough -g 60 -movflags +faststart output.mp4