Cannot hardware transcode from h265 to h264 using ffmpeg and vaapi

HW encoding with ffmpeg and Intel throught vaapi is working for H264 → H264.

$ ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i 'input.h264.mkv' -c:a copy -c:s copy -c:v h264_vaapi 'output.h264.mkv'

But not for H265 → H264.

$ ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i 'input.h265.mkv' -c:a copy -c:s copy -c:v h264_vaapi 'output.h264.mkv'

[h264_vaapi @ 0x5dd7efe1d040] No usable encoding profile found.
[vost#0:0/h264_vaapi @ 0x5dd7efe3f280] Error while opening encoder - maybe incorrect parameters such as bit_rate, rate, width or height.
Error while filtering: Function not implemented

Then again, SW decoding H265 → HW encoding H264 is working.

$ ffmpeg -vaapi_device /dev/dri/renderD128 -i 'input.h265.mkv' -vf 'format=nv12,hwupload' -c:a copy -c:s copy -c:v h264_vaapi 'output.h264.mkv'

CPU goes as high as 95% during this transcode.


CPU: Intel(R) Core™ i3-7100 CPU @ 3.90GHz

$ sudo inxi -Ga
  Device-1: Intel HD Graphics 630 vendor: Dell driver: i915 v: kernel
    arch: Gen-9.5 process: Intel 14nm built: 2016-20 ports: active: HDMI-A-1
    empty: DP-1 bus-ID: 00:02.0 chip-ID: 8086:5912 class-ID: 0300
  Display: x11 server: X.Org v: 21.1.11 compositor: Picom v: git-89c2c
    driver: X: loaded: modesetting dri: iris gpu: i915 display-ID: :0 screens: 1
  Screen-1: 0 s-res: 1920x1080 s-dpi: 96 s-size: 508x285mm (20.00x11.22")
    s-diag: 582mm (22.93")
  Monitor-1: HDMI-A-1 mapped: HDMI-1 model: HP 24es serial: 3CM7170X3K
    built: 2017 res: 1920x1080 hz: 60 dpi: 93 gamma: 1.2
    size: 527x296mm (20.75x11.65") diag: 604mm (23.8") ratio: 16:9 modes:
    max: 1920x1080 min: 720x400
  API: EGL v: 1.5 hw: drv: intel iris platforms: device: 0 drv: iris
    device: 1 drv: swrast surfaceless: drv: iris x11: drv: iris
    inactive: gbm,wayland
  API: OpenGL v: 4.6 compat-v: 4.5 vendor: intel mesa v: 24.0.1-arch1.1
    glx-v: 1.4 direct-render: yes renderer: Mesa Intel HD Graphics 630 (KBL GT2)
    device-ID: 8086:5912 memory: 3.65 GiB unified: yes

vaapi info

$ vainfo
Trying display: wayland
Trying display: x11
vainfo: VA-API version: 1.20 (libva 2.20.1)
vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 24.1.3 ()
vainfo: Supported profile and entrypoints
      VAProfileNone                   : VAEntrypointVideoProc
      VAProfileNone                   : VAEntrypointStats
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Simple            : VAEntrypointEncSlice
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointEncSlice
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointEncSlice
      VAProfileH264Main               : VAEntrypointFEI
      VAProfileH264Main               : VAEntrypointEncSliceLP
      VAProfileH264High               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointEncSlice
      VAProfileH264High               : VAEntrypointFEI
      VAProfileH264High               : VAEntrypointEncSliceLP
      VAProfileVC1Simple              : VAEntrypointVLD
      VAProfileVC1Main                : VAEntrypointVLD
      VAProfileVC1Advanced            : VAEntrypointVLD
      VAProfileJPEGBaseline           : VAEntrypointVLD
      VAProfileJPEGBaseline           : VAEntrypointEncPicture
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
      VAProfileH264ConstrainedBaseline: VAEntrypointFEI
      VAProfileH264ConstrainedBaseline: VAEntrypointEncSliceLP
      VAProfileVP8Version0_3          : VAEntrypointVLD
      VAProfileVP8Version0_3          : VAEntrypointEncSlice
      VAProfileHEVCMain               : VAEntrypointVLD
      VAProfileHEVCMain               : VAEntrypointEncSlice
      VAProfileHEVCMain               : VAEntrypointFEI
      VAProfileHEVCMain10             : VAEntrypointVLD
      VAProfileHEVCMain10             : VAEntrypointEncSlice
      VAProfileVP9Profile0            : VAEntrypointVLD
      VAProfileVP9Profile2            : VAEntrypointVLD

ffprobe of video I am trying to convert:

Stream #0:0[0x1](eng): Video: hevc (Main 10) (hev1 / 0x31766568), yuv420p10le(tv, progressive), 1920x804 [SAR 1:1 DAR 160:67], 2425 kb/s, 24 fps, 24 tbr, 12288 tbn (default)

Looks like the video is “HEVCMain10” but my hardware does seem to support encoding and decoding it, “VAProfileHEVCMain10 : VAEntrypointVLD, VAProfileHEVCMain10 : EntrypointEncSlice”.

So what gives?


Looks like I can HW transcode H265 (main) videos alright but not H265 (main 10).

Is the h.265 video played back with vaapi hardware video acceleration? The GPU should have h.265 main 10 decode support (and the entry point is shown in the vainfo output), but only for 4:2:0 videos, not 4:2:2 or 4:4:4.

If you do a ffprobe input.h265.mkv it should show a Stream ... Video: hevc ... yuv420p10le .... If it is a yuv422p10le or yuv444p10le that’s probably the culprit.

You can pass -loglevel info to ffmpeg for a maybe more helpful verbose output.

It’s yuv420p10le.

Well, that was my best guess. Can you post a short example of a clip failing the encoding, I have hardware from the same generation and could give it a try.

1 Like

Video Info

$ ffprobe 'P1000070.MP4'

Stream #0:0[0x1](und): Video: hevc (Main 10) (hvc1 / 0x31637668), yuv420p10le(tv, bt2020nc/bt2020/arib-std-b67), 3840x2160 [SAR 1:1 DAR 16:9], 13238 kb/s, 25 fps, 25 tbr, 90k tbn (default)

H265 → H264 Converstion with ffmpeg and vaapi Result

ffmpeg -loglevel info -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i 'P1000070.MP4' -map 0 -c:a copy -c:s copy -c:v h264_vaapi 'file:///home/moss/Videos/P1000070.h264.mp4'

[h264_vaapi @ 0x5f687b438fc0] No usable encoding profile found.
[vost#0:0/h264_vaapi @ 0x5f687b4e91c0] Error while opening encoder - maybe incorrect parameters such as bit_rate, rate, width or height.
Error while filtering: Function not implemented
[out#0/mp4 @ 0x5f687b3e0ac0] Nothing was written into output file, because at least one of its streams received no packets.

The Video File Download

Please give it a try and let me know.

On a quick look something is fishy. Even on latest Intel Arc hardware which should support everything.

Can you test -vf 'scale_vaapi=w=-1:h=-1:format=nv12' and see if that works for you?

ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i P1000070.MP4 -vf 'scale_vaapi=w=-1:h=-1:format=nv12' -c:a copy -c:s copy -c:v h264_vaapi 'output.h264.mkv'

From what I understand there’s no 10 bit support for vaapi h.264, that’s why the encoder fails. The SW h.265 → HW h.264 does the conversion to 8 bit implicitly, that’s why it works. The HW h.265 → HW h.264 looks for a 10 bit encoder and fails. So you have to manually convert to 8 bit (nv12). But that transformation doesn’t seem to work on vaapi if you don’t tell the transformation pipeline we are using vaapi, therefore the “fake” call to scale_vaapi is necessary.

That’s the limit of my paygrade, it may be worth to ask for a more authoritative answer in the ffmpeg forum.

1 Like

Excellent, this is working with no issue! Thank you.

Scaling but not really scaling to trigger 12bit → 8bit conversion definitely seems hacky so this is most definitely an issue with ffmpeg. I am not sure if I can navigate the ffmpeg forums at this time so I leave raising a bug report to someone more experienced than me.

1 Like

Glad that it worked. On an unrelated note regarding your avatar … BSG was so goood - at least in the first half, esp. the first season. My formative TV years. :wink:

It seems it completely fell of the the pop-culture radar in recent years. Together with Firefly, Lost, … We had such good shows at the time, even if they broke our hearts in the end. :sob:

1 Like

Yeah, BSG is some seriously high quality sci-fi TV. I prefer it to other popular space operas like Star Trek and Star Wars which are actually quite popular. I guess it is way too heavy for the general audience.

And yes Lost is an experience, never felt like that with any other TV show! And Firefly :broken_heart:.

BTW did you watch the original (1978) or (and) the re-imagined (2004) series? I only watched the latter.

Sam Esmail of the Mr Robot fame is supposed to work on a new BSG TV series for NBC’s Peacock streaming services. But it seems to be in development hell but he recently commented on it:

The brains behind ‘Mr. Robot’ and ‘Leave the World Behind’ on rebooting ‘Battlestar Galactica’: November 30, 2023

Q: And where are you with your Battlestar Galactica adaptation?
Sam Esmail: We have a great outline and we’re probably going to go to pilot soon.

I mean, when we started working on it, I obviously was aware of AI, but now, four or five years later, it’s in the public consciousness and now that’s so influential in how we’re going to tell the story. The allegory piece is something that is crystallized in a different way, too. The focus is the same, which is the fear of tech and how it might take over, but this idea of just “the robots are going to be our overlords” is a very facile and overly simplistic way of looking at it. Now that the audience is more sophisticated about the consequences, I think we have to match that with Battlestar.

Hopefully it happens.

A little something about my avatar: I actually used to run Manjaro for a long time (eclipsed by EOS now) and my account used to have a BSG avatar on those forums. I hate that I forgot his name (Manjaro forums got nuked so can’t just login and see) but he used to post cool Manjaro themed wallpapers and stuff. One day he created and posted avatars he made for some forum members. One of them is mine. You can see my username from that forum in the avatar!

I only really watched the new series. The original was often on TV when I was young, so I’m somewhat familiar with it. It’s impossible for me to judge it fairly, but as a kid I loved it. It had space battles you could replay with your own toys. That was enough to make it awesome. :smiley:

1 Like

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.