Choppy graphics/video using balenablocks/xserver

Product support
1

Hello all,

After following this post: looking for a light X11 server for production ready deployment of GUI app.

I’ve been able to get a simple GUI playing a video playing in balena. Issue now however is that the video plays back super choppy. Wondering if anyone has input on how to correct this. For reference, my docker-compose.yml is:

version: "2"
volumes:
  x11:
services:
  xserver:
    image: balenablocks/xserver
    restart: always
    privileged: true
    volumes:
      - 'x11:/tmp/.X11-unix'
    labels:
      io.balena.features.dbus: "1"
  multimedia:
    build: ./multimedia
    privileged: true
    restart: always
    devices:
      - /dev/dri
    group_add:
      - video
    volumes:
      - 'x11:/tmp/.X11-unix'
    labels:
      io.balena.features.dbus: "1"

And dockerfile in media directory is this:

FROM balenalib/%%BALENA_MACHINE_NAME%%-debian:bullseye-run

RUN install_packages \
    x11-apps mesa-utils mesa-utils-extra python3-wxgtk4.0 python3-wxgtk-media4.0 gstreamer1.0-gtk3 gstreamer1.0-libav dbus-x11 at-spi2-core

COPY ./app /app
WORKDIR /app

ENV UDEV=1
CMD ["python3", "multimedia.py"]

The python code that plays the video is a simple wx.media call:

import wx
import wx.media

class TestPanel(wx.Frame):
    def __init__(self):
        wx.Frame.__init__(self, None)
        self.testMedia = wx.media.MediaCtrl(self)
        self.testMedia.Bind(wx.media.EVT_MEDIA_LOADED, self.play)
        self.testMedia.Bind(wx.media.EVT_MEDIA_FINISHED, self.quit)
        self.testMedia.Load('/app/mymedia.mp4')

    def play(self, event):
        self.testMedia.Play()

    def quit(self, event):
        self.Destroy()

if __name__ == '__main__':
    app = wx.App()
    Frame = TestPanel()
    Frame.Show()
    app.MainLoop()

I’ve tested this locally running the video in macos and runs perfectly. Any input is appreciated!

2

Hi there,

What device type are you using for your test? What are the encoding details of the file? What is the resolution of the video?

Although the video runs on your mac, that’s quite a different hardware specification from a lot of the balena device types. Could the issue simply be that the video is too CPU intenstive for the device you’re trying to run it on?

Phil

4

Thanks @phil-d-wilson for looking into this.

I’m on a coral dev board (1gb ram model). I think it’s a codec issue as cli vlc (cvlc) works well, but other players, such as ffmpeg (using ffplay) is still choppy. Guessing I need to install a library for proper mpeg-2 decoding. Is that right?

Also for extra info for anyone wanting to chime in, Coral Dev Board mentions issues when trying to support any video output higher than 1080p, so I’ve set the resolution down to 1080p on the 4K monitor I’m testing on. That still didn’t help the choppy video alone though.

5

Also @phil-d-wilson the video parameters are:

Dimensions: 1920 x 1080 (so 1080p native)
Codecs: AAC, H.264

Does anyone know what package under debian provides these codecs?

Cheers

6

Hey Richard

Looking at the coral dev board(Dev Board datasheet | Coral) it seems that it supports hardware accelerated H264 decoding

4Kp60 VP9 and 4Kp30 AVC/H.264 decoder (requires full system resources)

Not sure which drivers or utilities are required exactly though.

It seems that someone managed to use gstreamer with decodebin(decodebin) for hardware decoding. see H264 hardware decoding · Issue #500 · google-coral/edgetpu · GitHub Perhaps you will be able to modify the pipeline to your use case?

Hope this is helpful

1 Like
8

Thanks a lot for this! I’ve figured a very hacky workaround and I’m now getting back to resolving this issue, in hopes that there would be more clarity in the future, so thanks for sharing!