Chromium Balena Browser React App Audio Playback on Videos DAC issues

Project help
, ,
1

I’m having a tough time playing audio from a React App in chromium, through a DAC (based on the discontinued PhatDAC from PiMoroni) on a custom Hat attached to a Raspberry Pi 4 with Define DT parameters:i2c_arm=on", "spi=on", "audio=off and Enable DT Params:hifiberry-dac. I am building my image based on the BalenaBlocks browser. But I included the pimoroni setup script found through their setup page. The script isn’t intended to work with Pi4 arch, but I hacked line 197, to be IS_ARMHF=true, and that seems to let the setup script do its thing. Guessing this is because of the arch differences between Pi3 (what we originally designed the hat for) and Pi4 (which we upgraded our next fleet to). But, the audio still will be enabled and play through the attached speak, except…

When I’m running the React App inside of a Balena device, the speaker is popping at varying intervals (I observed a correlation to when the React App is loading pages or doing something in the background) and the Audio will cut in an out. But most frustrating of all is video content loaded inside of the react app wont play any audio unless it sits for awhile, (guessing the entire file finished loading). This files are mp4, streaming from an on prem asset server.

The strange thing is when I point the chromium browser directly to the file on our asset server, the audio will play along with the video content as soon as its available enough to stream. But when the React App loads the same video, it will not play. Also, in all cases there is regular popping of the speaker.

When I load this app into chromium on a Pi4 with a normal raspbian OS (not balena’d), the video inside the react app plays normally, and no popping of the speaker.

After much searching around on forums, I came across one board saying that i could use BalenaSound and send the audio from the container running chromium into BalenaSound containers, with the AUDIO_OUTPUT:DAC set on the Device environment variables. and setting up the Audio Block, as called for in this The sound is perfect, without any popping, and the video in the React App plays audio, except its about 2 seconds delayed. Saw a forum (this one) where another person had this issue. Seems to have gone unresolved.

I’ve also tried the more simple approach of just using the BalenaBlocks browser, with turning on the AudioBlock, and setting AUDIO_OUTPUT:DAC and Define DT Overlays:hifiberry-dac. But also get regular popping of the speaker through the react app, and the audio from videos inside the react app never plays (vs will play after a long wait), but it will play when pointed directly to the file on the asset server.

Other small sound effects will play, only the audio of a video in the React App has consistent issues.

Thoughts?

2

I realized that the pimoroni setup script is probably doing the same things that the BalenaSound settings are doing. So I built a new container without that script, and was able to get sound, but the same issues persist, proving that the pimoroni script was indeed vestigial, but also has no noticeable changes to audio behavior.

3

Following this recent thread, and think I should attempt the same thing suggested in that reply.

4

Some additional information.

The ENABLE_GPU flag doesn’t seem to actually enable chromium GPU features.

When I check chrome://gpu I get chrome gpu webgl hardware ***accelerated*** but at reduced performance along with a bunch of other issues.
We just ran a profiler comparison between chromium in raspbian, and chromium in balena browser, and see a significant difference in its performance, and its showing the rendering is happening on the CPU in balena, vs on the GPU on raspbian.

Also, the following flags are set in my FLAGS variable

--disable-features=TranslateUI --disable-component-extensions-with-background-pages --disable-background-networking --disable-sync --metrics-recording-only --disable-default-apps --no-default-browser-check --no-first-run --disable-backgrounding-occluded-windows --disable-renderer-backgrounding --disable-background-timer-throttling --force-fieldtrials=*BackgroundTracing/default/ --window-size=1024 600 --window-position=0 0 --autoplay-policy=no-user-gesture-required --noerrdialogs --disable-session-crashed-bubble --check-for-update-interval=31536000 --disable-dev-shm-usage --enable-zero-copy --num-raster-threads=4 --ignore-gpu-blacklist --enable-gpu-rasterization --enable-native-gpu-memory-buffers --disable-gpu-compositing --enable-experimental-canvas-features

10

Can you post the react code for your video? If the video plays normally in its own tab, it’s probably something to do with react.

Try just using `__dangerouslyBoundInnerHTML="" and see if that is the problem in an empty app, and see if it can play. That way you can know if it’s the react process or something else messing up to narrow it down.

12

Hi there! Regarding the audio problems you describe, here a few pointers that might hep you troubleshoot. First, don’t disable audio "audio=off if using balenaSound/audio block. It’s not necessary and it will only cause problems :slight_smile: Next thing I would do is to completely ignore the DAC, what I mean with this is to use the onboard audio card (audio jack) and test your application using that. Make sure that everything runs as intended before adding the DAC into the mix. That way we can tell for sure where the problem lies (application/setup vs DAC/os config).