Fullscreen webkit browser with HW accelerated CSS, WebGL, and HTML5 video on the RaspberryPi 3

Project help
, , , ,
1

Hey guys!

I created a project that allows you to run a fullscreen Webkit browser on your RaspberryPi. It can do GPU accelerated HD video decoding, WebGL and CSS. This is made possible by the amazing work of Web Platform for Embedded

Here are some cool facts about it:

  • The docker image is tiny since it’s built with Yocto. It’s less than 80MB or 35MB compressed!
  • It renders directly to the framebuffer, without running on X11 or Wayland
  • I’ve added support for the PiTFT screen so that you can display things there and use the touchscreen

Here are some cool things you can do with it:

Let me know what you think :slight_smile:

14 Likes
2

Hi Petros,

I tried this yesterday. The outcomes are really impressive.

My company is working on a cloud-based digital signage solution and we need high GPU performances to run HD videos on a web browser.

It was impossible on a RB Pi with Raspbian. It seems to work really good with this project on an horizontal display. But, we tried on a vertical displays and the performances are significantly low regarding to the horizontal mode.

After some search, I found out that the RB Pi needs more GPU in vertical mode. Do you think you still can improve performance on the vertical mode ?

Best,
Benjamin.

3

Hi @bgauthie. What mechanism did you use to rotate the screen? It sounds weird that it would need more GPU memory.

4

@petrosagg Sounds great, but for me ENV WPE_URL="file:///var/lib/public_html/index.html" doesn’t render html, instead it literally shows the index.html's code.

Here are the steps to reproduce:

git clone https://github.com/resin-io-playground/resin-wpe
cd resin-wpe
mkdir public_html
wget https://github.com/h5bp/html5-boilerplate/releases/download/5.3.0/html5-boilerplate_v5.3.0.zip
unzip html5-boilerplate_v5.3.0.zip -d public_html
rm html5-boilerplate_v5.3.0.zip

then add

COPY public_html /var/lib/public_html
ENV WPE_URL="file:///var/lib/public_html/index.html"

to Dockerfile before CMD and push to resin.

IMG_1261
IMG_1261.JPG3264×2448 2.33 MB

5

Hi,

thanks for the awesome work, is there a way to run this on a raspberry pi 3 without the resin.io?

6

@fellowgeek I think it should run fine if you run the docker image on a pi with the appropriate privileges and the correct config.txt. Have you tried and hit some issue?

7

Hi Petros,

I tried this on my RPi3 and it seems awesome!

Now I’d like to display WebGL on an addressable LED array but am a little lost on grabbing video data from the framebuffer.

One way to get the image headlessly is to use Xvfb like texttop does, and I think it could also work using fbdump. So far I’ve tried installing them in the dockerfile and in

/resin-wpe/base-image/meta-resin-wpe/recipes-core/images/resin-wpe-image.bb

Is installing extra software possible on this minimal container?

8

Hey @dsk2070 :slight_smile: WebGL on an addressable LED array sounds awesome, please post pictures when you’re done!

It is possible to install extra software on the wpe container but it’s a bit cumbersome. Basically you have to create a Yocto recipe, drop it in the base-image folder (check this fbcp recipe I made as a guide) and then include it in the list of installed packages.

Alternatively you can try statically compiling your program, committing to git, and COPYing it in your Dockerfile. That’s a lot messier but saves you the lengthy yocto build.

As a side note, it might be worth checking out or even modifying fbcp to output the framebuffer to your LED array. It sounds like it does a very similar function to what you want and you have the yocto recipe ready.

1 Like
9

I am trying to run install python and completely lost with all the Yocto stuff.
Is there a way to make the WPE browser work on something like alpine? Not that hardcore, but small enough.

10

Thanks for your reply a few weeks ago @petrosagg I’ll be sure to share photos of my LED stuff here when it’s done!

With resin-wpe I looked into installing extra software but didn’t get far - Yocto feels almost like a different operating system compared to regular Linux :open_mouth:

@MiluchOK if your question is still open then maybe try this:

https://www.npmjs.com/package/wpe-webgl

I haven’t tested it yet but it looks like a WPE WebGL viewer that could work on a RPi running Debian? Not really sure

11

Hi @petrosagg,

It’s awesome, the performance is really good. I can play HD videos smoothly.
But I face a challenge that I want to capture a screenshot of the display. I tried to convert raw file from /dev/fb0 but only see the black screen. I also search it but seem no luck with WPE run on top.
So how can I take a screenshot of the display screen while WPE is running?

Thanks,

12

@petrosagg I am also looking to run it in rasberry pi 3 without resin. I tried running it, but initially it failed saying missing vchiq. So while starting docker I added the device as follows:

docker run --rm --device=/dev/vchiq --name pi resin. Here resin, is the name of the image build.

The WPELancher seems to run fine, but nothing is showing up in the display. I have set hdmi_group=1 and hdmi_mode=4, gpu_mem_256=128 and gpu_mem_512=196 in the config.txt file.

Can you please tell me what might be that I am doing wrong?

13

Where you able to run without resin?

14

Very nice.

  1. Do you have to rebuild to get patches to webkit? Or is there an easier way to patch?
  2. Can you make this read-only as well? I have a raspbian stretch lite image that I have read-only using overlayfs.

Maybe adding this? https://github.com/cmhe/meta-readonly-rootfs-overlay

16

Hi everyone

I need some help with this implementation… i’m trying to build a car dashboard based on CAN BUS data, with some low level eletronics integration, serial com with an Arduino and so on…

I rly loved the performance of WPE but I’m feeling very restricted with the use of resinOS, I can not get an easy SSH access to the service running on docker and i’m having to adapt to this architecture is delaying the project a lot…

The first version of project was based on an arch linux, all integration with the CAN BUS was done and everything worked, but the fullscreen browser performance was not as smooth as this one with WPE (I tried chromium and gtk + with webkit)

Well, I’m lost and confused as to continue the project, I do not know if it is easier to try to implement WPE in a pure arch linux or to better understand the operation of the resinOS…

I need some help from you guys

Tks a lot

1 Like
17

hey @vincentphan, were you able to capture the screenshot from the buffer while running wpe??
thanks

18

I’ve found a good solution using Electron JS and the following:

// Screenshot thru SSH //

DISPLAY=":0.0" import -window root beta.png

curl --upload-file ./beta.png https://transfer.sh/beta.png

transfer.sh returns a url to download the screen grab

1 Like
19

very nice. can you do inertial/momentum scrolling with one finger (like iOS/Android)

20

Very impressive, I like it a lot. Unfortuantely it has a problem with clip-paths:

28

@srlevitt I guess this is not supported in the wpewebkit version used in balena-wpe. You’ll need to wait until it’s updated (I don’t have any estimate for that).
We have an experimental version that you can test: docker image imrehg/balena-wpe:raspberrypi3-6925588 but it does not update wpewebkit so I don’t think it will make any difference.