Hi Stevie,
Just to let you know, we’ve found that in recent OS versions the CPU on the Pi isn’t automatically set for the most performance. In essence the CPU can be set into various modes, balancing performance and power consumption, much like a laptop or phone does. This project was born out of a need at the beginning of the Covid-19 pandemic to try and provide a video call solution, on low-cost hardware, to try and help isolated individuals. It was always going to be stretch to get it to run (at all) on a Raspberry Pi, even a 4b, since encoding your camera feed and decoding the other call participant’s feed - that’s hard work! We tested and tested the project, and really deliberated not putting it on the blog at all - but ultimately it did function and we felt could provide benefit to those people with no other method of seeing loved ones.
The other angle to consider here, is that there are quite a lot of moving parts, some of which we have no control over. Should Jitsi make changes to their jitsi.meet platform that stop this project performing at all - there is nothing we will be able to do.
For anyone else with this issue: we have found that the CPU scaling governor now needs to be set in projects that need all the performance the Pi can give. The CPU will still run at ~100%, because the video encoding/decoding is always going to be hard for it.
To set the governor you can run the following:
echo performance > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
This will run the CPU in performance mode: https://wiki.archlinux.org/index.php/CPU_frequency_scaling
Lastly, another thing to remember is that the Pi will throttle performance if the CPU (and GPU…etc) reaches a certain temperature, to save the hardware. From memory this is ~85 Celsius. Therefore if your Pi is in a closed case (such as the official Pi case) it will run hotter, and throttle. Adding a heatsink may help, or putting the Pi into a specialised cooling case will help considerably.
For brave souls, there is also lots of information online about overclocking your Pi4. There is obvious risk here - so proceed with caution. 
Phil