We’re currently in the middle of looking at ways to get our logs streamed (via Logstash) into ELK. As far as I can tell from the resin-supervisor code, you’re using pubnub to stream logs 32kb at a time - I’ve managed to glean a lot of useful from this github issue:
A couple of questions:
- Are you currently using any compression on what’s being sent to pubnub? I’ve found some mention of being able to use gzipped streams: https://support.pubnub.com/support/discussions/topics/14000006317 - I’m not sure if this is available in their golang SDK, and I couldn’t see any sign in the supervisor source that the logs are being compressed.
- I’ve seen that there’s an easy way (through environment vars) to disable log streaming completely. Is there any way to use a different log streaming implementation? For example, I’d like to look at using filebeat (within a container on the device) to stream our logs into logstash. I know that this has the side-effect that we wouldn’t get logging in the resin dashboard, however given all the beautiful things we can do with the log data when it hits ELK, this isn’t such a big problem
I know that there are other ways to achieve this, for example, we could use the resin SDK to continuously stream logs from each device wherever we need them to go, I just feel like that adds an unnecessary layer of complexity (i.e. keeping track of which devices need to be streamed from, having a central instance which actually makes the api calls etc), which wouldn’t exist if we had some flexibility about where / when / how our logs are sent from the device itself.
Thanks for any thoughts on the topic!