I setup a test raspberry pi with an application which print the date like this every 5 secs
2023-04-07 07:36:00.965886
Then I tried to read the device logs using the python balena-sdk:
from os import getenv
from balena import Balena
# from pprint import pprint
credentials = {
"username": getenv("BALENA_LOGIN_USER"),
"password": getenv("BALENA_LOGIN_PASSWORD"),
}
class BalenaManager:
def __init__(self, credentials: dict):
self.balena = Balena()
self.balena.settings.set("pine_endpoint", "https://api.balena.apicai.de/v6/")
self.balena.settings.set("api_endpoint", "https://api.balena.apicai.de/")
self.balena.auth.login(**credentials)
# pprint(self.balena.models.device.get_all())
def send_log_to_server(self, uuid: str, msg: str, error: bool):
if not error:
msg_content = msg["message"]
print(f"{uuid}|{msg_content}")
else:
print(f"{uuid}|{msg}")
def setup_subscribe(self, uuid: str):
def msg_callback(msg):
self.send_log_to_server(uuid, msg, error=False)
def error_callback(msg):
self.send_log_to_server(uuid, msg, error=True)
self.balena.logs.subscribe(uuid, msg_callback, error_callback, 1)
if __name__ == "__main__":
if credentials["username"] is None or credentials["password"] is None:
print("you need to specify the balena credentials via env variables")
exit(-1)
b = BalenaManager(credentials)
b.setup_subscribe("device_uuid_here")
This works fine for about 1 minute, but than stops printing the logs:
device_uuid_here|2023-04-07 07:44:41.005696
device_uuid_here|2023-04-07 07:44:46.006184
device_uuid_here|2023-04-07 07:44:51.006586
device_uuid_here|2023-04-07 07:44:56.006985
device_uuid_here|2023-04-07 07:45:01.007352
device_uuid_here|2023-04-07 07:45:06.007717
device_uuid_here|2023-04-07 07:45:11.008169
device_uuid_here|2023-04-07 07:45:16.008526
device_uuid_here|2023-04-07 07:45:21.008990
device_uuid_here|2023-04-07 07:45:26.009453
device_uuid_here|2023-04-07 07:45:31.009821
device_uuid_here|2023-04-07 07:45:36.010198
device_uuid_here|2023-04-07 07:45:41.010658
I could not find out why, so I tried reading the logs directly out of the redis log backend:
https://github.com/balena-io/open-balena-api/blob/master/src/features/device-logs/lib/backends/redis.ts
I implemented increasing the "{device:1}:subscribers"
key so that you can read the logs from the channel "{device:1}:logs"
. Then I used your avro schema:
{
name: 'log',
type: 'record',
fields: [
{ name: 'version', type: 'int', default: 1 },
{ name: 'createdAt', type: 'long' },
{ name: 'timestamp', type: 'long' },
{ name: 'isSystem', type: 'boolean', default: false },
{ name: 'isStdErr', type: 'boolean', default: false },
{ name: 'serviceId', type: ['null', 'int'], default: null },
{ name: 'message', type: 'string' },
],
}
to decode the logs from the channel:
device 1: {'version': 1, 'createdAt': 7037864708678381089, 'timestamp': -7037864708666847010, 'isSystem': False, 'isStdErr': False, 'serviceId': 3, 'message': '2023-04-07 07:54:06.053250\r'}
This is now working, except that the timestamps are wrong and look like the overflowed. Do you have any idea why?
I also think, every long value decoded from avro is wrong. When trying to decode the logs from another device, I also couldn’t read the log message, which in avro uses a long value to encode the string length.
I am not good at js, could it be some custom long type, in the avro library you are using: [Advanced usage · mtth/avsc Wiki · GitHub] ?
Thank you for your help