API Rate limit issue - Is the limit device or organization specific?

Product support
1

We are currently experiencing issues with the API rate limit (BalenaRequestError: Request error: Too Many Requests). This is due to polling the application, device, and authentication APIs every 10 seconds on each device in our fleet.

Here is our current implementation:

const login = async (): Promise<void> => {
    const apiKey = process.env.BALENA_API_KEY
    if (!apiKey) throw new Error('BALENA_API_KEY not set')
    await balena.auth.loginWithToken(apiKey)
}

const getReleaseTrackedToFleet = async (appId: string): Promise<Release> => {
    if (!(await balena.auth.isLoggedIn())) await login()

    const app = await balena.models.application.get(Number(appId), {
        $select: ['should_be_running__release'],
        $expand: { should_be_running__release: { $select: RELEASE_FIELDS } },
    })
    return convertToDto(app.should_be_running__release[0])
}

const getCurrentDeviceRelease = async (deviceUuid: string): Promise<Release> => {
    if (!(await balena.auth.isLoggedIn())) await login()

    const device = await balena.models.device.get(deviceUuid, {
        $select: ['is_running__release'],
        $expand: { should_be_running__release: { $select: RELEASE_FIELDS } },
    })
    return convertToDto(device.should_be_running__release[0])
}

setInterval(async () => {
    await Promise.all([getReleaseTrackedToFleet('appId'), getCurrentDeviceRelease('deviceUuid')])
}, 10000)

We are using this polling mechanism to check if the current device release differs from the fleet release. When a difference is detected, the user can manually choose to update the device to the new fleet release. The more frequently we poll this information, the quicker the user can be notified of a new release. While increasing the polling interval is an option, we are concerned that the issue may persist as more devices are added to the fleet.

2

@PatrickHallek have you considered using one of the methods that fetches all the device in one query, which will mean that it will always be one query, regardless of how many devices in the fleet? Example: Balena Node.js SDK | balena

Application in this case is the Fleet.

Let us know if this helps!

3

We are using this polling mechanism to check if the current device release differs from the fleet release.

The provided code runs on the device with its scoped BALENA_API_KEY. So each device needs to check if its current release matches the fleet release. We can reduce the requests each device does. But I want to know if the request limit is organization scoped or IP specific. If the Balena Cloud limits the requests each organization does, we will face scalability issues in the long run.

4

Right, it wasn’t clear from your original message that you were running this code on every device. If you need to know if the current release matches the fleet release, that information could be obtained directly from the supervisor API, as the device itself already knows if it’s up to date with the fleet or not: Interacting with the balena Supervisor | balena (look for the update_pending part).

The API limitations we have in place are designed to protect the platform, because as you’ve pointed out, it’s super easy to even accidentally scale requests to the point of creating what is essentially a botnet that DoSs the API when you have a fleet of 100,000 devices. In this case the limit can’t be device or IP specific, as it wouldn’t prevent the denial of service in the case that the devices are spread across the globe, as many large fleets are.

That said, obviously the aim of the platform is to enable what you’re doing and enable your use case, so it’s important to understand needs, and build them in to the platform when necessary.

You might also consider upvoting this feature request and sharing your use case (if you haven’t already): Implement webhooks/notifications/a means to be alerted of changes · Balena Feature Requests

Hope this helps!