What are we working on? The Balena Notes

Product support
1

As there’s always so much going on inside resin.io, I was thinking it would be cool to give a glimpse behind the scenes of what’s going on. There are a lot of moving parts, most of us are working remotely, and would definitely be good to see the variety of things people work on day to day to make resin.io tick. :smiley:

The idea is that each day someone from the resin.io team posts an update in this thread, telling the wider community what are they working on at the moment. Then they’d also nominate the next person to give an update the following day. (Let’s see how it will work out ;))

For the first update :cake:, I nominate @craig, one of our frontend foremans and demo dynamite :sparkles:

1 Like
2

@imrehg cool idea! I also appreciate the emoji garnishing :slight_smile:

Well, Fridays are hack days at resin, which is when we can take some time to experiment with new ideas.

One of my main tasks at resin is keeping our websites up to date. This was easy when it was just resin.io, but we now have etcher.io, resinos and a few more in the pipeline. They generally have a lot of common elements but a fairly manual process to update one or spin up a new site.

So the plan today is to build, an mvp of a custom static site generator.
It’ll be a CLI tool that consumes a conf file (something.conf.js) and spits out an index.html + perhaps allow you to push gh-pages too.

I’ll try drop later today to show any progress I’ve made. :fireworks:

1 Like
3

Anyone interested here’s where I got to today: https://github.com/resin-io-playground/lander

Right now it’s just a thin wrapper around webpack. It has a handlebar loader that allows you to build a page by requiring templates and exporting them as an array. (see below)

const jumbotron = require('lander/jumbotron.handlebars')
const navbar = require('lander/jumbotron.handlebars')
const custom = require('landerCustom/custom.handlebars') // resolves to `$cwd/.lander/templates`

module.exports = [
 navbar({title: 'hey!'}),
 jumbotron(data)
]

Ofc it also bundles your scripts and sass.

Next is to give the CLI a deploy option and add more templates.

1 Like
4

:rocket: cheers @craig! As I reacall there are a handful of internal projects that should be opened up by their creators, this should make the debut nicer :smiley: (And the THOUGHTS.md is interesting to record some work notes, I guess?)

:blush: BTW, who you’d nominate for the Monday note :calendar_spiral: ?

5

Cool, hmm… I’ve seen a lot of PRs from our new backend engineer @cameron. What’s going on with the builder? :construction_worker:

1 Like
6

Thanks for the nomination @craig , you’re right it was quite a hectic week with the builder!

As some of you may have noticed, our builders were brought down, which turned out to be due to a perfectly acceptable dockerfile. This later turned out to be a bug in our configuration of DIND (docker in docker). This was identified and fixed pretty quickly, but I’ve been working to ensure users don’t experience anymore downtime.

The first steps in this were to update qemu and docker, to pick up a few bugfixes that the docker and qemu community have released. We implemented more stringent checks in our load balancer to catch unruly builder behaviour and fix it quickly and automatically.

In addition to this I’ve been working on giving the builder some :heart:, squashing some bugs such as null containers, and adding some exciting features such as container size reporting, dockerfile linting, and some other speed improvements which I’ll speak more about in the coming weeks.

I know that our devices team :pager: is always working on exciting new projects, so I’ll nominate our device team engineer @telphan to keep us updated on that.

1 Like
7

Thank you for the nomination @cameron!

Ever since the appearance of ResinOS 2.0, our team had been mainly fixing bugs for its release. We also had a few features to polish, e.g. persistent logging, which requires reboot as per the current implementation.
We also continuously try to make ResinHUP (host OS updater) as reliable as possible, so we can start updating device in the wild.

I am also implementing new features, e.g. running resinOS from the SD card or allowing the user to place the data partition on an external storage drive (e.g. SD card) on devices that normally boot from internal storage.

The splash system is getting a revamp soon. I plan on supporting custom user backgrounds and even support GIFs for the foreground.

Nominating @curcuz to share with us how the gaming console is panning out.

1 Like
8

Thanks, cool :scream_cat:! Can’t wait for animated splash screens!

Maybe will tune in down a bit. :wink:

Let’s see how @curcuz is doing, he’s just on the way to Helsinki to Slush, let’s see how’ll be his day today.

9

:alarm_clock: Pinging @curcuz, who seem to be having a great time in Helsinki, btw :circus_tent::snowflake: ^___^

10

Hey guys! sorry for the late answer, it’s been a very, very intense week!

we showcased a cool demo on Artik modules running on resin and announced the first 2 upcoming “resin.io ready” certified devices!

now on my way to London, so we could have a joint hack friday @imrehg on headless-etcher!

1 Like
11

Thanks @curcuz! Hope you are doing okay, heard you have an layover in Iceland! :airplane_arriving:

In the meantime, who do you nominate as next to check in on Monday?

12

I nominate @taahirisaacs - curious about eventual new swag designs :wink:

2 Likes
13

Thanks for the nomination @curcuz :grin:

I’ve currently been working on a few tasks:

Task 1: Etcher Updates :raised_hands:

Since the arrival of our awesome new UX Designer, @konmouz, we’ve been closing design tickets as well as planning ahead for the future of Etcher. I’m currently working on variations for a better way for users to visualise the flashing process as well as implementing instructions for the post-flash process.

Task 2: Swag :santa:

I’m working with the team to get Etcher and resinOS tees out into the swag world. Some early iterations below :muscle:

I nominate @dfunckt – Go! Go! Go! :robot: :spy:

2 Likes
14

Great work @taahirisaacs, both logos look amazing!

2 Likes
15

Thanks for the nomination @taahirisaacs :dizzy:

So, there’s a bunch of tasks at various stages of completion I’m currently jumping between, spanning from the front-end to the backend. Here’s what’s cooking:

1 - Fixing the pull and push progress bars that appear when you git push resin master to be super-smooth once again. These broke when we upgraded our builder to a more recent Docker version.
2 - Merging our v1 and v2 Docker image registries to run behind a single proxy. When deployed, this will cause most devices to automatically pull from v2 which is faster and more reliable.
3 - Implementing self-serve billing for our users – i.e. choosing plans, making payments, etc. This is a larger project, both in number of implementation stages and moving parts and is destined to take a while.
4 - Reading up and playing with React because, errr… :zipper_mouth: :wink:

I’d like to nominate @andrei next, our ResinOS lord – go Andrei! :confetti_ball:

1 Like
16

What is @andrei doing… I do know this guy so I’ll tell you:

  1. He was very much into resinhup lately. This is our in-house baked solution for hostOS updates. He thinks that it actually works and upgraded the entire fleet of a client under very restrictive network constrains. Uh! I forgot to say he didn’t brick any of the devices. Can’t remember the number of them though.
  2. He, as well, was playing with watchdog in our OS trying to get it robust and working.
  3. Don’t tell him about releases. Cause he had to finalise some of those too. But hey! That is how we roll out our goodies so somebody needs to do it.
  4. Everybody says he is a pretty talkative person too. We put that at work and assigned him for some interviews.
  5. Last topping: devices team misc.

He (andrei) didn’t want to nominate anyone but I came up with a good suggestion: @petrosagg . He is the real master around.

PS 6. I stumbled upon him in the tube. He started to listed to audio books. Who the hell would listed to “A Short History of England”?! Stepped out and took the next train.

1 Like
17

Let’s see… I’m working on a couple of tricky bugs that happen on some devices.

I’m trying to figure out how to tune the virtual memory subsystem of the Beaglebone Black so that we don’t get DMA allocation failures. On top of that, I’m patching docker so that layer extraction disk syncs happen at the end of each file instead each layer to reduce the amount of dirty pages in the cache. I’m also integrating posix_fadvise() in docker to hint the kernel about page cache usage. You can see the WIP work on docker here https://github.com/resin-io/docker/pull/10

I’m also working on the Intel Quark which has a hardware CPU bug that is fixed by a special GCC flag that omits the LOCK prefix in instructions. Since this is a GCC flag it means that golang programs (like docker) don’t get the fix and so I’m looking into the golang compiler to make it emit the correct assembly for this board.

Apart from the above, I also spend my time thinking about how to improve the logging solution on the devices using journald, experimenting with Rust and NEON, study docker internals to improve IO utilisation, and interviewing some candidates.

For the next resiner, I nominate… @lifeeth

1 Like
18

What am I doing?

For the last few months I have been working with our team on building and setting up an automated hardware testing rig for all our boards. You can checkout the progress for this at https://github.com/resin-io/autohat-rig and https://github.com/resin-io/autohat

Last week we received a new iteration of the test rig - I was making sure there were no bugs in the hardware, so far the hardware seems rock solid :slight_smile:. This is how a desktop setup currently looks like:

Today I am going to be spending time writing up a specification for housing the rigs for all the boards supported by this version of the test rig at our London office. This is going to involve the following :

  • Using closed network racks with custom aluminium layers with mounting holes to safeguard the boards from @andrei :wink:

  • Revamping the office network with a new router (Thinking Ubiquity Edge Router X) - so that @shaunmulligan’s cat video downloads wont interfere with the bandwidth needed for tests.

  • Writing up a minimal/MVP API spec for integrating the rig into our CI/CD pipeline. I am thinking of a Resin powered NUC connected to multiple AutoHAT boards with a web URL end point that accepts a Robot test suite and a Yocto image to be tested.

  • Taking help from our wonderful operations team to place an order for more AutoHAT boards and components needed for our rig.

Next I nominate @shaunmulligan

1 Like
19

Very cool, @lifeeth, can’t wait to play with AutoHAT and automate all the things, for example testing the repos in github/resin-io-projects :slight_smile: I think there was some talk about putting a Faraday cage around each device, to cut down on interference, I wonder at what stage it will be totally necessary - and also badass looking.

1 Like
20

Thanks @lifeeth for the update on AutoHat, its gonna be amazing when we have all our supported boards continuously testing themselves :smile: , its definitely gonna improve the support teams life a bunch by catching bugs and regressions :thumbsup:.

For myself this week, I am mostly working on laying out some product features for 2017. In particular I am trying to scope out all the work done and still needed for the version 1 of our official multi-container feature. This feature is perhaps one of the most requested features we have ever had and I am really excited to get it built as soon as possible because it really allows for some cool usecases. Its also a really interesting feature to work on and spec out because it seeps into almost every part of our product from the Dashboard to the OS to the builders and API, so its really a full team effort. In its first form, the resin.io multicontainer feature should allow you to push a simple git repo with a docker-compose.yml file at its root and it should all get seamlessly built and deployed across your entire fleet, working in symphony with all the usual features you already know and love in resin.io, like delta image pulls, webterminal, update strategies, etc.

Another feature I’m working on defining this week is the notion of “Organisations” or team accounts, which I know is a feature that has a massive impact on our customers and their development teams, so look forward to that landing in 2017 as well.

Besides the formal product spec’ing stuff, I’m also hoping to get some time to write a few bits of documentation on the resin.io container lifecycle and clear up some confusion in the docs on Environment variables and Configuration variables.

Thats pretty much all on my end for this week, so I nominate @Page because I’m curious on how the mysterious PineJS and API stuff if getting on?

4 Likes