What are we working on? The Balena Notes

So I’ve mostly been working on support and code review lately. In the time that’s left after that I’ve been learning more typescript as part of looking into using it for pinejs and our api, and so far I’ve converted one module as part of the effort. I’ve also been looking at adapting some changes to our permissions system to work with a new feature others have been developing, which may end up with adding functionality to our api for expressing the relationships we want to follow between resources.

1 Like

And I nominate @andrei to go next

Given that I have written in this thread just a couple of days ago, I will proxy nominate @bryan to be next.

1 Like

Sorry about that, I must have missed you somehow when scanning through people who’d already posted

Hey everyone! As Christmas is coming and the year winds down, a big part of our team are taking their well deserved holidays over the holidays! I have started this thread, but haven’t checked in before, so taking this opportunity after @bryan :pen_fountain:

A lot of my day-to-day job involves taking and then writing up internal projects, and getting them out to a wider audience. Tech is pretty complex, and the more interesting examples we can provide for resin.io, the better off everyone is! For example, today I was writing up one of the projects from this forum to publish on Hackster: Voice Controlled LED Matrix Christmas Tree There are a few more projects that I’d like to document in the resn.io Hackster group.

Also, I’m making a draft specs for two resin.io kits:

The first is demo kit (hardware components, software to deploy, slides to go with), which can be easily taken to a meetup (either by us, or anyone who wants to impress others with resin.io:wink: :trophy: Will probably be based around a Raspberry Pi 3 + PiTFT.

The second is a whorkshop kit (hardware, software setup, example projects to work on), which can be used to get a resin.io workshop started in a very short time. Looks like recent experiments with cloud IDEs should help, and probably will be based on RPi3 + Blinkt :keyboard:

And there’s a lot of ongoing effort for finding interesting meetups to join, improve on these forums, and generally hack on projects (not just on Fridays, like today)

(On the personal side, I’m trying to learn Shogi or Japanese chess, any players around? :slight_smile:

Cheers, and :christmas_tree: Merry Christmas everyone! (will tag the next person in the new year:)

Back to our “regular program” in this prime year of 2017 :calendar_spiral: , pinging @alison for the new year’s first update and passing the baton!

Thanks @imrehg! And happy new year, everybody! :fireworks:

I’m starting off the year by sketching out how we plan to raise awareness, increase adoption, and create understanding of resin.io in 2017. This includes events, activities, content, communications, marketing assets, and so forth - all of the different ways we communicate with the world.

I’m also working with the team to update our pricing model to better align with the value we provide to our users. More to come on this soon.

On the product side, I’m strategizing about projects and programs we plan to release more broadly this year, including the “resin.io ready” program for device manufacturers, the AutoHAT project for automated hardware testing, the third iteration of the Beast, and a couple new ones that the team is working on. Again, more to come!

I’m really excited for the year ahead. We have a lot of exciting things planned that I can’t wait to share.

And with that teaser, I’ll pass it on to @sonya!

1 Like

Thanks @alisondavis17, I’ll gladly share!

Today I’m focusing on our customer support ticket process.

In the “front of the house”, I’m looking at how to triage incoming questions more efficiently. That’s a lovely bit of optimization, because it positively affects users who are :disappointed_relieved:, and for the developer who is looking for which tickets need help first :fire_engine:.

On the backend I’m working with @shaunmulligan to render every useful bit of insight we can from each support ticket. Each support ticket is like a geode of bug recreation, product insight, and user experience research. We crack 'em open and find the gems! :ring: We’re looking at how to efficiently run a ticket through a teardown and polish them into GitHub issues. We have a pretty solid process now, so we’re looking at what we can do to make it faster, better, stronger. :robot:

Tomorrow I’m planning on hacking on a Pi with a moisture sensor for my garden. :seedling:

I’ll pass the baton to the person with the best summit ignite talk, @hedley! :bee: :honey_pot: No pressure to write your post in couplets, that would be kind of tough-lets. :confounded: :no_good:

1 Like

Hey @imrehg, one of the most useful things (for me, anyway) about Resin is that I don’t have to pull SD cards to update devices. That may not sound like a huge deal, but I have huge hands, and I hate fiddling with micro SD media :slight_smile: A demo that would impress me would be adding new functionality to devices I can’t see or touch without having to put hands on hardware.
A demo environment, for your consideration: Have five Pi-Zeros, each with an Enviro pHAT, located in hard-to-reach places, and have them pushing telemetry for humidity to a time-series DB. Maybe Influxdata. With a code push, you can gather and transmit temperature, without laying hands on sensors. Another git push and you’re gathering light levels, all without having to interact with the hardware.

Hey @ashmastaflash that reminds me of the drone demo from last year’s Dockercon, when a thermal camera was enabled as a new piece of hardware on a drone, mid-flight (check the video here).

The Pi Zero + Enviro pHAT sounds good as enhancement demo, will definitely make some trial project around that. Pi Zero is a bit slow for demos sometimes (good enough for actual work, though), which would make me hesitate. Would work well with smaller audience or workshop, probably. A larger “brother” to that might be a Pi 3 + SenseHAT + battery pack combo.

The no-need-to-touch-the-SD-card is a very good angle, though, cheers!

Thank you, @sonya!

Whilst my ignite presentation was rhymed for the most, I think I’ll refrain for the rest of this post. :wink:

I’m currently working on several different bits of Resin, but some highlights from this week:

  • Looking into issues with resinOS discoverability on local networks, which has involved writing some C & BSD socket code for investigation (as an old embedded engineer, I secretly loved this!)
  • Updating our Development Environment to a new Linux kernel and Docker version, to allow engineers to work on some future Resin features
  • Working on internal processes to make the Resin workflow, from initial development through to deployment, as smooth as possible; at the moment I’m writing process bots that take Github webhooks and act on them (currently a versioning bot that modifies semver versions and Changelog files for a compoment based on merged PRs). We’re in the process of moving to TypeScript, so it’s been a great chance to use this to develop the process bots
  • Hoping if I get time today, to spend a couple of hours on my own hacks to get an RPi3 up and running as a serial programmer, via Resin, for some LPC microcontroller tinkering

And as the internal processes has me working very closely with him, I nominate @kostas for the next update! :smiley:

1 Like

Thank you @hedley !

What I’m currently working on:

  • Keeping track of roadmap items, prioritised projects and specification documents, which we often write to plan new features or address non-trivial issues. By the way, our plan is to give more information about our roadmap and ongoing specification document work to the community, which should happen in the following months.
  • Work on making our development and deployment processes more efficient. Currently this means moving forward with adopting conventions to automate changelog generation and versioning, with the ultimate goal being automated deployments that can account for inter-component dependencies.
  • Handle the logistics of other internal processes like our architecture calls
  • During this week I also plan to work on fixing some issues on rdt (https://github.com/resin-os/resin-device-toolbox) and resin sync (https://docs.resin.io/raspberrypi3/nodejs/getting-started/#using-resin-sync-to-develop-fast)

And I nominate @pablo !

1 Like

Thanks @kostas!

One of my main tasks is to maintain the Resin Supervisor, our agent on devices. I’ve recently been implementing a few bug fixes and adapting a few features to work correctly on ResinOS 2.0 (e.g. how we allow user apps to modprobe kernel modules).

A big thing I’m working on now is defining a re-architecture of the Supervisor code. As it grew organically when we developed features requested by users, some of the code is organized in ways that can be improved; the idea of the re-architecture is to optimize this so that the next features we have in mind (like multi-container as Shaun mentioned) are easier to implement and can be done in an elegant way. This should also allow us to progressively improve the Supervisor’s performance and resource usage, and we are thinking of a step-by-step transition to Rust. A challenge with this re-architecture is to do it without trying to rewrite the whole code, reusing as much as possible and avoiding a feature-freeze.

Besides this, I’m starting to implement a new version of the update locking mechanism that will solve some issues that are intrinsic to the current design - doing this will also hopefully imply some baby steps towards the new architecture.

I also always spend some time doing user support, or helping teammates do it, especially when there’s bugs or issues that may come from the Supervisor or its interaction with the host OS.

And now I nominate @jviotti for the next update!

1 Like

Thanks for the nomination @pablo!

I’ve been recently working on taking Reconfix to the next level (https://github.com/resin-io/reconfix/pull/6). Reconfix is a bidirectional configuration engine which with the help of some publisher-provided schemas, allows users to read the current device configuration, and re-configure it as needed, whatever “configuration” means for that device (wifi credentials, logging, etc), however I decided to take some time off from simplifying the schema languages to squash some pending accumulated bugs in order to prepare for the next release, as well as refactoring, increasing unit test coverage and other “cleanup” tasks.

Here’s a summary of what I did this week, in more detail:

  • I’ve been assigned on support for two slots per day, which kept me busy for half of my time.
  • I chased down a very subtle bug on etcher-image-write, Etcher’s backend which does the actual low-level writing, where errors coming from the first stream on a stream chain will go unhandled (https://github.com/resin-io-modules/etcher-image-write/pull/75)
  • I took some time to finish a couple of old PRs that re-implement NodeJS file-system readable and writable streams to retry I/O operations up to 10 times when EIO errors are encountered. Because of SDCard self-balancing mechanism, retrying failed operations (EIO) usually makes it work (https://github.com/resin-io-modules/etcher-image-write/pull/73 and https://github.com/resin-io-modules/etcher-image-write/pull/70)
  • I did some experimentation on adding node-gyp support to Etcher, so we can write certain critical pieces of functionality in C++ on the same repo, and have it compiled and loaded correctly
  • I debugged and fixed and bug in Etcher where the writer child process will send multiple state messages to the GUI in a short amount of time, causing the GUI IPC server to throttle them and attempt to parse them together, confusing the parser and eventually complaining with a JavaScript error (see https://github.com/resin-io/etcher/pull/997 and https://github.com/resin-io/etcher/pull/1008)
  • The Etcher GUI calls the Etcher CLI (which we’re working to publish as an standalone package) to actually perform the writing. We support a CLI option that we call “robot”, which causes the CLI to output state information in a way that can be easily parsed back by a machine (in this case, the GUI). Turns out that if an error happens during the initial option parsing stage of the CLI, the “robot” option argument will not be available yet, causing output to be printed in a normal CLI fashion, causing confusion for the GUI parser (which expects a certain structure out of the messages). In order to solve this problem, we allow the “robot” option to be specified as an environment variable, which we can access independently of the option parsing routines (see https://github.com/resin-io/etcher/pull/1010)
  • I chased down and fixed a weird configuration edge case where Windows will report multiple partitions of a single drive to belong to the same drive letter, confusing Etcher during the drive detection stage (https://github.com/resin-io-modules/drivelist/pull/130)
  • I carefully cleaned up some unneeded development dependencies from the npm-shrinkwrap.json file, who seems to have been added there accidentally long ago (https://github.com/resin-io/etcher/pull/1017)
  • I’m moving the GNU/Linux Etcher build system to run inside Docker containers in Travis CI, so we can more easily handle cross-architecture builds
  • I finished and merged a PR that provides support for Etcher unsigned snapshot builds. These builds will contain the short version of the git commit hash the refer to appended in the version (e.g: v1.0.0-beta.17+xxxxxxx) and will be published to some “development channel” very soon for testing purposes.
  • @kostas and I jumped into a call to so some pair debugging of a weird EBUSY error when configuring a resinOS image in OS X using the resin-device-toolbox CLI https://github.com/resin-os/resin-device-toolbox/issues/32, which we’re now working to fix

I think that’s all from me, and I’ll nominate @tim, who I know has been doing some interesting work on the Resin SDK.

1 Like

Thanks @jviotti!

I’ve been very busy too - right now we’re transforming the Resin SDK so you can use it in the browser :desktop:, not just in Node :pager:.

That means digging through all the modules we use in the SDK, and refactoring away everything that won’t work without access to the filesystem, for example. We’ve actually put out a first couple of beta releases of this up this week :champagne: so you can give it a go yourself right now if you like: npm install resin-sdk@6.0.0-beta2

The current step from here is updating everything that depends on the SDK to use this new version, which has had some necessary breaking API changes. That means larger projects like the resin cli, but also all sorts of smaller component libraries we use. The big exciting step that’s coming shortly though is to transform the Resin dashboard code to use the SDK too, rather than its own separately implemented Resin API interactions. This is going to nicely simplify the dashboard code, and should set us up well for some exciting new things we’ve got in the works over there.

And of course, all our work on this is open-source and discussed in the open on our Github repos, so you can dig and follow exactly what we’re doing and discussing day to day on Github for yourself if you like. Here’s a few recent more specific examples to check out:

Phew! I’ll nominate @Eugene.

1 Like

Thanks @tim!

Soo, actually Tim has stolen a good half of my occupation during the past week.
Migrating the Resin SDK from Node-only project to the universal codebase is a project I’ve started back in September.
Then different assignments, the Resin summit, and holidays have delayed it. Since Tim has joined the team things started moving faster, yet we’ve spent almost a month finalizing the code, stabilizing the tests, and simplifying and unifying the testing process across multiple modules.

I’ve been also currently working on improving our UX analytics. We collect some metrics to calculate how many users reach from registering to creating an application and from creating an application to deploying a real device. That helps us identify which parts of the UI may be slowing the users down. Recently we’ve also decided to start sending these events to Google Analytics to simplify some of our metrics calculations. So I’m working on refactoring some of our analytics-related code (which is also universal, running in Node.js and the browser), and experimenting with lerna a bit.

Finally, this week I’m doing 4 hours / day of support, following our support-driven development paradigm.

Aaaand I nominate @horia!

1 Like

Thank @Eugene.

For the last months I have been working on AutoHaT project, a project that aims testing the OS images on Single Board Computers. The test automation framework that we are using to write the test cases is Robot Framework.
The AutoHaT team and I have been writing test cases and keywords that are being tested in QEMU at first. We do have a first version of a rig that works with Raspberry Pi 3 but I am more excited to see the Mega Rig that will be able to test all of our supported boards.
Right now I am working on adding keywords and test cases that will make other’s life much easier to test and automate everything.

Our work is open-source and can be verified in Github and if you are interested, you can check our progress. Here are some examples:

For the next candidate, I nominate @florin

1 Like

For a while now I am finalizing the migration to Poky morty. This will start to be rolled out on boards in the following days, most likely on the Raspberry PI family. This means our host OS will have the new codebase from poky morty as the foundation, taking advantage of bugfixes / enhancements / security fixes in the newer packages versions as well as kernel upgrades for a few of the boards we support.

Aside from this, been working on getting a new board in the resin.io dashboard (to arrive in the following days, based on resinOS 2.X beta).

Also stay tuned for a new 1.26 release that will gradually take place end of this week / beginning of next week.

That’s pretty much it in a nutshell for me and I pass it on to @nghiant2710

1 Like

Thanks for the nomination @florin.

So let’s see what I’m having on my plate.

  • As usual, I’ve been working on a couple of updates for the Resin base images, we have some bug fixes, new python version and Alpine Linux 3.5 coming. Also, there are few issues need to be addressed: weird behavior of the base images on pure resinOS and rdt, new value for INITSYSTEM variable…

  • Besides the base images updates, I’m finalizing some new features for base images build script. The build script will now automatically store base images info (size, available tags…) on a Github repository, it’s really helpful for our users since they don’t need to find these info on the dockerhub anymore. Another coming feature is the automation test for base images which is the missing puzzle in our build process.

  • Aside from those, I’m playing with minideb (https://github.com/bitnami/minideb) a really cool base image introduced by my colleague @shaunmulligan. It’s a small base image designed for containers and hope that we can learn some awesome things to improve our base images from it.

And now, I nominate @ilias

1 Like

Thanks for the nomination @nghiant2710

So, I work as a data engineer developing methods for data analytics. We work with various data sources and at this time we are focusing on data integration and analysis. In particular, we are very interested in assessing data consistency between our data sources and our data warehouse. We are also testing some new services regarding data integration.

In general, we gather all the tools and scripts we write in a project that also provides visualizations, using some pretty cool Javascript libraries for charting. We have some methods for cohort analysis, regression and some basic prediction models.

Finally, I must say that, at this time, we are very happy because we recently welcomed a new data engineer in the team, so we are in the middle of the on-boarding process and we are trying to make him familiar with the the data and tools we use!

That’s the current status more or less and I pass it to @jack who helped me a lot recently with the setup of a new data warehouse.

1 Like