Not able to run development mode on Jetson Nano

smithandrewc · February 8, 2022, 4:58pm

Hi,

I’ve recently built a stock jetson-nano-emmc image from master of balena-jetson using the command ./balena-yocto-scripts/build/barys -d -k -m jetson-nano-emmc. The build compiles properly and I’m able to deploy to a nano (using jetson-flash). However, when it boots up, the balena dashboard shows it as “Production”. When I run cat /mnt/boot/config.json | jq I don’t see anything about development. When I compare the boot log of an image compiled without the “-d” flag, I do see a more verbose kernel boot log. However, the “-d” image isn’t listed as a Development image in the dashboard and the serial console is not enabled.

Anyone have any suggestions of things to try? I’ve tried forcing the append of the DISTRO_FEATURES “osdev-image” (thinking maybe the OS_DEVELOPMENT flag wasn’t being set) but that made no difference. Something else to try?

Thanks,

Andrew

andrewnhem · February 8, 2022, 5:49pm

Hello, Andrew. Andrew here.

I need to brush up on my CLI skills, but when I want a dev device, I provision it as such via the dashboard.

If I want to hack on a production device, I like to use local mode: Develop locally - Balena Documentation

Have you tried local mode on that Jetson Nano? Let me know how that goes or if I’m off-target on my recommendation here!

smithandrewc · February 8, 2022, 6:22pm

Thanks for the suggestion Andrew but local mode isn’t available for me, the tool tip says that “Local mode is only supported on development OS versions”, which is exactly what I’m trying to build and deploy. I’ve never used local mode before but I don’t think it’ll fit my use case, I want to deploy these images to a fleet but I want some debugging features (like the system console) available.

Thanks for trying,

Andrew

Langhalsdino · February 11, 2022, 2:08pm

Hi @smithandrewc,
i face the same challenge. Did you figure out how to solve it?

smithandrewc · February 11, 2022, 2:22pm

Hi @Langhalsdino, unfortunately no. I still haven’t resolved it. I’ll probably spend some time on it this weekend and see what I can find. I’ll report back here any findings.

Thanks.

dtischler · February 11, 2022, 9:32pm

I built a device from source last evening (an unsupported device type) and I used the -d flag, and once I got the device online (I had to masquerade as a different device with a recognized slug), it did indeed show as Development version of the OS. So I am wondering if something has changed … @alexgg could the recent work on merging of Dev and Prod images have an impact here when running local builds, even though these users are both passing the -d flag to barys?

alexgg · February 15, 2022, 12:08pm

The master branch of the balena-jetson repository is currently using balena-yocto-scripts at v1.17.0. This version correctly sets the OS_DEVELOPMENT variable when the -d flag is passed to barys. I can’t comment on the original problem as there is not enough versioning details in the thread, but it is likely that the balena-yocto-scripts version in used was not updated.

Langhalsdino · February 19, 2022, 2:56pm

Hi Alexgg,

thank you for pointing this out to me. Yes, i was stuck on v1.15.8 from last November. Switching to v1.17.1 is probably helping.

I will flash a new image on monday and see if it helped

Side Node: Your yocto book helped me getting comfortable with yocto project.

alexgg · February 21, 2022, 10:37am

Side Node: Your yocto book helped me getting comfortable with yocto project.

I am glad it is still helpful. Let us know how updating balena-yocto-scripts go.

smithandrewc · February 22, 2022, 2:22pm

Thanks @alexgg. However, I can confirm that I was building with the balena-yocto-scripts at v1.17.0 (commit f33225d87feff1a4e9b95ca7704c9b7ec27d6d61). What versioning information would you like to help debug? I’m going to sync up the repos now, try rebuilding and see if it helps.

Thanks.

alexgg · February 22, 2022, 2:55pm

It could be that you were building with a conf/local.conf that was created by an older version. In that case, barys won’t update it so the OS_DEVELOPMENT variable wouldn’t be set.
Please check whether your conf/local.conf file contains an OS_DEVELOPMENT variable, and if not either just add it or re-create the build directory from scratch.

smithandrewc · February 22, 2022, 2:59pm

Thanks, I confirmed that OS_DEVELOPMENT is there.

# Set this to 1 to disable quiet boot and allow bootloader shell access
OS_DEVELOPMENT = "1"

I’m rebuilding the image right now, it’s almost done. Once finished I will reflash and see if it works. Thanks.

smithandrewc · February 22, 2022, 3:19pm

OK still an issue. I can see a pretty verbose kernel boot log but the console isn’t available (no prompt from username, etc) and it still shows up as “Production”. Suggestions on things to check? Thanks.

Langhalsdino · February 22, 2022, 4:21pm

Its the same on my site, i still build images that are production images.

dtischler · February 22, 2022, 6:18pm

@acostach @alexgg Could this be something that is Jetson-repo specific? I built a custom Raspberry Pi image yesterday, and using the -d flag was enough for me to get a login terminal on HDMI and .dev in balenaCloud. Thanks!

Langhalsdino · February 22, 2022, 8:33pm

Furthermore, it might be emmc specific, since @smithandrewc and I are using the jetson nano emmc and jetson xavier nx emmc with the jetson-flash repo tooling by balena.

acostach · February 23, 2022, 12:09pm

Hi @Langhalsdino , can you please try set “developmentMode”:“true” in your /mnt/boot/config.json? You can do this at runtime on the device

smithandrewc · February 23, 2022, 8:21pm

Hi @acostach, thanks for the suggestion. I modified the config.json file and I can confirm that it does go into development mode and a serial console is now available. Before I start searching, any idea where in the source code is that flag set?

Thanks.

Langhalsdino · February 24, 2022, 1:11pm

The suggested way works for me too

smithandrewc · February 24, 2022, 4:59pm

I think I’ve pinpointed the issue, maybe I’m using the tool wrong.

After compiling the image with the “-d” flag, I confirmed in the balena-image-jetson-nano-emmc.balenaos-img file (in partition 12) that the config.json reads:

{
  "developmentMode": true,
  "deviceType": "jetson-nano-emmc",
  "localMode": true,
  "persistentLogging": false
}

However, after I run

balena os configure --fleet <my fleet> build/tmp/deploy/images/jetson-nano-emmc/balena-image-jetson-nano-emmc.balenaos-img

which configures the image to connect to my fleet (with wifi), the config.json file has been altered significantly (which is expected) but I don’t see “developmentMode”: true.

{
  "applicationId": <app id>,
  "deviceType": "jetson-nano-emmc",
  "userId": <user id>,
  "appUpdatePollInterval": 600000,
  "listenPort": 48484,
  "vpnPort": 443,
  "apiEndpoint": "https://api.balena-cloud.com",
  "vpnEndpoint": "vpn.balena-cloud.com",
  "registryEndpoint": "registry2.balena-cloud.com",
  "deltaEndpoint": "https://delta.balena-cloud.com",
  "mixpanelToken": <some token>,
  "wifiSsid": <my ssid>,
  "wifiKey": <my key>,
  "apiKey": <my api>,
  "files": {
    "network/network.config": "<some network info>"
  }
}

Is there something wrong with my deployment procedure (i.e. compile, configure image for my fleet, write image to device, connect)?

Thanks.