How to git pull from the balena remote?

I’ve done a bunch of development. Because of the nature of the workflow, what you end up doing is force pushing a lot to the Balena remote, then eventually when you figure out what you did to get your thing to work, you re-do that work in a sane git branch that has useful commit messages. All that work is present in the Balena branch, but I’ve lost the repo that I was using on my disk. So, how can I git pull from the balena remote? Currently I can’t do that, as it tells me I don’t have permissions to do so, and to check my ssh key. I can push just fine, but can’t pull from my application.

I’m not sure if this helps, but I can share my workflow that offers me rapid development time while maintaining sane git branches.

I have a device with local mode enabled. I use balena push (ip-address or hostname) to make quick changes, do tests, try demo repositories, etc. Once I have code I want to keep I’ll commit that to my local git branch and push it to my remote master (github, bitbucket, etc). Eventually, when I want to deploy on a remote or production device, I use balena deploy (appname) which builds my docker image layers on my local machine, uploads them to the Docker Registry, and publishes them as a release.

Don’t quote me on this, but I don’t think balena push is intended to act like a full fledged git repository. It’s more focused on versioning built docker images as releases than it is about source code management.

Lastly, if you haven’t yet, go through this page with a fine-tooth comb and you might find some helpful tips: https://www.balena.io/docs/learn/deploy/deployment/

Hope that helps.

I haven’t been using the Balena cli. I’ve simply been using git push. This doesn’t answer my question either, which is how do I pull the work I’ve already done on a balena remote since I’ve lost the local repo?

Hopefully, someone on the Balena team can chime in on that. I’ve never used that workflow.

I imagine worst-case scenario is that you could SSH into your containers and copy your filesystem back onto your local machine. Though I imagine someone can recommend a better way.

Hello there @matthewcroughan,

You should be able to clone the balena repository by carrying out the following:

git clone <user>@git.balena-cloud.com:<user>/<app>.git

Where <user> is your balena username and <app> is the name of the application you want the source for. As long as the SSH keys registered with your balena account are in your ssh-agent keychain, then this should work without issue.

However, I want to stress very strongly that we make absolutely no guarantee that any code pushed to our private git repositories will be present if required. They are used purely as a store for our Builders to access to build application images, and as such are considered volatile. As per this page: https://www.balena.io/docs/learn/deploy/deployment/#git-push we make the following warning:
“Warning: The balenaCloud git repository is not intended as a code hosting solution, and we cannot guarantee the persistence of data in balenaCloud git remotes. We recommend you use a service like Github to manage and share your code.”

I’d also like to use this opportunity to note that git push is now considered a legacy form of pushing to our builders, and we will not be improving or recommending this method. The correct way to push code to our Builders is now balena push.

Thanks and best regards,

Heds

matthew@thinkpad ~/git/octoprint-opizero $ git clone gh_matthewcroughan@git.balena-cloud.com:gh_matthewcroughan/opizero-octoprint.git
Cloning into 'opizero-octoprint'...
X11 forwarding request failed on channel 0
remote: Counting objects: 102, done.
remote: Compressing objects: 100% (92/92), done.
fatal: the remote end hung up unexpectedly
fatal: early EOFs:  99% (101/102)
fatal: index-pack failed

I’m going to assume that this means my stuff has been removed from your caches? I’ll read into the balena cli methods of building projects to prevent this from happening in future.

@matthewcroughan

Try again, they were having difficulties with their git server earlier.

Also can you not ssh into your device and pull your own code that way? Unless you delete the source code during the build process it should all still be there.

Good Luck

Hi @matthewcroughan, yes please try again. As Thomas says, we had an incident with our git server. It’s resolved now and I could clone my old repositories from balena remote fine. Hope that you find yours as well.

Unfortunately this still returns the same error EOF.

No, this is simply about the build process, there is no special code on the device. It’s just the Dockerfile that built the device that I want.

Hey matthewcroughan I would give a shot on the recommendations here.

Luckily, somehow, it seemed to work without any modifications just now, and I have my repo. I seem to have a lot of issues with Balena that are like this generally, specifically when pushing to you, or doing anything relating to cryptography. It’s as if your servers just don’t have the entropy to deal with the requests. Though this specific issue was quite long lasting and seemed repeatable, it has not been, and I have successfully cloned it.

Whenever these things happen, I’ll endeavor to make a thread about it since I’m now on the forums.

Hi @matthewcroughan, good to hear that you recovered your repo fine.

Could you please elaborate on the other issues you experienced? What do the error messages say exactly? Do you still experience them?

Unfortunately, we do experience incidents in our git servers now and then. We are working hard to improve our services as we have a quickly growing number of users. These are growth pains. Hope you understand.

Of course I understand, it’s not really a complaint. just a mention. I will log them as I run into them, but they’re often obscure git messages. Whenever this takes place, pushing a commit multiple times until the servers are not throwing errors resolves the problem, it’s just a temporary thing. But often times the most frustrating issues are when it denies permissions and tells me I don’t have write access, or that my keys are invalid. This is what leads me to believe the entropy on your servers is related. I can’t comment too much though, as this is only from memory. I’ll post about it when I have the errors at hand.

Thank you. Much appreciated!

Having the same issue

$ git pull balena master
remote: Enumerating objects: 102, done.
remote: Counting objects: 100% (102/102), done.
remote: Compressing objects: 100% (49/49), done.
fatal: the remote end hung up unexpectedly
fatal: early EOFs: 38% (39/102)
fatal: index-pack failed

It will eventually work. I’m not sure of the actual reason for why it is intermittent, but keep trying, perhaps even write a script to try every 5 minutes. Additionally, try to move to the balena push workflow.

Hey there! Do you face the same issue if you try pushing the application using the balena push command (https://www.balena.io/docs/learn/deploy/deployment/#balena-push) from our official CLI tool? That approach bypasses git and sends the build artifact directly to our build servers, and is therefore the recommended approach.

Knowing whether that works or not would also help us better determine if the problem is in our git or build server.