Git Push Stuck on Writing Objects

Product support
1

Resin / Raspberry Pi setup is good, attempting to follow instructions per https://github.com/resin-io-projects/simple-server-node.

All works well until the final step (git push resin master), and the command hangs indefinitely on ‘Writing objects’, stays stuck at 54%. I have tried this from a separate workstation, with exactly the same results. Any thoughts?

C:\simple-server-node> git push resin master
Counting objects: 55, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (33/33), done.
Writing objects: 54% (30/55)

4

Hey @clayshek,
Welcome to resin! I just ran a build on a new app and it all looks okay for me (logs attached below). Can you describe you setup a bit so we can try narrow down the issue.

shaun-mbp-2:Desktop hobochild$ cd simple-server-node/
shaun-mbp-2:simple-server-node hobochild$ git remote add resin shaun_projects@git.resin.io:shaun_projects/testgit.git
shaun-mbp-2:simple-server-node hobochild$ git push resin master
Counting objects: 55, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (54/54), done.
Writing objects: 100% (55/55), 379.40 KiB | 0 bytes/s, done.
Total 55 (delta 22), reused 0 (delta 0)

[Info]     Starting build for shaun_projects/testgit, user shaun_projects
[Info]     Dashboard link: https://dashboard.resin.io/apps/520304/devices
[Info]     Building on ARM01
[Info]     Fetching base image
[==================================================>] 100%
[Info]     Building Dockerfile.template project
[Build]    Step 1/7 : FROM resin/raspberrypi3-node:6-slim
[Build]     ---> 5f3054b374ff
[Build]    Step 2/7 : WORKDIR /usr/src/app
[Build]     ---> 5b63ae90ccec
[Build]    Removing intermediate container 2c5ff897cc9c
[Build]    Step 3/7 : COPY package.json package.json
[Build]     ---> 137770f7485d
[Build]    Removing intermediate container d0d31ef9f0e8
[Build]    Step 4/7 : RUN JOBS=MAX npm install --production --unsafe-perm && npm cache clean && rm -rf /tmp/*
[Build]     ---> Running in ab30f6489dd7
[Build]    simple-server-node@1.0.0 /usr/src/app
[Build]    └─┬ express@4.15.4
[Build]      ├─┬ accepts@1.3.4
[Build]      │ ├─┬ mime-types@2.1.16
[Build]      │ │ └── mime-db@1.29.0
[Build]      │ └── negotiator@0.6.1
[Build]      ├── array-flatten@1.1.1
[Build]      ├── content-disposition@0.5.2
[Build]      ├── content-type@1.0.2
[Build]      ├── cookie@0.3.1
[Build]      ├── cookie-signature@1.0.6
[Build]      ├─┬ debug@2.6.8
[Build]      │ └── ms@2.0.0
[Build]      ├── depd@1.1.1
[Build]      ├── encodeurl@1.0.1
[Build]      ├── escape-html@1.0.3
[Build]      ├── etag@1.8.0
[Build]      ├─┬ finalhandler@1.0.4
[Build]      │ └── unpipe@1.0.0
[Build]      ├── fresh@0.5.0
[Build]      ├── merge-descriptors@1.0.1
[Build]      ├── methods@1.1.2
[Build]      ├─┬ on-finished@2.3.0
[Build]      │ └── ee-first@1.1.1
[Build]      ├── parseurl@1.3.1
[Build]      ├── path-to-regexp@0.1.7
[Build]      ├─┬ proxy-addr@1.1.5
[Build]      │ ├── forwarded@0.1.0
[Build]      │ └── ipaddr.js@1.4.0
[Build]      ├── qs@6.5.0
[Build]      ├── range-parser@1.2.0
[Build]      ├─┬ send@0.15.4
[Build]      │ ├── destroy@1.0.4
[Build]      │ ├─┬ http-errors@1.6.2
[Build]      │ │ └── inherits@2.0.3
[Build]      │ └── mime@1.3.4
[Build]      ├── serve-static@1.12.4
[Build]      ├── setprototypeof@1.0.3
[Build]      ├── statuses@1.3.1
[Build]      ├─┬ type-is@1.6.15
[Build]      │ └── media-typer@0.3.0
[Build]      ├── utils-merge@1.0.0
[Build]      └── vary@1.1.1
[Build]     ---> e8549021a788
[Build]    Removing intermediate container ab30f6489dd7
[Build]    Step 5/7 : COPY . ./
[Build]     ---> cd5fbe6223b2
[Build]    Removing intermediate container b014ba9f5f96
[Build]    Step 6/7 : ENV INITSYSTEM on
[Build]     ---> Running in f93de4056177
[Build]     ---> 82026fa1a623
[Build]    Removing intermediate container f93de4056177
[Build]    Step 7/7 : CMD npm start
[Build]     ---> Running in 411fc00f9932
[Build]     ---> 855f565b4953
[Build]    Removing intermediate container 411fc00f9932
[Build]    Successfully built 855f565b4953
[Build]    Successfully tagged registry2.resin.io:443/testgit/0a40648adc7806d049bf6172e44e98aa5f53bedb:latest
[Success]  Image created successfully
[Info]     Verifying image integrity...
[Success]  Image passed integrity checks!
[Info]     Uploading image to registry...
[==================================================>] 100%
[Success]  Image uploaded successfully!
[Info]     Check your dashboard for device download progress:
[Info]       https://dashboard.resin.io/apps/520304/devices
[Info]     Build took 1 minute and 5 seconds
[Info]     Image size: 171.15 MB (+ 1.33 MB from resin/raspberrypi3-node:6-slim)
			    \
			     \
			      \\
			       \\
			        >\/7
			    _.-(6'  \
			   (=___._/` \
			        )  \ |
			       /   / |
			      /    > /
			     j    < _\
			 _.-' :      ``.
			 \ r=._\        `.
			<`\\_  \         .`-.
			 \ r-7  `-. ._  ' .  `\
			  \`,      `-.`7  7)   )
			   \/         \|  \'  / `-._
			              ||    .'
			               \\  (
			                >\  >
			            ,.-' >.'
			           <.'_.''
			             <'
Tip: Talk to other resin.io users in the https://gitter.im/resin-io/chat chat room!

To shaun_projects@git.resin.io:shaun_projects/testgit.git
 * [new branch]      master -> master
shaun-mbp-2:simple-server-node hobochild$ date
Thu 24 Aug 2017 22:01:08 BST
5

From your logs it looks like you are running on windows. Which git client are you using. perhaps trying with git bash or another one might help?

6

Correct, running on Windows. I have same results from both Windows 10 & Win 7 Enterprise. Updated to the latest version of Git (2.14.1.windows.1) on one of those to try to rectify the issue, no change. On v 2.12.0.windows.1 on the other laptop. Was trying with Git command, just tried git bash, same results, hanging on Writing objects. My ssh key was imported to resin.io from Github without issue.

7

@clayshek, are you having any trouble running git push to other repos?

8

I’m not, tested to several others, working fine.

10

Interesting, I’ll keep looking into it and let you know what I find.

11

Thanks. I just tried using the hello-python project, no change in behavior:

PS C:> git clone git@github.com:alexandrosm/hello-python.git
Cloning into ‘hello-python’…
remote: Counting objects: 80, done.
remote: Total 80 (delta 0), reused 0 (delta 0), pack-reused 80
Receiving objects: 100% (80/80), 9.78 KiB | 1.22 MiB/s, done.
Resolving deltas: 100% (16/16), done.
PS C:> cd .\hello-python
PS C:\hello-python> git remote add resin gh_clayshek@git.resin.io:gh_clayshek/pifleetapp.git
PS C:\hello-python> git push resin master
Counting objects: 80, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (62/62), done.
Writing objects: 62% (50/80)

Stalls here indefinitely.

12

@clayshek, It’s possible there’s an issue with the SSH key type. Could you try the following:

  • Create a new project with just a Dockerfile
  • In the Dockerfile, put FROM resin/raspberrypi3-debian
  • run git init, git add Dockerfile, etc. and try pushing again
13

@clayshek, you might also try running GIT_SSH_COMMAND="ssh -v" git push resin master to see if we can get a little more info

14

So a bit of further troubleshooting and I seem to be on to something, thanks to your suggestion to use GIT_SSH_COMMAND=“ssh -v” git push…

Pretty much any regular git command that interacts with the resin git server (git clone, git push) is getting hung up at some point. However, when I run it as above to spit out more verbose info, it works. Seems like an info response is being returned, and my terminal isn’t handling it. I see this with both git from windows command line / powershell, as well as git bash, and experiencing this on multiple machines / Windows OS. Still digging further, I may vaguely recall this from some time ago as a git / Windows related bug. However, interesting that I don’t see this problem with github, or my internal gitlab enterprise servers.

15

Hey @clayshek,

Do you think that you could provide us with the log of the failed commands by adding extra verbosity with GIT_SSH_COMMAND=“ssh -vvv” ? That will give our backend team more to work with while troubleshooting this.

Best, Kostas

16

Looks like this is related to using Pageant (Putty) for ssh key management. I had my GIT setup configured that way, and has worked elsewhere as mentioned prior. That was the default when running any git command (GIT_SSH environment variable set to ‘C:\Program Files (x86)\PuTTY\plink.exe’), however, when running GIT_SSH_COMMAND that overrides it, taking pageant out of the picture, and everything works find on the Resin git server.

Below is output of the commands run with GIT_TRACE=2, showing the difference. I updated to latest version of putty software (0.70) but issue still exists. I have just removed it from the picture by deleting the GIT_SSH env var, and using git bash with ssh-agent. Works fine now. Others using putty / pageant may experience same.

~/simple-server-node (master)
$ GIT_TRACE=2 git push resin master
21:37:42.241072 git.c:328
trace: built-in: git ‘push’ ‘resin’ 'master’
Pushing to gh_clayshek@git.resin.io:gh_clayshek/pifleetapp.git
21:37:42.251072 run-command.c:626
trace: run_command: ‘C:\Program Files (x86)\PuTTY\plink.exe’ ‘gh_clayshek@git.resin.io’ ‘git-receive-pack ‘’‘gh_clayshek/pifleetapp.git’’’’

~/simple-server-node (master)
$ GIT_SSH_COMMAND=“ssh -v” GIT_TRACE=2 git push resin master
21:38:31.524890 git.c:328
trace: built-in: git ‘push’ ‘resin’ 'master’
21:38:31.530891 run-command.c:626
trace: run_command: ‘ssh -v’ ‘gh_clayshek@git.resin.io’ ‘git-receive-pack ‘’‘gh_clayshek/pifleetapp.git’’’'
OpenSSH_7.5p1, OpenSSL 1.0.2k 26 Jan 2017
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Connecting to git.resin.io [54.165.162.194] port 22.
debug1: Connection established.

17

Hi, so the issue is solved now?

18

Yes, solved for me by taking Pageant out of the picture for ssh key management. Would still exist if I went back to using Pageant, so remediated might be a better word, but all good now. Thank you for the help!

19

Glad it’s working. Enjoy resin!

20

I get the same issue and confirm it is using git on windows with putty pageant. Using pageant to manage keys works with every every other git server server I use, (bitbucket, github, gitlab). It is also the default for the popular widows git client tortise git.

I guess it is a lower priority to fix but would be nice someday and may cause some windows users confusion and make it harder to give resin.io a go.