# Terminal giving "rpc error" over and over again

**URL:** <https://forums.balena.io/t/terminal-giving-rpc-error-over-and-over-again/1491>\
**Category:** Product support\
**Tags:** terminal\
**Created:** [July 19, 2017, 12:57am UTC](https://forums.balena.io/t/terminal-giving-rpc-error-over-and-over-again/1491 "2017-07-19T00:57:25Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![rglenn](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/rglenn/32/504_2.png) [@rglenn](https://forums.balena.io/u/rglenn)\
**Post date:** [July 19, 2017, 12:57am UTC](https://forums.balena.io/t/terminal-giving-rpc-error-over-and-over-again/1491/1 "2017-07-19T00:57:25Z")

</div>

Hi everyone. I’m trying to use the terminal after pushing an application up, however I’m running into issues. The terminal in the dashboard gives the following in a continuous loop, and never actually gives me a prompt:

```auto
Connecting to e96081[REDACTED]...
Spawning shell...
rpc error: code = 2 desc = invalid character 'e' looking for beginning of value
SSH session disconnected
SSH reconnecting...
Spawning shell...
rpc error: code = 2 desc = invalid character 'e' looking for beginning of value
SSH session disconnected
SSH reconnecting...
Spawning shell...

```

Hardware is a Pi 1 running Resin OS 2.0.8+rev1 (prod)

Has anyone encountered this before? Any suggestions for fixing it? Thanks!

---

<div class="post-metadata">

**Author:** ![michal](https://avatars.discourse-cdn.com/v4/letter/m/34f0e0/32.png) [@michal](https://forums.balena.io/u/michal)\
**Post date:** [July 19, 2017, 8:37am UTC](https://forums.balena.io/t/terminal-giving-rpc-error-over-and-over-again/1491/3 "2017-07-19T08:37:01Z")

</div>

Hi Randy,

I sent you a private message asking for some more details.

---

<div class="post-metadata">

**Author:** ![daghemo](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/daghemo/32/531_2.png) [@daghemo](https://forums.balena.io/u/daghemo)\
**Post date:** [August 3, 2017, 6:41pm UTC](https://forums.balena.io/t/terminal-giving-rpc-error-over-and-over-again/1491/44 "2017-08-03T18:41:26Z")

</div>

Same problem here. ☹

That is, same error reported by @rglenn, using a Raspberry Pi 1 B board with Resin OS 2.0.8.  
Also tried with the new 2.2.0, with the same result.  
Also tried with resin-cli version 5.10.2 and 6.3.0, with the same result.

Switching back to Resin OS 2.0.6 seems to solve the problem: I can connect to my Raspberry Pi 1 B.

Raspberry Pi 0 W, known to be based on the Raspberry Pi 1 image, is not affected by the problem.  
Also, obviously, Raspberry Pi 3 is not affected by the problem.

---

<div class="post-metadata">

**Author:** ![imrehg](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/imrehg/32/36_2.png) [@imrehg](https://forums.balena.io/u/imrehg)\
**Post date:** [August 6, 2017, 10:48am UTC](https://forums.balena.io/t/terminal-giving-rpc-error-over-and-over-again/1491/45 "2017-08-06T10:48:41Z")

</div>

hey @daghemo That’s very strange, thanks for letting us know. We’ll be checking it out and try to reproduce the issue. What base image are you using? Is it possible to share the code that you are deploying on the device?

The main difference between 2.0.6 and newer versions is the newer using Docker 17.03.

---

<div class="post-metadata">

**Author:** ![daghemo](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/daghemo/32/531_2.png) [@daghemo](https://forums.balena.io/u/daghemo)\
**Post date:** [August 6, 2017, 11:36am UTC](https://forums.balena.io/t/terminal-giving-rpc-error-over-and-over-again/1491/46 "2017-08-06T11:36:46Z")

</div>

Hi @imrehg, my Dockerfile starts with:

`FROM resin/%%RESIN_MACHINE_NAME%%-python`

Yes, me too I suspect that the error is related to Docker.

---

<div class="post-metadata">

**Author:** ![imrehg](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/imrehg/32/36_2.png) [@imrehg](https://forums.balena.io/u/imrehg)\
**Post date:** [August 7, 2017, 9:26am UTC](https://forums.balena.io/t/terminal-giving-rpc-error-over-and-over-again/1491/47 "2017-08-07T09:26:50Z")

</div>

Just set up a Raspberry Pi 1 with resinOS `2.2.0+rev1` and a basic Python project. That did not give me those issues, thus so far cannot replicate this.  
Can you share the code that you are deploying, or if you have a device doing that, send us the dashboard link of the device in private message? That would help quite a bit to triage this issue!

---

<div class="post-metadata">

**Author:** ![phynias](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/phynias/32/538_2.png) [@phynias](https://forums.balena.io/u/phynias)\
**Post date:** [August 9, 2017, 9:56pm UTC](https://forums.balena.io/t/terminal-giving-rpc-error-over-and-over-again/1491/48 "2017-08-09T21:56:01Z")

</div>

just setup a pi3 with 2.2.0+rev1 and this is happening to me.

---

<div class="post-metadata">

**Author:** ![EugeneGwon](https://avatars.discourse-cdn.com/v4/letter/e/53a042/32.png) [@EugeneGwon](https://forums.balena.io/u/EugeneGwon)\
**Post date:** [August 17, 2017, 5:21am UTC](https://forums.balena.io/t/terminal-giving-rpc-error-over-and-over-again/1491/49 "2017-08-17T05:21:06Z")

</div>

It happens me, too.  
after deploy my application, it works very well. but a day after, It shows me ‘rpc error’.  
It works again after restart my application.

---

<div class="post-metadata">

**Author:** ![imrehg](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/imrehg/32/36_2.png) [@imrehg](https://forums.balena.io/u/imrehg)\
**Post date:** [August 27, 2017, 9:25am UTC](https://forums.balena.io/t/terminal-giving-rpc-error-over-and-over-again/1491/50 "2017-08-27T09:25:00Z")

</div>

@phynias, @EugeneGwon, do you still have these issues? If you hit them, could you send us a heads up here to have a chance to debug when it’s happening? Or if you have a repeatable way to replicate it, that would be also helpful. It’s not great that the application needs to be restarted for any reason like this.

---

<div class="post-metadata">

**Author:** ![EugeneGwon](https://avatars.discourse-cdn.com/v4/letter/e/53a042/32.png) [@EugeneGwon](https://forums.balena.io/u/EugeneGwon)\
**Post date:** [August 28, 2017, 2:05am UTC](https://forums.balena.io/t/terminal-giving-rpc-error-over-and-over-again/1491/51 "2017-08-28T02:05:50Z")

</div>

Hi, @imrehg  
I still have this issue. usually, this happens when I sleep/wake up my mac book.  
here’s how to :

1. open multiple application terminals(2 or 3) on [resin.io](http://resin.io) dashboard
2. sleep my mac book (‘sleep’ menu under Apple icon)
3. wake up my mac book about 10 mins later
4. refresh browser; now I can see terminals are trying to respawn shell but failed with an error  
(rpc error: code=2 desc=invalid character ‘e’ looking for beginning of value)

most time, at least 1 terminal had succeeded to reconnect, but sometimes, all fail.  
If I keep open/close terminal multiple times, then it suddenly succeeded to connect ! 🙂

here’s my environment :

- browser: google chrome 60.0.3112.90
- mac book model: mac book air (mid-2011), Sierra
- Raspberry Pi 1 model B, USB Wifi

Please let me know if you need more information.

Thanks.

---

<div class="post-metadata">

**Author:** ![greenwalrus](https://avatars.discourse-cdn.com/v4/letter/g/ce7236/32.png) [@greenwalrus](https://forums.balena.io/u/greenwalrus)\
**Post date:** [October 12, 2017, 6:09am UTC](https://forums.balena.io/t/terminal-giving-rpc-error-over-and-over-again/1491/52 "2017-10-12T06:09:57Z")

</div>

I’m having this consistently happen on my raspberry pi zero w.

I ssh’d in via dev mode, and can reprpdouce with  
~# docker exec -it ef802e4585c4 bash -v  
rpc error: code = 2 desc = invalid character ‘e’ looking for beginning of value

Trying it without -t causes a full error message to be spit out  
~# docker exec -i ef802e4585c4 bash  
rpc error: code = 2 desc = oci runtime error: exec failed: container\_linux.go:248: starting container process caused “process\_linux.go:100: writing config to pipe caused “write parent: broken pipe””

---

<div class="post-metadata">

**Author:** ![greenwalrus](https://avatars.discourse-cdn.com/v4/letter/g/ce7236/32.png) [@greenwalrus](https://forums.balena.io/u/greenwalrus)\
**Post date:** [October 12, 2017, 6:16am UTC](https://forums.balena.io/t/terminal-giving-rpc-error-over-and-over-again/1491/53 "2017-10-12T06:16:22Z")

</div>

[https://github.com/moby/moby/issues/31383](https://github.com/moby/moby/issues/31383) appears to be related

Running  
journalctl -u docker

Shows the following.

Oct 12 06:05:55 919df58 dockerd[653]: time=“2017-10-12T06:05:55.795635713Z” level=error m  
sg="Error running exec in container: rpc error: code = 2 desc = invalid character ‘e’ loo  
king for beginning of value"  
Oct 12 06:05:55 919df58 dockerd[653]: time=“2017-10-12T06:05:55.804043677Z” level=error m  
sg="Handler for POST /v1.27/exec/0f8cc2d6819cd45c107da6b771fe463e4b5f6c3217cab116eee03a5f  
2e0d0f2b/resize returned error: rpc error: code = 2 desc = containerd: process not found  
for container"  
Oct 12 06:06:06 919df58 dockerd[653]: time=“2017-10-12T06:06:06.042497940Z” level=error m  
sg="Error running exec in container: rpc error: code = 2 desc = invalid character ‘e’ loo  
king for beginning of value"  
Oct 12 06:06:06 919df58 dockerd[653]: time=“2017-10-12T06:06:06.055061886Z” level=error m  
sg="Handler for POST /v1.27/exec/8949392bfeb6556727c1418a877e4859a7c6aeea14857c2c7f8e7cd8  
79e1956a/resize returned error: rpc error: code = 2 desc = containerd: process not found  
for container"  
Oct 12 06:06:35 919df58 dockerd[653]: time=“2017-10-12T06:06:35.159834262Z” level=error m  
sg=“Error running exec in container: rpc error: code = 2 desc = oci runtime error: exec f  
ailed: container\_linux.go:248: starting container process caused “process\_linux.go:100:  
writing config to pipe caused \“write parent: broken pipe\”””

Given that docker 17.03.1 was released in febuary, kicking up the version may be the easiest way to test the fix.

---

<div class="post-metadata">

**Author:** ![imrehg](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/imrehg/32/36_2.png) [@imrehg](https://forums.balena.io/u/imrehg)\
**Post date:** [October 12, 2017, 4:12pm UTC](https://forums.balena.io/t/terminal-giving-rpc-error-over-and-over-again/1491/54 "2017-10-12T16:12:47Z")

</div>

sorry for the issue, could you give our team support access on the device? Will PM you to ask you for your device dashboard URL

---

<div class="post-metadata">

**Author:** ![JoachimLindborg](https://avatars.discourse-cdn.com/v4/letter/j/5f9b8f/32.png) [@JoachimLindborg](https://forums.balena.io/u/JoachimLindborg)\
**Post date:** [November 5, 2017, 8:45am UTC](https://forums.balena.io/t/terminal-giving-rpc-error-over-and-over-again/1491/56 "2017-11-05T08:45:05Z")

</div>

I just also started to have this problem now. Worked fine last week.  
I have Resin OS 2.2.0+rev1 (prod) on a raspberry P1. it has been running for around a week on same software that I deployed on Oct 29th 2017. Application works fine but unable to use terminal. Haven’t tried any fixes yet

---

<div class="post-metadata">

**Author:** ![royillingworth](https://avatars.discourse-cdn.com/v4/letter/r/da6949/32.png) [@royillingworth](https://forums.balena.io/u/royillingworth)\
**Post date:** [November 5, 2017, 9:37pm UTC](https://forums.balena.io/t/terminal-giving-rpc-error-over-and-over-again/1491/59 "2017-11-05T21:37:13Z")

</div>

I also just started getting this message. Also running Raspberry Pi 1, running Resin OS 2.2.0+rev1(prod). Started in the last few days. Sometimes I can log in, most times not - today not at all.  
I have 2 units that are showing this problem. I think (need to find a way to verify) the ones that are failing were built as 2.2.0+rev1 rather than being upgraded to 2.2.0 (originally built with 2.0.0+rev3).

---

<div class="post-metadata">

**Author:** ![sqweelygig](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/sqweelygig/32/150_2.png) [@sqweelygig](https://forums.balena.io/u/sqweelygig)\
**Post date:** [November 6, 2017, 1:20pm UTC](https://forums.balena.io/t/terminal-giving-rpc-error-over-and-over-again/1491/60 "2017-11-06T13:20:22Z")

</div>

Would you mind granting support access, under actions, and then letting us know the device URL.

---

<div class="post-metadata">

**Author:** ![royillingworth](https://avatars.discourse-cdn.com/v4/letter/r/da6949/32.png) [@royillingworth](https://forums.balena.io/u/royillingworth)\
**Post date:** [November 8, 2017, 2:05am UTC](https://forums.balena.io/t/terminal-giving-rpc-error-over-and-over-again/1491/62 "2017-11-08T02:05:18Z")

</div>

Any update here? I gave you 12 hours of access - not sure if anything material occurred?

---

<div class="post-metadata">

**Author:** ![imrehg](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/imrehg/32/36_2.png) [@imrehg](https://forums.balena.io/u/imrehg)\
**Post date:** [November 11, 2017, 11:11am UTC](https://forums.balena.io/t/terminal-giving-rpc-error-over-and-over-again/1491/63 "2017-11-11T11:11:01Z")

</div>

Hi, sorry for the delay. Could you grant us a bit longer access, and provide the device’s dashboard URL? Otherwise we can’t know what device to look at.

---

<div class="post-metadata">

**Author:** ![royillingworth](https://avatars.discourse-cdn.com/v4/letter/r/da6949/32.png) [@royillingworth](https://forums.balena.io/u/royillingworth)\
**Post date:** [November 11, 2017, 1:01pm UTC](https://forums.balena.io/t/terminal-giving-rpc-error-over-and-over-again/1491/64 "2017-11-11T13:01:13Z")

</div>

OK, I have granted support access for 1 week. The UUID of a failing node is 608030f24d65e80db139536335940ad2.  
I originally responded to you by email and apparently I have an email configuration issue as you never received it (it had the UUIDs in that email)

---

<div class="post-metadata">

**Author:** ![PeterKAlvin](https://avatars.discourse-cdn.com/v4/letter/p/91b2a8/32.png) [@PeterKAlvin](https://forums.balena.io/u/PeterKAlvin)\
**Post date:** [December 5, 2017, 2:57pm UTC](https://forums.balena.io/t/terminal-giving-rpc-error-over-and-over-again/1491/67 "2017-12-05T14:57:47Z")

</div>

Hey Guys

we are getting the same error, after local push:

```auto
- Creating 'local-app' container
- Starting 'local-app' container
rdt push failed. Error: Error while starting container local-app: Error: (HTTP code 500) server error - invalid character 'c' looking for beginning of value Error: Error while starting container local-app: Error: (HTTP code 500) server error - invalid character 'c' looking for beginning of value
    at /usr/local/lib/node_modules/resin-cli/node_modules/resin-sync/build/docker-utils.js:309:15
    at tryCatcher (/usr/local/lib/node_modules/resin-cli/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/usr/local/lib/node_modules/resin-cli/node_modules/bluebird/js/release/promise.js:512:31)
    at Promise._settlePromise (/usr/local/lib/node_modules/resin-cli/node_modules/bluebird/js/release/promise.js:569:18)
    at Promise._settlePromise0 (/usr/local/lib/node_modules/resin-cli/node_modules/bluebird/js/release/promise.js:614:10)
    at Promise._settlePromises (/usr/local/lib/node_modules/resin-cli/node_modules/bluebird/js/release/promise.js:689:18)
    at Async._drainQueue (/usr/local/lib/node_modules/resin-cli/node_modules/bluebird/js/release/async.js:133:16)
    at Async._drainQueues (/usr/local/lib/node_modules/resin-cli/node_modules/bluebird/js/release/async.js:143:10)
    at Immediate.Async.drainQueues (/usr/local/lib/node_modules/resin-cli/node_modules/bluebird/js/release/async.js:17:14)
    at runCallback (timers.js:789:20)
    at tryOnImmediate (timers.js:751:5)
    at processImmediate [as _immediateCallback] (timers.js:722:5)

```

[Next page](https://forums.balena.io/t/terminal-giving-rpc-error-over-and-over-again/1491.md?page=2)
