Facing issue in openbalena configuration

openBalena
42

I have set the A record. I got the configuration using dig api.test.domain.com also.

When I try balena login I get following log:

➜ balena login
 _            _
| |__   __ _ | |  ____  _ __    __ _
| '_ \ / _` || | / __ \| '_ \  / _` |
| |_) | (_) || ||  ___/| | | || (_) |
|_.__/ \__,_||_| \____/|_| |_| \__,_|


Logging in to test.domain.com
? How would you like to login? Credentials
? Email: user@domain.com
? Password: [hidden]
ENOTFOUND: request to https://api.test.domain.com/login_ failed, reason: getaddrinfo ENOTFOUND api.test.domain.com api.test.domain.com:443

Additional information may be available by setting a DEBUG=1 environment
variable: "set DEBUG=1" on a Windows command prompt, or "export DEBUG=1"
on Linux or macOS.

If you need help, don't hesitate in contacting our support forums at
https://forums.balena.io

For bug reports or feature requests, have a look at the GitHub issues or
create a new one at: https://github.com/balena-io/balena-cli/issues/

After setting debug =1 I get following logs:

FetchError: request to https://api.test.domain.com/login_ failed, reason: getaddrinfo ENOTFOUND api.test.domain.com api.test.domain.com:443
    at ClientRequest.<anonymous> (/snapshot/balena-cli/node_modules/node-fetch/index.js:133:11)
    at emitOne (events.js:96:13)
    at ClientRequest.emit (events.js:188:7)
    at ClientRequest.emit (/snapshot/balena-cli/node_modules/raven/lib/instrumentation/http.js:51:23)
    at TLSSocket.socketErrorListener (_http_client.js:310:9)
    at emitOne (events.js:96:13)
    at TLSSocket.emit (events.js:188:7)
    at connectErrorNT (net.js:1025:8)
    at _combinedTickCallback (internal/process/next_tick.js:80:11)
    at process._tickDomainCallback (internal/process/next_tick.js:128:9)
43

Ok, so if your computer can resolve api.test.domain.com to an IP, you should not get this error with curl anymore:

root@machine:/home/root# curl http://test.domain.com/ping
curl: (6) Could not resolve host: api.test.domain.com
44

I have updated the Question can you please check the logs?

45

The error still means, that the computer you are running balena login on cannot resolve your host name to an IP. Can you provide the output of the dig command?

47

I have commented some of the section in this log.

Log of dig command:

➜ dig api.test.domain.com

; <<>> DiG 9.10.6 <<>> api.test.domain.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 5061
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;api.test.domain.com.		IN	A

;; AUTHORITY SECTION:
test.com.		274	IN	SOA	ns43.domaincontrol.com. dns.jomax.net. 2019050900 28800 7200 604800 600

;; Query time: 5 msec
;; SERVER: **********
;; WHEN: Tue May 14 15:59:12 IST 2019
;; MSG SIZE  rcvd: 116
48

Yeah, so this output shows you, that there are no A or CNAME records configured for your domain. If you compare this with www.google.com for example:

dig www.google.com

; <<>> DiG 9.11.3-1ubuntu1.7-Ubuntu <<>> www.google.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 44876
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;www.google.com.			IN	A

;; ANSWER SECTION:
www.google.com.		65	IN	A	216.58.206.4

;; Query time: 0 msec
;; SERVER: 127.0.0.53#53(127.0.0.53)
;; WHEN: Tue May 14 12:37:21 CEST 2019
;; MSG SIZE  rcvd: 59

You need to see an answer section in the dig response in that looks similar to this:

;; ANSWER SECTION:
api.test.domain.com.		65	IN	A	X.X.X.X

Once you see this answer section, you did set up your dns entries correctly.

49

That was output for api.test.domain.com but when I do dig test.domain.com it works

➜ dig test.domain.com

; <<>> DiG 9.10.6 <<>> test.domain.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 860
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 5

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;test.domain.com.		IN	A

;; ANSWER SECTION:
test.domain.com.	372	IN	A	****

;; AUTHORITY SECTION:
domain.com.		3306	IN	NS	ns44.domaincontrol.com.
domain.com.		3306	IN	NS	ns43.domaincontrol.com.

;; ADDITIONAL SECTION:
ns43.domaincontrol.com.	152718	IN	A	*****
ns43.domaincontrol.com.	152718	IN	AAAA	*****
ns44.domaincontrol.com.	65104	IN	A	*****
ns44.domaincontrol.com.	161720	IN	AAAA	*****

;; Query time: 5 msec
;; SERVER: 203.187.215.35#53(203.187.215.35)
;; WHEN: Tue May 14 16:21:53 IST 2019
;; MSG SIZE  rcvd: 200

I think the problem is that we can’t assign subdomain to subdomain. I have created a subdomain with A record named ‘test.domain.com’ (the real one is different this just specifies the format.

I think I need to create api.domain.com domain. But the problem is we are using api.domain.com already in our existing system. Can I make the change in the server so that it will expect other than api.domain.com ( i.e replace api.domain.com with something like a test.domain.com )

51

I am still facing the issue can anyone help me here?

53

Hi,
it looks like you are struggling with your DNS setup. You will have to make sure that api. is being resolved correctly so you will have to add appropriate records to you DNS configuration. As we do not know how you are configuring your DNS we can not assist you in doing so.
For us to help you, you will have to provide information on how you are setting up DNS.
It looks like you successfully set up test.domain.com - if you are struggling with setting up api.test.domain.com maybe for now it will be easier to set up the system without the test subdomain, going to api.domain.com if that is not taken on your host.
Regards
Thomas

56

Okay I’ll check into that But I am still confused if this type of API is possible

api.test.domain.com

I think I need to register domain.com and api.domain.com will be mapped as a subdomain.

Also, can I change the api.domain.com to test.domain.com for openbalena configuration?

57

Hey @Sharvin26, In current openbalena setup scripts, you can use an api.<domain> subdomain as the API endpoint see, where <domain> could be a valid FQDN . You need to modify the setup for changing the API FQDN into something different than api. prefix.

All you need to setup here, for using test.domain.com as your base domain, would be a few CNAMES pointing to your server, as described here: https://www.balena.io/open/docs/getting-started/#Domain-Configuration
Just replace mydomain.com with test.domain.com.
After this step is complete, you should be able to verify those CNAMES are pointing to your server, as soon as DNS propagation is complete by checking that each of the below:

dig +short api.test.domain.com
dig +short registry.mydomain.com
dig +short vpn.mydomain.com
dig +short s3.mydomain.com

is yielding your server ip.

59

Thanks this worked.

dig +short api.test.domain.com
dig +short registry.test.domain.com
dig +short vpn.test.domain.com
dig +short s3.test.domain.com

When I start the openBalena instance with:

./scripts/compose up

I get this log:

root@machine:/home/root/open-balena# ./scripts/compose up
Starting openbalena_s3_1_4fbc1fd9053b            ... done
Starting openbalena_db_1_a31c9cfcac3f            ... done
Starting openbalena_redis_1_41d5143bd969         ... done
Starting openbalena_cert-provider_1_65dd3f2ad90d ... done
Starting openbalena_api_1_c26c293fcb23           ... done
Starting openbalena_registry_1_fdeb54d92860      ... done
Starting openbalena_vpn_1_2cdf91b9b5d9           ... done
Starting openbalena_haproxy_1_7e5b800840ab       ... done
Attaching to openbalena_db_1_a31c9cfcac3f, openbalena_cert-provider_1_65dd3f2ad90d, openbalena_redis_1_41d5143bd969, openbalena_s3_1_4fbc1fd9053b, openbalena_api_1_c26c293fcb23, openbalena_vpn_1_2cdf91b9b5d9, openbalena_registry_1_fdeb54d92860, openbalena_haproxy_1_7e5b800840ab
db_1_a31c9cfcac3f | 2019-05-18 09:38:51.823 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
db_1_a31c9cfcac3f | 2019-05-18 09:38:51.823 UTC [1] LOG:  listening on IPv6 address "::", port 5432
db_1_a31c9cfcac3f | 2019-05-18 09:38:51.835 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1_a31c9cfcac3f | 2019-05-18 09:38:51.884 UTC [21] LOG:  database system was shut down at 2019-05-18 09:23:23 UTC
db_1_a31c9cfcac3f | 2019-05-18 09:38:51.931 UTC [1] LOG:  database system is ready to accept connections
cert-provider_1_65dd3f2ad90d | [Error] ACTIVE variable is not enabled. Value should be "true" or "yes" to continue.
cert-provider_1_65dd3f2ad90d | [Error] Unable to continue due to misconfiguration. See errors above. [Stopping]
redis_1_41d5143bd969 | 1:C 18 May 2019 09:38:52.310 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_1_41d5143bd969 | 1:C 18 May 2019 09:38:52.311 # Redis version=5.0.5, bits=64, commit=00000000, modified=0, pid=1, just started
redis_1_41d5143bd969 | 1:C 18 May 2019 09:38:52.311 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis_1_41d5143bd969 | 1:M 18 May 2019 09:38:52.314 * Running mode=standalone, port=6379.
redis_1_41d5143bd969 | 1:M 18 May 2019 09:38:52.315 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis_1_41d5143bd969 | 1:M 18 May 2019 09:38:52.315 # Server initialized
redis_1_41d5143bd969 | 1:M 18 May 2019 09:38:52.315 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
redis_1_41d5143bd969 | 1:M 18 May 2019 09:38:52.315 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
redis_1_41d5143bd969 | 1:M 18 May 2019 09:38:52.316 * DB loaded from disk: 0.001 seconds
redis_1_41d5143bd969 | 1:M 18 May 2019 09:38:52.316 * Ready to accept connections
s3_1_4fbc1fd9053b | Systemd init system enabled.
api_1_c26c293fcb23 | Systemd init system enabled.
vpn_1_2cdf91b9b5d9 | Systemd init system enabled.
registry_1_fdeb54d92860 | Systemd init system enabled.
haproxy_1_7e5b800840ab | Building certificate from environment variables...
haproxy_1_7e5b800840ab | Setting up watches.  Beware: since -r was given, this may take a while!
haproxy_1_7e5b800840ab | Watches established.
haproxy_1_7e5b800840ab | [NOTICE] 137/093856 (15) : New worker #1 (17) forked
haproxy_1_7e5b800840ab | [WARNING] 137/093856 (17) : Server backend_api/resin_api_1 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
haproxy_1_7e5b800840ab | [ALERT] 137/093856 (17) : backend 'backend_api' has no server available!
haproxy_1_7e5b800840ab | [WARNING] 137/093856 (17) : Server backend_registry/resin_registry_1 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
haproxy_1_7e5b800840ab | [ALERT] 137/093856 (17) : backend 'backend_registry' has no server available!
haproxy_1_7e5b800840ab | [WARNING] 137/093856 (17) : Server backend_vpn/resin_vpn_1 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
haproxy_1_7e5b800840ab | [ALERT] 137/093856 (17) : backend 'backend_vpn' has no server available!
haproxy_1_7e5b800840ab | [WARNING] 137/093857 (17) : Server vpn-tunnel/balena_vpn is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
haproxy_1_7e5b800840ab | [ALERT] 137/093857 (17) : proxy 'vpn-tunnel' has no server available!
db_1_a31c9cfcac3f | 2019-05-18 09:39:00.509 UTC [28] ERROR:  relation "uniq_model_model_type_vocab" already exists
db_1_a31c9cfcac3f | 2019-05-18 09:39:00.509 UTC [28] STATEMENT:  CREATE UNIQUE INDEX "uniq_model_model_type_vocab" ON "model" ("is of-vocabulary", "model type");
haproxy_1_7e5b800840ab | [WARNING] 137/093900 (17) : Server backend_registry/resin_registry_1 is UP, reason: Layer4 check passed, check duration: 0ms. 1 active and 0 backup servers online. 0 sessions requeued, 0 total in queue.
haproxy_1_7e5b800840ab | [WARNING] 137/093900 (17) : Server backend_vpn/resin_vpn_1 is UP, reason: Layer4 check passed, check duration: 0ms. 1 active and 0 backup servers online. 0 sessions requeued, 0 total in queue.
haproxy_1_7e5b800840ab | [WARNING] 137/093903 (17) : Server vpn-tunnel/balena_vpn is UP, reason: Layer4 check passed, check duration: 0ms. 1 active and 0 backup servers online. 0 sessions requeued, 0 total in queue.
haproxy_1_7e5b800840ab | [WARNING] 137/093906 (17) : Server backend_api/resin_api_1 is UP, reason: Layer4 check passed, check duration: 0ms. 1 active and 0 backup servers online. 0 sessions requeued, 0 total in queue.

And then I tried to open api.test.domain.com on google chrome it gave me an error This site can’t be reached

So I wrote a small flask server and I opened api.test.domain.com and I was able to successfully get the response.

I think something is going wrong for this command ./scripts/compose up

60

@sharvin26 did you use HTTPS and not HTTP - openBalena uses HTTPS.

61

Also, your comment ^^ you ran ./scripts/compose up and I just want to say that if you Ctrl-C after that then it will stop the services. You should use a -d flag to return you to a prompt.

62

Okay then I need to add -c flag in front of this command right?

./scripts/quickstart -U <email@address> -P <password> -d mydomain.com

63

@sharvin26 the -c flag just instructs the service stack to attempt to acquire a publicly trusted certificate from LetsEncrypt; this would replace the self-signed one produced during the quickstart phase, at runtime. No matter if you use the -c flag or not, openBalena will always be exposed over HTTPS and you should always use HTTPS to communicate with it.

According to you log above; haproxy_1_7e5b800840ab | [WARNING] 137/093906 (17) : Server backend_api/resin_api_1 is UP, reason: Layer4 check passed, check duration: 0ms. 1 active and 0 backup servers online. 0 sessions requeued, 0 total in queue. the API container is started and running fine as HAproxy is confirming its availability. If you do a cURL call, like so, curl -ik https://api.test.domain.com/ping you should see a OK response. This would mean the API is working.

65

Thanks it’s working

But for balena apps command I get following error

➜ balena apps
Cannot read property 'length' of undefined

Additional information may be available by setting a DEBUG=1 environment
variable: "set DEBUG=1" on a Windows command prompt, or "export DEBUG=1"
on Linux or macOS.

If you need help, don't hesitate in contacting our support forums at
https://forums.balena.io

For bug reports or feature requests, have a look at the GitHub issues or
create a new one at: https://github.com/balena-io/balena-cli/issues/
66

@sharvin26 can you enable debug and paste that output. Also, please make sure your balena CLI is up to date.

67

Yes I have recently downloaded the new version of balena cli

➜ balena version
10.13.3

LOGS:

 balena apps
TypeError: Cannot read property 'length' of undefined
    at /snapshot/balena-cli/build/actions/app.js:75:60
    at Array.forEach (native)
    at /snapshot/balena-cli/build/actions/app.js:74:20
    at tryCatcher (/snapshot/balena-cli/node_modules/bluebird/js/release/util.js:16:23)
    at Promise.module.exports.Promise._settlePromiseFromHandler (/snapshot/balena-cli/node_modules/bluebird/js/release/promise.js:512:31)
    at Promise.module.exports.Promise._settlePromise (/snapshot/balena-cli/node_modules/bluebird/js/release/promise.js:569:18)
    at Promise.module.exports.Promise._settlePromise0 (/snapshot/balena-cli/node_modules/bluebird/js/release/promise.js:614:10)
    at Promise.module.exports.Promise._settlePromises (/snapshot/balena-cli/node_modules/bluebird/js/release/promise.js:694:18)
    at Promise.module.exports.Promise._fulfill (/snapshot/balena-cli/node_modules/bluebird/js/release/promise.js:638:18)
    at MappingPromiseArray.module.exports.PromiseArray._resolve (/snapshot/balena-cli/node_modules/bluebird/js/release/promise_array.js:126:19)
    at MappingPromiseArray.module.exports.MappingPromiseArray._promiseFulfilled (/snapshot/balena-cli/node_modules/bluebird/js/release/map.js:101:18)
    at MappingPromiseArray.module.exports.PromiseArray._iterate (/snapshot/balena-cli/node_modules/bluebird/js/release/promise_array.js:114:31)
    at MappingPromiseArray.init (/snapshot/balena-cli/node_modules/bluebird/js/release/promise_array.js:78:10)
    at Promise.module.exports.Promise._settlePromise (/snapshot/balena-cli/node_modules/bluebird/js/release/promise.js:566:21)
    at Promise.module.exports.Promise._settlePromise0 (/snapshot/balena-cli/node_modules/bluebird/js/release/promise.js:614:10)
    at Promise.module.exports.Promise._settlePromises (/snapshot/balena-cli/node_modules/bluebird/js/release/promise.js:694:18)
    at _drainQueueStep (/snapshot/balena-cli/node_modules/bluebird/js/release/async.js:138:12)
    at _drainQueue (/snapshot/balena-cli/node_modules/bluebird/js/release/async.js:131:9)
    at Async._drainQueues (/snapshot/balena-cli/node_modules/bluebird/js/release/async.js:147:5)
    at Immediate.e.Async.drainQueues (/snapshot/balena-cli/node_modules/bluebird/js/release/async.js:17:14)
    at runCallback (timers.js:672:20)
    at tryOnImmediate (timers.js:645:5)
    at processImmediate [as _immediateCallback] (timers.js:617:5)

If you need help, don't hesitate in contacting our support forums at
https://forums.balena.io

For bug reports or feature requests, have a look at the GitHub issues or
create a new one at: https://github.com/balena-io/balena-cli/issues/
68

Can you just confirm the environment/command you used prior to this? Did you login to your instance fine etc?