Accessing ROS master from outside the container

Product support
#28

(Sorry, hit enter accidentally)

But, it’s being set to localhost in your case, which means it’ll never listen on an external interface (ie. you can’t connect to it if you’re not on the same device). It’s also using port 11311, so you’d need to EXPOSE 11311 in your Dockerfile.

I suspect what you actually want is the IP address of the device, ie:

ROS_MASTER_URI=http://192.168.1.116:11311/

or using port 80 if that’s been started as such.

This should then bind to the external interface instead of localhost. You might be able to use http://0.0.0.0:1131 to bind to all interfaces, depending on how the ROS code works.

#34

@hedss
I think even the rosmaster is run by local host, in normal use cases it should also be accessible from the outside world. I can connect to my rosmaster running on my laptop from the container, by doing

export ROS_MASTER_URI=http://<IP of laptop>:80

Therefore, running a roscore on a local laptop and accessing it from a container works.

However what doesn’t work, is running a rosmaster in the container and accessing it from the laptop.

When I run roscore on port 80 (in the container) and launch a ros node, XML-RPC server spits an error that the it cannot contact the master.

$ export ROS_MASTER_URI=http://<IP of laptop>:80
$ roslaunch <launchfile>.launch
started roslaunch server http://localhost:37337/

...

auto-starting new master
process[master]: started with pid [878]
ERROR: Unable to start XML-RPC server, port 80 is already in use
Unhandled exception in thread started by <bound method XmlRpcNode.run of <rosgraph.xmlrpc.XmlRpcNode object at 0x7f02189d42d0>>
Traceback (most recent call last):
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rosgraph/xmlrpc.py", line 215, in run
    self._run()
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rosgraph/xmlrpc.py", line 284, in _run
    self._run_init()
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rosgraph/xmlrpc.py", line 234, in _run_init
    self.server = ThreadingXMLRPCServer((bind_address, port), log_requests)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rosgraph/xmlrpc.py", line 115, in __init__
    SimpleXMLRPCServer.__init__(self, addr, SilenceableXMLRPCRequestHandler, log_requests)
  File "/usr/lib/python2.7/SimpleXMLRPCServer.py", line 593, in __init__
    SocketServer.TCPServer.__init__(self, addr, requestHandler, bind_and_activate)
  File "/usr/lib/python2.7/SocketServer.py", line 417, in __init__
    self.server_bind()
  File "/usr/lib/python2.7/SocketServer.py", line 431, in server_bind
    self.socket.bind(self.server_address)
  File "/usr/lib/python2.7/socket.py", line 228, in meth
    return getattr(self._sock,name)(*args)
socket.error: [Errno 98] Address already in use

I have read in the documentation that for single container applications I do not need to expose ports. Should I try and manually exposing the ports? Does it matter that it I am using it on runtime?

Thank you in advance,

#35

This is getting pretty convoluted, and a bit difficult to untangle without us having that much experience with ROS that you have.

To me the earlier CURL that resulted in a 501 error shows that the endpoint is listening properly - just needs different communication. Otherwise it would be just refusing to connect.

The master that you are trying to connect to is running in a resinOS system, right? Is that okay if we take a look at the system from the inside, so we can be more on the same page? Will send you a private message for the device ID, and can continue the debug on this thread.

#42

Hi, looks like the server is only exposed to localhost, where in the code would one set the server to be on 0.0.0.0 and port 80?

#43

I am running roscore as

roscore -p 80

You cannot set the rosmaster address as 0.0.0.0 as the other ros nodes should be able to connect to the rosmaster

Do you mean that I have to manually expose the port from the dockerfile?

#44

Sorry, can you explain what do you mean by that?

The point being, is that servers run with localhost:port setup are not going to be accessible outside of the machine, while servers run with 0.0.0.0:port should be (as 0.0.0.0 just means that expose the server on all interfaces).
When I check on the host OS:

  • curl -v localhost:80 works as expected, I think, giving a 501 error as the server expects different data, but at least connects
  • curl -v 192.x.x.x:80 which is the local network address on the other hand cannot connect, as expected if the server is just running on localhost.

Thus trying to figure out where’s the discrepancy, if this assessment above is correct

In the meantime we are reading up on ROS concepts, as it’s not totally clear what supposed to be going on, and how the service is being run. Will get back to you with anything that we figure out!

#45

Reading about the starting setup, this might be the way to tell roscore to set up the right server?

export ROS_MASTER_URI=http://0.0.0.0:80/
roscore -p 80

Have you tried this? (just guessing)

#46

@imrehg I understand, so does it mean that all local host ports are not accessible from the outside?

This might be helpful in how ros works

When a ros master is running on a specific port, different nodes register their address and topics to the rosmaster.

The problem I am facing is that I want to run a node on a different device(my laptop) while the ros master is running INSIDE the container. For my ros node to run(in my laptop), I need to set the ROS_MASTER_URI to the address /port where roscore is running(which is running in the container). That way, my ros node running in the laptop can get information of the other ros nodes that are communicating with the same ros master.

For eg. you can have a ros master running in a AWS server, while my laptop and other devices run a ros node with the same ROS_MASTER_URI pointing to the address/port of the rosmaster. This way, rosnodes can talk to eachother

#47 
$ export ROS_MASTER_URI=http://0.0.0.0:80/
$ roscore -p 80
... logging to /root/.ros/log/2ea8eec4-7fac-11e8-8e7a-f8633ff9a90c/roslaunch-12f197a-2039.log
Checking log directory for disk usage. This may take awhile.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://localhost:41753/
ros_comm version 1.12.13


SUMMARY
========

PARAMETERS
 * /rosdistro: kinetic
 * /rosversion: 1.12.13

NODES

WARNING: ROS_MASTER_URI [http://0.0.0.0:80/] host is not set to this machine
auto-starting new master
process[master]: started with pid [2049]
ROS_MASTER_URI=http://localhost:80/

setting /run_id to 2ea8eec4-7fac-11e8-8e7a-f8633ff9a90c
process[rosout-1]: started with pid [2062]
started core service [/rosout]

The roscore just starts a new rosmaster on the localhost.

#48

ROS_MASTER_URI specifies which rosmaster the ROS NODES should connect to,

you have to run a roscore on some place that other nodes can connect to

#49

No, all host ports should be accessible from the outside - if the server listens on those ports and on the relevant network interfaces:

  • localhost:port means the device listens only locally, on the port, on the loopback interface
  • 0.0.0.0:port means the device listens on that port, on all interfaces (ie. ethernet, wifi, everything available
  • ip:port means the device just listens on that particular IP (belonging to an interface), and that port

Okay, I understand your situation a bit better, but then the question is still:

  • how are you running the master inside the container?
  • as much as I can tell from the docs (at http://wiki.ros.org/ROS/Tutorials/UnderstandingNodes) , roscore is Master + rosout + parameter server thus when you are running roscore, it will just start a master!
  • are all those command lines you sent above are on your laptop? We thought that’s in the container.

Based on this feedback, here are some adjusted suggestions:

  1. if the master is running inside the container, it seems it exposes the port 80 - but only on localhost. Change the master inside the container to expose on all interfaces? How do you run the master there? By running roscore inside the container as well? Because then the environment variable and command line sent above needs to be in the container start script!
  2. on your local network device, there of course ROS_MASTER_URI needs to point to the actual IP or name of the device. Once the change was applied to the container, we can adjust that as well.

So what do you think about the first part above?

#50

@imrehg I was running the rosmaster inside the container by ssh ing into it.

For the answers to the question.

  • I was only running the rosmaster inside the container, and trying to access that from the outside. To clarify,
    (inside the container)
roscore -p 80

(on the laptop)

export ROS_MASTER_URI=http://localhost:80
rostopic list

For the feedbacks:

  • For the first feedback, I will try running as a CMD instruction in the dockerfile and see if it makes a difference.
  • On the second feedback, do you mean the actual IP the IP of the host OS? for me it is a little confusing how the IP / port works with the container and the hostOS

I really appreciate you looking into this and will get back to you after applying the changes to the container.

Thanks,

#51

This above is not quite right as much as I can tell

In the container should run:

export ROS_MASTER_URI=http://0.0.0.0:80
roscore -p 80

and on the laptop should run:

export ROS_MASTER_URI=http://<master address>:80
rostopic list

where <master address> is e.g. the device’s IP.

Not sure if this would work, but the one you wrote is definitely not on the right path.

I mean the device’s local network IP. Not sure what is the difference between “actual IP” and “IP of the host OS” is in that above. Basically the device has an IP address that is reachable from the outside (e.g. the local network), and that’s the address it needs connecting to. The container here doesn’t matter and doesn’t have its own address. All the application in this (privileged, host networking) container does is listening on certain ports. The localhost vs 0.0.0.0 difference is the same on all devices, doesn’t need to be containers.

#52

@imrehg Thank you for clarifying on the network address on container / hostOS

Although I am not sure if we are on the same page about setting the rosmaster address on 0.0.0.0

As far as I know, It is NOT possible to start a rosmaster on 0.0.0.0 The ROS_MASTER_URI environment variable is referenced by the rosnodes(even that are running locally), not the rosmaster. The rosnodes use the variable to connect to the rosmaster.

If you run roscore -p 80 the rosmaster will still start on localhost:80

If you try running the command as you have suggested, you will see that it still launches a rosmaster on localhost:80

export ROS_MASTER_URI=http://0.0.0.0:80
roscore -p 80

The log of the command above is

$ export ROS_MASTER_URI=http://0.0.0.0:80/
$ roscore -p 80
... logging to /root/.ros/log/2ea8eec4-7fac-11e8-8e7a-f8633ff9a90c/roslaunch-12f197a-2039.log
Checking log directory for disk usage. This may take awhile.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://localhost:41753/
ros_comm version 1.12.13


SUMMARY
========

PARAMETERS
 * /rosdistro: kinetic
 * /rosversion: 1.12.13

NODES

WARNING: ROS_MASTER_URI [http://0.0.0.0:80/] host is not set to this machine
auto-starting new master
process[master]: started with pid [2049]
ROS_MASTER_URI=http://localhost:80/

setting /run_id to 2ea8eec4-7fac-11e8-8e7a-f8633ff9a90c

If you set ROS_MASTER_URI to 0.0.0.0:80 the rosnodes will not be able to connect to the rosmaster that is running locally

#53

Assuming we can start roscore in the container only with localhost (‘limitation’ by roscore) and we want to connect from the laptop with

export ROS_MASTER_URI=http://<master address>:80
rostopic list

Is there a easy workaround to open the ports so that they are accessible even if roscore doesn’t do the ‘right’ thing for resin?

#54

hi @jalim it’s getting really complex to untangle things here. One thing is for sure, that ROS_MASTER_URI on your laptop is not going be much help, that is always connecting to the local device (your laptop).

Would it be possible to share your code with us, so we can take a look like that?

I’m pretty sure that the master has to be able to run on not just localhost. localhost should expose things only on the local device, and in the examples there are remote connections to the master. Thus I’m still thinking there’s a configuration issue to overcome here.

#55

Hi @imrehg

To clarify:

In a standard use case without resin and just 2 standard ubuntu computers this is what I would do as a user of ROS:

On computer A:
roscore

on computer B
export ROS_MASTER_URI=http://:80
rostopic list

ROS_MASTER_URI corresponds to the address of computer A, no need to set it on computer A.
I lack the insight into the resin networking setup why this does not work per default with resin?

#56

It all depends how things are set up on the resin device, are the ports exposed to the general network for example. That’s the reason we’d like to look at the code, as it’s easier to tell. W

If it’s a single-container setup (just a Dockerfile or Dockerfile.template), then the container will run with host networking, which means it should work the same way as you describe - if something listens on a port, it will be available to connect to.

If a multicontainer setup is used ( a docker-compose.yml to set up services), then the given ports need to be explicitly exposed, or host networking needs to be set explicitly.

And if it works as you describe, the master is not run with localhost, but likely the more permissive “listen on all interfaces” (ie. 0.0.0.0 setup). But again, this is not clear how things are started. From the wiki at http://wiki.ros.org/ROS/EnvironmentVariables#ROS_MASTER_URI it says:

Great care should be taken when using localhost, as that can lead to unintended behaviors with remotely launched nodes.

Thus there’s more to consider, I think.

Just wondering, do you also have a setup that doesn’t work, that we might be able to look at and provide feedback on?

#57

@imrehg I looked back on the issue and it turned out to be a simple configuration issue.

You simply need to set the ROS_HOSTNAME and the ROS_MASTER_URI as the local network address such as

export ROS_HOSTNAME=192.168.1.116 
export ROS_MASTER_URI=http://192.168.1.116:80

Thank you for your inputs.

#58

That’s awesome, @jalim! Thanks for letting us know.

I learned a bit about ROS as well, and looking forward seeing what will you make! :slight_smile: