[erlang-questions] Reconnect to nodes

[Roberto Ostinelli]

Dear list,
A simple question: am I correct that, when a node is removed because of a
net split, you need to have your own application logic to reconnect to it,
and nothing in the VM will try doing that for you?

Let me show you an example. I have two nodes: ***@127.0.0.1 and ***@127.0.0.1
that are connected to each other:

(***@127.0.0.1)1> nodes().
['***@127.0.0.1']

On node 2 I listen for nodedown events of node 1:

(***@127.0.0.1)1> monitor_node('***@127.0.0.1', true).
true

On node 1, I simulate a net splits with the best option I've found until
now, i.e suspending the net_kernel process:

(***@127.0.0.1)2> sys:suspend(net_kernel).
ok

After ~60 seconds on node 2 I get:

=ERROR REPORT==== 22-Apr-2016::11:28:21 ===
** Node '***@127.0.0.1' not responding **
** Removing (timedout) connection **
(***@127.0.0.1)2> flush().
Shell got {nodedown,'***@127.0.0.1'}

Now the two nodes are disconnected:

(***@127.0.0.1)3> nodes().
[]

(***@127.0.0.1)3> nodes().
[]

Even when I resume the net_kernel process:

(***@127.0.0.1)4> sys:resume(net_kernel).
ok

The nodes do not reconnect:

(***@127.0.0.1)5> nodes().
[]

I'm ok with this, though I would like to confirm that my understanding is
correct.
If so, does everyone just implement some standard connection manager that
does only reconnections?

Thank you,
r.

[Boroska András]

Hi Roberto,

Your assumptions are correct. Note, however, that rpc calls via the rpc
module will reconnect the nodes automatically which is a nice convenience.

Br,
Andras

[Serge Aleynikov]

Roberto,

This is the expected behavior. Note that the nodes will automatically
reconnect by default when either node has a process that sends a message to
a process on the remote node. This reconnection behavior can be modified
by setting the kernel's 'dist_auto_connect' option.

Other applications (such as mnesia) may require custom recovery from a
network split, which is one of the reasons why automatic reconnection may
not be desirable.

Regards,

Serge

Post by Roberto Ostinelli
Dear list,
A simple question: am I correct that, when a node is removed because of a
net split, you need to have your own application logic to reconnect to it,
and nothing in the VM will try doing that for you?
true
On node 1, I simulate a net splits with the best option I've found until
ok
=ERROR REPORT==== 22-Apr-2016::11:28:21 ===
** Removing (timedout) connection **
[]
[]
ok
[]
I'm ok with this, though I would like to confirm that my understanding is
correct.
If so, does everyone just implement some standard connection manager that
does only reconnections?
Thank you,
r.
_______________________________________________
erlang-questions mailing list
http://erlang.org/mailman/listinfo/erlang-questions

[Roberto Ostinelli]

Thank you both for confirming.
Hence I guess some kind of a basic reconnection manager might be helpful
here.

Thanks!

r.

Post by Boroska András
Roberto,
This is the expected behavior. Note that the nodes will automatically
reconnect by default when either node has a process that sends a message to
a process on the remote node. This reconnection behavior can be modified
by setting the kernel's 'dist_auto_connect' option.
Other applications (such as mnesia) may require custom recovery from a
network split, which is one of the reasons why automatic reconnection may
not be desirable.
Regards,
Serge

Post by Roberto Ostinelli
Dear list,
A simple question: am I correct that, when a node is removed because of a
net split, you need to have your own application logic to reconnect to it,
and nothing in the VM will try doing that for you?
true
On node 1, I simulate a net splits with the best option I've found until
ok
=ERROR REPORT==== 22-Apr-2016::11:28:21 ===
** Removing (timedout) connection **
[]
[]
ok
[]
I'm ok with this, though I would like to confirm that my understanding is
correct.
If so, does everyone just implement some standard connection manager that
does only reconnections?
Thank you,
r.
_______________________________________________
erlang-questions mailing list
http://erlang.org/mailman/listinfo/erlang-questions

[Serge Aleynikov]

You can take a look at this app that handles detection of network splits
and auto-reconnecting:

https://github.com/saleyn/netmon

Serge

Post by Roberto Ostinelli
Thank you both for confirming.
Hence I guess some kind of a basic reconnection manager might be helpful
here.
Thanks!
r.

Post by Boroska András
Roberto,
This is the expected behavior. Note that the nodes will automatically
reconnect by default when either node has a process that sends a message to
a process on the remote node. This reconnection behavior can be modified
by setting the kernel's 'dist_auto_connect' option.
Other applications (such as mnesia) may require custom recovery from a
network split, which is one of the reasons why automatic reconnection may
not be desirable.
Regards,
Serge

Post by Roberto Ostinelli
Dear list,
A simple question: am I correct that, when a node is removed because of
a net split, you need to have your own application logic to reconnect to
it, and nothing in the VM will try doing that for you?
true
On node 1, I simulate a net splits with the best option I've found until
ok
=ERROR REPORT==== 22-Apr-2016::11:28:21 ===
** Removing (timedout) connection **
[]
[]
ok
[]
I'm ok with this, though I would like to confirm that my understanding
is correct.
If so, does everyone just implement some standard connection manager
that does only reconnections?
Thank you,
r.
_______________________________________________
erlang-questions mailing list
http://erlang.org/mailman/listinfo/erlang-questions

[Roberto Ostinelli]

Thanks!

[Michael Truog]

This is in CloudI (in cloudi_core) as shown at http://cloudi.org/api.html#2_nodes_set with reconnect_start and reconnect_delay determining the interval for checking the node connections. If you need help using cloudi_core, that is at https://github.com/CloudI/CloudI/tree/master/examples/hello_world5 .

Post by Roberto Ostinelli
Thank you both for confirming.
Hence I guess some kind of a basic reconnection manager might be helpful here.
Thanks!
r.
Roberto,
This is the expected behavior. Note that the nodes will automatically reconnect by default when either node has a process that sends a message to a process on the remote node. This reconnection behavior can be modified by setting the kernel's 'dist_auto_connect' option.
Other applications (such as mnesia) may require custom recovery from a network split, which is one of the reasons why automatic reconnection may not be desirable.
Regards,
Serge
Dear list,
A simple question: am I correct that, when a node is removed because of a net split, you need to have your own application logic to reconnect to it, and nothing in the VM will try doing that for you?
true
ok
=ERROR REPORT==== 22-Apr-2016::11:28:21 ===
** Removing (timedout) connection **
[]
[]
ok
[]
I'm ok with this, though I would like to confirm that my understanding is correct.
If so, does everyone just implement some standard connection manager that does only reconnections?
Thank you,
r.
_______________________________________________
erlang-questions mailing list
http://erlang.org/mailman/listinfo/erlang-questions
_______________________________________________
erlang-questions mailing list
http://erlang.org/mailman/listinfo/erlang-questions