#include <conn.h>

Collaboration diagram for m0_rpc_conn:

Data Fields
uint64_t	c_sender_id

struct m0_uint128	c_uuid

uint64_t	c_flags

struct m0_rpc_machine *	c_rpc_machine

const struct m0_rpc_conn_ha_cfg *	c_ha_cfg

uint64_t	c_ha_attempts

struct m0_sm_timer	c_ha_timer

struct m0_tlink	c_link

uint64_t	c_nr_sessions

struct m0_tl	c_sessions

struct m0_tl	c_item_sources

struct m0_rpc_chan *	c_rpcchan

struct m0_sm	c_sm

struct m0_sm_ast	c_ast

bool	c_ast_in_progress

uint64_t	c_magic

struct m0_clink	c_ha_clink

struct m0_clink	c_conf_exp_clink

struct m0_clink	c_conf_ready_clink

struct m0_fid	c_svc_fid

Detailed Description

A rpc connection identifies a sender to the receiver. It acts as a parent object within which sessions are created. rpc connection has two identifiers.

UUID: Uniquely Identifies of the rpc connection globally within the cluster. UUID is generated by sender.
SenderID: A sender id is assigned by receiver. Sender Id uniquely identifies a rpc connection on receiver side. Same sender has different sender_id to communicate with different receiver.

UUID being larger in size compared to SenderID, it is efficient to use sender id to locate rpc connection object.

m0_rpc_machine maintains two lists of m0_rpc_conn

rm_outgoing_conns: list of m0_rpc_conn objects for which this node is sender
rm_incoming_conns: list of m0_rpc_conn object for which this node is receiver

Instance of m0_rpc_conn stores a list of all sessions currently active with the service.

At the time of creation of a m0_rpc_conn, a "special" session with SESSION_ID_0 is also created. It is special in the sense that it is "hand-made" and there is no need to communicate to receiver in order to create this session. Receiver assumes that there always exists a session 0 for each rpc connection. Receiver creates session 0 while creating the rpc connection itself. Session 0 is required to send special fops like

conn_establish or conn_terminate FOP
session_establish or session_terminate FOP.

State transition diagram:

                                  | m0_rpc_conn_init()
 m0_rpc_conn_establish() != 0     V
       +---------------------INITIALISED
       |                          |
       |                          |  m0_rpc_conn_establish()
       |                          |
       |                          V
       +---------------------- CONNECTING
       | time-out ||              |
       |     reply.rc != 0        | m0_rpc_conn_establish_reply_received() &&
       |                          |    reply.rc == 0
       V                          |
     FAILED                       |
       |  ^                       V
       |  |                    ACTIVE
       |  |                       |
       |  |                       | m0_rpc_conn_terminate()
       |  | failed || timeout     |
       |  |                       V
       |  +-------------------TERMINATING
       |                          |
       |                          | m0_rpc_conn_terminate_reply_received() &&
       |                          |              rc== 0
       |                          V
       |                        TERMINATED
       |                          |
       |m0_rpc_conn_fini()        V  m0_rpc_conn_fini()
       +--------------------->FINALISED

Liveness and Concurrency:

Sender side allocation and deallocation of m0_rpc_conn object is entirely handled by user (m0_rpc_conn object is not reference counted).
On receiver side, user is not expected to allocate or deallocate m0_rpc_conn objects explicitly.
Receiver side m0_rpc_conn object will be instantiated in response to rpc connection establish request and is deallocated while terminating the rpc connection.
Access to m0_rpc_conn is synchronized with conn->c_rpc_machine->rm_sm_grp.s_lock.

Typical sequence of API execution Note: error checking is omitted.

// ALLOCATE CONN
struct m0_rpc_conn *conn;
M0_ALLOC_PTR(conn);
// INITIALISE CONN
rc = m0_rpc_conn_init(conn, tgt_end_point, rpc_machine);
M0_ASSERT(ergo(rc == 0, conn_state(conn) == M0_RPC_CONN_INITIALISED));
// ESTABLISH RPC CONNECTION
rc = m0_rpc_conn_establish(conn);
if (rc != 0) {
      // some error occured. Cannot establish connection.
      // handle the situation and return
}
// WAIT UNTIL CONNECTION IS ESTABLISHED
rc = m0_rpc_conn_timedwait(conn, M0_BITS(M0_RPC_CONN_ACTIVE,
                                   M0_RPC_CONN_FAILED),
                     absolute_timeout);
if (rc == 0) {
      if (conn_state(conn) == M0_RPC_CONN_ACTIVE)
        // connection is established and is ready to be used
      else
        // connection establishing failed
} else {
      // timeout
}
// Assuming connection is established.
// Create one or more sessions using this connection. @see m0_rpc_session
// TERMINATING CONNECTION
// Make sure that all the sessions that were created on this connection are
// terminated
M0_ASSERT(conn->c_nr_sessions == 0);
rc = m0_rpc_conn_terminate(conn);
// WAIT UNTIL CONNECTION IS TERMINATED
rc = m0_rpc_conn_timedwait(conn, M0_BITS(M0_RPC_CONN_TERMINATED,
                                   M0_RPC_CONN_FAILED),
                     absolute_timeout);
if (rc == 0) {
      if (conn_state(conn) == M0_RPC_CONN_TERMINATED)
        // conn is successfully terminated
      else
        // conn terminate has failed
} else {
      // timeout
}
// assuming conn is terminated
m0_rpc_conn_fini(conn);
m0_free(conn);

On receiver side, user is not expected to call any of these APIs. Receiver side rpc-layer will internally allocate/deallocate and manage all the state transitions of conn internally.

m0_rpc_conn:c_ha_timer is armed when an rpc item related to the connection was sent or timed out. If the connection is subscribed to m0_conf_service object and the timer callback is triggered, Motr sends m0_ha_msg to HA about a problem. The message consists of one field that indicates m0_ha_obj_state of the service.

HA is notified about M0_NC_TRANSIENT state in case of item resend or timeout, and M0_NC_ONLINE in the case when a reply is received on the connection. When Motr sends message about M0_NC_TRANSIENT state, the flag RCF_TRANSIENT_SENT is set for the connection. The flag is checked when the timer callback is triggered again, and in case it was already set, then Motr doesn't notify HA repeatedly. The flag is cleared when the reply is received for an item and the service is in M0_NC_TRANSIENT state. This way Motr notifies HA about M0_NC_TRANSIENT state only once per connection trouble no matter how many items are sent over the connection. It is able to send message about M0_NC_TRANSIENT state only when Halon switches the service to this state and then back to M0_NC_ONLINE state.

Definition at line 267 of file conn.h.

Field Documentation

◆ c_ast

struct m0_sm_ast c_ast

Finalises and frees m0_rpc_conn when conn terminate reply is sent over network. Posted in: conn_terminate_reply_sent_cb() Invokes: conn_cleanup_ast() callback

Definition at line 330 of file conn.h.

◆ c_ast_in_progress

bool c_ast_in_progress

Used to prevent duplicate posting of ast

Definition at line 333 of file conn.h.

◆ c_conf_exp_clink

struct m0_clink c_conf_exp_clink

conf cache expired clink

Definition at line 342 of file conn.h.

◆ c_conf_ready_clink

struct m0_clink c_conf_ready_clink

conf cache ready clink

Definition at line 345 of file conn.h.

◆ c_flags

uint64_t c_flags

See also: m0_rpc_conn_flags for list of flags

Definition at line 275 of file conn.h.

◆ c_ha_attempts

uint64_t c_ha_attempts

Indicates how many attempts to notify HA occurred

Definition at line 284 of file conn.h.

◆ c_ha_cfg

const struct m0_rpc_conn_ha_cfg* c_ha_cfg

Configuration of HA notifications mechanism.

Definition at line 281 of file conn.h.

◆ c_ha_clink

struct m0_clink c_ha_clink

HA notification clink

Definition at line 339 of file conn.h.

◆ c_ha_timer

struct m0_sm_timer c_ha_timer

HA notify timer.

Invokes item_ha_timer_cb() after every c_ha_cfg->rchc_ha_interval.

Definition at line 291 of file conn.h.

◆ c_item_sources

struct m0_tl c_item_sources

List of m0_rpc_item_source instances. List link: m0_rpc_item_source::ris_tlink List descriptor: item_source

Definition at line 314 of file conn.h.

◆ c_link

struct m0_tlink c_link

list_link to put m0_rpc_conn in either m0_rpc_machine::rm_incoming_conns or m0_rpc_machine::rm_outgoing_conns. List descriptor: rpc_conn

Definition at line 298 of file conn.h.

◆ c_magic

uint64_t c_magic

M0_RPC_CONN_MAGIC

Definition at line 336 of file conn.h.

◆ c_nr_sessions

uint64_t c_nr_sessions

Counts number of sessions (excluding session 0)

Definition at line 301 of file conn.h.

◆ c_rpc_machine

struct m0_rpc_machine* c_rpc_machine

rpc_machine with which this conn is associated

Definition at line 278 of file conn.h.

◆ c_rpcchan

struct m0_rpc_chan* c_rpcchan

Identifies destination of this connection.

Definition at line 317 of file conn.h.

◆ c_sender_id

uint64_t c_sender_id

Sender ID, unique on receiver

Definition at line 269 of file conn.h.

◆ c_sessions

struct m0_tl c_sessions

List of all the sessions created under this rpc connection. m0_rpc_session objects are placed in this list using m0_rpc_session::s_link. List descriptor: session

Definition at line 308 of file conn.h.

◆ c_sm

struct m0_sm c_sm

RPC connection state machine

See also: m0_rpc_conn_state, conn_conf

Definition at line 322 of file conn.h.

◆ c_svc_fid

struct m0_fid c_svc_fid

Fid of c_svc_obj

Definition at line 350 of file conn.h.

◆ c_uuid

struct m0_uint128 c_uuid

Globally unique ID of rpc connection

Definition at line 272 of file conn.h.

The documentation for this struct was generated from the following file:

rpc/conn.h

Data Fields