Redundant Configuration Client (rconfc)

Redundant configuration client library – rconfc – provides an interface for Motr applications working in cluster with multiple configuration servers (confd).

Rconfc supplements confc functionality and processes situations when cluster dynamically changes configuration or loses connection to some of confd servers.

• Data Structures
• Subroutines
• Initialisation, finalisation and reading data
• Detailed Functional Specification

---

Data Structures

• m0_rconfc — an instance of redundant configuration client.

m0_rconfc structure includes a dedicated m0_confc instance m0_rconfc::rc_confc, which is intended for conventional configuration reading.

Beside of that, it contains two lists of confc instances, the one used for communication with multiple confd servers in cluster environment during configuration version election, and the other one used for keeping list of discovered confd servers that run the same version that has won in the most recent election.

---

Subroutines

• m0_rconfc_init() initialises rconfc internals.
• m0_rconfc_start() carries out configuration version election. During election all known confd servers are polled for a number of version they run. The elected version, i.e. the version having the quorum, is used in further configuration reading operation. In case quorum is reached, m0_rconfc connects the dedicated m0_rconfc::rc_confc to one of the confd servers from active list.
• m0_rconfc_start_sync() is synchronous version of m0_rconfc_start().
• m0_rconfc_stop() disconnects dedicated confc and release other internal resources.
• m0_rconfc_stop_sync() is synchronous version of m0_rconfc_stop().
• m0_rconfc_fini() finalises rconfc instance.
• m0_rconfc_ver_max_read() reports the maximum configuration version number the confd servers responded with during the most recent version election.

---

Initialisation, finalisation and reading data

To access configuration data the configuration consumer is allowed to follow two different scenarios:

• reading with standalone configuration client (confc)
• reading with confc exposed from redundant configuration client (rconfc)

In the first case the consumer reads data from particular confd server with no regard to whether its version reached quorum or not.

In the second case the consumer is guaranteed to read data from configuration that reached the quorum in the cluster, i.e. the most recent consistent one. As well, the consumer is guaranteed to be notified about the fact of configuration change, to let accommodate with the change if required. In addition, the ability to switch among confd servers in case of network error transparently for the consumer sufficiently strengthens the reading reliability, and due to this, cluster robustness in whole.

In the latter case the consumer initialises and starts m0_rconfc instance instead of m0_confc, and performs all further reading via m0_rconfc::rc_confc.

Also, the consumer using rconfc doesn't provide any confd addresses explicitly. The list of confd servers and other related information is centralised and is maintained by HA service. Rconfc queries this information on startup.

Example:

#include "conf/rconfc.h"
#include "conf/obj.h"

struct m0_sm_group    *grp = ...;
struct m0_rpc_machine *rpcm = ...;
struct m0_rconfc       rconfc;

void conf_exp_cb(struct m0_rconfc *rconfc)
{
        ...   clean up all local copies of configuration data    ...
        ... close all m0_conf_obj instances retained by consumer ...
}

startup(const struct m0_fid *profile, ...)
{
        rc = m0_rconfc_init(&rconfc, grp, rpcm, conf_exp_cb);
        if (rc == 0) {
             rc = m0_rconfc_start_sync(&rconfc);
             if (rc != 0)
                     m0_rconfc_stop_sync(&rconfc);
        }
        ...
}

... Access configuration objects, using confc interfaces. ...

reading(...)
{
       struct m0_conf_obj *obj;
       struct m0_confc    *confc = &rconfc.rc_confc;
       int                 rc;
       rc = m0_confc_open_sync(&obj, confc->cc_root, ... );
       ... read object data ...
       m0_confc_close(obj);
}

shutdown(...)
{
        m0_rconfc_stop_sync(&rconfc);
        m0_rconfc_fini(&rconfc);
}

For asynchronous rconfc start/stop please see details of m0_rconfc_start_sync(), m0_rconfc_stop_sync() implementation or documentation for m0_rconfc_start(), m0_rconfc_stop().

Note

Consumer is allowed to use any standard approach for opening configuration and traversing directories in accordance with confc specification. Using rconfc puts no additional limitation other than stipulated by proper m0_rconfc_init().

See also

Detailed Functional Specification