LNet Transport Core API

• API Overview
• API Data Structures
• Subroutines
• LNet Transport Core Interfaces

API Overview

The LNet Transport Core presents an address space agnostic API to the LNet Transport layer. These interfaces are declared in the file net/lnet/lnet_core.h.

The interface is implemented differently in the kernel and in user space. The kernel interface interacts directly with LNet; the user space interface uses a device driver to communicate with its kernel counterpart and uses shared memory to avoid event data copy.

The Core API offers no callback mechanism. Instead, the transport must poll for events. Typically this is done on one or more dedicated threads, which exhibit the desired processor affiliation required by the higher software layers.

The following sequence diagram illustrates the typical operational flow:

API Data Structures

The API requires that the transport application maintain API defined shared data for various network related objects:

• nlx_core_domain
• nlx_core_transfer_mc
• nlx_core_buffer

The sharing takes place between the transport layer and the core layer. This will span the kernel and user address space boundary when using the user space transport.

These shared data structures should be embedded in the transport application's own private data. This requirement results in an initialization call pattern that takes a pointer to the standard network layer data structure concerned and a pointer to the API's data structure.

Subsequent calls to the API only pass the API data structure pointer. The API data structure must be eventually finalized.

Subroutines

The API subroutines are described in LNet Transport Core Interfaces. The subroutines are categorized as follows:

• Initialization, finalization, cancellation and query subroutines:
  • nlx_core_buf_deregister()
  • nlx_core_buf_register()
  • nlx_core_dom_fini()
  • nlx_core_dom_init()
  • nlx_core_get_max_buffer_segment_size()
  • nlx_core_get_max_buffer_size()
  • nlx_core_tm_start()
  • nlx_core_tm_stop()
  These interfaces have names roughly similar to the associated m0_net_xprt_ops method from which they are intended to be directly or indirectly invoked. Note that there are no equivalents for the xo_tm_init(), xo_tm_fini() and xo_tm_confine() calls.
• End point address parsing subroutines:
  • nlx_core_ep_addr_decode()
  • nlx_core_ep_addr_encode()
• Buffer operation related subroutines:
  • nlx_core_buf_active_recv()
  • nlx_core_buf_active_send()
  • nlx_core_buf_del()
  • nlx_core_buf_msg_recv()
  • nlx_core_buf_msg_send()
  • nlx_core_buf_passive_recv()
  • nlx_core_buf_passive_send()
  • nlx_core_buf_desc_encode()
  • nlx_core_buf_desc_decode()
  The buffer operation initiation calls are all invoked in the context of the m0_net_buffer_add() subroutine. All operations are immediately initiated in the Lustre LNet kernel module, though results will be returned asynchronously through buffer events.
• Event processing calls:
  • nlx_core_buf_event_wait()
  • nlx_core_buf_event_get()
  The API assumes that only a single transport thread will be used to process events.

Invocation of the buffer operation initiation subroutines and the nlx_core_buf_event_get() subroutine should be serialized.

See also

LNet Transport Kernel Core DLD

LNet Transport User Space Core DLD

LNet Transport Device DLD