Network Buffer Pool

Network buffer pool allocates and manages a pool of network buffers. Users request a buffer from the pool and after its usage is over gives back to the pool. More...

Collaboration diagram for Network Buffer Pool:

Data Structures
struct	m0_net_buffer_pool_ops
struct	m0_net_buffer_pool

Enumerations
enum	{ M0_BUFFER_ANY_COLOUR = ~0, M0_NET_BUFFER_POOL_THRESHOLD = 2 }

Functions
	M0_TL_DESCR_DEFINE (m0_net_pool, "net_buffer_pool", M0_INTERNAL, struct m0_net_buffer, nb_lru, nb_magic, M0_NET_BUFFER_LINK_MAGIC, M0_NET_BUFFER_HEAD_MAGIC)
	M0_TL_DEFINE (m0_net_pool, M0_INTERNAL, struct m0_net_buffer)
static bool	pool_colour_check (const struct m0_net_buffer_pool *pool)
static bool	pool_lru_buffer_check (const struct m0_net_buffer_pool *pool)
static bool	colour_is_valid (const struct m0_net_buffer_pool *pool, uint32_t colour)
M0_INTERNAL bool	m0_net_buffer_pool_invariant (const struct m0_net_buffer_pool *pool)
M0_INTERNAL int	m0_net_buffer_pool_init (struct m0_net_buffer_pool *pool, struct m0_net_domain *ndom, uint32_t threshold, uint32_t seg_nr, m0_bcount_t seg_size, uint32_t colours, unsigned shift, bool dont_dump)
static bool	net_buffer_pool_grow (struct m0_net_buffer_pool *pool)
M0_INTERNAL int	m0_net_buffer_pool_provision (struct m0_net_buffer_pool *pool, uint32_t buf_nr)
static void	buffer_remove (struct m0_net_buffer_pool *pool, struct m0_net_buffer *nb)
M0_INTERNAL void	m0_net_buffer_pool_fini (struct m0_net_buffer_pool *pool)
M0_INTERNAL void	m0_net_buffer_pool_lock (struct m0_net_buffer_pool *pool)
M0_INTERNAL bool	m0_net_buffer_pool_is_locked (const struct m0_net_buffer_pool *pool)
M0_INTERNAL bool	m0_net_buffer_pool_is_not_locked (const struct m0_net_buffer_pool *pool)
M0_INTERNAL void	m0_net_buffer_pool_unlock (struct m0_net_buffer_pool *pool)
M0_INTERNAL struct m0_net_buffer *	m0_net_buffer_pool_get (struct m0_net_buffer_pool *pool, uint32_t colour)
M0_INTERNAL void	m0_net_buffer_pool_put (struct m0_net_buffer_pool *pool, struct m0_net_buffer *buf, uint32_t colour)
M0_INTERNAL bool	m0_net_buffer_pool_prune (struct m0_net_buffer_pool *pool)

Detailed Description

Network buffer pool allocates and manages a pool of network buffers. Users request a buffer from the pool and after its usage is over gives back to the pool.

It provides support for a pool of network buffers involving no higher level interfaces than the network module itself. It is associated with a single network domain. Non-blocking interfaces are available to get and put network buffers. Call-backs are provided to announce when the pool is non-empty or low on free buffers.

Upon receiving the not_empty call back user can put back buffers which are not in use into the pool.

The "coloured" variant of the get operation is done by returning the most recently used buffer that is associated with a specific colour (transfer machine), or if none such are found, the least recently used buffer from the pool, if any.

Pool is protected by a lock, to get or put a buffer into the pool user must acquire the lock and release the lock once its usage is over.

To finalize the pool all the buffers must be returned back to the pool (i.e number of free buffers must be equal to the total number of buffers).

To describe a typical buffer pool usage pattern, suppose that one wants a buffer pool of 10, size of each segment is 1024, number of segments is 64 and threshold is 10.

First, user needs to provide m0_net_buffer_pool_ops:

struct m0_net_buffer_pool_ops b_ops = {
 .nbpo_not_empty       = notempty,
 .nbpo_below_threshold = low,
};

• Then, buffer pool needs to be assigned to a network domain and initialized with above values:

  struct m0_net_buffer_pool bp;
  struct m0_net_xprt *xprt;
  ...
  bp.nbp_ops = &b_ops;
  rc = m0_net_buffer_pool_init(&bp, bp.nbp_ndom, 10, 64, 4096, 10, ...);
  ...

• Now, to add buffers into the pool need to acquire the lock and then specify the number of buffers to be added:

  m0_net_buffer_pool_lock(&bp);
  m0_net_buffer_pool_provision(&bp, 10);
  m0_net_buffer_pool_unlock(&bp);

  • To add a buffer in the pool:

    m0_net_buffer_pool_lock(&bp);
    m0_net_buffer_pool_provision(&bp, 1);
    m0_net_buffer_pool_unlock(&bp);

  • To get a buffer from the pool: To use any colour for the buffer variable colour should be M0_BUFFER_ANY_COLOUR.

    m0_net_buffer_pool_lock(&bp);
    nb = m0_net_buffer_pool_get(&bp, colour);
    if (nb != NULL)
     "Use the buffer"
    else
     "goto sleep until buffer is available"
    m0_net_buffer_pool_unlock(&bp);

• To put back the buffer in the pool: To use any colour for the buffer variable colour should be M0_BUFFER_ANY_COLOUR.

  m0_net_buffer_pool_lock(&bp);
  m0_net_buffer_pool_put(&bp, nb, colour);
  m0_net_buffer_pool_unlock(&bp);

  • To remove a buffer from the pool:

    m0_net_buffer_pool_lock(&bp);
    m0_net_buffer_pool_prune(&bp);
    m0_net_buffer_pool_unlock(&bp);

  • To finalize the pool:

    m0_net_buffer_pool_fini(&bp);

  See also

  Also see m0_net_tm_pool_attach() and Auo-Provisioning of Receive Message Queue Buffers.

Enumeration Type Documentation

anonymous enum

anonymous enum

Enumerator
M0_BUFFER_ANY_COLOUR
M0_NET_BUFFER_POOL_THRESHOLD

Definition at line 140 of file buffer_pool.h.

Function Documentation

buffer_remove()

static void buffer_remove ( struct m0_net_buffer_pool *  pool, struct m0_net_buffer *  nb  )

static

It removes the given buffer from the pool

Definition at line 141 of file buffer_pool.c.

Here is the call graph for this function:

Here is the caller graph for this function:

colour_is_valid()

static bool colour_is_valid ( const struct m0_net_buffer_pool *  pool, uint32_t  colour  )

static

Definition at line 208 of file buffer_pool.c.

Here is the caller graph for this function:

m0_net_buffer_pool_fini()

M0_INTERNAL void m0_net_buffer_pool_fini

(

struct m0_net_buffer_pool *

pool

)

Finalizes a buffer pool.

Precondition

m0_net_buffer_pool_is_not_locked(pool)

Definition at line 154 of file buffer_pool.c.

Here is the call graph for this function:

Here is the caller graph for this function:

m0_net_buffer_pool_get()

M0_INTERNAL struct m0_net_buffer * m0_net_buffer_pool_get	(	struct m0_net_buffer_pool *	pool,
		uint32_t	colour
	)

Gets a buffer from the pool. If the colour is specified (i.e non zero) and the corresponding coloured list is not empty then the buffer is taken from the head of this list. Otherwise the buffer is taken from the head of the per buffer pool list.

Precondition

m0_net_buffer_pool_is_locked(pool)

colour == M0_BUFFER_ANY_COLOUR || colour < pool->nbp_colours_nr

Postcondition

ergo(result != NULL, result->nb_flags & M0_NET_BUF_REGISTERED)

ergo(result != NULL, result->nb_pool == pool)

Definition at line 215 of file buffer_pool.c.

Here is the call graph for this function:

Here is the caller graph for this function:

m0_net_buffer_pool_init()

M0_INTERNAL int m0_net_buffer_pool_init	(	struct m0_net_buffer_pool *	pool,
		struct m0_net_domain *	ndom,
		uint32_t	threshold,
		uint32_t	seg_nr,
		m0_bcount_t	seg_size,
		uint32_t	colours,
		unsigned	shift,
		bool	dont_dump
	)

Initializes fields of a buffer pool and tlist, which are used to populate the pool using m0_net_buffer_pool_provision().

Precondition

ndom != NULL

Parameters

pool	Pool to initialize.
ndom	Network domain to associate with the pool.
threshold	Number of buffer below which to notify the user.
seg_nr	Number of segments in each buffer.
colours	Number of colours in the pool.
seg_size	Size of each segment in a buffer.
shift	Alignment needed for network buffers.

Precondition

seg_nr <= m0_net_domain_get_max_buffer_segments(ndom) && seg_size <= m0_net_domain_get_max_buffer_segment_size(ndom)

Definition at line 82 of file buffer_pool.c.

Here is the call graph for this function:

Here is the caller graph for this function:

m0_net_buffer_pool_invariant()

M0_INTERNAL bool m0_net_buffer_pool_invariant

(

const struct m0_net_buffer_pool *

pool

)

Checks the buffer pool.

Definition at line 50 of file buffer_pool.c.

Here is the call graph for this function:

Here is the caller graph for this function:

m0_net_buffer_pool_is_locked()

M0_INTERNAL bool m0_net_buffer_pool_is_locked

(

const struct m0_net_buffer_pool *

pool

)

Check whether buffer pool is locked or not.

Definition at line 191 of file buffer_pool.c.

Here is the call graph for this function:

Here is the caller graph for this function:

m0_net_buffer_pool_is_not_locked()

M0_INTERNAL bool m0_net_buffer_pool_is_not_locked

(

const struct m0_net_buffer_pool *

pool

)

Returns true when buffer pool is not locked.

Definition at line 197 of file buffer_pool.c.

Here is the call graph for this function:

Here is the caller graph for this function:

m0_net_buffer_pool_lock()

M0_INTERNAL void m0_net_buffer_pool_lock

(

struct m0_net_buffer_pool *

pool

)

Acquires the lock on buffer pool.

Definition at line 186 of file buffer_pool.c.

Here is the call graph for this function:

Here is the caller graph for this function:

m0_net_buffer_pool_provision()

M0_INTERNAL int m0_net_buffer_pool_provision	(	struct m0_net_buffer_pool *	pool,
		uint32_t	buf_nr
	)

It adds the buf_nr buffers in the buffer pool. Suppose to add 10 items to the pool, m0_net_buffer_pool_provision(pool, 10) can be used.

Precondition

m0_net_buffer_pool_is_locked(pool)

seg_size > 0 && seg_nr > 0 && buf_nr > 0

pool->nbp_ndom != NULL

Parameters

pool	Pool to provision.
buf_nr	Number of buffers to be added in the pool.

Returns

result number of buffers it managed to allocate.

Definition at line 125 of file buffer_pool.c.

Here is the call graph for this function:

Here is the caller graph for this function:

m0_net_buffer_pool_prune()

M0_INTERNAL bool m0_net_buffer_pool_prune

(

struct m0_net_buffer_pool *

pool

)

Removes a buffer from the pool to prune it.

Precondition

m0_net_buffer_pool_is_locked(pool)

Definition at line 310 of file buffer_pool.c.

Here is the call graph for this function:

Here is the caller graph for this function:

m0_net_buffer_pool_put()

M0_INTERNAL void m0_net_buffer_pool_put	(	struct m0_net_buffer_pool *	pool,
		struct m0_net_buffer *	buf,
		uint32_t	colour
	)

Puts the buffer back to the pool. If the colour is specfied then the buffer is put at the head of corresponding coloured list and also put at the tail of the global list.

Precondition

m0_net_buffer_pool_is_locked(pool)

colour == M0_BUFFER_ANY_COLOUR || colour < pool->nbp_colours_nr

pool->nbp_ndom == buf->nb_dom

(buf->nb_flags & M0_NET_BUF_REGISTERED) && !(buf->nb_flags & M0_NET_BUF_QUEUED)

Definition at line 243 of file buffer_pool.c.

Here is the call graph for this function:

Here is the caller graph for this function:

m0_net_buffer_pool_unlock()

M0_INTERNAL void m0_net_buffer_pool_unlock

(

struct m0_net_buffer_pool *

pool

)

Releases the lock on buffer pool.

Definition at line 203 of file buffer_pool.c.

Here is the call graph for this function:

Here is the caller graph for this function:

M0_TL_DEFINE()

M0_TL_DEFINE	(	m0_net_pool	,
		M0_INTERNAL	,
		struct m0_net_buffer
	)

M0_TL_DESCR_DEFINE()

M0_TL_DESCR_DEFINE	(	m0_net_pool	,
		"net_buffer_pool"	,
		M0_INTERNAL	,
		struct m0_net_buffer	,
		nb_lru	,
		nb_magic	,
		M0_NET_BUFFER_LINK_MAGIC	,
		M0_NET_BUFFER_HEAD_MAGIC
	)

Descriptor for the tlist of buffers.

net_buffer_pool_grow()

static bool net_buffer_pool_grow ( struct m0_net_buffer_pool *  pool )

static

Adds a buffer to the pool to increase the capacity.

Precondition

m0_net_buffer_pool_is_locked(pool)

Definition at line 270 of file buffer_pool.c.

Here is the call graph for this function:

Here is the caller graph for this function:

pool_colour_check()

static bool pool_colour_check ( const struct m0_net_buffer_pool *  pool )

static

Definition at line 68 of file buffer_pool.c.

Here is the caller graph for this function:

pool_lru_buffer_check()

static bool pool_lru_buffer_check ( const struct m0_net_buffer_pool *  pool )

static

Definition at line 75 of file buffer_pool.c.

Here is the caller graph for this function: