Data Structures | |
| struct | m0_layout_domain |
| struct | m0_layout |
| struct | m0_layout_ops |
| struct | m0_layout_type |
| struct | m0_layout_type_ops |
| struct | m0_layout_enum |
| struct | m0_layout_enum_ops |
| struct | m0_layout_enum_type |
| struct | m0_layout_enum_type_ops |
| struct | m0_striped_layout |
| struct | m0_layout_instance |
| struct | m0_layout_instance_ops |
| struct | m0_layout_rec |
| struct | m0_layout_plan |
| struct | m0_layout_plop |
| struct | m0_layout_plop_rel |
| struct | m0_layout_plop_ops |
| struct | m0_layout_inout_plop |
| struct | m0_layout_io_plop |
| struct | m0_layout_index_plop |
| struct | m0_layout_fun_plop |
Macros | |
| #define | M0_TRACE_SUBSYSTEM M0_TRACE_SUBSYS_LAYOUT |
| #define | M0_TRACE_SUBSYSTEM M0_TRACE_SUBSYS_LAYOUT |
Enumerations | |
| enum | { LNET_MAX_PAYLOAD = 1 << 20 } |
| enum | m0_layout_xcode_op { M0_LXO_BUFFER_OP, M0_LXO_DB_LOOKUP, M0_LXO_DB_ADD, M0_LXO_DB_UPDATE, M0_LXO_DB_DELETE } |
| enum | { M0_LAYOUT_TYPE_MAX = 32, M0_LAYOUT_ENUM_TYPE_MAX = 32 } |
| enum | { LID_NONE = 0, DEFAULT_DB_FLAG = 0, LDB_MAX_INLINE_COB_ENTRIES = 20, L_TABLE_INIT_ERR = -501, LTO_REG_ERR = -502, LETO_REG_ERR = -503, LO_DECODE_ERR = -504, LO_ENCODE_ERR = -505, L_TABLE_UPDATE_ERR = -506 } |
| enum | m0_layout_plop_type { M0_LAT_IN_WRITE, M0_LAT_READ, M0_LAT_WRITE, M0_LAT_GET, M0_LAT_PUT, M0_LAT_FUN, M0_LAT_OUT_READ, M0_LAT_DONE, M0_LAT_NR } |
| enum | m0_layout_plop_state { M0_LPS_INIT, M0_LPS_STARTED, M0_LPS_DONE } |
| enum | { M0_LAYOUT_PLOT_ANYCOLOUR = ~0ULL } |
Variables | |
| struct m0_layout_type | m0_pdclust_layout_type |
| struct m0_layout_enum_type | m0_linear_enum_type |
| static const struct m0_bob_type | layout_bob |
| static const struct m0_bob_type | enum_bob |
| static const struct m0_bob_type | layout_instance_bob |
Detailed Description
Terminology
- Layout
A 'layout' is an attribute of a file. It maps a file onto a set of network resources. viz. component objects. - Layout user count
Layout user count is the number of users associated with a particular layout. For example, files using that layout, other composite layouts using that layout.- User count of a layout does not have any impact on the liveness of an in-memory representation of a layout.
- A layout with non-zero user count can not be deleted from the layout DB.
- A layout with zero user count may continue to reside in memory or in the layout DB.
- Layout type
A 'layout type' specifies how a file is stored in a collection of targets. It provides the <offset-in-gob> to <traget-idx, offset-in-target> mapping. For example, PDCLUST, RAID1, RAID5 are some types of layout, while COMPOSITE being another special layout type. - Enumeration
An 'enumeration' provides <gfid, target-idx> to <cob-fid> mapping. Not all the layout types need an enumeration. For example, layouts with types composite, de-dup do not need an enumeration. - Enumeration type
An 'enumeration type' determines how a collection of component object identifiers (cob-fid) is specified. For example, it may be specified as a list or by means of some linear formula.
Supported layout and enum types
- Layout types supported currently are:
- PDCLUST
This layout type applies parity declustering feature to the striping process. Parity declustering feature is to keep the rebuild overhead low by striping a file over more servers or drives than there are units in the parity group. - COMPOSITE
This layout type partitions a file or a part of the file into various segments while each of those segment uses a different layout.
- PDCLUST
- Enumeration types (also referred as 'enum types') supported currently are:
- LINEAR
A layout with LINEAR enumeration type uses a formula to enumerate all its component object identifiers. - LIST
A layout with LIST enumeration type uses a list to enumerate all its component object identifiers.
- LINEAR
Layout Managed Resources
- A layout as well as a layout-id are resources (managed by the 'Motr Resource Manager').
- Layout being a resource, it can be cached by the clients and can be revoked when it is changed.
- Layout Id being a resource, a client can cache a range of layout ids that it uses to create new layouts without contacting the server.
- A layout can be assigned to a file both by the server and the client.
Sequence Of Layout Operation
The sequence of operation related to domain initialization/finalisation, layout type and enum type registration and unregistration is as follows:
- Initialise m0_layout_domain object.
- Register layout types and enum types using m0_layout_standard_types_register().
- Perform various required operations on the in-memory layouts including the usage of m0_pdclust_build(), m0_layout_get(), m0_layout_put(), m0_layout_encode(), m0_layout_decode(), m0_layout_enum_nr(), m0_layout_enum_get().
- Perform various BE-related operations on the layouts, like: m0_layout_lookup(), m0_layout_add(), m0_layout_update(), m0_layout_delete().
- Perform various operations on layout instances including the usage of m0_layout_instance_build(), m0_layout_instance_fini() and the relevant instance type specific operations. (Creating a layout instance is a way of associating a layout with a particular user, for example a file.)
- Finalise all the layout instances.
- Finalise all the in-memory layouts. (The layouts can continue to exist in the layout DB, even if the resepctive layout types and enum types are to be unregistered and the domain is to be finalised.)
- Unregister layout types and enum types using m0_layout_all_types_unregister().
- Finalise m0_layout_domain object.
Client Server Access to APIs
Regarding client/server access to various APIs from layout and layout-DB modules:
- The APIs exported through layout.h are available both to the client and the server.
- the APIs exported through layout_db.h are available only to the server.
Layout access plan, theory of operation
Overview
Layout access plan is an abstraction, which describes how a particular operation is to be executed on an object or index.
An access plan can be thought of as a directed graph, where vertices are "plan operations" (plops) and arcs correspond to dependencies between plops.
Take a network parity de-clustered layout as an example. To read an extent from an object with this layout, one has to read data from the cobs of the object. The plan for the object read would, therefore, contain a plop for each cob read operation, each having an outgoing arc to the "operation complete" plop.
In case of a write to an object with parity de-clustered layout, the plan contains additional plops corresponding to parity calculations. In case of partial group write, the plan, in addition, contains read-plops describing the logic of read-modify-write cycle. Degraded mode introduces more plops and so on.
Access plans are an intermediate abstraction designed for two main users:
- IO code in client API, including key-value indices and data objects, and
- in-storage compute (ISC, aka Function Shipping).
See also a quick introduction presentation at doc/PDF/layout-access-plan.pdf.
Interface
In the following, "implementation" means generic layout code and particular layout type code, while "user" denotes a caller of the layout plan interface (e.g., IO code).
An access plan is represented by m0_layout_plan.
The directed graph is not presented as a static data structure. Instead, a plan produces, via calls to m0_layout_plan_get(), plops as they become "ready", in the sense that all their dependencies are satisfied.
Interaction between the implementation and the user takes form of the loop (which is, basically, a graph traversal):
- the user calls m0_layout_plan_get() to obtain a ready plop. The implementation returns a plop, represented by a "sub-class" of m0_layout_plop. Some plop fields are set up by the implementation before the plop is handed to the user;
- the user processes the plop in the appropriate manner, e.g., executes read or write operation (which, in turn, might involve layout plans) and sets plop fields accordingly to the results of the execution (e.g., points plop to the data read from a cob);
- the user calls m0_layout_plop_done() to signal the implementation that the plop has been executed. This might make more plops ready, so the loop repeats.
All plops should be executed and m0_layout_plop_done() on them should be called by the user in the dependency order indicated by the m0_layout_plop::pl_deps list. Before starting the plop execution user should call m0_layout_plop_start() which verifies whether the dependencies are met and the plop is still actual (i.e. that it was not cancelled).
A plop is not necessarily immediately destroyed by the implementation after the user completes it. The implementation might keep plop alive for some time, for example, data read from a cob might be needed for a future parity calculation. When the implementation is done with the plop, it invokes m0_layout_plop_ops::po_fini() call-back that the user might optionally set before calling m0_layout_plop_done(). Between the call to m0_layout_plop_done() and invocation of ->po_fini() call-back the plop (and the user data buffers referenced by the plop fields) are shared between the user and the application and the user should neither modify nor free the plop or its data buffers.
As an optimisation, implementation deterministically colours plops, in such a way that a user can improve locality of reference by assigning colours to processor cores. Colouring is optional and does not affect correctness.
Use cases
Let's consider, for example, the i/o read operation. The user calls m0_layout_plan_build() passing m0_op as argument describing the operation. After this we can enter the "loop" traversing the graph. m0_layout_plan_get() is called and returns M0_LAT_READ plop. In this plop user gets cob fid (at m0_layout_plop::pl_ent) and m0_layout_io_plop::iop_ext indexvec describing where the unit to be read is located at the cob. Now, having this information the user is ready to send the io fop to the ioservice.
The next m0_layout_plan_get() call might return M0_LAT_OUT_READ indicating the object data is ready for the user. But the data will be actually ready only after the M0_LAT_READ plop on which M0_LAT_OUT_READ depends on is done.
On receiving the reply from ioservice, the user puts the received io data at m0_layout_io_plop::iop_data bufvec and calls m0_layout_plop_done(). Now the user can access the data using M0_LAT_OUT_READ plop and call plop_done() on it. One iteration of the graph traversing loop is done.
Now, if the object is small and the bufvec at m0_op operation specified by user spans only a single unit, the next call to m0_layout_plan_get() might return M0_LAT_DONE indicating that the plan is done. But usually objects consist of many units and a single m0_op might span many units which, preferrably, should be read from the ioservice(s) in parallel. So the user might call m0_layout_plan_get() many times and get a number of M0_LAT_READ plops before calling the first m0_layout_plop_done().
Let's look at the calls flow example below where u0, u1, u2 are the units for which the plops are returned by the plan:
m0_layout_plan_build() ->
u0: m0_layout_plan_get() -> M0_LAT_READ -> user may start reading...
u1: m0_layout_plan_get() -> M0_LAT_READ -> in parallel...
u2: m0_layout_plan_get() -> M0_LAT_READ -> ...
...
u0: m0_layout_plan_get() -> M0_LAT_OUT_READ -> wait for M0_LAT_READ...
u0: the data for M0_LAT_READ is ready -> m0_layout_plop_done()
u0: now the data in M0_LAT_OUT_READ is ready -> m0_layout_plop_done()
...
m0_layout_plan_get() -> M0_LAT_FUN -> user must check pl_deps:
if all plops in the list are M0_LPS_DONE -> call the function and
-> m0_layout_plop_done()
...
m0_layout_plan_get() -> +1 -> there are no more plops do to yet,
user should complete some previous plops.
...
m0_layout_plan_get() -> M0_LAT_DONE -> ... -> m0_layout_plop_done()
m0_layout_plan_fini()
* - Note
- M0_LAT_OUT_READ plop can be returned by the plan for the unit even before m0_layout_plop_done() is called for its M0_LAT_READ plop. It is user responsibility to track the dependencies between plops (via pl_deps), execute and call m0_layout_plop_done() on them in order.
The picture becomes a bit more complicated when some disk is failed and we are doing the degraded read. Or we are working in the read-verify mode. In this case the plan would involve reading of the parity units also, as well as running the parity calculation functions (M0_LAT_FUN plop) for degraded groups or for every group (in read-verify mode). The function would restore (or verify) data in a synchronous way.
- Note
- in case of the read-verify mode the verification status is indicated by the return code from the m0_layout_fun_plop::fp_fun() call.
The plan may change and adapt according to the situation. For example, let's consider the case when some data unit in a parity group cannot be read for some reason. The user should indicate the error to the implementation via pl_rc at plop_done() for the M0_LAT_READ plop. The implementation will change the plan and cancel all the plops that are no longer relevant (like M0_LAT_OUT_READ for the failed M0_LAT_READ in this case). On the next plan_get() calls the implementation will return M0_LAT_READ plop(s) for the group parity unit(s), followed by M0_LAT_FUN which would restore the failed data unit using the parity data, followed by the new M0_LAT_OUT_READ plop for this data unit. Here is the diagram:
ISC use case
ISC user should analyse the plops and their dependencies in order to figure out whether the user data can be obtained at the server side so that the computation could be delegated there. In some cases, however, the data may not be accessible at the server. For example, in degraded mode some data units in some parity groups have to be recovered from other data and parity units which may be located on different servers. In this case the computation on the recovered units is better to do at the client side.
If the graph dependency looks like a linear chain starting from M0_LAT_READ and finishing with M0_LAT_OUT_READ, this might be a good canditate for ISC:
M0_LAT_READ <-- [ M0_LAT_FUN <-- ... ] <-- M0_LAT_OUT_READ *
it means that the user data can be obtained at the server side independently from the other servers (i.e. there are no several M0_LAT_READ or parity calculation (M0_LAT_FUN) plops in the dependencies) and the execution of such a chain of plops can be delegated to the server side along with the ISC computation.
ISC code should only make sure that the functions which may happen in between the chain links can be run at the server side.
In all other cases the user data should be retrieved at the client side and the ISC computation on it should be done at the client side as well.
Layout Threading and Concurrency Model
- Arrays from the struct m0_layout_domain, storing registered layout types and registered enum types viz. ld_type[] and ld_enum[] are protected by using m0_layout_domain::ld_lock.
- Reference count is maintained for each of the layout types and enum types. This is to help verify that no layout type or enum type gets unregistered while any of the in-memory layout object or enum object is using it.
- The list of the in-memory layout objects stored in the struct m0_layout_domain viz. ld_layout_list is protected by using m0_layout_domain::ld_lock.
- Reference count viz. m0_layout::l_ref is maintained for each of the in-memory layout object.
- It is initialised to 1 during an in-memory layout object creation.
- It gets incremented during every m0_layout_find() and m0_layout_lookup() operations so that the respective user has a hold on that in-memory layout object, during its usage. The user explicitly needs to release this reference once done with the usage, by using m0_layout_put().
- User can explicitly acquire an additional reference on the layout using m0_layout_get() and needs to release it using m0_layout_put().
- Whenever it is the last reference being released, the in-memory layout gets deleted.
- The in-memory layout object is protected by using m0_layout::l_lock.
- DB takes its own locks internally to guarantee that concurrent calls to data-base operations for different layouts do not mess with each other.
- m0_layout::l_lock is used to serialise data-base operations for a particular layout. This relates to the concept of "key locking" in the data-base theory.
- m0_layout_domain::ld_lock is held during the following operations:
- Registration and unregistration routines for various layout types and enum types.
- While increasing/decreasing references on the layout types and enum types through layout_type_get(), layout_type_put(), enum_type_get() and enum_type_put().
- While adding/deleting an entry to/from the layout list that happens through m0_layout__populate() and m0_layout_put() respectively.
- While trying to locate an entry into the layout list using either m0_layout_find() or m0_layout_lookup().
- m0_layout::l_lock is held during the following operations:
- The in-memory operations: m0_layout_get(), m0_layout_put(), m0_layout_user_count_inc() and m0_layout_user_count_dec().
- The DB operation: m0_layout_lookup(), m0_layout_add() m0_layout_update() and m0_layout_delete().
- The user is required to perform lt->lt_ops->lto_allocate() before invoking m0_layout_decode() on a buffer. lt->lt_ops->lto_allocate() has the m0_layout::l_lock held for the layout allocated. As an effect, m0_layout::l_lock is locked throughout the m0_layout_decode() operation.
- The user is explicitly required to hold this lock while invoking m0_layout_encode().
Macro Definition Documentation
◆ M0_TRACE_SUBSYSTEM [1/2]
◆ M0_TRACE_SUBSYSTEM [2/2]
Enumeration Type Documentation
◆ anonymous enum
◆ anonymous enum
| anonymous enum |
◆ anonymous enum
| anonymous enum |
Definition at line 46 of file layout_internal.h.
◆ anonymous enum
◆ m0_layout_plop_state
| enum m0_layout_plop_state |
◆ m0_layout_plop_type
| enum m0_layout_plop_type |
Types of plops.
A value of this enum is placed by the implementation in m0_layout_plop::pl_type field of a plop, before it is returned by m0_layout_plan_get().
| Enumerator | |
|---|---|
| M0_LAT_IN_WRITE | Input plop, m0_layout_inout_plop. Input plop has no dependencies. It is used to signal the implementation that a part of input data is available. An input plop is ready immediately after the plan for a write operation is created. The user handles it by setting m0_layout_in_object_plop::iob_ready to the set of available data extents. More input plops are generated by the implementation until the entire range of data is available. This plop is useful in situations when the entire input data-set is not available at once.
|
| M0_LAT_READ | Read plop, m0_layout_io_plop. This plop instructs the user to read data from the specified object.
|
| M0_LAT_WRITE | Write plop, m0_layout_io_plop. This plop instructs the user to write data to the specified object.
|
| M0_LAT_GET | Get plop, m0_layout_index_plop. This plop instructs the user to look values up in the specified index.
|
| M0_LAT_PUT | Put plop, m0_layout_index_plop. This plop instructs the user to insert values into the specified index.
|
| M0_LAT_FUN | Fun plop, m0_layout_fun_plop. This plop instructs the user to execute a non-blocking function provided by the implementation.
|
| M0_LAT_OUT_READ | Output plop, m0_layout_inout_plop. This plop signals the user that a part of the read data is available.
|
| M0_LAT_DONE | Done plop, m0_layout_plop. This plop signals the final completion of the plan execution. The value of m0_layout_plop::pl_rc field of this plop contains the return code of the entire operation.
|
| M0_LAT_NR | |
◆ m0_layout_xcode_op
| enum m0_layout_xcode_op |
Operation on a layout record, performed through either m0_layout_decode() or m0_layout_encode() routines. M0_LXO_BUFFER_OP indicates that m0_layout_decode()/m0_layout_encode() has to operate upon a buffer.
| Enumerator | |
|---|---|
| M0_LXO_BUFFER_OP | |
| M0_LXO_DB_LOOKUP | |
| M0_LXO_DB_ADD | |
| M0_LXO_DB_UPDATE | |
| M0_LXO_DB_DELETE | |
Function Documentation
◆ add_plops_relation()
|
static |
◆ classify()
| enum m0_pdclust_unit_type classify | ( | const struct m0_pdclust_layout * | play, |
| int | unit | ||
| ) |
◆ del_plop_relations()
|
static |
◆ dummy_create()
|
static |
Definition at line 52 of file m0layout.c.
◆ enum_type_get()
|
static |
◆ enum_type_put()
|
static |
◆ layout_demo()
| void layout_demo | ( | struct m0_pdclust_instance * | pi, |
| struct m0_pdclust_layout * | pl, | ||
| int | R, | ||
| int | I, | ||
| bool | |||
| ) |
- Todo:
- Allocate the arrays globally so that it does not result into going beyond the stack limit in the kernel mode.
Definition at line 93 of file m0layout.c.
◆ layout_invariant_internal()
|
static |
◆ layout_type_get()
|
static |
◆ layout_type_put()
|
static |
◆ list_lookup()
|
static |
◆ M0_BASSERT()
| M0_BASSERT | ( | M0_IS_8ALIGNED(sizeof(struct m0_layout_rec)) | ) |
◆ M0_BOB_DEFINE() [1/3]
| M0_BOB_DEFINE | ( | static | , |
| & | layout_bob, | ||
| m0_layout | |||
| ) |
◆ M0_BOB_DEFINE() [2/3]
| M0_BOB_DEFINE | ( | static | , |
| & | enum_bob, | ||
| m0_layout_enum | |||
| ) |
◆ M0_BOB_DEFINE() [3/3]
| M0_BOB_DEFINE | ( | static | , |
| & | layout_instance_bob, | ||
| m0_layout_instance | |||
| ) |
◆ m0_layout__allocated_invariant()
| M0_INTERNAL bool m0_layout__allocated_invariant | ( | const struct m0_layout * | l | ) |
◆ m0_layout__delete()
| M0_INTERNAL void m0_layout__delete | ( | struct m0_layout * | l | ) |
◆ m0_layout__domain_invariant()
| M0_INTERNAL bool m0_layout__domain_invariant | ( | const struct m0_layout_domain * | dom | ) |
◆ m0_layout__enum_fini()
| M0_INTERNAL void m0_layout__enum_fini | ( | struct m0_layout_enum * | le | ) |
◆ m0_layout__enum_init()
| M0_INTERNAL void m0_layout__enum_init | ( | struct m0_layout_domain * | dom, |
| struct m0_layout_enum * | le, | ||
| struct m0_layout_enum_type * | let, | ||
| const struct m0_layout_enum_ops * | ops | ||
| ) |
◆ m0_layout__enum_invariant()
| M0_INTERNAL bool m0_layout__enum_invariant | ( | const struct m0_layout_enum * | e | ) |
◆ m0_layout__enum_max_recsize()
| M0_INTERNAL m0_bcount_t m0_layout__enum_max_recsize | ( | struct m0_layout_domain * | dom | ) |
◆ m0_layout__fini()
| M0_INTERNAL void m0_layout__fini | ( | struct m0_layout * | l | ) |
◆ m0_layout__fini_internal()
| M0_INTERNAL void m0_layout__fini_internal | ( | struct m0_layout * | l | ) |
◆ m0_layout__init()
| M0_INTERNAL void m0_layout__init | ( | struct m0_layout * | l, |
| struct m0_layout_domain * | dom, | ||
| uint64_t | lid, | ||
| struct m0_layout_type * | lt, | ||
| const struct m0_layout_ops * | ops | ||
| ) |
◆ m0_layout__instance_fini()
| M0_INTERNAL void m0_layout__instance_fini | ( | struct m0_layout_instance * | li | ) |
◆ m0_layout__instance_init()
| M0_INTERNAL void m0_layout__instance_init | ( | struct m0_layout_instance * | li, |
| const struct m0_fid * | gfid, | ||
| struct m0_layout * | l, | ||
| const struct m0_layout_instance_ops * | ops | ||
| ) |
◆ m0_layout__instance_invariant()
| M0_INTERNAL bool m0_layout__instance_invariant | ( | const struct m0_layout_instance * | li | ) |
◆ m0_layout__invariant()
| M0_INTERNAL bool m0_layout__invariant | ( | const struct m0_layout * | l | ) |
◆ m0_layout__list_lookup()
| M0_INTERNAL struct m0_layout * m0_layout__list_lookup | ( | const struct m0_layout_domain * | dom, |
| uint64_t | lid, | ||
| bool | ref_increment | ||
| ) |
Looks up for an entry from the layout list, with the specified layout id. If the entry is found, and if the argument ref_increment has its value set to 'true' then it acquires an additional reference on the layout object.
- Precondition
- m0_mutex_is_locked(&dom->ld_lock).
- Parameters
-
ref_increment Once layout with specified lid is found, an additional reference is acquired on it if value of ref_increment is true.
- Postcondition
- ergo(l != NULL && ref_increment, m0_ref_read(&l->l_ref) > 1);
Definition at line 282 of file layout.c.
◆ m0_layout__log()
| M0_INTERNAL void m0_layout__log | ( | const char * | fn_name, |
| const char * | err_msg, | ||
| uint64_t | lid, | ||
| int | rc | ||
| ) |
Adds a M0_LOG record (trace record), indicating failure, along with a short error message string and the error code.
- Parameters
-
fn_name Function name for the trace record. err_msg Message for the trace record. lid Layout id for the trace record. rc Return code for the trace records.
Definition at line 586 of file layout.c.
◆ m0_layout__populate()
| M0_INTERNAL void m0_layout__populate | ( | struct m0_layout * | l, |
| uint32_t | user_count | ||
| ) |
◆ m0_layout__striped_allocated_invariant()
| M0_INTERNAL bool m0_layout__striped_allocated_invariant | ( | const struct m0_striped_layout * | stl | ) |
◆ m0_layout__striped_delete()
| M0_INTERNAL void m0_layout__striped_delete | ( | struct m0_striped_layout * | stl | ) |
◆ m0_layout__striped_fini()
| M0_INTERNAL void m0_layout__striped_fini | ( | struct m0_striped_layout * | str_l | ) |
◆ m0_layout__striped_init()
| M0_INTERNAL void m0_layout__striped_init | ( | struct m0_striped_layout * | stl, |
| struct m0_layout_domain * | dom, | ||
| uint64_t | lid, | ||
| struct m0_layout_type * | type, | ||
| const struct m0_layout_ops * | ops | ||
| ) |
◆ m0_layout__striped_invariant()
| M0_INTERNAL bool m0_layout__striped_invariant | ( | const struct m0_striped_layout * | stl | ) |
◆ m0_layout__striped_populate()
| M0_INTERNAL void m0_layout__striped_populate | ( | struct m0_striped_layout * | str_l, |
| struct m0_layout_enum * | e, | ||
| uint32_t | user_count | ||
| ) |
Initialises a striped layout object, using provided enumeration object.
- Precondition
- The enumeration object e is already initialised by internally elevating reference of the respective enum type.
- Postcondition
- Pointer to the m0_layout object is set back in the m0_layout_enum object.
Definition at line 427 of file layout.c.
◆ m0_layout_add()
| M0_INTERNAL void m0_layout_add | ( | struct m0_layout_domain * | dom, |
| struct m0_layout * | l | ||
| ) |
◆ m0_layout_decode()
| M0_INTERNAL int m0_layout_decode | ( | struct m0_layout * | l, |
| struct m0_bufvec_cursor * | cur, | ||
| enum m0_layout_xcode_op | op, | ||
| struct m0_be_tx * | tx | ||
| ) |
This method
- Either continues to build an in-memory layout object from its representation 'stored in the Layout DB'
- Or builds an in-memory layout object from its representation 'received through a buffer'.
Two use cases of m0_layout_decode()
- Server decodes an on-disk layout record by reading it from the Layout DB, into an in-memory layout structure, using m0_layout_lookup() which internally calls m0_layout_decode().
- Client decodes a buffer received over the network, into an in-memory layout structure, using m0_layout_decode().
- Parameters
-
cur Cursor pointing to a buffer containing serialised representation of the layout. Regarding the size of the buffer: - In case m0_layout_decode() is called through m0_layout_add(), then the buffer should be containing all the data that is read specifically from the layouts table. It means its size needs to be at the most the size returned by m0_layout_max_recsize().
- In case m0_layout_decode() is called by some other caller, then the buffer should be containing all the data belonging to the specific layout. It may include data that spans over tables other than layouts as well. It means its size may need to be even more than the one returned by m0_layout_max_recsize(). For example, in case of LIST enumeration type, the buffer needs to contain the data that is stored in the cob_lists table.
op This enum parameter indicates what is the DB operation to be performed on the layout record. It could be LOOKUP if at all a DB operation. If it is BUFFER_OP, then the layout is decoded from its representation received through the buffer.
- Precondition
- m0_layout__allocated_invariant(l) implying:
- m0_ref_read(l->l_ref) == 1 and
- m0_mutex_is_locked(&l->l_lock)
- The buffer pointed by cur contains serialised representation of the whole layout in case op is M0_LXO_BUFFER_OP. It contains the data for the layout read from the primary table viz. "layouts" in case op is M0_LXO_DB_LOOKUP.
- m0_layout__allocated_invariant(l) implying:
- Postcondition
- Layout object is fully built (along with enumeration object being built if applicable) along with its ref count being intialised to 1. User needs to explicitly release this reference so as to delete this in-memory layout.
- ergo(rc == 0, m0_layout__invariant(l))
- ergo(rc != 0, m0_layout__allocated_invariant(l)
- m0_mutex_is_locked(&l->l_lock)
- The cursor cur is advanced by the size of the data that is read from it.
- ergo(rc == 0, m0_ref_read(l->l_ref) == 1)
- See also
- m0_layout_put()
Definition at line 947 of file layout.c.
◆ m0_layout_domain_cleanup()
| M0_INTERNAL void m0_layout_domain_cleanup | ( | struct m0_layout_domain * | dom | ) |
◆ m0_layout_domain_fini()
| M0_INTERNAL void m0_layout_domain_fini | ( | struct m0_layout_domain * | dom | ) |
Finalises the layout domain. Dual to m0_layout_domain_init().
- Precondition
- All the layout types and enum types should be unregistered.
Definition at line 633 of file layout.c.
◆ m0_layout_domain_init()
| M0_INTERNAL int m0_layout_domain_init | ( | struct m0_layout_domain * | dom | ) |
◆ m0_layout_encode()
| M0_INTERNAL int m0_layout_encode | ( | struct m0_layout * | l, |
| enum m0_layout_xcode_op | op, | ||
| struct m0_be_tx * | tx, | ||
| struct m0_bufvec_cursor * | out | ||
| ) |
This method uses an in-memory layout object and
- Either adds/updates/deletes it to/from the Layout DB
- Or converts it to a buffer.
Two use cases of m0_layout_encode()
- Server encodes an in-memory layout object into a buffer using m0_layout_encode(), so as to send it to the client.
- Server encodes an in-memory layout object using one of m0_layout_add(), m0_layout_update() or m0_layout_delete() and adds/updates/deletes it to or from the Layout DB.
- Parameters
-
op This enum parameter indicates what is the DB operation to be performed on the layout record if at all a DB operation which could be one of ADD/UPDATE/DELETE. If it is BUFFER_OP, then the layout is stored in the buffer provided by the caller. out Cursor pointing to a buffer. Regarding the size of the buffer: - In case m0_layout_encode() is called through m0_layout_add()| m0_layout_update()|m0_layout_delete(), then the buffer size should be large enough to contain the data that is to be written specifically to the layouts table. It means it needs to be at the most the size returned by m0_layout_max_recsize().
- In case m0_layout_encode() is called by some other caller, then the buffer size should be large enough to contain all the data belonging to the specific layout. It means the size required may even be more than the one returned by m0_layout_max_recsize(). For example, in case of LIST enumeration type, some data goes into table other than layouts, viz. cob_lists table.
- Precondition
- m0_layout__invariant(l)
- m0_mutex_is_locked(&l->l_lock)
- Postcondition
- If op is is either for M0_LXO_DB_<ADD|UPDATE|DELETE>, the respective DB operation is continued.
- If op is M0_LXO_BUFFER_OP, the buffer contains the serialised representation of the whole layout.
Definition at line 1000 of file layout.c.
◆ m0_layout_enum_find()
| M0_INTERNAL uint32_t m0_layout_enum_find | ( | const struct m0_layout_enum * | e, |
| const struct m0_fid * | gfid, | ||
| const struct m0_fid * | target | ||
| ) |
◆ m0_layout_enum_fini()
| M0_INTERNAL void m0_layout_enum_fini | ( | struct m0_layout_enum * | le | ) |
Finalises the enum object. Dual to enum type specific build procedure. The user will not invoke this API explicitly if the enum is used as a part of some layout object. Layout finalisation will take care of enum finalisation in that case. This API is expected to be used only in case the enum could not be made part of any layout for some reason.
- See also
- m0_layout_put()
Definition at line 518 of file layout.c.
◆ m0_layout_enum_get()
| M0_INTERNAL void m0_layout_enum_get | ( | const struct m0_layout_enum * | e, |
| uint32_t | idx, | ||
| const struct m0_fid * | gfid, | ||
| struct m0_fid * | out | ||
| ) |
Returns idx-th object in the layout enumeration.
Beware that this function is currently incorrect and may return unexpected results. The recommanded way to convert an object fid to a component fid is the function m0_poolmach_gob2cob in pool/poolmachine.h
[ref] https://jts.seagate.com/browse/MOTR-1834
- See also
- m0_poolmach_gob2cob()
Definition at line 1080 of file layout.c.
◆ m0_layout_enum_nr()
| M0_INTERNAL uint32_t m0_layout_enum_nr | ( | const struct m0_layout_enum * | e | ) |
◆ m0_layout_enum_type_register()
| M0_INTERNAL int m0_layout_enum_type_register | ( | struct m0_layout_domain * | dom, |
| struct m0_layout_enum_type * | et | ||
| ) |
Registers a new enumeration type with the enumeration types maintained by m0_layout_domain::ld_enum[] and initialises enum type specific tables, if applicable.
Definition at line 759 of file layout.c.
◆ m0_layout_enum_type_unregister()
| M0_INTERNAL void m0_layout_enum_type_unregister | ( | struct m0_layout_domain * | dom, |
| struct m0_layout_enum_type * | et | ||
| ) |
Unregisters an enumeration type from the enumeration types maintained by m0_layout_domain::ld_enum[] and finalises enum type specific tables, if applicable.
Definition at line 793 of file layout.c.
◆ m0_layout_find()
| M0_INTERNAL struct m0_layout * m0_layout_find | ( | struct m0_layout_domain * | dom, |
| uint64_t | lid | ||
| ) |
Returns the layout object if it exists in memory by incrementing a reference on it, else returns NULL. This interface does not attempt to read the layout from the layout database.
- Postcondition
- ergo( != NULL, m0_ref_read(l->l_ref) > 1)
- Note
- This API is required specifically on the client in the absence of layout DB APIs, m0_layout_lookup() to be specific.
Definition at line 861 of file layout.c.
◆ m0_layout_find_by_buffsize()
| M0_INTERNAL int64_t m0_layout_find_by_buffsize | ( | struct m0_layout_domain * | dom, |
| struct m0_fid * | pver, | ||
| size_t | buffsize | ||
| ) |
Find optimal layout id by pver and buffersize. Note, that returned lid is not pver+lid hash (client-side style id) but rather server-side one (currently in range 1-14).
If buffer is too small then returns the first very lid. If too large then returns the layout id describing biggest possible unit_size.
Returns -EINVAL if cannot find the right layout.
Definition at line 810 of file layout.c.
◆ m0_layout_find_by_objsz()
| M0_INTERNAL int64_t m0_layout_find_by_objsz | ( | struct m0_client * | cli, |
| struct m0_fid * | pool, | ||
| size_t | sz | ||
| ) |
Find optimal layout id by pool_id and supposed object size. It's another variant of m0_layout_find_by_buffsize() which might be easier to use. The returned layout id can be passed straight away to the m0_obj_init().
Returns -EINVAL if cannot find the right layout.
Definition at line 851 of file layout.c.
◆ m0_layout_get()
| M0_INTERNAL void m0_layout_get | ( | struct m0_layout * | l | ) |
Acquires an additional reference on the layout object.
- See also
- m0_layout_put()
- m0_layout_find()
Definition at line 880 of file layout.c.
◆ m0_layout_instance_build()
| M0_INTERNAL int m0_layout_instance_build | ( | struct m0_layout * | l, |
| const struct m0_fid * | fid, | ||
| struct m0_layout_instance ** | out | ||
| ) |
Allocates and builds a layout instance using the supplied layout; Acquires an additional reference on the layout pointed by 'l'.
- Postcondition
- ergo(rc == 0, m0_ref_read(&l->l_ref) > 1)
Dual to m0_layout_instance_fini()
Definition at line 1113 of file layout.c.
◆ m0_layout_instance_fini()
| M0_INTERNAL void m0_layout_instance_fini | ( | struct m0_layout_instance * | li | ) |
Finalises the layout instance object; releases reference on the layout that was obtained through m0_layout_instance_build().
Dual to m0_layout_instance_build()
Definition at line 1123 of file layout.c.
◆ m0_layout_instance_to_enum()
| M0_INTERNAL struct m0_layout_enum * m0_layout_instance_to_enum | ( | const struct m0_layout_instance * | li | ) |
◆ m0_layout_max_recsize()
| M0_INTERNAL m0_bcount_t m0_layout_max_recsize | ( | const struct m0_layout_domain * | dom | ) |
Returns maximum possible size for a record in the layouts table (without considering the data in the tables other than layouts), from what is maintained in the m0_layout_domain object.
Definition at line 1037 of file layout.c.
◆ m0_layout_plan_abort()
| M0_INTERNAL void m0_layout_plan_abort | ( | struct m0_layout_plan * | plan | ) |
Signals the implementation that operation execution should be aborted.
Operation abort might affect access plan. The user still has to drain the plan and execute all received plops until a DONE plop is produced.
◆ m0_layout_plan_build()
| M0_INTERNAL struct m0_layout_plan * m0_layout_plan_build | ( | struct m0_op * | op | ) |
◆ m0_layout_plan_fini()
| M0_INTERNAL void m0_layout_plan_fini | ( | struct m0_layout_plan * | plan | ) |
Finalises the plan.
This causes invocation of m0_layout_plop_ops::po_fini() for all still existing plops.
Definition at line 229 of file plan.c.
◆ m0_layout_plan_get()
| M0_INTERNAL int m0_layout_plan_get | ( | struct m0_layout_plan * | plan, |
| uint64_t | colour, | ||
| struct m0_layout_plop ** | out | ||
| ) |
Allocates and returns a ready plop.
If no plop is ready at the moment, +1 is returned.
If colour is equal to M0_LAYOUT_PLOT_ANYCOLOUR, any ready plop is returned, otherwise only a plop with the matching colour.
Definition at line 278 of file plan.c.
◆ m0_layout_plop_done()
| M0_INTERNAL void m0_layout_plop_done | ( | struct m0_layout_plop * | plop | ) |
Instructs the implementation that the user completed processing of the plop.
This might make more plop ready to be returned from m0_layout_plan_get().
If plop->pl_rc is non 0, the implementation might attempt to update the plan to mask or correct the failure.
Definition at line 311 of file plan.c.
◆ m0_layout_plop_start()
| M0_INTERNAL int m0_layout_plop_start | ( | struct m0_layout_plop * | plop | ) |
Instructs the implementation that the user starts processing of the plop.
- Return values
-
-EINVAL if the plop cannot be processed anymore for some reason. For example, if it was cancelled by the plan already.
Definition at line 299 of file plan.c.
◆ m0_layout_put()
| M0_INTERNAL void m0_layout_put | ( | struct m0_layout * | l | ) |
Releases a reference on the layout object. If it is the last reference being released, then it removes the layout entry from the layout list maintained in the layout domain and then finalises the layout along with finalising its enumeration object, if applicable.
- See also
- m0_layout_get()
- m0_layout_find()
Definition at line 893 of file layout.c.
◆ m0_layout_standard_types_register()
| M0_INTERNAL int m0_layout_standard_types_register | ( | struct m0_layout_domain * | dom | ) |
◆ m0_layout_standard_types_unregister()
| M0_INTERNAL void m0_layout_standard_types_unregister | ( | struct m0_layout_domain * | dom | ) |
◆ m0_layout_to_enum()
| M0_INTERNAL struct m0_layout_enum * m0_layout_to_enum | ( | const struct m0_layout * | l | ) |
◆ m0_layout_to_striped()
| M0_INTERNAL struct m0_striped_layout * m0_layout_to_striped | ( | const struct m0_layout * | l | ) |
Returns m0_striped_layout object for the specified m0_layout object.
Definition at line 1045 of file layout.c.
◆ m0_layout_type_register()
| M0_INTERNAL int m0_layout_type_register | ( | struct m0_layout_domain * | dom, |
| struct m0_layout_type * | lt | ||
| ) |
Registers a new layout type with the layout types maintained by m0_layout_domain::ld_type[] and initialises layout type specific tables, if applicable.
Definition at line 707 of file layout.c.
◆ m0_layout_type_unregister()
| M0_INTERNAL void m0_layout_type_unregister | ( | struct m0_layout_domain * | dom, |
| struct m0_layout_type * | lt | ||
| ) |
Unregisters a layout type from the layout types maintained by m0_layout_domain::ld_type[] and finalises layout type specific tables, if applicable.
Definition at line 741 of file layout.c.
◆ m0_layout_user_count_dec()
| M0_INTERNAL void m0_layout_user_count_dec | ( | struct m0_layout * | l | ) |
◆ m0_layout_user_count_inc()
| M0_INTERNAL void m0_layout_user_count_inc | ( | struct m0_layout * | l | ) |
◆ m0_layouts_fini()
◆ m0_layouts_init()
◆ m0_striped_layout_to_enum()
| M0_INTERNAL struct m0_layout_enum * m0_striped_layout_to_enum | ( | const struct m0_striped_layout * | stl | ) |
Returns m0_layout_enum object for the specified m0_striped_layout object.
Definition at line 1056 of file layout.c.
◆ M0_TL_DECLARE() [1/2]
| M0_TL_DECLARE | ( | pldeps | , |
| M0_EXTERN | , | ||
| struct m0_layout_plop_rel | |||
| ) |
◆ M0_TL_DECLARE() [2/2]
| M0_TL_DECLARE | ( | plrdeps | , |
| M0_EXTERN | , | ||
| struct m0_layout_plop_rel | |||
| ) |
◆ M0_TL_DEFINE() [1/4]
| M0_TL_DEFINE | ( | pplops | , |
| M0_INTERNAL | , | ||
| struct m0_layout_plop | |||
| ) |
◆ M0_TL_DEFINE() [2/4]
| M0_TL_DEFINE | ( | pldeps | , |
| M0_INTERNAL | , | ||
| struct m0_layout_plop_rel | |||
| ) |
◆ M0_TL_DEFINE() [3/4]
| M0_TL_DEFINE | ( | plrdeps | , |
| M0_INTERNAL | , | ||
| struct m0_layout_plop_rel | |||
| ) |
◆ M0_TL_DEFINE() [4/4]
| M0_TL_DEFINE | ( | layout | , |
| static | , | ||
| struct m0_layout | |||
| ) |
◆ M0_TL_DESCR_DECLARE() [1/2]
| M0_TL_DESCR_DECLARE | ( | pldeps | , |
| M0_EXTERN | |||
| ) |
◆ M0_TL_DESCR_DECLARE() [2/2]
| M0_TL_DESCR_DECLARE | ( | plrdeps | , |
| M0_EXTERN | |||
| ) |
◆ M0_TL_DESCR_DEFINE() [1/4]
| M0_TL_DESCR_DEFINE | ( | pplops | , |
| "plan plops" | , | ||
| M0_INTERNAL | , | ||
| struct m0_layout_plop | , | ||
| pl_linkage | , | ||
| pl_magix | , | ||
| M0_LAYOUT_PLAN_PLOP_MAGIC | , | ||
| M0_LAYOUT_PPLOPS_HMAGIC | |||
| ) |
◆ M0_TL_DESCR_DEFINE() [2/4]
| M0_TL_DESCR_DEFINE | ( | pldeps | , |
| "plop deps" | , | ||
| M0_INTERNAL | , | ||
| struct m0_layout_plop_rel | , | ||
| plr_dep_linkage | , | ||
| plr_magix | , | ||
| M0_LAYOUT_PLAN_PLOPR_MAGIC | , | ||
| M0_LAYOUT_PPLD_HMAGIC | |||
| ) |
◆ M0_TL_DESCR_DEFINE() [3/4]
| M0_TL_DESCR_DEFINE | ( | plrdeps | , |
| "plop rdeps" | , | ||
| M0_INTERNAL | , | ||
| struct m0_layout_plop_rel | , | ||
| plr_rdep_linkage | , | ||
| plr_magix | , | ||
| M0_LAYOUT_PLAN_PLOPR_MAGIC | , | ||
| M0_LAYOUT_PPLRD_HMAGIC | |||
| ) |
◆ M0_TL_DESCR_DEFINE() [4/4]
| M0_TL_DESCR_DEFINE | ( | layout | , |
| "layout-list" | , | ||
| static | , | ||
| struct m0_layout | , | ||
| l_list_linkage | , | ||
| l_magic | , | ||
| M0_LAYOUT_MAGIC | , | ||
| M0_LAYOUT_HEAD_MAGIC | |||
| ) |
◆ main()
| int main | ( | int | argc, |
| char ** | argv | ||
| ) |
◆ max_recsize_update()
|
static |
Maximum possible size for a record in the layouts table (without considering the data in the tables other than the layouts) is maintained in m0_layout_domain::ld_max_recsize. This function updates m0_layout_domain::ld_max_recsize, by re-calculating it.
Definition at line 555 of file layout.c.
◆ plop_alloc_init()
|
static |
Variable Documentation
◆ enum_bob
|
static |
◆ layout_bob
|
static |
◆ layout_instance_bob
|
static |
◆ m0_linear_enum_type
| struct m0_layout_enum_type m0_linear_enum_type |
Definition at line 342 of file linear_enum.c.
◆ m0_pdclust_layout_type
| struct m0_layout_type m0_pdclust_layout_type |
