Todo List

Global __confdb_free (struct m0_be_btree *btree, struct m0_be_seg *seg, struct m0_be_tx *tx)

check validity of key and record addresses and sizes. Specifically, check that val.b_addr points to an allocated region in a segment with appropriate size and alignment. Such checks should be done generally by (not existing) beobj interface.

also check that key (fid) matches m0_conf_objx_fid().

Global _confc_cache_clean (struct m0_confc *confc)

Confc root pointer is not valid anymore after cache cleanup, so it should be reinitialised. The easiest way would be to reinitialise confc completely, but user can create confc contexts during reelection, so let's reinitialise root object in different way.

Global addr_check (const char *addr)

additional checks

Module adieu

A natural way to extend this design is to introduce additional SIS_PREPARED state and to split IO operation submission into two stages: (i) "preparation" stage that is entered once "IO geometry" is known (i.e., once m0_vec of data pages and m0_vec storage objects are known) and (ii) "queueing" stage that is entered when in addition to IO geometry, actual data pages are allocated. The motivating example for this refinement is a data server handling read or write RPC from a client. The RPC contains enough information to build IO vectors, while data arrive later through RDMA. To avoid dead-locks, it is crucial to avoid dynamic resource allocations (first of all, memory allocations) in data path after resources are consumed by RDMA. To this end, IO operation must be completely set up and ready for queueing before RMDA starts, i.e., before data pages are available.

Module adieu

A natural way to extend this design is to introduce additional SIS_PREPARED state and to split IO operation submission into two stages: (i) "preparation" stage that is entered once "IO geometry" is known (i.e., once m0_vec of data pages and m0_vec storage objects are known) and (ii) "queueing" stage that is entered when in addition to IO geometry, actual data pages are allocated. The motivating example for this refinement is a data server handling read or write RPC from a client. The RPC contains enough information to build IO vectors, while data arrive later through RDMA. To avoid dead-locks, it is crucial to avoid dynamic resource allocations (first of all, memory allocations) in data path after resources are consumed by RDMA. To this end, IO operation must be completely set up and ready for queueing before RMDA starts, i.e., before data pages are available.

Module adieu

A natural way to extend this design is to introduce additional SIS_PREPARED state and to split IO operation submission into two stages: (i) "preparation" stage that is entered once "IO geometry" is known (i.e., once m0_vec of data pages and m0_vec storage objects are known) and (ii) "queueing" stage that is entered when in addition to IO geometry, actual data pages are allocated. The motivating example for this refinement is a data server handling read or write RPC from a client. The RPC contains enough information to build IO vectors, while data arrive later through RDMA. To avoid dead-locks, it is crucial to avoid dynamic resource allocations (first of all, memory allocations) in data path after resources are consumed by RDMA. To this end, IO operation must be completely set up and ready for queueing before RMDA starts, i.e., before data pages are available.

Global application_data_copy (struct pargrp_iomap *map, struct m0_obj *obj, m0_bindex_t start, m0_bindex_t end, struct m0_bufvec_cursor *datacur, enum copy_direction dir, enum page_attr filter)

reduce the number of arguments to this function, map+obj are in ioo

Global apply_filters (struct fdmi_sd_fom *sd_fom, struct m0_fdmi_src_rec *src_rec)

Mark FDMI filter as invalid (send HA not?) (phase 2)

Global autotm [1024]

Move it to m0 instance or make per-domain.

Global BALLOC_DEF_CONTAINER_SIZE

should equal to size of HDD

Module be

Interface tester

Global be_tx_ast_offset [M0_BTS_NR+1]

Find out why M0_BTS_NR + 1 is enough and M0_BTS_NR isn't.

Class buf_status_errno

add timestamp

Global capa_test (void)

struct m0_capa_issuer is empty, put proper values.

Global cas_fom_create (struct m0_fop *fop, struct m0_fom **out, struct m0_reqh *reqh)

Validity (cas_is_valid()) of input records is not checked here, so "out_nr" can be bogus. Cannot check validity at this point, because ->fto_create() errors are silently ignored.

Global cas_next_ast (struct m0_sm_group *grp, struct m0_sm_ast *ast)

Currently there is no way to pass several starting keys along with number of consecutive records for each key through m0_idx_op().

Module chan

The next scalability improvement is to allow m0_chan to use an externally specified mutex instead of a built-in one. This would allow larger state machines with multiple channels to operate under fewer locks, reducing coherency bus traffic.

Module client

A library on top of client for fast scalable identifier allocation will be provided as part of Motr.

an interface to register 8-bit combinations for application use (to introduce application-specific "entity-like" things).

to be described).

entity type structures (to provide constructors, 8-bit identifier tags and an ability to register new entity types).

handling of extensible attributes (check-sums, version numbers, etc.), which require interaction with the implementation on the service side.

Global cm_id_generate (void)

Rewrite this when mechanism to generate unique ids is in place.

Global cm_proxy_invariant (const struct m0_cm_proxy *pxy)

: Add checks for pxy::px_id when uid mechanism is implemented.

Module cond

Consider supporting other types of locks in addition to m0_mutex.

Global conf_dir_id_build (struct m0_fid *out, const struct m0_fid *parent, const struct m0_conf_obj_type *children_type)

This would produce non-unique identifier, if an object has two different directories with the same children types. Perhaps relation fid should be factored in somehow.

Global config_print (struct m0_net_test_console_cfg *cfg)

write text

Global confx_fill (struct m0_confx *dest, struct m0_be_btree *btree)

check validity of key and record addresses and sizes. Specifically, check that val.b_addr points to an allocated region in a segment with appropriate size and alignment. Such checks should be done generally by (not existing) beobj interface.

also check that key (fid) matches m0_conf_objx_fid().

handle iteration errors.

Global console_run (struct m0_net_test_console_ctx *ctx)

make command line parameter

can be interrupted

Global constraints_are_valid (const struct m0_rpc_frm_constraints *constraints)

XXX Check whether constraints are consistent

Page Copy Machine Functional Specification

This would be re-written when configuration api's would be implemented.

• m0_confc_open() Opens an individual confc object. processing.

This section would be re-written when the other copy machine functionalities would be implemented.

Global COUNT

Greater number of indices produces -E2BIG error in idx-deleteN test case.

Global cp_fom_tick (struct m0_fom *fom)

Revise it.

Global creq_niter_next (struct creq_niter *it)

Validate it.

Global crw_ivec

Perhaps it would be easier to change client to send io_index_vec as 2 sequences (of the same length): first indices, then counts. Such sequence can be converted to m0_indexvec without allocating intermediary structure and it might simplify client code too, if it uses indexvecs.

Global cs_conf_storage_pver_is_actual (const struct m0_conf_obj *obj)

XXX filter only actual pool versions till formulaic pool version creation in place.

Class cs_stob_file

XXX FIXME: confc has landed ages ago.

Global cs_storage_init (const char *stob_type, const char *stob_path, uint64_t dom_key, struct cs_stobs *stob, struct m0_be_seg *seg, bool mkfs, bool force, bool disable_direct_io)

Use generic mechanism to generate stob ids

Global cs_storage_setup (struct m0_motr *cctx)

Have a generic mechanism to generate unique cob domain id. Handle error messages properly.

Global cs_ut_ep_mixed_dup_cmd []

Remove passing of multiple endpoints to m0d, as is not needed.

Module desim

add m0_ prefixes to sim symbols.

Page Detailed Level Design for read-modify-write IO requests.

In future, optimizations could be done in order to speed up small file IO with the help of client side cache.

IO types like fault IO are not supported yet.

A client cache is missing at the moment. With addition of cache, the states of an IO request might add up.

In future, with introduction of Resource Manager, distributed extent locks have to be acquired or released as needed.

However, with all code in kernel and no present UT code for m0t1fs, it is still to be decided how to write UTs for this component.

Page Distributed transaction manager

Internal declarations leak in the global name-space. This should be fixed by making them static and including dtm .c files in a single dtm/dtm.c file.

pack, unpack, fop.

Page DIX copy machine DLD

It would be nice to find the solution where implementation of any kind of aggregation groups is not necessary.

• For rebalance, the same rule to determine responsible node for data reconstruction can be applied. DIX copy machine uses the same layout as used during DIX repair to map a unit to the target unit on new device. The newly added device may have a new UUID, but will have the same index in the pool and the component catalogues identifiers of the failed device and the replacement device will also be the same.

Global dix_cas_rops_send (struct m0_dix_req *req)

CAS request should copy cctg_id internally.

Global dix_cm_ag_is_frozen_on (struct m0_cm_aggr_group *ag, struct m0_cm_proxy *pxy)

What to return here?

Global dix_cm_iter_trans []

Revise this array.

Global dix_inst::di_index_pver

This field is temporary. Client interface should be extended, so user can define pool version for an index. Use constant pool version until this is done. Actually, this pool version equals to the one used for root index, which is defined in filesystem configuration object.

Global dix_next_ast (struct m0_sm_group *grp, struct m0_sm_ast *ast)

Currently there is no way to pass several starting keys along with number of consecutive records for each key through m0_idx_op().

Global dix_req_trans []

Check it.

Page DLD of Bulk Server

If buffer_pool reached to low threshold, Bulk I/O service may expand pool size. This can be done later to minimize waiting time for network buffer.

Page DLD of configuration caching

Implement cache eviction.

Global enable_filters (bool enable, struct m0_fid *filter_ids, uint32_t filter_count)

Phase 2: implement posting filter descriptions to filterd

Global eval_flt_node (struct m0_fdmi_eval_ctx *ctx, struct m0_fdmi_flt_node *node, struct m0_fdmi_flt_operand *res, struct m0_fdmi_eval_var_info *var_info)

Free resources from operands?

Global ext_punch (struct m0_stob *stob, struct m0_dtx *tx, struct m0_ext *todo)

the target ext to delete, which will be marked as a hole.

Global fail_idx_reed_solomon_recover (struct m0_parity_math *math, struct m0_buf *data, struct m0_buf *parity, const uint32_t failure_index)

Iterative reed-solomon decode to be implemented.

Module FDMI_DLD_fspec_filter

Insert example of usage here

Global fdmi_filter_check (const void *bob)

Phase 2: Do checks

Global fdmi_post_fop (struct m0_fop *fop, struct m0_rpc_session *session)

what deadline is better? (phase 2)

Global ffs_op_encode (struct m0_fdmi_src_rec *src_rec, struct m0_buf *buf)

Q: (for FOL owners) FOL record does not provide API call to calculate record size when encoded. For now, I'll do double allocation. Alloc internal buf of max size, then encode, then alloc with correct size, then copy, then dealloc inernal buf. Can be done properly once FOL record owner exports needed api call.

Global ffs_op_node_eval (struct m0_fdmi_src_rec *src_rec, struct m0_fdmi_flt_var_node *value_desc, struct m0_fdmi_flt_operand *value)

Phase 2: STUB: For now, we will not analyze filter, we just return FOL op code – always.

Global file_lock_encdec (struct m0_file *file, struct m0_bufvec_cursor *cur, enum m0_xcode_what what)

->rto_decode() decode for file resource should create an entire ambient object: an inode on client and a cob on server.

Module fom

load balancing is not implemented at the moment).

describe intended fom and reqh usage on client.

Global fom_failure (struct m0_fom *fom)

workaround to move be transaction into FAILED state, so that it can be finalised.

Global fom_queue_reply (struct m0_fom *fom)

Implement write back cache, during which we may perform updates on local objects and re-integrate with the server later, in that case we may block while, we caching fop, this requires more additions to the routine.

Global frm_try_merging_item (struct m0_rpc_frm *frm, struct m0_rpc_item *item, m0_bcount_t limit)

XXX implement item merging

Page Functional Specification

Outdated and not used now

Global ha_client_release (struct m0_ref *ref)

: Is it possible to move {m0}_ha_client_* functions to ha/note.[ch] files? Seems that functionality is related to HA notifications.

Global ha_msg_is_one_way (const struct m0_ha_msg *msg)

Move to ha/msg.[hc].

Global handle_fdmi_rec_not (struct m0_uint128 *rec_id, struct m0_buf fdmi_rec, struct m0_fid filter_id)

Call decode for fdmi record type (frt_rec_decode)

Consider special ret code if processing is complete ?

Global if (rc !=0)

When we have rm locking working, this will be changed to revalidate inode with checking cached lock. If lock is cached (not canceled), which means inode did not change, then we don't have to do getattr and can just use cached data.

Global incoming_check (struct m0_rm_incoming *in)

Here we introduce "thundering herd" problem, potentially waking up all requests waiting for reserved credit. It is necessary, because rio_conflict() won't be called for 'in' if waiting requests are not woken up.

Global incoming_check_held (struct m0_rm_incoming *in, struct m0_rm_credit *rest, struct m0_rm_credit *held, int *wait, bool *cr_used)

Ignore borrow requests for held non-conflicting credits. If it is the only credit that can satisfy incoming request, then eventually creditor will revoke it.

Global incoming_check_with (struct m0_rm_incoming *in, struct m0_rm_credit *credit)

Will be deleted once borrowing held non-conflicting credits is allowed.

Global io_item_replied (struct m0_rpc_item *item)

Rearrange IO item merging code to work with new formation code.

Global is_actual_pver (const struct m0_conf_obj *obj)

XXX filter only actual pool versions till formulaic pool version creation in place.

Global is_session_marked (struct io_request *req, struct m0_rpc_session *session)

This code is not required once MOTR-899 lands into dev. Returns true if a given session is already marked as failed. In case a session is not already marked for failure, the functions marks it and returns false.

Global is_session_marked (struct m0_op_io *ioo, struct m0_rpc_session *session)

This code is not required once MOTR-899 lands into dev. Returns true if a given session is already marked as failed. In case a session is not already marked for failure, the functions marks it and returns false.

Global iter_ut_pool_init ()

Ugly workaround to do finalisation successfully.

Global layout_demo (struct m0_pdclust_instance *pi, struct m0_pdclust_layout *pl, int R, int I, bool print)

Allocate the arrays globally so that it does not result into going beyond the stack limit in the kernel mode.

Global linux_location []

move vars to a context

Global m0_arch_alloc (size_t size)

The proper solution is to introduce an additional interface m0_alloc_safe(), to be called outside of critical locks and using GFP_KERNEL.

Class m0_be_io_credit

add bshift

Global m0_be_log::lg_store

this might be changed to something more complicated to support flexible deployment and grow-able logs. E.g., a log can be stored in a sequence of regions in segments, linked to each other through header blocks.

Class m0_be_log_io

document fields owned by m0_be_log and move fields reset there

Global m0_be_log_store_io_translate (struct m0_be_log_store *ls, m0_bindex_t position, struct m0_be_io *bio)

s/io_prepare/io_translate/g

Global m0_be_log_store_rbuf_write_buf (struct m0_be_log_store *ls)

Make interface for redundant buffer simplier.

Global m0_be_reg__invariant (const struct m0_be_reg *reg)

make m0_be_reg_copy_to(reg, dst_addr) and m0_be_reg_copy_from(reg, src_addr)

Global m0_be_reg_gen_idx (const struct m0_be_reg *reg)

add UT similar to libm0-ut:time.

Global m0_be_seg_reserved (const struct m0_be_seg *seg)

s/bs_reserved/m0_be_seg_reserved/ everywhere

Global m0_be_tx::t_fast

Remove when m0_be_tx_close_sync() is removed

Global m0_be_tx::t_fdmi_put_ast

Will be fixed when proper refcounting is implemented in second phase of FDMI work.

Global m0_be_tx::t_fdmi_ref

Fix this when proper refcounting is implemented.

Global m0_be_tx::t_payload

Don't allocate m0_be_tx::t_payload separately. Use m0_be_tx_group preallocated payload area.

Use m0_be_tx::t_filler callback to fill m0_be_tx::t_payload.

Global m0_be_tx__is_fast (struct m0_be_tx *tx)

Remove when m0_be_tx_close_sync() is removed.

Global m0_be_tx_group_fom_init (struct m0_be_tx_group_fom *m, struct m0_be_tx_group *gr, struct m0_reqh *reqh)

XXX TODO s/gf/m/ in function parameters

Global m0_be_ut_log_store_create_simple (void)

use ls_cfg_create and ls_cfg_open to test that create parameters aren't used in open()

Class m0_capa_issuer

Use proper capability issuer

Global m0_cm_configure (struct m0_cm *cm, struct m0_fop *fop)

Pass actual configuration fop data structure once configuration interfaces and datastructures are available.

Global m0_cm_fail (struct m0_cm *cm, int rc)

Rewrite this function when new ADDB infrastucture is in place.

Global m0_cob_init (struct m0_cob_domain *dom, struct m0_cob *cob)

Get di type from configuration.

Global m0_conf_confc_ha_update (struct m0_confc *confc)

XXX RELOCATEME: This function belongs ha subsystem, not conf.

Global m0_conf_dir_new (struct m0_conf_obj *parent, const struct m0_fid *relfid, const struct m0_conf_obj_type *children_type, const struct m0_fid_arr *children_ids, struct m0_conf_dir **out)

UT transactional property of m0_conf_dir_new().

Global m0_conf_obj_ha_update (const struct m0_fid *obj_fid)

XXX RELOCATEME: This function belongs ha subsystem, not conf.

Global m0_conf_objs_ha_update (struct m0_ha_nvec *nvec)

XXX RELOCATEME: This function belongs ha subsystem, not conf.

Global m0_conf_pool::pl_pver_policy

Replace with m0_conf_root::rt_pool_selection_policy.

Global m0_conf_process::pc_endpoint

Use an array to support several network interfaces.

Global m0_conf_root::rt_fdmi_flt_grps

Halon support is yet to be implemented; see HALON-730.

Global m0_conf_root::rt_params

Make it an array of name-value pairs (attributes).

Global m0_conf_sdev::sd_drive

XXX DELETEME

Global m0_conf_service::cs_params

Make it an array of name-value pairs (attributes).

Global m0_ctg_ctidx_delete_sync (const struct m0_cas_id *cid, struct m0_be_tx *tx)

Make it asynchronous.

Global m0_ctg_ctidx_insert_sync (const struct m0_cas_id *cid, struct m0_be_tx *tx)

Make it asynchronous.

Global m0_ctg_ctidx_lookup_sync (const struct m0_fid *fid, struct m0_dix_layout **layout)

Make it asynchronous.

Global m0_ctg_store_init (struct m0_be_domain *dom)

Use 0type.

Global m0_dtx::tx_state

placeholder for now.

Global m0_fdmi__rec_id_gen (struct m0_fdmi_src_rec *src_rec)

Phase 2: m0_fdmi__rec_id_gen() should return unique ID within whole Motr system. Some generic function should be used. For this moment, to provide unique values within several running Motr instances, lets populate hi value part with rand value, low part will be incremented. As a bare minimum, for phase 2 we need to make sure fsdc_instance_id is globally unique, and that re-start generates a new one, again globally unique. (Note this is required, if we leave the same ID over restart, new instance will try to process replies aimed at previous one.

Global m0_fdmi__src_dock_fom_start (struct m0_fdmi_src_dock *src_dock, const struct m0_filterc_ops *filterc_ops, struct m0_reqh *reqh)

FDMI service can't work without filterc. inform ADDB on critical error.

Global m0_fdmi__src_dock_fom_wakeup (struct fdmi_sd_fom *sd_fom)

Small possibility of races exist (Phase 2).

Class m0_fdmi_filter_desc

Phase 2: Internals will be understood during further development

Global m0_fdmi_flt_op_node_create (enum m0_fdmi_flt_op_code op_code, struct m0_fdmi_flt_node *left, struct m0_fdmi_flt_node *right)

Per-operation code check for number of operands and its types (phase 2)

Global m0_fdmi_source_register (struct m0_fdmi_src *src)

Phase 2: Check that fdmi_rec_type_id exists

Global m0_fom_dom_ops

Support fom timeout functionality.

Global m0_fom_domain_ops::fdo_time_is_out )(const struct m0_fom_domain *dom, const struct m0_fom *fom)

fom timeout implementation.

Global m0_fom_tick_generic (struct m0_fom *fom)

standard fom phases implementation, depends on the support routines for handling various standard operations on fop as mentioned above

Global m0_fop_cob_rw::crw_ivec

Perhaps it would be easier to change client to send io_index_vec as 2 sequences (of the same length): first indices, then counts. Such sequence can be converted to m0_indexvec without allocating intermediary structure and it might simplify client code too, if it uses indexvecs.

Global m0_idx_op (struct m0_idx *idx, enum m0_idx_opcode opcode, struct m0_bufvec *keys, struct m0_bufvec *vals, int32_t *rcs, uint32_t flags, struct m0_op **op)

For now 'rcs' may be NULL if index backend is not Motr KVS and operation code is not M0_IC_GET. All backends should be updated to fill 'rcs' for all operation codes.

Global m0_io_cob_stob_create (struct m0_fom *fom, struct m0_cob_domain *cdom, struct m0_fid *fid, struct m0_fid *pver, uint64_t lid, bool crow, struct m0_cob **out)

delete stob too.

Global m0_ios_cdom_get (struct m0_reqh *reqh, struct m0_cob_domain **out)

: This function is used by copy machine module, but not used by IO service. Corresponding ticket: MOTR-1190.

Global m0_layout_list_enum::lle_list_of_cobs

In kernel any allocation over 4KB is not safe. Thus, this array can safely hold only upto 256 number of COB identifiers, (m0_fid being 16 bytes in size). This issue is to be addressed later.

Global M0_LOCKERS_DEFINE (M0_INTERNAL, m0_locality, lo_lockers)

move m0_locality_lockers_type and ldata[] in locality_global, once lockers are updated to use non-global lockers type.

Global m0_mdstore_close (struct m0_mdstore *md, struct m0_cob *cob, struct m0_be_tx *tx)

:

• orphans handling?
• quota handling?

Global m0_mdstore_open (struct m0_mdstore *md, struct m0_cob *cob, m0_mdstore_locate_flags_t flags, struct m0_be_tx *tx)

: Place cob to open files table.

Global m0_net__buffer_add (struct m0_net_buffer *buf, struct m0_net_transfer_mc *tm)

should be m0_net_desc_free()?

Global m0_net_test_client_server_bulk_ut (void)

investigate strange m0_net_tm_stop() time on the bulk test client.

Global m0_net_test_mps_add (struct m0_net_test_mps *mps, unsigned long messages, m0_time_t timestamp)

problem with small mps->ntmps_time_interval can be here

Global m0_net_test_network_bd_serialize (enum m0_net_test_serialize_op op, struct m0_net_test_network_ctx *ctx, uint32_t buf_bulk_index, uint32_t buf_ping_index, m0_bcount_t offset)

possible security vulnerability because bounds are not checked

Global m0_net_test_network_buf_fill (struct m0_net_test_network_ctx *ctx, enum m0_net_test_network_buf_type buf_type, uint32_t buf_index, uint8_t fill)

use m0_bufvec_cursor_step

Global m0_net_test_network_buf_resize (struct m0_net_test_network_ctx *ctx, enum m0_net_test_network_buf_type buf_type, uint32_t buf_index, m0_bcount_t new_size)

isn't safe because net_test_buf_init() can fail

Global m0_net_test_network_ctx_init (struct m0_net_test_network_ctx *ctx, struct m0_net_test_network_cfg *cfg, const char *tm_addr)

create configuration structure instead a lot of parameters

Global m0_net_test_network_ut_ping (void)

timeddown

Global m0_net_test_nh::ntnh_sd

XXX check grammar

Global m0_net_test_nh_init (struct m0_net_test_nh *nh, const struct m0_net_test_cmd_init *icmd)

reset all stats

Global m0_net_test_node_ctx::ntnc_exit_flag

make it atomic

Global m0_op_io::ioo_rc

: remove this

Global m0_op_kick (struct m0_op *op)

: put pressure on the rpc system to send this rpc message

Global m0_poolmach_state_transit (struct m0_poolmach *pm, const struct m0_poolmach_event *event)

If this is a new node join event, the index might larger than the current number. Then we need to create a new larger array to hold nodes info.

Add ADDB error message here.

Global m0_pools_common::pc_dev2svc

Check whether concurrency needs to be handled after MOTR-1498 is in dev.

Global m0_pools_common_conf_ready_async_cb (struct m0_clink *clink)

XXX: See if we could do anything with the failed update here. But so far we just cross fingers and hope it succeeds.

Global m0_reqh_addb2_init (struct m0_reqh *reqh, const char *location, uint64_t key, bool mkfs, bool force, m0_bcount_t size)

replace size constant (size) with a value from confc.

Global m0_reqh_fdmi_service::rfdms_src_dock_inited

Temporary field to indicate whether source dock was successfully started. (phase 2)

Global m0_reqh_init (struct m0_reqh *reqh, const struct m0_reqh_init_args *reqh_args)

use iostores instead of m0_cob_domain

Global m0_rm_ha_subscribe_sync (struct m0_confc *confc, const char *rem_ep, struct m0_rm_ha_tracker *tracker)

Ideally this function should be implemented through asynchronous m0_rm_ha_subscribe(). The problem is that m0_rm_ha_subscriber internally locks confc sm group. For global confc it is locality0 sm group. So this group can't be provided to m0_rm_ha_subscriber_init(). Usually users requesting synchronous operation don't have another option, except locality0 sm group.

What if remote is already in M0_NC_FAILED state? We should check it somewhere.

Class m0_rm_incoming

a new type of incoming request M0_RIT_GRANT (M0_RIT_FOIEGRAS?) can be added to forcibly grant new credits to the owner, for example, as part of a coordinated global distributed resource usage balancing between owners. Processing of requests of this type would be very simple, because adding new credits never blocks. Similarly, a new outgoing request type M0_ROT_TAKE could be added.

Global m0_rm_remote_owner_set (struct m0_rm_remote *rem, uint64_t id)

Assigns a service to a given remote.

Global m0_rm_resource_type::rt_ha_events

:

1. Ideally queue shall be operated in lockfree manner to avoid taking locks in HA callback and in AST thread. Currently since only one producer and one consumer contend for the queue lock and critical section is not large this has been differed. Lockfree queue would also require a bit of memory management.
2. HA should not send the same state twice, and ideally Motr should assert on such incidence. This implementation would circumvent such behaviour by HA. Once Halon is completely replaced by Hare RM can assert on such spurious notifications.

Global m0_rm_resource_type::rt_id

Currently this is assigned manually and centrally. In the future, resource types identifiers (as well as rpc item opcodes) will be assigned dynamically by a special service (and then announced to the clients). Such identifier name-spaces are resources themselves, so, welcome to a minefield of bootstrapping.

Global m0_rms_register (void)

Contact confd and take list of resource types for this resource manager.

Global m0_rpc_at_init (struct m0_rpc_at_buf *ab)

M0_PRE(M0_IS0(ab));

Global m0_rpc_conn_pool_get_async (struct m0_rpc_conn_pool *pool, const char *remote_ep, struct m0_rpc_session **session)

Looks like rpc link connect could not be called twice, even in case first attempt fails (phase 2).

Potential race if connection is established before clink is added to session channel.

Global m0_rpc_conn_pool_session_established (struct m0_rpc_session *session)

Unprotected access to ->sm_state in this function.

Global m0_rpc_frm_constraints_get_defaults (struct m0_rpc_frm_constraints *c)

XXX decide default values for constraints

Global m0_rpc_item_dispatch (struct m0_rpc_item *item)

this assumes that the item is a fop.

Global M0_RPC_ITEM_RESEND_INTERVAL

Provide proper value for re-send interval

Global m0_rpc_item_type_ops::rito_eq )(const struct m0_rpc_item *i1, const struct m0_rpc_item *i2)

XXX Implement rito_eq for fops

Global m0_rpc_machine::rm_reqh

There needs to be generic mechanism to register a request handler (or any other handler for future use) with the rpc machine and a ops vector specifying a method to be invoked for futher processing, e.g. m0_reqh_fop_handle(), in case of reqh.

Class m0_sns_cm_file_ctx

Use the same object to hold layout context.

Global m0_sns_cm_op

: Remove once Halon supports successor enum m0_cm_op.

Global m0_sns_cm_status

: Remove once Halon supports successor enum m0_cm_status.

Global m0_spiel_pool_rebalance_abort (struct m0_spiel *spl, const struct m0_fid *pool_fid)

Remove once Halon supports m0_spiel_{sns,dix}_rebalance_abort().

Remove once Halon supports m0_spiel_{sns,dix}_rebalance_abort().

Global m0_spiel_pool_rebalance_continue (struct m0_spiel *spl, const struct m0_fid *pool_fid)

Remove once Halon supports m0_spiel_{sns,dix}_rebalance_continue().

Remove once Halon supports m0_spiel_{sns,dix}_rebalance_continue().

Global m0_spiel_pool_rebalance_quiesce (struct m0_spiel *spl, const struct m0_fid *pool_fid)

Remove once Halon supports m0_spiel_{sns,dix}_rebalance_quiesce().

Remove once Halon supports m0_spiel_{sns,dix}_rebalance_quiesce().

Global m0_spiel_pool_rebalance_start (struct m0_spiel *spl, const struct m0_fid *pool_fid)

Remove once Halon supports m0_spiel_{sns,dix}_rebalance_start().

Remove once Halon supports m0_spiel_{sns,dix}_rebalance_start().

Global m0_spiel_pool_rebalance_status (struct m0_spiel *spl, const struct m0_fid *pool_fid, struct m0_spiel_sns_status **statuses)

Remove once Halon supports m0_spiel_{sns,dix}_rebalance_status().

Global m0_spiel_pool_repair_abort (struct m0_spiel *spl, const struct m0_fid *pool_fid)

Remove once Halon supports m0_spiel_{sns,dix}_repair_abort().

Remove once Halon supports m0_spiel_{sns,dix}_repair_abort().

Global m0_spiel_pool_repair_continue (struct m0_spiel *spl, const struct m0_fid *pool_fid)

Remove once Halon supports m0_spiel_{sns,dix}_repair_continue().

Remove once Halon supports m0_spiel_{sns,dix}_repair_continue().

Global m0_spiel_pool_repair_quiesce (struct m0_spiel *spl, const struct m0_fid *pool_fid)

Remove once Halon supports m0_spiel_{sns,dix}_repair_quiesce().

Remove once Halon supports m0_spiel_{sns,dix}_repair_quiesce().

Global m0_spiel_pool_repair_start (struct m0_spiel *spl, const struct m0_fid *pool_fid)

Remove once Halon supports m0_spiel_{sns,dix}_repair_start().

Remove once Halon supports m0_spiel_{sns,dix}_repair_start().

Global m0_spiel_pool_repair_status (struct m0_spiel *spl, const struct m0_fid *pool_fid, struct m0_spiel_sns_status **statuses)

Remove once Halon supports m0_spiel_{sns,dix}_repair_status().

Remove once Halon supports m0_spiel_{sns,dix}_repair_status().

Class m0_spiel_sns_status

Remove once Halon supports successor m0_spiel_repreb_status.

Class m0_stob_cache

document

Global M0_STOB_CACHE_MAX_SIZE

make a parameter for stob domain.

Global m0_stob_find_by_key (struct m0_stob_domain *dom, const struct m0_fid *stob_fid, struct m0_stob **out)

move allocation out of cache lock if needed

Global m0_stob_io_opcode

implement barriers.

Global m0_stob_state

add M0_ prefix.

Global M0_TL_DESCR_DECLARE (rpcbulk, M0_EXTERN)

This value can be reduced after multiple message delivery in a single buffer is supported.

Global m0_trace_allot (const struct m0_trace_descr *td, const void *body)

put memory barrier here before writing the magic

Global M0_XCA_DOMAIN

Q: Is it possible release FOP to be ONEWAY?

Global M0_XCA_DOMAIN

at some point, we may replace the co_ref by taking a reference on the underlying co_stob. At that point, we will need a callback at last put. We wait to see how cob users will use these references, whether they need callbacks in turn, etc.

Q: Make it generic reply?

Q: Is it possible release FOP to be ONEWAY?

Global M0_XCA_DOMAIN

at some point, we may replace the co_ref by taking a reference on the underlying co_stob. At that point, we will need a callback at last put. We wait to see how cob users will use these references, whether they need callbacks in turn, etc.

Q: Make it generic reply?

Q: Is it possible release FOP to be ONEWAY?

Global M0_XCA_DOMAIN

Q: Make it generic reply?

Global M0_XCA_ENUM

Phase 2: Recursive data definition is not allowed in xcode operations.

Global M0_XCA_ENUM

Phase 2: Recursive data definition is not allowed in xcode operations.

Module m0ham

Fix memory leaks:

• m0_ha_entrypoint_req_fop::erf_git_rev_id
• m0_ha_entrypoint_rep_fop::hbp_active_rm_ep
• m0_ha_entrypoint_rep_fop::hbp_confd_fids.af_elems
• m0_ha_entrypoint_rep_fop::hbp_confd_eps.ab_elems<p>Implement "Connect/disconnect without m0_ha_msg traffic" scenario. See the comment in ham_link_disconnected().

"Server sends, client receives" scenario ==> client crashes.

ham_say() entrypoint attributes in ham_entrypoint_replied().

Global m0t1fs_file_lock_init (struct m0t1fs_inode *ci, struct m0t1fs_sb *csb)

Get di type from configuration.

Global m0t1fs_fsync (struct file *file, loff_t start, loff_t end, int datasync)

: Generate some addb here.

Global m0t1fs_mds_cob_op (struct m0t1fs_sb *csb, const struct m0t1fs_mdop *mo, struct m0_fop_type *ftype, struct m0_fop **rep_fop)

remid can be found generically, outside of this switch through the use of 'm0_xcode_find()' - this function should be cleaned up later.

Global m0t1fs_statfs (struct dentry *dentry, struct kstatfs *buf)

According to stob_ad_domain_cfg_create_parse(), current block size forcibly defaulted by BALLOC_DEF_BLOCK_SHIFT value. Need to understand if it's ever going to be any different in future, and if so, whether it's worth to be exposed to fs client. Besides, what if the balloc setup is going to vary among segments/pools/etc.?

Global main (int argc, char *argv[])

add to command line parameters

where is the error

add to command line parameters

add Ctrl+C handler m0_net_test_fini()+m0_net_test_config_fini()

atexit()

where is the error

Global md_fol_frag_redo (struct m0_fop_fol_frag *ffrag, struct m0_fol *fol)

Perform the redo operation for meta-data updates using the generic fop fol fragment.

Global md_fol_frag_undo (struct m0_fop_fol_frag *ffrag, struct m0_fol *fol)

Perform the undo operation for meta-data updates using the generic fop fol fragment.

Global mds_start (struct m0_reqh_service *service)

XXX change this when reqh will be started before services, see MOTR-317

Page Motr Network Benchmark

Outdated and not used now.

Outdated and not used now.

Global net_buf_data_eq (enum m0_net_test_network_buf_type buf_type, struct m0_net_test_network_ctx *ctx1, uint32_t buf_index1, struct m0_net_test_network_ctx *ctx2, uint32_t buf_index2)

too expensive, use m0_bufvec_cursor_step() + memcmp()

Global net_test_command_ut (size_t nr)

set callback

Global net_test_network_ctx_initfini (struct m0_net_test_network_ctx *ctx, struct m0_net_test_network_cfg *cfg, const char *tm_addr)

make ctx->ntc_dom embedded into ctx

replace gmc and ctx

Module netsock

It is not clear how to manage write-monitoring in case of multiple "parallel" sockets to the same end-point. If writeability of all such sockets is monitored, epoll_wait() can busy-loop. If not all of them are monitored, they are useless for concurrent writes.

Module NetTestNetworkDFS

m0_net_test_network_ prefix is too long. rename and align.

s/uint32_t/size_t/

Module NetTestNetworkInternals

add timeouts to channels and network buffers

align code (function parameters etc.)

cache m0_vec_count()

Module NetTestPingNodeInternals

nb_max_receive_msgs > 1 is not supported.

Module NetTestStatsDFS

Move to lib/stats.h

Global NKP_assign_and_pool_init (uint32_t enum_id, uint32_t inline_test, uint32_t list_nr_less, uint32_t list_nr_more, uint32_t linear_nr, uint32_t *N, uint32_t *K, uint32_t *S, uint32_t *P)

This is a workaround to be taken out along with the forthcoming patch for the layout module to use xcode and newer BE, with which the whole layout test suite will anyway be restructured.

Global nlx_dev_tm_cleanup (struct nlx_kcore_domain *kd, struct nlx_kcore_transfer_mc *ktm)

XXX list removal is protected by the lock, but iteration over the list isn't? –nikita.

Global node_bulk_cmd_start (void *ctx_, const struct m0_net_test_cmd *cmd, struct m0_net_test_cmd *reply)

copy-paste from node_ping.c. refactor it.

copy-paste from node_ping.c

Global node_bulk_cmd_stop (void *ctx_, const struct m0_net_test_cmd *cmd, struct m0_net_test_cmd *reply)

copy-paste from node_ping.c. refactor it.

Class node_bulk_ctx

move equal parts from node_bulk_ctx & node_ping_ctx to single struct

Global node_cmd_wait (struct m0_net_test_node_ctx *ctx, struct m0_net_test_cmd *cmd, enum m0_net_test_cmd_type type)

add retry count

Global node_options_add (struct cs_args *args, const struct m0_conf_node *node)

Node parameters cn_memsize and cn_flags options are not used currently. Options '-m' and '-q' options are used for maximum RPC message size and minimum length of TM receive queue. If required, change the option names accordingly.

Global nreq_asmbl_accept (struct m0_cas_req *req)

validate it

Global open_filter_group (struct m0_filterc_ctx *ctx, enum m0_fdmi_rec_type_id rec_type_id, struct m0_conf_obj **out)

Close opened groups?

Global outgoing_check (struct m0_rm_incoming *in, enum m0_rm_outgoing_type, struct m0_rm_credit *credit, struct m0_rm_remote *other)

adjust outgoing requests priority (priority inheritance)

Global pargrp_iomap_fullpages_count (struct pargrp_iomap *map)

: all client buffers should be PA_FULLPAGE_MODIFY - rip all this out

Global permute_column (struct m0_pdclust_instance *pi, uint64_t omega, uint64_t t)

Not sure if this should be replaced by an ADDB DP or a M0_LOG.

Global ping_init (struct nlx_ping_ctx *ctx)

replace gmc and ctx

Global prepare_rpc_env (struct test_rpc_env *env, struct m0_reqh *reqh, const struct m0_rpc_frm_ops *frm_ops, bool sender, struct m0_rpc_conn *rpc_conn, struct m0_rpc_session *rpc_session)

Add prefix

Global pthread_attr_default

move this in m0 instance.

Global pver_is_actual (const struct m0_conf_obj *obj)

XXX filter only actual pool versions till formulaic pool version creation in place.

Global rconfc_link_fini_ast (struct m0_sm_group *grp, struct m0_sm_ast *ast)

The dead confd fid is going to be removed from phony cache, and the link object to be finalised in rconfc_link_fom_fini(). Therefore, no way remains to listen for the confd state changes in future. You may be tempted by an idea of reviving the link in case the confd is announced M0_NC_ONLINE later, but consider the following analysis:

Global reqh_services_start (struct m0_reqh_context *rctx, struct m0_motr *cctx)

XXX Handle errors properly.

Global rev_session_clink_cb (struct m0_clink *link)

This would be fixed, when RM will be using normal locality sm groups, rather than per-resource-type groups

Global rm_ha_sbscr_diter_next (struct m0_rm_ha_subscriber *sbscr)

What if obj is already in M0_NC_FAILED state? We should check it somewhere.

Global rm_remote_death_handler (struct m0_rm_remote *remote)

if rc != 0, then credits remain in sub-let list and can't be revoked anymore. Also we can't return error code to user. Maybe we should notify HA about error?

Module rm_service

All the creditors for the resources reside inside RM service; i.e. RM service will be the creditor for all currently available resource types. When specific resource types are implemented, this can be changed.

Global root_encode (struct m0_confx_obj *dest, const struct m0_conf_obj *src)

Make spiel happy for now as it does not know about fdmi yet.

Module rpc-at

Another method of buffer transmission may be needed: "best". When client specifies this method it has to provide network buffer as with in-bulk case. The server sends the reply inline if possible, uses the netbuffer otherwise. This would allow return of large records (via in-bulk), while avoiding in-bulk overhead for small records.

: Perform negotiation on inbulk threshold during connection establishment?

Global rpc_conn__on_cache_ready_cb (struct m0_clink *clink)

See if we can act any smarter than just log the subscription error. Please note, -ENOENT code is normal in the situation when the connection previously was established to a service that appears abandoned when conf updates.

Global rpc_conn__on_service_event_cb (struct m0_clink *clink)

See if to __conn_ha_unsubscribe() right now, but not wait until rpc connection getting finalised.

See if anything, or what and when otherwise, we need to do on getting M0_NC_ONLINE notification.

Module rpc_session

currently, receiver assigns identifiers to connections and sessions and these identifiers are used by both parties. What we can do, is to allow sender to assign identifiers to sessions (this identifier is sent in SESSION_ESTABLISH). Then, whenever receiver uses the session to send a reply, it uses this identifier (instead of receiver assigned session-id). The advantage of this, is that sender can use an identifier that allows quick lookup (e.g., an index in some session array or simply a pointer). Similarly for connections (i.e., another sender generated identifier in addition to uuid, that is not guaranteed to be globally unique)

• Generate ADDB data points for important session events
• store replies in FOL
• Optimization: Cache misordered items at receiver, rather than discarding them.

Global rpc_tm_setup (struct m0_net_transfer_mc *tm, struct m0_net_domain *net_dom, const char *ep_addr, struct m0_net_buffer_pool *pool, uint32_t colour, m0_bcount_t msg_size, uint32_t qlen)

Find more appropriate err code. tm does not report cause of failure.

Global sc_key_pos_set (struct m0_dix_next_sort_ctx *ctx, uint32_t key_idx, const uint32_t *recs_nr)

: Why it is so?

Global sd_fom_process_matched_filters (struct m0_fdmi_src_dock *sd_ctx, struct m0_fdmi_src_rec *src_rec)

store map <fdmi record id, endpoint>, Phase 2

Global service_ctxs_create (struct m0_pools_common *pc, bool service_connect)

: This needs to be converted to process (m0d) context since it connects to specific process endpoint.

Global spiel_conf_ut_init (void)

Use fid convert function to set kind.

Global spiel_process_health_replied_ast (struct m0_sm_group *grp, struct m0_sm_ast *ast)

Need to understand if it would make sense from consumer's standpoint to interrupt stats collection here on a network error.

Global stats_add (struct m0_tl *stats_list, struct m0_stats_sum *sum)

Is it required to check valid m0_stats_sum::ss_id from some some list? How ? need some global stats id list.

Module stob

more scalable object index instead of a list.

Global stob_ad_io_launch (struct m0_stob_io *io)

: Needs to check performance impact of sorting each stobio on ad stob.

Global stob_linux_domain_create_destroy (struct m0_stob_type *type, const char *path, uint64_t dom_key, void *cfg, bool create)

get mode from create cfg

Module stoblinux

use explicit state machine instead of ioq threads

Global svc_find (const struct m0_op_idx *oi)

Currently, the implementation of non-distributed indices is very similar to the distributed indices implementation. This makes bug fixing more difficult: every error has to be fixed twice. To solve this issue the implementation should be generalised for both cases in this file. A non-distributed index should be treated as a special type of distributed index (with 1 component catalogue).

Page Test console command line parameters

Update obsoleted options. Use

Global test_init (void)

break UT into small transactions

Global test_start_stop (void)

Check addresses used by rconfc

Global tolerance_of_level (struct m0_op_io *ioo, uint64_t lv)

This code is not required once MOTR-899 lands into dev. Tolerance for the given level.

Global tolerance_of_level (struct io_request *req, uint64_t lv)

This code is not required once MOTR-899 lands into dev. Tolerance for the given level.

Global xcode_read_as (const struct m0_xcode_type *type, const char *str, int *rc)

XXX Move to xcode/xcode.[hc]. Use this function instead of m0_confstr_parse().