|
Crate index workload overview.
Crate execution for index measurements consists of two parts:
- Warmup.
- Common operations.
Warmup is supposed to prepare index to workload. For example, if we have to measure over fragmented index, we should setup next parameters:
It is means, that crate performs next operations:
Common operations are executed randomly (see cr_op_selector). Each operaion has 'percents' of total operations count (OP_COUNT). Total number of keys in use are calculated by OP_COUNT and NUM_KVP. OP_COUNT can be "unlimited" TODO: for now, it means INT_MAX / (128 * NUM_KVP) operations.
Crate index workload detailed description.
Workload has following parameters:
- WORKLOAD_TYPE - always 0 (idx operations).
- WORKLOAD_SEED - initial value for pseudo-random generator (int or "tstamp").
- KEY_SIZE - size of the key; can be defined as number or "random", KEY_SIZE should be >= 16
- VALUE_SIZE - size of the value; can be defined as number or "random"
- MAX_KEY_SIZE - max size of key; used in case, when KEY_SIZE = "random". Also, see CR_MAX_OF_MAX_KEY_SIZE.
- MAX_VALUE_SIZE - max size of value; used in case, when VALUE_SIZE = "random". Also, see CR_MAX_OF_MAX_VALUE_SIZE.
- OP_COUNT - total operations count; can be defined as number, number with units (1K), or "unlimited".
- PUT,GET,DEL,NEXT - percents of portion operations.
- EXEC_TIME - time limit for executing; can be defined as number (seconds), or "unlimited".
- NUM_KVP - number of records, which should be processed at operation.
- NXRECORDS - number of records, which should be processed at NEXT operation; can be defined as number, number with units (1K) or "default" (NXRECORDS == NUM_KVP).
- KEY_PREFIX - number, which suppose to define the key space (fid.f_container); can be defined as a number or "random".
- WARMUP_PUT_CNT - count of PUT operations in "warmup" step; defined as number of operations or as "all" (it means "fill all keys in the index").
- WARMUP_DEL_RATIO - ratio, which determines, which portion of WARMUP_PUT_CNT should deleted in random order (int).
- KEY_ORDER - defines key ordering in operations ("ordered" or "random").
- INDEX_FID - index fid (fid, for example,
<7800000000000001:0>). - LOG_LEVEL - logging level(err(0), warn(1), info(2), trace(3), debug(4)).
Operation order (see cr_idx_w_select_op)
Operations are executed in round-robin manner or randomly. If crate can't find appropriate keys for operation then it marks operation as "pending". Selector knows about it and can unmark pending ops, if it cannot find appropriate operations.
Keys order
Keys can be generated sequentially or randomly. There are two functions: cr_idx_w_find_rnd_k and cr_idx_w_find_seq_k. They both have the same signature, but different logic.
cr_idx_w_find_seq_k
This function selects next keys from starting key to nr_keys (the end of keys space). Starting key value depends on operation type: readonly operations (GET,NEXT) should store last key from previous key sequence; other operations (PUT,DEL) starts from first key. Anyway, they iterate over all available keys and fill key list.
cr_idx_w_find_rnd_k
This function selects random key, puts it in the key list and does it again until list is full or bitmap is full.
Different logic for different operations (see struct cr_idx_w_ops).
This struct describes how operation should be prepared for execution and how they change storage state.
Measurements
Currently, only execution time is measured during the test. It measures with m0_time* functions. Crate prints result to stdout when test is finished.
Logging
crate has own logging system, which based on fprintf(stderr...). (see crlog and see cr_log).
Execution time limits
There are two ways to limit execution time. The first, is operations time limit, which checks after operation executed (as described in HLD). The second way uses the watchdog, which detects staled client operations.
