DAOS_Documentation_Integration by cdurf1 · Pull Request #4 · daos-stack/daos

cdurf1 · 2017-05-02T21:51:40Z

Documentation to accompany the DAOS source is placed. Changes include the HLD sections and linking.

Adding model.md to client

Makito and Samir observed the following assertion failure after restarting engines. #0 raise () from /lib64/libc.so.6 #1 abort () from /lib64/libc.so.6 #2 __assert_fail_base () from /lib64/libc.so.6 #3 __assert_fail () from /lib64/libc.so.6 #4 pool_map_get_version (map=0x0) at src/common/pool_map.c:2852 #5 ds_pool_get_version (pool=0x7f0ca063c690, pool=0x7f0ca063c690) at src/include/daos_srv/pool.h:296 #6 pc=rpc@entry=0x7f0ca0998d30, p_rpt=p_rpt@entry=0x7f0ca83a77b0) at src/rebuild/srv.c:2101 #7 rebuild_tgt_scan_handler (rpc=0x7f0ca0998d30) at src/rebuild/scan.c:954 #8 crt_handle_rpc (arg=0x7f0ca0998d30) at src/cart/crt_rpc.c:1654 #9 ABTD_ythread_func_wrapper (p_arg=0x7f0ca83a78a0) at arch/abtd_ythread.c:21 #10 make_fcontext () from /usr/lib64/libabt.so.1 #11 ?? () The ds_pool_get_version call passed a NULL map argument to pool_map_get_version. The ds_pool.sp_map field may be NULL after the pool is started but before the pool receives the initial pool map from the pool service. This patch fixes ds_pool_get_version to return 0, which is less than all valid pool map versions, when sp_map is NULL, resulting in rebuild retries like this: Rebuild [queued] (pool=3bf68c9c ver=2) tgts=2 Rebuild [started] (pool 3bf68c9c ver=2) Rebuild [failed] (pool 3bf68c9c ver=2 status=DER_BUSY(-1012): 'Device or resource busy') Rebuild [queued] (pool=3bf68c9c ver=2) tgts=2 Rebuild [started] (pool 3bf68c9c ver=2) Rebuild [failed] (pool 3bf68c9c ver=2 status=DER_BUSY(-1012): 'Device or resource busy') Rebuild [queued] (pool=3bf68c9c ver=2) tgts=2 Rebuild [started] (pool 3bf68c9c ver=2) Rebuild [scanning] (pool 3bf68c9c ver=2, toberb_obj=0, rb_obj=0, [...] Rebuild [scanning] (pool 3bf68c9c ver=2, toberb_obj=0, rb_obj=0, [...] Rebuild [completed] (pool 3bf68c9c ver=2, toberb_obj=0, rb_obj=0,[...] Target[2] (rank 2 idx 0 status 16 ver 1) is excluded. Also, this patch removes some rebuild code that handles NULL ds_pool.sp_group fields. Those can not happen as we always initialize sp_group (as well as sp_iv_ns) before putting a ds_pool object into the LRU. Signed-off-by: Li Wei <wei.g.li@intel.com>

Makito and Samir observed the following assertion failure after restarting engines. #0 raise () from /lib64/libc.so.6 #1 abort () from /lib64/libc.so.6 #2 __assert_fail_base () from /lib64/libc.so.6 #3 __assert_fail () from /lib64/libc.so.6 #4 pool_map_get_version (map=0x0) at src/common/pool_map.c:2852 #5 ds_pool_get_version (pool=0x7f0ca063c690, pool=0x7f0ca063c690) at src/include/daos_srv/pool.h:296 #6 pc=rpc@entry=0x7f0ca0998d30, p_rpt=p_rpt@entry=0x7f0ca83a77b0) at src/rebuild/srv.c:2101 #7 rebuild_tgt_scan_handler (rpc=0x7f0ca0998d30) at src/rebuild/scan.c:954 #8 crt_handle_rpc (arg=0x7f0ca0998d30) at src/cart/crt_rpc.c:1654 #9 ABTD_ythread_func_wrapper (p_arg=0x7f0ca83a78a0) at arch/abtd_ythread.c:21 #10 make_fcontext () from /usr/lib64/libabt.so.1 #11 ?? () The ds_pool_get_version call passed a NULL map argument to pool_map_get_version. The ds_pool.sp_map field may be NULL after the pool is started but before the pool receives the initial pool map from the pool service. This patch fixes ds_pool_get_version to return 0, which is less than all valid pool map versions, when sp_map is NULL, resulting in rebuild retries like this: Rebuild [queued] (pool=3bf68c9c ver=2) tgts=2 Rebuild [started] (pool 3bf68c9c ver=2) Rebuild [failed] (pool 3bf68c9c ver=2 status=DER_BUSY(-1012): 'Device or resource busy') Rebuild [queued] (pool=3bf68c9c ver=2) tgts=2 Rebuild [started] (pool 3bf68c9c ver=2) Rebuild [failed] (pool 3bf68c9c ver=2 status=DER_BUSY(-1012): 'Device or resource busy') Rebuild [queued] (pool=3bf68c9c ver=2) tgts=2 Rebuild [started] (pool 3bf68c9c ver=2) Rebuild [scanning] (pool 3bf68c9c ver=2, toberb_obj=0, rb_obj=0, [...] Rebuild [scanning] (pool 3bf68c9c ver=2, toberb_obj=0, rb_obj=0, [...] Rebuild [completed] (pool 3bf68c9c ver=2, toberb_obj=0, rb_obj=0,[...] Target[2] (rank 2 idx 0 status 16 ver 1) is excluded. Also, this patch removes some rebuild code that handles NULL ds_pool.sp_group fields. Those can not happen as we always initialize sp_group (as well as sp_iv_ns) before putting a ds_pool object into the LRU. Skip-test: true Signed-off-by: Li Wei <wei.g.li@intel.com>

Makito and Samir observed the following assertion failure after restarting engines. #0 raise () from /lib64/libc.so.6 #1 abort () from /lib64/libc.so.6 #2 __assert_fail_base () from /lib64/libc.so.6 #3 __assert_fail () from /lib64/libc.so.6 #4 pool_map_get_version (map=0x0) at src/common/pool_map.c:2852 #5 ds_pool_get_version (pool=0x7f0ca063c690, pool=0x7f0ca063c690) at src/include/daos_srv/pool.h:296 #6 pc=rpc@entry=0x7f0ca0998d30, p_rpt=p_rpt@entry=0x7f0ca83a77b0) at src/rebuild/srv.c:2101 #7 rebuild_tgt_scan_handler (rpc=0x7f0ca0998d30) at src/rebuild/scan.c:954 #8 crt_handle_rpc (arg=0x7f0ca0998d30) at src/cart/crt_rpc.c:1654 #9 ABTD_ythread_func_wrapper (p_arg=0x7f0ca83a78a0) at arch/abtd_ythread.c:21 #10 make_fcontext () from /usr/lib64/libabt.so.1 #11 ?? () The ds_pool_get_version call passed a NULL map argument to pool_map_get_version. The ds_pool.sp_map field may be NULL after the pool is started but before the pool receives the initial pool map from the pool service. This patch fixes ds_pool_get_version to return 0, which is less than all valid pool map versions, when sp_map is NULL, resulting in rebuild retries like this: Rebuild [queued] (pool=3bf68c9c ver=2) tgts=2 Rebuild [started] (pool 3bf68c9c ver=2) Rebuild [failed] (pool 3bf68c9c ver=2 status=DER_BUSY(-1012): 'Device or resource busy') Rebuild [queued] (pool=3bf68c9c ver=2) tgts=2 Rebuild [started] (pool 3bf68c9c ver=2) Rebuild [failed] (pool 3bf68c9c ver=2 status=DER_BUSY(-1012): 'Device or resource busy') Rebuild [queued] (pool=3bf68c9c ver=2) tgts=2 Rebuild [started] (pool 3bf68c9c ver=2) Rebuild [scanning] (pool 3bf68c9c ver=2, toberb_obj=0, rb_obj=0, [...] Rebuild [scanning] (pool 3bf68c9c ver=2, toberb_obj=0, rb_obj=0, [...] Rebuild [completed] (pool 3bf68c9c ver=2, toberb_obj=0, rb_obj=0,[...] Target[2] (rank 2 idx 0 status 16 ver 1) is excluded. Also, this patch removes some rebuild code that handles NULL ds_pool.sp_group fields. Those can not happen as we always initialize sp_group (as well as sp_iv_ns) before putting a ds_pool object into the LRU. Signed-off-by: Li Wei <wei.g.li@intel.com>

Commit 53d66d7 might have introduced the following "epi != NULL" assertion failure in crt_context_req_untrack. #0 raise () from /usr/lib64/libc.so.6 #1 abort () from /usr/lib64/libc.so.6 #2 __assert_fail_base.cold.0 () from /usr/lib64/libc.so.6 #3 __assert_fail () from /usr/lib64/libc.so.6 #4 crt_context_req_untrack (rpc_priv=rpc_priv@entry=0x55a254b83150) at src/cart/crt_context.c:1358 crt_ctx = 0x55a25465f500 epi = 0x0 submit_list = {next = 0x0, prev = 0x4} tmp_rpc = <optimized out> rc = <optimized out> __func__ = "crt_context_req_untrack" __PRETTY_FUNCTION__ = "crt_context_req_untrack" #5 crt_hg_req_send_cb (hg_cbinfo=<optimized out>) at src/cart/crt_hg.c:1287 rpc_pub = <optimized out> rpc_priv = <optimized out> hg_ret = HG_SUCCESS rc = 0 __func__ = "crt_hg_req_send_cb" __PRETTY_FUNCTION__ = "crt_hg_req_send_cb" __rc = <optimized out> The RPC might be an outgoing URI_LOOKUP request, which we don't track. Hence its crp_epi field was NULL as expected. Commit 53d66d7 should have asserted "epi != NULL" _after_ making sure that the RPC is not an URI_LOOKUP request. This patch does that and removes a few useless lines from crt_context_req_untrack_internal. Test-tag: pr dynamic_server_pool OSAOfflineExtend NvmePoolExtend Signed-off-by: Li Wei <wei.g.li@intel.com> Required-githooks: true

Commit 53d66d7 might have introduced the following "epi != NULL" assertion failure in crt_context_req_untrack. #0 raise () from /usr/lib64/libc.so.6 #1 abort () from /usr/lib64/libc.so.6 #2 __assert_fail_base.cold.0 () from /usr/lib64/libc.so.6 #3 __assert_fail () from /usr/lib64/libc.so.6 #4 crt_context_req_untrack (rpc_priv=rpc_priv@entry=0x55a254b83150) at src/cart/crt_context.c:1358 crt_ctx = 0x55a25465f500 epi = 0x0 submit_list = {next = 0x0, prev = 0x4} tmp_rpc = <optimized out> rc = <optimized out> __func__ = "crt_context_req_untrack" __PRETTY_FUNCTION__ = "crt_context_req_untrack" #5 crt_hg_req_send_cb (hg_cbinfo=<optimized out>) at src/cart/crt_hg.c:1287 rpc_pub = <optimized out> rpc_priv = <optimized out> hg_ret = HG_SUCCESS rc = 0 __func__ = "crt_hg_req_send_cb" __PRETTY_FUNCTION__ = "crt_hg_req_send_cb" __rc = <optimized out> The RPC might be an outgoing URI_LOOKUP request, which we don't track. Hence its crp_epi field was NULL as expected. Commit 53d66d7 should have asserted "epi != NULL" _after_ making sure that the RPC is not an URI_LOOKUP request. This patch does that and removes a few useless lines from crt_context_req_untrack_internal. Signed-off-by: Li Wei <wei.g.li@intel.com>

Initial commit that installs the mpi file utils (mfu) and other dependencies for the io500 workload into the DAOS client image. Dependencies are installed into /usr/local/io500 Signed-off-by: Joel Rosenzweig <joel.b.rosenzweig@intel.com> Co-authored-by: Johann Lombardi <johann.lombardi@intel.com> Co-authored-by: lsitkiew <lukasz.sitkiewicz@intel.com>

dumps: refine dtx_update_abort/vos_ioc_create

Features: dfuse Allow-unstable-test: true Signed-off-by: Jeff Olivier <jeffolivier@google.com>

test_pool_create_slow_svc's 180s timeout had the thinnest margin in the suite (max observed 103s over 10 CI repeats in PR-19162 build #4, a 1.75x margin), because unlike its sibling sub-tests it also pays for the full agent/server startup: the most CI-load-dependent part of the run. Sibling yaml files use 240-300s for their equivalent startup test (e.g. create_all_vm.yaml: test_one_pool_vm: 240). Raise it to 240s for a more comfortable, consistent margin. Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@hpe.com>

cdurf1 added 30 commits April 21, 2017 10:07

Create model.md

101e166

Adding model.md to client

Add files via upload

7a83bae

Update model.md

3b47c13

Update model.md

b87ab16

Create layering.md

b5b6f4c

Update README.md

d31c6e6

Add files via upload

b994e4e

Update HLD.md

9c64835

Update HLD.md

0d6e04d

Update HLD.md

4f13f31

Update HLD.md

14cba3a

Update HLD.md

15b5ded

Update HLD.md

4673757

Update HLD.md

b176b78

Create use_cases.md

beaf836

Update use_cases.md

b7d5125

Create vos.md

ad6237a

Update HLD.md

46eaf7a

Update HLD.md

cf52d06

Update vos.md

be158d4

Add files via upload

81b1ef6

Update HLD.md

d8ed45c

Update key_array_object.md

3f60efd

Update model.md

9b6c509

Create posix.md

b1d21e6

Update HLD.md

5305103

Add files via upload

901088f

Update HLD.md

bae317a

Update DAOS_M_forGitHub.md

d8e6146

Update DAOS_M_forGitHub.md

0170c34

sylviachanoiyee mentioned this pull request May 2, 2021

DAOS-7220 Test: Test SameKeyDifferentValue, pool create fails, scm to small #5533

Merged

wangdi1 mentioned this pull request Aug 2, 2021

DAOS-8082 mgmt: missing collect req put #6340

Merged

sylviachanoiyee mentioned this pull request Aug 28, 2021

DAOS-7834 test: Add telemetry data to soak #6494

Merged

sylviachanoiyee mentioned this pull request Dec 13, 2021

DAOS-9180: remove unnecessary prop and pool map sync during rebuild #7550

Merged

sylviachanoiyee mentioned this pull request Jul 13, 2022

DAOS-10500 test: nvme/health.py: increase pool capacity (#9066) #9245

Merged

liw mentioned this pull request Jul 20, 2022

DAOS-10622 pool: Fix ds_pool_get_version for NULL sp_map (#9277) #9753

Merged

kccain mentioned this pull request Mar 10, 2023

DAOS-12636 container: set status property to unhealthy #11467

Merged

18 tasks

Nasf-Fan mentioned this pull request Mar 20, 2023

DAOS-12706 object: Some memory corruption issue #11624

Merged

18 tasks

wiliamhuang mentioned this pull request Jul 21, 2023

DAOS-13548 client: enable one NLT fault injection test for libpil4dfs #12664

Closed

18 tasks

osalyk pushed a commit to osalyk/daos that referenced this pull request Oct 18, 2023

Merge pull request daos-stack#4 from janekmi/dumps_dtx_update_abort-2

aa532b5

dumps: refine dtx_update_abort/vos_ioc_create

jolivier23 mentioned this pull request Dec 19, 2023

DAOS-14532 gurt: Replace environment APIs hook #13483

Merged

18 tasks

daltonbohning mentioned this pull request Feb 5, 2024

DAOS-15084 chk: aggregated patch for cat_recovery #13718

Merged

18 tasks

wiliamhuang mentioned this pull request Apr 16, 2024

DAOS-14149 client: add compatible mode for libpil4dfs #13294

Merged

18 tasks

phender mentioned this pull request May 1, 2024

DAOS-15774 test: fix nvme/enospace.py container destroy #14285

Merged

18 tasks

mchaarawi mentioned this pull request Jul 24, 2024

DAOS-16264 common: Fix incorrect assertion #14809

Merged

18 tasks

jolivier23 added a commit that referenced this pull request Jan 12, 2025

Try again to see if it picks up #4

3487bc6

Features: dfuse Allow-unstable-test: true Signed-off-by: Jeff Olivier <jeffolivier@google.com>

mjmac mentioned this pull request Jan 29, 2025

DAOS-16312 control: Always use --force for dmg system stop #15799

Merged

3 of 18 tasks

mchaarawi mentioned this pull request Nov 13, 2025

DAOS-18161 object: refine EC aggregation processing #17092

Merged

6 tasks

Navigation Menu

Repository navigation

DAOS_Documentation_Integration - #4

DAOS_Documentation_Integration#4
cdurf1 wants to merge 30 commits into
daos-stack:masterfrom
cdurf1:master

cdurf1 commented May 2, 2017

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Milestone

Development

Uh oh!

2 participants

Conversation

cdurf1 commented May 2, 2017

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Milestone

Development

Uh oh!

2 participants