Repository navigation
Nasf fan/daos 19616 b28 - #19226
Closed
Nasf-Fan wants to merge 257 commits into
Closed
Nasf fan/daos 19616 b28#19226Nasf-Fan wants to merge 257 commits into
Nasf-Fan wants to merge 257 commits into
Conversation
Fix two bugs causing -DER_CSUM failures with non-power-of-2 checksum chunk sizes on EC objects: 1. In __migrate_fetch_update_bulk(), checksums were computed while IOD recxs were still in DAOS-space (needed for the preceding fetch), but stored at VOS-space offsets. When cksum_size does not evenly divide ec_cell_sz, chunk boundaries differ between coordinate spaces, producing mismatched checksums. Move mrone_recx_daos2_vos() before migrate_csum_calc() so checksums align with VOS storage offsets. 2. In vos_csum_recalc.c, calc_csum_params() and csum_agg_verify() used unsigned int for extent offsets, truncating the PARITY_INDICATOR bit (0x8000000000000000) on parity extents. With non-power-of-2 chunk sizes, this truncation shifts chunk boundaries, causing aggregation checksum verification to always fail. Widen to uint64_t. Signed-off-by: Kenneth Cain <kenneth.cain@hpe.com>
…egation (#17829) (#17896) EC aggregation doesn't require full layout computation, instead it only requires the layout up to the specified group. This patch adds an option and allows the placement function to complete layout computation early. It can significantly reduce computation time for EC aggregation. Signed-off-by: Liang Zhen <gnailzenh@gmail.com> * DAOS: rebuild scan/reclaim also uses partial layout Signed-off-by: Liang Zhen <gnailzenh@gmail.com> --------- Signed-off-by: Liang Zhen <gnailzenh@gmail.com>
… (#17920) There's no need to set execute permission bit for the checkpoint files. Signed-off-by: Denis Barakhtanov <dbarahtanov@enakta.com>
This patch fixes several resource-handling bugs that could lead to memory leaks or crashes 1.Avoid leaking memory allocated in tse_task_cb() when tse_task_reinit_with_delay() fails by introducing a helper to clean up on failure. 2. Ensure sub_anchors_free() is called on error paths for enumeration and key2anchor case. 3. Invoke dc_tx_local_close() on error paths to properly close local transactions. 4. Make drpc_call() return an error when allocation fails to avoid potential NULL dereference and segfault. Signed-off-by: Wang Shilong <shilong.wang@hpe.com>
In large clusters, object layout calculation can become CPU-intensive. Under space pressure, the scheduler gives aggregation more CPU time, so EC aggregation should yield more frequently to avoid holding the CPU for too long. Signed-off-by: Wang Shilong <shilong.wang@hpe.com>
refresh mkdocs and doc/release/ for 2.8 Signed-off-by: Michael Hennecke <michael.hennecke@hpe.com>
SPDK, in all known applications—including daos_engine—is initialized only once during the lifetime of a process. DDB uses SPDK differently, allowing the user to initialize and re‑initialize SPDK multiple times within the same process. However, SPDK does not fully support re‑initialization. At the moment, this issue appears only when the SPDK configuration uses the VMD subsystem. When a VMD‑enabled SPDK configuration is in use, two conditions must be respected: - The VMD‑enabled configuration must be used during the first SPDK initialization. The VMD subsystem and DPDK will not be initialized on subsequent SPDK re‑initializations. - After a VMD‑enabled configuration has been used, the user cannot re‑initialize SPDK—whether with VMD enabled or disabled. The internal state of the VMD subsystem and DPDK becomes unsafe to use, even if the next SPDK configuration does not include VMD. These rules if not adhered to causes DDB to crash. Signed-off-by: Jan Michalski <jan-marian.michalski@hpe.com> Co-authored-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
client/dfs/mnt/dfs_query() updated to lookup file and directory object classes if default attributes fields are 0 Signed-off-by: Oleg Neverovitch oleg.neverovitch@hpe.com
- pool get-attr doesn't have '--path' option Signed-off-by: onever23 <oleg.neverovitch@hpe.com>
add draft 2.8 support matrix, incl. supported HPC fabrics Signed-off-by: Michael Hennecke <michael.hennecke@hpe.com>
In md-on-ssd phase2 mode, dkey iteration could yield on vos_iter_prepare() for loading object, and re-probe is required when it recurse back to object level. - missing btr_node_tx_add() when changing tn_flags. - missing btr_node_tx_add() on node splitting. - more messages & assertions. Signed-off-by: Niu Yawei <yawei.niu@hpe.com>
Usually, most of resent RPCs will be detected and handled on DTX leader. But when DTX leader is switched, such as old DTX leader is dead/evicted, the DTX for some inflight IO maybe in 'prepared' status on a non-leader while related client resends the RPC to new DTX leader. Under such case, DTX-resync may has not handled such DTX in time. Then IO handler on the non-leader needs to check whether related DTX has ever been prepared or not: if yes, directly reply to the DTX leader to avoid misguiding lower layer logic as to generate confused error. Signed-off-by: Fan Yong <fan.yong@hpe.com> Reviewed-by: Wang Shilong <shilong.wang@hpe.com> Reviewed-by: Xuezhao Liu <xuezhao.liu@hpe.com>
- for all providers except verbs - disable mrc on servers, enable on clients - for verbs -- disable mrc on servers and clients Signed-off-by: Alexander A Oganezov <alexander.oganezov@hpe.com>
…) (#18013) Updates `actions/upload-artifact` from 7.0.0 to 7.0.1 Signed-off-by: dependabot[bot] <support@github.com>
Replace the vos_obj_hold() call before vos_obj_incarnate() with vos_obj_acquire() to ensure the ts_set setup is fully done within transaction, that can make code cleaner and avoid following race: - Updater A called vos_obj_hold(), it failed to find the OI and mark the obj entry in ts_set as negative. - Updater A yield. - Concurrent updater B created OI and setup obj->obj_df. - Updater A resumed to call vos_obj_incarnate(), it found that the obj->obj_df is non-NULL so skipped OI creation. - The ts_set for updater A is now in an inconsistent state, it's negative but without 'se_create_idx' set. Signed-off-by: Niu Yawei <yawei.niu@hpe.com> Reviewed-by: Wang Shilong <shilong.wang@hpe.com> Reviewed-by: Fan Yong <fan.yong@hpe.com>
It appears that if the PS leader skips the ds_pool_iv_conn_hdls_update call during pool_svc_step_up_cb because there's no pool handle in the DB, IV fetches for pool handles will create invalid IV entries and return unexpected -DER_NOTLEADERs. To prevent that, this patch changes pool_svc_step_up_cb to call ds_pool_iv_conn_hdls_update even if there's no pool handle in the DB. Signed-off-by: Li Wei <liwei@hpe.com>
…18075) Wait for rebuild to start during the I/O loop once enough corruption injections have been issued, so fast providers such as verbs can also observe scrubber- triggered rebuild. Signed-off-by: Wang Shilong <shilong.wang@hpe.com>
|
Errors are component not formatted correctly,Ticket number prefix incorrect,PR title is malformatted. See https://daosio.atlassian.net/wiki/spaces/DC/pages/11133911069/Commit+Comments,Unable to load ticket data |
Nasf-Fan
force-pushed
the
Nasf-Fan/DAOS-19616_b28
branch
25 times, most recently
from
October 11, 2026 14:50
27b7deb to
40b2780
Compare
1. On each target, before scanning the container shard for rebuild, the logic will wait its global stable epoch (per container based) to exceed rebuild epoch. It will guarantee that all the inflight IO result that are triggered before rebuild start is visible to the subsequent rebuild logic whether on DTX leader or not. 2. Before handling some modification on DTX leader, need to check whether the epoch for current modification is newer than pool's rebuild epoch or not. If not, then return -DER_UPDATE_AGAIN to the client that will retry the modification sometime later. 3. In theory, it is possible that when exclude some engine, related ULT maybe trying to abort a DTX because of some failure happened during local modification (but related modification has succeeded on remote targets). Related DTX_ABORT RPC has been sent out but not been received or processed by related remote targets. Under such case, the DTX leader is switched because of related engine is exlcuded. And then DTX resync is triggered by new DTX leader who may regard such DTX as committable since all alive targets claim 'PREPARED'. Then the new DTX leader will make different decision that conflict with old DTX leader did. That may cause related DTX to be aborted on some targets but committed on the others, as to data inconsistency among related redundancy group. The new logic forbids to abort DTX from old DTX leader to avoid such inconsistency. 4. If the modification is in rebuilding/reintegrating, then related modification will use synchronous DTX to refresh related global stable epoch more quickly. 5. During DTX resync, the DTX CHECK RPC should not be sent to the target that will be reintegrated (with PO_COMP_ST_UP). Because such target may have removed related DTX entries. Add server side environment variable DAOS_REBUILD_CONTINUE_ON_TIMEOUT to control whether continue rebuild process or not if got timeout when wait for global stable epoch. It is false by default and rebuild will failout (and restart) when got timeout. If explicitly set it as true, then there is risk of data lost in subsequent rebuild process if race with inflight IO. It is a temporary solution. Signed-off-by: Fan Yong <fan.yong@hpe.com> Co-authored-by: Xuezhao Liu <xuezhao.liu@hpe.com> Co-authored-by: Wang Shilong <shilong.wang@hpe.com>
Nasf-Fan
force-pushed
the
Nasf-Fan/DAOS-19616_b28
branch
from
October 11, 2026 16:25
40b2780 to
9600127
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Steps for the author:
After all prior steps are complete: