KEYMGR Simulation Results

Thursday January 25 2024 20:03:30 UTC

GitHub Revision: `6d3d71452b`

Branch: os_regression

Testplan

Simulator: VCS

Build randomization enabled with --build-seed 2216296397

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	keymgr_smoke	1.276m	14.894ms	50	50	100.00
V1	random	keymgr_random	1.639m	46.001ms	50	50	100.00
V1	csr_hw_reset	keymgr_csr_hw_reset	1.450s	38.917us	5	5	100.00
V1	csr_rw	keymgr_csr_rw	1.460s	45.742us	18	20	90.00
V1	csr_bit_bash	keymgr_csr_bit_bash	18.200s	3.428ms	4	5	80.00
V1	csr_aliasing	keymgr_csr_aliasing	8.890s	368.944us	4	5	80.00
V1	csr_mem_rw_with_rand_reset	keymgr_csr_mem_rw_with_rand_reset	2.590s	114.553us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	keymgr_csr_rw	1.460s	45.742us	18	20	90.00
		keymgr_csr_aliasing	8.890s	368.944us	4	5	80.00
V1		TOTAL			151	155	97.42
V2	cfgen_during_op	keymgr_cfg_regwen	2.294m	4.937ms	49	50	98.00
V2	sideload	keymgr_sideload	1.031m	31.563ms	50	50	100.00
		keymgr_sideload_kmac	1.022m	1.982ms	50	50	100.00
		keymgr_sideload_aes	48.770s	1.927ms	50	50	100.00
		keymgr_sideload_otbn	1.072m	9.140ms	50	50	100.00
V2	direct_to_disabled_state	keymgr_direct_to_disabled	48.020s	1.631ms	50	50	100.00
V2	lc_disable	keymgr_lc_disable	7.040s	356.759us	49	50	98.00
V2	kmac_error_response	keymgr_kmac_rsp_err	1.351m	7.672ms	47	50	94.00
V2	invalid_sw_input	keymgr_sw_invalid_input	1.352m	7.733ms	50	50	100.00
V2	invalid_hw_input	keymgr_hwsw_invalid_input	2.541m	26.582ms	50	50	100.00
V2	sync_async_fault_cross	keymgr_sync_async_fault_cross	14.090s	3.672ms	50	50	100.00
V2	stress_all	keymgr_stress_all	3.972m	7.894ms	49	50	98.00
V2	intr_test	keymgr_intr_test	1.160s	28.896us	50	50	100.00
V2	alert_test	keymgr_alert_test	1.120s	25.937us	50	50	100.00
V2	tl_d_oob_addr_access	keymgr_tl_errors	4.410s	126.976us	20	20	100.00
V2	tl_d_illegal_access	keymgr_tl_errors	4.410s	126.976us	20	20	100.00
V2	tl_d_outstanding_access	keymgr_csr_hw_reset	1.450s	38.917us	5	5	100.00
		keymgr_csr_rw	1.460s	45.742us	18	20	90.00
		keymgr_csr_aliasing	8.890s	368.944us	4	5	80.00
		keymgr_same_csr_outstanding	2.910s	179.588us	14	20	70.00
V2	tl_d_partial_access	keymgr_csr_hw_reset	1.450s	38.917us	5	5	100.00
		keymgr_csr_rw	1.460s	45.742us	18	20	90.00
		keymgr_csr_aliasing	8.890s	368.944us	4	5	80.00
		keymgr_same_csr_outstanding	2.910s	179.588us	14	20	70.00
V2		TOTAL			728	740	98.38
V2S	sec_cm_additional_check	keymgr_sec_cm	1.049m	5.190ms	5	5	100.00
V2S	tl_intg_err	keymgr_sec_cm	1.049m	5.190ms	5	5	100.00
		keymgr_tl_intg_err	1.151m	7.754ms	14	20	70.00
V2S	shadow_reg_update_error	keymgr_shadow_reg_errors	17.320s	720.341us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	keymgr_shadow_reg_errors	17.320s	720.341us	20	20	100.00
V2S	shadow_reg_storage_error	keymgr_shadow_reg_errors	17.320s	720.341us	20	20	100.00
V2S	shadowed_reset_glitch	keymgr_shadow_reg_errors	17.320s	720.341us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	keymgr_shadow_reg_errors_with_csr_rw	13.820s	803.756us	15	20	75.00
V2S	prim_count_check	keymgr_sec_cm	1.049m	5.190ms	5	5	100.00
V2S	prim_fsm_check	keymgr_sec_cm	1.049m	5.190ms	5	5	100.00
V2S	sec_cm_bus_integrity	keymgr_tl_intg_err	1.151m	7.754ms	14	20	70.00
V2S	sec_cm_config_shadow	keymgr_shadow_reg_errors	17.320s	720.341us	20	20	100.00
V2S	sec_cm_op_config_regwen	keymgr_cfg_regwen	2.294m	4.937ms	49	50	98.00
V2S	sec_cm_reseed_config_regwen	keymgr_random	1.639m	46.001ms	50	50	100.00
		keymgr_csr_rw	1.460s	45.742us	18	20	90.00
V2S	sec_cm_sw_binding_config_regwen	keymgr_random	1.639m	46.001ms	50	50	100.00
		keymgr_csr_rw	1.460s	45.742us	18	20	90.00
V2S	sec_cm_max_key_ver_config_regwen	keymgr_random	1.639m	46.001ms	50	50	100.00
		keymgr_csr_rw	1.460s	45.742us	18	20	90.00
V2S	sec_cm_lc_ctrl_intersig_mubi	keymgr_lc_disable	7.040s	356.759us	49	50	98.00
V2S	sec_cm_constants_consistency	keymgr_hwsw_invalid_input	2.541m	26.582ms	50	50	100.00
V2S	sec_cm_intersig_consistency	keymgr_hwsw_invalid_input	2.541m	26.582ms	50	50	100.00
V2S	sec_cm_hw_key_sw_noaccess	keymgr_random	1.639m	46.001ms	50	50	100.00
V2S	sec_cm_output_keys_ctrl_redun	keymgr_sideload_protect	27.170s	3.407ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_sparse	keymgr_sec_cm	1.049m	5.190ms	5	5	100.00
V2S	sec_cm_data_fsm_sparse	keymgr_sec_cm	1.049m	5.190ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_local_esc	keymgr_sec_cm	1.049m	5.190ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_consistency	keymgr_custom_cm	1.206m	4.214ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_global_esc	keymgr_lc_disable	7.040s	356.759us	49	50	98.00
V2S	sec_cm_ctrl_ctr_redun	keymgr_sec_cm	1.049m	5.190ms	5	5	100.00
V2S	sec_cm_kmac_if_fsm_sparse	keymgr_sec_cm	1.049m	5.190ms	5	5	100.00
V2S	sec_cm_kmac_if_ctr_redun	keymgr_sec_cm	1.049m	5.190ms	5	5	100.00
V2S	sec_cm_kmac_if_cmd_ctrl_consistency	keymgr_custom_cm	1.206m	4.214ms	50	50	100.00
V2S	sec_cm_kmac_if_done_ctrl_consistency	keymgr_custom_cm	1.206m	4.214ms	50	50	100.00
V2S	sec_cm_reseed_ctr_redun	keymgr_sec_cm	1.049m	5.190ms	5	5	100.00
V2S	sec_cm_side_load_sel_ctrl_consistency	keymgr_custom_cm	1.206m	4.214ms	50	50	100.00
V2S	sec_cm_sideload_ctrl_fsm_sparse	keymgr_sec_cm	1.049m	5.190ms	5	5	100.00
V2S	sec_cm_ctrl_key_integrity	keymgr_custom_cm	1.206m	4.214ms	50	50	100.00
V2S		TOTAL			154	165	93.33
V3	stress_all_with_rand_reset	keymgr_stress_all_with_rand_reset	22.200s	2.762ms	44	50	88.00
V3		TOTAL			44	50	88.00
		TOTAL			1077	1110	97.03

Testplan Progress

Items	Total	Written	Passing	Progress
V1	7	7	4	57.14
V2	16	16	11	68.75
V2S	6	6	4	66.67
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
97.83	99.10	98.03	98.50	100.00	99.11	98.41	91.66

Failure Buckets

Offending '(d2h.d_error || ((d2h.d_data & *) == (exp_vals[*] & *)))' has 21 failures:

Test keymgr_same_csr_outstanding has 6 failures.

0.keymgr_same_csr_outstanding.1506588312
Line 255, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/0.keymgr_same_csr_outstanding/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[6] & 'hffffffff)))'
  UVM_ERROR @ 258188907 ps: (keymgr_csr_assert_fpv.sv:371) [ASSERT FAILED] sealing_sw_binding_1_rd_A
  UVM_INFO @ 258188907 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

2.keymgr_same_csr_outstanding.4202733004
Line 255, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/2.keymgr_same_csr_outstanding/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[19] & 'hffffffff)))'
  UVM_ERROR @  46830546 ps: (keymgr_csr_assert_fpv.sv:436) [ASSERT FAILED] attest_sw_binding_6_rd_A
  UVM_INFO @  46830546 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 4 more failures.

Test keymgr_csr_rw has 2 failures.

2.keymgr_csr_rw.77692010
Line 255, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/2.keymgr_csr_rw/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[7] & 'hffffffff)))'
  UVM_ERROR @  19589142 ps: (keymgr_csr_assert_fpv.sv:376) [ASSERT FAILED] sealing_sw_binding_2_rd_A
  UVM_INFO @  19589142 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

19.keymgr_csr_rw.3167026710
Line 256, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/19.keymgr_csr_rw/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[17] & 'hffffffff)))'
  UVM_ERROR @ 222030396 ps: (keymgr_csr_assert_fpv.sv:426) [ASSERT FAILED] attest_sw_binding_4_rd_A
  UVM_INFO @ 222030396 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test keymgr_csr_aliasing has 1 failures.

2.keymgr_csr_aliasing.2321156078
Line 255, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/2.keymgr_csr_aliasing/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[19] & 'hffffffff)))'
  UVM_ERROR @ 685427257 ps: (keymgr_csr_assert_fpv.sv:436) [ASSERT FAILED] attest_sw_binding_6_rd_A
  UVM_INFO @ 685427257 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test keymgr_csr_bit_bash has 1 failures.

3.keymgr_csr_bit_bash.3146640393
Line 255, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/3.keymgr_csr_bit_bash/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[7] & 'hffffffff)))'
  UVM_ERROR @ 1502810260 ps: (keymgr_csr_assert_fpv.sv:376) [ASSERT FAILED] sealing_sw_binding_2_rd_A
  UVM_INFO @ 1502810260 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test keymgr_tl_intg_err has 6 failures.

5.keymgr_tl_intg_err.1116511324
Line 305, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/5.keymgr_tl_intg_err/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[17] & 'hffffffff)))'
  UVM_ERROR @ 6834542659 ps: (keymgr_csr_assert_fpv.sv:426) [ASSERT FAILED] attest_sw_binding_4_rd_A
  UVM_INFO @ 6834542659 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

9.keymgr_tl_intg_err.1077376008
Line 265, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/9.keymgr_tl_intg_err/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[14] & 'hffffffff)))'
  UVM_ERROR @  13910053 ps: (keymgr_csr_assert_fpv.sv:411) [ASSERT FAILED] attest_sw_binding_1_rd_A
  UVM_INFO @  13910053 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 4 more failures.

... and 1 more tests.

UVM_ERROR (keymgr_scoreboard.sv:674) [scoreboard] Check failed item.d_data == addr_phase_op_status (* [*] vs * [*]) has 6 failures:

Test keymgr_stress_all_with_rand_reset has 6 failures.

2.keymgr_stress_all_with_rand_reset.196345129
Line 1116, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/2.keymgr_stress_all_with_rand_reset/latest/run.log

  UVM_ERROR @ 740662383 ps: (keymgr_scoreboard.sv:674) [uvm_test_top.env.scoreboard] Check failed item.d_data == addr_phase_op_status (1 [0x1] vs 2 [0x2])
  UVM_INFO @ 740662383 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

5.keymgr_stress_all_with_rand_reset.1533293741
Line 416, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/5.keymgr_stress_all_with_rand_reset/latest/run.log

  UVM_ERROR @  71198255 ps: (keymgr_scoreboard.sv:674) [uvm_test_top.env.scoreboard] Check failed item.d_data == addr_phase_op_status (1 [0x1] vs 2 [0x2])
  UVM_INFO @  71198255 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 4 more failures.

UVM_ERROR (keymgr_scoreboard.sv:750) [scoreboard] Check failed item.d_data ==gmv(csr) (* [] vs * []) reg name: keymgr_reg_block.sw_share1_output_*` has 2 failures:

Test keymgr_lc_disable has 1 failures.

3.keymgr_lc_disable.2809256577
Line 446, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/3.keymgr_lc_disable/latest/run.log

  UVM_ERROR @ 244749933 ps: (keymgr_scoreboard.sv:750) [uvm_test_top.env.scoreboard] Check failed item.d_data == `gmv(csr) (1618439013 [0x60776b65] vs 4111456055 [0xf50fd737]) reg name: keymgr_reg_block.sw_share1_output_0
  UVM_INFO @ 244749933 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test keymgr_stress_all has 1 failures.

34.keymgr_stress_all.2244296194
Line 1965, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/34.keymgr_stress_all/latest/run.log

  UVM_ERROR @ 2809865902 ps: (keymgr_scoreboard.sv:750) [uvm_test_top.env.scoreboard] Check failed item.d_data == `gmv(csr) (784549537 [0x2ec346a1] vs 0 [0x0]) reg name: keymgr_reg_block.sw_share1_output_7
  UVM_INFO @ 2809865902 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_ERROR (keymgr_scoreboard.sv:1019) [scoreboard] Check failed act != exp (* [*] vs * [*]) cdi_type: Attestation has 2 failures:

Test keymgr_kmac_rsp_err has 2 failures.

15.keymgr_kmac_rsp_err.1992034648
Line 281, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/15.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @  32561021 ps: (keymgr_scoreboard.sv:1019) [uvm_test_top.env.scoreboard] Check failed act != exp (1605930754565491250042339328357387143542526556207941497641627338815315533913320566769687182509137701065695722189679558343153940020735608349119969995870093535015371257132660123340833581788545627144360924574636511012844848459516227714577396400740941945866438143503124135799327475885683498299947770461929582807949124926219958827362812381635559637138173021942785231263302214931799316157863576983935435325727043936696218285202763 [0x3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f91f4a9f4f6a85e814c77b8e573fc767042c48bd05c76e1b3f20e0b49cc954fe9030e76644300fc5c9d77ee9eabf9d9d17eaddbc43574c46e67ccd230f456f179071837e3d79b75afc4c171353eac19d6899e2219085e5be36b12d832fd2c3ae40d94b19ef0a22c6e3267f88359e53306bf382ea119e9cce2c6d85a3f0f5eb962afda17b5f259c6f14d20e65f91c835d4b] vs 1605930754565491250042339328357387143542526556207941497641627338815315533913320566769687182509137701065695722189679558343153940020735608349119969995870093535015371257132660123340833581788545627144360924574636511012844848459516227714577396400740941945866438143503124135799327475885683498299947770461929582807949124926219958827362812381635559637138173021942785231263302214931799316157863576983935435325727043936696218285202763 [0x3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f91f4a9f4f6a85e814c77b8e573fc767042c48bd05c76e1b3f20e0b49cc954fe9030e76644300fc5c9d77ee9eabf9d9d17eaddbc43574c46e67ccd230f456f179071837e3d79b75afc4c171353eac19d6899e2219085e5be36b12d832fd2c3ae40d94b19ef0a22c6e3267f88359e53306bf382ea119e9cce2c6d85a3f0f5eb962afda17b5f259c6f14d20e65f91c835d4b]) cdi_type: Attestation
  
   DiversificationKey act: 0xf382ea119e9cce2c6d85a3f0f5eb962afda17b5f259c6f14d20e65f91c835d4b, exp: 0xf382ea119e9cce2c6d85a3f0f5eb962afda17b5f259c6f14d20e65f91c835d4b
   RomDigests act: 0xeaddbc43574c46e67ccd230f456f179071837e3d79b75afc4c171353eac19d6899e2219085e5be36b12d832fd2c3ae40d94b19ef0a22c6e3267f88359e53306b, exp: 0xeaddbc43574c46e67ccd230f456f179071837e3d79b75afc4c171353eac19d6899e2219085e5be36b12d832fd2c3ae40d94b19ef0a22c6e3267f88359e53306b
   HealthMeasurement act: 0x30e76644300fc5c9d77ee9eabf9d9d17, exp: 0x30e76644300fc5c9d77ee9eabf9d9d17

24.keymgr_kmac_rsp_err.3728930042
Line 397, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/24.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @  92818705 ps: (keymgr_scoreboard.sv:1019) [uvm_test_top.env.scoreboard] Check failed act != exp (565578307737125520330937310409163714961344609186800056550569205520268096790070411846809956906971449158322940045164081424349702758997277060152529146072856926102600042805470435061693469538033803437877332524057215851316247392379878734791203391518724142690878475575403807802987282819644963493850670501133652553160922322345741494529887933743431055734227435416958558096802963243403955156722791712901680063254548846934142449512616520940114629694115951788573938813327448927539306446168954659390273471580154924 [0xb087d376bce56fc276169caf3b48e6d1091ec4caf15bae4b8661a313b2f721223a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9c9c29e5459bd4814bfdb607a6357e51534b080b4a58e31392198242fd2d8981a18e6e617cc5306275e0ee91c5da1bdf0669bc247dd3eae18a5681bbcfee1fb201f395502e91d58cef57206920f676c4ce7f6b2704afd924d4347dc02bb3e3339922af6b9ad023e7d121bc1fef0e66172425361ef23a2aff53c91c5f113a56383adc6dc4798a4519afc2e188b75772c2c] vs 565578307737125520330937310409163714961344609186800056550569205520268096790070411846809956906971449158322940045164081424349702758997277060152529146072856926102600042805470435061693469538033803437877332524057215851316247392379878734791203391518724142690878475575403807802987282819644963493850670501133652553160922322345741494529887933743431055734227435416958558096802963243403955156722791712901680063254548846934142449512616520940114629694115951788573938813327448927539306446168954659390273471580154924 [0xb087d376bce56fc276169caf3b48e6d1091ec4caf15bae4b8661a313b2f721223a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9c9c29e5459bd4814bfdb607a6357e51534b080b4a58e31392198242fd2d8981a18e6e617cc5306275e0ee91c5da1bdf0669bc247dd3eae18a5681bbcfee1fb201f395502e91d58cef57206920f676c4ce7f6b2704afd924d4347dc02bb3e3339922af6b9ad023e7d121bc1fef0e66172425361ef23a2aff53c91c5f113a56383adc6dc4798a4519afc2e188b75772c2c]) cdi_type: Attestation
  
   DiversificationKey act: 0x425361ef23a2aff53c91c5f113a56383adc6dc4798a4519afc2e188b75772c2c, exp: 0x425361ef23a2aff53c91c5f113a56383adc6dc4798a4519afc2e188b75772c2c
   RomDigests act: 0x669bc247dd3eae18a5681bbcfee1fb201f395502e91d58cef57206920f676c4ce7f6b2704afd924d4347dc02bb3e3339922af6b9ad023e7d121bc1fef0e66172, exp: 0x669bc247dd3eae18a5681bbcfee1fb201f395502e91d58cef57206920f676c4ce7f6b2704afd924d4347dc02bb3e3339922af6b9ad023e7d121bc1fef0e66172
   HealthMeasurement act: 0x18e6e617cc5306275e0ee91c5da1bdf0, exp: 0x18e6e617cc5306275e0ee91c5da1bdf0

UVM_ERROR (keymgr_scoreboard.sv:750) [scoreboard] Check failed item.d_data ==gmv(csr) (* [] vs * []) reg name: keymgr_reg_block.start` has 1 failures:
- Test keymgr_cfg_regwen has 1 failures.
  - 1.keymgr_cfg_regwen.771603543
    Line 261, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/1.keymgr_cfg_regwen/latest/run.log
```
  UVM_ERROR @   3606707 ps: (keymgr_scoreboard.sv:750) [uvm_test_top.env.scoreboard] Check failed item.d_data == `gmv(csr) (0 [0x0] vs 1 [0x1]) reg name: keymgr_reg_block.start
  UVM_INFO @   3606707 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (cip_base_scoreboard.sv:281) scoreboard [scoreboard] alert recov_operation_err did not trigger max_delay:* has 1 failures:
- Test keymgr_kmac_rsp_err has 1 failures.
  - 28.keymgr_kmac_rsp_err.1530401101
    Line 709, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/28.keymgr_kmac_rsp_err/latest/run.log
```
  UVM_ERROR @ 126774118 ps: (cip_base_scoreboard.sv:281) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @ 126774118 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```