KEYMGR Simulation Results

Thursday February 22 2024 20:02:27 UTC

GitHub Revision: `952e4400c5`

Branch: os_regression

Testplan

Simulator: VCS

Build randomization enabled with --build-seed 3269902080

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	keymgr_smoke	37.920s	2.957ms	50	50	100.00
V1	random	keymgr_random	1.357m	27.976ms	50	50	100.00
V1	csr_hw_reset	keymgr_csr_hw_reset	1.560s	33.519us	5	5	100.00
V1	csr_rw	keymgr_csr_rw	1.270s	18.112us	16	20	80.00
V1	csr_bit_bash	keymgr_csr_bit_bash	23.520s	4.765ms	4	5	80.00
V1	csr_aliasing	keymgr_csr_aliasing	13.820s	377.227us	3	5	60.00
V1	csr_mem_rw_with_rand_reset	keymgr_csr_mem_rw_with_rand_reset	2.010s	98.288us	19	20	95.00
V1	regwen_csr_and_corresponding_lockable_csr	keymgr_csr_rw	1.270s	18.112us	16	20	80.00
		keymgr_csr_aliasing	13.820s	377.227us	3	5	60.00
V1		TOTAL			147	155	94.84
V2	cfgen_during_op	keymgr_cfg_regwen	1.654m	7.764ms	49	50	98.00
V2	sideload	keymgr_sideload	1.086m	3.650ms	50	50	100.00
		keymgr_sideload_kmac	1.497m	6.917ms	50	50	100.00
		keymgr_sideload_aes	1.528m	9.056ms	50	50	100.00
		keymgr_sideload_otbn	1.385m	7.787ms	50	50	100.00
V2	direct_to_disabled_state	keymgr_direct_to_disabled	1.036m	16.286ms	50	50	100.00
V2	lc_disable	keymgr_lc_disable	13.160s	467.864us	50	50	100.00
V2	kmac_error_response	keymgr_kmac_rsp_err	57.420s	13.546ms	46	50	92.00
V2	invalid_sw_input	keymgr_sw_invalid_input	36.260s	3.505ms	50	50	100.00
V2	invalid_hw_input	keymgr_hwsw_invalid_input	1.772m	11.052ms	50	50	100.00
V2	sync_async_fault_cross	keymgr_sync_async_fault_cross	34.130s	4.099ms	49	50	98.00
V2	stress_all	keymgr_stress_all	7.930m	79.556ms	49	50	98.00
V2	intr_test	keymgr_intr_test	0.870s	112.725us	50	50	100.00
V2	alert_test	keymgr_alert_test	1.010s	42.204us	50	50	100.00
V2	tl_d_oob_addr_access	keymgr_tl_errors	5.420s	274.581us	20	20	100.00
V2	tl_d_illegal_access	keymgr_tl_errors	5.420s	274.581us	20	20	100.00
V2	tl_d_outstanding_access	keymgr_csr_hw_reset	1.560s	33.519us	5	5	100.00
		keymgr_csr_rw	1.270s	18.112us	16	20	80.00
		keymgr_csr_aliasing	13.820s	377.227us	3	5	60.00
		keymgr_same_csr_outstanding	2.690s	323.261us	14	20	70.00
V2	tl_d_partial_access	keymgr_csr_hw_reset	1.560s	33.519us	5	5	100.00
		keymgr_csr_rw	1.270s	18.112us	16	20	80.00
		keymgr_csr_aliasing	13.820s	377.227us	3	5	60.00
		keymgr_same_csr_outstanding	2.690s	323.261us	14	20	70.00
V2		TOTAL			727	740	98.24
V2S	sec_cm_additional_check	keymgr_sec_cm	44.860s	10.476ms	5	5	100.00
V2S	tl_intg_err	keymgr_sec_cm	44.860s	10.476ms	5	5	100.00
		keymgr_tl_intg_err	1.304m	2.538ms	16	20	80.00
V2S	shadow_reg_update_error	keymgr_shadow_reg_errors	13.640s	681.596us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	keymgr_shadow_reg_errors	13.640s	681.596us	20	20	100.00
V2S	shadow_reg_storage_error	keymgr_shadow_reg_errors	13.640s	681.596us	20	20	100.00
V2S	shadowed_reset_glitch	keymgr_shadow_reg_errors	13.640s	681.596us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	keymgr_shadow_reg_errors_with_csr_rw	15.110s	1.606ms	10	20	50.00
V2S	prim_count_check	keymgr_sec_cm	44.860s	10.476ms	5	5	100.00
V2S	prim_fsm_check	keymgr_sec_cm	44.860s	10.476ms	5	5	100.00
V2S	sec_cm_bus_integrity	keymgr_tl_intg_err	1.304m	2.538ms	16	20	80.00
V2S	sec_cm_config_shadow	keymgr_shadow_reg_errors	13.640s	681.596us	20	20	100.00
V2S	sec_cm_op_config_regwen	keymgr_cfg_regwen	1.654m	7.764ms	49	50	98.00
V2S	sec_cm_reseed_config_regwen	keymgr_random	1.357m	27.976ms	50	50	100.00
		keymgr_csr_rw	1.270s	18.112us	16	20	80.00
V2S	sec_cm_sw_binding_config_regwen	keymgr_random	1.357m	27.976ms	50	50	100.00
		keymgr_csr_rw	1.270s	18.112us	16	20	80.00
V2S	sec_cm_max_key_ver_config_regwen	keymgr_random	1.357m	27.976ms	50	50	100.00
		keymgr_csr_rw	1.270s	18.112us	16	20	80.00
V2S	sec_cm_lc_ctrl_intersig_mubi	keymgr_lc_disable	13.160s	467.864us	50	50	100.00
V2S	sec_cm_constants_consistency	keymgr_hwsw_invalid_input	1.772m	11.052ms	50	50	100.00
V2S	sec_cm_intersig_consistency	keymgr_hwsw_invalid_input	1.772m	11.052ms	50	50	100.00
V2S	sec_cm_hw_key_sw_noaccess	keymgr_random	1.357m	27.976ms	50	50	100.00
V2S	sec_cm_output_keys_ctrl_redun	keymgr_sideload_protect	17.530s	825.948us	50	50	100.00
V2S	sec_cm_ctrl_fsm_sparse	keymgr_sec_cm	44.860s	10.476ms	5	5	100.00
V2S	sec_cm_data_fsm_sparse	keymgr_sec_cm	44.860s	10.476ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_local_esc	keymgr_sec_cm	44.860s	10.476ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_consistency	keymgr_custom_cm	1.216m	4.555ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_global_esc	keymgr_lc_disable	13.160s	467.864us	50	50	100.00
V2S	sec_cm_ctrl_ctr_redun	keymgr_sec_cm	44.860s	10.476ms	5	5	100.00
V2S	sec_cm_kmac_if_fsm_sparse	keymgr_sec_cm	44.860s	10.476ms	5	5	100.00
V2S	sec_cm_kmac_if_ctr_redun	keymgr_sec_cm	44.860s	10.476ms	5	5	100.00
V2S	sec_cm_kmac_if_cmd_ctrl_consistency	keymgr_custom_cm	1.216m	4.555ms	50	50	100.00
V2S	sec_cm_kmac_if_done_ctrl_consistency	keymgr_custom_cm	1.216m	4.555ms	50	50	100.00
V2S	sec_cm_reseed_ctr_redun	keymgr_sec_cm	44.860s	10.476ms	5	5	100.00
V2S	sec_cm_side_load_sel_ctrl_consistency	keymgr_custom_cm	1.216m	4.555ms	50	50	100.00
V2S	sec_cm_sideload_ctrl_fsm_sparse	keymgr_sec_cm	44.860s	10.476ms	5	5	100.00
V2S	sec_cm_ctrl_key_integrity	keymgr_custom_cm	1.216m	4.555ms	50	50	100.00
V2S		TOTAL			151	165	91.52
V3	stress_all_with_rand_reset	keymgr_stress_all_with_rand_reset	14.890s	1.156ms	43	50	86.00
V3		TOTAL			43	50	86.00
		TOTAL			1068	1110	96.22

Testplan Progress

Items	Total	Written	Passing	Progress
V1	7	7	3	42.86
V2	16	16	11	68.75
V2S	6	6	4	66.67
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
97.83	99.10	97.91	98.64	100.00	99.11	98.41	91.61

Failure Buckets

Offending '(d2h.d_error || ((d2h.d_data & *) == (exp_vals[*] & *)))' has 28 failures:

Test keymgr_shadow_reg_errors_with_csr_rw has 10 failures.

0.keymgr_shadow_reg_errors_with_csr_rw.3966898892
Line 258, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/0.keymgr_shadow_reg_errors_with_csr_rw/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[16] & 'hffffffff)))'
  UVM_ERROR @   8684859 ps: (keymgr_csr_assert_fpv.sv:421) [ASSERT FAILED] attest_sw_binding_3_rd_A
  UVM_INFO @   8684859 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

1.keymgr_shadow_reg_errors_with_csr_rw.402176749
Line 255, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/1.keymgr_shadow_reg_errors_with_csr_rw/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[8] & 'hffffffff)))'
  UVM_ERROR @  15749513 ps: (keymgr_csr_assert_fpv.sv:381) [ASSERT FAILED] sealing_sw_binding_3_rd_A
  UVM_INFO @  15749513 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 8 more failures.

Test keymgr_tl_intg_err has 4 failures.

1.keymgr_tl_intg_err.3808598631
Line 272, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/1.keymgr_tl_intg_err/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[18] & 'hffffffff)))'
  UVM_ERROR @  90342656 ps: (keymgr_csr_assert_fpv.sv:431) [ASSERT FAILED] attest_sw_binding_5_rd_A
  UVM_INFO @  90342656 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

3.keymgr_tl_intg_err.2238397913
Line 279, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/3.keymgr_tl_intg_err/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[6] & 'hffffffff)))'
  UVM_ERROR @ 184975097 ps: (keymgr_csr_assert_fpv.sv:371) [ASSERT FAILED] sealing_sw_binding_1_rd_A
  UVM_INFO @ 184975097 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 2 more failures.

Test keymgr_csr_aliasing has 2 failures.

1.keymgr_csr_aliasing.3991176949
Line 255, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/1.keymgr_csr_aliasing/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[15] & 'hffffffff)))'
  UVM_ERROR @ 544059965 ps: (keymgr_csr_assert_fpv.sv:416) [ASSERT FAILED] attest_sw_binding_2_rd_A
  UVM_INFO @ 544059965 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

4.keymgr_csr_aliasing.778508079
Line 255, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/4.keymgr_csr_aliasing/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[7] & 'hffffffff)))'
  UVM_ERROR @ 497128308 ps: (keymgr_csr_assert_fpv.sv:376) [ASSERT FAILED] sealing_sw_binding_2_rd_A
  UVM_INFO @ 497128308 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test keymgr_same_csr_outstanding has 6 failures.

1.keymgr_same_csr_outstanding.3832952011
Line 255, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/1.keymgr_same_csr_outstanding/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[14] & 'hffffffff)))'
  UVM_ERROR @ 149771302 ps: (keymgr_csr_assert_fpv.sv:411) [ASSERT FAILED] attest_sw_binding_1_rd_A
  UVM_INFO @ 149771302 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

6.keymgr_same_csr_outstanding.3052565541
Line 255, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/6.keymgr_same_csr_outstanding/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[8] & 'hffffffff)))'
  UVM_ERROR @ 325167578 ps: (keymgr_csr_assert_fpv.sv:381) [ASSERT FAILED] sealing_sw_binding_3_rd_A
  UVM_INFO @ 325167578 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 4 more failures.

Test keymgr_csr_bit_bash has 1 failures.

3.keymgr_csr_bit_bash.1089059241
Line 255, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/3.keymgr_csr_bit_bash/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[10] & 'hffffffff)))'
  UVM_ERROR @ 796587941 ps: (keymgr_csr_assert_fpv.sv:391) [ASSERT FAILED] sealing_sw_binding_5_rd_A
  UVM_INFO @ 796587941 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 2 more tests.

UVM_ERROR (keymgr_scoreboard.sv:1019) [scoreboard] Check failed act != exp (* [*] vs * [*]) cdi_type: Attestation has 4 failures:

Test keymgr_kmac_rsp_err has 4 failures.

21.keymgr_kmac_rsp_err.4209360825
Line 265, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/21.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @  68557656 ps: (keymgr_scoreboard.sv:1019) [uvm_test_top.env.scoreboard] Check failed act != exp (1605930754565491250042339328357387143542526556207941497641627338815315533913315164235698265151473594720783016455494486769939792816055781243019531535450590774806713733484859016567440419160423253412230185517569346086555888742239712247850362429873851631606275096022636997083914410499419650679497362434289154950058178075543115192649606194880916349344189663350255767611752278697032363794288055403962882915221762996794897307131600 [0x3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f908aeb4158d3996fcb43c8e73582fc0e2b2384ac8dc32b72027b18ac4ce7a9f3dd84aae3d38168eaeda9d608b03284d2d2936b5c4cd8f72cdfa5ea32dc1486ceb235d2dd59fbe39b9909518c1113849076ea172208a6fdc9b94b29a39e7876afe2fed674a27524bc583c7c0089154cc9c36ebe8f4fc0e03b8079771db4175dac9ad54e592f81b49ab298750df3ab1ded0] vs 1605930754565491250042339328357387143542526556207941497641627338815315533913315164235698265151473594720783016455494486769939792816055781243019531535450590774806713733484859016567440419160423253412230185517569346086555888742239712247850362429873851631606275096022636997083914410499419650679497362434289154950058178075543115192649606194880916349344189663350255767611752278697032363794288055403962882915221762996794897307131600 [0x3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f908aeb4158d3996fcb43c8e73582fc0e2b2384ac8dc32b72027b18ac4ce7a9f3dd84aae3d38168eaeda9d608b03284d2d2936b5c4cd8f72cdfa5ea32dc1486ceb235d2dd59fbe39b9909518c1113849076ea172208a6fdc9b94b29a39e7876afe2fed674a27524bc583c7c0089154cc9c36ebe8f4fc0e03b8079771db4175dac9ad54e592f81b49ab298750df3ab1ded0]) cdi_type: Attestation
  
   DiversificationKey act: 0x36ebe8f4fc0e03b8079771db4175dac9ad54e592f81b49ab298750df3ab1ded0, exp: 0x36ebe8f4fc0e03b8079771db4175dac9ad54e592f81b49ab298750df3ab1ded0
   RomDigests act: 0x2936b5c4cd8f72cdfa5ea32dc1486ceb235d2dd59fbe39b9909518c1113849076ea172208a6fdc9b94b29a39e7876afe2fed674a27524bc583c7c0089154cc9c, exp: 0x2936b5c4cd8f72cdfa5ea32dc1486ceb235d2dd59fbe39b9909518c1113849076ea172208a6fdc9b94b29a39e7876afe2fed674a27524bc583c7c0089154cc9c
   HealthMeasurement act: 0xd84aae3d38168eaeda9d608b03284d2d, exp: 0xd84aae3d38168eaeda9d608b03284d2d

25.keymgr_kmac_rsp_err.2716651485
Line 354, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/25.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @  70944741 ps: (keymgr_scoreboard.sv:1019) [uvm_test_top.env.scoreboard] Check failed act != exp (186405200219984053504460145677895304085899648830499127046112759181894122501764667386743159759457862492426443987817165030702395875780259894889043704119254436956606276947125706437234856104942713642239022564939571720701623878889859450430897567273566511238494799012314749383180944204184919158406812172772779244259637090051226822613993791814230997798372749501412707801712791865662737450413047639545350202295820501123965803801930478598627601441851958741530819389124977717203890543092644925655107418 [0xf9e347504d5105b4000000006aa513920000000000000000000000003a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f929196ca08a89053f240188161f9b1439591755d74fcc3c8e7e5ae07a658e21650b25e4a33f934d4df60e459629750b238aaf7fad2f9018dfd3c3efcee233e646d7422c11ec7ace4453c61de19a355e27f1dc7a3b7231834f09206f4d5b31b2a0e541325a44c86d4355f48a964fc54e9eaebb1e98c97e109f79a01aa8de2869012429a6f91447b9d7731422b46446475a] vs 186405200219984053504460145677895304085899648830499127046112759181894122501764667386743159759457862492426443987817165030702395875780259894889043704119254436956606276947125706437234856104942713642239022564939571720701623878889859450430897567273566511238494799012314749383180944204184919158406812172772779244259637090051226822613993791814230997798372749501412707801712791865662737450413047639545350202295820501123965803801930478598627601441851958741530819389124977717203890543092644925655107418 [0xf9e347504d5105b4000000006aa513920000000000000000000000003a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f929196ca08a89053f240188161f9b1439591755d74fcc3c8e7e5ae07a658e21650b25e4a33f934d4df60e459629750b238aaf7fad2f9018dfd3c3efcee233e646d7422c11ec7ace4453c61de19a355e27f1dc7a3b7231834f09206f4d5b31b2a0e541325a44c86d4355f48a964fc54e9eaebb1e98c97e109f79a01aa8de2869012429a6f91447b9d7731422b46446475a]) cdi_type: Attestation
  
   DiversificationKey act: 0xaebb1e98c97e109f79a01aa8de2869012429a6f91447b9d7731422b46446475a, exp: 0xaebb1e98c97e109f79a01aa8de2869012429a6f91447b9d7731422b46446475a
   RomDigests act: 0x8aaf7fad2f9018dfd3c3efcee233e646d7422c11ec7ace4453c61de19a355e27f1dc7a3b7231834f09206f4d5b31b2a0e541325a44c86d4355f48a964fc54e9e, exp: 0x8aaf7fad2f9018dfd3c3efcee233e646d7422c11ec7ace4453c61de19a355e27f1dc7a3b7231834f09206f4d5b31b2a0e541325a44c86d4355f48a964fc54e9e
   HealthMeasurement act: 0xb25e4a33f934d4df60e459629750b23, exp: 0xb25e4a33f934d4df60e459629750b23

... and 2 more failures.

UVM_ERROR (keymgr_scoreboard.sv:674) [scoreboard] Check failed item.d_data == addr_phase_op_status (* [*] vs * [*]) has 3 failures:
- Test keymgr_stress_all_with_rand_reset has 3 failures.
  - 14.keymgr_stress_all_with_rand_reset.3408501908
    Line 1206, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/14.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 323458834 ps: (keymgr_scoreboard.sv:674) [uvm_test_top.env.scoreboard] Check failed item.d_data == addr_phase_op_status (1 [0x1] vs 2 [0x2])
  UVM_INFO @ 323458834 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 28.keymgr_stress_all_with_rand_reset.2638197384
    Line 1240, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/28.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 663043341 ps: (keymgr_scoreboard.sv:674) [uvm_test_top.env.scoreboard] Check failed item.d_data == addr_phase_op_status (1 [0x1] vs 2 [0x2])
  UVM_INFO @ 663043341 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 1 more failures.

UVM_ERROR (keymgr_scoreboard.sv:1017) [scoreboard] Check failed act == exp (* [*] vs * [*]) cdi_type: Attestation has 2 failures:

Test keymgr_stress_all_with_rand_reset has 2 failures.

0.keymgr_stress_all_with_rand_reset.2866425689
Line 971, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/0.keymgr_stress_all_with_rand_reset/latest/run.log

  UVM_ERROR @ 543713728 ps: (keymgr_scoreboard.sv:1017) [uvm_test_top.env.scoreboard] Check failed act == exp (728324755937801495846352324893612756384215570738525594572581697585705491455739705938732666856261115860679018494372410345013232769757026289752786955609767207670781851939879130978291886513194155238895548211430937564787229778825068602410041732890727868180574695570434276555912127540421349078910378680359249877560343636655232571607024947297761795292684025196051974156980562342703508702360856577980255064082803201643541950471226940687279602680930370776695944320624892548691956380889209805322666924698441036 [0xe353e086e9965c345f027dd834d4c6e0a7d21b7ac81c6b1946c1bc2983b6a7343a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9753a4d53c80cda4f5305d6a75aea05717817bb6d97c25b244b920d362950de7b83dcfc47372cbf44f75154286a2a6dfa4e4f34fddab357b5dd1a05868e404a52a460abdabbe2f643ae33fe02d176586bd0b07a9a01f3ec39d490c616fe70c7ee0794992df85f30522e1f11c3f73c3f4560aa6ca7f6e93fdf9458e908ff39c7b1998da81c933344a04ada8ff55a7f154c] vs 704447636866115358131534362012618747162332146296713075502518282645985125757310106070000772623158027591556899190361384835091338006390720409849166219731929181498808757869246619397162638934429760600811892664195402913140419043716480238278916970540035275145728297023155526027718831750484997674229731798263909236794138705697669024891230445704473007305624409010474098129712089778330980668576427476104606493832339740104837923397025948233423479861487803470989150627774140728110871502045690820463881959235327308 [0xdbe0017a8efdc5aacab5e128db4dc44e306d07fb22dba7e1ba5d44fd234c82133a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9753a4d53c80cda4f5305d6a75aea05717817bb6d97c25b244b920d362950de7b83dcfc47372cbf44f75154286a2a6dfa4e4f34fddab357b5dd1a05868e404a52a460abdabbe2f643ae33fe02d176586bd0b07a9a01f3ec39d490c616fe70c7ee0794992df85f30522e1f11c3f73c3f4560aa6ca7f6e93fdf9458e908ff39c7b1998da81c933344a04ada8ff55a7f154c]) cdi_type: Attestation
  
   DiversificationKey act: 0x60aa6ca7f6e93fdf9458e908ff39c7b1998da81c933344a04ada8ff55a7f154c, exp: 0x60aa6ca7f6e93fdf9458e908ff39c7b1998da81c933344a04ada8ff55a7f154c
   RomDigests act: 0x4e4f34fddab357b5dd1a05868e404a52a460abdabbe2f643ae33fe02d176586bd0b07a9a01f3ec39d490c616fe70c7ee0794992df85f30522e1f11c3f73c3f45, exp: 0x4e4f34fddab357b5dd1a05868e404a52a460abdabbe2f643ae33fe02d176586bd0b07a9a01f3ec39d490c616fe70c7ee0794992df85f30522e1f11c3f73c3f45
   HealthMeasurement act: 0x83dcfc47372cbf44f75154286a2a6dfa, exp: 0x83dcfc47372cbf44f75154286a2a6dfa

38.keymgr_stress_all_with_rand_reset.3121223248
Line 1206, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/38.keymgr_stress_all_with_rand_reset/latest/run.log

  UVM_ERROR @ 597523876 ps: (keymgr_scoreboard.sv:1017) [uvm_test_top.env.scoreboard] Check failed act == exp (116144598137127101363509290096020494228958185164922403418729293494155882371331401341996396854390864267581418964585926368985221307736151173937876581130398447810123610403727972248963272067340854408540424221135309072132127738258393720996080878302768050031735949710148863336956363642772725522082158296217302016380938614845434231424468723699992778473579675404559635270696345279706212367727234429003901507011991018284648001479565543924454520591374499644684529978000550858107972814226222806895457751 [0x9bb2f72a0000000000000000be942ab70000000000cf42e6624aa73d3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9b65e81c092c571dc089c43aa02ecf7589083844b5eb6f0319b1583e10019e57564a22e6996c3a12f29a1e931c374fbfb9b8b4c86066c11619953744dd4a43b0cbb13584651abe5c26f52360580f7bb26b554ddc38b30d496c3beb38df0ae9209094f459aead47b2655d3dd8ea37f05eb0cb60f189da2b7d7197512094b891111679a0d0c14494b413fd521529f9075d7] vs 237288139860368115248479340291604605579397300339455791281113305309730844958416925327218252276299626352386532589197960580742594165582827305833773379457361130340085284949260079249122636365963308923846995402001640778906297374687056165234658440994424040807261387569552136511538850819839830969292819319627269889028403607116354121754675985050440971527150392650206278105707963341851937795310047804311121620502223193492940743927470723811508853994176242225399398649964065893084021819229760085701044235744081367 [0x4a103625000000000000000000000000000000000324630a00000000000000003a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9b65e81c092c571dc089c43aa02ecf7589083844b5eb6f0319b1583e10019e57564a22e6996c3a12f29a1e931c374fbfb9b8b4c86066c11619953744dd4a43b0cbb13584651abe5c26f52360580f7bb26b554ddc38b30d496c3beb38df0ae9209094f459aead47b2655d3dd8ea37f05eb0cb60f189da2b7d7197512094b891111679a0d0c14494b413fd521529f9075d7]) cdi_type: Attestation
  
   DiversificationKey act: 0xcb60f189da2b7d7197512094b891111679a0d0c14494b413fd521529f9075d7, exp: 0xcb60f189da2b7d7197512094b891111679a0d0c14494b413fd521529f9075d7
   RomDigests act: 0x9b8b4c86066c11619953744dd4a43b0cbb13584651abe5c26f52360580f7bb26b554ddc38b30d496c3beb38df0ae9209094f459aead47b2655d3dd8ea37f05eb, exp: 0x9b8b4c86066c11619953744dd4a43b0cbb13584651abe5c26f52360580f7bb26b554ddc38b30d496c3beb38df0ae9209094f459aead47b2655d3dd8ea37f05eb
   HealthMeasurement act: 0x64a22e6996c3a12f29a1e931c374fbfb, exp: 0x64a22e6996c3a12f29a1e931c374fbfb

UVM_ERROR (cip_base_scoreboard.sv:281) scoreboard [scoreboard] alert recov_operation_err did not trigger max_delay:* has 2 failures:

Test keymgr_stress_all has 1 failures.

8.keymgr_stress_all.3124186491
Line 1095, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/8.keymgr_stress_all/latest/run.log

  UVM_ERROR @ 222629840 ps: (cip_base_scoreboard.sv:281) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @ 222629840 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test keymgr_sync_async_fault_cross has 1 failures.

41.keymgr_sync_async_fault_cross.460659210
Line 268, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/41.keymgr_sync_async_fault_cross/latest/run.log

  UVM_ERROR @   1912108 ps: (cip_base_scoreboard.sv:281) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @   1912108 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_ERROR (keymgr_scoreboard.sv:736) [scoreboard] Check failed item.d_data !=gmv(csr) (* [] vs * [])` has 2 failures:

Test keymgr_stress_all_with_rand_reset has 2 failures.

8.keymgr_stress_all_with_rand_reset.1485254269
Line 613, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/8.keymgr_stress_all_with_rand_reset/latest/run.log

  UVM_ERROR @ 200038248 ps: (keymgr_scoreboard.sv:736) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (759015069 [0x2d3da69d] vs 759015069 [0x2d3da69d])
  UVM_INFO @ 200038248 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

44.keymgr_stress_all_with_rand_reset.2179973339
Line 694, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/44.keymgr_stress_all_with_rand_reset/latest/run.log

  UVM_ERROR @ 430688813 ps: (keymgr_scoreboard.sv:736) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (1576802926 [0x5dfc1a6e] vs 1576802926 [0x5dfc1a6e])
  UVM_INFO @ 430688813 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_ERROR (keymgr_scoreboard.sv:750) [scoreboard] Check failed item.d_data ==gmv(csr) (* [] vs * []) reg name: keymgr_reg_block.start` has 1 failures:
- Test keymgr_cfg_regwen has 1 failures.
  - 18.keymgr_cfg_regwen.1517335999
    Line 411, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/18.keymgr_cfg_regwen/latest/run.log
```
  UVM_ERROR @  40469047 ps: (keymgr_scoreboard.sv:750) [uvm_test_top.env.scoreboard] Check failed item.d_data == `gmv(csr) (0 [0x0] vs 1 [0x1]) reg name: keymgr_reg_block.start
  UVM_INFO @  40469047 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```