KEYMGR Simulation Results

Monday February 05 2024 20:02:51 UTC

GitHub Revision: `0d61ea543e`

Branch: os_regression

Testplan

Simulator: VCS

Build randomization enabled with --build-seed 226220803

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	keymgr_smoke	26.940s	1.778ms	50	50	100.00
V1	random	keymgr_random	1.109m	5.261ms	50	50	100.00
V1	csr_hw_reset	keymgr_csr_hw_reset	1.430s	28.062us	5	5	100.00
V1	csr_rw	keymgr_csr_rw	1.360s	432.253us	18	20	90.00
V1	csr_bit_bash	keymgr_csr_bit_bash	14.490s	557.418us	3	5	60.00
V1	csr_aliasing	keymgr_csr_aliasing	8.840s	779.688us	2	5	40.00
V1	csr_mem_rw_with_rand_reset	keymgr_csr_mem_rw_with_rand_reset	1.840s	145.640us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	keymgr_csr_rw	1.360s	432.253us	18	20	90.00
		keymgr_csr_aliasing	8.840s	779.688us	2	5	40.00
V1		TOTAL			148	155	95.48
V2	cfgen_during_op	keymgr_cfg_regwen	2.739m	3.129ms	48	50	96.00
V2	sideload	keymgr_sideload	1.176m	2.156ms	48	50	96.00
		keymgr_sideload_kmac	1.060m	9.389ms	50	50	100.00
		keymgr_sideload_aes	1.087m	9.746ms	50	50	100.00
		keymgr_sideload_otbn	47.420s	10.269ms	50	50	100.00
V2	direct_to_disabled_state	keymgr_direct_to_disabled	45.120s	4.164ms	50	50	100.00
V2	lc_disable	keymgr_lc_disable	24.450s	2.657ms	50	50	100.00
V2	kmac_error_response	keymgr_kmac_rsp_err	1.630m	2.641ms	45	50	90.00
V2	invalid_sw_input	keymgr_sw_invalid_input	46.940s	4.200ms	50	50	100.00
V2	invalid_hw_input	keymgr_hwsw_invalid_input	1.273m	3.119ms	49	50	98.00
V2	sync_async_fault_cross	keymgr_sync_async_fault_cross	24.300s	4.088ms	50	50	100.00
V2	stress_all	keymgr_stress_all	10.482m	44.858ms	47	50	94.00
V2	intr_test	keymgr_intr_test	0.970s	13.365us	50	50	100.00
V2	alert_test	keymgr_alert_test	0.960s	13.985us	50	50	100.00
V2	tl_d_oob_addr_access	keymgr_tl_errors	4.890s	244.254us	20	20	100.00
V2	tl_d_illegal_access	keymgr_tl_errors	4.890s	244.254us	20	20	100.00
V2	tl_d_outstanding_access	keymgr_csr_hw_reset	1.430s	28.062us	5	5	100.00
		keymgr_csr_rw	1.360s	432.253us	18	20	90.00
		keymgr_csr_aliasing	8.840s	779.688us	2	5	40.00
		keymgr_same_csr_outstanding	3.640s	452.956us	19	20	95.00
V2	tl_d_partial_access	keymgr_csr_hw_reset	1.430s	28.062us	5	5	100.00
		keymgr_csr_rw	1.360s	432.253us	18	20	90.00
		keymgr_csr_aliasing	8.840s	779.688us	2	5	40.00
		keymgr_same_csr_outstanding	3.640s	452.956us	19	20	95.00
V2		TOTAL			726	740	98.11
V2S	sec_cm_additional_check	keymgr_sec_cm	9.992m	35.342ms	5	5	100.00
V2S	tl_intg_err	keymgr_sec_cm	9.992m	35.342ms	5	5	100.00
		keymgr_tl_intg_err	9.390s	228.688us	16	20	80.00
V2S	shadow_reg_update_error	keymgr_shadow_reg_errors	16.710s	644.306us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	keymgr_shadow_reg_errors	16.710s	644.306us	20	20	100.00
V2S	shadow_reg_storage_error	keymgr_shadow_reg_errors	16.710s	644.306us	20	20	100.00
V2S	shadowed_reset_glitch	keymgr_shadow_reg_errors	16.710s	644.306us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	keymgr_shadow_reg_errors_with_csr_rw	12.300s	481.182us	14	20	70.00
V2S	prim_count_check	keymgr_sec_cm	9.992m	35.342ms	5	5	100.00
V2S	prim_fsm_check	keymgr_sec_cm	9.992m	35.342ms	5	5	100.00
V2S	sec_cm_bus_integrity	keymgr_tl_intg_err	9.390s	228.688us	16	20	80.00
V2S	sec_cm_config_shadow	keymgr_shadow_reg_errors	16.710s	644.306us	20	20	100.00
V2S	sec_cm_op_config_regwen	keymgr_cfg_regwen	2.739m	3.129ms	48	50	96.00
V2S	sec_cm_reseed_config_regwen	keymgr_random	1.109m	5.261ms	50	50	100.00
		keymgr_csr_rw	1.360s	432.253us	18	20	90.00
V2S	sec_cm_sw_binding_config_regwen	keymgr_random	1.109m	5.261ms	50	50	100.00
		keymgr_csr_rw	1.360s	432.253us	18	20	90.00
V2S	sec_cm_max_key_ver_config_regwen	keymgr_random	1.109m	5.261ms	50	50	100.00
		keymgr_csr_rw	1.360s	432.253us	18	20	90.00
V2S	sec_cm_lc_ctrl_intersig_mubi	keymgr_lc_disable	24.450s	2.657ms	50	50	100.00
V2S	sec_cm_constants_consistency	keymgr_hwsw_invalid_input	1.273m	3.119ms	49	50	98.00
V2S	sec_cm_intersig_consistency	keymgr_hwsw_invalid_input	1.273m	3.119ms	49	50	98.00
V2S	sec_cm_hw_key_sw_noaccess	keymgr_random	1.109m	5.261ms	50	50	100.00
V2S	sec_cm_output_keys_ctrl_redun	keymgr_sideload_protect	11.770s	1.290ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_sparse	keymgr_sec_cm	9.992m	35.342ms	5	5	100.00
V2S	sec_cm_data_fsm_sparse	keymgr_sec_cm	9.992m	35.342ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_local_esc	keymgr_sec_cm	9.992m	35.342ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_consistency	keymgr_custom_cm	54.000s	6.366ms	48	50	96.00
V2S	sec_cm_ctrl_fsm_global_esc	keymgr_lc_disable	24.450s	2.657ms	50	50	100.00
V2S	sec_cm_ctrl_ctr_redun	keymgr_sec_cm	9.992m	35.342ms	5	5	100.00
V2S	sec_cm_kmac_if_fsm_sparse	keymgr_sec_cm	9.992m	35.342ms	5	5	100.00
V2S	sec_cm_kmac_if_ctr_redun	keymgr_sec_cm	9.992m	35.342ms	5	5	100.00
V2S	sec_cm_kmac_if_cmd_ctrl_consistency	keymgr_custom_cm	54.000s	6.366ms	48	50	96.00
V2S	sec_cm_kmac_if_done_ctrl_consistency	keymgr_custom_cm	54.000s	6.366ms	48	50	96.00
V2S	sec_cm_reseed_ctr_redun	keymgr_sec_cm	9.992m	35.342ms	5	5	100.00
V2S	sec_cm_side_load_sel_ctrl_consistency	keymgr_custom_cm	54.000s	6.366ms	48	50	96.00
V2S	sec_cm_sideload_ctrl_fsm_sparse	keymgr_sec_cm	9.992m	35.342ms	5	5	100.00
V2S	sec_cm_ctrl_key_integrity	keymgr_custom_cm	54.000s	6.366ms	48	50	96.00
V2S		TOTAL			153	165	92.73
V3	stress_all_with_rand_reset	keymgr_stress_all_with_rand_reset	12.930s	212.775us	40	50	80.00
V3		TOTAL			40	50	80.00
		TOTAL			1067	1110	96.13

Testplan Progress

Items	Total	Written	Passing	Progress
V1	7	7	4	57.14
V2	16	16	10	62.50
V2S	6	6	3	50.00
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
97.84	99.10	97.99	98.69	100.00	99.11	98.41	91.58

Failure Buckets

Offending '(d2h.d_error || ((d2h.d_data & *) == (exp_vals[*] & *)))' has 18 failures:

Test keymgr_csr_bit_bash has 2 failures.

0.keymgr_csr_bit_bash.1854518773
Line 255, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/0.keymgr_csr_bit_bash/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[17] & 'hffffffff)))'
  UVM_ERROR @ 508972449 ps: (keymgr_csr_assert_fpv.sv:426) [ASSERT FAILED] attest_sw_binding_4_rd_A
  UVM_INFO @ 508972449 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

3.keymgr_csr_bit_bash.1862860273
Line 255, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/3.keymgr_csr_bit_bash/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[17] & 'hffffffff)))'
  UVM_ERROR @ 3955198200 ps: (keymgr_csr_assert_fpv.sv:426) [ASSERT FAILED] attest_sw_binding_4_rd_A
  UVM_INFO @ 3955198200 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test keymgr_tl_intg_err has 4 failures.

1.keymgr_tl_intg_err.2216479478
Line 258, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/1.keymgr_tl_intg_err/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[11] & 'hffffffff)))'
  UVM_ERROR @  10636910 ps: (keymgr_csr_assert_fpv.sv:396) [ASSERT FAILED] sealing_sw_binding_6_rd_A
  UVM_INFO @  10636910 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

2.keymgr_tl_intg_err.3195867192
Line 272, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/2.keymgr_tl_intg_err/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[19] & 'hffffffff)))'
  UVM_ERROR @  79868538 ps: (keymgr_csr_assert_fpv.sv:436) [ASSERT FAILED] attest_sw_binding_6_rd_A
  UVM_INFO @  79868538 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 2 more failures.

Test keymgr_csr_aliasing has 3 failures.

2.keymgr_csr_aliasing.1390129480
Line 255, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/2.keymgr_csr_aliasing/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[13] & 'hffffffff)))'
  UVM_ERROR @ 297097699 ps: (keymgr_csr_assert_fpv.sv:406) [ASSERT FAILED] attest_sw_binding_0_rd_A
  UVM_INFO @ 297097699 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

3.keymgr_csr_aliasing.3172521524
Line 255, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/3.keymgr_csr_aliasing/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[17] & 'hffffffff)))'
  UVM_ERROR @ 779688373 ps: (keymgr_csr_assert_fpv.sv:426) [ASSERT FAILED] attest_sw_binding_4_rd_A
  UVM_INFO @ 779688373 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 1 more failures.

Test keymgr_shadow_reg_errors_with_csr_rw has 6 failures.

4.keymgr_shadow_reg_errors_with_csr_rw.4031257987
Line 255, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/4.keymgr_shadow_reg_errors_with_csr_rw/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[13] & 'hffffffff)))'
  UVM_ERROR @  26778040 ps: (keymgr_csr_assert_fpv.sv:406) [ASSERT FAILED] attest_sw_binding_0_rd_A
  UVM_INFO @  26778040 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

5.keymgr_shadow_reg_errors_with_csr_rw.98975731
Line 255, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/5.keymgr_shadow_reg_errors_with_csr_rw/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[19] & 'hffffffff)))'
  UVM_ERROR @   9346363 ps: (keymgr_csr_assert_fpv.sv:436) [ASSERT FAILED] attest_sw_binding_6_rd_A
  UVM_INFO @   9346363 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 4 more failures.

Test keymgr_csr_rw has 2 failures.

11.keymgr_csr_rw.22870314
Line 256, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/11.keymgr_csr_rw/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[17] & 'hffffffff)))'
  UVM_ERROR @ 138542080 ps: (keymgr_csr_assert_fpv.sv:426) [ASSERT FAILED] attest_sw_binding_4_rd_A
  UVM_INFO @ 138542080 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

14.keymgr_csr_rw.1793426544
Line 255, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/14.keymgr_csr_rw/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[16] & 'hffffffff)))'
  UVM_ERROR @  17503426 ps: (keymgr_csr_assert_fpv.sv:421) [ASSERT FAILED] attest_sw_binding_3_rd_A
  UVM_INFO @  17503426 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 1 more tests.

UVM_ERROR (keymgr_scoreboard.sv:674) [scoreboard] Check failed item.d_data == addr_phase_op_status (* [*] vs * [*]) has 7 failures:

Test keymgr_stress_all_with_rand_reset has 7 failures.

0.keymgr_stress_all_with_rand_reset.2920748454
Line 714, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/0.keymgr_stress_all_with_rand_reset/latest/run.log

  UVM_ERROR @  72393670 ps: (keymgr_scoreboard.sv:674) [uvm_test_top.env.scoreboard] Check failed item.d_data == addr_phase_op_status (1 [0x1] vs 2 [0x2])
  UVM_INFO @  72393670 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

7.keymgr_stress_all_with_rand_reset.4048563141
Line 499, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/7.keymgr_stress_all_with_rand_reset/latest/run.log

  UVM_ERROR @ 124472978 ps: (keymgr_scoreboard.sv:674) [uvm_test_top.env.scoreboard] Check failed item.d_data == addr_phase_op_status (1 [0x1] vs 2 [0x2])
  UVM_INFO @ 124472978 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 5 more failures.

UVM_ERROR (cip_base_scoreboard.sv:281) scoreboard [scoreboard] alert recov_operation_err did not trigger max_delay:* has 6 failures:

Test keymgr_sideload has 2 failures.

2.keymgr_sideload.896014975
Line 288, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/2.keymgr_sideload/latest/run.log

  UVM_ERROR @  11152050 ps: (cip_base_scoreboard.sv:281) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @  11152050 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

26.keymgr_sideload.1717037488
Line 273, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/26.keymgr_sideload/latest/run.log

  UVM_ERROR @   2649086 ps: (cip_base_scoreboard.sv:281) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @   2649086 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test keymgr_cfg_regwen has 2 failures.

3.keymgr_cfg_regwen.2329098183
Line 317, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/3.keymgr_cfg_regwen/latest/run.log

  UVM_ERROR @   5633988 ps: (cip_base_scoreboard.sv:281) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @   5633988 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

41.keymgr_cfg_regwen.1874879301
Line 374, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/41.keymgr_cfg_regwen/latest/run.log

  UVM_ERROR @   8942354 ps: (cip_base_scoreboard.sv:281) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @   8942354 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test keymgr_stress_all has 1 failures.

44.keymgr_stress_all.185534038
Line 275, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/44.keymgr_stress_all/latest/run.log

  UVM_ERROR @  23094634 ps: (cip_base_scoreboard.sv:281) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @  23094634 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test keymgr_hwsw_invalid_input has 1 failures.

45.keymgr_hwsw_invalid_input.214634098
Line 844, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/45.keymgr_hwsw_invalid_input/latest/run.log

  UVM_ERROR @ 143395107 ps: (cip_base_scoreboard.sv:281) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @ 143395107 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_ERROR (keymgr_scoreboard.sv:1019) [scoreboard] Check failed act != exp (* [*] vs * [*]) cdi_type: Attestation has 5 failures:

Test keymgr_kmac_rsp_err has 5 failures.

10.keymgr_kmac_rsp_err.1427718366
Line 523, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/10.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @ 132899428 ps: (keymgr_scoreboard.sv:1019) [uvm_test_top.env.scoreboard] Check failed act != exp (42470344716759233896650459921296529632543422674617808282448344998573789502952981905683688477282694751718331899629797253626571673028681284183155443272369973263992858487178036531872165247239075486407174513074266570320481165679569795675907890461212016262828615363008993268526668227601738576912386372588183691236605990907461890944824453992344808554720616357615965257718787331753313612161999785228186597275730933747717348464480251791703426923731737219213291279597087425697409753283984583961903398821786771 [0xd418a2885196fa7b55c8db9955b3ca8d7fa4ca9aa249017a58ec4dbf9dff2783a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f974ca27466b78d2d01ccc1095b0a8d50ad1e9366cbae5d0664125440c5b07cd0db7771bb1596e5168d99e51b61b4512d2a81e3bcbbe7768ded78aa672a505af0f5c4503ac12369fc0890803a7ebd0070c932448283498ba696b133ee0c07c8659cf0c0ee6c53f9007f5c40e1281d1c798226b49146e03d7cd807f9c879fe809dd76c07b393c754db9d9f7a41f09bf7493] vs 42470344716759233896650459921296529632543422674617808282448344998573789502952981905683688477282694751718331899629797253626571673028681284183155443272369973263992858487178036531872165247239075486407174513074266570320481165679569795675907890461212016262828615363008993268526668227601738576912386372588183691236605990907461890944824453992344808554720616357615965257718787331753313612161999785228186597275730933747717348464480251791703426923731737219213291279597087425697409753283984583961903398821786771 [0xd418a2885196fa7b55c8db9955b3ca8d7fa4ca9aa249017a58ec4dbf9dff2783a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f974ca27466b78d2d01ccc1095b0a8d50ad1e9366cbae5d0664125440c5b07cd0db7771bb1596e5168d99e51b61b4512d2a81e3bcbbe7768ded78aa672a505af0f5c4503ac12369fc0890803a7ebd0070c932448283498ba696b133ee0c07c8659cf0c0ee6c53f9007f5c40e1281d1c798226b49146e03d7cd807f9c879fe809dd76c07b393c754db9d9f7a41f09bf7493]) cdi_type: Attestation
  
   DiversificationKey act: 0x226b49146e03d7cd807f9c879fe809dd76c07b393c754db9d9f7a41f09bf7493, exp: 0x226b49146e03d7cd807f9c879fe809dd76c07b393c754db9d9f7a41f09bf7493
   RomDigests act: 0xa81e3bcbbe7768ded78aa672a505af0f5c4503ac12369fc0890803a7ebd0070c932448283498ba696b133ee0c07c8659cf0c0ee6c53f9007f5c40e1281d1c798, exp: 0xa81e3bcbbe7768ded78aa672a505af0f5c4503ac12369fc0890803a7ebd0070c932448283498ba696b133ee0c07c8659cf0c0ee6c53f9007f5c40e1281d1c798
   HealthMeasurement act: 0xb7771bb1596e5168d99e51b61b4512d2, exp: 0xb7771bb1596e5168d99e51b61b4512d2

14.keymgr_kmac_rsp_err.793772879
Line 583, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/14.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @  35644852 ps: (keymgr_scoreboard.sv:1019) [uvm_test_top.env.scoreboard] Check failed act != exp (107964857665945418893719998301402259817396561562313985713325584781164530460689459745678350243016166295980685328150262377786553529371262326036155732371650643720775764167491764450003810555470374787764587050485666995638133360456332814973449013883743171248307023474342541677053762120426824345767499557543104710516282294704237494866511136909305684155206593157977803690320251607508630430351479095238584122426687716251719937074389600714015959380028881513425394807423721340367132237452750331645918869 [0x90bbcf41638e766200000000c42a3bf4fc6ffec600000000000000003a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f970022086812594e81e8c81a7a705197875ed989fb397163d3ede45420ca1cf51e9297d3df486fb08dfefce8d5c04a48f102a15ed9663f4a01acfb754a28a5fe1a39830f62b8ac563a250f6f6a5dee8f6f4f1542384abf2057ae4d46e611150f38612cd2ca504936c39cecd3f46c4c1fdfa5132797e074ad5132e381ca24ed3fb6a0b71c56c9d4dc0dc0984a212c5fe95] vs 107964857665945418893719998301402259817396561562313985713325584781164530460689459745678350243016166295980685328150262377786553529371262326036155732371650643720775764167491764450003810555470374787764587050485666995638133360456332814973449013883743171248307023474342541677053762120426824345767499557543104710516282294704237494866511136909305684155206593157977803690320251607508630430351479095238584122426687716251719937074389600714015959380028881513425394807423721340367132237452750331645918869 [0x90bbcf41638e766200000000c42a3bf4fc6ffec600000000000000003a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f970022086812594e81e8c81a7a705197875ed989fb397163d3ede45420ca1cf51e9297d3df486fb08dfefce8d5c04a48f102a15ed9663f4a01acfb754a28a5fe1a39830f62b8ac563a250f6f6a5dee8f6f4f1542384abf2057ae4d46e611150f38612cd2ca504936c39cecd3f46c4c1fdfa5132797e074ad5132e381ca24ed3fb6a0b71c56c9d4dc0dc0984a212c5fe95]) cdi_type: Attestation
  
   DiversificationKey act: 0xfa5132797e074ad5132e381ca24ed3fb6a0b71c56c9d4dc0dc0984a212c5fe95, exp: 0xfa5132797e074ad5132e381ca24ed3fb6a0b71c56c9d4dc0dc0984a212c5fe95
   RomDigests act: 0x102a15ed9663f4a01acfb754a28a5fe1a39830f62b8ac563a250f6f6a5dee8f6f4f1542384abf2057ae4d46e611150f38612cd2ca504936c39cecd3f46c4c1fd, exp: 0x102a15ed9663f4a01acfb754a28a5fe1a39830f62b8ac563a250f6f6a5dee8f6f4f1542384abf2057ae4d46e611150f38612cd2ca504936c39cecd3f46c4c1fd
   HealthMeasurement act: 0xe9297d3df486fb08dfefce8d5c04a48f, exp: 0xe9297d3df486fb08dfefce8d5c04a48f

... and 3 more failures.

UVM_ERROR (keymgr_scoreboard.sv:1017) [scoreboard] Check failed act == exp (* [*] vs * [*]) cdi_type: Attestation has 2 failures:

Test keymgr_stress_all_with_rand_reset has 2 failures.

25.keymgr_stress_all_with_rand_reset.2307828682
Line 1142, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/25.keymgr_stress_all_with_rand_reset/latest/run.log

  UVM_ERROR @ 472200408 ps: (keymgr_scoreboard.sv:1017) [uvm_test_top.env.scoreboard] Check failed act == exp (1605930754565491250042339328357387143542526556207941497641627338815315533913343792416847097020780030926134817559296691855112087050265005160101844746343053303243798981442381720823014214392146196538098266574928167323872432477366871274540485180933343421551683519177516921824757957893583103663892414734404707027132619669544981232601272130451055931864797522805987290437896570521232033518365031447112854340150882726469925183503719 [0x3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9807d0cc9150d9487a210f001f007406a7f81b40f105ac6beccd9bc9b9518dc70d587d35d5cc72087f44a65e00230c4760af2eb102544f84cea4649e6939826681f10b51f9f43c647c00bc82660c5e16b60215d39b4e113e7a96d1b855a7f5879af85f5a655a1e6e8a35d68d94d0e448c127ace981ba76499386f51e47dec8829f96c599764fe9f62e9fecc9d32d93d67] vs 282228480323324432690257092198252496171812082161307359123694466782635224490614963244567337796756110670449898528944785807759024220088509426047057357615080018073476550650716635062368023792076439918483041330926731580172787323689802648981351217756730936345188688234629569118758674498004070361413550243096771406773339852596545782344258472429826666278980384658411229568250634224824081440177769593519863563526892560113356040933074112872935737762601825296415469776917881681072377789296885937682203746770697575 [0x58171cf4000000000000000000000000000000000000000000000000709a67363a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9807d0cc9150d9487a210f001f007406a7f81b40f105ac6beccd9bc9b9518dc70d587d35d5cc72087f44a65e00230c4760af2eb102544f84cea4649e6939826681f10b51f9f43c647c00bc82660c5e16b60215d39b4e113e7a96d1b855a7f5879af85f5a655a1e6e8a35d68d94d0e448c127ace981ba76499386f51e47dec8829f96c599764fe9f62e9fecc9d32d93d67]) cdi_type: Attestation
  
   DiversificationKey act: 0x127ace981ba76499386f51e47dec8829f96c599764fe9f62e9fecc9d32d93d67, exp: 0x127ace981ba76499386f51e47dec8829f96c599764fe9f62e9fecc9d32d93d67
   RomDigests act: 0xaf2eb102544f84cea4649e6939826681f10b51f9f43c647c00bc82660c5e16b60215d39b4e113e7a96d1b855a7f5879af85f5a655a1e6e8a35d68d94d0e448c, exp: 0xaf2eb102544f84cea4649e6939826681f10b51f9f43c647c00bc82660c5e16b60215d39b4e113e7a96d1b855a7f5879af85f5a655a1e6e8a35d68d94d0e448c
   HealthMeasurement act: 0xd587d35d5cc72087f44a65e00230c476, exp: 0xd587d35d5cc72087f44a65e00230c476

43.keymgr_stress_all_with_rand_reset.3636051829
Line 704, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/43.keymgr_stress_all_with_rand_reset/latest/run.log

  UVM_ERROR @  52371557 ps: (keymgr_scoreboard.sv:1017) [uvm_test_top.env.scoreboard] Check failed act == exp (482293050894456972961859179217341032316562900058320572071476166085120463647158467591115874773577465619793113316422198930547558199381677782229760802663621487997893376523249311234414965113035112734352181812225756120888674369602687552073116309783091679622467905172615304009027513800722800398060007816913822191998432320989898592448962474634635780624385842647238451923725402476929746091795005480322153340200039092985217805499331508869273475717052278862034242231011771732935834433232777734197365763844264568 [0x9689059d983041756a3e518197326547a36fc8acee4ee07fd8aa7610122846c53a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9672ac686e3341580e27e7ed7a8dc6c64f2a16e571e7fcfa7c344823e171a4fd75271e610c2a9454dc3a84bce59863acb976f391103f008f1a682a35c58f7ad949ecc69e0f7031840e8d70f2a1cd7c43fbe71bdcba5bf83f13efe9441da438c1cf610150db9f41bc314477ad640ceca5b2b7a2dfdeae2f73c450b72fa9b8fa33a4fabaf3b6ab1a173392a2a11491ad278] vs 110854726438421283432129237208126811989024690138492386270341191127555462938137332886946588533136696511338818046008201940133402182437418333839982775266954488151265611367322595732219215502920294673587648839090148474791931950239728668952990635657003066889205683956094183280277524730163594307788429061329327515184544232110862903449592111045528118450653967632394652217632032636886045909679657794049293334249731947138637099030499179811617622648063517393109615819463303167580610081256143028804191844233302648 [0x2299b540bd46ff8f737cfc4c3c61c125d235a6fd4c29c1c4e9c8aa6ee764e9753a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9672ac686e3341580e27e7ed7a8dc6c64f2a16e571e7fcfa7c344823e171a4fd75271e610c2a9454dc3a84bce59863acb976f391103f008f1a682a35c58f7ad949ecc69e0f7031840e8d70f2a1cd7c43fbe71bdcba5bf83f13efe9441da438c1cf610150db9f41bc314477ad640ceca5b2b7a2dfdeae2f73c450b72fa9b8fa33a4fabaf3b6ab1a173392a2a11491ad278]) cdi_type: Attestation
  
   DiversificationKey act: 0x2b7a2dfdeae2f73c450b72fa9b8fa33a4fabaf3b6ab1a173392a2a11491ad278, exp: 0x2b7a2dfdeae2f73c450b72fa9b8fa33a4fabaf3b6ab1a173392a2a11491ad278
   RomDigests act: 0x976f391103f008f1a682a35c58f7ad949ecc69e0f7031840e8d70f2a1cd7c43fbe71bdcba5bf83f13efe9441da438c1cf610150db9f41bc314477ad640ceca5b, exp: 0x976f391103f008f1a682a35c58f7ad949ecc69e0f7031840e8d70f2a1cd7c43fbe71bdcba5bf83f13efe9441da438c1cf610150db9f41bc314477ad640ceca5b
   HealthMeasurement act: 0x5271e610c2a9454dc3a84bce59863acb, exp: 0x5271e610c2a9454dc3a84bce59863acb

UVM_FATAL (keymgr_custom_cm_vseq.sv:81) [keymgr_custom_cm_vseq] wait timeout occurred! has 1 failures:
- Test keymgr_custom_cm has 1 failures.
  - 1.keymgr_custom_cm.1888254537
    Line 506, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/1.keymgr_custom_cm/latest/run.log
```
  UVM_FATAL @ 11258460566 ps: (keymgr_custom_cm_vseq.sv:81) [uvm_test_top.env.virtual_sequencer.keymgr_custom_cm_vseq] wait timeout occurred!
  UVM_INFO @ 11258460566 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (keymgr_scoreboard.sv:750) [scoreboard] Check failed item.d_data ==gmv(csr) (* [] vs * []) reg name: keymgr_reg_block.sw_share1_output_*` has 1 failures:
- Test keymgr_stress_all has 1 failures.
  - 14.keymgr_stress_all.2558544678
    Line 4078, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/14.keymgr_stress_all/latest/run.log
```
  UVM_ERROR @ 6768202783 ps: (keymgr_scoreboard.sv:750) [uvm_test_top.env.scoreboard] Check failed item.d_data == `gmv(csr) (3290077365 [0xc41a98b5] vs 226677131 [0xd82d18b]) reg name: keymgr_reg_block.sw_share1_output_0
  UVM_INFO @ 6768202783 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (keymgr_scoreboard.sv:736) [scoreboard] Check failed item.d_data !=gmv(csr) (* [] vs * [])` has 1 failures:
- Test keymgr_stress_all_with_rand_reset has 1 failures.
  - 19.keymgr_stress_all_with_rand_reset.1394985142
    Line 558, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/19.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 349016422 ps: (keymgr_scoreboard.sv:736) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (2732109288 [0xa2d8ade8] vs 2732109288 [0xa2d8ade8])
  UVM_INFO @ 349016422 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (keymgr_scoreboard.sv:633) [scoreboard] Check failed act_state == addr_phase_working_state (* [*] vs * [*]) has 1 failures:
- Test keymgr_stress_all has 1 failures.
  - 33.keymgr_stress_all.769790220
    Line 1559, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/33.keymgr_stress_all/latest/run.log
```
  UVM_ERROR @ 1889490928 ps: (keymgr_scoreboard.sv:633) [uvm_test_top.env.scoreboard] Check failed act_state == addr_phase_working_state (1 [0x1] vs 6 [0x6])
  UVM_INFO @ 1889490928 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (cip_base_vseq.sv:818) [keymgr_custom_cm_vseq] Check failed cfg.m_alert_agent_cfgs[alert_name].vif.get_alert() == * (* [*] vs * [*]) fatal error fatal_fault_err does not trigger! has 1 failures:
- Test keymgr_custom_cm has 1 failures.
  - 34.keymgr_custom_cm.2513635199
    Line 452, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/34.keymgr_custom_cm/latest/run.log
```
  UVM_ERROR @ 476670236 ps: (cip_base_vseq.sv:818) [uvm_test_top.env.virtual_sequencer.keymgr_custom_cm_vseq] Check failed cfg.m_alert_agent_cfgs[alert_name].vif.get_alert() == 1 (0 [0x0] vs 1 [0x1]) fatal error fatal_fault_err does not trigger!
  UVM_INFO @ 476670236 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```