KEYMGR Simulation Results

Monday January 22 2024 20:02:58 UTC

GitHub Revision: `509f2f46b9`

Branch: os_regression

Testplan

Simulator: VCS

Build randomization enabled with --build-seed 3521792850

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	keymgr_smoke	43.620s	4.458ms	50	50	100.00
V1	random	keymgr_random	1.227m	10.410ms	48	50	96.00
V1	csr_hw_reset	keymgr_csr_hw_reset	1.470s	60.936us	5	5	100.00
V1	csr_rw	keymgr_csr_rw	1.550s	25.277us	16	20	80.00
V1	csr_bit_bash	keymgr_csr_bit_bash	12.830s	268.488us	3	5	60.00
V1	csr_aliasing	keymgr_csr_aliasing	13.650s	767.979us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	keymgr_csr_mem_rw_with_rand_reset	1.740s	40.628us	19	20	95.00
V1	regwen_csr_and_corresponding_lockable_csr	keymgr_csr_rw	1.550s	25.277us	16	20	80.00
		keymgr_csr_aliasing	13.650s	767.979us	5	5	100.00
V1		TOTAL			146	155	94.19
V2	cfgen_during_op	keymgr_cfg_regwen	2.305m	40.552ms	49	50	98.00
V2	sideload	keymgr_sideload	56.640s	4.058ms	48	50	96.00
		keymgr_sideload_kmac	1.340m	7.815ms	47	50	94.00
		keymgr_sideload_aes	1.151m	4.510ms	50	50	100.00
		keymgr_sideload_otbn	1.282m	2.289ms	47	50	94.00
V2	direct_to_disabled_state	keymgr_direct_to_disabled	1.076m	11.680ms	47	50	94.00
V2	lc_disable	keymgr_lc_disable	20.780s	689.782us	48	50	96.00
V2	kmac_error_response	keymgr_kmac_rsp_err	1.151m	3.143ms	44	50	88.00
V2	invalid_sw_input	keymgr_sw_invalid_input	1.578m	4.979ms	50	50	100.00
V2	invalid_hw_input	keymgr_hwsw_invalid_input	55.600s	6.701ms	49	50	98.00
V2	sync_async_fault_cross	keymgr_sync_async_fault_cross	18.350s	11.260ms	48	50	96.00
V2	stress_all	keymgr_stress_all	9.589m	35.526ms	48	50	96.00
V2	intr_test	keymgr_intr_test	1.160s	30.280us	50	50	100.00
V2	alert_test	keymgr_alert_test	1.130s	308.246us	50	50	100.00
V2	tl_d_oob_addr_access	keymgr_tl_errors	4.960s	173.963us	19	20	95.00
V2	tl_d_illegal_access	keymgr_tl_errors	4.960s	173.963us	19	20	95.00
V2	tl_d_outstanding_access	keymgr_csr_hw_reset	1.470s	60.936us	5	5	100.00
		keymgr_csr_rw	1.550s	25.277us	16	20	80.00
		keymgr_csr_aliasing	13.650s	767.979us	5	5	100.00
		keymgr_same_csr_outstanding	2.680s	112.553us	13	20	65.00
V2	tl_d_partial_access	keymgr_csr_hw_reset	1.470s	60.936us	5	5	100.00
		keymgr_csr_rw	1.550s	25.277us	16	20	80.00
		keymgr_csr_aliasing	13.650s	767.979us	5	5	100.00
		keymgr_same_csr_outstanding	2.680s	112.553us	13	20	65.00
V2		TOTAL			707	740	95.54
V2S	sec_cm_additional_check	keymgr_sec_cm	2.014m	5.576ms	5	5	100.00
V2S	tl_intg_err	keymgr_sec_cm	2.014m	5.576ms	5	5	100.00
		keymgr_tl_intg_err	18.000s	1.820ms	15	20	75.00
V2S	shadow_reg_update_error	keymgr_shadow_reg_errors	29.560s	1.729ms	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	keymgr_shadow_reg_errors	29.560s	1.729ms	20	20	100.00
V2S	shadow_reg_storage_error	keymgr_shadow_reg_errors	29.560s	1.729ms	20	20	100.00
V2S	shadowed_reset_glitch	keymgr_shadow_reg_errors	29.560s	1.729ms	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	keymgr_shadow_reg_errors_with_csr_rw	21.210s	724.795us	16	20	80.00
V2S	prim_count_check	keymgr_sec_cm	2.014m	5.576ms	5	5	100.00
V2S	prim_fsm_check	keymgr_sec_cm	2.014m	5.576ms	5	5	100.00
V2S	sec_cm_bus_integrity	keymgr_tl_intg_err	18.000s	1.820ms	15	20	75.00
V2S	sec_cm_config_shadow	keymgr_shadow_reg_errors	29.560s	1.729ms	20	20	100.00
V2S	sec_cm_op_config_regwen	keymgr_cfg_regwen	2.305m	40.552ms	49	50	98.00
V2S	sec_cm_reseed_config_regwen	keymgr_random	1.227m	10.410ms	48	50	96.00
		keymgr_csr_rw	1.550s	25.277us	16	20	80.00
V2S	sec_cm_sw_binding_config_regwen	keymgr_random	1.227m	10.410ms	48	50	96.00
		keymgr_csr_rw	1.550s	25.277us	16	20	80.00
V2S	sec_cm_max_key_ver_config_regwen	keymgr_random	1.227m	10.410ms	48	50	96.00
		keymgr_csr_rw	1.550s	25.277us	16	20	80.00
V2S	sec_cm_lc_ctrl_intersig_mubi	keymgr_lc_disable	20.780s	689.782us	48	50	96.00
V2S	sec_cm_constants_consistency	keymgr_hwsw_invalid_input	55.600s	6.701ms	49	50	98.00
V2S	sec_cm_intersig_consistency	keymgr_hwsw_invalid_input	55.600s	6.701ms	49	50	98.00
V2S	sec_cm_hw_key_sw_noaccess	keymgr_random	1.227m	10.410ms	48	50	96.00
V2S	sec_cm_output_keys_ctrl_redun	keymgr_sideload_protect	30.960s	1.591ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_sparse	keymgr_sec_cm	2.014m	5.576ms	5	5	100.00
V2S	sec_cm_data_fsm_sparse	keymgr_sec_cm	2.014m	5.576ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_local_esc	keymgr_sec_cm	2.014m	5.576ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_consistency	keymgr_custom_cm	38.680s	1.544ms	49	50	98.00
V2S	sec_cm_ctrl_fsm_global_esc	keymgr_lc_disable	20.780s	689.782us	48	50	96.00
V2S	sec_cm_ctrl_ctr_redun	keymgr_sec_cm	2.014m	5.576ms	5	5	100.00
V2S	sec_cm_kmac_if_fsm_sparse	keymgr_sec_cm	2.014m	5.576ms	5	5	100.00
V2S	sec_cm_kmac_if_ctr_redun	keymgr_sec_cm	2.014m	5.576ms	5	5	100.00
V2S	sec_cm_kmac_if_cmd_ctrl_consistency	keymgr_custom_cm	38.680s	1.544ms	49	50	98.00
V2S	sec_cm_kmac_if_done_ctrl_consistency	keymgr_custom_cm	38.680s	1.544ms	49	50	98.00
V2S	sec_cm_reseed_ctr_redun	keymgr_sec_cm	2.014m	5.576ms	5	5	100.00
V2S	sec_cm_side_load_sel_ctrl_consistency	keymgr_custom_cm	38.680s	1.544ms	49	50	98.00
V2S	sec_cm_sideload_ctrl_fsm_sparse	keymgr_sec_cm	2.014m	5.576ms	5	5	100.00
V2S	sec_cm_ctrl_key_integrity	keymgr_custom_cm	38.680s	1.544ms	49	50	98.00
V2S		TOTAL			155	165	93.94
V3	stress_all_with_rand_reset	keymgr_stress_all_with_rand_reset	17.740s	1.725ms	48	50	96.00
V3		TOTAL			48	50	96.00
		TOTAL			1056	1110	95.14

Testplan Progress

Items	Total	Written	Passing	Progress
V1	7	7	3	42.86
V2	16	16	4	25.00
V2S	6	6	3	50.00
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
97.83	99.10	98.07	98.58	100.00	99.11	98.41	91.58

Failure Buckets

Offending '(d2h.d_error || ((d2h.d_data & *) == (exp_vals[*] & *)))' has 23 failures:

Test keymgr_shadow_reg_errors_with_csr_rw has 4 failures.

0.keymgr_shadow_reg_errors_with_csr_rw.1811673147
Line 258, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/0.keymgr_shadow_reg_errors_with_csr_rw/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[8] & 'hffffffff)))'
  UVM_ERROR @  52203429 ps: (keymgr_csr_assert_fpv.sv:381) [ASSERT FAILED] sealing_sw_binding_3_rd_A
  UVM_INFO @  52203429 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

8.keymgr_shadow_reg_errors_with_csr_rw.3222268614
Line 255, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/8.keymgr_shadow_reg_errors_with_csr_rw/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[20] & 'hffffffff)))'
  UVM_ERROR @  12941614 ps: (keymgr_csr_assert_fpv.sv:441) [ASSERT FAILED] attest_sw_binding_7_rd_A
  UVM_INFO @  12941614 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 2 more failures.

Test keymgr_tl_intg_err has 5 failures.

1.keymgr_tl_intg_err.3630019773
Line 271, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/1.keymgr_tl_intg_err/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[13] & 'hffffffff)))'
  UVM_ERROR @  23885962 ps: (keymgr_csr_assert_fpv.sv:406) [ASSERT FAILED] attest_sw_binding_0_rd_A
  UVM_INFO @  23885962 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

2.keymgr_tl_intg_err.413894637
Line 262, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/2.keymgr_tl_intg_err/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[9] & 'hffffffff)))'
  UVM_ERROR @  12383231 ps: (keymgr_csr_assert_fpv.sv:386) [ASSERT FAILED] sealing_sw_binding_4_rd_A
  UVM_INFO @  12383231 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 3 more failures.

Test keymgr_csr_rw has 4 failures.

1.keymgr_csr_rw.2259281889
Line 254, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/1.keymgr_csr_rw/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[6] & 'hffffffff)))'
  UVM_ERROR @  10133690 ps: (keymgr_csr_assert_fpv.sv:371) [ASSERT FAILED] sealing_sw_binding_1_rd_A
  UVM_INFO @  10133690 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

2.keymgr_csr_rw.2701893686
Line 255, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/2.keymgr_csr_rw/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[15] & 'hffffffff)))'
  UVM_ERROR @  12004248 ps: (keymgr_csr_assert_fpv.sv:416) [ASSERT FAILED] attest_sw_binding_2_rd_A
  UVM_INFO @  12004248 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 2 more failures.

Test keymgr_csr_bit_bash has 2 failures.

1.keymgr_csr_bit_bash.2167881009
Line 255, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/1.keymgr_csr_bit_bash/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[8] & 'hffffffff)))'
  UVM_ERROR @ 359358886 ps: (keymgr_csr_assert_fpv.sv:381) [ASSERT FAILED] sealing_sw_binding_3_rd_A
  UVM_INFO @ 359358886 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

3.keymgr_csr_bit_bash.1834623650
Line 255, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/3.keymgr_csr_bit_bash/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[19] & 'hffffffff)))'
  UVM_ERROR @ 350144798 ps: (keymgr_csr_assert_fpv.sv:436) [ASSERT FAILED] attest_sw_binding_6_rd_A
  UVM_INFO @ 350144798 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test keymgr_same_csr_outstanding has 7 failures.

1.keymgr_same_csr_outstanding.3770036922
Line 255, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/1.keymgr_same_csr_outstanding/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[20] & 'hffffffff)))'
  UVM_ERROR @  49507325 ps: (keymgr_csr_assert_fpv.sv:441) [ASSERT FAILED] attest_sw_binding_7_rd_A
  UVM_INFO @  49507325 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

4.keymgr_same_csr_outstanding.2612648294
Line 255, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/4.keymgr_same_csr_outstanding/latest/run.log

      Offending '(d2h.d_error || ((d2h.d_data & 'hffffffff) == (exp_vals[16] & 'hffffffff)))'
  UVM_ERROR @  41440490 ps: (keymgr_csr_assert_fpv.sv:421) [ASSERT FAILED] attest_sw_binding_3_rd_A
  UVM_INFO @  41440490 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 5 more failures.

... and 1 more tests.

Job keymgr-sim-vcs_run_default killed due to: Exit reason: Error: * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *] has 7 failures:
- Test keymgr_direct_to_disabled has 1 failures.
  - 1.keymgr_direct_to_disabled.950631206
    Log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/1.keymgr_direct_to_disabled/latest/run.log
```
  Job ID: smart:0b8b56ea-9970-4aed-8d04-9657813baf90
```
- Test keymgr_sideload_kmac has 1 failures.
  - 15.keymgr_sideload_kmac.844202238
    Log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/15.keymgr_sideload_kmac/latest/run.log
```
  Job ID: smart:81ad909b-b055-465e-9105-1ed4371089c6
```
- Test keymgr_random has 1 failures.
  - 15.keymgr_random.2856923112
    Log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/15.keymgr_random/latest/run.log
```
  Job ID: smart:36f3db21-d4ac-4743-9c8a-8073f515ffa3
```
- Test keymgr_cfg_regwen has 1 failures.
  - 15.keymgr_cfg_regwen.3571288329
    Log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/15.keymgr_cfg_regwen/latest/run.log
```
  Job ID: smart:27a4d22f-2ada-46a8-a400-9b55d0a7c168
```
- Test keymgr_sideload_otbn has 1 failures.
  - 17.keymgr_sideload_otbn.3133135309
    Log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/17.keymgr_sideload_otbn/latest/run.log
```
  Job ID: smart:fc4001d5-4b8b-46e8-84de-ec19b0e67e0f
```
- ... and 2 more tests.

UVM_ERROR (keymgr_scoreboard.sv:1019) [scoreboard] Check failed act != exp (* [*] vs * [*]) cdi_type: Attestation has 6 failures:

Test keymgr_kmac_rsp_err has 6 failures.

5.keymgr_kmac_rsp_err.1255629690
Line 357, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/5.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @  16468188 ps: (keymgr_scoreboard.sv:1019) [uvm_test_top.env.scoreboard] Check failed act != exp (684694319956708761557606310199492018446428158828356470887358636353936599802419538829624016168491622037638140841955287080608427118318074253855455420556452100756580492369007515581826363510373055671504938252747809497994107489802431433714367181090768782376390186983866135276818828175933711994607490305801938139509651335472057725575573247125234693262128540338706462484126340739760921718185838924070612407823166988865870626135288869535196969153350534898997642259728107300956101192484170165805411677222071160 [0xd5b5a449000000000000000000000000000000000000000000000000000000003a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9000000000000000000000000000000000000000000000000000000000000f0f000000000000000005cfbd765ce33f34ea20a046cf42e6eac560a3f82bfa76285b5c1d4aea7c915e49a32d1c89be0f507a20a046cf42e6eac560a3f82bfa76285b5c1d4aea7c915e49a32d1c89be0f507fa365df52da48cd752fb3a026a8e608f0098cfe5fa9810494829d0cd9479eb78] vs 684694319956708761557606310199492018446428158828356470887358636353936599802419538829624016168491622037638140841955287080608427118318074253855455420556452100756580492369007515581826363510373055671504938252747809497994107489802431433714367181090768782376390186983866135276818828175933711994607490305801938139509651335472057725575573247125234693262128540338706462484126340739760921718185838924070612407823166988865870626135288869535196969153350534898997642259728107300956101192484170165805411677222071160 [0xd5b5a449000000000000000000000000000000000000000000000000000000003a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9000000000000000000000000000000000000000000000000000000000000f0f000000000000000005cfbd765ce33f34ea20a046cf42e6eac560a3f82bfa76285b5c1d4aea7c915e49a32d1c89be0f507a20a046cf42e6eac560a3f82bfa76285b5c1d4aea7c915e49a32d1c89be0f507fa365df52da48cd752fb3a026a8e608f0098cfe5fa9810494829d0cd9479eb78]) cdi_type: Attestation
  
   DiversificationKey act: 0xfa365df52da48cd752fb3a026a8e608f0098cfe5fa9810494829d0cd9479eb78, exp: 0xfa365df52da48cd752fb3a026a8e608f0098cfe5fa9810494829d0cd9479eb78
   RomDigests act: 0xa20a046cf42e6eac560a3f82bfa76285b5c1d4aea7c915e49a32d1c89be0f507a20a046cf42e6eac560a3f82bfa76285b5c1d4aea7c915e49a32d1c89be0f507, exp: 0xa20a046cf42e6eac560a3f82bfa76285b5c1d4aea7c915e49a32d1c89be0f507a20a046cf42e6eac560a3f82bfa76285b5c1d4aea7c915e49a32d1c89be0f507
   HealthMeasurement act: 0x5cfbd765ce33f34e, exp: 0x5cfbd765ce33f34e

9.keymgr_kmac_rsp_err.2066422251
Line 514, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/9.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @ 259706806 ps: (keymgr_scoreboard.sv:1019) [uvm_test_top.env.scoreboard] Check failed act != exp (357834974160613204051181520284181172677131377898226017552964950666474179437384426756151676014163532943627534105075732115647633017080777155257542255375103848028438688236134306760386480214157022295951514467078073850101318019233891127218600084268356606910360862109998383023542198076829682212658352314297635335112678269896900654366650826671950527655025778286796101441570339699207941683423160694243419983362177563616647628694569319091562120163442047653621792931309865629149411098979201600011089030330002266 [0x6fb05af20ec5ea49a7dde53dcf1d04483acdf61d5d78075378ac3bec0556214f3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9016470eea5b7b8ba7373f189b06c4db69f69258a1fd13b43d1f0ba09af498a0c331394a8bea98b93360a46ccb857e9997b3090e12c7e25ce7e2e81382246160d71526e1b1fa20e72599985fee5797ce9333aaf0754cdb2a4e140d596e6750845c19454ffbbbf15b8ac99ea644705fbfb0b7a0e1ce54af8379aad351d69f1fe372db0c66841b7951f295ef39c72b92b5a] vs 357834974160613204051181520284181172677131377898226017552964950666474179437384426756151676014163532943627534105075732115647633017080777155257542255375103848028438688236134306760386480214157022295951514467078073850101318019233891127218600084268356606910360862109998383023542198076829682212658352314297635335112678269896900654366650826671950527655025778286796101441570339699207941683423160694243419983362177563616647628694569319091562120163442047653621792931309865629149411098979201600011089030330002266 [0x6fb05af20ec5ea49a7dde53dcf1d04483acdf61d5d78075378ac3bec0556214f3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9016470eea5b7b8ba7373f189b06c4db69f69258a1fd13b43d1f0ba09af498a0c331394a8bea98b93360a46ccb857e9997b3090e12c7e25ce7e2e81382246160d71526e1b1fa20e72599985fee5797ce9333aaf0754cdb2a4e140d596e6750845c19454ffbbbf15b8ac99ea644705fbfb0b7a0e1ce54af8379aad351d69f1fe372db0c66841b7951f295ef39c72b92b5a]) cdi_type: Attestation
  
   DiversificationKey act: 0xb7a0e1ce54af8379aad351d69f1fe372db0c66841b7951f295ef39c72b92b5a, exp: 0xb7a0e1ce54af8379aad351d69f1fe372db0c66841b7951f295ef39c72b92b5a
   RomDigests act: 0x7b3090e12c7e25ce7e2e81382246160d71526e1b1fa20e72599985fee5797ce9333aaf0754cdb2a4e140d596e6750845c19454ffbbbf15b8ac99ea644705fbfb, exp: 0x7b3090e12c7e25ce7e2e81382246160d71526e1b1fa20e72599985fee5797ce9333aaf0754cdb2a4e140d596e6750845c19454ffbbbf15b8ac99ea644705fbfb
   HealthMeasurement act: 0x331394a8bea98b93360a46ccb857e999, exp: 0x331394a8bea98b93360a46ccb857e999

... and 4 more failures.

Job keymgr-sim-vcs_run_default killed due to: Exit reason: Error: * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *] has 5 failures:
- Test keymgr_sideload_kmac has 1 failures.
  - 2.keymgr_sideload_kmac.2819881260
    Log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/2.keymgr_sideload_kmac/latest/run.log
```
  Job ID: smart:5fc85e90-fa80-4916-92b0-77ec892e9284
```
- Test keymgr_sync_async_fault_cross has 1 failures.
  - 10.keymgr_sync_async_fault_cross.4272615530
    Log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/10.keymgr_sync_async_fault_cross/latest/run.log
```
  Job ID: smart:3ac8dc22-c776-470a-9686-8f69ffa7c680
```
- Test keymgr_hwsw_invalid_input has 1 failures.
  - 19.keymgr_hwsw_invalid_input.1444554021
    Log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/19.keymgr_hwsw_invalid_input/latest/run.log
```
  Job ID: smart:59c0a5c3-b853-4e6d-9e01-27728f3eee03
```
- Test keymgr_sideload_otbn has 1 failures.
  - 29.keymgr_sideload_otbn.305769371
    Log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/29.keymgr_sideload_otbn/latest/run.log
```
  Job ID: smart:ae7b0b04-a52c-4be9-8375-784f411eee9c
```
- Test keymgr_sideload has 1 failures.
  - 32.keymgr_sideload.3439162554
    Log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/32.keymgr_sideload/latest/run.log
```
  Job ID: smart:a2774211-b51e-40b9-b232-a10354190340
```

UVM_ERROR (cip_base_scoreboard.sv:281) scoreboard [scoreboard] alert recov_operation_err did not trigger max_delay:* has 4 failures:

Test keymgr_sideload_otbn has 1 failures.

1.keymgr_sideload_otbn.2111142002
Line 268, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/1.keymgr_sideload_otbn/latest/run.log

  UVM_ERROR @   4138912 ps: (cip_base_scoreboard.sv:281) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @   4138912 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test keymgr_random has 1 failures.

10.keymgr_random.3226457199
Line 319, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/10.keymgr_random/latest/run.log

  UVM_ERROR @   9151230 ps: (cip_base_scoreboard.sv:281) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @   9151230 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test keymgr_sideload has 1 failures.

20.keymgr_sideload.3263253177
Line 302, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/20.keymgr_sideload/latest/run.log

  UVM_ERROR @  10519041 ps: (cip_base_scoreboard.sv:281) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @  10519041 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test keymgr_stress_all has 1 failures.

30.keymgr_stress_all.1269309640
Line 3976, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/30.keymgr_stress_all/latest/run.log

  UVM_ERROR @ 647470985 ps: (cip_base_scoreboard.sv:281) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @ 647470985 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Job keymgr-sim-vcs_run_default killed due to: Exit reason: Error: * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *] has 4 failures:
- Test keymgr_direct_to_disabled has 1 failures.
  - 2.keymgr_direct_to_disabled.752876635
    Log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/2.keymgr_direct_to_disabled/latest/run.log
```
  Job ID: smart:4816a307-790e-4b8c-9cce-7b25b4f114cd
```
- Test keymgr_stress_all_with_rand_reset has 1 failures.
  - 17.keymgr_stress_all_with_rand_reset.2604594206
    Log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/17.keymgr_stress_all_with_rand_reset/latest/run.log
```
  Job ID: smart:31fdea77-63ff-4a0c-b197-2e8a82b9575a
```
- Test keymgr_lc_disable has 1 failures.
  - 25.keymgr_lc_disable.126610448
    Log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/25.keymgr_lc_disable/latest/run.log
```
  Job ID: smart:2512fc40-0993-4ab7-a565-d89954b15fde
```
- Test keymgr_sideload_kmac has 1 failures.
  - 47.keymgr_sideload_kmac.2974330223
    Log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/47.keymgr_sideload_kmac/latest/run.log
```
  Job ID: smart:2183f6d0-d247-4e1a-a5b9-80f033c97c79
```
Job keymgr-sim-vcs_run_default killed due to: Exit reason: Error: * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *] has 2 failures:
- Test keymgr_direct_to_disabled has 1 failures.
  - 31.keymgr_direct_to_disabled.2569871745
    Log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/31.keymgr_direct_to_disabled/latest/run.log
```
  Job ID: smart:44da5687-7f42-4f6e-81cb-10651f0b3fda
```
- Test keymgr_sync_async_fault_cross has 1 failures.
  - 46.keymgr_sync_async_fault_cross.3429547554
    Log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/46.keymgr_sync_async_fault_cross/latest/run.log
```
  Job ID: smart:01d897c7-0ca2-4f05-a708-9fc007accd7f
```
UVM_ERROR (keymgr_scoreboard.sv:750) [scoreboard] Check failed item.d_data ==gmv(csr) (* [] vs * []) reg name: keymgr_reg_block.sw_share0_output_*` has 1 failures:
- Test keymgr_stress_all has 1 failures.
  - 10.keymgr_stress_all.792163091
    Line 1501, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/10.keymgr_stress_all/latest/run.log
```
  UVM_ERROR @ 1847813405 ps: (keymgr_scoreboard.sv:750) [uvm_test_top.env.scoreboard] Check failed item.d_data == `gmv(csr) (2227104401 [0x84beea91] vs 4247105252 [0xfd25aee4]) reg name: keymgr_reg_block.sw_share0_output_1
  UVM_INFO @ 1847813405 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
Job keymgr-sim-vcs_run_cover_reg_top killed due to: Exit reason: Job lost from admin server: generic::not_found: generic::not_found: job is not found has 1 failures:
- Test keymgr_tl_errors has 1 failures.
  - 15.keymgr_tl_errors.3333062289
    Log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/15.keymgr_tl_errors/latest/run.log
```
  Job ID: smart:4c87e0b7-3757-4728-b546-c6b670dbe70c
```
UVM_ERROR (keymgr_scoreboard.sv:674) [scoreboard] Check failed item.d_data == addr_phase_op_status (* [*] vs * [*]) has 1 failures:
- Test keymgr_stress_all_with_rand_reset has 1 failures.
  - 35.keymgr_stress_all_with_rand_reset.3193414209
    Line 781, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/35.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 335945424 ps: (keymgr_scoreboard.sv:674) [uvm_test_top.env.scoreboard] Check failed item.d_data == addr_phase_op_status (1 [0x1] vs 2 [0x2])
  UVM_INFO @ 335945424 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```