KEYMGR Simulation Results

Saturday August 24 2024 20:58:08 UTC

GitHub Revision: `e733a8ef8a`

Branch: os_regression_2024_08_24

Simulator: VCS

Build randomization enabled with --build-seed 36240513409906943553650221581975102764006655953510936167454320581301243659163

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	keymgr_smoke	30.870s	2.823ms	50	50	100.00
V1	random	keymgr_random	1.047m	2.250ms	50	50	100.00
V1	csr_hw_reset	keymgr_csr_hw_reset	1.720s	20.068us	5	5	100.00
V1	csr_rw	keymgr_csr_rw	2.380s	53.593us	20	20	100.00
V1	csr_bit_bash	keymgr_csr_bit_bash	36.300s	3.579ms	5	5	100.00
V1	csr_aliasing	keymgr_csr_aliasing	23.880s	2.026ms	5	5	100.00
V1	csr_mem_rw_with_rand_reset	keymgr_csr_mem_rw_with_rand_reset	3.600s	51.455us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	keymgr_csr_rw	2.380s	53.593us	20	20	100.00
		keymgr_csr_aliasing	23.880s	2.026ms	5	5	100.00
V1		TOTAL			155	155	100.00
V2	cfgen_during_op	keymgr_cfg_regwen	2.131m	7.117ms	50	50	100.00
V2	sideload	keymgr_sideload	58.300s	6.394ms	50	50	100.00
		keymgr_sideload_kmac	1.843m	9.599ms	50	50	100.00
		keymgr_sideload_aes	1.067m	3.518ms	50	50	100.00
		keymgr_sideload_otbn	1.337m	17.659ms	50	50	100.00
V2	direct_to_disabled_state	keymgr_direct_to_disabled	13.990s	363.360us	50	50	100.00
V2	lc_disable	keymgr_lc_disable	39.120s	594.348us	49	50	98.00
V2	kmac_error_response	keymgr_kmac_rsp_err	8.710s	510.620us	50	50	100.00
V2	invalid_sw_input	keymgr_sw_invalid_input	1.879m	8.641ms	50	50	100.00
V2	invalid_hw_input	keymgr_hwsw_invalid_input	1.294m	28.752ms	50	50	100.00
V2	sync_async_fault_cross	keymgr_sync_async_fault_cross	20.070s	561.059us	49	50	98.00
V2	stress_all	keymgr_stress_all	3.837m	9.869ms	47	50	94.00
V2	intr_test	keymgr_intr_test	1.560s	22.035us	50	50	100.00
V2	alert_test	keymgr_alert_test	1.530s	73.313us	50	50	100.00
V2	tl_d_oob_addr_access	keymgr_tl_errors	6.690s	430.569us	20	20	100.00
V2	tl_d_illegal_access	keymgr_tl_errors	6.690s	430.569us	20	20	100.00
V2	tl_d_outstanding_access	keymgr_csr_hw_reset	1.720s	20.068us	5	5	100.00
		keymgr_csr_rw	2.380s	53.593us	20	20	100.00
		keymgr_csr_aliasing	23.880s	2.026ms	5	5	100.00
		keymgr_same_csr_outstanding	5.690s	1.006ms	20	20	100.00
V2	tl_d_partial_access	keymgr_csr_hw_reset	1.720s	20.068us	5	5	100.00
		keymgr_csr_rw	2.380s	53.593us	20	20	100.00
		keymgr_csr_aliasing	23.880s	2.026ms	5	5	100.00
		keymgr_same_csr_outstanding	5.690s	1.006ms	20	20	100.00
V2		TOTAL			735	740	99.32
V2S	sec_cm_additional_check	keymgr_sec_cm	24.460s	1.114ms	5	5	100.00
V2S	tl_intg_err	keymgr_sec_cm	24.460s	1.114ms	5	5	100.00
		keymgr_tl_intg_err	13.710s	441.519us	20	20	100.00
V2S	shadow_reg_update_error	keymgr_shadow_reg_errors	6.450s	131.956us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	keymgr_shadow_reg_errors	6.450s	131.956us	20	20	100.00
V2S	shadow_reg_storage_error	keymgr_shadow_reg_errors	6.450s	131.956us	20	20	100.00
V2S	shadowed_reset_glitch	keymgr_shadow_reg_errors	6.450s	131.956us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	keymgr_shadow_reg_errors_with_csr_rw	22.690s	7.030ms	20	20	100.00
V2S	prim_count_check	keymgr_sec_cm	24.460s	1.114ms	5	5	100.00
V2S	prim_fsm_check	keymgr_sec_cm	24.460s	1.114ms	5	5	100.00
V2S	sec_cm_bus_integrity	keymgr_tl_intg_err	13.710s	441.519us	20	20	100.00
V2S	sec_cm_config_shadow	keymgr_shadow_reg_errors	6.450s	131.956us	20	20	100.00
V2S	sec_cm_op_config_regwen	keymgr_cfg_regwen	2.131m	7.117ms	50	50	100.00
V2S	sec_cm_reseed_config_regwen	keymgr_random	1.047m	2.250ms	50	50	100.00
		keymgr_csr_rw	2.380s	53.593us	20	20	100.00
V2S	sec_cm_sw_binding_config_regwen	keymgr_random	1.047m	2.250ms	50	50	100.00
		keymgr_csr_rw	2.380s	53.593us	20	20	100.00
V2S	sec_cm_max_key_ver_config_regwen	keymgr_random	1.047m	2.250ms	50	50	100.00
		keymgr_csr_rw	2.380s	53.593us	20	20	100.00
V2S	sec_cm_lc_ctrl_intersig_mubi	keymgr_lc_disable	39.120s	594.348us	49	50	98.00
V2S	sec_cm_constants_consistency	keymgr_hwsw_invalid_input	1.294m	28.752ms	50	50	100.00
V2S	sec_cm_intersig_consistency	keymgr_hwsw_invalid_input	1.294m	28.752ms	50	50	100.00
V2S	sec_cm_hw_key_sw_noaccess	keymgr_random	1.047m	2.250ms	50	50	100.00
V2S	sec_cm_output_keys_ctrl_redun	keymgr_sideload_protect	26.090s	1.624ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_sparse	keymgr_sec_cm	24.460s	1.114ms	5	5	100.00
V2S	sec_cm_data_fsm_sparse	keymgr_sec_cm	24.460s	1.114ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_local_esc	keymgr_sec_cm	24.460s	1.114ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_consistency	keymgr_custom_cm	45.470s	2.282ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_global_esc	keymgr_lc_disable	39.120s	594.348us	49	50	98.00
V2S	sec_cm_ctrl_ctr_redun	keymgr_sec_cm	24.460s	1.114ms	5	5	100.00
V2S	sec_cm_kmac_if_fsm_sparse	keymgr_sec_cm	24.460s	1.114ms	5	5	100.00
V2S	sec_cm_kmac_if_ctr_redun	keymgr_sec_cm	24.460s	1.114ms	5	5	100.00
V2S	sec_cm_kmac_if_cmd_ctrl_consistency	keymgr_custom_cm	45.470s	2.282ms	50	50	100.00
V2S	sec_cm_kmac_if_done_ctrl_consistency	keymgr_custom_cm	45.470s	2.282ms	50	50	100.00
V2S	sec_cm_reseed_ctr_redun	keymgr_sec_cm	24.460s	1.114ms	5	5	100.00
V2S	sec_cm_side_load_sel_ctrl_consistency	keymgr_custom_cm	45.470s	2.282ms	50	50	100.00
V2S	sec_cm_sideload_ctrl_fsm_sparse	keymgr_sec_cm	24.460s	1.114ms	5	5	100.00
V2S	sec_cm_ctrl_key_integrity	keymgr_custom_cm	45.470s	2.282ms	50	50	100.00
V2S		TOTAL			165	165	100.00
V3	stress_all_with_rand_reset	keymgr_stress_all_with_rand_reset	44.080s	10.848ms	24	50	48.00
V3		TOTAL			24	50	48.00
		TOTAL			1079	1110	97.21

Testplan Progress

Items	Total	Written	Passing	Progress
V1	7	7	7	100.00
V2	16	16	13	81.25
V2S	6	6	6	100.00
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
97.79	99.04	98.11	98.60	100.00	99.02	98.63	91.12

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:867) [keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 24 failures:
- Test keymgr_stress_all_with_rand_reset has 24 failures.
  - 3.keymgr_stress_all_with_rand_reset.53841723968800925228737870088522910703828265867470231323704997162593357163644
    Line 728, in log /workspaces/repo/scratch/os_regression_2024_08_24/keymgr-sim-vcs/3.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 338584475 ps: (cip_base_vseq.sv:867) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 338584475 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 4.keymgr_stress_all_with_rand_reset.106084919385961593187325843923085576580151119249185521386968018562433878616510
    Line 118, in log /workspaces/repo/scratch/os_regression_2024_08_24/keymgr-sim-vcs/4.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 504425550 ps: (cip_base_vseq.sv:867) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10002 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 504425550 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 22 more failures.

UVM_ERROR (cip_base_scoreboard.sv:304) scoreboard [scoreboard] alert recov_operation_err did not trigger max_delay:* has 4 failures:

Test keymgr_stress_all_with_rand_reset has 2 failures.

6.keymgr_stress_all_with_rand_reset.4919593436735047647311768826733601166159594318778195126717420099015010109310
Line 231, in log /workspaces/repo/scratch/os_regression_2024_08_24/keymgr-sim-vcs/6.keymgr_stress_all_with_rand_reset/latest/run.log

  UVM_ERROR @  42876288 ps: (cip_base_scoreboard.sv:304) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @  42876288 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

24.keymgr_stress_all_with_rand_reset.27016024884976543810997055347401770340886051511839672261593347428534127199581
Line 556, in log /workspaces/repo/scratch/os_regression_2024_08_24/keymgr-sim-vcs/24.keymgr_stress_all_with_rand_reset/latest/run.log

  UVM_ERROR @ 179523981 ps: (cip_base_scoreboard.sv:304) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @ 179523981 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test keymgr_stress_all has 2 failures.

19.keymgr_stress_all.41054155119204588909294730424896820742283968424606598141577747339674759931329
Line 512, in log /workspaces/repo/scratch/os_regression_2024_08_24/keymgr-sim-vcs/19.keymgr_stress_all/latest/run.log

  UVM_ERROR @ 674399968 ps: (cip_base_scoreboard.sv:304) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @ 674399968 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

35.keymgr_stress_all.44983512411925569286840479166767043891548486573788860100014570095727924163595
Line 181, in log /workspaces/repo/scratch/os_regression_2024_08_24/keymgr-sim-vcs/35.keymgr_stress_all/latest/run.log

  UVM_ERROR @ 770360680 ps: (cip_base_scoreboard.sv:304) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @ 770360680 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_ERROR (keymgr_if.sv:557) [keymgr_if] Check failed {act_key.key[*], act_key.key[*]} !== keys_a_array[state][cdi][dest] (* [*] vs * [*]) AES key at state StDisabled for Sealing Aes has 1 failures:

Test keymgr_lc_disable has 1 failures.

29.keymgr_lc_disable.80948173489143692980161687217140388126547539864424298094030275352435203470948
Line 483, in log /workspaces/repo/scratch/os_regression_2024_08_24/keymgr-sim-vcs/29.keymgr_lc_disable/latest/run.log

  UVM_ERROR @ 820890332 ps: (keymgr_if.sv:557) [keymgr_if] Check failed {act_key.key[1], act_key.key[0]} !== keys_a_array[state][cdi][dest] (1360998882361469252447336379188813650191916996195979350733985192753799538263936326286256170393306399902272488413401364980788360540623948926675039562178559 [0x19fc6c88ca4d14260e3c0a47d27b16f991974dae3a30c2eba78a7073fa21e7e6b22d2789806fadd6b175fdf96f6a69ecfcb6f40e253d9c8228609b556d031fff] vs 1360998882361469252447336379188813650191916996195979350733985192753799538263936326286256170393306399902272488413401364980788360540623948926675039562178559 [0x19fc6c88ca4d14260e3c0a47d27b16f991974dae3a30c2eba78a7073fa21e7e6b22d2789806fadd6b175fdf96f6a69ecfcb6f40e253d9c8228609b556d031fff]) AES key at state StDisabled for Sealing Aes
  UVM_INFO @ 820890332 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_ERROR (keymgr_if.sv:557) [keymgr_if] Check failed {act_key.key[*], act_key.key[*]} !== keys_a_array[state][cdi][dest] (* [*] vs * [*]) KMAC key at state StDisabled for Sealing Kmac has 1 failures:

Test keymgr_stress_all has 1 failures.

38.keymgr_stress_all.19042070622204963368311644258505764998965709028821258068804144227616921832656
Line 1628, in log /workspaces/repo/scratch/os_regression_2024_08_24/keymgr-sim-vcs/38.keymgr_stress_all/latest/run.log

  UVM_ERROR @ 148689939 ps: (keymgr_if.sv:557) [keymgr_if] Check failed {act_key.key[1], act_key.key[0]} !== keys_a_array[state][cdi][dest] (6521219210597505410432800896928952043804835953681386170957007896546833985145510147756977824536555484076309217640527613144173776133638353829381573096389186 [0x7c830e36a4ae8a48431f2f1c6928847f8fd82dfefdf5b710cfbdaab8ae885635141e8a315064aace65d1acedf060a1e7d690aa639f7abd19f7544ed1484e3a42] vs 6521219210597505410432800896928952043804835953681386170957007896546833985145510147756977824536555484076309217640527613144173776133638353829381573096389186 [0x7c830e36a4ae8a48431f2f1c6928847f8fd82dfefdf5b710cfbdaab8ae885635141e8a315064aace65d1acedf060a1e7d690aa639f7abd19f7544ed1484e3a42]) KMAC key at state StDisabled for Sealing Kmac
  UVM_INFO @ 148689939 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_ERROR (cip_base_scoreboard.sv:262) scoreboard [scoreboard] alert recov_operation_err is not received! has 1 failures:
- Test keymgr_sync_async_fault_cross has 1 failures.
  - 47.keymgr_sync_async_fault_cross.90043453818333172820848712294970310372401397377315970946709905378509740940400
    Line 115, in log /workspaces/repo/scratch/os_regression_2024_08_24/keymgr-sim-vcs/47.keymgr_sync_async_fault_cross/latest/run.log
```
  UVM_ERROR @  65577684 ps: (cip_base_scoreboard.sv:262) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err is not received!
  UVM_INFO @  65577684 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```