KEYMGR Simulation Results

Sunday June 02 2024 19:02:53 UTC

GitHub Revision: `01a208901a`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 50418669159766293903157726892781832882154091083197082086235277423705989875584

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	keymgr_smoke	50.570s	11.132ms	50	50	100.00
V1	random	keymgr_random	1.027m	11.139ms	50	50	100.00
V1	csr_hw_reset	keymgr_csr_hw_reset	1.620s	72.461us	5	5	100.00
V1	csr_rw	keymgr_csr_rw	1.570s	116.951us	20	20	100.00
V1	csr_bit_bash	keymgr_csr_bit_bash	16.250s	2.692ms	5	5	100.00
V1	csr_aliasing	keymgr_csr_aliasing	14.680s	757.576us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	keymgr_csr_mem_rw_with_rand_reset	2.470s	106.560us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	keymgr_csr_rw	1.570s	116.951us	20	20	100.00
		keymgr_csr_aliasing	14.680s	757.576us	5	5	100.00
V1		TOTAL			155	155	100.00
V2	cfgen_during_op	keymgr_cfg_regwen	2.334m	9.005ms	49	50	98.00
V2	sideload	keymgr_sideload	44.660s	1.425ms	50	50	100.00
		keymgr_sideload_kmac	1.039m	6.447ms	50	50	100.00
		keymgr_sideload_aes	46.630s	1.907ms	50	50	100.00
		keymgr_sideload_otbn	1.045m	6.258ms	50	50	100.00
V2	direct_to_disabled_state	keymgr_direct_to_disabled	39.470s	1.216ms	50	50	100.00
V2	lc_disable	keymgr_lc_disable	52.320s	1.859ms	48	50	96.00
V2	kmac_error_response	keymgr_kmac_rsp_err	7.160s	255.650us	50	50	100.00
V2	invalid_sw_input	keymgr_sw_invalid_input	48.560s	2.184ms	50	50	100.00
V2	invalid_hw_input	keymgr_hwsw_invalid_input	47.110s	4.139ms	50	50	100.00
V2	sync_async_fault_cross	keymgr_sync_async_fault_cross	7.470s	1.503ms	49	50	98.00
V2	stress_all	keymgr_stress_all	5.680m	61.623ms	49	50	98.00
V2	intr_test	keymgr_intr_test	0.910s	16.365us	50	50	100.00
V2	alert_test	keymgr_alert_test	1.070s	22.191us	50	50	100.00
V2	tl_d_oob_addr_access	keymgr_tl_errors	6.090s	326.443us	20	20	100.00
V2	tl_d_illegal_access	keymgr_tl_errors	6.090s	326.443us	20	20	100.00
V2	tl_d_outstanding_access	keymgr_csr_hw_reset	1.620s	72.461us	5	5	100.00
		keymgr_csr_rw	1.570s	116.951us	20	20	100.00
		keymgr_csr_aliasing	14.680s	757.576us	5	5	100.00
		keymgr_same_csr_outstanding	3.750s	101.900us	20	20	100.00
V2	tl_d_partial_access	keymgr_csr_hw_reset	1.620s	72.461us	5	5	100.00
		keymgr_csr_rw	1.570s	116.951us	20	20	100.00
		keymgr_csr_aliasing	14.680s	757.576us	5	5	100.00
		keymgr_same_csr_outstanding	3.750s	101.900us	20	20	100.00
V2		TOTAL			735	740	99.32
V2S	sec_cm_additional_check	keymgr_sec_cm	29.540s	1.357ms	5	5	100.00
V2S	tl_intg_err	keymgr_sec_cm	29.540s	1.357ms	5	5	100.00
		keymgr_tl_intg_err	9.560s	313.077us	20	20	100.00
V2S	shadow_reg_update_error	keymgr_shadow_reg_errors	5.780s	222.381us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	keymgr_shadow_reg_errors	5.780s	222.381us	20	20	100.00
V2S	shadow_reg_storage_error	keymgr_shadow_reg_errors	5.780s	222.381us	20	20	100.00
V2S	shadowed_reset_glitch	keymgr_shadow_reg_errors	5.780s	222.381us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	keymgr_shadow_reg_errors_with_csr_rw	14.400s	6.303ms	20	20	100.00
V2S	prim_count_check	keymgr_sec_cm	29.540s	1.357ms	5	5	100.00
V2S	prim_fsm_check	keymgr_sec_cm	29.540s	1.357ms	5	5	100.00
V2S	sec_cm_bus_integrity	keymgr_tl_intg_err	9.560s	313.077us	20	20	100.00
V2S	sec_cm_config_shadow	keymgr_shadow_reg_errors	5.780s	222.381us	20	20	100.00
V2S	sec_cm_op_config_regwen	keymgr_cfg_regwen	2.334m	9.005ms	49	50	98.00
V2S	sec_cm_reseed_config_regwen	keymgr_random	1.027m	11.139ms	50	50	100.00
		keymgr_csr_rw	1.570s	116.951us	20	20	100.00
V2S	sec_cm_sw_binding_config_regwen	keymgr_random	1.027m	11.139ms	50	50	100.00
		keymgr_csr_rw	1.570s	116.951us	20	20	100.00
V2S	sec_cm_max_key_ver_config_regwen	keymgr_random	1.027m	11.139ms	50	50	100.00
		keymgr_csr_rw	1.570s	116.951us	20	20	100.00
V2S	sec_cm_lc_ctrl_intersig_mubi	keymgr_lc_disable	52.320s	1.859ms	48	50	96.00
V2S	sec_cm_constants_consistency	keymgr_hwsw_invalid_input	47.110s	4.139ms	50	50	100.00
V2S	sec_cm_intersig_consistency	keymgr_hwsw_invalid_input	47.110s	4.139ms	50	50	100.00
V2S	sec_cm_hw_key_sw_noaccess	keymgr_random	1.027m	11.139ms	50	50	100.00
V2S	sec_cm_output_keys_ctrl_redun	keymgr_sideload_protect	22.820s	2.317ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_sparse	keymgr_sec_cm	29.540s	1.357ms	5	5	100.00
V2S	sec_cm_data_fsm_sparse	keymgr_sec_cm	29.540s	1.357ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_local_esc	keymgr_sec_cm	29.540s	1.357ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_consistency	keymgr_custom_cm	36.950s	2.873ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_global_esc	keymgr_lc_disable	52.320s	1.859ms	48	50	96.00
V2S	sec_cm_ctrl_ctr_redun	keymgr_sec_cm	29.540s	1.357ms	5	5	100.00
V2S	sec_cm_kmac_if_fsm_sparse	keymgr_sec_cm	29.540s	1.357ms	5	5	100.00
V2S	sec_cm_kmac_if_ctr_redun	keymgr_sec_cm	29.540s	1.357ms	5	5	100.00
V2S	sec_cm_kmac_if_cmd_ctrl_consistency	keymgr_custom_cm	36.950s	2.873ms	50	50	100.00
V2S	sec_cm_kmac_if_done_ctrl_consistency	keymgr_custom_cm	36.950s	2.873ms	50	50	100.00
V2S	sec_cm_reseed_ctr_redun	keymgr_sec_cm	29.540s	1.357ms	5	5	100.00
V2S	sec_cm_side_load_sel_ctrl_consistency	keymgr_custom_cm	36.950s	2.873ms	50	50	100.00
V2S	sec_cm_sideload_ctrl_fsm_sparse	keymgr_sec_cm	29.540s	1.357ms	5	5	100.00
V2S	sec_cm_ctrl_key_integrity	keymgr_custom_cm	36.950s	2.873ms	50	50	100.00
V2S		TOTAL			165	165	100.00
V3	stress_all_with_rand_reset	keymgr_stress_all_with_rand_reset	24.230s	2.597ms	27	50	54.00
V3		TOTAL			27	50	54.00
		TOTAL			1082	1110	97.48

Testplan Progress

Items	Total	Written	Passing	Progress
V1	7	7	7	100.00
V2	16	16	12	75.00
V2S	6	6	6	100.00
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
97.46	99.04	98.19	98.72	97.67	99.02	98.41	91.14

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:828) [keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 22 failures:
- Test keymgr_stress_all_with_rand_reset has 22 failures.
  - 1.keymgr_stress_all_with_rand_reset.90812480104333750696404477176701321777626649021689074998403053479219839921589
    Line 740, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/1.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 233563609 ps: (cip_base_vseq.sv:828) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 233563609 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 6.keymgr_stress_all_with_rand_reset.49870793859959075158146910245575355437450122349147765515047282908500234558185
    Line 1364, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/6.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 901273261 ps: (cip_base_vseq.sv:828) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 901273261 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 20 more failures.
UVM_ERROR (cip_base_scoreboard.sv:301) scoreboard [scoreboard] alert recov_operation_err did not trigger max_delay:* has 3 failures:
- Test keymgr_cfg_regwen has 1 failures.
  - 10.keymgr_cfg_regwen.46560088596811292122761736172776622678862382418349048631962552309849360730273
    Line 686, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/10.keymgr_cfg_regwen/latest/run.log
```
  UVM_ERROR @  50388856 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @  50388856 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test keymgr_stress_all has 1 failures.
  - 19.keymgr_stress_all.64613273106378082538728403452088767306846103567443364182301582636512904528099
    Line 1009, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/19.keymgr_stress_all/latest/run.log
```
  UVM_ERROR @ 584808915 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @ 584808915 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test keymgr_stress_all_with_rand_reset has 1 failures.
  - 19.keymgr_stress_all_with_rand_reset.24727705611926982579882321702078357547299897093433535003807521797021051417993
    Line 1277, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/19.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 1038698422 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @ 1038698422 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```

UVM_ERROR (keymgr_if.sv:557) [keymgr_if] Check failed {act_key.key[*], act_key.key[*]} !== keys_a_array[state][cdi][dest] (* [*] vs * [*]) KMAC key at state StOwnerIntKey for Sealing Kmac has 1 failures:

Test keymgr_lc_disable has 1 failures.

14.keymgr_lc_disable.29332387783397513272434258516047505658805056745649192247182277991907512250112
Line 472, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/14.keymgr_lc_disable/latest/run.log

  UVM_ERROR @  40324179 ps: (keymgr_if.sv:557) [keymgr_if] Check failed {act_key.key[1], act_key.key[0]} !== keys_a_array[state][cdi][dest] (5620732998037601898773505109498625449043439175636339182507156767735720415938882024699389178103866578671803551155069294837165880457824596447127107907795873 [0x6b519212dd8c8e83c25f59f2d018d49a5f7932c0a08ea5d793990be26ba5e44f98ef864b3064d2132233d9d53159e45024195a174b408c60c90d4101c51abba1] vs 5620732998037601898773505109498625449043439175636339182507156767735720415938882024699389178103866578671803551155069294837165880457824596447127107907795873 [0x6b519212dd8c8e83c25f59f2d018d49a5f7932c0a08ea5d793990be26ba5e44f98ef864b3064d2132233d9d53159e45024195a174b408c60c90d4101c51abba1]) KMAC key at state StOwnerIntKey for Sealing Kmac
  UVM_INFO @  40324179 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_ERROR (keymgr_sync_async_fault_cross_vseq.sv:38) [keymgr_sync_async_fault_cross_vseq] Check failed act_fault_status[keymgr_pkg::FaultKmacOp] || act_fault_status[keymgr_pkg::FaultKmacOut] == * (* [*] vs * [*]) has 1 failures:
- Test keymgr_sync_async_fault_cross has 1 failures.
  - 32.keymgr_sync_async_fault_cross.29304694176153744929601246044652772226683471574678591022491744541810552311053
    Line 331, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/32.keymgr_sync_async_fault_cross/latest/run.log
```
  UVM_ERROR @ 334263151 ps: (keymgr_sync_async_fault_cross_vseq.sv:38) [uvm_test_top.env.virtual_sequencer.keymgr_sync_async_fault_cross_vseq] Check failed act_fault_status[keymgr_pkg::FaultKmacOp] ||                 act_fault_status[keymgr_pkg::FaultKmacOut] == 1 (0 [0x0] vs 1 [0x1])
  UVM_INFO @ 334263151 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```

UVM_ERROR (keymgr_if.sv:557) [keymgr_if] Check failed {act_key.key[*], act_key.key[*]} !== keys_a_array[state][cdi][dest] (* [*] vs * [*]) KMAC key at state StOwnerKey for Attestation Kmac has 1 failures:

Test keymgr_lc_disable has 1 failures.

36.keymgr_lc_disable.24802097222679213548629002690258087450202923553601622956761937148359561911570
Line 628, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/36.keymgr_lc_disable/latest/run.log

  UVM_ERROR @  37277245 ps: (keymgr_if.sv:557) [keymgr_if] Check failed {act_key.key[1], act_key.key[0]} !== keys_a_array[state][cdi][dest] (10081313865131905208201754676895968325326980201224143716793662447027433621844999152886932279432684093558058579906072149266312393672801728763884589094697355 [0xc07c6f8aa89324e0d222efc8ed1998a46a1947db14842c040361c59420313b88fb2e6930bcf0781ccfe8a414a7c488aa281fc8a66ee09fb128a22f76ed62798b] vs 10081313865131905208201754676895968325326980201224143716793662447027433621844999152886932279432684093558058579906072149266312393672801728763884589094697355 [0xc07c6f8aa89324e0d222efc8ed1998a46a1947db14842c040361c59420313b88fb2e6930bcf0781ccfe8a414a7c488aa281fc8a66ee09fb128a22f76ed62798b]) KMAC key at state StOwnerKey for Attestation Kmac
  UVM_INFO @  37277245 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---