KEYMGR Simulation Results

Monday July 29 2024 23:02:32 UTC

GitHub Revision: `39f3866b56`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 91682663165753342493852681547271085771042321116470426223748766059309541455602

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	keymgr_smoke	24.490s	4.539ms	50	50	100.00
V1	random	keymgr_random	31.640s	3.340ms	50	50	100.00
V1	csr_hw_reset	keymgr_csr_hw_reset	1.440s	137.159us	5	5	100.00
V1	csr_rw	keymgr_csr_rw	1.510s	98.683us	20	20	100.00
V1	csr_bit_bash	keymgr_csr_bit_bash	31.420s	2.629ms	5	5	100.00
V1	csr_aliasing	keymgr_csr_aliasing	6.390s	297.692us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	keymgr_csr_mem_rw_with_rand_reset	2.250s	72.662us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	keymgr_csr_rw	1.510s	98.683us	20	20	100.00
		keymgr_csr_aliasing	6.390s	297.692us	5	5	100.00
V1		TOTAL			155	155	100.00
V2	cfgen_during_op	keymgr_cfg_regwen	2.078m	13.938ms	50	50	100.00
V2	sideload	keymgr_sideload	55.630s	5.683ms	50	50	100.00
		keymgr_sideload_kmac	1.078m	26.633ms	50	50	100.00
		keymgr_sideload_aes	55.550s	6.908ms	50	50	100.00
		keymgr_sideload_otbn	34.430s	5.436ms	50	50	100.00
V2	direct_to_disabled_state	keymgr_direct_to_disabled	33.620s	1.494ms	50	50	100.00
V2	lc_disable	keymgr_lc_disable	19.620s	1.353ms	50	50	100.00
V2	kmac_error_response	keymgr_kmac_rsp_err	11.010s	409.831us	49	50	98.00
V2	invalid_sw_input	keymgr_sw_invalid_input	55.530s	5.446ms	50	50	100.00
V2	invalid_hw_input	keymgr_hwsw_invalid_input	1.478m	31.791ms	50	50	100.00
V2	sync_async_fault_cross	keymgr_sync_async_fault_cross	14.550s	1.653ms	50	50	100.00
V2	stress_all	keymgr_stress_all	7.484m	15.879ms	47	50	94.00
V2	intr_test	keymgr_intr_test	0.970s	31.403us	50	50	100.00
V2	alert_test	keymgr_alert_test	0.970s	59.739us	50	50	100.00
V2	tl_d_oob_addr_access	keymgr_tl_errors	5.430s	587.165us	20	20	100.00
V2	tl_d_illegal_access	keymgr_tl_errors	5.430s	587.165us	20	20	100.00
V2	tl_d_outstanding_access	keymgr_csr_hw_reset	1.440s	137.159us	5	5	100.00
		keymgr_csr_rw	1.510s	98.683us	20	20	100.00
		keymgr_csr_aliasing	6.390s	297.692us	5	5	100.00
		keymgr_same_csr_outstanding	4.610s	280.942us	20	20	100.00
V2	tl_d_partial_access	keymgr_csr_hw_reset	1.440s	137.159us	5	5	100.00
		keymgr_csr_rw	1.510s	98.683us	20	20	100.00
		keymgr_csr_aliasing	6.390s	297.692us	5	5	100.00
		keymgr_same_csr_outstanding	4.610s	280.942us	20	20	100.00
V2		TOTAL			736	740	99.46
V2S	sec_cm_additional_check	keymgr_sec_cm	15.870s	1.069ms	5	5	100.00
V2S	tl_intg_err	keymgr_sec_cm	15.870s	1.069ms	5	5	100.00
		keymgr_tl_intg_err	9.980s	1.184ms	20	20	100.00
V2S	shadow_reg_update_error	keymgr_shadow_reg_errors	5.070s	652.990us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	keymgr_shadow_reg_errors	5.070s	652.990us	20	20	100.00
V2S	shadow_reg_storage_error	keymgr_shadow_reg_errors	5.070s	652.990us	20	20	100.00
V2S	shadowed_reset_glitch	keymgr_shadow_reg_errors	5.070s	652.990us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	keymgr_shadow_reg_errors_with_csr_rw	13.010s	2.002ms	20	20	100.00
V2S	prim_count_check	keymgr_sec_cm	15.870s	1.069ms	5	5	100.00
V2S	prim_fsm_check	keymgr_sec_cm	15.870s	1.069ms	5	5	100.00
V2S	sec_cm_bus_integrity	keymgr_tl_intg_err	9.980s	1.184ms	20	20	100.00
V2S	sec_cm_config_shadow	keymgr_shadow_reg_errors	5.070s	652.990us	20	20	100.00
V2S	sec_cm_op_config_regwen	keymgr_cfg_regwen	2.078m	13.938ms	50	50	100.00
V2S	sec_cm_reseed_config_regwen	keymgr_random	31.640s	3.340ms	50	50	100.00
		keymgr_csr_rw	1.510s	98.683us	20	20	100.00
V2S	sec_cm_sw_binding_config_regwen	keymgr_random	31.640s	3.340ms	50	50	100.00
		keymgr_csr_rw	1.510s	98.683us	20	20	100.00
V2S	sec_cm_max_key_ver_config_regwen	keymgr_random	31.640s	3.340ms	50	50	100.00
		keymgr_csr_rw	1.510s	98.683us	20	20	100.00
V2S	sec_cm_lc_ctrl_intersig_mubi	keymgr_lc_disable	19.620s	1.353ms	50	50	100.00
V2S	sec_cm_constants_consistency	keymgr_hwsw_invalid_input	1.478m	31.791ms	50	50	100.00
V2S	sec_cm_intersig_consistency	keymgr_hwsw_invalid_input	1.478m	31.791ms	50	50	100.00
V2S	sec_cm_hw_key_sw_noaccess	keymgr_random	31.640s	3.340ms	50	50	100.00
V2S	sec_cm_output_keys_ctrl_redun	keymgr_sideload_protect	18.410s	5.289ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_sparse	keymgr_sec_cm	15.870s	1.069ms	5	5	100.00
V2S	sec_cm_data_fsm_sparse	keymgr_sec_cm	15.870s	1.069ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_local_esc	keymgr_sec_cm	15.870s	1.069ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_consistency	keymgr_custom_cm	51.630s	7.264ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_global_esc	keymgr_lc_disable	19.620s	1.353ms	50	50	100.00
V2S	sec_cm_ctrl_ctr_redun	keymgr_sec_cm	15.870s	1.069ms	5	5	100.00
V2S	sec_cm_kmac_if_fsm_sparse	keymgr_sec_cm	15.870s	1.069ms	5	5	100.00
V2S	sec_cm_kmac_if_ctr_redun	keymgr_sec_cm	15.870s	1.069ms	5	5	100.00
V2S	sec_cm_kmac_if_cmd_ctrl_consistency	keymgr_custom_cm	51.630s	7.264ms	50	50	100.00
V2S	sec_cm_kmac_if_done_ctrl_consistency	keymgr_custom_cm	51.630s	7.264ms	50	50	100.00
V2S	sec_cm_reseed_ctr_redun	keymgr_sec_cm	15.870s	1.069ms	5	5	100.00
V2S	sec_cm_side_load_sel_ctrl_consistency	keymgr_custom_cm	51.630s	7.264ms	50	50	100.00
V2S	sec_cm_sideload_ctrl_fsm_sparse	keymgr_sec_cm	15.870s	1.069ms	5	5	100.00
V2S	sec_cm_ctrl_key_integrity	keymgr_custom_cm	51.630s	7.264ms	50	50	100.00
V2S		TOTAL			165	165	100.00
V3	stress_all_with_rand_reset	keymgr_stress_all_with_rand_reset	31.060s	797.899us	26	50	52.00
V3		TOTAL			26	50	52.00
		TOTAL			1082	1110	97.48

Testplan Progress

Items	Total	Written	Passing	Progress
V1	7	7	7	100.00
V2	16	16	14	87.50
V2S	6	6	6	100.00
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
97.74	99.04	98.03	98.58	100.00	99.02	98.41	91.09

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:836) [keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 23 failures:
- Test keymgr_stress_all_with_rand_reset has 23 failures.
  - 0.keymgr_stress_all_with_rand_reset.17254903801659345531973549580435111519187307250137200350557905485587519180617
    Line 1377, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/0.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 538733435 ps: (cip_base_vseq.sv:836) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 538733435 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 1.keymgr_stress_all_with_rand_reset.8744680452273501252812915113050425080437754545016152386480521696575569788346
    Line 1236, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/1.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 287883157 ps: (cip_base_vseq.sv:836) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 287883157 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 21 more failures.
UVM_ERROR (keymgr_scoreboard.sv:766) [scoreboard] Check failed item.d_data !=gmv(csr) (* [] vs * []) reg name: keymgr_reg_block.sw_share1_output_*` has 1 failures:
- Test keymgr_stress_all has 1 failures.
  - 13.keymgr_stress_all.112943982389111804817821779812994240281863714695900146720306147287481929676676
    Line 656, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/13.keymgr_stress_all/latest/run.log
```
  UVM_ERROR @ 531758659 ps: (keymgr_scoreboard.sv:766) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (1193314034 [0x472086f2] vs 1193314034 [0x472086f2]) reg name: keymgr_reg_block.sw_share1_output_6
  UVM_INFO @ 531758659 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (cip_base_scoreboard.sv:241) [scoreboard] Check failed expected_alert[alert_name].expected == * (* [*] vs * [*]) alert recov_operation_err triggered unexpectedly has 1 failures:
- Test keymgr_kmac_rsp_err has 1 failures.
  - 26.keymgr_kmac_rsp_err.79507825899011162347256073367474654505304726229022908738719733984600299248103
    Line 436, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/26.keymgr_kmac_rsp_err/latest/run.log
```
  UVM_ERROR @   7883397 ps: (cip_base_scoreboard.sv:241) [uvm_test_top.env.scoreboard] Check failed expected_alert[alert_name].expected == 1 (0 [0x0] vs 1 [0x1]) alert recov_operation_err triggered unexpectedly
  UVM_INFO @   7883397 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (cip_base_vseq.sv:755) [keymgr_common_vseq] Check failed (!has_outstanding_access()) Trying to trigger a reset with outstanding CSR items. has 1 failures:
- Test keymgr_stress_all_with_rand_reset has 1 failures.
  - 26.keymgr_stress_all_with_rand_reset.79682882314660145369757123927503428203547917136418491107801013465005552192286
    Line 615, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/26.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 558725051 ps: (cip_base_vseq.sv:755) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Trying to trigger a reset with outstanding CSR items.
  UVM_INFO @ 558725051 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (cip_base_scoreboard.sv:301) scoreboard [scoreboard] alert recov_operation_err did not trigger max_delay:* has 1 failures:
- Test keymgr_stress_all has 1 failures.
  - 35.keymgr_stress_all.28440642286271349747699919798018133997880782091899285519164041945926700528094
    Line 289, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/35.keymgr_stress_all/latest/run.log
```
  UVM_ERROR @   8029063 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @   8029063 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```

UVM_ERROR (keymgr_if.sv:557) [keymgr_if] Check failed {act_key.key[*], act_key.key[*]} !== keys_a_array[state][cdi][dest] (* [*] vs * [*]) KMAC key at state StDisabled for Sealing Kmac has 1 failures:

Test keymgr_stress_all has 1 failures.

41.keymgr_stress_all.113117238678424328644358882444223355727208479328912902622620511022147074228405
Line 1314, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/41.keymgr_stress_all/latest/run.log

  UVM_ERROR @ 2879802418 ps: (keymgr_if.sv:557) [keymgr_if] Check failed {act_key.key[1], act_key.key[0]} !== keys_a_array[state][cdi][dest] (4964666000607510399073904814135580627950793027786158707399075388148608271321798670412628066191926143603710003548293355595953450908693399785919716591464913 [0x5ecac82baffd0645a3b05467d9764eb63df132d7f761b649394176d39fac628f429826e42b898f4ad44e5730a9e7e157c7f8a8a06e717e2f8df45f542164f1d1] vs 4964666000607510399073904814135580627950793027786158707399075388148608271321798670412628066191926143603710003548293355595953450908693399785919716591464913 [0x5ecac82baffd0645a3b05467d9764eb63df132d7f761b649394176d39fac628f429826e42b898f4ad44e5730a9e7e157c7f8a8a06e717e2f8df45f542164f1d1]) KMAC key at state StDisabled for Sealing Kmac
  UVM_INFO @ 2879802418 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---