KEYMGR Simulation Results

Tuesday April 23 2024 19:02:21 UTC

GitHub Revision: `41bc3e0c7f`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 60193594966460162319774997373112005644450303415496697929754976735654535188776

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	keymgr_smoke	1.107m	3.728ms	50	50	100.00
V1	random	keymgr_random	57.130s	1.818ms	50	50	100.00
V1	csr_hw_reset	keymgr_csr_hw_reset	1.400s	113.990us	5	5	100.00
V1	csr_rw	keymgr_csr_rw	1.890s	541.741us	20	20	100.00
V1	csr_bit_bash	keymgr_csr_bit_bash	27.180s	1.334ms	5	5	100.00
V1	csr_aliasing	keymgr_csr_aliasing	10.470s	474.608us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	keymgr_csr_mem_rw_with_rand_reset	2.770s	182.572us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	keymgr_csr_rw	1.890s	541.741us	20	20	100.00
		keymgr_csr_aliasing	10.470s	474.608us	5	5	100.00
V1		TOTAL			155	155	100.00
V2	cfgen_during_op	keymgr_cfg_regwen	1.130m	5.324ms	50	50	100.00
V2	sideload	keymgr_sideload	1.177m	6.360ms	50	50	100.00
		keymgr_sideload_kmac	48.690s	10.380ms	50	50	100.00
		keymgr_sideload_aes	53.990s	8.058ms	50	50	100.00
		keymgr_sideload_otbn	44.130s	1.356ms	50	50	100.00
V2	direct_to_disabled_state	keymgr_direct_to_disabled	26.520s	2.834ms	50	50	100.00
V2	lc_disable	keymgr_lc_disable	45.230s	848.953us	50	50	100.00
V2	kmac_error_response	keymgr_kmac_rsp_err	43.980s	1.276ms	46	50	92.00
V2	invalid_sw_input	keymgr_sw_invalid_input	52.870s	12.769ms	50	50	100.00
V2	invalid_hw_input	keymgr_hwsw_invalid_input	1.716m	9.233ms	50	50	100.00
V2	sync_async_fault_cross	keymgr_sync_async_fault_cross	15.990s	3.161ms	49	50	98.00
V2	stress_all	keymgr_stress_all	8.562m	16.122ms	50	50	100.00
V2	intr_test	keymgr_intr_test	0.970s	44.507us	50	50	100.00
V2	alert_test	keymgr_alert_test	0.960s	16.091us	50	50	100.00
V2	tl_d_oob_addr_access	keymgr_tl_errors	5.820s	705.841us	20	20	100.00
V2	tl_d_illegal_access	keymgr_tl_errors	5.820s	705.841us	20	20	100.00
V2	tl_d_outstanding_access	keymgr_csr_hw_reset	1.400s	113.990us	5	5	100.00
		keymgr_csr_rw	1.890s	541.741us	20	20	100.00
		keymgr_csr_aliasing	10.470s	474.608us	5	5	100.00
		keymgr_same_csr_outstanding	3.840s	91.777us	20	20	100.00
V2	tl_d_partial_access	keymgr_csr_hw_reset	1.400s	113.990us	5	5	100.00
		keymgr_csr_rw	1.890s	541.741us	20	20	100.00
		keymgr_csr_aliasing	10.470s	474.608us	5	5	100.00
		keymgr_same_csr_outstanding	3.840s	91.777us	20	20	100.00
V2		TOTAL			735	740	99.32
V2S	sec_cm_additional_check	keymgr_sec_cm	36.840s	2.733ms	5	5	100.00
V2S	tl_intg_err	keymgr_sec_cm	36.840s	2.733ms	5	5	100.00
		keymgr_tl_intg_err	36.100s	5.862ms	20	20	100.00
V2S	shadow_reg_update_error	keymgr_shadow_reg_errors	11.240s	3.894ms	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	keymgr_shadow_reg_errors	11.240s	3.894ms	20	20	100.00
V2S	shadow_reg_storage_error	keymgr_shadow_reg_errors	11.240s	3.894ms	20	20	100.00
V2S	shadowed_reset_glitch	keymgr_shadow_reg_errors	11.240s	3.894ms	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	keymgr_shadow_reg_errors_with_csr_rw	13.860s	411.525us	20	20	100.00
V2S	prim_count_check	keymgr_sec_cm	36.840s	2.733ms	5	5	100.00
V2S	prim_fsm_check	keymgr_sec_cm	36.840s	2.733ms	5	5	100.00
V2S	sec_cm_bus_integrity	keymgr_tl_intg_err	36.100s	5.862ms	20	20	100.00
V2S	sec_cm_config_shadow	keymgr_shadow_reg_errors	11.240s	3.894ms	20	20	100.00
V2S	sec_cm_op_config_regwen	keymgr_cfg_regwen	1.130m	5.324ms	50	50	100.00
V2S	sec_cm_reseed_config_regwen	keymgr_random	57.130s	1.818ms	50	50	100.00
		keymgr_csr_rw	1.890s	541.741us	20	20	100.00
V2S	sec_cm_sw_binding_config_regwen	keymgr_random	57.130s	1.818ms	50	50	100.00
		keymgr_csr_rw	1.890s	541.741us	20	20	100.00
V2S	sec_cm_max_key_ver_config_regwen	keymgr_random	57.130s	1.818ms	50	50	100.00
		keymgr_csr_rw	1.890s	541.741us	20	20	100.00
V2S	sec_cm_lc_ctrl_intersig_mubi	keymgr_lc_disable	45.230s	848.953us	50	50	100.00
V2S	sec_cm_constants_consistency	keymgr_hwsw_invalid_input	1.716m	9.233ms	50	50	100.00
V2S	sec_cm_intersig_consistency	keymgr_hwsw_invalid_input	1.716m	9.233ms	50	50	100.00
V2S	sec_cm_hw_key_sw_noaccess	keymgr_random	57.130s	1.818ms	50	50	100.00
V2S	sec_cm_output_keys_ctrl_redun	keymgr_sideload_protect	18.510s	1.197ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_sparse	keymgr_sec_cm	36.840s	2.733ms	5	5	100.00
V2S	sec_cm_data_fsm_sparse	keymgr_sec_cm	36.840s	2.733ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_local_esc	keymgr_sec_cm	36.840s	2.733ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_consistency	keymgr_custom_cm	20.220s	558.931us	50	50	100.00
V2S	sec_cm_ctrl_fsm_global_esc	keymgr_lc_disable	45.230s	848.953us	50	50	100.00
V2S	sec_cm_ctrl_ctr_redun	keymgr_sec_cm	36.840s	2.733ms	5	5	100.00
V2S	sec_cm_kmac_if_fsm_sparse	keymgr_sec_cm	36.840s	2.733ms	5	5	100.00
V2S	sec_cm_kmac_if_ctr_redun	keymgr_sec_cm	36.840s	2.733ms	5	5	100.00
V2S	sec_cm_kmac_if_cmd_ctrl_consistency	keymgr_custom_cm	20.220s	558.931us	50	50	100.00
V2S	sec_cm_kmac_if_done_ctrl_consistency	keymgr_custom_cm	20.220s	558.931us	50	50	100.00
V2S	sec_cm_reseed_ctr_redun	keymgr_sec_cm	36.840s	2.733ms	5	5	100.00
V2S	sec_cm_side_load_sel_ctrl_consistency	keymgr_custom_cm	20.220s	558.931us	50	50	100.00
V2S	sec_cm_sideload_ctrl_fsm_sparse	keymgr_sec_cm	36.840s	2.733ms	5	5	100.00
V2S	sec_cm_ctrl_key_integrity	keymgr_custom_cm	20.220s	558.931us	50	50	100.00
V2S		TOTAL			165	165	100.00
V3	stress_all_with_rand_reset	keymgr_stress_all_with_rand_reset	23.280s	758.760us	22	50	44.00
V3		TOTAL			22	50	44.00
		TOTAL			1077	1110	97.03

Testplan Progress

Items	Total	Written	Passing	Progress
V1	7	7	7	100.00
V2	16	16	14	87.50
V2S	6	6	6	100.00
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
97.81	99.07	98.06	98.29	100.00	99.19	98.41	91.61

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:829) [keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 21 failures:
- Test keymgr_stress_all_with_rand_reset has 21 failures.
  - 0.keymgr_stress_all_with_rand_reset.98496319452973519739735169662588113707367017804261402803935931675353529420574
    Line 321, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/0.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 394459511 ps: (cip_base_vseq.sv:829) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 394459511 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 3.keymgr_stress_all_with_rand_reset.43415284893381550046488799069058478127799579795402779311515912782874335314477
    Line 530, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/3.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 162920542 ps: (cip_base_vseq.sv:829) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 162920542 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 19 more failures.
UVM_ERROR (keymgr_scoreboard.sv:736) [scoreboard] Check failed item.d_data !=gmv(csr) (* [] vs * [])` has 5 failures:
- Test keymgr_stress_all_with_rand_reset has 5 failures.
  - 30.keymgr_stress_all_with_rand_reset.52689326836834878534012554792426729532640947503611834431118553977292030525950
    Line 1040, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/30.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 1047666219 ps: (keymgr_scoreboard.sv:736) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (3130444093 [0xba96c93d] vs 3130444093 [0xba96c93d])
  UVM_INFO @ 1047666219 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 35.keymgr_stress_all_with_rand_reset.29090281356178137806792394058632045280143566819931869123757180065896172499833
    Line 516, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/35.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 229466862 ps: (keymgr_scoreboard.sv:736) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (3654517574 [0xd9d38346] vs 3654517574 [0xd9d38346])
  UVM_INFO @ 229466862 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 3 more failures.

UVM_ERROR (keymgr_scoreboard.sv:1013) [scoreboard] Check failed act != exp (* [*] vs * [*]) cdi_type: Attestation has 4 failures:

Test keymgr_kmac_rsp_err has 4 failures.

2.keymgr_kmac_rsp_err.55006849351530403406187263120918042038049980555583396749261530432369050459838
Line 396, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/2.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @  26824441 ps: (keymgr_scoreboard.sv:1013) [uvm_test_top.env.scoreboard] Check failed act != exp (2793962260147392452771925903965965610633380263163384411939425171183506856001537603022389033042824947893238127539274551641967497869495074143089920102279579529851221345242522745385544685917380224897842396502413112993516284983611197473807662351251972329691647791313889190852161563006164995628601548345079252423608131429145326159444951917872001926378804370090776609760961483184803125164316760636852429554860533263387362443181486 [0x64fa5bca0d9ef14d57d52d8fef78074ce6d8b5b800000000000000008eea86603a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9f46c99a30f2912882dee71ad6c732518a43b8abc5066da34d4f99af6b8e511bbaae8ff084f82475cd5ebf78bf1e2a3bd4c4e4084079e9112004bff9aa17bbd9a589edb161634e877b38dc2cc20a4ede3fe1b59d31ae2a2962fd3dbf56e36ca5fdcd624f356245ad726f47070b774bdae] vs 2793962260147392452771925903965965610633380263163384411939425171183506856001537603022389033042824947893238127539274551641967497869495074143089920102279579529851221345242522745385544685917380224897842396502413112993516284983611197473807662351251972329691647791313889190852161563006164995628601548345079252423608131429145326159444951917872001926378804370090776609760961483184803125164316760636852429554860533263387362443181486 [0x64fa5bca0d9ef14d57d52d8fef78074ce6d8b5b800000000000000008eea86603a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9f46c99a30f2912882dee71ad6c732518a43b8abc5066da34d4f99af6b8e511bbaae8ff084f82475cd5ebf78bf1e2a3bd4c4e4084079e9112004bff9aa17bbd9a589edb161634e877b38dc2cc20a4ede3fe1b59d31ae2a2962fd3dbf56e36ca5fdcd624f356245ad726f47070b774bdae]) cdi_type: Attestation
  
   DiversificationKey act: 0xfe1b59d31ae2a2962fd3dbf56e36ca5fdcd624f356245ad726f47070b774bdae, exp: 0xfe1b59d31ae2a2962fd3dbf56e36ca5fdcd624f356245ad726f47070b774bdae
   RomDigest act: 0x4c4e4084079e9112004bff9aa17bbd9a589edb161634e877b38dc2cc20a4ede3, exp: 0x4c4e4084079e9112004bff9aa17bbd9a589edb161634e877b38dc2cc20a4ede3
   HealthMeasurement act: 0xaae8ff084f82475cd5ebf78bf1e2a3bd, exp: 0xaae8ff084f82475cd5ebf78bf1e2a3bd

29.keymgr_kmac_rsp_err.86574930468252175717108558560207171224404907097256750216481731526078570355992
Line 483, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/29.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @ 456437282 ps: (keymgr_scoreboard.sv:1013) [uvm_test_top.env.scoreboard] Check failed act != exp (2291744071973839943985886842755947045419110065497934679632378724570145542430666760626099113549212335012421445419322575476101745584148665404107299960327164198413953477049369600131740394362035653841439134823308643815444447274334544393476168348581091155888663341086319624084680373214166256879414001767115993017157689008674857787439816024789552266017493967795720574889232129048823828860901362311684907138376286050149473542072963 [0x52d3b97778c3d1793d0d6206920b03c745ead2b2d9012de8bbe4874be52aad023a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f98069b48dac2e662d4edebcb0aa556260c437ba51ae34fb4d7d221c9268bada2b8649e209ae5c2a30997755a3681f002ef4735214e306619f9f3afc837856a604d8631a6b64ecbc8d2eb7f5d5b7774091648e7e770fa041b5cf56c50d918f14fff9fa5bf6ffa472c72ebe660df2ddea83] vs 2291744071973839943985886842755947045419110065497934679632378724570145542430666760626099113549212335012421445419322575476101745584148665404107299960327164198413953477049369600131740394362035653841439134823308643815444447274334544393476168348581091155888663341086319624084680373214166256879414001767115993017157689008674857787439816024789552266017493967795720574889232129048823828860901362311684907138376286050149473542072963 [0x52d3b97778c3d1793d0d6206920b03c745ead2b2d9012de8bbe4874be52aad023a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f98069b48dac2e662d4edebcb0aa556260c437ba51ae34fb4d7d221c9268bada2b8649e209ae5c2a30997755a3681f002ef4735214e306619f9f3afc837856a604d8631a6b64ecbc8d2eb7f5d5b7774091648e7e770fa041b5cf56c50d918f14fff9fa5bf6ffa472c72ebe660df2ddea83]) cdi_type: Attestation
  
   DiversificationKey act: 0x648e7e770fa041b5cf56c50d918f14fff9fa5bf6ffa472c72ebe660df2ddea83, exp: 0x648e7e770fa041b5cf56c50d918f14fff9fa5bf6ffa472c72ebe660df2ddea83
   RomDigest act: 0xf4735214e306619f9f3afc837856a604d8631a6b64ecbc8d2eb7f5d5b7774091, exp: 0xf4735214e306619f9f3afc837856a604d8631a6b64ecbc8d2eb7f5d5b7774091
   HealthMeasurement act: 0x8649e209ae5c2a30997755a3681f002e, exp: 0x8649e209ae5c2a30997755a3681f002e

... and 2 more failures.

UVM_ERROR (cip_base_vseq.sv:753) [keymgr_common_vseq] Check failed (!has_outstanding_access()) Trying to trigger a reset with outstanding CSR items. has 2 failures:
- Test keymgr_stress_all_with_rand_reset has 2 failures.
  - 1.keymgr_stress_all_with_rand_reset.24485999603010259417347716932427725680632188584196872917296524181954035512362
    Line 277, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/1.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 864975314 ps: (cip_base_vseq.sv:753) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Trying to trigger a reset with outstanding CSR items.
  UVM_INFO @ 864975314 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 5.keymgr_stress_all_with_rand_reset.30243455463592722507380197558392555235972533023843426740374118098761190235069
    Line 547, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/5.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 787314432 ps: (cip_base_vseq.sv:753) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Trying to trigger a reset with outstanding CSR items.
  UVM_INFO @ 787314432 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (cip_base_scoreboard.sv:261) scoreboard [scoreboard] alert recov_operation_err is not received! has 1 failures:
- Test keymgr_sync_async_fault_cross has 1 failures.
  - 34.keymgr_sync_async_fault_cross.102865763841014761462457807753900078115112578017064462974947611840453337901270
    Line 297, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/34.keymgr_sync_async_fault_cross/latest/run.log
```
  UVM_ERROR @  58177533 ps: (cip_base_scoreboard.sv:261) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err is not received!
  UVM_INFO @  58177533 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```