KEYMGR Simulation Results

Tuesday April 16 2024 19:02:32 UTC

GitHub Revision: `1c75f24e99`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 47053888840936652465110085351243654616760492049444303115123736462709488656445

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	keymgr_smoke	44.740s	1.822ms	50	50	100.00
V1	random	keymgr_random	1.411m	8.217ms	50	50	100.00
V1	csr_hw_reset	keymgr_csr_hw_reset	1.190s	21.368us	5	5	100.00
V1	csr_rw	keymgr_csr_rw	1.570s	31.084us	20	20	100.00
V1	csr_bit_bash	keymgr_csr_bit_bash	16.740s	1.053ms	5	5	100.00
V1	csr_aliasing	keymgr_csr_aliasing	16.690s	4.149ms	5	5	100.00
V1	csr_mem_rw_with_rand_reset	keymgr_csr_mem_rw_with_rand_reset	2.280s	51.870us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	keymgr_csr_rw	1.570s	31.084us	20	20	100.00
		keymgr_csr_aliasing	16.690s	4.149ms	5	5	100.00
V1		TOTAL			155	155	100.00
V2	cfgen_during_op	keymgr_cfg_regwen	2.560m	2.895ms	49	50	98.00
V2	sideload	keymgr_sideload	41.330s	3.618ms	50	50	100.00
		keymgr_sideload_kmac	58.050s	4.503ms	50	50	100.00
		keymgr_sideload_aes	47.070s	3.389ms	50	50	100.00
		keymgr_sideload_otbn	47.320s	7.144ms	50	50	100.00
V2	direct_to_disabled_state	keymgr_direct_to_disabled	27.950s	2.755ms	50	50	100.00
V2	lc_disable	keymgr_lc_disable	17.330s	9.588ms	48	50	96.00
V2	kmac_error_response	keymgr_kmac_rsp_err	1.654m	4.971ms	43	50	86.00
V2	invalid_sw_input	keymgr_sw_invalid_input	49.470s	18.223ms	50	50	100.00
V2	invalid_hw_input	keymgr_hwsw_invalid_input	1.121m	10.266ms	50	50	100.00
V2	sync_async_fault_cross	keymgr_sync_async_fault_cross	14.530s	2.984ms	49	50	98.00
V2	stress_all	keymgr_stress_all	10.083m	28.936ms	48	50	96.00
V2	intr_test	keymgr_intr_test	0.900s	16.748us	50	50	100.00
V2	alert_test	keymgr_alert_test	1.000s	64.535us	50	50	100.00
V2	tl_d_oob_addr_access	keymgr_tl_errors	5.000s	159.746us	20	20	100.00
V2	tl_d_illegal_access	keymgr_tl_errors	5.000s	159.746us	20	20	100.00
V2	tl_d_outstanding_access	keymgr_csr_hw_reset	1.190s	21.368us	5	5	100.00
		keymgr_csr_rw	1.570s	31.084us	20	20	100.00
		keymgr_csr_aliasing	16.690s	4.149ms	5	5	100.00
		keymgr_same_csr_outstanding	4.080s	122.663us	20	20	100.00
V2	tl_d_partial_access	keymgr_csr_hw_reset	1.190s	21.368us	5	5	100.00
		keymgr_csr_rw	1.570s	31.084us	20	20	100.00
		keymgr_csr_aliasing	16.690s	4.149ms	5	5	100.00
		keymgr_same_csr_outstanding	4.080s	122.663us	20	20	100.00
V2		TOTAL			727	740	98.24
V2S	sec_cm_additional_check	keymgr_sec_cm	5.200m	16.453ms	5	5	100.00
V2S	tl_intg_err	keymgr_sec_cm	5.200m	16.453ms	5	5	100.00
		keymgr_tl_intg_err	21.130s	833.395us	20	20	100.00
V2S	shadow_reg_update_error	keymgr_shadow_reg_errors	11.880s	1.996ms	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	keymgr_shadow_reg_errors	11.880s	1.996ms	20	20	100.00
V2S	shadow_reg_storage_error	keymgr_shadow_reg_errors	11.880s	1.996ms	20	20	100.00
V2S	shadowed_reset_glitch	keymgr_shadow_reg_errors	11.880s	1.996ms	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	keymgr_shadow_reg_errors_with_csr_rw	16.190s	543.164us	20	20	100.00
V2S	prim_count_check	keymgr_sec_cm	5.200m	16.453ms	5	5	100.00
V2S	prim_fsm_check	keymgr_sec_cm	5.200m	16.453ms	5	5	100.00
V2S	sec_cm_bus_integrity	keymgr_tl_intg_err	21.130s	833.395us	20	20	100.00
V2S	sec_cm_config_shadow	keymgr_shadow_reg_errors	11.880s	1.996ms	20	20	100.00
V2S	sec_cm_op_config_regwen	keymgr_cfg_regwen	2.560m	2.895ms	49	50	98.00
V2S	sec_cm_reseed_config_regwen	keymgr_random	1.411m	8.217ms	50	50	100.00
		keymgr_csr_rw	1.570s	31.084us	20	20	100.00
V2S	sec_cm_sw_binding_config_regwen	keymgr_random	1.411m	8.217ms	50	50	100.00
		keymgr_csr_rw	1.570s	31.084us	20	20	100.00
V2S	sec_cm_max_key_ver_config_regwen	keymgr_random	1.411m	8.217ms	50	50	100.00
		keymgr_csr_rw	1.570s	31.084us	20	20	100.00
V2S	sec_cm_lc_ctrl_intersig_mubi	keymgr_lc_disable	17.330s	9.588ms	48	50	96.00
V2S	sec_cm_constants_consistency	keymgr_hwsw_invalid_input	1.121m	10.266ms	50	50	100.00
V2S	sec_cm_intersig_consistency	keymgr_hwsw_invalid_input	1.121m	10.266ms	50	50	100.00
V2S	sec_cm_hw_key_sw_noaccess	keymgr_random	1.411m	8.217ms	50	50	100.00
V2S	sec_cm_output_keys_ctrl_redun	keymgr_sideload_protect	31.040s	4.304ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_sparse	keymgr_sec_cm	5.200m	16.453ms	5	5	100.00
V2S	sec_cm_data_fsm_sparse	keymgr_sec_cm	5.200m	16.453ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_local_esc	keymgr_sec_cm	5.200m	16.453ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_consistency	keymgr_custom_cm	49.480s	2.126ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_global_esc	keymgr_lc_disable	17.330s	9.588ms	48	50	96.00
V2S	sec_cm_ctrl_ctr_redun	keymgr_sec_cm	5.200m	16.453ms	5	5	100.00
V2S	sec_cm_kmac_if_fsm_sparse	keymgr_sec_cm	5.200m	16.453ms	5	5	100.00
V2S	sec_cm_kmac_if_ctr_redun	keymgr_sec_cm	5.200m	16.453ms	5	5	100.00
V2S	sec_cm_kmac_if_cmd_ctrl_consistency	keymgr_custom_cm	49.480s	2.126ms	50	50	100.00
V2S	sec_cm_kmac_if_done_ctrl_consistency	keymgr_custom_cm	49.480s	2.126ms	50	50	100.00
V2S	sec_cm_reseed_ctr_redun	keymgr_sec_cm	5.200m	16.453ms	5	5	100.00
V2S	sec_cm_side_load_sel_ctrl_consistency	keymgr_custom_cm	49.480s	2.126ms	50	50	100.00
V2S	sec_cm_sideload_ctrl_fsm_sparse	keymgr_sec_cm	5.200m	16.453ms	5	5	100.00
V2S	sec_cm_ctrl_key_integrity	keymgr_custom_cm	49.480s	2.126ms	50	50	100.00
V2S		TOTAL			165	165	100.00
V3	stress_all_with_rand_reset	keymgr_stress_all_with_rand_reset	34.760s	13.851ms	22	50	44.00
V3		TOTAL			22	50	44.00
		TOTAL			1069	1110	96.31

Testplan Progress

Items	Total	Written	Passing	Progress
V1	7	7	7	100.00
V2	16	16	11	68.75
V2S	6	6	6	100.00
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
97.85	99.07	98.06	98.66	100.00	99.11	98.41	91.61

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:829) [keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 20 failures:
- Test keymgr_stress_all_with_rand_reset has 20 failures.
  - 0.keymgr_stress_all_with_rand_reset.21061873274495155203709700821444404278259988468757297468846888026659503531042
    Line 560, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/0.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 240956061 ps: (cip_base_vseq.sv:829) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10001 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 240956061 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 1.keymgr_stress_all_with_rand_reset.12047799322511554768538006618472434915367897627550092291206688348368400973147
    Line 305, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/1.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 213593067 ps: (cip_base_vseq.sv:829) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 213593067 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 18 more failures.
UVM_ERROR (keymgr_scoreboard.sv:736) [scoreboard] Check failed item.d_data !=gmv(csr) (* [] vs * [])` has 7 failures:
- Test keymgr_stress_all_with_rand_reset has 7 failures.
  - 14.keymgr_stress_all_with_rand_reset.105725264254620845944512334684911217382487993852009494919497665934606475647950
    Line 1454, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/14.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 326406966 ps: (keymgr_scoreboard.sv:736) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (855232640 [0x32f9d080] vs 855232640 [0x32f9d080])
  UVM_INFO @ 326406966 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 33.keymgr_stress_all_with_rand_reset.32161474623559164648806353967120473085792772037216366236152014442002612726517
    Line 833, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/33.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 1289283745 ps: (keymgr_scoreboard.sv:736) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (4065268546 [0xf24f1342] vs 4065268546 [0xf24f1342])
  UVM_INFO @ 1289283745 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 5 more failures.

UVM_ERROR (keymgr_scoreboard.sv:1013) [scoreboard] Check failed act != exp (* [*] vs * [*]) cdi_type: Attestation has 6 failures:

Test keymgr_kmac_rsp_err has 6 failures.

6.keymgr_kmac_rsp_err.51818842363600358416859927415157239421426970176595697498577984965893769649548
Line 284, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/6.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @ 619011267 ps: (keymgr_scoreboard.sv:1013) [uvm_test_top.env.scoreboard] Check failed act != exp (13869088684237589750254652523805745455607483863801036890679746459828093165537624428249434082572210646356401831540157857537939020899035064295279308795360165904414059789301483672837524838380982581302039951484695174235694842716416549168309821160551626573831721136876865538121171276497498537367910003611142072470377873037906127626824553249766433745784 [0x3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9000000000000000000000000000000000000000000000000000000000000f0f000000000000000005cfbd765ce33f34ea20a046cf42e6eac560a3f82bfa76285b5c1d4aea7c915e49a32d1c89be0f507fa365df52da48cd752fb3a026a8e608f0098cfe5fa9810494829d0cd9479eb78] vs 13869088684237589750254652523805745455607483863801036890679746459828093165537624428249434082572210646356401831540157857537939020899035064295279308795360165904414059789301483672837524838380982581302039951484695174235694842716416549168309821160551626573831721136876865538121171276497498537367910003611142072470377873037906127626824553249766433745784 [0x3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9000000000000000000000000000000000000000000000000000000000000f0f000000000000000005cfbd765ce33f34ea20a046cf42e6eac560a3f82bfa76285b5c1d4aea7c915e49a32d1c89be0f507fa365df52da48cd752fb3a026a8e608f0098cfe5fa9810494829d0cd9479eb78]) cdi_type: Attestation
  
   DiversificationKey act: 0xfa365df52da48cd752fb3a026a8e608f0098cfe5fa9810494829d0cd9479eb78, exp: 0xfa365df52da48cd752fb3a026a8e608f0098cfe5fa9810494829d0cd9479eb78
   RomDigest act: 0xa20a046cf42e6eac560a3f82bfa76285b5c1d4aea7c915e49a32d1c89be0f507, exp: 0xa20a046cf42e6eac560a3f82bfa76285b5c1d4aea7c915e49a32d1c89be0f507
   HealthMeasurement act: 0x5cfbd765ce33f34e, exp: 0x5cfbd765ce33f34e

18.keymgr_kmac_rsp_err.4723300383338158398851443416714945676300904748941391119337565847548100631868
Line 490, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/18.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @  59891868 ps: (keymgr_scoreboard.sv:1013) [uvm_test_top.env.scoreboard] Check failed act != exp (1216591447860717958362826411372803351209615380214772469651592387487130645645101383665391926562270572917594588998081620697155417936028900208846378343681491489114277743166803597558744069351804475270249675323687213060908340957134840538605174007423110893721110401442730389319860078856720669601971376144470089515949695181341397651297355206974070302560875330765758409912510224605535576882923486438377957202441898557405501708091573 [0x2bf82c864e718ac8def75f186bdf950cff8632585fda340cd6a38415895bdefd3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9fc5a478e54610427f764dde2ecdd2f3c04a81b8f3c4f32f4a2d0e89ae6d88329355ca532e3f960d31e3b78c83c33f5cda634ec2b4f30d0c983068693f5493cf8c484294134a7d927d495c4f6224f08d07ceb2d5d32df67fe51c914ea6f811dbcd2b48c002548ce561bb4218bfd90c0b5] vs 1216591447860717958362826411372803351209615380214772469651592387487130645645101383665391926562270572917594588998081620697155417936028900208846378343681491489114277743166803597558744069351804475270249675323687213060908340957134840538605174007423110893721110401442730389319860078856720669601971376144470089515949695181341397651297355206974070302560875330765758409912510224605535576882923486438377957202441898557405501708091573 [0x2bf82c864e718ac8def75f186bdf950cff8632585fda340cd6a38415895bdefd3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9fc5a478e54610427f764dde2ecdd2f3c04a81b8f3c4f32f4a2d0e89ae6d88329355ca532e3f960d31e3b78c83c33f5cda634ec2b4f30d0c983068693f5493cf8c484294134a7d927d495c4f6224f08d07ceb2d5d32df67fe51c914ea6f811dbcd2b48c002548ce561bb4218bfd90c0b5]) cdi_type: Attestation
  
   DiversificationKey act: 0x7ceb2d5d32df67fe51c914ea6f811dbcd2b48c002548ce561bb4218bfd90c0b5, exp: 0x7ceb2d5d32df67fe51c914ea6f811dbcd2b48c002548ce561bb4218bfd90c0b5
   RomDigest act: 0xa634ec2b4f30d0c983068693f5493cf8c484294134a7d927d495c4f6224f08d0, exp: 0xa634ec2b4f30d0c983068693f5493cf8c484294134a7d927d495c4f6224f08d0
   HealthMeasurement act: 0x355ca532e3f960d31e3b78c83c33f5cd, exp: 0x355ca532e3f960d31e3b78c83c33f5cd

... and 4 more failures.

UVM_ERROR (cip_base_scoreboard.sv:301) scoreboard [scoreboard] alert recov_operation_err did not trigger max_delay:* has 4 failures:

Test keymgr_stress_all has 2 failures.

19.keymgr_stress_all.71445830715018693425876933702113853275336862606518259738938925569459828245288
Line 2280, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/19.keymgr_stress_all/latest/run.log

  UVM_ERROR @ 258883541 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @ 258883541 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

45.keymgr_stress_all.16991125093622993570680910128484014351590153318474362306954975168306427056843
Line 423, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/45.keymgr_stress_all/latest/run.log

  UVM_ERROR @  95987099 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @  95987099 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test keymgr_cfg_regwen has 1 failures.

29.keymgr_cfg_regwen.50268177344262495489324799444805220561242100971702770808905620514028688387470
Line 269, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/29.keymgr_cfg_regwen/latest/run.log

  UVM_ERROR @   5086896 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @   5086896 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test keymgr_kmac_rsp_err has 1 failures.

29.keymgr_kmac_rsp_err.10848618609903398308787836256403691842916893135411199516102862861489589581171
Line 283, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/29.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @  17403810 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @  17403810 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_ERROR (keymgr_scoreboard.sv:750) [scoreboard] Check failed item.d_data ==gmv(csr) (* [] vs * []) reg name: keymgr_reg_block.sw_share1_output_*` has 2 failures:
- Test keymgr_lc_disable has 1 failures.
  - 12.keymgr_lc_disable.113767605045785370413701279869111541486120405770231111244130823185263246408471
    Line 473, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/12.keymgr_lc_disable/latest/run.log
```
  UVM_ERROR @  80498070 ps: (keymgr_scoreboard.sv:750) [uvm_test_top.env.scoreboard] Check failed item.d_data == `gmv(csr) (4153190536 [0xf78ca888] vs 0 [0x0]) reg name: keymgr_reg_block.sw_share1_output_2
  UVM_INFO @  80498070 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test keymgr_stress_all_with_rand_reset has 1 failures.
  - 26.keymgr_stress_all_with_rand_reset.10591236833129932379037025579440150310824848819110047178964402626701164763115
    Line 1176, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/26.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 170029992 ps: (keymgr_scoreboard.sv:750) [uvm_test_top.env.scoreboard] Check failed item.d_data == `gmv(csr) (3517792881 [0xd1ad4271] vs 436253028 [0x1a00b164]) reg name: keymgr_reg_block.sw_share1_output_4
  UVM_INFO @ 170029992 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (keymgr_scoreboard.sv:750) [scoreboard] Check failed item.d_data ==gmv(csr) (* [] vs * []) reg name: keymgr_reg_block.sw_share0_output_*` has 1 failures:
- Test keymgr_lc_disable has 1 failures.
  - 21.keymgr_lc_disable.81427661354569779187177023988509349221995401791552272971282045460367100646405
    Line 609, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/21.keymgr_lc_disable/latest/run.log
```
  UVM_ERROR @  69645033 ps: (keymgr_scoreboard.sv:750) [uvm_test_top.env.scoreboard] Check failed item.d_data == `gmv(csr) (1626261028 [0x60eec624] vs 0 [0x0]) reg name: keymgr_reg_block.sw_share0_output_2
  UVM_INFO @  69645033 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (cip_base_scoreboard.sv:261) scoreboard [scoreboard] alert recov_operation_err is not received! has 1 failures:
- Test keymgr_sync_async_fault_cross has 1 failures.
  - 24.keymgr_sync_async_fault_cross.87342417125828740322199824389569207252185794974011193768901709666349023792928
    Line 340, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/24.keymgr_sync_async_fault_cross/latest/run.log
```
  UVM_ERROR @ 145241492 ps: (cip_base_scoreboard.sv:261) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err is not received!
  UVM_INFO @ 145241492 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```