KEYMGR Simulation Results

Wednesday August 21 2024 01:12:47 UTC

GitHub Revision: `34b8fc33e3`

Branch: earlgrey_1_0_0_2024_08_20_RC0

Simulator: VCS

Build randomization enabled with --build-seed 77645589415139663032322841827996135987237190720163469870959218015679941996572

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	keymgr_smoke	35.760s	1.517ms	50	50	100.00
V1	random	keymgr_random	51.420s	1.860ms	50	50	100.00
V1	csr_hw_reset	keymgr_csr_hw_reset	1.640s	31.280us	5	5	100.00
V1	csr_rw	keymgr_csr_rw	1.960s	60.087us	20	20	100.00
V1	csr_bit_bash	keymgr_csr_bit_bash	23.840s	949.854us	5	5	100.00
V1	csr_aliasing	keymgr_csr_aliasing	12.360s	789.204us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	keymgr_csr_mem_rw_with_rand_reset	2.560s	35.760us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	keymgr_csr_rw	1.960s	60.087us	20	20	100.00
		keymgr_csr_aliasing	12.360s	789.204us	5	5	100.00
V1		TOTAL			155	155	100.00
V2	cfgen_during_op	keymgr_cfg_regwen	1.416m	8.015ms	50	50	100.00
V2	sideload	keymgr_sideload	54.510s	2.593ms	49	50	98.00
		keymgr_sideload_kmac	44.410s	7.057ms	50	50	100.00
		keymgr_sideload_aes	59.910s	6.116ms	50	50	100.00
		keymgr_sideload_otbn	36.800s	1.597ms	50	50	100.00
V2	direct_to_disabled_state	keymgr_direct_to_disabled	30.170s	1.046ms	50	50	100.00
V2	lc_disable	keymgr_lc_disable	17.370s	425.651us	49	50	98.00
V2	kmac_error_response	keymgr_kmac_rsp_err	7.800s	1.112ms	49	50	98.00
V2	invalid_sw_input	keymgr_sw_invalid_input	50.780s	15.988ms	50	50	100.00
V2	invalid_hw_input	keymgr_hwsw_invalid_input	58.410s	2.082ms	50	50	100.00
V2	sync_async_fault_cross	keymgr_sync_async_fault_cross	14.510s	2.533ms	50	50	100.00
V2	stress_all	keymgr_stress_all	2.941m	8.827ms	48	50	96.00
V2	intr_test	keymgr_intr_test	1.250s	40.786us	50	50	100.00
V2	alert_test	keymgr_alert_test	1.470s	317.970us	50	50	100.00
V2	tl_d_oob_addr_access	keymgr_tl_errors	4.560s	559.826us	20	20	100.00
V2	tl_d_illegal_access	keymgr_tl_errors	4.560s	559.826us	20	20	100.00
V2	tl_d_outstanding_access	keymgr_csr_hw_reset	1.640s	31.280us	5	5	100.00
		keymgr_csr_rw	1.960s	60.087us	20	20	100.00
		keymgr_csr_aliasing	12.360s	789.204us	5	5	100.00
		keymgr_same_csr_outstanding	2.980s	417.206us	20	20	100.00
V2	tl_d_partial_access	keymgr_csr_hw_reset	1.640s	31.280us	5	5	100.00
		keymgr_csr_rw	1.960s	60.087us	20	20	100.00
		keymgr_csr_aliasing	12.360s	789.204us	5	5	100.00
		keymgr_same_csr_outstanding	2.980s	417.206us	20	20	100.00
V2		TOTAL			735	740	99.32
V2S	sec_cm_additional_check	keymgr_sec_cm	20.360s	665.899us	5	5	100.00
V2S	tl_intg_err	keymgr_sec_cm	20.360s	665.899us	5	5	100.00
		keymgr_tl_intg_err	7.660s	226.036us	20	20	100.00
V2S	shadow_reg_update_error	keymgr_shadow_reg_errors	8.900s	509.230us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	keymgr_shadow_reg_errors	8.900s	509.230us	20	20	100.00
V2S	shadow_reg_storage_error	keymgr_shadow_reg_errors	8.900s	509.230us	20	20	100.00
V2S	shadowed_reset_glitch	keymgr_shadow_reg_errors	8.900s	509.230us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	keymgr_shadow_reg_errors_with_csr_rw	8.560s	380.975us	20	20	100.00
V2S	prim_count_check	keymgr_sec_cm	20.360s	665.899us	5	5	100.00
V2S	prim_fsm_check	keymgr_sec_cm	20.360s	665.899us	5	5	100.00
V2S	sec_cm_bus_integrity	keymgr_tl_intg_err	7.660s	226.036us	20	20	100.00
V2S	sec_cm_config_shadow	keymgr_shadow_reg_errors	8.900s	509.230us	20	20	100.00
V2S	sec_cm_op_config_regwen	keymgr_cfg_regwen	1.416m	8.015ms	50	50	100.00
V2S	sec_cm_reseed_config_regwen	keymgr_random	51.420s	1.860ms	50	50	100.00
		keymgr_csr_rw	1.960s	60.087us	20	20	100.00
V2S	sec_cm_sw_binding_config_regwen	keymgr_random	51.420s	1.860ms	50	50	100.00
		keymgr_csr_rw	1.960s	60.087us	20	20	100.00
V2S	sec_cm_max_key_ver_config_regwen	keymgr_random	51.420s	1.860ms	50	50	100.00
		keymgr_csr_rw	1.960s	60.087us	20	20	100.00
V2S	sec_cm_lc_ctrl_intersig_mubi	keymgr_lc_disable	17.370s	425.651us	49	50	98.00
V2S	sec_cm_constants_consistency	keymgr_hwsw_invalid_input	58.410s	2.082ms	50	50	100.00
V2S	sec_cm_intersig_consistency	keymgr_hwsw_invalid_input	58.410s	2.082ms	50	50	100.00
V2S	sec_cm_hw_key_sw_noaccess	keymgr_random	51.420s	1.860ms	50	50	100.00
V2S	sec_cm_output_keys_ctrl_redun	keymgr_sideload_protect	28.470s	869.597us	50	50	100.00
V2S	sec_cm_ctrl_fsm_sparse	keymgr_sec_cm	20.360s	665.899us	5	5	100.00
V2S	sec_cm_data_fsm_sparse	keymgr_sec_cm	20.360s	665.899us	5	5	100.00
V2S	sec_cm_ctrl_fsm_local_esc	keymgr_sec_cm	20.360s	665.899us	5	5	100.00
V2S	sec_cm_ctrl_fsm_consistency	keymgr_custom_cm	34.990s	823.754us	50	50	100.00
V2S	sec_cm_ctrl_fsm_global_esc	keymgr_lc_disable	17.370s	425.651us	49	50	98.00
V2S	sec_cm_ctrl_ctr_redun	keymgr_sec_cm	20.360s	665.899us	5	5	100.00
V2S	sec_cm_kmac_if_fsm_sparse	keymgr_sec_cm	20.360s	665.899us	5	5	100.00
V2S	sec_cm_kmac_if_ctr_redun	keymgr_sec_cm	20.360s	665.899us	5	5	100.00
V2S	sec_cm_kmac_if_cmd_ctrl_consistency	keymgr_custom_cm	34.990s	823.754us	50	50	100.00
V2S	sec_cm_kmac_if_done_ctrl_consistency	keymgr_custom_cm	34.990s	823.754us	50	50	100.00
V2S	sec_cm_reseed_ctr_redun	keymgr_sec_cm	20.360s	665.899us	5	5	100.00
V2S	sec_cm_side_load_sel_ctrl_consistency	keymgr_custom_cm	34.990s	823.754us	50	50	100.00
V2S	sec_cm_sideload_ctrl_fsm_sparse	keymgr_sec_cm	20.360s	665.899us	5	5	100.00
V2S	sec_cm_ctrl_key_integrity	keymgr_custom_cm	34.990s	823.754us	50	50	100.00
V2S		TOTAL			165	165	100.00
V3	stress_all_with_rand_reset	keymgr_stress_all_with_rand_reset	29.770s	7.483ms	27	50	54.00
V3		TOTAL			27	50	54.00
		TOTAL			1082	1110	97.48

Testplan Progress

Items	Total	Written	Passing	Progress
V1	7	7	7	100.00
V2	16	16	12	75.00
V2S	6	6	6	100.00
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
97.77	99.04	97.79	98.70	100.00	99.02	98.63	91.19

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:867) [keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 22 failures:
- Test keymgr_stress_all_with_rand_reset has 22 failures.
  - 5.keymgr_stress_all_with_rand_reset.54841703406726054690564223673753691641828802806987144805803210769809923570528
    Line 184, in log /workspaces/lowrisc/opentitan/scratch/earlgrey_1_0_0_2024_08_20_RC0/keymgr-sim-vcs/5.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 493857286 ps: (cip_base_vseq.sv:867) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 493857286 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 7.keymgr_stress_all_with_rand_reset.30069536813733778961367851087931069249863387039027950785183880829623757547562
    Line 551, in log /workspaces/lowrisc/opentitan/scratch/earlgrey_1_0_0_2024_08_20_RC0/keymgr-sim-vcs/7.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 217433103 ps: (cip_base_vseq.sv:867) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 217433103 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 20 more failures.

UVM_ERROR (keymgr_scoreboard.sv:766) [scoreboard] Check failed item.d_data !=gmv(csr) (* [] vs * []) reg name: keymgr_reg_block.sw_share1_output_*` has 3 failures:

Test keymgr_lc_disable has 1 failures.

5.keymgr_lc_disable.113866362157749586819601165919089655190528739715460669252734116580250571106562
Line 329, in log /workspaces/lowrisc/opentitan/scratch/earlgrey_1_0_0_2024_08_20_RC0/keymgr-sim-vcs/5.keymgr_lc_disable/latest/run.log

  UVM_ERROR @ 192603481 ps: (keymgr_scoreboard.sv:766) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (2382946206 [0x8e08df9e] vs 2382946206 [0x8e08df9e]) reg name: keymgr_reg_block.sw_share1_output_3
  UVM_INFO @ 192603481 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test keymgr_stress_all has 2 failures.

6.keymgr_stress_all.20982371607659638525508871660731233605919853342553200052500592532829976715821
Line 156, in log /workspaces/lowrisc/opentitan/scratch/earlgrey_1_0_0_2024_08_20_RC0/keymgr-sim-vcs/6.keymgr_stress_all/latest/run.log

  UVM_ERROR @  34505866 ps: (keymgr_scoreboard.sv:766) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (0 [0x0] vs 0 [0x0]) reg name: keymgr_reg_block.sw_share1_output_3
  UVM_INFO @  34505866 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

14.keymgr_stress_all.38451408391264857075867710246267961247702546532331179673041891526786783463295
Line 2081, in log /workspaces/lowrisc/opentitan/scratch/earlgrey_1_0_0_2024_08_20_RC0/keymgr-sim-vcs/14.keymgr_stress_all/latest/run.log

  UVM_ERROR @ 3356029665 ps: (keymgr_scoreboard.sv:766) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (0 [0x0] vs 0 [0x0]) reg name: keymgr_reg_block.sw_share1_output_2
  UVM_INFO @ 3356029665 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_ERROR (keymgr_scoreboard.sv:1064) [scoreboard] Check failed act == exp (* [*] vs * [*]) cdi_type: Attestation has 1 failures:

Test keymgr_stress_all_with_rand_reset has 1 failures.

2.keymgr_stress_all_with_rand_reset.28685265396837416229333172869186145457083741076270528697875708404463765902599
Line 332, in log /workspaces/lowrisc/opentitan/scratch/earlgrey_1_0_0_2024_08_20_RC0/keymgr-sim-vcs/2.keymgr_stress_all_with_rand_reset/latest/run.log

  UVM_ERROR @ 173832160 ps: (keymgr_scoreboard.sv:1064) [uvm_test_top.env.scoreboard] Check failed act == exp (17473282431871643895894400773827752444036501438026918448402420114383552373230344920419031497313544908429971099806777154315394979215162265899911714474158503148586925612970698171278373139570332583682402781862325005336035298027872215446468265304441903519337895382603011244662619555047926512574875484877097516895445707712496708497226502338449783304953 [0x491fb922dfb10c515d7616e4ad2bfcbe6032ec8ae01c88879b27776e2ec3e76a8b1e64e07f1f92af15b9227f225a3fca0670075952bee2b33086865a24c086cffa65394d72447d68f6c2e13e6d1a4334ef852e99021e27fa9deb418cbfce0528d138afccaaed711ff6905acebdf3a4303a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9] vs 16667549483362017062068632699886767979932818020237701417840856770347963923385782372530966770906292996273611978358614501400487899035371322208425600384877427495527732724024477708744155865323350786016615424950799993793739572260596395850459426478229350249643148400839776185350357853720184421958808072095013385759343773770641436959243423382173068210937 [0x45c08396341e37004757d1e0904d7541cac2a6e8fdd168df7da6a20f57fac6098b1e64e07f1f92af15b9227f225a3fca0670075952bee2b33086865a24c086cffa65394d72447d68f6c2e13e6d1a4334ef852e99021e27fa9deb418cbfce0528d138afccaaed711ff6905acebdf3a4303a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9]) cdi_type: Attestation
  
   HardwareRevisionSecret act: 0x3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9, exp: 0x3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9
   RomDigest act: 0xef852e99021e27fa9deb418cbfce0528d138afccaaed711ff6905acebdf3a430, exp: 0xef852e99021e27fa9deb418cbfce0528d138afccaaed711ff6905acebdf3a430
   HealthMeasurement act: 0xfa65394d72447d68f6c2e13e6d1a4334, exp: 0xfa65394d72447d68f6c2e13e6d1a4334

UVM_ERROR (cip_base_scoreboard.sv:242) [scoreboard] Check failed expected_alert[alert_name].expected == * (* [*] vs * [*]) alert recov_operation_err triggered unexpectedly has 1 failures:
- Test keymgr_kmac_rsp_err has 1 failures.
  - 10.keymgr_kmac_rsp_err.102380749593493791332634948189936783270354842050800082135337849427143956779138
    Line 232, in log /workspaces/lowrisc/opentitan/scratch/earlgrey_1_0_0_2024_08_20_RC0/keymgr-sim-vcs/10.keymgr_kmac_rsp_err/latest/run.log
```
  UVM_ERROR @  56175223 ps: (cip_base_scoreboard.sv:242) [uvm_test_top.env.scoreboard] Check failed expected_alert[alert_name].expected == 1 (0 [0x0] vs 1 [0x1]) alert recov_operation_err triggered unexpectedly
  UVM_INFO @  56175223 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (cip_base_scoreboard.sv:304) scoreboard [scoreboard] alert recov_operation_err did not trigger max_delay:* has 1 failures:
- Test keymgr_sideload has 1 failures.
  - 29.keymgr_sideload.64093263553369058080250488420322739236141744676040553789391234085915060403722
    Line 101, in log /workspaces/lowrisc/opentitan/scratch/earlgrey_1_0_0_2024_08_20_RC0/keymgr-sim-vcs/29.keymgr_sideload/latest/run.log
```
  UVM_ERROR @  12464445 ps: (cip_base_scoreboard.sv:304) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @  12464445 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```