KEYMGR Simulation Results

Sunday March 17 2024 19:02:52 UTC

GitHub Revision: `c187a82ee8`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 28440605375541353837496064678278045899395893237469128852560697715229879921060

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	keymgr_smoke	59.500s	12.299ms	50	50	100.00
V1	random	keymgr_random	1.212m	31.114ms	50	50	100.00
V1	csr_hw_reset	keymgr_csr_hw_reset	1.450s	146.123us	5	5	100.00
V1	csr_rw	keymgr_csr_rw	1.520s	377.600us	20	20	100.00
V1	csr_bit_bash	keymgr_csr_bit_bash	30.910s	1.112ms	5	5	100.00
V1	csr_aliasing	keymgr_csr_aliasing	18.100s	524.442us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	keymgr_csr_mem_rw_with_rand_reset	2.700s	145.576us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	keymgr_csr_rw	1.520s	377.600us	20	20	100.00
		keymgr_csr_aliasing	18.100s	524.442us	5	5	100.00
V1		TOTAL			155	155	100.00
V2	cfgen_during_op	keymgr_cfg_regwen	2.325m	2.667ms	49	50	98.00
V2	sideload	keymgr_sideload	1.349m	18.161ms	50	50	100.00
		keymgr_sideload_kmac	1.716m	28.752ms	50	50	100.00
		keymgr_sideload_aes	1.530m	8.415ms	50	50	100.00
		keymgr_sideload_otbn	1.138m	10.028ms	50	50	100.00
V2	direct_to_disabled_state	keymgr_direct_to_disabled	35.580s	1.187ms	49	50	98.00
V2	lc_disable	keymgr_lc_disable	11.410s	417.180us	50	50	100.00
V2	kmac_error_response	keymgr_kmac_rsp_err	30.420s	2.780ms	45	50	90.00
V2	invalid_sw_input	keymgr_sw_invalid_input	45.930s	17.201ms	49	50	98.00
V2	invalid_hw_input	keymgr_hwsw_invalid_input	1.204m	6.591ms	50	50	100.00
V2	sync_async_fault_cross	keymgr_sync_async_fault_cross	25.820s	3.683ms	50	50	100.00
V2	stress_all	keymgr_stress_all	11.687m	109.130ms	50	50	100.00
V2	intr_test	keymgr_intr_test	0.920s	15.187us	50	50	100.00
V2	alert_test	keymgr_alert_test	1.050s	20.838us	50	50	100.00
V2	tl_d_oob_addr_access	keymgr_tl_errors	6.210s	1.108ms	20	20	100.00
V2	tl_d_illegal_access	keymgr_tl_errors	6.210s	1.108ms	20	20	100.00
V2	tl_d_outstanding_access	keymgr_csr_hw_reset	1.450s	146.123us	5	5	100.00
		keymgr_csr_rw	1.520s	377.600us	20	20	100.00
		keymgr_csr_aliasing	18.100s	524.442us	5	5	100.00
		keymgr_same_csr_outstanding	4.600s	469.871us	20	20	100.00
V2	tl_d_partial_access	keymgr_csr_hw_reset	1.450s	146.123us	5	5	100.00
		keymgr_csr_rw	1.520s	377.600us	20	20	100.00
		keymgr_csr_aliasing	18.100s	524.442us	5	5	100.00
		keymgr_same_csr_outstanding	4.600s	469.871us	20	20	100.00
V2		TOTAL			732	740	98.92
V2S	sec_cm_additional_check	keymgr_sec_cm	19.110s	2.354ms	5	5	100.00
V2S	tl_intg_err	keymgr_sec_cm	19.110s	2.354ms	5	5	100.00
		keymgr_tl_intg_err	59.440s	2.123ms	20	20	100.00
V2S	shadow_reg_update_error	keymgr_shadow_reg_errors	9.760s	911.431us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	keymgr_shadow_reg_errors	9.760s	911.431us	20	20	100.00
V2S	shadow_reg_storage_error	keymgr_shadow_reg_errors	9.760s	911.431us	20	20	100.00
V2S	shadowed_reset_glitch	keymgr_shadow_reg_errors	9.760s	911.431us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	keymgr_shadow_reg_errors_with_csr_rw	11.290s	1.480ms	20	20	100.00
V2S	prim_count_check	keymgr_sec_cm	19.110s	2.354ms	5	5	100.00
V2S	prim_fsm_check	keymgr_sec_cm	19.110s	2.354ms	5	5	100.00
V2S	sec_cm_bus_integrity	keymgr_tl_intg_err	59.440s	2.123ms	20	20	100.00
V2S	sec_cm_config_shadow	keymgr_shadow_reg_errors	9.760s	911.431us	20	20	100.00
V2S	sec_cm_op_config_regwen	keymgr_cfg_regwen	2.325m	2.667ms	49	50	98.00
V2S	sec_cm_reseed_config_regwen	keymgr_random	1.212m	31.114ms	50	50	100.00
		keymgr_csr_rw	1.520s	377.600us	20	20	100.00
V2S	sec_cm_sw_binding_config_regwen	keymgr_random	1.212m	31.114ms	50	50	100.00
		keymgr_csr_rw	1.520s	377.600us	20	20	100.00
V2S	sec_cm_max_key_ver_config_regwen	keymgr_random	1.212m	31.114ms	50	50	100.00
		keymgr_csr_rw	1.520s	377.600us	20	20	100.00
V2S	sec_cm_lc_ctrl_intersig_mubi	keymgr_lc_disable	11.410s	417.180us	50	50	100.00
V2S	sec_cm_constants_consistency	keymgr_hwsw_invalid_input	1.204m	6.591ms	50	50	100.00
V2S	sec_cm_intersig_consistency	keymgr_hwsw_invalid_input	1.204m	6.591ms	50	50	100.00
V2S	sec_cm_hw_key_sw_noaccess	keymgr_random	1.212m	31.114ms	50	50	100.00
V2S	sec_cm_output_keys_ctrl_redun	keymgr_sideload_protect	18.300s	635.783us	50	50	100.00
V2S	sec_cm_ctrl_fsm_sparse	keymgr_sec_cm	19.110s	2.354ms	5	5	100.00
V2S	sec_cm_data_fsm_sparse	keymgr_sec_cm	19.110s	2.354ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_local_esc	keymgr_sec_cm	19.110s	2.354ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_consistency	keymgr_custom_cm	32.220s	2.113ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_global_esc	keymgr_lc_disable	11.410s	417.180us	50	50	100.00
V2S	sec_cm_ctrl_ctr_redun	keymgr_sec_cm	19.110s	2.354ms	5	5	100.00
V2S	sec_cm_kmac_if_fsm_sparse	keymgr_sec_cm	19.110s	2.354ms	5	5	100.00
V2S	sec_cm_kmac_if_ctr_redun	keymgr_sec_cm	19.110s	2.354ms	5	5	100.00
V2S	sec_cm_kmac_if_cmd_ctrl_consistency	keymgr_custom_cm	32.220s	2.113ms	50	50	100.00
V2S	sec_cm_kmac_if_done_ctrl_consistency	keymgr_custom_cm	32.220s	2.113ms	50	50	100.00
V2S	sec_cm_reseed_ctr_redun	keymgr_sec_cm	19.110s	2.354ms	5	5	100.00
V2S	sec_cm_side_load_sel_ctrl_consistency	keymgr_custom_cm	32.220s	2.113ms	50	50	100.00
V2S	sec_cm_sideload_ctrl_fsm_sparse	keymgr_sec_cm	19.110s	2.354ms	5	5	100.00
V2S	sec_cm_ctrl_key_integrity	keymgr_custom_cm	32.220s	2.113ms	50	50	100.00
V2S		TOTAL			165	165	100.00
V3	stress_all_with_rand_reset	keymgr_stress_all_with_rand_reset	25.900s	1.059ms	22	50	44.00
V3		TOTAL			22	50	44.00
		TOTAL			1074	1110	96.76

Testplan Progress

Items	Total	Written	Passing	Progress
V1	7	7	7	100.00
V2	16	16	12	75.00
V2S	6	6	6	100.00
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
97.94	99.07	97.99	99.36	100.00	99.11	98.41	91.63

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:827) [keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 23 failures:
- Test keymgr_stress_all_with_rand_reset has 23 failures.
  - 0.keymgr_stress_all_with_rand_reset.106614636244807549602213142309953528499950623855115779357460798775187438245236
    Line 282, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/0.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 698546387 ps: (cip_base_vseq.sv:827) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 698546387 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 2.keymgr_stress_all_with_rand_reset.33355766774586239032067403532680943647411859954583513382314784744850594662872
    Line 1149, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/2.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 662399833 ps: (cip_base_vseq.sv:827) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 662399833 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 21 more failures.

UVM_ERROR (keymgr_scoreboard.sv:1013) [scoreboard] Check failed act != exp (* [*] vs * [*]) cdi_type: Attestation has 5 failures:

Test keymgr_kmac_rsp_err has 5 failures.

2.keymgr_kmac_rsp_err.85661052059685194256290042616850836437992741925467235387674354796304853974416
Line 323, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/2.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @  67450214 ps: (keymgr_scoreboard.sv:1013) [uvm_test_top.env.scoreboard] Check failed act != exp (5517581618861415672393912256334261950619221579860668537599350644700277865727854431006184551879828610386586105306813974371987822637324695399699328070475917251981270706968390291663278752693903405877341013828758736646090480058417293423352871090844163065620488385201941580480765343495721874983197656839218334062052047601189940062345141199647783637914599990203271087174066053980279216181243591098980706111045381017650711242508906 [0xc769e394000000008e3694b4000000000000000000000000984f0daed5fa50d23a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f972819b31b888adc0a3c85d406dd1395c1ec98f99a2f16f2c26623c4f4411bfcb394eefc5ca1431f8aba89465d3beae3d90bad11540fd33abdfb2cd4d88bb1b7e5cfbc766260e2758a7773f5bc2ce1546c6b30bb81b5462147bc1ef4115355deeff9c59d8c9e8bbe357b3d16456988a6a] vs 5517581618861415672393912256334261950619221579860668537599350644700277865727854431006184551879828610386586105306813974371987822637324695399699328070475917251981270706968390291663278752693903405877341013828758736646090480058417293423352871090844163065620488385201941580480765343495721874983197656839218334062052047601189940062345141199647783637914599990203271087174066053980279216181243591098980706111045381017650711242508906 [0xc769e394000000008e3694b4000000000000000000000000984f0daed5fa50d23a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f972819b31b888adc0a3c85d406dd1395c1ec98f99a2f16f2c26623c4f4411bfcb394eefc5ca1431f8aba89465d3beae3d90bad11540fd33abdfb2cd4d88bb1b7e5cfbc766260e2758a7773f5bc2ce1546c6b30bb81b5462147bc1ef4115355deeff9c59d8c9e8bbe357b3d16456988a6a]) cdi_type: Attestation
  
   DiversificationKey act: 0xc6b30bb81b5462147bc1ef4115355deeff9c59d8c9e8bbe357b3d16456988a6a, exp: 0xc6b30bb81b5462147bc1ef4115355deeff9c59d8c9e8bbe357b3d16456988a6a
   RomDigest act: 0x90bad11540fd33abdfb2cd4d88bb1b7e5cfbc766260e2758a7773f5bc2ce1546, exp: 0x90bad11540fd33abdfb2cd4d88bb1b7e5cfbc766260e2758a7773f5bc2ce1546
   HealthMeasurement act: 0x394eefc5ca1431f8aba89465d3beae3d, exp: 0x394eefc5ca1431f8aba89465d3beae3d

8.keymgr_kmac_rsp_err.98843527600673898324121981800792338382930640934209356490556896979269902436121
Line 305, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/8.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @  12891901 ps: (keymgr_scoreboard.sv:1013) [uvm_test_top.env.scoreboard] Check failed act != exp (13869088684237589750254652523805745455607483863801036890679746459828093165537939935887096654247164594929776882497712621079698203267226285354970261501773355039096796333224940548605755985658595266936671954126671257169472523293290900176047420928494560727880565863637625602255031886572127739397650945410521773882568576810139776157851783631849466583755 [0x3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f998e35b87e75fa842f76510ab4818b63be52289249dcbb900f37f24ac1d6bbe8248687bea5cbe754ff7bce0e03b71a848449f1dc39dac09b9562a44750404fec318b4507ae4e5470318b335a7db871802991c39867bbd5def000d51c402a3fd91559a5271ff1fa26034fb62235faf5acb] vs 13869088684237589750254652523805745455607483863801036890679746459828093165537939935887096654247164594929776882497712621079698203267226285354970261501773355039096796333224940548605755985658595266936671954126671257169472523293290900176047420928494560727880565863637625602255031886572127739397650945410521773882568576810139776157851783631849466583755 [0x3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f998e35b87e75fa842f76510ab4818b63be52289249dcbb900f37f24ac1d6bbe8248687bea5cbe754ff7bce0e03b71a848449f1dc39dac09b9562a44750404fec318b4507ae4e5470318b335a7db871802991c39867bbd5def000d51c402a3fd91559a5271ff1fa26034fb62235faf5acb]) cdi_type: Attestation
  
   DiversificationKey act: 0x991c39867bbd5def000d51c402a3fd91559a5271ff1fa26034fb62235faf5acb, exp: 0x991c39867bbd5def000d51c402a3fd91559a5271ff1fa26034fb62235faf5acb
   RomDigest act: 0x449f1dc39dac09b9562a44750404fec318b4507ae4e5470318b335a7db871802, exp: 0x449f1dc39dac09b9562a44750404fec318b4507ae4e5470318b335a7db871802
   HealthMeasurement act: 0x48687bea5cbe754ff7bce0e03b71a848, exp: 0x48687bea5cbe754ff7bce0e03b71a848

... and 3 more failures.

UVM_ERROR (keymgr_scoreboard.sv:736) [scoreboard] Check failed item.d_data !=gmv(csr) (* [] vs * [])` has 4 failures:
- Test keymgr_stress_all_with_rand_reset has 4 failures.
  - 36.keymgr_stress_all_with_rand_reset.81746493180211601396168605172374053239358223217226182462586153773326550896097
    Line 493, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/36.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 110198107 ps: (keymgr_scoreboard.sv:736) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (2384700896 [0x8e23a5e0] vs 2384700896 [0x8e23a5e0])
  UVM_INFO @ 110198107 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 41.keymgr_stress_all_with_rand_reset.23480719213189929958449096580988075278804809155275089736713059987764659542268
    Line 899, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/41.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 255771404 ps: (keymgr_scoreboard.sv:736) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (2573897299 [0x996a8e53] vs 2573897299 [0x996a8e53])
  UVM_INFO @ 255771404 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 2 more failures.
UVM_ERROR (cip_base_scoreboard.sv:301) scoreboard [scoreboard] alert recov_operation_err did not trigger max_delay:* has 3 failures:
- Test keymgr_cfg_regwen has 1 failures.
  - 35.keymgr_cfg_regwen.48407012116229086074091604997366820898845075971128900111015450608423892388024
    Line 329, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/35.keymgr_cfg_regwen/latest/run.log
```
  UVM_ERROR @  41636578 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @  41636578 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test keymgr_sw_invalid_input has 1 failures.
  - 41.keymgr_sw_invalid_input.93058780864948991945763793853597771231292338534616925628226785486391934775862
    Line 442, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/41.keymgr_sw_invalid_input/latest/run.log
```
  UVM_ERROR @  59659632 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @  59659632 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test keymgr_direct_to_disabled has 1 failures.
  - 43.keymgr_direct_to_disabled.95068069475122704688242003904766391540786370677887975760064248882080823407394
    Line 327, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/43.keymgr_direct_to_disabled/latest/run.log
```
  UVM_ERROR @   4824929 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @   4824929 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (keymgr_scoreboard.sv:750) [scoreboard] Check failed item.d_data ==gmv(csr) (* [] vs * []) reg name: keymgr_reg_block.sw_share1_output_*` has 1 failures:
- Test keymgr_stress_all_with_rand_reset has 1 failures.
  - 4.keymgr_stress_all_with_rand_reset.60516174939533317749492451217901413762688609331435708006876068176701952961711
    Line 1801, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/4.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 2136057720 ps: (keymgr_scoreboard.sv:750) [uvm_test_top.env.scoreboard] Check failed item.d_data == `gmv(csr) (1681080656 [0x64334150] vs 0 [0x0]) reg name: keymgr_reg_block.sw_share1_output_4
  UVM_INFO @ 2136057720 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```