KEYMGR Simulation Results

Tuesday August 13 2024 23:04:47 UTC

GitHub Revision: `098010d125`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 12185085088694708177096441863424670920996379189869351644310607217057882846251

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	keymgr_smoke	22.380s	2.729ms	50	50	100.00
V1	random	keymgr_random	1.303m	8.585ms	50	50	100.00
V1	csr_hw_reset	keymgr_csr_hw_reset	1.270s	69.984us	5	5	100.00
V1	csr_rw	keymgr_csr_rw	1.660s	456.537us	20	20	100.00
V1	csr_bit_bash	keymgr_csr_bit_bash	24.960s	3.268ms	5	5	100.00
V1	csr_aliasing	keymgr_csr_aliasing	9.400s	921.474us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	keymgr_csr_mem_rw_with_rand_reset	2.330s	80.950us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	keymgr_csr_rw	1.660s	456.537us	20	20	100.00
		keymgr_csr_aliasing	9.400s	921.474us	5	5	100.00
V1		TOTAL			155	155	100.00
V2	cfgen_during_op	keymgr_cfg_regwen	49.350s	3.333ms	50	50	100.00
V2	sideload	keymgr_sideload	43.130s	1.481ms	50	50	100.00
		keymgr_sideload_kmac	33.670s	1.010ms	49	50	98.00
		keymgr_sideload_aes	49.170s	10.618ms	50	50	100.00
		keymgr_sideload_otbn	37.530s	6.932ms	50	50	100.00
V2	direct_to_disabled_state	keymgr_direct_to_disabled	30.660s	1.012ms	50	50	100.00
V2	lc_disable	keymgr_lc_disable	9.610s	785.175us	48	50	96.00
V2	kmac_error_response	keymgr_kmac_rsp_err	7.180s	154.928us	50	50	100.00
V2	invalid_sw_input	keymgr_sw_invalid_input	1.602m	9.539ms	50	50	100.00
V2	invalid_hw_input	keymgr_hwsw_invalid_input	19.390s	717.147us	50	50	100.00
V2	sync_async_fault_cross	keymgr_sync_async_fault_cross	12.480s	2.463ms	49	50	98.00
V2	stress_all	keymgr_stress_all	10.554m	19.420ms	46	50	92.00
V2	intr_test	keymgr_intr_test	0.880s	102.927us	50	50	100.00
V2	alert_test	keymgr_alert_test	1.040s	17.976us	50	50	100.00
V2	tl_d_oob_addr_access	keymgr_tl_errors	4.160s	461.448us	20	20	100.00
V2	tl_d_illegal_access	keymgr_tl_errors	4.160s	461.448us	20	20	100.00
V2	tl_d_outstanding_access	keymgr_csr_hw_reset	1.270s	69.984us	5	5	100.00
		keymgr_csr_rw	1.660s	456.537us	20	20	100.00
		keymgr_csr_aliasing	9.400s	921.474us	5	5	100.00
		keymgr_same_csr_outstanding	3.980s	116.146us	20	20	100.00
V2	tl_d_partial_access	keymgr_csr_hw_reset	1.270s	69.984us	5	5	100.00
		keymgr_csr_rw	1.660s	456.537us	20	20	100.00
		keymgr_csr_aliasing	9.400s	921.474us	5	5	100.00
		keymgr_same_csr_outstanding	3.980s	116.146us	20	20	100.00
V2		TOTAL			732	740	98.92
V2S	sec_cm_additional_check	keymgr_sec_cm	13.600s	940.668us	5	5	100.00
V2S	tl_intg_err	keymgr_sec_cm	13.600s	940.668us	5	5	100.00
		keymgr_tl_intg_err	10.820s	977.382us	20	20	100.00
V2S	shadow_reg_update_error	keymgr_shadow_reg_errors	5.360s	644.731us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	keymgr_shadow_reg_errors	5.360s	644.731us	20	20	100.00
V2S	shadow_reg_storage_error	keymgr_shadow_reg_errors	5.360s	644.731us	20	20	100.00
V2S	shadowed_reset_glitch	keymgr_shadow_reg_errors	5.360s	644.731us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	keymgr_shadow_reg_errors_with_csr_rw	17.320s	595.723us	20	20	100.00
V2S	prim_count_check	keymgr_sec_cm	13.600s	940.668us	5	5	100.00
V2S	prim_fsm_check	keymgr_sec_cm	13.600s	940.668us	5	5	100.00
V2S	sec_cm_bus_integrity	keymgr_tl_intg_err	10.820s	977.382us	20	20	100.00
V2S	sec_cm_config_shadow	keymgr_shadow_reg_errors	5.360s	644.731us	20	20	100.00
V2S	sec_cm_op_config_regwen	keymgr_cfg_regwen	49.350s	3.333ms	50	50	100.00
V2S	sec_cm_reseed_config_regwen	keymgr_random	1.303m	8.585ms	50	50	100.00
		keymgr_csr_rw	1.660s	456.537us	20	20	100.00
V2S	sec_cm_sw_binding_config_regwen	keymgr_random	1.303m	8.585ms	50	50	100.00
		keymgr_csr_rw	1.660s	456.537us	20	20	100.00
V2S	sec_cm_max_key_ver_config_regwen	keymgr_random	1.303m	8.585ms	50	50	100.00
		keymgr_csr_rw	1.660s	456.537us	20	20	100.00
V2S	sec_cm_lc_ctrl_intersig_mubi	keymgr_lc_disable	9.610s	785.175us	48	50	96.00
V2S	sec_cm_constants_consistency	keymgr_hwsw_invalid_input	19.390s	717.147us	50	50	100.00
V2S	sec_cm_intersig_consistency	keymgr_hwsw_invalid_input	19.390s	717.147us	50	50	100.00
V2S	sec_cm_hw_key_sw_noaccess	keymgr_random	1.303m	8.585ms	50	50	100.00
V2S	sec_cm_output_keys_ctrl_redun	keymgr_sideload_protect	15.740s	568.253us	50	50	100.00
V2S	sec_cm_ctrl_fsm_sparse	keymgr_sec_cm	13.600s	940.668us	5	5	100.00
V2S	sec_cm_data_fsm_sparse	keymgr_sec_cm	13.600s	940.668us	5	5	100.00
V2S	sec_cm_ctrl_fsm_local_esc	keymgr_sec_cm	13.600s	940.668us	5	5	100.00
V2S	sec_cm_ctrl_fsm_consistency	keymgr_custom_cm	17.780s	533.214us	50	50	100.00
V2S	sec_cm_ctrl_fsm_global_esc	keymgr_lc_disable	9.610s	785.175us	48	50	96.00
V2S	sec_cm_ctrl_ctr_redun	keymgr_sec_cm	13.600s	940.668us	5	5	100.00
V2S	sec_cm_kmac_if_fsm_sparse	keymgr_sec_cm	13.600s	940.668us	5	5	100.00
V2S	sec_cm_kmac_if_ctr_redun	keymgr_sec_cm	13.600s	940.668us	5	5	100.00
V2S	sec_cm_kmac_if_cmd_ctrl_consistency	keymgr_custom_cm	17.780s	533.214us	50	50	100.00
V2S	sec_cm_kmac_if_done_ctrl_consistency	keymgr_custom_cm	17.780s	533.214us	50	50	100.00
V2S	sec_cm_reseed_ctr_redun	keymgr_sec_cm	13.600s	940.668us	5	5	100.00
V2S	sec_cm_side_load_sel_ctrl_consistency	keymgr_custom_cm	17.780s	533.214us	50	50	100.00
V2S	sec_cm_sideload_ctrl_fsm_sparse	keymgr_sec_cm	13.600s	940.668us	5	5	100.00
V2S	sec_cm_ctrl_key_integrity	keymgr_custom_cm	17.780s	533.214us	50	50	100.00
V2S		TOTAL			165	165	100.00
V3	stress_all_with_rand_reset	keymgr_stress_all_with_rand_reset	34.620s	6.641ms	33	50	66.00
V3		TOTAL			33	50	66.00
		TOTAL			1085	1110	97.75

Testplan Progress

Items	Total	Written	Passing	Progress
V1	7	7	7	100.00
V2	16	16	12	75.00
V2S	6	6	6	100.00
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
97.44	99.00	98.11	98.79	97.67	98.93	98.41	91.19

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:867) [keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 17 failures:
- Test keymgr_stress_all_with_rand_reset has 17 failures.
  - 0.keymgr_stress_all_with_rand_reset.78695544100821481952151183988438545838872525736038636895267375235847189270913
    Line 512, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/0.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 654826764 ps: (cip_base_vseq.sv:867) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 654826764 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 3.keymgr_stress_all_with_rand_reset.38794362722560512222395038488198268439974213601553590313563499733960077300734
    Line 561, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/3.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 763881616 ps: (cip_base_vseq.sv:867) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 763881616 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 15 more failures.

UVM_ERROR (keymgr_scoreboard.sv:766) [scoreboard] Check failed item.d_data !=gmv(csr) (* [] vs * []) reg name: keymgr_reg_block.sw_share0_output_*` has 4 failures:

Test keymgr_stress_all has 3 failures.

3.keymgr_stress_all.16197319581407208477778412132539085548747923592939316199975470778251120907927
Line 4076, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/3.keymgr_stress_all/latest/run.log

  UVM_ERROR @ 3910432619 ps: (keymgr_scoreboard.sv:766) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (0 [0x0] vs 0 [0x0]) reg name: keymgr_reg_block.sw_share0_output_1
  UVM_INFO @ 3910432619 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

5.keymgr_stress_all.110142245847202255126813055352596406569065746298120423383257200031136046305623
Line 333, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/5.keymgr_stress_all/latest/run.log

  UVM_ERROR @  99366617 ps: (keymgr_scoreboard.sv:766) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (0 [0x0] vs 0 [0x0]) reg name: keymgr_reg_block.sw_share0_output_1
  UVM_INFO @  99366617 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 1 more failures.

Test keymgr_lc_disable has 1 failures.

49.keymgr_lc_disable.76828452835984836554347977146395760002832610622246917391561331350900193987492
Line 483, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/49.keymgr_lc_disable/latest/run.log

  UVM_ERROR @  42786584 ps: (keymgr_scoreboard.sv:766) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (3630247607 [0xd8612eb7] vs 3630247607 [0xd8612eb7]) reg name: keymgr_reg_block.sw_share0_output_1
  UVM_INFO @  42786584 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_ERROR (cip_base_scoreboard.sv:304) scoreboard [scoreboard] alert recov_operation_err did not trigger max_delay:* has 2 failures:
- Test keymgr_stress_all has 1 failures.
  - 23.keymgr_stress_all.5399737201325315920394052275075918857435770787962031211628078235038244724031
    Line 2250, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/23.keymgr_stress_all/latest/run.log
```
  UVM_ERROR @ 1116909181 ps: (cip_base_scoreboard.sv:304) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:2000
  UVM_INFO @ 1116909181 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test keymgr_sideload_kmac has 1 failures.
  - 48.keymgr_sideload_kmac.89501337955993037777040490048613841660176974458744615940326999907732176070413
    Line 274, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/48.keymgr_sideload_kmac/latest/run.log
```
  UVM_ERROR @   8772035 ps: (cip_base_scoreboard.sv:304) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @   8772035 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (keymgr_sync_async_fault_cross_vseq.sv:38) [keymgr_sync_async_fault_cross_vseq] Check failed act_fault_status[keymgr_pkg::FaultKmacOp] || act_fault_status[keymgr_pkg::FaultKmacOut] == * (* [*] vs * [*]) has 1 failures:
- Test keymgr_sync_async_fault_cross has 1 failures.
  - 8.keymgr_sync_async_fault_cross.14923545418979203314093397106798910151924162529273191056274084282523490014638
    Line 272, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/8.keymgr_sync_async_fault_cross/latest/run.log
```
  UVM_ERROR @  66600402 ps: (keymgr_sync_async_fault_cross_vseq.sv:38) [uvm_test_top.env.virtual_sequencer.keymgr_sync_async_fault_cross_vseq] Check failed act_fault_status[keymgr_pkg::FaultKmacOp] ||                 act_fault_status[keymgr_pkg::FaultKmacOut] == 1 (0 [0x0] vs 1 [0x1])
  UVM_INFO @  66600402 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```

UVM_ERROR (keymgr_if.sv:557) [keymgr_if] Check failed {act_key.key[*], act_key.key[*]} !== keys_a_array[state][cdi][dest] (* [*] vs * [*]) KMAC key at state StCreatorRootKey for Sealing Kmac has 1 failures:

Test keymgr_lc_disable has 1 failures.

17.keymgr_lc_disable.104528277743198567720539839237291777054787144887605698131486759984072594133774
Line 411, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/17.keymgr_lc_disable/latest/run.log

  UVM_ERROR @ 595365726 ps: (keymgr_if.sv:557) [keymgr_if] Check failed {act_key.key[1], act_key.key[0]} !== keys_a_array[state][cdi][dest] (10054929577932339944112549062108974192823361961894838785639823257181909360907250483685063802496017004670234361346808521476822021425335135491689166273159172 [0xbffb78d4ff618ebfddfae08a88d4a7135b208d17af95cbc23608a851e4e3f743a854ef564f83707c57d1cb63c27b675f3c1525aa27ab311cc34f072d79baa004] vs 10054929577932339944112549062108974192823361961894838785639823257181909360907250483685063802496017004670234361346808521476822021425335135491689166273159172 [0xbffb78d4ff618ebfddfae08a88d4a7135b208d17af95cbc23608a851e4e3f743a854ef564f83707c57d1cb63c27b675f3c1525aa27ab311cc34f072d79baa004]) KMAC key at state StCreatorRootKey for Sealing Kmac
  UVM_INFO @ 595365726 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---