KEYMGR Simulation Results

Tuesday September 24 2024 01:05:57 UTC

GitHub Revision: `78ad89d1aa`

Branch: os_regression_2024_09_23

Simulator: VCS

Build randomization enabled with --build-seed 34048022127553017884926631616394166155118623175048314192737094530054579848544

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	keymgr_smoke	36.040s	27.324ms	50	50	100.00
V1	random	keymgr_random	1.177m	6.636ms	49	50	98.00
V1	csr_hw_reset	keymgr_csr_hw_reset	1.830s	64.832us	5	5	100.00
V1	csr_rw	keymgr_csr_rw	2.150s	24.901us	20	20	100.00
V1	csr_bit_bash	keymgr_csr_bit_bash	26.560s	6.139ms	5	5	100.00
V1	csr_aliasing	keymgr_csr_aliasing	15.970s	3.065ms	5	5	100.00
V1	csr_mem_rw_with_rand_reset	keymgr_csr_mem_rw_with_rand_reset	3.970s	43.429us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	keymgr_csr_rw	2.150s	24.901us	20	20	100.00
		keymgr_csr_aliasing	15.970s	3.065ms	5	5	100.00
V1		TOTAL			154	155	99.35
V2	cfgen_during_op	keymgr_cfg_regwen	2.080m	7.959ms	50	50	100.00
V2	sideload	keymgr_sideload	50.210s	1.750ms	50	50	100.00
		keymgr_sideload_kmac	49.100s	24.891ms	49	50	98.00
		keymgr_sideload_aes	54.520s	5.236ms	50	50	100.00
		keymgr_sideload_otbn	1.221m	10.358ms	50	50	100.00
V2	direct_to_disabled_state	keymgr_direct_to_disabled	21.210s	2.259ms	50	50	100.00
V2	lc_disable	keymgr_lc_disable	8.610s	1.864ms	49	50	98.00
V2	kmac_error_response	keymgr_kmac_rsp_err	13.220s	376.486us	49	50	98.00
V2	invalid_sw_input	keymgr_sw_invalid_input	31.580s	2.879ms	48	50	96.00
V2	invalid_hw_input	keymgr_hwsw_invalid_input	55.740s	11.026ms	49	50	98.00
V2	sync_async_fault_cross	keymgr_sync_async_fault_cross	20.130s	2.411ms	49	50	98.00
V2	stress_all	keymgr_stress_all	7.813m	76.724ms	50	50	100.00
V2	intr_test	keymgr_intr_test	1.390s	14.486us	50	50	100.00
V2	alert_test	keymgr_alert_test	1.630s	26.345us	50	50	100.00
V2	tl_d_oob_addr_access	keymgr_tl_errors	5.770s	1.137ms	20	20	100.00
V2	tl_d_illegal_access	keymgr_tl_errors	5.770s	1.137ms	20	20	100.00
V2	tl_d_outstanding_access	keymgr_csr_hw_reset	1.830s	64.832us	5	5	100.00
		keymgr_csr_rw	2.150s	24.901us	20	20	100.00
		keymgr_csr_aliasing	15.970s	3.065ms	5	5	100.00
		keymgr_same_csr_outstanding	6.490s	200.595us	20	20	100.00
V2	tl_d_partial_access	keymgr_csr_hw_reset	1.830s	64.832us	5	5	100.00
		keymgr_csr_rw	2.150s	24.901us	20	20	100.00
		keymgr_csr_aliasing	15.970s	3.065ms	5	5	100.00
		keymgr_same_csr_outstanding	6.490s	200.595us	20	20	100.00
V2		TOTAL			733	740	99.05
V2S	sec_cm_additional_check	keymgr_sec_cm	22.910s	814.616us	5	5	100.00
V2S	tl_intg_err	keymgr_sec_cm	22.910s	814.616us	5	5	100.00
		keymgr_tl_intg_err	10.450s	261.767us	20	20	100.00
V2S	shadow_reg_update_error	keymgr_shadow_reg_errors	5.500s	316.249us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	keymgr_shadow_reg_errors	5.500s	316.249us	20	20	100.00
V2S	shadow_reg_storage_error	keymgr_shadow_reg_errors	5.500s	316.249us	20	20	100.00
V2S	shadowed_reset_glitch	keymgr_shadow_reg_errors	5.500s	316.249us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	keymgr_shadow_reg_errors_with_csr_rw	11.140s	1.482ms	20	20	100.00
V2S	prim_count_check	keymgr_sec_cm	22.910s	814.616us	5	5	100.00
V2S	prim_fsm_check	keymgr_sec_cm	22.910s	814.616us	5	5	100.00
V2S	sec_cm_bus_integrity	keymgr_tl_intg_err	10.450s	261.767us	20	20	100.00
V2S	sec_cm_config_shadow	keymgr_shadow_reg_errors	5.500s	316.249us	20	20	100.00
V2S	sec_cm_op_config_regwen	keymgr_cfg_regwen	2.080m	7.959ms	50	50	100.00
V2S	sec_cm_reseed_config_regwen	keymgr_random	1.177m	6.636ms	49	50	98.00
		keymgr_csr_rw	2.150s	24.901us	20	20	100.00
V2S	sec_cm_sw_binding_config_regwen	keymgr_random	1.177m	6.636ms	49	50	98.00
		keymgr_csr_rw	2.150s	24.901us	20	20	100.00
V2S	sec_cm_max_key_ver_config_regwen	keymgr_random	1.177m	6.636ms	49	50	98.00
		keymgr_csr_rw	2.150s	24.901us	20	20	100.00
V2S	sec_cm_lc_ctrl_intersig_mubi	keymgr_lc_disable	8.610s	1.864ms	49	50	98.00
V2S	sec_cm_constants_consistency	keymgr_hwsw_invalid_input	55.740s	11.026ms	49	50	98.00
V2S	sec_cm_intersig_consistency	keymgr_hwsw_invalid_input	55.740s	11.026ms	49	50	98.00
V2S	sec_cm_hw_key_sw_noaccess	keymgr_random	1.177m	6.636ms	49	50	98.00
V2S	sec_cm_output_keys_ctrl_redun	keymgr_sideload_protect	39.690s	3.923ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_sparse	keymgr_sec_cm	22.910s	814.616us	5	5	100.00
V2S	sec_cm_data_fsm_sparse	keymgr_sec_cm	22.910s	814.616us	5	5	100.00
V2S	sec_cm_ctrl_fsm_local_esc	keymgr_sec_cm	22.910s	814.616us	5	5	100.00
V2S	sec_cm_ctrl_fsm_consistency	keymgr_custom_cm	13.790s	702.887us	49	50	98.00
V2S	sec_cm_ctrl_fsm_global_esc	keymgr_lc_disable	8.610s	1.864ms	49	50	98.00
V2S	sec_cm_ctrl_ctr_redun	keymgr_sec_cm	22.910s	814.616us	5	5	100.00
V2S	sec_cm_kmac_if_fsm_sparse	keymgr_sec_cm	22.910s	814.616us	5	5	100.00
V2S	sec_cm_kmac_if_ctr_redun	keymgr_sec_cm	22.910s	814.616us	5	5	100.00
V2S	sec_cm_kmac_if_cmd_ctrl_consistency	keymgr_custom_cm	13.790s	702.887us	49	50	98.00
V2S	sec_cm_kmac_if_done_ctrl_consistency	keymgr_custom_cm	13.790s	702.887us	49	50	98.00
V2S	sec_cm_reseed_ctr_redun	keymgr_sec_cm	22.910s	814.616us	5	5	100.00
V2S	sec_cm_side_load_sel_ctrl_consistency	keymgr_custom_cm	13.790s	702.887us	49	50	98.00
V2S	sec_cm_sideload_ctrl_fsm_sparse	keymgr_sec_cm	22.910s	814.616us	5	5	100.00
V2S	sec_cm_ctrl_key_integrity	keymgr_custom_cm	13.790s	702.887us	49	50	98.00
V2S		TOTAL			164	165	99.39
V3	stress_all_with_rand_reset	keymgr_stress_all_with_rand_reset	32.300s	1.044ms	25	50	50.00
V3		TOTAL			25	50	50.00
		TOTAL			1076	1110	96.94

Testplan Progress

Items	Total	Written	Passing	Progress
V1	7	7	6	85.71
V2	16	16	10	62.50
V2S	6	6	5	83.33
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
97.43	98.99	98.07	98.43	97.67	98.92	98.63	91.27

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:867) [keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 22 failures:
- Test keymgr_stress_all_with_rand_reset has 22 failures.
  - 1.keymgr_stress_all_with_rand_reset.110313212687025549029070022312104435086350032134312062660506160088479972507247
    Line 138, in log /workspaces/repo/scratch/os_regression_2024_09_23/keymgr-sim-vcs/1.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 118601322 ps: (cip_base_vseq.sv:867) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 118601322 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 2.keymgr_stress_all_with_rand_reset.21248337656203178347778642775437321294558242553489688497830750172088802995777
    Line 164, in log /workspaces/repo/scratch/os_regression_2024_09_23/keymgr-sim-vcs/2.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 126644946 ps: (cip_base_vseq.sv:867) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10002 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 126644946 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 20 more failures.

UVM_ERROR (cip_base_scoreboard.sv:304) scoreboard [scoreboard] alert recov_operation_err did not trigger max_delay:* has 7 failures:

Test keymgr_sideload_kmac has 1 failures.

0.keymgr_sideload_kmac.4131622070519321670494710409367362925192261676458352218255623532030658915231
Line 117, in log /workspaces/repo/scratch/os_regression_2024_09_23/keymgr-sim-vcs/0.keymgr_sideload_kmac/latest/run.log

  UVM_ERROR @  74119957 ps: (cip_base_scoreboard.sv:304) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @  74119957 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test keymgr_sw_invalid_input has 2 failures.

10.keymgr_sw_invalid_input.53877343354890931613761164300472828058344888454026816717988642432885353411988
Line 136, in log /workspaces/repo/scratch/os_regression_2024_09_23/keymgr-sim-vcs/10.keymgr_sw_invalid_input/latest/run.log

  UVM_ERROR @  13069051 ps: (cip_base_scoreboard.sv:304) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @  13069051 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

23.keymgr_sw_invalid_input.13130597447007992017556891818937718866277253741748826197107098907577195415691
Line 476, in log /workspaces/repo/scratch/os_regression_2024_09_23/keymgr-sim-vcs/23.keymgr_sw_invalid_input/latest/run.log

  UVM_ERROR @  30413508 ps: (cip_base_scoreboard.sv:304) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @  30413508 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test keymgr_custom_cm has 1 failures.

14.keymgr_custom_cm.23162107937920438171206037301911528953671898878490027953859993950027782883687
Line 184, in log /workspaces/repo/scratch/os_regression_2024_09_23/keymgr-sim-vcs/14.keymgr_custom_cm/latest/run.log

  UVM_ERROR @  97434151 ps: (cip_base_scoreboard.sv:304) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @  97434151 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test keymgr_random has 1 failures.

24.keymgr_random.16615182583324649390734987816219809941435151081923833531004796598549719128270
Line 479, in log /workspaces/repo/scratch/os_regression_2024_09_23/keymgr-sim-vcs/24.keymgr_random/latest/run.log

  UVM_ERROR @ 100857665 ps: (cip_base_scoreboard.sv:304) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @ 100857665 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test keymgr_lc_disable has 1 failures.

41.keymgr_lc_disable.4410565661752012354280960059939626457239737260873134337798932846182243612
Line 75, in log /workspaces/repo/scratch/os_regression_2024_09_23/keymgr-sim-vcs/41.keymgr_lc_disable/latest/run.log

  UVM_ERROR @  44240246 ps: (cip_base_scoreboard.sv:304) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:2000
  UVM_INFO @  44240246 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 1 more tests.

UVM_ERROR (cip_base_vseq.sv:771) [keymgr_common_vseq] Check failed (cfg.can_reset_with_csr_accesses || !has_outstanding_access()) Trying to trigger a reset with outstanding CSR items. has 2 failures:
- Test keymgr_stress_all_with_rand_reset has 2 failures.
  - 35.keymgr_stress_all_with_rand_reset.75668479831507569991377723902370601647367662164331456447095106354585213723207
    Line 129, in log /workspaces/repo/scratch/os_regression_2024_09_23/keymgr-sim-vcs/35.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 211756066 ps: (cip_base_vseq.sv:771) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (cfg.can_reset_with_csr_accesses || !has_outstanding_access()) Trying to trigger a reset with outstanding CSR items.
  UVM_INFO @ 211756066 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 47.keymgr_stress_all_with_rand_reset.71016976015166050404447592499368416276791874642225534228573304272657833439994
    Line 557, in log /workspaces/repo/scratch/os_regression_2024_09_23/keymgr-sim-vcs/47.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 764041503 ps: (cip_base_vseq.sv:771) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (cfg.can_reset_with_csr_accesses || !has_outstanding_access()) Trying to trigger a reset with outstanding CSR items.
  UVM_INFO @ 764041503 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (cip_base_scoreboard.sv:262) scoreboard [scoreboard] alert recov_operation_err is not received! has 1 failures:
- Test keymgr_sync_async_fault_cross has 1 failures.
  - 22.keymgr_sync_async_fault_cross.105131219856416560085569984159170924690222329741058517090013758120839291585011
    Line 168, in log /workspaces/repo/scratch/os_regression_2024_09_23/keymgr-sim-vcs/22.keymgr_sync_async_fault_cross/latest/run.log
```
  UVM_ERROR @ 221862684 ps: (cip_base_scoreboard.sv:262) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err is not received!
  UVM_INFO @ 221862684 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (keymgr_scoreboard.sv:692) [scoreboard] Check failed item.d_data == addr_phase_op_status (* [*] vs * [*]) has 1 failures:
- Test keymgr_stress_all_with_rand_reset has 1 failures.
  - 25.keymgr_stress_all_with_rand_reset.101958862025164952574245435869782724576286252983313093983133039308383690065502
    Line 599, in log /workspaces/repo/scratch/os_regression_2024_09_23/keymgr-sim-vcs/25.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 474575515 ps: (keymgr_scoreboard.sv:692) [uvm_test_top.env.scoreboard] Check failed item.d_data == addr_phase_op_status (1 [0x1] vs 2 [0x2])
  UVM_INFO @ 474575515 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (cip_base_scoreboard.sv:242) [scoreboard] Check failed expected_alert[alert_name].expected == * (* [*] vs * [*]) alert recov_operation_err triggered unexpectedly has 1 failures:
- Test keymgr_kmac_rsp_err has 1 failures.
  - 49.keymgr_kmac_rsp_err.65495663430183342889721674706058545947357790931303847177410391342522790986980
    Line 420, in log /workspaces/repo/scratch/os_regression_2024_09_23/keymgr-sim-vcs/49.keymgr_kmac_rsp_err/latest/run.log
```
  UVM_ERROR @ 150933293 ps: (cip_base_scoreboard.sv:242) [uvm_test_top.env.scoreboard] Check failed expected_alert[alert_name].expected == 1 (0 [0x0] vs 1 [0x1]) alert recov_operation_err triggered unexpectedly
  UVM_INFO @ 150933293 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```