KEYMGR Simulation Results

Monday October 14 2024 17:26:15 UTC

GitHub Revision: `12e3b8572e`

Branch: os_regression_2024_10_14

Simulator: VCS

Build randomization enabled with --build-seed 85025606402499621082521464627961092918263397067038954055071960501195381950243

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	keymgr_smoke	21.510s	1.112ms	50	50	100.00
V1	random	keymgr_random	1.011m	7.407ms	50	50	100.00
V1	csr_hw_reset	keymgr_csr_hw_reset	1.340s	29.553us	5	5	100.00
V1	csr_rw	keymgr_csr_rw	1.460s	94.404us	20	20	100.00
V1	csr_bit_bash	keymgr_csr_bit_bash	21.290s	877.294us	5	5	100.00
V1	csr_aliasing	keymgr_csr_aliasing	12.510s	1.115ms	5	5	100.00
V1	csr_mem_rw_with_rand_reset	keymgr_csr_mem_rw_with_rand_reset	2.220s	274.357us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	keymgr_csr_rw	1.460s	94.404us	20	20	100.00
		keymgr_csr_aliasing	12.510s	1.115ms	5	5	100.00
V1		TOTAL			155	155	100.00
V2	cfgen_during_op	keymgr_cfg_regwen	1.690m	4.225ms	49	50	98.00
V2	sideload	keymgr_sideload	30.060s	2.762ms	50	50	100.00
		keymgr_sideload_kmac	1.115m	4.382ms	50	50	100.00
		keymgr_sideload_aes	36.360s	5.313ms	50	50	100.00
		keymgr_sideload_otbn	50.480s	6.691ms	50	50	100.00
V2	direct_to_disabled_state	keymgr_direct_to_disabled	49.040s	5.821ms	50	50	100.00
V2	lc_disable	keymgr_lc_disable	13.700s	385.543us	49	50	98.00
V2	kmac_error_response	keymgr_kmac_rsp_err	10.270s	293.678us	50	50	100.00
V2	invalid_sw_input	keymgr_sw_invalid_input	1.234m	8.690ms	50	50	100.00
V2	invalid_hw_input	keymgr_hwsw_invalid_input	51.060s	1.902ms	50	50	100.00
V2	sync_async_fault_cross	keymgr_sync_async_fault_cross	11.010s	528.707us	49	50	98.00
V2	stress_all	keymgr_stress_all	8.379m	30.601ms	48	50	96.00
V2	intr_test	keymgr_intr_test	0.950s	29.926us	50	50	100.00
V2	alert_test	keymgr_alert_test	1.540s	70.000us	50	50	100.00
V2	tl_d_oob_addr_access	keymgr_tl_errors	4.210s	433.694us	20	20	100.00
V2	tl_d_illegal_access	keymgr_tl_errors	4.210s	433.694us	20	20	100.00
V2	tl_d_outstanding_access	keymgr_csr_hw_reset	1.340s	29.553us	5	5	100.00
		keymgr_csr_rw	1.460s	94.404us	20	20	100.00
		keymgr_csr_aliasing	12.510s	1.115ms	5	5	100.00
		keymgr_same_csr_outstanding	3.900s	245.350us	20	20	100.00
V2	tl_d_partial_access	keymgr_csr_hw_reset	1.340s	29.553us	5	5	100.00
		keymgr_csr_rw	1.460s	94.404us	20	20	100.00
		keymgr_csr_aliasing	12.510s	1.115ms	5	5	100.00
		keymgr_same_csr_outstanding	3.900s	245.350us	20	20	100.00
V2		TOTAL			735	740	99.32
V2S	sec_cm_additional_check	keymgr_sec_cm	24.240s	1.014ms	5	5	100.00
V2S	tl_intg_err	keymgr_sec_cm	24.240s	1.014ms	5	5	100.00
		keymgr_tl_intg_err	9.190s	558.523us	20	20	100.00
V2S	shadow_reg_update_error	keymgr_shadow_reg_errors	3.940s	688.686us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	keymgr_shadow_reg_errors	3.940s	688.686us	20	20	100.00
V2S	shadow_reg_storage_error	keymgr_shadow_reg_errors	3.940s	688.686us	20	20	100.00
V2S	shadowed_reset_glitch	keymgr_shadow_reg_errors	3.940s	688.686us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	keymgr_shadow_reg_errors_with_csr_rw	14.920s	548.650us	20	20	100.00
V2S	prim_count_check	keymgr_sec_cm	24.240s	1.014ms	5	5	100.00
V2S	prim_fsm_check	keymgr_sec_cm	24.240s	1.014ms	5	5	100.00
V2S	sec_cm_bus_integrity	keymgr_tl_intg_err	9.190s	558.523us	20	20	100.00
V2S	sec_cm_config_shadow	keymgr_shadow_reg_errors	3.940s	688.686us	20	20	100.00
V2S	sec_cm_op_config_regwen	keymgr_cfg_regwen	1.690m	4.225ms	49	50	98.00
V2S	sec_cm_reseed_config_regwen	keymgr_random	1.011m	7.407ms	50	50	100.00
		keymgr_csr_rw	1.460s	94.404us	20	20	100.00
V2S	sec_cm_sw_binding_config_regwen	keymgr_random	1.011m	7.407ms	50	50	100.00
		keymgr_csr_rw	1.460s	94.404us	20	20	100.00
V2S	sec_cm_max_key_ver_config_regwen	keymgr_random	1.011m	7.407ms	50	50	100.00
		keymgr_csr_rw	1.460s	94.404us	20	20	100.00
V2S	sec_cm_lc_ctrl_intersig_mubi	keymgr_lc_disable	13.700s	385.543us	49	50	98.00
V2S	sec_cm_constants_consistency	keymgr_hwsw_invalid_input	51.060s	1.902ms	50	50	100.00
V2S	sec_cm_intersig_consistency	keymgr_hwsw_invalid_input	51.060s	1.902ms	50	50	100.00
V2S	sec_cm_hw_key_sw_noaccess	keymgr_random	1.011m	7.407ms	50	50	100.00
V2S	sec_cm_output_keys_ctrl_redun	keymgr_sideload_protect	36.980s	4.148ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_sparse	keymgr_sec_cm	24.240s	1.014ms	5	5	100.00
V2S	sec_cm_data_fsm_sparse	keymgr_sec_cm	24.240s	1.014ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_local_esc	keymgr_sec_cm	24.240s	1.014ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_consistency	keymgr_custom_cm	35.660s	3.649ms	49	50	98.00
V2S	sec_cm_ctrl_fsm_global_esc	keymgr_lc_disable	13.700s	385.543us	49	50	98.00
V2S	sec_cm_ctrl_ctr_redun	keymgr_sec_cm	24.240s	1.014ms	5	5	100.00
V2S	sec_cm_kmac_if_fsm_sparse	keymgr_sec_cm	24.240s	1.014ms	5	5	100.00
V2S	sec_cm_kmac_if_ctr_redun	keymgr_sec_cm	24.240s	1.014ms	5	5	100.00
V2S	sec_cm_kmac_if_cmd_ctrl_consistency	keymgr_custom_cm	35.660s	3.649ms	49	50	98.00
V2S	sec_cm_kmac_if_done_ctrl_consistency	keymgr_custom_cm	35.660s	3.649ms	49	50	98.00
V2S	sec_cm_reseed_ctr_redun	keymgr_sec_cm	24.240s	1.014ms	5	5	100.00
V2S	sec_cm_side_load_sel_ctrl_consistency	keymgr_custom_cm	35.660s	3.649ms	49	50	98.00
V2S	sec_cm_sideload_ctrl_fsm_sparse	keymgr_sec_cm	24.240s	1.014ms	5	5	100.00
V2S	sec_cm_ctrl_key_integrity	keymgr_custom_cm	35.660s	3.649ms	49	50	98.00
V2S		TOTAL			164	165	99.39
V3	stress_all_with_rand_reset	keymgr_stress_all_with_rand_reset	38.330s	5.464ms	26	50	52.00
V3		TOTAL			26	50	52.00
		TOTAL			1080	1110	97.30

Testplan Progress

Items	Total	Written	Passing	Progress
V1	7	7	7	100.00
V2	16	16	12	75.00
V2S	6	6	5	83.33
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
97.76	99.03	98.11	98.40	100.00	99.01	98.63	91.14

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:867) [keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 20 failures:
- Test keymgr_stress_all_with_rand_reset has 20 failures.
  - 0.keymgr_stress_all_with_rand_reset.21226254987736935236300376934825580159608152247460122665299332980659461221678
    Line 1174, in log /workspaces/repo/scratch/os_regression_2024_10_14/keymgr-sim-vcs/0.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 382864044 ps: (cip_base_vseq.sv:867) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 382864044 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 2.keymgr_stress_all_with_rand_reset.99184806155715960284053853099631175786880241739205626043286055932400366548831
    Line 219, in log /workspaces/repo/scratch/os_regression_2024_10_14/keymgr-sim-vcs/2.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 1475031032 ps: (cip_base_vseq.sv:867) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 1475031032 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 18 more failures.

UVM_ERROR (cip_base_scoreboard.sv:304) scoreboard [scoreboard] alert recov_operation_err did not trigger max_delay:* has 4 failures:

Test keymgr_stress_all has 2 failures.

1.keymgr_stress_all.111703309467996553579243181515506666853167652017050477850070391702539735406682
Line 709, in log /workspaces/repo/scratch/os_regression_2024_10_14/keymgr-sim-vcs/1.keymgr_stress_all/latest/run.log

  UVM_ERROR @ 575340717 ps: (cip_base_scoreboard.sv:304) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @ 575340717 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

43.keymgr_stress_all.110578596487129341196034244823221077709195177639891670180536067735209751688703
Line 4269, in log /workspaces/repo/scratch/os_regression_2024_10_14/keymgr-sim-vcs/43.keymgr_stress_all/latest/run.log

  UVM_ERROR @ 1239279848 ps: (cip_base_scoreboard.sv:304) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @ 1239279848 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test keymgr_sync_async_fault_cross has 1 failures.
- 11.keymgr_sync_async_fault_cross.4045838455617703509659121392349700460826333472612833982522824987139188222219
  Line 81, in log /workspaces/repo/scratch/os_regression_2024_10_14/keymgr-sim-vcs/11.keymgr_sync_async_fault_cross/latest/run.log
```
  UVM_ERROR @   9109522 ps: (cip_base_scoreboard.sv:304) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @   9109522 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```

Test keymgr_custom_cm has 1 failures.

35.keymgr_custom_cm.72930122408523696374841069293699834660270343660153357779293422582421910189063
Line 138, in log /workspaces/repo/scratch/os_regression_2024_10_14/keymgr-sim-vcs/35.keymgr_custom_cm/latest/run.log

  UVM_ERROR @   7917779 ps: (cip_base_scoreboard.sv:304) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @   7917779 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_ERROR (cip_base_vseq.sv:771) [keymgr_common_vseq] Check failed (cfg.can_reset_with_csr_accesses || !has_outstanding_access()) Trying to trigger a reset with outstanding CSR items. has 2 failures:
- Test keymgr_stress_all_with_rand_reset has 2 failures.
  - 35.keymgr_stress_all_with_rand_reset.91150824000159030258203634977578408347052655318468189389050887714513555308764
    Line 588, in log /workspaces/repo/scratch/os_regression_2024_10_14/keymgr-sim-vcs/35.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 983403110 ps: (cip_base_vseq.sv:771) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (cfg.can_reset_with_csr_accesses || !has_outstanding_access()) Trying to trigger a reset with outstanding CSR items.
  UVM_INFO @ 983403110 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 49.keymgr_stress_all_with_rand_reset.54424057701254343738179922984497179175101740115598458259783433903728356440864
    Line 310, in log /workspaces/repo/scratch/os_regression_2024_10_14/keymgr-sim-vcs/49.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 159164425 ps: (cip_base_vseq.sv:771) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (cfg.can_reset_with_csr_accesses || !has_outstanding_access()) Trying to trigger a reset with outstanding CSR items.
  UVM_INFO @ 159164425 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```

UVM_ERROR (keymgr_scoreboard.sv:1064) [scoreboard] Check failed act == exp (* [*] vs * [*]) cdi_type: Attestation has 1 failures:

Test keymgr_stress_all_with_rand_reset has 1 failures.

15.keymgr_stress_all_with_rand_reset.84814641655109639143603200056818680300494528541733165812279522474150869491
Line 949, in log /workspaces/repo/scratch/os_regression_2024_10_14/keymgr-sim-vcs/15.keymgr_stress_all_with_rand_reset/latest/run.log

  UVM_ERROR @ 352646280 ps: (keymgr_scoreboard.sv:1064) [uvm_test_top.env.scoreboard] Check failed act == exp (557446023491622274898390731887398682439183419953224550101116816157204008246021365772783112356040351387432981858007335801399625835484825451193015587636099481479239732035196029788509303437989181378799520312784074360586609201145358831152136436150921049119561223364831030939840608091295489248388995073526349036785968001785 [0xb8d3ecb400000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000f0f000000000000000005cfbd765ce33f34ea20a046cf42e6eac560a3f82bfa76285b5c1d4aea7c915e49a32d1c89be0f5073a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9] vs 39102934483625847309710412667911521144103784599974043089806252297659997698042382523164882992268468712919090153955312971988394941778369487024521624705800184787732517859806461912257990710432030364500363545506453951948611766781960387539617583653393550223857018996337458372247196931715888708458608133926629683074322007858651580322843868852010625446649 [0xa3a44e8e00000000ecd8a7b00000000027decffde4520c98b97e366200000000000000000000000000000000000000000000000000000000000000000000f0f000000000000000005cfbd765ce33f34ea20a046cf42e6eac560a3f82bfa76285b5c1d4aea7c915e49a32d1c89be0f5073a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9]) cdi_type: Attestation
  
   HardwareRevisionSecret act: 0x3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9, exp: 0x3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9
   RomDigest act: 0xa20a046cf42e6eac560a3f82bfa76285b5c1d4aea7c915e49a32d1c89be0f507, exp: 0xa20a046cf42e6eac560a3f82bfa76285b5c1d4aea7c915e49a32d1c89be0f507
   HealthMeasurement act: 0x5cfbd765ce33f34e, exp: 0x5cfbd765ce33f34e

UVM_ERROR (keymgr_scoreboard.sv:692) [scoreboard] Check failed item.d_data == addr_phase_op_status (* [*] vs * [*]) has 1 failures:
- Test keymgr_stress_all_with_rand_reset has 1 failures.
  - 28.keymgr_stress_all_with_rand_reset.113106086430338455995212538717968423476317662508593238635867946768311787868144
    Line 183, in log /workspaces/repo/scratch/os_regression_2024_10_14/keymgr-sim-vcs/28.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 163556902 ps: (keymgr_scoreboard.sv:692) [uvm_test_top.env.scoreboard] Check failed item.d_data == addr_phase_op_status (1 [0x1] vs 2 [0x2])
  UVM_INFO @ 163556902 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (keymgr_scoreboard.sv:794) [scoreboard] Check failed item.d_data ==gmv(csr) (* [] vs * []) reg name: keymgr_reg_block.start` has 1 failures:
- Test keymgr_cfg_regwen has 1 failures.
  - 38.keymgr_cfg_regwen.54254483114636735401194353785322752143812731072964054884789672737957473822421
    Line 74, in log /workspaces/repo/scratch/os_regression_2024_10_14/keymgr-sim-vcs/38.keymgr_cfg_regwen/latest/run.log
```
  UVM_ERROR @   5012790 ps: (keymgr_scoreboard.sv:794) [uvm_test_top.env.scoreboard] Check failed item.d_data == `gmv(csr) (0 [0x0] vs 1 [0x1]) reg name: keymgr_reg_block.start
  UVM_INFO @   5012790 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (keymgr_scoreboard.sv:651) [scoreboard] Check failed act_state == addr_phase_working_state (* [*] vs * [*]) has 1 failures:
- Test keymgr_lc_disable has 1 failures.
  - 49.keymgr_lc_disable.22313220536116424786044849073226231675887156286822583013941428062164818975332
    Line 173, in log /workspaces/repo/scratch/os_regression_2024_10_14/keymgr-sim-vcs/49.keymgr_lc_disable/latest/run.log
```
  UVM_ERROR @  26846200 ps: (keymgr_scoreboard.sv:651) [uvm_test_top.env.scoreboard] Check failed act_state == addr_phase_working_state (1 [0x1] vs 6 [0x6])
  UVM_INFO @  26846200 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```