KEYMGR Simulation Results

Friday August 02 2024 23:02:48 UTC

GitHub Revision: `c8985d6745`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 75989420798843487383163268541581889763599806834398027919895759109584083292465

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	keymgr_smoke	27.370s	10.061ms	50	50	100.00
V1	random	keymgr_random	43.720s	4.727ms	50	50	100.00
V1	csr_hw_reset	keymgr_csr_hw_reset	1.410s	215.746us	5	5	100.00
V1	csr_rw	keymgr_csr_rw	1.580s	36.741us	20	20	100.00
V1	csr_bit_bash	keymgr_csr_bit_bash	15.770s	2.342ms	5	5	100.00
V1	csr_aliasing	keymgr_csr_aliasing	10.240s	2.565ms	5	5	100.00
V1	csr_mem_rw_with_rand_reset	keymgr_csr_mem_rw_with_rand_reset	2.640s	62.917us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	keymgr_csr_rw	1.580s	36.741us	20	20	100.00
		keymgr_csr_aliasing	10.240s	2.565ms	5	5	100.00
V1		TOTAL			155	155	100.00
V2	cfgen_during_op	keymgr_cfg_regwen	46.390s	3.213ms	49	50	98.00
V2	sideload	keymgr_sideload	44.720s	3.456ms	49	50	98.00
		keymgr_sideload_kmac	24.970s	4.041ms	49	50	98.00
		keymgr_sideload_aes	51.820s	3.949ms	50	50	100.00
		keymgr_sideload_otbn	1.004m	6.415ms	50	50	100.00
V2	direct_to_disabled_state	keymgr_direct_to_disabled	21.010s	4.034ms	50	50	100.00
V2	lc_disable	keymgr_lc_disable	5.340s	578.061us	48	50	96.00
V2	kmac_error_response	keymgr_kmac_rsp_err	6.140s	272.095us	50	50	100.00
V2	invalid_sw_input	keymgr_sw_invalid_input	1.480m	8.325ms	50	50	100.00
V2	invalid_hw_input	keymgr_hwsw_invalid_input	1.243m	4.612ms	50	50	100.00
V2	sync_async_fault_cross	keymgr_sync_async_fault_cross	41.870s	6.859ms	49	50	98.00
V2	stress_all	keymgr_stress_all	5.137m	168.367ms	49	50	98.00
V2	intr_test	keymgr_intr_test	0.950s	27.282us	50	50	100.00
V2	alert_test	keymgr_alert_test	0.970s	13.389us	50	50	100.00
V2	tl_d_oob_addr_access	keymgr_tl_errors	3.810s	504.592us	20	20	100.00
V2	tl_d_illegal_access	keymgr_tl_errors	3.810s	504.592us	20	20	100.00
V2	tl_d_outstanding_access	keymgr_csr_hw_reset	1.410s	215.746us	5	5	100.00
		keymgr_csr_rw	1.580s	36.741us	20	20	100.00
		keymgr_csr_aliasing	10.240s	2.565ms	5	5	100.00
		keymgr_same_csr_outstanding	3.560s	220.444us	20	20	100.00
V2	tl_d_partial_access	keymgr_csr_hw_reset	1.410s	215.746us	5	5	100.00
		keymgr_csr_rw	1.580s	36.741us	20	20	100.00
		keymgr_csr_aliasing	10.240s	2.565ms	5	5	100.00
		keymgr_same_csr_outstanding	3.560s	220.444us	20	20	100.00
V2		TOTAL			733	740	99.05
V2S	sec_cm_additional_check	keymgr_sec_cm	16.340s	2.794ms	5	5	100.00
V2S	tl_intg_err	keymgr_sec_cm	16.340s	2.794ms	5	5	100.00
		keymgr_tl_intg_err	9.700s	482.920us	20	20	100.00
V2S	shadow_reg_update_error	keymgr_shadow_reg_errors	4.720s	168.131us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	keymgr_shadow_reg_errors	4.720s	168.131us	20	20	100.00
V2S	shadow_reg_storage_error	keymgr_shadow_reg_errors	4.720s	168.131us	20	20	100.00
V2S	shadowed_reset_glitch	keymgr_shadow_reg_errors	4.720s	168.131us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	keymgr_shadow_reg_errors_with_csr_rw	15.220s	783.108us	20	20	100.00
V2S	prim_count_check	keymgr_sec_cm	16.340s	2.794ms	5	5	100.00
V2S	prim_fsm_check	keymgr_sec_cm	16.340s	2.794ms	5	5	100.00
V2S	sec_cm_bus_integrity	keymgr_tl_intg_err	9.700s	482.920us	20	20	100.00
V2S	sec_cm_config_shadow	keymgr_shadow_reg_errors	4.720s	168.131us	20	20	100.00
V2S	sec_cm_op_config_regwen	keymgr_cfg_regwen	46.390s	3.213ms	49	50	98.00
V2S	sec_cm_reseed_config_regwen	keymgr_random	43.720s	4.727ms	50	50	100.00
		keymgr_csr_rw	1.580s	36.741us	20	20	100.00
V2S	sec_cm_sw_binding_config_regwen	keymgr_random	43.720s	4.727ms	50	50	100.00
		keymgr_csr_rw	1.580s	36.741us	20	20	100.00
V2S	sec_cm_max_key_ver_config_regwen	keymgr_random	43.720s	4.727ms	50	50	100.00
		keymgr_csr_rw	1.580s	36.741us	20	20	100.00
V2S	sec_cm_lc_ctrl_intersig_mubi	keymgr_lc_disable	5.340s	578.061us	48	50	96.00
V2S	sec_cm_constants_consistency	keymgr_hwsw_invalid_input	1.243m	4.612ms	50	50	100.00
V2S	sec_cm_intersig_consistency	keymgr_hwsw_invalid_input	1.243m	4.612ms	50	50	100.00
V2S	sec_cm_hw_key_sw_noaccess	keymgr_random	43.720s	4.727ms	50	50	100.00
V2S	sec_cm_output_keys_ctrl_redun	keymgr_sideload_protect	25.520s	1.066ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_sparse	keymgr_sec_cm	16.340s	2.794ms	5	5	100.00
V2S	sec_cm_data_fsm_sparse	keymgr_sec_cm	16.340s	2.794ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_local_esc	keymgr_sec_cm	16.340s	2.794ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_consistency	keymgr_custom_cm	56.530s	4.966ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_global_esc	keymgr_lc_disable	5.340s	578.061us	48	50	96.00
V2S	sec_cm_ctrl_ctr_redun	keymgr_sec_cm	16.340s	2.794ms	5	5	100.00
V2S	sec_cm_kmac_if_fsm_sparse	keymgr_sec_cm	16.340s	2.794ms	5	5	100.00
V2S	sec_cm_kmac_if_ctr_redun	keymgr_sec_cm	16.340s	2.794ms	5	5	100.00
V2S	sec_cm_kmac_if_cmd_ctrl_consistency	keymgr_custom_cm	56.530s	4.966ms	50	50	100.00
V2S	sec_cm_kmac_if_done_ctrl_consistency	keymgr_custom_cm	56.530s	4.966ms	50	50	100.00
V2S	sec_cm_reseed_ctr_redun	keymgr_sec_cm	16.340s	2.794ms	5	5	100.00
V2S	sec_cm_side_load_sel_ctrl_consistency	keymgr_custom_cm	56.530s	4.966ms	50	50	100.00
V2S	sec_cm_sideload_ctrl_fsm_sparse	keymgr_sec_cm	16.340s	2.794ms	5	5	100.00
V2S	sec_cm_ctrl_key_integrity	keymgr_custom_cm	56.530s	4.966ms	50	50	100.00
V2S		TOTAL			165	165	100.00
V3	stress_all_with_rand_reset	keymgr_stress_all_with_rand_reset	27.540s	3.786ms	25	50	50.00
V3		TOTAL			25	50	50.00
		TOTAL			1078	1110	97.12

Testplan Progress

Items	Total	Written	Passing	Progress
V1	7	7	7	100.00
V2	16	16	10	62.50
V2S	6	6	6	100.00
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
97.73	99.04	98.11	98.42	100.00	99.02	98.41	91.14

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:836) [keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 23 failures:
- Test keymgr_stress_all_with_rand_reset has 23 failures.
  - 1.keymgr_stress_all_with_rand_reset.80404655869371587859493481508432075118078842089424124569441924990370469421641
    Line 355, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/1.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 482962103 ps: (cip_base_vseq.sv:836) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 482962103 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 2.keymgr_stress_all_with_rand_reset.15741548054733079238984236950994333827705708867221122614173027186140054605407
    Line 655, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/2.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 4033677591 ps: (cip_base_vseq.sv:836) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 4033677591 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 21 more failures.
UVM_ERROR (cip_base_scoreboard.sv:301) scoreboard [scoreboard] alert recov_operation_err did not trigger max_delay:* has 4 failures:
- Test keymgr_sideload_kmac has 1 failures.
  - 17.keymgr_sideload_kmac.77541078074925599558717623052337248535989581100295737423988680404820080965900
    Line 283, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/17.keymgr_sideload_kmac/latest/run.log
```
  UVM_ERROR @   9064437 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @   9064437 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test keymgr_cfg_regwen has 1 failures.
  - 17.keymgr_cfg_regwen.86542295698506414516941444908479507812868835370802420156876668532650558871160
    Line 312, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/17.keymgr_cfg_regwen/latest/run.log
```
  UVM_ERROR @  19914520 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @  19914520 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test keymgr_sideload has 1 failures.
  - 21.keymgr_sideload.58069153872966316039090349549073917022261589616531325389029359423107463624714
    Line 278, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/21.keymgr_sideload/latest/run.log
```
  UVM_ERROR @   5733583 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @   5733583 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test keymgr_sync_async_fault_cross has 1 failures.
  - 40.keymgr_sync_async_fault_cross.45359209489212135742563378783540743021344596500398614223384557261333542793484
    Line 263, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/40.keymgr_sync_async_fault_cross/latest/run.log
```
  UVM_ERROR @   3088971 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @   3088971 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```

UVM_ERROR (keymgr_scoreboard.sv:766) [scoreboard] Check failed item.d_data !=gmv(csr) (* [] vs * []) reg name: keymgr_reg_block.sw_share1_output_*` has 2 failures:

Test keymgr_stress_all has 1 failures.

40.keymgr_stress_all.5049528295352626828208221897286829848769363286446382740021651035377492053932
Line 407, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/40.keymgr_stress_all/latest/run.log

  UVM_ERROR @  65922700 ps: (keymgr_scoreboard.sv:766) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (3341347154 [0xc728e952] vs 3341347154 [0xc728e952]) reg name: keymgr_reg_block.sw_share1_output_7
  UVM_INFO @  65922700 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test keymgr_lc_disable has 1 failures.

46.keymgr_lc_disable.107345084903207516810000168571529838148219199962335219094198771840956440979834
Line 528, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/46.keymgr_lc_disable/latest/run.log

  UVM_ERROR @ 120870549 ps: (keymgr_scoreboard.sv:766) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (3519541256 [0xd1c7f008] vs 3519541256 [0xd1c7f008]) reg name: keymgr_reg_block.sw_share1_output_6
  UVM_INFO @ 120870549 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_ERROR (keymgr_if.sv:557) [keymgr_if] Check failed {act_key.key[*], act_key.key[*]} !== keys_a_array[state][cdi][dest] (* [*] vs * [*]) AES key at state StCreatorRootKey for Attestation Aes has 1 failures:

Test keymgr_lc_disable has 1 failures.

0.keymgr_lc_disable.88925619277109699413117722431275539656592226015758652358284530795194130400542
Line 418, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/0.keymgr_lc_disable/latest/run.log

  UVM_ERROR @ 134119103 ps: (keymgr_if.sv:557) [keymgr_if] Check failed {act_key.key[1], act_key.key[0]} !== keys_a_array[state][cdi][dest] (6252470001749377791300002102822550718805154556992070596801434021244816942330554937431151960695675680585912394764344012935795130577687296468148970011992107 [0x77616fcefb924ef0adf87f7de24a2fd8224e9819d4b54708f438880afbbde9d8c7dc466f951dca59cbb72f1ad8b4d5dbac218868ea20a4797ed983fc6f87a02b] vs 6252470001749377791300002102822550718805154556992070596801434021244816942330554937431151960695675680585912394764344012935795130577687296468148970011992107 [0x77616fcefb924ef0adf87f7de24a2fd8224e9819d4b54708f438880afbbde9d8c7dc466f951dca59cbb72f1ad8b4d5dbac218868ea20a4797ed983fc6f87a02b]) AES key at state StCreatorRootKey for Attestation Aes
  UVM_INFO @ 134119103 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_ERROR (cip_base_vseq.sv:755) [keymgr_common_vseq] Check failed (!has_outstanding_access()) Trying to trigger a reset with outstanding CSR items. has 1 failures:
- Test keymgr_stress_all_with_rand_reset has 1 failures.
  - 31.keymgr_stress_all_with_rand_reset.101985402226737521361104879486431035822677107509665565721628751567536830306313
    Line 724, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/31.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 1220508916 ps: (cip_base_vseq.sv:755) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Trying to trigger a reset with outstanding CSR items.
  UVM_INFO @ 1220508916 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (keymgr_scoreboard.sv:766) [scoreboard] Check failed item.d_data !=gmv(csr) (* [] vs * []) reg name: keymgr_reg_block.sw_share0_output_*` has 1 failures:
- Test keymgr_stress_all_with_rand_reset has 1 failures.
  - 47.keymgr_stress_all_with_rand_reset.98960389466899933929147609277453646235327487860040324207132934514825662027436
    Line 482, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/47.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 556701287 ps: (keymgr_scoreboard.sv:766) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (0 [0x0] vs 0 [0x0]) reg name: keymgr_reg_block.sw_share0_output_5
  UVM_INFO @ 556701287 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```