KEYMGR Simulation Results

Thursday May 02 2024 19:03:09 UTC

GitHub Revision: `ecd9f08747`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 19770536698299155636913061839112149222426010608929753156399703507865583879800

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	keymgr_smoke	1.196m	7.712ms	50	50	100.00
V1	random	keymgr_random	2.119m	19.017ms	50	50	100.00
V1	csr_hw_reset	keymgr_csr_hw_reset	1.210s	212.226us	5	5	100.00
V1	csr_rw	keymgr_csr_rw	1.650s	160.026us	20	20	100.00
V1	csr_bit_bash	keymgr_csr_bit_bash	17.500s	3.165ms	5	5	100.00
V1	csr_aliasing	keymgr_csr_aliasing	11.280s	472.657us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	keymgr_csr_mem_rw_with_rand_reset	2.310s	203.691us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	keymgr_csr_rw	1.650s	160.026us	20	20	100.00
		keymgr_csr_aliasing	11.280s	472.657us	5	5	100.00
V1		TOTAL			155	155	100.00
V2	cfgen_during_op	keymgr_cfg_regwen	1.298m	3.100ms	48	50	96.00
V2	sideload	keymgr_sideload	1.052m	3.921ms	49	50	98.00
		keymgr_sideload_kmac	1.068m	7.134ms	50	50	100.00
		keymgr_sideload_aes	40.330s	5.671ms	50	50	100.00
		keymgr_sideload_otbn	52.070s	18.472ms	50	50	100.00
V2	direct_to_disabled_state	keymgr_direct_to_disabled	41.740s	2.236ms	50	50	100.00
V2	lc_disable	keymgr_lc_disable	29.930s	989.090us	50	50	100.00
V2	kmac_error_response	keymgr_kmac_rsp_err	1.253m	4.631ms	42	50	84.00
V2	invalid_sw_input	keymgr_sw_invalid_input	1.052m	2.794ms	50	50	100.00
V2	invalid_hw_input	keymgr_hwsw_invalid_input	1.151m	7.556ms	50	50	100.00
V2	sync_async_fault_cross	keymgr_sync_async_fault_cross	13.330s	2.233ms	50	50	100.00
V2	stress_all	keymgr_stress_all	4.426m	11.166ms	48	50	96.00
V2	intr_test	keymgr_intr_test	0.880s	13.947us	50	50	100.00
V2	alert_test	keymgr_alert_test	1.090s	53.344us	50	50	100.00
V2	tl_d_oob_addr_access	keymgr_tl_errors	6.150s	1.109ms	20	20	100.00
V2	tl_d_illegal_access	keymgr_tl_errors	6.150s	1.109ms	20	20	100.00
V2	tl_d_outstanding_access	keymgr_csr_hw_reset	1.210s	212.226us	5	5	100.00
		keymgr_csr_rw	1.650s	160.026us	20	20	100.00
		keymgr_csr_aliasing	11.280s	472.657us	5	5	100.00
		keymgr_same_csr_outstanding	4.570s	240.620us	20	20	100.00
V2	tl_d_partial_access	keymgr_csr_hw_reset	1.210s	212.226us	5	5	100.00
		keymgr_csr_rw	1.650s	160.026us	20	20	100.00
		keymgr_csr_aliasing	11.280s	472.657us	5	5	100.00
		keymgr_same_csr_outstanding	4.570s	240.620us	20	20	100.00
V2		TOTAL			727	740	98.24
V2S	sec_cm_additional_check	keymgr_sec_cm	30.820s	5.063ms	5	5	100.00
V2S	tl_intg_err	keymgr_sec_cm	30.820s	5.063ms	5	5	100.00
		keymgr_tl_intg_err	47.180s	1.848ms	20	20	100.00
V2S	shadow_reg_update_error	keymgr_shadow_reg_errors	10.250s	2.387ms	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	keymgr_shadow_reg_errors	10.250s	2.387ms	20	20	100.00
V2S	shadow_reg_storage_error	keymgr_shadow_reg_errors	10.250s	2.387ms	20	20	100.00
V2S	shadowed_reset_glitch	keymgr_shadow_reg_errors	10.250s	2.387ms	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	keymgr_shadow_reg_errors_with_csr_rw	16.250s	551.547us	20	20	100.00
V2S	prim_count_check	keymgr_sec_cm	30.820s	5.063ms	5	5	100.00
V2S	prim_fsm_check	keymgr_sec_cm	30.820s	5.063ms	5	5	100.00
V2S	sec_cm_bus_integrity	keymgr_tl_intg_err	47.180s	1.848ms	20	20	100.00
V2S	sec_cm_config_shadow	keymgr_shadow_reg_errors	10.250s	2.387ms	20	20	100.00
V2S	sec_cm_op_config_regwen	keymgr_cfg_regwen	1.298m	3.100ms	48	50	96.00
V2S	sec_cm_reseed_config_regwen	keymgr_random	2.119m	19.017ms	50	50	100.00
		keymgr_csr_rw	1.650s	160.026us	20	20	100.00
V2S	sec_cm_sw_binding_config_regwen	keymgr_random	2.119m	19.017ms	50	50	100.00
		keymgr_csr_rw	1.650s	160.026us	20	20	100.00
V2S	sec_cm_max_key_ver_config_regwen	keymgr_random	2.119m	19.017ms	50	50	100.00
		keymgr_csr_rw	1.650s	160.026us	20	20	100.00
V2S	sec_cm_lc_ctrl_intersig_mubi	keymgr_lc_disable	29.930s	989.090us	50	50	100.00
V2S	sec_cm_constants_consistency	keymgr_hwsw_invalid_input	1.151m	7.556ms	50	50	100.00
V2S	sec_cm_intersig_consistency	keymgr_hwsw_invalid_input	1.151m	7.556ms	50	50	100.00
V2S	sec_cm_hw_key_sw_noaccess	keymgr_random	2.119m	19.017ms	50	50	100.00
V2S	sec_cm_output_keys_ctrl_redun	keymgr_sideload_protect	28.450s	4.373ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_sparse	keymgr_sec_cm	30.820s	5.063ms	5	5	100.00
V2S	sec_cm_data_fsm_sparse	keymgr_sec_cm	30.820s	5.063ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_local_esc	keymgr_sec_cm	30.820s	5.063ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_consistency	keymgr_custom_cm	46.330s	1.314ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_global_esc	keymgr_lc_disable	29.930s	989.090us	50	50	100.00
V2S	sec_cm_ctrl_ctr_redun	keymgr_sec_cm	30.820s	5.063ms	5	5	100.00
V2S	sec_cm_kmac_if_fsm_sparse	keymgr_sec_cm	30.820s	5.063ms	5	5	100.00
V2S	sec_cm_kmac_if_ctr_redun	keymgr_sec_cm	30.820s	5.063ms	5	5	100.00
V2S	sec_cm_kmac_if_cmd_ctrl_consistency	keymgr_custom_cm	46.330s	1.314ms	50	50	100.00
V2S	sec_cm_kmac_if_done_ctrl_consistency	keymgr_custom_cm	46.330s	1.314ms	50	50	100.00
V2S	sec_cm_reseed_ctr_redun	keymgr_sec_cm	30.820s	5.063ms	5	5	100.00
V2S	sec_cm_side_load_sel_ctrl_consistency	keymgr_custom_cm	46.330s	1.314ms	50	50	100.00
V2S	sec_cm_sideload_ctrl_fsm_sparse	keymgr_sec_cm	30.820s	5.063ms	5	5	100.00
V2S	sec_cm_ctrl_key_integrity	keymgr_custom_cm	46.330s	1.314ms	50	50	100.00
V2S		TOTAL			165	165	100.00
V3	stress_all_with_rand_reset	keymgr_stress_all_with_rand_reset	26.300s	800.895us	32	50	64.00
V3		TOTAL			32	50	64.00
		TOTAL			1079	1110	97.21

Testplan Progress

Items	Total	Written	Passing	Progress
V1	7	7	7	100.00
V2	16	16	12	75.00
V2S	6	6	6	100.00
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
97.80	99.07	98.03	98.32	100.00	99.11	98.41	91.66

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:829) [keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 11 failures:
- Test keymgr_stress_all_with_rand_reset has 11 failures.
  - 1.keymgr_stress_all_with_rand_reset.108145255514704383535412268350973433610685266456226006920955851681458633697420
    Line 326, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/1.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 116124151 ps: (cip_base_vseq.sv:829) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 116124151 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 9.keymgr_stress_all_with_rand_reset.72051313942545000611082874524336727604645768317665466109379135654078895924718
    Line 316, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/9.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 111907960 ps: (cip_base_vseq.sv:829) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10004 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 111907960 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 9 more failures.

UVM_ERROR (keymgr_scoreboard.sv:1013) [scoreboard] Check failed act != exp (* [*] vs * [*]) cdi_type: Attestation has 8 failures:

Test keymgr_kmac_rsp_err has 8 failures.

0.keymgr_kmac_rsp_err.67335826142762822127939236631030786472186624751675392805931461945415735323760
Line 343, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/0.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @  88981794 ps: (keymgr_scoreboard.sv:1013) [uvm_test_top.env.scoreboard] Check failed act != exp (4062100553003322069712623437001934829935705399780895867972387898951124770294656226139148952738731395675338738102503036156934994871966205975880125230788464527935840017387987119876499306579941547106572467222283612896536467594559294762370357996019409672522951578314980753009518370599327311861133764214581316386739756352051391231662021794948700620601460244283478655092757017956122449156737694809180501134307797707424314955412805 [0x92cf748fd266fbb8aa952aec9b99d52bd3b4ea7c44954e6ed1a54c502c9f53e13a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9418f8216fec4e1a38a01c92aeb638830dbd991221927afb7e39172a586ca667a6b9a1c4e1a775a963a7c48396f99275db50973c9adc76c0765a80fb24be1a0254b8f6e7d2e40149a5d5a0d68030690b61fa116763ad0d22f3220c9c1f6ecbf13588bb7855877a1a343aa7b0800983545] vs 4062100553003322069712623437001934829935705399780895867972387898951124770294656226139148952738731395675338738102503036156934994871966205975880125230788464527935840017387987119876499306579941547106572467222283612896536467594559294762370357996019409672522951578314980753009518370599327311861133764214581316386739756352051391231662021794948700620601460244283478655092757017956122449156737694809180501134307797707424314955412805 [0x92cf748fd266fbb8aa952aec9b99d52bd3b4ea7c44954e6ed1a54c502c9f53e13a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9418f8216fec4e1a38a01c92aeb638830dbd991221927afb7e39172a586ca667a6b9a1c4e1a775a963a7c48396f99275db50973c9adc76c0765a80fb24be1a0254b8f6e7d2e40149a5d5a0d68030690b61fa116763ad0d22f3220c9c1f6ecbf13588bb7855877a1a343aa7b0800983545]) cdi_type: Attestation
  
   DiversificationKey act: 0x1fa116763ad0d22f3220c9c1f6ecbf13588bb7855877a1a343aa7b0800983545, exp: 0x1fa116763ad0d22f3220c9c1f6ecbf13588bb7855877a1a343aa7b0800983545
   RomDigest act: 0xb50973c9adc76c0765a80fb24be1a0254b8f6e7d2e40149a5d5a0d68030690b6, exp: 0xb50973c9adc76c0765a80fb24be1a0254b8f6e7d2e40149a5d5a0d68030690b6
   HealthMeasurement act: 0x6b9a1c4e1a775a963a7c48396f99275d, exp: 0x6b9a1c4e1a775a963a7c48396f99275d

6.keymgr_kmac_rsp_err.12445689401767655566212839841994654612184802407379593435537377821672095843737
Line 627, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/6.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @ 120509234 ps: (keymgr_scoreboard.sv:1013) [uvm_test_top.env.scoreboard] Check failed act != exp (181624644361600479349601643162781292851735935534471292618301431859522664897200468313335075081426152819264400054444935347337480109822362898340284336584167476015794101078675562459272066020061185763700011776778516681160220690438496715776683335234474542928954949588854838996710908321987931238396935139371630263349401098561678038526489326753962097881593091364019283429162142427802746559084129806028963981197219659421294 [0x1c3165f80000000000000000000000000000000000000000000000003a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9a33bd5850b365eb7848f3ada069dfd4408f4fb7d4d00c5250a4467a34d25aafb25b28bd18870ec84c048003d6a1d11c40fe181bdff4b4b5d201ab977e4cec6f9b5b66191dcd400eb1ebdcde1679ced7c472b7648ccde6539845ca0c59ccd22a920b7a915958917cabcde6d4c0912966e] vs 181624644361600479349601643162781292851735935534471292618301431859522664897200468313335075081426152819264400054444935347337480109822362898340284336584167476015794101078675562459272066020061185763700011776778516681160220690438496715776683335234474542928954949588854838996710908321987931238396935139371630263349401098561678038526489326753962097881593091364019283429162142427802746559084129806028963981197219659421294 [0x1c3165f80000000000000000000000000000000000000000000000003a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9a33bd5850b365eb7848f3ada069dfd4408f4fb7d4d00c5250a4467a34d25aafb25b28bd18870ec84c048003d6a1d11c40fe181bdff4b4b5d201ab977e4cec6f9b5b66191dcd400eb1ebdcde1679ced7c472b7648ccde6539845ca0c59ccd22a920b7a915958917cabcde6d4c0912966e]) cdi_type: Attestation
  
   DiversificationKey act: 0x472b7648ccde6539845ca0c59ccd22a920b7a915958917cabcde6d4c0912966e, exp: 0x472b7648ccde6539845ca0c59ccd22a920b7a915958917cabcde6d4c0912966e
   RomDigest act: 0xfe181bdff4b4b5d201ab977e4cec6f9b5b66191dcd400eb1ebdcde1679ced7c, exp: 0xfe181bdff4b4b5d201ab977e4cec6f9b5b66191dcd400eb1ebdcde1679ced7c
   HealthMeasurement act: 0x25b28bd18870ec84c048003d6a1d11c4, exp: 0x25b28bd18870ec84c048003d6a1d11c4

... and 6 more failures.

UVM_ERROR (keymgr_scoreboard.sv:736) [scoreboard] Check failed item.d_data !=gmv(csr) (* [] vs * [])` has 7 failures:
- Test keymgr_stress_all_with_rand_reset has 7 failures.
  - 4.keymgr_stress_all_with_rand_reset.102691973655303506282954115295584603904615829443979012159626285185893642388136
    Line 685, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/4.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 389211193 ps: (keymgr_scoreboard.sv:736) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (3665360867 [0xda78f7e3] vs 3665360867 [0xda78f7e3])
  UVM_INFO @ 389211193 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 8.keymgr_stress_all_with_rand_reset.77166522766000759397921530597551638574084493056791756008249554498226513873310
    Line 898, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/8.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 516374865 ps: (keymgr_scoreboard.sv:736) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (2218439971 [0x843ab523] vs 2218439971 [0x843ab523])
  UVM_INFO @ 516374865 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 5 more failures.
UVM_ERROR (cip_base_scoreboard.sv:301) scoreboard [scoreboard] alert recov_operation_err did not trigger max_delay:* has 2 failures:
- Test keymgr_sideload has 1 failures.
  - 31.keymgr_sideload.22950082987410826380776684036285262842324359421086900085809918455362265195656
    Line 298, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/31.keymgr_sideload/latest/run.log
```
  UVM_ERROR @  18988479 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @  18988479 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test keymgr_cfg_regwen has 1 failures.
  - 38.keymgr_cfg_regwen.24566218585944036383470471630636164741138478334505758698295752981916932304028
    Line 415, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/38.keymgr_cfg_regwen/latest/run.log
```
  UVM_ERROR @ 109499431 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @ 109499431 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (keymgr_scoreboard.sv:750) [scoreboard] Check failed item.d_data ==gmv(csr) (* [] vs * []) reg name: keymgr_reg_block.sw_share1_output_*` has 1 failures:
- Test keymgr_stress_all has 1 failures.
  - 33.keymgr_stress_all.27519104008562994585023648198264951178484625918806141532967223296426861254762
    Line 1864, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/33.keymgr_stress_all/latest/run.log
```
  UVM_ERROR @ 221884238 ps: (keymgr_scoreboard.sv:750) [uvm_test_top.env.scoreboard] Check failed item.d_data == `gmv(csr) (1161922771 [0x454188d3] vs 1237527142 [0x49c32a66]) reg name: keymgr_reg_block.sw_share1_output_7
  UVM_INFO @ 221884238 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (keymgr_scoreboard.sv:750) [scoreboard] Check failed item.d_data ==gmv(csr) (* [] vs * []) reg name: keymgr_reg_block.sw_share0_output_*` has 1 failures:
- Test keymgr_stress_all has 1 failures.
  - 39.keymgr_stress_all.22974480825232777467590143452412621518661578366024888056931781585133204003235
    Line 1282, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/39.keymgr_stress_all/latest/run.log
```
  UVM_ERROR @ 983045052 ps: (keymgr_scoreboard.sv:750) [uvm_test_top.env.scoreboard] Check failed item.d_data == `gmv(csr) (3905687331 [0xe8cc0f23] vs 0 [0x0]) reg name: keymgr_reg_block.sw_share0_output_0
  UVM_INFO @ 983045052 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (keymgr_scoreboard.sv:750) [scoreboard] Check failed item.d_data ==gmv(csr) (* [] vs * []) reg name: keymgr_reg_block.start` has 1 failures:
- Test keymgr_cfg_regwen has 1 failures.
  - 40.keymgr_cfg_regwen.26002469344514554445408666577472905424802798289889762066001463242949145309847
    Line 262, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/40.keymgr_cfg_regwen/latest/run.log
```
  UVM_ERROR @   6366860 ps: (keymgr_scoreboard.sv:750) [uvm_test_top.env.scoreboard] Check failed item.d_data == `gmv(csr) (0 [0x0] vs 1 [0x1]) reg name: keymgr_reg_block.start
  UVM_INFO @   6366860 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```