KEYMGR Simulation Results

Sunday April 21 2024 19:02:51 UTC

GitHub Revision: `4fd94db59a`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 89274329416853274976097168471417145417282051311181377329444669936981619711436

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	keymgr_smoke	29.060s	1.517ms	50	50	100.00
V1	random	keymgr_random	1.431m	12.777ms	50	50	100.00
V1	csr_hw_reset	keymgr_csr_hw_reset	1.390s	41.530us	5	5	100.00
V1	csr_rw	keymgr_csr_rw	1.860s	31.910us	20	20	100.00
V1	csr_bit_bash	keymgr_csr_bit_bash	18.730s	1.333ms	5	5	100.00
V1	csr_aliasing	keymgr_csr_aliasing	14.180s	1.447ms	5	5	100.00
V1	csr_mem_rw_with_rand_reset	keymgr_csr_mem_rw_with_rand_reset	2.190s	109.500us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	keymgr_csr_rw	1.860s	31.910us	20	20	100.00
		keymgr_csr_aliasing	14.180s	1.447ms	5	5	100.00
V1		TOTAL			155	155	100.00
V2	cfgen_during_op	keymgr_cfg_regwen	2.177m	4.866ms	48	50	96.00
V2	sideload	keymgr_sideload	42.710s	1.666ms	50	50	100.00
		keymgr_sideload_kmac	1.185m	6.202ms	50	50	100.00
		keymgr_sideload_aes	1.037m	10.549ms	50	50	100.00
		keymgr_sideload_otbn	53.900s	4.668ms	50	50	100.00
V2	direct_to_disabled_state	keymgr_direct_to_disabled	22.540s	849.520us	50	50	100.00
V2	lc_disable	keymgr_lc_disable	25.340s	2.092ms	48	50	96.00
V2	kmac_error_response	keymgr_kmac_rsp_err	1.370m	2.953ms	47	50	94.00
V2	invalid_sw_input	keymgr_sw_invalid_input	1.114m	4.129ms	50	50	100.00
V2	invalid_hw_input	keymgr_hwsw_invalid_input	42.840s	1.419ms	50	50	100.00
V2	sync_async_fault_cross	keymgr_sync_async_fault_cross	18.180s	2.040ms	48	50	96.00
V2	stress_all	keymgr_stress_all	6.651m	16.778ms	50	50	100.00
V2	intr_test	keymgr_intr_test	1.010s	178.603us	50	50	100.00
V2	alert_test	keymgr_alert_test	1.040s	20.968us	50	50	100.00
V2	tl_d_oob_addr_access	keymgr_tl_errors	5.590s	354.027us	20	20	100.00
V2	tl_d_illegal_access	keymgr_tl_errors	5.590s	354.027us	20	20	100.00
V2	tl_d_outstanding_access	keymgr_csr_hw_reset	1.390s	41.530us	5	5	100.00
		keymgr_csr_rw	1.860s	31.910us	20	20	100.00
		keymgr_csr_aliasing	14.180s	1.447ms	5	5	100.00
		keymgr_same_csr_outstanding	3.870s	207.092us	20	20	100.00
V2	tl_d_partial_access	keymgr_csr_hw_reset	1.390s	41.530us	5	5	100.00
		keymgr_csr_rw	1.860s	31.910us	20	20	100.00
		keymgr_csr_aliasing	14.180s	1.447ms	5	5	100.00
		keymgr_same_csr_outstanding	3.870s	207.092us	20	20	100.00
V2		TOTAL			731	740	98.78
V2S	sec_cm_additional_check	keymgr_sec_cm	8.234m	95.302ms	5	5	100.00
V2S	tl_intg_err	keymgr_sec_cm	8.234m	95.302ms	5	5	100.00
		keymgr_tl_intg_err	36.920s	1.405ms	20	20	100.00
V2S	shadow_reg_update_error	keymgr_shadow_reg_errors	37.530s	1.353ms	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	keymgr_shadow_reg_errors	37.530s	1.353ms	20	20	100.00
V2S	shadow_reg_storage_error	keymgr_shadow_reg_errors	37.530s	1.353ms	20	20	100.00
V2S	shadowed_reset_glitch	keymgr_shadow_reg_errors	37.530s	1.353ms	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	keymgr_shadow_reg_errors_with_csr_rw	15.210s	466.806us	20	20	100.00
V2S	prim_count_check	keymgr_sec_cm	8.234m	95.302ms	5	5	100.00
V2S	prim_fsm_check	keymgr_sec_cm	8.234m	95.302ms	5	5	100.00
V2S	sec_cm_bus_integrity	keymgr_tl_intg_err	36.920s	1.405ms	20	20	100.00
V2S	sec_cm_config_shadow	keymgr_shadow_reg_errors	37.530s	1.353ms	20	20	100.00
V2S	sec_cm_op_config_regwen	keymgr_cfg_regwen	2.177m	4.866ms	48	50	96.00
V2S	sec_cm_reseed_config_regwen	keymgr_random	1.431m	12.777ms	50	50	100.00
		keymgr_csr_rw	1.860s	31.910us	20	20	100.00
V2S	sec_cm_sw_binding_config_regwen	keymgr_random	1.431m	12.777ms	50	50	100.00
		keymgr_csr_rw	1.860s	31.910us	20	20	100.00
V2S	sec_cm_max_key_ver_config_regwen	keymgr_random	1.431m	12.777ms	50	50	100.00
		keymgr_csr_rw	1.860s	31.910us	20	20	100.00
V2S	sec_cm_lc_ctrl_intersig_mubi	keymgr_lc_disable	25.340s	2.092ms	48	50	96.00
V2S	sec_cm_constants_consistency	keymgr_hwsw_invalid_input	42.840s	1.419ms	50	50	100.00
V2S	sec_cm_intersig_consistency	keymgr_hwsw_invalid_input	42.840s	1.419ms	50	50	100.00
V2S	sec_cm_hw_key_sw_noaccess	keymgr_random	1.431m	12.777ms	50	50	100.00
V2S	sec_cm_output_keys_ctrl_redun	keymgr_sideload_protect	11.560s	733.145us	50	50	100.00
V2S	sec_cm_ctrl_fsm_sparse	keymgr_sec_cm	8.234m	95.302ms	5	5	100.00
V2S	sec_cm_data_fsm_sparse	keymgr_sec_cm	8.234m	95.302ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_local_esc	keymgr_sec_cm	8.234m	95.302ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_consistency	keymgr_custom_cm	1.624m	8.185ms	49	50	98.00
V2S	sec_cm_ctrl_fsm_global_esc	keymgr_lc_disable	25.340s	2.092ms	48	50	96.00
V2S	sec_cm_ctrl_ctr_redun	keymgr_sec_cm	8.234m	95.302ms	5	5	100.00
V2S	sec_cm_kmac_if_fsm_sparse	keymgr_sec_cm	8.234m	95.302ms	5	5	100.00
V2S	sec_cm_kmac_if_ctr_redun	keymgr_sec_cm	8.234m	95.302ms	5	5	100.00
V2S	sec_cm_kmac_if_cmd_ctrl_consistency	keymgr_custom_cm	1.624m	8.185ms	49	50	98.00
V2S	sec_cm_kmac_if_done_ctrl_consistency	keymgr_custom_cm	1.624m	8.185ms	49	50	98.00
V2S	sec_cm_reseed_ctr_redun	keymgr_sec_cm	8.234m	95.302ms	5	5	100.00
V2S	sec_cm_side_load_sel_ctrl_consistency	keymgr_custom_cm	1.624m	8.185ms	49	50	98.00
V2S	sec_cm_sideload_ctrl_fsm_sparse	keymgr_sec_cm	8.234m	95.302ms	5	5	100.00
V2S	sec_cm_ctrl_key_integrity	keymgr_custom_cm	1.624m	8.185ms	49	50	98.00
V2S		TOTAL			164	165	99.39
V3	stress_all_with_rand_reset	keymgr_stress_all_with_rand_reset	22.910s	366.955us	25	50	50.00
V3		TOTAL			25	50	50.00
		TOTAL			1075	1110	96.85

Testplan Progress

Items	Total	Written	Passing	Progress
V1	7	7	7	100.00
V2	16	16	12	75.00
V2S	6	6	5	83.33
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
97.82	99.07	98.06	98.34	100.00	99.19	98.41	91.63

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:829) [keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 18 failures:
- Test keymgr_stress_all_with_rand_reset has 18 failures.
  - 0.keymgr_stress_all_with_rand_reset.1058122756828038937732091180207850776290242339128626420708962825477786847204
    Line 608, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/0.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 216081245 ps: (cip_base_vseq.sv:829) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 216081245 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 5.keymgr_stress_all_with_rand_reset.62923170309325615367374106555426230639959248045579918519565069511861899010601
    Line 1720, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/5.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 1991634440 ps: (cip_base_vseq.sv:829) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 1991634440 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 16 more failures.
UVM_ERROR (keymgr_scoreboard.sv:736) [scoreboard] Check failed item.d_data !=gmv(csr) (* [] vs * [])` has 6 failures:
- Test keymgr_stress_all_with_rand_reset has 6 failures.
  - 2.keymgr_stress_all_with_rand_reset.98041184234514169907019883539309491890728813846709346037703571567149570986636
    Line 1145, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/2.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 281453892 ps: (keymgr_scoreboard.sv:736) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (758486523 [0x2d3595fb] vs 758486523 [0x2d3595fb])
  UVM_INFO @ 281453892 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 3.keymgr_stress_all_with_rand_reset.53137903484340232616172384605910766707036028001162166604856998018182840040849
    Line 748, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/3.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 191095042 ps: (keymgr_scoreboard.sv:736) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (2116661181 [0x7e29afbd] vs 2116661181 [0x7e29afbd])
  UVM_INFO @ 191095042 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 4 more failures.
UVM_ERROR (cip_base_scoreboard.sv:301) scoreboard [scoreboard] alert recov_operation_err did not trigger max_delay:* has 3 failures:
- Test keymgr_stress_all_with_rand_reset has 1 failures.
  - 1.keymgr_stress_all_with_rand_reset.16886532094045477201583594904211417171336982999850348892182750189938618521099
    Line 892, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/1.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 356409402 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @ 356409402 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test keymgr_cfg_regwen has 1 failures.
  - 23.keymgr_cfg_regwen.28551589782552010044044666620545408633534219306926412124761759751327644799872
    Line 441, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/23.keymgr_cfg_regwen/latest/run.log
```
  UVM_ERROR @  50273042 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @  50273042 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test keymgr_custom_cm has 1 failures.
  - 32.keymgr_custom_cm.23951028082923090935379931597914660660235708013646741420082824285424140104968
    Line 395, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/32.keymgr_custom_cm/latest/run.log
```
  UVM_ERROR @  44160637 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @  44160637 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```

UVM_ERROR (keymgr_scoreboard.sv:1013) [scoreboard] Check failed act != exp (* [*] vs * [*]) cdi_type: Attestation has 3 failures:

Test keymgr_kmac_rsp_err has 3 failures.

2.keymgr_kmac_rsp_err.79919608838259508462076322397918654773541533558961144330427147264912163643933
Line 356, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/2.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @ 1415956193 ps: (keymgr_scoreboard.sv:1013) [uvm_test_top.env.scoreboard] Check failed act != exp (4665859572150931630771659744203522632169813053376688787660720265133442268067421190847703487916464612929154694320030886173569217097413501768931580861902045405691557249376708947055985722467892299120484012737681396265063285220124540442931835272325507350789629609001491726118270746835706576552531076460549100185743372147265739174054902250425848182274534340368562059978070752101423314670445536510248770227733757531104924831806743 [0xa8a19153a8f756360e9ca3b91bc5d4efd077ad56b1454707eb1d4f664a23fd863a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9b8c21a80342e5370505595aa70a85aff9d4bedb0dee00add3d6c826ec9940cd4554168eb3eac17eb6dff9453aa67b3fcc08622f301cf237e0b9f57a177813937b7b74f7e95d566eb5ced91e481e5a37a354a5f5aca5c1f184fe4efe1c826f1a2edce7992fa72c082d67352ab7ce37517] vs 4665859572150931630771659744203522632169813053376688787660720265133442268067421190847703487916464612929154694320030886173569217097413501768931580861902045405691557249376708947055985722467892299120484012737681396265063285220124540442931835272325507350789629609001491726118270746835706576552531076460549100185743372147265739174054902250425848182274534340368562059978070752101423314670445536510248770227733757531104924831806743 [0xa8a19153a8f756360e9ca3b91bc5d4efd077ad56b1454707eb1d4f664a23fd863a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9b8c21a80342e5370505595aa70a85aff9d4bedb0dee00add3d6c826ec9940cd4554168eb3eac17eb6dff9453aa67b3fcc08622f301cf237e0b9f57a177813937b7b74f7e95d566eb5ced91e481e5a37a354a5f5aca5c1f184fe4efe1c826f1a2edce7992fa72c082d67352ab7ce37517]) cdi_type: Attestation
  
   DiversificationKey act: 0x354a5f5aca5c1f184fe4efe1c826f1a2edce7992fa72c082d67352ab7ce37517, exp: 0x354a5f5aca5c1f184fe4efe1c826f1a2edce7992fa72c082d67352ab7ce37517
   RomDigest act: 0xc08622f301cf237e0b9f57a177813937b7b74f7e95d566eb5ced91e481e5a37a, exp: 0xc08622f301cf237e0b9f57a177813937b7b74f7e95d566eb5ced91e481e5a37a
   HealthMeasurement act: 0x554168eb3eac17eb6dff9453aa67b3fc, exp: 0x554168eb3eac17eb6dff9453aa67b3fc

18.keymgr_kmac_rsp_err.39326771988039948613019391508589835348301338975985334604026870162813596167382
Line 294, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/18.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @  76358114 ps: (keymgr_scoreboard.sv:1013) [uvm_test_top.env.scoreboard] Check failed act != exp (13869088684237589750254652523805745455607483863801036890679746459828093165538113979218865451845315999915601255944331604618858335982329456980894376916824293590700126751403838709822420310810302570251120658732306395456962314126943086099407444312161607754144976332268382936715797963817192548307170754942864691749133858461861744818613842781861437825310 [0x3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9ed39c87eaf03731faaf6cca1d20669396e66cabaf8bf8d4d704959ba0d43b3eeec8222a780c9fe06d8fc2fae09420827d99d278c5ccdc14d2ec4c1bf1a06dd30de751f100435ffbe907f7392ee2794d2ac03aae0411bcfbbc0dd27705d178b1ce12333c87d12ea778dd779b1dfd8e11e] vs 13869088684237589750254652523805745455607483863801036890679746459828093165538113979218865451845315999915601255944331604618858335982329456980894376916824293590700126751403838709822420310810302570251120658732306395456962314126943086099407444312161607754144976332268382936715797963817192548307170754942864691749133858461861744818613842781861437825310 [0x3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9ed39c87eaf03731faaf6cca1d20669396e66cabaf8bf8d4d704959ba0d43b3eeec8222a780c9fe06d8fc2fae09420827d99d278c5ccdc14d2ec4c1bf1a06dd30de751f100435ffbe907f7392ee2794d2ac03aae0411bcfbbc0dd27705d178b1ce12333c87d12ea778dd779b1dfd8e11e]) cdi_type: Attestation
  
   DiversificationKey act: 0xac03aae0411bcfbbc0dd27705d178b1ce12333c87d12ea778dd779b1dfd8e11e, exp: 0xac03aae0411bcfbbc0dd27705d178b1ce12333c87d12ea778dd779b1dfd8e11e
   RomDigest act: 0xd99d278c5ccdc14d2ec4c1bf1a06dd30de751f100435ffbe907f7392ee2794d2, exp: 0xd99d278c5ccdc14d2ec4c1bf1a06dd30de751f100435ffbe907f7392ee2794d2
   HealthMeasurement act: 0xec8222a780c9fe06d8fc2fae09420827, exp: 0xec8222a780c9fe06d8fc2fae09420827

... and 1 more failures.

UVM_ERROR (cip_base_scoreboard.sv:261) scoreboard [scoreboard] alert recov_operation_err is not received! has 2 failures:
- Test keymgr_sync_async_fault_cross has 2 failures.
  - 42.keymgr_sync_async_fault_cross.9422741232180896379086327865863154618906452411424897750732581790007062797686
    Line 347, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/42.keymgr_sync_async_fault_cross/latest/run.log
```
  UVM_ERROR @ 247277358 ps: (cip_base_scoreboard.sv:261) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err is not received!
  UVM_INFO @ 247277358 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 49.keymgr_sync_async_fault_cross.72188574318094698519071875539538528197675620215235171491553742123898692815520
    Line 317, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/49.keymgr_sync_async_fault_cross/latest/run.log
```
  UVM_ERROR @ 246673353 ps: (cip_base_scoreboard.sv:261) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err is not received!
  UVM_INFO @ 246673353 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (keymgr_scoreboard.sv:671) [scoreboard] Check failed edn_fifos[*].used() == * (* [*] vs * [*]) has 1 failures:
- Test keymgr_lc_disable has 1 failures.
  - 12.keymgr_lc_disable.65105667161457153106174957784272016824324068704918073110018747691999945799733
    Line 380, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/12.keymgr_lc_disable/latest/run.log
```
  UVM_ERROR @  51677527 ps: (keymgr_scoreboard.sv:671) [uvm_test_top.env.scoreboard] Check failed edn_fifos[0].used() == 2 (0 [0x0] vs 2 [0x2])
  UVM_INFO @  51677527 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (keymgr_scoreboard.sv:750) [scoreboard] Check failed item.d_data ==gmv(csr) (* [] vs * []) reg name: keymgr_reg_block.sw_share1_output_*` has 1 failures:
- Test keymgr_lc_disable has 1 failures.
  - 37.keymgr_lc_disable.63135877254149591015826783660913449851281321569967696158534496690197115016320
    Line 550, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/37.keymgr_lc_disable/latest/run.log
```
  UVM_ERROR @  25540950 ps: (keymgr_scoreboard.sv:750) [uvm_test_top.env.scoreboard] Check failed item.d_data == `gmv(csr) (954113711 [0x38de9eaf] vs 2459216586 [0x9294aaca]) reg name: keymgr_reg_block.sw_share1_output_1
  UVM_INFO @  25540950 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (keymgr_scoreboard.sv:750) [scoreboard] Check failed item.d_data ==gmv(csr) (* [] vs * []) reg name: keymgr_reg_block.start` has 1 failures:
- Test keymgr_cfg_regwen has 1 failures.
  - 49.keymgr_cfg_regwen.46871434709786970795974762369278231588755781205533276259244703344502741999647
    Line 316, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/49.keymgr_cfg_regwen/latest/run.log
```
  UVM_ERROR @   4340285 ps: (keymgr_scoreboard.sv:750) [uvm_test_top.env.scoreboard] Check failed item.d_data == `gmv(csr) (0 [0x0] vs 1 [0x1]) reg name: keymgr_reg_block.start
  UVM_INFO @   4340285 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```