KEYMGR Simulation Results

Sunday March 10 2024 19:02:34 UTC

GitHub Revision: `8d1fda3660`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 55344925760588090643748974780216117977546302496149780891974223483299136808506

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	keymgr_smoke	43.220s	9.165ms	50	50	100.00
V1	random	keymgr_random	57.740s	2.240ms	50	50	100.00
V1	csr_hw_reset	keymgr_csr_hw_reset	1.370s	17.932us	5	5	100.00
V1	csr_rw	keymgr_csr_rw	1.570s	27.518us	20	20	100.00
V1	csr_bit_bash	keymgr_csr_bit_bash	31.040s	5.337ms	5	5	100.00
V1	csr_aliasing	keymgr_csr_aliasing	17.900s	2.040ms	5	5	100.00
V1	csr_mem_rw_with_rand_reset	keymgr_csr_mem_rw_with_rand_reset	2.440s	51.744us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	keymgr_csr_rw	1.570s	27.518us	20	20	100.00
		keymgr_csr_aliasing	17.900s	2.040ms	5	5	100.00
V1		TOTAL			155	155	100.00
V2	cfgen_during_op	keymgr_cfg_regwen	2.840m	3.078ms	49	50	98.00
V2	sideload	keymgr_sideload	40.970s	1.780ms	50	50	100.00
		keymgr_sideload_kmac	1.312m	13.761ms	50	50	100.00
		keymgr_sideload_aes	59.800s	7.943ms	50	50	100.00
		keymgr_sideload_otbn	59.810s	4.498ms	50	50	100.00
V2	direct_to_disabled_state	keymgr_direct_to_disabled	51.120s	5.188ms	50	50	100.00
V2	lc_disable	keymgr_lc_disable	8.060s	152.257us	50	50	100.00
V2	kmac_error_response	keymgr_kmac_rsp_err	1.339m	18.303ms	42	50	84.00
V2	invalid_sw_input	keymgr_sw_invalid_input	1.603m	9.457ms	50	50	100.00
V2	invalid_hw_input	keymgr_hwsw_invalid_input	52.790s	1.664ms	50	50	100.00
V2	sync_async_fault_cross	keymgr_sync_async_fault_cross	41.100s	7.193ms	49	50	98.00
V2	stress_all	keymgr_stress_all	6.649m	60.634ms	49	50	98.00
V2	intr_test	keymgr_intr_test	0.940s	16.292us	50	50	100.00
V2	alert_test	keymgr_alert_test	1.030s	43.247us	50	50	100.00
V2	tl_d_oob_addr_access	keymgr_tl_errors	4.420s	247.431us	20	20	100.00
V2	tl_d_illegal_access	keymgr_tl_errors	4.420s	247.431us	20	20	100.00
V2	tl_d_outstanding_access	keymgr_csr_hw_reset	1.370s	17.932us	5	5	100.00
		keymgr_csr_rw	1.570s	27.518us	20	20	100.00
		keymgr_csr_aliasing	17.900s	2.040ms	5	5	100.00
		keymgr_same_csr_outstanding	4.510s	455.147us	20	20	100.00
V2	tl_d_partial_access	keymgr_csr_hw_reset	1.370s	17.932us	5	5	100.00
		keymgr_csr_rw	1.570s	27.518us	20	20	100.00
		keymgr_csr_aliasing	17.900s	2.040ms	5	5	100.00
		keymgr_same_csr_outstanding	4.510s	455.147us	20	20	100.00
V2		TOTAL			729	740	98.51
V2S	sec_cm_additional_check	keymgr_sec_cm	2.178m	12.127ms	5	5	100.00
V2S	tl_intg_err	keymgr_sec_cm	2.178m	12.127ms	5	5	100.00
		keymgr_tl_intg_err	13.050s	483.991us	20	20	100.00
V2S	shadow_reg_update_error	keymgr_shadow_reg_errors	33.020s	1.051ms	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	keymgr_shadow_reg_errors	33.020s	1.051ms	20	20	100.00
V2S	shadow_reg_storage_error	keymgr_shadow_reg_errors	33.020s	1.051ms	20	20	100.00
V2S	shadowed_reset_glitch	keymgr_shadow_reg_errors	33.020s	1.051ms	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	keymgr_shadow_reg_errors_with_csr_rw	13.390s	1.371ms	20	20	100.00
V2S	prim_count_check	keymgr_sec_cm	2.178m	12.127ms	5	5	100.00
V2S	prim_fsm_check	keymgr_sec_cm	2.178m	12.127ms	5	5	100.00
V2S	sec_cm_bus_integrity	keymgr_tl_intg_err	13.050s	483.991us	20	20	100.00
V2S	sec_cm_config_shadow	keymgr_shadow_reg_errors	33.020s	1.051ms	20	20	100.00
V2S	sec_cm_op_config_regwen	keymgr_cfg_regwen	2.840m	3.078ms	49	50	98.00
V2S	sec_cm_reseed_config_regwen	keymgr_random	57.740s	2.240ms	50	50	100.00
		keymgr_csr_rw	1.570s	27.518us	20	20	100.00
V2S	sec_cm_sw_binding_config_regwen	keymgr_random	57.740s	2.240ms	50	50	100.00
		keymgr_csr_rw	1.570s	27.518us	20	20	100.00
V2S	sec_cm_max_key_ver_config_regwen	keymgr_random	57.740s	2.240ms	50	50	100.00
		keymgr_csr_rw	1.570s	27.518us	20	20	100.00
V2S	sec_cm_lc_ctrl_intersig_mubi	keymgr_lc_disable	8.060s	152.257us	50	50	100.00
V2S	sec_cm_constants_consistency	keymgr_hwsw_invalid_input	52.790s	1.664ms	50	50	100.00
V2S	sec_cm_intersig_consistency	keymgr_hwsw_invalid_input	52.790s	1.664ms	50	50	100.00
V2S	sec_cm_hw_key_sw_noaccess	keymgr_random	57.740s	2.240ms	50	50	100.00
V2S	sec_cm_output_keys_ctrl_redun	keymgr_sideload_protect	29.450s	2.480ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_sparse	keymgr_sec_cm	2.178m	12.127ms	5	5	100.00
V2S	sec_cm_data_fsm_sparse	keymgr_sec_cm	2.178m	12.127ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_local_esc	keymgr_sec_cm	2.178m	12.127ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_consistency	keymgr_custom_cm	53.800s	2.770ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_global_esc	keymgr_lc_disable	8.060s	152.257us	50	50	100.00
V2S	sec_cm_ctrl_ctr_redun	keymgr_sec_cm	2.178m	12.127ms	5	5	100.00
V2S	sec_cm_kmac_if_fsm_sparse	keymgr_sec_cm	2.178m	12.127ms	5	5	100.00
V2S	sec_cm_kmac_if_ctr_redun	keymgr_sec_cm	2.178m	12.127ms	5	5	100.00
V2S	sec_cm_kmac_if_cmd_ctrl_consistency	keymgr_custom_cm	53.800s	2.770ms	50	50	100.00
V2S	sec_cm_kmac_if_done_ctrl_consistency	keymgr_custom_cm	53.800s	2.770ms	50	50	100.00
V2S	sec_cm_reseed_ctr_redun	keymgr_sec_cm	2.178m	12.127ms	5	5	100.00
V2S	sec_cm_side_load_sel_ctrl_consistency	keymgr_custom_cm	53.800s	2.770ms	50	50	100.00
V2S	sec_cm_sideload_ctrl_fsm_sparse	keymgr_sec_cm	2.178m	12.127ms	5	5	100.00
V2S	sec_cm_ctrl_key_integrity	keymgr_custom_cm	53.800s	2.770ms	50	50	100.00
V2S		TOTAL			165	165	100.00
V3	stress_all_with_rand_reset	keymgr_stress_all_with_rand_reset	28.430s	2.356ms	16	50	32.00
V3		TOTAL			16	50	32.00
		TOTAL			1065	1110	95.95

Testplan Progress

Items	Total	Written	Passing	Progress
V1	7	7	7	100.00
V2	16	16	12	75.00
V2S	6	6	6	100.00
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
97.84	99.07	97.99	98.67	100.00	99.11	98.41	91.63

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:827) [keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 26 failures:
- Test keymgr_stress_all_with_rand_reset has 26 failures.
  - 1.keymgr_stress_all_with_rand_reset.59853312909547244139183817401763915371088495460238216926152682193766252329120
    Line 909, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/1.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 1292307079 ps: (cip_base_vseq.sv:827) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 1292307079 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 3.keymgr_stress_all_with_rand_reset.18663909222875950214860926019202595751595083527051704119929677249060492493000
    Line 330, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/3.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 753046255 ps: (cip_base_vseq.sv:827) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 753046255 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 24 more failures.
UVM_ERROR (keymgr_scoreboard.sv:736) [scoreboard] Check failed item.d_data !=gmv(csr) (* [] vs * [])` has 8 failures:
- Test keymgr_stress_all_with_rand_reset has 8 failures.
  - 0.keymgr_stress_all_with_rand_reset.79252384237073927211937015657992797315615997308417063650427446993002930076568
    Line 808, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/0.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 387898840 ps: (keymgr_scoreboard.sv:736) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (4234148003 [0xfc5ff8a3] vs 4234148003 [0xfc5ff8a3])
  UVM_INFO @ 387898840 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 2.keymgr_stress_all_with_rand_reset.113217802470665839476351828908545069948009358297785603094016864569023504486257
    Line 2301, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/2.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 894602335 ps: (keymgr_scoreboard.sv:736) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (658482619 [0x273fa5bb] vs 658482619 [0x273fa5bb])
  UVM_INFO @ 894602335 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 6 more failures.

UVM_ERROR (keymgr_scoreboard.sv:1013) [scoreboard] Check failed act != exp (* [*] vs * [*]) cdi_type: Attestation has 7 failures:

Test keymgr_kmac_rsp_err has 7 failures.

7.keymgr_kmac_rsp_err.23590215546659828201145367732865847420017265487079262292966612836107782766517
Line 403, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/7.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @  97493598 ps: (keymgr_scoreboard.sv:1013) [uvm_test_top.env.scoreboard] Check failed act != exp (23159459280695583989571988166912667679579670698922062287040512011246225530331771797977004307970901423579678808045335438720278487797896555122050287141975379722720681924140785740492938388272220161697732635863021709473429953608130088950151105883185555483776297480198990832462879102088494110714724605347688704534599728908621437405947331614944737572492746490195864290039168550693773704502153636076214 [0x4250bad00000000000000000a5932541000000003a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9cc1ae5d404bd96ae788797e9eab537e0be570bca10e02c8bac7a0f3b7507567a8b3e1c689b09030eb371df978df85c783b70245cb9c52ca286f4b3075d12111a34ff430a1b1f1a5126fdd28687891d17d6fbff02eb585699e710d3e651bd2b3e60b5ee7f797e62a75b262a19e10ddab6] vs 23159459280695583989571988166912667679579670698922062287040512011246225530331771797977004307970901423579678808045335438720278487797896555122050287141975379722720681924140785740492938388272220161697732635863021709473429953608130088950151105883185555483776297480198990832462879102088494110714724605347688704534599728908621437405947331614944737572492746490195864290039168550693773704502153636076214 [0x4250bad00000000000000000a5932541000000003a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9cc1ae5d404bd96ae788797e9eab537e0be570bca10e02c8bac7a0f3b7507567a8b3e1c689b09030eb371df978df85c783b70245cb9c52ca286f4b3075d12111a34ff430a1b1f1a5126fdd28687891d17d6fbff02eb585699e710d3e651bd2b3e60b5ee7f797e62a75b262a19e10ddab6]) cdi_type: Attestation
  
   DiversificationKey act: 0xd6fbff02eb585699e710d3e651bd2b3e60b5ee7f797e62a75b262a19e10ddab6, exp: 0xd6fbff02eb585699e710d3e651bd2b3e60b5ee7f797e62a75b262a19e10ddab6
   RomDigest act: 0x3b70245cb9c52ca286f4b3075d12111a34ff430a1b1f1a5126fdd28687891d17, exp: 0x3b70245cb9c52ca286f4b3075d12111a34ff430a1b1f1a5126fdd28687891d17
   HealthMeasurement act: 0x8b3e1c689b09030eb371df978df85c78, exp: 0x8b3e1c689b09030eb371df978df85c78

12.keymgr_kmac_rsp_err.106700100333218361463427240392390820823157232553939468907738466878229654037287
Line 386, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/12.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @ 589877326 ps: (keymgr_scoreboard.sv:1013) [uvm_test_top.env.scoreboard] Check failed act != exp (6065446236836113038664476185494132449911077384817968268871970110365931597655423510562939308414606241063124308747135244714472537135314266496866391042383094532006677755888595515441681936852718682239137826540061137917382787128349810434346464696866011979459117905087834804058963106123485719148355939342870658531019933210305455688649267282515758704735842047923318399978125480377776214599391059479907630097277639691390908513359372 [0xdb36da9c0000000000000000000000007fd605680000000000000000000000003a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9dd2c4e414bb0f01f0c2318a2f4f8e6fa508fb3c8cfe84524b80f3f972c0c645a3e21789b33661e47aec039ae1c5e966ff63844754a47719f61a023eb292a3f4d2113e082b6589db607b58cb47b7b7e253126f8d7ece8a84ef54c4c1224dcf138cae3d5c04217432d38ebe4aaa59b960c] vs 6065446236836113038664476185494132449911077384817968268871970110365931597655423510562939308414606241063124308747135244714472537135314266496866391042383094532006677755888595515441681936852718682239137826540061137917382787128349810434346464696866011979459117905087834804058963106123485719148355939342870658531019933210305455688649267282515758704735842047923318399978125480377776214599391059479907630097277639691390908513359372 [0xdb36da9c0000000000000000000000007fd605680000000000000000000000003a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9dd2c4e414bb0f01f0c2318a2f4f8e6fa508fb3c8cfe84524b80f3f972c0c645a3e21789b33661e47aec039ae1c5e966ff63844754a47719f61a023eb292a3f4d2113e082b6589db607b58cb47b7b7e253126f8d7ece8a84ef54c4c1224dcf138cae3d5c04217432d38ebe4aaa59b960c]) cdi_type: Attestation
  
   DiversificationKey act: 0x3126f8d7ece8a84ef54c4c1224dcf138cae3d5c04217432d38ebe4aaa59b960c, exp: 0x3126f8d7ece8a84ef54c4c1224dcf138cae3d5c04217432d38ebe4aaa59b960c
   RomDigest act: 0xf63844754a47719f61a023eb292a3f4d2113e082b6589db607b58cb47b7b7e25, exp: 0xf63844754a47719f61a023eb292a3f4d2113e082b6589db607b58cb47b7b7e25
   HealthMeasurement act: 0x3e21789b33661e47aec039ae1c5e966f, exp: 0x3e21789b33661e47aec039ae1c5e966f

... and 5 more failures.

UVM_ERROR (cip_base_scoreboard.sv:287) scoreboard [scoreboard] alert recov_operation_err did not trigger max_delay:* has 3 failures:
- Test keymgr_stress_all has 1 failures.
  - 16.keymgr_stress_all.33872586177924273858350670919342029456783578528616348547638172375539878635813
    Line 1431, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/16.keymgr_stress_all/latest/run.log
```
  UVM_ERROR @ 335461700 ps: (cip_base_scoreboard.sv:287) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @ 335461700 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test keymgr_sync_async_fault_cross has 1 failures.
  - 20.keymgr_sync_async_fault_cross.11805201200299562488700453522739524459825537936833474187522974155878344965653
    Line 273, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/20.keymgr_sync_async_fault_cross/latest/run.log
```
  UVM_ERROR @   5857945 ps: (cip_base_scoreboard.sv:287) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @   5857945 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test keymgr_kmac_rsp_err has 1 failures.
  - 45.keymgr_kmac_rsp_err.101001249974262136177691359853673654696884962730294467414400323282622832967851
    Line 370, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/45.keymgr_kmac_rsp_err/latest/run.log
```
  UVM_ERROR @  11471397 ps: (cip_base_scoreboard.sv:287) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @  11471397 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (keymgr_scoreboard.sv:750) [scoreboard] Check failed item.d_data ==gmv(csr) (* [] vs * []) reg name: keymgr_reg_block.start` has 1 failures:
- Test keymgr_cfg_regwen has 1 failures.
  - 30.keymgr_cfg_regwen.101330020479754170929403282205128929141305526700773931876796001327870540322281
    Line 321, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/30.keymgr_cfg_regwen/latest/run.log
```
  UVM_ERROR @  12999109 ps: (keymgr_scoreboard.sv:750) [uvm_test_top.env.scoreboard] Check failed item.d_data == `gmv(csr) (0 [0x0] vs 1 [0x1]) reg name: keymgr_reg_block.start
  UVM_INFO @  12999109 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```