KEYMGR Simulation Results

Monday April 15 2024 18:56:04 UTC

GitHub Revision: `9f4903e77a`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 40268988864630991006175718979742731758115610160637428218057845043020955930762

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	keymgr_smoke	50.030s	18.302ms	50	50	100.00
V1	random	keymgr_random	49.910s	5.118ms	50	50	100.00
V1	csr_hw_reset	keymgr_csr_hw_reset	1.160s	17.330us	5	5	100.00
V1	csr_rw	keymgr_csr_rw	1.390s	42.768us	20	20	100.00
V1	csr_bit_bash	keymgr_csr_bit_bash	25.220s	907.490us	5	5	100.00
V1	csr_aliasing	keymgr_csr_aliasing	14.520s	366.192us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	keymgr_csr_mem_rw_with_rand_reset	2.390s	38.587us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	keymgr_csr_rw	1.390s	42.768us	20	20	100.00
		keymgr_csr_aliasing	14.520s	366.192us	5	5	100.00
V1		TOTAL			155	155	100.00
V2	cfgen_during_op	keymgr_cfg_regwen	46.150s	880.556us	50	50	100.00
V2	sideload	keymgr_sideload	55.050s	6.870ms	50	50	100.00
		keymgr_sideload_kmac	1.095m	3.953ms	50	50	100.00
		keymgr_sideload_aes	1.093m	2.018ms	50	50	100.00
		keymgr_sideload_otbn	1.190m	2.054ms	49	50	98.00
V2	direct_to_disabled_state	keymgr_direct_to_disabled	29.760s	834.258us	50	50	100.00
V2	lc_disable	keymgr_lc_disable	9.340s	473.481us	50	50	100.00
V2	kmac_error_response	keymgr_kmac_rsp_err	32.530s	4.054ms	45	50	90.00
V2	invalid_sw_input	keymgr_sw_invalid_input	58.540s	12.957ms	50	50	100.00
V2	invalid_hw_input	keymgr_hwsw_invalid_input	1.209m	6.036ms	50	50	100.00
V2	sync_async_fault_cross	keymgr_sync_async_fault_cross	35.760s	5.690ms	49	50	98.00
V2	stress_all	keymgr_stress_all	9.234m	18.553ms	47	50	94.00
V2	intr_test	keymgr_intr_test	1.050s	23.569us	50	50	100.00
V2	alert_test	keymgr_alert_test	1.170s	27.198us	50	50	100.00
V2	tl_d_oob_addr_access	keymgr_tl_errors	3.920s	358.085us	20	20	100.00
V2	tl_d_illegal_access	keymgr_tl_errors	3.920s	358.085us	20	20	100.00
V2	tl_d_outstanding_access	keymgr_csr_hw_reset	1.160s	17.330us	5	5	100.00
		keymgr_csr_rw	1.390s	42.768us	20	20	100.00
		keymgr_csr_aliasing	14.520s	366.192us	5	5	100.00
		keymgr_same_csr_outstanding	4.870s	516.298us	20	20	100.00
V2	tl_d_partial_access	keymgr_csr_hw_reset	1.160s	17.330us	5	5	100.00
		keymgr_csr_rw	1.390s	42.768us	20	20	100.00
		keymgr_csr_aliasing	14.520s	366.192us	5	5	100.00
		keymgr_same_csr_outstanding	4.870s	516.298us	20	20	100.00
V2		TOTAL			730	740	98.65
V2S	sec_cm_additional_check	keymgr_sec_cm	1.150m	7.929ms	5	5	100.00
V2S	tl_intg_err	keymgr_sec_cm	1.150m	7.929ms	5	5	100.00
		keymgr_tl_intg_err	15.770s	2.318ms	20	20	100.00
V2S	shadow_reg_update_error	keymgr_shadow_reg_errors	32.690s	1.697ms	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	keymgr_shadow_reg_errors	32.690s	1.697ms	20	20	100.00
V2S	shadow_reg_storage_error	keymgr_shadow_reg_errors	32.690s	1.697ms	20	20	100.00
V2S	shadowed_reset_glitch	keymgr_shadow_reg_errors	32.690s	1.697ms	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	keymgr_shadow_reg_errors_with_csr_rw	15.300s	812.937us	20	20	100.00
V2S	prim_count_check	keymgr_sec_cm	1.150m	7.929ms	5	5	100.00
V2S	prim_fsm_check	keymgr_sec_cm	1.150m	7.929ms	5	5	100.00
V2S	sec_cm_bus_integrity	keymgr_tl_intg_err	15.770s	2.318ms	20	20	100.00
V2S	sec_cm_config_shadow	keymgr_shadow_reg_errors	32.690s	1.697ms	20	20	100.00
V2S	sec_cm_op_config_regwen	keymgr_cfg_regwen	46.150s	880.556us	50	50	100.00
V2S	sec_cm_reseed_config_regwen	keymgr_random	49.910s	5.118ms	50	50	100.00
		keymgr_csr_rw	1.390s	42.768us	20	20	100.00
V2S	sec_cm_sw_binding_config_regwen	keymgr_random	49.910s	5.118ms	50	50	100.00
		keymgr_csr_rw	1.390s	42.768us	20	20	100.00
V2S	sec_cm_max_key_ver_config_regwen	keymgr_random	49.910s	5.118ms	50	50	100.00
		keymgr_csr_rw	1.390s	42.768us	20	20	100.00
V2S	sec_cm_lc_ctrl_intersig_mubi	keymgr_lc_disable	9.340s	473.481us	50	50	100.00
V2S	sec_cm_constants_consistency	keymgr_hwsw_invalid_input	1.209m	6.036ms	50	50	100.00
V2S	sec_cm_intersig_consistency	keymgr_hwsw_invalid_input	1.209m	6.036ms	50	50	100.00
V2S	sec_cm_hw_key_sw_noaccess	keymgr_random	49.910s	5.118ms	50	50	100.00
V2S	sec_cm_output_keys_ctrl_redun	keymgr_sideload_protect	23.970s	3.304ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_sparse	keymgr_sec_cm	1.150m	7.929ms	5	5	100.00
V2S	sec_cm_data_fsm_sparse	keymgr_sec_cm	1.150m	7.929ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_local_esc	keymgr_sec_cm	1.150m	7.929ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_consistency	keymgr_custom_cm	51.600s	6.792ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_global_esc	keymgr_lc_disable	9.340s	473.481us	50	50	100.00
V2S	sec_cm_ctrl_ctr_redun	keymgr_sec_cm	1.150m	7.929ms	5	5	100.00
V2S	sec_cm_kmac_if_fsm_sparse	keymgr_sec_cm	1.150m	7.929ms	5	5	100.00
V2S	sec_cm_kmac_if_ctr_redun	keymgr_sec_cm	1.150m	7.929ms	5	5	100.00
V2S	sec_cm_kmac_if_cmd_ctrl_consistency	keymgr_custom_cm	51.600s	6.792ms	50	50	100.00
V2S	sec_cm_kmac_if_done_ctrl_consistency	keymgr_custom_cm	51.600s	6.792ms	50	50	100.00
V2S	sec_cm_reseed_ctr_redun	keymgr_sec_cm	1.150m	7.929ms	5	5	100.00
V2S	sec_cm_side_load_sel_ctrl_consistency	keymgr_custom_cm	51.600s	6.792ms	50	50	100.00
V2S	sec_cm_sideload_ctrl_fsm_sparse	keymgr_sec_cm	1.150m	7.929ms	5	5	100.00
V2S	sec_cm_ctrl_key_integrity	keymgr_custom_cm	51.600s	6.792ms	50	50	100.00
V2S		TOTAL			165	165	100.00
V3	stress_all_with_rand_reset	keymgr_stress_all_with_rand_reset	27.660s	734.843us	24	50	48.00
V3		TOTAL			24	50	48.00
		TOTAL			1074	1110	96.76

Testplan Progress

Items	Total	Written	Passing	Progress
V1	7	7	7	100.00
V2	16	16	12	75.00
V2S	6	6	6	100.00
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
97.84	99.07	98.14	98.69	100.00	99.11	98.41	91.46

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:829) [keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 19 failures:
- Test keymgr_stress_all_with_rand_reset has 19 failures.
  - 0.keymgr_stress_all_with_rand_reset.1828561699316915901058313313606290632678831537140429358107430865633844324408
    Line 852, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/0.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 1806735584 ps: (cip_base_vseq.sv:829) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 1806735584 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 2.keymgr_stress_all_with_rand_reset.39907774381259230233966977840551026190083997850572924376280353788550974267996
    Line 511, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/2.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 232677432 ps: (cip_base_vseq.sv:829) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 232677432 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 17 more failures.
UVM_ERROR (keymgr_scoreboard.sv:736) [scoreboard] Check failed item.d_data !=gmv(csr) (* [] vs * [])` has 7 failures:
- Test keymgr_stress_all_with_rand_reset has 7 failures.
  - 1.keymgr_stress_all_with_rand_reset.59261489639351822945999460757110591160170583979111267458684049493266405506282
    Line 1069, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/1.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 249166729 ps: (keymgr_scoreboard.sv:736) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (849997443 [0x32a9ee83] vs 849997443 [0x32a9ee83])
  UVM_INFO @ 249166729 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 11.keymgr_stress_all_with_rand_reset.29102963449439572186276695954972776932787397224482970144703557643805499597526
    Line 936, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/11.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 904956171 ps: (keymgr_scoreboard.sv:736) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (748577660 [0x2c9e637c] vs 748577660 [0x2c9e637c])
  UVM_INFO @ 904956171 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 5 more failures.

UVM_ERROR (keymgr_scoreboard.sv:1013) [scoreboard] Check failed act != exp (* [*] vs * [*]) cdi_type: Attestation has 5 failures:

Test keymgr_kmac_rsp_err has 5 failures.

3.keymgr_kmac_rsp_err.70965065513866486562602314502980725928064594638329226388540838339481079111837
Line 412, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/3.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @  47443852 ps: (keymgr_scoreboard.sv:1013) [uvm_test_top.env.scoreboard] Check failed act != exp (13869088684237589750254652523805745455607483863801036890679746459828093165538107035103374043284313498406587330742322756496017987063972304962207990639218254041081196967570567412350203702437984226041833878972203360840790456939094388674432615048386558953923056674364911009316220240695202116090481255143140931266627930104621982111048617201239108525056 [0x3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9e9dc5a079899c3706c171ce635a1be8979d78f87fe005eecc9345896f6cfdcf11b7ce8e1c7d65bf6f749bdb8286009b1dc6b555b39a00951bf486c3b0a729205571ffb8fc0d980a13fac70dc5904b55bf3a2694b393b3336507016d0c3cfc27909db687093c6a4ab4aa105b10c1e9c00] vs 13869088684237589750254652523805745455607483863801036890679746459828093165538107035103374043284313498406587330742322756496017987063972304962207990639218254041081196967570567412350203702437984226041833878972203360840790456939094388674432615048386558953923056674364911009316220240695202116090481255143140931266627930104621982111048617201239108525056 [0x3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9e9dc5a079899c3706c171ce635a1be8979d78f87fe005eecc9345896f6cfdcf11b7ce8e1c7d65bf6f749bdb8286009b1dc6b555b39a00951bf486c3b0a729205571ffb8fc0d980a13fac70dc5904b55bf3a2694b393b3336507016d0c3cfc27909db687093c6a4ab4aa105b10c1e9c00]) cdi_type: Attestation
  
   DiversificationKey act: 0xf3a2694b393b3336507016d0c3cfc27909db687093c6a4ab4aa105b10c1e9c00, exp: 0xf3a2694b393b3336507016d0c3cfc27909db687093c6a4ab4aa105b10c1e9c00
   RomDigest act: 0xdc6b555b39a00951bf486c3b0a729205571ffb8fc0d980a13fac70dc5904b55b, exp: 0xdc6b555b39a00951bf486c3b0a729205571ffb8fc0d980a13fac70dc5904b55b
   HealthMeasurement act: 0x1b7ce8e1c7d65bf6f749bdb8286009b1, exp: 0x1b7ce8e1c7d65bf6f749bdb8286009b1

4.keymgr_kmac_rsp_err.104949293585618157702855544658729507734182720331689902600596966315035776349657
Line 388, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/4.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @ 396749385 ps: (keymgr_scoreboard.sv:1013) [uvm_test_top.env.scoreboard] Check failed act != exp (20080110197977749145887525945433534553590857077021135947483750375558624533740383411697985922862800000214461825206671816550639463983227828187299786323083660473729186679786877701016980648121254407244510633052659201286180176261964525009968332831278550173970862171318019365038068877865773152797829063820126012918338943374756002283128680280095698206862579700995946824989229739927137191842923213072877 [0x397f7451000000000000000000000000140d47ba3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9526c37a3a824b2f1c6d9b6fc0990d01d0b943fa24a980f6df5d2e520bc7b4e61704abe7b3fbcfab568836750b76412493d6fe53e5ea81f7b7c229f75ae151deee4f538d0ae2558d858246fae1a3dbf3c050f7a0793e2311f85893a1b7156d1464a85049547b68be9b24655018fada9ed] vs 20080110197977749145887525945433534553590857077021135947483750375558624533740383411697985922862800000214461825206671816550639463983227828187299786323083660473729186679786877701016980648121254407244510633052659201286180176261964525009968332831278550173970862171318019365038068877865773152797829063820126012918338943374756002283128680280095698206862579700995946824989229739927137191842923213072877 [0x397f7451000000000000000000000000140d47ba3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9526c37a3a824b2f1c6d9b6fc0990d01d0b943fa24a980f6df5d2e520bc7b4e61704abe7b3fbcfab568836750b76412493d6fe53e5ea81f7b7c229f75ae151deee4f538d0ae2558d858246fae1a3dbf3c050f7a0793e2311f85893a1b7156d1464a85049547b68be9b24655018fada9ed]) cdi_type: Attestation
  
   DiversificationKey act: 0x50f7a0793e2311f85893a1b7156d1464a85049547b68be9b24655018fada9ed, exp: 0x50f7a0793e2311f85893a1b7156d1464a85049547b68be9b24655018fada9ed
   RomDigest act: 0x3d6fe53e5ea81f7b7c229f75ae151deee4f538d0ae2558d858246fae1a3dbf3c, exp: 0x3d6fe53e5ea81f7b7c229f75ae151deee4f538d0ae2558d858246fae1a3dbf3c
   HealthMeasurement act: 0x704abe7b3fbcfab568836750b7641249, exp: 0x704abe7b3fbcfab568836750b7641249

... and 3 more failures.

UVM_ERROR (cip_base_scoreboard.sv:301) scoreboard [scoreboard] alert recov_operation_err did not trigger max_delay:* has 2 failures:
- Test keymgr_stress_all has 1 failures.
  - 7.keymgr_stress_all.62938995808764358926526786082533386270797142422752929774188428816440365550817
    Line 1469, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/7.keymgr_stress_all/latest/run.log
```
  UVM_ERROR @ 613570460 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @ 613570460 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test keymgr_sideload_otbn has 1 failures.
  - 45.keymgr_sideload_otbn.41481749397185012422987545507335607856144469516152258961061501990962026643615
    Line 259, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/45.keymgr_sideload_otbn/latest/run.log
```
  UVM_ERROR @   2152833 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @   2152833 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```

UVM_ERROR (keymgr_scoreboard.sv:750) [scoreboard] Check failed item.d_data ==gmv(csr) (* [] vs * []) reg name: keymgr_reg_block.sw_share0_output_*` has 2 failures:

Test keymgr_stress_all has 2 failures.

42.keymgr_stress_all.46626581802893227742323917172043235755343083707146765927446193499290610152145
Line 340, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/42.keymgr_stress_all/latest/run.log

  UVM_ERROR @  64491073 ps: (keymgr_scoreboard.sv:750) [uvm_test_top.env.scoreboard] Check failed item.d_data == `gmv(csr) (2031994638 [0x791dc70e] vs 0 [0x0]) reg name: keymgr_reg_block.sw_share0_output_0
  UVM_INFO @  64491073 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

45.keymgr_stress_all.84677715746198123965442760520574050664605051640319462207432499621181786699925
Line 736, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/45.keymgr_stress_all/latest/run.log

  UVM_ERROR @ 272715693 ps: (keymgr_scoreboard.sv:750) [uvm_test_top.env.scoreboard] Check failed item.d_data == `gmv(csr) (3576961636 [0xd5341a64] vs 876937961 [0x344502e9]) reg name: keymgr_reg_block.sw_share0_output_2
  UVM_INFO @ 272715693 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_ERROR (cip_base_scoreboard.sv:261) scoreboard [scoreboard] alert recov_operation_err is not received! has 1 failures:
- Test keymgr_sync_async_fault_cross has 1 failures.
  - 36.keymgr_sync_async_fault_cross.24671430559496060726308260126995356878250913193542042114650913738725402268563
    Line 373, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/36.keymgr_sync_async_fault_cross/latest/run.log
```
  UVM_ERROR @ 2377674384 ps: (cip_base_scoreboard.sv:261) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err is not received!
  UVM_INFO @ 2377674384 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```