KEYMGR Simulation Results

Sunday April 28 2024 19:02:25 UTC

GitHub Revision: `ae68723071`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 39039922970915743742128251849028328647614073777998354662703170901147801110391

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	keymgr_smoke	34.450s	3.434ms	48	50	96.00
V1	random	keymgr_random	45.840s	5.739ms	50	50	100.00
V1	csr_hw_reset	keymgr_csr_hw_reset	1.500s	130.566us	5	5	100.00
V1	csr_rw	keymgr_csr_rw	1.620s	28.518us	20	20	100.00
V1	csr_bit_bash	keymgr_csr_bit_bash	27.830s	3.575ms	5	5	100.00
V1	csr_aliasing	keymgr_csr_aliasing	15.250s	733.787us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	keymgr_csr_mem_rw_with_rand_reset	2.490s	223.190us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	keymgr_csr_rw	1.620s	28.518us	20	20	100.00
		keymgr_csr_aliasing	15.250s	733.787us	5	5	100.00
V1		TOTAL			153	155	98.71
V2	cfgen_during_op	keymgr_cfg_regwen	1.416m	6.784ms	49	50	98.00
V2	sideload	keymgr_sideload	40.710s	4.405ms	50	50	100.00
		keymgr_sideload_kmac	1.556m	20.604ms	50	50	100.00
		keymgr_sideload_aes	1.238m	11.531ms	49	50	98.00
		keymgr_sideload_otbn	42.650s	1.253ms	50	50	100.00
V2	direct_to_disabled_state	keymgr_direct_to_disabled	24.430s	1.315ms	50	50	100.00
V2	lc_disable	keymgr_lc_disable	1.023m	2.275ms	49	50	98.00
V2	kmac_error_response	keymgr_kmac_rsp_err	57.330s	2.080ms	44	50	88.00
V2	invalid_sw_input	keymgr_sw_invalid_input	1.254m	3.414ms	50	50	100.00
V2	invalid_hw_input	keymgr_hwsw_invalid_input	53.200s	1.711ms	50	50	100.00
V2	sync_async_fault_cross	keymgr_sync_async_fault_cross	12.610s	937.057us	50	50	100.00
V2	stress_all	keymgr_stress_all	17.472m	636.067ms	50	50	100.00
V2	intr_test	keymgr_intr_test	1.040s	25.983us	50	50	100.00
V2	alert_test	keymgr_alert_test	1.260s	33.042us	50	50	100.00
V2	tl_d_oob_addr_access	keymgr_tl_errors	5.450s	160.639us	20	20	100.00
V2	tl_d_illegal_access	keymgr_tl_errors	5.450s	160.639us	20	20	100.00
V2	tl_d_outstanding_access	keymgr_csr_hw_reset	1.500s	130.566us	5	5	100.00
		keymgr_csr_rw	1.620s	28.518us	20	20	100.00
		keymgr_csr_aliasing	15.250s	733.787us	5	5	100.00
		keymgr_same_csr_outstanding	3.920s	689.658us	20	20	100.00
V2	tl_d_partial_access	keymgr_csr_hw_reset	1.500s	130.566us	5	5	100.00
		keymgr_csr_rw	1.620s	28.518us	20	20	100.00
		keymgr_csr_aliasing	15.250s	733.787us	5	5	100.00
		keymgr_same_csr_outstanding	3.920s	689.658us	20	20	100.00
V2		TOTAL			731	740	98.78
V2S	sec_cm_additional_check	keymgr_sec_cm	2.532m	41.999ms	5	5	100.00
V2S	tl_intg_err	keymgr_sec_cm	2.532m	41.999ms	5	5	100.00
		keymgr_tl_intg_err	2.262m	25.461ms	20	20	100.00
V2S	shadow_reg_update_error	keymgr_shadow_reg_errors	25.210s	1.177ms	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	keymgr_shadow_reg_errors	25.210s	1.177ms	20	20	100.00
V2S	shadow_reg_storage_error	keymgr_shadow_reg_errors	25.210s	1.177ms	20	20	100.00
V2S	shadowed_reset_glitch	keymgr_shadow_reg_errors	25.210s	1.177ms	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	keymgr_shadow_reg_errors_with_csr_rw	14.880s	444.518us	20	20	100.00
V2S	prim_count_check	keymgr_sec_cm	2.532m	41.999ms	5	5	100.00
V2S	prim_fsm_check	keymgr_sec_cm	2.532m	41.999ms	5	5	100.00
V2S	sec_cm_bus_integrity	keymgr_tl_intg_err	2.262m	25.461ms	20	20	100.00
V2S	sec_cm_config_shadow	keymgr_shadow_reg_errors	25.210s	1.177ms	20	20	100.00
V2S	sec_cm_op_config_regwen	keymgr_cfg_regwen	1.416m	6.784ms	49	50	98.00
V2S	sec_cm_reseed_config_regwen	keymgr_random	45.840s	5.739ms	50	50	100.00
		keymgr_csr_rw	1.620s	28.518us	20	20	100.00
V2S	sec_cm_sw_binding_config_regwen	keymgr_random	45.840s	5.739ms	50	50	100.00
		keymgr_csr_rw	1.620s	28.518us	20	20	100.00
V2S	sec_cm_max_key_ver_config_regwen	keymgr_random	45.840s	5.739ms	50	50	100.00
		keymgr_csr_rw	1.620s	28.518us	20	20	100.00
V2S	sec_cm_lc_ctrl_intersig_mubi	keymgr_lc_disable	1.023m	2.275ms	49	50	98.00
V2S	sec_cm_constants_consistency	keymgr_hwsw_invalid_input	53.200s	1.711ms	50	50	100.00
V2S	sec_cm_intersig_consistency	keymgr_hwsw_invalid_input	53.200s	1.711ms	50	50	100.00
V2S	sec_cm_hw_key_sw_noaccess	keymgr_random	45.840s	5.739ms	50	50	100.00
V2S	sec_cm_output_keys_ctrl_redun	keymgr_sideload_protect	38.150s	4.010ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_sparse	keymgr_sec_cm	2.532m	41.999ms	5	5	100.00
V2S	sec_cm_data_fsm_sparse	keymgr_sec_cm	2.532m	41.999ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_local_esc	keymgr_sec_cm	2.532m	41.999ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_consistency	keymgr_custom_cm	23.530s	865.983us	50	50	100.00
V2S	sec_cm_ctrl_fsm_global_esc	keymgr_lc_disable	1.023m	2.275ms	49	50	98.00
V2S	sec_cm_ctrl_ctr_redun	keymgr_sec_cm	2.532m	41.999ms	5	5	100.00
V2S	sec_cm_kmac_if_fsm_sparse	keymgr_sec_cm	2.532m	41.999ms	5	5	100.00
V2S	sec_cm_kmac_if_ctr_redun	keymgr_sec_cm	2.532m	41.999ms	5	5	100.00
V2S	sec_cm_kmac_if_cmd_ctrl_consistency	keymgr_custom_cm	23.530s	865.983us	50	50	100.00
V2S	sec_cm_kmac_if_done_ctrl_consistency	keymgr_custom_cm	23.530s	865.983us	50	50	100.00
V2S	sec_cm_reseed_ctr_redun	keymgr_sec_cm	2.532m	41.999ms	5	5	100.00
V2S	sec_cm_side_load_sel_ctrl_consistency	keymgr_custom_cm	23.530s	865.983us	50	50	100.00
V2S	sec_cm_sideload_ctrl_fsm_sparse	keymgr_sec_cm	2.532m	41.999ms	5	5	100.00
V2S	sec_cm_ctrl_key_integrity	keymgr_custom_cm	23.530s	865.983us	50	50	100.00
V2S		TOTAL			165	165	100.00
V3	stress_all_with_rand_reset	keymgr_stress_all_with_rand_reset	26.930s	467.291us	17	50	34.00
V3		TOTAL			17	50	34.00
		TOTAL			1066	1110	96.04

Testplan Progress

Items	Total	Written	Passing	Progress
V1	7	7	6	85.71
V2	16	16	12	75.00
V2S	6	6	6	100.00
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
97.80	99.07	98.10	98.24	100.00	99.11	98.41	91.68

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:829) [keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 27 failures:
- Test keymgr_stress_all_with_rand_reset has 27 failures.
  - 0.keymgr_stress_all_with_rand_reset.100652646037657503581987914614269007999352677269736368685228787575016441987057
    Line 430, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/0.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 115223431 ps: (cip_base_vseq.sv:829) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10002 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 115223431 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 2.keymgr_stress_all_with_rand_reset.104259942629203602373156633213017677949666974536339675391038131988102930867193
    Line 323, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/2.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 436398985 ps: (cip_base_vseq.sv:829) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 436398985 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 25 more failures.

UVM_ERROR (keymgr_scoreboard.sv:1013) [scoreboard] Check failed act != exp (* [*] vs * [*]) cdi_type: Attestation has 6 failures:

Test keymgr_kmac_rsp_err has 6 failures.

3.keymgr_kmac_rsp_err.31778124157209056670832053921963995540464223139996291861272576755273476263414
Line 405, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/3.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @ 327100487 ps: (keymgr_scoreboard.sv:1013) [uvm_test_top.env.scoreboard] Check failed act != exp (6090238956535179115435899803206002182014651544499333858719185305045684502187837711594241488857636292531352589655773185041524762859036017224329195699500897029107694485192853399709356175165545661194990856118716088752417987078369493796342416878628115090269682196405431883182945478288236403613282207962366556507902991506373238381248693211485161177580081409567705671018349757917453178335264840202459259048673224220314899572515005 [0xdc1c3ddf0cee1f2f9ba2646700000000ab5111f8854d65afb4239aeab0e1cf773a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f97fc0e7c96e1845b0e5bcdbe57ae70db413768fcef04f1617d2e5758504cf439fd7b3cf150e77f95957c9f1865971e74f4c39c4ff83ab44d229e43518c328511c0f6eec028426b4a29a67cce045ea50b2f94ccaa55c087347b0ade6af6eeaf4385c167aa39b83684609f5b92e41dee0bd] vs 6090238956535179115435899803206002182014651544499333858719185305045684502187837711594241488857636292531352589655773185041524762859036017224329195699500897029107694485192853399709356175165545661194990856118716088752417987078369493796342416878628115090269682196405431883182945478288236403613282207962366556507902991506373238381248693211485161177580081409567705671018349757917453178335264840202459259048673224220314899572515005 [0xdc1c3ddf0cee1f2f9ba2646700000000ab5111f8854d65afb4239aeab0e1cf773a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f97fc0e7c96e1845b0e5bcdbe57ae70db413768fcef04f1617d2e5758504cf439fd7b3cf150e77f95957c9f1865971e74f4c39c4ff83ab44d229e43518c328511c0f6eec028426b4a29a67cce045ea50b2f94ccaa55c087347b0ade6af6eeaf4385c167aa39b83684609f5b92e41dee0bd]) cdi_type: Attestation
  
   DiversificationKey act: 0xf94ccaa55c087347b0ade6af6eeaf4385c167aa39b83684609f5b92e41dee0bd, exp: 0xf94ccaa55c087347b0ade6af6eeaf4385c167aa39b83684609f5b92e41dee0bd
   RomDigest act: 0x4c39c4ff83ab44d229e43518c328511c0f6eec028426b4a29a67cce045ea50b2, exp: 0x4c39c4ff83ab44d229e43518c328511c0f6eec028426b4a29a67cce045ea50b2
   HealthMeasurement act: 0xd7b3cf150e77f95957c9f1865971e74f, exp: 0xd7b3cf150e77f95957c9f1865971e74f

7.keymgr_kmac_rsp_err.75210310615896539238906900416024996120867962214113548255083559278227458327412
Line 620, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/7.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @ 518889822 ps: (keymgr_scoreboard.sv:1013) [uvm_test_top.env.scoreboard] Check failed act != exp (6590556812262783825339731814185971266930948640911201469609917840709634307784011234804904502834017888279642391983842954109907349025955157847876917100855796854651851774682271124267901089468223363068945122699334207813923420123499754825758415478577367143400838035753875713339472217030680152513174347169884978351477812474486683112983835315855111180513366330165376742945901230109858199479680434077282803187404019359105399465541745 [0xee314b203240a69b67d0a5d407a9a6e8b69377c2bb2fbb9c86e53c22ae2aa5953a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f981e587a41341c6bfdf74d4047298da670cbe1538bbb9f3b66a8fb2541e5839f503a32f7c18d7cb1ca772f508debca982f407b4e9735a405842816b9d9073adf6e27208cf1b30a3a949ea179495ff825814b85a627a95e606cffdf73bf8cdf00a87db273c413b91ec8c1767dadffd8071] vs 6590556812262783825339731814185971266930948640911201469609917840709634307784011234804904502834017888279642391983842954109907349025955157847876917100855796854651851774682271124267901089468223363068945122699334207813923420123499754825758415478577367143400838035753875713339472217030680152513174347169884978351477812474486683112983835315855111180513366330165376742945901230109858199479680434077282803187404019359105399465541745 [0xee314b203240a69b67d0a5d407a9a6e8b69377c2bb2fbb9c86e53c22ae2aa5953a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f981e587a41341c6bfdf74d4047298da670cbe1538bbb9f3b66a8fb2541e5839f503a32f7c18d7cb1ca772f508debca982f407b4e9735a405842816b9d9073adf6e27208cf1b30a3a949ea179495ff825814b85a627a95e606cffdf73bf8cdf00a87db273c413b91ec8c1767dadffd8071]) cdi_type: Attestation
  
   DiversificationKey act: 0x14b85a627a95e606cffdf73bf8cdf00a87db273c413b91ec8c1767dadffd8071, exp: 0x14b85a627a95e606cffdf73bf8cdf00a87db273c413b91ec8c1767dadffd8071
   RomDigest act: 0xf407b4e9735a405842816b9d9073adf6e27208cf1b30a3a949ea179495ff8258, exp: 0xf407b4e9735a405842816b9d9073adf6e27208cf1b30a3a949ea179495ff8258
   HealthMeasurement act: 0x3a32f7c18d7cb1ca772f508debca982, exp: 0x3a32f7c18d7cb1ca772f508debca982

... and 4 more failures.

UVM_ERROR (keymgr_scoreboard.sv:736) [scoreboard] Check failed item.d_data !=gmv(csr) (* [] vs * [])` has 4 failures:
- Test keymgr_stress_all_with_rand_reset has 4 failures.
  - 1.keymgr_stress_all_with_rand_reset.99223329355193354861056235635392192943572070988977706083666582438209828173939
    Line 2410, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/1.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 1266735794 ps: (keymgr_scoreboard.sv:736) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (2621255622 [0x9c3d2fc6] vs 2621255622 [0x9c3d2fc6])
  UVM_INFO @ 1266735794 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 3.keymgr_stress_all_with_rand_reset.32120694522559162040013621641250240851765910845257003550051834190698077026304
    Line 1228, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/3.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 290902089 ps: (keymgr_scoreboard.sv:736) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (3804599275 [0xe2c593eb] vs 3804599275 [0xe2c593eb])
  UVM_INFO @ 290902089 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 2 more failures.

UVM_ERROR (cip_base_scoreboard.sv:301) scoreboard [scoreboard] alert recov_operation_err did not trigger max_delay:* has 4 failures:

Test keymgr_cfg_regwen has 1 failures.

10.keymgr_cfg_regwen.31835536336041626565469929737843413274054471250140823383682321980986309538758
Line 314, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/10.keymgr_cfg_regwen/latest/run.log

  UVM_ERROR @   8976049 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @   8976049 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test keymgr_smoke has 2 failures.

31.keymgr_smoke.28785557249417468856910298006298619974431175629752855765556650501666682296312
Line 283, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/31.keymgr_smoke/latest/run.log

  UVM_ERROR @  56325194 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @  56325194 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

49.keymgr_smoke.40175882262762878472316489526449040200151012508320295672190396752362090940280
Line 268, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/49.keymgr_smoke/latest/run.log

  UVM_ERROR @   4858045 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @   4858045 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test keymgr_sideload_aes has 1 failures.

38.keymgr_sideload_aes.87837637246217741436344217840975909711279943460010130439841102812985667984450
Line 266, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/38.keymgr_sideload_aes/latest/run.log

  UVM_ERROR @   3672014 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @   3672014 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_ERROR (keymgr_scoreboard.sv:750) [scoreboard] Check failed item.d_data ==gmv(csr) (* [] vs * []) reg name: keymgr_reg_block.sw_share1_output_*` has 2 failures:
- Test keymgr_stress_all_with_rand_reset has 1 failures.
  - 8.keymgr_stress_all_with_rand_reset.103210916447573376633066752787658627558861314026428499988195258214066288857587
    Line 575, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/8.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 205445433 ps: (keymgr_scoreboard.sv:750) [uvm_test_top.env.scoreboard] Check failed item.d_data == `gmv(csr) (3852062518 [0xe599cf36] vs 3622561428 [0xd7ebe694]) reg name: keymgr_reg_block.sw_share1_output_1
  UVM_INFO @ 205445433 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test keymgr_lc_disable has 1 failures.
  - 14.keymgr_lc_disable.44063137620289697066789852147028115461439634791257398955100513788743770118811
    Line 619, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/14.keymgr_lc_disable/latest/run.log
```
  UVM_ERROR @ 408541209 ps: (keymgr_scoreboard.sv:750) [uvm_test_top.env.scoreboard] Check failed item.d_data == `gmv(csr) (3110983161 [0xb96dd5f9] vs 1115578936 [0x427e6238]) reg name: keymgr_reg_block.sw_share1_output_1
  UVM_INFO @ 408541209 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (keymgr_scoreboard.sv:674) [scoreboard] Check failed item.d_data == addr_phase_op_status (* [*] vs * [*]) has 1 failures:
- Test keymgr_stress_all_with_rand_reset has 1 failures.
  - 38.keymgr_stress_all_with_rand_reset.38701171724578293900320672592256676730807381804942418243005796905935344201925
    Line 814, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/38.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 184156279 ps: (keymgr_scoreboard.sv:674) [uvm_test_top.env.scoreboard] Check failed item.d_data == addr_phase_op_status (1 [0x1] vs 2 [0x2])
  UVM_INFO @ 184156279 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```