KEYMGR Simulation Results

Wednesday February 07 2024 20:02:46 UTC

GitHub Revision: `5c87d18988`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 42540109002295994234923032062842839138270099951232798724643629525632267455156

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	keymgr_smoke	58.020s	24.023ms	50	50	100.00
V1	random	keymgr_random	1.334m	2.604ms	50	50	100.00
V1	csr_hw_reset	keymgr_csr_hw_reset	1.380s	34.862us	5	5	100.00
V1	csr_rw	keymgr_csr_rw	1.520s	47.177us	20	20	100.00
V1	csr_bit_bash	keymgr_csr_bit_bash	16.240s	562.049us	5	5	100.00
V1	csr_aliasing	keymgr_csr_aliasing	14.980s	1.495ms	5	5	100.00
V1	csr_mem_rw_with_rand_reset	keymgr_csr_mem_rw_with_rand_reset	2.010s	109.622us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	keymgr_csr_rw	1.520s	47.177us	20	20	100.00
		keymgr_csr_aliasing	14.980s	1.495ms	5	5	100.00
V1		TOTAL			155	155	100.00
V2	cfgen_during_op	keymgr_cfg_regwen	1.253m	5.027ms	48	50	96.00
V2	sideload	keymgr_sideload	47.560s	7.962ms	50	50	100.00
		keymgr_sideload_kmac	59.020s	8.588ms	50	50	100.00
		keymgr_sideload_aes	44.540s	3.290ms	50	50	100.00
		keymgr_sideload_otbn	42.860s	1.893ms	50	50	100.00
V2	direct_to_disabled_state	keymgr_direct_to_disabled	32.230s	4.858ms	50	50	100.00
V2	lc_disable	keymgr_lc_disable	9.300s	184.839us	50	50	100.00
V2	kmac_error_response	keymgr_kmac_rsp_err	2.001m	21.420ms	42	50	84.00
V2	invalid_sw_input	keymgr_sw_invalid_input	1.125m	2.792ms	50	50	100.00
V2	invalid_hw_input	keymgr_hwsw_invalid_input	1.229m	2.429ms	49	50	98.00
V2	sync_async_fault_cross	keymgr_sync_async_fault_cross	18.720s	3.582ms	50	50	100.00
V2	stress_all	keymgr_stress_all	16.596m	31.194ms	50	50	100.00
V2	intr_test	keymgr_intr_test	0.920s	19.886us	50	50	100.00
V2	alert_test	keymgr_alert_test	1.010s	66.947us	50	50	100.00
V2	tl_d_oob_addr_access	keymgr_tl_errors	3.990s	759.110us	20	20	100.00
V2	tl_d_illegal_access	keymgr_tl_errors	3.990s	759.110us	20	20	100.00
V2	tl_d_outstanding_access	keymgr_csr_hw_reset	1.380s	34.862us	5	5	100.00
		keymgr_csr_rw	1.520s	47.177us	20	20	100.00
		keymgr_csr_aliasing	14.980s	1.495ms	5	5	100.00
		keymgr_same_csr_outstanding	4.800s	1.323ms	20	20	100.00
V2	tl_d_partial_access	keymgr_csr_hw_reset	1.380s	34.862us	5	5	100.00
		keymgr_csr_rw	1.520s	47.177us	20	20	100.00
		keymgr_csr_aliasing	14.980s	1.495ms	5	5	100.00
		keymgr_same_csr_outstanding	4.800s	1.323ms	20	20	100.00
V2		TOTAL			729	740	98.51
V2S	sec_cm_additional_check	keymgr_sec_cm	2.688m	5.759ms	5	5	100.00
V2S	tl_intg_err	keymgr_sec_cm	2.688m	5.759ms	5	5	100.00
		keymgr_tl_intg_err	1.113m	3.162ms	20	20	100.00
V2S	shadow_reg_update_error	keymgr_shadow_reg_errors	21.280s	1.242ms	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	keymgr_shadow_reg_errors	21.280s	1.242ms	20	20	100.00
V2S	shadow_reg_storage_error	keymgr_shadow_reg_errors	21.280s	1.242ms	20	20	100.00
V2S	shadowed_reset_glitch	keymgr_shadow_reg_errors	21.280s	1.242ms	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	keymgr_shadow_reg_errors_with_csr_rw	10.890s	904.710us	20	20	100.00
V2S	prim_count_check	keymgr_sec_cm	2.688m	5.759ms	5	5	100.00
V2S	prim_fsm_check	keymgr_sec_cm	2.688m	5.759ms	5	5	100.00
V2S	sec_cm_bus_integrity	keymgr_tl_intg_err	1.113m	3.162ms	20	20	100.00
V2S	sec_cm_config_shadow	keymgr_shadow_reg_errors	21.280s	1.242ms	20	20	100.00
V2S	sec_cm_op_config_regwen	keymgr_cfg_regwen	1.253m	5.027ms	48	50	96.00
V2S	sec_cm_reseed_config_regwen	keymgr_random	1.334m	2.604ms	50	50	100.00
		keymgr_csr_rw	1.520s	47.177us	20	20	100.00
V2S	sec_cm_sw_binding_config_regwen	keymgr_random	1.334m	2.604ms	50	50	100.00
		keymgr_csr_rw	1.520s	47.177us	20	20	100.00
V2S	sec_cm_max_key_ver_config_regwen	keymgr_random	1.334m	2.604ms	50	50	100.00
		keymgr_csr_rw	1.520s	47.177us	20	20	100.00
V2S	sec_cm_lc_ctrl_intersig_mubi	keymgr_lc_disable	9.300s	184.839us	50	50	100.00
V2S	sec_cm_constants_consistency	keymgr_hwsw_invalid_input	1.229m	2.429ms	49	50	98.00
V2S	sec_cm_intersig_consistency	keymgr_hwsw_invalid_input	1.229m	2.429ms	49	50	98.00
V2S	sec_cm_hw_key_sw_noaccess	keymgr_random	1.334m	2.604ms	50	50	100.00
V2S	sec_cm_output_keys_ctrl_redun	keymgr_sideload_protect	38.120s	4.444ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_sparse	keymgr_sec_cm	2.688m	5.759ms	5	5	100.00
V2S	sec_cm_data_fsm_sparse	keymgr_sec_cm	2.688m	5.759ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_local_esc	keymgr_sec_cm	2.688m	5.759ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_consistency	keymgr_custom_cm	45.800s	1.560ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_global_esc	keymgr_lc_disable	9.300s	184.839us	50	50	100.00
V2S	sec_cm_ctrl_ctr_redun	keymgr_sec_cm	2.688m	5.759ms	5	5	100.00
V2S	sec_cm_kmac_if_fsm_sparse	keymgr_sec_cm	2.688m	5.759ms	5	5	100.00
V2S	sec_cm_kmac_if_ctr_redun	keymgr_sec_cm	2.688m	5.759ms	5	5	100.00
V2S	sec_cm_kmac_if_cmd_ctrl_consistency	keymgr_custom_cm	45.800s	1.560ms	50	50	100.00
V2S	sec_cm_kmac_if_done_ctrl_consistency	keymgr_custom_cm	45.800s	1.560ms	50	50	100.00
V2S	sec_cm_reseed_ctr_redun	keymgr_sec_cm	2.688m	5.759ms	5	5	100.00
V2S	sec_cm_side_load_sel_ctrl_consistency	keymgr_custom_cm	45.800s	1.560ms	50	50	100.00
V2S	sec_cm_sideload_ctrl_fsm_sparse	keymgr_sec_cm	2.688m	5.759ms	5	5	100.00
V2S	sec_cm_ctrl_key_integrity	keymgr_custom_cm	45.800s	1.560ms	50	50	100.00
V2S		TOTAL			165	165	100.00
V3	stress_all_with_rand_reset	keymgr_stress_all_with_rand_reset	13.420s	457.894us	44	50	88.00
V3		TOTAL			44	50	88.00
		TOTAL			1093	1110	98.47

Testplan Progress

Items	Total	Written	Passing	Progress
V1	7	7	7	100.00
V2	16	16	13	81.25
V2S	6	6	6	100.00
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
97.81	99.07	97.91	98.53	100.00	99.11	98.41	91.63

Failure Buckets

UVM_ERROR (keymgr_scoreboard.sv:1013) [scoreboard] Check failed act != exp (* [*] vs * [*]) cdi_type: Attestation has 8 failures:

Test keymgr_kmac_rsp_err has 8 failures.

5.keymgr_kmac_rsp_err.20363613617382784842669998695966542794287394524860105540532006123042090743261
Line 274, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/5.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @  28956185 ps: (keymgr_scoreboard.sv:1013) [uvm_test_top.env.scoreboard] Check failed act != exp (13869088684237589750254652523805745455607483863801036890679746459828093165537774794131959621954707501000610381923164545543259440617491324416227215074861405528674700251204859742206783587870679941142430325730373453064486625394900956660952335665610961029996816478142072719153073641397471531937515247832311133896877211985369837521208999830572777529208 [0x3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f948dd30be224c1f111e9283e10220cf64b76b88736e77cd740fd38d28d9f35358959831d51fd60b8de80a943bcce1b0365ea4e55c360fb016b12b6c95c6d796c5bb9e1163e032b7be3db98720b05beb7ebc7010bcc5f48205579c77530ae13e63c889f35f679b30572cb4045defdadf78] vs 13869088684237589750254652523805745455607483863801036890679746459828093165537774794131959621954707501000610381923164545543259440617491324416227215074861405528674700251204859742206783587870679941142430325730373453064486625394900956660952335665610961029996816478142072719153073641397471531937515247832311133896877211985369837521208999830572777529208 [0x3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f948dd30be224c1f111e9283e10220cf64b76b88736e77cd740fd38d28d9f35358959831d51fd60b8de80a943bcce1b0365ea4e55c360fb016b12b6c95c6d796c5bb9e1163e032b7be3db98720b05beb7ebc7010bcc5f48205579c77530ae13e63c889f35f679b30572cb4045defdadf78]) cdi_type: Attestation
  
   DiversificationKey act: 0xbc7010bcc5f48205579c77530ae13e63c889f35f679b30572cb4045defdadf78, exp: 0xbc7010bcc5f48205579c77530ae13e63c889f35f679b30572cb4045defdadf78
   RomDigest act: 0x5ea4e55c360fb016b12b6c95c6d796c5bb9e1163e032b7be3db98720b05beb7e, exp: 0x5ea4e55c360fb016b12b6c95c6d796c5bb9e1163e032b7be3db98720b05beb7e
   HealthMeasurement act: 0x959831d51fd60b8de80a943bcce1b036, exp: 0x959831d51fd60b8de80a943bcce1b036

14.keymgr_kmac_rsp_err.5917810418384772668024070402921195878331345887500916383886717532021648787626
Line 420, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/14.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @  97328716 ps: (keymgr_scoreboard.sv:1013) [uvm_test_top.env.scoreboard] Check failed act != exp (6830007924310302639415831560460182482061348796427548051097229724467615990663398758039886947313675948904120264096143692015933449250087304144854569145494276710959031247758986122561258153079393114370336792773068162196967358022419751548727714131749067960141431866135741017957423189636749917331482402630993473384801645209839656431120580558307424448696876676677979767566220499474444526923259481983369315522941406268715689581097518 [0xf6d8bf8e02e929bccf74a4054b457adc47dc646967eccc80342b8fc11d7ed4543a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9ca587ff177353c6096e8cf38c24631d003c1657d002ef46bac570b4b1f7d4c2eb84bac3e4e7f925a58aae5e822b1362f8fe4fbcc3c79a163e91f5bec8c8a118308eb1d26b2afb19f1d5f95a67dd3bc4903089b1827c893ecac3acb15059d12800eaa628b71d7d0a16a1182d5c3c6322e] vs 6830007924310302639415831560460182482061348796427548051097229724467615990663398758039886947313675948904120264096143692015933449250087304144854569145494276710959031247758986122561258153079393114370336792773068162196967358022419751548727714131749067960141431866135741017957423189636749917331482402630993473384801645209839656431120580558307424448696876676677979767566220499474444526923259481983369315522941406268715689581097518 [0xf6d8bf8e02e929bccf74a4054b457adc47dc646967eccc80342b8fc11d7ed4543a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9ca587ff177353c6096e8cf38c24631d003c1657d002ef46bac570b4b1f7d4c2eb84bac3e4e7f925a58aae5e822b1362f8fe4fbcc3c79a163e91f5bec8c8a118308eb1d26b2afb19f1d5f95a67dd3bc4903089b1827c893ecac3acb15059d12800eaa628b71d7d0a16a1182d5c3c6322e]) cdi_type: Attestation
  
   DiversificationKey act: 0x3089b1827c893ecac3acb15059d12800eaa628b71d7d0a16a1182d5c3c6322e, exp: 0x3089b1827c893ecac3acb15059d12800eaa628b71d7d0a16a1182d5c3c6322e
   RomDigest act: 0x8fe4fbcc3c79a163e91f5bec8c8a118308eb1d26b2afb19f1d5f95a67dd3bc49, exp: 0x8fe4fbcc3c79a163e91f5bec8c8a118308eb1d26b2afb19f1d5f95a67dd3bc49
   HealthMeasurement act: 0xb84bac3e4e7f925a58aae5e822b1362f, exp: 0xb84bac3e4e7f925a58aae5e822b1362f

... and 6 more failures.

UVM_ERROR (keymgr_scoreboard.sv:736) [scoreboard] Check failed item.d_data !=gmv(csr) (* [] vs * [])` has 2 failures:
- Test keymgr_stress_all_with_rand_reset has 2 failures.
  - 4.keymgr_stress_all_with_rand_reset.33354532158099486835523572442990734620180113210739520136400953506422364002185
    Line 605, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/4.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 277147002 ps: (keymgr_scoreboard.sv:736) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (1147844206 [0x446ab66e] vs 1147844206 [0x446ab66e])
  UVM_INFO @ 277147002 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 6.keymgr_stress_all_with_rand_reset.49365886518784987207811189101455276167982713894978460045312346909520915022690
    Line 1714, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/6.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 608922206 ps: (keymgr_scoreboard.sv:736) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (2139657583 [0x7f88956f] vs 2139657583 [0x7f88956f])
  UVM_INFO @ 608922206 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (keymgr_scoreboard.sv:674) [scoreboard] Check failed item.d_data == addr_phase_op_status (* [*] vs * [*]) has 2 failures:
- Test keymgr_stress_all_with_rand_reset has 2 failures.
  - 14.keymgr_stress_all_with_rand_reset.99740413566260684453460437050267030244090002206974017624456058634601944526929
    Line 914, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/14.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 808708140 ps: (keymgr_scoreboard.sv:674) [uvm_test_top.env.scoreboard] Check failed item.d_data == addr_phase_op_status (1 [0x1] vs 2 [0x2])
  UVM_INFO @ 808708140 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 49.keymgr_stress_all_with_rand_reset.7449067087924046147803528894814573830753841127354131445777352704844944953394
    Line 807, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/49.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 370527945 ps: (keymgr_scoreboard.sv:674) [uvm_test_top.env.scoreboard] Check failed item.d_data == addr_phase_op_status (1 [0x1] vs 2 [0x2])
  UVM_INFO @ 370527945 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (keymgr_scoreboard.sv:671) [scoreboard] Check failed edn_fifos[*].used() == * (* [*] vs * [*]) has 2 failures:
- Test keymgr_stress_all_with_rand_reset has 2 failures.
  - 22.keymgr_stress_all_with_rand_reset.106573225236335232437975785990333843511310763702849640769360011739664601728935
    Line 260, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/22.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @  51102972 ps: (keymgr_scoreboard.sv:671) [uvm_test_top.env.scoreboard] Check failed edn_fifos[0].used() == 2 (3 [0x3] vs 2 [0x2])
  UVM_INFO @  51102972 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 25.keymgr_stress_all_with_rand_reset.46545024520138958941173411746653068604068650714716605046549208583591524329733
    Line 929, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/25.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 226984601 ps: (keymgr_scoreboard.sv:671) [uvm_test_top.env.scoreboard] Check failed edn_fifos[0].used() == 2 (3 [0x3] vs 2 [0x2])
  UVM_INFO @ 226984601 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (cip_base_scoreboard.sv:287) scoreboard [scoreboard] alert recov_operation_err did not trigger max_delay:* has 2 failures:
- Test keymgr_cfg_regwen has 1 failures.
  - 26.keymgr_cfg_regwen.42306262190060636644615142506243049712893415060508665411617428364786518286247
    Line 262, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/26.keymgr_cfg_regwen/latest/run.log
```
  UVM_ERROR @   4682779 ps: (cip_base_scoreboard.sv:287) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @   4682779 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test keymgr_hwsw_invalid_input has 1 failures.
  - 35.keymgr_hwsw_invalid_input.103146450296948056584915539897785783484115900009481501137022940832524419126772
    Line 316, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/35.keymgr_hwsw_invalid_input/latest/run.log
```
  UVM_ERROR @  25936798 ps: (cip_base_scoreboard.sv:287) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @  25936798 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (keymgr_scoreboard.sv:750) [scoreboard] Check failed item.d_data ==gmv(csr) (* [] vs * []) reg name: keymgr_reg_block.start` has 1 failures:
- Test keymgr_cfg_regwen has 1 failures.
  - 32.keymgr_cfg_regwen.25591854014330580054498448880586867833968604085368613699542254667292137006598
    Line 309, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/32.keymgr_cfg_regwen/latest/run.log
```
  UVM_ERROR @   9145989 ps: (keymgr_scoreboard.sv:750) [uvm_test_top.env.scoreboard] Check failed item.d_data == `gmv(csr) (0 [0x0] vs 1 [0x1]) reg name: keymgr_reg_block.start
  UVM_INFO @   9145989 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```