KMAC/UNMASKED Simulation Results

Tuesday May 21 2024 19:02:35 UTC

GitHub Revision: `be3d980075`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 85829748320245376283659198434338498577935164172956485671224275001047693479661

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	kmac_smoke	1.165m	16.687ms	50	50	100.00
V1	csr_hw_reset	kmac_csr_hw_reset	1.130s	33.269us	5	5	100.00
V1	csr_rw	kmac_csr_rw	1.230s	98.662us	20	20	100.00
V1	csr_bit_bash	kmac_csr_bit_bash	15.860s	1.169ms	5	5	100.00
V1	csr_aliasing	kmac_csr_aliasing	10.420s	1.085ms	5	5	100.00
V1	csr_mem_rw_with_rand_reset	kmac_csr_mem_rw_with_rand_reset	2.660s	41.232us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	kmac_csr_rw	1.230s	98.662us	20	20	100.00
		kmac_csr_aliasing	10.420s	1.085ms	5	5	100.00
V1	mem_walk	kmac_mem_walk	0.730s	13.853us	5	5	100.00
V1	mem_partial_access	kmac_mem_partial_access	1.450s	118.682us	5	5	100.00
V1		TOTAL			115	115	100.00
V2	long_msg_and_output	kmac_long_msg_and_output	45.766m	407.535ms	50	50	100.00
V2	burst_write	kmac_burst_write	13.268m	105.353ms	49	50	98.00
V2	test_vectors	kmac_test_vectors_sha3_224	35.319m	1.384s	50	50	100.00
		kmac_test_vectors_sha3_256	32.362m	618.762ms	50	50	100.00
		kmac_test_vectors_sha3_384	28.805m	1.238s	50	50	100.00
		kmac_test_vectors_sha3_512	19.588m	867.427ms	50	50	100.00
		kmac_test_vectors_shake_128	1.766h	3.215s	50	50	100.00
		kmac_test_vectors_shake_256	1.345h	908.525ms	50	50	100.00
		kmac_test_vectors_kmac	5.400s	3.570ms	50	50	100.00
		kmac_test_vectors_kmac_xof	5.460s	259.257us	50	50	100.00
V2	sideload	kmac_sideload	7.371m	204.019ms	50	50	100.00
V2	app	kmac_app	4.967m	113.833ms	49	50	98.00
V2	app_with_partial_data	kmac_app_with_partial_data	4.919m	8.231ms	9	10	90.00
V2	entropy_refresh	kmac_entropy_refresh	5.538m	16.665ms	49	50	98.00
V2	error	kmac_error	7.739m	81.878ms	49	50	98.00
V2	key_error	kmac_key_error	10.010s	2.380ms	50	50	100.00
V2	edn_timeout_error	kmac_edn_timeout_error	43.510s	2.327ms	20	20	100.00
V2	entropy_mode_error	kmac_entropy_mode_error	44.170s	9.160ms	20	20	100.00
V2	entropy_ready_error	kmac_entropy_ready_error	1.213m	8.078ms	10	10	100.00
V2	lc_escalation	kmac_lc_escalation	42.610s	3.271ms	50	50	100.00
V2	stress_all	kmac_stress_all	30.397m	100.889ms	50	50	100.00
V2	intr_test	kmac_intr_test	0.880s	13.806us	50	50	100.00
V2	alert_test	kmac_alert_test	0.890s	16.097us	50	50	100.00
V2	tl_d_oob_addr_access	kmac_tl_errors	3.470s	651.815us	20	20	100.00
V2	tl_d_illegal_access	kmac_tl_errors	3.470s	651.815us	20	20	100.00
V2	tl_d_outstanding_access	kmac_csr_hw_reset	1.130s	33.269us	5	5	100.00
		kmac_csr_rw	1.230s	98.662us	20	20	100.00
		kmac_csr_aliasing	10.420s	1.085ms	5	5	100.00
		kmac_same_csr_outstanding	2.760s	137.329us	20	20	100.00
V2	tl_d_partial_access	kmac_csr_hw_reset	1.130s	33.269us	5	5	100.00
		kmac_csr_rw	1.230s	98.662us	20	20	100.00
		kmac_csr_aliasing	10.420s	1.085ms	5	5	100.00
		kmac_same_csr_outstanding	2.760s	137.329us	20	20	100.00
V2		TOTAL			1045	1050	99.52
V2S	shadow_reg_update_error	kmac_shadow_reg_errors	1.520s	105.685us	19	20	95.00
V2S	shadow_reg_read_clear_staged_value	kmac_shadow_reg_errors	1.520s	105.685us	19	20	95.00
V2S	shadow_reg_storage_error	kmac_shadow_reg_errors	1.520s	105.685us	19	20	95.00
V2S	shadowed_reset_glitch	kmac_shadow_reg_errors	1.520s	105.685us	19	20	95.00
V2S	shadow_reg_update_error_with_csr_rw	kmac_shadow_reg_errors_with_csr_rw	2.990s	433.349us	18	20	90.00
V2S	tl_intg_err	kmac_sec_cm	1.095m	5.330ms	5	5	100.00
		kmac_tl_intg_err	5.590s	508.834us	20	20	100.00
V2S	sec_cm_bus_integrity	kmac_tl_intg_err	5.590s	508.834us	20	20	100.00
V2S	sec_cm_lc_escalate_en_intersig_mubi	kmac_lc_escalation	42.610s	3.271ms	50	50	100.00
V2S	sec_cm_sw_key_key_masking	kmac_smoke	1.165m	16.687ms	50	50	100.00
V2S	sec_cm_key_sideload	kmac_sideload	7.371m	204.019ms	50	50	100.00
V2S	sec_cm_cfg_shadowed_config_shadow	kmac_shadow_reg_errors	1.520s	105.685us	19	20	95.00
V2S	sec_cm_fsm_sparse	kmac_sec_cm	1.095m	5.330ms	5	5	100.00
V2S	sec_cm_ctr_redun	kmac_sec_cm	1.095m	5.330ms	5	5	100.00
V2S	sec_cm_packer_ctr_redun	kmac_sec_cm	1.095m	5.330ms	5	5	100.00
V2S	sec_cm_cfg_shadowed_config_regwen	kmac_smoke	1.165m	16.687ms	50	50	100.00
V2S	sec_cm_fsm_global_esc	kmac_lc_escalation	42.610s	3.271ms	50	50	100.00
V2S	sec_cm_fsm_local_esc	kmac_sec_cm	1.095m	5.330ms	5	5	100.00
V2S	sec_cm_absorbed_ctrl_mubi	kmac_mubi	4.155m	3.733ms	10	10	100.00
V2S	sec_cm_sw_cmd_ctrl_sparse	kmac_smoke	1.165m	16.687ms	50	50	100.00
V2S		TOTAL			72	75	96.00
V3	throughput	kmac_throughput			0	0	--
V3	stress_all_with_rand_reset	kmac_stress_all_with_rand_reset	34.926m	651.822ms	8	50	16.00
V3		TOTAL			8	50	16.00
		TOTAL			1240	1290	96.12

Testplan Progress

Items	Total	Written	Passing	Progress
V1	8	8	8	100.00
V2	25	25	20	80.00
V2S	5	5	3	60.00
V3	2	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
92.19	95.88	92.30	100.00	67.77	94.11	98.84	96.43

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:829) [kmac_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 38 failures:
- Test kmac_stress_all_with_rand_reset has 38 failures.
  - 0.kmac_stress_all_with_rand_reset.113195489049742158876091002853005286815393547833131726577239346566492527373683
    Line 389, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/0.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 23942475905 ps: (cip_base_vseq.sv:829) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 23942475905 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 2.kmac_stress_all_with_rand_reset.76209249308652362685778645343114541656705160082842489411948319761808392273038
    Line 362, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/2.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 4751026505 ps: (cip_base_vseq.sv:829) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 4751026505 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 36 more failures.
UVM_ERROR (kmac_scoreboard.sv:1175) [scoreboard] Check failed csr.get_mirrored_value() == item.d_data (* [*] vs * [*]) reg name: kmac_reg_block.err_code has 4 failures:
- Test kmac_stress_all_with_rand_reset has 4 failures.
  - 13.kmac_stress_all_with_rand_reset.85050025426394834087976865868930642864655075663597764411130996710995621245889
    Line 763, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/13.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 124733559065 ps: (kmac_scoreboard.sv:1175) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (0 [0x0] vs 2147483728 [0x80000050]) reg name: kmac_reg_block.err_code
  UVM_INFO @ 124733559065 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 25.kmac_stress_all_with_rand_reset.66415507679795588792369628244536863866664074835229808674151955160426955436582
    Line 741, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/25.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 136584507021 ps: (kmac_scoreboard.sv:1175) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (0 [0x0] vs 2147483728 [0x80000050]) reg name: kmac_reg_block.err_code
  UVM_INFO @ 136584507021 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 2 more failures.

UVM_FATAL (kmac_scoreboard.sv:1519) [scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (* [*] vs * [*]) Mismatch between unmasked_digest[*] and dpi_digest[*] has 3 failures:

Test kmac_app_with_partial_data has 1 failures.

0.kmac_app_with_partial_data.2283067489059370587771807502243802757911790456794803347490025980623418255348
Line 791, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/0.kmac_app_with_partial_data/latest/run.log

  UVM_FATAL @ 61220073469 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (108 [0x6c] vs 150 [0x96]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 61220073469 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test kmac_app has 1 failures.

32.kmac_app.52621569646661725494612752305846077222844367533568859285689024823771276336212
Line 321, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/32.kmac_app/latest/run.log

  UVM_FATAL @ 900195319 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (178 [0xb2] vs 117 [0x75]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 900195319 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test kmac_entropy_refresh has 1 failures.

37.kmac_entropy_refresh.86912650132834565369157815447887636842593762959564947809909738421714450052735
Line 453, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/37.kmac_entropy_refresh/latest/run.log

  UVM_FATAL @ 2270643841 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (145 [0x91] vs 213 [0xd5]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 2270643841 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_ERROR (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (* [*] vs * [*]) Regname: kmac_reg_block.status.alert_recov_ctrl_update_err reset value: * Check_csr_read_clear_staged_val task: check update_err status has 3 failures:

Test kmac_shadow_reg_errors_with_csr_rw has 2 failures.

13.kmac_shadow_reg_errors_with_csr_rw.111082353635983876349087734904936640850164672839544465998770059143671910099212
Line 264, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/13.kmac_shadow_reg_errors_with_csr_rw/latest/run.log

  UVM_ERROR @ 129608377 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (1 [0x1] vs 0 [0x0]) Regname: kmac_reg_block.status.alert_recov_ctrl_update_err reset value: 0x0  Check_csr_read_clear_staged_val task: check update_err status
  UVM_INFO @ 129608377 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

14.kmac_shadow_reg_errors_with_csr_rw.12313314528281317257149471529604967183551329977639054643667445287110665203218
Line 265, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/14.kmac_shadow_reg_errors_with_csr_rw/latest/run.log

  UVM_ERROR @  29965380 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (1 [0x1] vs 0 [0x0]) Regname: kmac_reg_block.status.alert_recov_ctrl_update_err reset value: 0x0  Check_csr_read_clear_staged_val task: check update_err status
  UVM_INFO @  29965380 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test kmac_shadow_reg_errors has 1 failures.

15.kmac_shadow_reg_errors.104662426826381981814790355455285855015392531947596676723788411394311878346116
Line 264, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/15.kmac_shadow_reg_errors/latest/run.log

  UVM_ERROR @   4273050 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (1 [0x1] vs 0 [0x0]) Regname: kmac_reg_block.status.alert_recov_ctrl_update_err reset value: 0x0  Check_csr_read_clear_staged_val task: check update_err status
  UVM_INFO @   4273050 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_FATAL (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of * ps hit, indicating a probable testbench issue has 2 failures:
- Test kmac_burst_write has 1 failures.
  - 0.kmac_burst_write.104369567659269884639717454383222523988783694070106425180646435110586328869887
    Line 1082, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/0.kmac_burst_write/latest/run.log
```
  UVM_FATAL @ 200000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 200000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 200000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test kmac_error has 1 failures.
  - 36.kmac_error.7162044610577971879718583476938670892129268313807759475287006186655291341742
    Line 761, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/36.kmac_error/latest/run.log
```
  UVM_FATAL @ 200000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 200000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 200000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```