KMAC/UNMASKED Simulation Results

Saturday August 10 2024 23:02:23 UTC

GitHub Revision: `07b417ef03`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 2196818177928134427831197337249851347498377272679561983541244979366753055772

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	kmac_smoke	1.133m	4.265ms	50	50	100.00
V1	csr_hw_reset	kmac_csr_hw_reset	1.140s	58.125us	5	5	100.00
V1	csr_rw	kmac_csr_rw	1.180s	296.452us	20	20	100.00
V1	csr_bit_bash	kmac_csr_bit_bash	20.940s	1.487ms	5	5	100.00
V1	csr_aliasing	kmac_csr_aliasing	9.070s	442.493us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	kmac_csr_mem_rw_with_rand_reset	2.530s	308.515us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	kmac_csr_rw	1.180s	296.452us	20	20	100.00
		kmac_csr_aliasing	9.070s	442.493us	5	5	100.00
V1	mem_walk	kmac_mem_walk	0.800s	17.520us	5	5	100.00
V1	mem_partial_access	kmac_mem_partial_access	1.440s	37.091us	5	5	100.00
V1		TOTAL			115	115	100.00
V2	long_msg_and_output	kmac_long_msg_and_output	1.479h	302.913ms	47	50	94.00
V2	burst_write	kmac_burst_write	19.097m	141.462ms	50	50	100.00
V2	test_vectors	kmac_test_vectors_sha3_224	1.006h	968.776ms	50	50	100.00
		kmac_test_vectors_sha3_256	54.321m	367.879ms	50	50	100.00
		kmac_test_vectors_sha3_384	40.709m	136.182ms	50	50	100.00
		kmac_test_vectors_sha3_512	26.268m	197.115ms	50	50	100.00
		kmac_test_vectors_shake_128	2.955h	342.920ms	20	50	40.00
		kmac_test_vectors_shake_256	2.982h	941.173ms	49	50	98.00
		kmac_test_vectors_kmac	6.120s	3.455ms	50	50	100.00
		kmac_test_vectors_kmac_xof	6.030s	5.095ms	50	50	100.00
V2	sideload	kmac_sideload	8.202m	32.914ms	50	50	100.00
V2	app	kmac_app	5.988m	63.647ms	47	50	94.00
V2	app_with_partial_data	kmac_app_with_partial_data	5.886m	45.998ms	10	10	100.00
V2	entropy_refresh	kmac_entropy_refresh	6.647m	19.271ms	49	50	98.00
V2	error	kmac_error	8.456m	21.010ms	48	50	96.00
V2	key_error	kmac_key_error	10.870s	7.548ms	50	50	100.00
V2	edn_timeout_error	kmac_edn_timeout_error	41.030s	8.057ms	20	20	100.00
V2	entropy_mode_error	kmac_entropy_mode_error	42.120s	2.842ms	20	20	100.00
V2	entropy_ready_error	kmac_entropy_ready_error	1.210m	33.797ms	10	10	100.00
V2	lc_escalation	kmac_lc_escalation	39.790s	3.856ms	50	50	100.00
V2	stress_all	kmac_stress_all	45.344m	396.277ms	50	50	100.00
V2	intr_test	kmac_intr_test	0.900s	15.963us	50	50	100.00
V2	alert_test	kmac_alert_test	0.850s	35.271us	50	50	100.00
V2	tl_d_oob_addr_access	kmac_tl_errors	3.750s	142.165us	20	20	100.00
V2	tl_d_illegal_access	kmac_tl_errors	3.750s	142.165us	20	20	100.00
V2	tl_d_outstanding_access	kmac_csr_hw_reset	1.140s	58.125us	5	5	100.00
		kmac_csr_rw	1.180s	296.452us	20	20	100.00
		kmac_csr_aliasing	9.070s	442.493us	5	5	100.00
		kmac_same_csr_outstanding	2.520s	392.452us	20	20	100.00
V2	tl_d_partial_access	kmac_csr_hw_reset	1.140s	58.125us	5	5	100.00
		kmac_csr_rw	1.180s	296.452us	20	20	100.00
		kmac_csr_aliasing	9.070s	442.493us	5	5	100.00
		kmac_same_csr_outstanding	2.520s	392.452us	20	20	100.00
V2		TOTAL			1010	1050	96.19
V2S	shadow_reg_update_error	kmac_shadow_reg_errors	1.360s	53.127us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	kmac_shadow_reg_errors	1.360s	53.127us	20	20	100.00
V2S	shadow_reg_storage_error	kmac_shadow_reg_errors	1.360s	53.127us	20	20	100.00
V2S	shadowed_reset_glitch	kmac_shadow_reg_errors	1.360s	53.127us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	kmac_shadow_reg_errors_with_csr_rw	2.760s	219.601us	20	20	100.00
V2S	tl_intg_err	kmac_sec_cm	1.319m	46.321ms	5	5	100.00
		kmac_tl_intg_err	5.310s	1.559ms	20	20	100.00
V2S	sec_cm_bus_integrity	kmac_tl_intg_err	5.310s	1.559ms	20	20	100.00
V2S	sec_cm_lc_escalate_en_intersig_mubi	kmac_lc_escalation	39.790s	3.856ms	50	50	100.00
V2S	sec_cm_sw_key_key_masking	kmac_smoke	1.133m	4.265ms	50	50	100.00
V2S	sec_cm_key_sideload	kmac_sideload	8.202m	32.914ms	50	50	100.00
V2S	sec_cm_cfg_shadowed_config_shadow	kmac_shadow_reg_errors	1.360s	53.127us	20	20	100.00
V2S	sec_cm_fsm_sparse	kmac_sec_cm	1.319m	46.321ms	5	5	100.00
V2S	sec_cm_ctr_redun	kmac_sec_cm	1.319m	46.321ms	5	5	100.00
V2S	sec_cm_packer_ctr_redun	kmac_sec_cm	1.319m	46.321ms	5	5	100.00
V2S	sec_cm_cfg_shadowed_config_regwen	kmac_smoke	1.133m	4.265ms	50	50	100.00
V2S	sec_cm_fsm_global_esc	kmac_lc_escalation	39.790s	3.856ms	50	50	100.00
V2S	sec_cm_fsm_local_esc	kmac_sec_cm	1.319m	46.321ms	5	5	100.00
V2S	sec_cm_absorbed_ctrl_mubi	kmac_mubi	5.292m	21.349ms	10	10	100.00
V2S	sec_cm_sw_cmd_ctrl_sparse	kmac_smoke	1.133m	4.265ms	50	50	100.00
V2S		TOTAL			75	75	100.00
V3	throughput	kmac_throughput			0	0	--
V3	stress_all_with_rand_reset	kmac_stress_all_with_rand_reset	13.764m	33.800ms	2	10	20.00
V3		TOTAL			2	10	20.00
		TOTAL			1202	1250	96.16

Testplan Progress

Items	Total	Written	Passing	Progress
V1	8	8	8	100.00
V2	25	25	19	76.00
V2S	5	5	5	100.00
V3	2	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
92.28	95.89	92.27	100.00	68.60	94.11	98.84	96.29

Failure Buckets

Job kmac_unmasked-sim-vcs_run_default killed due to: Exit reason: User job exceeded runlimit: User job timed out has 34 failures:
- Test kmac_test_vectors_shake_128 has 30 failures.
  - 2.kmac_test_vectors_shake_128.48505640221618296110073992308660754385300902469559182816729430482883842403147
    Log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/2.kmac_test_vectors_shake_128/latest/run.log
```
  Job ID: smart:48f7b448-0cba-4d04-bdee-d0b2d73bfe7d
```
  - 3.kmac_test_vectors_shake_128.13600549845954940225409775873201093651032791999641158086085201164317708396767
    Log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/3.kmac_test_vectors_shake_128/latest/run.log
```
  Job ID: smart:c7bd19fa-fba6-43a5-a1c1-c50cb751089f
```
  - ... and 28 more failures.
- Test kmac_long_msg_and_output has 3 failures.
  - 5.kmac_long_msg_and_output.50363501910419009496960519411225332963511237170579069756852756095362650990311
    Log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/5.kmac_long_msg_and_output/latest/run.log
```
  Job ID: smart:0a6092b9-43e1-4dad-b0a8-f5c233febba0
```
  - 7.kmac_long_msg_and_output.70988983209908445598431294061133107005140986868946334326608027682735800653658
    Log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/7.kmac_long_msg_and_output/latest/run.log
```
  Job ID: smart:0b3cf3e8-03ae-468c-bec4-96652df61161
```
  - ... and 1 more failures.
- Test kmac_test_vectors_shake_256 has 1 failures.
  - 44.kmac_test_vectors_shake_256.50635768257182553767263969336460059066961385149605898327037258506749645191280
    Log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/44.kmac_test_vectors_shake_256/latest/run.log
```
  Job ID: smart:f6aba059-be82-48a7-8365-9b9aec5416b8
```
UVM_ERROR (cip_base_vseq.sv:836) [kmac_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 8 failures:
- Test kmac_stress_all_with_rand_reset has 8 failures.
  - 0.kmac_stress_all_with_rand_reset.86723212263023552084740103774483190649266530708939282719817903416230678040798
    Line 356, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/0.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 5675414401 ps: (cip_base_vseq.sv:836) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 5675414401 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 2.kmac_stress_all_with_rand_reset.27483906677876341859561364584136136454670694144627292180626600085170373088855
    Line 920, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/2.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 33799790893 ps: (cip_base_vseq.sv:836) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 33799790893 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 6 more failures.

UVM_FATAL (kmac_scoreboard.sv:1638) [scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (* [*] vs * [*]) Mismatch between unmasked_digest[*] and dpi_digest[*] has 5 failures:

Test kmac_error has 1 failures.

5.kmac_error.96498251304178668417797621111450519011172027675072473646245405916890252257750
Line 688, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/5.kmac_error/latest/run.log

  UVM_FATAL @ 16357898869 ps: (kmac_scoreboard.sv:1638) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (173 [0xad] vs 142 [0x8e]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 16357898869 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test kmac_app has 3 failures.

23.kmac_app.83250147166699779932819838674960561591763370943819811411304538919505908977165
Line 611, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/23.kmac_app/latest/run.log

  UVM_FATAL @ 24352462562 ps: (kmac_scoreboard.sv:1638) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (35 [0x23] vs 198 [0xc6]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 24352462562 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

26.kmac_app.25697358533597580168751243276116354870654649985973250313736403654252277371299
Line 427, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/26.kmac_app/latest/run.log

  UVM_FATAL @ 2166526802 ps: (kmac_scoreboard.sv:1638) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (238 [0xee] vs 200 [0xc8]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 2166526802 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 1 more failures.

Test kmac_entropy_refresh has 1 failures.

28.kmac_entropy_refresh.92385237572548001674054274209903000022777899229898477971376327091764787573767
Line 1117, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/28.kmac_entropy_refresh/latest/run.log

  UVM_FATAL @ 32838517278 ps: (kmac_scoreboard.sv:1638) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (240 [0xf0] vs 223 [0xdf]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 32838517278 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_FATAL (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of * ps hit, indicating a probable testbench issue has 1 failures:
- Test kmac_error has 1 failures.
  - 44.kmac_error.59233112254200592938833295259398829164739470675726582217226835319909836452794
    Line 1179, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/44.kmac_error/latest/run.log
```
  UVM_FATAL @ 200000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 200000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 200000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```