KMAC/MASKED Simulation Results

Sunday June 23 2024 23:02:35 UTC

GitHub Revision: `25e609d6bb`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 43520053114197278442322840927374150239284669988213580416404649115121474470865

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	kmac_smoke	1.671m	18.003ms	50	50	100.00
V1	csr_hw_reset	kmac_csr_hw_reset	1.200s	38.255us	5	5	100.00
V1	csr_rw	kmac_csr_rw	1.240s	69.193us	20	20	100.00
V1	csr_bit_bash	kmac_csr_bit_bash	21.390s	1.536ms	5	5	100.00
V1	csr_aliasing	kmac_csr_aliasing	10.900s	2.076ms	5	5	100.00
V1	csr_mem_rw_with_rand_reset	kmac_csr_mem_rw_with_rand_reset	2.600s	281.790us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	kmac_csr_rw	1.240s	69.193us	20	20	100.00
		kmac_csr_aliasing	10.900s	2.076ms	5	5	100.00
V1	mem_walk	kmac_mem_walk	0.810s	11.257us	5	5	100.00
V1	mem_partial_access	kmac_mem_partial_access	1.550s	135.011us	5	5	100.00
V1		TOTAL			115	115	100.00
V2	long_msg_and_output	kmac_long_msg_and_output	56.280m	128.596ms	50	50	100.00
V2	burst_write	kmac_burst_write	24.625m	52.765ms	50	50	100.00
V2	test_vectors	kmac_test_vectors_sha3_224	42.486m	403.224ms	50	50	100.00
		kmac_test_vectors_sha3_256	44.922m	1.521s	50	50	100.00
		kmac_test_vectors_sha3_384	32.687m	162.991ms	50	50	100.00
		kmac_test_vectors_sha3_512	28.086m	834.003ms	48	50	96.00
		kmac_test_vectors_shake_128	1.842h	1.698s	50	50	100.00
		kmac_test_vectors_shake_256	1.649h	1.575s	50	50	100.00
		kmac_test_vectors_kmac	7.920s	3.565ms	50	50	100.00
		kmac_test_vectors_kmac_xof	7.710s	2.935ms	49	50	98.00
V2	sideload	kmac_sideload	9.245m	20.291ms	50	50	100.00
V2	app	kmac_app	7.372m	48.613ms	48	50	96.00
V2	app_with_partial_data	kmac_app_with_partial_data	6.814m	73.530ms	9	10	90.00
V2	entropy_refresh	kmac_entropy_refresh	6.977m	32.821ms	49	50	98.00
V2	error	kmac_error	8.717m	153.791ms	49	50	98.00
V2	key_error	kmac_key_error	13.450s	2.014ms	50	50	100.00
V2	edn_timeout_error	kmac_edn_timeout_error	54.730s	14.625ms	20	20	100.00
V2	entropy_mode_error	kmac_entropy_mode_error	42.130s	12.993ms	20	20	100.00
V2	entropy_ready_error	kmac_entropy_ready_error	1.169m	6.265ms	10	10	100.00
V2	lc_escalation	kmac_lc_escalation	42.070s	3.367ms	50	50	100.00
V2	stress_all	kmac_stress_all	54.564m	485.109ms	47	50	94.00
V2	intr_test	kmac_intr_test	0.890s	22.559us	50	50	100.00
V2	alert_test	kmac_alert_test	0.980s	32.265us	50	50	100.00
V2	tl_d_oob_addr_access	kmac_tl_errors	3.210s	130.496us	20	20	100.00
V2	tl_d_illegal_access	kmac_tl_errors	3.210s	130.496us	20	20	100.00
V2	tl_d_outstanding_access	kmac_csr_hw_reset	1.200s	38.255us	5	5	100.00
		kmac_csr_rw	1.240s	69.193us	20	20	100.00
		kmac_csr_aliasing	10.900s	2.076ms	5	5	100.00
		kmac_same_csr_outstanding	2.800s	126.082us	20	20	100.00
V2	tl_d_partial_access	kmac_csr_hw_reset	1.200s	38.255us	5	5	100.00
		kmac_csr_rw	1.240s	69.193us	20	20	100.00
		kmac_csr_aliasing	10.900s	2.076ms	5	5	100.00
		kmac_same_csr_outstanding	2.800s	126.082us	20	20	100.00
V2		TOTAL			1039	1050	98.95
V2S	shadow_reg_update_error	kmac_shadow_reg_errors	1.430s	102.950us	19	20	95.00
V2S	shadow_reg_read_clear_staged_value	kmac_shadow_reg_errors	1.430s	102.950us	19	20	95.00
V2S	shadow_reg_storage_error	kmac_shadow_reg_errors	1.430s	102.950us	19	20	95.00
V2S	shadowed_reset_glitch	kmac_shadow_reg_errors	1.430s	102.950us	19	20	95.00
V2S	shadow_reg_update_error_with_csr_rw	kmac_shadow_reg_errors_with_csr_rw	3.440s	1.740ms	17	20	85.00
V2S	tl_intg_err	kmac_sec_cm	2.010m	33.520ms	5	5	100.00
		kmac_tl_intg_err	5.840s	3.525ms	20	20	100.00
V2S	sec_cm_bus_integrity	kmac_tl_intg_err	5.840s	3.525ms	20	20	100.00
V2S	sec_cm_lc_escalate_en_intersig_mubi	kmac_lc_escalation	42.070s	3.367ms	50	50	100.00
V2S	sec_cm_sw_key_key_masking	kmac_smoke	1.671m	18.003ms	50	50	100.00
V2S	sec_cm_key_sideload	kmac_sideload	9.245m	20.291ms	50	50	100.00
V2S	sec_cm_cfg_shadowed_config_shadow	kmac_shadow_reg_errors	1.430s	102.950us	19	20	95.00
V2S	sec_cm_fsm_sparse	kmac_sec_cm	2.010m	33.520ms	5	5	100.00
V2S	sec_cm_ctr_redun	kmac_sec_cm	2.010m	33.520ms	5	5	100.00
V2S	sec_cm_packer_ctr_redun	kmac_sec_cm	2.010m	33.520ms	5	5	100.00
V2S	sec_cm_cfg_shadowed_config_regwen	kmac_smoke	1.671m	18.003ms	50	50	100.00
V2S	sec_cm_fsm_global_esc	kmac_lc_escalation	42.070s	3.367ms	50	50	100.00
V2S	sec_cm_fsm_local_esc	kmac_sec_cm	2.010m	33.520ms	5	5	100.00
V2S	sec_cm_absorbed_ctrl_mubi	kmac_mubi	7.027m	96.543ms	9	10	90.00
V2S	sec_cm_sw_cmd_ctrl_sparse	kmac_smoke	1.671m	18.003ms	50	50	100.00
V2S		TOTAL			70	75	93.33
V3	throughput	kmac_throughput			0	0	--
V3	stress_all_with_rand_reset	kmac_stress_all_with_rand_reset	39.749m	165.459ms	2	10	20.00
V3		TOTAL			2	10	20.00
		TOTAL			1226	1250	98.08

Testplan Progress

Items	Total	Written	Passing	Progress
V1	8	8	8	100.00
V2	25	25	18	72.00
V2S	5	5	2	40.00
V3	2	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
94.24	97.91	92.62	99.89	76.76	95.59	99.05	97.88

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:828) [kmac_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 8 failures:
- Test kmac_stress_all_with_rand_reset has 8 failures.
  - 1.kmac_stress_all_with_rand_reset.41445025733460178638784514283479167808317688987926007088748211794508035986129
    Line 444, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/1.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 25469491677 ps: (cip_base_vseq.sv:828) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 25469491677 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 2.kmac_stress_all_with_rand_reset.23223373669452103326393352169246311824287302656045490119196594023429844823945
    Line 905, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/2.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 18988279183 ps: (cip_base_vseq.sv:828) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 18988279183 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 6 more failures.

UVM_FATAL (kmac_scoreboard.sv:1519) [scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (* [*] vs * [*]) Mismatch between unmasked_digest[*] and dpi_digest[*] has 6 failures:

Test kmac_app has 2 failures.

1.kmac_app.59453162473621731464573925078419658175835530921968054008937572576216039507276
Line 367, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/1.kmac_app/latest/run.log

  UVM_FATAL @ 2901121778 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (186 [0xba] vs 202 [0xca]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 2901121778 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

25.kmac_app.58338579095026849141814065971651103236774872936232457398400005998666034727208
Line 951, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/25.kmac_app/latest/run.log

  UVM_FATAL @ 105475939613 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (73 [0x49] vs 116 [0x74]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 105475939613 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test kmac_mubi has 1 failures.

2.kmac_mubi.57538632371498344836471263000040916285593757909818473842884193031100513872739
Line 275, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/2.kmac_mubi/latest/run.log

  UVM_FATAL @ 784256070 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (123 [0x7b] vs 190 [0xbe]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 784256070 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test kmac_stress_all has 2 failures.

2.kmac_stress_all.26724181895284868849249190478761220824707618984196942616205727527647900332966
Line 373, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/2.kmac_stress_all/latest/run.log

  UVM_FATAL @ 3485166633 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (95 [0x5f] vs 184 [0xb8]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 3485166633 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

13.kmac_stress_all.65436877642480078306664942917177936411127097877410460569889608431035397940511
Line 1639, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/13.kmac_stress_all/latest/run.log

  UVM_FATAL @ 22674580006 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (221 [0xdd] vs 109 [0x6d]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 22674580006 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test kmac_entropy_refresh has 1 failures.

37.kmac_entropy_refresh.57522090025601805061131816805360909924824914560631607979445116823129649803708
Line 691, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/37.kmac_entropy_refresh/latest/run.log

  UVM_FATAL @ 19598547178 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (192 [0xc0] vs 34 [0x22]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 19598547178 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_ERROR (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (* [*] vs * [*]) Regname: kmac_reg_block.intr_state.kmac_done reset value: * has 4 failures:
- Test kmac_stress_all has 1 failures.
  - 1.kmac_stress_all.92573998666125889147014096763981999868287763473317006265456041808722124662187
    Line 1682, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/1.kmac_stress_all/latest/run.log
```
  UVM_ERROR @ 70669462932 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @ 70669462932 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test kmac_test_vectors_kmac_xof has 1 failures.
  - 25.kmac_test_vectors_kmac_xof.34206110609022645660996736912197260638537104315941349573921905157825339567169
    Line 253, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/25.kmac_test_vectors_kmac_xof/latest/run.log
```
  UVM_ERROR @  29765444 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @  29765444 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test kmac_test_vectors_sha3_512 has 2 failures.
  - 32.kmac_test_vectors_sha3_512.40530157896470583177644443101909264065248600968605233875033030913278026730153
    Line 253, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/32.kmac_test_vectors_sha3_512/latest/run.log
```
  UVM_ERROR @  25878741 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @  25878741 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 34.kmac_test_vectors_sha3_512.65417570145402114978171364362954011908829980294883578315471146731689930330233
    Line 253, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/34.kmac_test_vectors_sha3_512/latest/run.log
```
  UVM_ERROR @  29885647 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @  29885647 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```

UVM_ERROR (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (* [*] vs * [*]) Regname: kmac_reg_block.status.alert_recov_ctrl_update_err reset value: * Check_csr_read_clear_staged_val task: check update_err status has 3 failures:

Test kmac_shadow_reg_errors_with_csr_rw has 2 failures.

5.kmac_shadow_reg_errors_with_csr_rw.108608043840340029225766111405330123724133149549678177983916887571006531919520
Line 255, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/5.kmac_shadow_reg_errors_with_csr_rw/latest/run.log

  UVM_ERROR @  58393586 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (1 [0x1] vs 0 [0x0]) Regname: kmac_reg_block.status.alert_recov_ctrl_update_err reset value: 0x0  Check_csr_read_clear_staged_val task: check update_err status
  UVM_INFO @  58393586 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

14.kmac_shadow_reg_errors_with_csr_rw.64050820224663895485425725730768403862871627594125647564310227531111525612147
Line 255, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/14.kmac_shadow_reg_errors_with_csr_rw/latest/run.log

  UVM_ERROR @ 315863003 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (1 [0x1] vs 0 [0x0]) Regname: kmac_reg_block.status.alert_recov_ctrl_update_err reset value: 0x0  Check_csr_read_clear_staged_val task: check update_err status
  UVM_INFO @ 315863003 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test kmac_shadow_reg_errors has 1 failures.

8.kmac_shadow_reg_errors.61851323112774557687432405029978473301386841351703215651309829328151020215864
Line 254, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/8.kmac_shadow_reg_errors/latest/run.log

  UVM_ERROR @   5530569 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (1 [0x1] vs 0 [0x0]) Regname: kmac_reg_block.status.alert_recov_ctrl_update_err reset value: 0x0  Check_csr_read_clear_staged_val task: check update_err status
  UVM_INFO @   5530569 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_FATAL (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of * ps hit, indicating a probable testbench issue has 2 failures:
- Test kmac_app_with_partial_data has 1 failures.
  - 4.kmac_app_with_partial_data.109656622604057983559799387485306111143558962812316663738322763358511885304789
    Line 847, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/4.kmac_app_with_partial_data/latest/run.log
```
  UVM_FATAL @ 200000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 200000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 200000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test kmac_error has 1 failures.
  - 40.kmac_error.52589626524899406729302864842685826369467753171292438716299586212002656792650
    Line 1161, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/40.kmac_error/latest/run.log
```
  UVM_FATAL @ 200000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 200000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 200000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (* [*] vs * [*]) Regname: kmac_reg_block.prefix_* reset value: * has 1 failures:
- Test kmac_shadow_reg_errors_with_csr_rw has 1 failures.
  - 6.kmac_shadow_reg_errors_with_csr_rw.45036710729414021109840699830356558997380941650907849546003922967803574881160
    Line 255, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/6.kmac_shadow_reg_errors_with_csr_rw/latest/run.log
```
  UVM_ERROR @  95759744 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (1805098438 [0x6b979dc6] vs 947440296 [0x3878caa8]) Regname: kmac_reg_block.prefix_6 reset value: 0x0
  UVM_INFO @  95759744 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```