KMAC/UNMASKED Simulation Results

Sunday May 12 2024 19:02:35 UTC

GitHub Revision: `69c572b503`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 72674276607041733394622960695970595070180537542023880499199659375034056632550

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	kmac_smoke	1.140m	8.826ms	50	50	100.00
V1	csr_hw_reset	kmac_csr_hw_reset	1.160s	33.170us	5	5	100.00
V1	csr_rw	kmac_csr_rw	1.250s	275.645us	20	20	100.00
V1	csr_bit_bash	kmac_csr_bit_bash	19.920s	3.843ms	5	5	100.00
V1	csr_aliasing	kmac_csr_aliasing	9.740s	1.056ms	5	5	100.00
V1	csr_mem_rw_with_rand_reset	kmac_csr_mem_rw_with_rand_reset	2.660s	349.781us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	kmac_csr_rw	1.250s	275.645us	20	20	100.00
		kmac_csr_aliasing	9.740s	1.056ms	5	5	100.00
V1	mem_walk	kmac_mem_walk	0.770s	14.020us	5	5	100.00
V1	mem_partial_access	kmac_mem_partial_access	1.170s	36.322us	5	5	100.00
V1		TOTAL			115	115	100.00
V2	long_msg_and_output	kmac_long_msg_and_output	49.438m	99.690ms	50	50	100.00
V2	burst_write	kmac_burst_write	15.409m	36.831ms	49	50	98.00
V2	test_vectors	kmac_test_vectors_sha3_224	34.944m	404.471ms	50	50	100.00
		kmac_test_vectors_sha3_256	32.055m	368.099ms	50	50	100.00
		kmac_test_vectors_sha3_384	27.895m	1.193s	50	50	100.00
		kmac_test_vectors_sha3_512	18.113m	492.138ms	50	50	100.00
		kmac_test_vectors_shake_128	1.602h	531.468ms	50	50	100.00
		kmac_test_vectors_shake_256	1.377h	2.673s	50	50	100.00
		kmac_test_vectors_kmac	5.190s	226.149us	50	50	100.00
		kmac_test_vectors_kmac_xof	5.260s	251.918us	50	50	100.00
V2	sideload	kmac_sideload	7.150m	207.197ms	50	50	100.00
V2	app	kmac_app	5.829m	45.993ms	48	50	96.00
V2	app_with_partial_data	kmac_app_with_partial_data	3.157m	11.776ms	10	10	100.00
V2	entropy_refresh	kmac_entropy_refresh	5.049m	70.930ms	50	50	100.00
V2	error	kmac_error	6.673m	25.399ms	50	50	100.00
V2	key_error	kmac_key_error	11.180s	11.931ms	49	50	98.00
V2	edn_timeout_error	kmac_edn_timeout_error	38.200s	2.118ms	20	20	100.00
V2	entropy_mode_error	kmac_entropy_mode_error	42.820s	9.369ms	20	20	100.00
V2	entropy_ready_error	kmac_entropy_ready_error	1.102m	40.608ms	10	10	100.00
V2	lc_escalation	kmac_lc_escalation	33.140s	730.331us	50	50	100.00
V2	stress_all	kmac_stress_all	44.608m	742.821ms	49	50	98.00
V2	intr_test	kmac_intr_test	0.880s	114.795us	50	50	100.00
V2	alert_test	kmac_alert_test	0.890s	74.496us	50	50	100.00
V2	tl_d_oob_addr_access	kmac_tl_errors	2.550s	157.747us	20	20	100.00
V2	tl_d_illegal_access	kmac_tl_errors	2.550s	157.747us	20	20	100.00
V2	tl_d_outstanding_access	kmac_csr_hw_reset	1.160s	33.170us	5	5	100.00
		kmac_csr_rw	1.250s	275.645us	20	20	100.00
		kmac_csr_aliasing	9.740s	1.056ms	5	5	100.00
		kmac_same_csr_outstanding	2.680s	225.332us	20	20	100.00
V2	tl_d_partial_access	kmac_csr_hw_reset	1.160s	33.170us	5	5	100.00
		kmac_csr_rw	1.250s	275.645us	20	20	100.00
		kmac_csr_aliasing	9.740s	1.056ms	5	5	100.00
		kmac_same_csr_outstanding	2.680s	225.332us	20	20	100.00
V2		TOTAL			1045	1050	99.52
V2S	shadow_reg_update_error	kmac_shadow_reg_errors	1.340s	96.835us	18	20	90.00
V2S	shadow_reg_read_clear_staged_value	kmac_shadow_reg_errors	1.340s	96.835us	18	20	90.00
V2S	shadow_reg_storage_error	kmac_shadow_reg_errors	1.340s	96.835us	18	20	90.00
V2S	shadowed_reset_glitch	kmac_shadow_reg_errors	1.340s	96.835us	18	20	90.00
V2S	shadow_reg_update_error_with_csr_rw	kmac_shadow_reg_errors_with_csr_rw	3.200s	147.030us	19	20	95.00
V2S	tl_intg_err	kmac_sec_cm	1.166m	21.379ms	5	5	100.00
		kmac_tl_intg_err	5.420s	902.935us	20	20	100.00
V2S	sec_cm_bus_integrity	kmac_tl_intg_err	5.420s	902.935us	20	20	100.00
V2S	sec_cm_lc_escalate_en_intersig_mubi	kmac_lc_escalation	33.140s	730.331us	50	50	100.00
V2S	sec_cm_sw_key_key_masking	kmac_smoke	1.140m	8.826ms	50	50	100.00
V2S	sec_cm_key_sideload	kmac_sideload	7.150m	207.197ms	50	50	100.00
V2S	sec_cm_cfg_shadowed_config_shadow	kmac_shadow_reg_errors	1.340s	96.835us	18	20	90.00
V2S	sec_cm_fsm_sparse	kmac_sec_cm	1.166m	21.379ms	5	5	100.00
V2S	sec_cm_ctr_redun	kmac_sec_cm	1.166m	21.379ms	5	5	100.00
V2S	sec_cm_packer_ctr_redun	kmac_sec_cm	1.166m	21.379ms	5	5	100.00
V2S	sec_cm_cfg_shadowed_config_regwen	kmac_smoke	1.140m	8.826ms	50	50	100.00
V2S	sec_cm_fsm_global_esc	kmac_lc_escalation	33.140s	730.331us	50	50	100.00
V2S	sec_cm_fsm_local_esc	kmac_sec_cm	1.166m	21.379ms	5	5	100.00
V2S	sec_cm_absorbed_ctrl_mubi	kmac_mubi	3.788m	8.798ms	10	10	100.00
V2S	sec_cm_sw_cmd_ctrl_sparse	kmac_smoke	1.140m	8.826ms	50	50	100.00
V2S		TOTAL			72	75	96.00
V3	throughput	kmac_throughput			0	0	--
V3	stress_all_with_rand_reset	kmac_stress_all_with_rand_reset	30.023m	172.305ms	7	50	14.00
V3		TOTAL			7	50	14.00
		TOTAL			1239	1290	96.05

Testplan Progress

Items	Total	Written	Passing	Progress
V1	8	8	8	100.00
V2	25	25	21	84.00
V2S	5	5	3	60.00
V3	2	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
91.91	95.88	92.30	100.00	66.12	94.11	98.67	96.29

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:829) [kmac_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 34 failures:
- Test kmac_stress_all_with_rand_reset has 34 failures.
  - 0.kmac_stress_all_with_rand_reset.99941569637171933760218091852268972010826509890495148296972367118722438157716
    Line 499, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/0.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 12274487784 ps: (cip_base_vseq.sv:829) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 12274487784 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 1.kmac_stress_all_with_rand_reset.77867675084167179262756813883751821622062017550846741530312457086549265355408
    Line 982, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/1.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 43500109103 ps: (cip_base_vseq.sv:829) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 43500109103 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 32 more failures.
UVM_ERROR (kmac_scoreboard.sv:1175) [scoreboard] Check failed csr.get_mirrored_value() == item.d_data (* [*] vs * [*]) reg name: kmac_reg_block.err_code has 9 failures:
- Test kmac_stress_all_with_rand_reset has 9 failures.
  - 7.kmac_stress_all_with_rand_reset.58564154399712316342596979114290642831039019910617702358841300339851800033408
    Line 1779, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/7.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 30515004380 ps: (kmac_scoreboard.sv:1175) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (0 [0x0] vs 2147483656 [0x80000008]) reg name: kmac_reg_block.err_code
  UVM_INFO @ 30515004380 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 14.kmac_stress_all_with_rand_reset.46473955038346421512681156512601910221789966719287856906249071278865680495040
    Line 1994, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/14.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 33302066138 ps: (kmac_scoreboard.sv:1175) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (0 [0x0] vs 2147483672 [0x80000018]) reg name: kmac_reg_block.err_code
  UVM_INFO @ 33302066138 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 7 more failures.

UVM_ERROR (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (* [*] vs * [*]) Regname: kmac_reg_block.status.alert_recov_ctrl_update_err reset value: * Check_csr_read_clear_staged_val task: check update_err status has 3 failures:

Test kmac_shadow_reg_errors_with_csr_rw has 1 failures.

6.kmac_shadow_reg_errors_with_csr_rw.107370485153738112221875055989064411058154893613988165684378534511472674836918
Line 264, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/6.kmac_shadow_reg_errors_with_csr_rw/latest/run.log

  UVM_ERROR @  16113013 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (1 [0x1] vs 0 [0x0]) Regname: kmac_reg_block.status.alert_recov_ctrl_update_err reset value: 0x0  Check_csr_read_clear_staged_val task: check update_err status
  UVM_INFO @  16113013 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test kmac_shadow_reg_errors has 2 failures.

15.kmac_shadow_reg_errors.2673882891433933505840409493382254057405080474611838217296529078007328762199
Line 265, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/15.kmac_shadow_reg_errors/latest/run.log

  UVM_ERROR @ 130664617 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (1 [0x1] vs 0 [0x0]) Regname: kmac_reg_block.status.alert_recov_ctrl_update_err reset value: 0x0  Check_csr_read_clear_staged_val task: check update_err status
  UVM_INFO @ 130664617 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

17.kmac_shadow_reg_errors.57665638125592316931680908604930549531529356704813426709698971668849045902709
Line 265, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/17.kmac_shadow_reg_errors/latest/run.log

  UVM_ERROR @ 249151992 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (1 [0x1] vs 0 [0x0]) Regname: kmac_reg_block.status.alert_recov_ctrl_update_err reset value: 0x0  Check_csr_read_clear_staged_val task: check update_err status
  UVM_INFO @ 249151992 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_FATAL (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of * ps hit, indicating a probable testbench issue has 2 failures:
- Test kmac_app has 1 failures.
  - 4.kmac_app.55923156322204838247464664886643934652417864667887824153491679051195722763503
    Line 1064, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/4.kmac_app/latest/run.log
```
  UVM_FATAL @ 200000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 200000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 200000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test kmac_burst_write has 1 failures.
  - 38.kmac_burst_write.33631291056560221161254230935740884016110656666951486138273796733421796384498
    Line 999, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/38.kmac_burst_write/latest/run.log
```
  UVM_FATAL @ 200000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 200000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 200000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_FATAL (kmac_scoreboard.sv:1519) [scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (* [*] vs * [*]) Mismatch between unmasked_digest[*] and dpi_digest[*] has 2 failures:
- Test kmac_stress_all has 1 failures.
  - 29.kmac_stress_all.85728468103741417362614975920889486590204227824589477511229194972827044704144
    Line 2313, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/29.kmac_stress_all/latest/run.log
```
  UVM_FATAL @ 533136233308 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (33 [0x21] vs 133 [0x85]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 533136233308 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test kmac_app has 1 failures.
  - 40.kmac_app.61928780902492304260188053567428347912972420715929684707539959873787181208466
    Line 321, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/40.kmac_app/latest/run.log
```
  UVM_FATAL @ 3441246485 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (109 [0x6d] vs 159 [0x9f]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 3441246485 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (kmac_base_vseq.sv:382) [kmac_key_error_vseq] Check failed (intr_pins[KmacErr] == *) intr_pins[KmacErr] is not set! has 1 failures:
- Test kmac_key_error has 1 failures.
  - 21.kmac_key_error.10443038485828642670788577745181902133190763120699998679946282353703236223944
    Line 273, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/21.kmac_key_error/latest/run.log
```
  UVM_ERROR @ 3837050019 ps: (kmac_base_vseq.sv:382) [uvm_test_top.env.virtual_sequencer.kmac_key_error_vseq] Check failed (intr_pins[KmacErr] == 1) intr_pins[KmacErr] is not set!
  UVM_INFO @ 3837050019 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```