KMAC/UNMASKED Simulation Results

Tuesday June 11 2024 19:02:38 UTC

GitHub Revision: `dd5ad5fb77`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 66418170746903624595625818392428707033482455256751560525176982524210226376736

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	kmac_smoke	1.096m	16.650ms	50	50	100.00
V1	csr_hw_reset	kmac_csr_hw_reset	1.090s	41.960us	5	5	100.00
V1	csr_rw	kmac_csr_rw	1.240s	36.252us	20	20	100.00
V1	csr_bit_bash	kmac_csr_bit_bash	19.610s	4.176ms	5	5	100.00
V1	csr_aliasing	kmac_csr_aliasing	10.230s	3.058ms	5	5	100.00
V1	csr_mem_rw_with_rand_reset	kmac_csr_mem_rw_with_rand_reset	2.510s	136.683us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	kmac_csr_rw	1.240s	36.252us	20	20	100.00
		kmac_csr_aliasing	10.230s	3.058ms	5	5	100.00
V1	mem_walk	kmac_mem_walk	0.770s	15.234us	5	5	100.00
V1	mem_partial_access	kmac_mem_partial_access	1.570s	168.834us	5	5	100.00
V1		TOTAL			115	115	100.00
V2	long_msg_and_output	kmac_long_msg_and_output	50.213m	663.960ms	50	50	100.00
V2	burst_write	kmac_burst_write	13.785m	36.960ms	49	50	98.00
V2	test_vectors	kmac_test_vectors_sha3_224	35.259m	480.313ms	50	50	100.00
		kmac_test_vectors_sha3_256	32.796m	433.323ms	50	50	100.00
		kmac_test_vectors_sha3_384	27.496m	589.239ms	50	50	100.00
		kmac_test_vectors_sha3_512	17.610m	642.337ms	50	50	100.00
		kmac_test_vectors_shake_128	1.671h	2.563s	50	50	100.00
		kmac_test_vectors_shake_256	1.334h	1.286s	50	50	100.00
		kmac_test_vectors_kmac	5.610s	2.128ms	50	50	100.00
		kmac_test_vectors_kmac_xof	5.560s	2.637ms	50	50	100.00
V2	sideload	kmac_sideload	7.079m	22.741ms	50	50	100.00
V2	app	kmac_app	6.250m	13.405ms	49	50	98.00
V2	app_with_partial_data	kmac_app_with_partial_data	4.365m	13.245ms	10	10	100.00
V2	entropy_refresh	kmac_entropy_refresh	6.095m	16.288ms	50	50	100.00
V2	error	kmac_error	6.419m	13.817ms	50	50	100.00
V2	key_error	kmac_key_error	10.210s	3.724ms	50	50	100.00
V2	edn_timeout_error	kmac_edn_timeout_error	38.190s	1.354ms	20	20	100.00
V2	entropy_mode_error	kmac_entropy_mode_error	49.750s	2.455ms	20	20	100.00
V2	entropy_ready_error	kmac_entropy_ready_error	1.218m	31.631ms	10	10	100.00
V2	lc_escalation	kmac_lc_escalation	48.480s	894.649us	50	50	100.00
V2	stress_all	kmac_stress_all	35.273m	124.774ms	49	50	98.00
V2	intr_test	kmac_intr_test	0.830s	15.518us	50	50	100.00
V2	alert_test	kmac_alert_test	0.900s	28.872us	50	50	100.00
V2	tl_d_oob_addr_access	kmac_tl_errors	3.080s	147.480us	20	20	100.00
V2	tl_d_illegal_access	kmac_tl_errors	3.080s	147.480us	20	20	100.00
V2	tl_d_outstanding_access	kmac_csr_hw_reset	1.090s	41.960us	5	5	100.00
		kmac_csr_rw	1.240s	36.252us	20	20	100.00
		kmac_csr_aliasing	10.230s	3.058ms	5	5	100.00
		kmac_same_csr_outstanding	2.810s	491.227us	20	20	100.00
V2	tl_d_partial_access	kmac_csr_hw_reset	1.090s	41.960us	5	5	100.00
		kmac_csr_rw	1.240s	36.252us	20	20	100.00
		kmac_csr_aliasing	10.230s	3.058ms	5	5	100.00
		kmac_same_csr_outstanding	2.810s	491.227us	20	20	100.00
V2		TOTAL			1047	1050	99.71
V2S	shadow_reg_update_error	kmac_shadow_reg_errors	1.560s	59.023us	17	20	85.00
V2S	shadow_reg_read_clear_staged_value	kmac_shadow_reg_errors	1.560s	59.023us	17	20	85.00
V2S	shadow_reg_storage_error	kmac_shadow_reg_errors	1.560s	59.023us	17	20	85.00
V2S	shadowed_reset_glitch	kmac_shadow_reg_errors	1.560s	59.023us	17	20	85.00
V2S	shadow_reg_update_error_with_csr_rw	kmac_shadow_reg_errors_with_csr_rw	2.830s	531.630us	18	20	90.00
V2S	tl_intg_err	kmac_sec_cm	45.500s	13.587ms	5	5	100.00
		kmac_tl_intg_err	6.270s	546.509us	20	20	100.00
V2S	sec_cm_bus_integrity	kmac_tl_intg_err	6.270s	546.509us	20	20	100.00
V2S	sec_cm_lc_escalate_en_intersig_mubi	kmac_lc_escalation	48.480s	894.649us	50	50	100.00
V2S	sec_cm_sw_key_key_masking	kmac_smoke	1.096m	16.650ms	50	50	100.00
V2S	sec_cm_key_sideload	kmac_sideload	7.079m	22.741ms	50	50	100.00
V2S	sec_cm_cfg_shadowed_config_shadow	kmac_shadow_reg_errors	1.560s	59.023us	17	20	85.00
V2S	sec_cm_fsm_sparse	kmac_sec_cm	45.500s	13.587ms	5	5	100.00
V2S	sec_cm_ctr_redun	kmac_sec_cm	45.500s	13.587ms	5	5	100.00
V2S	sec_cm_packer_ctr_redun	kmac_sec_cm	45.500s	13.587ms	5	5	100.00
V2S	sec_cm_cfg_shadowed_config_regwen	kmac_smoke	1.096m	16.650ms	50	50	100.00
V2S	sec_cm_fsm_global_esc	kmac_lc_escalation	48.480s	894.649us	50	50	100.00
V2S	sec_cm_fsm_local_esc	kmac_sec_cm	45.500s	13.587ms	5	5	100.00
V2S	sec_cm_absorbed_ctrl_mubi	kmac_mubi	4.780m	19.719ms	9	10	90.00
V2S	sec_cm_sw_cmd_ctrl_sparse	kmac_smoke	1.096m	16.650ms	50	50	100.00
V2S		TOTAL			69	75	92.00
V3	throughput	kmac_throughput			0	0	--
V3	stress_all_with_rand_reset	kmac_stress_all_with_rand_reset	45.739m	102.350ms	8	50	16.00
V3		TOTAL			8	50	16.00
		TOTAL			1239	1290	96.05

Testplan Progress

Items	Total	Written	Passing	Progress
V1	8	8	8	100.00
V2	25	25	22	88.00
V2S	5	5	2	40.00
V3	2	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
92.21	95.88	92.30	100.00	67.77	94.11	98.84	96.58

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:828) [kmac_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 36 failures:
- Test kmac_stress_all_with_rand_reset has 36 failures.
  - 2.kmac_stress_all_with_rand_reset.46204361537315299363046881726320053280484023111157662598739314099497251617449
    Line 276, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/2.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 12798567806 ps: (cip_base_vseq.sv:828) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 12798567806 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 3.kmac_stress_all_with_rand_reset.74006718983663493107415064937014011641131501239414189326265406390997654796065
    Line 264, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/3.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 429124575 ps: (cip_base_vseq.sv:828) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 429124575 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 34 more failures.

UVM_ERROR (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (* [*] vs * [*]) Regname: kmac_reg_block.status.alert_recov_ctrl_update_err reset value: * Check_csr_read_clear_staged_val task: check update_err status has 5 failures:

Test kmac_shadow_reg_errors_with_csr_rw has 2 failures.

0.kmac_shadow_reg_errors_with_csr_rw.51913264256821825810311462011114508348668253371233260322632602024105392709752
Line 265, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/0.kmac_shadow_reg_errors_with_csr_rw/latest/run.log

  UVM_ERROR @ 420425022 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (1 [0x1] vs 0 [0x0]) Regname: kmac_reg_block.status.alert_recov_ctrl_update_err reset value: 0x0  Check_csr_read_clear_staged_val task: check update_err status
  UVM_INFO @ 420425022 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

8.kmac_shadow_reg_errors_with_csr_rw.21783983049256109292439839957308579567303824456119315888361433559985437349305
Line 265, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/8.kmac_shadow_reg_errors_with_csr_rw/latest/run.log

  UVM_ERROR @  66344322 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (1 [0x1] vs 0 [0x0]) Regname: kmac_reg_block.status.alert_recov_ctrl_update_err reset value: 0x0  Check_csr_read_clear_staged_val task: check update_err status
  UVM_INFO @  66344322 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test kmac_shadow_reg_errors has 3 failures.

2.kmac_shadow_reg_errors.110336291537278546953072885665612653645302345502685730199550662459405005836644
Line 264, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/2.kmac_shadow_reg_errors/latest/run.log

  UVM_ERROR @ 132502488 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (1 [0x1] vs 0 [0x0]) Regname: kmac_reg_block.status.alert_recov_ctrl_update_err reset value: 0x0  Check_csr_read_clear_staged_val task: check update_err status
  UVM_INFO @ 132502488 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

5.kmac_shadow_reg_errors.7690789014856552292755593691546943710566951238078656169475392492804295517251
Line 265, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/5.kmac_shadow_reg_errors/latest/run.log

  UVM_ERROR @  34543055 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (1 [0x1] vs 0 [0x0]) Regname: kmac_reg_block.status.alert_recov_ctrl_update_err reset value: 0x0  Check_csr_read_clear_staged_val task: check update_err status
  UVM_INFO @  34543055 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 1 more failures.

UVM_ERROR (kmac_scoreboard.sv:1175) [scoreboard] Check failed csr.get_mirrored_value() == item.d_data (* [*] vs * [*]) reg name: kmac_reg_block.err_code has 5 failures:
- Test kmac_stress_all_with_rand_reset has 5 failures.
  - 7.kmac_stress_all_with_rand_reset.49555533138149029291264829575670463813690129504999261970552166020282268120166
    Line 284, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/7.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 1031039317 ps: (kmac_scoreboard.sv:1175) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (0 [0x0] vs 2147483768 [0x80000078]) reg name: kmac_reg_block.err_code
  UVM_INFO @ 1031039317 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 30.kmac_stress_all_with_rand_reset.38693473584102898161351994014727796059486519623766494417972971294268550352752
    Line 935, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/30.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 34850268746 ps: (kmac_scoreboard.sv:1175) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (0 [0x0] vs 2147483760 [0x80000070]) reg name: kmac_reg_block.err_code
  UVM_INFO @ 34850268746 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 3 more failures.

UVM_FATAL (kmac_scoreboard.sv:1519) [scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (* [*] vs * [*]) Mismatch between unmasked_digest[*] and dpi_digest[*] has 4 failures:

Test kmac_mubi has 1 failures.

3.kmac_mubi.88797441092261874451733309599411790848091456236420894870570541839886977579552
Line 491, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/3.kmac_mubi/latest/run.log

  UVM_FATAL @ 1242694954 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (59 [0x3b] vs 245 [0xf5]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 1242694954 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test kmac_stress_all_with_rand_reset has 1 failures.

26.kmac_stress_all_with_rand_reset.65288217422046050874942411245125104330947182303197865214001284604708994948932
Line 2381, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/26.kmac_stress_all_with_rand_reset/latest/run.log

  UVM_FATAL @ 207865924546 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (15 [0xf] vs 3 [0x3]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 207865924546 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test kmac_stress_all has 1 failures.

29.kmac_stress_all.61167039476043778735670528716026573598648701428721807014443125940678658538372
Line 683, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/29.kmac_stress_all/latest/run.log

  UVM_FATAL @ 28637732298 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (210 [0xd2] vs 80 [0x50]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 28637732298 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test kmac_app has 1 failures.

45.kmac_app.15098128002895840133367937936515774992064025867386751680229875287011788938489
Line 373, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/45.kmac_app/latest/run.log

  UVM_FATAL @ 1622926958 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (113 [0x71] vs 232 [0xe8]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 1622926958 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_FATAL (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of * ps hit, indicating a probable testbench issue has 1 failures:
- Test kmac_burst_write has 1 failures.
  - 42.kmac_burst_write.34736116620053239275398965691770155929419829645918733391102871398128104535407
    Line 1214, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/42.kmac_burst_write/latest/run.log
```
  UVM_FATAL @ 200000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 200000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 200000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```