KMAC/UNMASKED Simulation Results

Thursday June 06 2024 19:04:47 UTC

GitHub Revision: `32d52b8d41`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 59908589074363629542901507660786833114562191729708937078847065421241135561861

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	kmac_smoke	1.150m	4.161ms	50	50	100.00
V1	csr_hw_reset	kmac_csr_hw_reset	1.120s	34.760us	5	5	100.00
V1	csr_rw	kmac_csr_rw	1.140s	105.830us	20	20	100.00
V1	csr_bit_bash	kmac_csr_bit_bash	15.540s	565.250us	5	5	100.00
V1	csr_aliasing	kmac_csr_aliasing	9.120s	762.579us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	kmac_csr_mem_rw_with_rand_reset	2.350s	69.023us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	kmac_csr_rw	1.140s	105.830us	20	20	100.00
		kmac_csr_aliasing	9.120s	762.579us	5	5	100.00
V1	mem_walk	kmac_mem_walk	0.770s	15.415us	5	5	100.00
V1	mem_partial_access	kmac_mem_partial_access	1.230s	29.501us	5	5	100.00
V1		TOTAL			115	115	100.00
V2	long_msg_and_output	kmac_long_msg_and_output	46.387m	131.286ms	50	50	100.00
V2	burst_write	kmac_burst_write	12.933m	55.432ms	49	50	98.00
V2	test_vectors	kmac_test_vectors_sha3_224	37.643m	899.558ms	50	50	100.00
		kmac_test_vectors_sha3_256	31.652m	384.603ms	50	50	100.00
		kmac_test_vectors_sha3_384	24.849m	532.713ms	50	50	100.00
		kmac_test_vectors_sha3_512	18.373m	607.923ms	50	50	100.00
		kmac_test_vectors_shake_128	1.530h	1.504s	50	50	100.00
		kmac_test_vectors_shake_256	1.262h	863.757ms	50	50	100.00
		kmac_test_vectors_kmac	5.670s	1.840ms	50	50	100.00
		kmac_test_vectors_kmac_xof	5.140s	465.069us	50	50	100.00
V2	sideload	kmac_sideload	6.814m	28.947ms	50	50	100.00
V2	app	kmac_app	5.165m	125.989ms	50	50	100.00
V2	app_with_partial_data	kmac_app_with_partial_data	4.969m	15.993ms	8	10	80.00
V2	entropy_refresh	kmac_entropy_refresh	5.738m	94.421ms	48	50	96.00
V2	error	kmac_error	6.395m	20.227ms	49	50	98.00
V2	key_error	kmac_key_error	9.780s	3.855ms	50	50	100.00
V2	edn_timeout_error	kmac_edn_timeout_error	42.210s	2.161ms	20	20	100.00
V2	entropy_mode_error	kmac_entropy_mode_error	44.230s	9.785ms	20	20	100.00
V2	entropy_ready_error	kmac_entropy_ready_error	1.100m	18.317ms	10	10	100.00
V2	lc_escalation	kmac_lc_escalation	30.430s	3.661ms	50	50	100.00
V2	stress_all	kmac_stress_all	40.107m	85.418ms	49	50	98.00
V2	intr_test	kmac_intr_test	0.870s	52.678us	50	50	100.00
V2	alert_test	kmac_alert_test	0.870s	13.933us	50	50	100.00
V2	tl_d_oob_addr_access	kmac_tl_errors	3.210s	124.018us	20	20	100.00
V2	tl_d_illegal_access	kmac_tl_errors	3.210s	124.018us	20	20	100.00
V2	tl_d_outstanding_access	kmac_csr_hw_reset	1.120s	34.760us	5	5	100.00
		kmac_csr_rw	1.140s	105.830us	20	20	100.00
		kmac_csr_aliasing	9.120s	762.579us	5	5	100.00
		kmac_same_csr_outstanding	2.400s	187.147us	20	20	100.00
V2	tl_d_partial_access	kmac_csr_hw_reset	1.120s	34.760us	5	5	100.00
		kmac_csr_rw	1.140s	105.830us	20	20	100.00
		kmac_csr_aliasing	9.120s	762.579us	5	5	100.00
		kmac_same_csr_outstanding	2.400s	187.147us	20	20	100.00
V2		TOTAL			1043	1050	99.33
V2S	shadow_reg_update_error	kmac_shadow_reg_errors	1.370s	63.177us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	kmac_shadow_reg_errors	1.370s	63.177us	20	20	100.00
V2S	shadow_reg_storage_error	kmac_shadow_reg_errors	1.370s	63.177us	20	20	100.00
V2S	shadowed_reset_glitch	kmac_shadow_reg_errors	1.370s	63.177us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	kmac_shadow_reg_errors_with_csr_rw	2.820s	495.455us	18	20	90.00
V2S	tl_intg_err	kmac_sec_cm	1.136m	5.770ms	5	5	100.00
		kmac_tl_intg_err	4.470s	449.623us	20	20	100.00
V2S	sec_cm_bus_integrity	kmac_tl_intg_err	4.470s	449.623us	20	20	100.00
V2S	sec_cm_lc_escalate_en_intersig_mubi	kmac_lc_escalation	30.430s	3.661ms	50	50	100.00
V2S	sec_cm_sw_key_key_masking	kmac_smoke	1.150m	4.161ms	50	50	100.00
V2S	sec_cm_key_sideload	kmac_sideload	6.814m	28.947ms	50	50	100.00
V2S	sec_cm_cfg_shadowed_config_shadow	kmac_shadow_reg_errors	1.370s	63.177us	20	20	100.00
V2S	sec_cm_fsm_sparse	kmac_sec_cm	1.136m	5.770ms	5	5	100.00
V2S	sec_cm_ctr_redun	kmac_sec_cm	1.136m	5.770ms	5	5	100.00
V2S	sec_cm_packer_ctr_redun	kmac_sec_cm	1.136m	5.770ms	5	5	100.00
V2S	sec_cm_cfg_shadowed_config_regwen	kmac_smoke	1.150m	4.161ms	50	50	100.00
V2S	sec_cm_fsm_global_esc	kmac_lc_escalation	30.430s	3.661ms	50	50	100.00
V2S	sec_cm_fsm_local_esc	kmac_sec_cm	1.136m	5.770ms	5	5	100.00
V2S	sec_cm_absorbed_ctrl_mubi	kmac_mubi	5.945m	43.850ms	10	10	100.00
V2S	sec_cm_sw_cmd_ctrl_sparse	kmac_smoke	1.150m	4.161ms	50	50	100.00
V2S		TOTAL			73	75	97.33
V3	throughput	kmac_throughput			0	0	--
V3	stress_all_with_rand_reset	kmac_stress_all_with_rand_reset	48.862m	122.738ms	9	50	18.00
V3		TOTAL			9	50	18.00
		TOTAL			1240	1290	96.12

Testplan Progress

Items	Total	Written	Passing	Progress
V1	8	8	8	100.00
V2	25	25	20	80.00
V2S	5	5	4	80.00
V3	2	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
92.43	95.88	92.30	100.00	69.42	94.11	98.84	96.43

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:828) [kmac_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 33 failures:
- Test kmac_stress_all_with_rand_reset has 33 failures.
  - 1.kmac_stress_all_with_rand_reset.49404387845333948796759249066049261712877305597212722238281820659623116396406
    Line 788, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/1.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 109026051297 ps: (cip_base_vseq.sv:828) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 109026051297 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 2.kmac_stress_all_with_rand_reset.34473264524039693462414480058066957618964265699445853046798842882226670994442
    Line 584, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/2.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 11703997068 ps: (cip_base_vseq.sv:828) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 11703997068 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 31 more failures.
UVM_ERROR (kmac_scoreboard.sv:1175) [scoreboard] Check failed csr.get_mirrored_value() == item.d_data (* [*] vs * [*]) reg name: kmac_reg_block.err_code has 8 failures:
- Test kmac_stress_all_with_rand_reset has 8 failures.
  - 0.kmac_stress_all_with_rand_reset.60867056607507508090217159018143953359769373985832559821727452348433537467445
    Line 2945, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/0.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 77597986860 ps: (kmac_scoreboard.sv:1175) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (0 [0x0] vs 2147483768 [0x80000078]) reg name: kmac_reg_block.err_code
  UVM_INFO @ 77597986860 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 4.kmac_stress_all_with_rand_reset.88948369333268861074729473316551503735641603341398135009902836058097873974798
    Line 1078, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/4.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 90265813758 ps: (kmac_scoreboard.sv:1175) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (0 [0x0] vs 2147483664 [0x80000010]) reg name: kmac_reg_block.err_code
  UVM_INFO @ 90265813758 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 6 more failures.

UVM_FATAL (kmac_scoreboard.sv:1519) [scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (* [*] vs * [*]) Mismatch between unmasked_digest[*] and dpi_digest[*] has 5 failures:

Test kmac_entropy_refresh has 2 failures.

1.kmac_entropy_refresh.52142561340169447929527216225680288287762311916377744927186736956498215875159
Line 609, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/1.kmac_entropy_refresh/latest/run.log

  UVM_FATAL @ 28633972904 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (151 [0x97] vs 179 [0xb3]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 28633972904 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

42.kmac_entropy_refresh.49063794154017156501522339747052476517278409257872622030552563177286257979731
Line 361, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/42.kmac_entropy_refresh/latest/run.log

  UVM_FATAL @ 1341734724 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (147 [0x93] vs 165 [0xa5]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 1341734724 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test kmac_app_with_partial_data has 2 failures.

2.kmac_app_with_partial_data.99305089819975238097281291990589145795243553854168573454843835959541429222489
Line 711, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/2.kmac_app_with_partial_data/latest/run.log

  UVM_FATAL @ 12214120061 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (206 [0xce] vs 161 [0xa1]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 12214120061 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

5.kmac_app_with_partial_data.79870572017690122228505067274465446473304467651518317261171672892592227060015
Line 489, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/5.kmac_app_with_partial_data/latest/run.log

  UVM_FATAL @ 2199577954 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (155 [0x9b] vs 1 [0x1]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 2199577954 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test kmac_stress_all has 1 failures.

46.kmac_stress_all.15027758271196660739610025199340702483002859455421150153517312333111221479658
Line 757, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/46.kmac_stress_all/latest/run.log

  UVM_FATAL @ 27802451681 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (101 [0x65] vs 23 [0x17]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 27802451681 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_FATAL (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of * ps hit, indicating a probable testbench issue has 2 failures:
- Test kmac_error has 1 failures.
  - 4.kmac_error.32399723675539635727538113919522251640513601169452473104164986635515093734265
    Line 1047, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/4.kmac_error/latest/run.log
```
  UVM_FATAL @ 200000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 200000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 200000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test kmac_burst_write has 1 failures.
  - 33.kmac_burst_write.50635278709734773429351530378382550035943310140179677466398312181122890226332
    Line 819, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/33.kmac_burst_write/latest/run.log
```
  UVM_FATAL @ 200000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 200000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 200000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (* [*] vs * [*]) Regname: kmac_reg_block.prefix_*.prefix_* reset value: * has 1 failures:
- Test kmac_shadow_reg_errors_with_csr_rw has 1 failures.
  - 3.kmac_shadow_reg_errors_with_csr_rw.61333327231503787923371785219642544458827813961257325586150524491008642342407
    Line 264, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/3.kmac_shadow_reg_errors_with_csr_rw/latest/run.log
```
  UVM_ERROR @  33382105 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (3192275368 [0xbe4641a8] vs 0 [0x0]) Regname: kmac_reg_block.prefix_7.prefix_0 reset value: 0x0
  UVM_INFO @  33382105 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (* [*] vs * [*]) Regname: kmac_reg_block.status.alert_recov_ctrl_update_err reset value: * Check_csr_read_clear_staged_val task: check update_err status has 1 failures:
- Test kmac_shadow_reg_errors_with_csr_rw has 1 failures.
  - 11.kmac_shadow_reg_errors_with_csr_rw.6965883410826837438536531199171938420255142273329661223135880403027477826631
    Line 265, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/11.kmac_shadow_reg_errors_with_csr_rw/latest/run.log
```
  UVM_ERROR @ 378358607 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (1 [0x1] vs 0 [0x0]) Regname: kmac_reg_block.status.alert_recov_ctrl_update_err reset value: 0x0  Check_csr_read_clear_staged_val task: check update_err status
  UVM_INFO @ 378358607 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```