KMAC/MASKED Simulation Results

Thursday August 22 2024 22:02:20 UTC

GitHub Revision: `0825c81be0`

Branch: os_regression_2024_08_22

Simulator: VCS

Build randomization enabled with --build-seed 102736032995262985039236458937944411119924968439319752111682827040046827694889

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	kmac_smoke	1.269m	4.671ms	50	50	100.00
V1	csr_hw_reset	kmac_csr_hw_reset	1.000s	104.362us	5	5	100.00
V1	csr_rw	kmac_csr_rw	1.100s	124.673us	20	20	100.00
V1	csr_bit_bash	kmac_csr_bit_bash	18.630s	1.504ms	5	5	100.00
V1	csr_aliasing	kmac_csr_aliasing	8.380s	1.715ms	5	5	100.00
V1	csr_mem_rw_with_rand_reset	kmac_csr_mem_rw_with_rand_reset	2.370s	283.876us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	kmac_csr_rw	1.100s	124.673us	20	20	100.00
		kmac_csr_aliasing	8.380s	1.715ms	5	5	100.00
V1	mem_walk	kmac_mem_walk	0.690s	29.661us	5	5	100.00
V1	mem_partial_access	kmac_mem_partial_access	1.360s	197.743us	5	5	100.00
V1		TOTAL			115	115	100.00
V2	long_msg_and_output	kmac_long_msg_and_output	52.881m	102.471ms	50	50	100.00
V2	burst_write	kmac_burst_write	23.277m	31.217ms	50	50	100.00
V2	test_vectors	kmac_test_vectors_sha3_224	36.117m	64.449ms	5	5	100.00
		kmac_test_vectors_sha3_256	40.367m	88.580ms	5	5	100.00
		kmac_test_vectors_sha3_384	29.042m	228.060ms	5	5	100.00
		kmac_test_vectors_sha3_512	20.983m	49.709ms	5	5	100.00
		kmac_test_vectors_shake_128	29.976m	21.361ms	5	5	100.00
		kmac_test_vectors_shake_256	6.644m	28.361ms	5	5	100.00
		kmac_test_vectors_kmac	2.730s	1.534ms	5	5	100.00
		kmac_test_vectors_kmac_xof	2.710s	187.791us	5	5	100.00
V2	sideload	kmac_sideload	8.067m	45.894ms	50	50	100.00
V2	app	kmac_app	5.677m	16.797ms	47	50	94.00
V2	app_with_partial_data	kmac_app_with_partial_data	5.807m	14.907ms	10	10	100.00
V2	entropy_refresh	kmac_entropy_refresh	6.447m	39.526ms	49	50	98.00
V2	error	kmac_error	7.175m	59.323ms	50	50	100.00
V2	key_error	kmac_key_error	16.200s	24.202ms	50	50	100.00
V2	edn_timeout_error	kmac_edn_timeout_error	36.650s	3.797ms	20	20	100.00
V2	entropy_mode_error	kmac_entropy_mode_error	42.030s	19.612ms	20	20	100.00
V2	entropy_ready_error	kmac_entropy_ready_error	57.150s	22.025ms	10	10	100.00
V2	lc_escalation	kmac_lc_escalation	49.760s	7.447ms	50	50	100.00
V2	stress_all	kmac_stress_all	40.528m	389.084ms	48	50	96.00
V2	intr_test	kmac_intr_test	0.800s	150.976us	50	50	100.00
V2	alert_test	kmac_alert_test	0.850s	281.242us	50	50	100.00
V2	tl_d_oob_addr_access	kmac_tl_errors	3.640s	681.865us	20	20	100.00
V2	tl_d_illegal_access	kmac_tl_errors	3.640s	681.865us	20	20	100.00
V2	tl_d_outstanding_access	kmac_csr_hw_reset	1.000s	104.362us	5	5	100.00
		kmac_csr_rw	1.100s	124.673us	20	20	100.00
		kmac_csr_aliasing	8.380s	1.715ms	5	5	100.00
		kmac_same_csr_outstanding	2.280s	1.869ms	20	20	100.00
V2	tl_d_partial_access	kmac_csr_hw_reset	1.000s	104.362us	5	5	100.00
		kmac_csr_rw	1.100s	124.673us	20	20	100.00
		kmac_csr_aliasing	8.380s	1.715ms	5	5	100.00
		kmac_same_csr_outstanding	2.280s	1.869ms	20	20	100.00
V2		TOTAL			684	690	99.13
V2S	shadow_reg_update_error	kmac_shadow_reg_errors	1.450s	231.670us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	kmac_shadow_reg_errors	1.450s	231.670us	20	20	100.00
V2S	shadow_reg_storage_error	kmac_shadow_reg_errors	1.450s	231.670us	20	20	100.00
V2S	shadowed_reset_glitch	kmac_shadow_reg_errors	1.450s	231.670us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	kmac_shadow_reg_errors_with_csr_rw	2.670s	586.357us	20	20	100.00
V2S	tl_intg_err	kmac_sec_cm	1.685m	28.117ms	5	5	100.00
		kmac_tl_intg_err	4.350s	234.005us	20	20	100.00
V2S	sec_cm_bus_integrity	kmac_tl_intg_err	4.350s	234.005us	20	20	100.00
V2S	sec_cm_lc_escalate_en_intersig_mubi	kmac_lc_escalation	49.760s	7.447ms	50	50	100.00
V2S	sec_cm_sw_key_key_masking	kmac_smoke	1.269m	4.671ms	50	50	100.00
V2S	sec_cm_key_sideload	kmac_sideload	8.067m	45.894ms	50	50	100.00
V2S	sec_cm_cfg_shadowed_config_shadow	kmac_shadow_reg_errors	1.450s	231.670us	20	20	100.00
V2S	sec_cm_fsm_sparse	kmac_sec_cm	1.685m	28.117ms	5	5	100.00
V2S	sec_cm_ctr_redun	kmac_sec_cm	1.685m	28.117ms	5	5	100.00
V2S	sec_cm_packer_ctr_redun	kmac_sec_cm	1.685m	28.117ms	5	5	100.00
V2S	sec_cm_cfg_shadowed_config_regwen	kmac_smoke	1.269m	4.671ms	50	50	100.00
V2S	sec_cm_fsm_global_esc	kmac_lc_escalation	49.760s	7.447ms	50	50	100.00
V2S	sec_cm_fsm_local_esc	kmac_sec_cm	1.685m	28.117ms	5	5	100.00
V2S	sec_cm_absorbed_ctrl_mubi	kmac_mubi	6.196m	13.808ms	10	10	100.00
V2S	sec_cm_sw_cmd_ctrl_sparse	kmac_smoke	1.269m	4.671ms	50	50	100.00
V2S		TOTAL			75	75	100.00
V3	throughput	kmac_throughput			0	0	--
V3	stress_all_with_rand_reset	kmac_stress_all_with_rand_reset	1.962m	7.740ms	4	10	40.00
V3		TOTAL			4	10	40.00
		TOTAL			878	890	98.65

Testplan Progress

Items	Total	Written	Passing	Progress
V1	8	8	8	100.00
V2	25	25	22	88.00
V2S	5	5	5	100.00
V3	2	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
94.25	97.91	92.65	99.89	76.76	95.59	99.05	97.88

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:867) [kmac_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 6 failures:
- Test kmac_stress_all_with_rand_reset has 6 failures.
  - 1.kmac_stress_all_with_rand_reset.106064101837745014883877743206644593730324502374763354602371852765189293352277
    Line 100, in log /workspaces/repo/scratch/os_regression_2024_08_22/kmac_masked-sim-vcs/1.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 3321598075 ps: (cip_base_vseq.sv:867) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 3321598075 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 2.kmac_stress_all_with_rand_reset.24686732311830118300415389136596455458783634625717461070659335879134301115846
    Line 71, in log /workspaces/repo/scratch/os_regression_2024_08_22/kmac_masked-sim-vcs/2.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 849794501 ps: (cip_base_vseq.sv:867) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 849794501 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 4 more failures.

UVM_FATAL (kmac_scoreboard.sv:1638) [scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (* [*] vs * [*]) Mismatch between unmasked_digest[*] and dpi_digest[*] has 4 failures:

Test kmac_app has 3 failures.

10.kmac_app.35916629099673281132915044109371752187112672890884488603977012217422851309531
Line 350, in log /workspaces/repo/scratch/os_regression_2024_08_22/kmac_masked-sim-vcs/10.kmac_app/latest/run.log

  UVM_FATAL @ 19848634758 ps: (kmac_scoreboard.sv:1638) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (223 [0xdf] vs 218 [0xda]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 19848634758 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

13.kmac_app.70456934983103129286287271455649190511333134652844037297080674329326682018709
Line 390, in log /workspaces/repo/scratch/os_regression_2024_08_22/kmac_masked-sim-vcs/13.kmac_app/latest/run.log

  UVM_FATAL @ 8249463452 ps: (kmac_scoreboard.sv:1638) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (124 [0x7c] vs 103 [0x67]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 8249463452 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 1 more failures.

Test kmac_entropy_refresh has 1 failures.

11.kmac_entropy_refresh.80044017870364952807922042292640234799116812178622956497812052927025166411373
Line 570, in log /workspaces/repo/scratch/os_regression_2024_08_22/kmac_masked-sim-vcs/11.kmac_entropy_refresh/latest/run.log

  UVM_FATAL @ 22109629750 ps: (kmac_scoreboard.sv:1638) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (62 [0x3e] vs 59 [0x3b]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 22109629750 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_ERROR (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (* [*] vs * [*]) Regname: kmac_reg_block.intr_state.kmac_done reset value: * has 2 failures:
- Test kmac_stress_all has 2 failures.
  - 10.kmac_stress_all.56210178326049516452269089768092578217431799908461100158033983795661874811384
    Line 69, in log /workspaces/repo/scratch/os_regression_2024_08_22/kmac_masked-sim-vcs/10.kmac_stress_all/latest/run.log
```
  UVM_ERROR @  63444206 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @  63444206 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 33.kmac_stress_all.65526333473685425397170243445277027362021944310396585346070198648903386434418
    Line 1553, in log /workspaces/repo/scratch/os_regression_2024_08_22/kmac_masked-sim-vcs/33.kmac_stress_all/latest/run.log
```
  UVM_ERROR @ 59151486052 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @ 59151486052 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```