KMAC/MASKED Simulation Results

Thursday July 11 2024 23:02:31 UTC

GitHub Revision: `edf2fd5092`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 110991919330983905489672005724934609038320729526710604109871030362225161447318

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	kmac_smoke	1.752m	9.428ms	50	50	100.00
V1	csr_hw_reset	kmac_csr_hw_reset	1.190s	48.393us	5	5	100.00
V1	csr_rw	kmac_csr_rw	1.290s	491.685us	20	20	100.00
V1	csr_bit_bash	kmac_csr_bit_bash	22.810s	6.548ms	5	5	100.00
V1	csr_aliasing	kmac_csr_aliasing	9.490s	1.466ms	5	5	100.00
V1	csr_mem_rw_with_rand_reset	kmac_csr_mem_rw_with_rand_reset	2.480s	70.959us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	kmac_csr_rw	1.290s	491.685us	20	20	100.00
		kmac_csr_aliasing	9.490s	1.466ms	5	5	100.00
V1	mem_walk	kmac_mem_walk	0.780s	31.863us	5	5	100.00
V1	mem_partial_access	kmac_mem_partial_access	1.540s	45.018us	5	5	100.00
V1		TOTAL			115	115	100.00
V2	long_msg_and_output	kmac_long_msg_and_output	56.161m	126.681ms	50	50	100.00
V2	burst_write	kmac_burst_write	28.333m	31.380ms	50	50	100.00
V2	test_vectors	kmac_test_vectors_sha3_224	44.171m	411.627ms	50	50	100.00
		kmac_test_vectors_sha3_256	43.281m	1.317s	49	50	98.00
		kmac_test_vectors_sha3_384	32.771m	699.605ms	50	50	100.00
		kmac_test_vectors_sha3_512	25.319m	610.845ms	50	50	100.00
		kmac_test_vectors_shake_128	1.784h	1.033s	50	50	100.00
		kmac_test_vectors_shake_256	1.733h	2.442s	50	50	100.00
		kmac_test_vectors_kmac	7.350s	3.768ms	49	50	98.00
		kmac_test_vectors_kmac_xof	7.290s	685.367us	50	50	100.00
V2	sideload	kmac_sideload	9.215m	189.392ms	50	50	100.00
V2	app	kmac_app	6.173m	54.098ms	50	50	100.00
V2	app_with_partial_data	kmac_app_with_partial_data	7.117m	36.494ms	10	10	100.00
V2	entropy_refresh	kmac_entropy_refresh	7.592m	16.341ms	47	50	94.00
V2	error	kmac_error	8.570m	78.459ms	49	50	98.00
V2	key_error	kmac_key_error	13.450s	3.713ms	50	50	100.00
V2	edn_timeout_error	kmac_edn_timeout_error	42.110s	8.107ms	20	20	100.00
V2	entropy_mode_error	kmac_entropy_mode_error	33.360s	4.309ms	20	20	100.00
V2	entropy_ready_error	kmac_entropy_ready_error	1.346m	8.152ms	10	10	100.00
V2	lc_escalation	kmac_lc_escalation	59.030s	4.501ms	50	50	100.00
V2	stress_all	kmac_stress_all	37.639m	261.027ms	44	50	88.00
V2	intr_test	kmac_intr_test	0.870s	18.040us	50	50	100.00
V2	alert_test	kmac_alert_test	0.910s	16.290us	50	50	100.00
V2	tl_d_oob_addr_access	kmac_tl_errors	3.190s	398.917us	20	20	100.00
V2	tl_d_illegal_access	kmac_tl_errors	3.190s	398.917us	20	20	100.00
V2	tl_d_outstanding_access	kmac_csr_hw_reset	1.190s	48.393us	5	5	100.00
		kmac_csr_rw	1.290s	491.685us	20	20	100.00
		kmac_csr_aliasing	9.490s	1.466ms	5	5	100.00
		kmac_same_csr_outstanding	2.750s	749.586us	20	20	100.00
V2	tl_d_partial_access	kmac_csr_hw_reset	1.190s	48.393us	5	5	100.00
		kmac_csr_rw	1.290s	491.685us	20	20	100.00
		kmac_csr_aliasing	9.490s	1.466ms	5	5	100.00
		kmac_same_csr_outstanding	2.750s	749.586us	20	20	100.00
V2		TOTAL			1038	1050	98.86
V2S	shadow_reg_update_error	kmac_shadow_reg_errors	1.440s	212.480us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	kmac_shadow_reg_errors	1.440s	212.480us	20	20	100.00
V2S	shadow_reg_storage_error	kmac_shadow_reg_errors	1.440s	212.480us	20	20	100.00
V2S	shadowed_reset_glitch	kmac_shadow_reg_errors	1.440s	212.480us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	kmac_shadow_reg_errors_with_csr_rw	2.980s	231.867us	20	20	100.00
V2S	tl_intg_err	kmac_sec_cm	1.910m	32.836ms	5	5	100.00
		kmac_tl_intg_err	6.340s	5.256ms	20	20	100.00
V2S	sec_cm_bus_integrity	kmac_tl_intg_err	6.340s	5.256ms	20	20	100.00
V2S	sec_cm_lc_escalate_en_intersig_mubi	kmac_lc_escalation	59.030s	4.501ms	50	50	100.00
V2S	sec_cm_sw_key_key_masking	kmac_smoke	1.752m	9.428ms	50	50	100.00
V2S	sec_cm_key_sideload	kmac_sideload	9.215m	189.392ms	50	50	100.00
V2S	sec_cm_cfg_shadowed_config_shadow	kmac_shadow_reg_errors	1.440s	212.480us	20	20	100.00
V2S	sec_cm_fsm_sparse	kmac_sec_cm	1.910m	32.836ms	5	5	100.00
V2S	sec_cm_ctr_redun	kmac_sec_cm	1.910m	32.836ms	5	5	100.00
V2S	sec_cm_packer_ctr_redun	kmac_sec_cm	1.910m	32.836ms	5	5	100.00
V2S	sec_cm_cfg_shadowed_config_regwen	kmac_smoke	1.752m	9.428ms	50	50	100.00
V2S	sec_cm_fsm_global_esc	kmac_lc_escalation	59.030s	4.501ms	50	50	100.00
V2S	sec_cm_fsm_local_esc	kmac_sec_cm	1.910m	32.836ms	5	5	100.00
V2S	sec_cm_absorbed_ctrl_mubi	kmac_mubi	6.335m	15.761ms	9	10	90.00
V2S	sec_cm_sw_cmd_ctrl_sparse	kmac_smoke	1.752m	9.428ms	50	50	100.00
V2S		TOTAL			74	75	98.67
V3	throughput	kmac_throughput			0	0	--
V3	stress_all_with_rand_reset	kmac_stress_all_with_rand_reset	44.472m	677.201ms	3	10	30.00
V3		TOTAL			3	10	30.00
		TOTAL			1230	1250	98.40

Testplan Progress

Items	Total	Written	Passing	Progress
V1	8	8	8	100.00
V2	25	25	20	80.00
V2S	5	5	4	80.00
V3	2	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
94.40	97.89	92.55	99.89	78.17	95.53	98.89	97.88

Failure Buckets

UVM_FATAL (kmac_scoreboard.sv:1519) [scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (* [*] vs * [*]) Mismatch between unmasked_digest[*] and dpi_digest[*]

has 7 failures:

Test kmac_mubi has 1 failures.

3.kmac_mubi.15902202608168438058653338643850046129513852243327130036668237694526064897842
Line 387, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/3.kmac_mubi/latest/run.log

  UVM_FATAL @ 3103452730 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (27 [0x1b] vs 158 [0x9e]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 3103452730 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test kmac_entropy_refresh has 3 failures.

26.kmac_entropy_refresh.13404287531178857355599145137603603176054535185644970503580896534714481497341
Line 275, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/26.kmac_entropy_refresh/latest/run.log

  UVM_FATAL @ 632408469 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (97 [0x61] vs 198 [0xc6]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 632408469 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

41.kmac_entropy_refresh.18407817119094648847421667309055021077738514018807979268834442777644216612088
Line 527, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/41.kmac_entropy_refresh/latest/run.log

  UVM_FATAL @ 6376417695 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (174 [0xae] vs 248 [0xf8]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 6376417695 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 1 more failures.

Test kmac_stress_all has 3 failures.

32.kmac_stress_all.47604713637577999520018168452346426198719990348776503298642636313899865864315
Line 977, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/32.kmac_stress_all/latest/run.log

  UVM_FATAL @ 18540154137 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (109 [0x6d] vs 182 [0xb6]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 18540154137 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

44.kmac_stress_all.112586020071262455346757072776060996601896098252704975394873957482084941610935
Line 1327, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/44.kmac_stress_all/latest/run.log

  UVM_FATAL @ 37427609748 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (192 [0xc0] vs 170 [0xaa]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 37427609748 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 1 more failures.

UVM_ERROR (cip_base_vseq.sv:825) [kmac_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 6 failures:
- Test kmac_stress_all_with_rand_reset has 6 failures.
  - 0.kmac_stress_all_with_rand_reset.37008258253951133294411129824744363284677288457460158687661309666525937200460
    Line 784, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/0.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 19887619576 ps: (cip_base_vseq.sv:825) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 19887619576 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 2.kmac_stress_all_with_rand_reset.69263957692892955300377564199599099109533912609198539280902570500765348951757
    Line 785, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/2.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 70876606098 ps: (cip_base_vseq.sv:825) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 70876606098 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 4 more failures.
UVM_FATAL (kmac_scoreboard.sv:218) [scoreboard] Wait EDN request has 5 failures:
- Test kmac_stress_all_with_rand_reset has 1 failures.
  - 3.kmac_stress_all_with_rand_reset.53817368645259790925892759352714791378752029255008319361524158848644399295960
    Line 2715, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/3.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_FATAL @ 138127775556 ps: (kmac_scoreboard.sv:218) [uvm_test_top.env.scoreboard] Wait EDN request
  UVM_INFO @ 138127775556 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test kmac_error has 1 failures.
  - 7.kmac_error.101979453114222852607587464849313348008914317513828215878976183467426315541184
    Line 615, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/7.kmac_error/latest/run.log
```
  UVM_FATAL @ 10053621052 ps: (kmac_scoreboard.sv:218) [uvm_test_top.env.scoreboard] Wait EDN request
  UVM_INFO @ 10053621052 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test kmac_stress_all has 3 failures.
  - 20.kmac_stress_all.73679424872151748679598196763675061967317099534017038833343647585877936360180
    Line 2572, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/20.kmac_stress_all/latest/run.log
```
  UVM_FATAL @ 381377246192 ps: (kmac_scoreboard.sv:218) [uvm_test_top.env.scoreboard] Wait EDN request
  UVM_INFO @ 381377246192 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 25.kmac_stress_all.1505361234490319433260488787354367095736645561569764544637598539750725506115
    Line 444, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/25.kmac_stress_all/latest/run.log
```
  UVM_FATAL @ 26926732430 ps: (kmac_scoreboard.sv:218) [uvm_test_top.env.scoreboard] Wait EDN request
  UVM_INFO @ 26926732430 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 1 more failures.
UVM_ERROR (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (* [*] vs * [*]) Regname: kmac_reg_block.intr_state.kmac_done reset value: * has 2 failures:
- Test kmac_test_vectors_kmac has 1 failures.
  - 38.kmac_test_vectors_kmac.107530658902944638330896698615086494711833469786754993797080374610548211382846
    Line 253, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/38.kmac_test_vectors_kmac/latest/run.log
```
  UVM_ERROR @  59851637 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @  59851637 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test kmac_test_vectors_sha3_256 has 1 failures.
  - 42.kmac_test_vectors_sha3_256.108323305507932439086702411515278931782623332753431119490847081675955746504991
    Line 253, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/42.kmac_test_vectors_sha3_256/latest/run.log
```
  UVM_ERROR @  81661271 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @  81661271 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```