KMAC/MASKED Simulation Results

Tuesday August 06 2024 23:02:29 UTC

GitHub Revision: `5fd4ecc0fc`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 56304622830272859824235340993951659280265419461975949533183046575604373639200

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	kmac_smoke	1.519m	26.248ms	50	50	100.00
V1	csr_hw_reset	kmac_csr_hw_reset	1.260s	54.361us	5	5	100.00
V1	csr_rw	kmac_csr_rw	1.260s	145.632us	20	20	100.00
V1	csr_bit_bash	kmac_csr_bit_bash	20.570s	6.022ms	5	5	100.00
V1	csr_aliasing	kmac_csr_aliasing	9.560s	1.363ms	5	5	100.00
V1	csr_mem_rw_with_rand_reset	kmac_csr_mem_rw_with_rand_reset	2.700s	80.606us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	kmac_csr_rw	1.260s	145.632us	20	20	100.00
		kmac_csr_aliasing	9.560s	1.363ms	5	5	100.00
V1	mem_walk	kmac_mem_walk	0.790s	15.089us	5	5	100.00
V1	mem_partial_access	kmac_mem_partial_access	1.540s	150.785us	5	5	100.00
V1		TOTAL			115	115	100.00
V2	long_msg_and_output	kmac_long_msg_and_output	55.774m	102.291ms	40	50	80.00
V2	burst_write	kmac_burst_write	28.095m	220.883ms	50	50	100.00
V2	test_vectors	kmac_test_vectors_sha3_224	58.928m	85.965ms	38	50	76.00
		kmac_test_vectors_sha3_256	59.714m	1.569s	43	50	86.00
		kmac_test_vectors_sha3_384	50.326m	995.423ms	49	50	98.00
		kmac_test_vectors_sha3_512	33.851m	215.358ms	50	50	100.00
		kmac_test_vectors_shake_128	1.830h	251.888ms	23	50	46.00
		kmac_test_vectors_shake_256	1.631h	1.040s	23	50	46.00
		kmac_test_vectors_kmac	7.770s	516.514us	50	50	100.00
		kmac_test_vectors_kmac_xof	9.040s	849.473us	50	50	100.00
V2	sideload	kmac_sideload	9.405m	15.491ms	50	50	100.00
V2	app	kmac_app	8.848m	98.910ms	49	50	98.00
V2	app_with_partial_data	kmac_app_with_partial_data	6.768m	23.777ms	9	10	90.00
V2	entropy_refresh	kmac_entropy_refresh	8.462m	16.506ms	49	50	98.00
V2	error	kmac_error	9.641m	42.440ms	50	50	100.00
V2	key_error	kmac_key_error	13.770s	4.558ms	50	50	100.00
V2	edn_timeout_error	kmac_edn_timeout_error	41.760s	1.426ms	20	20	100.00
V2	entropy_mode_error	kmac_entropy_mode_error	48.720s	1.642ms	20	20	100.00
V2	entropy_ready_error	kmac_entropy_ready_error	50.700s	3.396ms	10	10	100.00
V2	lc_escalation	kmac_lc_escalation	50.750s	4.750ms	50	50	100.00
V2	stress_all	kmac_stress_all	1.063h	90.979ms	49	50	98.00
V2	intr_test	kmac_intr_test	0.890s	29.300us	50	50	100.00
V2	alert_test	kmac_alert_test	0.920s	287.890us	50	50	100.00
V2	tl_d_oob_addr_access	kmac_tl_errors	3.880s	132.137us	20	20	100.00
V2	tl_d_illegal_access	kmac_tl_errors	3.880s	132.137us	20	20	100.00
V2	tl_d_outstanding_access	kmac_csr_hw_reset	1.260s	54.361us	5	5	100.00
		kmac_csr_rw	1.260s	145.632us	20	20	100.00
		kmac_csr_aliasing	9.560s	1.363ms	5	5	100.00
		kmac_same_csr_outstanding	2.620s	232.188us	20	20	100.00
V2	tl_d_partial_access	kmac_csr_hw_reset	1.260s	54.361us	5	5	100.00
		kmac_csr_rw	1.260s	145.632us	20	20	100.00
		kmac_csr_aliasing	9.560s	1.363ms	5	5	100.00
		kmac_same_csr_outstanding	2.620s	232.188us	20	20	100.00
V2		TOTAL			962	1050	91.62
V2S	shadow_reg_update_error	kmac_shadow_reg_errors	1.430s	73.157us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	kmac_shadow_reg_errors	1.430s	73.157us	20	20	100.00
V2S	shadow_reg_storage_error	kmac_shadow_reg_errors	1.430s	73.157us	20	20	100.00
V2S	shadowed_reset_glitch	kmac_shadow_reg_errors	1.430s	73.157us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	kmac_shadow_reg_errors_with_csr_rw	2.900s	491.481us	20	20	100.00
V2S	tl_intg_err	kmac_sec_cm	1.742m	27.670ms	5	5	100.00
		kmac_tl_intg_err	5.490s	354.706us	20	20	100.00
V2S	sec_cm_bus_integrity	kmac_tl_intg_err	5.490s	354.706us	20	20	100.00
V2S	sec_cm_lc_escalate_en_intersig_mubi	kmac_lc_escalation	50.750s	4.750ms	50	50	100.00
V2S	sec_cm_sw_key_key_masking	kmac_smoke	1.519m	26.248ms	50	50	100.00
V2S	sec_cm_key_sideload	kmac_sideload	9.405m	15.491ms	50	50	100.00
V2S	sec_cm_cfg_shadowed_config_shadow	kmac_shadow_reg_errors	1.430s	73.157us	20	20	100.00
V2S	sec_cm_fsm_sparse	kmac_sec_cm	1.742m	27.670ms	5	5	100.00
V2S	sec_cm_ctr_redun	kmac_sec_cm	1.742m	27.670ms	5	5	100.00
V2S	sec_cm_packer_ctr_redun	kmac_sec_cm	1.742m	27.670ms	5	5	100.00
V2S	sec_cm_cfg_shadowed_config_regwen	kmac_smoke	1.519m	26.248ms	50	50	100.00
V2S	sec_cm_fsm_global_esc	kmac_lc_escalation	50.750s	4.750ms	50	50	100.00
V2S	sec_cm_fsm_local_esc	kmac_sec_cm	1.742m	27.670ms	5	5	100.00
V2S	sec_cm_absorbed_ctrl_mubi	kmac_mubi	6.382m	28.555ms	10	10	100.00
V2S	sec_cm_sw_cmd_ctrl_sparse	kmac_smoke	1.519m	26.248ms	50	50	100.00
V2S		TOTAL			75	75	100.00
V3	throughput	kmac_throughput			0	0	--
V3	stress_all_with_rand_reset	kmac_stress_all_with_rand_reset	58.563m	151.791ms	6	10	60.00
V3		TOTAL			6	10	60.00
		TOTAL			1158	1250	92.64

Testplan Progress

Items	Total	Written	Passing	Progress
V1	8	8	8	100.00
V2	25	25	15	60.00
V2S	5	5	5	100.00
V3	2	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
94.10	97.89	92.58	99.89	76.06	95.53	98.89	97.88

Failure Buckets

Job kmac_masked-sim-vcs_run_default killed due to: Exit reason: User job exceeded runlimit: User job timed out has 80 failures:
- Test kmac_test_vectors_shake_128 has 26 failures.
  - 0.kmac_test_vectors_shake_128.109596583609387983074434771585978855665961987500168172880828652866801767914799
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/0.kmac_test_vectors_shake_128/latest/run.log
```
  Job ID: smart:a6e93f08-cc51-4ca5-8be7-ad6f0fdcfabc
```
  - 3.kmac_test_vectors_shake_128.66970643483017016951408413859121244840411013339720436479508473290439150836639
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/3.kmac_test_vectors_shake_128/latest/run.log
```
  Job ID: smart:0acff5d3-8d11-4f75-b1be-ea6051536b4a
```
  - ... and 24 more failures.
- Test kmac_test_vectors_shake_256 has 27 failures.
  - 0.kmac_test_vectors_shake_256.106076214158298521419183146322098409603725839401074833988947317382426373169996
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/0.kmac_test_vectors_shake_256/latest/run.log
```
  Job ID: smart:6bb0700b-3fa4-4263-aecc-3ffe01ee5b62
```
  - 1.kmac_test_vectors_shake_256.103799072255016719619960144331111689216563164639446474477766301045878784754914
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/1.kmac_test_vectors_shake_256/latest/run.log
```
  Job ID: smart:c58d21d0-7b80-42ef-8514-dfa84c68b4e5
```
  - ... and 25 more failures.
- Test kmac_test_vectors_sha3_256 has 5 failures.
  - 1.kmac_test_vectors_sha3_256.6795564978868089788443733758950860871627961797800575171003306769582180145137
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/1.kmac_test_vectors_sha3_256/latest/run.log
```
  Job ID: smart:225d5c22-a2c3-42e4-977e-27902202a8c6
```
  - 11.kmac_test_vectors_sha3_256.50722882698044449015149204455689475870107267700060352171384089983606271891416
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/11.kmac_test_vectors_sha3_256/latest/run.log
```
  Job ID: smart:cc562ab8-8e25-41b8-bceb-049bdc097fd1
```
  - ... and 3 more failures.
- Test kmac_long_msg_and_output has 10 failures.
  - 7.kmac_long_msg_and_output.44894197395026153883329624372046487697818718349740851803127457982612264490681
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/7.kmac_long_msg_and_output/latest/run.log
```
  Job ID: smart:ee9df05c-b59e-4fec-8cf9-5ee67c69d2b5
```
  - 14.kmac_long_msg_and_output.109674548409253923411402041294271467973890991683826791889887061165584812952537
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/14.kmac_long_msg_and_output/latest/run.log
```
  Job ID: smart:74099208-fa4d-47fa-a1af-b15cfe1d2ce3
```
  - ... and 8 more failures.
- Test kmac_test_vectors_sha3_224 has 12 failures.
  - 15.kmac_test_vectors_sha3_224.47893980976136188241921556997307610244964029308451631734732468974165166557579
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/15.kmac_test_vectors_sha3_224/latest/run.log
```
  Job ID: smart:929ea702-f9ce-4b47-903f-f3a3b319d991
```
  - 16.kmac_test_vectors_sha3_224.101935745799164731343884132711909789927108502544649933232911807985875360504833
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/16.kmac_test_vectors_sha3_224/latest/run.log
```
  Job ID: smart:dc27f2a3-c2fb-4f96-9b75-10e6428f1765
```
  - ... and 10 more failures.
UVM_ERROR (csr_utils_pkg.sv:476) [csr_utils::csr_rd_check] Check failed obs == exp (* [*] vs * [*]) Regname: kmac_reg_block.intr_state.kmac_done reset value: * has 4 failures:
- Test kmac_test_vectors_shake_128 has 1 failures.
  - 1.kmac_test_vectors_shake_128.105009040144649073599860112693926927348861650066148000159617213076464519839001
    Line 253, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/1.kmac_test_vectors_shake_128/latest/run.log
```
  UVM_ERROR @  60495097 ps: (csr_utils_pkg.sv:476) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @  60495097 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test kmac_test_vectors_sha3_256 has 2 failures.
  - 26.kmac_test_vectors_sha3_256.709511022855221053866803936182826688162487160484895744988714371051004946659
    Line 253, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/26.kmac_test_vectors_sha3_256/latest/run.log
```
  UVM_ERROR @ 147000246 ps: (csr_utils_pkg.sv:476) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @ 147000246 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 34.kmac_test_vectors_sha3_256.45528965561080636010069683783660999562416918878548602660469932716062051857558
    Line 253, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/34.kmac_test_vectors_sha3_256/latest/run.log
```
  UVM_ERROR @  36725542 ps: (csr_utils_pkg.sv:476) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @  36725542 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test kmac_test_vectors_sha3_384 has 1 failures.
  - 31.kmac_test_vectors_sha3_384.35923704069095280581829473908189464038432673570183687062578097053777426090257
    Line 253, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/31.kmac_test_vectors_sha3_384/latest/run.log
```
  UVM_ERROR @  87498494 ps: (csr_utils_pkg.sv:476) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @  87498494 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (cip_base_vseq.sv:836) [kmac_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 4 failures:
- Test kmac_stress_all_with_rand_reset has 4 failures.
  - 1.kmac_stress_all_with_rand_reset.66377942996614646997231264365539055164968757622874671813136048839246504575596
    Line 394, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/1.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 23030935325 ps: (cip_base_vseq.sv:836) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 23030935325 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 3.kmac_stress_all_with_rand_reset.114323499465259411067802330640094282426429739932208153853121192079216405237340
    Line 1389, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/3.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 62146418526 ps: (cip_base_vseq.sv:836) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 62146418526 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 2 more failures.

UVM_FATAL (kmac_scoreboard.sv:1638) [scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (* [*] vs * [*]) Mismatch between unmasked_digest[*] and dpi_digest[*] has 3 failures:

Test kmac_app_with_partial_data has 1 failures.

5.kmac_app_with_partial_data.94523340625622034149707904104413786405360578144198274309665348871969498297506
Line 341, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/5.kmac_app_with_partial_data/latest/run.log

  UVM_FATAL @ 3585853826 ps: (kmac_scoreboard.sv:1638) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (90 [0x5a] vs 76 [0x4c]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 3585853826 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test kmac_stress_all has 1 failures.

39.kmac_stress_all.92339552999360902483546997595158643551791257158289226010304235455761202046911
Line 1243, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/39.kmac_stress_all/latest/run.log

  UVM_FATAL @ 90716695103 ps: (kmac_scoreboard.sv:1638) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (162 [0xa2] vs 142 [0x8e]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 90716695103 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test kmac_app has 1 failures.

43.kmac_app.108638240168518185870367210895128862988619292822724186735108959495407335333199
Line 527, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/43.kmac_app/latest/run.log

  UVM_FATAL @ 4548963419 ps: (kmac_scoreboard.sv:1638) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (179 [0xb3] vs 192 [0xc0]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 4548963419 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_FATAL (kmac_scoreboard.sv:1458) [scoreboard] Check failed digest_share*.size() == output_len_bytes (* [*] vs * [*]) Calculated output length(32) doesn't match actual output length(48)! has 1 failures:
- Test kmac_entropy_refresh has 1 failures.
  - 28.kmac_entropy_refresh.5744615842663017724937203668597340048476317431344426822097223053467894330623
    Line 691, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/28.kmac_entropy_refresh/latest/run.log
```
  UVM_FATAL @ 9974123498 ps: (kmac_scoreboard.sv:1458) [uvm_test_top.env.scoreboard] Check failed digest_share0.size() == output_len_bytes (48 [0x30] vs 32 [0x20]) Calculated output length(32) doesn't match actual output length(48)!
  UVM_INFO @ 9974123498 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```