KMAC/MASKED Simulation Results

Saturday July 27 2024 23:02:25 UTC

GitHub Revision: `eca25c0ff8`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 6528518538521148567139195500524222710943459299328477504124649113671643189924

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	kmac_smoke	1.572m	7.818ms	50	50	100.00
V1	csr_hw_reset	kmac_csr_hw_reset	1.150s	107.977us	5	5	100.00
V1	csr_rw	kmac_csr_rw	1.260s	28.609us	20	20	100.00
V1	csr_bit_bash	kmac_csr_bit_bash	14.330s	1.168ms	5	5	100.00
V1	csr_aliasing	kmac_csr_aliasing	10.710s	2.144ms	5	5	100.00
V1	csr_mem_rw_with_rand_reset	kmac_csr_mem_rw_with_rand_reset	2.730s	175.413us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	kmac_csr_rw	1.260s	28.609us	20	20	100.00
		kmac_csr_aliasing	10.710s	2.144ms	5	5	100.00
V1	mem_walk	kmac_mem_walk	0.790s	22.462us	5	5	100.00
V1	mem_partial_access	kmac_mem_partial_access	1.640s	301.558us	5	5	100.00
V1		TOTAL			115	115	100.00
V2	long_msg_and_output	kmac_long_msg_and_output	58.999m	26.781ms	41	50	82.00
V2	burst_write	kmac_burst_write	30.776m	101.983ms	50	50	100.00
V2	test_vectors	kmac_test_vectors_sha3_224	58.320m	138.571ms	39	50	78.00
		kmac_test_vectors_sha3_256	59.866m	191.191ms	46	50	92.00
		kmac_test_vectors_sha3_384	46.502m	392.877ms	50	50	100.00
		kmac_test_vectors_sha3_512	32.291m	51.334ms	47	50	94.00
		kmac_test_vectors_shake_128	1.927h	872.187ms	17	50	34.00
		kmac_test_vectors_shake_256	1.611h	74.660ms	20	50	40.00
		kmac_test_vectors_kmac	11.800s	1.713ms	49	50	98.00
		kmac_test_vectors_kmac_xof	8.340s	4.310ms	50	50	100.00
V2	sideload	kmac_sideload	9.902m	20.216ms	50	50	100.00
V2	app	kmac_app	7.640m	51.944ms	50	50	100.00
V2	app_with_partial_data	kmac_app_with_partial_data	8.769m	74.083ms	10	10	100.00
V2	entropy_refresh	kmac_entropy_refresh	7.521m	18.973ms	47	50	94.00
V2	error	kmac_error	8.950m	76.142ms	49	50	98.00
V2	key_error	kmac_key_error	16.370s	14.066ms	50	50	100.00
V2	edn_timeout_error	kmac_edn_timeout_error	50.980s	667.504us	20	20	100.00
V2	entropy_mode_error	kmac_entropy_mode_error	42.240s	1.800ms	20	20	100.00
V2	entropy_ready_error	kmac_entropy_ready_error	1.235m	26.633ms	10	10	100.00
V2	lc_escalation	kmac_lc_escalation	1.245m	4.256ms	50	50	100.00
V2	stress_all	kmac_stress_all	50.533m	1.278s	49	50	98.00
V2	intr_test	kmac_intr_test	0.890s	29.069us	50	50	100.00
V2	alert_test	kmac_alert_test	0.910s	65.613us	50	50	100.00
V2	tl_d_oob_addr_access	kmac_tl_errors	3.110s	201.862us	20	20	100.00
V2	tl_d_illegal_access	kmac_tl_errors	3.110s	201.862us	20	20	100.00
V2	tl_d_outstanding_access	kmac_csr_hw_reset	1.150s	107.977us	5	5	100.00
		kmac_csr_rw	1.260s	28.609us	20	20	100.00
		kmac_csr_aliasing	10.710s	2.144ms	5	5	100.00
		kmac_same_csr_outstanding	2.780s	110.972us	20	20	100.00
V2	tl_d_partial_access	kmac_csr_hw_reset	1.150s	107.977us	5	5	100.00
		kmac_csr_rw	1.260s	28.609us	20	20	100.00
		kmac_csr_aliasing	10.710s	2.144ms	5	5	100.00
		kmac_same_csr_outstanding	2.780s	110.972us	20	20	100.00
V2		TOTAL			954	1050	90.86
V2S	shadow_reg_update_error	kmac_shadow_reg_errors	1.590s	419.961us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	kmac_shadow_reg_errors	1.590s	419.961us	20	20	100.00
V2S	shadow_reg_storage_error	kmac_shadow_reg_errors	1.590s	419.961us	20	20	100.00
V2S	shadowed_reset_glitch	kmac_shadow_reg_errors	1.590s	419.961us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	kmac_shadow_reg_errors_with_csr_rw	3.120s	523.602us	20	20	100.00
V2S	tl_intg_err	kmac_sec_cm	1.827m	14.980ms	5	5	100.00
		kmac_tl_intg_err	5.150s	953.538us	20	20	100.00
V2S	sec_cm_bus_integrity	kmac_tl_intg_err	5.150s	953.538us	20	20	100.00
V2S	sec_cm_lc_escalate_en_intersig_mubi	kmac_lc_escalation	1.245m	4.256ms	50	50	100.00
V2S	sec_cm_sw_key_key_masking	kmac_smoke	1.572m	7.818ms	50	50	100.00
V2S	sec_cm_key_sideload	kmac_sideload	9.902m	20.216ms	50	50	100.00
V2S	sec_cm_cfg_shadowed_config_shadow	kmac_shadow_reg_errors	1.590s	419.961us	20	20	100.00
V2S	sec_cm_fsm_sparse	kmac_sec_cm	1.827m	14.980ms	5	5	100.00
V2S	sec_cm_ctr_redun	kmac_sec_cm	1.827m	14.980ms	5	5	100.00
V2S	sec_cm_packer_ctr_redun	kmac_sec_cm	1.827m	14.980ms	5	5	100.00
V2S	sec_cm_cfg_shadowed_config_regwen	kmac_smoke	1.572m	7.818ms	50	50	100.00
V2S	sec_cm_fsm_global_esc	kmac_lc_escalation	1.245m	4.256ms	50	50	100.00
V2S	sec_cm_fsm_local_esc	kmac_sec_cm	1.827m	14.980ms	5	5	100.00
V2S	sec_cm_absorbed_ctrl_mubi	kmac_mubi	5.711m	40.874ms	10	10	100.00
V2S	sec_cm_sw_cmd_ctrl_sparse	kmac_smoke	1.572m	7.818ms	50	50	100.00
V2S		TOTAL			75	75	100.00
V3	throughput	kmac_throughput			0	0	--
V3	stress_all_with_rand_reset	kmac_stress_all_with_rand_reset	48.516m	275.954ms	5	10	50.00
V3		TOTAL			5	10	50.00
		TOTAL			1149	1250	91.92

Testplan Progress

Items	Total	Written	Passing	Progress
V1	8	8	8	100.00
V2	25	25	15	60.00
V2S	5	5	5	100.00
V3	2	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
94.43	97.91	92.65	99.89	78.17	95.59	99.05	97.73

Failure Buckets

Job kmac_masked-sim-vcs_run_default killed due to: Exit reason: User job exceeded runlimit: User job timed out has 86 failures:
- Test kmac_test_vectors_sha3_256 has 4 failures.
  - 0.kmac_test_vectors_sha3_256.76211920596485356525337215639977617706711210134174596680701585660521851625140
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/0.kmac_test_vectors_sha3_256/latest/run.log
```
  Job ID: smart:5417c56c-2705-4f96-8bb6-eb178b458f0f
```
  - 19.kmac_test_vectors_sha3_256.55239569940583245677145336618000912024593269276290952632107031535182698340561
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/19.kmac_test_vectors_sha3_256/latest/run.log
```
  Job ID: smart:d48cf727-8efc-40da-a998-31b547e78003
```
  - ... and 2 more failures.
- Test kmac_test_vectors_shake_256 has 30 failures.
  - 0.kmac_test_vectors_shake_256.43611730625352790184022269839630006126827248696987740817788995154443801459880
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/0.kmac_test_vectors_shake_256/latest/run.log
```
  Job ID: smart:7bcba32e-f4b0-4e78-b2b5-8aa6aa07db40
```
  - 5.kmac_test_vectors_shake_256.38874565160444128943946355755028178369953424949833166437194422948596708292932
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/5.kmac_test_vectors_shake_256/latest/run.log
```
  Job ID: smart:5fbd27b2-2b7a-40d6-82ea-7f087a07f7f0
```
  - ... and 28 more failures.
- Test kmac_long_msg_and_output has 9 failures.
  - 1.kmac_long_msg_and_output.103758532822959300869079759184737611956213613336441204885236039612115241164085
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/1.kmac_long_msg_and_output/latest/run.log
```
  Job ID: smart:55c92c1c-73fe-4f32-a6bf-50e36a36ef97
```
  - 7.kmac_long_msg_and_output.65623801213179997641292275496005349539309557951677131325626359360943789947959
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/7.kmac_long_msg_and_output/latest/run.log
```
  Job ID: smart:669bd668-bdab-48f3-a6a6-2456154b0ad5
```
  - ... and 7 more failures.
- Test kmac_test_vectors_shake_128 has 33 failures.
  - 1.kmac_test_vectors_shake_128.46840134296443284427521581772124776380345654333632580364433631569628794455925
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/1.kmac_test_vectors_shake_128/latest/run.log
```
  Job ID: smart:d738c232-0f41-49dd-8563-2d5e84932908
```
  - 2.kmac_test_vectors_shake_128.57150479343641339961506138490052438966445734862542779137678930304968067031694
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/2.kmac_test_vectors_shake_128/latest/run.log
```
  Job ID: smart:ff26eb15-9338-4f10-8227-92c322d7b6a2
```
  - ... and 31 more failures.
- Test kmac_test_vectors_sha3_224 has 10 failures.
  - 8.kmac_test_vectors_sha3_224.51215380018847325709936081930772184389395205931911603540846648463031877602523
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/8.kmac_test_vectors_sha3_224/latest/run.log
```
  Job ID: smart:b4c14a5c-0742-49f3-84d1-3b0f11dfe7fb
```
  - 10.kmac_test_vectors_sha3_224.88790122439678748762892125319724822234919724402931115483421251462100108411784
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/10.kmac_test_vectors_sha3_224/latest/run.log
```
  Job ID: smart:5ac46f74-a530-4561-9bec-b680aaafab64
```
  - ... and 8 more failures.
UVM_ERROR (csr_utils_pkg.sv:476) [csr_utils::csr_rd_check] Check failed obs == exp (* [*] vs * [*]) Regname: kmac_reg_block.intr_state.kmac_done reset value: * has 5 failures:
- Test kmac_test_vectors_sha3_512 has 3 failures.
  - 2.kmac_test_vectors_sha3_512.29228174975521134820772390769314261037617684426549837157314936926400045976790
    Line 253, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/2.kmac_test_vectors_sha3_512/latest/run.log
```
  UVM_ERROR @  40148134 ps: (csr_utils_pkg.sv:476) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @  40148134 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 13.kmac_test_vectors_sha3_512.90558623095171381201252271928355216563655255419754676151612110979882004622872
    Line 253, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/13.kmac_test_vectors_sha3_512/latest/run.log
```
  UVM_ERROR @  59044442 ps: (csr_utils_pkg.sv:476) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @  59044442 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 1 more failures.
- Test kmac_test_vectors_kmac has 1 failures.
  - 15.kmac_test_vectors_kmac.97610493500201203465980363312743806621694356780705770075171458013099438481008
    Line 253, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/15.kmac_test_vectors_kmac/latest/run.log
```
  UVM_ERROR @  57092354 ps: (csr_utils_pkg.sv:476) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @  57092354 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test kmac_test_vectors_sha3_224 has 1 failures.
  - 38.kmac_test_vectors_sha3_224.109873849951291543751575749954533542430433821190315299173757003384405669258691
    Line 253, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/38.kmac_test_vectors_sha3_224/latest/run.log
```
  UVM_ERROR @ 125214723 ps: (csr_utils_pkg.sv:476) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @ 125214723 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (cip_base_vseq.sv:839) [kmac_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 4 failures:
- Test kmac_stress_all_with_rand_reset has 4 failures.
  - 0.kmac_stress_all_with_rand_reset.18391060694416047496278500770895996824486942483884855090576314661658239115511
    Line 281, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/0.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 1996556145 ps: (cip_base_vseq.sv:839) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 1996556145 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 1.kmac_stress_all_with_rand_reset.103729403964758534855925200452780898082384468079870495077601918746034087929665
    Line 363, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/1.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 7903270841 ps: (cip_base_vseq.sv:839) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 7903270841 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 2 more failures.

UVM_FATAL (kmac_scoreboard.sv:1638) [scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (* [*] vs * [*]) Mismatch between unmasked_digest[*] and dpi_digest[*] has 4 failures:

Test kmac_entropy_refresh has 3 failures.

10.kmac_entropy_refresh.26740388349593690657554815517439613545072639911257274675071456976134623960971
Line 999, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/10.kmac_entropy_refresh/latest/run.log

  UVM_FATAL @ 40343133191 ps: (kmac_scoreboard.sv:1638) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (108 [0x6c] vs 36 [0x24]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 40343133191 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

22.kmac_entropy_refresh.35275804043860134391448779090872983993953113794818451199552239965390964633205
Line 309, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/22.kmac_entropy_refresh/latest/run.log

  UVM_FATAL @ 1243434473 ps: (kmac_scoreboard.sv:1638) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (52 [0x34] vs 185 [0xb9]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 1243434473 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 1 more failures.

Test kmac_stress_all has 1 failures.

37.kmac_stress_all.50208937128902734210646343513150957376687619810341856997364091640312154128795
Line 455, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/37.kmac_stress_all/latest/run.log

  UVM_FATAL @ 3530786897 ps: (kmac_scoreboard.sv:1638) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (8 [0x8] vs 183 [0xb7]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 3530786897 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_ERROR (kmac_scoreboard.sv:1188) [scoreboard] Check failed csr.get_mirrored_value() == item.d_data (* [*] vs * [*]) reg name: kmac_reg_block.err_code has 1 failures:
- Test kmac_stress_all_with_rand_reset has 1 failures.
  - 4.kmac_stress_all_with_rand_reset.2391983567401468687388787120255638058541468482636559911084858336596638354078
    Line 695, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/4.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 62416853012 ps: (kmac_scoreboard.sv:1188) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (0 [0x0] vs 2147483768 [0x80000078]) reg name: kmac_reg_block.err_code
  UVM_INFO @ 62416853012 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_FATAL (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of * ps hit, indicating a probable testbench issue has 1 failures:
- Test kmac_error has 1 failures.
  - 25.kmac_error.83311466018849939661711052660687207519304991643833225165168711163202588213061
    Line 1012, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/25.kmac_error/latest/run.log
```
  UVM_FATAL @ 200000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 200000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 200000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```