KMAC/MASKED Simulation Results

Sunday July 07 2024 23:02:38 UTC

GitHub Revision: `2e5d86c9b5`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 75221189197949424635294305394615322888112457483844341597147780944629972574676

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	kmac_smoke	1.622m	17.521ms	50	50	100.00
V1	csr_hw_reset	kmac_csr_hw_reset	1.230s	61.969us	5	5	100.00
V1	csr_rw	kmac_csr_rw	1.250s	111.726us	20	20	100.00
V1	csr_bit_bash	kmac_csr_bit_bash	20.860s	8.002ms	5	5	100.00
V1	csr_aliasing	kmac_csr_aliasing	8.240s	705.185us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	kmac_csr_mem_rw_with_rand_reset	2.770s	186.585us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	kmac_csr_rw	1.250s	111.726us	20	20	100.00
		kmac_csr_aliasing	8.240s	705.185us	5	5	100.00
V1	mem_walk	kmac_mem_walk	0.820s	15.174us	5	5	100.00
V1	mem_partial_access	kmac_mem_partial_access	1.550s	45.099us	5	5	100.00
V1		TOTAL			115	115	100.00
V2	long_msg_and_output	kmac_long_msg_and_output	58.889m	557.223ms	48	50	96.00
V2	burst_write	kmac_burst_write	27.772m	14.547ms	50	50	100.00
V2	test_vectors	kmac_test_vectors_sha3_224	46.168m	977.673ms	49	50	98.00
		kmac_test_vectors_sha3_256	41.741m	604.040ms	50	50	100.00
		kmac_test_vectors_sha3_384	33.081m	458.723ms	50	50	100.00
		kmac_test_vectors_sha3_512	24.484m	259.416ms	50	50	100.00
		kmac_test_vectors_shake_128	1.893h	1.362s	49	50	98.00
		kmac_test_vectors_shake_256	1.631h	559.898ms	50	50	100.00
		kmac_test_vectors_kmac	7.150s	739.896us	50	50	100.00
		kmac_test_vectors_kmac_xof	7.140s	3.904ms	48	50	96.00
V2	sideload	kmac_sideload	8.889m	21.184ms	50	50	100.00
V2	app	kmac_app	7.335m	62.456ms	49	50	98.00
V2	app_with_partial_data	kmac_app_with_partial_data	6.054m	138.490ms	10	10	100.00
V2	entropy_refresh	kmac_entropy_refresh	7.174m	33.981ms	47	50	94.00
V2	error	kmac_error	8.227m	26.773ms	48	50	96.00
V2	key_error	kmac_key_error	13.000s	2.072ms	50	50	100.00
V2	edn_timeout_error	kmac_edn_timeout_error	57.850s	1.594ms	20	20	100.00
V2	entropy_mode_error	kmac_entropy_mode_error	45.530s	2.040ms	20	20	100.00
V2	entropy_ready_error	kmac_entropy_ready_error	1.269m	14.178ms	10	10	100.00
V2	lc_escalation	kmac_lc_escalation	1.054m	13.714ms	50	50	100.00
V2	stress_all	kmac_stress_all	1.515h	401.635ms	46	50	92.00
V2	intr_test	kmac_intr_test	0.880s	15.013us	50	50	100.00
V2	alert_test	kmac_alert_test	0.970s	21.745us	50	50	100.00
V2	tl_d_oob_addr_access	kmac_tl_errors	3.470s	975.802us	20	20	100.00
V2	tl_d_illegal_access	kmac_tl_errors	3.470s	975.802us	20	20	100.00
V2	tl_d_outstanding_access	kmac_csr_hw_reset	1.230s	61.969us	5	5	100.00
		kmac_csr_rw	1.250s	111.726us	20	20	100.00
		kmac_csr_aliasing	8.240s	705.185us	5	5	100.00
		kmac_same_csr_outstanding	3.010s	257.878us	20	20	100.00
V2	tl_d_partial_access	kmac_csr_hw_reset	1.230s	61.969us	5	5	100.00
		kmac_csr_rw	1.250s	111.726us	20	20	100.00
		kmac_csr_aliasing	8.240s	705.185us	5	5	100.00
		kmac_same_csr_outstanding	3.010s	257.878us	20	20	100.00
V2		TOTAL			1034	1050	98.48
V2S	shadow_reg_update_error	kmac_shadow_reg_errors	1.440s	77.391us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	kmac_shadow_reg_errors	1.440s	77.391us	20	20	100.00
V2S	shadow_reg_storage_error	kmac_shadow_reg_errors	1.440s	77.391us	20	20	100.00
V2S	shadowed_reset_glitch	kmac_shadow_reg_errors	1.440s	77.391us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	kmac_shadow_reg_errors_with_csr_rw	3.310s	406.730us	20	20	100.00
V2S	tl_intg_err	kmac_sec_cm	1.485m	8.327ms	5	5	100.00
		kmac_tl_intg_err	5.420s	497.374us	20	20	100.00
V2S	sec_cm_bus_integrity	kmac_tl_intg_err	5.420s	497.374us	20	20	100.00
V2S	sec_cm_lc_escalate_en_intersig_mubi	kmac_lc_escalation	1.054m	13.714ms	50	50	100.00
V2S	sec_cm_sw_key_key_masking	kmac_smoke	1.622m	17.521ms	50	50	100.00
V2S	sec_cm_key_sideload	kmac_sideload	8.889m	21.184ms	50	50	100.00
V2S	sec_cm_cfg_shadowed_config_shadow	kmac_shadow_reg_errors	1.440s	77.391us	20	20	100.00
V2S	sec_cm_fsm_sparse	kmac_sec_cm	1.485m	8.327ms	5	5	100.00
V2S	sec_cm_ctr_redun	kmac_sec_cm	1.485m	8.327ms	5	5	100.00
V2S	sec_cm_packer_ctr_redun	kmac_sec_cm	1.485m	8.327ms	5	5	100.00
V2S	sec_cm_cfg_shadowed_config_regwen	kmac_smoke	1.622m	17.521ms	50	50	100.00
V2S	sec_cm_fsm_global_esc	kmac_lc_escalation	1.054m	13.714ms	50	50	100.00
V2S	sec_cm_fsm_local_esc	kmac_sec_cm	1.485m	8.327ms	5	5	100.00
V2S	sec_cm_absorbed_ctrl_mubi	kmac_mubi	6.036m	28.219ms	10	10	100.00
V2S	sec_cm_sw_cmd_ctrl_sparse	kmac_smoke	1.622m	17.521ms	50	50	100.00
V2S		TOTAL			75	75	100.00
V3	throughput	kmac_throughput			0	0	--
V3	stress_all_with_rand_reset	kmac_stress_all_with_rand_reset	1.003h	425.898ms	5	10	50.00
V3		TOTAL			5	10	50.00
		TOTAL			1229	1250	98.32

Testplan Progress

Items	Total	Written	Passing	Progress
V1	8	8	8	100.00
V2	25	25	17	68.00
V2S	5	5	5	100.00
V3	2	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
94.24	97.91	92.62	99.89	76.76	95.59	99.05	97.88

Failure Buckets

UVM_FATAL (kmac_scoreboard.sv:1519) [scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (* [*] vs * [*]) Mismatch between unmasked_digest[*] and dpi_digest[*] has 6 failures:

Test kmac_stress_all has 3 failures.

5.kmac_stress_all.58574484276376406577680115736064950790587955723768015450293235005532737690516
Line 597, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/5.kmac_stress_all/latest/run.log

  UVM_FATAL @ 43876906022 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (233 [0xe9] vs 206 [0xce]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 43876906022 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

7.kmac_stress_all.35156957375044401367678344201623636326054010097552842189703492524815861529065
Line 1479, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/7.kmac_stress_all/latest/run.log

  UVM_FATAL @ 32802611358 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (138 [0x8a] vs 213 [0xd5]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 32802611358 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 1 more failures.

Test kmac_entropy_refresh has 2 failures.

9.kmac_entropy_refresh.102020070521307875343603830327914930282219809126911224820371892233634198114187
Line 413, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/9.kmac_entropy_refresh/latest/run.log

  UVM_FATAL @ 3348273156 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (144 [0x90] vs 114 [0x72]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 3348273156 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

42.kmac_entropy_refresh.62733758018362343516366029545706739512930186669352123158351335332293851689798
Line 355, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/42.kmac_entropy_refresh/latest/run.log

  UVM_FATAL @ 9392115871 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (56 [0x38] vs 231 [0xe7]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 9392115871 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test kmac_app has 1 failures.

34.kmac_app.54617389677907965169443242570365358275597976077586198729493345664737749813634
Line 509, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/34.kmac_app/latest/run.log

  UVM_FATAL @ 7353028031 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (154 [0x9a] vs 30 [0x1e]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 7353028031 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_ERROR (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (* [*] vs * [*]) Regname: kmac_reg_block.intr_state.kmac_done reset value: * has 5 failures:
- Test kmac_test_vectors_kmac_xof has 2 failures.
  - 0.kmac_test_vectors_kmac_xof.42690302719141609160036856224886329640146019885248253932343656475636295948706
    Line 253, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/0.kmac_test_vectors_kmac_xof/latest/run.log
```
  UVM_ERROR @  93266562 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @  93266562 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 12.kmac_test_vectors_kmac_xof.19062256224784567008023385223814186392010094942987044388277209997182877772880
    Line 253, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/12.kmac_test_vectors_kmac_xof/latest/run.log
```
  UVM_ERROR @ 193036116 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @ 193036116 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test kmac_entropy_refresh has 1 failures.
  - 19.kmac_entropy_refresh.7402720202039668119452955485049263996356993359829225212458279442801422463577
    Line 254, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/19.kmac_entropy_refresh/latest/run.log
```
  UVM_ERROR @  81439174 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @  81439174 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test kmac_test_vectors_shake_128 has 1 failures.
  - 29.kmac_test_vectors_shake_128.32386149244584907718858578550100154542887930506736498389600670947256470013824
    Line 253, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/29.kmac_test_vectors_shake_128/latest/run.log
```
  UVM_ERROR @  96467799 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @  96467799 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test kmac_test_vectors_sha3_224 has 1 failures.
  - 47.kmac_test_vectors_sha3_224.49627263039659001217538527930366021306732262634717228709251844230079961815003
    Line 253, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/47.kmac_test_vectors_sha3_224/latest/run.log
```
  UVM_ERROR @  55626553 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @  55626553 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_FATAL (kmac_scoreboard.sv:218) [scoreboard] Wait EDN request has 4 failures:
- Test kmac_stress_all_with_rand_reset has 1 failures.
  - 8.kmac_stress_all_with_rand_reset.108804904807635217177054662456132278125600055569146686804147234131663648919151
    Line 659, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/8.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_FATAL @ 17811892149 ps: (kmac_scoreboard.sv:218) [uvm_test_top.env.scoreboard] Wait EDN request
  UVM_INFO @ 17811892149 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test kmac_error has 2 failures.
  - 25.kmac_error.49959639612794525560403234753095620206379201018458265266970686876667652914242
    Line 407, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/25.kmac_error/latest/run.log
```
  UVM_FATAL @ 10139123050 ps: (kmac_scoreboard.sv:218) [uvm_test_top.env.scoreboard] Wait EDN request
  UVM_INFO @ 10139123050 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 39.kmac_error.27090313073507707520329327340669126482518503501349059702599672777485452702797
    Line 725, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/39.kmac_error/latest/run.log
```
  UVM_FATAL @ 10040918400 ps: (kmac_scoreboard.sv:218) [uvm_test_top.env.scoreboard] Wait EDN request
  UVM_INFO @ 10040918400 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test kmac_stress_all has 1 failures.
  - 32.kmac_stress_all.102272465507350399264921621731215122579369559445842463156675893240394980432962
    Line 1546, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/32.kmac_stress_all/latest/run.log
```
  UVM_FATAL @ 139122815407 ps: (kmac_scoreboard.sv:218) [uvm_test_top.env.scoreboard] Wait EDN request
  UVM_INFO @ 139122815407 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (kmac_scoreboard.sv:1175) [scoreboard] Check failed csr.get_mirrored_value() == item.d_data (* [*] vs * [*]) reg name: kmac_reg_block.err_code has 2 failures:
- Test kmac_stress_all_with_rand_reset has 2 failures.
  - 1.kmac_stress_all_with_rand_reset.6828813209678997836592907545477770992600645525770048328371969794708072217018
    Line 448, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/1.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 11101376541 ps: (kmac_scoreboard.sv:1175) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (0 [0x0] vs 2147483760 [0x80000070]) reg name: kmac_reg_block.err_code
  UVM_INFO @ 11101376541 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 9.kmac_stress_all_with_rand_reset.90327383758818789858998117510466827433825433169330680068085619479697407866423
    Line 596, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/9.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 93165179506 ps: (kmac_scoreboard.sv:1175) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (0 [0x0] vs 2147483760 [0x80000070]) reg name: kmac_reg_block.err_code
  UVM_INFO @ 93165179506 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (cip_base_vseq.sv:825) [kmac_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 2 failures:
- Test kmac_stress_all_with_rand_reset has 2 failures.
  - 3.kmac_stress_all_with_rand_reset.92563095399507159681295337669360003039238836420600952468556393640697352629370
    Line 620, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/3.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 26579926482 ps: (cip_base_vseq.sv:825) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 26579926482 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 6.kmac_stress_all_with_rand_reset.44856059506140681787809320274083654217670613425196648093310466963261070330150
    Line 1747, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/6.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 204749808589 ps: (cip_base_vseq.sv:825) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 204749808589 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
Job kmac_masked-sim-vcs_run_default killed due to: Exit reason: User job exceeded runlimit: User job timed out has 2 failures:
- Test kmac_long_msg_and_output has 2 failures.
  - 10.kmac_long_msg_and_output.100163364175126721403349536257124194819779709361474155850471337666675736858022
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/10.kmac_long_msg_and_output/latest/run.log
```
  Job ID: smart:27c01f03-67fe-4954-8e16-da365f5bc663
```
  - 33.kmac_long_msg_and_output.34495471326602594812276080578108283449420572649390205161891201079691923511261
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/33.kmac_long_msg_and_output/latest/run.log
```
  Job ID: smart:6742eb11-1987-4588-9f1a-f649aa5ff39e
```