KMAC/UNMASKED Simulation Results

Saturday July 27 2024 23:02:25 UTC

GitHub Revision: `eca25c0ff8`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 6528518538521148567139195500524222710943459299328477504124649113671643189924

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	kmac_smoke	1.038m	2.666ms	50	50	100.00
V1	csr_hw_reset	kmac_csr_hw_reset	1.100s	144.057us	5	5	100.00
V1	csr_rw	kmac_csr_rw	1.110s	99.431us	20	20	100.00
V1	csr_bit_bash	kmac_csr_bit_bash	19.460s	1.439ms	5	5	100.00
V1	csr_aliasing	kmac_csr_aliasing	9.610s	2.717ms	5	5	100.00
V1	csr_mem_rw_with_rand_reset	kmac_csr_mem_rw_with_rand_reset	2.470s	75.253us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	kmac_csr_rw	1.110s	99.431us	20	20	100.00
		kmac_csr_aliasing	9.610s	2.717ms	5	5	100.00
V1	mem_walk	kmac_mem_walk	0.710s	16.002us	5	5	100.00
V1	mem_partial_access	kmac_mem_partial_access	1.320s	44.154us	5	5	100.00
V1		TOTAL			115	115	100.00
V2	long_msg_and_output	kmac_long_msg_and_output	56.996m	64.131ms	40	50	80.00
V2	burst_write	kmac_burst_write	18.605m	157.184ms	50	50	100.00
V2	test_vectors	kmac_test_vectors_sha3_224	58.578m	402.338ms	49	50	98.00
		kmac_test_vectors_sha3_256	57.949m	1.277s	49	50	98.00
		kmac_test_vectors_sha3_384	40.924m	72.016ms	50	50	100.00
		kmac_test_vectors_sha3_512	26.031m	94.506ms	50	50	100.00
		kmac_test_vectors_shake_128	1.691h	203.355ms	20	50	40.00
		kmac_test_vectors_shake_256	1.375h	172.924ms	18	50	36.00
		kmac_test_vectors_kmac	5.770s	1.047ms	50	50	100.00
		kmac_test_vectors_kmac_xof	7.730s	5.104ms	50	50	100.00
V2	sideload	kmac_sideload	8.457m	20.987ms	50	50	100.00
V2	app	kmac_app	6.488m	18.297ms	49	50	98.00
V2	app_with_partial_data	kmac_app_with_partial_data	5.318m	42.458ms	10	10	100.00
V2	entropy_refresh	kmac_entropy_refresh	6.007m	114.636ms	48	50	96.00
V2	error	kmac_error	7.667m	23.489ms	50	50	100.00
V2	key_error	kmac_key_error	8.570s	12.262ms	50	50	100.00
V2	edn_timeout_error	kmac_edn_timeout_error	50.450s	19.272ms	20	20	100.00
V2	entropy_mode_error	kmac_entropy_mode_error	38.180s	2.166ms	20	20	100.00
V2	entropy_ready_error	kmac_entropy_ready_error	55.560s	7.996ms	10	10	100.00
V2	lc_escalation	kmac_lc_escalation	57.200s	3.437ms	50	50	100.00
V2	stress_all	kmac_stress_all	58.762m	392.850ms	50	50	100.00
V2	intr_test	kmac_intr_test	0.850s	21.096us	50	50	100.00
V2	alert_test	kmac_alert_test	0.840s	94.376us	50	50	100.00
V2	tl_d_oob_addr_access	kmac_tl_errors	3.160s	2.160ms	20	20	100.00
V2	tl_d_illegal_access	kmac_tl_errors	3.160s	2.160ms	20	20	100.00
V2	tl_d_outstanding_access	kmac_csr_hw_reset	1.100s	144.057us	5	5	100.00
		kmac_csr_rw	1.110s	99.431us	20	20	100.00
		kmac_csr_aliasing	9.610s	2.717ms	5	5	100.00
		kmac_same_csr_outstanding	2.600s	125.758us	20	20	100.00
V2	tl_d_partial_access	kmac_csr_hw_reset	1.100s	144.057us	5	5	100.00
		kmac_csr_rw	1.110s	99.431us	20	20	100.00
		kmac_csr_aliasing	9.610s	2.717ms	5	5	100.00
		kmac_same_csr_outstanding	2.600s	125.758us	20	20	100.00
V2		TOTAL			973	1050	92.67
V2S	shadow_reg_update_error	kmac_shadow_reg_errors	1.340s	42.239us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	kmac_shadow_reg_errors	1.340s	42.239us	20	20	100.00
V2S	shadow_reg_storage_error	kmac_shadow_reg_errors	1.340s	42.239us	20	20	100.00
V2S	shadowed_reset_glitch	kmac_shadow_reg_errors	1.340s	42.239us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	kmac_shadow_reg_errors_with_csr_rw	2.910s	922.997us	20	20	100.00
V2S	tl_intg_err	kmac_sec_cm	1.151m	35.611ms	5	5	100.00
		kmac_tl_intg_err	5.480s	4.001ms	20	20	100.00
V2S	sec_cm_bus_integrity	kmac_tl_intg_err	5.480s	4.001ms	20	20	100.00
V2S	sec_cm_lc_escalate_en_intersig_mubi	kmac_lc_escalation	57.200s	3.437ms	50	50	100.00
V2S	sec_cm_sw_key_key_masking	kmac_smoke	1.038m	2.666ms	50	50	100.00
V2S	sec_cm_key_sideload	kmac_sideload	8.457m	20.987ms	50	50	100.00
V2S	sec_cm_cfg_shadowed_config_shadow	kmac_shadow_reg_errors	1.340s	42.239us	20	20	100.00
V2S	sec_cm_fsm_sparse	kmac_sec_cm	1.151m	35.611ms	5	5	100.00
V2S	sec_cm_ctr_redun	kmac_sec_cm	1.151m	35.611ms	5	5	100.00
V2S	sec_cm_packer_ctr_redun	kmac_sec_cm	1.151m	35.611ms	5	5	100.00
V2S	sec_cm_cfg_shadowed_config_regwen	kmac_smoke	1.038m	2.666ms	50	50	100.00
V2S	sec_cm_fsm_global_esc	kmac_lc_escalation	57.200s	3.437ms	50	50	100.00
V2S	sec_cm_fsm_local_esc	kmac_sec_cm	1.151m	35.611ms	5	5	100.00
V2S	sec_cm_absorbed_ctrl_mubi	kmac_mubi	5.272m	12.174ms	10	10	100.00
V2S	sec_cm_sw_cmd_ctrl_sparse	kmac_smoke	1.038m	2.666ms	50	50	100.00
V2S		TOTAL			75	75	100.00
V3	throughput	kmac_throughput			0	0	--
V3	stress_all_with_rand_reset	kmac_stress_all_with_rand_reset	1.270h	54.208ms	3	10	30.00
V3		TOTAL			3	10	30.00
		TOTAL			1166	1250	93.28

Testplan Progress

Items	Total	Written	Passing	Progress
V1	8	8	8	100.00
V2	25	25	18	72.00
V2S	5	5	5	100.00
V3	2	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
92.03	95.89	92.27	100.00	66.94	94.11	98.84	96.15

Failure Buckets

Job kmac_unmasked-sim-vcs_run_default killed due to: Exit reason: User job exceeded runlimit: User job timed out has 74 failures:
- Test kmac_long_msg_and_output has 10 failures.
  - 0.kmac_long_msg_and_output.20398070520035588354737762817351473682974847446937386495439896028409166333609
    Log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/0.kmac_long_msg_and_output/latest/run.log
```
  Job ID: smart:cd9e9a39-2deb-4191-8bc0-c2a3cb4cebe1
```
  - 8.kmac_long_msg_and_output.65657652470520914534889886066917000400207284650995450921523317634565406869705
    Log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/8.kmac_long_msg_and_output/latest/run.log
```
  Job ID: smart:20d87954-86e8-4320-b43b-a293b58a98ac
```
  - ... and 8 more failures.
- Test kmac_test_vectors_shake_128 has 30 failures.
  - 1.kmac_test_vectors_shake_128.50588826619143646066646338604669504747073954453143565632313701112309002695217
    Log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/1.kmac_test_vectors_shake_128/latest/run.log
```
  Job ID: smart:0683b966-3e98-4c09-ad17-e3784e6f6021
```
  - 5.kmac_test_vectors_shake_128.79102806167814605974048720904860830317286149617876529721081895554794413134601
    Log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/5.kmac_test_vectors_shake_128/latest/run.log
```
  Job ID: smart:24682a5d-a94a-4375-952e-0ba921204f0b
```
  - ... and 28 more failures.
- Test kmac_test_vectors_shake_256 has 32 failures.
  - 1.kmac_test_vectors_shake_256.21544393561782913395661273733515080249337992391701492065058588483421043490526
    Log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/1.kmac_test_vectors_shake_256/latest/run.log
```
  Job ID: smart:49cf1f09-ce5f-43f4-b1e2-bcfb26d4bdf7
```
  - 4.kmac_test_vectors_shake_256.61864593530507517751877275456523279301444631670341416035906862865139316346264
    Log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/4.kmac_test_vectors_shake_256/latest/run.log
```
  Job ID: smart:fd6ab0f1-7942-49eb-8897-3691c7bf146a
```
  - ... and 30 more failures.
- Test kmac_test_vectors_sha3_256 has 1 failures.
  - 39.kmac_test_vectors_sha3_256.7629414819515300520956741553040228476023515034489283123651630327156162635087
    Log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/39.kmac_test_vectors_sha3_256/latest/run.log
```
  Job ID: smart:afc79878-7e9a-49ba-8ead-b41c5fbbf5eb
```
- Test kmac_test_vectors_sha3_224 has 1 failures.
  - 47.kmac_test_vectors_sha3_224.30064984234263192688189724794308114379835952974308926199890145167405061875615
    Log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/47.kmac_test_vectors_sha3_224/latest/run.log
```
  Job ID: smart:f4448a9d-c302-4f25-b9a3-5ae3e1e15802
```
UVM_ERROR (cip_base_vseq.sv:839) [kmac_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 6 failures:
- Test kmac_stress_all_with_rand_reset has 6 failures.
  - 0.kmac_stress_all_with_rand_reset.103890592901492386624726636692588472383806699546868424057190795097425192631594
    Line 1204, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/0.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 170274293100 ps: (cip_base_vseq.sv:839) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 170274293100 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 1.kmac_stress_all_with_rand_reset.88196388097717968219663260084268661786278787909266421671552345796545084903117
    Line 789, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/1.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 6333819237 ps: (cip_base_vseq.sv:839) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 6333819237 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 4 more failures.

UVM_FATAL (kmac_scoreboard.sv:1638) [scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (* [*] vs * [*]) Mismatch between unmasked_digest[*] and dpi_digest[*] has 3 failures:

Test kmac_app has 1 failures.

34.kmac_app.78770930066319364414791595764697035903056295378986557003908066942504253612736
Line 513, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/34.kmac_app/latest/run.log

  UVM_FATAL @ 14017540043 ps: (kmac_scoreboard.sv:1638) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (131 [0x83] vs 173 [0xad]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 14017540043 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test kmac_entropy_refresh has 2 failures.

38.kmac_entropy_refresh.15376643646695365246197046807349139818141397318307290462315465034611016167080
Line 459, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/38.kmac_entropy_refresh/latest/run.log

  UVM_FATAL @ 34790997475 ps: (kmac_scoreboard.sv:1638) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (174 [0xae] vs 153 [0x99]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 34790997475 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

44.kmac_entropy_refresh.20503756580276979997607202860542047883430111279455291600318185356728329451847
Line 473, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/44.kmac_entropy_refresh/latest/run.log

  UVM_FATAL @ 53886257290 ps: (kmac_scoreboard.sv:1638) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (28 [0x1c] vs 245 [0xf5]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 53886257290 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_ERROR (kmac_scoreboard.sv:1188) [scoreboard] Check failed csr.get_mirrored_value() == item.d_data (* [*] vs * [*]) reg name: kmac_reg_block.err_code has 1 failures:
- Test kmac_stress_all_with_rand_reset has 1 failures.
  - 6.kmac_stress_all_with_rand_reset.65177905013165543702447436943856237104657219780874576134469252794780677567125
    Line 1158, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/6.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 20980640390 ps: (kmac_scoreboard.sv:1188) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (0 [0x0] vs 2147483744 [0x80000060]) reg name: kmac_reg_block.err_code
  UVM_INFO @ 20980640390 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```