KMAC/MASKED Simulation Results

Tuesday July 30 2024 23:02:08 UTC

GitHub Revision: `fdfa12db04`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 101467584611478134588291649782725219255540557286164709436567235390830780957271

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	kmac_smoke	1.486m	6.876ms	50	50	100.00
V1	csr_hw_reset	kmac_csr_hw_reset	1.220s	63.278us	5	5	100.00
V1	csr_rw	kmac_csr_rw	1.230s	34.361us	20	20	100.00
V1	csr_bit_bash	kmac_csr_bit_bash	20.690s	973.009us	5	5	100.00
V1	csr_aliasing	kmac_csr_aliasing	9.360s	401.812us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	kmac_csr_mem_rw_with_rand_reset	2.680s	308.807us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	kmac_csr_rw	1.230s	34.361us	20	20	100.00
		kmac_csr_aliasing	9.360s	401.812us	5	5	100.00
V1	mem_walk	kmac_mem_walk	0.770s	15.621us	5	5	100.00
V1	mem_partial_access	kmac_mem_partial_access	1.530s	37.797us	5	5	100.00
V1		TOTAL			115	115	100.00
V2	long_msg_and_output	kmac_long_msg_and_output	58.733m	94.581ms	33	50	66.00
V2	burst_write	kmac_burst_write	28.635m	30.940ms	50	50	100.00
V2	test_vectors	kmac_test_vectors_sha3_224	58.954m	632.908ms	40	50	80.00
		kmac_test_vectors_sha3_256	59.856m	356.080ms	44	50	88.00
		kmac_test_vectors_sha3_384	48.672m	143.544ms	49	50	98.00
		kmac_test_vectors_sha3_512	31.985m	50.412ms	50	50	100.00
		kmac_test_vectors_shake_128	1.915h	212.832ms	16	50	32.00
		kmac_test_vectors_shake_256	1.687h	221.576ms	26	50	52.00
		kmac_test_vectors_kmac	7.900s	1.873ms	50	50	100.00
		kmac_test_vectors_kmac_xof	8.090s	3.206ms	50	50	100.00
V2	sideload	kmac_sideload	11.463m	256.470ms	50	50	100.00
V2	app	kmac_app	8.267m	40.043ms	49	50	98.00
V2	app_with_partial_data	kmac_app_with_partial_data	5.950m	15.847ms	10	10	100.00
V2	entropy_refresh	kmac_entropy_refresh	8.375m	178.059ms	48	50	96.00
V2	error	kmac_error	10.614m	20.738ms	50	50	100.00
V2	key_error	kmac_key_error	13.990s	1.945ms	50	50	100.00
V2	edn_timeout_error	kmac_edn_timeout_error	54.430s	8.076ms	20	20	100.00
V2	entropy_mode_error	kmac_entropy_mode_error	53.160s	3.365ms	20	20	100.00
V2	entropy_ready_error	kmac_entropy_ready_error	1.129m	48.636ms	10	10	100.00
V2	lc_escalation	kmac_lc_escalation	43.270s	1.546ms	50	50	100.00
V2	stress_all	kmac_stress_all	49.683m	70.261ms	49	50	98.00
V2	intr_test	kmac_intr_test	0.890s	38.621us	50	50	100.00
V2	alert_test	kmac_alert_test	0.900s	18.618us	50	50	100.00
V2	tl_d_oob_addr_access	kmac_tl_errors	3.910s	581.883us	20	20	100.00
V2	tl_d_illegal_access	kmac_tl_errors	3.910s	581.883us	20	20	100.00
V2	tl_d_outstanding_access	kmac_csr_hw_reset	1.220s	63.278us	5	5	100.00
		kmac_csr_rw	1.230s	34.361us	20	20	100.00
		kmac_csr_aliasing	9.360s	401.812us	5	5	100.00
		kmac_same_csr_outstanding	2.790s	402.314us	20	20	100.00
V2	tl_d_partial_access	kmac_csr_hw_reset	1.220s	63.278us	5	5	100.00
		kmac_csr_rw	1.230s	34.361us	20	20	100.00
		kmac_csr_aliasing	9.360s	401.812us	5	5	100.00
		kmac_same_csr_outstanding	2.790s	402.314us	20	20	100.00
V2		TOTAL			954	1050	90.86
V2S	shadow_reg_update_error	kmac_shadow_reg_errors	1.560s	102.715us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	kmac_shadow_reg_errors	1.560s	102.715us	20	20	100.00
V2S	shadow_reg_storage_error	kmac_shadow_reg_errors	1.560s	102.715us	20	20	100.00
V2S	shadowed_reset_glitch	kmac_shadow_reg_errors	1.560s	102.715us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	kmac_shadow_reg_errors_with_csr_rw	3.330s	139.589us	20	20	100.00
V2S	tl_intg_err	kmac_sec_cm	2.020m	16.297ms	5	5	100.00
		kmac_tl_intg_err	5.600s	988.023us	20	20	100.00
V2S	sec_cm_bus_integrity	kmac_tl_intg_err	5.600s	988.023us	20	20	100.00
V2S	sec_cm_lc_escalate_en_intersig_mubi	kmac_lc_escalation	43.270s	1.546ms	50	50	100.00
V2S	sec_cm_sw_key_key_masking	kmac_smoke	1.486m	6.876ms	50	50	100.00
V2S	sec_cm_key_sideload	kmac_sideload	11.463m	256.470ms	50	50	100.00
V2S	sec_cm_cfg_shadowed_config_shadow	kmac_shadow_reg_errors	1.560s	102.715us	20	20	100.00
V2S	sec_cm_fsm_sparse	kmac_sec_cm	2.020m	16.297ms	5	5	100.00
V2S	sec_cm_ctr_redun	kmac_sec_cm	2.020m	16.297ms	5	5	100.00
V2S	sec_cm_packer_ctr_redun	kmac_sec_cm	2.020m	16.297ms	5	5	100.00
V2S	sec_cm_cfg_shadowed_config_regwen	kmac_smoke	1.486m	6.876ms	50	50	100.00
V2S	sec_cm_fsm_global_esc	kmac_lc_escalation	43.270s	1.546ms	50	50	100.00
V2S	sec_cm_fsm_local_esc	kmac_sec_cm	2.020m	16.297ms	5	5	100.00
V2S	sec_cm_absorbed_ctrl_mubi	kmac_mubi	7.256m	52.955ms	10	10	100.00
V2S	sec_cm_sw_cmd_ctrl_sparse	kmac_smoke	1.486m	6.876ms	50	50	100.00
V2S		TOTAL			75	75	100.00
V3	throughput	kmac_throughput			0	0	--
V3	stress_all_with_rand_reset	kmac_stress_all_with_rand_reset	44.322m	80.363ms	4	10	40.00
V3		TOTAL			4	10	40.00
		TOTAL			1148	1250	91.84

Testplan Progress

Items	Total	Written	Passing	Progress
V1	8	8	8	100.00
V2	25	25	16	64.00
V2S	5	5	5	100.00
V3	2	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
94.41	97.89	92.65	99.89	78.17	95.53	98.89	97.88

Failure Buckets

Job kmac_masked-sim-vcs_run_default killed due to: Exit reason: User job exceeded runlimit: User job timed out has 90 failures:
- Test kmac_test_vectors_shake_128 has 34 failures.
  - 0.kmac_test_vectors_shake_128.21261677128317481473882222120630512864761039861803713385061078897914370442726
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/0.kmac_test_vectors_shake_128/latest/run.log
```
  Job ID: smart:a202a1ac-d1ef-49c9-9877-aaf6f6709465
```
  - 1.kmac_test_vectors_shake_128.60208766482744816001685885343244880507519884818381163704085796579327806754081
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/1.kmac_test_vectors_shake_128/latest/run.log
```
  Job ID: smart:a40b3d24-bef7-487d-b23e-dcb8fb5a0f45
```
  - ... and 32 more failures.
- Test kmac_test_vectors_shake_256 has 23 failures.
  - 0.kmac_test_vectors_shake_256.79145133663000220837314452213075987585367597786399145631748327426057996340599
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/0.kmac_test_vectors_shake_256/latest/run.log
```
  Job ID: smart:ce6b98e1-1b27-424c-a3bc-160a1ec0b432
```
  - 1.kmac_test_vectors_shake_256.115237747901092300266541228531854169992059696451488785684501536271865211080028
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/1.kmac_test_vectors_shake_256/latest/run.log
```
  Job ID: smart:844d46f1-0637-4d26-ac25-cce359327e4f
```
  - ... and 21 more failures.
- Test kmac_long_msg_and_output has 17 failures.
  - 1.kmac_long_msg_and_output.39301077286164382258630846289625063883275454101855308322769458842658494469270
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/1.kmac_long_msg_and_output/latest/run.log
```
  Job ID: smart:ffb6601b-5206-4307-a591-a8d94eef4ce5
```
  - 2.kmac_long_msg_and_output.6868193046768357668649788148993045484965927657696440531482391002993907787155
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/2.kmac_long_msg_and_output/latest/run.log
```
  Job ID: smart:7620ea7e-1ff1-4a78-966a-532d3ac89407
```
  - ... and 15 more failures.
- Test kmac_test_vectors_sha3_256 has 6 failures.
  - 9.kmac_test_vectors_sha3_256.87559498878915133262084881252496883882570628294952592203657465292865912598809
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/9.kmac_test_vectors_sha3_256/latest/run.log
```
  Job ID: smart:ba21b85c-3949-4fb1-9d60-16581e09b62d
```
  - 11.kmac_test_vectors_sha3_256.109269375119082389413359222928856519746407192371052861936138169269120668015172
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/11.kmac_test_vectors_sha3_256/latest/run.log
```
  Job ID: smart:d5e309e5-4c26-4997-bed5-ab8820ed4d15
```
  - ... and 4 more failures.
- Test kmac_test_vectors_sha3_224 has 10 failures.
  - 12.kmac_test_vectors_sha3_224.30450494994107605415872553063481095148719502859310778643752744708996058136899
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/12.kmac_test_vectors_sha3_224/latest/run.log
```
  Job ID: smart:375cdfe6-7197-4227-8991-66ad92bfa643
```
  - 13.kmac_test_vectors_sha3_224.41986696633006772573657469253993397842894624224207548935780925697233225869347
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/13.kmac_test_vectors_sha3_224/latest/run.log
```
  Job ID: smart:34bd236a-22cf-4d00-a9be-44680df09393
```
  - ... and 8 more failures.
UVM_ERROR (cip_base_vseq.sv:836) [kmac_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 4 failures:
- Test kmac_stress_all_with_rand_reset has 4 failures.
  - 0.kmac_stress_all_with_rand_reset.109160191860854662056412685815746066664836833804985522348730494997777275956359
    Line 1994, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/0.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 63126269384 ps: (cip_base_vseq.sv:836) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 63126269384 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 4.kmac_stress_all_with_rand_reset.90494621368800140318960721889801849593829473690898338833065280396379448337166
    Line 644, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/4.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 14648927341 ps: (cip_base_vseq.sv:836) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 14648927341 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 2 more failures.

UVM_FATAL (kmac_scoreboard.sv:1638) [scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (* [*] vs * [*]) Mismatch between unmasked_digest[*] and dpi_digest[*] has 3 failures:

Test kmac_entropy_refresh has 1 failures.

4.kmac_entropy_refresh.105898291949632748532009290761556346731024395396876931257792533817990783674261
Line 321, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/4.kmac_entropy_refresh/latest/run.log

  UVM_FATAL @ 1355779467 ps: (kmac_scoreboard.sv:1638) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (106 [0x6a] vs 8 [0x8]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 1355779467 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test kmac_app has 1 failures.

25.kmac_app.86478093809008106835841995586322911263978088328212213644112923076525173209384
Line 337, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/25.kmac_app/latest/run.log

  UVM_FATAL @ 2731936445 ps: (kmac_scoreboard.sv:1638) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (134 [0x86] vs 152 [0x98]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 2731936445 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test kmac_stress_all has 1 failures.

49.kmac_stress_all.95743653725346651477158879576558632087870534479514085086847441357788951975991
Line 829, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/49.kmac_stress_all/latest/run.log

  UVM_FATAL @ 9976559151 ps: (kmac_scoreboard.sv:1638) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (142 [0x8e] vs 235 [0xeb]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 9976559151 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_ERROR (csr_utils_pkg.sv:476) [csr_utils::csr_rd_check] Check failed obs == exp (* [*] vs * [*]) Regname: kmac_reg_block.intr_state.kmac_done reset value: * has 3 failures:
- Test kmac_test_vectors_shake_256 has 1 failures.
  - 12.kmac_test_vectors_shake_256.83792598229443443700728638806230703653676157963923602435942310063035696792328
    Line 253, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/12.kmac_test_vectors_shake_256/latest/run.log
```
  UVM_ERROR @  38193714 ps: (csr_utils_pkg.sv:476) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @  38193714 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test kmac_test_vectors_sha3_384 has 1 failures.
  - 24.kmac_test_vectors_sha3_384.97889252806861383483258005936033477385023503496853885199910656654874862937245
    Line 253, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/24.kmac_test_vectors_sha3_384/latest/run.log
```
  UVM_ERROR @  93301052 ps: (csr_utils_pkg.sv:476) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @  93301052 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test kmac_entropy_refresh has 1 failures.
  - 49.kmac_entropy_refresh.28354241000171511860249875678998902716098700853834419475712970546652286236258
    Line 598, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/49.kmac_entropy_refresh/latest/run.log
```
  UVM_ERROR @ 6075999704 ps: (csr_utils_pkg.sv:476) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @ 6075999704 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (kmac_scoreboard.sv:1188) [scoreboard] Check failed csr.get_mirrored_value() == item.d_data (* [*] vs * [*]) reg name: kmac_reg_block.err_code has 2 failures:
- Test kmac_stress_all_with_rand_reset has 2 failures.
  - 2.kmac_stress_all_with_rand_reset.20429423539723610466021716184848224445858325222999252302302674512407987027901
    Line 260, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/2.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @  25230082 ps: (kmac_scoreboard.sv:1188) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (0 [0x0] vs 2147483680 [0x80000020]) reg name: kmac_reg_block.err_code
  UVM_INFO @  25230082 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 5.kmac_stress_all_with_rand_reset.38108110052645628735206257155139781318676931772942081525708247119093868993125
    Line 1498, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/5.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 191974506712 ps: (kmac_scoreboard.sv:1188) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (0 [0x0] vs 2147483672 [0x80000018]) reg name: kmac_reg_block.err_code
  UVM_INFO @ 191974506712 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```