KMAC/MASKED Simulation Results

Tuesday April 16 2024 19:02:32 UTC

GitHub Revision: `1c75f24e99`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 47053888840936652465110085351243654616760492049444303115123736462709488656445

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	kmac_smoke	1.650m	10.966ms	48	50	96.00
V1	csr_hw_reset	kmac_csr_hw_reset	1.210s	75.405us	5	5	100.00
V1	csr_rw	kmac_csr_rw	1.290s	115.944us	20	20	100.00
V1	csr_bit_bash	kmac_csr_bit_bash	22.200s	5.782ms	5	5	100.00
V1	csr_aliasing	kmac_csr_aliasing	9.540s	389.259us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	kmac_csr_mem_rw_with_rand_reset	2.770s	333.954us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	kmac_csr_rw	1.290s	115.944us	20	20	100.00
		kmac_csr_aliasing	9.540s	389.259us	5	5	100.00
V1	mem_walk	kmac_mem_walk	0.800s	18.039us	5	5	100.00
V1	mem_partial_access	kmac_mem_partial_access	1.520s	68.884us	5	5	100.00
V1		TOTAL			113	115	98.26
V2	long_msg_and_output	kmac_long_msg_and_output	55.006m	111.317ms	50	50	100.00
V2	burst_write	kmac_burst_write	23.342m	39.760ms	47	50	94.00
V2	test_vectors	kmac_test_vectors_sha3_224	42.984m	1.318s	50	50	100.00
		kmac_test_vectors_sha3_256	41.932m	1.283s	50	50	100.00
		kmac_test_vectors_sha3_384	32.779m	889.236ms	50	50	100.00
		kmac_test_vectors_sha3_512	24.091m	615.014ms	50	50	100.00
		kmac_test_vectors_shake_128	1.984h	3.231s	50	50	100.00
		kmac_test_vectors_shake_256	1.634h	315.711ms	50	50	100.00
		kmac_test_vectors_kmac	7.110s	1.667ms	49	50	98.00
		kmac_test_vectors_kmac_xof	7.430s	270.450us	49	50	98.00
V2	sideload	kmac_sideload	9.214m	22.201ms	50	50	100.00
V2	app	kmac_app	6.905m	68.709ms	49	50	98.00
V2	app_with_partial_data	kmac_app_with_partial_data	5.627m	28.322ms	10	10	100.00
V2	entropy_refresh	kmac_entropy_refresh	7.020m	8.950ms	50	50	100.00
V2	error	kmac_error	7.971m	20.432ms	50	50	100.00
V2	key_error	kmac_key_error	6.970s	5.061ms	49	50	98.00
V2	edn_timeout_error	kmac_edn_timeout_error	44.730s	793.306us	20	20	100.00
V2	entropy_mode_error	kmac_entropy_mode_error	33.000s	467.181us	20	20	100.00
V2	entropy_ready_error	kmac_entropy_ready_error	1.264m	23.092ms	10	10	100.00
V2	lc_escalation	kmac_lc_escalation	45.150s	3.799ms	50	50	100.00
V2	stress_all	kmac_stress_all	38.858m	74.736ms	49	50	98.00
V2	intr_test	kmac_intr_test	0.930s	16.351us	50	50	100.00
V2	alert_test	kmac_alert_test	0.930s	23.464us	50	50	100.00
V2	tl_d_oob_addr_access	kmac_tl_errors	3.210s	2.438ms	20	20	100.00
V2	tl_d_illegal_access	kmac_tl_errors	3.210s	2.438ms	20	20	100.00
V2	tl_d_outstanding_access	kmac_csr_hw_reset	1.210s	75.405us	5	5	100.00
		kmac_csr_rw	1.290s	115.944us	20	20	100.00
		kmac_csr_aliasing	9.540s	389.259us	5	5	100.00
		kmac_same_csr_outstanding	2.630s	100.760us	20	20	100.00
V2	tl_d_partial_access	kmac_csr_hw_reset	1.210s	75.405us	5	5	100.00
		kmac_csr_rw	1.290s	115.944us	20	20	100.00
		kmac_csr_aliasing	9.540s	389.259us	5	5	100.00
		kmac_same_csr_outstanding	2.630s	100.760us	20	20	100.00
V2		TOTAL			1042	1050	99.24
V2S	shadow_reg_update_error	kmac_shadow_reg_errors	1.500s	65.168us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	kmac_shadow_reg_errors	1.500s	65.168us	20	20	100.00
V2S	shadow_reg_storage_error	kmac_shadow_reg_errors	1.500s	65.168us	20	20	100.00
V2S	shadowed_reset_glitch	kmac_shadow_reg_errors	1.500s	65.168us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	kmac_shadow_reg_errors_with_csr_rw	3.070s	525.380us	19	20	95.00
V2S	tl_intg_err	kmac_sec_cm	1.766m	8.829ms	5	5	100.00
		kmac_tl_intg_err	5.310s	1.020ms	20	20	100.00
V2S	sec_cm_bus_integrity	kmac_tl_intg_err	5.310s	1.020ms	20	20	100.00
V2S	sec_cm_lc_escalate_en_intersig_mubi	kmac_lc_escalation	45.150s	3.799ms	50	50	100.00
V2S	sec_cm_sw_key_key_masking	kmac_smoke	1.650m	10.966ms	48	50	96.00
V2S	sec_cm_key_sideload	kmac_sideload	9.214m	22.201ms	50	50	100.00
V2S	sec_cm_cfg_shadowed_config_shadow	kmac_shadow_reg_errors	1.500s	65.168us	20	20	100.00
V2S	sec_cm_fsm_sparse	kmac_sec_cm	1.766m	8.829ms	5	5	100.00
V2S	sec_cm_ctr_redun	kmac_sec_cm	1.766m	8.829ms	5	5	100.00
V2S	sec_cm_packer_ctr_redun	kmac_sec_cm	1.766m	8.829ms	5	5	100.00
V2S	sec_cm_cfg_shadowed_config_regwen	kmac_smoke	1.650m	10.966ms	48	50	96.00
V2S	sec_cm_fsm_global_esc	kmac_lc_escalation	45.150s	3.799ms	50	50	100.00
V2S	sec_cm_fsm_local_esc	kmac_sec_cm	1.766m	8.829ms	5	5	100.00
V2S	sec_cm_absorbed_ctrl_mubi	kmac_mubi	6.414m	58.628ms	10	10	100.00
V2S	sec_cm_sw_cmd_ctrl_sparse	kmac_smoke	1.650m	10.966ms	48	50	96.00
V2S		TOTAL			74	75	98.67
V3	throughput	kmac_throughput			0	0	--
V3	stress_all_with_rand_reset	kmac_stress_all_with_rand_reset	1.050h	214.333ms	17	50	34.00
V3		TOTAL			17	50	34.00
		TOTAL			1246	1290	96.59

Testplan Progress

Items	Total	Written	Passing	Progress
V1	8	8	7	87.50
V2	25	25	19	76.00
V2S	5	5	4	80.00
V3	2	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
97.07	98.10	92.43	99.89	96.36	95.91	98.89	97.89

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:829) [kmac_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 24 failures:
- Test kmac_stress_all_with_rand_reset has 24 failures.
  - 0.kmac_stress_all_with_rand_reset.83704252790510420959463414670150176031572919613561305260318857227817333727020
    Line 298, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/0.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 7530679474 ps: (cip_base_vseq.sv:829) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 7530679474 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 3.kmac_stress_all_with_rand_reset.13639399901663121956356705587777350077708447222884223175355877148894226943173
    Line 667, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/3.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 12470140633 ps: (cip_base_vseq.sv:829) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 12470140633 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 22 more failures.
UVM_ERROR (kmac_scoreboard.sv:1175) [scoreboard] Check failed csr.get_mirrored_value() == item.d_data (* [*] vs * [*]) reg name: kmac_reg_block.err_code has 8 failures:
- Test kmac_stress_all_with_rand_reset has 8 failures.
  - 2.kmac_stress_all_with_rand_reset.59456108790418869891043733907377259775601239357912347262991471757111798603550
    Line 487, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/2.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 29407965827 ps: (kmac_scoreboard.sv:1175) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (0 [0x0] vs 2147483768 [0x80000078]) reg name: kmac_reg_block.err_code
  UVM_INFO @ 29407965827 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 4.kmac_stress_all_with_rand_reset.52882747111652372799838311002268548829252274181996042495422287428327117385899
    Line 1236, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/4.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 44420728995 ps: (kmac_scoreboard.sv:1175) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (0 [0x0] vs 2147483728 [0x80000050]) reg name: kmac_reg_block.err_code
  UVM_INFO @ 44420728995 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 6 more failures.

UVM_ERROR (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (* [*] vs * [*]) Regname: kmac_reg_block.intr_state.kmac_done reset value: * has 4 failures:

Test kmac_smoke has 2 failures.

29.kmac_smoke.67110841266551229241144602190189232271021418214371691162257831164514351383259
Line 254, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/29.kmac_smoke/latest/run.log

  UVM_ERROR @  77039268 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @  77039268 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

38.kmac_smoke.69992236594954865767696914661264147312548287544355941571732187192938030146877
Line 254, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/38.kmac_smoke/latest/run.log

  UVM_ERROR @  50911931 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @  50911931 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test kmac_test_vectors_kmac has 1 failures.

33.kmac_test_vectors_kmac.87715724695335962610622575640848767438957192733896539551371023916536259338473
Line 253, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/33.kmac_test_vectors_kmac/latest/run.log

  UVM_ERROR @  89908741 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @  89908741 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test kmac_test_vectors_kmac_xof has 1 failures.

45.kmac_test_vectors_kmac_xof.60399013762700664263659938703919527581772794985090843921720965718135407380337
Line 253, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/45.kmac_test_vectors_kmac_xof/latest/run.log

  UVM_ERROR @ 103555037 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @ 103555037 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_FATAL (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of * ps hit, indicating a probable testbench issue has 3 failures:
- Test kmac_burst_write has 3 failures.
  - 4.kmac_burst_write.59005020498915116116434800442790560890905658152740133397001526141083298627711
    Line 776, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/4.kmac_burst_write/latest/run.log
```
  UVM_FATAL @ 200000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 200000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 200000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 27.kmac_burst_write.23190912263104493137309747715400812119466051486515918248117631816669846660974
    Line 1155, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/27.kmac_burst_write/latest/run.log
```
  UVM_FATAL @ 200000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 200000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 200000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 1 more failures.
UVM_FATAL (kmac_scoreboard.sv:218) [scoreboard] Wait EDN request has 2 failures:
- Test kmac_stress_all_with_rand_reset has 1 failures.
  - 40.kmac_stress_all_with_rand_reset.92084398775537414850390779643561528154043406141386737456793708352025161684689
    Line 891, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/40.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_FATAL @ 26916168932 ps: (kmac_scoreboard.sv:218) [uvm_test_top.env.scoreboard] Wait EDN request
  UVM_INFO @ 26916168932 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test kmac_stress_all has 1 failures.
  - 44.kmac_stress_all.41693711030217050413132510629455172853357298712869854901408041728815896370938
    Line 852, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/44.kmac_stress_all/latest/run.log
```
  UVM_FATAL @ 76423214163 ps: (kmac_scoreboard.sv:218) [uvm_test_top.env.scoreboard] Wait EDN request
  UVM_INFO @ 76423214163 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (* [*] vs * [*]) Regname: kmac_reg_block.err_code reset value: * has 1 failures:
- Test kmac_key_error has 1 failures.
  - 2.kmac_key_error.13666060990739719632086102841380519030390901480034303580102333617104610298385
    Line 251, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/2.kmac_key_error/latest/run.log
```
  UVM_ERROR @ 124397190 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 16777216 [0x1000000]) Regname: kmac_reg_block.err_code reset value: 0x0
  UVM_INFO @ 124397190 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (* [*] vs * [*]) Regname: kmac_reg_block.prefix_* reset value: * has 1 failures:
- Test kmac_shadow_reg_errors_with_csr_rw has 1 failures.
  - 11.kmac_shadow_reg_errors_with_csr_rw.7260045971354091836587530072925035319953177610285105635682521165053597057439
    Line 255, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/11.kmac_shadow_reg_errors_with_csr_rw/latest/run.log
```
  UVM_ERROR @  82706017 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (1431232787 [0x554ee113] vs 310259302 [0x127e2e66]) Regname: kmac_reg_block.prefix_3 reset value: 0x0
  UVM_INFO @  82706017 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_FATAL (kmac_scoreboard.sv:1519) [scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (* [*] vs * [*]) Mismatch between unmasked_digest[*] and dpi_digest[*] has 1 failures:
- Test kmac_app has 1 failures.
  - 14.kmac_app.57056075796877509603450106393917643762958544204689169196121051360805467201385
    Line 559, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/14.kmac_app/latest/run.log
```
  UVM_FATAL @ 4758069595 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (248 [0xf8] vs 131 [0x83]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 4758069595 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```