KMAC/MASKED Simulation Results

Saturday July 20 2024 23:02:34 UTC

GitHub Revision: `e971cd9798`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 85433431889345478971181747401055702269263498582281270185582621732035232392187

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	kmac_smoke	1.367m	8.268ms	50	50	100.00
V1	csr_hw_reset	kmac_csr_hw_reset	1.210s	57.463us	5	5	100.00
V1	csr_rw	kmac_csr_rw	1.200s	27.973us	20	20	100.00
V1	csr_bit_bash	kmac_csr_bit_bash	20.550s	4.802ms	5	5	100.00
V1	csr_aliasing	kmac_csr_aliasing	9.020s	383.367us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	kmac_csr_mem_rw_with_rand_reset	2.630s	385.006us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	kmac_csr_rw	1.200s	27.973us	20	20	100.00
		kmac_csr_aliasing	9.020s	383.367us	5	5	100.00
V1	mem_walk	kmac_mem_walk	0.890s	27.506us	5	5	100.00
V1	mem_partial_access	kmac_mem_partial_access	1.250s	106.131us	5	5	100.00
V1		TOTAL			115	115	100.00
V2	long_msg_and_output	kmac_long_msg_and_output	57.025m	265.522ms	50	50	100.00
V2	burst_write	kmac_burst_write	30.053m	16.620ms	49	50	98.00
V2	test_vectors	kmac_test_vectors_sha3_224	42.404m	351.191ms	50	50	100.00
		kmac_test_vectors_sha3_256	39.062m	95.022ms	48	50	96.00
		kmac_test_vectors_sha3_384	35.401m	960.267ms	47	50	94.00
		kmac_test_vectors_sha3_512	24.191m	448.292ms	50	50	100.00
		kmac_test_vectors_shake_128	1.719h	1.124s	50	50	100.00
		kmac_test_vectors_shake_256	1.566h	3.022s	50	50	100.00
		kmac_test_vectors_kmac	7.590s	988.978us	50	50	100.00
		kmac_test_vectors_kmac_xof	7.690s	2.673ms	50	50	100.00
V2	sideload	kmac_sideload	9.072m	26.409ms	50	50	100.00
V2	app	kmac_app	7.693m	73.131ms	49	50	98.00
V2	app_with_partial_data	kmac_app_with_partial_data	6.153m	37.909ms	10	10	100.00
V2	entropy_refresh	kmac_entropy_refresh	8.141m	20.642ms	47	50	94.00
V2	error	kmac_error	8.384m	38.855ms	49	50	98.00
V2	key_error	kmac_key_error	16.780s	14.288ms	49	50	98.00
V2	edn_timeout_error	kmac_edn_timeout_error	52.350s	742.911us	20	20	100.00
V2	entropy_mode_error	kmac_entropy_mode_error	20.050s	321.982us	20	20	100.00
V2	entropy_ready_error	kmac_entropy_ready_error	1.102m	26.233ms	10	10	100.00
V2	lc_escalation	kmac_lc_escalation	48.000s	1.384ms	50	50	100.00
V2	stress_all	kmac_stress_all	44.475m	77.248ms	50	50	100.00
V2	intr_test	kmac_intr_test	0.940s	63.538us	50	50	100.00
V2	alert_test	kmac_alert_test	1.020s	328.019us	50	50	100.00
V2	tl_d_oob_addr_access	kmac_tl_errors	3.000s	207.413us	20	20	100.00
V2	tl_d_illegal_access	kmac_tl_errors	3.000s	207.413us	20	20	100.00
V2	tl_d_outstanding_access	kmac_csr_hw_reset	1.210s	57.463us	5	5	100.00
		kmac_csr_rw	1.200s	27.973us	20	20	100.00
		kmac_csr_aliasing	9.020s	383.367us	5	5	100.00
		kmac_same_csr_outstanding	2.640s	459.206us	20	20	100.00
V2	tl_d_partial_access	kmac_csr_hw_reset	1.210s	57.463us	5	5	100.00
		kmac_csr_rw	1.200s	27.973us	20	20	100.00
		kmac_csr_aliasing	9.020s	383.367us	5	5	100.00
		kmac_same_csr_outstanding	2.640s	459.206us	20	20	100.00
V2		TOTAL			1038	1050	98.86
V2S	shadow_reg_update_error	kmac_shadow_reg_errors	1.480s	189.858us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	kmac_shadow_reg_errors	1.480s	189.858us	20	20	100.00
V2S	shadow_reg_storage_error	kmac_shadow_reg_errors	1.480s	189.858us	20	20	100.00
V2S	shadowed_reset_glitch	kmac_shadow_reg_errors	1.480s	189.858us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	kmac_shadow_reg_errors_with_csr_rw	2.870s	237.804us	20	20	100.00
V2S	tl_intg_err	kmac_sec_cm	2.087m	8.568ms	5	5	100.00
		kmac_tl_intg_err	4.930s	308.128us	20	20	100.00
V2S	sec_cm_bus_integrity	kmac_tl_intg_err	4.930s	308.128us	20	20	100.00
V2S	sec_cm_lc_escalate_en_intersig_mubi	kmac_lc_escalation	48.000s	1.384ms	50	50	100.00
V2S	sec_cm_sw_key_key_masking	kmac_smoke	1.367m	8.268ms	50	50	100.00
V2S	sec_cm_key_sideload	kmac_sideload	9.072m	26.409ms	50	50	100.00
V2S	sec_cm_cfg_shadowed_config_shadow	kmac_shadow_reg_errors	1.480s	189.858us	20	20	100.00
V2S	sec_cm_fsm_sparse	kmac_sec_cm	2.087m	8.568ms	5	5	100.00
V2S	sec_cm_ctr_redun	kmac_sec_cm	2.087m	8.568ms	5	5	100.00
V2S	sec_cm_packer_ctr_redun	kmac_sec_cm	2.087m	8.568ms	5	5	100.00
V2S	sec_cm_cfg_shadowed_config_regwen	kmac_smoke	1.367m	8.268ms	50	50	100.00
V2S	sec_cm_fsm_global_esc	kmac_lc_escalation	48.000s	1.384ms	50	50	100.00
V2S	sec_cm_fsm_local_esc	kmac_sec_cm	2.087m	8.568ms	5	5	100.00
V2S	sec_cm_absorbed_ctrl_mubi	kmac_mubi	7.195m	70.847ms	10	10	100.00
V2S	sec_cm_sw_cmd_ctrl_sparse	kmac_smoke	1.367m	8.268ms	50	50	100.00
V2S		TOTAL			75	75	100.00
V3	throughput	kmac_throughput			0	0	--
V3	stress_all_with_rand_reset	kmac_stress_all_with_rand_reset	37.868m	89.928ms	2	10	20.00
V3		TOTAL			2	10	20.00
		TOTAL			1230	1250	98.40

Testplan Progress

Items	Total	Written	Passing	Progress
V1	8	8	8	100.00
V2	25	25	18	72.00
V2S	5	5	5	100.00
V3	2	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
94.34	97.91	92.62	99.89	77.46	95.59	99.05	97.88

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:839) [kmac_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 6 failures:
- Test kmac_stress_all_with_rand_reset has 6 failures.
  - 0.kmac_stress_all_with_rand_reset.7794237020794898697848930589287118223584132341193357417443447158090358768512
    Line 2618, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/0.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 89928353385 ps: (cip_base_vseq.sv:839) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 89928353385 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 2.kmac_stress_all_with_rand_reset.66301996691781400482071067407844359772097144168468590038569506740601549920272
    Line 532, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/2.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 9333397604 ps: (cip_base_vseq.sv:839) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 9333397604 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 4 more failures.
UVM_ERROR (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (* [*] vs * [*]) Regname: kmac_reg_block.intr_state.kmac_done reset value: * has 5 failures:
- Test kmac_test_vectors_sha3_256 has 2 failures.
  - 15.kmac_test_vectors_sha3_256.59156493161780149267381263502900422071346481874183568603585907318159993926086
    Line 253, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/15.kmac_test_vectors_sha3_256/latest/run.log
```
  UVM_ERROR @  30137265 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @  30137265 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 45.kmac_test_vectors_sha3_256.71580071615695108319220493068275202317238738483594996101888213670841859230962
    Line 253, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/45.kmac_test_vectors_sha3_256/latest/run.log
```
  UVM_ERROR @  44097117 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @  44097117 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test kmac_test_vectors_sha3_384 has 3 failures.
  - 34.kmac_test_vectors_sha3_384.4289985427179607673941255573661743639003334576394891347017102405902707560676
    Line 253, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/34.kmac_test_vectors_sha3_384/latest/run.log
```
  UVM_ERROR @  82640868 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @  82640868 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 44.kmac_test_vectors_sha3_384.58048108737723191140341866914316053383262394447468347715567585353813203114531
    Line 253, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/44.kmac_test_vectors_sha3_384/latest/run.log
```
  UVM_ERROR @  71054201 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @  71054201 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 1 more failures.

UVM_FATAL (kmac_scoreboard.sv:1519) [scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (* [*] vs * [*]) Mismatch between unmasked_digest[*] and dpi_digest[*] has 4 failures:

Test kmac_app has 1 failures.

4.kmac_app.109208037903356322769504715051796849208955364365185820208156964431156006529856
Line 387, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/4.kmac_app/latest/run.log

  UVM_FATAL @ 3449460475 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (152 [0x98] vs 126 [0x7e]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 3449460475 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test kmac_entropy_refresh has 3 failures.

10.kmac_entropy_refresh.91027857215112348499118881714485444500100539614460131264594586364584047137656
Line 373, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/10.kmac_entropy_refresh/latest/run.log

  UVM_FATAL @ 7062614424 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (236 [0xec] vs 47 [0x2f]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 7062614424 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

36.kmac_entropy_refresh.13451369711809217877326279682356172589715683870112050676178356182246235847449
Line 543, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/36.kmac_entropy_refresh/latest/run.log

  UVM_FATAL @ 19248008980 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (109 [0x6d] vs 206 [0xce]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 19248008980 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 1 more failures.

UVM_ERROR (kmac_scoreboard.sv:1175) [scoreboard] Check failed csr.get_mirrored_value() == item.d_data (* [*] vs * [*]) reg name: kmac_reg_block.err_code has 2 failures:
- Test kmac_stress_all_with_rand_reset has 2 failures.
  - 1.kmac_stress_all_with_rand_reset.59228257388434747981374352064167368370852692163444712053537110056468700200215
    Line 510, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/1.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 34075897812 ps: (kmac_scoreboard.sv:1175) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (0 [0x0] vs 2147483680 [0x80000020]) reg name: kmac_reg_block.err_code
  UVM_INFO @ 34075897812 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 8.kmac_stress_all_with_rand_reset.95247354377494180112478078437295745104646775238209742475752459689476266513956
    Line 576, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/8.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 12091501551 ps: (kmac_scoreboard.sv:1175) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (0 [0x0] vs 2147483760 [0x80000070]) reg name: kmac_reg_block.err_code
  UVM_INFO @ 12091501551 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_FATAL (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of * ps hit, indicating a probable testbench issue has 2 failures:
- Test kmac_error has 1 failures.
  - 19.kmac_error.45142177968274177070279448674364218933547762269506257053558775945169591818878
    Line 1314, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/19.kmac_error/latest/run.log
```
  UVM_FATAL @ 200000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 200000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 200000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test kmac_burst_write has 1 failures.
  - 28.kmac_burst_write.49766405896801880576043087040959183376382790846505269005498815741214253586699
    Line 614, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/28.kmac_burst_write/latest/run.log
```
  UVM_FATAL @ 200000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 200000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 200000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (kmac_base_vseq.sv:382) [kmac_key_error_vseq] Check failed (intr_pins[KmacErr] == *) intr_pins[KmacErr] is not set! has 1 failures:
- Test kmac_key_error has 1 failures.
  - 46.kmac_key_error.63065022011741276708824102422688954349207168440810349093473365511284633782477
    Line 275, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/46.kmac_key_error/latest/run.log
```
  UVM_ERROR @ 3147345088 ps: (kmac_base_vseq.sv:382) [uvm_test_top.env.virtual_sequencer.kmac_key_error_vseq] Check failed (intr_pins[KmacErr] == 1) intr_pins[KmacErr] is not set!
  UVM_INFO @ 3147345088 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```