KMAC/UNMASKED Simulation Results

Wednesday July 03 2024 23:02:32 UTC

GitHub Revision: `e6706fcc7b`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 8083624550445280117614176890238357255195852125596561370221115831648066795492

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	kmac_smoke	1.109m	4.239ms	50	50	100.00
V1	csr_hw_reset	kmac_csr_hw_reset	1.120s	178.585us	5	5	100.00
V1	csr_rw	kmac_csr_rw	1.160s	34.178us	20	20	100.00
V1	csr_bit_bash	kmac_csr_bit_bash	15.760s	286.883us	5	5	100.00
V1	csr_aliasing	kmac_csr_aliasing	9.370s	889.572us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	kmac_csr_mem_rw_with_rand_reset	2.500s	113.835us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	kmac_csr_rw	1.160s	34.178us	20	20	100.00
		kmac_csr_aliasing	9.370s	889.572us	5	5	100.00
V1	mem_walk	kmac_mem_walk	0.740s	12.277us	5	5	100.00
V1	mem_partial_access	kmac_mem_partial_access	1.450s	164.306us	5	5	100.00
V1		TOTAL			115	115	100.00
V2	long_msg_and_output	kmac_long_msg_and_output	43.370m	28.306ms	50	50	100.00
V2	burst_write	kmac_burst_write	14.585m	148.293ms	49	50	98.00
V2	test_vectors	kmac_test_vectors_sha3_224	34.886m	100.427ms	50	50	100.00
		kmac_test_vectors_sha3_256	33.662m	432.848ms	50	50	100.00
		kmac_test_vectors_sha3_384	25.974m	935.973ms	50	50	100.00
		kmac_test_vectors_sha3_512	18.952m	490.749ms	50	50	100.00
		kmac_test_vectors_shake_128	1.642h	2.146s	50	50	100.00
		kmac_test_vectors_shake_256	1.408h	2.715s	50	50	100.00
		kmac_test_vectors_kmac	5.460s	265.171us	50	50	100.00
		kmac_test_vectors_kmac_xof	5.390s	2.466ms	50	50	100.00
V2	sideload	kmac_sideload	7.641m	86.724ms	50	50	100.00
V2	app	kmac_app	5.016m	188.732ms	50	50	100.00
V2	app_with_partial_data	kmac_app_with_partial_data	4.340m	69.679ms	10	10	100.00
V2	entropy_refresh	kmac_entropy_refresh	6.139m	37.271ms	50	50	100.00
V2	error	kmac_error	6.430m	55.351ms	50	50	100.00
V2	key_error	kmac_key_error	12.190s	10.404ms	50	50	100.00
V2	edn_timeout_error	kmac_edn_timeout_error	40.790s	5.538ms	20	20	100.00
V2	entropy_mode_error	kmac_entropy_mode_error	36.580s	3.304ms	20	20	100.00
V2	entropy_ready_error	kmac_entropy_ready_error	54.300s	31.091ms	10	10	100.00
V2	lc_escalation	kmac_lc_escalation	36.070s	1.943ms	50	50	100.00
V2	stress_all	kmac_stress_all	40.765m	736.055ms	50	50	100.00
V2	intr_test	kmac_intr_test	0.900s	43.830us	50	50	100.00
V2	alert_test	kmac_alert_test	0.930s	206.061us	50	50	100.00
V2	tl_d_oob_addr_access	kmac_tl_errors	3.390s	130.054us	20	20	100.00
V2	tl_d_illegal_access	kmac_tl_errors	3.390s	130.054us	20	20	100.00
V2	tl_d_outstanding_access	kmac_csr_hw_reset	1.120s	178.585us	5	5	100.00
		kmac_csr_rw	1.160s	34.178us	20	20	100.00
		kmac_csr_aliasing	9.370s	889.572us	5	5	100.00
		kmac_same_csr_outstanding	2.660s	462.903us	20	20	100.00
V2	tl_d_partial_access	kmac_csr_hw_reset	1.120s	178.585us	5	5	100.00
		kmac_csr_rw	1.160s	34.178us	20	20	100.00
		kmac_csr_aliasing	9.370s	889.572us	5	5	100.00
		kmac_same_csr_outstanding	2.660s	462.903us	20	20	100.00
V2		TOTAL			1049	1050	99.90
V2S	shadow_reg_update_error	kmac_shadow_reg_errors	1.530s	76.652us	19	20	95.00
V2S	shadow_reg_read_clear_staged_value	kmac_shadow_reg_errors	1.530s	76.652us	19	20	95.00
V2S	shadow_reg_storage_error	kmac_shadow_reg_errors	1.530s	76.652us	19	20	95.00
V2S	shadowed_reset_glitch	kmac_shadow_reg_errors	1.530s	76.652us	19	20	95.00
V2S	shadow_reg_update_error_with_csr_rw	kmac_shadow_reg_errors_with_csr_rw	3.010s	138.908us	17	20	85.00
V2S	tl_intg_err	kmac_sec_cm	57.190s	12.070ms	5	5	100.00
		kmac_tl_intg_err	5.190s	335.244us	20	20	100.00
V2S	sec_cm_bus_integrity	kmac_tl_intg_err	5.190s	335.244us	20	20	100.00
V2S	sec_cm_lc_escalate_en_intersig_mubi	kmac_lc_escalation	36.070s	1.943ms	50	50	100.00
V2S	sec_cm_sw_key_key_masking	kmac_smoke	1.109m	4.239ms	50	50	100.00
V2S	sec_cm_key_sideload	kmac_sideload	7.641m	86.724ms	50	50	100.00
V2S	sec_cm_cfg_shadowed_config_shadow	kmac_shadow_reg_errors	1.530s	76.652us	19	20	95.00
V2S	sec_cm_fsm_sparse	kmac_sec_cm	57.190s	12.070ms	5	5	100.00
V2S	sec_cm_ctr_redun	kmac_sec_cm	57.190s	12.070ms	5	5	100.00
V2S	sec_cm_packer_ctr_redun	kmac_sec_cm	57.190s	12.070ms	5	5	100.00
V2S	sec_cm_cfg_shadowed_config_regwen	kmac_smoke	1.109m	4.239ms	50	50	100.00
V2S	sec_cm_fsm_global_esc	kmac_lc_escalation	36.070s	1.943ms	50	50	100.00
V2S	sec_cm_fsm_local_esc	kmac_sec_cm	57.190s	12.070ms	5	5	100.00
V2S	sec_cm_absorbed_ctrl_mubi	kmac_mubi	4.711m	17.649ms	10	10	100.00
V2S	sec_cm_sw_cmd_ctrl_sparse	kmac_smoke	1.109m	4.239ms	50	50	100.00
V2S		TOTAL			71	75	94.67
V3	throughput	kmac_throughput			0	0	--
V3	stress_all_with_rand_reset	kmac_stress_all_with_rand_reset	25.618m	74.059ms	0	10	0.00
V3		TOTAL			0	10	0.00
		TOTAL			1235	1250	98.80

Testplan Progress

Items	Total	Written	Passing	Progress
V1	8	8	8	100.00
V2	25	25	24	96.00
V2S	5	5	3	60.00
V3	2	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
91.99	95.77	90.51	100.00	68.60	93.67	98.84	96.58

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:825) [kmac_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 9 failures:
- Test kmac_stress_all_with_rand_reset has 9 failures.
  - 0.kmac_stress_all_with_rand_reset.20041609432356552755091206576654274432250446332894282214377176867509328407132
    Line 748, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/0.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 94998107729 ps: (cip_base_vseq.sv:825) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 94998107729 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 1.kmac_stress_all_with_rand_reset.87851391898297950106863604259771082415006902217474871051073483064038186194831
    Line 2418, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/1.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 74058783288 ps: (cip_base_vseq.sv:825) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 74058783288 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 7 more failures.

UVM_ERROR (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (* [*] vs * [*]) Regname: kmac_reg_block.status.alert_recov_ctrl_update_err reset value: * Check_csr_read_clear_staged_val task: check update_err status has 4 failures:

Test kmac_shadow_reg_errors_with_csr_rw has 3 failures.

3.kmac_shadow_reg_errors_with_csr_rw.111885118361593950621797155870147555842283447927663689966309590661389721717138
Line 265, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/3.kmac_shadow_reg_errors_with_csr_rw/latest/run.log

  UVM_ERROR @ 135999645 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (1 [0x1] vs 0 [0x0]) Regname: kmac_reg_block.status.alert_recov_ctrl_update_err reset value: 0x0  Check_csr_read_clear_staged_val task: check update_err status
  UVM_INFO @ 135999645 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

9.kmac_shadow_reg_errors_with_csr_rw.97399999774721842134172263295189707887200469840994714201454819662766732522918
Line 264, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/9.kmac_shadow_reg_errors_with_csr_rw/latest/run.log

  UVM_ERROR @ 152465568 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (1 [0x1] vs 0 [0x0]) Regname: kmac_reg_block.status.alert_recov_ctrl_update_err reset value: 0x0  Check_csr_read_clear_staged_val task: check update_err status
  UVM_INFO @ 152465568 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 1 more failures.

Test kmac_shadow_reg_errors has 1 failures.

18.kmac_shadow_reg_errors.85097061495736447511592691737648222987173703695261955633160754784797578280242
Line 265, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/18.kmac_shadow_reg_errors/latest/run.log

  UVM_ERROR @  19710973 ps: (csr_utils_pkg.sv:466) [csr_utils::csr_rd_check] Check failed obs == exp (1 [0x1] vs 0 [0x0]) Regname: kmac_reg_block.status.alert_recov_ctrl_update_err reset value: 0x0  Check_csr_read_clear_staged_val task: check update_err status
  UVM_INFO @  19710973 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_ERROR (kmac_scoreboard.sv:1175) [scoreboard] Check failed csr.get_mirrored_value() == item.d_data (* [*] vs * [*]) reg name: kmac_reg_block.err_code has 1 failures:
- Test kmac_stress_all_with_rand_reset has 1 failures.
  - 6.kmac_stress_all_with_rand_reset.103209718832385617790609375004593370717892635650305432841966621642672208532873
    Line 530, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/6.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 7003328333 ps: (kmac_scoreboard.sv:1175) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (0 [0x0] vs 2147483744 [0x80000060]) reg name: kmac_reg_block.err_code
  UVM_INFO @ 7003328333 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_FATAL (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of * ps hit, indicating a probable testbench issue has 1 failures:
- Test kmac_burst_write has 1 failures.
  - 7.kmac_burst_write.3651598713956703044566464436973947272122753982787345251810596887056842108955
    Line 914, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/7.kmac_burst_write/latest/run.log
```
  UVM_FATAL @ 200000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 200000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 200000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```