KMAC/UNMASKED Simulation Results

Friday July 05 2024 23:02:55 UTC

GitHub Revision: `9edf84e236`

Branch: os_regression

Testplan

Simulator: VCS

Build randomization enabled with --build-seed 47623749544922802985321435118963335754001495105472137721881337469861493653463

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	kmac_smoke	1.036m	12.266ms	50	50	100.00
V1	csr_hw_reset	kmac_csr_hw_reset	1.190s	182.907us	5	5	100.00
V1	csr_rw	kmac_csr_rw	1.120s	151.194us	20	20	100.00
V1	csr_bit_bash	kmac_csr_bit_bash	20.220s	2.936ms	5	5	100.00
V1	csr_aliasing	kmac_csr_aliasing	9.410s	394.528us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	kmac_csr_mem_rw_with_rand_reset	2.640s	525.330us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	kmac_csr_rw	1.120s	151.194us	20	20	100.00
		kmac_csr_aliasing	9.410s	394.528us	5	5	100.00
V1	mem_walk	kmac_mem_walk	0.790s	15.089us	5	5	100.00
V1	mem_partial_access	kmac_mem_partial_access	1.640s	46.058us	5	5	100.00
V1		TOTAL			115	115	100.00
V2	long_msg_and_output	kmac_long_msg_and_output	46.784m	1.868s	50	50	100.00
V2	burst_write	kmac_burst_write	13.550m	104.100ms	49	50	98.00
V2	test_vectors	kmac_test_vectors_sha3_224	34.661m	1.105s	50	50	100.00
		kmac_test_vectors_sha3_256	37.881m	1.137s	50	50	100.00
		kmac_test_vectors_sha3_384	26.429m	297.514ms	50	50	100.00
		kmac_test_vectors_sha3_512	18.421m	439.474ms	50	50	100.00
		kmac_test_vectors_shake_128	1.688h	1.433s	50	50	100.00
		kmac_test_vectors_shake_256	1.402h	866.925ms	50	50	100.00
		kmac_test_vectors_kmac	6.090s	2.775ms	50	50	100.00
		kmac_test_vectors_kmac_xof	5.790s	1.014ms	50	50	100.00
V2	sideload	kmac_sideload	7.112m	24.664ms	50	50	100.00
V2	app	kmac_app	5.274m	51.812ms	49	50	98.00
V2	app_with_partial_data	kmac_app_with_partial_data	3.791m	26.014ms	10	10	100.00
V2	entropy_refresh	kmac_entropy_refresh	5.377m	61.271ms	48	50	96.00
V2	error	kmac_error	7.275m	71.245ms	50	50	100.00
V2	key_error	kmac_key_error	9.670s	1.989ms	50	50	100.00
V2	edn_timeout_error	kmac_edn_timeout_error	50.740s	8.997ms	20	20	100.00
V2	entropy_mode_error	kmac_entropy_mode_error	39.810s	4.296ms	20	20	100.00
V2	entropy_ready_error	kmac_entropy_ready_error	1.113m	40.949ms	10	10	100.00
V2	lc_escalation	kmac_lc_escalation	43.270s	844.125us	50	50	100.00
V2	stress_all	kmac_stress_all	33.050m	86.480ms	50	50	100.00
V2	intr_test	kmac_intr_test	0.880s	32.684us	50	50	100.00
V2	alert_test	kmac_alert_test	0.900s	278.985us	50	50	100.00
V2	tl_d_oob_addr_access	kmac_tl_errors	3.410s	167.263us	20	20	100.00
V2	tl_d_illegal_access	kmac_tl_errors	3.410s	167.263us	20	20	100.00
V2	tl_d_outstanding_access	kmac_csr_hw_reset	1.190s	182.907us	5	5	100.00
		kmac_csr_rw	1.120s	151.194us	20	20	100.00
		kmac_csr_aliasing	9.410s	394.528us	5	5	100.00
		kmac_same_csr_outstanding	2.850s	490.284us	20	20	100.00
V2	tl_d_partial_access	kmac_csr_hw_reset	1.190s	182.907us	5	5	100.00
		kmac_csr_rw	1.120s	151.194us	20	20	100.00
		kmac_csr_aliasing	9.410s	394.528us	5	5	100.00
		kmac_same_csr_outstanding	2.850s	490.284us	20	20	100.00
V2		TOTAL			1046	1050	99.62
V2S	shadow_reg_update_error	kmac_shadow_reg_errors	1.470s	176.767us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	kmac_shadow_reg_errors	1.470s	176.767us	20	20	100.00
V2S	shadow_reg_storage_error	kmac_shadow_reg_errors	1.470s	176.767us	20	20	100.00
V2S	shadowed_reset_glitch	kmac_shadow_reg_errors	1.470s	176.767us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	kmac_shadow_reg_errors_with_csr_rw	3.010s	121.681us	20	20	100.00
V2S	tl_intg_err	kmac_sec_cm	1.036m	5.457ms	5	5	100.00
		kmac_tl_intg_err	5.160s	951.612us	20	20	100.00
V2S	sec_cm_bus_integrity	kmac_tl_intg_err	5.160s	951.612us	20	20	100.00
V2S	sec_cm_lc_escalate_en_intersig_mubi	kmac_lc_escalation	43.270s	844.125us	50	50	100.00
V2S	sec_cm_sw_key_key_masking	kmac_smoke	1.036m	12.266ms	50	50	100.00
V2S	sec_cm_key_sideload	kmac_sideload	7.112m	24.664ms	50	50	100.00
V2S	sec_cm_cfg_shadowed_config_shadow	kmac_shadow_reg_errors	1.470s	176.767us	20	20	100.00
V2S	sec_cm_fsm_sparse	kmac_sec_cm	1.036m	5.457ms	5	5	100.00
V2S	sec_cm_ctr_redun	kmac_sec_cm	1.036m	5.457ms	5	5	100.00
V2S	sec_cm_packer_ctr_redun	kmac_sec_cm	1.036m	5.457ms	5	5	100.00
V2S	sec_cm_cfg_shadowed_config_regwen	kmac_smoke	1.036m	12.266ms	50	50	100.00
V2S	sec_cm_fsm_global_esc	kmac_lc_escalation	43.270s	844.125us	50	50	100.00
V2S	sec_cm_fsm_local_esc	kmac_sec_cm	1.036m	5.457ms	5	5	100.00
V2S	sec_cm_absorbed_ctrl_mubi	kmac_mubi	4.274m	4.199ms	10	10	100.00
V2S	sec_cm_sw_cmd_ctrl_sparse	kmac_smoke	1.036m	12.266ms	50	50	100.00
V2S		TOTAL			75	75	100.00
V3	throughput	kmac_throughput			0	0	--
V3	stress_all_with_rand_reset	kmac_stress_all_with_rand_reset	21.828m	279.788ms	2	10	20.00
V3		TOTAL			2	10	20.00
		TOTAL			1238	1250	99.04

Testplan Progress

Items	Total	Written	Passing	Progress
V1	8	8	8	100.00
V2	25	25	22	88.00
V2S	5	5	5	100.00
V3	2	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
92.21	95.89	92.30	100.00	67.77	94.11	98.84	96.58

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:825) [kmac_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 7 failures:
- Test kmac_stress_all_with_rand_reset has 7 failures.
  - 0.kmac_stress_all_with_rand_reset.22526051090024218174796744373016431560894047443304067800536682377402207964535
    Line 692, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/0.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 23889824668 ps: (cip_base_vseq.sv:825) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 23889824668 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 1.kmac_stress_all_with_rand_reset.10544079554249996957779286212812758487036265233662448591269003950139678197508
    Line 992, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/1.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 30842536608 ps: (cip_base_vseq.sv:825) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 30842536608 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 5 more failures.
UVM_FATAL (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of * ps hit, indicating a probable testbench issue has 2 failures:
- Test kmac_burst_write has 1 failures.
  - 1.kmac_burst_write.72279913123900197875466116849530619035884458083545073437854988228275064686004
    Line 1160, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/1.kmac_burst_write/latest/run.log
```
  UVM_FATAL @ 200000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 200000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 200000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test kmac_app has 1 failures.
  - 12.kmac_app.84407080674207487047405024827195206351942205016459882967582426151488018421714
    Line 810, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/12.kmac_app/latest/run.log
```
  UVM_FATAL @ 200000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 200000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 200000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_FATAL (kmac_scoreboard.sv:1519) [scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (* [*] vs * [*]) Mismatch between unmasked_digest[*] and dpi_digest[*] has 2 failures:
- Test kmac_entropy_refresh has 2 failures.
  - 16.kmac_entropy_refresh.96744285512971636380248631877572105163861568529753394031008331780794462721898
    Line 521, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/16.kmac_entropy_refresh/latest/run.log
```
  UVM_FATAL @ 6763757526 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (20 [0x14] vs 91 [0x5b]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 6763757526 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 47.kmac_entropy_refresh.105627409162870371401923415003549931633484679905502389533525255107624388149377
    Line 579, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/47.kmac_entropy_refresh/latest/run.log
```
  UVM_FATAL @ 18879345557 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (107 [0x6b] vs 176 [0xb0]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 18879345557 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (kmac_scoreboard.sv:1175) [scoreboard] Check failed csr.get_mirrored_value() == item.d_data (* [*] vs * [*]) reg name: kmac_reg_block.err_code has 1 failures:
- Test kmac_stress_all_with_rand_reset has 1 failures.
  - 2.kmac_stress_all_with_rand_reset.23609424029367914740022442415964344769861097110898192412339232341921044992478
    Line 1026, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/2.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 140377406038 ps: (kmac_scoreboard.sv:1175) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (0 [0x0] vs 2147483768 [0x80000078]) reg name: kmac_reg_block.err_code
  UVM_INFO @ 140377406038 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```

KMAC/UNMASKED Simulation Results

Friday July 05 2024 23:02:55 UTC

GitHub Revision: 9edf84e236

Branch: os_regression

Testplan

Simulator: VCS

Build randomization enabled with --build-seed 47623749544922802985321435118963335754001495105472137721881337469861493653463

Test Results

Testplan Progress

Coverage Results

Coverage Dashboard

Failure Buckets

Past Results

GitHub Revision: `9edf84e236`