HMAC Simulation Results

Monday June 24 2024 23:02:35 UTC

GitHub Revision: `6e698b4dfe`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 102849012855470111388983783327793201144267754054590670930996118558901483180117

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	hmac_smoke	47.320s	13.837ms	50	50	100.00
V1	csr_hw_reset	hmac_csr_hw_reset	1.000s	37.397us	5	5	100.00
V1	csr_rw	hmac_csr_rw	1.010s	286.499us	20	20	100.00
V1	csr_bit_bash	hmac_csr_bit_bash	15.160s	5.694ms	5	5	100.00
V1	csr_aliasing	hmac_csr_aliasing	8.120s	164.122us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	hmac_csr_mem_rw_with_rand_reset	12.716m	53.619ms	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	hmac_csr_rw	1.010s	286.499us	20	20	100.00
		hmac_csr_aliasing	8.120s	164.122us	5	5	100.00
V1		TOTAL			105	105	100.00
V2	long_msg	hmac_long_msg	2.426m	2.456ms	50	50	100.00
V2	back_pressure	hmac_back_pressure	54.640s	2.425ms	50	50	100.00
V2	test_vectors	hmac_test_sha256_vectors	9.623m	31.126ms	50	50	100.00
		hmac_test_sha384_vectors	39.698m	177.065ms	46	50	92.00
		hmac_test_sha512_vectors	37.159m	709.936ms	48	50	96.00
		hmac_test_hmac_vectors	1.510s	875.609us	50	50	100.00
V2	burst_wr	hmac_burst_wr	1.202m	8.610ms	50	50	100.00
V2	datapath_stress	hmac_datapath_stress	25.221m	12.539ms	50	50	100.00
V2	error	hmac_error	3.859m	12.518ms	50	50	100.00
V2	wipe_secret	hmac_wipe_secret	1.702m	7.604ms	50	50	100.00
V2	save_and_restore	hmac_smoke	47.320s	13.837ms	50	50	100.00
		hmac_long_msg	2.426m	2.456ms	50	50	100.00
		hmac_back_pressure	54.640s	2.425ms	50	50	100.00
		hmac_datapath_stress	25.221m	12.539ms	50	50	100.00
		hmac_burst_wr	1.202m	8.610ms	50	50	100.00
		hmac_stress_all	44.488m	25.363ms	11	50	22.00
V2	fifo_empty_status_interrupt	fifo_empty_status_interrupt			0	0	--
V2	wide_digest_configurable_key_length	hmac_smoke	47.320s	13.837ms	50	50	100.00
		hmac_long_msg	2.426m	2.456ms	50	50	100.00
		hmac_back_pressure	54.640s	2.425ms	50	50	100.00
		hmac_datapath_stress	25.221m	12.539ms	50	50	100.00
		hmac_burst_wr	1.202m	8.610ms	50	50	100.00
		hmac_error	3.859m	12.518ms	50	50	100.00
		hmac_wipe_secret	1.702m	7.604ms	50	50	100.00
		hmac_test_sha256_vectors	9.623m	31.126ms	50	50	100.00
		hmac_test_sha384_vectors	39.698m	177.065ms	46	50	92.00
		hmac_test_sha512_vectors	37.159m	709.936ms	48	50	96.00
		hmac_test_hmac_vectors	1.510s	875.609us	50	50	100.00
		hmac_stress_all	44.488m	25.363ms	11	50	22.00
V2	stress_all	hmac_stress_all	44.488m	25.363ms	11	50	22.00
V2	alert_test	hmac_alert_test	0.670s	24.752us	50	50	100.00
V2	intr_test	hmac_intr_test	0.690s	16.816us	50	50	100.00
V2	tl_d_oob_addr_access	hmac_tl_errors	3.920s	718.599us	20	20	100.00
V2	tl_d_illegal_access	hmac_tl_errors	3.920s	718.599us	20	20	100.00
V2	tl_d_outstanding_access	hmac_csr_hw_reset	1.000s	37.397us	5	5	100.00
		hmac_csr_rw	1.010s	286.499us	20	20	100.00
		hmac_csr_aliasing	8.120s	164.122us	5	5	100.00
		hmac_same_csr_outstanding	2.300s	602.504us	20	20	100.00
V2	tl_d_partial_access	hmac_csr_hw_reset	1.000s	37.397us	5	5	100.00
		hmac_csr_rw	1.010s	286.499us	20	20	100.00
		hmac_csr_aliasing	8.120s	164.122us	5	5	100.00
		hmac_same_csr_outstanding	2.300s	602.504us	20	20	100.00
V2		TOTAL			645	690	93.48
V2S	tl_intg_err	hmac_sec_cm	1.010s	86.271us	5	5	100.00
		hmac_tl_intg_err	4.210s	160.695us	20	20	100.00
V2S	sec_cm_bus_integrity	hmac_tl_intg_err	4.210s	160.695us	20	20	100.00
V2S		TOTAL			25	25	100.00
V3	write_config_and_secret_key_during_msg_wr	hmac_smoke	47.320s	13.837ms	50	50	100.00
V3	stress_all_with_rand_reset	hmac_stress_all_with_rand_reset	14.922m	5.219ms	0	10	0.00
V3		TOTAL			0	10	0.00
		TOTAL			775	830	93.37

Testplan Progress

Items	Total	Written	Passing	Progress
V1	6	6	6	100.00
V2	16	15	12	75.00
V2S	2	2	2	100.00
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
92.58	95.85	92.94	100.00	74.36	91.89	99.49	93.58

Failure Buckets

UVM_FATAL (hmac_test_vectors_sha_vseq.sv:28) virtual_sequencer [hmac_test_vectors_hmac_vseq] This test sequence requires a command-line argument: sha2_digest_size=SHA2_*/SHA2_*/SHA2_* has 26 failures:

Test hmac_stress_all has 21 failures.

0.hmac_stress_all.82987332384564122107125590624386737287775865753365483337852498843192502749591
Line 23758, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/0.hmac_stress_all/latest/run.log

  UVM_FATAL @ 2416353496 ps: (hmac_test_vectors_sha_vseq.sv:28) uvm_test_top.env.virtual_sequencer [uvm_test_top.env.virtual_sequencer.hmac_test_vectors_hmac_vseq] This test sequence requires a command-line argument: sha2_digest_size=SHA2_256/SHA2_384/SHA2_512
  UVM_INFO @ 2416353496 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

2.hmac_stress_all.4339591930363152689262944455784930021690516718168168452282613960316016936418
Line 45123, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/2.hmac_stress_all/latest/run.log

  UVM_FATAL @ 9375413886 ps: (hmac_test_vectors_sha_vseq.sv:28) uvm_test_top.env.virtual_sequencer [uvm_test_top.env.virtual_sequencer.hmac_test_vectors_hmac_vseq] This test sequence requires a command-line argument: sha2_digest_size=SHA2_256/SHA2_384/SHA2_512
  UVM_INFO @ 9375413886 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 19 more failures.

Test hmac_stress_all_with_rand_reset has 5 failures.

0.hmac_stress_all_with_rand_reset.89773220806848058479173750857515424719296500199753514564987752764315267930909
Line 87794, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/0.hmac_stress_all_with_rand_reset/latest/run.log

  UVM_FATAL @ 5218900685 ps: (hmac_test_vectors_sha_vseq.sv:28) uvm_test_top.env.virtual_sequencer [uvm_test_top.env.virtual_sequencer.hmac_test_vectors_hmac_vseq] This test sequence requires a command-line argument: sha2_digest_size=SHA2_256/SHA2_384/SHA2_512
  UVM_INFO @ 5218900685 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

2.hmac_stress_all_with_rand_reset.98339889545564429785998749134537532121880108020408798112837638420183010223084
Line 251, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/2.hmac_stress_all_with_rand_reset/latest/run.log

  UVM_FATAL @   1419069 ps: (hmac_test_vectors_sha_vseq.sv:28) uvm_test_top.env.virtual_sequencer [uvm_test_top.env.virtual_sequencer.hmac_test_vectors_hmac_vseq] This test sequence requires a command-line argument: sha2_digest_size=SHA2_256/SHA2_384/SHA2_512
  UVM_INFO @   1419069 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 3 more failures.

UVM_FATAL (hmac_test_vectors_sha_vseq.sv:28) virtual_sequencer [hmac_test_vectors_sha_vseq] This test sequence requires a command-line argument: sha2_digest_size=SHA2_*/SHA2_*/SHA2_* has 21 failures:

Test hmac_stress_all_with_rand_reset has 3 failures.

1.hmac_stress_all_with_rand_reset.102933519651126683520510929847308531373276550618532202754024992918715706058863
Line 20801, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/1.hmac_stress_all_with_rand_reset/latest/run.log

  UVM_FATAL @ 5551571311 ps: (hmac_test_vectors_sha_vseq.sv:28) uvm_test_top.env.virtual_sequencer [uvm_test_top.env.virtual_sequencer.hmac_test_vectors_sha_vseq] This test sequence requires a command-line argument: sha2_digest_size=SHA2_256/SHA2_384/SHA2_512
  UVM_INFO @ 5551571311 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

4.hmac_stress_all_with_rand_reset.46006120885195601039991997035858562915935051549244896828604552646599127656785
Line 251, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/4.hmac_stress_all_with_rand_reset/latest/run.log

  UVM_FATAL @   6591226 ps: (hmac_test_vectors_sha_vseq.sv:28) uvm_test_top.env.virtual_sequencer [uvm_test_top.env.virtual_sequencer.hmac_test_vectors_sha_vseq] This test sequence requires a command-line argument: sha2_digest_size=SHA2_256/SHA2_384/SHA2_512
  UVM_INFO @   6591226 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 1 more failures.

Test hmac_stress_all has 18 failures.

6.hmac_stress_all.108758231000946611900502017724062327395077953753170760030873830128978447223251
Line 249, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/6.hmac_stress_all/latest/run.log

  UVM_FATAL @   1014544 ps: (hmac_test_vectors_sha_vseq.sv:28) uvm_test_top.env.virtual_sequencer [uvm_test_top.env.virtual_sequencer.hmac_test_vectors_sha_vseq] This test sequence requires a command-line argument: sha2_digest_size=SHA2_256/SHA2_384/SHA2_512
  UVM_INFO @   1014544 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

10.hmac_stress_all.81822543270436828861649703512065429073945999218630529888915577314136913323999
Line 50365, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/10.hmac_stress_all/latest/run.log

  UVM_FATAL @ 9264880571 ps: (hmac_test_vectors_sha_vseq.sv:28) uvm_test_top.env.virtual_sequencer [uvm_test_top.env.virtual_sequencer.hmac_test_vectors_sha_vseq] This test sequence requires a command-line argument: sha2_digest_size=SHA2_256/SHA2_384/SHA2_512
  UVM_INFO @ 9264880571 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 16 more failures.

UVM_FATAL (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of * ps hit, indicating a probable testbench issue has 6 failures:
- Test hmac_test_sha384_vectors has 4 failures.
  - 12.hmac_test_sha384_vectors.24079989589340504956896001827800526111714781159302752830117345619317926838903
    Line 281751, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/12.hmac_test_sha384_vectors/latest/run.log
```
  UVM_FATAL @ 1000000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 1000000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 1000000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 16.hmac_test_sha384_vectors.58659518068891733347327115498434521904900339778508067396894786005415160224634
    Line 290444, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/16.hmac_test_sha384_vectors/latest/run.log
```
  UVM_FATAL @ 1000000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 1000000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 1000000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 2 more failures.
- Test hmac_test_sha512_vectors has 2 failures.
  - 29.hmac_test_sha512_vectors.15906476820322879394306711213334856625740834328931360562662336001995967573669
    Line 120070, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/29.hmac_test_sha512_vectors/latest/run.log
```
  UVM_FATAL @ 1000000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 1000000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 1000000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 34.hmac_test_sha512_vectors.111442667570702577954252887478471568061410192784441399560845929606016814087448
    Line 231947, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/34.hmac_test_sha512_vectors/latest/run.log
```
  UVM_FATAL @ 1000000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 1000000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 1000000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (cip_base_vseq.sv:828) [hmac_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 2 failures:
- Test hmac_stress_all_with_rand_reset has 2 failures.
  - 8.hmac_stress_all_with_rand_reset.9341105546298639235318458334111019662870500146408445050063468009846330508684
    Line 20239, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/8.hmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 36253318672 ps: (cip_base_vseq.sv:828) [uvm_test_top.env.virtual_sequencer.hmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 36253318672 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 9.hmac_stress_all_with_rand_reset.21334304868195258224960482980812668850592238466998738150802398292165984929330
    Line 3050, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/9.hmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 937830251 ps: (cip_base_vseq.sv:828) [uvm_test_top.env.virtual_sequencer.hmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 937830251 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```