HMAC Simulation Results

Wednesday June 26 2024 23:02:36 UTC

GitHub Revision: `be1c4a4f52`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 44766564427213563291105655232733134394512207819884794315335669279596867428010

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	hmac_smoke	18.910s	3.988ms	50	50	100.00
V1	csr_hw_reset	hmac_csr_hw_reset	1.010s	43.085us	5	5	100.00
V1	csr_rw	hmac_csr_rw	1.030s	61.734us	20	20	100.00
V1	csr_bit_bash	hmac_csr_bit_bash	17.070s	1.643ms	5	5	100.00
V1	csr_aliasing	hmac_csr_aliasing	8.340s	1.762ms	5	5	100.00
V1	csr_mem_rw_with_rand_reset	hmac_csr_mem_rw_with_rand_reset	26.105m	619.406ms	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	hmac_csr_rw	1.030s	61.734us	20	20	100.00
		hmac_csr_aliasing	8.340s	1.762ms	5	5	100.00
V1		TOTAL			105	105	100.00
V2	long_msg	hmac_long_msg	2.504m	11.265ms	50	50	100.00
V2	back_pressure	hmac_back_pressure	59.960s	5.059ms	50	50	100.00
V2	test_vectors	hmac_test_sha256_vectors	9.467m	83.274ms	50	50	100.00
		hmac_test_sha384_vectors	38.212m	173.286ms	48	50	96.00
		hmac_test_sha512_vectors	36.089m	705.273ms	47	50	94.00
		hmac_test_hmac_vectors	1.500s	168.844us	50	50	100.00
V2	burst_wr	hmac_burst_wr	1.198m	12.276ms	50	50	100.00
V2	datapath_stress	hmac_datapath_stress	24.359m	5.096ms	50	50	100.00
V2	error	hmac_error	3.284m	64.176ms	50	50	100.00
V2	wipe_secret	hmac_wipe_secret	1.467m	2.098ms	50	50	100.00
V2	save_and_restore	hmac_smoke	18.910s	3.988ms	50	50	100.00
		hmac_long_msg	2.504m	11.265ms	50	50	100.00
		hmac_back_pressure	59.960s	5.059ms	50	50	100.00
		hmac_datapath_stress	24.359m	5.096ms	50	50	100.00
		hmac_burst_wr	1.198m	12.276ms	50	50	100.00
		hmac_stress_all	39.008m	29.524ms	8	50	16.00
V2	fifo_empty_status_interrupt	fifo_empty_status_interrupt			0	0	--
V2	wide_digest_configurable_key_length	hmac_smoke	18.910s	3.988ms	50	50	100.00
		hmac_long_msg	2.504m	11.265ms	50	50	100.00
		hmac_back_pressure	59.960s	5.059ms	50	50	100.00
		hmac_datapath_stress	24.359m	5.096ms	50	50	100.00
		hmac_burst_wr	1.198m	12.276ms	50	50	100.00
		hmac_error	3.284m	64.176ms	50	50	100.00
		hmac_wipe_secret	1.467m	2.098ms	50	50	100.00
		hmac_test_sha256_vectors	9.467m	83.274ms	50	50	100.00
		hmac_test_sha384_vectors	38.212m	173.286ms	48	50	96.00
		hmac_test_sha512_vectors	36.089m	705.273ms	47	50	94.00
		hmac_test_hmac_vectors	1.500s	168.844us	50	50	100.00
		hmac_stress_all	39.008m	29.524ms	8	50	16.00
V2	stress_all	hmac_stress_all	39.008m	29.524ms	8	50	16.00
V2	alert_test	hmac_alert_test	0.690s	23.517us	50	50	100.00
V2	intr_test	hmac_intr_test	0.720s	33.235us	50	50	100.00
V2	tl_d_oob_addr_access	hmac_tl_errors	4.220s	226.300us	20	20	100.00
V2	tl_d_illegal_access	hmac_tl_errors	4.220s	226.300us	20	20	100.00
V2	tl_d_outstanding_access	hmac_csr_hw_reset	1.010s	43.085us	5	5	100.00
		hmac_csr_rw	1.030s	61.734us	20	20	100.00
		hmac_csr_aliasing	8.340s	1.762ms	5	5	100.00
		hmac_same_csr_outstanding	2.370s	154.769us	20	20	100.00
V2	tl_d_partial_access	hmac_csr_hw_reset	1.010s	43.085us	5	5	100.00
		hmac_csr_rw	1.030s	61.734us	20	20	100.00
		hmac_csr_aliasing	8.340s	1.762ms	5	5	100.00
		hmac_same_csr_outstanding	2.370s	154.769us	20	20	100.00
V2		TOTAL			643	690	93.19
V2S	tl_intg_err	hmac_sec_cm	1.010s	424.708us	5	5	100.00
		hmac_tl_intg_err	4.480s	253.493us	20	20	100.00
V2S	sec_cm_bus_integrity	hmac_tl_intg_err	4.480s	253.493us	20	20	100.00
V2S		TOTAL			25	25	100.00
V3	write_config_and_secret_key_during_msg_wr	hmac_smoke	18.910s	3.988ms	50	50	100.00
V3	stress_all_with_rand_reset	hmac_stress_all_with_rand_reset	20.064m	123.617ms	0	10	0.00
V3		TOTAL			0	10	0.00
		TOTAL			773	830	93.13

Testplan Progress

Items	Total	Written	Passing	Progress
V1	6	6	6	100.00
V2	16	15	12	75.00
V2S	2	2	2	100.00
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
92.58	95.85	93.09	100.00	74.36	91.89	99.49	93.40

Failure Buckets

UVM_FATAL (hmac_test_vectors_sha_vseq.sv:28) virtual_sequencer [hmac_test_vectors_hmac_vseq] This test sequence requires a command-line argument: sha2_digest_size=SHA2_*/SHA2_*/SHA2_* has 26 failures:

Test hmac_stress_all_with_rand_reset has 3 failures.

0.hmac_stress_all_with_rand_reset.62936680295009322573599691072399875700809540395575204616933266275990148930527
Line 121738, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/0.hmac_stress_all_with_rand_reset/latest/run.log

  UVM_FATAL @ 123617183264 ps: (hmac_test_vectors_sha_vseq.sv:28) uvm_test_top.env.virtual_sequencer [uvm_test_top.env.virtual_sequencer.hmac_test_vectors_hmac_vseq] This test sequence requires a command-line argument: sha2_digest_size=SHA2_256/SHA2_384/SHA2_512
  UVM_INFO @ 123617183264 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

5.hmac_stress_all_with_rand_reset.32781548374581996876751841759542928290559018234152887828723478060587470245136
Line 18146, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/5.hmac_stress_all_with_rand_reset/latest/run.log

  UVM_FATAL @ 8479771564 ps: (hmac_test_vectors_sha_vseq.sv:28) uvm_test_top.env.virtual_sequencer [uvm_test_top.env.virtual_sequencer.hmac_test_vectors_hmac_vseq] This test sequence requires a command-line argument: sha2_digest_size=SHA2_256/SHA2_384/SHA2_512
  UVM_INFO @ 8479771564 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 1 more failures.

Test hmac_stress_all has 23 failures.

1.hmac_stress_all.40741876102868543522846102615874406848460111170426274386103217198625532227945
Line 89994, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/1.hmac_stress_all/latest/run.log

  UVM_FATAL @ 10338353152 ps: (hmac_test_vectors_sha_vseq.sv:28) uvm_test_top.env.virtual_sequencer [uvm_test_top.env.virtual_sequencer.hmac_test_vectors_hmac_vseq] This test sequence requires a command-line argument: sha2_digest_size=SHA2_256/SHA2_384/SHA2_512
  UVM_INFO @ 10338353152 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

3.hmac_stress_all.65812152057352311536660782310288822937619341765774963547776360701861980535966
Line 38589, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/3.hmac_stress_all/latest/run.log

  UVM_FATAL @ 6717485040 ps: (hmac_test_vectors_sha_vseq.sv:28) uvm_test_top.env.virtual_sequencer [uvm_test_top.env.virtual_sequencer.hmac_test_vectors_hmac_vseq] This test sequence requires a command-line argument: sha2_digest_size=SHA2_256/SHA2_384/SHA2_512
  UVM_INFO @ 6717485040 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 21 more failures.

UVM_FATAL (hmac_test_vectors_sha_vseq.sv:28) virtual_sequencer [hmac_test_vectors_sha_vseq] This test sequence requires a command-line argument: sha2_digest_size=SHA2_*/SHA2_*/SHA2_* has 22 failures:

Test hmac_stress_all has 19 failures.

0.hmac_stress_all.43857911552356865253312388016522507875153743251086632085841070202621378508807
Line 34837, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/0.hmac_stress_all/latest/run.log

  UVM_FATAL @ 3655710989 ps: (hmac_test_vectors_sha_vseq.sv:28) uvm_test_top.env.virtual_sequencer [uvm_test_top.env.virtual_sequencer.hmac_test_vectors_sha_vseq] This test sequence requires a command-line argument: sha2_digest_size=SHA2_256/SHA2_384/SHA2_512
  UVM_INFO @ 3655710989 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

2.hmac_stress_all.57286781755169192356047350400958523505043017232373784798834227178907575116412
Line 249, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/2.hmac_stress_all/latest/run.log

  UVM_FATAL @   4412144 ps: (hmac_test_vectors_sha_vseq.sv:28) uvm_test_top.env.virtual_sequencer [uvm_test_top.env.virtual_sequencer.hmac_test_vectors_sha_vseq] This test sequence requires a command-line argument: sha2_digest_size=SHA2_256/SHA2_384/SHA2_512
  UVM_INFO @   4412144 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 17 more failures.

Test hmac_stress_all_with_rand_reset has 3 failures.

1.hmac_stress_all_with_rand_reset.30829136786803055967994452733503561850696160533542758609800489539827814130260
Line 251, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/1.hmac_stress_all_with_rand_reset/latest/run.log

  UVM_FATAL @   7257052 ps: (hmac_test_vectors_sha_vseq.sv:28) uvm_test_top.env.virtual_sequencer [uvm_test_top.env.virtual_sequencer.hmac_test_vectors_sha_vseq] This test sequence requires a command-line argument: sha2_digest_size=SHA2_256/SHA2_384/SHA2_512
  UVM_INFO @   7257052 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

4.hmac_stress_all_with_rand_reset.75930516251951118597893447868753497692178501961311964591621360068446503037875
Line 10305, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/4.hmac_stress_all_with_rand_reset/latest/run.log

  UVM_FATAL @ 664620258 ps: (hmac_test_vectors_sha_vseq.sv:28) uvm_test_top.env.virtual_sequencer [uvm_test_top.env.virtual_sequencer.hmac_test_vectors_sha_vseq] This test sequence requires a command-line argument: sha2_digest_size=SHA2_256/SHA2_384/SHA2_512
  UVM_INFO @ 664620258 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 1 more failures.

UVM_FATAL (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of * ps hit, indicating a probable testbench issue has 5 failures:
- Test hmac_test_sha512_vectors has 3 failures.
  - 9.hmac_test_sha512_vectors.86705510332554612783856010953633691182250007247579088118112228531245633529419
    Line 263944, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/9.hmac_test_sha512_vectors/latest/run.log
```
  UVM_FATAL @ 1000000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 1000000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 1000000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 14.hmac_test_sha512_vectors.19385410466765861634121442117640314936467179563457786108664389635614201927563
    Line 152291, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/14.hmac_test_sha512_vectors/latest/run.log
```
  UVM_FATAL @ 1000000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 1000000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 1000000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 1 more failures.
- Test hmac_test_sha384_vectors has 2 failures.
  - 11.hmac_test_sha384_vectors.19491193230808536644814837012055979025439047554616022030492698653541002113340
    Line 234814, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/11.hmac_test_sha384_vectors/latest/run.log
```
  UVM_FATAL @ 1000000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 1000000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 1000000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 22.hmac_test_sha384_vectors.95265366650998784929070600172099426836695181358897673441516669937057879873318
    Line 83995, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/22.hmac_test_sha384_vectors/latest/run.log
```
  UVM_FATAL @ 1000000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 1000000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 1000000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (cip_base_vseq.sv:828) [hmac_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 4 failures:
- Test hmac_stress_all_with_rand_reset has 4 failures.
  - 2.hmac_stress_all_with_rand_reset.27588925510912765022517754789392903064113269078442909322195129084030897396938
    Line 2976, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/2.hmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 10004214532 ps: (cip_base_vseq.sv:828) [uvm_test_top.env.virtual_sequencer.hmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 10004214532 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 3.hmac_stress_all_with_rand_reset.66535893628911935522186941207087918890137250189043260596519773614125133176415
    Line 20932, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/3.hmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 41337990863 ps: (cip_base_vseq.sv:828) [uvm_test_top.env.virtual_sequencer.hmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 41337990863 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 2 more failures.