HMAC Simulation Results

Thursday March 21 2024 19:02:46 UTC

GitHub Revision: `e3ca274e77`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 110450978848188291656921294920309436568649534904994074551053469482156204817270

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	hmac_smoke	7.380s	4.928ms	50	50	100.00
V1	csr_hw_reset	hmac_csr_hw_reset	1.020s	79.244us	5	5	100.00
V1	csr_rw	hmac_csr_rw	0.980s	31.435us	20	20	100.00
V1	csr_bit_bash	hmac_csr_bit_bash	15.940s	1.053ms	5	5	100.00
V1	csr_aliasing	hmac_csr_aliasing	6.130s	737.549us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	hmac_csr_mem_rw_with_rand_reset	25.764m	1.258s	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	hmac_csr_rw	0.980s	31.435us	20	20	100.00
		hmac_csr_aliasing	6.130s	737.549us	5	5	100.00
V1		TOTAL			105	105	100.00
V2	long_msg	hmac_long_msg	2.376m	29.191ms	50	50	100.00
V2	back_pressure	hmac_back_pressure	1.281m	14.133ms	50	50	100.00
V2	test_vectors	hmac_test_sha_vectors	9.631m	191.246ms	50	50	100.00
		hmac_test_hmac_vectors	1.440s	218.909us	50	50	100.00
V2	burst_wr	hmac_burst_wr	1.253m	21.417ms	50	50	100.00
V2	datapath_stress	hmac_datapath_stress	2.420m	4.654ms	50	50	100.00
V2	error	hmac_error	4.224m	103.641ms	50	50	100.00
V2	wipe_secret	hmac_wipe_secret	1.524m	23.013ms	50	50	100.00
V2	stress_all	hmac_stress_all	34.840m	1.666s	48	50	96.00
V2	alert_test	hmac_alert_test	0.650s	14.645us	50	50	100.00
V2	intr_test	hmac_intr_test	0.690s	14.678us	50	50	100.00
V2	tl_d_oob_addr_access	hmac_tl_errors	4.350s	398.129us	20	20	100.00
V2	tl_d_illegal_access	hmac_tl_errors	4.350s	398.129us	20	20	100.00
V2	tl_d_outstanding_access	hmac_csr_hw_reset	1.020s	79.244us	5	5	100.00
		hmac_csr_rw	0.980s	31.435us	20	20	100.00
		hmac_csr_aliasing	6.130s	737.549us	5	5	100.00
		hmac_same_csr_outstanding	2.540s	155.322us	20	20	100.00
V2	tl_d_partial_access	hmac_csr_hw_reset	1.020s	79.244us	5	5	100.00
		hmac_csr_rw	0.980s	31.435us	20	20	100.00
		hmac_csr_aliasing	6.130s	737.549us	5	5	100.00
		hmac_same_csr_outstanding	2.540s	155.322us	20	20	100.00
V2		TOTAL			588	590	99.66
V2S	tl_intg_err	hmac_sec_cm	0.980s	172.419us	5	5	100.00
		hmac_tl_intg_err	4.650s	227.482us	20	20	100.00
V2S	sec_cm_bus_integrity	hmac_tl_intg_err	4.650s	227.482us	20	20	100.00
V2S		TOTAL			25	25	100.00
V3	write_config_and_secret_key_during_msg_wr	hmac_smoke	7.380s	4.928ms	50	50	100.00
V3	stress_all_with_rand_reset	hmac_stress_all_with_rand_reset	1.118h	160.087ms	13	200	6.50
V3		TOTAL			13	200	6.50
		TOTAL			731	920	79.46

Testplan Progress

Items	Total	Written	Passing	Progress
V1	6	6	6	100.00
V2	13	13	12	92.31
V2S	2	2	2	100.00
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
87.78	92.80	85.98	100.00	78.95	88.15	99.49	69.08

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:829) [hmac_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 185 failures:
- Test hmac_stress_all_with_rand_reset has 185 failures.
  - 0.hmac_stress_all_with_rand_reset.20252419669121621312679423876039460099512040154062901874565253515169278753142
    Line 19781, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/0.hmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 54627509527 ps: (cip_base_vseq.sv:829) [uvm_test_top.env.virtual_sequencer.hmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 54627509527 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 1.hmac_stress_all_with_rand_reset.87054328763736031158921176651883765529844744895305796077409937725642737943606
    Line 45183, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/1.hmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 19872236763 ps: (cip_base_vseq.sv:829) [uvm_test_top.env.virtual_sequencer.hmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 19872236763 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 183 more failures.

UVM_ERROR (hmac_scoreboard.sv:289) [scoreboard] Check failed real_digest_val == exp_digest[digest_idx] (* [*] vs * [*]) has 3 failures:

Test hmac_stress_all_with_rand_reset has 1 failures.

2.hmac_stress_all_with_rand_reset.86813598845833727550419690945778338203626897160228824171092041721779601896013
Line 6714, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/2.hmac_stress_all_with_rand_reset/latest/run.log

  UVM_ERROR @ 998033045 ps: (hmac_scoreboard.sv:289) [uvm_test_top.env.scoreboard] Check failed real_digest_val == exp_digest[digest_idx] (0 [0x0] vs 2772209687 [0xa53c9017])
  UVM_INFO @ 998033045 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test hmac_stress_all has 2 failures.

31.hmac_stress_all.8077226432532644161432188963774723566763640600793189554027738647500061181827
Line 47536, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/31.hmac_stress_all/latest/run.log

  UVM_ERROR @ 3012907983 ps: (hmac_scoreboard.sv:289) [uvm_test_top.env.scoreboard] Check failed real_digest_val == exp_digest[digest_idx] (0 [0x0] vs 3505336185 [0xd0ef2f79])
  UVM_INFO @ 3012907983 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

32.hmac_stress_all.8059248559885269977309992356345097525063422775181604164908454186144401820556
Line 24982, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/32.hmac_stress_all/latest/run.log

  UVM_ERROR @ 5752250708 ps: (hmac_scoreboard.sv:289) [uvm_test_top.env.scoreboard] Check failed real_digest_val == exp_digest[digest_idx] (0 [0x0] vs 1311061055 [0x4e25343f])
  UVM_INFO @ 5752250708 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_ERROR (cip_base_vseq.sv:753) [hmac_common_vseq] Check failed (!has_outstanding_access()) Trying to trigger a reset with outstanding CSR items. has 1 failures:
- Test hmac_stress_all_with_rand_reset has 1 failures.
  - 175.hmac_stress_all_with_rand_reset.47728186225968444507109633750613314238384925463145746670426659984630572903584
    Line 33026, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/175.hmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 83865086591 ps: (cip_base_vseq.sv:753) [uvm_test_top.env.virtual_sequencer.hmac_common_vseq] Check failed (!has_outstanding_access()) Trying to trigger a reset with outstanding CSR items.
  UVM_INFO @ 83865086591 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```