HMAC Simulation Results

Wednesday February 21 2024 20:04:41 UTC

GitHub Revision: `df66f8a42e`

Branch: os_regression

Testplan

Simulator: VCS

Build randomization enabled with --build-seed 105428938048998514387352931012238053576571450380985277214810281406530880002461

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	hmac_smoke	5.010s	862.637us	50	50	100.00
V1	csr_hw_reset	hmac_csr_hw_reset	0.750s	26.819us	5	5	100.00
V1	csr_rw	hmac_csr_rw	0.760s	65.415us	20	20	100.00
V1	csr_bit_bash	hmac_csr_bit_bash	9.560s	16.048ms	5	5	100.00
V1	csr_aliasing	hmac_csr_aliasing	2.670s	159.978us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	hmac_csr_mem_rw_with_rand_reset	6.871m	84.128ms	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	hmac_csr_rw	0.760s	65.415us	20	20	100.00
		hmac_csr_aliasing	2.670s	159.978us	5	5	100.00
V1		TOTAL			105	105	100.00
V2	long_msg	hmac_long_msg	1.702m	14.969ms	50	50	100.00
V2	back_pressure	hmac_back_pressure	1.078m	1.626ms	50	50	100.00
V2	test_vectors	hmac_test_sha_vectors	9.222m	45.215ms	46	50	92.00
		hmac_test_hmac_vectors	1.250s	279.110us	50	50	100.00
V2	burst_wr	hmac_burst_wr	1.184m	5.704ms	50	50	100.00
V2	datapath_stress	hmac_datapath_stress	2.420m	10.254ms	50	50	100.00
V2	error	hmac_error	3.793m	27.205ms	49	50	98.00
V2	wipe_secret	hmac_wipe_secret	1.359m	10.700ms	50	50	100.00
V2	stress_all	hmac_stress_all	43.755m	735.549ms	50	50	100.00
V2	alert_test	hmac_alert_test	0.620s	52.314us	50	50	100.00
V2	intr_test	hmac_intr_test	0.670s	28.472us	50	50	100.00
V2	tl_d_oob_addr_access	hmac_tl_errors	4.060s	860.283us	20	20	100.00
V2	tl_d_illegal_access	hmac_tl_errors	4.060s	860.283us	20	20	100.00
V2	tl_d_outstanding_access	hmac_csr_hw_reset	0.750s	26.819us	5	5	100.00
		hmac_csr_rw	0.760s	65.415us	20	20	100.00
		hmac_csr_aliasing	2.670s	159.978us	5	5	100.00
		hmac_same_csr_outstanding	1.500s	80.511us	20	20	100.00
V2	tl_d_partial_access	hmac_csr_hw_reset	0.750s	26.819us	5	5	100.00
		hmac_csr_rw	0.760s	65.415us	20	20	100.00
		hmac_csr_aliasing	2.670s	159.978us	5	5	100.00
		hmac_same_csr_outstanding	1.500s	80.511us	20	20	100.00
V2		TOTAL			585	590	99.15
V2S	tl_intg_err	hmac_sec_cm	0.890s	126.620us	5	5	100.00
		hmac_tl_intg_err	2.890s	2.209ms	20	20	100.00
V2S	sec_cm_bus_integrity	hmac_tl_intg_err	2.890s	2.209ms	20	20	100.00
V2S		TOTAL			25	25	100.00
V3	write_config_and_secret_key_during_msg_wr	hmac_smoke	5.010s	862.637us	50	50	100.00
V3	stress_all_with_rand_reset	hmac_stress_all_with_rand_reset	49.747m	83.451ms	12	200	6.00
V3		TOTAL			12	200	6.00
		TOTAL			727	920	79.02

Testplan Progress

Items	Total	Written	Passing	Progress
V1	6	6	6	100.00
V2	13	13	11	84.62
V2S	2	2	2	100.00
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
98.81	99.55	98.55	100.00	96.30	98.47	99.49	99.31

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:774) [hmac_common_vseq] Check failed (!has_outstanding_access()) Waited too long to issue a reset with no outstanding accesses. has 181 failures:
- Test hmac_stress_all_with_rand_reset has 181 failures.
  - 0.hmac_stress_all_with_rand_reset.69405786368698111340722402982904394198125253962564447429068516754824083876638
    Line 286, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/0.hmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 10491298030 ps: (cip_base_vseq.sv:774) [uvm_test_top.env.virtual_sequencer.hmac_common_vseq] Check failed (!has_outstanding_access()) Waited too long to issue a reset with no outstanding accesses.
  UVM_INFO @ 10491298030 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 1.hmac_stress_all_with_rand_reset.85591656939545506470425789980892017391659357084741912249156844049790655269718
    Line 530, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/1.hmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 26711152785 ps: (cip_base_vseq.sv:774) [uvm_test_top.env.virtual_sequencer.hmac_common_vseq] Check failed (!has_outstanding_access()) Waited too long to issue a reset with no outstanding accesses.
  UVM_INFO @ 26711152785 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 179 more failures.
UVM_ERROR (cip_base_vseq.sv:719) [hmac_common_vseq] Check failed (!has_outstanding_access()) Trying to trigger a reset with outstanding CSR items. has 7 failures:
- Test hmac_stress_all_with_rand_reset has 7 failures.
  - 24.hmac_stress_all_with_rand_reset.27340834979414741345579863778646388382431271075847814272536054231716830150138
    Line 319, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/24.hmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 4835764793 ps: (cip_base_vseq.sv:719) [uvm_test_top.env.virtual_sequencer.hmac_common_vseq] Check failed (!has_outstanding_access()) Trying to trigger a reset with outstanding CSR items.
  UVM_INFO @ 4835764793 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 102.hmac_stress_all_with_rand_reset.1040649858692305110739863726744845531073819647730236797400128648675303609138
    Line 478, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/102.hmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 16226730789 ps: (cip_base_vseq.sv:719) [uvm_test_top.env.virtual_sequencer.hmac_common_vseq] Check failed (!has_outstanding_access()) Trying to trigger a reset with outstanding CSR items.
  UVM_INFO @ 16226730789 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 5 more failures.
UVM_FATAL (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of * ps hit, indicating a probable testbench issue has 5 failures:
- Test hmac_error has 1 failures.
  - 3.hmac_error.51411030622798797922813230183415070207299178049892493574393384868837149917466
    Line 291, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/3.hmac_error/latest/run.log
```
  UVM_FATAL @ 200000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 200000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 200000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test hmac_test_sha_vectors has 4 failures.
  - 8.hmac_test_sha_vectors.78247367253465383814471633127432460520906882079927211805769043469001297746542
    Line 248, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/8.hmac_test_sha_vectors/latest/run.log
```
  UVM_FATAL @ 200000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 200000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 200000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 15.hmac_test_sha_vectors.9987416049486716413294769123343553405981932303070631643141448552949042771343
    Line 248, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/15.hmac_test_sha_vectors/latest/run.log
```
  UVM_FATAL @ 200000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 200000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 200000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 2 more failures.

HMAC Simulation Results

Wednesday February 21 2024 20:04:41 UTC

GitHub Revision: df66f8a42e

Branch: os_regression

Testplan

Simulator: VCS

Build randomization enabled with --build-seed 105428938048998514387352931012238053576571450380985277214810281406530880002461

Test Results

Testplan Progress

Coverage Results

Coverage Dashboard

Failure Buckets

Past Results

GitHub Revision: `df66f8a42e`