HMAC Simulation Results

Thursday April 04 2024 19:02:33 UTC

GitHub Revision: `2723ca659d`

Branch: os_regression

Testplan

Simulator: VCS

Build randomization enabled with --build-seed 9870132716819564205271541124341458297216848204999383102382742091236484427981

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	hmac_smoke	7.290s	2.596ms	50	50	100.00
V1	csr_hw_reset	hmac_csr_hw_reset	1.000s	44.458us	5	5	100.00
V1	csr_rw	hmac_csr_rw	0.990s	96.062us	20	20	100.00
V1	csr_bit_bash	hmac_csr_bit_bash	14.480s	323.995us	5	5	100.00
V1	csr_aliasing	hmac_csr_aliasing	8.960s	1.860ms	5	5	100.00
V1	csr_mem_rw_with_rand_reset	hmac_csr_mem_rw_with_rand_reset	15.487m	319.964ms	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	hmac_csr_rw	0.990s	96.062us	20	20	100.00
		hmac_csr_aliasing	8.960s	1.860ms	5	5	100.00
V1		TOTAL			105	105	100.00
V2	long_msg	hmac_long_msg	2.334m	25.178ms	50	50	100.00
V2	back_pressure	hmac_back_pressure	1.157m	3.466ms	50	50	100.00
V2	test_vectors	hmac_test_sha_vectors	9.152m	92.405ms	49	50	98.00
		hmac_test_hmac_vectors	1.460s	182.134us	50	50	100.00
V2	burst_wr	hmac_burst_wr	1.100m	19.809ms	50	50	100.00
V2	datapath_stress	hmac_datapath_stress	3.235m	3.286ms	50	50	100.00
V2	error	hmac_error	5.331m	35.659ms	50	50	100.00
V2	wipe_secret	hmac_wipe_secret	1.746m	25.859ms	50	50	100.00
V2	save_and_restore	save_and_restore			0	0	--
V2	fifo_empty_status_interrupt	fifo_empty_status_interrupt			0	0	--
V2	wide_digest_configurable_key_length	wide_digest_configurable_key_length			0	0	--
V2	stress_all	hmac_stress_all	46.391m	224.965ms	50	50	100.00
V2	alert_test	hmac_alert_test	0.690s	16.234us	50	50	100.00
V2	intr_test	hmac_intr_test	0.640s	28.349us	50	50	100.00
V2	tl_d_oob_addr_access	hmac_tl_errors	3.640s	231.988us	20	20	100.00
V2	tl_d_illegal_access	hmac_tl_errors	3.640s	231.988us	20	20	100.00
V2	tl_d_outstanding_access	hmac_csr_hw_reset	1.000s	44.458us	5	5	100.00
		hmac_csr_rw	0.990s	96.062us	20	20	100.00
		hmac_csr_aliasing	8.960s	1.860ms	5	5	100.00
		hmac_same_csr_outstanding	2.420s	230.946us	20	20	100.00
V2	tl_d_partial_access	hmac_csr_hw_reset	1.000s	44.458us	5	5	100.00
		hmac_csr_rw	0.990s	96.062us	20	20	100.00
		hmac_csr_aliasing	8.960s	1.860ms	5	5	100.00
		hmac_same_csr_outstanding	2.420s	230.946us	20	20	100.00
V2		TOTAL			589	590	99.83
V2S	tl_intg_err	hmac_sec_cm	0.990s	62.558us	5	5	100.00
		hmac_tl_intg_err	5.060s	557.921us	20	20	100.00
V2S	sec_cm_bus_integrity	hmac_tl_intg_err	5.060s	557.921us	20	20	100.00
V2S		TOTAL			25	25	100.00
V3	write_config_and_secret_key_during_msg_wr	hmac_smoke	7.290s	2.596ms	50	50	100.00
V3	stress_all_with_rand_reset	hmac_stress_all_with_rand_reset	1.101h	101.907ms	20	200	10.00
V3		TOTAL			20	200	10.00
		TOTAL			739	920	80.33

Testplan Progress

Items	Total	Written	Passing	Progress
V1	6	6	6	100.00
V2	16	13	12	75.00
V2S	2	2	2	100.00
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
86.93	92.46	85.23	100.00	76.32	85.93	99.49	69.08

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:829) [hmac_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 178 failures:
- Test hmac_stress_all_with_rand_reset has 178 failures.
  - 0.hmac_stress_all_with_rand_reset.108028147313635349753258973425154976411142952505630266314795268483500897211143
    Line 26488, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/0.hmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 13260739494 ps: (cip_base_vseq.sv:829) [uvm_test_top.env.virtual_sequencer.hmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 13260739494 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 1.hmac_stress_all_with_rand_reset.10974350999169265580362013987764080967562720637779980736779941840372268916677
    Line 10589, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/1.hmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 23533556760 ps: (cip_base_vseq.sv:829) [uvm_test_top.env.virtual_sequencer.hmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 23533556760 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 176 more failures.
UVM_FATAL (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of * ps hit, indicating a probable testbench issue has 1 failures:
- Test hmac_test_sha_vectors has 1 failures.
  - 24.hmac_test_sha_vectors.25818586837219651306245018283470264611421250880413807749948553192261652530109
    Line 80694, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/24.hmac_test_sha_vectors/latest/run.log
```
  UVM_FATAL @ 800000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 800000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 800000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (hmac_scoreboard.sv:289) [scoreboard] Check failed real_digest_val == exp_digest[digest_idx] (* [*] vs * [*]) has 1 failures:
- Test hmac_stress_all_with_rand_reset has 1 failures.
  - 111.hmac_stress_all_with_rand_reset.105476982698555085376311712282645293956862059417813018370505003462212068274983
    Line 9438, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/111.hmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 1417033480 ps: (hmac_scoreboard.sv:289) [uvm_test_top.env.scoreboard] Check failed real_digest_val == exp_digest[digest_idx] (0 [0x0] vs 1184355213 [0x4697d38d])
  UVM_INFO @ 1417033480 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (cip_base_vseq.sv:753) [hmac_common_vseq] Check failed (!has_outstanding_access()) Trying to trigger a reset with outstanding CSR items. has 1 failures:
- Test hmac_stress_all_with_rand_reset has 1 failures.
  - 150.hmac_stress_all_with_rand_reset.67903836483758419838995573315154712203985524078032368657833804423071337080995
    Line 322, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/150.hmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @  22706766 ps: (cip_base_vseq.sv:753) [uvm_test_top.env.virtual_sequencer.hmac_common_vseq] Check failed (!has_outstanding_access()) Trying to trigger a reset with outstanding CSR items.
  UVM_INFO @  22706766 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```

HMAC Simulation Results

Thursday April 04 2024 19:02:33 UTC

GitHub Revision: 2723ca659d

Branch: os_regression

Testplan

Simulator: VCS

Build randomization enabled with --build-seed 9870132716819564205271541124341458297216848204999383102382742091236484427981

Test Results

Testplan Progress

Coverage Results

Coverage Dashboard

Failure Buckets

Past Results

GitHub Revision: `2723ca659d`