HMAC Simulation Results

Tuesday September 24 2024 01:05:57 UTC

GitHub Revision: `78ad89d1aa`

Branch: os_regression_2024_09_23

Testplan

Simulator: VCS

Build randomization enabled with --build-seed 34048022127553017884926631616394166155118623175048314192737094530054579848544

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	hmac_smoke	26.320s	3.236ms	50	50	100.00
V1	csr_hw_reset	hmac_csr_hw_reset	1.090s	35.093us	5	5	100.00
V1	csr_rw	hmac_csr_rw	1.060s	17.527us	20	20	100.00
V1	csr_bit_bash	hmac_csr_bit_bash	10.920s	1.015ms	5	5	100.00
V1	csr_aliasing	hmac_csr_aliasing	8.190s	1.804ms	5	5	100.00
V1	csr_mem_rw_with_rand_reset	hmac_csr_mem_rw_with_rand_reset	23.968m	507.168ms	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	hmac_csr_rw	1.060s	17.527us	20	20	100.00
		hmac_csr_aliasing	8.190s	1.804ms	5	5	100.00
V1		TOTAL			105	105	100.00
V2	long_msg	hmac_long_msg	3.844m	42.936ms	50	50	100.00
V2	back_pressure	hmac_back_pressure	2.128m	1.646ms	50	50	100.00
V2	test_vectors	hmac_test_sha256_vectors	14.899m	59.101ms	5	5	100.00
		hmac_test_sha384_vectors	57.828m	836.232ms	5	5	100.00
		hmac_test_sha512_vectors	50.564m	85.878ms	5	5	100.00
		hmac_test_hmac256_vectors	1.415m	9.525ms	5	5	100.00
		hmac_test_hmac384_vectors	1.899m	2.285ms	5	5	100.00
		hmac_test_hmac512_vectors	2.320m	15.631ms	5	5	100.00
V2	burst_wr	hmac_burst_wr	1.635m	20.673ms	50	50	100.00
V2	datapath_stress	hmac_datapath_stress	33.748m	33.142ms	50	50	100.00
V2	error	hmac_error	5.641m	16.548ms	50	50	100.00
V2	wipe_secret	hmac_wipe_secret	3.432m	8.336ms	50	50	100.00
V2	save_and_restore	hmac_smoke	26.320s	3.236ms	50	50	100.00
		hmac_long_msg	3.844m	42.936ms	50	50	100.00
		hmac_back_pressure	2.128m	1.646ms	50	50	100.00
		hmac_datapath_stress	33.748m	33.142ms	50	50	100.00
		hmac_burst_wr	1.635m	20.673ms	50	50	100.00
		hmac_stress_all	1.686h	104.532ms	50	50	100.00
V2	fifo_empty_status_interrupt	hmac_smoke	26.320s	3.236ms	50	50	100.00
		hmac_long_msg	3.844m	42.936ms	50	50	100.00
		hmac_back_pressure	2.128m	1.646ms	50	50	100.00
		hmac_datapath_stress	33.748m	33.142ms	50	50	100.00
		hmac_wipe_secret	3.432m	8.336ms	50	50	100.00
		hmac_test_sha256_vectors	14.899m	59.101ms	5	5	100.00
		hmac_test_sha384_vectors	57.828m	836.232ms	5	5	100.00
		hmac_test_sha512_vectors	50.564m	85.878ms	5	5	100.00
		hmac_test_hmac256_vectors	1.415m	9.525ms	5	5	100.00
		hmac_test_hmac384_vectors	1.899m	2.285ms	5	5	100.00
		hmac_test_hmac512_vectors	2.320m	15.631ms	5	5	100.00
V2	wide_digest_configurable_key_length	hmac_smoke	26.320s	3.236ms	50	50	100.00
		hmac_long_msg	3.844m	42.936ms	50	50	100.00
		hmac_back_pressure	2.128m	1.646ms	50	50	100.00
		hmac_datapath_stress	33.748m	33.142ms	50	50	100.00
		hmac_burst_wr	1.635m	20.673ms	50	50	100.00
		hmac_error	5.641m	16.548ms	50	50	100.00
		hmac_wipe_secret	3.432m	8.336ms	50	50	100.00
		hmac_test_sha256_vectors	14.899m	59.101ms	5	5	100.00
		hmac_test_sha384_vectors	57.828m	836.232ms	5	5	100.00
		hmac_test_sha512_vectors	50.564m	85.878ms	5	5	100.00
		hmac_test_hmac256_vectors	1.415m	9.525ms	5	5	100.00
		hmac_test_hmac384_vectors	1.899m	2.285ms	5	5	100.00
		hmac_test_hmac512_vectors	2.320m	15.631ms	5	5	100.00
		hmac_stress_all	1.686h	104.532ms	50	50	100.00
V2	stress_all	hmac_stress_all	1.686h	104.532ms	50	50	100.00
V2	alert_test	hmac_alert_test	1.010s	11.580us	50	50	100.00
V2	intr_test	hmac_intr_test	0.880s	15.666us	50	50	100.00
V2	tl_d_oob_addr_access	hmac_tl_errors	3.890s	385.247us	20	20	100.00
V2	tl_d_illegal_access	hmac_tl_errors	3.890s	385.247us	20	20	100.00
V2	tl_d_outstanding_access	hmac_csr_hw_reset	1.090s	35.093us	5	5	100.00
		hmac_csr_rw	1.060s	17.527us	20	20	100.00
		hmac_csr_aliasing	8.190s	1.804ms	5	5	100.00
		hmac_same_csr_outstanding	2.430s	136.140us	20	20	100.00
V2	tl_d_partial_access	hmac_csr_hw_reset	1.090s	35.093us	5	5	100.00
		hmac_csr_rw	1.060s	17.527us	20	20	100.00
		hmac_csr_aliasing	8.190s	1.804ms	5	5	100.00
		hmac_same_csr_outstanding	2.430s	136.140us	20	20	100.00
V2		TOTAL			520	520	100.00
V2S	tl_intg_err	hmac_sec_cm	1.410s	332.988us	5	5	100.00
		hmac_tl_intg_err	4.360s	571.179us	20	20	100.00
V2S	sec_cm_bus_integrity	hmac_tl_intg_err	4.360s	571.179us	20	20	100.00
V2S		TOTAL			25	25	100.00
V3	write_config_and_secret_key_during_msg_wr	hmac_smoke	26.320s	3.236ms	50	50	100.00
V3	stress_all_with_rand_reset	hmac_stress_all_with_rand_reset	11.908m	120.477ms	4	10	40.00
V3		TOTAL			4	10	40.00
		TOTAL			654	660	99.09

Testplan Progress

Items	Total	Written	Passing	Progress
V1	6	6	6	100.00
V2	17	17	17	100.00
V2S	2	2	2	100.00
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
97.61	95.37	97.17	100.00	94.12	98.25	98.52	99.85

Failure Buckets

UVM_ERROR (hmac_base_vseq.sv:80) [hmac_long_msg_vseq] Check failed cfg.hmac_vif.is_idle() == *'b* (* [*] vs * [*]) has 2 failures:
- Test hmac_stress_all_with_rand_reset has 2 failures.
  - 8.hmac_stress_all_with_rand_reset.96582141768626283070194038235608188628620107698494308353888579932898360771526
    Line 29048, in log /workspaces/repo/scratch/os_regression_2024_09_23/hmac-sim-vcs/8.hmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 38147440424 ps: (hmac_base_vseq.sv:80) [uvm_test_top.env.virtual_sequencer.hmac_long_msg_vseq] Check failed cfg.hmac_vif.is_idle() == 1'b1 (0 [0x0] vs 1 [0x1])
  UVM_INFO @ 38147440424 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 9.hmac_stress_all_with_rand_reset.94438121779052271089469142914421208585408301138001612917584755877872744672245
    Line 27487, in log /workspaces/repo/scratch/os_regression_2024_09_23/hmac-sim-vcs/9.hmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 5565377921 ps: (hmac_base_vseq.sv:80) [uvm_test_top.env.virtual_sequencer.hmac_long_msg_vseq] Check failed cfg.hmac_vif.is_idle() == 1'b1 (0 [0x0] vs 1 [0x1])
  UVM_INFO @ 5565377921 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (hmac_base_vseq.sv:80) [hmac_common_vseq] Check failed cfg.hmac_vif.is_idle() == *'b* (* [*] vs * [*]) has 1 failures:
- Test hmac_stress_all_with_rand_reset has 1 failures.
  - 0.hmac_stress_all_with_rand_reset.100916392929535049494144433325492778914241978151222200653080134862670975281151
    Line 6746, in log /workspaces/repo/scratch/os_regression_2024_09_23/hmac-sim-vcs/0.hmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 4880766897 ps: (hmac_base_vseq.sv:80) [uvm_test_top.env.virtual_sequencer.hmac_common_vseq] Check failed cfg.hmac_vif.is_idle() == 1'b1 (0 [0x0] vs 1 [0x1])
  UVM_INFO @ 4880766897 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (cip_base_vseq.sv:771) [hmac_common_vseq] Check failed (cfg.can_reset_with_csr_accesses || !has_outstanding_access()) Trying to trigger a reset with outstanding CSR items. has 1 failures:
- Test hmac_stress_all_with_rand_reset has 1 failures.
  - 4.hmac_stress_all_with_rand_reset.2596216155314323907565779185443350682807872088072052494640598041419938220988
    Line 4358, in log /workspaces/repo/scratch/os_regression_2024_09_23/hmac-sim-vcs/4.hmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 2638176649 ps: (cip_base_vseq.sv:771) [uvm_test_top.env.virtual_sequencer.hmac_common_vseq] Check failed (cfg.can_reset_with_csr_accesses || !has_outstanding_access()) Trying to trigger a reset with outstanding CSR items.
  UVM_INFO @ 2638176649 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (hmac_base_vseq.sv:80) [hmac_back_pressure_vseq] Check failed cfg.hmac_vif.is_idle() == *'b* (* [*] vs * [*]) has 1 failures:
- Test hmac_stress_all_with_rand_reset has 1 failures.
  - 5.hmac_stress_all_with_rand_reset.84032239910315204439182986575211246985780364730873743345733123539557596987718
    Line 9322, in log /workspaces/repo/scratch/os_regression_2024_09_23/hmac-sim-vcs/5.hmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 3003400716 ps: (hmac_base_vseq.sv:80) [uvm_test_top.env.virtual_sequencer.hmac_back_pressure_vseq] Check failed cfg.hmac_vif.is_idle() == 1'b1 (0 [0x0] vs 1 [0x1])
  UVM_INFO @ 3003400716 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (hmac_base_vseq.sv:80) [hmac_wipe_secret_vseq] Check failed cfg.hmac_vif.is_idle() == *'b* (* [*] vs * [*]) has 1 failures:
- Test hmac_stress_all_with_rand_reset has 1 failures.
  - 6.hmac_stress_all_with_rand_reset.29789541928219563468926828704948582713136241834567272256199642768893217186930
    Line 3980, in log /workspaces/repo/scratch/os_regression_2024_09_23/hmac-sim-vcs/6.hmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 4654175461 ps: (hmac_base_vseq.sv:80) [uvm_test_top.env.virtual_sequencer.hmac_wipe_secret_vseq] Check failed cfg.hmac_vif.is_idle() == 1'b1 (0 [0x0] vs 1 [0x1])
  UVM_INFO @ 4654175461 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```

HMAC Simulation Results

Tuesday September 24 2024 01:05:57 UTC

GitHub Revision: 78ad89d1aa

Branch: os_regression_2024_09_23

Testplan

Simulator: VCS

Build randomization enabled with --build-seed 34048022127553017884926631616394166155118623175048314192737094530054579848544

Test Results

Testplan Progress

Coverage Results

Coverage Dashboard

Failure Buckets

Past Results

GitHub Revision: `78ad89d1aa`