HMAC Simulation Results

Sunday December 31 2023 20:02:18 UTC

GitHub Revision: `a9c19f09f3`

Branch: os_regression

Testplan

Simulator: VCS

Build randomization enabled with --build-seed 36521940887861431083267591129785326983863798057293121812910170439117479843669

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	hmac_smoke	4.200s	1.656ms	50	50	100.00
V1	csr_hw_reset	hmac_csr_hw_reset	0.740s	16.245us	5	5	100.00
V1	csr_rw	hmac_csr_rw	0.770s	68.809us	20	20	100.00
V1	csr_bit_bash	hmac_csr_bit_bash	8.680s	1.631ms	5	5	100.00
V1	csr_aliasing	hmac_csr_aliasing	2.580s	537.177us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	hmac_csr_mem_rw_with_rand_reset	2.500s	76.294us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	hmac_csr_rw	0.770s	68.809us	20	20	100.00
		hmac_csr_aliasing	2.580s	537.177us	5	5	100.00
V1		TOTAL			105	105	100.00
V2	long_msg	hmac_long_msg	1.758m	38.837ms	50	50	100.00
V2	back_pressure	hmac_back_pressure	59.370s	6.664ms	50	50	100.00
V2	test_vectors	hmac_test_sha_vectors	8.237m	43.318ms	46	50	92.00
		hmac_test_hmac_vectors	1.200s	903.376us	50	50	100.00
V2	burst_wr	hmac_burst_wr	1.164m	6.983ms	50	50	100.00
V2	datapath_stress	hmac_datapath_stress	2.356m	2.780ms	50	50	100.00
V2	error	hmac_error	2.472m	52.296ms	50	50	100.00
V2	wipe_secret	hmac_wipe_secret	1.388m	18.510ms	50	50	100.00
V2	stress_all	hmac_stress_all	39.103m	153.976ms	50	50	100.00
V2	alert_test	hmac_alert_test	0.650s	31.692us	50	50	100.00
V2	intr_test	hmac_intr_test	0.680s	48.926us	50	50	100.00
V2	tl_d_oob_addr_access	hmac_tl_errors	3.170s	62.234us	20	20	100.00
V2	tl_d_illegal_access	hmac_tl_errors	3.170s	62.234us	20	20	100.00
V2	tl_d_outstanding_access	hmac_csr_hw_reset	0.740s	16.245us	5	5	100.00
		hmac_csr_rw	0.770s	68.809us	20	20	100.00
		hmac_csr_aliasing	2.580s	537.177us	5	5	100.00
		hmac_same_csr_outstanding	1.470s	107.510us	20	20	100.00
V2	tl_d_partial_access	hmac_csr_hw_reset	0.740s	16.245us	5	5	100.00
		hmac_csr_rw	0.770s	68.809us	20	20	100.00
		hmac_csr_aliasing	2.580s	537.177us	5	5	100.00
		hmac_same_csr_outstanding	1.470s	107.510us	20	20	100.00
V2		TOTAL			586	590	99.32
V2S	tl_intg_err	hmac_sec_cm	0.990s	110.383us	5	5	100.00
		hmac_tl_intg_err	2.490s	1.401ms	19	20	95.00
V2S	sec_cm_bus_integrity	hmac_tl_intg_err	2.490s	1.401ms	19	20	95.00
V2S		TOTAL			24	25	96.00
V3	write_config_and_secret_key_during_msg_wr	hmac_smoke	4.200s	1.656ms	50	50	100.00
V3	stress_all_with_rand_reset	hmac_stress_all_with_rand_reset	2.194h	1.143s	194	200	97.00
V3		TOTAL			194	200	97.00
		TOTAL			909	920	98.80

Testplan Progress

Items	Total	Written	Passing	Progress
V1	6	6	6	100.00
V2	13	13	12	92.31
V2S	2	2	1	50.00
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
99.57	99.53	98.47	100.00	100.00	99.76	99.49	99.72

Failure Buckets

UVM_FATAL (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of * ps hit, indicating a probable testbench issue has 4 failures:
- Test hmac_test_sha_vectors has 4 failures.
  - 9.hmac_test_sha_vectors.86780380373971649753192176338920137983939205378883665222513621102209071995752
    Line 248, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/9.hmac_test_sha_vectors/latest/run.log
```
  UVM_FATAL @ 200000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 200000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 200000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 11.hmac_test_sha_vectors.56474136888180138344947237992381172359479846865183610318567720886454977918381
    Line 248, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/11.hmac_test_sha_vectors/latest/run.log
```
  UVM_FATAL @ 200000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 200000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 200000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 2 more failures.
UVM_ERROR (hmac_base_vseq.sv:37) [hmac_stress_all_vseq] Check failed cfg.hmac_vif.is_idle() == *'b* (* [*] vs * [*]) has 4 failures:
- Test hmac_stress_all_with_rand_reset has 4 failures.
  - 94.hmac_stress_all_with_rand_reset.29483534583321267010484065994753541963106026217434273731043302171832873668697
    Line 387, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/94.hmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 25347505799 ps: (hmac_base_vseq.sv:37) [uvm_test_top.env.virtual_sequencer.hmac_stress_all_vseq] Check failed cfg.hmac_vif.is_idle() == 1'b1 (0 [0x0] vs 1 [0x1])
  UVM_INFO @ 25347505799 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 130.hmac_stress_all_with_rand_reset.32488283511356643498281659661670231527891236852656673155798956804369755416748
    Line 540, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/130.hmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 13552118228 ps: (hmac_base_vseq.sv:37) [uvm_test_top.env.virtual_sequencer.hmac_stress_all_vseq] Check failed cfg.hmac_vif.is_idle() == 1'b1 (0 [0x0] vs 1 [0x1])
  UVM_INFO @ 13552118228 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 2 more failures.

Exit reason: Error: User command failed Job returned non-zero exit code has 1 failures:

Test hmac_tl_intg_err has 1 failures.

10.hmac_tl_intg_err.88853496893687502328016190653061182652753889649671480524415175941625500199385
Log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/10.hmac_tl_intg_err/latest/run.log

  [make]: simulate
  cd /workspace/10.hmac_tl_intg_err/latest && /workspace/cover_reg_top/simv +run_tl_intg_err +en_scb=0 +cdc_instrumentation_enabled=1 +UVM_NO_RELNOTES +UVM_VERBOSITY=UVM_LOW -licqueue -ucli -do /workspace/mnt/repo_top/hw/dv/tools/sim.tcl +ntb_random_seed=734063065 -assert nopostproc +UVM_TESTNAME=hmac_base_test +UVM_TEST_SEQ=hmac_common_vseq +en_cov=1 -cm line+cond+fsm+tgl+branch+assert -cm_dir /workspace/coverage/cover_reg_top.vdb -cm_log /dev/null -cm_name 10.hmac_tl_intg_err.734063065
  Chronologic VCS simulator copyright 1991-2022
  Contains Synopsys proprietary information.
  Compiler version T-2022.06-SP2_Full64; Runtime version T-2022.06-SP2_Full64;  Dec 31 12:25 2023
  Cannot find license file.
  Make sure that you have a license file and that your
  LM_LICENSE_FILE is pointing to the right location.
  
  make: *** [/workspace/mnt/repo_top/hw/dv/tools/dvsim/sim.mk:175: simulate] Error 255

UVM_ERROR (hmac_base_vseq.sv:37) [hmac_burst_wr_vseq] Check failed cfg.hmac_vif.is_idle() == *'b* (* [*] vs * [*]) has 1 failures:
- Test hmac_stress_all_with_rand_reset has 1 failures.
  - 19.hmac_stress_all_with_rand_reset.47747956591035337788690508235726944696460926187431233187105414148096717461621
    Line 369, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/19.hmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 7808469143 ps: (hmac_base_vseq.sv:37) [uvm_test_top.env.virtual_sequencer.hmac_burst_wr_vseq] Check failed cfg.hmac_vif.is_idle() == 1'b1 (0 [0x0] vs 1 [0x1])
  UVM_INFO @ 7808469143 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (hmac_base_vseq.sv:37) [hmac_long_msg_vseq] Check failed cfg.hmac_vif.is_idle() == *'b* (* [*] vs * [*]) has 1 failures:
- Test hmac_stress_all_with_rand_reset has 1 failures.
  - 109.hmac_stress_all_with_rand_reset.96305031459414599211339585532971212009087145632177149735037447559754104843727
    Line 504, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/109.hmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 40714543181 ps: (hmac_base_vseq.sv:37) [uvm_test_top.env.virtual_sequencer.hmac_long_msg_vseq] Check failed cfg.hmac_vif.is_idle() == 1'b1 (0 [0x0] vs 1 [0x1])
  UVM_INFO @ 40714543181 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```