HMAC Simulation Results

Sunday March 03 2024 20:02:47 UTC

GitHub Revision: 0cdf265eaa

Branch: os_regression

Testplan

Simulator: VCS

Build randomization enabled with --build-seed 82530437672810453765703374940713112405319051694331588453064008042331386550559

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	hmac_smoke	4.930s	1.351ms	50	50	100.00
V1	csr_hw_reset	hmac_csr_hw_reset	0.730s	45.318us	5	5	100.00
V1	csr_rw	hmac_csr_rw	0.750s	19.475us	20	20	100.00
V1	csr_bit_bash	hmac_csr_bit_bash	9.120s	4.010ms	5	5	100.00
V1	csr_aliasing	hmac_csr_aliasing	2.470s	128.409us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	hmac_csr_mem_rw_with_rand_reset	15.413m	176.726ms	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	hmac_csr_rw	0.750s	19.475us	20	20	100.00
		hmac_csr_aliasing	2.470s	128.409us	5	5	100.00
V1		TOTAL			105	105	100.00
V2	long_msg	hmac_long_msg	2.042m	9.907ms	50	50	100.00
V2	back_pressure	hmac_back_pressure	1.088m	3.721ms	50	50	100.00
V2	test_vectors	hmac_test_sha_vectors	8.736m	51.255ms	47	50	94.00
		hmac_test_hmac_vectors	1.400s	279.621us	50	50	100.00
V2	burst_wr	hmac_burst_wr	1.337m	1.661ms	50	50	100.00
V2	datapath_stress	hmac_datapath_stress	2.579m	8.839ms	50	50	100.00
V2	error	hmac_error	3.862m	13.747ms	50	50	100.00
V2	wipe_secret	hmac_wipe_secret	1.470m	12.010ms	50	50	100.00
V2	stress_all	hmac_stress_all	39.054m	193.120ms	50	50	100.00
V2	alert_test	hmac_alert_test	0.610s	20.115us	50	50	100.00
V2	intr_test	hmac_intr_test	0.660s	86.125us	50	50	100.00
V2	tl_d_oob_addr_access	hmac_tl_errors	3.730s	68.420us	20	20	100.00
V2	tl_d_illegal_access	hmac_tl_errors	3.730s	68.420us	20	20	100.00
V2	tl_d_outstanding_access	hmac_csr_hw_reset	0.730s	45.318us	5	5	100.00
		hmac_csr_rw	0.750s	19.475us	20	20	100.00
		hmac_csr_aliasing	2.470s	128.409us	5	5	100.00
		hmac_same_csr_outstanding	1.430s	66.595us	20	20	100.00
V2	tl_d_partial_access	hmac_csr_hw_reset	0.730s	45.318us	5	5	100.00
		hmac_csr_rw	0.750s	19.475us	20	20	100.00
		hmac_csr_aliasing	2.470s	128.409us	5	5	100.00
		hmac_same_csr_outstanding	1.430s	66.595us	20	20	100.00
V2		TOTAL			587	590	99.49
V2S	tl_intg_err	hmac_sec_cm	0.970s	329.264us	5	5	100.00
		hmac_tl_intg_err	2.270s	267.678us	20	20	100.00
V2S	sec_cm_bus_integrity	hmac_tl_intg_err	2.270s	267.678us	20	20	100.00
V2S		TOTAL			25	25	100.00
V3	write_config_and_secret_key_during_msg_wr	hmac_smoke	4.930s	1.351ms	50	50	100.00
V3	stress_all_with_rand_reset	hmac_stress_all_with_rand_reset	59.603m	165.203ms	9	200	4.50
V3		TOTAL			9	200	4.50
		TOTAL			726	920	78.91

Testplan Progress

Items	Total	Written	Passing	Progress
V1	6	6	6	100.00
V2	13	13	12	92.31
V2S	2	2	2	100.00
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
96.73	98.36	96.63	100.00	87.50	95.83	99.49	99.31

Failure Buckets

• UVM_ERROR (cip_base_vseq.sv:815) [hmac_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 183 failures:
  • Test hmac_stress_all_with_rand_reset has 183 failures.

    • 0.hmac_stress_all_with_rand_reset.98495911297471808302477810759671402835163650765172361592047147888434923230570
      Line 262, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/0.hmac_stress_all_with_rand_reset/latest/run.log

        UVM_ERROR @ 1228656221 ps: (cip_base_vseq.sv:815) [uvm_test_top.env.virtual_sequencer.hmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
        UVM_INFO @ 1228656221 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
        --- UVM Report catcher Summary ---
        
        
      

    • 2.hmac_stress_all_with_rand_reset.59273370437164538366094879665726867729142658324589647581405861553178622291199
      Line 308, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/2.hmac_stress_all_with_rand_reset/latest/run.log

        UVM_ERROR @ 3021066210 ps: (cip_base_vseq.sv:815) [uvm_test_top.env.virtual_sequencer.hmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
        UVM_INFO @ 3021066210 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
        --- UVM Report catcher Summary ---
        
        
      

    • ... and 181 more failures.

• UVM_ERROR (cip_base_vseq.sv:741) [hmac_common_vseq] Check failed (!has_outstanding_access()) Trying to trigger a reset with outstanding CSR items. has 8 failures:
  • Test hmac_stress_all_with_rand_reset has 8 failures.

    • 1.hmac_stress_all_with_rand_reset.1405094675157600763471410008529562144016986198277557735055280236557117428900
      Line 1502, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/1.hmac_stress_all_with_rand_reset/latest/run.log

        UVM_ERROR @ 206027146966 ps: (cip_base_vseq.sv:741) [uvm_test_top.env.virtual_sequencer.hmac_common_vseq] Check failed (!has_outstanding_access()) Trying to trigger a reset with outstanding CSR items.
        UVM_INFO @ 206027146966 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
        --- UVM Report catcher Summary ---
        
        
      

    • 20.hmac_stress_all_with_rand_reset.88983723087265497198087216002923197473419215137324702457009269408227937892131
      Line 260, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/20.hmac_stress_all_with_rand_reset/latest/run.log

        UVM_ERROR @  80823892 ps: (cip_base_vseq.sv:741) [uvm_test_top.env.virtual_sequencer.hmac_common_vseq] Check failed (!has_outstanding_access()) Trying to trigger a reset with outstanding CSR items.
        UVM_INFO @  80823892 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
        --- UVM Report catcher Summary ---
        
        
      

    • ... and 6 more failures.

• UVM_FATAL (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of * ps hit, indicating a probable testbench issue has 3 failures:
  • Test hmac_test_sha_vectors has 3 failures.

    • 22.hmac_test_sha_vectors.40541293859484951168185786078304194482242610010279976364376231244201198614009
      Line 378, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/22.hmac_test_sha_vectors/latest/run.log

        UVM_FATAL @ 200000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 200000000000 ps hit, indicating a probable testbench issue
        UVM_INFO @ 200000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
        --- UVM Report catcher Summary ---
        
        
      

    • 24.hmac_test_sha_vectors.11717656146212430484272755939677230490608578068072376747245820945191514691824
      Line 417, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/24.hmac_test_sha_vectors/latest/run.log

        UVM_FATAL @ 200000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 200000000000 ps hit, indicating a probable testbench issue
        UVM_INFO @ 200000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
        --- UVM Report catcher Summary ---
        
        
      

    • ... and 1 more failures.

Past Results

• Latest
• lint
• 2024.03.01_07.57.40
• 2024.02.29_12.17.36
• 2024.02.26_03.37.47
• 2024.02.22_04.55.05
• 2024.02.19_06.08.01
• 2024.02.14_21.58.20