HMAC Simulation Results

Sunday January 21 2024 20:02:56 UTC

GitHub Revision: 796f9fb805

Branch: os_regression

Testplan

Simulator: VCS

Build randomization enabled with --build-seed 82526748448873323296379810788667205332667151893362240729689214265893867671108

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	hmac_smoke	4.480s	417.528us	50	50	100.00
V1	csr_hw_reset	hmac_csr_hw_reset	0.710s	26.343us	5	5	100.00
V1	csr_rw	hmac_csr_rw	0.760s	17.865us	20	20	100.00
V1	csr_bit_bash	hmac_csr_bit_bash	9.630s	2.009ms	5	5	100.00
V1	csr_aliasing	hmac_csr_aliasing	2.000s	1.741ms	5	5	100.00
V1	csr_mem_rw_with_rand_reset	hmac_csr_mem_rw_with_rand_reset	14.361m	128.107ms	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	hmac_csr_rw	0.760s	17.865us	20	20	100.00
		hmac_csr_aliasing	2.000s	1.741ms	5	5	100.00
V1		TOTAL			105	105	100.00
V2	long_msg	hmac_long_msg	1.839m	28.451ms	50	50	100.00
V2	back_pressure	hmac_back_pressure	52.870s	1.556ms	50	50	100.00
V2	test_vectors	hmac_test_sha_vectors	9.633m	89.644ms	48	50	96.00
		hmac_test_hmac_vectors	1.430s	633.824us	50	50	100.00
V2	burst_wr	hmac_burst_wr	1.101m	6.594ms	50	50	100.00
V2	datapath_stress	hmac_datapath_stress	2.634m	2.950ms	50	50	100.00
V2	error	hmac_error	3.857m	78.167ms	50	50	100.00
V2	wipe_secret	hmac_wipe_secret	1.505m	9.271ms	50	50	100.00
V2	stress_all	hmac_stress_all	36.701m	125.317ms	50	50	100.00
V2	alert_test	hmac_alert_test	0.630s	23.740us	49	50	98.00
V2	intr_test	hmac_intr_test	0.650s	31.058us	50	50	100.00
V2	tl_d_oob_addr_access	hmac_tl_errors	3.590s	2.662ms	20	20	100.00
V2	tl_d_illegal_access	hmac_tl_errors	3.590s	2.662ms	20	20	100.00
V2	tl_d_outstanding_access	hmac_csr_hw_reset	0.710s	26.343us	5	5	100.00
		hmac_csr_rw	0.760s	17.865us	20	20	100.00
		hmac_csr_aliasing	2.000s	1.741ms	5	5	100.00
		hmac_same_csr_outstanding	1.450s	66.757us	20	20	100.00
V2	tl_d_partial_access	hmac_csr_hw_reset	0.710s	26.343us	5	5	100.00
		hmac_csr_rw	0.760s	17.865us	20	20	100.00
		hmac_csr_aliasing	2.000s	1.741ms	5	5	100.00
		hmac_same_csr_outstanding	1.450s	66.757us	20	20	100.00
V2		TOTAL			587	590	99.49
V2S	tl_intg_err	hmac_sec_cm	1.020s	93.181us	5	5	100.00
		hmac_tl_intg_err	2.630s	595.475us	20	20	100.00
V2S	sec_cm_bus_integrity	hmac_tl_intg_err	2.630s	595.475us	20	20	100.00
V2S		TOTAL			25	25	100.00
V3	write_config_and_secret_key_during_msg_wr	hmac_smoke	4.480s	417.528us	50	50	100.00
V3	stress_all_with_rand_reset	hmac_stress_all_with_rand_reset	1.494h	106.355ms	187	200	93.50
V3		TOTAL			187	200	93.50
		TOTAL			904	920	98.26

Testplan Progress

Items	Total	Written	Passing	Progress
V1	6	6	6	100.00
V2	13	13	11	84.62
V2S	2	2	2	100.00
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
99.62	99.53	98.69	100.00	100.00	99.76	99.49	99.86

Failure Buckets

• UVM_ERROR (hmac_base_vseq.sv:37) [hmac_stress_all_vseq] Check failed cfg.hmac_vif.is_idle() == *'b* (* [*] vs * [*]) has 10 failures:

  • Test hmac_stress_all_with_rand_reset has 10 failures.

    • 4.hmac_stress_all_with_rand_reset.12488031392524182067965726939311872841973495305162880050639082620508589665685
      Line 1390, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/4.hmac_stress_all_with_rand_reset/latest/run.log

        UVM_ERROR @ 674778559348 ps: (hmac_base_vseq.sv:37) [uvm_test_top.env.virtual_sequencer.hmac_stress_all_vseq] Check failed cfg.hmac_vif.is_idle() == 1'b1 (0 [0x0] vs 1 [0x1])
        UVM_INFO @ 674778559348 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
        --- UVM Report catcher Summary ---
        
        
      

    • 8.hmac_stress_all_with_rand_reset.114374137426274716356924648247066787731715731248223350089614377211601012732820
      Line 460, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/8.hmac_stress_all_with_rand_reset/latest/run.log

        UVM_ERROR @ 14195590631 ps: (hmac_base_vseq.sv:37) [uvm_test_top.env.virtual_sequencer.hmac_stress_all_vseq] Check failed cfg.hmac_vif.is_idle() == 1'b1 (0 [0x0] vs 1 [0x1])
        UVM_INFO @ 14195590631 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
        --- UVM Report catcher Summary ---
        
        
      

    • ... and 8 more failures.

• UVM_ERROR (hmac_base_vseq.sv:37) [hmac_datapath_stress_vseq] Check failed cfg.hmac_vif.is_idle() == *'b* (* [*] vs * [*]) has 2 failures:

  • Test hmac_stress_all_with_rand_reset has 2 failures.

    • 111.hmac_stress_all_with_rand_reset.63355284378366116457429431659256893077023165416018571100273383369555006210878
      Line 1703, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/111.hmac_stress_all_with_rand_reset/latest/run.log

        UVM_ERROR @ 142966067365 ps: (hmac_base_vseq.sv:37) [uvm_test_top.env.virtual_sequencer.hmac_datapath_stress_vseq] Check failed cfg.hmac_vif.is_idle() == 1'b1 (0 [0x0] vs 1 [0x1])
        UVM_INFO @ 142966067365 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
        --- UVM Report catcher Summary ---
        
        
      

    • 178.hmac_stress_all_with_rand_reset.113734950014093704962874487204824767447899774609402932360064192397638328724520
      Line 252, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/178.hmac_stress_all_with_rand_reset/latest/run.log

        UVM_ERROR @   6212555 ps: (hmac_base_vseq.sv:37) [uvm_test_top.env.virtual_sequencer.hmac_datapath_stress_vseq] Check failed cfg.hmac_vif.is_idle() == 1'b1 (0 [0x0] vs 1 [0x1])
        UVM_INFO @   6212555 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
        --- UVM Report catcher Summary ---
      

• Job hmac-sim-vcs_run_default killed due to: Exit reason: Error: * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *] has 1 failures:

  • Test hmac_test_sha_vectors has 1 failures.

    • 3.hmac_test_sha_vectors.76216867446851166608515333578907927630097976850012464907144188400815012130920
      Log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/3.hmac_test_sha_vectors/latest/run.log

        Job ID: smart:3d8acf22-63a1-4e15-a9d1-eda8acf5589a
      

• Job hmac-sim-vcs_run_default killed due to: Exit reason: Error: * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *] has 1 failures:

  • Test hmac_alert_test has 1 failures.

    • 31.hmac_alert_test.55865375943441609213407653996545639102349359911042139295477533444146973217483
      Log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/31.hmac_alert_test/latest/run.log

        Job ID: smart:df39b550-c731-4cae-be2e-263970810786
      

• UVM_FATAL (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of * ps hit, indicating a probable testbench issue has 1 failures:

  • Test hmac_test_sha_vectors has 1 failures.

    • 41.hmac_test_sha_vectors.39300736501967638444917957306941414290388370571457187385030508424521949739309
      Line 248, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/41.hmac_test_sha_vectors/latest/run.log

        UVM_FATAL @ 200000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 200000000000 ps hit, indicating a probable testbench issue
        UVM_INFO @ 200000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
        --- UVM Report catcher Summary ---
      

• UVM_ERROR (hmac_base_vseq.sv:37) [hmac_common_vseq] Check failed cfg.hmac_vif.is_idle() == *'b* (* [*] vs * [*]) has 1 failures:

  • Test hmac_stress_all_with_rand_reset has 1 failures.

    • 110.hmac_stress_all_with_rand_reset.85120917140552792964225184967504511433871962700087102315360084962384884508878
      Line 252, in log /container/opentitan-public/scratch/os_regression/hmac-sim-vcs/110.hmac_stress_all_with_rand_reset/latest/run.log

        UVM_ERROR @   1116457 ps: (hmac_base_vseq.sv:37) [uvm_test_top.env.virtual_sequencer.hmac_common_vseq] Check failed cfg.hmac_vif.is_idle() == 1'b1 (0 [0x0] vs 1 [0x1])
        UVM_INFO @   1116457 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
        --- UVM Report catcher Summary ---
      

Past Results

• Latest
• lint
• 2024.01.18_08.33.23
• 2024.01.15_03.51.00
• 2024.01.11_03.05.38
• 2024.01.08_02.52.38
• 2024.01.04_01.16.52
• 2024.01.01_00.44.25