USBDEV Simulation Results

Wednesday July 31 2024 23:02:38 UTC

GitHub Revision: e9b7e615a7

Branch: os_regression

Testplan

Simulator: VCS

Build randomization enabled with --build-seed 25204348267605859133056659113100703417171299070132656462514712657132693373848

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	usbdev_smoke	1.200s	271.403us	50	50	100.00
V1	csr_hw_reset	usbdev_csr_hw_reset	1.160s	281.372us	5	5	100.00
V1	csr_rw	usbdev_csr_rw	1.100s	96.622us	20	20	100.00
V1	csr_bit_bash	usbdev_csr_bit_bash	8.240s	1.853ms	5	5	100.00
V1	csr_aliasing	usbdev_csr_aliasing	3.520s	348.867us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	usbdev_csr_mem_rw_with_rand_reset	2.500s	88.087us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	usbdev_csr_rw	1.100s	96.622us	20	20	100.00
		usbdev_csr_aliasing	3.520s	348.867us	5	5	100.00
V1	mem_walk	usbdev_mem_walk	4.970s	734.400us	5	5	100.00
V1	mem_partial_access	usbdev_mem_partial_access	2.570s	195.546us	5	5	100.00
V1		TOTAL			115	115	100.00
V2	in_trans	usbdev_in_trans	1.160s	279.412us	50	50	100.00
V2	data_toggle_clear	usbdev_data_toggle_clear	2.100s	649.920us	50	50	100.00
V2	phy_pins_sense	usbdev_phy_pins_sense	0.810s	107.970us	50	50	100.00
V2	av_buffer	usbdev_av_buffer	1.050s	149.204us	50	50	100.00
V2	rx_fifo	usbdev_pkt_buffer	1.096m	22.199ms	50	50	100.00
V2	phy_config_tx_osc_test_mode	usbdev_phy_config_tx_osc_test_mode	1.100s	314.353us	1	1	100.00
V2	phy_config_eop_single_bit_handling	usbdev_phy_config_eop_single_bit_handling	0.910s	183.452us	1	1	100.00
V2	phy_config_pinflip	usbdev_phy_config_pinflip	1.190s	303.905us	50	50	100.00
V2	phy_config_rand_bus_type	usbdev_phy_config_rand_bus_type	1.100s	267.506us	5	5	100.00
V2	phy_config_rx_dp_dn	usbdev_phy_config_rx_dp_dn	1.120s	256.456us	1	1	100.00
V2	phy_config_tx_use_d_se0	usbdev_phy_config_tx_use_d_se0	1.040s	274.977us	1	1	100.00
V2	phy_config_usb_ref_disable	usbdev_phy_config_usb_ref_disable	0.940s	142.716us	50	50	100.00
V2	max_length_out_transaction	usbdev_max_length_out_transaction	1.080s	201.435us	50	50	100.00
		usbdev_stream_len_max	3.610s	1.374ms	50	50	100.00
V2	max_length_in_transaction	usbdev_max_length_in_transaction	1.130s	279.423us	50	50	100.00
V2	min_length_out_transaction	usbdev_min_length_out_transaction	0.990s	153.399us	50	50	100.00
V2	min_length_in_transaction	usbdev_min_length_in_transaction	1.040s	189.858us	50	50	100.00
V2	random_length_out_transaction	usbdev_random_length_out_transaction	1.070s	212.364us	50	50	100.00
V2	random_length_in_transaction	usbdev_random_length_in_transaction	1.070s	226.591us	50	50	100.00
V2	out_stall	usbdev_out_stall	1.030s	247.869us	50	50	100.00
V2	in_stall	usbdev_in_stall	0.910s	182.202us	50	50	100.00
V2	out_iso	usbdev_out_iso	1.070s	215.814us	50	50	100.00
V2	in_iso	usbdev_in_iso	1.320s	239.273us	50	50	100.00
V2	pkt_received	usbdev_pkt_received	1.110s	228.225us	50	50	100.00
V2	pkt_sent	usbdev_pkt_sent	1.160s	320.308us	50	50	100.00
V2	disconnected	usbdev_disconnected	0.960s	151.876us	50	50	100.00
V2	host_lost	usbdev_host_lost	10.060s	4.165ms	1	1	100.00
V2	link_reset	usbdev_link_reset	0.950s	165.613us	1	1	100.00
V2	link_suspend	usbdev_link_suspend	6.070s	3.328ms	50	50	100.00
V2	link_resume	usbdev_link_resume	36.060s	23.276ms	50	50	100.00
V2	av_empty	usbdev_av_empty	0.990s	179.201us	5	5	100.00
V2	rx_full	usbdev_rx_full	1.270s	361.667us	1	1	100.00
V2	av_overflow	usbdev_av_overflow	1.000s	161.303us	5	5	100.00
V2	link_in_err	usbdev_link_in_err	1.100s	230.580us	50	50	100.00
V2	rx_crc_err	usbdev_rx_crc_err	1.020s	200.506us	50	50	100.00
V2	rx_pid_err	usbdev_rx_pid_err	0.920s	201.908us	5	5	100.00
V2	rx_bitstuff_err	usbdev_bitstuff_err	1.020s	155.778us	50	50	100.00
V2	link_out_err	usbdev_link_out_err	1.470s	476.660us	1	1	100.00
V2	enable	usbdev_enable	0.810s	40.849us	50	50	100.00
V2	resume_link_active	usbdev_resume_link_active	26.160s	20.219ms	1	1	100.00
V2	device_address	usbdev_device_address	53.410s	20.735ms	50	50	100.00
V2	invalid_data1_data0_toggle_test	usbdev_invalid_data1_data0_toggle_test	1.460s	430.234us	1	1	100.00
V2	setup_stage	usbdev_setup_stage	0.970s	208.218us	50	50	100.00
V2	endpoint_access	usbdev_endpoint_access	3.050s	954.366us	50	50	100.00
V2	disable_endpoint	usbdev_disable_endpoint	1.850s	490.269us	50	50	100.00
V2	out_trans_nak	usbdev_out_trans_nak	1.060s	195.354us	50	50	100.00
V2	setup_trans_ignored	usbdev_setup_trans_ignored	0.980s	241.228us	50	50	100.00
V2	nak_trans	usbdev_nak_trans	1.060s	266.865us	50	50	100.00
V2	stall_trans	usbdev_stall_trans	1.190s	210.341us	50	50	100.00
V2	setup_priority_over_stall_response	usbdev_setup_priority_over_stall_response	1.120s	299.898us	5	5	100.00
V2	stall_priority_over_nak	usbdev_stall_priority_over_nak	1.040s	255.255us	50	50	100.00
V2	pending_in_trans	usbdev_pending_in_trans	1.020s	151.678us	50	50	100.00
V2	streaming_test	usbdev_streaming_out	3.411m	7.456ms	50	50	100.00
V2	max_clock_error_untracked	usbdev_freq_hiclk	3.143m	119.200ms	5	5	100.00
		usbdev_freq_loclk	3.093m	118.103ms	5	5	100.00
V2	max_clock_error_tracking	usbdev_freq_hiclk_max	3.091m	118.058ms	5	5	100.00
		usbdev_freq_loclk_max	3.843m	119.084ms	5	5	100.00
V2	max_phase_error	usbdev_freq_phase	2.948m	120.172ms	5	5	100.00
V2	min_inter_pkt_delay	usbdev_min_inter_pkt_delay	3.921m	7.884ms	50	50	100.00
V2	max_inter_pkt_delay	usbdev_max_inter_pkt_delay	3.677m	7.028ms	50	50	100.00
V2	device_timeout_missing_host_handshake	usbdev_timeout_missing_host_handshake	54.620s	7.899ms	50	50	100.00
V2	device_timeout	usbdev_device_timeout	39.610s	1.691ms	50	50	100.00
V2	packet_buffer	usbdev_pkt_buffer	1.096m	22.199ms	50	50	100.00
V2	nak_to_out_trans_when_avbuffer_empty_rxfifo_full	usbdev_nak_to_out_trans_when_avbuffer_empty_rxfifo_full	1.700s	497.204us	1	1	100.00
V2	aon_wake_resume	usbdev_aon_wake_resume	35.430s	23.413ms	50	50	100.00
V2	aon_wake_reset	usbdev_aon_wake_reset	19.960s	13.332ms	50	50	100.00
V2	aon_wake_disconnect	usbdev_aon_wake_disconnect	7.230s	4.386ms	50	50	100.00
V2	invalid_sync	usbdev_invalid_sync	4.894m	10.005ms	50	50	100.00
V2	spurious_pids_ignored	usbdev_spurious_pids_ignored	3.129m	6.133ms	50	50	100.00
V2	low_speed_traffic	usbdev_low_speed_traffic	4.918m	9.673ms	50	50	100.00
V2	rand_bus_resets	usbdev_rand_bus_resets	2.548m	17.593ms	9	10	90.00
V2	rand_disconnects	usbdev_rand_bus_disconnects	4.651m	13.700ms	10	10	100.00
V2	rand_suspends	usbdev_rand_suspends	6.370m	15.849ms	10	10	100.00
V2	max_usb_traffic	usbdev_max_usb_traffic	3.196m	6.445ms	50	50	100.00
V2	stress_usb_traffic	usbdev_stress_usb_traffic	7.779m	19.311ms	5	5	100.00
V2	in_packet_retraction	usbdev_iso_retraction	2.679m	12.429ms	45	50	90.00
V2	data_toggle_restore	usbdev_data_toggle_restore	3.470s	1.331ms	50	50	100.00
V2	setup_priority	usbdev_setup_priority	1.570s	428.210us	5	5	100.00
V2	fifo_resets	usbdev_fifo_rst	2.900s	377.279us	50	50	100.00
V2	intr_test	usbdev_intr_test	0.850s	110.078us	50	50	100.00
V2	alert_test	usbdev_alert_test	0.740s	51.812us	50	50	100.00
V2	tl_d_oob_addr_access	usbdev_tl_errors	3.430s	109.859us	20	20	100.00
V2	tl_d_illegal_access	usbdev_tl_errors	3.430s	109.859us	20	20	100.00
V2	tl_d_outstanding_access	usbdev_csr_hw_reset	1.160s	281.372us	5	5	100.00
		usbdev_csr_rw	1.100s	96.622us	20	20	100.00
		usbdev_csr_aliasing	3.520s	348.867us	5	5	100.00
		usbdev_same_csr_outstanding	1.840s	441.965us	20	20	100.00
V2	tl_d_partial_access	usbdev_csr_hw_reset	1.160s	281.372us	5	5	100.00
		usbdev_csr_rw	1.100s	96.622us	20	20	100.00
		usbdev_csr_aliasing	3.520s	348.867us	5	5	100.00
		usbdev_same_csr_outstanding	1.840s	441.965us	20	20	100.00
V2		TOTAL			2835	2841	99.79
V2S	tl_intg_err	usbdev_sec_cm	1.910s	1.128ms	5	5	100.00
		usbdev_tl_intg_err	6.880s	2.296ms	20	20	100.00
V2S	sec_cm_bus_integrity	usbdev_tl_intg_err	6.880s	2.296ms	20	20	100.00
V2S		TOTAL			25	25	100.00
V3	dpi_config_host	usbdev_dpi_config_host	52.410s	5.205ms	1	1	100.00
V3		TOTAL			1	1	100.00
	Unmapped tests	usbdev_stress_all_with_rand_reset	0.690s	17.901us	0	10	0.00
		usbdev_stress_all	0.630s		0	50	0.00
		TOTAL			2976	3042	97.83

Testplan Progress

Items	Total	Written	Passing	Progress
N.A.	2	2	0	0.00
V1	8	8	8	100.00
V2	82	82	80	97.56
V2S	2	2	2	100.00
V3	1	1	1	100.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
90.37	97.82	93.79	97.44	73.44	96.21	98.17	75.75

Failure Buckets

• UVM_WARNING [BDTYP] Cannot create an object of type 'usbdev_stress_all_vseq' because it is not registered with the factory. has 60 failures:

  • Test usbdev_stress_all_with_rand_reset has 10 failures.

    • 0.usbdev_stress_all_with_rand_reset.3318459470905838350513875565705042034810794802708996217821272533896194277780
      Line 251, in log /container/opentitan-public/scratch/os_regression/usbdev-sim-vcs/0.usbdev_stress_all_with_rand_reset/latest/run.log

        UVM_WARNING @  69039709 ps: [BDTYP] Cannot create an object of type 'usbdev_stress_all_vseq' because it is not registered with the factory.
        UVM_INFO @  69039709 ps: (uvm_factory.svh:1645) [UVM/FACTORY/PRINT]
        #### Factory Configuration (*)
        
        No instance overrides are registered with this factory
      

    • 1.usbdev_stress_all_with_rand_reset.75435923127221161509732771468254346965686613171344326480851739339938810192925
      Line 251, in log /container/opentitan-public/scratch/os_regression/usbdev-sim-vcs/1.usbdev_stress_all_with_rand_reset/latest/run.log

        UVM_WARNING @  29184186 ps: [BDTYP] Cannot create an object of type 'usbdev_stress_all_vseq' because it is not registered with the factory.
        UVM_INFO @  29184186 ps: (uvm_factory.svh:1645) [UVM/FACTORY/PRINT]
        #### Factory Configuration (*)
        
        No instance overrides are registered with this factory
      

    • ... and 8 more failures.

  • Test usbdev_stress_all has 50 failures.

    • 0.usbdev_stress_all.105331946933732851569514515530786217255925818245463228044168613000276250646790
      Line 251, in log /container/opentitan-public/scratch/os_regression/usbdev-sim-vcs/0.usbdev_stress_all/latest/run.log

        UVM_WARNING @         0 ps: [BDTYP] Cannot create an object of type 'usbdev_stress_all_vseq' because it is not registered with the factory.
        UVM_INFO @         0 ps: (uvm_factory.svh:1645) [UVM/FACTORY/PRINT]
        #### Factory Configuration (*)
        
        No instance overrides are registered with this factory
      

    • 1.usbdev_stress_all.17677238245913743562826894707972950579445783160213384186939901373490110790913
      Line 251, in log /container/opentitan-public/scratch/os_regression/usbdev-sim-vcs/1.usbdev_stress_all/latest/run.log

        UVM_WARNING @         0 ps: [BDTYP] Cannot create an object of type 'usbdev_stress_all_vseq' because it is not registered with the factory.
        UVM_INFO @         0 ps: (uvm_factory.svh:1645) [UVM/FACTORY/PRINT]
        #### Factory Configuration (*)
        
        No instance overrides are registered with this factory
      

    • ... and 48 more failures.

• UVM_ERROR (usbdev_scoreboard.sv:200) [scoreboard] Check failed act_item.compare(exp_item) == * (* [*] vs * [*]) DUT response does not match expectation has 3 failures:

  • Test usbdev_iso_retraction has 3 failures.

    • 26.usbdev_iso_retraction.102183283244878602143601405100787223672138002195808177195268395779312371298229
      Line 869, in log /container/opentitan-public/scratch/os_regression/usbdev-sim-vcs/26.usbdev_iso_retraction/latest/run.log

        UVM_ERROR @ 3610212087 ps: (usbdev_scoreboard.sv:200) [uvm_test_top.env.scoreboard] Check failed act_item.compare(exp_item) == 1 (0 [0x0] vs 1 [0x1]) DUT response does not match expectation
        UVM_INFO @ 3610212087 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
        --- UVM Report catcher Summary ---
        
        
      

    • 28.usbdev_iso_retraction.5149917165865120583941584091448836637625553319646753910800631912662820515253
      Line 271, in log /container/opentitan-public/scratch/os_regression/usbdev-sim-vcs/28.usbdev_iso_retraction/latest/run.log

        UVM_ERROR @ 189197038 ps: (usbdev_scoreboard.sv:200) [uvm_test_top.env.scoreboard] Check failed act_item.compare(exp_item) == 1 (0 [0x0] vs 1 [0x1]) DUT response does not match expectation
        UVM_INFO @ 189197038 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
        --- UVM Report catcher Summary ---
        
        
      

    • ... and 1 more failures.

• UVM_ERROR (usbdev_scoreboard.sv:879) [scoreboard] Check failed item.d_data == csr.get_mirrored_value() (* [*] vs * [*]) CSR read data mismatches prediction (usbdev_reg_block.in_sent) has 2 failures:

  • Test usbdev_iso_retraction has 2 failures.

    • 38.usbdev_iso_retraction.111546427801750126363807473288466420683676399002823553350957770564099539537019
      Line 808, in log /container/opentitan-public/scratch/os_regression/usbdev-sim-vcs/38.usbdev_iso_retraction/latest/run.log

        UVM_ERROR @ 3302279005 ps: (usbdev_scoreboard.sv:879) [uvm_test_top.env.scoreboard] Check failed item.d_data == csr.get_mirrored_value() (128 [0x80] vs 0 [0x0]) CSR read data mismatches prediction (usbdev_reg_block.in_sent)
        UVM_INFO @ 3302279005 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
        --- UVM Report catcher Summary ---
        
        
      

    • 45.usbdev_iso_retraction.1381621027761414545222758731390719320672275754740305809263365204981315008044
      Line 1887, in log /container/opentitan-public/scratch/os_regression/usbdev-sim-vcs/45.usbdev_iso_retraction/latest/run.log

        UVM_ERROR @ 9489441532 ps: (usbdev_scoreboard.sv:879) [uvm_test_top.env.scoreboard] Check failed item.d_data == csr.get_mirrored_value() (2048 [0x800] vs 0 [0x0]) CSR read data mismatches prediction (usbdev_reg_block.in_sent)
        UVM_INFO @ 9489441532 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
        --- UVM Report catcher Summary ---
      

• UVM_ERROR (usbdev_scoreboard.sv:879) [scoreboard] Check failed item.d_data == csr.get_mirrored_value() (* [*] vs * [*]) CSR read data mismatches prediction (usbdev_reg_block.usbstat) has 1 failures:

  • Test usbdev_rand_bus_resets has 1 failures.

    • 3.usbdev_rand_bus_resets.56537918051578178090487158496347777086266354472742141517374350253504839895170
      Line 650, in log /container/opentitan-public/scratch/os_regression/usbdev-sim-vcs/3.usbdev_rand_bus_resets/latest/run.log

        UVM_ERROR @ 3496774545 ps: (usbdev_scoreboard.sv:879) [uvm_test_top.env.scoreboard] Check failed item.d_data == csr.get_mirrored_value() (3234390019 [0xc0c8e003] vs 3234381827 [0xc0c8c003]) CSR read data mismatches prediction (usbdev_reg_block.usbstat)
        UVM_INFO @ 3496774545 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
        --- UVM Report catcher Summary ---
      

Past Results

• Latest
• lint
• 2024.07.31_09.21.24
• 2024.07.30_09.40.16
• 2024.07.29_08.05.13
• 2024.07.28_11.04.56
• 2024.07.27_09.57.20
• 2024.07.26_09.18.41