ENTROPY_SRC Simulation Results

Sunday April 07 2024 19:02:41 UTC

GitHub Revision: `7773b039d0`

Branch: os_regression

Simulator: XCELIUM

Build randomization enabled with --build-seed 110372901762865644007400082009110088154180821215015477169464044145224727696933

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	entropy_src_smoke	13.000s	59.785us	50	50	100.00
V1	csr_hw_reset	entropy_src_csr_hw_reset	4.000s	161.869us	5	5	100.00
V1	csr_rw	entropy_src_csr_rw	8.000s	23.399us	20	20	100.00
V1	csr_bit_bash	entropy_src_csr_bit_bash	15.000s	1.347ms	5	5	100.00
V1	csr_aliasing	entropy_src_csr_aliasing	15.000s	152.787us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	entropy_src_csr_mem_rw_with_rand_reset	9.000s	74.128us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	entropy_src_csr_rw	8.000s	23.399us	20	20	100.00
		entropy_src_csr_aliasing	15.000s	152.787us	5	5	100.00
V1		TOTAL			105	105	100.00
V2	firmware	entropy_src_smoke	13.000s	59.785us	50	50	100.00
		entropy_src_rng	4.950m	10.012ms	300	300	100.00
		entropy_src_fw_ov	2.450m	5.033ms	292	300	97.33
V2	firmware_mode	entropy_src_fw_ov	2.450m	5.033ms	292	300	97.33
V2	rng_mode	entropy_src_rng	4.950m	10.012ms	300	300	100.00
V2	rng_max_rate	entropy_src_rng_max_rate	8.900m	10.030ms	388	400	97.00
V2	health_checks	entropy_src_rng	4.950m	10.012ms	300	300	100.00
V2	conditioning	entropy_src_rng	4.950m	10.012ms	300	300	100.00
V2	interrupts	entropy_src_rng	4.950m	10.012ms	300	300	100.00
V2	alerts	entropy_src_rng	4.950m	10.012ms	300	300	100.00
		entropy_src_functional_alerts	18.000s	55.944us	50	50	100.00
V2	stress_all	entropy_src_stress_all	20.000s	1.334ms	50	50	100.00
V2	functional_errors	entropy_src_functional_errors	9.333m	10.012ms	975	1000	97.50
V2	firmware_ov_read_contiguous_data	entropy_src_fw_ov_contiguous	28.000s	340.988us	50	50	100.00
V2	intr_test	entropy_src_intr_test	13.000s	32.386us	50	50	100.00
V2	alert_test	entropy_src_alert_test	18.000s	16.551us	50	50	100.00
V2	tl_d_oob_addr_access	entropy_src_tl_errors	16.000s	838.185us	20	20	100.00
V2	tl_d_illegal_access	entropy_src_tl_errors	16.000s	838.185us	20	20	100.00
V2	tl_d_outstanding_access	entropy_src_csr_hw_reset	4.000s	161.869us	5	5	100.00
		entropy_src_csr_rw	8.000s	23.399us	20	20	100.00
		entropy_src_csr_aliasing	15.000s	152.787us	5	5	100.00
		entropy_src_same_csr_outstanding	14.000s	262.303us	20	20	100.00
V2	tl_d_partial_access	entropy_src_csr_hw_reset	4.000s	161.869us	5	5	100.00
		entropy_src_csr_rw	8.000s	23.399us	20	20	100.00
		entropy_src_csr_aliasing	15.000s	152.787us	5	5	100.00
		entropy_src_same_csr_outstanding	14.000s	262.303us	20	20	100.00
V2		TOTAL			2245	2290	98.03
V2S	tl_intg_err	entropy_src_sec_cm	4.000s	489.954us	5	5	100.00
		entropy_src_tl_intg_err	10.000s	198.593us	20	20	100.00
V2S	sec_cm_config_regwen	entropy_src_rng	4.950m	10.012ms	300	300	100.00
		entropy_src_cfg_regwen	23.000s	41.399us	50	50	100.00
V2S	sec_cm_config_mubi	entropy_src_rng	4.950m	10.012ms	300	300	100.00
V2S	sec_cm_config_redun	entropy_src_rng	4.950m	10.012ms	300	300	100.00
V2S	sec_cm_intersig_mubi	entropy_src_rng	4.950m	10.012ms	300	300	100.00
		entropy_src_fw_ov	2.450m	5.033ms	292	300	97.33
V2S	sec_cm_main_sm_fsm_sparse	entropy_src_functional_errors	9.333m	10.012ms	975	1000	97.50
		entropy_src_sec_cm	4.000s	489.954us	5	5	100.00
V2S	sec_cm_ack_sm_fsm_sparse	entropy_src_functional_errors	9.333m	10.012ms	975	1000	97.50
		entropy_src_sec_cm	4.000s	489.954us	5	5	100.00
V2S	sec_cm_rng_bkgn_chk	entropy_src_rng	4.950m	10.012ms	300	300	100.00
V2S	sec_cm_fifo_ctr_redun	entropy_src_functional_errors	9.333m	10.012ms	975	1000	97.50
		entropy_src_sec_cm	4.000s	489.954us	5	5	100.00
V2S	sec_cm_ctr_redun	entropy_src_functional_errors	9.333m	10.012ms	975	1000	97.50
		entropy_src_sec_cm	4.000s	489.954us	5	5	100.00
V2S	sec_cm_ctr_local_esc	entropy_src_functional_errors	9.333m	10.012ms	975	1000	97.50
V2S	sec_cm_esfinal_rdata_bus_consistency	entropy_src_functional_alerts	18.000s	55.944us	50	50	100.00
V2S	sec_cm_tile_link_bus_integrity	entropy_src_tl_intg_err	10.000s	198.593us	20	20	100.00
V2S		TOTAL			75	75	100.00
V3	external_health_tests	entropy_src_rng_with_xht_rsps	4.933m	10.026ms	50	50	100.00
V3	stress_all_with_rand_reset	entropy_src_stress_all_with_rand_reset			0	0	--
V3		TOTAL			50	50	100.00
	Unmapped tests	entropy_src_intr	33.000s	3.959ms	48	50	96.00
		TOTAL			2523	2570	98.17

Testplan Progress

Items	Total	Written	Passing	Progress
N.A.	1	1	0	0.00
V1	6	6	6	100.00
V2	11	11	8	72.73
V2S	3	3	3	100.00
V3	2	1	1	50.00

Coverage Results

Coverage Dashboard

Score	Block	Branch	Statement	Expression	Toggle	Fsm	Assertion	CoverGroup
86.60	98.17	95.37	98.33	95.84	88.27	96.88	90.46	58.59

Failure Buckets

UVM_FATAL (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_read_err (addr=*) == * has 11 failures:
- Test entropy_src_functional_errors has 11 failures.
  - 28.entropy_src_functional_errors.27946729482092522695634908561140569622108159958823643373652383503511797631630
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/28.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10016320180 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_read_err (addr=0x1a2a44d8) == 0x1
  UVM_INFO @ 10016320180 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 88.entropy_src_functional_errors.34659641415404917020604580620505764937176990134518884848236936806765575347069
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/88.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10028738337 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_read_err (addr=0xb779abd8) == 0x1
  UVM_INFO @ 10028738337 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 9 more failures.
UVM_FATAL (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_state_err (addr=*) == * has 5 failures:
- Test entropy_src_functional_errors has 5 failures.
  - 34.entropy_src_functional_errors.40558040597942167729952923615167560273987599987410771964043966737449642978877
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/34.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10015928966 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_state_err (addr=0x54d6dad8) == 0x1
  UVM_INFO @ 10015928966 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 141.entropy_src_functional_errors.67748194935965879174804881652378545490230000843569673711718005825536990611092
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/141.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10034288952 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_state_err (addr=0xddf730d8) == 0x1
  UVM_INFO @ 10034288952 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 3 more failures.
UVM_ERROR (cip_base_scoreboard.sv:242) [scoreboard] Check failed expected_alert[alert_name].expected == * (* [*] vs * [*]) alert fatal_alert triggered unexpectedly has 5 failures:
- Test entropy_src_fw_ov has 5 failures.
  - 112.entropy_src_fw_ov.48322438868838398720460298547217185182947099975532647190414747673724792711619
    Line 430, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/112.entropy_src_fw_ov/latest/run.log
```
  UVM_ERROR @ 1691039467 ps: (cip_base_scoreboard.sv:242) [uvm_test_top.env.scoreboard] Check failed expected_alert[alert_name].expected == 1 (0 [0x0] vs 1 [0x1]) alert fatal_alert triggered unexpectedly
  UVM_INFO @ 1691039467 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 171.entropy_src_fw_ov.39974427899227578087135598366301319856763039598702198954684963330633155674329
    Line 634, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/171.entropy_src_fw_ov/latest/run.log
```
  UVM_ERROR @ 4571922571 ps: (cip_base_scoreboard.sv:242) [uvm_test_top.env.scoreboard] Check failed expected_alert[alert_name].expected == 1 (0 [0x0] vs 1 [0x1]) alert fatal_alert triggered unexpectedly
  UVM_INFO @ 4571922571 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 3 more failures.

Exit reason: Error: User command failed xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_sha3_*/rtl/sha*.sv,505): Assertion KeccakIdleWhenNoRunHs_A has failed has 4 failures:

Test entropy_src_rng_max_rate has 4 failures.

21.entropy_src_rng_max_rate.49339822842359311636425171975859586907955531921235534084866346932969334206531
Line 2578, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/21.entropy_src_rng_max_rate/latest/run.log

  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_sha3_0.1/rtl/sha3.sv,505): (time 10044754299 PS) Assertion tb.dut.u_entropy_src_core.u_sha3.KeccakIdleWhenNoRunHs_A has failed
  UVM_ERROR @ 10044754299 ps: (sha3.sv:505) [ASSERT FAILED] KeccakIdleWhenNoRunHs_A
  UVM_INFO @ 10044754299 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

164.entropy_src_rng_max_rate.40007852145485762443022952834939806798295452404512603103397650115384647475329
Line 3392, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/164.entropy_src_rng_max_rate/latest/run.log

  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_sha3_0.1/rtl/sha3.sv,505): (time 10109928128 PS) Assertion tb.dut.u_entropy_src_core.u_sha3.KeccakIdleWhenNoRunHs_A has failed
  UVM_ERROR @ 10109928128 ps: (sha3.sv:505) [ASSERT FAILED] KeccakIdleWhenNoRunHs_A
  UVM_INFO @ 10109928128 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 2 more failures.

UVM_FATAL (entropy_src_base_vseq.sv:442) [entropy_src_rng_vseq] Timeout encountered while reading TlSrcObserveFIFO has 4 failures:
- Test entropy_src_rng_max_rate has 4 failures.
  - 67.entropy_src_rng_max_rate.79156794890249650738452155539579990509136736722449301260126293897827635762719
    Line 1051, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/67.entropy_src_rng_max_rate/latest/run.log
```
  UVM_FATAL @ 8804477617 ps: (entropy_src_base_vseq.sv:442) [uvm_test_top.env.virtual_sequencer.entropy_src_rng_vseq] Timeout encountered while reading TlSrcObserveFIFO
  UVM_INFO @ 8804477617 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 208.entropy_src_rng_max_rate.55965395911668223360334785551583417368552139390772093302247936006772056216513
    Line 1001, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/208.entropy_src_rng_max_rate/latest/run.log
```
  UVM_FATAL @ 4766559691 ps: (entropy_src_base_vseq.sv:442) [uvm_test_top.env.virtual_sequencer.entropy_src_rng_vseq] Timeout encountered while reading TlSrcObserveFIFO
  UVM_INFO @ 4766559691 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 2 more failures.
UVM_FATAL (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.sfifo_observe_err (addr=*) == * has 4 failures:
- Test entropy_src_functional_errors has 4 failures.
  - 232.entropy_src_functional_errors.111526948357351490781390463563709103348220804543985854957357877823838260686541
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/232.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10028533648 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.sfifo_observe_err (addr=0x230fb1d8) == 0x1
  UVM_INFO @ 10028533648 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 291.entropy_src_functional_errors.102805437559616088924704184308224860711356506083416000426721363473775427174020
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/291.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10044051074 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.sfifo_observe_err (addr=0xf3367d8) == 0x1
  UVM_INFO @ 10044051074 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 2 more failures.
Exit reason: Error: User command failed UVM_ERROR (cip_base_scoreboard.sv:242) [scoreboard] Check failed expected_alert[alert_name].expected == * (* [*] vs * [*]) alert fatal_alert triggered unexpectedly has 3 failures:
- Test entropy_src_fw_ov has 3 failures.
  - 172.entropy_src_fw_ov.71637807210664781076564112268584597990506289981994137635358544678309275872979
    Line 430, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/172.entropy_src_fw_ov/latest/run.log
```
  UVM_ERROR @ 1751002158 ps: (cip_base_scoreboard.sv:242) [uvm_test_top.env.scoreboard] Check failed expected_alert[alert_name].expected == 1 (0 [0x0] vs 1 [0x1]) alert fatal_alert triggered unexpectedly
  UVM_INFO @ 1751002158 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 212.entropy_src_fw_ov.34172707169273813620460139688496348811624717400487482251652819350227733955728
    Line 481, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/212.entropy_src_fw_ov/latest/run.log
```
  UVM_ERROR @ 3126847284 ps: (cip_base_scoreboard.sv:242) [uvm_test_top.env.scoreboard] Check failed expected_alert[alert_name].expected == 1 (0 [0x0] vs 1 [0x1]) alert fatal_alert triggered unexpectedly
  UVM_INFO @ 3126847284 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 1 more failures.

Exit reason: Error: User command failed xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_entropy_src_*/rtl/entropy_src_core.sv,3390): Assertion Final_PreconFifoPushedPostStartup_A has failed has 2 failures:

Test entropy_src_intr has 2 failures.

11.entropy_src_intr.99954323965246284749546147778892060003656741789544529949755260542107640665051
Line 375, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/11.entropy_src_intr/latest/run.log

  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_entropy_src_0.1/rtl/entropy_src_core.sv,3390): (time 3958520610 PS) Assertion tb.dut.u_entropy_src_core.Final_PreconFifoPushedPostStartup_A has failed
  xmsim: *W,SLFINV: Call to process::self() from invalid process; returning null.
  xmsim: *W,SLFINV: Call to process::self() from invalid process; returning null.
  xmsim: *W,SLFINV: Call to process::self() from invalid process; returning null.
  xmsim: *W,SLFINV: Call to process::self() from invalid process; returning null.

48.entropy_src_intr.66988067007624860697634318950121398429997772945754815848741735460910354326167
Line 375, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/48.entropy_src_intr/latest/run.log

  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_entropy_src_0.1/rtl/entropy_src_core.sv,3390): (time 2188022901 PS) Assertion tb.dut.u_entropy_src_core.Final_PreconFifoPushedPostStartup_A has failed
  xmsim: *W,SLFINV: Call to process::self() from invalid process; returning null.
  xmsim: *W,SLFINV: Call to process::self() from invalid process; returning null.
  xmsim: *W,SLFINV: Call to process::self() from invalid process; returning null.
  xmsim: *W,SLFINV: Call to process::self() from invalid process; returning null.

Exit reason: Error: User command failed UVM_FATAL (entropy_src_base_vseq.sv:442) [entropy_src_rng_vseq] Timeout encountered while reading TlSrcObserveFIFO has 2 failures:
- Test entropy_src_rng_max_rate has 2 failures.
  - 174.entropy_src_rng_max_rate.112924070664693898297588619836163550580320062099378247980018747925453260496949
    Line 1155, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/174.entropy_src_rng_max_rate/latest/run.log
```
  UVM_FATAL @ 1729571762 ps: (entropy_src_base_vseq.sv:442) [uvm_test_top.env.virtual_sequencer.entropy_src_rng_vseq] Timeout encountered while reading TlSrcObserveFIFO
  UVM_INFO @ 1729571762 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 301.entropy_src_rng_max_rate.66204247724177029979397763140142179761399856020995646389689107197597732283639
    Line 2072, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/301.entropy_src_rng_max_rate/latest/run.log
```
  UVM_FATAL @ 7049297307 ps: (entropy_src_base_vseq.sv:442) [uvm_test_top.env.virtual_sequencer.entropy_src_rng_vseq] Timeout encountered while reading TlSrcObserveFIFO
  UVM_INFO @ 7049297307 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
Exit reason: Error: User command failed UVM_FATAL (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_state_err (addr=*) == * has 2 failures:
- Test entropy_src_functional_errors has 2 failures.
  - 247.entropy_src_functional_errors.100807491314300798007429030647892645437831461522163634128021344857048911421703
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/247.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10046706370 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_state_err (addr=0x6632cbd8) == 0x1
  UVM_INFO @ 10046706370 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 955.entropy_src_functional_errors.91693776069595776949182477341870389278117033382405234081785895891374328122308
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/955.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10017002090 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_state_err (addr=0x4fc5cfd8) == 0x1
  UVM_INFO @ 10017002090 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
Exit reason: Error: User command failed UVM_FATAL (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.sfifo_observe_err (addr=*) == * has 2 failures:
- Test entropy_src_functional_errors has 2 failures.
  - 625.entropy_src_functional_errors.60321033235333290800349251047074560839766141451042975311069737782841035604859
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/625.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10063385852 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.sfifo_observe_err (addr=0xa34472d8) == 0x1
  UVM_INFO @ 10063385852 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 849.entropy_src_functional_errors.2584970328292794511465258456103818516528405496005234491677145711613051710780
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/849.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10050159077 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.sfifo_observe_err (addr=0x5ce39fd8) == 0x1
  UVM_INFO @ 10050159077 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (cip_base_scoreboard.sv:302) scoreboard [scoreboard] alert recov_alert did not trigger max_delay:* has 1 failures:
- Test entropy_src_rng_max_rate has 1 failures.
  - 52.entropy_src_rng_max_rate.4541670059765871168808512563334173000987995761761389051401113099251948990438
    Line 1556, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/52.entropy_src_rng_max_rate/latest/run.log
```
  UVM_ERROR @ 8046885720 ps: (cip_base_scoreboard.sv:302) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_alert did not trigger max_delay:5
  UVM_INFO @ 8046885720 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
Exit reason: Error: User command failed UVM_ERROR (cip_base_scoreboard.sv:302) scoreboard [scoreboard] alert recov_alert did not trigger max_delay:* has 1 failures:
- Test entropy_src_rng_max_rate has 1 failures.
  - 189.entropy_src_rng_max_rate.38376344156178741155566112165656105812066169785693132413114631384093551205418
    Line 897, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/189.entropy_src_rng_max_rate/latest/run.log
```
  UVM_ERROR @ 3374444242 ps: (cip_base_scoreboard.sv:302) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_alert did not trigger max_delay:5
  UVM_INFO @ 3374444242 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
Exit reason: Error: User command failed UVM_FATAL (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_read_err (addr=*) == * has 1 failures:
- Test entropy_src_functional_errors has 1 failures.
  - 393.entropy_src_functional_errors.19186605794560771075710891024740259318931587877813072094439736287935670521802
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/393.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10011819995 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_read_err (addr=0x50b1ddd8) == 0x1
  UVM_INFO @ 10011819995 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```