ENTROPY_SRC Simulation Results

Sunday May 19 2024 19:02:23 UTC

GitHub Revision: `eb776817a5`

Branch: os_regression

Simulator: XCELIUM

Build randomization enabled with --build-seed 56458776725427632834749451790671712939002859133119076946547796163671543192855

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	entropy_src_smoke	13.000s	182.735us	50	50	100.00
V1	csr_hw_reset	entropy_src_csr_hw_reset	8.000s	53.930us	5	5	100.00
V1	csr_rw	entropy_src_csr_rw	12.000s	95.270us	20	20	100.00
V1	csr_bit_bash	entropy_src_csr_bit_bash	13.000s	528.215us	5	5	100.00
V1	csr_aliasing	entropy_src_csr_aliasing	9.000s	44.867us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	entropy_src_csr_mem_rw_with_rand_reset	8.000s	319.948us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	entropy_src_csr_rw	12.000s	95.270us	20	20	100.00
		entropy_src_csr_aliasing	9.000s	44.867us	5	5	100.00
V1		TOTAL			105	105	100.00
V2	firmware	entropy_src_smoke	13.000s	182.735us	50	50	100.00
		entropy_src_rng	4.617m	10.030ms	298	300	99.33
		entropy_src_fw_ov	2.283m	5.073ms	294	300	98.00
V2	firmware_mode	entropy_src_fw_ov	2.283m	5.073ms	294	300	98.00
V2	rng_mode	entropy_src_rng	4.617m	10.030ms	298	300	99.33
V2	rng_max_rate	entropy_src_rng_max_rate	8.717m	10.053ms	388	400	97.00
V2	health_checks	entropy_src_rng	4.617m	10.030ms	298	300	99.33
V2	conditioning	entropy_src_rng	4.617m	10.030ms	298	300	99.33
V2	interrupts	entropy_src_rng	4.617m	10.030ms	298	300	99.33
V2	alerts	entropy_src_rng	4.617m	10.030ms	298	300	99.33
		entropy_src_functional_alerts	18.000s	105.095us	50	50	100.00
V2	stress_all	entropy_src_stress_all	17.000s	494.697us	50	50	100.00
V2	functional_errors	entropy_src_functional_errors	9.917m	10.012ms	977	1000	97.70
V2	firmware_ov_read_contiguous_data	entropy_src_fw_ov_contiguous	37.000s	698.529us	50	50	100.00
V2	intr_test	entropy_src_intr_test	13.000s	53.796us	50	50	100.00
V2	alert_test	entropy_src_alert_test	12.000s	90.211us	50	50	100.00
V2	tl_d_oob_addr_access	entropy_src_tl_errors	16.000s	116.954us	20	20	100.00
V2	tl_d_illegal_access	entropy_src_tl_errors	16.000s	116.954us	20	20	100.00
V2	tl_d_outstanding_access	entropy_src_csr_hw_reset	8.000s	53.930us	5	5	100.00
		entropy_src_csr_rw	12.000s	95.270us	20	20	100.00
		entropy_src_csr_aliasing	9.000s	44.867us	5	5	100.00
		entropy_src_same_csr_outstanding	4.000s	261.758us	20	20	100.00
V2	tl_d_partial_access	entropy_src_csr_hw_reset	8.000s	53.930us	5	5	100.00
		entropy_src_csr_rw	12.000s	95.270us	20	20	100.00
		entropy_src_csr_aliasing	9.000s	44.867us	5	5	100.00
		entropy_src_same_csr_outstanding	4.000s	261.758us	20	20	100.00
V2		TOTAL			2247	2290	98.12
V2S	tl_intg_err	entropy_src_sec_cm	4.000s	98.041us	5	5	100.00
		entropy_src_tl_intg_err	13.000s	60.095us	20	20	100.00
V2S	sec_cm_config_regwen	entropy_src_rng	4.617m	10.030ms	298	300	99.33
		entropy_src_cfg_regwen	12.000s	36.933us	50	50	100.00
V2S	sec_cm_config_mubi	entropy_src_rng	4.617m	10.030ms	298	300	99.33
V2S	sec_cm_config_redun	entropy_src_rng	4.617m	10.030ms	298	300	99.33
V2S	sec_cm_intersig_mubi	entropy_src_rng	4.617m	10.030ms	298	300	99.33
		entropy_src_fw_ov	2.283m	5.073ms	294	300	98.00
V2S	sec_cm_main_sm_fsm_sparse	entropy_src_functional_errors	9.917m	10.012ms	977	1000	97.70
		entropy_src_sec_cm	4.000s	98.041us	5	5	100.00
V2S	sec_cm_ack_sm_fsm_sparse	entropy_src_functional_errors	9.917m	10.012ms	977	1000	97.70
		entropy_src_sec_cm	4.000s	98.041us	5	5	100.00
V2S	sec_cm_rng_bkgn_chk	entropy_src_rng	4.617m	10.030ms	298	300	99.33
V2S	sec_cm_fifo_ctr_redun	entropy_src_functional_errors	9.917m	10.012ms	977	1000	97.70
		entropy_src_sec_cm	4.000s	98.041us	5	5	100.00
V2S	sec_cm_ctr_redun	entropy_src_functional_errors	9.917m	10.012ms	977	1000	97.70
		entropy_src_sec_cm	4.000s	98.041us	5	5	100.00
V2S	sec_cm_ctr_local_esc	entropy_src_functional_errors	9.917m	10.012ms	977	1000	97.70
V2S	sec_cm_esfinal_rdata_bus_consistency	entropy_src_functional_alerts	18.000s	105.095us	50	50	100.00
V2S	sec_cm_tile_link_bus_integrity	entropy_src_tl_intg_err	13.000s	60.095us	20	20	100.00
V2S		TOTAL			75	75	100.00
V3	external_health_tests	entropy_src_rng_with_xht_rsps	4.333m	10.060ms	50	50	100.00
V3	stress_all_with_rand_reset	entropy_src_stress_all_with_rand_reset			0	0	--
V3		TOTAL			50	50	100.00
	Unmapped tests	entropy_src_intr	22.000s	3.997ms	40	50	80.00
		TOTAL			2517	2570	97.94

Testplan Progress

Items	Total	Written	Passing	Progress
N.A.	1	1	0	0.00
V1	6	6	6	100.00
V2	11	11	7	63.64
V2S	3	3	3	100.00
V3	2	1	1	50.00

Coverage Results

Coverage Dashboard

Score	Block	Branch	Statement	Expression	Toggle	Fsm	Assertion	CoverGroup
86.59	98.17	95.37	98.33	95.84	88.05	96.88	90.46	58.66

Failure Buckets

Exit reason: Error: User command failed xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_entropy_src_*/rtl/entropy_src_core.sv,3392): Assertion Final_PreconFifoPushedPostStartup_A has failed has 10 failures:

Test entropy_src_intr has 10 failures.

0.entropy_src_intr.27937633899025716574966845425713099211882742596351280890796535495536692613606
Line 375, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/0.entropy_src_intr/latest/run.log

  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_entropy_src_0.1/rtl/entropy_src_core.sv,3392): (time 151330255 PS) Assertion tb.dut.u_entropy_src_core.Final_PreconFifoPushedPostStartup_A has failed
  xmsim: *W,SLFINV: Call to process::self() from invalid process; returning null.
  xmsim: *W,SLFINV: Call to process::self() from invalid process; returning null.
  xmsim: *W,SLFINV: Call to process::self() from invalid process; returning null.
  xmsim: *W,SLFINV: Call to process::self() from invalid process; returning null.

2.entropy_src_intr.53028850239107421824224140393070611178744514148341641689183845952530811884385
Line 375, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/2.entropy_src_intr/latest/run.log

  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_entropy_src_0.1/rtl/entropy_src_core.sv,3392): (time 268518671 PS) Assertion tb.dut.u_entropy_src_core.Final_PreconFifoPushedPostStartup_A has failed
  xmsim: *W,SLFINV: Call to process::self() from invalid process; returning null.
  xmsim: *W,SLFINV: Call to process::self() from invalid process; returning null.
  xmsim: *W,SLFINV: Call to process::self() from invalid process; returning null.
  xmsim: *W,SLFINV: Call to process::self() from invalid process; returning null.

... and 8 more failures.

Exit reason: Error: User command failed xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_sha3_*/rtl/sha*.sv,505): Assertion KeccakIdleWhenNoRunHs_A has failed has 8 failures:

Test entropy_src_rng_max_rate has 6 failures.

1.entropy_src_rng_max_rate.107844210201878680309272502197924830526365385298115470992635095872764325843901
Line 4406, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/1.entropy_src_rng_max_rate/latest/run.log

  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_sha3_0.1/rtl/sha3.sv,505): (time 10051329494 PS) Assertion tb.dut.u_entropy_src_core.u_sha3.KeccakIdleWhenNoRunHs_A has failed
  UVM_ERROR @ 10051329494 ps: (sha3.sv:505) [ASSERT FAILED] KeccakIdleWhenNoRunHs_A
  UVM_INFO @ 10051329494 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

9.entropy_src_rng_max_rate.90628554638183618969634619433084144651850990628517917491137777064890148214776
Line 4070, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/9.entropy_src_rng_max_rate/latest/run.log

  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_sha3_0.1/rtl/sha3.sv,505): (time 10052997695 PS) Assertion tb.dut.u_entropy_src_core.u_sha3.KeccakIdleWhenNoRunHs_A has failed
  UVM_ERROR @ 10052997695 ps: (sha3.sv:505) [ASSERT FAILED] KeccakIdleWhenNoRunHs_A
  UVM_INFO @ 10052997695 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 4 more failures.

Test entropy_src_rng has 1 failures.

128.entropy_src_rng.11187806621488867620308037985150908662576373120976641312819288878125829400398
Line 1380, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/128.entropy_src_rng/latest/run.log

  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_sha3_0.1/rtl/sha3.sv,505): (time 10046039349 PS) Assertion tb.dut.u_entropy_src_core.u_sha3.KeccakIdleWhenNoRunHs_A has failed
  UVM_ERROR @ 10046039349 ps: (sha3.sv:505) [ASSERT FAILED] KeccakIdleWhenNoRunHs_A
  UVM_INFO @ 10046039349 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test entropy_src_fw_ov has 1 failures.

193.entropy_src_fw_ov.98172822900312171193811093361329261878812485835332152471041821216325626467273
Line 819, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/193.entropy_src_fw_ov/latest/run.log

  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_sha3_0.1/rtl/sha3.sv,505): (time 5001002232 PS) Assertion tb.dut.u_entropy_src_core.u_sha3.KeccakIdleWhenNoRunHs_A has failed
  UVM_ERROR @ 5001002232 ps: (sha3.sv:505) [ASSERT FAILED] KeccakIdleWhenNoRunHs_A
  UVM_INFO @ 5001002232 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_FATAL (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.sfifo_observe_err (addr=*) == * has 8 failures:
- Test entropy_src_functional_errors has 8 failures.
  - 33.entropy_src_functional_errors.90629256180620158413110758943338951377871777731819747064691532395269146900355
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/33.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10060525528 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.sfifo_observe_err (addr=0xaf624d8) == 0x1
  UVM_INFO @ 10060525528 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 125.entropy_src_functional_errors.94044860561272736257920032428271827967826838447159513382816068853680714217382
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/125.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10011811472 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.sfifo_observe_err (addr=0xbcf35ed8) == 0x1
  UVM_INFO @ 10011811472 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 6 more failures.
UVM_ERROR (cip_base_scoreboard.sv:242) [scoreboard] Check failed expected_alert[alert_name].expected == * (* [*] vs * [*]) alert fatal_alert triggered unexpectedly has 4 failures:
- Test entropy_src_fw_ov has 4 failures.
  - 63.entropy_src_fw_ov.42573769508346993131301123574311004664167700455850924272895906409802857530104
    Line 708, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/63.entropy_src_fw_ov/latest/run.log
```
  UVM_ERROR @ 2896807821 ps: (cip_base_scoreboard.sv:242) [uvm_test_top.env.scoreboard] Check failed expected_alert[alert_name].expected == 1 (0 [0x0] vs 1 [0x1]) alert fatal_alert triggered unexpectedly
  UVM_INFO @ 2896807821 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 69.entropy_src_fw_ov.47863305126392496664114156738211894722152850592487611742560139359406070348323
    Line 379, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/69.entropy_src_fw_ov/latest/run.log
```
  UVM_ERROR @ 1961982470 ps: (cip_base_scoreboard.sv:242) [uvm_test_top.env.scoreboard] Check failed expected_alert[alert_name].expected == 1 (0 [0x0] vs 1 [0x1]) alert fatal_alert triggered unexpectedly
  UVM_INFO @ 1961982470 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 2 more failures.
Exit reason: Error: User command failed UVM_FATAL (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_read_err (addr=*) == * has 4 failures:
- Test entropy_src_functional_errors has 4 failures.
  - 186.entropy_src_functional_errors.63229271291952923451010466363993005690341214348409353980234588140515027995199
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/186.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10068140565 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_read_err (addr=0x7299ecd8) == 0x1
  UVM_INFO @ 10068140565 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 294.entropy_src_functional_errors.40040925564590789651755676862561195813204935065478985118459024987577207810088
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/294.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10012865192 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_read_err (addr=0x379eddd8) == 0x1
  UVM_INFO @ 10012865192 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 2 more failures.
UVM_FATAL (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_state_err (addr=*) == * has 4 failures:
- Test entropy_src_functional_errors has 4 failures.
  - 190.entropy_src_functional_errors.16299003149384883533281409348852659554234174016842149734214801136613290928353
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/190.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10164102283 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_state_err (addr=0xced98d8) == 0x1
  UVM_INFO @ 10164102283 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 311.entropy_src_functional_errors.79819444648113581597495177958090317065141884144514239442476848819391146773609
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/311.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10066002325 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_state_err (addr=0x9e4117d8) == 0x1
  UVM_INFO @ 10066002325 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 2 more failures.
UVM_ERROR (entropy_src_scoreboard.sv:1885) [scoreboard] Check failed csr.get_mirrored_value() == item.d_data (* [*] vs * [*]) reg name: entropy_src_reg_block.alert_summary_fail_counts has 3 failures:
- Test entropy_src_rng_max_rate has 3 failures.
  - 5.entropy_src_rng_max_rate.84563730026997434149533948423145709076678059564834942187662656016465730420356
    Line 3451, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/5.entropy_src_rng_max_rate/latest/run.log
```
  UVM_ERROR @ 7765177317 ps: (entropy_src_scoreboard.sv:1885) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (0 [0x0] vs 1 [0x1]) reg name: entropy_src_reg_block.alert_summary_fail_counts
  UVM_INFO @ 7765177317 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 180.entropy_src_rng_max_rate.68262591640966404381839262960029133332831688630248810028909853873941973087462
    Line 2944, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/180.entropy_src_rng_max_rate/latest/run.log
```
  UVM_ERROR @ 5772077000 ps: (entropy_src_scoreboard.sv:1885) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (3 [0x3] vs 0 [0x0]) reg name: entropy_src_reg_block.alert_summary_fail_counts
  UVM_INFO @ 5772077000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 1 more failures.
UVM_FATAL (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_read_err (addr=*) == * has 3 failures:
- Test entropy_src_functional_errors has 3 failures.
  - 31.entropy_src_functional_errors.107982198210572567579250475143659031563421437714442326631704645739777619223444
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/31.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10020063495 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_read_err (addr=0x2a754ed8) == 0x1
  UVM_INFO @ 10020063495 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 46.entropy_src_functional_errors.72131145386900789835331482535418989408200251256952958121180060486172808578580
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/46.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10012393702 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_read_err (addr=0x31e6d8) == 0x1
  UVM_INFO @ 10012393702 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 1 more failures.
Exit reason: Error: User command failed UVM_FATAL (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_state_err (addr=*) == * has 3 failures:
- Test entropy_src_functional_errors has 3 failures.
  - 82.entropy_src_functional_errors.85364543950739341107652667858504380822893039083797146572321673671031867686388
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/82.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10012375936 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_state_err (addr=0xc8f3e9d8) == 0x1
  UVM_INFO @ 10012375936 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 131.entropy_src_functional_errors.39168374580589787958054034470681316869591614936745204269492308331121365254288
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/131.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10012443314 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_state_err (addr=0x26592bd8) == 0x1
  UVM_INFO @ 10012443314 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 1 more failures.
UVM_FATAL (entropy_src_base_vseq.sv:442) [entropy_src_rng_vseq] Timeout encountered while reading TlSrcObserveFIFO has 2 failures:
- Test entropy_src_rng_max_rate has 2 failures.
  - 226.entropy_src_rng_max_rate.7198580272528657305563413357485926157273739044100534606761356787304421251920
    Line 2534, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/226.entropy_src_rng_max_rate/latest/run.log
```
  UVM_FATAL @ 7300022238 ps: (entropy_src_base_vseq.sv:442) [uvm_test_top.env.virtual_sequencer.entropy_src_rng_vseq] Timeout encountered while reading TlSrcObserveFIFO
  UVM_INFO @ 7300022238 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 264.entropy_src_rng_max_rate.94160854099590287867511311440796227929259017510288395988892683151853296481645
    Line 1344, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/264.entropy_src_rng_max_rate/latest/run.log
```
  UVM_FATAL @ 7111317383 ps: (entropy_src_base_vseq.sv:442) [uvm_test_top.env.virtual_sequencer.entropy_src_rng_vseq] Timeout encountered while reading TlSrcObserveFIFO
  UVM_INFO @ 7111317383 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
Exit reason: Error: User command failed UVM_FATAL (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.sfifo_observe_err (addr=*) == * has 1 failures:
- Test entropy_src_functional_errors has 1 failures.
  - 4.entropy_src_functional_errors.80390796379983229079160286698739923559402368599330270957949935949248169114827
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/4.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10055975737 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.sfifo_observe_err (addr=0x335a67d8) == 0x1
  UVM_INFO @ 10055975737 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (entropy_src_scoreboard.sv:1885) [scoreboard] Check failed csr.get_mirrored_value() == item.d_data (* [*] vs * [*]) reg name: entropy_src_reg_block.recov_alert_sts has 1 failures:
- Test entropy_src_rng has 1 failures.
  - 87.entropy_src_rng.80681078300502331603808373128607530837099557282888034009366529716964608859168
    Line 543, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/87.entropy_src_rng/latest/run.log
```
  UVM_ERROR @ 6165314457 ps: (entropy_src_scoreboard.sv:1885) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (4096 [0x1000] vs 0 [0x0]) reg name: entropy_src_reg_block.recov_alert_sts
  UVM_INFO @ 6165314457 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
Exit reason: Error: User command failed UVM_FATAL (entropy_src_base_vseq.sv:442) [entropy_src_rng_vseq] Timeout encountered while reading TlSrcObserveFIFO has 1 failures:
- Test entropy_src_rng_max_rate has 1 failures.
  - 108.entropy_src_rng_max_rate.43233420546102996999711656165949568483439937023233010241747863763787771300921
    Line 795, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/108.entropy_src_rng_max_rate/latest/run.log
```
  UVM_FATAL @ 2685233686 ps: (entropy_src_base_vseq.sv:442) [uvm_test_top.env.virtual_sequencer.entropy_src_rng_vseq] Timeout encountered while reading TlSrcObserveFIFO
  UVM_INFO @ 2685233686 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
Exit reason: Error: User command failed UVM_ERROR (cip_base_scoreboard.sv:242) [scoreboard] Check failed expected_alert[alert_name].expected == * (* [*] vs * [*]) alert fatal_alert triggered unexpectedly has 1 failures:
- Test entropy_src_fw_ov has 1 failures.
  - 144.entropy_src_fw_ov.65760134940872765711287369456164972251290729185249522060744429986630591592567
    Line 814, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/144.entropy_src_fw_ov/latest/run.log
```
  UVM_ERROR @ 4844581864 ps: (cip_base_scoreboard.sv:242) [uvm_test_top.env.scoreboard] Check failed expected_alert[alert_name].expected == 1 (0 [0x0] vs 1 [0x1]) alert fatal_alert triggered unexpectedly
  UVM_INFO @ 4844581864 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```