ENTROPY_SRC Simulation Results

Sunday May 05 2024 19:05:13 UTC

GitHub Revision: `d0c52cdadd`

Branch: os_regression

Simulator: XCELIUM

Build randomization enabled with --build-seed 81579111587980121648949789282063322266496016209500883225240730864920651071561

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	entropy_src_smoke	4.000s	29.975us	50	50	100.00
V1	csr_hw_reset	entropy_src_csr_hw_reset	3.000s	101.213us	5	5	100.00
V1	csr_rw	entropy_src_csr_rw	3.000s	59.101us	20	20	100.00
V1	csr_bit_bash	entropy_src_csr_bit_bash	14.000s	3.179ms	5	5	100.00
V1	csr_aliasing	entropy_src_csr_aliasing	7.000s	683.867us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	entropy_src_csr_mem_rw_with_rand_reset	4.000s	35.154us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	entropy_src_csr_rw	3.000s	59.101us	20	20	100.00
		entropy_src_csr_aliasing	7.000s	683.867us	5	5	100.00
V1		TOTAL			105	105	100.00
V2	firmware	entropy_src_smoke	4.000s	29.975us	50	50	100.00
		entropy_src_rng	5.050m	10.059ms	298	300	99.33
		entropy_src_fw_ov	2.417m	5.025ms	289	300	96.33
V2	firmware_mode	entropy_src_fw_ov	2.417m	5.025ms	289	300	96.33
V2	rng_mode	entropy_src_rng	5.050m	10.059ms	298	300	99.33
V2	rng_max_rate	entropy_src_rng_max_rate	9.550m	10.050ms	388	400	97.00
V2	health_checks	entropy_src_rng	5.050m	10.059ms	298	300	99.33
V2	conditioning	entropy_src_rng	5.050m	10.059ms	298	300	99.33
V2	interrupts	entropy_src_rng	5.050m	10.059ms	298	300	99.33
V2	alerts	entropy_src_rng	5.050m	10.059ms	298	300	99.33
		entropy_src_functional_alerts	5.000s	56.578us	50	50	100.00
V2	stress_all	entropy_src_stress_all	10.000s	2.502ms	50	50	100.00
V2	functional_errors	entropy_src_functional_errors	10.383m	10.013ms	963	1000	96.30
V2	firmware_ov_read_contiguous_data	entropy_src_fw_ov_contiguous	24.000s	356.929us	50	50	100.00
V2	intr_test	entropy_src_intr_test	3.000s	62.550us	50	50	100.00
V2	alert_test	entropy_src_alert_test	4.000s	36.117us	50	50	100.00
V2	tl_d_oob_addr_access	entropy_src_tl_errors	7.000s	127.132us	20	20	100.00
V2	tl_d_illegal_access	entropy_src_tl_errors	7.000s	127.132us	20	20	100.00
V2	tl_d_outstanding_access	entropy_src_csr_hw_reset	3.000s	101.213us	5	5	100.00
		entropy_src_csr_rw	3.000s	59.101us	20	20	100.00
		entropy_src_csr_aliasing	7.000s	683.867us	5	5	100.00
		entropy_src_same_csr_outstanding	5.000s	89.000us	20	20	100.00
V2	tl_d_partial_access	entropy_src_csr_hw_reset	3.000s	101.213us	5	5	100.00
		entropy_src_csr_rw	3.000s	59.101us	20	20	100.00
		entropy_src_csr_aliasing	7.000s	683.867us	5	5	100.00
		entropy_src_same_csr_outstanding	5.000s	89.000us	20	20	100.00
V2		TOTAL			2228	2290	97.29
V2S	tl_intg_err	entropy_src_sec_cm	4.000s	92.019us	5	5	100.00
		entropy_src_tl_intg_err	7.000s	213.614us	20	20	100.00
V2S	sec_cm_config_regwen	entropy_src_rng	5.050m	10.059ms	298	300	99.33
		entropy_src_cfg_regwen	4.000s	63.299us	50	50	100.00
V2S	sec_cm_config_mubi	entropy_src_rng	5.050m	10.059ms	298	300	99.33
V2S	sec_cm_config_redun	entropy_src_rng	5.050m	10.059ms	298	300	99.33
V2S	sec_cm_intersig_mubi	entropy_src_rng	5.050m	10.059ms	298	300	99.33
		entropy_src_fw_ov	2.417m	5.025ms	289	300	96.33
V2S	sec_cm_main_sm_fsm_sparse	entropy_src_functional_errors	10.383m	10.013ms	963	1000	96.30
		entropy_src_sec_cm	4.000s	92.019us	5	5	100.00
V2S	sec_cm_ack_sm_fsm_sparse	entropy_src_functional_errors	10.383m	10.013ms	963	1000	96.30
		entropy_src_sec_cm	4.000s	92.019us	5	5	100.00
V2S	sec_cm_rng_bkgn_chk	entropy_src_rng	5.050m	10.059ms	298	300	99.33
V2S	sec_cm_fifo_ctr_redun	entropy_src_functional_errors	10.383m	10.013ms	963	1000	96.30
		entropy_src_sec_cm	4.000s	92.019us	5	5	100.00
V2S	sec_cm_ctr_redun	entropy_src_functional_errors	10.383m	10.013ms	963	1000	96.30
		entropy_src_sec_cm	4.000s	92.019us	5	5	100.00
V2S	sec_cm_ctr_local_esc	entropy_src_functional_errors	10.383m	10.013ms	963	1000	96.30
V2S	sec_cm_esfinal_rdata_bus_consistency	entropy_src_functional_alerts	5.000s	56.578us	50	50	100.00
V2S	sec_cm_tile_link_bus_integrity	entropy_src_tl_intg_err	7.000s	213.614us	20	20	100.00
V2S		TOTAL			75	75	100.00
V3	external_health_tests	entropy_src_rng_with_xht_rsps	4.817m	10.091ms	50	50	100.00
V3	stress_all_with_rand_reset	entropy_src_stress_all_with_rand_reset			0	0	--
V3		TOTAL			50	50	100.00
	Unmapped tests	entropy_src_intr	19.000s	516.817us	43	50	86.00
		TOTAL			2501	2570	97.32

Testplan Progress

Items	Total	Written	Passing	Progress
N.A.	1	1	0	0.00
V1	6	6	6	100.00
V2	11	11	7	63.64
V2S	3	3	3	100.00
V3	2	1	1	50.00

Coverage Results

Coverage Dashboard

Score	Block	Branch	Statement	Expression	Toggle	Fsm	Assertion	CoverGroup
86.64	98.21	95.47	98.36	95.84	88.10	96.88	90.46	58.73

Failure Buckets

UVM_FATAL (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.sfifo_observe_err (addr=*) == * has 9 failures:
- Test entropy_src_functional_errors has 9 failures.
  - 93.entropy_src_functional_errors.51018765931422734791743539942383072549843731865211535480283175901920664355274
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/93.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10124895941 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.sfifo_observe_err (addr=0x83978ed8) == 0x1
  UVM_INFO @ 10124895941 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 101.entropy_src_functional_errors.75845291685087394668924363613012490890617887490400919639825359945567648018056
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/101.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10022760174 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.sfifo_observe_err (addr=0x594f9dd8) == 0x1
  UVM_INFO @ 10022760174 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 7 more failures.

Exit reason: Error: User command failed xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_sha3_*/rtl/sha*.sv,505): Assertion KeccakIdleWhenNoRunHs_A has failed has 8 failures:

Test entropy_src_rng_max_rate has 6 failures.

5.entropy_src_rng_max_rate.22352247446188685229613697164506002684485379769255728611943407249576236193403
Line 4986, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/5.entropy_src_rng_max_rate/latest/run.log

  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_sha3_0.1/rtl/sha3.sv,505): (time 10061459910 PS) Assertion tb.dut.u_entropy_src_core.u_sha3.KeccakIdleWhenNoRunHs_A has failed
  UVM_ERROR @ 10061459910 ps: (sha3.sv:505) [ASSERT FAILED] KeccakIdleWhenNoRunHs_A
  UVM_INFO @ 10061459910 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

66.entropy_src_rng_max_rate.96579326538107428955066190884832345658141787534107527625127156143411414120517
Line 1968, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/66.entropy_src_rng_max_rate/latest/run.log

  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_sha3_0.1/rtl/sha3.sv,505): (time 10198678141 PS) Assertion tb.dut.u_entropy_src_core.u_sha3.KeccakIdleWhenNoRunHs_A has failed
  UVM_ERROR @ 10198678141 ps: (sha3.sv:505) [ASSERT FAILED] KeccakIdleWhenNoRunHs_A
  UVM_INFO @ 10198678141 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 4 more failures.

Test entropy_src_fw_ov has 1 failures.

67.entropy_src_fw_ov.38065550079955925879131053715454447483699663530133090196543192968972425467926
Line 1178, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/67.entropy_src_fw_ov/latest/run.log

  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_sha3_0.1/rtl/sha3.sv,505): (time 5001065792 PS) Assertion tb.dut.u_entropy_src_core.u_sha3.KeccakIdleWhenNoRunHs_A has failed
  UVM_ERROR @ 5001065792 ps: (sha3.sv:505) [ASSERT FAILED] KeccakIdleWhenNoRunHs_A
  UVM_INFO @ 5001065792 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test entropy_src_rng has 1 failures.

293.entropy_src_rng.18029589201065957944735746188561839719725175332087820007887045209977191520157
Line 3326, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/293.entropy_src_rng/latest/run.log

  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_sha3_0.1/rtl/sha3.sv,505): (time 10028079916 PS) Assertion tb.dut.u_entropy_src_core.u_sha3.KeccakIdleWhenNoRunHs_A has failed
  UVM_ERROR @ 10028079916 ps: (sha3.sv:505) [ASSERT FAILED] KeccakIdleWhenNoRunHs_A
  UVM_INFO @ 10028079916 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_ERROR (cip_base_scoreboard.sv:242) [scoreboard] Check failed expected_alert[alert_name].expected == * (* [*] vs * [*]) alert fatal_alert triggered unexpectedly has 8 failures:
- Test entropy_src_fw_ov has 8 failures.
  - 9.entropy_src_fw_ov.59806641710059547860952196595861330582481724033137698307072660639774922075930
    Line 1155, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/9.entropy_src_fw_ov/latest/run.log
```
  UVM_ERROR @ 3921532065 ps: (cip_base_scoreboard.sv:242) [uvm_test_top.env.scoreboard] Check failed expected_alert[alert_name].expected == 1 (0 [0x0] vs 1 [0x1]) alert fatal_alert triggered unexpectedly
  UVM_INFO @ 3921532065 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 24.entropy_src_fw_ov.94454812239648555423129038031557082172984704338599905150987214472663186057522
    Line 793, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/24.entropy_src_fw_ov/latest/run.log
```
  UVM_ERROR @ 3717706893 ps: (cip_base_scoreboard.sv:242) [uvm_test_top.env.scoreboard] Check failed expected_alert[alert_name].expected == 1 (0 [0x0] vs 1 [0x1]) alert fatal_alert triggered unexpectedly
  UVM_INFO @ 3717706893 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 6 more failures.
UVM_FATAL (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_read_err (addr=*) == * has 8 failures:
- Test entropy_src_functional_errors has 8 failures.
  - 57.entropy_src_functional_errors.81740493933439793061802335943320009052022580731338915471925712664666519654320
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/57.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10043788283 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_read_err (addr=0x63dca1d8) == 0x1
  UVM_INFO @ 10043788283 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 225.entropy_src_functional_errors.114849468070444729388063691001914878230929649430186442268767907750940213893292
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/225.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10012639917 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_read_err (addr=0xbdf437d8) == 0x1
  UVM_INFO @ 10012639917 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 6 more failures.
UVM_FATAL (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_state_err (addr=*) == * has 8 failures:
- Test entropy_src_functional_errors has 8 failures.
  - 152.entropy_src_functional_errors.101497041880308846023639695585827804434159709516170742587941910267208838186276
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/152.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10029371967 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_state_err (addr=0x786f8dd8) == 0x1
  UVM_INFO @ 10029371967 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 174.entropy_src_functional_errors.77514769411806560927670382161828949073047603005531451516850817517978344950094
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/174.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10152227287 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_state_err (addr=0x414657d8) == 0x1
  UVM_INFO @ 10152227287 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 6 more failures.

Exit reason: Error: User command failed xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_entropy_src_*/rtl/entropy_src_core.sv,3392): Assertion Final_PreconFifoPushedPostStartup_A has failed has 7 failures:

Test entropy_src_intr has 7 failures.

6.entropy_src_intr.101458453151745248693078728928746127490595561950531606493838111185318463599175
Line 375, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/6.entropy_src_intr/latest/run.log

  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_entropy_src_0.1/rtl/entropy_src_core.sv,3392): (time 963768542 PS) Assertion tb.dut.u_entropy_src_core.Final_PreconFifoPushedPostStartup_A has failed
  xmsim: *W,SLFINV: Call to process::self() from invalid process; returning null.
  xmsim: *W,SLFINV: Call to process::self() from invalid process; returning null.
  xmsim: *W,SLFINV: Call to process::self() from invalid process; returning null.
  xmsim: *W,SLFINV: Call to process::self() from invalid process; returning null.

22.entropy_src_intr.80828249379953398951085493234197747024369402336714682376927340056673264346200
Line 375, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/22.entropy_src_intr/latest/run.log

  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_entropy_src_0.1/rtl/entropy_src_core.sv,3392): (time 587268287 PS) Assertion tb.dut.u_entropy_src_core.Final_PreconFifoPushedPostStartup_A has failed
  xmsim: *W,SLFINV: Call to process::self() from invalid process; returning null.
  xmsim: *W,SLFINV: Call to process::self() from invalid process; returning null.
  xmsim: *W,SLFINV: Call to process::self() from invalid process; returning null.
  xmsim: *W,SLFINV: Call to process::self() from invalid process; returning null.

... and 5 more failures.

Exit reason: Error: User command failed UVM_FATAL (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_state_err (addr=*) == * has 6 failures:
- Test entropy_src_functional_errors has 6 failures.
  - 100.entropy_src_functional_errors.43114629682869183071983368159185551289351672064039394444637611531583522703370
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/100.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10017327746 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_state_err (addr=0x1c5849d8) == 0x1
  UVM_INFO @ 10017327746 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 206.entropy_src_functional_errors.86784357477149890362970163631704575737770410861508734716519265340674923451501
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/206.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10017624349 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_state_err (addr=0xaf819ad8) == 0x1
  UVM_INFO @ 10017624349 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 4 more failures.
Exit reason: Error: User command failed UVM_FATAL (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_read_err (addr=*) == * has 4 failures:
- Test entropy_src_functional_errors has 4 failures.
  - 233.entropy_src_functional_errors.25304750237666229191493079539684237898718042407953984768912683640985537208555
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/233.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10033743580 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_read_err (addr=0xf1d0cdd8) == 0x1
  UVM_INFO @ 10033743580 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 441.entropy_src_functional_errors.33102993303178205756004233028824482229528318449095047354604950955517966838275
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/441.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10061150375 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_read_err (addr=0x737e88d8) == 0x1
  UVM_INFO @ 10061150375 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 2 more failures.
Exit reason: Error: User command failed UVM_FATAL (entropy_src_base_vseq.sv:442) [entropy_src_rng_vseq] Timeout encountered while reading TlSrcObserveFIFO has 2 failures:
- Test entropy_src_rng_max_rate has 2 failures.
  - 226.entropy_src_rng_max_rate.59028612509978460788842776340756006011791189066287050151243769446835554678926
    Line 884, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/226.entropy_src_rng_max_rate/latest/run.log
```
  UVM_FATAL @ 3814237923 ps: (entropy_src_base_vseq.sv:442) [uvm_test_top.env.virtual_sequencer.entropy_src_rng_vseq] Timeout encountered while reading TlSrcObserveFIFO
  UVM_INFO @ 3814237923 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 230.entropy_src_rng_max_rate.105419146415023069867048139150796830176560203443510441058178019567373915679648
    Line 2940, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/230.entropy_src_rng_max_rate/latest/run.log
```
  UVM_FATAL @ 6457670617 ps: (entropy_src_base_vseq.sv:442) [uvm_test_top.env.virtual_sequencer.entropy_src_rng_vseq] Timeout encountered while reading TlSrcObserveFIFO
  UVM_INFO @ 6457670617 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
Exit reason: Error: User command failed UVM_ERROR (cip_base_scoreboard.sv:242) [scoreboard] Check failed expected_alert[alert_name].expected == * (* [*] vs * [*]) alert fatal_alert triggered unexpectedly has 2 failures:
- Test entropy_src_fw_ov has 2 failures.
  - 236.entropy_src_fw_ov.54454223870793596985300070528280099192207958509417094456548787317648060619117
    Line 430, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/236.entropy_src_fw_ov/latest/run.log
```
  UVM_ERROR @ 3130411015 ps: (cip_base_scoreboard.sv:242) [uvm_test_top.env.scoreboard] Check failed expected_alert[alert_name].expected == 1 (0 [0x0] vs 1 [0x1]) alert fatal_alert triggered unexpectedly
  UVM_INFO @ 3130411015 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 265.entropy_src_fw_ov.51552556335716095566811408326600448655278812600664050616681838076612892457475
    Line 379, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/265.entropy_src_fw_ov/latest/run.log
```
  UVM_ERROR @ 1780499696 ps: (cip_base_scoreboard.sv:242) [uvm_test_top.env.scoreboard] Check failed expected_alert[alert_name].expected == 1 (0 [0x0] vs 1 [0x1]) alert fatal_alert triggered unexpectedly
  UVM_INFO @ 1780499696 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_FATAL (entropy_src_base_vseq.sv:442) [entropy_src_rng_vseq] Timeout encountered while reading TlSrcObserveFIFO has 2 failures:
- Test entropy_src_rng_max_rate has 2 failures.
  - 292.entropy_src_rng_max_rate.48773156895158964696289936630993857812308770532377724096873533436692062360667
    Line 1795, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/292.entropy_src_rng_max_rate/latest/run.log
```
  UVM_FATAL @ 9345921674 ps: (entropy_src_base_vseq.sv:442) [uvm_test_top.env.virtual_sequencer.entropy_src_rng_vseq] Timeout encountered while reading TlSrcObserveFIFO
  UVM_INFO @ 9345921674 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 305.entropy_src_rng_max_rate.114249711601798222209721501645994193494508034524530775822593081684067499492175
    Line 1819, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/305.entropy_src_rng_max_rate/latest/run.log
```
  UVM_FATAL @ 7016505346 ps: (entropy_src_base_vseq.sv:442) [uvm_test_top.env.virtual_sequencer.entropy_src_rng_vseq] Timeout encountered while reading TlSrcObserveFIFO
  UVM_INFO @ 7016505346 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
Exit reason: Error: User command failed UVM_FATAL (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.sfifo_observe_err (addr=*) == * has 2 failures:
- Test entropy_src_functional_errors has 2 failures.
  - 909.entropy_src_functional_errors.115095472182117159804505074251957983329076639737943562592965980592737943592119
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/909.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10015038576 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.sfifo_observe_err (addr=0xdfc69dd8) == 0x1
  UVM_INFO @ 10015038576 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 992.entropy_src_functional_errors.87110154136366683554342823659093955567237031901543451658177795399532150269013
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/992.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10017443048 ps: (csr_utils_pkg.sv:577) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.sfifo_observe_err (addr=0xa7e4f5d8) == 0x1
  UVM_INFO @ 10017443048 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (entropy_src_scoreboard.sv:1885) [scoreboard] Check failed csr.get_mirrored_value() == item.d_data (* [*] vs * [*]) reg name: entropy_src_reg_block.alert_summary_fail_counts has 1 failures:
- Test entropy_src_rng_max_rate has 1 failures.
  - 44.entropy_src_rng_max_rate.87886719489140244393754065638874333372837425149225499081281677504928830478388
    Line 1411, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/44.entropy_src_rng_max_rate/latest/run.log
```
  UVM_ERROR @ 3836996096 ps: (entropy_src_scoreboard.sv:1885) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (187 [0xbb] vs 0 [0x0]) reg name: entropy_src_reg_block.alert_summary_fail_counts
  UVM_INFO @ 3836996096 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
Exit reason: Error: User command failed UVM_ERROR (cip_base_scoreboard.sv:302) scoreboard [scoreboard] alert recov_alert did not trigger max_delay:* has 1 failures:
- Test entropy_src_rng_max_rate has 1 failures.
  - 160.entropy_src_rng_max_rate.18048701411940659214589683605814317992828582924210388798502887936069303803944
    Line 2269, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/160.entropy_src_rng_max_rate/latest/run.log
```
  UVM_ERROR @ 4577066252 ps: (cip_base_scoreboard.sv:302) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_alert did not trigger max_delay:5
  UVM_INFO @ 4577066252 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (cip_base_scoreboard.sv:302) scoreboard [scoreboard] alert recov_alert did not trigger max_delay:* has 1 failures:
- Test entropy_src_rng has 1 failures.
  - 288.entropy_src_rng.33412396248735923535791534545164709743288331847482790902445971959701257916827
    Line 1167, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/288.entropy_src_rng/latest/run.log
```
  UVM_ERROR @ 5780933293 ps: (cip_base_scoreboard.sv:302) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_alert did not trigger max_delay:5
  UVM_INFO @ 5780933293 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```