ENTROPY_SRC Simulation Results

Monday July 29 2024 23:02:32 UTC

GitHub Revision: `39f3866b56`

Branch: os_regression

Simulator: XCELIUM

Build randomization enabled with --build-seed 91682663165753342493852681547271085771042321116470426223748766059309541455602

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	entropy_src_smoke	8.000s	22.202us	50	50	100.00
V1	csr_hw_reset	entropy_src_csr_hw_reset	4.000s	67.434us	5	5	100.00
V1	csr_rw	entropy_src_csr_rw	4.000s	31.005us	20	20	100.00
V1	csr_bit_bash	entropy_src_csr_bit_bash	14.000s	514.395us	5	5	100.00
V1	csr_aliasing	entropy_src_csr_aliasing	8.000s	237.613us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	entropy_src_csr_mem_rw_with_rand_reset	5.000s	135.104us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	entropy_src_csr_rw	4.000s	31.005us	20	20	100.00
		entropy_src_csr_aliasing	8.000s	237.613us	5	5	100.00
V1		TOTAL			105	105	100.00
V2	firmware	entropy_src_smoke	8.000s	22.202us	50	50	100.00
		entropy_src_rng	5.583m	10.019ms	298	300	99.33
		entropy_src_fw_ov	3.100m	5.054ms	287	300	95.67
V2	firmware_mode	entropy_src_fw_ov	3.100m	5.054ms	287	300	95.67
V2	rng_mode	entropy_src_rng	5.583m	10.019ms	298	300	99.33
V2	rng_max_rate	entropy_src_rng_max_rate	9.783m	10.017ms	394	400	98.50
V2	health_checks	entropy_src_rng	5.583m	10.019ms	298	300	99.33
V2	conditioning	entropy_src_rng	5.583m	10.019ms	298	300	99.33
V2	interrupts	entropy_src_rng	5.583m	10.019ms	298	300	99.33
		entropy_src_intr	33.000s	2.042ms	50	50	100.00
V2	alerts	entropy_src_rng	5.583m	10.019ms	298	300	99.33
		entropy_src_functional_alerts	5.000s	91.309us	50	50	100.00
V2	stress_all	entropy_src_stress_all	14.000s	745.994us	50	50	100.00
V2	functional_errors	entropy_src_functional_errors	9.750m	10.012ms	959	1000	95.90
V2	firmware_ov_read_contiguous_data	entropy_src_fw_ov_contiguous	25.000s	1.191ms	50	50	100.00
V2	intr_test	entropy_src_intr_test	4.000s	32.093us	50	50	100.00
V2	alert_test	entropy_src_alert_test	8.000s	35.182us	50	50	100.00
V2	tl_d_oob_addr_access	entropy_src_tl_errors	6.000s	168.107us	20	20	100.00
V2	tl_d_illegal_access	entropy_src_tl_errors	6.000s	168.107us	20	20	100.00
V2	tl_d_outstanding_access	entropy_src_csr_hw_reset	4.000s	67.434us	5	5	100.00
		entropy_src_csr_rw	4.000s	31.005us	20	20	100.00
		entropy_src_csr_aliasing	8.000s	237.613us	5	5	100.00
		entropy_src_same_csr_outstanding	5.000s	174.661us	20	20	100.00
V2	tl_d_partial_access	entropy_src_csr_hw_reset	4.000s	67.434us	5	5	100.00
		entropy_src_csr_rw	4.000s	31.005us	20	20	100.00
		entropy_src_csr_aliasing	8.000s	237.613us	5	5	100.00
		entropy_src_same_csr_outstanding	5.000s	174.661us	20	20	100.00
V2		TOTAL			2278	2340	97.35
V2S	tl_intg_err	entropy_src_sec_cm	3.000s	55.906us	5	5	100.00
		entropy_src_tl_intg_err	7.000s	177.188us	20	20	100.00
V2S	sec_cm_config_regwen	entropy_src_rng	5.583m	10.019ms	298	300	99.33
		entropy_src_cfg_regwen	12.000s	86.298us	50	50	100.00
V2S	sec_cm_config_mubi	entropy_src_rng	5.583m	10.019ms	298	300	99.33
V2S	sec_cm_config_redun	entropy_src_rng	5.583m	10.019ms	298	300	99.33
V2S	sec_cm_intersig_mubi	entropy_src_rng	5.583m	10.019ms	298	300	99.33
		entropy_src_fw_ov	3.100m	5.054ms	287	300	95.67
V2S	sec_cm_main_sm_fsm_sparse	entropy_src_functional_errors	9.750m	10.012ms	959	1000	95.90
		entropy_src_sec_cm	3.000s	55.906us	5	5	100.00
V2S	sec_cm_ack_sm_fsm_sparse	entropy_src_functional_errors	9.750m	10.012ms	959	1000	95.90
		entropy_src_sec_cm	3.000s	55.906us	5	5	100.00
V2S	sec_cm_rng_bkgn_chk	entropy_src_rng	5.583m	10.019ms	298	300	99.33
V2S	sec_cm_fifo_ctr_redun	entropy_src_functional_errors	9.750m	10.012ms	959	1000	95.90
		entropy_src_sec_cm	3.000s	55.906us	5	5	100.00
V2S	sec_cm_ctr_redun	entropy_src_functional_errors	9.750m	10.012ms	959	1000	95.90
		entropy_src_sec_cm	3.000s	55.906us	5	5	100.00
V2S	sec_cm_ctr_local_esc	entropy_src_functional_errors	9.750m	10.012ms	959	1000	95.90
V2S	sec_cm_esfinal_rdata_bus_consistency	entropy_src_functional_alerts	5.000s	91.309us	50	50	100.00
V2S	sec_cm_tile_link_bus_integrity	entropy_src_tl_intg_err	7.000s	177.188us	20	20	100.00
V2S		TOTAL			75	75	100.00
V3	external_health_tests	entropy_src_rng_with_xht_rsps	5.267m	10.020ms	50	50	100.00
V3	stress_all_with_rand_reset	entropy_src_stress_all_with_rand_reset			0	0	--
V3		TOTAL			50	50	100.00
		TOTAL			2508	2570	97.59

Testplan Progress

Items	Total	Written	Passing	Progress
V1	6	6	6	100.00
V2	12	12	8	66.67
V2S	3	3	3	100.00
V3	2	1	1	50.00

Coverage Results

Coverage Dashboard

Score	Block	Branch	Statement	Expression	Toggle	Fsm	Assertion	CoverGroup
97.08	98.15	95.32	98.33	95.79	96.59	96.88	90.48	95.47

Failure Buckets

UVM_FATAL (csr_utils_pkg.sv:597) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_state_err (addr=*, Comparison=CompareOpEq, exp_data=*, call_count=2) has 10 failures:
- Test entropy_src_functional_errors has 10 failures.
  - 119.entropy_src_functional_errors.37360149602684568458859737298001496748141239559983438477607633465102343935134
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/119.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10011784787 ps: (csr_utils_pkg.sv:597) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_state_err (addr=0xdd89dfd8, Comparison=CompareOpEq, exp_data=0x1, call_count=2)
  UVM_INFO @ 10011784787 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 195.entropy_src_functional_errors.47280869799015235949827269774189481135950292860263998629233419797382454136475
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/195.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10014864647 ps: (csr_utils_pkg.sv:597) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_state_err (addr=0x98df6fd8, Comparison=CompareOpEq, exp_data=0x1, call_count=2)
  UVM_INFO @ 10014864647 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 8 more failures.

UVM_ERROR (entropy_src_scoreboard.sv:2031) [scoreboard] Check failed csr.get_mirrored_value() == item.d_data (* [*] vs * [*]) reg name: entropy_src_reg_block.err_code has 9 failures:

Test entropy_src_fw_ov has 9 failures.

21.entropy_src_fw_ov.12539345063167677728019049708674554671292144867933752699008948261305357841407
Line 635, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/21.entropy_src_fw_ov/latest/run.log

  UVM_ERROR @ 4596789224 ps: (entropy_src_scoreboard.sv:2031) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (1048576 [0x100000] vs 3145728 [0x300000]) reg name: entropy_src_reg_block.err_code
  UVM_INFO @ 4596789224 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

63.entropy_src_fw_ov.6904203251166242784286038376250003069001481936471105563871184179374990916124
Line 481, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/63.entropy_src_fw_ov/latest/run.log

  UVM_ERROR @ 2848689011 ps: (entropy_src_scoreboard.sv:2031) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (1048576 [0x100000] vs 3145728 [0x300000]) reg name: entropy_src_reg_block.err_code
  UVM_INFO @ 2848689011 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 7 more failures.

UVM_FATAL (csr_utils_pkg.sv:597) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_read_err (addr=*, Comparison=CompareOpEq, exp_data=*, call_count=2) has 8 failures:
- Test entropy_src_functional_errors has 8 failures.
  - 62.entropy_src_functional_errors.58143279044987563570873847091123213425628614187467719903880427740461298464690
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/62.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10094719392 ps: (csr_utils_pkg.sv:597) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_read_err (addr=0x1eea20d8, Comparison=CompareOpEq, exp_data=0x1, call_count=2)
  UVM_INFO @ 10094719392 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 114.entropy_src_functional_errors.31749887907458014278060263317778584599336619629690653504191962103480355078299
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/114.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10069051941 ps: (csr_utils_pkg.sv:597) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_read_err (addr=0x36e33fd8, Comparison=CompareOpEq, exp_data=0x1, call_count=2)
  UVM_INFO @ 10069051941 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 6 more failures.
Exit reason: Error: User command failed UVM_FATAL (csr_utils_pkg.sv:597) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_read_err (addr=*, Comparison=CompareOpEq, exp_data=*, call_count=2) has 8 failures:
- Test entropy_src_functional_errors has 8 failures.
  - 110.entropy_src_functional_errors.1811721187052794719677707017823791350826626848798376767961851524068454026763
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/110.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10016513960 ps: (csr_utils_pkg.sv:597) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_read_err (addr=0xa0e97fd8, Comparison=CompareOpEq, exp_data=0x1, call_count=2)
  UVM_INFO @ 10016513960 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 157.entropy_src_functional_errors.45665506037715431560215402831415740718225942720862224693711567522083514680748
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/157.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10018114987 ps: (csr_utils_pkg.sv:597) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_read_err (addr=0x32b052d8, Comparison=CompareOpEq, exp_data=0x1, call_count=2)
  UVM_INFO @ 10018114987 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 6 more failures.
UVM_FATAL (csr_utils_pkg.sv:597) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.sfifo_observe_err (addr=*, Comparison=CompareOpEq, exp_data=*, call_count=2) has 8 failures:
- Test entropy_src_functional_errors has 8 failures.
  - 115.entropy_src_functional_errors.81933481215116949210946453799999466610463102776293131622169863728213467292338
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/115.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10045224334 ps: (csr_utils_pkg.sv:597) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.sfifo_observe_err (addr=0xd0a2a7d8, Comparison=CompareOpEq, exp_data=0x1, call_count=2)
  UVM_INFO @ 10045224334 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 175.entropy_src_functional_errors.74927501234736011301947341687770918216192708681257421515925713079326579216590
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/175.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10096103510 ps: (csr_utils_pkg.sv:597) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.sfifo_observe_err (addr=0x374774d8, Comparison=CompareOpEq, exp_data=0x1, call_count=2)
  UVM_INFO @ 10096103510 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 6 more failures.
Exit reason: Error: User command failed UVM_FATAL (csr_utils_pkg.sv:597) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_state_err (addr=*, Comparison=CompareOpEq, exp_data=*, call_count=2) has 6 failures:
- Test entropy_src_functional_errors has 6 failures.
  - 10.entropy_src_functional_errors.35596860214994024365154761362132352022657649230183493329756484049420156585887
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/10.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10071319530 ps: (csr_utils_pkg.sv:597) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_state_err (addr=0xea2c93d8, Comparison=CompareOpEq, exp_data=0x1, call_count=2)
  UVM_INFO @ 10071319530 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 483.entropy_src_functional_errors.57392622390946813599604631408907985709708560428631224439242886476244273877220
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/483.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10018985481 ps: (csr_utils_pkg.sv:597) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_state_err (addr=0x9b4104d8, Comparison=CompareOpEq, exp_data=0x1, call_count=2)
  UVM_INFO @ 10018985481 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 4 more failures.
UVM_ERROR (cip_base_scoreboard.sv:302) scoreboard [scoreboard] alert recov_alert did not trigger max_delay:* has 4 failures:
- Test entropy_src_rng has 1 failures.
  - 123.entropy_src_rng.41442822311237227701849450746234097266519643356547952920132380147146617614390
    Line 1697, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/123.entropy_src_rng/latest/run.log
```
  UVM_ERROR @ 9160925884 ps: (cip_base_scoreboard.sv:302) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_alert did not trigger max_delay:5
  UVM_INFO @ 9160925884 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test entropy_src_rng_max_rate has 3 failures.
  - 138.entropy_src_rng_max_rate.97425723696404742487561852253717497282535733549102726281562496341020427994019
    Line 1404, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/138.entropy_src_rng_max_rate/latest/run.log
```
  UVM_ERROR @ 3435057377 ps: (cip_base_scoreboard.sv:302) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_alert did not trigger max_delay:5
  UVM_INFO @ 3435057377 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 298.entropy_src_rng_max_rate.89046311004021263506048292216176610905512224024635026062810525274237257176050
    Line 2130, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/298.entropy_src_rng_max_rate/latest/run.log
```
  UVM_ERROR @ 7529383120 ps: (cip_base_scoreboard.sv:302) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_alert did not trigger max_delay:5
  UVM_INFO @ 7529383120 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 1 more failures.
UVM_ERROR (cip_base_scoreboard.sv:242) [scoreboard] Check failed expected_alert[alert_name].expected == * (* [*] vs * [*]) alert fatal_alert triggered unexpectedly has 3 failures:
- Test entropy_src_fw_ov has 3 failures.
  - 71.entropy_src_fw_ov.28530333905931507456786811935983268869366299016188383468055244748609820790858
    Line 583, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/71.entropy_src_fw_ov/latest/run.log
```
  UVM_ERROR @ 3279777382 ps: (cip_base_scoreboard.sv:242) [uvm_test_top.env.scoreboard] Check failed expected_alert[alert_name].expected == 1 (0 [0x0] vs 1 [0x1]) alert fatal_alert triggered unexpectedly
  UVM_INFO @ 3279777382 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 216.entropy_src_fw_ov.84865456182696141939450543738128132434502361577758479521860497605822232236170
    Line 644, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/216.entropy_src_fw_ov/latest/run.log
```
  UVM_ERROR @ 2694691496 ps: (cip_base_scoreboard.sv:242) [uvm_test_top.env.scoreboard] Check failed expected_alert[alert_name].expected == 1 (0 [0x0] vs 1 [0x1]) alert fatal_alert triggered unexpectedly
  UVM_INFO @ 2694691496 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 1 more failures.
UVM_ERROR (entropy_src_scoreboard.sv:2031) [scoreboard] Check failed csr.get_mirrored_value() == item.d_data (* [*] vs * [*]) reg name: entropy_src_reg_block.alert_summary_fail_counts has 2 failures:
- Test entropy_src_rng_max_rate has 2 failures.
  - 135.entropy_src_rng_max_rate.1674443308692394744816818601361193262145522812763524147909702731401177606823
    Line 2161, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/135.entropy_src_rng_max_rate/latest/run.log
```
  UVM_ERROR @ 9854837171 ps: (entropy_src_scoreboard.sv:2031) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (5 [0x5] vs 4 [0x4]) reg name: entropy_src_reg_block.alert_summary_fail_counts
  UVM_INFO @ 9854837171 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 277.entropy_src_rng_max_rate.39348506621877964838628781977177608541857313849245580106798432544009047176215
    Line 893, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/277.entropy_src_rng_max_rate/latest/run.log
```
  UVM_ERROR @ 791819031 ps: (entropy_src_scoreboard.sv:2031) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (2 [0x2] vs 1 [0x1]) reg name: entropy_src_reg_block.alert_summary_fail_counts
  UVM_INFO @ 791819031 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (entropy_src_scoreboard.sv:2031) [scoreboard] Check failed csr.get_mirrored_value() == item.d_data (* [*] vs * [*]) reg name: entropy_src_reg_block.recov_alert_sts has 1 failures:
- Test entropy_src_rng has 1 failures.
  - 25.entropy_src_rng.106972216380255003451081501823799156512779246565489177239431619699707777025064
    Line 1381, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/25.entropy_src_rng/latest/run.log
```
  UVM_ERROR @ 6885337615 ps: (entropy_src_scoreboard.sv:2031) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (4096 [0x1000] vs 0 [0x0]) reg name: entropy_src_reg_block.recov_alert_sts
  UVM_INFO @ 6885337615 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
Exit reason: Error: User command failed UVM_ERROR (cip_base_scoreboard.sv:242) [scoreboard] Check failed expected_alert[alert_name].expected == * (* [*] vs * [*]) alert fatal_alert triggered unexpectedly has 1 failures:
- Test entropy_src_fw_ov has 1 failures.
  - 117.entropy_src_fw_ov.43747707854999922881816056258352000164199070094867284199791770494413037338528
    Line 484, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/117.entropy_src_fw_ov/latest/run.log
```
  UVM_ERROR @ 2218895464 ps: (cip_base_scoreboard.sv:242) [uvm_test_top.env.scoreboard] Check failed expected_alert[alert_name].expected == 1 (0 [0x0] vs 1 [0x1]) alert fatal_alert triggered unexpectedly
  UVM_INFO @ 2218895464 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
Exit reason: Error: User command failed UVM_ERROR (cip_base_scoreboard.sv:302) scoreboard [scoreboard] alert recov_alert did not trigger max_delay:* has 1 failures:
- Test entropy_src_rng_max_rate has 1 failures.
  - 306.entropy_src_rng_max_rate.87624780422332018992202839375142326721729931338138470480669841698970245301379
    Line 481, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/306.entropy_src_rng_max_rate/latest/run.log
```
  UVM_ERROR @ 625956459 ps: (cip_base_scoreboard.sv:302) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_alert did not trigger max_delay:5
  UVM_INFO @ 625956459 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
Exit reason: Error: User command failed UVM_FATAL (csr_utils_pkg.sv:597) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.sfifo_observe_err (addr=*, Comparison=CompareOpEq, exp_data=*, call_count=2) has 1 failures:
- Test entropy_src_functional_errors has 1 failures.
  - 582.entropy_src_functional_errors.28664518316960229690777975895758469526836408396991349554172935294792713926651
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/582.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10017640944 ps: (csr_utils_pkg.sv:597) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.sfifo_observe_err (addr=0x49fdd5d8, Comparison=CompareOpEq, exp_data=0x1, call_count=2)
  UVM_INFO @ 10017640944 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```