ENTROPY_SRC Simulation Results

Saturday August 10 2024 23:02:23 UTC

GitHub Revision: `07b417ef03`

Branch: os_regression

Simulator: XCELIUM

Build randomization enabled with --build-seed 2196818177928134427831197337249851347498377272679561983541244979366753055772

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	entropy_src_smoke	4.000s	95.424us	50	50	100.00
V1	csr_hw_reset	entropy_src_csr_hw_reset	3.000s	145.115us	5	5	100.00
V1	csr_rw	entropy_src_csr_rw	4.000s	22.555us	20	20	100.00
V1	csr_bit_bash	entropy_src_csr_bit_bash	17.000s	801.635us	5	5	100.00
V1	csr_aliasing	entropy_src_csr_aliasing	6.000s	145.481us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	entropy_src_csr_mem_rw_with_rand_reset	4.000s	32.589us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	entropy_src_csr_rw	4.000s	22.555us	20	20	100.00
		entropy_src_csr_aliasing	6.000s	145.481us	5	5	100.00
V1		TOTAL			105	105	100.00
V2	firmware	entropy_src_smoke	4.000s	95.424us	50	50	100.00
		entropy_src_rng	10.483m	20.073ms	300	300	100.00
		entropy_src_fw_ov	10.033m	20.056ms	254	300	84.67
V2	firmware_mode	entropy_src_fw_ov	10.033m	20.056ms	254	300	84.67
V2	rng_mode	entropy_src_rng	10.483m	20.073ms	300	300	100.00
V2	rng_max_rate	entropy_src_rng_max_rate	18.700m	20.056ms	396	400	99.00
V2	health_checks	entropy_src_rng	10.483m	20.073ms	300	300	100.00
V2	conditioning	entropy_src_rng	10.483m	20.073ms	300	300	100.00
V2	interrupts	entropy_src_rng	10.483m	20.073ms	300	300	100.00
		entropy_src_intr	24.000s	3.971ms	50	50	100.00
V2	alerts	entropy_src_rng	10.483m	20.073ms	300	300	100.00
		entropy_src_functional_alerts	5.000s	59.617us	50	50	100.00
V2	stress_all	entropy_src_stress_all	11.033m	19.274ms	50	50	100.00
V2	functional_errors	entropy_src_functional_errors	9.850m	10.013ms	973	1000	97.30
V2	firmware_ov_read_contiguous_data	entropy_src_fw_ov_contiguous	23.000s	1.715ms	50	50	100.00
V2	intr_test	entropy_src_intr_test	4.000s	21.265us	50	50	100.00
V2	alert_test	entropy_src_alert_test	4.000s	24.864us	50	50	100.00
V2	tl_d_oob_addr_access	entropy_src_tl_errors	8.000s	163.680us	20	20	100.00
V2	tl_d_illegal_access	entropy_src_tl_errors	8.000s	163.680us	20	20	100.00
V2	tl_d_outstanding_access	entropy_src_csr_hw_reset	3.000s	145.115us	5	5	100.00
		entropy_src_csr_rw	4.000s	22.555us	20	20	100.00
		entropy_src_csr_aliasing	6.000s	145.481us	5	5	100.00
		entropy_src_same_csr_outstanding	4.000s	57.056us	20	20	100.00
V2	tl_d_partial_access	entropy_src_csr_hw_reset	3.000s	145.115us	5	5	100.00
		entropy_src_csr_rw	4.000s	22.555us	20	20	100.00
		entropy_src_csr_aliasing	6.000s	145.481us	5	5	100.00
		entropy_src_same_csr_outstanding	4.000s	57.056us	20	20	100.00
V2		TOTAL			2263	2340	96.71
V2S	tl_intg_err	entropy_src_sec_cm	4.000s	100.295us	5	5	100.00
		entropy_src_tl_intg_err	7.000s	376.952us	20	20	100.00
V2S	sec_cm_config_regwen	entropy_src_rng	10.483m	20.073ms	300	300	100.00
		entropy_src_cfg_regwen	3.000s	20.228us	50	50	100.00
V2S	sec_cm_config_mubi	entropy_src_rng	10.483m	20.073ms	300	300	100.00
V2S	sec_cm_config_redun	entropy_src_rng	10.483m	20.073ms	300	300	100.00
V2S	sec_cm_intersig_mubi	entropy_src_rng	10.483m	20.073ms	300	300	100.00
		entropy_src_fw_ov	10.033m	20.056ms	254	300	84.67
V2S	sec_cm_main_sm_fsm_sparse	entropy_src_functional_errors	9.850m	10.013ms	973	1000	97.30
		entropy_src_sec_cm	4.000s	100.295us	5	5	100.00
V2S	sec_cm_ack_sm_fsm_sparse	entropy_src_functional_errors	9.850m	10.013ms	973	1000	97.30
		entropy_src_sec_cm	4.000s	100.295us	5	5	100.00
V2S	sec_cm_rng_bkgn_chk	entropy_src_rng	10.483m	20.073ms	300	300	100.00
V2S	sec_cm_fifo_ctr_redun	entropy_src_functional_errors	9.850m	10.013ms	973	1000	97.30
		entropy_src_sec_cm	4.000s	100.295us	5	5	100.00
V2S	sec_cm_ctr_redun	entropy_src_functional_errors	9.850m	10.013ms	973	1000	97.30
		entropy_src_sec_cm	4.000s	100.295us	5	5	100.00
V2S	sec_cm_ctr_local_esc	entropy_src_functional_errors	9.850m	10.013ms	973	1000	97.30
V2S	sec_cm_esfinal_rdata_bus_consistency	entropy_src_functional_alerts	5.000s	59.617us	50	50	100.00
V2S	sec_cm_tile_link_bus_integrity	entropy_src_tl_intg_err	7.000s	376.952us	20	20	100.00
V2S		TOTAL			75	75	100.00
V3	external_health_tests	entropy_src_rng_with_xht_rsps	10.067m	20.051ms	49	50	98.00
V3	stress_all_with_rand_reset	entropy_src_stress_all_with_rand_reset			0	0	--
V3		TOTAL			49	50	98.00
		TOTAL			2492	2570	96.96

Testplan Progress

Items	Total	Written	Passing	Progress
V1	6	6	6	100.00
V2	12	12	9	75.00
V2S	3	3	3	100.00
V3	2	1	0	0.00

Coverage Results

Coverage Dashboard

Score	Block	Branch	Statement	Expression	Toggle	Fsm	Assertion	CoverGroup
97.17	98.17	95.38	98.36	95.84	96.62	96.88	90.49	96.12

Failure Buckets

UVM_ERROR (cip_base_scoreboard.sv:243) [scoreboard] Check failed expected_alert[alert_name].expected == * (* [*] vs * [*]) alert fatal_alert triggered unexpectedly has 20 failures:
- Test entropy_src_fw_ov has 20 failures.
  - 9.entropy_src_fw_ov.31063773239092772179647506763145897986522374423770454739830565108273299069826
    Line 2305, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/9.entropy_src_fw_ov/latest/run.log
```
  UVM_ERROR @ 9690366234 ps: (cip_base_scoreboard.sv:243) [uvm_test_top.env.scoreboard] Check failed expected_alert[alert_name].expected == 1 (0 [0x0] vs 1 [0x1]) alert fatal_alert triggered unexpectedly
  UVM_INFO @ 9690366234 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 16.entropy_src_fw_ov.26461652823500591475270617134071495090648265952075734137825167466398277358453
    Line 2423, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/16.entropy_src_fw_ov/latest/run.log
```
  UVM_ERROR @ 14942114407 ps: (cip_base_scoreboard.sv:243) [uvm_test_top.env.scoreboard] Check failed expected_alert[alert_name].expected == 1 (0 [0x0] vs 1 [0x1]) alert fatal_alert triggered unexpectedly
  UVM_INFO @ 14942114407 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 18 more failures.

UVM_ERROR (entropy_src_scoreboard.sv:2035) [scoreboard] Check failed csr.get_mirrored_value() == item.d_data (* [*] vs * [*]) reg name: entropy_src_reg_block.err_code has 19 failures:

Test entropy_src_fw_ov has 19 failures.

49.entropy_src_fw_ov.7677543908538288802338720619162922722975938504934919518066273828592257932871
Line 1108, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/49.entropy_src_fw_ov/latest/run.log

  UVM_ERROR @ 7534224119 ps: (entropy_src_scoreboard.sv:2035) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (1048576 [0x100000] vs 3145728 [0x300000]) reg name: entropy_src_reg_block.err_code
  UVM_INFO @ 7534224119 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

61.entropy_src_fw_ov.93524038145472758075278663488703134637883018247347290208466661721702146806373
Line 481, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/61.entropy_src_fw_ov/latest/run.log

  UVM_ERROR @ 3363151139 ps: (entropy_src_scoreboard.sv:2035) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (1048576 [0x100000] vs 3145728 [0x300000]) reg name: entropy_src_reg_block.err_code
  UVM_INFO @ 3363151139 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 17 more failures.

UVM_FATAL (csr_utils_pkg.sv:587) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_state_err (addr=*, Comparison=CompareOpEq, exp_data=*, call_count=2) has 9 failures:
- Test entropy_src_functional_errors has 9 failures.
  - 5.entropy_src_functional_errors.7661447855124833339661459848211178096280650816013827340509850121993765288273
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/5.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10059226873 ps: (csr_utils_pkg.sv:587) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_state_err (addr=0x1dce3ad8, Comparison=CompareOpEq, exp_data=0x1, call_count=2)
  UVM_INFO @ 10059226873 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 158.entropy_src_functional_errors.82259765085218296408643010513376487042927793626105750516646852809909859798693
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/158.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10015437063 ps: (csr_utils_pkg.sv:587) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_state_err (addr=0x409be5d8, Comparison=CompareOpEq, exp_data=0x1, call_count=2)
  UVM_INFO @ 10015437063 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 7 more failures.
UVM_FATAL (csr_utils_pkg.sv:587) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_read_err (addr=*, Comparison=CompareOpEq, exp_data=*, call_count=2) has 9 failures:
- Test entropy_src_functional_errors has 9 failures.
  - 18.entropy_src_functional_errors.38121795514301225176053285592391609577199514329640206669919432134378107080758
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/18.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10046455081 ps: (csr_utils_pkg.sv:587) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_read_err (addr=0x1776f4d8, Comparison=CompareOpEq, exp_data=0x1, call_count=2)
  UVM_INFO @ 10046455081 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 109.entropy_src_functional_errors.34199292627186859766495403950870687671051697947916187868815431582831330754923
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/109.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10085559068 ps: (csr_utils_pkg.sv:587) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_read_err (addr=0xac0a0dd8, Comparison=CompareOpEq, exp_data=0x1, call_count=2)
  UVM_INFO @ 10085559068 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 7 more failures.
UVM_FATAL (csr_utils_pkg.sv:587) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.sfifo_observe_err (addr=*, Comparison=CompareOpEq, exp_data=*, call_count=2) has 4 failures:
- Test entropy_src_functional_errors has 4 failures.
  - 56.entropy_src_functional_errors.89518939180466034691991121235761978289299624654474233431493132213399710084670
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/56.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10068376911 ps: (csr_utils_pkg.sv:587) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.sfifo_observe_err (addr=0x1c7aecd8, Comparison=CompareOpEq, exp_data=0x1, call_count=2)
  UVM_INFO @ 10068376911 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 270.entropy_src_functional_errors.114451182460696245638385793430964678367922097422247101083216030518799426719777
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/270.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10060920842 ps: (csr_utils_pkg.sv:587) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.sfifo_observe_err (addr=0x8e54dad8, Comparison=CompareOpEq, exp_data=0x1, call_count=2)
  UVM_INFO @ 10060920842 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 2 more failures.
Exit reason: Error: User command failed UVM_FATAL (csr_utils_pkg.sv:587) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_read_err (addr=*, Comparison=CompareOpEq, exp_data=*, call_count=2) has 4 failures:
- Test entropy_src_functional_errors has 4 failures.
  - 197.entropy_src_functional_errors.88059777382447885069106508514854244406705217649893546869045822747963807454197
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/197.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10013407090 ps: (csr_utils_pkg.sv:587) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_read_err (addr=0x32a45d8, Comparison=CompareOpEq, exp_data=0x1, call_count=2)
  UVM_INFO @ 10013407090 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 372.entropy_src_functional_errors.39446025663337197363586024592548748211023652107922488971853576872370030523230
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/372.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10017820185 ps: (csr_utils_pkg.sv:587) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.fifo_read_err (addr=0x1e335ed8, Comparison=CompareOpEq, exp_data=0x1, call_count=2)
  UVM_INFO @ 10017820185 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 2 more failures.
Exit reason: Error: User command failed UVM_ERROR (cip_base_scoreboard.sv:243) [scoreboard] Check failed expected_alert[alert_name].expected == * (* [*] vs * [*]) alert fatal_alert triggered unexpectedly has 3 failures:
- Test entropy_src_fw_ov has 3 failures.
  - 65.entropy_src_fw_ov.21000794274415698684687189859188903549889093047351277451024974986347561684221
    Line 1507, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/65.entropy_src_fw_ov/latest/run.log
```
  UVM_ERROR @ 15613223094 ps: (cip_base_scoreboard.sv:243) [uvm_test_top.env.scoreboard] Check failed expected_alert[alert_name].expected == 1 (0 [0x0] vs 1 [0x1]) alert fatal_alert triggered unexpectedly
  UVM_INFO @ 15613223094 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 256.entropy_src_fw_ov.45905067745761252629475837610299173274937043294507888652476295569237663563984
    Line 1802, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/256.entropy_src_fw_ov/latest/run.log
```
  UVM_ERROR @ 14258923338 ps: (cip_base_scoreboard.sv:243) [uvm_test_top.env.scoreboard] Check failed expected_alert[alert_name].expected == 1 (0 [0x0] vs 1 [0x1]) alert fatal_alert triggered unexpectedly
  UVM_INFO @ 14258923338 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 1 more failures.
UVM_ERROR (entropy_src_scoreboard.sv:2035) [scoreboard] Check failed csr.get_mirrored_value() == item.d_data (* [*] vs * [*]) reg name: entropy_src_reg_block.recov_alert_sts has 2 failures:
- Test entropy_src_rng_with_xht_rsps has 1 failures.
  - 11.entropy_src_rng_with_xht_rsps.108093324136848756118372467582444379692195156468676500792888100181137323171078
    Line 1381, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/11.entropy_src_rng_with_xht_rsps/latest/run.log
```
  UVM_ERROR @ 4918214635 ps: (entropy_src_scoreboard.sv:2035) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (4096 [0x1000] vs 0 [0x0]) reg name: entropy_src_reg_block.recov_alert_sts
  UVM_INFO @ 4918214635 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test entropy_src_rng_max_rate has 1 failures.
  - 31.entropy_src_rng_max_rate.7420309239465226105561728991241070436995500178383512557669216050151068586019
    Line 2007, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/31.entropy_src_rng_max_rate/latest/run.log
```
  UVM_ERROR @ 5174029192 ps: (entropy_src_scoreboard.sv:2035) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (4096 [0x1000] vs 0 [0x0]) reg name: entropy_src_reg_block.recov_alert_sts
  UVM_INFO @ 5174029192 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```

Exit reason: Error: User command failed xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_entropy_src_*/rtl/entropy_src_core.sv,3405): Assertion AtReset_EsfinalFifoPushedPostStartup_A has failed has 2 failures:

Test entropy_src_rng_max_rate has 2 failures.

17.entropy_src_rng_max_rate.18697816662178423769809311866077114796916152714892388371748985811588705576997
Line 2379, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/17.entropy_src_rng_max_rate/latest/run.log

  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_entropy_src_0.1/rtl/entropy_src_core.sv,3405): (time 13376069280 PS) Assertion tb.dut.u_entropy_src_core.AtReset_EsfinalFifoPushedPostStartup_A has failed
  UVM_ERROR @ 13376069280 ps: (entropy_src_core.sv:3405) [ASSERT FAILED] AtReset_EsfinalFifoPushedPostStartup_A
  UVM_INFO @ 13376069280 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

327.entropy_src_rng_max_rate.67844032633088719965138490788282791679009517260592949779935445885632221773607
Line 2059, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/327.entropy_src_rng_max_rate/latest/run.log

  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_entropy_src_0.1/rtl/entropy_src_core.sv,3405): (time 11535522072 PS) Assertion tb.dut.u_entropy_src_core.AtReset_EsfinalFifoPushedPostStartup_A has failed
  UVM_ERROR @ 11535522072 ps: (entropy_src_core.sv:3405) [ASSERT FAILED] AtReset_EsfinalFifoPushedPostStartup_A
  UVM_INFO @ 11535522072 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Exit reason: Error: User command failed UVM_FATAL (entropy_src_scoreboard.sv:2653) [scoreboard] Check failed match_found == * (* [*] vs * [*]) All candidate csrng seeds have been checked, with no match has 2 failures:
- Test entropy_src_fw_ov has 2 failures.
  - 34.entropy_src_fw_ov.59833423497524321049167048088254211971313481710441591793043724671065674199297
    Line 2567, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/34.entropy_src_fw_ov/latest/run.log
```
  UVM_FATAL @ 19974538980 ps: (entropy_src_scoreboard.sv:2653) [uvm_test_top.env.scoreboard] Check failed match_found == 1 (0 [0x0] vs 1 [0x1]) All candidate csrng seeds have been checked, with no match
  UVM_INFO @ 19974538980 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 81.entropy_src_fw_ov.30825073202935674797144908638591400082649231028888081634234780610017617021897
    Line 1474, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/81.entropy_src_fw_ov/latest/run.log
```
  UVM_FATAL @ 4027133162 ps: (entropy_src_scoreboard.sv:2653) [uvm_test_top.env.scoreboard] Check failed match_found == 1 (0 [0x0] vs 1 [0x1]) All candidate csrng seeds have been checked, with no match
  UVM_INFO @ 4027133162 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_FATAL (entropy_src_scoreboard.sv:2653) [scoreboard] Check failed match_found == * (* [*] vs * [*]) All candidate csrng seeds have been checked, with no match has 2 failures:
- Test entropy_src_fw_ov has 2 failures.
  - 98.entropy_src_fw_ov.102025632565533758544746651444618194171613111154731478219523780517310027709239
    Line 1214, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/98.entropy_src_fw_ov/latest/run.log
```
  UVM_FATAL @ 9538096849 ps: (entropy_src_scoreboard.sv:2653) [uvm_test_top.env.scoreboard] Check failed match_found == 1 (0 [0x0] vs 1 [0x1]) All candidate csrng seeds have been checked, with no match
  UVM_INFO @ 9538096849 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 172.entropy_src_fw_ov.113876652533225793516933468647194722161949376258515745275227263930381725273155
    Line 3124, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/172.entropy_src_fw_ov/latest/run.log
```
  UVM_FATAL @ 15096625067 ps: (entropy_src_scoreboard.sv:2653) [uvm_test_top.env.scoreboard] Check failed match_found == 1 (0 [0x0] vs 1 [0x1]) All candidate csrng seeds have been checked, with no match
  UVM_INFO @ 15096625067 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (cip_base_scoreboard.sv:305) scoreboard [scoreboard] alert recov_alert did not trigger max_delay:* has 1 failures:
- Test entropy_src_rng_max_rate has 1 failures.
  - 248.entropy_src_rng_max_rate.45993919957946071903684339086800898379008349692187661352651959074660332944559
    Line 1293, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/248.entropy_src_rng_max_rate/latest/run.log
```
  UVM_ERROR @ 2775030439 ps: (cip_base_scoreboard.sv:305) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_alert did not trigger max_delay:5
  UVM_INFO @ 2775030439 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
Exit reason: Error: User command failed UVM_FATAL (csr_utils_pkg.sv:587) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.sfifo_observe_err (addr=*, Comparison=CompareOpEq, exp_data=*, call_count=2) has 1 failures:
- Test entropy_src_functional_errors has 1 failures.
  - 402.entropy_src_functional_errors.83215830075179650489858556612787121005586597629463143501217411729355054617854
    Line 332, in log /container/opentitan-public/scratch/os_regression/entropy_src-sim-xcelium/402.entropy_src_functional_errors/latest/run.log
```
  UVM_FATAL @ 10015588080 ps: (csr_utils_pkg.sv:587) [csr_utils::csr_spinwait] timeout entropy_src_reg_block.err_code.sfifo_observe_err (addr=0x98c85d8, Comparison=CompareOpEq, exp_data=0x1, call_count=2)
  UVM_INFO @ 10015588080 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```