AES/UNMASKED Simulation Results

Monday August 26 2024 23:33:20 UTC

GitHub Revision: `4674f625b3`

Branch: os_regression_2024_08_26

Simulator: XCELIUM

Build randomization enabled with --build-seed 27137705585251537962012108482438895412147493342955425380690984800523869492310

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	wake_up	aes_wake_up	28.000s	76.167us	1	1	100.00
V1	smoke	aes_smoke	1.650m	78.806us	50	50	100.00
V1	csr_hw_reset	aes_csr_hw_reset	1.067m	120.845us	5	5	100.00
V1	csr_rw	aes_csr_rw	58.000s	127.207us	20	20	100.00
V1	csr_bit_bash	aes_csr_bit_bash	1.183m	2.302ms	5	5	100.00
V1	csr_aliasing	aes_csr_aliasing	30.000s	166.459us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	aes_csr_mem_rw_with_rand_reset	1.117m	61.389us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	aes_csr_rw	58.000s	127.207us	20	20	100.00
		aes_csr_aliasing	30.000s	166.459us	5	5	100.00
V1		TOTAL			106	106	100.00
V2	algorithm	aes_smoke	1.650m	78.806us	50	50	100.00
		aes_config_error	1.983m	56.306us	50	50	100.00
		aes_stress	1.467m	85.597us	50	50	100.00
V2	key_length	aes_smoke	1.650m	78.806us	50	50	100.00
		aes_config_error	1.983m	56.306us	50	50	100.00
		aes_stress	1.467m	85.597us	50	50	100.00
V2	back2back	aes_stress	1.467m	85.597us	50	50	100.00
		aes_b2b	2.517m	298.894us	50	50	100.00
V2	backpressure	aes_stress	1.467m	85.597us	50	50	100.00
V2	multi_message	aes_smoke	1.650m	78.806us	50	50	100.00
		aes_config_error	1.983m	56.306us	50	50	100.00
		aes_stress	1.467m	85.597us	50	50	100.00
		aes_alert_reset	1.967m	120.620us	50	50	100.00
V2	failure_test	aes_man_cfg_err	2.067m	67.122us	50	50	100.00
		aes_config_error	1.983m	56.306us	50	50	100.00
		aes_alert_reset	1.967m	120.620us	50	50	100.00
V2	trigger_clear_test	aes_clear	2.450m	91.517us	50	50	100.00
V2	nist_test_vectors	aes_nist_vectors	31.000s	522.153us	1	1	100.00
V2	reset_recovery	aes_alert_reset	1.967m	120.620us	50	50	100.00
V2	stress	aes_stress	1.467m	85.597us	50	50	100.00
V2	sideload	aes_stress	1.467m	85.597us	50	50	100.00
		aes_sideload	2.483m	63.644us	50	50	100.00
V2	deinitialization	aes_deinit	1.650m	110.042us	50	50	100.00
V2	stress_all	aes_stress_all	1.583m	1.229ms	10	10	100.00
V2	alert_test	aes_alert_test	2.300m	92.020us	50	50	100.00
V2	tl_d_oob_addr_access	aes_tl_errors	32.000s	299.262us	20	20	100.00
V2	tl_d_illegal_access	aes_tl_errors	32.000s	299.262us	20	20	100.00
V2	tl_d_outstanding_access	aes_csr_hw_reset	1.067m	120.845us	5	5	100.00
		aes_csr_rw	58.000s	127.207us	20	20	100.00
		aes_csr_aliasing	30.000s	166.459us	5	5	100.00
		aes_same_csr_outstanding	57.000s	783.414us	20	20	100.00
V2	tl_d_partial_access	aes_csr_hw_reset	1.067m	120.845us	5	5	100.00
		aes_csr_rw	58.000s	127.207us	20	20	100.00
		aes_csr_aliasing	30.000s	166.459us	5	5	100.00
		aes_same_csr_outstanding	57.000s	783.414us	20	20	100.00
V2		TOTAL			501	501	100.00
V2S	reseeding	aes_reseed	1.950m	399.559us	50	50	100.00
V2S	fault_inject	aes_fi	1.483m	243.682us	50	50	100.00
		aes_control_fi	1.033m	50.745us	260	300	86.67
		aes_cipher_fi	1.000m	79.666us	296	350	84.57
V2S	shadow_reg_update_error	aes_shadow_reg_errors	1.567m	80.059us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	aes_shadow_reg_errors	1.567m	80.059us	20	20	100.00
V2S	shadow_reg_storage_error	aes_shadow_reg_errors	1.567m	80.059us	20	20	100.00
V2S	shadowed_reset_glitch	aes_shadow_reg_errors	1.567m	80.059us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	aes_shadow_reg_errors_with_csr_rw	35.000s	584.594us	20	20	100.00
V2S	tl_intg_err	aes_sec_cm	1.650m	567.437us	5	5	100.00
		aes_tl_intg_err	1.083m	210.167us	20	20	100.00
V2S	sec_cm_bus_integrity	aes_tl_intg_err	1.083m	210.167us	20	20	100.00
V2S	sec_cm_lc_escalate_en_intersig_mubi	aes_alert_reset	1.967m	120.620us	50	50	100.00
V2S	sec_cm_main_config_shadow	aes_shadow_reg_errors	1.567m	80.059us	20	20	100.00
V2S	sec_cm_main_config_sparse	aes_smoke	1.650m	78.806us	50	50	100.00
		aes_stress	1.467m	85.597us	50	50	100.00
		aes_alert_reset	1.967m	120.620us	50	50	100.00
		aes_core_fi	2.267m	10.017ms	65	70	92.86
V2S	sec_cm_aux_config_shadow	aes_shadow_reg_errors	1.567m	80.059us	20	20	100.00
V2S	sec_cm_aux_config_regwen	aes_readability	1.500m	64.671us	50	50	100.00
		aes_stress	1.467m	85.597us	50	50	100.00
V2S	sec_cm_key_sideload	aes_stress	1.467m	85.597us	50	50	100.00
		aes_sideload	2.483m	63.644us	50	50	100.00
V2S	sec_cm_key_sw_unreadable	aes_readability	1.500m	64.671us	50	50	100.00
V2S	sec_cm_data_reg_sw_unreadable	aes_readability	1.500m	64.671us	50	50	100.00
V2S	sec_cm_key_sec_wipe	aes_readability	1.500m	64.671us	50	50	100.00
V2S	sec_cm_iv_config_sec_wipe	aes_readability	1.500m	64.671us	50	50	100.00
V2S	sec_cm_data_reg_sec_wipe	aes_readability	1.500m	64.671us	50	50	100.00
V2S	sec_cm_data_reg_key_sca	aes_stress	1.467m	85.597us	50	50	100.00
V2S	sec_cm_key_masking	aes_stress	1.467m	85.597us	50	50	100.00
V2S	sec_cm_main_fsm_sparse	aes_fi	1.483m	243.682us	50	50	100.00
V2S	sec_cm_main_fsm_redun	aes_fi	1.483m	243.682us	50	50	100.00
		aes_control_fi	1.033m	50.745us	260	300	86.67
		aes_cipher_fi	1.000m	79.666us	296	350	84.57
		aes_ctr_fi	59.000s	67.250us	45	50	90.00
V2S	sec_cm_cipher_fsm_sparse	aes_fi	1.483m	243.682us	50	50	100.00
V2S	sec_cm_cipher_fsm_redun	aes_fi	1.483m	243.682us	50	50	100.00
		aes_control_fi	1.033m	50.745us	260	300	86.67
		aes_cipher_fi	1.000m	79.666us	296	350	84.57
V2S	sec_cm_cipher_ctr_redun	aes_cipher_fi	1.000m	79.666us	296	350	84.57
V2S	sec_cm_ctr_fsm_sparse	aes_fi	1.483m	243.682us	50	50	100.00
V2S	sec_cm_ctr_fsm_redun	aes_fi	1.483m	243.682us	50	50	100.00
		aes_control_fi	1.033m	50.745us	260	300	86.67
		aes_ctr_fi	59.000s	67.250us	45	50	90.00
V2S	sec_cm_ctrl_sparse	aes_fi	1.483m	243.682us	50	50	100.00
		aes_control_fi	1.033m	50.745us	260	300	86.67
		aes_cipher_fi	1.000m	79.666us	296	350	84.57
		aes_ctr_fi	59.000s	67.250us	45	50	90.00
V2S	sec_cm_main_fsm_global_esc	aes_alert_reset	1.967m	120.620us	50	50	100.00
V2S	sec_cm_main_fsm_local_esc	aes_fi	1.483m	243.682us	50	50	100.00
		aes_control_fi	1.033m	50.745us	260	300	86.67
		aes_cipher_fi	1.000m	79.666us	296	350	84.57
		aes_ctr_fi	59.000s	67.250us	45	50	90.00
V2S	sec_cm_cipher_fsm_local_esc	aes_fi	1.483m	243.682us	50	50	100.00
		aes_control_fi	1.033m	50.745us	260	300	86.67
		aes_cipher_fi	1.000m	79.666us	296	350	84.57
		aes_ctr_fi	59.000s	67.250us	45	50	90.00
V2S	sec_cm_ctr_fsm_local_esc	aes_fi	1.483m	243.682us	50	50	100.00
		aes_control_fi	1.033m	50.745us	260	300	86.67
		aes_ctr_fi	59.000s	67.250us	45	50	90.00
V2S	sec_cm_data_reg_local_esc	aes_fi	1.483m	243.682us	50	50	100.00
		aes_control_fi	1.033m	50.745us	260	300	86.67
		aes_cipher_fi	1.000m	79.666us	296	350	84.57
V2S		TOTAL			881	985	89.44
V3	stress_all_with_rand_reset	aes_stress_all_with_rand_reset	2.083m	971.629us	0	10	0.00
V3		TOTAL			0	10	0.00
		TOTAL			1488	1602	92.88

Testplan Progress

Items	Total	Written	Passing	Progress
V1	7	7	7	100.00
V2	13	13	13	100.00
V2S	11	11	7	63.64
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

Score	Block	Branch	Statement	Expression	Toggle	Fsm	Assertion	CoverGroup
97.17	97.43	94.18	98.81	93.57	97.72	93.33	98.85	96.81

Failure Buckets

Job timed out after * minutes has 85 failures:
- Test aes_control_fi has 36 failures.
  - 4.aes_control_fi.52932655556030270874451405018860683801181280247223927543705799696105726088688
    Log /workspaces/repo/scratch/os_regression_2024_08_26/aes_unmasked-sim-xcelium/4.aes_control_fi/latest/run.log
```
  Job timed out after 1 minutes
```
  - 7.aes_control_fi.4582914830421643895369807563553363289014702630772397488990560083238853644320
    Log /workspaces/repo/scratch/os_regression_2024_08_26/aes_unmasked-sim-xcelium/7.aes_control_fi/latest/run.log
```
  Job timed out after 1 minutes
```
  - ... and 34 more failures.
- Test aes_cipher_fi has 44 failures.
  - 4.aes_cipher_fi.78199934747584251115624641156526096614966457026484892041575323016485154718554
    Log /workspaces/repo/scratch/os_regression_2024_08_26/aes_unmasked-sim-xcelium/4.aes_cipher_fi/latest/run.log
```
  Job timed out after 1 minutes
```
  - 12.aes_cipher_fi.11500173771474724656356420582016959055607086693747841232326957931420969084483
    Log /workspaces/repo/scratch/os_regression_2024_08_26/aes_unmasked-sim-xcelium/12.aes_cipher_fi/latest/run.log
```
  Job timed out after 1 minutes
```
  - ... and 42 more failures.
- Test aes_ctr_fi has 5 failures.
  - 4.aes_ctr_fi.48038779677726139479626951896125721546307498719300862181778569571638051552146
    Log /workspaces/repo/scratch/os_regression_2024_08_26/aes_unmasked-sim-xcelium/4.aes_ctr_fi/latest/run.log
```
  Job timed out after 1 minutes
```
  - 14.aes_ctr_fi.86616809977394687150636021171486912715414547811491047163363002978614913337460
    Log /workspaces/repo/scratch/os_regression_2024_08_26/aes_unmasked-sim-xcelium/14.aes_ctr_fi/latest/run.log
```
  Job timed out after 1 minutes
```
  - ... and 3 more failures.
UVM_FATAL (aes_cipher_fi_vseq.sv:62) [aes_cipher_fi_vseq] wait timeout occurred! has 10 failures:
- Test aes_cipher_fi has 10 failures.
  - 21.aes_cipher_fi.94154850374372175307706689785055016314593273405337493915274582712870629217623
    Line 137, in log /workspaces/repo/scratch/os_regression_2024_08_26/aes_unmasked-sim-xcelium/21.aes_cipher_fi/latest/run.log
```
  UVM_FATAL @ 10020659177 ps: (aes_cipher_fi_vseq.sv:62) [uvm_test_top.env.virtual_sequencer.aes_cipher_fi_vseq] wait timeout occurred!
  UVM_INFO @ 10020659177 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 106.aes_cipher_fi.68384402247463293350786689245704097133599949984350214874969936080583470896464
    Line 137, in log /workspaces/repo/scratch/os_regression_2024_08_26/aes_unmasked-sim-xcelium/106.aes_cipher_fi/latest/run.log
```
  UVM_FATAL @ 10004893019 ps: (aes_cipher_fi_vseq.sv:62) [uvm_test_top.env.virtual_sequencer.aes_cipher_fi_vseq] wait timeout occurred!
  UVM_INFO @ 10004893019 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 8 more failures.

UVM_ERROR (uvm_sequencer_base.svh:757) sequencer [SEQREQZMB] The task responsible for requesting a wait_for_grant on sequencer 'sequencer' for sequence 'sideload_seq' has been killed, to avoid a deadlock the sequence will be removed from the arbitration queues has 5 failures:

Test aes_stress_all_with_rand_reset has 5 failures.

0.aes_stress_all_with_rand_reset.11082219542251336569733698300874128222700599522193778064559953007667881230704
Line 1501, in log /workspaces/repo/scratch/os_regression_2024_08_26/aes_unmasked-sim-xcelium/0.aes_stress_all_with_rand_reset/latest/run.log

  UVM_ERROR @ 1003881102 ps: (uvm_sequencer_base.svh:757) uvm_test_top.env.keymgr_sideload_agent.sequencer [SEQREQZMB] The task responsible for requesting a wait_for_grant on sequencer 'uvm_test_top.env.keymgr_sideload_agent.sequencer' for sequence 'uvm_test_top.env.virtual_sequencer.aes_stress_vseq.sideload_seq' has been killed, to avoid a deadlock the sequence will be removed from the arbitration queues
  UVM_INFO @ 1003881102 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

4.aes_stress_all_with_rand_reset.58569940056729758567705091728731058382344906968754998619739326426730723379507
Line 403, in log /workspaces/repo/scratch/os_regression_2024_08_26/aes_unmasked-sim-xcelium/4.aes_stress_all_with_rand_reset/latest/run.log

  UVM_ERROR @ 3921873388 ps: (uvm_sequencer_base.svh:757) uvm_test_top.env.keymgr_sideload_agent.sequencer [SEQREQZMB] The task responsible for requesting a wait_for_grant on sequencer 'uvm_test_top.env.keymgr_sideload_agent.sequencer' for sequence 'uvm_test_top.env.virtual_sequencer.aes_reseed_vseq.sideload_seq' has been killed, to avoid a deadlock the sequence will be removed from the arbitration queues
  UVM_INFO @ 3921873388 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 3 more failures.

UVM_FATAL (aes_control_fi_vseq.sv:62) [aes_control_fi_vseq] wait timeout occurred! has 4 failures:
- Test aes_control_fi has 4 failures.
  - 34.aes_control_fi.106805182936655031644178752353513202448798765653665597685726139576684648722286
    Line 130, in log /workspaces/repo/scratch/os_regression_2024_08_26/aes_unmasked-sim-xcelium/34.aes_control_fi/latest/run.log
```
  UVM_FATAL @ 10005976682 ps: (aes_control_fi_vseq.sv:62) [uvm_test_top.env.virtual_sequencer.aes_control_fi_vseq] wait timeout occurred!
  UVM_INFO @ 10005976682 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 69.aes_control_fi.95249779391425277815353049082992328947469444894376401631526918352845544290980
    Line 133, in log /workspaces/repo/scratch/os_regression_2024_08_26/aes_unmasked-sim-xcelium/69.aes_control_fi/latest/run.log
```
  UVM_FATAL @ 10009622912 ps: (aes_control_fi_vseq.sv:62) [uvm_test_top.env.virtual_sequencer.aes_control_fi_vseq] wait timeout occurred!
  UVM_INFO @ 10009622912 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 2 more failures.
UVM_FATAL (aes_core_fi_vseq.sv:66) [aes_core_fi_vseq] wait timeout occurred! has 3 failures:
- Test aes_core_fi has 3 failures.
  - 32.aes_core_fi.66035452669728666883670395126885388301389905183334796531171753510217903261427
    Line 132, in log /workspaces/repo/scratch/os_regression_2024_08_26/aes_unmasked-sim-xcelium/32.aes_core_fi/latest/run.log
```
  UVM_FATAL @ 10003450709 ps: (aes_core_fi_vseq.sv:66) [uvm_test_top.env.virtual_sequencer.aes_core_fi_vseq] wait timeout occurred!
  UVM_INFO @ 10003450709 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 35.aes_core_fi.56166791287803547228030386898509991573094322188190953016938497433871626832079
    Line 136, in log /workspaces/repo/scratch/os_regression_2024_08_26/aes_unmasked-sim-xcelium/35.aes_core_fi/latest/run.log
```
  UVM_FATAL @ 10002805067 ps: (aes_core_fi_vseq.sv:66) [uvm_test_top.env.virtual_sequencer.aes_core_fi_vseq] wait timeout occurred!
  UVM_INFO @ 10002805067 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 1 more failures.
UVM_FATAL (aes_base_vseq.sv:74) [aes_alert_reset_vseq] Check failed (aes_ctrl_aux[*] == cfg.do_reseed) has 2 failures:
- Test aes_stress_all_with_rand_reset has 2 failures.
  - 1.aes_stress_all_with_rand_reset.100693763367154043098505763234499304993990362852351295827352378953578111659361
    Line 868, in log /workspaces/repo/scratch/os_regression_2024_08_26/aes_unmasked-sim-xcelium/1.aes_stress_all_with_rand_reset/latest/run.log
```
  UVM_FATAL @ 1069968894 ps: (aes_base_vseq.sv:74) [uvm_test_top.env.virtual_sequencer.aes_alert_reset_vseq] Check failed (aes_ctrl_aux[0] == cfg.do_reseed)
  UVM_INFO @ 1069968894 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 9.aes_stress_all_with_rand_reset.83198152572356807614481871170510818285437053022156619704890996428086480719483
    Line 139, in log /workspaces/repo/scratch/os_regression_2024_08_26/aes_unmasked-sim-xcelium/9.aes_stress_all_with_rand_reset/latest/run.log
```
  UVM_FATAL @  13734708 ps: (aes_base_vseq.sv:74) [uvm_test_top.env.virtual_sequencer.aes_alert_reset_vseq] Check failed (aes_ctrl_aux[0] == cfg.do_reseed)
  UVM_INFO @  13734708 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_FATAL (aes_base_vseq.sv:74) [aes_stress_vseq] Check failed (aes_ctrl_aux[*] == cfg.do_reseed) has 2 failures:
- Test aes_stress_all_with_rand_reset has 2 failures.
  - 2.aes_stress_all_with_rand_reset.89298889473305932883118541611997096146095687950777363665880103438091476123149
    Line 139, in log /workspaces/repo/scratch/os_regression_2024_08_26/aes_unmasked-sim-xcelium/2.aes_stress_all_with_rand_reset/latest/run.log
```
  UVM_FATAL @  38502581 ps: (aes_base_vseq.sv:74) [uvm_test_top.env.virtual_sequencer.aes_stress_vseq] Check failed (aes_ctrl_aux[0] == cfg.do_reseed)
  UVM_INFO @  38502581 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 6.aes_stress_all_with_rand_reset.65489448429303296958754519981643757559709590258115873052531752208538959429771
    Line 219, in log /workspaces/repo/scratch/os_regression_2024_08_26/aes_unmasked-sim-xcelium/6.aes_stress_all_with_rand_reset/latest/run.log
```
  UVM_FATAL @ 140742708 ps: (aes_base_vseq.sv:74) [uvm_test_top.env.virtual_sequencer.aes_stress_vseq] Check failed (aes_ctrl_aux[0] == cfg.do_reseed)
  UVM_INFO @ 140742708 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (cip_base_vseq.sv:557) [aes_alert_reset_vseq] Check failed * == cfg.m_alert_agent_cfgs[alert_name].vif.get_alert() (* [*] vs * [*]) Alert fatal_fault fired unexpectedly! has 1 failures:
- Test aes_stress_all_with_rand_reset has 1 failures.
  - 3.aes_stress_all_with_rand_reset.65454038660657338143480945546012871863222686907275919946250531044370485732957
    Line 699, in log /workspaces/repo/scratch/os_regression_2024_08_26/aes_unmasked-sim-xcelium/3.aes_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 971629354 ps: (cip_base_vseq.sv:557) [uvm_test_top.env.virtual_sequencer.aes_alert_reset_vseq] Check failed 0 == cfg.m_alert_agent_cfgs[alert_name].vif.get_alert() (0 [0x0] vs 1 [0x1]) Alert fatal_fault fired unexpectedly!
  UVM_INFO @ 971629354 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_FATAL (csr_utils_pkg.sv:587) [csr_utils::csr_spinwait] timeout aes_reg_block.status.idle (addr=*, Comparison=CompareOpEq, exp_data=*, call_count=7) has 1 failures:
- Test aes_core_fi has 1 failures.
  - 17.aes_core_fi.4328201195145210493242190653056747335250622055627676056828689127040687310757
    Line 128, in log /workspaces/repo/scratch/os_regression_2024_08_26/aes_unmasked-sim-xcelium/17.aes_core_fi/latest/run.log
```
  UVM_FATAL @ 10017004478 ps: (csr_utils_pkg.sv:587) [csr_utils::csr_spinwait] timeout aes_reg_block.status.idle (addr=0x8e495084, Comparison=CompareOpEq, exp_data=0x0, call_count=7)
  UVM_INFO @ 10017004478 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_FATAL (aes_core_fi_vseq.sv:89) [aes_core_fi_vseq] wait timeout occurred! has 1 failures:
- Test aes_core_fi has 1 failures.
  - 29.aes_core_fi.62378794649555320427844928733529557221940014288338280029879191538851489235290
    Line 130, in log /workspaces/repo/scratch/os_regression_2024_08_26/aes_unmasked-sim-xcelium/29.aes_core_fi/latest/run.log
```
  UVM_FATAL @ 10027066014 ps: (aes_core_fi_vseq.sv:89) [uvm_test_top.env.virtual_sequencer.aes_core_fi_vseq] wait timeout occurred!
  UVM_INFO @ 10027066014 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```