FLASH_CTRL Simulation Results

Sunday January 21 2024 20:02:56 UTC

GitHub Revision: `796f9fb805`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 82526748448873323296379810788667205332667151893362240729689214265893867671108

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	flash_ctrl_smoke	3.680m	240.925us	48	50	96.00
V1	smoke_hw	flash_ctrl_smoke_hw	26.260s	94.299us	5	5	100.00
V1	csr_hw_reset	flash_ctrl_csr_hw_reset	45.800s	100.666us	5	5	100.00
V1	csr_rw	flash_ctrl_csr_rw	17.820s	139.771us	20	20	100.00
V1	csr_bit_bash	flash_ctrl_csr_bit_bash	1.321m	4.773ms	5	5	100.00
V1	csr_aliasing	flash_ctrl_csr_aliasing	1.143m	14.076ms	5	5	100.00
V1	csr_mem_rw_with_rand_reset	flash_ctrl_csr_mem_rw_with_rand_reset	19.630s	32.086us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	flash_ctrl_csr_rw	17.820s	139.771us	20	20	100.00
		flash_ctrl_csr_aliasing	1.143m	14.076ms	5	5	100.00
V1	mem_walk	flash_ctrl_mem_walk	13.910s	47.397us	5	5	100.00
V1	mem_partial_access	flash_ctrl_mem_partial_access	13.800s	29.464us	5	5	100.00
V1		TOTAL			118	120	98.33
V2	sw_op	flash_ctrl_sw_op	27.530s	62.131us	5	5	100.00
V2	host_read_direct	flash_ctrl_host_dir_rd	1.509m	212.592us	5	5	100.00
V2	rma_hw_if	flash_ctrl_hw_rma	29.678m	147.705ms	3	3	100.00
		flash_ctrl_hw_rma_reset	14.178m	110.163ms	10	20	50.00
		flash_ctrl_lcmgr_intg	13.960s	48.531us	20	20	100.00
V2	host_controller_arb	flash_ctrl_host_ctrl_arb	39.240m	401.242ms	5	5	100.00
V2	erase_suspend	flash_ctrl_erase_suspend	9.435m	18.643ms	5	5	100.00
V2	program_reset	flash_ctrl_prog_reset	21.310s	118.540us	30	30	100.00
V2	full_memory_access	flash_ctrl_full_mem_access	44.193m	330.672ms	2	5	40.00
V2	rd_buff_eviction	flash_ctrl_rd_buff_evict	3.566m	1.794ms	5	5	100.00
V2	rd_buff_eviction_w_ecc	flash_ctrl_rw_evict	37.260s	1.476ms	39	40	97.50
		flash_ctrl_rw_evict_all_en	36.490s	103.736us	40	40	100.00
		flash_ctrl_re_evict	39.000s	498.556us	19	20	95.00
V2	host_arb	flash_ctrl_phy_arb	8.813m	1.497ms	20	20	100.00
V2	host_interleave	flash_ctrl_phy_arb	8.813m	1.497ms	20	20	100.00
V2	memory_protection	flash_ctrl_mp_regions	18.986m	268.403ms	19	20	95.00
V2	fetch_code	flash_ctrl_fetch_code	31.710s	2.432ms	10	10	100.00
V2	all_partitions	flash_ctrl_rand_ops	24.462m	165.303us	20	20	100.00
V2	error_mp	flash_ctrl_error_mp	41.072m	4.522ms	10	10	100.00
V2	error_prog_win	flash_ctrl_error_prog_win	20.769m	504.839us	10	10	100.00
V2	error_prog_type	flash_ctrl_error_prog_type	46.240m	1.083ms	5	5	100.00
V2	error_read_seed	flash_ctrl_hw_read_seed_err	14.310s	15.319us	20	20	100.00
V2	read_write_overflow	flash_ctrl_oversize_error	3.348m	1.209ms	5	5	100.00
V2	flash_ctrl_disable	flash_ctrl_disable	11.139m	10.007ms	28	50	56.00
V2	flash_ctrl_connect	flash_ctrl_connect	16.650s	79.600us	80	80	100.00
V2	stress_all	flash_ctrl_stress_all	18.402m	451.116us	5	5	100.00
V2	secret_partition	flash_ctrl_hw_sec_otp	4.725m	23.191ms	50	50	100.00
		flash_ctrl_otp_reset	2.474m	74.800us	33	80	41.25
V2	isolation_partition	flash_ctrl_hw_rma	29.678m	147.705ms	3	3	100.00
V2	interrupts	flash_ctrl_intr_rd	3.066m	4.792ms	40	40	100.00
		flash_ctrl_intr_wr	2.075m	4.925ms	10	10	100.00
		flash_ctrl_intr_rd_slow_flash	5.006m	123.290ms	40	40	100.00
		flash_ctrl_intr_wr_slow_flash	12.465m	304.550ms	10	10	100.00
V2	invalid_op	flash_ctrl_invalid_op	1.598m	3.774ms	20	20	100.00
V2	mid_op_rst	flash_ctrl_mid_op_rst	1.210m	1.827ms	5	5	100.00
V2	double_bit_err	flash_ctrl_read_word_sweep_derr	23.010s	52.519us	5	5	100.00
		flash_ctrl_ro_derr	2.534m	1.350ms	10	10	100.00
		flash_ctrl_rw_derr	12.770m	4.932ms	10	10	100.00
		flash_ctrl_derr_detect	1.770m	179.127us	5	5	100.00
		flash_ctrl_integrity	10.985m	55.392ms	5	5	100.00
V2	single_bit_err	flash_ctrl_read_word_sweep_serr	23.730s	24.316us	5	5	100.00
		flash_ctrl_ro_serr	2.468m	6.012ms	10	10	100.00
		flash_ctrl_rw_serr	10.810m	3.808ms	10	10	100.00
V2	singlebit_err_counter	flash_ctrl_serr_counter	1.065m	1.034ms	5	5	100.00
V2	singlebit_err_address	flash_ctrl_serr_address	1.262m	3.093ms	5	5	100.00
V2	scramble	flash_ctrl_wo	4.099m	10.989ms	20	20	100.00
		flash_ctrl_write_word_sweep	17.170s	66.653us	1	1	100.00
		flash_ctrl_read_word_sweep	13.290s	15.567us	1	1	100.00
		flash_ctrl_ro	2.333m	1.793ms	20	20	100.00
		flash_ctrl_rw	9.292m	11.891ms	20	20	100.00
V2	filesystem_support	flash_ctrl_fs_sup	40.360s	539.816us	5	5	100.00
V2	rma_write_process_error	flash_ctrl_rma_err	14.744m	332.119ms	3	3	100.00
		flash_ctrl_hw_prog_rma_wipe_err	5.215m	10.012ms	20	20	100.00
V2	alert_test	flash_ctrl_alert_test	15.650s	109.356us	50	50	100.00
V2	intr_test	flash_ctrl_intr_test	14.580s	31.121us	50	50	100.00
V2	tl_d_oob_addr_access	flash_ctrl_tl_errors	20.780s	102.419us	20	20	100.00
V2	tl_d_illegal_access	flash_ctrl_tl_errors	20.780s	102.419us	20	20	100.00
V2	tl_d_outstanding_access	flash_ctrl_csr_hw_reset	45.800s	100.666us	5	5	100.00
		flash_ctrl_csr_rw	17.820s	139.771us	20	20	100.00
		flash_ctrl_csr_aliasing	1.143m	14.076ms	5	5	100.00
		flash_ctrl_same_csr_outstanding	37.030s	1.739ms	20	20	100.00
V2	tl_d_partial_access	flash_ctrl_csr_hw_reset	45.800s	100.666us	5	5	100.00
		flash_ctrl_csr_rw	17.820s	139.771us	20	20	100.00
		flash_ctrl_csr_aliasing	1.143m	14.076ms	5	5	100.00
		flash_ctrl_same_csr_outstanding	37.030s	1.739ms	20	20	100.00
V2		TOTAL			928	1013	91.61
V2S	shadow_reg_update_error	flash_ctrl_shadow_reg_errors	16.210s	45.365us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	flash_ctrl_shadow_reg_errors	16.210s	45.365us	20	20	100.00
V2S	shadow_reg_storage_error	flash_ctrl_shadow_reg_errors	16.210s	45.365us	20	20	100.00
V2S	shadowed_reset_glitch	flash_ctrl_shadow_reg_errors	16.210s	45.365us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	flash_ctrl_shadow_reg_errors_with_csr_rw	16.770s	32.176us	20	20	100.00
V2S	tl_intg_err	flash_ctrl_sec_cm	1.375h	1.997ms	5	5	100.00
		flash_ctrl_tl_intg_err	15.195m	713.176us	20	20	100.00
V2S	sec_cm_reg_bus_integrity	flash_ctrl_tl_intg_err	15.195m	713.176us	20	20	100.00
V2S	sec_cm_host_bus_integrity	flash_ctrl_tl_intg_err	15.195m	713.176us	20	20	100.00
V2S	sec_cm_mem_bus_integrity	flash_ctrl_rd_intg	33.220s	342.086us	3	3	100.00
		flash_ctrl_wr_intg	15.000s	175.602us	3	3	100.00
V2S	sec_cm_scramble_key_sideload	flash_ctrl_smoke	3.680m	240.925us	48	50	96.00
V2S	sec_cm_lc_ctrl_intersig_mubi	flash_ctrl_otp_reset	2.474m	74.800us	33	80	41.25
		flash_ctrl_disable	11.139m	10.007ms	28	50	56.00
		flash_ctrl_sec_info_access	1.441m	9.511ms	50	50	100.00
		flash_ctrl_connect	16.650s	79.600us	80	80	100.00
V2S	sec_cm_ctrl_config_regwen	flash_ctrl_config_regwen	13.830s	72.854us	5	5	100.00
V2S	sec_cm_data_regions_config_regwen	flash_ctrl_csr_rw	17.820s	139.771us	20	20	100.00
V2S	sec_cm_data_regions_config_shadow	flash_ctrl_shadow_reg_errors	16.210s	45.365us	20	20	100.00
V2S	sec_cm_info_regions_config_regwen	flash_ctrl_csr_rw	17.820s	139.771us	20	20	100.00
V2S	sec_cm_info_regions_config_shadow	flash_ctrl_shadow_reg_errors	16.210s	45.365us	20	20	100.00
V2S	sec_cm_bank_config_regwen	flash_ctrl_csr_rw	17.820s	139.771us	20	20	100.00
V2S	sec_cm_bank_config_shadow	flash_ctrl_shadow_reg_errors	16.210s	45.365us	20	20	100.00
V2S	sec_cm_mem_ctrl_global_esc	flash_ctrl_disable	11.139m	10.007ms	28	50	56.00
V2S	sec_cm_mem_ctrl_local_esc	flash_ctrl_rd_intg	33.220s	342.086us	3	3	100.00
		flash_ctrl_access_after_disable	14.090s	37.901us	3	3	100.00
V2S	sec_cm_mem_disable_config_mubi	flash_ctrl_disable	11.139m	10.007ms	28	50	56.00
V2S	sec_cm_exec_config_redun	flash_ctrl_fetch_code	31.710s	2.432ms	10	10	100.00
V2S	sec_cm_mem_scramble	flash_ctrl_rw	9.292m	11.891ms	20	20	100.00
V2S	sec_cm_mem_integrity	flash_ctrl_rw_serr	10.810m	3.808ms	10	10	100.00
		flash_ctrl_rw_derr	12.770m	4.932ms	10	10	100.00
		flash_ctrl_integrity	10.985m	55.392ms	5	5	100.00
V2S	sec_cm_rma_entry_mem_sec_wipe	flash_ctrl_hw_rma	29.678m	147.705ms	3	3	100.00
V2S	sec_cm_ctrl_fsm_sparse	flash_ctrl_sec_cm	1.375h	1.997ms	5	5	100.00
V2S	sec_cm_phy_fsm_sparse	flash_ctrl_sec_cm	1.375h	1.997ms	5	5	100.00
V2S	sec_cm_phy_prog_fsm_sparse	flash_ctrl_sec_cm	1.375h	1.997ms	5	5	100.00
V2S	sec_cm_ctr_redun	flash_ctrl_sec_cm	1.375h	1.997ms	5	5	100.00
V2S	sec_cm_phy_arbiter_ctrl_redun	flash_ctrl_phy_arb_redun	18.800s	95.400us	5	5	100.00
V2S	sec_cm_phy_host_grant_ctrl_consistency	flash_ctrl_phy_host_grant_err	14.730s	44.457us	5	5	100.00
V2S	sec_cm_phy_ack_ctrl_consistency	flash_ctrl_phy_ack_consistency	19.710s	149.412us	5	5	100.00
V2S	sec_cm_fifo_ctr_redun	flash_ctrl_sec_cm	1.375h	1.997ms	5	5	100.00
V2S	sec_cm_mem_tl_lc_gate_fsm_sparse	flash_ctrl_sec_cm	1.375h	1.997ms	5	5	100.00
V2S	sec_cm_prog_tl_lc_gate_fsm_sparse	flash_ctrl_sec_cm	1.375h	1.997ms	5	5	100.00
V2S		TOTAL			144	144	100.00
V3	asymmetric_read_path	flash_ctrl_rd_ooo	45.060s	81.188us	1	1	100.00
V3	stress_all_with_rand_reset	flash_ctrl_stress_all_with_rand_reset			0	0	--
V3		TOTAL			1	1	100.00
		TOTAL			1191	1278	93.19

Testplan Progress

Items	Total	Written	Passing	Progress
V1	9	9	8	88.89
V2	55	55	48	87.27
V2S	12	12	12	100.00
V3	2	1	1	50.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
96.25	95.87	94.08	98.95	89.80	98.46	98.20	98.42

Failure Buckets

UVM_WARNING (uvm_reg_field.svh:1191) [RegModel] Trying to predict value of field 'val' while register 'flash_ctrl_core_reg_block.dis' is being accessed has 78 failures:

Test flash_ctrl_otp_reset has 46 failures.

0.flash_ctrl_otp_reset.36819845740803297534194064222684524021769524365477403887331963915750937144015
Line 288, in log /container/opentitan-public/scratch/os_regression/flash_ctrl-sim-vcs/0.flash_ctrl_otp_reset/latest/run.log

  UVM_WARNING @ 3478.1 ns: (uvm_reg_field.svh:1191) [RegModel] Trying to predict value of field 'val' while register 'flash_ctrl_core_reg_block.dis' is being accessed
  UVM_INFO @ 3658.5 ns: (flash_ctrl_otp_reset_vseq.sv:130) uvm_test_top.env.virtual_sequencer [uvm_test_top.env.virtual_sequencer.flash_ctrl_otp_reset_vseq] RESET
  UVM_INFO @ 5019.7 ns: (flash_mem_bkdr_util.sv:26) [mem_bkdr_util[FlashPartData][0]] Randomizing flash mem contents
  UVM_INFO @ 5019.7 ns: (flash_mem_bkdr_util.sv:26) [mem_bkdr_util[FlashPartData][1]] Randomizing flash mem contents
  UVM_INFO @ 5019.7 ns: (flash_mem_bkdr_util.sv:26) [mem_bkdr_util[FlashPartInfo][0]] Randomizing flash mem contents

2.flash_ctrl_otp_reset.28767143925269524381520248318015206345561392207942662077572192036255000700470
Line 326, in log /container/opentitan-public/scratch/os_regression/flash_ctrl-sim-vcs/2.flash_ctrl_otp_reset/latest/run.log

  UVM_WARNING @ 84337.7 ns: (uvm_reg_field.svh:1191) [RegModel] Trying to predict value of field 'val' while register 'flash_ctrl_core_reg_block.dis' is being accessed
  UVM_INFO @ 85437.7 ns: (flash_ctrl_otp_reset_vseq.sv:130) uvm_test_top.env.virtual_sequencer [uvm_test_top.env.virtual_sequencer.flash_ctrl_otp_reset_vseq] RESET
  UVM_INFO @ 93837.7 ns: (flash_mem_bkdr_util.sv:26) [mem_bkdr_util[FlashPartData][0]] Randomizing flash mem contents
  UVM_INFO @ 93837.7 ns: (flash_mem_bkdr_util.sv:26) [mem_bkdr_util[FlashPartData][1]] Randomizing flash mem contents
  UVM_INFO @ 93837.7 ns: (flash_mem_bkdr_util.sv:26) [mem_bkdr_util[FlashPartInfo][0]] Randomizing flash mem contents

... and 44 more failures.

Test flash_ctrl_disable has 22 failures.

2.flash_ctrl_disable.36419046376206463101593305619451065369476074185526874683081131060001928246811
Line 283, in log /container/opentitan-public/scratch/os_regression/flash_ctrl-sim-vcs/2.flash_ctrl_disable/latest/run.log

  UVM_WARNING @ 46462.8 ns: (uvm_reg_field.svh:1191) [RegModel] Trying to predict value of field 'val' while register 'flash_ctrl_core_reg_block.dis' is being accessed
  UVM_INFO @ 52512.8 ns: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

3.flash_ctrl_disable.34197111348291107363480572063003653652855086098648782729710681972989762286269
Line 283, in log /container/opentitan-public/scratch/os_regression/flash_ctrl-sim-vcs/3.flash_ctrl_disable/latest/run.log

  UVM_WARNING @ 5164.6 ns: (uvm_reg_field.svh:1191) [RegModel] Trying to predict value of field 'val' while register 'flash_ctrl_core_reg_block.dis' is being accessed
  UVM_INFO @ 11054.6 ns: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 20 more failures.

Test flash_ctrl_hw_rma_reset has 10 failures.

3.flash_ctrl_hw_rma_reset.76552157243718636120576982895360599287981277907129564443177089544274088172465
Line 291, in log /container/opentitan-public/scratch/os_regression/flash_ctrl-sim-vcs/3.flash_ctrl_hw_rma_reset/latest/run.log

  UVM_WARNING @ 25904.6 ns: (uvm_reg_field.svh:1191) [RegModel] Trying to predict value of field 'val' while register 'flash_ctrl_core_reg_block.dis' is being accessed
  UVM_INFO @ 26201.6 ns: (flash_ctrl_hw_rma_reset_vseq.sv:72) uvm_test_top.env.virtual_sequencer [Test] RESET
  UVM_INFO @ 28280.6 ns: (flash_mem_bkdr_util.sv:26) [mem_bkdr_util[FlashPartData][0]] Randomizing flash mem contents
  UVM_INFO @ 28280.6 ns: (flash_mem_bkdr_util.sv:26) [mem_bkdr_util[FlashPartData][1]] Randomizing flash mem contents
  UVM_INFO @ 28280.6 ns: (flash_mem_bkdr_util.sv:26) [mem_bkdr_util[FlashPartInfo][0]] Randomizing flash mem contents

4.flash_ctrl_hw_rma_reset.93432997902822284599397177889257785814103535454544728200311678113313826644422
Line 291, in log /container/opentitan-public/scratch/os_regression/flash_ctrl-sim-vcs/4.flash_ctrl_hw_rma_reset/latest/run.log

  UVM_WARNING @ 10873.8 ns: (uvm_reg_field.svh:1191) [RegModel] Trying to predict value of field 'val' while register 'flash_ctrl_core_reg_block.dis' is being accessed
  UVM_INFO @ 11012.4 ns: (flash_ctrl_hw_rma_reset_vseq.sv:72) uvm_test_top.env.virtual_sequencer [Test] RESET
  UVM_INFO @ 11944.8 ns: (flash_mem_bkdr_util.sv:26) [mem_bkdr_util[FlashPartData][0]] Randomizing flash mem contents
  UVM_INFO @ 11944.8 ns: (flash_mem_bkdr_util.sv:26) [mem_bkdr_util[FlashPartData][1]] Randomizing flash mem contents
  UVM_INFO @ 11944.8 ns: (flash_mem_bkdr_util.sv:26) [mem_bkdr_util[FlashPartInfo][0]] Randomizing flash mem contents

... and 8 more failures.

Job flash_ctrl-sim-vcs_run_default killed due to: Exit reason: User job exceeded runlimit: User job timed out has 3 failures:
- Test flash_ctrl_full_mem_access has 3 failures.
  - 2.flash_ctrl_full_mem_access.57981094848220771416941747146782827538588995304677065157825377120288904854618
    Log /container/opentitan-public/scratch/os_regression/flash_ctrl-sim-vcs/2.flash_ctrl_full_mem_access/latest/run.log
```
  Job ID: smart:2e29296c-714d-4bfa-bddc-83b8e0461d0b
```
  - 3.flash_ctrl_full_mem_access.53929472375820047369739264408063937040419394615451143428328478703294579667249
    Log /container/opentitan-public/scratch/os_regression/flash_ctrl-sim-vcs/3.flash_ctrl_full_mem_access/latest/run.log
```
  Job ID: smart:e4100052-c3e1-4df7-9ee6-e3b9d213a91a
```
  - ... and 1 more failures.
Job flash_ctrl-sim-vcs_run_default killed due to: Exit reason: Error: * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *] has 2 failures:
- Test flash_ctrl_smoke has 2 failures.
  - 5.flash_ctrl_smoke.53317185838367669027060774390955889404501005334929987702015716405560924263153
    Log /container/opentitan-public/scratch/os_regression/flash_ctrl-sim-vcs/5.flash_ctrl_smoke/latest/run.log
```
  Job ID: smart:ad57301c-1590-4f34-abae-0c29b446b75c
```
  - 21.flash_ctrl_smoke.44052798728394947058051511417003755240317380768155796368674523008917383410117
    Log /container/opentitan-public/scratch/os_regression/flash_ctrl-sim-vcs/21.flash_ctrl_smoke/latest/run.log
```
  Job ID: smart:c80c407b-d38f-40ba-9e6a-2eb15e0af2fc
```
Job flash_ctrl-sim-vcs_run_default killed due to: Exit reason: Error: * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *] has 2 failures:
- Test flash_ctrl_otp_reset has 1 failures.
  - 14.flash_ctrl_otp_reset.90734148736564111558452096686161112285516354902601374801242373316763104134433
    Log /container/opentitan-public/scratch/os_regression/flash_ctrl-sim-vcs/14.flash_ctrl_otp_reset/latest/run.log
```
  Job ID: smart:cfc1eb1c-63aa-4160-b9a4-4a75345f5293
```
- Test flash_ctrl_re_evict has 1 failures.
  - 16.flash_ctrl_re_evict.7479540653172405566769645901837604717302174114157629177450974640694855187474
    Log /container/opentitan-public/scratch/os_regression/flash_ctrl-sim-vcs/16.flash_ctrl_re_evict/latest/run.log
```
  Job ID: smart:d5e92b56-8f37-4a89-953d-1f6e5bd40b6b
```
UVM_ERROR (cip_base_scoreboard.sv:287) scoreboard [scoreboard] alert recov_err did not trigger max_delay:* has 1 failures:
- Test flash_ctrl_mp_regions has 1 failures.
  - 1.flash_ctrl_mp_regions.26010267031159307251674912353259931716593191920059018072264224932364003176839
    Line 1652, in log /container/opentitan-public/scratch/os_regression/flash_ctrl-sim-vcs/1.flash_ctrl_mp_regions/latest/run.log
```
  UVM_ERROR @ 1652029.9 ns: (cip_base_scoreboard.sv:287) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_err did not trigger max_delay:2000
  UVM_INFO @ 1652029.9 ns: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (csr_utils_pkg.sv:457) [csr_utils::csr_rd_check] Check failed obs == exp (* [*] vs * [*]) Regname: flash_ctrl_core_reg_block.op_status.err reset value: * has 1 failures:
- Test flash_ctrl_rw_evict has 1 failures.
  - 25.flash_ctrl_rw_evict.7539307032542168147179250162223426720817936549676934596801104361289384756621
    Line 282, in log /container/opentitan-public/scratch/os_regression/flash_ctrl-sim-vcs/25.flash_ctrl_rw_evict/latest/run.log
```
  UVM_ERROR @ 38291.6 ns: (csr_utils_pkg.sv:457) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: flash_ctrl_core_reg_block.op_status.err reset value: 0x0
  UVM_INFO @ 38291.6 ns: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```