OTBN Simulation Results

Saturday May 20 2023 07:05:26 UTC

GitHub Revision: `e3fb01b5e`

Branch: os_regression

Simulator: XCELIUM

Build randomization enabled with --build-seed 2781625531

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	otbn_smoke	9.000s	24.441us	1	1	100.00
V1	single_binary	otbn_single	3.333m	1.032ms	92	100	92.00
V1	csr_hw_reset	otbn_csr_hw_reset	8.000s	37.643us	5	5	100.00
V1	csr_rw	otbn_csr_rw	6.000s	18.455us	20	20	100.00
V1	csr_bit_bash	otbn_csr_bit_bash	9.000s	140.347us	5	5	100.00
V1	csr_aliasing	otbn_csr_aliasing	6.000s	27.797us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	otbn_csr_mem_rw_with_rand_reset	8.000s	77.080us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	otbn_csr_rw	6.000s	18.455us	20	20	100.00
		otbn_csr_aliasing	6.000s	27.797us	5	5	100.00
V1	mem_walk	otbn_mem_walk	36.000s	1.141ms	5	5	100.00
V1	mem_partial_access	otbn_mem_partial_access	16.000s	1.306ms	5	5	100.00
V1		TOTAL			158	166	95.18
V2	reset_recovery	otbn_reset	34.000s	340.324us	8	10	80.00
V2	multi_error	otbn_multi_err	53.000s	658.354us	1	1	100.00
V2	back_to_back	otbn_multi	1.000m	230.178us	9	10	90.00
V2	stress_all	otbn_stress_all	1.367m	296.487us	8	10	80.00
V2	lc_escalation	otbn_escalate	27.000s	320.554us	47	60	78.33
V2	zero_state_err_urnd	otbn_zero_state_err_urnd	9.000s	40.053us	2	5	40.00
V2	sw_errs_fatal_chk	otbn_sw_errs_fatal_chk	21.000s	64.682us	9	10	90.00
V2	alert_test	otbn_alert_test	9.000s	52.691us	50	50	100.00
V2	intr_test	otbn_intr_test	6.000s	37.460us	50	50	100.00
V2	tl_d_oob_addr_access	otbn_tl_errors	11.000s	129.033us	20	20	100.00
V2	tl_d_illegal_access	otbn_tl_errors	11.000s	129.033us	20	20	100.00
V2	tl_d_outstanding_access	otbn_csr_hw_reset	8.000s	37.643us	5	5	100.00
		otbn_csr_rw	6.000s	18.455us	20	20	100.00
		otbn_csr_aliasing	6.000s	27.797us	5	5	100.00
		otbn_same_csr_outstanding	7.000s	32.543us	20	20	100.00
V2	tl_d_partial_access	otbn_csr_hw_reset	8.000s	37.643us	5	5	100.00
		otbn_csr_rw	6.000s	18.455us	20	20	100.00
		otbn_csr_aliasing	6.000s	27.797us	5	5	100.00
		otbn_same_csr_outstanding	7.000s	32.543us	20	20	100.00
V2		TOTAL			224	246	91.06
V2S	mem_integrity	otbn_imem_err	11.000s	69.502us	9	10	90.00
		otbn_dmem_err	11.000s	24.495us	14	15	93.33
V2S	internal_integrity	otbn_alu_bignum_mod_err	10.000s	101.653us	5	5	100.00
		otbn_controller_ispr_rdata_err	16.000s	95.498us	5	5	100.00
		otbn_mac_bignum_acc_err	19.000s	74.375us	5	5	100.00
		otbn_urnd_err	6.000s	17.123us	2	2	100.00
V2S	illegal_bus_access	otbn_illegal_mem_acc	8.000s	16.841us	5	5	100.00
V2S	otbn_mem_gnt_acc_err	otbn_mem_gnt_acc_err	58.000s	10.003ms	1	2	50.00
V2S	tl_intg_err	otbn_sec_cm	4.967m	2.921ms	5	5	100.00
		otbn_tl_intg_err	28.000s	205.779us	20	20	100.00
V2S	passthru_mem_tl_intg_err	otbn_passthru_mem_tl_intg_err	28.000s	2.981ms	20	20	100.00
V2S	prim_fsm_check	otbn_sec_cm	4.967m	2.921ms	5	5	100.00
V2S	prim_count_check	otbn_sec_cm	4.967m	2.921ms	5	5	100.00
V2S	sec_cm_mem_scramble	otbn_smoke	9.000s	24.441us	1	1	100.00
V2S	sec_cm_data_mem_integrity	otbn_dmem_err	11.000s	24.495us	14	15	93.33
V2S	sec_cm_instruction_mem_integrity	otbn_imem_err	11.000s	69.502us	9	10	90.00
V2S	sec_cm_bus_integrity	otbn_tl_intg_err	28.000s	205.779us	20	20	100.00
V2S	sec_cm_controller_fsm_global_esc	otbn_escalate	27.000s	320.554us	47	60	78.33
V2S	sec_cm_controller_fsm_local_esc	otbn_imem_err	11.000s	69.502us	9	10	90.00
		otbn_dmem_err	11.000s	24.495us	14	15	93.33
		otbn_zero_state_err_urnd	9.000s	40.053us	2	5	40.00
		otbn_illegal_mem_acc	8.000s	16.841us	5	5	100.00
		otbn_sec_cm	4.967m	2.921ms	5	5	100.00
V2S	sec_cm_controller_fsm_sparse	otbn_sec_cm	4.967m	2.921ms	5	5	100.00
V2S	sec_cm_scramble_key_sideload	otbn_single	3.333m	1.032ms	92	100	92.00
V2S	sec_cm_scramble_ctrl_fsm_local_esc	otbn_imem_err	11.000s	69.502us	9	10	90.00
		otbn_dmem_err	11.000s	24.495us	14	15	93.33
		otbn_zero_state_err_urnd	9.000s	40.053us	2	5	40.00
		otbn_illegal_mem_acc	8.000s	16.841us	5	5	100.00
		otbn_sec_cm	4.967m	2.921ms	5	5	100.00
V2S	sec_cm_scramble_ctrl_fsm_sparse	otbn_sec_cm	4.967m	2.921ms	5	5	100.00
V2S	sec_cm_start_stop_ctrl_fsm_global_esc	otbn_escalate	27.000s	320.554us	47	60	78.33
V2S	sec_cm_start_stop_ctrl_fsm_local_esc	otbn_imem_err	11.000s	69.502us	9	10	90.00
		otbn_dmem_err	11.000s	24.495us	14	15	93.33
		otbn_zero_state_err_urnd	9.000s	40.053us	2	5	40.00
		otbn_illegal_mem_acc	8.000s	16.841us	5	5	100.00
		otbn_sec_cm	4.967m	2.921ms	5	5	100.00
V2S	sec_cm_start_stop_ctrl_fsm_sparse	otbn_sec_cm	4.967m	2.921ms	5	5	100.00
V2S	sec_cm_data_reg_sw_sca	otbn_single	3.333m	1.032ms	92	100	92.00
V2S	sec_cm_ctrl_redun	otbn_ctrl_redun	10.000s	35.226us	7	12	58.33
V2S	sec_cm_pc_ctrl_flow_redun	otbn_pc_ctrl_flow_redun	9.000s	16.904us	5	5	100.00
V2S	sec_cm_rnd_bus_consistency	otbn_rnd_sec_cm	30.000s	99.416us	5	5	100.00
V2S	sec_cm_rnd_rng_digest	otbn_rnd_sec_cm	30.000s	99.416us	5	5	100.00
V2S	sec_cm_rf_base_data_reg_sw_integrity	otbn_rf_base_intg_err	10.000s	57.307us	9	10	90.00
V2S	sec_cm_rf_base_data_reg_sw_glitch_detect	otbn_sec_cm	4.967m	2.921ms	5	5	100.00
V2S	sec_cm_stack_wr_ptr_ctr_redun	otbn_sec_cm	4.967m	2.921ms	5	5	100.00
V2S	sec_cm_rf_bignum_data_reg_sw_integrity	otbn_rf_bignum_intg_err	10.000s	131.385us	10	10	100.00
V2S	sec_cm_rf_bignum_data_reg_sw_glitch_detect	otbn_sec_cm	4.967m	2.921ms	5	5	100.00
V2S	sec_cm_loop_stack_ctr_redun	otbn_sec_cm	4.967m	2.921ms	5	5	100.00
V2S	sec_cm_loop_stack_addr_integrity	otbn_stack_addr_integ_chk	42.000s	10.015ms	3	5	60.00
V2S	sec_cm_call_stack_addr_integrity	otbn_stack_addr_integ_chk	42.000s	10.015ms	3	5	60.00
V2S	sec_cm_start_stop_ctrl_state_consistency	otbn_sec_wipe_err	13.000s	50.275us	7	7	100.00
V2S	sec_cm_data_mem_sec_wipe	otbn_single	3.333m	1.032ms	92	100	92.00
V2S	sec_cm_instruction_mem_sec_wipe	otbn_single	3.333m	1.032ms	92	100	92.00
V2S	sec_cm_data_reg_sw_sec_wipe	otbn_single	3.333m	1.032ms	92	100	92.00
V2S	sec_cm_write_mem_integrity	otbn_multi	1.000m	230.178us	9	10	90.00
V2S	sec_cm_ctrl_flow_count	otbn_single	3.333m	1.032ms	92	100	92.00
V2S	sec_cm_ctrl_flow_sca	otbn_single	3.333m	1.032ms	92	100	92.00
V2S	sec_cm_data_mem_sw_noaccess	otbn_sw_no_acc	15.000s	105.878us	5	5	100.00
V2S	sec_cm_key_sideload	otbn_single	3.333m	1.032ms	92	100	92.00
V2S	sec_cm_tlul_fifo_ctr_redun	otbn_sec_cm	4.967m	2.921ms	5	5	100.00
V2S		TOTAL			142	153	92.81
V3	stress_all_with_rand_reset	otbn_stress_all_with_rand_reset	10.800m	8.172ms	3	10	30.00
V3		TOTAL			3	10	30.00
		TOTAL			527	575	91.65

Testplan Progress

Items	Total	Written	Passing	Progress
V1	9	9	8	88.89
V2	11	11	5	45.45
V2S	19	19	13	68.42
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

Score	Block	Branch	Statement	Expression	Toggle	Fsm	Assertion	CoverGroup
98.75	99.51	94.20	99.61	90.81	93.46	94.87	91.17	99.16

Failure Buckets

Exit reason: Error: User command failed xmsim: *E,ASRTST (/workspace/default/src/lowrisc_dv_otbn_sim_*/tb.sv,270): Assertion MatchingStatus_A has failed has 5 failures:

Test otbn_escalate has 4 failures.

0.otbn_escalate.2716628107
Line 241, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/0.otbn_escalate/latest/run.log

  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_dv_otbn_sim_0.1/tb.sv,270): (time 81516675 PS) Assertion tb.MatchingStatus_A has failed
  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_dv_otbn_env_0.1/otbn_insn_cnt_if.sv,21): (time 81516675 PS) Assertion tb.insn_cnt_if.InsnCntMatches_A has failed
  UVM_ERROR @  81516675 ps: (tb.sv:270) [ASSERT FAILED] MatchingStatus_A
  UVM_INFO @  81516675 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

13.otbn_escalate.3473181139
Line 241, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/13.otbn_escalate/latest/run.log

  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_dv_otbn_sim_0.1/tb.sv,270): (time 7528947 PS) Assertion tb.MatchingStatus_A has failed
  UVM_ERROR @   7528947 ps: (tb.sv:270) [ASSERT FAILED] MatchingStatus_A
  UVM_INFO @   7528947 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 2 more failures.

Test otbn_rf_base_intg_err has 1 failures.

1.otbn_rf_base_intg_err.2793694558
Line 243, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/1.otbn_rf_base_intg_err/latest/run.log

  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_dv_otbn_sim_0.1/tb.sv,270): (time 62038424 PS) Assertion tb.MatchingStatus_A has failed
  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_dv_otbn_env_0.1/otbn_insn_cnt_if.sv,21): (time 62038424 PS) Assertion tb.insn_cnt_if.InsnCntMatches_A has failed
  UVM_ERROR @  62038424 ps: (tb.sv:270) [ASSERT FAILED] MatchingStatus_A
  UVM_INFO @  62038424 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Exit reason: Error: User command failed UVM_FATAL (otbn_base_vseq.sv:582) [otbn_single_vseq] Check failed (!(cfg.loop_vif.loop_stack_push || cfg.loop_vif.loop_stack_pop) && !cfg.trace_vif.locking_o) has 5 failures:

Test otbn_stress_all_with_rand_reset has 3 failures.

1.otbn_stress_all_with_rand_reset.3187011891
Line 286, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/1.otbn_stress_all_with_rand_reset/latest/run.log

  UVM_FATAL @ 457762896 ps: (otbn_base_vseq.sv:582) [uvm_test_top.env.virtual_sequencer.otbn_single_vseq] Check failed (!(cfg.loop_vif.loop_stack_push || cfg.loop_vif.loop_stack_pop) &&                       !cfg.trace_vif.locking_o)
  UVM_INFO @ 457762896 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

5.otbn_stress_all_with_rand_reset.3109617970
Line 294, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/5.otbn_stress_all_with_rand_reset/latest/run.log

  UVM_FATAL @ 208565234 ps: (otbn_base_vseq.sv:582) [uvm_test_top.env.virtual_sequencer.otbn_single_vseq] Check failed (!(cfg.loop_vif.loop_stack_push || cfg.loop_vif.loop_stack_pop) &&                       !cfg.trace_vif.locking_o)
  UVM_INFO @ 208565234 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 1 more failures.

Test otbn_single has 2 failures.

6.otbn_single.2597054946
Line 242, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/6.otbn_single/latest/run.log

  UVM_FATAL @  45962657 ps: (otbn_base_vseq.sv:582) [uvm_test_top.env.virtual_sequencer.otbn_single_vseq] Check failed (!(cfg.loop_vif.loop_stack_push || cfg.loop_vif.loop_stack_pop) &&                       !cfg.trace_vif.locking_o)
  UVM_INFO @  45962657 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

31.otbn_single.1844727367
Line 241, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/31.otbn_single/latest/run.log

  UVM_FATAL @  99562912 ps: (otbn_base_vseq.sv:582) [uvm_test_top.env.virtual_sequencer.otbn_single_vseq] Check failed (!(cfg.loop_vif.loop_stack_push || cfg.loop_vif.loop_stack_pop) &&                       !cfg.trace_vif.locking_o)
  UVM_INFO @  99562912 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_FATAL (otbn_base_vseq.sv:369) [otbn_single_vseq] Check failed (cfg.model_agent_cfg.vif.status == otbn_pkg::StatusIdle) Timed out waiting for OTBN to be idle before execution has 5 failures:

Test otbn_single has 5 failures.

13.otbn_single.3730720329
Line 242, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/13.otbn_single/latest/run.log

  UVM_FATAL @  13211120 ps: (otbn_base_vseq.sv:369) [uvm_test_top.env.virtual_sequencer.otbn_single_vseq] Check failed (cfg.model_agent_cfg.vif.status == otbn_pkg::StatusIdle) Timed out waiting for OTBN to be idle before execution
  UVM_INFO @  13211120 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

39.otbn_single.2643381536
Line 241, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/39.otbn_single/latest/run.log

  UVM_FATAL @  10554354 ps: (otbn_base_vseq.sv:369) [uvm_test_top.env.virtual_sequencer.otbn_single_vseq] Check failed (cfg.model_agent_cfg.vif.status == otbn_pkg::StatusIdle) Timed out waiting for OTBN to be idle before execution
  UVM_INFO @  10554354 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 3 more failures.

Exit reason: Error: User command failed xmsim: *E,ASRTST (/workspace/default/src/lowrisc_dv_otbn_env_*/otbn_insn_cnt_if.sv,21): Assertion InsnCntMatches_A has failed has 4 failures:

Test otbn_zero_state_err_urnd has 2 failures.

1.otbn_zero_state_err_urnd.3195020776
Line 242, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/1.otbn_zero_state_err_urnd/latest/run.log

  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_dv_otbn_env_0.1/otbn_insn_cnt_if.sv,21): (time 6897088 PS) Assertion tb.insn_cnt_if.InsnCntMatches_A has failed
  UVM_ERROR @   6897088 ps: (otbn_insn_cnt_if.sv:21) [ASSERT FAILED] InsnCntMatches_A
  UVM_INFO @   6897088 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

3.otbn_zero_state_err_urnd.1356520946
Line 241, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/3.otbn_zero_state_err_urnd/latest/run.log

  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_dv_otbn_env_0.1/otbn_insn_cnt_if.sv,21): (time 9783523 PS) Assertion tb.insn_cnt_if.InsnCntMatches_A has failed
  UVM_ERROR @   9783523 ps: (otbn_insn_cnt_if.sv:21) [ASSERT FAILED] InsnCntMatches_A
  UVM_INFO @   9783523 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test otbn_stack_addr_integ_chk has 1 failures.

2.otbn_stack_addr_integ_chk.1942412440
Line 249, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/2.otbn_stack_addr_integ_chk/latest/run.log

  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_dv_otbn_env_0.1/otbn_insn_cnt_if.sv,21): (time 37809896 PS) Assertion tb.insn_cnt_if.InsnCntMatches_A has failed
  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_dv_otbn_model_agent_0.1/otbn_model_if.sv,137): (time 37809896 PS) Assertion tb.model_if.NoModelErrs has failed
  UVM_ERROR @  37809896 ps: (otbn_insn_cnt_if.sv:21) [ASSERT FAILED] InsnCntMatches_A
  UVM_INFO @  37809896 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test otbn_escalate has 1 failures.

46.otbn_escalate.3125697734
Line 242, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/46.otbn_escalate/latest/run.log

  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_dv_otbn_env_0.1/otbn_insn_cnt_if.sv,21): (time 29991440 PS) Assertion tb.insn_cnt_if.InsnCntMatches_A has failed
  UVM_ERROR @  29991440 ps: (otbn_insn_cnt_if.sv:21) [ASSERT FAILED] InsnCntMatches_A
  UVM_INFO @  29991440 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Exit reason: Error: User command failed xmsim: *E,ASRTST (/workspace/default/src/lowrisc_dv_otbn_model_agent_*/otbn_model_if.sv,137): Assertion NoModelErrs has failed has 3 failures:

Test otbn_escalate has 3 failures.

10.otbn_escalate.2574497132
Line 261, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/10.otbn_escalate/latest/run.log

  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_dv_otbn_model_agent_0.1/otbn_model_if.sv,137): (time 1669885 PS) Assertion tb.model_if.NoModelErrs has failed
  UVM_ERROR @   1669885 ps: (otbn_model_if.sv:137) [ASSERT FAILED] NoModelErrs
  UVM_INFO @   1669885 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

43.otbn_escalate.3767341499
Line 261, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/43.otbn_escalate/latest/run.log

  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_dv_otbn_model_agent_0.1/otbn_model_if.sv,137): (time 3466515 PS) Assertion tb.model_if.NoModelErrs has failed
  UVM_ERROR @   3466515 ps: (otbn_model_if.sv:137) [ASSERT FAILED] NoModelErrs
  UVM_INFO @   3466515 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 1 more failures.

UVM_FATAL (otbn_scoreboard.sv:538) scoreboard [scoreboard] We saw a STATUS change * cycles ago that implied we'd get a fatal alert but it still hasn't arrived. has 3 failures:

Test otbn_escalate has 3 failures.

30.otbn_escalate.3938701672
Line 242, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/30.otbn_escalate/latest/run.log

  UVM_FATAL @   8149734 ps: (otbn_scoreboard.sv:538) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] We saw a STATUS change 30 cycles ago that implied we'd get a fatal alert but it still hasn't arrived.
  UVM_INFO @   8149734 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

47.otbn_escalate.1541486062
Line 242, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/47.otbn_escalate/latest/run.log

  UVM_FATAL @  44310444 ps: (otbn_scoreboard.sv:538) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] We saw a STATUS change 30 cycles ago that implied we'd get a fatal alert but it still hasn't arrived.
  UVM_INFO @  44310444 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 1 more failures.

UVM_FATAL (otbn_base_vseq.sv:369) [otbn_reset_vseq] Check failed (cfg.model_agent_cfg.vif.status == otbn_pkg::StatusIdle) Timed out waiting for OTBN to be idle before execution has 2 failures:

Test otbn_reset has 2 failures.

0.otbn_reset.731028425
Line 242, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/0.otbn_reset/latest/run.log

  UVM_FATAL @  10041274 ps: (otbn_base_vseq.sv:369) [uvm_test_top.env.virtual_sequencer.otbn_reset_vseq] Check failed (cfg.model_agent_cfg.vif.status == otbn_pkg::StatusIdle) Timed out waiting for OTBN to be idle before execution
  UVM_INFO @  10041274 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

4.otbn_reset.1074633853
Line 242, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/4.otbn_reset/latest/run.log

  UVM_FATAL @  10188823 ps: (otbn_base_vseq.sv:369) [uvm_test_top.env.virtual_sequencer.otbn_reset_vseq] Check failed (cfg.model_agent_cfg.vif.status == otbn_pkg::StatusIdle) Timed out waiting for OTBN to be idle before execution
  UVM_INFO @  10188823 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_FATAL (otbn_base_vseq.sv:369) [otbn_ctrl_redun_vseq] Check failed (cfg.model_agent_cfg.vif.status == otbn_pkg::StatusIdle) Timed out waiting for OTBN to be idle before execution has 2 failures:

Test otbn_ctrl_redun has 2 failures.

0.otbn_ctrl_redun.22377881
Line 242, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/0.otbn_ctrl_redun/latest/run.log

  UVM_FATAL @  11158851 ps: (otbn_base_vseq.sv:369) [uvm_test_top.env.virtual_sequencer.otbn_ctrl_redun_vseq] Check failed (cfg.model_agent_cfg.vif.status == otbn_pkg::StatusIdle) Timed out waiting for OTBN to be idle before execution
  UVM_INFO @  11158851 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

2.otbn_ctrl_redun.4062641114
Line 241, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/2.otbn_ctrl_redun/latest/run.log

  UVM_FATAL @  11802472 ps: (otbn_base_vseq.sv:369) [uvm_test_top.env.virtual_sequencer.otbn_ctrl_redun_vseq] Check failed (cfg.model_agent_cfg.vif.status == otbn_pkg::StatusIdle) Timed out waiting for OTBN to be idle before execution
  UVM_INFO @  11802472 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_FATAL (otbn_base_vseq.sv:582) [otbn_single_vseq] Check failed (!(cfg.loop_vif.loop_stack_push || cfg.loop_vif.loop_stack_pop) && !cfg.trace_vif.locking_o) has 2 failures:

Test otbn_stress_all_with_rand_reset has 1 failures.

4.otbn_stress_all_with_rand_reset.2887939493
Line 301, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/4.otbn_stress_all_with_rand_reset/latest/run.log

  UVM_FATAL @ 283983727 ps: (otbn_base_vseq.sv:582) [uvm_test_top.env.virtual_sequencer.otbn_single_vseq] Check failed (!(cfg.loop_vif.loop_stack_push || cfg.loop_vif.loop_stack_pop) &&                       !cfg.trace_vif.locking_o)
  UVM_INFO @ 283983727 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test otbn_single has 1 failures.

51.otbn_single.3656134204
Line 240, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/51.otbn_single/latest/run.log

  UVM_FATAL @  11734144 ps: (otbn_base_vseq.sv:582) [uvm_test_top.env.virtual_sequencer.otbn_single_vseq] Check failed (!(cfg.loop_vif.loop_stack_push || cfg.loop_vif.loop_stack_pop) &&                       !cfg.trace_vif.locking_o)
  UVM_INFO @  11734144 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Exit reason: Error: User command failed xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_otbn_*/rtl/otbn.sv,1258): Assertion InitSecWipeNonZeroWideRegs_A has failed has 2 failures:

Test otbn_ctrl_redun has 2 failures.

4.otbn_ctrl_redun.1493221459
Line 243, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/4.otbn_ctrl_redun/latest/run.log

  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_otbn_0.1/rtl/otbn.sv,1258): (time 3381065 PS) Assertion tb.dut.gen_sec_wipe_wdr_asserts[31].InitSecWipeNonZeroWideRegs_A has failed
  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_otbn_0.1/rtl/otbn.sv,1258): (time 3381065 PS) Assertion tb.dut.gen_sec_wipe_wdr_asserts[30].InitSecWipeNonZeroWideRegs_A has failed
  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_otbn_0.1/rtl/otbn.sv,1258): (time 3381065 PS) Assertion tb.dut.gen_sec_wipe_wdr_asserts[29].InitSecWipeNonZeroWideRegs_A has failed
  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_otbn_0.1/rtl/otbn.sv,1258): (time 3381065 PS) Assertion tb.dut.gen_sec_wipe_wdr_asserts[28].InitSecWipeNonZeroWideRegs_A has failed
  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_otbn_0.1/rtl/otbn.sv,1258): (time 3381065 PS) Assertion tb.dut.gen_sec_wipe_wdr_asserts[27].InitSecWipeNonZeroWideRegs_A has failed

6.otbn_ctrl_redun.1659087028
Line 241, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/6.otbn_ctrl_redun/latest/run.log

  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_otbn_0.1/rtl/otbn.sv,1258): (time 6278653 PS) Assertion tb.dut.gen_sec_wipe_wdr_asserts[31].InitSecWipeNonZeroWideRegs_A has failed
  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_otbn_0.1/rtl/otbn.sv,1258): (time 6278653 PS) Assertion tb.dut.gen_sec_wipe_wdr_asserts[30].InitSecWipeNonZeroWideRegs_A has failed
  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_otbn_0.1/rtl/otbn.sv,1258): (time 6278653 PS) Assertion tb.dut.gen_sec_wipe_wdr_asserts[29].InitSecWipeNonZeroWideRegs_A has failed
  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_otbn_0.1/rtl/otbn.sv,1258): (time 6278653 PS) Assertion tb.dut.gen_sec_wipe_wdr_asserts[28].InitSecWipeNonZeroWideRegs_A has failed
  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_otbn_0.1/rtl/otbn.sv,1258): (time 6278653 PS) Assertion tb.dut.gen_sec_wipe_wdr_asserts[27].InitSecWipeNonZeroWideRegs_A has failed

UVM_FATAL (otbn_base_vseq.sv:582) [otbn_imem_err_vseq] Check failed (!(cfg.loop_vif.loop_stack_push || cfg.loop_vif.loop_stack_pop) && !cfg.trace_vif.locking_o) has 2 failures:

Test otbn_stress_all_with_rand_reset has 2 failures.

7.otbn_stress_all_with_rand_reset.2242572737
Line 291, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/7.otbn_stress_all_with_rand_reset/latest/run.log

  UVM_FATAL @ 784626641 ps: (otbn_base_vseq.sv:582) [uvm_test_top.env.virtual_sequencer.otbn_imem_err_vseq] Check failed (!(cfg.loop_vif.loop_stack_push || cfg.loop_vif.loop_stack_pop) &&                       !cfg.trace_vif.locking_o)
  UVM_INFO @ 784626641 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

9.otbn_stress_all_with_rand_reset.1490008616
Line 302, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/9.otbn_stress_all_with_rand_reset/latest/run.log

  UVM_FATAL @ 820293424 ps: (otbn_base_vseq.sv:582) [uvm_test_top.env.virtual_sequencer.otbn_imem_err_vseq] Check failed (!(cfg.loop_vif.loop_stack_push || cfg.loop_vif.loop_stack_pop) &&                       !cfg.trace_vif.locking_o)
  UVM_INFO @ 820293424 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_FATAL (otbn_mem_gnt_acc_err_vseq.sv:40) [otbn_mem_gnt_acc_err_vseq] timeout occurred! has 1 failures:
- Test otbn_mem_gnt_acc_err has 1 failures.
  - 1.otbn_mem_gnt_acc_err.2404620979
    Line 244, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/1.otbn_mem_gnt_acc_err/latest/run.log
```
  UVM_FATAL @ 10003361570 ps: (otbn_mem_gnt_acc_err_vseq.sv:40) [uvm_test_top.env.virtual_sequencer.otbn_mem_gnt_acc_err_vseq] timeout occurred!
  UVM_INFO @ 10003361570 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```

UVM_FATAL (otbn_base_vseq.sv:582) [otbn_multi_vseq] Check failed (!(cfg.loop_vif.loop_stack_push || cfg.loop_vif.loop_stack_pop) && !cfg.trace_vif.locking_o) has 1 failures:

Test otbn_multi has 1 failures.

3.otbn_multi.3431104146
Line 284, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/3.otbn_multi/latest/run.log

  UVM_FATAL @ 307994977 ps: (otbn_base_vseq.sv:582) [uvm_test_top.env.virtual_sequencer.otbn_multi_vseq] Check failed (!(cfg.loop_vif.loop_stack_push || cfg.loop_vif.loop_stack_pop) &&                       !cfg.trace_vif.locking_o)
  UVM_INFO @ 307994977 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Exit reason: Error: User command failed UVM_FATAL (otbn_model_if.sv:77) [otbn_model_if] Check failed (u_model.otbn_model_step_crc(handle, item, crc_state) == *) Failed to update CRC has 1 failures:
- Test otbn_escalate has 1 failures.
  - 3.otbn_escalate.3823709275
    Line 262, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/3.otbn_escalate/latest/run.log
```
  UVM_FATAL @  11139784 ps: (otbn_model_if.sv:77) [otbn_model_if] Check failed (u_model.otbn_model_step_crc(handle, item, crc_state) == 0) Failed to update CRC
  UVM_INFO @  11139784 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
Exit reason: Error: User command failed UVM_FATAL (otbn_base_vseq.sv:582) [otbn_imem_err_vseq] Check failed (!(cfg.loop_vif.loop_stack_push || cfg.loop_vif.loop_stack_pop) && !cfg.trace_vif.locking_o) has 1 failures:
- Test otbn_stress_all_with_rand_reset has 1 failures.
  - 3.otbn_stress_all_with_rand_reset.2327621060
    Line 303, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/3.otbn_stress_all_with_rand_reset/latest/run.log
```
  UVM_FATAL @ 892586090 ps: (otbn_base_vseq.sv:582) [uvm_test_top.env.virtual_sequencer.otbn_imem_err_vseq] Check failed (!(cfg.loop_vif.loop_stack_push || cfg.loop_vif.loop_stack_pop) &&                       !cfg.trace_vif.locking_o)
  UVM_INFO @ 892586090 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_FATAL (otbn_base_vseq.sv:369) [otbn_imem_err_vseq] Check failed (cfg.model_agent_cfg.vif.status == otbn_pkg::StatusIdle) Timed out waiting for OTBN to be idle before execution has 1 failures:
- Test otbn_imem_err has 1 failures.
  - 4.otbn_imem_err.1616237938
    Line 243, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/4.otbn_imem_err/latest/run.log
```
  UVM_FATAL @  19930098 ps: (otbn_base_vseq.sv:369) [uvm_test_top.env.virtual_sequencer.otbn_imem_err_vseq] Check failed (cfg.model_agent_cfg.vif.status == otbn_pkg::StatusIdle) Timed out waiting for OTBN to be idle before execution
  UVM_INFO @  19930098 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_FATAL (otbn_base_vseq.sv:369) [otbn_dmem_err_vseq] Check failed (cfg.model_agent_cfg.vif.status == otbn_pkg::StatusIdle) Timed out waiting for OTBN to be idle before execution has 1 failures:
- Test otbn_dmem_err has 1 failures.
  - 4.otbn_dmem_err.3741533113
    Line 242, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/4.otbn_dmem_err/latest/run.log
```
  UVM_FATAL @  29829349 ps: (otbn_base_vseq.sv:369) [uvm_test_top.env.virtual_sequencer.otbn_dmem_err_vseq] Check failed (cfg.model_agent_cfg.vif.status == otbn_pkg::StatusIdle) Timed out waiting for OTBN to be idle before execution
  UVM_INFO @  29829349 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
Exit reason: Error: User command failed UVM_FATAL (otbn_base_vseq.sv:582) [otbn_dmem_err_vseq] Check failed (!(cfg.loop_vif.loop_stack_push || cfg.loop_vif.loop_stack_pop) && !cfg.trace_vif.locking_o) has 1 failures:
- Test otbn_stress_all has 1 failures.
  - 4.otbn_stress_all.1779509598
    Line 279, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/4.otbn_stress_all/latest/run.log
```
  UVM_FATAL @  73990685 ps: (otbn_base_vseq.sv:582) [uvm_test_top.env.virtual_sequencer.otbn_dmem_err_vseq] Check failed (!(cfg.loop_vif.loop_stack_push || cfg.loop_vif.loop_stack_pop) &&                       !cfg.trace_vif.locking_o)
  UVM_INFO @  73990685 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```

Exit reason: Error: User command failed xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_otbn_*/rtl/otbn.sv,1230): Assertion InitSecWipeNonZeroBaseRegs_A has failed has 1 failures:

Test otbn_zero_state_err_urnd has 1 failures.

4.otbn_zero_state_err_urnd.1932026380
Line 241, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/4.otbn_zero_state_err_urnd/latest/run.log

  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_otbn_0.1/rtl/otbn.sv,1230): (time 12331035 PS) Assertion tb.dut.gen_sec_wipe_gpr_asserts[17].InitSecWipeNonZeroBaseRegs_A has failed
  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_otbn_0.1/rtl/otbn.sv,1230): (time 12331035 PS) Assertion tb.dut.gen_sec_wipe_gpr_asserts[16].InitSecWipeNonZeroBaseRegs_A has failed
  UVM_ERROR @  12331035 ps: (otbn.sv:1230) [ASSERT FAILED] InitSecWipeNonZeroBaseRegs_A
  UVM_INFO @  12331035 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_FATAL (otbn_stack_addr_integ_chk_vseq.sv:72) [otbn_stack_addr_integ_chk_vseq] timeout occurred! has 1 failures:
- Test otbn_stack_addr_integ_chk has 1 failures.
  - 4.otbn_stack_addr_integ_chk.4133348405
    Line 241, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/4.otbn_stack_addr_integ_chk/latest/run.log
```
  UVM_FATAL @ 10015169868 ps: (otbn_stack_addr_integ_chk_vseq.sv:72) [uvm_test_top.env.virtual_sequencer.otbn_stack_addr_integ_chk_vseq] timeout occurred!
  UVM_INFO @ 10015169868 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_FATAL (otbn_scoreboard.sv:495) scoreboard [scoreboard] A fatal alert arrived * cycles ago and we still don't think it should have done. has 1 failures:
- Test otbn_ctrl_redun has 1 failures.
  - 7.otbn_ctrl_redun.1079814288
    Line 240, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/7.otbn_ctrl_redun/latest/run.log
```
  UVM_FATAL @  35225996 ps: (otbn_scoreboard.sv:495) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] A fatal alert arrived 400 cycles ago and we still don't think it should have done.
  UVM_INFO @  35225996 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_FATAL (otbn_base_vseq.sv:582) [otbn_dmem_err_vseq] Check failed (!(cfg.loop_vif.loop_stack_push || cfg.loop_vif.loop_stack_pop) && !cfg.trace_vif.locking_o) has 1 failures:
- Test otbn_stress_all has 1 failures.
  - 9.otbn_stress_all.2901284389
    Line 281, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/9.otbn_stress_all/latest/run.log
```
  UVM_FATAL @ 110216320 ps: (otbn_base_vseq.sv:582) [uvm_test_top.env.virtual_sequencer.otbn_dmem_err_vseq] Check failed (!(cfg.loop_vif.loop_stack_push || cfg.loop_vif.loop_stack_pop) &&                       !cfg.trace_vif.locking_o)
  UVM_INFO @ 110216320 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_FATAL (otbn_base_vseq.sv:582) [otbn_sw_errs_fatal_chk_vseq] Check failed (!(cfg.loop_vif.loop_stack_push || cfg.loop_vif.loop_stack_pop) && !cfg.trace_vif.locking_o) has 1 failures:
- Test otbn_sw_errs_fatal_chk has 1 failures.
  - 9.otbn_sw_errs_fatal_chk.2897734782
    Line 241, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/9.otbn_sw_errs_fatal_chk/latest/run.log
```
  UVM_FATAL @  90248375 ps: (otbn_base_vseq.sv:582) [uvm_test_top.env.virtual_sequencer.otbn_sw_errs_fatal_chk_vseq] Check failed (!(cfg.loop_vif.loop_stack_push || cfg.loop_vif.loop_stack_pop) &&                       !cfg.trace_vif.locking_o)
  UVM_INFO @  90248375 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
Exit reason: Error: User command failed UVM_FATAL (otbn_scoreboard.sv:538) scoreboard [scoreboard] We saw a STATUS change * cycles ago that implied we'd get a fatal alert but it still hasn't arrived. has 1 failures:
- Test otbn_escalate has 1 failures.
  - 23.otbn_escalate.526377249
    Line 241, in log /container/opentitan-public/scratch/os_regression/otbn-sim-xcelium/23.otbn_escalate/latest/run.log
```
  UVM_FATAL @  20698069 ps: (otbn_scoreboard.sv:538) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] We saw a STATUS change 30 cycles ago that implied we'd get a fatal alert but it still hasn't arrived.
  UVM_INFO @  20698069 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```