AES/MASKED Simulation Results

Thursday May 16 2024 19:02:11 UTC

GitHub Revision: `349bab6601`

Branch: os_regression

Simulator: XCELIUM

Build randomization enabled with --build-seed 60729333463373082946889975499553948547086354767408862399987151421185145065082

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	wake_up	aes_wake_up	3.000s	133.776us	1	1	100.00
V1	smoke	aes_smoke	14.000s	952.640us	50	50	100.00
V1	csr_hw_reset	aes_csr_hw_reset	4.000s	209.457us	5	5	100.00
V1	csr_rw	aes_csr_rw	9.000s	56.735us	20	20	100.00
V1	csr_bit_bash	aes_csr_bit_bash	10.000s	1.009ms	5	5	100.00
V1	csr_aliasing	aes_csr_aliasing	6.000s	865.638us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	aes_csr_mem_rw_with_rand_reset	13.000s	113.157us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	aes_csr_rw	9.000s	56.735us	20	20	100.00
		aes_csr_aliasing	6.000s	865.638us	5	5	100.00
V1		TOTAL			106	106	100.00
V2	algorithm	aes_smoke	14.000s	952.640us	50	50	100.00
		aes_config_error	20.000s	502.162us	50	50	100.00
		aes_stress	1.400m	2.744ms	50	50	100.00
V2	key_length	aes_smoke	14.000s	952.640us	50	50	100.00
		aes_config_error	20.000s	502.162us	50	50	100.00
		aes_stress	1.400m	2.744ms	50	50	100.00
V2	back2back	aes_stress	1.400m	2.744ms	50	50	100.00
		aes_b2b	1.050m	725.320us	50	50	100.00
V2	backpressure	aes_stress	1.400m	2.744ms	50	50	100.00
V2	multi_message	aes_smoke	14.000s	952.640us	50	50	100.00
		aes_config_error	20.000s	502.162us	50	50	100.00
		aes_stress	1.400m	2.744ms	50	50	100.00
		aes_alert_reset	1.017m	2.436ms	49	50	98.00
V2	failure_test	aes_man_cfg_err	15.000s	118.244us	50	50	100.00
		aes_config_error	20.000s	502.162us	50	50	100.00
		aes_alert_reset	1.017m	2.436ms	49	50	98.00
V2	trigger_clear_test	aes_clear	22.000s	553.810us	50	50	100.00
V2	nist_test_vectors	aes_nist_vectors	12.000s	180.253us	1	1	100.00
V2	reset_recovery	aes_alert_reset	1.017m	2.436ms	49	50	98.00
V2	stress	aes_stress	1.400m	2.744ms	50	50	100.00
V2	sideload	aes_stress	1.400m	2.744ms	50	50	100.00
		aes_sideload	26.000s	759.964us	50	50	100.00
V2	deinitialization	aes_deinit	24.000s	199.331us	50	50	100.00
V2	stress_all	aes_stress_all	4.167m	13.501ms	10	10	100.00
V2	alert_test	aes_alert_test	8.000s	66.969us	50	50	100.00
V2	tl_d_oob_addr_access	aes_tl_errors	9.000s	152.858us	20	20	100.00
V2	tl_d_illegal_access	aes_tl_errors	9.000s	152.858us	20	20	100.00
V2	tl_d_outstanding_access	aes_csr_hw_reset	4.000s	209.457us	5	5	100.00
		aes_csr_rw	9.000s	56.735us	20	20	100.00
		aes_csr_aliasing	6.000s	865.638us	5	5	100.00
		aes_same_csr_outstanding	14.000s	169.159us	20	20	100.00
V2	tl_d_partial_access	aes_csr_hw_reset	4.000s	209.457us	5	5	100.00
		aes_csr_rw	9.000s	56.735us	20	20	100.00
		aes_csr_aliasing	6.000s	865.638us	5	5	100.00
		aes_same_csr_outstanding	14.000s	169.159us	20	20	100.00
V2		TOTAL			500	501	99.80
V2S	reseeding	aes_reseed	51.000s	1.536ms	50	50	100.00
V2S	fault_inject	aes_fi	32.000s	2.154ms	50	50	100.00
		aes_control_fi	49.000s	10.011ms	286	300	95.33
		aes_cipher_fi	49.000s	10.008ms	340	350	97.14
V2S	shadow_reg_update_error	aes_shadow_reg_errors	13.000s	72.066us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	aes_shadow_reg_errors	13.000s	72.066us	20	20	100.00
V2S	shadow_reg_storage_error	aes_shadow_reg_errors	13.000s	72.066us	20	20	100.00
V2S	shadowed_reset_glitch	aes_shadow_reg_errors	13.000s	72.066us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	aes_shadow_reg_errors_with_csr_rw	9.000s	119.674us	20	20	100.00
V2S	tl_intg_err	aes_sec_cm	11.000s	974.354us	5	5	100.00
		aes_tl_intg_err	9.000s	290.967us	20	20	100.00
V2S	sec_cm_bus_integrity	aes_tl_intg_err	9.000s	290.967us	20	20	100.00
V2S	sec_cm_lc_escalate_en_intersig_mubi	aes_alert_reset	1.017m	2.436ms	49	50	98.00
V2S	sec_cm_main_config_shadow	aes_shadow_reg_errors	13.000s	72.066us	20	20	100.00
V2S	sec_cm_main_config_sparse	aes_smoke	14.000s	952.640us	50	50	100.00
		aes_stress	1.400m	2.744ms	50	50	100.00
		aes_alert_reset	1.017m	2.436ms	49	50	98.00
		aes_core_fi	1.300m	10.014ms	67	70	95.71
V2S	sec_cm_aux_config_shadow	aes_shadow_reg_errors	13.000s	72.066us	20	20	100.00
V2S	sec_cm_aux_config_regwen	aes_readability	14.000s	87.813us	50	50	100.00
		aes_stress	1.400m	2.744ms	50	50	100.00
V2S	sec_cm_key_sideload	aes_stress	1.400m	2.744ms	50	50	100.00
		aes_sideload	26.000s	759.964us	50	50	100.00
V2S	sec_cm_key_sw_unreadable	aes_readability	14.000s	87.813us	50	50	100.00
V2S	sec_cm_data_reg_sw_unreadable	aes_readability	14.000s	87.813us	50	50	100.00
V2S	sec_cm_key_sec_wipe	aes_readability	14.000s	87.813us	50	50	100.00
V2S	sec_cm_iv_config_sec_wipe	aes_readability	14.000s	87.813us	50	50	100.00
V2S	sec_cm_data_reg_sec_wipe	aes_readability	14.000s	87.813us	50	50	100.00
V2S	sec_cm_data_reg_key_sca	aes_stress	1.400m	2.744ms	50	50	100.00
V2S	sec_cm_key_masking	aes_stress	1.400m	2.744ms	50	50	100.00
V2S	sec_cm_main_fsm_sparse	aes_fi	32.000s	2.154ms	50	50	100.00
V2S	sec_cm_main_fsm_redun	aes_fi	32.000s	2.154ms	50	50	100.00
		aes_control_fi	49.000s	10.011ms	286	300	95.33
		aes_cipher_fi	49.000s	10.008ms	340	350	97.14
		aes_ctr_fi	8.000s	67.025us	50	50	100.00
V2S	sec_cm_cipher_fsm_sparse	aes_fi	32.000s	2.154ms	50	50	100.00
V2S	sec_cm_cipher_fsm_redun	aes_fi	32.000s	2.154ms	50	50	100.00
		aes_control_fi	49.000s	10.011ms	286	300	95.33
		aes_cipher_fi	49.000s	10.008ms	340	350	97.14
V2S	sec_cm_cipher_ctr_redun	aes_cipher_fi	49.000s	10.008ms	340	350	97.14
V2S	sec_cm_ctr_fsm_sparse	aes_fi	32.000s	2.154ms	50	50	100.00
V2S	sec_cm_ctr_fsm_redun	aes_fi	32.000s	2.154ms	50	50	100.00
		aes_control_fi	49.000s	10.011ms	286	300	95.33
		aes_ctr_fi	8.000s	67.025us	50	50	100.00
V2S	sec_cm_ctrl_sparse	aes_fi	32.000s	2.154ms	50	50	100.00
		aes_control_fi	49.000s	10.011ms	286	300	95.33
		aes_cipher_fi	49.000s	10.008ms	340	350	97.14
		aes_ctr_fi	8.000s	67.025us	50	50	100.00
V2S	sec_cm_main_fsm_global_esc	aes_alert_reset	1.017m	2.436ms	49	50	98.00
V2S	sec_cm_main_fsm_local_esc	aes_fi	32.000s	2.154ms	50	50	100.00
		aes_control_fi	49.000s	10.011ms	286	300	95.33
		aes_cipher_fi	49.000s	10.008ms	340	350	97.14
		aes_ctr_fi	8.000s	67.025us	50	50	100.00
V2S	sec_cm_cipher_fsm_local_esc	aes_fi	32.000s	2.154ms	50	50	100.00
		aes_control_fi	49.000s	10.011ms	286	300	95.33
		aes_cipher_fi	49.000s	10.008ms	340	350	97.14
		aes_ctr_fi	8.000s	67.025us	50	50	100.00
V2S	sec_cm_ctr_fsm_local_esc	aes_fi	32.000s	2.154ms	50	50	100.00
		aes_control_fi	49.000s	10.011ms	286	300	95.33
		aes_ctr_fi	8.000s	67.025us	50	50	100.00
V2S	sec_cm_data_reg_local_esc	aes_fi	32.000s	2.154ms	50	50	100.00
		aes_control_fi	49.000s	10.011ms	286	300	95.33
		aes_cipher_fi	49.000s	10.008ms	340	350	97.14
V2S		TOTAL			958	985	97.26
V3	stress_all_with_rand_reset	aes_stress_all_with_rand_reset	1.017m	1.656ms	0	10	0.00
V3		TOTAL			0	10	0.00
		TOTAL			1564	1602	97.63

Testplan Progress

Items	Total	Written	Passing	Progress
V1	7	7	7	100.00
V2	13	13	12	92.31
V2S	11	11	8	72.73
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

Score	Block	Branch	Statement	Expression	Toggle	Fsm	Assertion	CoverGroup
98.34	98.54	96.28	99.43	95.67	97.72	97.78	99.11	97.60

Failure Buckets

Job aes_masked-sim-xcelium_run_default killed due to: Exit reason: User job exceeded runlimit: User job timed out has 11 failures:
- Test aes_control_fi has 10 failures.
  - 3.aes_control_fi.45532757938331368642243408950850838813317560360408031647479050577865517303998
    Log /container/opentitan-public/scratch/os_regression/aes_masked-sim-xcelium/3.aes_control_fi/latest/run.log
```
  Job ID: smart:9d37fcbb-1d34-495e-9697-9b08efecb3ea
```
  - 23.aes_control_fi.53786579192202461393642044991069577368950886150376086239342837551488825906783
    Log /container/opentitan-public/scratch/os_regression/aes_masked-sim-xcelium/23.aes_control_fi/latest/run.log
```
  Job ID: smart:a936e7cc-50b9-4555-b543-b16c7e64b7e5
```
  - ... and 8 more failures.
- Test aes_cipher_fi has 1 failures.
  - 285.aes_cipher_fi.66715477772976713422384497825606029159802778275062878628300847987514118032325
    Log /container/opentitan-public/scratch/os_regression/aes_masked-sim-xcelium/285.aes_cipher_fi/latest/run.log
```
  Job ID: smart:f737f964-afa0-4bd9-aecf-6e54a6dfb4cd
```
UVM_FATAL (aes_cipher_fi_vseq.sv:62) [aes_cipher_fi_vseq] wait timeout occurred! has 9 failures:
- Test aes_cipher_fi has 9 failures.
  - 3.aes_cipher_fi.6838493518975541278116896324089889923774147927538528232247745770013832286794
    Line 318, in log /container/opentitan-public/scratch/os_regression/aes_masked-sim-xcelium/3.aes_cipher_fi/latest/run.log
```
  UVM_FATAL @ 10049409158 ps: (aes_cipher_fi_vseq.sv:62) [uvm_test_top.env.virtual_sequencer.aes_cipher_fi_vseq] wait timeout occurred!
  UVM_INFO @ 10049409158 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 72.aes_cipher_fi.87296013092679881416270623006530125029965989604465696619365209561443868885657
    Line 327, in log /container/opentitan-public/scratch/os_regression/aes_masked-sim-xcelium/72.aes_cipher_fi/latest/run.log
```
  UVM_FATAL @ 10011598886 ps: (aes_cipher_fi_vseq.sv:62) [uvm_test_top.env.virtual_sequencer.aes_cipher_fi_vseq] wait timeout occurred!
  UVM_INFO @ 10011598886 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 7 more failures.

UVM_ERROR (uvm_sequencer_base.svh:757) sequencer [SEQREQZMB] The task responsible for requesting a wait_for_grant on sequencer 'sequencer' for sequence 'sideload_seq' has been killed, to avoid a deadlock the sequence will be removed from the arbitration queues has 5 failures:

Test aes_stress_all_with_rand_reset has 5 failures.

0.aes_stress_all_with_rand_reset.111056902482605762810984607141605604208218889920452465225458676447353268046142
Line 708, in log /container/opentitan-public/scratch/os_regression/aes_masked-sim-xcelium/0.aes_stress_all_with_rand_reset/latest/run.log

  UVM_ERROR @ 259841636 ps: (uvm_sequencer_base.svh:757) uvm_test_top.env.keymgr_sideload_agent.sequencer [SEQREQZMB] The task responsible for requesting a wait_for_grant on sequencer 'uvm_test_top.env.keymgr_sideload_agent.sequencer' for sequence 'uvm_test_top.env.virtual_sequencer.aes_reseed_vseq.sideload_seq' has been killed, to avoid a deadlock the sequence will be removed from the arbitration queues
  UVM_INFO @ 259841636 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

1.aes_stress_all_with_rand_reset.18318229698208343616788418358009862304747737986250769504461618540460320754469
Line 1525, in log /container/opentitan-public/scratch/os_regression/aes_masked-sim-xcelium/1.aes_stress_all_with_rand_reset/latest/run.log

  UVM_ERROR @ 709833420 ps: (uvm_sequencer_base.svh:757) uvm_test_top.env.keymgr_sideload_agent.sequencer [SEQREQZMB] The task responsible for requesting a wait_for_grant on sequencer 'uvm_test_top.env.keymgr_sideload_agent.sequencer' for sequence 'uvm_test_top.env.virtual_sequencer.aes_reseed_vseq.sideload_seq' has been killed, to avoid a deadlock the sequence will be removed from the arbitration queues
  UVM_INFO @ 709833420 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 3 more failures.

UVM_FATAL (aes_control_fi_vseq.sv:62) [aes_control_fi_vseq] wait timeout occurred! has 4 failures:
- Test aes_control_fi has 4 failures.
  - 73.aes_control_fi.8725784706401641931937479051035671775888306009216191574925772092490318095699
    Line 322, in log /container/opentitan-public/scratch/os_regression/aes_masked-sim-xcelium/73.aes_control_fi/latest/run.log
```
  UVM_FATAL @ 10077608965 ps: (aes_control_fi_vseq.sv:62) [uvm_test_top.env.virtual_sequencer.aes_control_fi_vseq] wait timeout occurred!
  UVM_INFO @ 10077608965 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 192.aes_control_fi.36109801385297482784937624330019071866407512247898274973775643875855427290579
    Line 313, in log /container/opentitan-public/scratch/os_regression/aes_masked-sim-xcelium/192.aes_control_fi/latest/run.log
```
  UVM_FATAL @ 10009895175 ps: (aes_control_fi_vseq.sv:62) [uvm_test_top.env.virtual_sequencer.aes_control_fi_vseq] wait timeout occurred!
  UVM_INFO @ 10009895175 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 2 more failures.

Exit reason: Error: User command failed UVM_ERROR (uvm_sequencer_base.svh:757) sequencer [SEQREQZMB] The task responsible for requesting a wait_for_grant on sequencer 'sequencer' for sequence 'sideload_seq' has been killed, to avoid a deadlock the sequence will be removed from the arbitration queues has 3 failures:

Test aes_stress_all_with_rand_reset has 3 failures.

3.aes_stress_all_with_rand_reset.40702401708807472564908628670968235255576282532461064746230391196456096006167
Line 1625, in log /container/opentitan-public/scratch/os_regression/aes_masked-sim-xcelium/3.aes_stress_all_with_rand_reset/latest/run.log

  UVM_ERROR @ 1769997918 ps: (uvm_sequencer_base.svh:757) uvm_test_top.env.keymgr_sideload_agent.sequencer [SEQREQZMB] The task responsible for requesting a wait_for_grant on sequencer 'uvm_test_top.env.keymgr_sideload_agent.sequencer' for sequence 'uvm_test_top.env.virtual_sequencer.aes_stress_vseq.sideload_seq' has been killed, to avoid a deadlock the sequence will be removed from the arbitration queues
  UVM_INFO @ 1769997918 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

7.aes_stress_all_with_rand_reset.1827760655945291106224624672129331750895177679472112271524165287003396112213
Line 1049, in log /container/opentitan-public/scratch/os_regression/aes_masked-sim-xcelium/7.aes_stress_all_with_rand_reset/latest/run.log

  UVM_ERROR @ 1922912946 ps: (uvm_sequencer_base.svh:757) uvm_test_top.env.keymgr_sideload_agent.sequencer [SEQREQZMB] The task responsible for requesting a wait_for_grant on sequencer 'uvm_test_top.env.keymgr_sideload_agent.sequencer' for sequence 'uvm_test_top.env.virtual_sequencer.aes_stress_vseq.sideload_seq' has been killed, to avoid a deadlock the sequence will be removed from the arbitration queues
  UVM_INFO @ 1922912946 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 1 more failures.

UVM_FATAL (aes_core_fi_vseq.sv:66) [aes_core_fi_vseq] wait timeout occurred! has 3 failures:
- Test aes_core_fi has 3 failures.
  - 23.aes_core_fi.95680181279359018695662771746479339651164100664152333148261713127906101819099
    Line 315, in log /container/opentitan-public/scratch/os_regression/aes_masked-sim-xcelium/23.aes_core_fi/latest/run.log
```
  UVM_FATAL @ 10014153010 ps: (aes_core_fi_vseq.sv:66) [uvm_test_top.env.virtual_sequencer.aes_core_fi_vseq] wait timeout occurred!
  UVM_INFO @ 10014153010 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 52.aes_core_fi.37032838369938263102126646322213751675538188607047100629087818852020070893118
    Line 320, in log /container/opentitan-public/scratch/os_regression/aes_masked-sim-xcelium/52.aes_core_fi/latest/run.log
```
  UVM_FATAL @ 10008529436 ps: (aes_core_fi_vseq.sv:66) [uvm_test_top.env.virtual_sequencer.aes_core_fi_vseq] wait timeout occurred!
  UVM_INFO @ 10008529436 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 1 more failures.
Exit reason: Error: User command failed UVM_FATAL (aes_reseed_vseq.sv:40) [aes_reseed_vseq] Check failed request_seen | finished_all_msgs == *'b* (* [*] vs * [*]) has 2 failures:
- Test aes_stress_all_with_rand_reset has 2 failures.
  - 2.aes_stress_all_with_rand_reset.77684545039591871459703230806515599401537728697521465591344607757694204011462
    Line 1319, in log /container/opentitan-public/scratch/os_regression/aes_masked-sim-xcelium/2.aes_stress_all_with_rand_reset/latest/run.log
```
  UVM_FATAL @ 4646675484 ps: (aes_reseed_vseq.sv:40) [uvm_test_top.env.virtual_sequencer.aes_reseed_vseq] Check failed request_seen | finished_all_msgs == 1'b1 (0 [0x0] vs 1 [0x1])
  UVM_INFO @ 4646675484 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 4.aes_stress_all_with_rand_reset.86775654061756576634825988262478766588308804927417013471300195455686907688120
    Line 335, in log /container/opentitan-public/scratch/os_regression/aes_masked-sim-xcelium/4.aes_stress_all_with_rand_reset/latest/run.log
```
  UVM_FATAL @ 232113791 ps: (aes_reseed_vseq.sv:40) [uvm_test_top.env.virtual_sequencer.aes_reseed_vseq] Check failed request_seen | finished_all_msgs == 1'b1 (0 [0x0] vs 1 [0x1])
  UVM_INFO @ 232113791 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
Exit reason: Error: User command failed xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_aes_*/rtl/aes_core.sv,987): Assertion AesSecCmDataRegLocalEscDataOut has failed (* cycles, starting * PS) has 1 failures:
- Test aes_alert_reset has 1 failures.
  - 23.aes_alert_reset.34381222388223871061170849563048754034721193908402370434104477993438761615367
    Line 2801, in log /container/opentitan-public/scratch/os_regression/aes_masked-sim-xcelium/23.aes_alert_reset/latest/run.log
```
  xmsim: *E,ASRTST (/workspace/default/src/lowrisc_ip_aes_1.0/rtl/aes_core.sv,987): (time 13552129 PS) Assertion tb.dut.u_aes_core.AesSecCmDataRegLocalEscDataOut has failed (2 cycles, starting 13531721 PS)
  UVM_ERROR @  13552129 ps: (aes_core.sv:987) [ASSERT FAILED] AesSecCmDataRegLocalEscDataOut
  UVM_INFO @  13552129 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```