AES/MASKED Simulation Results

Tuesday May 16 2023 07:02:31 UTC

GitHub Revision: `50278df8b`

Branch: os_regression

Testplan

Simulator: XCELIUM

Build randomization enabled with --build-seed 1341560578

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	wake_up	aes_wake_up	3.000s	62.040us	1	1	100.00
V1	smoke	aes_smoke	43.000s	1.314ms	50	50	100.00
V1	csr_hw_reset	aes_csr_hw_reset	3.000s	54.427us	5	5	100.00
V1	csr_rw	aes_csr_rw	4.000s	54.228us	20	20	100.00
V1	csr_bit_bash	aes_csr_bit_bash	11.000s	730.310us	5	5	100.00
V1	csr_aliasing	aes_csr_aliasing	6.000s	215.610us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	aes_csr_mem_rw_with_rand_reset	4.000s	99.872us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	aes_csr_rw	4.000s	54.228us	20	20	100.00
		aes_csr_aliasing	6.000s	215.610us	5	5	100.00
V1		TOTAL			106	106	100.00
V2	algorithm	aes_smoke	43.000s	1.314ms	50	50	100.00
		aes_config_error	47.000s	1.468ms	50	50	100.00
		aes_stress	2.200m	5.035ms	50	50	100.00
V2	key_length	aes_smoke	43.000s	1.314ms	50	50	100.00
		aes_config_error	47.000s	1.468ms	50	50	100.00
		aes_stress	2.200m	5.035ms	50	50	100.00
V2	back2back	aes_stress	2.200m	5.035ms	50	50	100.00
		aes_b2b	33.000s	337.377us	50	50	100.00
V2	backpressure	aes_stress	2.200m	5.035ms	50	50	100.00
V2	multi_message	aes_smoke	43.000s	1.314ms	50	50	100.00
		aes_config_error	47.000s	1.468ms	50	50	100.00
		aes_stress	2.200m	5.035ms	50	50	100.00
		aes_alert_reset	1.017m	3.223ms	50	50	100.00
V2	failure_test	aes_config_error	47.000s	1.468ms	50	50	100.00
		aes_alert_reset	1.017m	3.223ms	50	50	100.00
		aes_man_cfg_err	6.000s	145.700us	50	50	100.00
V2	trigger_clear_test	aes_clear	12.000s	173.264us	50	50	100.00
V2	nist_test_vectors	aes_nist_vectors	12.000s	416.642us	1	1	100.00
V2	reset_recovery	aes_alert_reset	1.017m	3.223ms	50	50	100.00
V2	stress	aes_stress	2.200m	5.035ms	50	50	100.00
V2	sideload	aes_stress	2.200m	5.035ms	50	50	100.00
		aes_sideload	13.000s	671.293us	50	50	100.00
V2	deinitialization	aes_deinit	9.000s	477.246us	50	50	100.00
V2	alert_test	aes_alert_test	5.000s	99.366us	50	50	100.00
V2	tl_d_oob_addr_access	aes_tl_errors	6.000s	278.919us	20	20	100.00
V2	tl_d_illegal_access	aes_tl_errors	6.000s	278.919us	20	20	100.00
V2	tl_d_outstanding_access	aes_csr_hw_reset	3.000s	54.427us	5	5	100.00
		aes_csr_rw	4.000s	54.228us	20	20	100.00
		aes_csr_aliasing	6.000s	215.610us	5	5	100.00
		aes_same_csr_outstanding	5.000s	382.111us	20	20	100.00
V2	tl_d_partial_access	aes_csr_hw_reset	3.000s	54.427us	5	5	100.00
		aes_csr_rw	4.000s	54.228us	20	20	100.00
		aes_csr_aliasing	6.000s	215.610us	5	5	100.00
		aes_same_csr_outstanding	5.000s	382.111us	20	20	100.00
V2		TOTAL			491	491	100.00
V2S	reseeding	aes_reseed	2.383m	4.849ms	50	50	100.00
V2S	fault_inject	aes_fi	19.000s	906.737us	50	50	100.00
		aes_control_fi	26.000s	10.047ms	284	300	94.67
		aes_cipher_fi	50.000s	10.004ms	333	350	95.14
V2S	shadow_reg_update_error	aes_shadow_reg_errors	5.000s	104.047us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	aes_shadow_reg_errors	5.000s	104.047us	20	20	100.00
V2S	shadow_reg_storage_error	aes_shadow_reg_errors	5.000s	104.047us	20	20	100.00
V2S	shadowed_reset_glitch	aes_shadow_reg_errors	5.000s	104.047us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	aes_shadow_reg_errors_with_csr_rw	5.000s	167.472us	20	20	100.00
V2S	tl_intg_err	aes_sec_cm	34.000s	3.722ms	5	5	100.00
		aes_tl_intg_err	6.000s	525.955us	20	20	100.00
V2S	sec_cm_bus_integrity	aes_tl_intg_err	6.000s	525.955us	20	20	100.00
V2S	sec_cm_lc_escalate_en_intersig_mubi	aes_alert_reset	1.017m	3.223ms	50	50	100.00
V2S	sec_cm_main_config_shadow	aes_shadow_reg_errors	5.000s	104.047us	20	20	100.00
V2S	sec_cm_main_config_sparse	aes_smoke	43.000s	1.314ms	50	50	100.00
		aes_stress	2.200m	5.035ms	50	50	100.00
		aes_alert_reset	1.017m	3.223ms	50	50	100.00
		aes_core_fi	3.083m	10.435ms	66	70	94.29
V2S	sec_cm_aux_config_shadow	aes_shadow_reg_errors	5.000s	104.047us	20	20	100.00
V2S	sec_cm_aux_config_regwen	aes_stress	2.200m	5.035ms	50	50	100.00
		aes_readability	6.000s	55.163us	50	50	100.00
V2S	sec_cm_key_sideload	aes_stress	2.200m	5.035ms	50	50	100.00
		aes_sideload	13.000s	671.293us	50	50	100.00
V2S	sec_cm_key_sw_unreadable	aes_readability	6.000s	55.163us	50	50	100.00
V2S	sec_cm_data_reg_sw_unreadable	aes_readability	6.000s	55.163us	50	50	100.00
V2S	sec_cm_key_sec_wipe	aes_readability	6.000s	55.163us	50	50	100.00
V2S	sec_cm_iv_config_sec_wipe	aes_readability	6.000s	55.163us	50	50	100.00
V2S	sec_cm_data_reg_sec_wipe	aes_readability	6.000s	55.163us	50	50	100.00
V2S	sec_cm_data_reg_key_sca	aes_stress	2.200m	5.035ms	50	50	100.00
V2S	sec_cm_key_masking	aes_stress	2.200m	5.035ms	50	50	100.00
V2S	sec_cm_main_fsm_sparse	aes_fi	19.000s	906.737us	50	50	100.00
V2S	sec_cm_main_fsm_redun	aes_fi	19.000s	906.737us	50	50	100.00
		aes_control_fi	26.000s	10.047ms	284	300	94.67
		aes_cipher_fi	50.000s	10.004ms	333	350	95.14
		aes_ctr_fi	5.000s	68.405us	50	50	100.00
V2S	sec_cm_cipher_fsm_sparse	aes_fi	19.000s	906.737us	50	50	100.00
V2S	sec_cm_cipher_fsm_redun	aes_fi	19.000s	906.737us	50	50	100.00
		aes_control_fi	26.000s	10.047ms	284	300	94.67
		aes_cipher_fi	50.000s	10.004ms	333	350	95.14
V2S	sec_cm_cipher_ctr_redun	aes_cipher_fi	50.000s	10.004ms	333	350	95.14
V2S	sec_cm_ctr_fsm_sparse	aes_fi	19.000s	906.737us	50	50	100.00
V2S	sec_cm_ctr_fsm_redun	aes_fi	19.000s	906.737us	50	50	100.00
		aes_control_fi	26.000s	10.047ms	284	300	94.67
		aes_ctr_fi	5.000s	68.405us	50	50	100.00
V2S	sec_cm_ctrl_sparse	aes_fi	19.000s	906.737us	50	50	100.00
		aes_control_fi	26.000s	10.047ms	284	300	94.67
		aes_cipher_fi	50.000s	10.004ms	333	350	95.14
		aes_ctr_fi	5.000s	68.405us	50	50	100.00
V2S	sec_cm_main_fsm_global_esc	aes_alert_reset	1.017m	3.223ms	50	50	100.00
V2S	sec_cm_main_fsm_local_esc	aes_fi	19.000s	906.737us	50	50	100.00
		aes_control_fi	26.000s	10.047ms	284	300	94.67
		aes_cipher_fi	50.000s	10.004ms	333	350	95.14
		aes_ctr_fi	5.000s	68.405us	50	50	100.00
V2S	sec_cm_cipher_fsm_local_esc	aes_fi	19.000s	906.737us	50	50	100.00
		aes_control_fi	26.000s	10.047ms	284	300	94.67
		aes_cipher_fi	50.000s	10.004ms	333	350	95.14
		aes_ctr_fi	5.000s	68.405us	50	50	100.00
V2S	sec_cm_ctr_fsm_local_esc	aes_fi	19.000s	906.737us	50	50	100.00
		aes_control_fi	26.000s	10.047ms	284	300	94.67
		aes_ctr_fi	5.000s	68.405us	50	50	100.00
V2S	sec_cm_data_reg_local_esc	aes_fi	19.000s	906.737us	50	50	100.00
		aes_control_fi	26.000s	10.047ms	284	300	94.67
		aes_cipher_fi	50.000s	10.004ms	333	350	95.14
V2S		TOTAL			948	985	96.24
V3		TOTAL			0	0	--
		TOTAL			1545	1582	97.66

Testplan Progress

Items	Total	Written	Passing	Progress
V1	7	7	7	100.00
V2	12	12	12	100.00
V2S	11	11	8	72.73

Coverage Results

Coverage Dashboard

Score	Block	Branch	Statement	Expression	Toggle	Fsm	Assertion	CoverGroup
98.44	98.99	97.49	99.37	95.89	95.60	97.78	98.82	92.29

Failure Buckets

Job aes_masked-sim-xcelium_run_default killed due to: Exit reason: User job exceeded runlimit: User job timed out has 23 failures:
- Test aes_cipher_fi has 9 failures.
  - 1.aes_cipher_fi.1396778062
    Log /container/opentitan-public/scratch/os_regression/aes_masked-sim-xcelium/1.aes_cipher_fi/latest/run.log
```
  Job ID: smart:230046cb-8444-4651-8c99-e9b837326395
```
  - 42.aes_cipher_fi.3906516170
    Log /container/opentitan-public/scratch/os_regression/aes_masked-sim-xcelium/42.aes_cipher_fi/latest/run.log
```
  Job ID: smart:e362bf51-a1d4-459a-aa79-002a0d4a0d62
```
  - ... and 7 more failures.
- Test aes_control_fi has 14 failures.
  - 4.aes_control_fi.164492873
    Log /container/opentitan-public/scratch/os_regression/aes_masked-sim-xcelium/4.aes_control_fi/latest/run.log
```
  Job ID: smart:1e0a264a-739e-47b2-ab7b-d81ccdb4d733
```
  - 15.aes_control_fi.4271883348
    Log /container/opentitan-public/scratch/os_regression/aes_masked-sim-xcelium/15.aes_control_fi/latest/run.log
```
  Job ID: smart:da056193-f8a9-4856-9250-884da5e11fa4
```
  - ... and 12 more failures.

UVM_FATAL (aes_cipher_fi_vseq.sv:62) [aes_cipher_fi_vseq] wait timeout occurred! has 8 failures:

Test aes_cipher_fi has 8 failures.

95.aes_cipher_fi.2697036208
Line 285, in log /container/opentitan-public/scratch/os_regression/aes_masked-sim-xcelium/95.aes_cipher_fi/latest/run.log

  UVM_FATAL @ 10010889539 ps: (aes_cipher_fi_vseq.sv:62) [uvm_test_top.env.virtual_sequencer.aes_cipher_fi_vseq] wait timeout occurred!
  UVM_INFO @ 10010889539 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

116.aes_cipher_fi.3473786850
Line 285, in log /container/opentitan-public/scratch/os_regression/aes_masked-sim-xcelium/116.aes_cipher_fi/latest/run.log

  UVM_FATAL @ 10004664270 ps: (aes_cipher_fi_vseq.sv:62) [uvm_test_top.env.virtual_sequencer.aes_cipher_fi_vseq] wait timeout occurred!
  UVM_INFO @ 10004664270 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 6 more failures.

UVM_FATAL (aes_core_fi_vseq.sv:66) [aes_core_fi_vseq] wait timeout occurred! has 4 failures:

Test aes_core_fi has 4 failures.

27.aes_core_fi.37821343
Line 276, in log /container/opentitan-public/scratch/os_regression/aes_masked-sim-xcelium/27.aes_core_fi/latest/run.log

  UVM_FATAL @ 10030419825 ps: (aes_core_fi_vseq.sv:66) [uvm_test_top.env.virtual_sequencer.aes_core_fi_vseq] wait timeout occurred!
  UVM_INFO @ 10030419825 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

43.aes_core_fi.144270955
Line 275, in log /container/opentitan-public/scratch/os_regression/aes_masked-sim-xcelium/43.aes_core_fi/latest/run.log

  UVM_FATAL @ 10015755807 ps: (aes_core_fi_vseq.sv:66) [uvm_test_top.env.virtual_sequencer.aes_core_fi_vseq] wait timeout occurred!
  UVM_INFO @ 10015755807 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 2 more failures.

UVM_FATAL (aes_control_fi_vseq.sv:62) [aes_control_fi_vseq] wait timeout occurred! has 2 failures:

Test aes_control_fi has 2 failures.

49.aes_control_fi.1693403867
Line 282, in log /container/opentitan-public/scratch/os_regression/aes_masked-sim-xcelium/49.aes_control_fi/latest/run.log

  UVM_FATAL @ 10155371368 ps: (aes_control_fi_vseq.sv:62) [uvm_test_top.env.virtual_sequencer.aes_control_fi_vseq] wait timeout occurred!
  UVM_INFO @ 10155371368 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

73.aes_control_fi.513950038
Line 277, in log /container/opentitan-public/scratch/os_regression/aes_masked-sim-xcelium/73.aes_control_fi/latest/run.log

  UVM_FATAL @ 10046649287 ps: (aes_control_fi_vseq.sv:62) [uvm_test_top.env.virtual_sequencer.aes_control_fi_vseq] wait timeout occurred!
  UVM_INFO @ 10046649287 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---