AES/UNMASKED Simulation Results

Wednesday May 17 2023 07:05:42 UTC

GitHub Revision: `3df77bec1`

Branch: os_regression

Testplan

Simulator: XCELIUM

Build randomization enabled with --build-seed 2320738200

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	wake_up	aes_wake_up	3.000s	67.329us	1	1	100.00
V1	smoke	aes_smoke	5.000s	55.124us	50	50	100.00
V1	csr_hw_reset	aes_csr_hw_reset	3.000s	70.052us	5	5	100.00
V1	csr_rw	aes_csr_rw	4.000s	55.080us	20	20	100.00
V1	csr_bit_bash	aes_csr_bit_bash	8.000s	318.705us	5	5	100.00
V1	csr_aliasing	aes_csr_aliasing	4.000s	384.181us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	aes_csr_mem_rw_with_rand_reset	4.000s	79.625us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	aes_csr_rw	4.000s	55.080us	20	20	100.00
		aes_csr_aliasing	4.000s	384.181us	5	5	100.00
V1		TOTAL			106	106	100.00
V2	algorithm	aes_smoke	5.000s	55.124us	50	50	100.00
		aes_config_error	10.000s	386.913us	50	50	100.00
		aes_stress	39.000s	592.395us	50	50	100.00
V2	key_length	aes_smoke	5.000s	55.124us	50	50	100.00
		aes_config_error	10.000s	386.913us	50	50	100.00
		aes_stress	39.000s	592.395us	50	50	100.00
V2	back2back	aes_stress	39.000s	592.395us	50	50	100.00
		aes_b2b	11.000s	130.793us	50	50	100.00
V2	backpressure	aes_stress	39.000s	592.395us	50	50	100.00
V2	multi_message	aes_smoke	5.000s	55.124us	50	50	100.00
		aes_config_error	10.000s	386.913us	50	50	100.00
		aes_stress	39.000s	592.395us	50	50	100.00
		aes_alert_reset	6.000s	96.004us	50	50	100.00
V2	failure_test	aes_config_error	10.000s	386.913us	50	50	100.00
		aes_alert_reset	6.000s	96.004us	50	50	100.00
		aes_man_cfg_err	4.000s	62.411us	50	50	100.00
V2	trigger_clear_test	aes_clear	7.000s	218.267us	50	50	100.00
V2	nist_test_vectors	aes_nist_vectors	7.000s	271.925us	1	1	100.00
V2	reset_recovery	aes_alert_reset	6.000s	96.004us	50	50	100.00
V2	stress	aes_stress	39.000s	592.395us	50	50	100.00
V2	sideload	aes_stress	39.000s	592.395us	50	50	100.00
		aes_sideload	5.000s	97.825us	50	50	100.00
V2	deinitialization	aes_deinit	6.000s	181.782us	50	50	100.00
V2	alert_test	aes_alert_test	4.000s	69.138us	50	50	100.00
V2	tl_d_oob_addr_access	aes_tl_errors	6.000s	173.089us	20	20	100.00
V2	tl_d_illegal_access	aes_tl_errors	6.000s	173.089us	20	20	100.00
V2	tl_d_outstanding_access	aes_csr_hw_reset	3.000s	70.052us	5	5	100.00
		aes_csr_rw	4.000s	55.080us	20	20	100.00
		aes_csr_aliasing	4.000s	384.181us	5	5	100.00
		aes_same_csr_outstanding	4.000s	218.050us	20	20	100.00
V2	tl_d_partial_access	aes_csr_hw_reset	3.000s	70.052us	5	5	100.00
		aes_csr_rw	4.000s	55.080us	20	20	100.00
		aes_csr_aliasing	4.000s	384.181us	5	5	100.00
		aes_same_csr_outstanding	4.000s	218.050us	20	20	100.00
V2		TOTAL			491	491	100.00
V2S	reseeding	aes_reseed	53.000s	4.130ms	48	50	96.00
V2S	fault_inject	aes_fi	5.000s	261.660us	49	50	98.00
		aes_control_fi	48.000s	16.110ms	276	300	92.00
		aes_cipher_fi	49.000s	32.845ms	317	350	90.57
V2S	shadow_reg_update_error	aes_shadow_reg_errors	4.000s	350.018us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	aes_shadow_reg_errors	4.000s	350.018us	20	20	100.00
V2S	shadow_reg_storage_error	aes_shadow_reg_errors	4.000s	350.018us	20	20	100.00
V2S	shadowed_reset_glitch	aes_shadow_reg_errors	4.000s	350.018us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	aes_shadow_reg_errors_with_csr_rw	5.000s	116.405us	20	20	100.00
V2S	tl_intg_err	aes_sec_cm	7.000s	1.026ms	5	5	100.00
		aes_tl_intg_err	5.000s	413.481us	20	20	100.00
V2S	sec_cm_bus_integrity	aes_tl_intg_err	5.000s	413.481us	20	20	100.00
V2S	sec_cm_lc_escalate_en_intersig_mubi	aes_alert_reset	6.000s	96.004us	50	50	100.00
V2S	sec_cm_main_config_shadow	aes_shadow_reg_errors	4.000s	350.018us	20	20	100.00
V2S	sec_cm_main_config_sparse	aes_smoke	5.000s	55.124us	50	50	100.00
		aes_stress	39.000s	592.395us	50	50	100.00
		aes_alert_reset	6.000s	96.004us	50	50	100.00
		aes_core_fi	15.000s	10.010ms	67	70	95.71
V2S	sec_cm_aux_config_shadow	aes_shadow_reg_errors	4.000s	350.018us	20	20	100.00
V2S	sec_cm_aux_config_regwen	aes_stress	39.000s	592.395us	50	50	100.00
		aes_readability	4.000s	92.484us	50	50	100.00
V2S	sec_cm_key_sideload	aes_stress	39.000s	592.395us	50	50	100.00
		aes_sideload	5.000s	97.825us	50	50	100.00
V2S	sec_cm_key_sw_unreadable	aes_readability	4.000s	92.484us	50	50	100.00
V2S	sec_cm_data_reg_sw_unreadable	aes_readability	4.000s	92.484us	50	50	100.00
V2S	sec_cm_key_sec_wipe	aes_readability	4.000s	92.484us	50	50	100.00
V2S	sec_cm_iv_config_sec_wipe	aes_readability	4.000s	92.484us	50	50	100.00
V2S	sec_cm_data_reg_sec_wipe	aes_readability	4.000s	92.484us	50	50	100.00
V2S	sec_cm_data_reg_key_sca	aes_stress	39.000s	592.395us	50	50	100.00
V2S	sec_cm_key_masking	aes_stress	39.000s	592.395us	50	50	100.00
V2S	sec_cm_main_fsm_sparse	aes_fi	5.000s	261.660us	49	50	98.00
V2S	sec_cm_main_fsm_redun	aes_fi	5.000s	261.660us	49	50	98.00
		aes_control_fi	48.000s	16.110ms	276	300	92.00
		aes_cipher_fi	49.000s	32.845ms	317	350	90.57
		aes_ctr_fi	4.000s	99.760us	50	50	100.00
V2S	sec_cm_cipher_fsm_sparse	aes_fi	5.000s	261.660us	49	50	98.00
V2S	sec_cm_cipher_fsm_redun	aes_fi	5.000s	261.660us	49	50	98.00
		aes_control_fi	48.000s	16.110ms	276	300	92.00
		aes_cipher_fi	49.000s	32.845ms	317	350	90.57
V2S	sec_cm_cipher_ctr_redun	aes_cipher_fi	49.000s	32.845ms	317	350	90.57
V2S	sec_cm_ctr_fsm_sparse	aes_fi	5.000s	261.660us	49	50	98.00
V2S	sec_cm_ctr_fsm_redun	aes_fi	5.000s	261.660us	49	50	98.00
		aes_control_fi	48.000s	16.110ms	276	300	92.00
		aes_ctr_fi	4.000s	99.760us	50	50	100.00
V2S	sec_cm_ctrl_sparse	aes_fi	5.000s	261.660us	49	50	98.00
		aes_control_fi	48.000s	16.110ms	276	300	92.00
		aes_cipher_fi	49.000s	32.845ms	317	350	90.57
		aes_ctr_fi	4.000s	99.760us	50	50	100.00
V2S	sec_cm_main_fsm_global_esc	aes_alert_reset	6.000s	96.004us	50	50	100.00
V2S	sec_cm_main_fsm_local_esc	aes_fi	5.000s	261.660us	49	50	98.00
		aes_control_fi	48.000s	16.110ms	276	300	92.00
		aes_cipher_fi	49.000s	32.845ms	317	350	90.57
		aes_ctr_fi	4.000s	99.760us	50	50	100.00
V2S	sec_cm_cipher_fsm_local_esc	aes_fi	5.000s	261.660us	49	50	98.00
		aes_control_fi	48.000s	16.110ms	276	300	92.00
		aes_cipher_fi	49.000s	32.845ms	317	350	90.57
		aes_ctr_fi	4.000s	99.760us	50	50	100.00
V2S	sec_cm_ctr_fsm_local_esc	aes_fi	5.000s	261.660us	49	50	98.00
		aes_control_fi	48.000s	16.110ms	276	300	92.00
		aes_ctr_fi	4.000s	99.760us	50	50	100.00
V2S	sec_cm_data_reg_local_esc	aes_fi	5.000s	261.660us	49	50	98.00
		aes_control_fi	48.000s	16.110ms	276	300	92.00
		aes_cipher_fi	49.000s	32.845ms	317	350	90.57
V2S		TOTAL			922	985	93.60
V3		TOTAL			0	0	--
		TOTAL			1519	1582	96.02

Testplan Progress

Items	Total	Written	Passing	Progress
V1	7	7	7	100.00
V2	12	12	12	100.00
V2S	11	11	6	54.55

Coverage Results

Coverage Dashboard

Score	Block	Branch	Statement	Expression	Toggle	Fsm	Assertion	CoverGroup
97.14	97.54	94.43	98.81	93.63	97.64	91.85	98.26	92.90

Failure Buckets

Job aes_unmasked-sim-xcelium_run_default killed due to: Exit reason: User job exceeded runlimit: User job timed out has 33 failures:
- Test aes_control_fi has 17 failures.
  - 1.aes_control_fi.1401371500
    Log /container/opentitan-public/scratch/os_regression/aes_unmasked-sim-xcelium/1.aes_control_fi/latest/run.log
```
  Job ID: smart:9f7945e0-08ce-4960-9799-16e2f72bbaf2
```
  - 37.aes_control_fi.4154357259
    Log /container/opentitan-public/scratch/os_regression/aes_unmasked-sim-xcelium/37.aes_control_fi/latest/run.log
```
  Job ID: smart:5c9318c4-578d-429b-9f5d-587ef99fa61f
```
  - ... and 15 more failures.
- Test aes_cipher_fi has 16 failures.
  - 6.aes_cipher_fi.3983133054
    Log /container/opentitan-public/scratch/os_regression/aes_unmasked-sim-xcelium/6.aes_cipher_fi/latest/run.log
```
  Job ID: smart:e30ab956-3bde-4743-b2a9-a83f30142d1b
```
  - 9.aes_cipher_fi.2770828245
    Log /container/opentitan-public/scratch/os_regression/aes_unmasked-sim-xcelium/9.aes_cipher_fi/latest/run.log
```
  Job ID: smart:1236a012-23a0-4c68-8fac-d295c546b85a
```
  - ... and 14 more failures.

UVM_FATAL (aes_cipher_fi_vseq.sv:62) [aes_cipher_fi_vseq] wait timeout occurred! has 17 failures:

Test aes_cipher_fi has 17 failures.

2.aes_cipher_fi.1921066353
Line 269, in log /container/opentitan-public/scratch/os_regression/aes_unmasked-sim-xcelium/2.aes_cipher_fi/latest/run.log

  UVM_FATAL @ 10002585079 ps: (aes_cipher_fi_vseq.sv:62) [uvm_test_top.env.virtual_sequencer.aes_cipher_fi_vseq] wait timeout occurred!
  UVM_INFO @ 10002585079 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

52.aes_cipher_fi.2643872004
Line 280, in log /container/opentitan-public/scratch/os_regression/aes_unmasked-sim-xcelium/52.aes_cipher_fi/latest/run.log

  UVM_FATAL @ 10006880656 ps: (aes_cipher_fi_vseq.sv:62) [uvm_test_top.env.virtual_sequencer.aes_cipher_fi_vseq] wait timeout occurred!
  UVM_INFO @ 10006880656 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 15 more failures.

UVM_FATAL (aes_control_fi_vseq.sv:62) [aes_control_fi_vseq] wait timeout occurred! has 7 failures:

Test aes_control_fi has 7 failures.

17.aes_control_fi.4160203486
Line 284, in log /container/opentitan-public/scratch/os_regression/aes_unmasked-sim-xcelium/17.aes_control_fi/latest/run.log

  UVM_FATAL @ 10015427646 ps: (aes_control_fi_vseq.sv:62) [uvm_test_top.env.virtual_sequencer.aes_control_fi_vseq] wait timeout occurred!
  UVM_INFO @ 10015427646 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

40.aes_control_fi.705097800
Line 269, in log /container/opentitan-public/scratch/os_regression/aes_unmasked-sim-xcelium/40.aes_control_fi/latest/run.log

  UVM_FATAL @ 10003603399 ps: (aes_control_fi_vseq.sv:62) [uvm_test_top.env.virtual_sequencer.aes_control_fi_vseq] wait timeout occurred!
  UVM_INFO @ 10003603399 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 5 more failures.

UVM_FATAL (aes_core_fi_vseq.sv:66) [aes_core_fi_vseq] wait timeout occurred! has 3 failures:

Test aes_core_fi has 3 failures.

10.aes_core_fi.86582183
Line 271, in log /container/opentitan-public/scratch/os_regression/aes_unmasked-sim-xcelium/10.aes_core_fi/latest/run.log

  UVM_FATAL @ 10012076871 ps: (aes_core_fi_vseq.sv:66) [uvm_test_top.env.virtual_sequencer.aes_core_fi_vseq] wait timeout occurred!
  UVM_INFO @ 10012076871 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

12.aes_core_fi.2868614283
Line 269, in log /container/opentitan-public/scratch/os_regression/aes_unmasked-sim-xcelium/12.aes_core_fi/latest/run.log

  UVM_FATAL @ 10009577480 ps: (aes_core_fi_vseq.sv:66) [uvm_test_top.env.virtual_sequencer.aes_core_fi_vseq] wait timeout occurred!
  UVM_INFO @ 10009577480 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

... and 1 more failures.

UVM_FATAL (aes_scoreboard.sv:621) scoreboard [scoreboard] # * has 2 failures:

Test aes_reseed has 2 failures.

4.aes_reseed.1241869550
Line 5513, in log /container/opentitan-public/scratch/os_regression/aes_unmasked-sim-xcelium/4.aes_reseed/latest/run.log

  UVM_FATAL @  38597120 ps: (aes_scoreboard.sv:621) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard]  # 1
            TEST FAILED MESSAGES DID NOT MATCH
  
                 0 b4      5d      94      0
                 1 00      92      4a      0

46.aes_reseed.4166706876
Line 3501, in log /container/opentitan-public/scratch/os_regression/aes_unmasked-sim-xcelium/46.aes_reseed/latest/run.log

  UVM_FATAL @  16022008 ps: (aes_scoreboard.sv:621) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard]  # 0
            TEST FAILED MESSAGES DID NOT MATCH
  
                 0 d3      fd      7f      0
                 1 00      50      05      0

UVM_FATAL (aes_fi_vseq.sv:69) virtual_sequencer [aes_fi_vseq] Was Able to finish without clearing reset has 1 failures:

Test aes_fi has 1 failures.

7.aes_fi.2754103955
Line 5137, in log /container/opentitan-public/scratch/os_regression/aes_unmasked-sim-xcelium/7.aes_fi/latest/run.log

  UVM_FATAL @  69583964 ps: (aes_fi_vseq.sv:69) uvm_test_top.env.virtual_sequencer [uvm_test_top.env.virtual_sequencer.aes_fi_vseq] Was Able to finish without clearing reset
  UVM_INFO @  69583964 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---