KEYMGR Simulation Results

Thursday April 18 2024 19:02:27 UTC

GitHub Revision: `d3942ca074`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 25859338206198790995583629940734127463564215244480240139741775999763579929205

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	keymgr_smoke	33.410s	1.622ms	49	50	98.00
V1	random	keymgr_random	1.552m	34.490ms	50	50	100.00
V1	csr_hw_reset	keymgr_csr_hw_reset	1.140s	207.351us	5	5	100.00
V1	csr_rw	keymgr_csr_rw	1.500s	149.661us	20	20	100.00
V1	csr_bit_bash	keymgr_csr_bit_bash	25.560s	903.611us	5	5	100.00
V1	csr_aliasing	keymgr_csr_aliasing	10.490s	453.607us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	keymgr_csr_mem_rw_with_rand_reset	2.350s	34.225us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	keymgr_csr_rw	1.500s	149.661us	20	20	100.00
		keymgr_csr_aliasing	10.490s	453.607us	5	5	100.00
V1		TOTAL			154	155	99.35
V2	cfgen_during_op	keymgr_cfg_regwen	1.983m	9.207ms	50	50	100.00
V2	sideload	keymgr_sideload	45.420s	9.999ms	50	50	100.00
		keymgr_sideload_kmac	40.800s	6.485ms	49	50	98.00
		keymgr_sideload_aes	33.690s	1.248ms	50	50	100.00
		keymgr_sideload_otbn	55.690s	7.709ms	50	50	100.00
V2	direct_to_disabled_state	keymgr_direct_to_disabled	16.020s	541.059us	50	50	100.00
V2	lc_disable	keymgr_lc_disable	9.600s	268.129us	49	50	98.00
V2	kmac_error_response	keymgr_kmac_rsp_err	1.471m	2.487ms	45	50	90.00
V2	invalid_sw_input	keymgr_sw_invalid_input	1.473m	7.728ms	50	50	100.00
V2	invalid_hw_input	keymgr_hwsw_invalid_input	1.821m	13.722ms	50	50	100.00
V2	sync_async_fault_cross	keymgr_sync_async_fault_cross	29.600s	3.751ms	49	50	98.00
V2	stress_all	keymgr_stress_all	12.826m	183.563ms	48	50	96.00
V2	intr_test	keymgr_intr_test	0.990s	24.585us	50	50	100.00
V2	alert_test	keymgr_alert_test	1.060s	21.322us	50	50	100.00
V2	tl_d_oob_addr_access	keymgr_tl_errors	3.760s	392.888us	20	20	100.00
V2	tl_d_illegal_access	keymgr_tl_errors	3.760s	392.888us	20	20	100.00
V2	tl_d_outstanding_access	keymgr_csr_hw_reset	1.140s	207.351us	5	5	100.00
		keymgr_csr_rw	1.500s	149.661us	20	20	100.00
		keymgr_csr_aliasing	10.490s	453.607us	5	5	100.00
		keymgr_same_csr_outstanding	2.910s	234.246us	20	20	100.00
V2	tl_d_partial_access	keymgr_csr_hw_reset	1.140s	207.351us	5	5	100.00
		keymgr_csr_rw	1.500s	149.661us	20	20	100.00
		keymgr_csr_aliasing	10.490s	453.607us	5	5	100.00
		keymgr_same_csr_outstanding	2.910s	234.246us	20	20	100.00
V2		TOTAL			730	740	98.65
V2S	sec_cm_additional_check	keymgr_sec_cm	2.139m	23.887ms	5	5	100.00
V2S	tl_intg_err	keymgr_sec_cm	2.139m	23.887ms	5	5	100.00
		keymgr_tl_intg_err	15.190s	563.608us	20	20	100.00
V2S	shadow_reg_update_error	keymgr_shadow_reg_errors	16.900s	654.658us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	keymgr_shadow_reg_errors	16.900s	654.658us	20	20	100.00
V2S	shadow_reg_storage_error	keymgr_shadow_reg_errors	16.900s	654.658us	20	20	100.00
V2S	shadowed_reset_glitch	keymgr_shadow_reg_errors	16.900s	654.658us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	keymgr_shadow_reg_errors_with_csr_rw	15.700s	6.814ms	20	20	100.00
V2S	prim_count_check	keymgr_sec_cm	2.139m	23.887ms	5	5	100.00
V2S	prim_fsm_check	keymgr_sec_cm	2.139m	23.887ms	5	5	100.00
V2S	sec_cm_bus_integrity	keymgr_tl_intg_err	15.190s	563.608us	20	20	100.00
V2S	sec_cm_config_shadow	keymgr_shadow_reg_errors	16.900s	654.658us	20	20	100.00
V2S	sec_cm_op_config_regwen	keymgr_cfg_regwen	1.983m	9.207ms	50	50	100.00
V2S	sec_cm_reseed_config_regwen	keymgr_random	1.552m	34.490ms	50	50	100.00
		keymgr_csr_rw	1.500s	149.661us	20	20	100.00
V2S	sec_cm_sw_binding_config_regwen	keymgr_random	1.552m	34.490ms	50	50	100.00
		keymgr_csr_rw	1.500s	149.661us	20	20	100.00
V2S	sec_cm_max_key_ver_config_regwen	keymgr_random	1.552m	34.490ms	50	50	100.00
		keymgr_csr_rw	1.500s	149.661us	20	20	100.00
V2S	sec_cm_lc_ctrl_intersig_mubi	keymgr_lc_disable	9.600s	268.129us	49	50	98.00
V2S	sec_cm_constants_consistency	keymgr_hwsw_invalid_input	1.821m	13.722ms	50	50	100.00
V2S	sec_cm_intersig_consistency	keymgr_hwsw_invalid_input	1.821m	13.722ms	50	50	100.00
V2S	sec_cm_hw_key_sw_noaccess	keymgr_random	1.552m	34.490ms	50	50	100.00
V2S	sec_cm_output_keys_ctrl_redun	keymgr_sideload_protect	18.740s	893.437us	50	50	100.00
V2S	sec_cm_ctrl_fsm_sparse	keymgr_sec_cm	2.139m	23.887ms	5	5	100.00
V2S	sec_cm_data_fsm_sparse	keymgr_sec_cm	2.139m	23.887ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_local_esc	keymgr_sec_cm	2.139m	23.887ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_consistency	keymgr_custom_cm	53.030s	5.437ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_global_esc	keymgr_lc_disable	9.600s	268.129us	49	50	98.00
V2S	sec_cm_ctrl_ctr_redun	keymgr_sec_cm	2.139m	23.887ms	5	5	100.00
V2S	sec_cm_kmac_if_fsm_sparse	keymgr_sec_cm	2.139m	23.887ms	5	5	100.00
V2S	sec_cm_kmac_if_ctr_redun	keymgr_sec_cm	2.139m	23.887ms	5	5	100.00
V2S	sec_cm_kmac_if_cmd_ctrl_consistency	keymgr_custom_cm	53.030s	5.437ms	50	50	100.00
V2S	sec_cm_kmac_if_done_ctrl_consistency	keymgr_custom_cm	53.030s	5.437ms	50	50	100.00
V2S	sec_cm_reseed_ctr_redun	keymgr_sec_cm	2.139m	23.887ms	5	5	100.00
V2S	sec_cm_side_load_sel_ctrl_consistency	keymgr_custom_cm	53.030s	5.437ms	50	50	100.00
V2S	sec_cm_sideload_ctrl_fsm_sparse	keymgr_sec_cm	2.139m	23.887ms	5	5	100.00
V2S	sec_cm_ctrl_key_integrity	keymgr_custom_cm	53.030s	5.437ms	50	50	100.00
V2S		TOTAL			165	165	100.00
V3	stress_all_with_rand_reset	keymgr_stress_all_with_rand_reset	33.730s	2.841ms	26	50	52.00
V3		TOTAL			26	50	52.00
		TOTAL			1075	1110	96.85

Testplan Progress

Items	Total	Written	Passing	Progress
V1	7	7	6	85.71
V2	16	16	11	68.75
V2S	6	6	6	100.00
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
97.81	99.07	98.18	98.33	100.00	99.11	98.41	91.58

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:829) [keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 16 failures:
- Test keymgr_stress_all_with_rand_reset has 16 failures.
  - 4.keymgr_stress_all_with_rand_reset.71737496100071451947873695006572009100263509827346450853402162346221350461536
    Line 802, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/4.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 237760598 ps: (cip_base_vseq.sv:829) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 237760598 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 7.keymgr_stress_all_with_rand_reset.29766493542658251968135055064760747985829072831086199058700018939122187103132
    Line 716, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/7.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 131720946 ps: (cip_base_vseq.sv:829) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10001 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 131720946 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 14 more failures.
UVM_ERROR (keymgr_scoreboard.sv:736) [scoreboard] Check failed item.d_data !=gmv(csr) (* [] vs * [])` has 6 failures:
- Test keymgr_stress_all_with_rand_reset has 6 failures.
  - 19.keymgr_stress_all_with_rand_reset.9057015625096138804180765856856868468505615446519613710936089437463865172726
    Line 1954, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/19.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 445766115 ps: (keymgr_scoreboard.sv:736) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (3768821466 [0xe0a3a6da] vs 3768821466 [0xe0a3a6da])
  UVM_INFO @ 445766115 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 20.keymgr_stress_all_with_rand_reset.65847941637672744243149020379562568238942869376176159785365345265784416609370
    Line 1053, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/20.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 1583287825 ps: (keymgr_scoreboard.sv:736) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (3474149178 [0xcf134f3a] vs 3474149178 [0xcf134f3a])
  UVM_INFO @ 1583287825 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 4 more failures.

UVM_ERROR (keymgr_scoreboard.sv:1013) [scoreboard] Check failed act != exp (* [*] vs * [*]) cdi_type: Attestation has 5 failures:

Test keymgr_kmac_rsp_err has 5 failures.

4.keymgr_kmac_rsp_err.76744405132971433207858249672030687424177754977738337290125687625566425085543
Line 420, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/4.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @ 228457607 ps: (keymgr_scoreboard.sv:1013) [uvm_test_top.env.scoreboard] Check failed act != exp (1296527165279679267334116419099986222478055895839056782476369790989935869575012281840281462417330287367136053448516421488846327370768004629217093737117272769467565822880902795338306865112186628270664767423485537088098008520973489079289861398962572622016458644263439055583265691075365851035740391284952448282988337196798967794399814198308371403982025141074527022581435628453557234477759032927776573233231443838964369253129852 [0x2edbc1c50000000000000000de4121157c534b2605677934b7b1182b000000003a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f909387d2eb47eaef1638784610b7b52ce85cc8cb74bd0d5058acb861ae4f63368b59574e20081ce2141f463cfbee927e06c4dbbac7cce0fed7746b8536ba444f7ba027426ab608db53f4bb602441b0d34e20170e17ff7b5866fe31aa87fb7d77e142da49ccbaec441e47e5f7775f6de7c] vs 1296527165279679267334116419099986222478055895839056782476369790989935869575012281840281462417330287367136053448516421488846327370768004629217093737117272769467565822880902795338306865112186628270664767423485537088098008520973489079289861398962572622016458644263439055583265691075365851035740391284952448282988337196798967794399814198308371403982025141074527022581435628453557234477759032927776573233231443838964369253129852 [0x2edbc1c50000000000000000de4121157c534b2605677934b7b1182b000000003a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f909387d2eb47eaef1638784610b7b52ce85cc8cb74bd0d5058acb861ae4f63368b59574e20081ce2141f463cfbee927e06c4dbbac7cce0fed7746b8536ba444f7ba027426ab608db53f4bb602441b0d34e20170e17ff7b5866fe31aa87fb7d77e142da49ccbaec441e47e5f7775f6de7c]) cdi_type: Attestation
  
   DiversificationKey act: 0xe20170e17ff7b5866fe31aa87fb7d77e142da49ccbaec441e47e5f7775f6de7c, exp: 0xe20170e17ff7b5866fe31aa87fb7d77e142da49ccbaec441e47e5f7775f6de7c
   RomDigest act: 0x6c4dbbac7cce0fed7746b8536ba444f7ba027426ab608db53f4bb602441b0d34, exp: 0x6c4dbbac7cce0fed7746b8536ba444f7ba027426ab608db53f4bb602441b0d34
   HealthMeasurement act: 0xb59574e20081ce2141f463cfbee927e0, exp: 0xb59574e20081ce2141f463cfbee927e0

11.keymgr_kmac_rsp_err.74568218560662100788807486603793086510110766184435802034329013777975747448927
Line 370, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/11.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @  73147581 ps: (keymgr_scoreboard.sv:1013) [uvm_test_top.env.scoreboard] Check failed act != exp (245931741855230042465868446538266789388964370743673387774282730933423926545627375026712124746315286907369713202326740049626379979466781829621447100255318579173822767710439286423693672365023629403484814651388678014945389901355181468427365464714883893289914406699333786348348471867647622841799263054083914522580072062082022659591060138435431213164693880034297387239980656056371465997689749384318760955538296 [0xa3f5ffa62761c5e500000000000000009942b696000000003a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9000000000000000000000000000000000000000000000000000000000000f0f000000000000000005cfbd765ce33f34ea20a046cf42e6eac560a3f82bfa76285b5c1d4aea7c915e49a32d1c89be0f507fa365df52da48cd752fb3a026a8e608f0098cfe5fa9810494829d0cd9479eb78] vs 245931741855230042465868446538266789388964370743673387774282730933423926545627375026712124746315286907369713202326740049626379979466781829621447100255318579173822767710439286423693672365023629403484814651388678014945389901355181468427365464714883893289914406699333786348348471867647622841799263054083914522580072062082022659591060138435431213164693880034297387239980656056371465997689749384318760955538296 [0xa3f5ffa62761c5e500000000000000009942b696000000003a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9000000000000000000000000000000000000000000000000000000000000f0f000000000000000005cfbd765ce33f34ea20a046cf42e6eac560a3f82bfa76285b5c1d4aea7c915e49a32d1c89be0f507fa365df52da48cd752fb3a026a8e608f0098cfe5fa9810494829d0cd9479eb78]) cdi_type: Attestation
  
   DiversificationKey act: 0xfa365df52da48cd752fb3a026a8e608f0098cfe5fa9810494829d0cd9479eb78, exp: 0xfa365df52da48cd752fb3a026a8e608f0098cfe5fa9810494829d0cd9479eb78
   RomDigest act: 0xa20a046cf42e6eac560a3f82bfa76285b5c1d4aea7c915e49a32d1c89be0f507, exp: 0xa20a046cf42e6eac560a3f82bfa76285b5c1d4aea7c915e49a32d1c89be0f507
   HealthMeasurement act: 0x5cfbd765ce33f34e, exp: 0x5cfbd765ce33f34e

... and 3 more failures.

UVM_ERROR (cip_base_scoreboard.sv:301) scoreboard [scoreboard] alert recov_operation_err did not trigger max_delay:* has 5 failures:

Test keymgr_stress_all has 2 failures.

7.keymgr_stress_all.89644699945108261186104364876384083279602514761993658669676938608711702586852
Line 1423, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/7.keymgr_stress_all/latest/run.log

  UVM_ERROR @ 246580776 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @ 246580776 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

31.keymgr_stress_all.70440142572699265130735580235924704533757425871357398573649511558585538192417
Line 574, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/31.keymgr_stress_all/latest/run.log

  UVM_ERROR @  36461938 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @  36461938 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test keymgr_smoke has 1 failures.

24.keymgr_smoke.32384873268752867298332472859415330424590060203900509253968773317310742983926
Line 309, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/24.keymgr_smoke/latest/run.log

  UVM_ERROR @  21652549 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @  21652549 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test keymgr_sideload_kmac has 1 failures.

31.keymgr_sideload_kmac.106830054955020184450468289199646363218015249064754173551845109315811691636778
Line 263, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/31.keymgr_sideload_kmac/latest/run.log

  UVM_ERROR @   1598431 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @   1598431 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test keymgr_sync_async_fault_cross has 1 failures.
- 44.keymgr_sync_async_fault_cross.21705401824304229553828568507806793127835422768937104836891995926398301186010
  Line 278, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/44.keymgr_sync_async_fault_cross/latest/run.log
```
  UVM_ERROR @   5914968 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @   5914968 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```

UVM_ERROR (keymgr_scoreboard.sv:750) [scoreboard] Check failed item.d_data ==gmv(csr) (* [] vs * []) reg name: keymgr_reg_block.sw_share1_output_*` has 2 failures:
- Test keymgr_lc_disable has 1 failures.
  - 14.keymgr_lc_disable.24929582872147898667866573874495357633534688027876989492494540584968787668841
    Line 483, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/14.keymgr_lc_disable/latest/run.log
```
  UVM_ERROR @  35256821 ps: (keymgr_scoreboard.sv:750) [uvm_test_top.env.scoreboard] Check failed item.d_data == `gmv(csr) (4015955805 [0xef5e9f5d] vs 0 [0x0]) reg name: keymgr_reg_block.sw_share1_output_4
  UVM_INFO @  35256821 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test keymgr_stress_all_with_rand_reset has 1 failures.
  - 36.keymgr_stress_all_with_rand_reset.14591758020170575737559217646908113861629491682028742034005948857593086055704
    Line 749, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/36.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 141868599 ps: (keymgr_scoreboard.sv:750) [uvm_test_top.env.scoreboard] Check failed item.d_data == `gmv(csr) (1361141020 [0x51215d1c] vs 0 [0x0]) reg name: keymgr_reg_block.sw_share1_output_1
  UVM_INFO @ 141868599 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (keymgr_scoreboard.sv:674) [scoreboard] Check failed item.d_data == addr_phase_op_status (* [*] vs * [*]) has 1 failures:
- Test keymgr_stress_all_with_rand_reset has 1 failures.
  - 27.keymgr_stress_all_with_rand_reset.7564095543452637189910942451740464912710825032268270371167163520604696391607
    Line 387, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/27.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 798079662 ps: (keymgr_scoreboard.sv:674) [uvm_test_top.env.scoreboard] Check failed item.d_data == addr_phase_op_status (1 [0x1] vs 2 [0x2])
  UVM_INFO @ 798079662 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```