KEYMGR Simulation Results

Wednesday February 28 2024 23:53:28 UTC

GitHub Revision: `32ed2c4230`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 10708067410766204292161266966839433462058030635847883045650346145926493105783

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	keymgr_smoke	48.240s	4.913ms	49	50	98.00
V1	random	keymgr_random	1.542m	9.493ms	50	50	100.00
V1	csr_hw_reset	keymgr_csr_hw_reset	1.380s	105.970us	5	5	100.00
V1	csr_rw	keymgr_csr_rw	1.580s	59.952us	20	20	100.00
V1	csr_bit_bash	keymgr_csr_bit_bash	23.830s	865.846us	5	5	100.00
V1	csr_aliasing	keymgr_csr_aliasing	14.450s	859.411us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	keymgr_csr_mem_rw_with_rand_reset	2.670s	61.910us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	keymgr_csr_rw	1.580s	59.952us	20	20	100.00
		keymgr_csr_aliasing	14.450s	859.411us	5	5	100.00
V1		TOTAL			154	155	99.35
V2	cfgen_during_op	keymgr_cfg_regwen	18.360s	652.608us	50	50	100.00
V2	sideload	keymgr_sideload	47.420s	7.223ms	50	50	100.00
		keymgr_sideload_kmac	50.170s	1.516ms	50	50	100.00
		keymgr_sideload_aes	58.380s	7.681ms	50	50	100.00
		keymgr_sideload_otbn	1.063m	6.252ms	50	50	100.00
V2	direct_to_disabled_state	keymgr_direct_to_disabled	43.740s	2.264ms	50	50	100.00
V2	lc_disable	keymgr_lc_disable	1.281m	3.050ms	49	50	98.00
V2	kmac_error_response	keymgr_kmac_rsp_err	1.911m	5.435ms	47	50	94.00
V2	invalid_sw_input	keymgr_sw_invalid_input	1.309m	39.095ms	49	50	98.00
V2	invalid_hw_input	keymgr_hwsw_invalid_input	48.970s	9.359ms	50	50	100.00
V2	sync_async_fault_cross	keymgr_sync_async_fault_cross	14.420s	4.456ms	50	50	100.00
V2	stress_all	keymgr_stress_all	5.927m	14.856ms	48	50	96.00
V2	intr_test	keymgr_intr_test	0.920s	58.844us	50	50	100.00
V2	alert_test	keymgr_alert_test	1.130s	23.167us	50	50	100.00
V2	tl_d_oob_addr_access	keymgr_tl_errors	4.940s	538.559us	20	20	100.00
V2	tl_d_illegal_access	keymgr_tl_errors	4.940s	538.559us	20	20	100.00
V2	tl_d_outstanding_access	keymgr_csr_hw_reset	1.380s	105.970us	5	5	100.00
		keymgr_csr_rw	1.580s	59.952us	20	20	100.00
		keymgr_csr_aliasing	14.450s	859.411us	5	5	100.00
		keymgr_same_csr_outstanding	3.040s	440.221us	20	20	100.00
V2	tl_d_partial_access	keymgr_csr_hw_reset	1.380s	105.970us	5	5	100.00
		keymgr_csr_rw	1.580s	59.952us	20	20	100.00
		keymgr_csr_aliasing	14.450s	859.411us	5	5	100.00
		keymgr_same_csr_outstanding	3.040s	440.221us	20	20	100.00
V2		TOTAL			733	740	99.05
V2S	sec_cm_additional_check	keymgr_sec_cm	26.340s	4.774ms	5	5	100.00
V2S	tl_intg_err	keymgr_sec_cm	26.340s	4.774ms	5	5	100.00
		keymgr_tl_intg_err	20.090s	1.663ms	20	20	100.00
V2S	shadow_reg_update_error	keymgr_shadow_reg_errors	9.940s	484.303us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	keymgr_shadow_reg_errors	9.940s	484.303us	20	20	100.00
V2S	shadow_reg_storage_error	keymgr_shadow_reg_errors	9.940s	484.303us	20	20	100.00
V2S	shadowed_reset_glitch	keymgr_shadow_reg_errors	9.940s	484.303us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	keymgr_shadow_reg_errors_with_csr_rw	12.940s	8.487ms	20	20	100.00
V2S	prim_count_check	keymgr_sec_cm	26.340s	4.774ms	5	5	100.00
V2S	prim_fsm_check	keymgr_sec_cm	26.340s	4.774ms	5	5	100.00
V2S	sec_cm_bus_integrity	keymgr_tl_intg_err	20.090s	1.663ms	20	20	100.00
V2S	sec_cm_config_shadow	keymgr_shadow_reg_errors	9.940s	484.303us	20	20	100.00
V2S	sec_cm_op_config_regwen	keymgr_cfg_regwen	18.360s	652.608us	50	50	100.00
V2S	sec_cm_reseed_config_regwen	keymgr_random	1.542m	9.493ms	50	50	100.00
		keymgr_csr_rw	1.580s	59.952us	20	20	100.00
V2S	sec_cm_sw_binding_config_regwen	keymgr_random	1.542m	9.493ms	50	50	100.00
		keymgr_csr_rw	1.580s	59.952us	20	20	100.00
V2S	sec_cm_max_key_ver_config_regwen	keymgr_random	1.542m	9.493ms	50	50	100.00
		keymgr_csr_rw	1.580s	59.952us	20	20	100.00
V2S	sec_cm_lc_ctrl_intersig_mubi	keymgr_lc_disable	1.281m	3.050ms	49	50	98.00
V2S	sec_cm_constants_consistency	keymgr_hwsw_invalid_input	48.970s	9.359ms	50	50	100.00
V2S	sec_cm_intersig_consistency	keymgr_hwsw_invalid_input	48.970s	9.359ms	50	50	100.00
V2S	sec_cm_hw_key_sw_noaccess	keymgr_random	1.542m	9.493ms	50	50	100.00
V2S	sec_cm_output_keys_ctrl_redun	keymgr_sideload_protect	27.670s	1.494ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_sparse	keymgr_sec_cm	26.340s	4.774ms	5	5	100.00
V2S	sec_cm_data_fsm_sparse	keymgr_sec_cm	26.340s	4.774ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_local_esc	keymgr_sec_cm	26.340s	4.774ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_consistency	keymgr_custom_cm	24.410s	4.929ms	49	50	98.00
V2S	sec_cm_ctrl_fsm_global_esc	keymgr_lc_disable	1.281m	3.050ms	49	50	98.00
V2S	sec_cm_ctrl_ctr_redun	keymgr_sec_cm	26.340s	4.774ms	5	5	100.00
V2S	sec_cm_kmac_if_fsm_sparse	keymgr_sec_cm	26.340s	4.774ms	5	5	100.00
V2S	sec_cm_kmac_if_ctr_redun	keymgr_sec_cm	26.340s	4.774ms	5	5	100.00
V2S	sec_cm_kmac_if_cmd_ctrl_consistency	keymgr_custom_cm	24.410s	4.929ms	49	50	98.00
V2S	sec_cm_kmac_if_done_ctrl_consistency	keymgr_custom_cm	24.410s	4.929ms	49	50	98.00
V2S	sec_cm_reseed_ctr_redun	keymgr_sec_cm	26.340s	4.774ms	5	5	100.00
V2S	sec_cm_side_load_sel_ctrl_consistency	keymgr_custom_cm	24.410s	4.929ms	49	50	98.00
V2S	sec_cm_sideload_ctrl_fsm_sparse	keymgr_sec_cm	26.340s	4.774ms	5	5	100.00
V2S	sec_cm_ctrl_key_integrity	keymgr_custom_cm	24.410s	4.929ms	49	50	98.00
V2S		TOTAL			164	165	99.39
V3	stress_all_with_rand_reset	keymgr_stress_all_with_rand_reset	38.210s	1.674ms	21	50	42.00
V3		TOTAL			21	50	42.00
		TOTAL			1072	1110	96.58

Testplan Progress

Items	Total	Written	Passing	Progress
V1	7	7	6	85.71
V2	16	16	12	75.00
V2S	6	6	5	83.33
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
97.85	99.07	98.14	98.62	100.00	99.11	98.41	91.61

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:788) [keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 25 failures:
- Test keymgr_stress_all_with_rand_reset has 25 failures.
  - 3.keymgr_stress_all_with_rand_reset.104010718847493525001638623223684603120978360209747903406019388448650333629299
    Line 725, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/3.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 249235906 ps: (cip_base_vseq.sv:788) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 249235906 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 6.keymgr_stress_all_with_rand_reset.41083647485650237060671303339802839396045665743960984815623584266195631543931
    Line 293, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/6.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 968448662 ps: (cip_base_vseq.sv:788) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 968448662 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 23 more failures.

UVM_ERROR (cip_base_scoreboard.sv:287) scoreboard [scoreboard] alert recov_operation_err did not trigger max_delay:* has 5 failures:

Test keymgr_stress_all has 2 failures.

3.keymgr_stress_all.54895186067403744590400661662342536378822819949172382465718354837904077635340
Line 659, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/3.keymgr_stress_all/latest/run.log

  UVM_ERROR @ 465247713 ps: (cip_base_scoreboard.sv:287) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @ 465247713 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

33.keymgr_stress_all.50770506275920630321129021776106126535975022286548630294878699952002017837888
Line 2272, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/33.keymgr_stress_all/latest/run.log

  UVM_ERROR @ 1936839814 ps: (cip_base_scoreboard.sv:287) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @ 1936839814 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test keymgr_smoke has 1 failures.

14.keymgr_smoke.89052864603066840252754109484720842641033938358935433759494143361580764968877
Line 274, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/14.keymgr_smoke/latest/run.log

  UVM_ERROR @   6966504 ps: (cip_base_scoreboard.sv:287) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @   6966504 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test keymgr_sw_invalid_input has 1 failures.

19.keymgr_sw_invalid_input.7676131119006868701002107396873063303692727530007559987740624868812807128543
Line 442, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/19.keymgr_sw_invalid_input/latest/run.log

  UVM_ERROR @   8574874 ps: (cip_base_scoreboard.sv:287) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @   8574874 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test keymgr_custom_cm has 1 failures.

40.keymgr_custom_cm.53588209374902028545686128821335837158346767783457174829278213462742563016590
Line 436, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/40.keymgr_custom_cm/latest/run.log

  UVM_ERROR @  46688526 ps: (cip_base_scoreboard.sv:287) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @  46688526 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_ERROR (keymgr_scoreboard.sv:736) [scoreboard] Check failed item.d_data !=gmv(csr) (* [] vs * [])` has 3 failures:
- Test keymgr_stress_all_with_rand_reset has 3 failures.
  - 5.keymgr_stress_all_with_rand_reset.29146556395582165785256753453850795486729791978298857761207910273139981080168
    Line 3193, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/5.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 7328511697 ps: (keymgr_scoreboard.sv:736) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (2005461460 [0x7788e9d4] vs 2005461460 [0x7788e9d4])
  UVM_INFO @ 7328511697 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 24.keymgr_stress_all_with_rand_reset.8721128411927817927640735680538582644059351537606865393979216170847650700250
    Line 1813, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/24.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 460568771 ps: (keymgr_scoreboard.sv:736) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (2135507867 [0x7f49439b] vs 2135507867 [0x7f49439b])
  UVM_INFO @ 460568771 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 1 more failures.

UVM_ERROR (keymgr_scoreboard.sv:1013) [scoreboard] Check failed act != exp (* [*] vs * [*]) cdi_type: Attestation has 3 failures:

Test keymgr_kmac_rsp_err has 3 failures.

26.keymgr_kmac_rsp_err.56371829992449151345525902413628710673096891100140387073687360237542321865829
Line 503, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/26.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @  39141704 ps: (keymgr_scoreboard.sv:1013) [uvm_test_top.env.scoreboard] Check failed act != exp (181827500455998736810702955668246866123667144185569461398230915259537522277099174835083587562563974003980135201557682538618372029447176770743699296249348962529290445314977313069428292795552881744492306903665685883169633841495359589281323104579910516986114811031265795607492667789128307113410855020182087426590028559685783280976741670333985322620229750704070876263486607789796016522477783025824738220388195640566940677021506 [0x6924ef10000000000000000729977645c747b860000000096c376f0679a77da3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9f312f0d18d29aea64c1e56f37c8486094d9f4ece4b4469233b8206bf65a99a10cb8a9c0b987876530bcba3da90702b68cf827dbc3d80d039d5ac4e6ab851263565fab4b20e46ab02adbc6e81fe989c5d04e71bc4afde4e91f89136c9f594086d842109144c207199e53a5b2e31959f42] vs 181827500455998736810702955668246866123667144185569461398230915259537522277099174835083587562563974003980135201557682538618372029447176770743699296249348962529290445314977313069428292795552881744492306903665685883169633841495359589281323104579910516986114811031265795607492667789128307113410855020182087426590028559685783280976741670333985322620229750704070876263486607789796016522477783025824738220388195640566940677021506 [0x6924ef10000000000000000729977645c747b860000000096c376f0679a77da3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9f312f0d18d29aea64c1e56f37c8486094d9f4ece4b4469233b8206bf65a99a10cb8a9c0b987876530bcba3da90702b68cf827dbc3d80d039d5ac4e6ab851263565fab4b20e46ab02adbc6e81fe989c5d04e71bc4afde4e91f89136c9f594086d842109144c207199e53a5b2e31959f42]) cdi_type: Attestation
  
   DiversificationKey act: 0x4e71bc4afde4e91f89136c9f594086d842109144c207199e53a5b2e31959f42, exp: 0x4e71bc4afde4e91f89136c9f594086d842109144c207199e53a5b2e31959f42
   RomDigest act: 0xcf827dbc3d80d039d5ac4e6ab851263565fab4b20e46ab02adbc6e81fe989c5d, exp: 0xcf827dbc3d80d039d5ac4e6ab851263565fab4b20e46ab02adbc6e81fe989c5d
   HealthMeasurement act: 0xcb8a9c0b987876530bcba3da90702b68, exp: 0xcb8a9c0b987876530bcba3da90702b68

37.keymgr_kmac_rsp_err.70449756557746418004015501807340015010911803323776947921906713212883447709688
Line 425, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/37.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @  27509121 ps: (keymgr_scoreboard.sv:1013) [uvm_test_top.env.scoreboard] Check failed act != exp (1314858830569888130155893982114739967926870035052756616337957546195225295527938837348849950140029214094830405949404303632807341980653951391791612074882332614493530000784617337936627785384115375249880950183207578157580609287216479052006203158686933940923421989124503550855918286605317430801082126597291603838619151408253131344837804236038106119869664367886741317678670332095721604552365942862289068220518007928075459870873665 [0x2f855d94655a3a11a22d247114a4e6116c3c3f8df99c5b82793608179d0293e63a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9fbf89eaccc84137c1452876e923ceef1ca7e973628966ef736e2d5f9c6556e6c7b13f531e5c2eb18654da99b911a7b311811e3a2fea25fc87188483ab8c80cfb2709f275b7f5e86e5d8b2e57f38815566a969de9eab1a75a587cd496826fbd21a38504813504da5c57b4dbe2f5d85841] vs 1314858830569888130155893982114739967926870035052756616337957546195225295527938837348849950140029214094830405949404303632807341980653951391791612074882332614493530000784617337936627785384115375249880950183207578157580609287216479052006203158686933940923421989124503550855918286605317430801082126597291603838619151408253131344837804236038106119869664367886741317678670332095721604552365942862289068220518007928075459870873665 [0x2f855d94655a3a11a22d247114a4e6116c3c3f8df99c5b82793608179d0293e63a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9fbf89eaccc84137c1452876e923ceef1ca7e973628966ef736e2d5f9c6556e6c7b13f531e5c2eb18654da99b911a7b311811e3a2fea25fc87188483ab8c80cfb2709f275b7f5e86e5d8b2e57f38815566a969de9eab1a75a587cd496826fbd21a38504813504da5c57b4dbe2f5d85841]) cdi_type: Attestation
  
   DiversificationKey act: 0x6a969de9eab1a75a587cd496826fbd21a38504813504da5c57b4dbe2f5d85841, exp: 0x6a969de9eab1a75a587cd496826fbd21a38504813504da5c57b4dbe2f5d85841
   RomDigest act: 0x1811e3a2fea25fc87188483ab8c80cfb2709f275b7f5e86e5d8b2e57f3881556, exp: 0x1811e3a2fea25fc87188483ab8c80cfb2709f275b7f5e86e5d8b2e57f3881556
   HealthMeasurement act: 0x7b13f531e5c2eb18654da99b911a7b31, exp: 0x7b13f531e5c2eb18654da99b911a7b31

... and 1 more failures.

UVM_ERROR (keymgr_scoreboard.sv:750) [scoreboard] Check failed item.d_data ==gmv(csr) (* [] vs * []) reg name: keymgr_reg_block.sw_share0_output_*` has 1 failures:
- Test keymgr_stress_all_with_rand_reset has 1 failures.
  - 1.keymgr_stress_all_with_rand_reset.84570017935148253067425970378840664982556062191872346681076325775770483607406
    Line 703, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/1.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 235489672 ps: (keymgr_scoreboard.sv:750) [uvm_test_top.env.scoreboard] Check failed item.d_data == `gmv(csr) (4026524732 [0xefffe43c] vs 0 [0x0]) reg name: keymgr_reg_block.sw_share0_output_7
  UVM_INFO @ 235489672 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```

UVM_ERROR (keymgr_if.sv:550) [keymgr_if] Check failed {act_key.key[*], act_key.key[*]} !== keys_a_array[state][cdi][dest] (* [*] vs * [*]) KMAC key at state StCreatorRootKey for Attestation Kmac has 1 failures:

Test keymgr_lc_disable has 1 failures.

4.keymgr_lc_disable.83255023209708633980626961677195465983199135033886751188940788681607831737805
Line 379, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/4.keymgr_lc_disable/latest/run.log

  UVM_ERROR @ 114092060 ps: (keymgr_if.sv:550) [keymgr_if] Check failed {act_key.key[1], act_key.key[0]} !== keys_a_array[state][cdi][dest] (11117461786202510682755942938739474515913863471059876370152610695184484933861554233702146154401160677333638535305301820556332885577077618764582736103523928 [0xd44505708f5ded0b93433241c41805a2ec017871dec09f948082bcd3e71fd8c2dbf34f49f016cf5d9c8f0c09b1a3107fa288dbc388db39022f975d918960c658] vs 11117461786202510682755942938739474515913863471059876370152610695184484933861554233702146154401160677333638535305301820556332885577077618764582736103523928 [0xd44505708f5ded0b93433241c41805a2ec017871dec09f948082bcd3e71fd8c2dbf34f49f016cf5d9c8f0c09b1a3107fa288dbc388db39022f975d918960c658]) KMAC key at state StCreatorRootKey for Attestation Kmac
  UVM_INFO @ 114092060 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---