KEYMGR Simulation Results

Sunday March 24 2024 19:02:40 UTC

GitHub Revision: `70ad420931`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 56687816123908180356912499273064417112757374299033127319246303583078997854118

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	keymgr_smoke	43.580s	1.321ms	50	50	100.00
V1	random	keymgr_random	1.459m	7.309ms	50	50	100.00
V1	csr_hw_reset	keymgr_csr_hw_reset	1.590s	34.575us	5	5	100.00
V1	csr_rw	keymgr_csr_rw	1.600s	116.668us	20	20	100.00
V1	csr_bit_bash	keymgr_csr_bit_bash	16.920s	2.693ms	5	5	100.00
V1	csr_aliasing	keymgr_csr_aliasing	14.660s	1.491ms	5	5	100.00
V1	csr_mem_rw_with_rand_reset	keymgr_csr_mem_rw_with_rand_reset	2.510s	38.669us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	keymgr_csr_rw	1.600s	116.668us	20	20	100.00
		keymgr_csr_aliasing	14.660s	1.491ms	5	5	100.00
V1		TOTAL			155	155	100.00
V2	cfgen_during_op	keymgr_cfg_regwen	1.512m	1.646ms	49	50	98.00
V2	sideload	keymgr_sideload	31.570s	3.062ms	50	50	100.00
		keymgr_sideload_kmac	56.670s	9.409ms	50	50	100.00
		keymgr_sideload_aes	1.212m	7.000ms	50	50	100.00
		keymgr_sideload_otbn	55.880s	7.921ms	50	50	100.00
V2	direct_to_disabled_state	keymgr_direct_to_disabled	44.130s	2.115ms	50	50	100.00
V2	lc_disable	keymgr_lc_disable	19.340s	401.055us	50	50	100.00
V2	kmac_error_response	keymgr_kmac_rsp_err	1.196m	12.485ms	41	50	82.00
V2	invalid_sw_input	keymgr_sw_invalid_input	1.418m	14.036ms	50	50	100.00
V2	invalid_hw_input	keymgr_hwsw_invalid_input	1.982m	11.624ms	50	50	100.00
V2	sync_async_fault_cross	keymgr_sync_async_fault_cross	58.280s	6.694ms	50	50	100.00
V2	stress_all	keymgr_stress_all	14.764m	87.508ms	49	50	98.00
V2	intr_test	keymgr_intr_test	0.930s	13.009us	50	50	100.00
V2	alert_test	keymgr_alert_test	0.980s	29.266us	50	50	100.00
V2	tl_d_oob_addr_access	keymgr_tl_errors	5.400s	631.484us	20	20	100.00
V2	tl_d_illegal_access	keymgr_tl_errors	5.400s	631.484us	20	20	100.00
V2	tl_d_outstanding_access	keymgr_csr_hw_reset	1.590s	34.575us	5	5	100.00
		keymgr_csr_rw	1.600s	116.668us	20	20	100.00
		keymgr_csr_aliasing	14.660s	1.491ms	5	5	100.00
		keymgr_same_csr_outstanding	3.970s	414.058us	20	20	100.00
V2	tl_d_partial_access	keymgr_csr_hw_reset	1.590s	34.575us	5	5	100.00
		keymgr_csr_rw	1.600s	116.668us	20	20	100.00
		keymgr_csr_aliasing	14.660s	1.491ms	5	5	100.00
		keymgr_same_csr_outstanding	3.970s	414.058us	20	20	100.00
V2		TOTAL			729	740	98.51
V2S	sec_cm_additional_check	keymgr_sec_cm	28.780s	1.037ms	5	5	100.00
V2S	tl_intg_err	keymgr_sec_cm	28.780s	1.037ms	5	5	100.00
		keymgr_tl_intg_err	13.320s	1.002ms	20	20	100.00
V2S	shadow_reg_update_error	keymgr_shadow_reg_errors	54.910s	9.568ms	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	keymgr_shadow_reg_errors	54.910s	9.568ms	20	20	100.00
V2S	shadow_reg_storage_error	keymgr_shadow_reg_errors	54.910s	9.568ms	20	20	100.00
V2S	shadowed_reset_glitch	keymgr_shadow_reg_errors	54.910s	9.568ms	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	keymgr_shadow_reg_errors_with_csr_rw	15.760s	1.496ms	20	20	100.00
V2S	prim_count_check	keymgr_sec_cm	28.780s	1.037ms	5	5	100.00
V2S	prim_fsm_check	keymgr_sec_cm	28.780s	1.037ms	5	5	100.00
V2S	sec_cm_bus_integrity	keymgr_tl_intg_err	13.320s	1.002ms	20	20	100.00
V2S	sec_cm_config_shadow	keymgr_shadow_reg_errors	54.910s	9.568ms	20	20	100.00
V2S	sec_cm_op_config_regwen	keymgr_cfg_regwen	1.512m	1.646ms	49	50	98.00
V2S	sec_cm_reseed_config_regwen	keymgr_random	1.459m	7.309ms	50	50	100.00
		keymgr_csr_rw	1.600s	116.668us	20	20	100.00
V2S	sec_cm_sw_binding_config_regwen	keymgr_random	1.459m	7.309ms	50	50	100.00
		keymgr_csr_rw	1.600s	116.668us	20	20	100.00
V2S	sec_cm_max_key_ver_config_regwen	keymgr_random	1.459m	7.309ms	50	50	100.00
		keymgr_csr_rw	1.600s	116.668us	20	20	100.00
V2S	sec_cm_lc_ctrl_intersig_mubi	keymgr_lc_disable	19.340s	401.055us	50	50	100.00
V2S	sec_cm_constants_consistency	keymgr_hwsw_invalid_input	1.982m	11.624ms	50	50	100.00
V2S	sec_cm_intersig_consistency	keymgr_hwsw_invalid_input	1.982m	11.624ms	50	50	100.00
V2S	sec_cm_hw_key_sw_noaccess	keymgr_random	1.459m	7.309ms	50	50	100.00
V2S	sec_cm_output_keys_ctrl_redun	keymgr_sideload_protect	44.670s	7.095ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_sparse	keymgr_sec_cm	28.780s	1.037ms	5	5	100.00
V2S	sec_cm_data_fsm_sparse	keymgr_sec_cm	28.780s	1.037ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_local_esc	keymgr_sec_cm	28.780s	1.037ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_consistency	keymgr_custom_cm	1.168m	10.311ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_global_esc	keymgr_lc_disable	19.340s	401.055us	50	50	100.00
V2S	sec_cm_ctrl_ctr_redun	keymgr_sec_cm	28.780s	1.037ms	5	5	100.00
V2S	sec_cm_kmac_if_fsm_sparse	keymgr_sec_cm	28.780s	1.037ms	5	5	100.00
V2S	sec_cm_kmac_if_ctr_redun	keymgr_sec_cm	28.780s	1.037ms	5	5	100.00
V2S	sec_cm_kmac_if_cmd_ctrl_consistency	keymgr_custom_cm	1.168m	10.311ms	50	50	100.00
V2S	sec_cm_kmac_if_done_ctrl_consistency	keymgr_custom_cm	1.168m	10.311ms	50	50	100.00
V2S	sec_cm_reseed_ctr_redun	keymgr_sec_cm	28.780s	1.037ms	5	5	100.00
V2S	sec_cm_side_load_sel_ctrl_consistency	keymgr_custom_cm	1.168m	10.311ms	50	50	100.00
V2S	sec_cm_sideload_ctrl_fsm_sparse	keymgr_sec_cm	28.780s	1.037ms	5	5	100.00
V2S	sec_cm_ctrl_key_integrity	keymgr_custom_cm	1.168m	10.311ms	50	50	100.00
V2S		TOTAL			165	165	100.00
V3	stress_all_with_rand_reset	keymgr_stress_all_with_rand_reset	28.320s	1.531ms	29	50	58.00
V3		TOTAL			29	50	58.00
		TOTAL			1078	1110	97.12

Testplan Progress

Items	Total	Written	Passing	Progress
V1	7	7	7	100.00
V2	16	16	13	81.25
V2S	6	6	6	100.00
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
97.81	99.07	98.06	98.42	100.00	99.11	98.41	91.61

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:829) [keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 14 failures:
- Test keymgr_stress_all_with_rand_reset has 14 failures.
  - 2.keymgr_stress_all_with_rand_reset.59774096286765345274499915021586924198066716407621180225048397698994979684882
    Line 1001, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/2.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 2158244477 ps: (cip_base_vseq.sv:829) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 2158244477 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 6.keymgr_stress_all_with_rand_reset.93628827471016056433446435873677831778194304921052863187547721740909988998119
    Line 346, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/6.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 590189554 ps: (cip_base_vseq.sv:829) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 590189554 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 12 more failures.

UVM_ERROR (keymgr_scoreboard.sv:1013) [scoreboard] Check failed act != exp (* [*] vs * [*]) cdi_type: Attestation has 9 failures:

Test keymgr_kmac_rsp_err has 9 failures.

15.keymgr_kmac_rsp_err.73563021542204959920953166142239808206440024704429672983931589205681850363249
Line 397, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/15.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @  95570990 ps: (keymgr_scoreboard.sv:1013) [uvm_test_top.env.scoreboard] Check failed act != exp (2075116541444382265511754392348313428527339651186170736876130763353147004984170284201248924359098019798275465617455513475475409546151860371188618303508908341946960713399338917709247625773845790594433717423619876944487373077633200349219542177885554978769152665404528919496315234574009861401139624680143854011761025759738509627225914721357710201770803975287112267499322624215842829244594657549844992928806551675218877253883964 [0x4aff7042aa6fdf4b8ed41962a0d8fa05a18bf2f921bf489d6ca739782aebd64f3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f932554998d0e8d7f9dd8a7659462a58dcac150b4ec74130004b9d158513427c41e180615694407ea6c77f90cb58e81324b77e8aa14458019137bab0db3f31fe456cb6c7cb7beb4074066f0d02a2a5b0363908ac8fd360f2629e373702e52f94f2342222ca7a4148ae4166f8dda057183c] vs 2075116541444382265511754392348313428527339651186170736876130763353147004984170284201248924359098019798275465617455513475475409546151860371188618303508908341946960713399338917709247625773845790594433717423619876944487373077633200349219542177885554978769152665404528919496315234574009861401139624680143854011761025759738509627225914721357710201770803975287112267499322624215842829244594657549844992928806551675218877253883964 [0x4aff7042aa6fdf4b8ed41962a0d8fa05a18bf2f921bf489d6ca739782aebd64f3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f932554998d0e8d7f9dd8a7659462a58dcac150b4ec74130004b9d158513427c41e180615694407ea6c77f90cb58e81324b77e8aa14458019137bab0db3f31fe456cb6c7cb7beb4074066f0d02a2a5b0363908ac8fd360f2629e373702e52f94f2342222ca7a4148ae4166f8dda057183c]) cdi_type: Attestation
  
   DiversificationKey act: 0x3908ac8fd360f2629e373702e52f94f2342222ca7a4148ae4166f8dda057183c, exp: 0x3908ac8fd360f2629e373702e52f94f2342222ca7a4148ae4166f8dda057183c
   RomDigest act: 0xb77e8aa14458019137bab0db3f31fe456cb6c7cb7beb4074066f0d02a2a5b036, exp: 0xb77e8aa14458019137bab0db3f31fe456cb6c7cb7beb4074066f0d02a2a5b036
   HealthMeasurement act: 0xe180615694407ea6c77f90cb58e81324, exp: 0xe180615694407ea6c77f90cb58e81324

21.keymgr_kmac_rsp_err.40340119165413990439094547040542021641909281096482004531463690259138349625840
Line 497, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/21.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @ 285097184 ps: (keymgr_scoreboard.sv:1013) [uvm_test_top.env.scoreboard] Check failed act != exp (2755192691834652109308696955318113389135490559374083636974075045070744818389219212756519710490533943973187243335476601823164173572403331531937373100307583928532668910996378891726294609236470173795624368714268963342540112562965906366735015584570944190895613927067075788725160405262075857248214959125581884148214862054900477283359002377625261629088587395630432559035551172504147979454923750798316539866035802983944702618614112 [0x6393a766698f12a19b768762335a8259cd839079d16e06a2113b88decf4c91dd3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9e00f879395e0d85c0535c28be577503be2c394d3daa293966213b547d7d3a3dd24f7fd70f46c19a8f8d1635277e25855b13f96b66c40e1c2560d86915b1b94e988f10824cf04378615c4b6880197d0cb128149df21c62ec95a5ac3ce9894f32f0e025f56a464496822a798285f5db960] vs 2755192691834652109308696955318113389135490559374083636974075045070744818389219212756519710490533943973187243335476601823164173572403331531937373100307583928532668910996378891726294609236470173795624368714268963342540112562965906366735015584570944190895613927067075788725160405262075857248214959125581884148214862054900477283359002377625261629088587395630432559035551172504147979454923750798316539866035802983944702618614112 [0x6393a766698f12a19b768762335a8259cd839079d16e06a2113b88decf4c91dd3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9e00f879395e0d85c0535c28be577503be2c394d3daa293966213b547d7d3a3dd24f7fd70f46c19a8f8d1635277e25855b13f96b66c40e1c2560d86915b1b94e988f10824cf04378615c4b6880197d0cb128149df21c62ec95a5ac3ce9894f32f0e025f56a464496822a798285f5db960]) cdi_type: Attestation
  
   DiversificationKey act: 0x128149df21c62ec95a5ac3ce9894f32f0e025f56a464496822a798285f5db960, exp: 0x128149df21c62ec95a5ac3ce9894f32f0e025f56a464496822a798285f5db960
   RomDigest act: 0xb13f96b66c40e1c2560d86915b1b94e988f10824cf04378615c4b6880197d0cb, exp: 0xb13f96b66c40e1c2560d86915b1b94e988f10824cf04378615c4b6880197d0cb
   HealthMeasurement act: 0x24f7fd70f46c19a8f8d1635277e25855, exp: 0x24f7fd70f46c19a8f8d1635277e25855

... and 7 more failures.

UVM_ERROR (keymgr_scoreboard.sv:736) [scoreboard] Check failed item.d_data !=gmv(csr) (* [] vs * [])` has 6 failures:
- Test keymgr_stress_all_with_rand_reset has 6 failures.
  - 1.keymgr_stress_all_with_rand_reset.107366045150062706811660628766817944014546365290591249168070170492830833404569
    Line 1048, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/1.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 1235939372 ps: (keymgr_scoreboard.sv:736) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (2972829191 [0xb131c607] vs 2972829191 [0xb131c607])
  UVM_INFO @ 1235939372 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 4.keymgr_stress_all_with_rand_reset.102751808145340050624763782312259460254498439584925862181954849581932634988427
    Line 2098, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/4.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 776287999 ps: (keymgr_scoreboard.sv:736) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (1409791759 [0x5407b70f] vs 1409791759 [0x5407b70f])
  UVM_INFO @ 776287999 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 4 more failures.
UVM_ERROR (keymgr_scoreboard.sv:750) [scoreboard] Check failed item.d_data ==gmv(csr) (* [] vs * []) reg name: keymgr_reg_block.start` has 1 failures:
- Test keymgr_cfg_regwen has 1 failures.
  - 13.keymgr_cfg_regwen.42612427047591201818396342158852674327461733696847654328043195005866564466655
    Line 316, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/13.keymgr_cfg_regwen/latest/run.log
```
  UVM_ERROR @   4385552 ps: (keymgr_scoreboard.sv:750) [uvm_test_top.env.scoreboard] Check failed item.d_data == `gmv(csr) (0 [0x0] vs 1 [0x1]) reg name: keymgr_reg_block.start
  UVM_INFO @   4385552 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (cip_base_vseq.sv:753) [keymgr_common_vseq] Check failed (!has_outstanding_access()) Trying to trigger a reset with outstanding CSR items. has 1 failures:
- Test keymgr_stress_all_with_rand_reset has 1 failures.
  - 18.keymgr_stress_all_with_rand_reset.90261363590553338063301757633916509600265949813563530695721639463590561442567
    Line 1062, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/18.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 943412032 ps: (cip_base_vseq.sv:753) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Trying to trigger a reset with outstanding CSR items.
  UVM_INFO @ 943412032 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (keymgr_scoreboard.sv:750) [scoreboard] Check failed item.d_data ==gmv(csr) (* [] vs * []) reg name: keymgr_reg_block.sw_share0_output_*` has 1 failures:
- Test keymgr_stress_all has 1 failures.
  - 49.keymgr_stress_all.114567763962729186073089310497397971358128616986804274965661446984141539566029
    Line 2167, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/49.keymgr_stress_all/latest/run.log
```
  UVM_ERROR @ 349756956 ps: (keymgr_scoreboard.sv:750) [uvm_test_top.env.scoreboard] Check failed item.d_data == `gmv(csr) (2772631977 [0xa54301a9] vs 0 [0x0]) reg name: keymgr_reg_block.sw_share0_output_4
  UVM_INFO @ 349756956 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```