KEYMGR Simulation Results

Tuesday March 12 2024 19:02:37 UTC

GitHub Revision: `bc285b7382`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 8078106501385188224785993882809517173695187907049792415947230968390919037084

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	keymgr_smoke	56.680s	1.875ms	50	50	100.00
V1	random	keymgr_random	1.212m	1.996ms	50	50	100.00
V1	csr_hw_reset	keymgr_csr_hw_reset	1.610s	68.726us	5	5	100.00
V1	csr_rw	keymgr_csr_rw	1.610s	37.588us	20	20	100.00
V1	csr_bit_bash	keymgr_csr_bit_bash	15.310s	673.540us	5	5	100.00
V1	csr_aliasing	keymgr_csr_aliasing	11.350s	2.044ms	5	5	100.00
V1	csr_mem_rw_with_rand_reset	keymgr_csr_mem_rw_with_rand_reset	2.390s	56.615us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	keymgr_csr_rw	1.610s	37.588us	20	20	100.00
		keymgr_csr_aliasing	11.350s	2.044ms	5	5	100.00
V1		TOTAL			155	155	100.00
V2	cfgen_during_op	keymgr_cfg_regwen	2.596m	2.766ms	50	50	100.00
V2	sideload	keymgr_sideload	49.110s	1.998ms	50	50	100.00
		keymgr_sideload_kmac	1.082m	7.887ms	50	50	100.00
		keymgr_sideload_aes	51.690s	16.419ms	50	50	100.00
		keymgr_sideload_otbn	58.140s	3.365ms	50	50	100.00
V2	direct_to_disabled_state	keymgr_direct_to_disabled	49.600s	9.391ms	50	50	100.00
V2	lc_disable	keymgr_lc_disable	46.200s	3.274ms	50	50	100.00
V2	kmac_error_response	keymgr_kmac_rsp_err	1.036m	5.549ms	38	50	76.00
V2	invalid_sw_input	keymgr_sw_invalid_input	47.500s	6.716ms	50	50	100.00
V2	invalid_hw_input	keymgr_hwsw_invalid_input	1.394m	9.081ms	50	50	100.00
V2	sync_async_fault_cross	keymgr_sync_async_fault_cross	11.170s	1.933ms	50	50	100.00
V2	stress_all	keymgr_stress_all	8.346m	111.820ms	49	50	98.00
V2	intr_test	keymgr_intr_test	1.000s	30.406us	50	50	100.00
V2	alert_test	keymgr_alert_test	1.300s	36.195us	50	50	100.00
V2	tl_d_oob_addr_access	keymgr_tl_errors	5.130s	151.827us	20	20	100.00
V2	tl_d_illegal_access	keymgr_tl_errors	5.130s	151.827us	20	20	100.00
V2	tl_d_outstanding_access	keymgr_csr_hw_reset	1.610s	68.726us	5	5	100.00
		keymgr_csr_rw	1.610s	37.588us	20	20	100.00
		keymgr_csr_aliasing	11.350s	2.044ms	5	5	100.00
		keymgr_same_csr_outstanding	4.530s	450.138us	20	20	100.00
V2	tl_d_partial_access	keymgr_csr_hw_reset	1.610s	68.726us	5	5	100.00
		keymgr_csr_rw	1.610s	37.588us	20	20	100.00
		keymgr_csr_aliasing	11.350s	2.044ms	5	5	100.00
		keymgr_same_csr_outstanding	4.530s	450.138us	20	20	100.00
V2		TOTAL			727	740	98.24
V2S	sec_cm_additional_check	keymgr_sec_cm	3.587m	15.121ms	5	5	100.00
V2S	tl_intg_err	keymgr_sec_cm	3.587m	15.121ms	5	5	100.00
		keymgr_tl_intg_err	1.311m	22.992ms	20	20	100.00
V2S	shadow_reg_update_error	keymgr_shadow_reg_errors	14.820s	2.525ms	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	keymgr_shadow_reg_errors	14.820s	2.525ms	20	20	100.00
V2S	shadow_reg_storage_error	keymgr_shadow_reg_errors	14.820s	2.525ms	20	20	100.00
V2S	shadowed_reset_glitch	keymgr_shadow_reg_errors	14.820s	2.525ms	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	keymgr_shadow_reg_errors_with_csr_rw	12.970s	428.461us	20	20	100.00
V2S	prim_count_check	keymgr_sec_cm	3.587m	15.121ms	5	5	100.00
V2S	prim_fsm_check	keymgr_sec_cm	3.587m	15.121ms	5	5	100.00
V2S	sec_cm_bus_integrity	keymgr_tl_intg_err	1.311m	22.992ms	20	20	100.00
V2S	sec_cm_config_shadow	keymgr_shadow_reg_errors	14.820s	2.525ms	20	20	100.00
V2S	sec_cm_op_config_regwen	keymgr_cfg_regwen	2.596m	2.766ms	50	50	100.00
V2S	sec_cm_reseed_config_regwen	keymgr_random	1.212m	1.996ms	50	50	100.00
		keymgr_csr_rw	1.610s	37.588us	20	20	100.00
V2S	sec_cm_sw_binding_config_regwen	keymgr_random	1.212m	1.996ms	50	50	100.00
		keymgr_csr_rw	1.610s	37.588us	20	20	100.00
V2S	sec_cm_max_key_ver_config_regwen	keymgr_random	1.212m	1.996ms	50	50	100.00
		keymgr_csr_rw	1.610s	37.588us	20	20	100.00
V2S	sec_cm_lc_ctrl_intersig_mubi	keymgr_lc_disable	46.200s	3.274ms	50	50	100.00
V2S	sec_cm_constants_consistency	keymgr_hwsw_invalid_input	1.394m	9.081ms	50	50	100.00
V2S	sec_cm_intersig_consistency	keymgr_hwsw_invalid_input	1.394m	9.081ms	50	50	100.00
V2S	sec_cm_hw_key_sw_noaccess	keymgr_random	1.212m	1.996ms	50	50	100.00
V2S	sec_cm_output_keys_ctrl_redun	keymgr_sideload_protect	39.740s	2.301ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_sparse	keymgr_sec_cm	3.587m	15.121ms	5	5	100.00
V2S	sec_cm_data_fsm_sparse	keymgr_sec_cm	3.587m	15.121ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_local_esc	keymgr_sec_cm	3.587m	15.121ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_consistency	keymgr_custom_cm	52.330s	9.506ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_global_esc	keymgr_lc_disable	46.200s	3.274ms	50	50	100.00
V2S	sec_cm_ctrl_ctr_redun	keymgr_sec_cm	3.587m	15.121ms	5	5	100.00
V2S	sec_cm_kmac_if_fsm_sparse	keymgr_sec_cm	3.587m	15.121ms	5	5	100.00
V2S	sec_cm_kmac_if_ctr_redun	keymgr_sec_cm	3.587m	15.121ms	5	5	100.00
V2S	sec_cm_kmac_if_cmd_ctrl_consistency	keymgr_custom_cm	52.330s	9.506ms	50	50	100.00
V2S	sec_cm_kmac_if_done_ctrl_consistency	keymgr_custom_cm	52.330s	9.506ms	50	50	100.00
V2S	sec_cm_reseed_ctr_redun	keymgr_sec_cm	3.587m	15.121ms	5	5	100.00
V2S	sec_cm_side_load_sel_ctrl_consistency	keymgr_custom_cm	52.330s	9.506ms	50	50	100.00
V2S	sec_cm_sideload_ctrl_fsm_sparse	keymgr_sec_cm	3.587m	15.121ms	5	5	100.00
V2S	sec_cm_ctrl_key_integrity	keymgr_custom_cm	52.330s	9.506ms	50	50	100.00
V2S		TOTAL			165	165	100.00
V3	stress_all_with_rand_reset	keymgr_stress_all_with_rand_reset	24.030s	4.280ms	17	50	34.00
V3		TOTAL			17	50	34.00
		TOTAL			1064	1110	95.86

Testplan Progress

Items	Total	Written	Passing	Progress
V1	7	7	7	100.00
V2	16	16	14	87.50
V2S	6	6	6	100.00
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
97.81	99.07	98.03	98.30	100.00	99.11	98.41	91.76

Failure Buckets

UVM_ERROR (cip_base_vseq.sv:827) [keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 25 failures:
- Test keymgr_stress_all_with_rand_reset has 25 failures.
  - 1.keymgr_stress_all_with_rand_reset.11867110053065421555776033700476719925966874655315881905188620270111617763484
    Line 413, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/1.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 301719413 ps: (cip_base_vseq.sv:827) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 301719413 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 2.keymgr_stress_all_with_rand_reset.16994349880775615419099116421666856961507290552692782990168841935627174607764
    Line 499, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/2.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 152988806 ps: (cip_base_vseq.sv:827) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 152988806 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 23 more failures.

UVM_ERROR (keymgr_scoreboard.sv:1013) [scoreboard] Check failed act != exp (* [*] vs * [*]) cdi_type: Attestation has 11 failures:

Test keymgr_kmac_rsp_err has 11 failures.

4.keymgr_kmac_rsp_err.98953476626777348642243472206694624749365687567035457470235018053139662447913
Line 424, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/4.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @ 213484688 ps: (keymgr_scoreboard.sv:1013) [uvm_test_top.env.scoreboard] Check failed act != exp (4460648657356150075028719226021262926145812374831201567325402897337894233960466160857471402549381622335866798947935033419589157160479038810091869954145358616511160324183284165662873388026290070474996602408057145935701003490375718200659512867266673258479673949422912353010986002881637464093960236570325742939928007238133903382657175746958089669008196006525461485382703821629860938276857150643531509783749869700153111798637789 [0xa136e92900000000d1d88db60000000066a072670000000000000000000000003a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f96e326939e950f1ddb9b4183532d8edf830f5d108640106c4e9baa2b201984fba99138027e16cb28f3790619531a265e2744cb34d6e064c1a10ee53af0ee6f682541d9ec211107146923b560e9bb97208916cc9eb1617178b57cdd7f283cc35b574362a627dcc386fd0d3ce18b4b864dd] vs 4460648657356150075028719226021262926145812374831201567325402897337894233960466160857471402549381622335866798947935033419589157160479038810091869954145358616511160324183284165662873388026290070474996602408057145935701003490375718200659512867266673258479673949422912353010986002881637464093960236570325742939928007238133903382657175746958089669008196006525461485382703821629860938276857150643531509783749869700153111798637789 [0xa136e92900000000d1d88db60000000066a072670000000000000000000000003a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f96e326939e950f1ddb9b4183532d8edf830f5d108640106c4e9baa2b201984fba99138027e16cb28f3790619531a265e2744cb34d6e064c1a10ee53af0ee6f682541d9ec211107146923b560e9bb97208916cc9eb1617178b57cdd7f283cc35b574362a627dcc386fd0d3ce18b4b864dd]) cdi_type: Attestation
  
   DiversificationKey act: 0x916cc9eb1617178b57cdd7f283cc35b574362a627dcc386fd0d3ce18b4b864dd, exp: 0x916cc9eb1617178b57cdd7f283cc35b574362a627dcc386fd0d3ce18b4b864dd
   RomDigest act: 0x744cb34d6e064c1a10ee53af0ee6f682541d9ec211107146923b560e9bb97208, exp: 0x744cb34d6e064c1a10ee53af0ee6f682541d9ec211107146923b560e9bb97208
   HealthMeasurement act: 0x99138027e16cb28f3790619531a265e2, exp: 0x99138027e16cb28f3790619531a265e2

6.keymgr_kmac_rsp_err.49254210077595270956266509896977549923685487273095933716326713551714511653843
Line 501, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/6.keymgr_kmac_rsp_err/latest/run.log

  UVM_ERROR @ 154161722 ps: (keymgr_scoreboard.sv:1013) [uvm_test_top.env.scoreboard] Check failed act != exp (13869088684237589750254652523805745455607483863801036890679746459828093165537749425697538482082568494966035882412806376581101037984299532902452652630554884915203692211350915422524595010893013083228828519885444105958146264065118630655586790933197844345028163247837549325546897800412463059189949137462138648402134553849508099829070178809759152278334 [0x3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f93c922f5f391ce8e20bc3c11662f1f20224df2830a5ebe865977231f0fe25043bcaf7b7a4baba58bfcd68a50768da4ff5e0f1d525ac3e1404a7a255e972fc86cf8ccbe7e5c702ba172c0123c19120f0e18d7b5cfe1ac489c677f0a248d53da8f7d9b981fb9313e86ce2d3d002e51d0b3e] vs 13869088684237589750254652523805745455607483863801036890679746459828093165537749425697538482082568494966035882412806376581101037984299532902452652630554884915203692211350915422524595010893013083228828519885444105958146264065118630655586790933197844345028163247837549325546897800412463059189949137462138648402134553849508099829070178809759152278334 [0x3a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f93c922f5f391ce8e20bc3c11662f1f20224df2830a5ebe865977231f0fe25043bcaf7b7a4baba58bfcd68a50768da4ff5e0f1d525ac3e1404a7a255e972fc86cf8ccbe7e5c702ba172c0123c19120f0e18d7b5cfe1ac489c677f0a248d53da8f7d9b981fb9313e86ce2d3d002e51d0b3e]) cdi_type: Attestation
  
   DiversificationKey act: 0x8d7b5cfe1ac489c677f0a248d53da8f7d9b981fb9313e86ce2d3d002e51d0b3e, exp: 0x8d7b5cfe1ac489c677f0a248d53da8f7d9b981fb9313e86ce2d3d002e51d0b3e
   RomDigest act: 0xe0f1d525ac3e1404a7a255e972fc86cf8ccbe7e5c702ba172c0123c19120f0e1, exp: 0xe0f1d525ac3e1404a7a255e972fc86cf8ccbe7e5c702ba172c0123c19120f0e1
   HealthMeasurement act: 0xcaf7b7a4baba58bfcd68a50768da4ff5, exp: 0xcaf7b7a4baba58bfcd68a50768da4ff5

... and 9 more failures.

UVM_ERROR (keymgr_scoreboard.sv:736) [scoreboard] Check failed item.d_data !=gmv(csr) (* [] vs * [])` has 6 failures:
- Test keymgr_stress_all_with_rand_reset has 6 failures.
  - 6.keymgr_stress_all_with_rand_reset.14040163980743677765142676133914219396365790182797083341604786816007907751037
    Line 745, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/6.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 275315892 ps: (keymgr_scoreboard.sv:736) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (1099451519 [0x41884c7f] vs 1099451519 [0x41884c7f])
  UVM_INFO @ 275315892 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 14.keymgr_stress_all_with_rand_reset.79096669475172107694841682038526387883457539026369835933016976505005777411748
    Line 1373, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/14.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 254997330 ps: (keymgr_scoreboard.sv:736) [uvm_test_top.env.scoreboard] Check failed item.d_data != `gmv(csr) (1025191197 [0x3d1b2d1d] vs 1025191197 [0x3d1b2d1d])
  UVM_INFO @ 254997330 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 4 more failures.

UVM_ERROR (keymgr_scoreboard.sv:1011) [scoreboard] Check failed act == exp (* [*] vs * [*]) cdi_type: Attestation has 1 failures:

Test keymgr_stress_all_with_rand_reset has 1 failures.

30.keymgr_stress_all_with_rand_reset.13693910433200072652800708061954168720109882650827148388607635825354363787842
Line 843, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/30.keymgr_stress_all_with_rand_reset/latest/run.log

  UVM_ERROR @ 299011340 ps: (keymgr_scoreboard.sv:1011) [uvm_test_top.env.scoreboard] Check failed act == exp (6094195813472608927701539711588257093063779284376967115784033592858563222764418154407065349821148949010896060154766775640559509097437652317992393978958791459946367191629263501496406942496616002763054332367782280310746033985580178540527286250859976530402905626592808134520340391843875856722966330962266009257282582411030030054389151462937838592598229886071474371043810988465006044686030548574511505484577866909857579356509818 [0xdc40d9f58842e6760194ec9405d5bb6dfa6af5d404f4d8f179d5654fd44027753a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9e8b704a096b4326a5cec38aa47e87eabea503e4fc5f13f1c1253eed0031b5960ce6ee4cb0546bafee96c8c94a72c4c9b77c4cd4a04c14b998c07f81fd1d6d24f013d4f86160099d6ba2d4728aa4c96722ddaf61fbb0f4de8787bdc001176ecbec4c5a2def799f1877517b708a8a5d67a] vs 3068736036098672323402001328404016612548396671983175369378119520394591818485557687000850148692910262979925843118634577033942640968799172982001909442437572237224291090723545272722602834318176397282098068184091521365135730080863194367524770144787320483449698747487276099597995103481480819013879371977000424136247581355793833393464813752796347265794010330572304529875346097966841690387388650008494278895790278653729424009451130 [0x6ee8a0609545fe729df7eec0cc4eb3dd2096e506369b2077a2c15816424d54d73a0a6d73cd50897de4d744bd65ebdb3837ea77087d878651c517c18a5742b2f9e8b704a096b4326a5cec38aa47e87eabea503e4fc5f13f1c1253eed0031b5960ce6ee4cb0546bafee96c8c94a72c4c9b77c4cd4a04c14b998c07f81fd1d6d24f013d4f86160099d6ba2d4728aa4c96722ddaf61fbb0f4de8787bdc001176ecbec4c5a2def799f1877517b708a8a5d67a]) cdi_type: Attestation
  
   DiversificationKey act: 0x2ddaf61fbb0f4de8787bdc001176ecbec4c5a2def799f1877517b708a8a5d67a, exp: 0x2ddaf61fbb0f4de8787bdc001176ecbec4c5a2def799f1877517b708a8a5d67a
   RomDigest act: 0x77c4cd4a04c14b998c07f81fd1d6d24f013d4f86160099d6ba2d4728aa4c9672, exp: 0x77c4cd4a04c14b998c07f81fd1d6d24f013d4f86160099d6ba2d4728aa4c9672
   HealthMeasurement act: 0xce6ee4cb0546bafee96c8c94a72c4c9b, exp: 0xce6ee4cb0546bafee96c8c94a72c4c9b

UVM_ERROR (cip_base_scoreboard.sv:287) scoreboard [scoreboard] alert recov_operation_err did not trigger max_delay:* has 1 failures:
- Test keymgr_kmac_rsp_err has 1 failures.
  - 35.keymgr_kmac_rsp_err.54254319416821331576262815701771064214574804178708146059446900877073112351460
    Line 441, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/35.keymgr_kmac_rsp_err/latest/run.log
```
  UVM_ERROR @  91896299 ps: (cip_base_scoreboard.sv:287) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
  UVM_INFO @  91896299 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (keymgr_scoreboard.sv:674) [scoreboard] Check failed item.d_data == addr_phase_op_status (* [*] vs * [*]) has 1 failures:
- Test keymgr_stress_all has 1 failures.
  - 37.keymgr_stress_all.11740642158557260312776858617391938592611003867044417681697954299981200398791
    Line 392, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/37.keymgr_stress_all/latest/run.log
```
  UVM_ERROR @ 619071278 ps: (keymgr_scoreboard.sv:674) [uvm_test_top.env.scoreboard] Check failed item.d_data == addr_phase_op_status (2 [0x2] vs 3 [0x3])
  UVM_INFO @ 619071278 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (keymgr_scoreboard.sv:750) [scoreboard] Check failed item.d_data ==gmv(csr) (* [] vs * []) reg name: keymgr_reg_block.sw_share1_output_*` has 1 failures:
- Test keymgr_stress_all_with_rand_reset has 1 failures.
  - 44.keymgr_stress_all_with_rand_reset.10474542245003907849608160355219136999898433618386184294562741806851536650310
    Line 314, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/44.keymgr_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @  52704561 ps: (keymgr_scoreboard.sv:750) [uvm_test_top.env.scoreboard] Check failed item.d_data == `gmv(csr) (3567682036 [0xd4a681f4] vs 0 [0x0]) reg name: keymgr_reg_block.sw_share1_output_3
  UVM_INFO @  52704561 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```