KEYMGR Simulation Results

Saturday July 20 2024 23:02:34 UTC

GitHub Revision: e971cd9798

Branch: os_regression

Testplan

Simulator: VCS

Build randomization enabled with --build-seed 85433431889345478971181747401055702269263498582281270185582621732035232392187

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	keymgr_smoke	28.560s	1.655ms	50	50	100.00
V1	random	keymgr_random	47.480s	9.715ms	50	50	100.00
V1	csr_hw_reset	keymgr_csr_hw_reset	1.420s	103.836us	5	5	100.00
V1	csr_rw	keymgr_csr_rw	1.570s	52.435us	20	20	100.00
V1	csr_bit_bash	keymgr_csr_bit_bash	27.100s	3.885ms	5	5	100.00
V1	csr_aliasing	keymgr_csr_aliasing	7.420s	248.836us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	keymgr_csr_mem_rw_with_rand_reset	2.070s	33.046us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	keymgr_csr_rw	1.570s	52.435us	20	20	100.00
		keymgr_csr_aliasing	7.420s	248.836us	5	5	100.00
V1		TOTAL			155	155	100.00
V2	cfgen_during_op	keymgr_cfg_regwen	2.333m	34.210ms	50	50	100.00
V2	sideload	keymgr_sideload	29.330s	5.287ms	50	50	100.00
		keymgr_sideload_kmac	59.210s	3.219ms	50	50	100.00
		keymgr_sideload_aes	48.440s	7.361ms	50	50	100.00
		keymgr_sideload_otbn	41.520s	5.748ms	49	50	98.00
V2	direct_to_disabled_state	keymgr_direct_to_disabled	35.590s	3.564ms	50	50	100.00
V2	lc_disable	keymgr_lc_disable	7.450s	749.884us	49	50	98.00
V2	kmac_error_response	keymgr_kmac_rsp_err	17.220s	2.405ms	50	50	100.00
V2	invalid_sw_input	keymgr_sw_invalid_input	38.120s	3.771ms	49	50	98.00
V2	invalid_hw_input	keymgr_hwsw_invalid_input	28.180s	1.284ms	49	50	98.00
V2	sync_async_fault_cross	keymgr_sync_async_fault_cross	15.270s	1.882ms	50	50	100.00
V2	stress_all	keymgr_stress_all	5.378m	11.287ms	49	50	98.00
V2	intr_test	keymgr_intr_test	0.940s	33.935us	50	50	100.00
V2	alert_test	keymgr_alert_test	1.040s	27.048us	50	50	100.00
V2	tl_d_oob_addr_access	keymgr_tl_errors	5.590s	666.803us	20	20	100.00
V2	tl_d_illegal_access	keymgr_tl_errors	5.590s	666.803us	20	20	100.00
V2	tl_d_outstanding_access	keymgr_csr_hw_reset	1.420s	103.836us	5	5	100.00
		keymgr_csr_rw	1.570s	52.435us	20	20	100.00
		keymgr_csr_aliasing	7.420s	248.836us	5	5	100.00
		keymgr_same_csr_outstanding	4.190s	119.294us	20	20	100.00
V2	tl_d_partial_access	keymgr_csr_hw_reset	1.420s	103.836us	5	5	100.00
		keymgr_csr_rw	1.570s	52.435us	20	20	100.00
		keymgr_csr_aliasing	7.420s	248.836us	5	5	100.00
		keymgr_same_csr_outstanding	4.190s	119.294us	20	20	100.00
V2		TOTAL			735	740	99.32
V2S	sec_cm_additional_check	keymgr_sec_cm	15.640s	1.972ms	5	5	100.00
V2S	tl_intg_err	keymgr_sec_cm	15.640s	1.972ms	5	5	100.00
		keymgr_tl_intg_err	12.220s	1.392ms	20	20	100.00
V2S	shadow_reg_update_error	keymgr_shadow_reg_errors	5.210s	190.259us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	keymgr_shadow_reg_errors	5.210s	190.259us	20	20	100.00
V2S	shadow_reg_storage_error	keymgr_shadow_reg_errors	5.210s	190.259us	20	20	100.00
V2S	shadowed_reset_glitch	keymgr_shadow_reg_errors	5.210s	190.259us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	keymgr_shadow_reg_errors_with_csr_rw	14.750s	755.255us	20	20	100.00
V2S	prim_count_check	keymgr_sec_cm	15.640s	1.972ms	5	5	100.00
V2S	prim_fsm_check	keymgr_sec_cm	15.640s	1.972ms	5	5	100.00
V2S	sec_cm_bus_integrity	keymgr_tl_intg_err	12.220s	1.392ms	20	20	100.00
V2S	sec_cm_config_shadow	keymgr_shadow_reg_errors	5.210s	190.259us	20	20	100.00
V2S	sec_cm_op_config_regwen	keymgr_cfg_regwen	2.333m	34.210ms	50	50	100.00
V2S	sec_cm_reseed_config_regwen	keymgr_random	47.480s	9.715ms	50	50	100.00
		keymgr_csr_rw	1.570s	52.435us	20	20	100.00
V2S	sec_cm_sw_binding_config_regwen	keymgr_random	47.480s	9.715ms	50	50	100.00
		keymgr_csr_rw	1.570s	52.435us	20	20	100.00
V2S	sec_cm_max_key_ver_config_regwen	keymgr_random	47.480s	9.715ms	50	50	100.00
		keymgr_csr_rw	1.570s	52.435us	20	20	100.00
V2S	sec_cm_lc_ctrl_intersig_mubi	keymgr_lc_disable	7.450s	749.884us	49	50	98.00
V2S	sec_cm_constants_consistency	keymgr_hwsw_invalid_input	28.180s	1.284ms	49	50	98.00
V2S	sec_cm_intersig_consistency	keymgr_hwsw_invalid_input	28.180s	1.284ms	49	50	98.00
V2S	sec_cm_hw_key_sw_noaccess	keymgr_random	47.480s	9.715ms	50	50	100.00
V2S	sec_cm_output_keys_ctrl_redun	keymgr_sideload_protect	21.590s	5.220ms	50	50	100.00
V2S	sec_cm_ctrl_fsm_sparse	keymgr_sec_cm	15.640s	1.972ms	5	5	100.00
V2S	sec_cm_data_fsm_sparse	keymgr_sec_cm	15.640s	1.972ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_local_esc	keymgr_sec_cm	15.640s	1.972ms	5	5	100.00
V2S	sec_cm_ctrl_fsm_consistency	keymgr_custom_cm	15.910s	644.341us	50	50	100.00
V2S	sec_cm_ctrl_fsm_global_esc	keymgr_lc_disable	7.450s	749.884us	49	50	98.00
V2S	sec_cm_ctrl_ctr_redun	keymgr_sec_cm	15.640s	1.972ms	5	5	100.00
V2S	sec_cm_kmac_if_fsm_sparse	keymgr_sec_cm	15.640s	1.972ms	5	5	100.00
V2S	sec_cm_kmac_if_ctr_redun	keymgr_sec_cm	15.640s	1.972ms	5	5	100.00
V2S	sec_cm_kmac_if_cmd_ctrl_consistency	keymgr_custom_cm	15.910s	644.341us	50	50	100.00
V2S	sec_cm_kmac_if_done_ctrl_consistency	keymgr_custom_cm	15.910s	644.341us	50	50	100.00
V2S	sec_cm_reseed_ctr_redun	keymgr_sec_cm	15.640s	1.972ms	5	5	100.00
V2S	sec_cm_side_load_sel_ctrl_consistency	keymgr_custom_cm	15.910s	644.341us	50	50	100.00
V2S	sec_cm_sideload_ctrl_fsm_sparse	keymgr_sec_cm	15.640s	1.972ms	5	5	100.00
V2S	sec_cm_ctrl_key_integrity	keymgr_custom_cm	15.910s	644.341us	50	50	100.00
V2S		TOTAL			165	165	100.00
V3	stress_all_with_rand_reset	keymgr_stress_all_with_rand_reset	26.720s	1.756ms	25	50	50.00
V3		TOTAL			25	50	50.00
		TOTAL			1080	1110	97.30

Testplan Progress

Items	Total	Written	Passing	Progress
V1	7	7	7	100.00
V2	16	16	11	68.75
V2S	6	6	6	100.00
V3	1	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
97.76	99.04	98.15	98.52	100.00	99.02	98.41	91.19

Failure Buckets

• UVM_ERROR (cip_base_vseq.sv:839) [keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 25 failures:

  • Test keymgr_stress_all_with_rand_reset has 25 failures.

    • 1.keymgr_stress_all_with_rand_reset.65596071438208773597323394933740940103291306786294034916249996487315696335466
      Line 920, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/1.keymgr_stress_all_with_rand_reset/latest/run.log

        UVM_ERROR @ 223515164 ps: (cip_base_vseq.sv:839) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
        UVM_INFO @ 223515164 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
        --- UVM Report catcher Summary ---
        
        
      

    • 3.keymgr_stress_all_with_rand_reset.30975444910590617925077074157996153361174330997204131926892759320352220428563
      Line 480, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/3.keymgr_stress_all_with_rand_reset/latest/run.log

        UVM_ERROR @ 1227122655 ps: (cip_base_vseq.sv:839) [uvm_test_top.env.virtual_sequencer.keymgr_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
        UVM_INFO @ 1227122655 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
        --- UVM Report catcher Summary ---
        
        
      

    • ... and 23 more failures.

• UVM_ERROR (cip_base_scoreboard.sv:301) scoreboard [scoreboard] alert recov_operation_err did not trigger max_delay:* has 4 failures:

  • Test keymgr_stress_all has 1 failures.

    • 4.keymgr_stress_all.15121291246522236946374290757841141235597142377329246434633655703509292530369
      Line 1936, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/4.keymgr_stress_all/latest/run.log

        UVM_ERROR @ 277391185 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
        UVM_INFO @ 277391185 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
        --- UVM Report catcher Summary ---
      

  • Test keymgr_hwsw_invalid_input has 1 failures.

    • 14.keymgr_hwsw_invalid_input.7205705034106153490192886087340142003352929475248482360349186768619372004868
      Line 463, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/14.keymgr_hwsw_invalid_input/latest/run.log

        UVM_ERROR @ 487456834 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
        UVM_INFO @ 487456834 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
        --- UVM Report catcher Summary ---
      

  • Test keymgr_sideload_otbn has 1 failures.

    • 30.keymgr_sideload_otbn.31376386558065976675247892938560933309270709998060745649820844980858299794235
      Line 259, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/30.keymgr_sideload_otbn/latest/run.log

        UVM_ERROR @   2678340 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
        UVM_INFO @   2678340 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
        --- UVM Report catcher Summary ---
      

  • Test keymgr_sw_invalid_input has 1 failures.

    • 35.keymgr_sw_invalid_input.50645800445086985070614010128129908258143766471663654576826931189898567154270
      Line 678, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/35.keymgr_sw_invalid_input/latest/run.log

        UVM_ERROR @  83590181 ps: (cip_base_scoreboard.sv:301) uvm_test_top.env.scoreboard [uvm_test_top.env.scoreboard] alert recov_operation_err did not trigger max_delay:0
        UVM_INFO @  83590181 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
        --- UVM Report catcher Summary ---
      

• UVM_ERROR (keymgr_scoreboard.sv:689) [scoreboard] Check failed edn_fifos[*].used() == * (* [*] vs * [*]) has 1 failures:

  • Test keymgr_lc_disable has 1 failures.

    • 13.keymgr_lc_disable.40253675462686576688661897434572458111330044065727614520722469379252236839459
      Line 329, in log /container/opentitan-public/scratch/os_regression/keymgr-sim-vcs/13.keymgr_lc_disable/latest/run.log

        UVM_ERROR @   8653050 ps: (keymgr_scoreboard.sv:689) [uvm_test_top.env.scoreboard] Check failed edn_fifos[0].used() == 2 (0 [0x0] vs 2 [0x2])
        UVM_INFO @   8653050 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
        --- UVM Report catcher Summary ---
      

Past Results

• Latest
• lint
• 2024.07.20_09.44.45
• 2024.07.19_08.47.27
• 2024.07.18_09.18.50
• 2024.07.17_09.28.30
• 2024.07.16_09.35.23
• 2024.07.15_09.24.00