KMAC/UNMASKED Simulation Results

Tuesday April 16 2024 19:02:32 UTC

GitHub Revision: 1c75f24e99

Branch: os_regression

Testplan

Simulator: VCS

Build randomization enabled with --build-seed 47053888840936652465110085351243654616760492049444303115123736462709488656445

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	kmac_smoke	1.313m	56.758ms	50	50	100.00
V1	csr_hw_reset	kmac_csr_hw_reset	1.160s	52.750us	5	5	100.00
V1	csr_rw	kmac_csr_rw	1.200s	37.347us	20	20	100.00
V1	csr_bit_bash	kmac_csr_bit_bash	10.550s	1.452ms	5	5	100.00
V1	csr_aliasing	kmac_csr_aliasing	9.950s	490.848us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	kmac_csr_mem_rw_with_rand_reset	2.790s	619.412us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	kmac_csr_rw	1.200s	37.347us	20	20	100.00
		kmac_csr_aliasing	9.950s	490.848us	5	5	100.00
V1	mem_walk	kmac_mem_walk	0.740s	15.633us	5	5	100.00
V1	mem_partial_access	kmac_mem_partial_access	1.460s	48.454us	5	5	100.00
V1		TOTAL			115	115	100.00
V2	long_msg_and_output	kmac_long_msg_and_output	47.520m	387.771ms	50	50	100.00
V2	burst_write	kmac_burst_write	13.894m	74.555ms	48	50	96.00
V2	test_vectors	kmac_test_vectors_sha3_224	33.198m	198.043ms	50	50	100.00
		kmac_test_vectors_sha3_256	33.543m	1.211s	50	50	100.00
		kmac_test_vectors_sha3_384	26.713m	259.592ms	50	50	100.00
		kmac_test_vectors_sha3_512	18.351m	453.788ms	50	50	100.00
		kmac_test_vectors_shake_128	1.544h	1.071s	50	50	100.00
		kmac_test_vectors_shake_256	1.215h	920.376ms	50	50	100.00
		kmac_test_vectors_kmac	5.290s	255.115us	50	50	100.00
		kmac_test_vectors_kmac_xof	5.320s	997.464us	50	50	100.00
V2	sideload	kmac_sideload	6.788m	19.116ms	49	50	98.00
V2	app	kmac_app	5.396m	35.985ms	50	50	100.00
V2	app_with_partial_data	kmac_app_with_partial_data	5.290m	18.710ms	10	10	100.00
V2	entropy_refresh	kmac_entropy_refresh	5.636m	37.608ms	50	50	100.00
V2	error	kmac_error	7.607m	20.488ms	49	50	98.00
V2	key_error	kmac_key_error	10.880s	19.777ms	50	50	100.00
V2	edn_timeout_error	kmac_edn_timeout_error	42.950s	1.936ms	20	20	100.00
V2	entropy_mode_error	kmac_entropy_mode_error	49.610s	10.390ms	20	20	100.00
V2	entropy_ready_error	kmac_entropy_ready_error	1.099m	7.659ms	10	10	100.00
V2	lc_escalation	kmac_lc_escalation	31.120s	578.960us	50	50	100.00
V2	stress_all	kmac_stress_all	34.147m	335.514ms	49	50	98.00
V2	intr_test	kmac_intr_test	0.930s	15.515us	50	50	100.00
V2	alert_test	kmac_alert_test	0.950s	218.999us	50	50	100.00
V2	tl_d_oob_addr_access	kmac_tl_errors	3.060s	861.363us	20	20	100.00
V2	tl_d_illegal_access	kmac_tl_errors	3.060s	861.363us	20	20	100.00
V2	tl_d_outstanding_access	kmac_csr_hw_reset	1.160s	52.750us	5	5	100.00
		kmac_csr_rw	1.200s	37.347us	20	20	100.00
		kmac_csr_aliasing	9.950s	490.848us	5	5	100.00
		kmac_same_csr_outstanding	2.540s	227.450us	20	20	100.00
V2	tl_d_partial_access	kmac_csr_hw_reset	1.160s	52.750us	5	5	100.00
		kmac_csr_rw	1.200s	37.347us	20	20	100.00
		kmac_csr_aliasing	9.950s	490.848us	5	5	100.00
		kmac_same_csr_outstanding	2.540s	227.450us	20	20	100.00
V2		TOTAL			1045	1050	99.52
V2S	shadow_reg_update_error	kmac_shadow_reg_errors	1.380s	35.702us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	kmac_shadow_reg_errors	1.380s	35.702us	20	20	100.00
V2S	shadow_reg_storage_error	kmac_shadow_reg_errors	1.380s	35.702us	20	20	100.00
V2S	shadowed_reset_glitch	kmac_shadow_reg_errors	1.380s	35.702us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	kmac_shadow_reg_errors_with_csr_rw	3.090s	1.034ms	20	20	100.00
V2S	tl_intg_err	kmac_sec_cm	1.302m	43.104ms	5	5	100.00
		kmac_tl_intg_err	4.840s	492.131us	20	20	100.00
V2S	sec_cm_bus_integrity	kmac_tl_intg_err	4.840s	492.131us	20	20	100.00
V2S	sec_cm_lc_escalate_en_intersig_mubi	kmac_lc_escalation	31.120s	578.960us	50	50	100.00
V2S	sec_cm_sw_key_key_masking	kmac_smoke	1.313m	56.758ms	50	50	100.00
V2S	sec_cm_key_sideload	kmac_sideload	6.788m	19.116ms	49	50	98.00
V2S	sec_cm_cfg_shadowed_config_shadow	kmac_shadow_reg_errors	1.380s	35.702us	20	20	100.00
V2S	sec_cm_fsm_sparse	kmac_sec_cm	1.302m	43.104ms	5	5	100.00
V2S	sec_cm_ctr_redun	kmac_sec_cm	1.302m	43.104ms	5	5	100.00
V2S	sec_cm_packer_ctr_redun	kmac_sec_cm	1.302m	43.104ms	5	5	100.00
V2S	sec_cm_cfg_shadowed_config_regwen	kmac_smoke	1.313m	56.758ms	50	50	100.00
V2S	sec_cm_fsm_global_esc	kmac_lc_escalation	31.120s	578.960us	50	50	100.00
V2S	sec_cm_fsm_local_esc	kmac_sec_cm	1.302m	43.104ms	5	5	100.00
V2S	sec_cm_absorbed_ctrl_mubi	kmac_mubi	3.106m	12.957ms	10	10	100.00
V2S	sec_cm_sw_cmd_ctrl_sparse	kmac_smoke	1.313m	56.758ms	50	50	100.00
V2S		TOTAL			75	75	100.00
V3	throughput	kmac_throughput			0	0	--
V3	stress_all_with_rand_reset	kmac_stress_all_with_rand_reset	27.902m	63.923ms	7	50	14.00
V3		TOTAL			7	50	14.00
		TOTAL			1242	1290	96.28

Testplan Progress

Items	Total	Written	Passing	Progress
V1	8	8	8	100.00
V2	25	25	21	84.00
V2S	5	5	5	100.00
V3	2	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
95.27	96.18	92.13	100.00	88.64	94.52	98.84	96.60

Failure Buckets

• UVM_ERROR (cip_base_vseq.sv:829) [kmac_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 37 failures:

  • Test kmac_stress_all_with_rand_reset has 37 failures.

    • 1.kmac_stress_all_with_rand_reset.27117276902027277768130891359797370622509470053894696736454672975983659603793
      Line 1194, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/1.kmac_stress_all_with_rand_reset/latest/run.log

        UVM_ERROR @ 176027316195 ps: (cip_base_vseq.sv:829) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
        UVM_INFO @ 176027316195 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
        --- UVM Report catcher Summary ---
        
        
      

    • 2.kmac_stress_all_with_rand_reset.88150249658293023815696331240804625913234283008245929705916525056253436693501
      Line 476, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/2.kmac_stress_all_with_rand_reset/latest/run.log

        UVM_ERROR @ 3176430492 ps: (cip_base_vseq.sv:829) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
        UVM_INFO @ 3176430492 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
        --- UVM Report catcher Summary ---
        
        
      

    • ... and 35 more failures.

• UVM_ERROR (kmac_scoreboard.sv:1175) [scoreboard] Check failed csr.get_mirrored_value() == item.d_data (* [*] vs * [*]) reg name: kmac_reg_block.err_code has 6 failures:

  • Test kmac_stress_all_with_rand_reset has 6 failures.

    • 7.kmac_stress_all_with_rand_reset.106617749866276169080953960438246061003642868466464150780061659816200169220396
      Line 2314, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/7.kmac_stress_all_with_rand_reset/latest/run.log

        UVM_ERROR @ 79215195466 ps: (kmac_scoreboard.sv:1175) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (0 [0x0] vs 2147483704 [0x80000038]) reg name: kmac_reg_block.err_code
        UVM_INFO @ 79215195466 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
        --- UVM Report catcher Summary ---
        
        
      

    • 8.kmac_stress_all_with_rand_reset.84777807799128617905089089415255325141377955740465528845682180522043700452592
      Line 337, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/8.kmac_stress_all_with_rand_reset/latest/run.log

        UVM_ERROR @ 830620135 ps: (kmac_scoreboard.sv:1175) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (0 [0x0] vs 2147483736 [0x80000058]) reg name: kmac_reg_block.err_code
        UVM_INFO @ 830620135 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
        --- UVM Report catcher Summary ---
        
        
      

    • ... and 4 more failures.

• UVM_FATAL (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of * ps hit, indicating a probable testbench issue has 4 failures:

  • Test kmac_sideload has 1 failures.

    • 6.kmac_sideload.3581136557447491673190033836856765166713202117075234452572507915075176562059
      Line 1251, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/6.kmac_sideload/latest/run.log

        UVM_FATAL @ 200000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 200000000000 ps hit, indicating a probable testbench issue
        UVM_INFO @ 200000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
        --- UVM Report catcher Summary ---
      

  • Test kmac_error has 1 failures.

    • 24.kmac_error.92999874046426539213278976653470123851229037561320345060189904008811344403699
      Line 901, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/24.kmac_error/latest/run.log

        UVM_FATAL @ 200000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 200000000000 ps hit, indicating a probable testbench issue
        UVM_INFO @ 200000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
        --- UVM Report catcher Summary ---
      

  • Test kmac_burst_write has 2 failures.

    • 28.kmac_burst_write.9738846300250581044211643246089477511329666827566775333209639279012648904144
      Line 632, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/28.kmac_burst_write/latest/run.log

        UVM_FATAL @ 200000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 200000000000 ps hit, indicating a probable testbench issue
        UVM_INFO @ 200000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
        --- UVM Report catcher Summary ---
        
        
      

    • 45.kmac_burst_write.67931402487945139875558620125289026763508429225268403479747724714547316959989
      Line 777, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/45.kmac_burst_write/latest/run.log

        UVM_FATAL @ 200000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 200000000000 ps hit, indicating a probable testbench issue
        UVM_INFO @ 200000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
        --- UVM Report catcher Summary ---
      

• UVM_FATAL (kmac_scoreboard.sv:1519) [scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (* [*] vs * [*]) Mismatch between unmasked_digest[*] and dpi_digest[*] has 1 failures:

  • Test kmac_stress_all has 1 failures.

    • 37.kmac_stress_all.57764296671949555270001054753533023463532728182026340933010271405487795299199
      Line 977, in log /container/opentitan-public/scratch/os_regression/kmac_unmasked-sim-vcs/37.kmac_stress_all/latest/run.log

        UVM_FATAL @ 4398557531 ps: (kmac_scoreboard.sv:1519) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (232 [0xe8] vs 121 [0x79]) Mismatch between unmasked_digest[0] and dpi_digest[0]
        UVM_INFO @ 4398557531 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
        --- UVM Report catcher Summary ---
      

Past Results

• Latest
• 2024.04.16_04.11.25
• 2024.04.12_19.14.08
• 2024.04.08_19.16.41
• 2024.04.05_04.21.30
• 2024.04.03_03.40.12
• 2024.04.01_04.36.51
• 2024.03.29_04.05.36