KMAC/MASKED Simulation Results

Wednesday February 07 2024 20:02:46 UTC

GitHub Revision: `5c87d18988`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 42540109002295994234923032062842839138270099951232798724643629525632267455156

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	kmac_smoke	1.583m	10.424ms	49	50	98.00
V1	csr_hw_reset	kmac_csr_hw_reset	1.260s	37.887us	5	5	100.00
V1	csr_rw	kmac_csr_rw	1.250s	57.519us	20	20	100.00
V1	csr_bit_bash	kmac_csr_bit_bash	26.310s	11.855ms	5	5	100.00
V1	csr_aliasing	kmac_csr_aliasing	11.290s	554.878us	5	5	100.00
V1	csr_mem_rw_with_rand_reset	kmac_csr_mem_rw_with_rand_reset	2.430s	107.481us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	kmac_csr_rw	1.250s	57.519us	20	20	100.00
		kmac_csr_aliasing	11.290s	554.878us	5	5	100.00
V1	mem_walk	kmac_mem_walk	0.770s	11.984us	5	5	100.00
V1	mem_partial_access	kmac_mem_partial_access	1.420s	17.664us	5	5	100.00
V1		TOTAL			114	115	99.13
V2	long_msg_and_output	kmac_long_msg_and_output	58.627m	89.460ms	49	50	98.00
V2	burst_write	kmac_burst_write	26.118m	57.806ms	50	50	100.00
V2	test_vectors	kmac_test_vectors_sha3_224	44.122m	399.000ms	49	50	98.00
		kmac_test_vectors_sha3_256	42.279m	188.942ms	50	50	100.00
		kmac_test_vectors_sha3_384	35.166m	509.529ms	50	50	100.00
		kmac_test_vectors_sha3_512	25.602m	546.695ms	50	50	100.00
		kmac_test_vectors_shake_128	1.972h	1.536s	48	50	96.00
		kmac_test_vectors_shake_256	1.692h	1.878s	50	50	100.00
		kmac_test_vectors_kmac	7.080s	768.471us	50	50	100.00
		kmac_test_vectors_kmac_xof	7.960s	3.352ms	50	50	100.00
V2	sideload	kmac_sideload	10.079m	85.292ms	50	50	100.00
V2	app	kmac_app	6.907m	20.882ms	50	50	100.00
V2	app_with_partial_data	kmac_app_with_partial_data	5.632m	12.475ms	9	10	90.00
V2	entropy_refresh	kmac_entropy_refresh	7.305m	30.912ms	50	50	100.00
V2	error	kmac_error	9.256m	166.850ms	49	50	98.00
V2	key_error	kmac_key_error	7.120s	1.138ms	50	50	100.00
V2	edn_timeout_error	kmac_edn_timeout_error	54.660s	2.597ms	20	20	100.00
V2	entropy_mode_error	kmac_entropy_mode_error	13.080s	391.226us	20	20	100.00
V2	entropy_ready_error	kmac_entropy_ready_error	1.300m	28.170ms	10	10	100.00
V2	lc_escalation	kmac_lc_escalation	1.024m	961.140us	50	50	100.00
V2	stress_all	kmac_stress_all	48.175m	210.411ms	47	50	94.00
V2	intr_test	kmac_intr_test	0.930s	14.873us	50	50	100.00
V2	alert_test	kmac_alert_test	0.930s	21.687us	50	50	100.00
V2	tl_d_oob_addr_access	kmac_tl_errors	4.000s	472.901us	20	20	100.00
V2	tl_d_illegal_access	kmac_tl_errors	4.000s	472.901us	20	20	100.00
V2	tl_d_outstanding_access	kmac_csr_hw_reset	1.260s	37.887us	5	5	100.00
		kmac_csr_rw	1.250s	57.519us	20	20	100.00
		kmac_csr_aliasing	11.290s	554.878us	5	5	100.00
		kmac_same_csr_outstanding	3.080s	120.115us	20	20	100.00
V2	tl_d_partial_access	kmac_csr_hw_reset	1.260s	37.887us	5	5	100.00
		kmac_csr_rw	1.250s	57.519us	20	20	100.00
		kmac_csr_aliasing	11.290s	554.878us	5	5	100.00
		kmac_same_csr_outstanding	3.080s	120.115us	20	20	100.00
V2		TOTAL			1041	1050	99.14
V2S	shadow_reg_update_error	kmac_shadow_reg_errors	1.370s	160.743us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	kmac_shadow_reg_errors	1.370s	160.743us	20	20	100.00
V2S	shadow_reg_storage_error	kmac_shadow_reg_errors	1.370s	160.743us	20	20	100.00
V2S	shadowed_reset_glitch	kmac_shadow_reg_errors	1.370s	160.743us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	kmac_shadow_reg_errors_with_csr_rw	3.460s	1.282ms	20	20	100.00
V2S	tl_intg_err	kmac_sec_cm	1.962m	9.334ms	5	5	100.00
		kmac_tl_intg_err	6.260s	997.621us	20	20	100.00
V2S	sec_cm_bus_integrity	kmac_tl_intg_err	6.260s	997.621us	20	20	100.00
V2S	sec_cm_lc_escalate_en_intersig_mubi	kmac_lc_escalation	1.024m	961.140us	50	50	100.00
V2S	sec_cm_sw_key_key_masking	kmac_smoke	1.583m	10.424ms	49	50	98.00
V2S	sec_cm_key_sideload	kmac_sideload	10.079m	85.292ms	50	50	100.00
V2S	sec_cm_cfg_shadowed_config_shadow	kmac_shadow_reg_errors	1.370s	160.743us	20	20	100.00
V2S	sec_cm_fsm_sparse	kmac_sec_cm	1.962m	9.334ms	5	5	100.00
V2S	sec_cm_ctr_redun	kmac_sec_cm	1.962m	9.334ms	5	5	100.00
V2S	sec_cm_packer_ctr_redun	kmac_sec_cm	1.962m	9.334ms	5	5	100.00
V2S	sec_cm_cfg_shadowed_config_regwen	kmac_smoke	1.583m	10.424ms	49	50	98.00
V2S	sec_cm_fsm_global_esc	kmac_lc_escalation	1.024m	961.140us	50	50	100.00
V2S	sec_cm_fsm_local_esc	kmac_sec_cm	1.962m	9.334ms	5	5	100.00
V2S	sec_cm_absorbed_ctrl_mubi	kmac_mubi	7.165m	52.558ms	9	10	90.00
V2S	sec_cm_sw_cmd_ctrl_sparse	kmac_smoke	1.583m	10.424ms	49	50	98.00
V2S		TOTAL			74	75	98.67
V3	throughput	kmac_throughput			0	0	--
V3	stress_all_with_rand_reset	kmac_stress_all_with_rand_reset	1.286h	173.581ms	36	50	72.00
V3		TOTAL			36	50	72.00
		TOTAL			1265	1290	98.06

Testplan Progress

Items	Total	Written	Passing	Progress
V1	8	8	7	87.50
V2	25	25	19	76.00
V2S	5	5	4	80.00
V3	2	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
97.29	98.38	93.14	99.93	96.36	96.04	98.89	98.31

Failure Buckets

UVM_ERROR (kmac_scoreboard.sv:1176) [scoreboard] Check failed csr.get_mirrored_value() == item.d_data (* [*] vs * [*]) reg name: kmac_reg_block.err_code has 12 failures:
- Test kmac_stress_all_with_rand_reset has 12 failures.
  - 0.kmac_stress_all_with_rand_reset.31448920053876042963312559292647742755999110248417640452624029190499952712024
    Line 570, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/0.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 2974356578 ps: (kmac_scoreboard.sv:1176) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (0 [0x0] vs 2147483688 [0x80000028]) reg name: kmac_reg_block.err_code
  UVM_INFO @ 2974356578 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 3.kmac_stress_all_with_rand_reset.29861337637833641447395732761672228937988253723553505418407608609629502002469
    Line 2359, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/3.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 281255450132 ps: (kmac_scoreboard.sv:1176) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (0 [0x0] vs 2147483656 [0x80000008]) reg name: kmac_reg_block.err_code
  UVM_INFO @ 281255450132 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 10 more failures.

UVM_FATAL (kmac_scoreboard.sv:1520) [scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (* [*] vs * [*]) Mismatch between unmasked_digest[*] and dpi_digest[*] has 3 failures:

Test kmac_app_with_partial_data has 1 failures.

1.kmac_app_with_partial_data.90949867747850219354248800636643962252808635617691113618576247854682642676714
Line 594, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/1.kmac_app_with_partial_data/latest/run.log

  UVM_FATAL @ 23951435431 ps: (kmac_scoreboard.sv:1520) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (174 [0xae] vs 185 [0xb9]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 23951435431 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test kmac_stress_all has 2 failures.

20.kmac_stress_all.105474896643033693362129653679655192925985760509303081968347775338705688970462
Line 1266, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/20.kmac_stress_all/latest/run.log

  UVM_FATAL @ 23989652957 ps: (kmac_scoreboard.sv:1520) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (187 [0xbb] vs 181 [0xb5]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 23989652957 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

42.kmac_stress_all.101521119970783876205631120193145680739639066608049033244741198297800140532697
Line 2256, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/42.kmac_stress_all/latest/run.log

  UVM_FATAL @ 114041103355 ps: (kmac_scoreboard.sv:1520) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (79 [0x4f] vs 217 [0xd9]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 114041103355 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_ERROR (csr_utils_pkg.sv:457) [csr_utils::csr_rd_check] Check failed obs == exp (* [*] vs * [*]) Regname: kmac_reg_block.intr_state.kmac_done reset value: * has 3 failures:
- Test kmac_test_vectors_sha3_224 has 1 failures.
  - 6.kmac_test_vectors_sha3_224.68529916336757401399040950905678853627303448068637754398470048264231609683427
    Line 278, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/6.kmac_test_vectors_sha3_224/latest/run.log
```
  UVM_ERROR @ 326592729 ps: (csr_utils_pkg.sv:457) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @ 326592729 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test kmac_test_vectors_shake_128 has 1 failures.
  - 7.kmac_test_vectors_shake_128.62525319660006546685921713309818347973857829803877173552468774756704195388377
    Line 278, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/7.kmac_test_vectors_shake_128/latest/run.log
```
  UVM_ERROR @  45799247 ps: (csr_utils_pkg.sv:457) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @  45799247 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test kmac_smoke has 1 failures.
  - 12.kmac_smoke.2913617124151601852747968613979152972047330449451944512507018802808593151657
    Line 279, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/12.kmac_smoke/latest/run.log
```
  UVM_ERROR @ 120609614 ps: (csr_utils_pkg.sv:457) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @ 120609614 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_FATAL (kmac_scoreboard.sv:218) [scoreboard] Wait EDN request has 2 failures:
- Test kmac_stress_all has 1 failures.
  - 0.kmac_stress_all.47725883412514618817059936669650972184061250599783290671415908424512999544173
    Line 451, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/0.kmac_stress_all/latest/run.log
```
  UVM_FATAL @ 10923150444 ps: (kmac_scoreboard.sv:218) [uvm_test_top.env.scoreboard] Wait EDN request
  UVM_INFO @ 10923150444 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test kmac_error has 1 failures.
  - 22.kmac_error.57553060074998956845227907450549833445318415028538743303901389048843614963862
    Line 473, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/22.kmac_error/latest/run.log
```
  UVM_FATAL @ 10048712789 ps: (kmac_scoreboard.sv:218) [uvm_test_top.env.scoreboard] Wait EDN request
  UVM_INFO @ 10048712789 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_FATAL (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of * ps hit, indicating a probable testbench issue has 2 failures:
- Test kmac_mubi has 1 failures.
  - 6.kmac_mubi.10812393111737096065691800423642350569580833257441937202693416976794313979578
    Line 918, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/6.kmac_mubi/latest/run.log
```
  UVM_FATAL @ 200000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 200000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 200000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test kmac_test_vectors_shake_128 has 1 failures.
  - 20.kmac_test_vectors_shake_128.106495776434732442875531800505248737914739732782902563169936390862874443340324
    Line 3909, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/20.kmac_test_vectors_shake_128/latest/run.log
```
  UVM_FATAL @ 5000000000000 ps: (uvm_phase.svh:1512) [PH_TIMEOUT] Explicit timeout of 5000000000000 ps hit, indicating a probable testbench issue
  UVM_INFO @ 5000000000000 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
Job kmac_masked-sim-vcs_run_default killed due to: Exit reason: User job exceeded runlimit: User job timed out has 1 failures:
- Test kmac_long_msg_and_output has 1 failures.
  - 16.kmac_long_msg_and_output.52367221724837430574372701524553058101420936307019292773567494742492375316790
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/16.kmac_long_msg_and_output/latest/run.log
```
  Job ID: smart:2d33cd54-5bb5-4a29-9745-f9e8d9e52edb
```
UVM_FATAL (kmac_test_vectors_base_vseq.sv:125) [kmac_test_vectors_kmac_xof_vseq] Check failed exp_digest[i] == act_digest[i] (* [*] vs * [*]) Mismatch between exp_digest[*] and act_digest[*] has 1 failures:
- Test kmac_stress_all_with_rand_reset has 1 failures.
  - 23.kmac_stress_all_with_rand_reset.5207642129883929064718613438451407408839471563043633432390679013429471103276
    Line 708, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/23.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_FATAL @ 28859458475 ps: (kmac_test_vectors_base_vseq.sv:125) [uvm_test_top.env.virtual_sequencer.kmac_test_vectors_kmac_xof_vseq] Check failed exp_digest[i] == act_digest[i] (23 [0x17] vs 252 [0xfc]) Mismatch between exp_digest[0] and act_digest[0]
  UVM_INFO @ 28859458475 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_FATAL (kmac_test_vectors_base_vseq.sv:125) [kmac_test_vectors_kmac_vseq] Check failed exp_digest[i] == act_digest[i] (* [*] vs * [*]) Mismatch between exp_digest[*] and act_digest[*] has 1 failures:
- Test kmac_stress_all_with_rand_reset has 1 failures.
  - 39.kmac_stress_all_with_rand_reset.92896326970859168844669506921920777127326511373043053140028221197268628114833
    Line 760, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/39.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_FATAL @ 92793347617 ps: (kmac_test_vectors_base_vseq.sv:125) [uvm_test_top.env.virtual_sequencer.kmac_test_vectors_kmac_vseq] Check failed exp_digest[i] == act_digest[i] (31 [0x1f] vs 138 [0x8a]) Mismatch between exp_digest[0] and act_digest[0]
  UVM_INFO @ 92793347617 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```