KMAC/MASKED Simulation Results

Wednesday August 07 2024 23:02:33 UTC

GitHub Revision: `bbf435ceff`

Branch: os_regression

Simulator: VCS

Build randomization enabled with --build-seed 6888687353677204195542416712589698377810102273194685652880785004967849651007

Test Results

Stage	Name	Tests	Max Job Runtime	Simulated Time	Passing	Total	Pass Rate
V1	smoke	kmac_smoke	1.463m	6.710ms	50	50	100.00
V1	csr_hw_reset	kmac_csr_hw_reset	1.220s	129.042us	5	5	100.00
V1	csr_rw	kmac_csr_rw	1.230s	132.003us	20	20	100.00
V1	csr_bit_bash	kmac_csr_bit_bash	19.350s	1.981ms	5	5	100.00
V1	csr_aliasing	kmac_csr_aliasing	11.530s	2.738ms	5	5	100.00
V1	csr_mem_rw_with_rand_reset	kmac_csr_mem_rw_with_rand_reset	2.660s	386.863us	20	20	100.00
V1	regwen_csr_and_corresponding_lockable_csr	kmac_csr_rw	1.230s	132.003us	20	20	100.00
		kmac_csr_aliasing	11.530s	2.738ms	5	5	100.00
V1	mem_walk	kmac_mem_walk	0.770s	16.431us	5	5	100.00
V1	mem_partial_access	kmac_mem_partial_access	1.480s	124.020us	5	5	100.00
V1		TOTAL			115	115	100.00
V2	long_msg_and_output	kmac_long_msg_and_output	59.761m	440.693ms	39	50	78.00
V2	burst_write	kmac_burst_write	27.235m	56.015ms	50	50	100.00
V2	test_vectors	kmac_test_vectors_sha3_224	59.701m	112.529ms	37	50	74.00
		kmac_test_vectors_sha3_256	59.101m	92.711ms	48	50	96.00
		kmac_test_vectors_sha3_384	48.999m	305.485ms	49	50	98.00
		kmac_test_vectors_sha3_512	33.518m	182.347ms	50	50	100.00
		kmac_test_vectors_shake_128	1.980h	447.532ms	18	50	36.00
		kmac_test_vectors_shake_256	1.637h	754.726ms	19	50	38.00
		kmac_test_vectors_kmac	7.560s	263.069us	50	50	100.00
		kmac_test_vectors_kmac_xof	7.460s	1.106ms	50	50	100.00
V2	sideload	kmac_sideload	11.215m	42.190ms	50	50	100.00
V2	app	kmac_app	7.832m	60.278ms	49	50	98.00
V2	app_with_partial_data	kmac_app_with_partial_data	6.319m	17.419ms	10	10	100.00
V2	entropy_refresh	kmac_entropy_refresh	7.088m	41.045ms	48	50	96.00
V2	error	kmac_error	8.994m	31.548ms	50	50	100.00
V2	key_error	kmac_key_error	23.670s	35.027ms	49	50	98.00
V2	edn_timeout_error	kmac_edn_timeout_error	52.660s	7.136ms	20	20	100.00
V2	entropy_mode_error	kmac_entropy_mode_error	30.150s	1.950ms	20	20	100.00
V2	entropy_ready_error	kmac_entropy_ready_error	1.351m	10.311ms	10	10	100.00
V2	lc_escalation	kmac_lc_escalation	41.600s	2.403ms	50	50	100.00
V2	stress_all	kmac_stress_all	1.116h	90.648ms	49	50	98.00
V2	intr_test	kmac_intr_test	0.930s	16.316us	50	50	100.00
V2	alert_test	kmac_alert_test	0.940s	139.345us	50	50	100.00
V2	tl_d_oob_addr_access	kmac_tl_errors	4.530s	1.851ms	20	20	100.00
V2	tl_d_illegal_access	kmac_tl_errors	4.530s	1.851ms	20	20	100.00
V2	tl_d_outstanding_access	kmac_csr_hw_reset	1.220s	129.042us	5	5	100.00
		kmac_csr_rw	1.230s	132.003us	20	20	100.00
		kmac_csr_aliasing	11.530s	2.738ms	5	5	100.00
		kmac_same_csr_outstanding	2.520s	395.724us	20	20	100.00
V2	tl_d_partial_access	kmac_csr_hw_reset	1.220s	129.042us	5	5	100.00
		kmac_csr_rw	1.230s	132.003us	20	20	100.00
		kmac_csr_aliasing	11.530s	2.738ms	5	5	100.00
		kmac_same_csr_outstanding	2.520s	395.724us	20	20	100.00
V2		TOTAL			955	1050	90.95
V2S	shadow_reg_update_error	kmac_shadow_reg_errors	1.520s	65.216us	20	20	100.00
V2S	shadow_reg_read_clear_staged_value	kmac_shadow_reg_errors	1.520s	65.216us	20	20	100.00
V2S	shadow_reg_storage_error	kmac_shadow_reg_errors	1.520s	65.216us	20	20	100.00
V2S	shadowed_reset_glitch	kmac_shadow_reg_errors	1.520s	65.216us	20	20	100.00
V2S	shadow_reg_update_error_with_csr_rw	kmac_shadow_reg_errors_with_csr_rw	3.250s	489.732us	19	20	95.00
V2S	tl_intg_err	kmac_sec_cm	1.507m	6.643ms	5	5	100.00
		kmac_tl_intg_err	5.020s	194.119us	20	20	100.00
V2S	sec_cm_bus_integrity	kmac_tl_intg_err	5.020s	194.119us	20	20	100.00
V2S	sec_cm_lc_escalate_en_intersig_mubi	kmac_lc_escalation	41.600s	2.403ms	50	50	100.00
V2S	sec_cm_sw_key_key_masking	kmac_smoke	1.463m	6.710ms	50	50	100.00
V2S	sec_cm_key_sideload	kmac_sideload	11.215m	42.190ms	50	50	100.00
V2S	sec_cm_cfg_shadowed_config_shadow	kmac_shadow_reg_errors	1.520s	65.216us	20	20	100.00
V2S	sec_cm_fsm_sparse	kmac_sec_cm	1.507m	6.643ms	5	5	100.00
V2S	sec_cm_ctr_redun	kmac_sec_cm	1.507m	6.643ms	5	5	100.00
V2S	sec_cm_packer_ctr_redun	kmac_sec_cm	1.507m	6.643ms	5	5	100.00
V2S	sec_cm_cfg_shadowed_config_regwen	kmac_smoke	1.463m	6.710ms	50	50	100.00
V2S	sec_cm_fsm_global_esc	kmac_lc_escalation	41.600s	2.403ms	50	50	100.00
V2S	sec_cm_fsm_local_esc	kmac_sec_cm	1.507m	6.643ms	5	5	100.00
V2S	sec_cm_absorbed_ctrl_mubi	kmac_mubi	7.455m	66.346ms	10	10	100.00
V2S	sec_cm_sw_cmd_ctrl_sparse	kmac_smoke	1.463m	6.710ms	50	50	100.00
V2S		TOTAL			74	75	98.67
V3	throughput	kmac_throughput			0	0	--
V3	stress_all_with_rand_reset	kmac_stress_all_with_rand_reset	1.527h	387.973ms	3	10	30.00
V3		TOTAL			3	10	30.00
		TOTAL			1147	1250	91.76

Testplan Progress

Items	Total	Written	Passing	Progress
V1	8	8	8	100.00
V2	25	25	15	60.00
V2S	5	5	4	80.00
V3	2	1	0	0.00

Coverage Results

Coverage Dashboard

SCORE	LINE	COND	TOGGLE	FSM	BRANCH	ASSERT	GROUP
94.00	97.89	92.55	99.89	75.35	95.53	98.89	97.88

Failure Buckets

Job kmac_masked-sim-vcs_run_default killed due to: Exit reason: User job exceeded runlimit: User job timed out has 88 failures:
- Test kmac_test_vectors_shake_256 has 31 failures.
  - 0.kmac_test_vectors_shake_256.50771294869872314187883876696892807525687207394443351644870089262435807623511
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/0.kmac_test_vectors_shake_256/latest/run.log
```
  Job ID: smart:b562a83e-022d-4097-be05-9bce865de3d5
```
  - 1.kmac_test_vectors_shake_256.72793192228547030052613831113327449773864150480962390968959430285003164310255
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/1.kmac_test_vectors_shake_256/latest/run.log
```
  Job ID: smart:518e1be3-942a-4f1c-94ea-be148ffe022c
```
  - ... and 29 more failures.
- Test kmac_test_vectors_shake_128 has 31 failures.
  - 1.kmac_test_vectors_shake_128.65395685958545141250366073729777835662935497709765396080203339016295305790362
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/1.kmac_test_vectors_shake_128/latest/run.log
```
  Job ID: smart:fac228a5-1256-4fec-9638-c98b0d4866dc
```
  - 3.kmac_test_vectors_shake_128.112767394103435099162341527464146425748009602255731060254034344287976307022664
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/3.kmac_test_vectors_shake_128/latest/run.log
```
  Job ID: smart:0fcb3c61-e835-4137-beec-afe8e74ee2f2
```
  - ... and 29 more failures.
- Test kmac_long_msg_and_output has 11 failures.
  - 3.kmac_long_msg_and_output.109665303309404584622607339685495176386877227974634661474418331947834458219059
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/3.kmac_long_msg_and_output/latest/run.log
```
  Job ID: smart:4c2ed18f-f169-42b9-b5a1-a8b82116d44c
```
  - 15.kmac_long_msg_and_output.87514644347119937418905312093105792854406842552422139491313266877133950353172
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/15.kmac_long_msg_and_output/latest/run.log
```
  Job ID: smart:3befa5d8-69b3-4d15-afd4-93df42c98b58
```
  - ... and 9 more failures.
- Test kmac_test_vectors_sha3_224 has 13 failures.
  - 3.kmac_test_vectors_sha3_224.208438610612599039761692702289536506481982455835567388453779325304270650005
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/3.kmac_test_vectors_sha3_224/latest/run.log
```
  Job ID: smart:e8bcf796-957f-4509-85c4-5bc9d4cbb311
```
  - 7.kmac_test_vectors_sha3_224.39932915649325017740747715911917851941901281305600095193310539234641559676282
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/7.kmac_test_vectors_sha3_224/latest/run.log
```
  Job ID: smart:4627fda0-020b-479a-98a8-602db677ecf0
```
  - ... and 11 more failures.
- Test kmac_test_vectors_sha3_256 has 2 failures.
  - 31.kmac_test_vectors_sha3_256.11969687778571455192720884742849219642477815571041186867086685776139906110753
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/31.kmac_test_vectors_sha3_256/latest/run.log
```
  Job ID: smart:420d2a6d-8162-4df7-ac6b-1e404ae3af45
```
  - 44.kmac_test_vectors_sha3_256.61108950816831707616035999467668388056581943208104507133681770736979148107710
    Log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/44.kmac_test_vectors_sha3_256/latest/run.log
```
  Job ID: smart:32372aa9-73fc-4afc-b386-5cbedd33755c
```
UVM_ERROR (cip_base_vseq.sv:836) [kmac_common_vseq] Check failed (!has_outstanding_access()) Waited * cycles to issue a reset with no outstanding accesses. has 5 failures:
- Test kmac_stress_all_with_rand_reset has 5 failures.
  - 1.kmac_stress_all_with_rand_reset.48746062958674904708848882219468351280369429301333912949615506685637472000175
    Line 1177, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/1.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 173211623383 ps: (cip_base_vseq.sv:836) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 173211623383 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 2.kmac_stress_all_with_rand_reset.112209818238379684317964702962334358923400595895434597078102121578888415210330
    Line 329, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/2.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 7531704947 ps: (cip_base_vseq.sv:836) [uvm_test_top.env.virtual_sequencer.kmac_common_vseq] Check failed (!has_outstanding_access()) Waited 10000 cycles to issue a reset with no outstanding accesses.
  UVM_INFO @ 7531704947 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - ... and 3 more failures.

UVM_FATAL (kmac_scoreboard.sv:1638) [scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (* [*] vs * [*]) Mismatch between unmasked_digest[*] and dpi_digest[*] has 3 failures:

Test kmac_entropy_refresh has 2 failures.

29.kmac_entropy_refresh.68635124260662395983123460192689803693551472204792045457806607570454419602361
Line 271, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/29.kmac_entropy_refresh/latest/run.log

  UVM_FATAL @ 499503076 ps: (kmac_scoreboard.sv:1638) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (232 [0xe8] vs 32 [0x20]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 499503076 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

49.kmac_entropy_refresh.66116025603215377681021999310202144720370477083371369499815560791424402300590
Line 489, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/49.kmac_entropy_refresh/latest/run.log

  UVM_FATAL @ 18951023238 ps: (kmac_scoreboard.sv:1638) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (242 [0xf2] vs 135 [0x87]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 18951023238 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Test kmac_app has 1 failures.

37.kmac_app.103666194343277155391117033237132100631910152895418547134033854915506532343483
Line 293, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/37.kmac_app/latest/run.log

  UVM_FATAL @ 1520199676 ps: (kmac_scoreboard.sv:1638) [uvm_test_top.env.scoreboard] Check failed unmasked_digest[i] == dpi_digest[i] (228 [0xe4] vs 191 [0xbf]) Mismatch between unmasked_digest[0] and dpi_digest[0]
  UVM_INFO @ 1520199676 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

UVM_ERROR (csr_utils_pkg.sv:476) [csr_utils::csr_rd_check] Check failed obs == exp (* [*] vs * [*]) Regname: kmac_reg_block.intr_state.kmac_done reset value: * has 3 failures:
- Test kmac_test_vectors_shake_128 has 1 failures.
  - 38.kmac_test_vectors_shake_128.57220424370948576400181629163897479659836956043087208590507613605588127632074
    Line 253, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/38.kmac_test_vectors_shake_128/latest/run.log
```
  UVM_ERROR @  80087828 ps: (csr_utils_pkg.sv:476) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @  80087828 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test kmac_stress_all has 1 failures.
  - 43.kmac_stress_all.94611225651576294738729978842360410338860794417794758521664211972693193729237
    Line 2920, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/43.kmac_stress_all/latest/run.log
```
  UVM_ERROR @ 85817985640 ps: (csr_utils_pkg.sv:476) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @ 85817985640 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
- Test kmac_test_vectors_sha3_384 has 1 failures.
  - 46.kmac_test_vectors_sha3_384.115168521778878923035630890828905227258690315720526444726897425619493257160348
    Line 253, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/46.kmac_test_vectors_sha3_384/latest/run.log
```
  UVM_ERROR @  35098270 ps: (csr_utils_pkg.sv:476) [csr_utils::csr_rd_check] Check failed obs == exp (0 [0x0] vs 1 [0x1]) Regname: kmac_reg_block.intr_state.kmac_done reset value: 0x0
  UVM_INFO @  35098270 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (kmac_scoreboard.sv:1188) [scoreboard] Check failed csr.get_mirrored_value() == item.d_data (* [*] vs * [*]) reg name: kmac_reg_block.err_code has 2 failures:
- Test kmac_stress_all_with_rand_reset has 2 failures.
  - 3.kmac_stress_all_with_rand_reset.19481584902224424691290833324475325875178222546489818244380573727738861781595
    Line 1559, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/3.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 56238779349 ps: (kmac_scoreboard.sv:1188) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (0 [0x0] vs 2147483688 [0x80000028]) reg name: kmac_reg_block.err_code
  UVM_INFO @ 56238779349 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
  
  
```
  - 5.kmac_stress_all_with_rand_reset.115164169245353269406900736377518428311252193006359571481932016785066731864731
    Line 493, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/5.kmac_stress_all_with_rand_reset/latest/run.log
```
  UVM_ERROR @ 10073935569 ps: (kmac_scoreboard.sv:1188) [uvm_test_top.env.scoreboard] Check failed csr.get_mirrored_value() == item.d_data (0 [0x0] vs 2147483744 [0x80000060]) reg name: kmac_reg_block.err_code
  UVM_INFO @ 10073935569 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (csr_utils_pkg.sv:476) [csr_utils::csr_rd_check] Check failed obs == exp (* [*] vs * [*]) Regname: kmac_reg_block.prefix_*.prefix_* reset value: * has 1 failures:
- Test kmac_shadow_reg_errors_with_csr_rw has 1 failures.
  - 13.kmac_shadow_reg_errors_with_csr_rw.1830330670591663214167426312647315132195480392744532302705105052979537532925
    Line 254, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/13.kmac_shadow_reg_errors_with_csr_rw/latest/run.log
```
  UVM_ERROR @  27324731 ps: (csr_utils_pkg.sv:476) [csr_utils::csr_rd_check] Check failed obs == exp (1050095580 [0x3e972fdc] vs 1505698409 [0x59bf2269]) Regname: kmac_reg_block.prefix_3.prefix_0 reset value: 0x0
  UVM_INFO @  27324731 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```
UVM_ERROR (kmac_base_vseq.sv:382) [kmac_key_error_vseq] Check failed (intr_pins[KmacErr] == *) intr_pins[KmacErr] is not set! has 1 failures:
- Test kmac_key_error has 1 failures.
  - 28.kmac_key_error.52920493616205778456411954628164833899042706470534718997111365504379170462162
    Line 273, in log /container/opentitan-public/scratch/os_regression/kmac_masked-sim-vcs/28.kmac_key_error/latest/run.log
```
  UVM_ERROR @ 1373151364 ps: (kmac_base_vseq.sv:382) [uvm_test_top.env.virtual_sequencer.kmac_key_error_vseq] Check failed (intr_pins[KmacErr] == 1) intr_pins[KmacErr] is not set!
  UVM_INFO @ 1373151364 ps: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---
```