Minor fix: dff8 reference design

[luca-valente]

Hello,

I found a minor issue with the dff8 reference design.

The reference solution implements a

  initial
    q = 8'h0;

which makes the test fail when the LLM's answer is simply

module top_module (
  input clk,
  input [7:0] d,
  output reg [7:0] q
);

  always @(posedge clk)
    q <= d;

endmodule

which in my opinion is correct.

Thank you for your attention!!

Luca

[npfet]

Hi Luca, good catch- I will add this to a list of fixes to get in before an upcoming update. We have a handful of consistency fixes for the VerilogEval v2 release as well (I'd expect it to be pushed somewhere in the next month or two... we're mostly finalizing benchmarking SOTA LLMs against v2 internally before pushing the fixes). We will only maintain an errata list for the v1 release since we want to have a consistent dataset to compare against prior papers.

Thanks,

Nate

[luca-valente-sony]

Thank you for the feedback 😄