[Fixing Test-Cases] Functions with image inputs but checking lists

[ian-coccimiglio]

If our test asks for images, I think we should make sure that the test-case is actually providing the images/arrays (rather than requiring the LLM to convert the input into images first)

Here's the sum_images test. The problem is that it's asking for images, but our test function is actually checking lists.

# "sum_images.ipynb"
def check(candidate):
    import numpy as np

    image1 = np.random.random((5,6))
    image2 = np.random.random((5,6))
    sum_image = image1 + image2
    
    assert np.allclose(candidate(image1, image2), sum_image)

    image1 =    [[1,2,3], [4,5,6]]
    image2 =    [[5,6,7], [0,1,2]]
    sum_image = [[6,8,10],[4,6,8]]
    
    assert np.allclose(candidate(image1, image2), sum_image)

This also applies to the mask_images test:

# "mask_image.ipynb"
def check(candidate):
    import numpy as np
    
    image = [
        [2,2,2,2,2],
        [2,2,3,2,2],
        [2,3,3,3,2],
        [2,2,3,2,2],
        [2,2,2,2,2],
    ]
    mask = [
        [0,0,0,0,0],
        [0,0,1,0,0],
        [0,1,1,1,0],
        [0,0,1,0,0],
        [0,0,0,0,0],
    ]
    reference = [
        [0,0,0,0,0],
        [0,0,3,0,0],
        [0,3,3,3,0],
        [0,0,3,0,0],
        [0,0,0,0,0],
    ]
    masked_image = candidate(image,mask)
    assert np.array_equal(masked_image, reference)

As well as mean squared error test:

# "mean_squared_error.ipynb"
def check(candidate):
    image1 = [
        [0,0,0,0,0],
        [0,1,0,0,0],
        [0,0,0,0,0],
        [0,0,0,2,0],
        [0,0,0,0,0],
    ]
    image2 = [
        [0,0,0,0,0],
        [0,1,0,0,0],
        [0,0,0,0,0],
        [0,0,0,2,0],
        [0,0,0,0,0],
    ]

    mse = candidate(image1,image2)
    print(mse)
    assert mse == 0

    image3 = [
        [0,0,0,0,0],
        [0,0,0,0,0],
        [0,0,0,0,0],
        [0,0,0,0,0],
        [0,0,0,0,0],
    ]

    mse = candidate(image1,image3)
    print(mse)
    assert mse == 5 / 25
    

This is just a sample, there may be more places where this is the case. Fixing these issues brings these (below 50%) benchmarks much more in line with our expectations.

Self-assigning this one.