Image Visual Check

Adding image processing tests and checking them visually

With numeric asserts alone, image processing bugs like vertical flips, swapped RGB, or shifted interpolation are easy to miss. So pin down the spec with numeric tests, then build a comparison image that lays out input | Matft | OpenCV | diff side by side, and have both Claude and the user check it visually. The comparison images are committed to the repository so they can be reviewed in the PR.

How it works

ImageSnapshot.save writes files only when run with the environment variable MATFT_IMAGE_SNAPSHOT=1, so that a regular swift test does not modify files in the repository.

0. Pre-checks

python3 -c "import cv2, numpy; print(cv2.__version__, numpy.__version__)"

If cv2 is missing, suggest pip3 install --user opencv-python-headless. Install only with the user's consent.

1. Write the test (Red)

Follow TDD as CLAUDE.md requires. If ImageTest.swift does not exist, create it in this form. Wrap the whole file in #if so it still builds where Accelerate/ImageIO are unavailable (WASI, Linux).

#if canImport(Accelerate) && canImport(ImageIO)
import XCTest

@testable import Matft

final class ImageTest: XCTestCase {
    func test_resize() {
        let image = Matft.image.cgimage2mfarray(ImageSnapshot.loadFixture())   // Float [0, 1], RGBA, shape=(225, 225, 4)
        let ret = Matft.image.resize(image, width: 300, height: 150)

        XCTAssertEqual(ret.shape, [150, 300, 4])
        XCTAssertEqual(ret.mftype, .Float)
        // Assert whatever can be checked numerically, e.g. representative pixel values

        ImageSnapshot.check(ret, as: "resize_300x150")
    }
}
#endif

Tips for numeric asserts:

• Get expected values by computing them from rena.png in Python (numpy / cv2). Write the Python expression in a comment so it is clear where the embedded values come from.
• For operations involving interpolation (resize, warpAffine), vImage and OpenCV do not match pixel for pixel. So assert algorithm-independent properties: shape, dtype, value range, pixels in flat regions, border values (warpAffine's borderValue), etc.
• For operations that are exactly defined (flip, channel swap, grayscale), compare representative pixels with the numpy / cv2 values (divide by 255 for Float; tolerance around 1e-2).
• Give each check a case name that tells the operation and its conditions (e.g. warpAffine_rotate30_edgeExtend).
• check fails until the case has a tolerance in ImageSnapshot.tolerances and a committed reference opencv/<case>.png (step 4). Pick .exact, .rounding(n) or .interpolation(meanAbs:minPSNR:) from the metrics of step 4, with a little margin, and write the measured values in a comment.

Confirm it fails with swift test --filter MatftTests.ImageTest.

2. Register the OpenCV version

Add the OpenCV operation to CASES in scripts/image_compare.py under the same case name.

"resize_300x150": Case("rena.png",
                       lambda x: cv2.resize(x, (300, 150), interpolation=cv2.INTER_LANCZOS4),
                       "Matft.image.resize(width: 300, height: 150) vs cv2.resize(LANCZOS4)"),

op receives an RGBA uint8 array. It may return either uint8 or float in [0, 1] (converted to uint8 automatically). Differences in conventions between Matft and OpenCV often make a correct result look "off". Watch for:

• Channel order: Matft is RGBA, OpenCV is BGR(A). The script passes RGBA, so do not use BGR-based conversions (COLOR_BGR2GRAY, etc.) inside op.
• Size arguments: cv2.resize takes (width, height); Matft's shape is (height, width, channel).
• Value range: Matft's Float images are [0, 1]; OpenCV uses 0–255. Multiply borderValue and the like by 255.
• Interpolation: vImage's resize uses Lanczos-style interpolation, whose edges differ from INTER_LINEAR. Pick the closest interpolation and state in the description what it was compared against.
• Savable channel counts: mfarray2cgimage supports only 1 and 4 channels. For results with 3 channels, such as RGBA2RGB, convert back to 4 channels with Matft.image.color(ret, conversion: .RGB2RGBA) before save. Do the same on the OpenCV side.

3. Implement and pass the test (Green)

Write the minimal implementation that passes the test, and confirm all tests pass with swift test. When no change to Matft itself is needed (just adding tests and a visual check to an existing feature), this step is only a check.