Files marked different - but actually are not

[baumaeschi]

When using Winmerge to compare two folder/file trees, I realized that many files were marked grey to exist "only right" or "only left" but exist on both sides and their contents are equal. Can anyone explain to me why these files are marked as shown in the attached example?

[sdottaka]

This appears to be the case when the file name contains accented characters.
My guess is that one side of ä is represented by Unicode U+00E4, and the other side is represented by Unicode's combining character sequence of U+0061 and U+0308.
If you copy the left and right file names and compare them with WinMerge, you will see the difference.
Unfortunately, WinMerge is currently unable to ignore such differences in the internal representation of file names.

[baumaeschi]

Hi sdottaka
Thank you for your reply. Your guess sounds likely. The only thing I did, was copy these files from my local storage to my cloud provider (pColud). This means Windows and pCloud's Unicode representation of special characters seems to be different. What a pity.

If you copy the left and right file names and compare them with WinMerge, you will see the difference

Could you please explain in detail how I could do such a comparison and see the different encodings?

Unfortunately, WinMerge is currently unable to ignore such differences in the internal representation of file names.

Is this possibly an "Issue" in the further development of Winmerge?

[sdottaka]

Could you please explain in detail how I could do such a comparison and see the different encodings?

Select the file on the left in Explorer or WinMerge and press the F2 key to select the file name, then press Ctrl+C to copy it to the clipboard. Copy the file name on the right in the same way.
Start WinMerge and select File→Open Clipboard menu item to compare the contents of the clipboard.
(However, you must have Windows Clipboard History enabled. You can enable it with Windows+V key)

Is this possibly an "Issue" in the further development of Winmerge?

It won't be implemented anytime soon. I think this will come after the ability to ignore differences in specific patterns in file names is achieved.

[baumaeschi]

Thank you for the step-by-step guidance on comparing Clipboard entries. The result is shown in the screenshot below. But I don't know how to analyze the actual difference in the encodings. Is there a way to show the UTF8 encoding in Hexadecimal representation?

[sdottaka]

I think it would be useful to have a tool like the one below that converts input strings into code points and displays them.

https://onlinetools.com/unicode/convert-unicode-to-code-points

[baumaeschi]

Hi, sdottaka
Thanks for the link. Works perfectly. Below the result for comparing "Länzlinger" on the left and the right side:

Länzlinger
pCloud:  U+004C U+0061 U+0308 U+006E U+007A U+006C U+0069 U+006E U+0067 U+0065 U+0072
Windows: U+004C U+00E4        U+006E U+007A U+006C U+0069 U+006E U+0067 U+0065 U+0072

So - your initial guess is absolutely right!

[baumaeschi]

It won't be implemented anytime soon. I think this will come after the ability to ignore differences in specific patterns in file names is achieved.

Could you please comment on what you mean by "differences in specific patterns"?
Does it e.g. mean Länzlinger would be equal to LànzlÎger, Lanzlinger, Laenzlingér, etc. ?

[sdottaka]

Could you please comment on what you mean by "differences in specific patterns"?

Maybe I should have said "specified patterns" instead of "specific patterns".

There are some requests to ignore differences in file names and compare them as shown below. To achieve this, it is necessary to introduce a mechanism to specify or map the part of the file name that you want to ignore using a pattern.
#993
#1469
#1301
#1552
#2102

[baumaeschi]

Ok, thank you. I can now understand in which direction this feature "Ignore certain parts of a filename when comparing" will go. Something different to my problem "Taking into account different UTF8 encoding schemes".

But - I have another output of winmerge, where differences are recogniced for the left and the right side, but I can't see any. Could you please study the screenshot below and tell me your assumptions for this results of winmerge.

I've marked 2 differences in blue. For me they are identical. This is also true for the rest of the comparison (grey)

[sdottaka]

The parent folder name of the selected files "CD Hüllen" contains accented characters, so it is considered a different folder, and the match was not made.

[baumaeschi]

Ohhh...thank you for the hint. The same problem as with the files (sorry, I've concentrated on the file names, not the directory names)
Is there already a feature request to make allowance for different UTF8 encodings?

[sdottaka]

There is probably no such feature request.

[baumaeschi]

Could you make such a feature request? Or shall I do it? Or shall I write an "Issue" in git and you mark it as a feature request?
Sorry, but I don't know how such "Wishes" are usually handled so that they are not forgotten.

[sdottaka]

Created issue #2132