Identifier based csv import ignored

[devfaz]

Bug description
I am running Firefly III version 5.4.6 and csv-import 2.5.0, and my problem is:

Im trying to use the "identifier based" deduplication functions, but my transactions get imported twice.

Steps to reproduce

• import the following csv

https://pastebin.com/geC1aVyE

here my config:

https://pastebin.com/iftZDZuN

• import the csv again
• every transaction got imported twice
• already tried with different "Unique column type" - didnt change anything.

Bonus points

• I searched and nobody reported this bug before
• I was able to replicate it on the demo site https://demo.firefly-iii.org/ (does demo has a csv-import?)
• I donated money ;)

[JC5]

I'm sorry, I can't replicate this.

In your config file there is a bracket missing }. Final character. Is this a copy/paste error or something else?

[devfaz]

Strange..

yes, its a copy&paste err, my config contains the missing }.

Maybe an issue with the used database backend? Im using postgres-10 ?

[devfaz]

I migrated my data to "yobasystems/alpine-mariadb:latest" -> issue persists.
I removed all data and started from scratch (empty data-volume) -> issue persists.

I also tried to disable the "set import tag", but even without import-tags, the issue persists.

[devfaz]

some more information:

• im using personal access token
• im behind an reverse proxy (TRUSTED_PROXIES="**")
• my URL and VANITY_URL differ (once internal docker, once external URL)

[devfaz]

I took a look into the logs and it seems csv-importer is trying to do a query/search against the firefly iii api.
cvs-importer is using the correct field:value, but it seems the firefly iii api is answering something like "not found".
I did the request manually and firefly-iii returned:

{
   "meta" : {
      "pagination" : {
         "per_page" : 50,
         "current_page" : 1,
         "total" : 7,
         "total_pages" : 1,
         "count" : 0
      }
   },
   "data" : [],
   "links" : {
      "self" : "http://firefly_iii_app:8080/api/v1/search/transactions?search=internal_reference%3A%2274987501056002277004949%22&page=1",
      "first" : "http://firefly_iii_app:8080/api/v1/search/transactions?search=internal_reference%3A%2274987501056002277004949%22&page=1",
      "last" : "http://firefly_iii_app:8080/api/v1/search/transactions?search=internal_reference%3A%2274987501056002277004949%22&page=1"
   }
}

total: 7 seems to be the number of results, but why is data: empty - bug or feature?

[devfaz]

Offtopic: Did the search in the WebGUI and got the results, the list contains the 7 entries, but the HTML-output contains some wired text: "Firefly III found 50 transactions in 0.01 seconds. " - just FYI.

[devfaz]

took a look in the code.. seems csv-import is just looking for the "count"-field which contains "0", so I would assume this is the bug/issue here.

[devfaz]

And finally.. the current beta version-5.5.0-beta.1 has fixed the issue. Now im getting a filled data[] array and count is no longer zero. Also csv-import is skipping the entries.

[JC5]

Thanks for the deep-dive! The "50 transactions"-bug is fixed in the next release.

I'm not sure why the count is zero. If the next release fixes it, I'll leave it for now. I hope to release 5.5.0 soon.