Add support for alternative decimal separator

[jschueller]

I would like to add support to use comma as decimal separator instead of dot
it is possible to do this by overloading std::numpunct and instanciating a custom locale from this:

  struct CSVParserFormat : std::numpunct<char>
  {
    explicit CSVParserFormat(const char_type decimalSeparator)
    : decimalSeparator_(decimalSeparator) {}

    char_type do_decimal_point() const { return decimalSeparator_; }
    char_type decimalSeparator_;
  };
...
std::istringstream iss(pStr);
iss.imbue(std::locale(std::locale::classic(), new CSVParserFormat(decimalSeparator_)));
double pVal = 0.0;
iss >> pVal;

what do you think ?

[d99kris]

Hi @jschueller - thanks for suggesting a feature. Similar to #180 I am a little hesitant, as I like to keep this library small and simple.

Would your use case be possible to solve by simply creating a stream, setting its locale and then using the Document constructor accepting a stream? Alternatively using custom data conversion routines.

Btw, please take note of mNumericLocale introduced for #102 which allows disabling locale-dependent parsing.

[jschueller]

I dont know if I can use the Converter<>::ToVal converter since the numerical separator is an attribute of one of my classes and I cannot access it from the converter template (the accessor to this attribute cannot be global)
So for now I have to loop over the array of strings and convert it myself which is a loat of code that should be done by rapidcsv itself

[d99kris]

So rapidcsv allows for a per-call converter (in addition the global converter). So let's say you have a semi-colon separated document with , as decimal separator you should be able to do something like this:

#include <iostream>
#include <vector>
#include "rapidcsv.h"

void WriteFile(const std::string& pPath, const std::string& pData)
{
  std::ofstream outfile;
  outfile.open(pPath, std::ifstream::out | std::ifstream::binary);
  outfile << pData;
  outfile.close();
}

struct CSVParserFormat : std::numpunct<char>
{
  explicit CSVParserFormat(const char_type decimalSeparator)
    : decimalSeparator_(decimalSeparator) {}

  char_type do_decimal_point() const { return decimalSeparator_; }
  char_type decimalSeparator_;
};

int main()
{
  std::string csvData =
    "a;b;c\n"
    "-inf;0,123;0,111\n"
    "1,121;inf;2,12\n";

  std::string csvPath = "test.csv";
  WriteFile(csvPath, csvData);

  std::ifstream csvStream(csvPath);

  rapidcsv::Document doc(csvStream,
                         rapidcsv::LabelParams(),
                         rapidcsv::SeparatorParams(';'));

  auto convLambda =
    [](const std::string& pStr, double& pVal)
    {
      static const std::map<std::string, double> infMap =
      {
        { "-inf", -std::numeric_limits<double>::infinity() },
        { "inf", std::numeric_limits<double>::infinity() },
      };

      if (infMap.count(pStr))
      {
        pVal = infMap.find(pStr)->second;
      }
      else
      {
        std::istringstream iss(pStr);
        iss.imbue(std::locale(std::locale::classic(), new CSVParserFormat(',')));
        iss >> pVal;
      }
    };

  auto cells = doc.GetColumn<double>("a", convLambda);
  for (const auto& cell : cells)
  {
    std::cout << cell << "\n";
  }
}

Of course convLambda could capture a local variable if you want dynamic/different decimal separators per call.

It is true that the rapidcsv library could incorporate this type of functionality, but for each suggested feature I have to weigh the benefits vs. "costs" in terms of tests / performance / etc.