Optimize ESP8266 miner

[mlesniew]

This commit changes how the sequence of strings representing consecutive integers is generated for the hasher. Instead of utilizing the Arduino String conversions, a Counter class is introduced, which can generate consecutive integers much faster.

On my Wemos D1 mini this gives a hash rate increase from around 37 kH/s to around 53 kH/s, which is a 43% improvement.

[revoxhere]

Nice job! Can you check if the ESP32 code can benefit from something similar?

[mlesniew]

The trick can be easily reused in the Arduino miner. To use it in the ESP32 code some additional modifications would be required, but there's potentially even more room for improvement there.

[snakeye]

I'm testing with the ESP-01 module (ESP8266). The update gives a stable improvement over fmt/format.h from 40 to 45 kH/s:

[mlesniew]

@snakeye great that it's working for you!

Still I would expect slightly higher hashrates -- ESP-01 is internally almost the same chip as my Wemos D1 mini. Are you using the unmodified code from this PR or have you made some additional changes?

[mlesniew]

@snakeye I tried running the code on my ESP-01 and was able to reach 53 kH/s, just like on the Wemos D1 Mini with the original miner.

Unfortunately, I needed to disable some features to achieve that. Otherwise the output binary doesn't fit into the chip's smaller flash. I created PR #1581 with changes related to that.

Anyway, my ESP's hashes get rejected -- it seems the server thinks the hashrate was manipulated. @revoxhere, can you advise?

[snakeye]

I'm using my own fork because I'm building the firmware with PlatformIO. But that's the only difference - I have taken the counter code as is. Probably I'm missing something.

My hashes are rejected as well and I received a bunch of emails about suspicious activity in my account.

[snakeye]

I would suggest using recursion and operators for better readability. The performance does not change in this case.

counter.h:

#pragma once

template <unsigned int max_digits>
class Counter
{
public:
    Counter() { reset(); }

    void reset()
    {
        memset(buffer, '0', max_digits);
        buffer[max_digits] = '\0';
        val = 0;
        len = 1;
    }

    Counter &operator++()
    {
        inc_string(max_digits - 1);
        ++val;
        return *this;
    }

    operator unsigned int () const { return val; }
    operator const char *() const { return buffer + max_digits - len; }
    size_t strlen() const { return len; }

protected:
    inline void inc_string(int pos)
    {
        if (pos < 0)
            return;

        if (buffer[pos] < '9')
        {
            buffer[pos]++;
        }
        else
        {
            buffer[pos] = '0';
            inc_string(pos - 1);
        }

        len = max(max_digits - pos, len);
    }

protected:
    char buffer[max_digits + 1];
    unsigned int val;
    size_t len;
};

Usage:

for (Counter<8> counter; counter < context.difficulty; ++counter)
{
  br_sha1_update(&sha1_ctx, (const char*)counter, counter.strlen());
  
  duco_numeric_result = counter;
}

[revoxhere]

Anyway, my ESP's hashes get rejected -- it seems the server thinks the hashrate was manipulated. @revoxhere, can you advise?

Yep, I'll prepare a custom difficulty that will prepare the nodes for these changes.

[mlesniew]

I thought I've added a comment here yesterday, but it seems I haven't published it in the end, so here's a few updates:

• I updated the code to use the recursive approach suggested by @snakeye. Hashrates are the same and the code is more readable. I wish the rest of the file was this tidy!
• I skipped the char * conversion operator and left the c_str() method there instead. The conversion operator would in my opinion lead to ambiguities. Even using Counter instances as String parameters would at least give us warnings. At the same time c_str() follows the conventions used in other classes, including std::string and String itself.
• I experimented with different versions of the code -- the variant submitted now seems to run the fastest. It's only a few hundred hashes per sec, but still. I left these changes in a fixup in case you want to reject it. I understand that we can sacrifice a little bit of efficiency for better readability -- the project is mainly educational after all.
• While the hashes are no longer rejected (or at least not always), the difficulty that I'm getting from the server is jumping from very small to very large values, to the point where the 20s watchdog timed out, so I increased it for now.
• I rebased the code to the newest upstream master today.

[flaviopuhl]

What is pending here?

[matteocrippa]

@mlesniew is String getValue(String data, char separator, int index) a leftover? I was trying to catch where it was used

[matteocrippa]

@revoxhere did you had a chance to look into this? I was testing with a D1 and noticed that the performance can be improved up to 50/54 kH/s but the miner is later not listed in the stats

[revoxhere]

Hmm.. looks like we've got two branches that optimize the ESP8266 performance now. I assume the one suggested in pull #1688 is an indirect continuation of @mlesniew's branch?

[matteocrippa]

Yep #1688 is an improved version of this one, with some changes around

[revoxhere]

Thanks for letting me know. Closing this one then