Tremendous amount of memory usage

[amosbird]

When doing helm-dash-install-docset, emacs ends up using more than 10GB real memory. And the memory never gets released until I restart emacs.

env:
Archlinux
emacs 25.2
helm-dash 20170531.135

[areina]

Hello @amosbird!
Which docset are you trying to install?

[amosbird]

I tried c, c++, python, elisp, go, rust . Any of these can reproduce this issue.

[areina]

Great, thanks, will take a look.

[dchrzanowski]

Exact same issue here (same setup: Archlinux and Emacs 25.2) , however, installing Python 3 docs and Typescript docs went ok and pretty quickly.

[fikovnik]

I have the same issue when installing Scala docset including 100% CPU usage

[jeeger]

The problem is that during extraction of the docset, the full list of files in the .tgz file kept in memory and passed around as a string. This is used to extract the folder the docset is stored into. The C docset for example is very large.

However, this is only necessary once during installation. A fix would be to split the extraction of the file from the parsing of the folder name, so all the output could be thrown away during extraction. Afterwards, we could use something like 'tar tf <docset.tgz> | head -n -1' to only get the last folder. This should be more efficient, but it more error-prone since this requires calling into the shell.

[kidd]

should we provide an alternative function that uses the shell and document it? idk, not ideal, but....

Otherwise, @jeeger do you have that function that uses the shell? if it can work as a replacement for an existing function, could you paste it here for future reference?