errors when completing pathnames with expansions

[calestyo]

Describe the bug

Hey there.

This is on Debian sid, bash-completion 2.11 and bash 5.2.15.

The background is that that I recently tried to make some improvements to fzf and its bash and completion integration.

In particular this lead me to - amongst many other questions, caused by my lack of expert bash completion knowledge ^^ - the question how to expand only parameters with no side-effects in a command string.

By pure chance I noted that apparently this is already done, to some extent by bash itself and I'd guess bash-completion might also somehow interfere, cause I found an error that only happens when bash-completion is loaded, but not when I have a bash --noprofile --norc.

To reproduce

(I always write after pressing the <Tab> at the position where that's written the resulting completed line after the =>.)

1. with bash --noprofile --norc:

mkdir -p 1/foo
v=1

ls $v/<Tab>          => ls $v/foo/
ls '$v'/<Tab>        => no completion
ls $(echo 1)/<Tab>   => no completion / no execution of the echo
ls '$(echo 1)'/<Tab> => no completion / no execution of the echo

v=2

ls $v/<Tab>          => no completion
ls '$v'/<Tab>        => no completion
ls $(echo 1)/<Tab>   => no completion / no execution of the echo
ls '$(echo 1)'/<Tab> => no completion / no execution of the echo

So here it seems to somehow expand $v in the first case and then completes foo therein.

rm -rf 1
unset -v v
mkdir -p '$v/foo' '$(echo 1)/bar'

ls $v/<Tab>          => ls \$v/foo/   (not sure whether I'd expect this or not, but ok)
ls '$v'/<Tab>        => ls \$v/foo/
ls $(echo 1)/<Tab>   => no completion / no execution of the echo    (if the above is completed, this should have been perhaps too?)
ls '$(echo 1)'/<Tab> => ls \$\(echo\ 1\)/bar/

So here it doesn't expand anything. The results are the same if v is e.g. set to 1, which is okay, as no dir 1/ exists.

But it does complete in some cases and change the quoting.

Now both together:

mkdir -p 1/baz
v=1
mkdir -p '$v/foo' '$(echo 1)/bar'

ls $v/<Tab>          => ls \$v/foo/    (unexpected, above it expanded $v to 1, here if both the literal $v/ and 1/ exist, it merely changes the quoting&completes)
ls '$v'/<Tab>        => ls \$v/foo/
ls $(echo 1)/<Tab>   => no completion / no execution of the echo
ls '$(echo 1)'/<Tab> => ls \$\(echo\ 1\)/bar/

unset -v v

ls $v/<Tab>          => ls \$v/foo/     (not sure whether I'd expect this or not, but ok)
ls '$v'/<Tab>        => ls \$v/foo/
ls $(echo 1)/<Tab>   => no completion / no execution of the echo
ls '$(echo 1)'/<Tab> => ls \$\(echo\ 1\)/bar/

Guess there's at least that case,... when both the literal $v/ and 1/ exist and bash does not expand $v to 1, whereas it does so if the literal $v/ does not exist,... which is IMO a bit fishy.

---

2. Without --noprofile --norc (and thus with bash-completion):

Starting from a fresh empty dir:

mkdir -p 1/foo
v=1

ls $v/<Tab>          => ls $v/foo/
ls '$v'/<Tab>        => gives: ls '$v'/-bash: unexpected EOF while looking for matching `''
                               ls $v/foo/
ls $(echo 1)/<Tab>   => no completion / no execution of the echo
ls '$(echo 1)'/<Tab> => gives: ls '$(echo 1)'/-bash: unexpected EOF while looking for matching `''
                               ls '$(echo 1)'/

v=2

ls $v/<Tab>          => no completion
ls '$v'/<Tab>        => gives: ls '$v'/-bash: unexpected EOF while looking for matching `''
                               ls '$v'/
ls $(echo 1)/<Tab>   => no completion / no execution of the echo
ls '$(echo 1)'/<Tab> => gives: ls '$(echo 1)'/-bash: unexpected EOF while looking for matching `''
                               ls '$(echo 1)'/

So here it seems to somehow expand $v in the first case and then completes foo therein.

Also, and this might be some issue in bash-completion(?) one gets errors in several cases.

rm -rf 1
unset -v v
mkdir -p '$v/foo' '$(echo 1)/bar'

ls $v/<Tab>          => ls \$v/foo/   (not sure whether I'd expect this or not, but ok)
ls '$v'/<Tab>        => gives: ls '$v'/-bash: unexpected EOF while looking for matching `''
                               ls \$v/foo/
ls $(echo 1)/<Tab>   => ls $(echo \$\(echo\ 1\)/bar/   (that seems wrong in any case)
ls '$(echo 1)'/<Tab> => gives: ls '$(echo 1)'/-bash: unexpected EOF while looking for matching `''
                               ls \$\(echo\ 1\)/bar/

So here it doesn't expand anything. The results are the same if v is e.g. set to 1, which is okay, as no dir 1/ exists.

But it does complete in some cases and change the quoting... and it causes the same error than above and that messed up completion to ls $(echo \$\(echo\ 1\)/bar/.

Now both together:

mkdir -p 1/baz
v=1
mkdir -p '$v/foo' '$(echo 1)/bar'

ls $v/<Tab>          => ls \$v/foo/    (unexpected, above it expanded $v to 1, here if both the literal $v/ and 1/ exist, it merely changes the quoting&completes)
ls '$v'/<Tab>        => gives: ls '$v'/-bash: unexpected EOF while looking for matching `''
                               ls \$v/foo/
ls $(echo 1)/<Tab>   => ls $(echo \$\(echo\ 1\)/bar/   (that seems wrong in any case)
ls '$(echo 1)'/<Tab> => gives: ls '$(echo 1)'/-bash: unexpected EOF while looking for matching `''
                               ls \$\(echo\ 1\)/bar/

unset -v v

ls $v/<Tab>          => ls \$v/foo/     (not sure whether I'd expect this or not, but ok)
ls '$v'/<Tab>        => gives: ls '$v'/-bash: unexpected EOF while looking for matching `''
                               ls \$v/foo/
ls $(echo 1)/<Tab>   => ls $(echo \$\(echo\ 1\)/bar/   (that seems wrong in any case)
ls '$(echo 1)'/<Tab> => gives: $ ls '$(echo 1)'/-bash: unexpected EOF while looking for matching `''
                               ls \$\(echo\ 1\)/bar/

Guess there's again at least that case,... when both the literal $v/ and 1/ exist and bash does not expand $v to 1, whereas it does so if the literal $v/ does not exist,... which is IMO a bit fishy.

Also, as above, the error messages and the messed up completion to ls $(echo \$\(echo\ 1\)/bar/.

Expected behavior

Well I guess that's clear from above ;-)

In some cases, bash alone produces IMO already results which at least I wouldn't expect.
Especially that $v is completed to $v when the literal dir $v/ exists, but that $v is expanded as variable, when the literal dir does not exist.

But that would needed be brought up at bash (or @akinomyoga… any ideas? You're always close to all knowing in these things :-) )

But these:

-bash: unexpected EOF while looking for matching `''

and the mess up completions to ls $(echo \$\(echo\ 1\)/bar/ look like they could be bash-completion bugs, as these don't happen without.

Additional context

As mentioned before, I came to this via fzf and there in particular: junegunn/fzf#3459

You may want to look especially at two comments there:

• my dangerous eval's in the shell code junegunn/fzf#3459 (comment) and there points (4) and (5)
  As far as I understand, both could happen somewhere inside bash-completion (either by a bug from bash-completion, or by fzf misusing completion functions from that.
  (4) is the same error message than that from above. (5) is a bit different but happens under similar circumstances than (4)
• the fzf upstream author’s dangerous eval's in the shell code junegunn/fzf#3459 (comment)
  It's about a unexpected completion result that neither of us does understand.
  Actually, it's probably unrelated to bash-completion, it just reminded me a bit to the messed up completion (ls $(echo \$\(echo\ 1\)/bar/) I got above

Versions (please complete the following information)

• Operating system name/distribution and version: Debian stable/unstable
• bash version, echo "$BASH_VERSION": 5.2.15(1)-release
• bash-completion version, (IFS=.; echo "${BASH_COMPLETION_VERSINFO[*]}"): 2.11

Thanks,
Chris :-)

[akinomyoga]

This is on Debian sid, bash-completion 2.11 and bash 5.2.15.

So, are these observations made in the Debian package of bash-completion 2.11? There are many changes in this area in the current master. Could you first test the same in the current master of bash-completion?

[calestyo]

Speaking of which... the most recent release is quite old… when are there going to be any new releases, which distros will then pick up?

3. Without --noprofile --norc (and thus with bash-completion, this time on 15b74b1):

Starting from a fresh empty dir:

mkdir -p 1/foo
v=1

ls $v/<Tab>          => ls $v/foo/
ls '$v'/<Tab>        => ls $v/foo/   (I'd say that this should NOT happen)
ls $(echo 1)/<Tab>   => no completion / no execution of the echo
ls '$(echo 1)'/<Tab> => no completion / no execution of the echo

v=2

ls $v/<Tab>          => no completion
ls '$v'/<Tab>        => no completion
ls $(echo 1)/<Tab>   => no completion / no execution of the echo
ls '$(echo 1)'/<Tab> => no completion / no execution of the echo

So here it seems to somehow expand $v in the first case and then completes foo therein.

Also, and this seems wrong to me, it expands the quoted $v in the 2nd case.

rm -rf 1
unset -v v
mkdir -p '$v/foo' '$(echo 1)/bar'

ls $v/<Tab>          => ls \$v/foo/   (not sure whether I'd expect this or not, but ok)
ls '$v'/<Tab>        => ls \$v/foo/   (I'd say that this should NOT happen)
ls $(echo 1)/<Tab>   => ls $(echo \$\(echo\ 1\)/bar/   (that seems wrong in any case)
ls '$(echo 1)'/<Tab> => ls \$\(echo\ 1\)/bar/

So here it doesn't expand anything. The results are the same if v is e.g. set to 1, which is okay, as no dir 1/ exists.

But it does complete in some cases and change the quoting... and it causes the same error than above and that messed up completion to ls $(echo \$\(echo\ 1\)/bar/.

Now both together:

mkdir -p 1/baz
v=1
mkdir -p '$v/foo' '$(echo 1)/bar'

ls $v/<Tab>          => ls \$v/foo/    (unexpected, above it expanded $v to 1, here if both the literal $v/ and 1/ exist, it merely changes the quoting&completes)
ls '$v'/<Tab>        => ls \$v/foo/    (I'd say that this should NOT happen)
ls $(echo 1)/<Tab>   => ls $(echo \$\(echo\ 1\)/bar/   (that seems wrong in any case)
ls '$(echo 1)'/<Tab> => ls \$\(echo\ 1\)/bar/

unset -v v

ls $v/<Tab>          => ls \$v/foo/     (not sure whether I'd expect this or not, but ok)
ls '$v'/<Tab>        => ls \$v/foo/     (I'd say that this should NOT happen)
ls $(echo 1)/<Tab>   => ls $(echo \$\(echo\ 1\)/bar/   (that seems wrong in any case)
ls '$(echo 1)'/<Tab> => ls \$\(echo\ 1\)/bar/

Guess there's again at least that case,... when both the literal $v/ and 1/ exist and bash does not expand $v to 1, whereas it does so if the literal $v/ does not exist,... which is IMO a bit fishy.

Also, as above, it expands the quoted '$v', which I think should not happen and the messed up completion to ls $(echo \$\(echo\ 1\)/bar/.

So while the error messages are in fact gone in master, some problems remain.

btw: It's not a big deal and technically of course find... and I'm not sure whether it's not even better anyway... but would it in principle also be possibl to not change the quoting on completon?
E.g. from '$(echo 1)'/<Tab> to ls \$\(echo\ 1\)/bar/?