Add pipeline support for nsh commandline

[JianyuWang0623]

Summary

Add pipeline support for nsh commandline through pipe2() and posix_spawn_file_actions_adddup2()

1. Pack parameters of nsh_execute() as struct nsh_exec_param_s
   • Input redirect flags currently is hardcode, passing by arguments maybe better.
   • Only support redirect to path_name, redirect to fd is needed for pipeline.
2. Add pipeline support for commandline
3. Concat variable arguments failed - nshlib/pipeline: Concat variable arguments failed
   • Without this patch

   nsh> set var_test `uname`
   nsh> echo $var_test
   NuttX
   nsh> echo $var_test | cat
   sh [5:100]
   
   nsh>
   

   • With this patch

   nsh> set var_test `uname`
   nsh> echo $var_test
   NuttX
   nsh> echo $var_test | cat
   sh [4:100]
   NuttX
   nsh>
   

   Reported by @jasonbu and @[email protected], thank you ;-)

And nesting pipeline supported.

Impact

nsh: nsh_parse_command()

Testing

General

# Redirectin
cat < /etc/init.d/rc.sysinit

# Pipe
ls | cat

# Pipe (nested)
ls | cat | cat | cat

# FIFO
mkfifo /dev/testfifo
cat /dev/testfifo &
ls > /dev/testfifo

# CMDPARAMS
set name `uname`
echo $name

# binfmt/sotest
sotest

With dd

Depends: apache/nuttx-apps#2751

# Env
# sim:nsh with `PIPES`, `NSH_PIPELINE_NEST_DEPTH=5`, `FS_HOSTFS` and `SIM_HOSTFS` enabled.

# Read from stdin and write to "of" 
mount -t hostfs -o fs=. /data 
ls | cat | dd | cat | dd of=/data/test_dd_stdin

#Read from stdin and write to stdout
ls | cat | dd | cat | dd

• patch

$ git diff
diff --git a/nshlib/nsh_ddcmd.c b/nshlib/nsh_ddcmd.c
index 601e64408d..812d51c103 100644
--- a/nshlib/nsh_ddcmd.c
+++ b/nshlib/nsh_ddcmd.c
@@ -107,7 +107,9 @@ static int dd_write(FAR struct dd_s *dd)
   written = 0;
   do
     {
+      write(dd->outfd, "<dd ", 4);
       nbytes = write(dd->outfd, buffer, dd->nbytes - written);
+      write(dd->outfd, " dd>", 4);
       if (nbytes < 0)
         { 
           FAR struct nsh_vtbl_s *vtbl = dd->vtbl;
diff --git a/nshlib/nsh_fscmds.c b/nshlib/nsh_fscmds.c
index 413ed24634..e6a453ab5b 100644
--- a/nshlib/nsh_fscmds.c
+++ b/nshlib/nsh_fscmds.c
@@ -806,7 +806,9 @@ int cmd_cat(FAR struct nsh_vtbl_s *vtbl, int argc, FAR char **argv)
           if (n == 0)
             break;

+          nsh_write(vtbl, "<cat ", 5);
           nsh_write(vtbl, buf, n);
+          nsh_write(vtbl, " cat>", 5);
         }

       free(buf);

• runtime

nsh> ls | cat | cat | dd | dd
<dd <dd <cat <cat /:
 bin/
 data/
 dev/
 etc/
 proc/
 tmp/
 var/
 cat> cat> dd> dd>nsh>
nsh> ls | dd | cat | cat | dd
<dd <cat <cat <dd /:
 bin/
 data/
 dev/
 etc/
 proc/
 tmp/
 var/
 dd> cat cat><cat > cat> dd>nsh>
nsh>

Apache:NuttX CI

[xiaoxiang781216]

@JianyuWang0623 ltp fail with this patchset.

[JianyuWang0623]

Polling docker ghcr.io/open-vela/openvela-ci-linux and try locally.
The test_framework, test_example and test_libuv are also failed:

test_framework/test_cmocka.py::test_cmocka FAILED                        [  0%]
test_example/test_example.py::test_hello FAILED                          [  0%]
test_libuv/test_libuv.py::TestLibuv::test_test_macros FAILED             [  0%]
test_open_posix/test_openposix_.py::test_ltp_interfaces_aio_write_9_1 FAILED [  6%]

[JianyuWang0623]

TODO pick:

1. nshlib/pipeline: Concat variable arguments failed apache/nuttx-apps#2841
2. nshlib/nsh_parse: Fix variable arguments concat error of nsh_execute() apache/nuttx-apps#2850