Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
syntax:keywords:coproc [2012/08/15 13:08]
ormaaj I suppose I could mention the print -p thing here too.
syntax:keywords:coproc [2013/04/14 12:36] (current)
thebonsai Don't tread version 4 special - it has been around for a long time now
Line 1: Line 1:
 ====== The coproc keyword ====== ====== The coproc keyword ======
-:V4: 
  
 ===== Synopsis ===== ===== Synopsis =====
 +
 <​code>​ <​code>​
  ​coproc [NAME] command [redirections]  ​coproc [NAME] command [redirections]
 </​code>​ </​code>​
- 
  
 ===== Description ===== ===== Description =====
-Bash 4.0 introduced the coprocesses,​ a feature certainly familiar to ksh users. 
  
-''​coproc''​ starts a command ​in the backgound ​setting up pipes so that you can +Bash 4.0 introduced //​coprocesses//,​ a feature certainly familiar to ksh users. The ''​coproc'' ​keyword ​starts a command ​as a background job, setting up pipes connected to both its stdin and stdout ​so that you can interact with it bidirectionally. Optionally, the co-process can have a name ''​NAME''​. If ''​NAME''​ is given, the command that follows **must be a compound command**. If no ''​NAME''​ is given, then the command can be either simple or compound.
-interact with it. Optionally, the co-process can have a name ''​NAME''​.+
  
-If ''​NAME'' ​is given, the following command **must be compound command**. If no ''​NAME'' ​ist giventhe command can be a simple command or a compound ​command.+The process ID of the shell spawned to execute the coprocess is available through the value of the variable named by ''​NAME'' ​followed by a ''​_PID''​ suffix. For example, the variable name used to store the PID of coproc started with no ''​NAME''​ given would be ''​COPROC_PID''​ (because ''​COPROC''​ is the default ''​NAME''​). [[commands:​builtin:​wait]] may be used to wait for the coprocess to terminate. Additionally,​ coprocesses may be manipulated through their ''​jobspec''​. 
 + 
 +==== Return status ==== 
 + 
 +The return status of coprocess is the exit status of its command.
  
 ==== Redirections ==== ==== Redirections ====
  
-The redirections are normal redirections that are set after the pipe +The optional ​redirections are applied ​after the pipes have been set up. Some examples:
-has been set up, some examples:+
  
-<​code>​+<​code ​bash>
 # redirecting stderr in the pipe # redirecting stderr in the pipe
-$ coproc { ls thisfiledoesntexist;​ read ;} 2>&1+$ coproc { ls thisfiledoesntexist;​ read; } 2>&1
 [2] 23084 [2] 23084
-$ read -${COPROC[0]};​printf ​"%s\n" ​"$REPLY"+IFS= read -ru ${COPROC[0]} ​x; printf ​'%s\n' ​"$x"
 ls: cannot access thisfiledoesntexist:​ No such file or directory ls: cannot access thisfiledoesntexist:​ No such file or directory
 </​code>​ </​code>​
  
-<​code>​+<​code ​bash>
 #let the output of the coprocess go to stdout #let the output of the coprocess go to stdout
-$ { coproc mycoproc { awk '​{print "​foo"​ $0;​fflush()}'​ ;} >&3 ;} 3>&1+$ { coproc mycoproc { awk '​{print "​foo"​ $0;​fflush()}';​ } >&3; } 3>&1
 [2] 23092 [2] 23092
 $ echo bar >&​${mycoproc[1]} $ echo bar >&​${mycoproc[1]}
Line 37: Line 37:
 </​code>​ </​code>​
  
-Here we need to save the previous file descriptor of stdout, because by the time +Here we need to save the previous file descriptor of stdout, because by the time we redirect the fds of the coprocessstdout has already ​been redirected to the pipe. 
-we want to redirect the fds of the coprocess stdout has been redirected +
-to the pipe.  +
  
 ==== Pitfalls ==== ==== Pitfalls ====
  
-=== Avoid the command | while read subshell ===+=== Avoid the final pipeline ​subshell ===
  
-The traditional ​KSH workaround to avoid the subshell when doing +The traditional ​Ksh workaround to avoid the subshell when doing ''​command | while read'' ​is to use a coprocess. UnfortunatelyBash'​s ​behavior differs.
-command | while read is to use a coprocess, ​unfortunately,​ it seems  +
-that bash'​s ​behaviour differ from KSH.+
  
-In KSH you would do: +In Ksh you would do: 
-<​code>​ +<​code ​bash> 
-ls |& #start a coprocess +# ksh93 or mksh/pdksh derivatives 
-while read -file;​do ​echo "​$file";​done #read its output+ls |& # start a coprocess 
 +while IFS= read -rp file; do print -r -- "​$file";​ done # read its output
 </​code>​ </​code>​
  
 In bash: In bash:
-<​code>​+<​code ​bash>
 #​DOESN'​T WORK #​DOESN'​T WORK
 $ coproc ls $ coproc ls
 [1] 23232 [1] 23232
-$ while read -${COPROC[0]} line;​do ​echo "​$line";​done+$ while IFS= read -ru ${COPROC[0]} line; do printf '​%s\n' ​"​$line";​ done
 bash: read: line: invalid file descriptor specification bash: read: line: invalid file descriptor specification
 [1]+  Done                    coproc COPROC ls [1]+  Done                    coproc COPROC ls
 </​code>​ </​code>​
  
-By the time we start reading from the output of the coprocess, the +By the time we start reading from the output of the coprocess, the file descriptor has been closed
-file descriptor has been closed.+ 
 +See [[http://​mywiki.wooledge.org/​BashFAQ/​024 | this FAQ entry on Greg's wiki]] for other pipeline subshell workarounds.
  
 === Buffering === === Buffering ===
  
-In the first example, we used ''​fflush()''​ in the ''​awk''​ command, ​this was +In the first example, we GNU awk'​s ​''​fflush()''​ command. As always, when you use pipes the I/O operations are buffered. Let's see what happens with ''​sed'':​
-done on purpose, as always ​when you use pipes the I/O operations are buffered+
-let's see what happens with ''​sed'':​+
  
-<​code>​+<​code ​bash>
 $ coproc sed s/^/foo/ $ coproc sed s/^/foo/
 [1] 22981 [1] 22981
 $ echo bar >&​${COPROC[1]} $ echo bar >&​${COPROC[1]}
-$ read -t 3 -${COPROC[0]};​ (( $? >127 )) && echo "​nothing read"+$ read -t 3 -ru ${COPROC[0]} ​_; (( $? > 127 )) && echo "​nothing read"
 nothing read nothing read
 </​code>​ </​code>​
  
-Even though this example is the same as the first ''​awk''​ example, the +Even though this example is the same as the first ''​awk''​ example, the ''​read'' ​doesn'​t return because the output is waiting in a buffer.
-read doesn'​t return, simply ​because the output is waiting in a buffer.+
  
-See [[http://​wooledge.org:8000/​BashFAQ/​009|this faq entry on Greg's wiki]] for some workarounds.+See [[http://mywiki.wooledge.org/​BashFAQ/​009 | this faq entry on Greg's wiki]] for some workarounds ​and more information on buffering issues.
  
 === background processes === === background processes ===
  
-The file descriptors ​of the coprocesses ​are available ​to the +A coprocess' ​file descriptors are accessible only to the process from which the ''​coproc'' ​was started. They are not inherited ​by subshells.
-shell where you run ''​coproc''​, but they are not inherited. +
-Here a not so meaningful illustration,​ suppose we want something that +
-continuely reads the output of our coprocess and echo the result:+
  
-<​code>​+Here is a not-so-meaningful illustration. Suppose we want to continuously read the output of a coprocess and ''​echo''​ the result: 
 + 
 +<​code ​bash>
 #NOT WORKING #NOT WORKING
 $ coproc awk '​{print "​foo"​ $0;​fflush()}'​ $ coproc awk '​{print "​foo"​ $0;​fflush()}'​
 [2] 23100 [2] 23100
-$ while read -${COPROC[0]};​do ​echo "$REPLY";done &+$ while IFS= read -ru ${COPROC[0]} ​x; do printf '​%s\n' ​"$x"; done &
 [3] 23104 [3] 23104
-$ ./bash: line 243: read: 61: invalid file descriptor: Bad file +bash: line 243: read: 61: invalid file descriptor: Bad file descriptor
-descriptor+
 </​code>​ </​code>​
-it failsbecause the descriptor is not avalaible in the subshell + 
-created by &.+This fails because the file descriptors created by the parent are not available to the subshell created by &.
  
 A possible workaround: A possible workaround:
  
-<​code>​+<​code ​bash>
 #WARNING: for illustration purpose ONLY #WARNING: for illustration purpose ONLY
 # this is not the way to make the coprocess print its output # this is not the way to make the coprocess print its output
Line 116: Line 108:
 [2] 23109 [2] 23109
 $ exec 3<&​${COPROC[0]} $ exec 3<&​${COPROC[0]}
-$ while read -3;do echo "$REPLY";done &+$ while IFS= read -ru x; do printf '​%s\n' ​"$x"; done &
 [3] 23110 [3] 23110
 $ echo bar >&​${COPROC[1]} $ echo bar >&​${COPROC[1]}
Line 122: Line 114:
 </​code>​ </​code>​
  
-Here the fd 3 is inherited.+Herefd 3 is inherited.
  
 ===== Examples ===== ===== Examples =====
Line 128: Line 120:
 ==== Anonymous Coprocess ==== ==== Anonymous Coprocess ====
  
-__**First let's see an example without ​''​NAME'':​**__+Unlike ksh, Bash doesn't have true anonymous coprocesses. Instead, Bash assigns FDs to a default array named ''​COPROC''​ if no ''​NAME'' ​is supplied. Here's an example:
  
-<​code>​+<​code ​bash>
 $ coproc awk '​{print "​foo"​ $0;​fflush()}'​ $ coproc awk '​{print "​foo"​ $0;​fflush()}'​
 [1] 22978 [1] 22978
 </​code>​ </​code>​
  
-The command starts in the background, coproc returns ​immedately. +This command starts in the background, ​and ''​coproc'' ​returns ​immediatelyTwo new file descriptors are now available via the ''​COPROC'' ​arrayWe can send data to our command:
-new files descriptors are now available via the COPROC array+
-We can send data to our command:+
  
-<​code>​+<​code ​bash>
 $ echo bar >&​${COPROC[1]} $ echo bar >&​${COPROC[1]}
 </​code>​ </​code>​
Line 145: Line 135:
 And then read its output: And then read its output:
  
-<​code>​ +<​code ​bash
-$ read -${COPROC[0]};​printf ​"%s\n" ​"$REPLY"+IFS= read -ru ${COPROC[0]} ​x; printf ​'%s\n' ​"$x"
 foobar foobar
 </​code>​ </​code>​
Line 160: Line 150:
 ==== Named Coprocess ==== ==== Named Coprocess ====
  
-__**Using a named coprocess is as simple, we just need a compound command like when defining a function:**__+Using a named coprocess is simple. We just need a compound command ​(like when defining a function), and the resulting FDs will be assigned to the indexed array ''​NAME''​ we supply instead.
  
-<​code>​+<​code ​bash>
 $ coproc mycoproc { awk '​{print "​foo"​ $0;​fflush()}'​ ;} $ coproc mycoproc { awk '​{print "​foo"​ $0;​fflush()}'​ ;}
 [1] 23058 [1] 23058
 $ echo bar >&​${mycoproc[1]} $ echo bar >&​${mycoproc[1]}
-$ read -${mycoproc[0]};​printf ​"%s\n" ​"$REPLY"+IFS= read -ru ${mycoproc[0]} ​x; printf ​'%s\n' ​"$x"
 foobar foobar
 $ kill $mycoproc_PID $ kill $mycoproc_PID
Line 175: Line 165:
 ==== Redirecting the output of a script to a file and to the screen ==== ==== Redirecting the output of a script to a file and to the screen ====
  
-<​code>​+<​code ​bash>
 #!/bin/bash #!/bin/bash
 # we start tee in the background # we start tee in the background
Line 183: Line 173:
 exec >&​${tee[1]} 2>&1 exec >&​${tee[1]} 2>&1
 </​code>​ </​code>​
- 
  
 ===== Portability considerations ===== ===== Portability considerations =====
Line 190: Line 179:
   * The ''​coproc''​ keyword appeared in Bash version 4.0-alpha   * The ''​coproc''​ keyword appeared in Bash version 4.0-alpha
   * The ''​-p''​ option to Bash's ''​print''​ loadable is a NOOP and not connected to Bash coprocesses in any way. It is only recognized as an option for ksh compatibility,​ and has no effect.   * The ''​-p''​ option to Bash's ''​print''​ loadable is a NOOP and not connected to Bash coprocesses in any way. It is only recognized as an option for ksh compatibility,​ and has no effect.
 +  * The ''​-p''​ option to Bash's ''​[[commands:​builtin:​read | read]]''​ builtin conflicts with that of all kshes and zsh. The equivalent in those shells is to add a ''​\?​prompt''​ suffix to the first variable name argument to ''​read''​. i.e., if the first variable name given contains a ''?''​ character, the remainder of the argument is used as the prompt string. Since this feature is pointless and redundant, I suggest not using it in either shell. Simply precede the ''​read''​ command with a ''​printf %s prompt >&​2''​. ​
  
 ==== Other shells ==== ==== Other shells ====
Line 197: Line 187:
 zsh coprocesses are very similar to ksh except in the way they are started. zsh adds the shell reserved word ''​coproc''​ to the pipeline syntax (similar to the way Bash's ''​time''​ keyword works), so that the pipeline that follows is started as a coproc. The coproc'​s input and output FDs can then be accessed and moved using the same ''​read''/''​print''​ ''​-p''​ and redirects used by the ksh shells. zsh coprocesses are very similar to ksh except in the way they are started. zsh adds the shell reserved word ''​coproc''​ to the pipeline syntax (similar to the way Bash's ''​time''​ keyword works), so that the pipeline that follows is started as a coproc. The coproc'​s input and output FDs can then be accessed and moved using the same ''​read''/''​print''​ ''​-p''​ and redirects used by the ksh shells.
  
-It is unfortunate that Bash chose to go against existing practice in their coproc implementation,​ especially considering it was the last of the major shells to incorporate this feature.+It is unfortunate that Bash chose to go against existing practice in their coproc implementation,​ especially considering it was the last of the major shells to incorporate this feature. However, Bash's method accomplishes the same without requiring nearly as much additional syntax. The ''​coproc''​ keyword is easy enough to wrap in a function such that it takes Bash code as an ordinary argument and/or stdin like ''​eval''​. Coprocess functionality in other shells can be similarly wrapped to create a ''​COPROC''​ array automatically.
  
 ==== Only one coprocess at a time ==== ==== Only one coprocess at a time ====
  • syntax/keywords/coproc.1345036126.txt
  • Last modified: 2012/08/15 13:08
  • by ormaaj