commands:builtin:mapfile

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
Last revision Both sides next revision
commands:builtin:mapfile [2011/10/28 07:48]
ormaaj Finally came up with some examples. Also rewrite much of the article in 3rd person.
commands:builtin:mapfile [2013/04/14 12:37]
thebonsai Don't tread version 4 special - it has been around for a long time now
Line 1: Line 1:
 ====== The mapfile builtin command ====== ====== The mapfile builtin command ======
-:V4: 
  
 ===== Synopsis ===== ===== Synopsis =====
Line 73: Line 72:
 Since redirects are syntactically allowed anywhere in a command, we put it before the printf to stay out of the way of additional arguments. Rather than opening "​outfile<​n>"​ for appending on each call by calculating the filename, open an FD for each first and calculate which FD to send output to by measuring the size of x mod 2. The zero-width format specification is used to absorb the index number argument. Since redirects are syntactically allowed anywhere in a command, we put it before the printf to stay out of the way of additional arguments. Rather than opening "​outfile<​n>"​ for appending on each call by calculating the filename, open an FD for each first and calculate which FD to send output to by measuring the size of x mod 2. The zero-width format specification is used to absorb the index number argument.
  
-Another variation might be to add each of these lines to the elements of separate arrays. I'll leave dissecting this one as an exercise for the reader. ​Exploiting array assignment like this is somewhat obscure ​and potentially dangerous behavior.+Another variation might be to add each of these lines to the elements of separate arrays. I'll leave dissecting this one as an exercise for the reader. ​This is quite the hack but illustrates some interesting properties of printf -v and mapfile -C (which you should probably never use in real code).
  
-<​code>​$ y=( 'a[j]' 'b[j++]' ); printf '​input%s\n'​ {1..10} | mapfile -tc 1 -C '​printf -v "${y[${#a[@]} + ${#b[@]} ) % 2]}" -- "​%.sprefix %s"'​ x +<​code>​$ y=( 'odd[j]' 'even[j++]' ); printf '​input%s\n'​ {1..10} | mapfile -tc 1 -C '​printf -v "​${y[${#​x[@]} % 2]}" -- "​%.sprefix %s"'​ xprintf '%s\n' "${odd[@]}" ​'' ​"${even[@]}"; }
-printf '%b\n' "${a[@]}\n" "${b[@]}"+
 prefix input1 prefix input1
 prefix input3 prefix input3
Line 90: Line 88:
 </​code>​ </​code>​
  
-==== Bugs ====+This example based on yet another #bash question illustrates mapfile in combination with read. The sample input is the heredoc to ''​main''​. The goal is to build a "​struct"​ based upon records in the input file made up of the numbers following the colon on each line. Every 3rd line is a key followed by 2 corresponding fields. The showRecord function takes a key and returns the record.
  
-Early implementations were buggy. For example, ''​mapfile'' ​filling the readline history buffer with calls to the ''​CALLBACK''​. ​This was fixed in 4.1 betaSome bugs may still exist.+<​code>​ 
 +#​!/​usr/​bin/​env bash 
 + 
 +showRecord() { 
 +    printf '​key[%d] = %d, %d\n' "​$1"​ "​${vals[@]:​keys[$1]*2:​2}"​ 
 +
 + 
 +parseRecords() { 
 +    trap 'unset -f _f' RETURN 
 +    _f() { 
 +        local x 
 +        IFS=: read -r _ x 
 +        ((keys[x]=n++)) 
 +    } 
 +    local n 
 + 
 +    _f 
 +    mapfile -tc2 -C _f "​$1"​ 
 +    eval "​$1"'​=("​${'"​$1"'​[@]##​*:​}"​)'​ # Return the array with some modification 
 +
 + 
 +main() { 
 +    local -a keys vals 
 +    parseRecords vals 
 +    showRecord "​$1"​ 
 +
 + 
 +main "​$1"​ <<​-"​EOF"​ 
 +fabric.domain:​123 
 +routex:1 
 +routey:2 
 +fabric.domain:​321 
 +routex:6 
 +routey:4 
 +EOF 
 +</​code>​ 
 + 
 +For example, ​running ''​scriptname 321''​ would output ''​key[321] = 6, 4''​. Every 2 lines read by ''​mapfile''​the function ​''​_f'' ​is called, which reads one additional lineSince the first line in the file is a key, and ''​_f''​ is responsible for the keys, it gets called first so that ''​mapfile''​ starts by reading the second line of input, calling ''​_f''​ with each subsequent 2 iterationsThe RETURN trap is unimportant. 
 +===== Bugs =====
  
-==== To Do ====+  * Early implementations were buggy. For example, ''​mapfile''​ filling the readline history buffer with calls to the ''​CALLBACK''​. This was fixed in 4.1 beta. 
 +  * <​del>''​mapfile -n''​ reads an extra line beyond the last line assigned to the array, through Bash.</​del>​ [[ftp://​ftp.gnu.org/​gnu/​bash/​bash-4.2-patches/​bash42-035 | Fixed in 4.2.35]]. 
 +===== To Do =====
   * Create an implementation as a shell function that's portable between Ksh, Zsh, and Bash (and possibly other bourne-like shells with array support).   * Create an implementation as a shell function that's portable between Ksh, Zsh, and Bash (and possibly other bourne-like shells with array support).
  
  • commands/builtin/mapfile.txt
  • Last modified: 2013/08/19 08:09
  • by ormaaj