Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
howto:pax [2014/07/14 09:38]
geirha missing -f or >
howto:pax [2015/08/09 04:55] (current)
bill_thomson
Line 3: Line 3:
 {{keywords>​bash shell scripting POSIX archive tar packing zip}} {{keywords>​bash shell scripting POSIX archive tar packing zip}}
  
-pax can do a lot of fancy stuff, feel free to contribute ​and more awesome pax tricks!+pax can do a lot of fancy stuff, feel free to contribute more awesome pax tricks!
  
 ===== Introduction ===== ===== Introduction =====
  
-The POSIX archiver, ''​pax'',​ is an attempt ​to have a standardized archiver ​that +The POSIX archiver, ''​pax'',​ is an attempt ​at a standardized archiver ​with 
-has the best features of the classic big two (''​tar''​ and ''​cpio''​) combined, +the best features of ''​tar''​ and ''​cpio'',​ able to handle ​all common archive types.
-while being able to work on all common archive types.+
  
-However, this is **not a manpage**, it will **not** list you all possible options, +However, this is **not a manpage**, it will **not** list all possible options, 
-it will **not** ​give you detailed ​descriptions ​about every corner of ''​pax''​. It's +it will **not** you detailed ​information ​about ''​pax''​. It'​s ​only an introduction.
-just an introduction.+
  
 This article is based on the debianized Berkeley implementation of ''​pax'',​ but implementation-specific things should be tagged as such. Unfortunately,​ the Debian package doesn'​t seem to be maintained anymore. This article is based on the debianized Berkeley implementation of ''​pax'',​ but implementation-specific things should be tagged as such. Unfortunately,​ the Debian package doesn'​t seem to be maintained anymore.
Line 22: Line 20:
 ==== Operation modes ==== ==== Operation modes ====
  
-There are four basic operation modesto //list//, //read//, //write// and +There are four basic operation modes to //list//, //read//, //write// and 
-//copy// archives. They'​re switched with the combinations of the ''​-r''​ and +//copy// archives. They'​re switched with combinations of ''​-r''​ and ''​-w''​ 
-''​-w'' ​commandline ​options:+command line options:
  
 ^ Mode ^ RW-Options ^ ^ Mode ^ RW-Options ^
Line 31: Line 29:
 | Write | ''​-w''​ | | Write | ''​-w''​ |
 | Copy | ''​-r -w''​ | | Copy | ''​-r -w''​ |
 +
  
 === List === === List ===
  
-In //list mode//, ''​pax''​ writes the list of archive members to the standard +In //list mode//, ''​pax''​ writes the list of archive members to standard 
-output (a table of contents). If a pattern ​to match is specified ​in the +output (a table of contents). If a pattern match is specified ​on the 
-commandline, only the filenames ​matching these patterns ​are printed.+command line, only matching ​filenames are printed.
  
 === Read === === Read ===
  
 //Read// an archive. ''​pax''​ will read archive data and extract the members to the //Read// an archive. ''​pax''​ will read archive data and extract the members to the
-current directory. If a pattern ​to match is specified ​in the commandline, only +current directory. If a pattern match is specified ​on the command line, only matching 
-the filenames ​matching these patterns ​are extracted.+filenames are extracted.
  
-When reading an archive, the archive type is guessed ​from the archive data.+When reading an archive, the archive type is determined ​from the archive data.
  
 === Write === === Write ===
  
-//Write// an archive, which means to create a new one or to append to an +//Write// an archive, which means create a new one or append to an 
-existing one. All files and directories specified on commandline ​are put into+existing one. All files and directories specified on the command line are inserted ​into
 the archive. The archive is written to standard output by default. the archive. The archive is written to standard output by default.
  
-If no files are given at commandlinethe filenames ​of the files to pack +If no files are specified on the command line, filenames are read from ''​STDIN''​.
-into the archives ​are read from ''​STDIN''​.+
  
-The write mode is the only mode where you need to give the desired ​archive +The write mode is the only mode where you need to specify ​the archive type 
-type with ''​-x <​TYPE>'',​ e.g. ''​-x ustar''​.+with ''​-x <​TYPE>'',​ e.g. ''​-x ustar''​.
  
 === Copy === === Copy ===
  
-//Copy//-mode is similar to the passthrough mode of ''​cpio''​. ​ It provides a way to replicate a complete or partial file hierarchy (with all the +//Copy// mode is similar to ''​cpio'' ​passthrough mode.  It provides a way to replicate a complete or partial file hierarchy 
-possibilities ​''​pax'' ​gives you, e.g. rewriting groups ​etc.) to another +(with all the ''​pax'' ​options, e.g. rewriting groups) to another location.
-location.+
  
 ==== Archive data ==== ==== Archive data ====
  
 When you don't specify anything special, ''​pax''​ will attempt to read archive When you don't specify anything special, ''​pax''​ will attempt to read archive
-data from the standard input (read/list modes) and write archive data to +data from standard input (read/list modes) and write archive data to 
-the standard output (write mode). This ensures ​that ''​pax''​ can be easily +standard output (write mode). This ensures ''​pax''​ can be easily 
-used as member ​of a typical ​shell pipe construct, e.g. to read a compressed+used as part of a shell pipe construct, e.g. to read a compressed
 archive that's decompressed in the pipe. archive that's decompressed in the pipe.
  
-The option to specify ​pathname of a file to be used as archive ​is the ''​-f''​ +The option to specify ​the pathname of a file to be archived ​is ''​-f''​ 
-option. ​This file will be used as in- or output, depending on what you do+This file will be used as input or output, depending on the operation
 (read/​write/​list). (read/​write/​list).
  
-Whenever ​pax reads an archive, no matter from where, it tries to guess the type +When pax reads an archive, it tries to guess the archive ​type. 
-of the archive. ​However, in //write// mode, you must give it the information +However, in //write// mode, you must specify ​which type of archive 
-which type of archive to appen, ​using the ''​-x <​TYPE>''​ switch. ​When you omit +to append ​using the ''​-x <​TYPE>''​ switch. ​If you omit this switch, 
-this switch, ​an archive ​of the default type will be created (POSIX says it's +a default ​archive will be created (POSIX says it's implementation defined, 
-implementation defined, ​my Berkeley ''​pax''​ creates ''​ustar''​ if no options ​given).+Berkeley ''​pax''​ creates ''​ustar''​ if no options ​are specified).
  
 The following archive formats are supported (Berkeley implementation):​ The following archive formats are supported (Berkeley implementation):​
Line 90: Line 87:
 |sv4crc ​ |SVR4 CPIO format with CRC  | |sv4crc ​ |SVR4 CPIO format with CRC  |
  
-Additionally,​ the Berkeley ''​pax''​ supports ​the options ''​-z''​ and ''​-j'',​ similar to GNU ''​tar'',​ to filter archive files through GZIP/BZIP2.+Berkeley ''​pax''​ supports options ''​-z''​ and ''​-j'',​ similar to GNU ''​tar'',​ to filter archive files through GZIP/BZIP2.
  
 ==== Matching archive members ==== ==== Matching archive members ====
  
-In //read// and //list// modes, you can specify patterns ​which ''​pax''​ +In //read// and //list// modes, you can specify patterns to determine ​which files to list or extract.
-will use to match against the archive members ​to decide, ​which files to +
-list or extract.+
  
   * the pattern notation is the one known by a POSIX-shell,​ i.e. the one known by Bash without ''​extglob''​   * the pattern notation is the one known by a POSIX-shell,​ i.e. the one known by Bash without ''​extglob''​
-  * if the specified pattern matches a complete directory, ​then it affects all files rooted at this directory +  * if the specified pattern matches a complete directory, it affects all files and subdirectories of the specified ​directory 
-  * if you specify the ''​-c''​ option, ''​pax''​ will negate ​the matches, i.e. it will match all filenames **but** the ones matched by the specified patterns+  * if you specify the ''​-c''​ option, ''​pax''​ will invert ​the matches, i.e. it matches ​all filenames **except** those matching ​the specified patterns
   * if no patterns are given, ''​pax''​ will "​match"​ (list or extract) all files from the archive   * if no patterns are given, ''​pax''​ will "​match"​ (list or extract) all files from the archive
-  * **To avoid conflicts with shell'​s ​pathname expansion, it's wise to quote those patterns!**+  * **To avoid conflicts with shell pathname expansion, it's wise to quote patterns!**
  
 === Some assorted examples of patterns === === Some assorted examples of patterns ===
Line 124: Line 119:
 ==== Creating an archive ==== ==== Creating an archive ====
  
-This task is done by the basic syntax+This task is done with basic syntax
 <​code>​ <​code>​
 # archive contents to stdout # archive contents to stdout
 pax -w >​archive.tar README.txt *.png data/ pax -w >​archive.tar README.txt *.png data/
  
-# equivalent, archive contents directly to a file+# equivalent, ​extract ​archive contents directly to a file
 pax -w -x ustar -f archive.tar README.txt *.png data/ pax -w -x ustar -f archive.tar README.txt *.png data/
 </​code>​ </​code>​
  
-The ''​pax''​ is in //write// mode, the given filenames are packed into the+''​pax''​ is in //write// mode, the given filenames are packed into an
 archive: archive:
   * ''​README.txt''​ is a normal file, it will be packed   * ''​README.txt''​ is a normal file, it will be packed
-  * ''​*.png''​ is a pathname glob **for your shell**, the shell will substitute ​it to all matching filenames **before** ''​pax''​ is executed. The result is a list of filenames that will be packed like the ''​README.txt''​ above +  * ''​*.png''​ is a pathname glob **for your shell**, the shell will substitute all matching filenames **before** ''​pax''​ is executed. The result is a list of filenames that will be packed like the ''​README.txt'' ​example ​above 
-  * ''​data/''​ is a directory. **Everything** ​that's rooted ​in this directory will be packed into the archive, not only the empty directory ​itself+  * ''​data/''​ is a directory. **Everything** in this directory will be packed into the archive, ​i.e. not just an empty directory
  
 When you specify the ''​-v''​ option, ''​pax''​ will write the pathnames of the When you specify the ''​-v''​ option, ''​pax''​ will write the pathnames of the
-files it puts into the archive to ''​STDERR''​.+files inserted ​into the archive to ''​STDERR''​.
  
-When, and only when no filename arguments are specified, ''​pax''​ attempts to +When, and only whenno filename arguments are specified, ''​pax''​ attempts to 
-read filenames ​of the files to archive ​from ''​STDIN'',​ separated by newlines. +read filenames from ''​STDIN'',​ separated by newlines. 
-This way you can easily ​compine ​''​find''​ with ''​pax'':​+This way you can easily ​combine ​''​find''​ with ''​pax'':​
 <​code>​ <​code>​
 find . -name '​*.txt'​ | pax -wf textfiles.tar -x ustar find . -name '​*.txt'​ | pax -wf textfiles.tar -x ustar
Line 152: Line 147:
  
 The standard output format to list archive members simply is to print each The standard output format to list archive members simply is to print each
-filename ​in a separate line. But the output format can be customizedto include +filename ​to a separate line. But the output format can be customized to include 
-permissions,​ timestamps, etc., using the ''​-o listopt=<​FORMAT>''​ specification.+permissions,​ timestamps, etc. with the ''​-o listopt=<​FORMAT>''​ specification.
 The syntax of the format specification is strongly derived from the The syntax of the format specification is strongly derived from the
-''​printf(3)''​ format specification ​syntax.+''​printf(3)''​ format specification.
  
 **Unfortunately** the ''​pax''​ utility delivered with Debian doesn'​t seem to **Unfortunately** the ''​pax''​ utility delivered with Debian doesn'​t seem to
Line 170: Line 165:
 ==== Extracting from an archive ==== ==== Extracting from an archive ====
  
-You can extract files, ​all or files (not) matching specific patterns from an +You can extract ​all files, or files (not) matching specific patterns from an 
-archive using constructs like+archive using constructs like:
 <​code>​ <​code>​
 # "​normal"​ extraction # "​normal"​ extraction
Line 178: Line 173:
  
 <​code>​ <​code>​
-# with negated ​pattern+# with inverted ​pattern
 pax -rf myarchive.tar -c '​*.txt'​ pax -rf myarchive.tar -c '​*.txt'​
 </​code>​ </​code>​
Line 185: Line 180:
 ==== Copying files ==== ==== Copying files ====
  
-To simply ​copy directory contents ​over to another directory, similar to a +To copy directory contents to another directory, similar to a 
-''​cp -a''​ command, ​just do:+''​cp -a''​ command, ​use:
 <​code>​ <​code>​
 mkdir destdir mkdir destdir
-pax -rw dir destdir #creates a copy of dir in destdir/, ​ie destdir/​dir ​+pax -rw dir destdir #creates a copy of dir in destdir/, ​i.e. destdir/​dir ​
 </​code>​ </​code>​
  
Line 195: Line 190:
  
  
-==== Copying files over ssh ====+==== Copying files via ssh ====
  
-To simply ​copy directory contents ​over to another directory on a distant machinejust do:+To copy directory contents to another directory on a remote systemuse:
 <​code>​ <​code>​
 pax -w localdir | ssh user@host "cd distantdest && pax -r -v" pax -w localdir | ssh user@host "cd distantdest && pax -r -v"
-pax -w localdir | gzip | ssh user@host "cd distantdir && gunzip | pax -r -v" #send the data compressed+pax -w localdir | gzip | ssh user@host "cd distantdir && gunzip | pax -r -v" #compress ​the sent data
 </​code>​ </​code>​
-These commands create a copy of localdir in distandir (distantdir/​dir) on the distant ​machine. ​+These commands create a copy of localdir in distandir (distantdir/​dir) on the remote ​machine. ​
  
 ===== Advanced usage ===== ===== Advanced usage =====
Line 211: Line 206:
 __**Note:​**__ ''​-T''​ is an extension and is not defined by POSIX. __**Note:​**__ ''​-T''​ is an extension and is not defined by POSIX.
  
-Sayyou have write-access to some fileserver mounted ​into your system+Say you have write-access to fileserver mounted ​on your filesystem tree
-In //copy// mode, you can tell ''​pax''​ to only copy the files that were+In //copy// mode, you can tell ''​pax''​ to copy only files that were
 modified today: modified today:
 <​code>​ <​code>​
Line 218: Line 213:
 pax -rw -T 0000 data/ /​n/​mybackups/​$(date +%A)/ pax -rw -T 0000 data/ /​n/​mybackups/​$(date +%A)/
 </​code>​ </​code>​
-This is done using the ''​-T''​ switch, which normally allows to give +This is done using the ''​-T''​ switch, which normally allows ​you to specify ​
-range of time, but in this case only the start point which means +time window, but in this caseonly the start time which means "​today ​at midnight"​.
-"today midnight"​.+
  
-When you execute this (of course ​very simple!) backup after your daily work, +When you execute this "very simple backup" ​after your daily work, 
-you will always ​have a copy of the files you modify.+you will have a copy of the modified ​files.
  
 __**Note:​**__ The ''​%A''​ format from ''​date''​ expands to the name of the __**Note:​**__ The ''​%A''​ format from ''​date''​ expands to the name of the
 current day, localized, e.g. "​Friday"​ (en) or "​Mittwoch"​ (de). current day, localized, e.g. "​Friday"​ (en) or "​Mittwoch"​ (de).
  
-The very same, but using an archive, can of course ​be achieved ​by:+The same, but with an archive, can be accomplished ​by:
 <​code>​ <​code>​
 pax -w -T 0000 -f /​n/​mybackups/​$(date +%A) pax -w -T 0000 -f /​n/​mybackups/​$(date +%A)
 </​code>​ </​code>​
-In this case, of course, the day-name is an archive-file (you don't need a filename +In this case, the day-name is an archive-file (you don't need a filename 
-extension like ''​.tar'' ​of course, ​but if you feel better, ​add one).+extension like ''​.tar''​ but you can add one, if desired).
  
  
Line 239: Line 233:
 ==== Changing filenames while archiving ==== ==== Changing filenames while archiving ====
  
-''​pax''​ is able to rewrite ​the filenames while archiving or while extracting from the archive. This simple ​example ​will create ​a tar archive containing the ''​holiday_2007/''​ directory, but the directory name inside the archive will be ''​holiday_pics/'':​+''​pax''​ is able to rewrite filenames while archiving or while extracting from an archive. This example ​creates ​a tar archive containing the ''​holiday_2007/''​ directory, but the directory name inside the archive will be ''​holiday_pics/'':​
 <​code>​ <​code>​
 pax -x ustar -w -f holiday_pictures.tar -s '/​^holiday_2007/​holiday_pics/'​ holiday_2007/​ pax -x ustar -w -f holiday_pictures.tar -s '/​^holiday_2007/​holiday_pics/'​ holiday_2007/​
 </​code>​ </​code>​
  
-The option responsible for the string manipulation is the ''​-s <​REWRITE-SPECIFICATION>''​. It takes the string rewrite specification as argument, in the form ''/​OLD/​NEW/​[gp]'',​ which is an ''​ed(1)''​-like regular expression (BRE) for ''​old''​ and generally can be used like the popolar ​''​s/​from/​to/​''​ command of ''​ed''​ or ''​sed''​. ​Every non-null character can be used as delimiter, so to mangle pathnames (containing slashes), you could use ''#/​old/​path#/​new/​path#''​.+The option responsible for the string manipulation is the ''​-s <​REWRITE-SPECIFICATION>''​. It takes the string rewrite specification as an argument, in the form ''/​OLD/​NEW/​[gp]'',​ which is an ''​ed(1)''​-like regular expression (BRE) for ''​old''​ and generally can be used like the popular sed construct ​''​s/​from/​to/''​. ​Any non-null character can be used as delimiter, so to mangle pathnames (containing slashes), you could use ''#/​old/​path#/​new/​path#''​.
  
-The optional ''​g''​ and ''​p''​ flags are used to apply the substitution **(g)**lobally to the line or to **(p)**rint the original and rewritten strings to ''​STDERR''​.+The optional ''​g''​ and ''​p''​ flags are used to apply substitution **(g)**lobally to the line or to **(p)**rint the original and rewritten strings to ''​STDERR''​.
  
-Multiple ''​-s''​ options can be specified on the commandline. They are all applied to the pathname strings of the files or archive members. This happens in the order as they are specified.+Multiple ''​-s''​ options can be specified on the command line. They are applied to the pathname strings of the files or archive members. This happens in the order they are specified.
  
 ==== Excluding files from an archive ==== ==== Excluding files from an archive ====
  
-The -s command seen above can be used to exclude ​completely ​a file, for this the substitution must result in a null string: +The -s command seen above can be used to exclude a file. The substitution must result in a null string: 
-For example let's say that you want to exclude all the CVS directories to create ​an archive with the source code (you should really use cvs export for this but...), for this  +For examplelet's say that you want to exclude all the CVS directories to create ​source code archive
-we are going to replace the names containing /CVS/ with nothing, note the .* they are needed because we need to match the whole pathname.+We are going to replace the names containing /CVS/ with nothing, note the .* they are needed because we need to match the entire ​pathname.
 <​code>​ <​code>​
   pax -w -x ustar -f release.tar -s',​.*/​CVS/​.*,,'​ myapplication ​   pax -w -x ustar -f release.tar -s',​.*/​CVS/​.*,,'​ myapplication ​
 </​code>​ </​code>​
-You can use several -s options, for instance let'​s ​suppose that you also want to remove ​the emacs backup ​files ending in ~:+You can use several -s options, for instancelet'​s ​say you also want to remove files ending in ~:
 <​code>​ <​code>​
   pax -w -x ustar -f release.tar -'​s,​.*/​CVS/​.*,,'​ -'​s/​.*~//'​ myapplication ​   pax -w -x ustar -f release.tar -'​s,​.*/​CVS/​.*,,'​ -'​s/​.*~//'​ myapplication ​
 </​code>​ </​code>​
  
-This also can be done while reading an archive, for instance suppose ​that you have an archive containing a "​usr"​ and a "​etc"​ directory but that you only want to extract the "​usr"​ directory:+This can also be done while reading an archive, for instancesuppose you have an archive containing a "​usr"​ and a "​etc"​ directory but that you want to extract ​only the "​usr"​ directory:
 <​code>​ <​code>​
 pax -r -f archive.tar -s',​^etc/​.*,,'​ #the etc/ dir is not extracted pax -r -f archive.tar -s',​^etc/​.*,,'​ #the etc/ dir is not extracted
 </​code>​ </​code>​
  
-==== Getting ​filenames to archive from STDIN ====+==== Getting archive ​filenames ​from STDIN ====
  
-Like ''​cpio'',​ pax can read filenames from standard input (''​stdin''​). ​ This provides great flexibility - for example, a ''​find(1)''​ command may select files/​directories in ways that pax can't do by itself. ​ In **write** mode (creating an archive) or **copy** mode, when no filenames ​to archive ​are given, pax expects to read filenames from standard input. For example:+Like ''​cpio'',​ pax can read filenames from standard input (''​stdin''​). ​ This provides great flexibility - for example, a ''​find(1)''​ command may select files/​directories in ways pax can't do itself. ​ In **write** mode (creating an archive) or **copy** mode, when no filenames are given, pax expects to read filenames from standard input. For example:
  
 <​code>​ <​code>​
Line 286: Line 280:
 </​code>​ </​code>​
  
-The ''​-d''​ option tells pax ''​not''​ to recurse into any directories ​that it reads (''​cpio''​-style). ​ Without ''​-d'',​ pax recurses into all directories (''​tar''​-style).+The ''​-d''​ option tells pax ''​not''​ to recurse into directories it reads (''​cpio''​-style). ​ Without ''​-d'',​ pax recurses into all directories (''​tar''​-style).
  
 **Note**: the ''​-0''​ option is not standard, but is present in some implementations. **Note**: the ''​-0''​ option is not standard, but is present in some implementations.
Line 294: Line 288:
 ''​pax''​ can handle the ''​tar''​ archive format, if you want to switch to the standard tool an alias like: ''​pax''​ can handle the ''​tar''​ archive format, if you want to switch to the standard tool an alias like:
 <​code>​ <​code>​
-alias tar='​echo USE PAX, you idiot. pax is the standard archiver!; # '+alias tar='​echo USE PAX, idiot. pax is the standard archiver!; # '
 </​code>​ </​code>​
 in your ''​~/​.bashrc''​ can be useful :-D. in your ''​~/​.bashrc''​ can be useful :-D.
Line 306: Line 300:
 | ''​tar tzvf file.tar.gz''​ | ''​pax -vz -f  file.tar.gz''​ | ''​-z''​ is an extension, POSIXly: ''​gunzip <​file.tar.gz %%|%% pax -v''​ | | ''​tar tzvf file.tar.gz''​ | ''​pax -vz -f  file.tar.gz''​ | ''​-z''​ is an extension, POSIXly: ''​gunzip <​file.tar.gz %%|%% pax -v''​ |
  
-''​pax''​ might not create ustar (''​tar''​) archives by default but its own pax format, add ''​-x ustar''​ if you want to be sure to create ​tar archives!+''​pax''​ might not create ustar (''​tar''​) archives by default but its own pax format, add ''​-x ustar''​ if you want to ensure pax creates ​tar archives!
  
 ===== Implementations ===== ===== Implementations =====
  • howto/pax.1405330698.txt
  • Last modified: 2014/07/14 09:38
  • by geirha