Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
scripting:basics [2012/05/10 11:06]
szg [Mass commenting]
scripting:basics [2015/08/02 06:31] (current)
bill_thomson
Line 4: Line 4:
  
 ===== Script files ===== ===== Script files =====
-shellscript ​usually resides inside a file. This file can be executable, but you can also call a Bash with that filename as parameter:+shell script ​usually resides inside a file. The file can be executable, but you can call a Bash script ​with that filename as parameter:
 <​code>​ <​code>​
 bash ./myfile bash ./myfile
 </​code>​ </​code>​
-There is **no need to add a boring filename extension** like ''​.bash''​ or ''​.sh''​. ​This is UNIX(r), where executables are not tagged by the extension, but by **permissions** (filemode). ​Of course, you can name the file like you want! To add proper filename ​extensions ​is a convention, nothing else.+There is **no need to add a boring filename extension** like ''​.bash''​ or ''​.sh''​. ​That is a holdover from UNIX(r), where executables are not tagged by the extension, but by **permissions** (filemode). ​The file name can be any combination of legal filename characters. Adding a proper filename ​extension ​is a convention, nothing else.
 <​code>​ <​code>​
 chmod +x ./myfile chmod +x ./myfile
 </​code>​ </​code>​
  
-If the file is executable ​(and you want to use it by just calling the script name)you need to specify ​the shebang!+If the file is executableand you want to use it by calling ​only the script name, the shebang ​must be included in the file.
  
  
Line 22: Line 22:
  
 ===== The Shebang ===== ===== The Shebang =====
-The in-file specification of the interpreter of that file, like:+The in-file specification of the interpreter of that file, for example:
 <code bash> <code bash>
 #!/bin/bash #!/bin/bash
 echo "Hello world..."​ echo "Hello world..."​
 </​code>​ </​code>​
-This thing usually ​((sometimes, or under specific circumstances also by the shell itself)) ​is interpreted by the kernel ​of your system. In general, if a file is executable, but actually ​not an executable (binary) program, and such a line is present, the program specified after ''#​!''​ is started with the scriptname and all its arguments. These two characters ''#''​ and ''​!'' ​have to be **the first two bytes** in the file!+This is interpreted by the kernel ​((under specific circumstancesalso by the shell itself)) of your system. In general, if a file is executable, but not an executable (binary) program, and such a line is present, the program specified after ''#​!''​ is started with the scriptname and all its arguments. These two characters ''#''​ and ''​!'' ​must be **the first two bytes** in the file!
  
-You can follow ​it by using the ''​echo'' ​program ​as fake-interpreter:​+You can follow ​the process ​by using ''​echo''​ as fake interpreter:​
 <​code>​ <​code>​
 #!/bin/echo #!/bin/echo
 </​code>​ </​code>​
-We don't need a script-body here, as the file will never be interpreted and executed by "''​echo''"​, but you can see what the system ​does, it calls "''/​bin/​echo''"​ with the name of the executable file and all what follows.+We don't need a script body here, as the file will never be interpreted and executed by "''​echo''"​. You can see what the Operating System ​does, it calls "''/​bin/​echo''"​ with the name of the executable file and following arguments.
 <​code>​ <​code>​
 $ /​home/​bash/​bin/​test testword hello $ /​home/​bash/​bin/​test testword hello
Line 39: Line 39:
 </​code>​ </​code>​
  
-The same way, with ''#​!/​bin/​bash''​ the shell "''/​bin/​bash''"​ is called with the script-file as argument. It'​s ​exactly ​the same to execute ​"''/​bin/​bash /​home/​bash/​bin/​test testword hello''" ​here!+The same way, with ''#​!/​bin/​bash''​ the shell "''/​bin/​bash''"​ is called with the script ​filename ​as an argument. It's the same as executing ​"''/​bin/​bash /​home/​bash/​bin/​test testword hello''"​
  
 If the interpreter can be specified with arguments and how long it can be is system-specific (see [[http://​www.in-ulm.de/​~mascheck/​various/​shebang/​|#​!-magic]]). If the interpreter can be specified with arguments and how long it can be is system-specific (see [[http://​www.in-ulm.de/​~mascheck/​various/​shebang/​|#​!-magic]]).
-Just by the way: When Bash executes a file with a #!/bin/bash-shebang, ​it ignores ​the shebang itself, ​because its first character is the hashmark "#",​ which introduces ​a comment. The shebang is for the operating system, not for the shell. Programs that don't ignore such lines may not work as shebang-driven interpreters.+When Bash executes a file with a #!/bin/bash shebang, the shebang itself ​is ignoredsince the first character is hashmark "#",​ which indicates ​a comment. The shebang is for the operating system, not for the shell. Programs that don't ignore such linesmay not work as shebang driven interpreters.
  
 <WRAP center round important 60%> <WRAP center round important 60%>
-__**Attention:​**__When the specified interpreter is unavailable or not executable (permissions),​ you'​ll ​get a "''​bad interpreter''"​ error message., ​usually. ​If you get nothing and it fails, ​it's still worth to check the shebang. Older Bash versions will throw a "''​no such file or directory''"​ for a nonexisting ​interpreter specified ​in the shebang.+__**Attention:​**__When the specified interpreter is unavailable or not executable (permissions),​ you usually ​get a "''​bad interpreter''"​ error message., If you get nothing and it fails, check the shebang. Older Bash versions will respond with a "''​no such file or directory''" ​error for a nonexistant ​interpreter specified ​by the shebang.
 </​WRAP>​ </​WRAP>​
  
Line 59: Line 59:
   * the needed ''​bash''​ binary must be located in ''​PATH''​   * the needed ''​bash''​ binary must be located in ''​PATH''​
  
-If you need the one or the other, or if you think the one or the other way is good/bad - it'​s ​up to you. It's just to say there is no waterproof ​portable way to specify an interpreter. **It is a common ​misinterpretion ​that it solves all problems. Period.**+Which one you need, or whether ​you think which one is good, or bad, is up to you. There is no bulletproof ​portable way to specify an interpreter. **It'​s ​a common ​misconception ​that it solves all problems. Period.**
  
 ===== The standard filedescriptors ===== ===== The standard filedescriptors =====
-Every normal UNIX(r)-program has, once initialized, ​//at least 3 open files//:+Once Initialized,​ every normal UNIX(r)-program has //at least 3 open files//:
  
   * **stdin**: standard input   * **stdin**: standard input
Line 68: Line 68:
   * **stderr**: standard error output   * **stderr**: standard error output
  
-Usually they'​re all connected to your terminal, stdin as input file (keyboard), stdout and stderr as output files (screen). When calling such a program, the invoking shell can change these filedescriptor'​s ​connections away from the terminal to any other file (see redirection). +Usuallythey'​re all connected to your terminal, stdin as input file (keyboard), stdout and stderr as output files (screen). When calling such a program, the invoking shell can change these filedescriptor connections away from the terminal to any other file (see redirection). 
-Why two different output filedescriptors? ​Well, it'​s ​convention to output debug logs, error messages and warnings ​on stderr and only useful data on stdout. ​That way the user (you!) can decide if he wants to see nothing, only the data, only the errors, both - and where he wants to see them.+Why two different output filedescriptors? ​It's convention to send error messages and warnings ​to stderr and only program output to stdout. ​This enables ​the user to decide if they want to see nothing, only the data, only the errors, ​or both - and where they want to see them.
  
-You should follow some rules when you write a script: +When you write a script: 
-  * read user-input ​always ​from ''​stdin''​ +  * always ​read user-input from ''​stdin''​ 
-  * always write diagnostic/​error/​warning-messages to ''​stderr''​+  * always write diagnostic/​error/​warning messages to ''​stderr''​
  
-To learn more about the standard filedescriptors,​ especially about redirecting ​and piping ​them, see:+To learn more about the standard filedescriptors,​ especially about redirection ​and piping, see:
   * [[howto:​redirection_tutorial | An illustrated redirection tutorial]]   * [[howto:​redirection_tutorial | An illustrated redirection tutorial]]
  
 ===== Variable names ===== ===== Variable names =====
-It'​s ​common ​to use lowercase ​variable-names for yourself, as usually ​shelland system-variables ​are all in UPPERCASE, ​however ​you should avoid to use the following ​variable-names for your own purposes ​(incomplete list!):+It'​s ​good practice ​to use lowercase names for your variables, as shell and system-variable names are usually ​all in UPPERCASE. However, you should avoid naming your variables any of the following (incomplete list!):
 |''​BASH''​|''​BASH_ARGC''​|''​BASH_ARGV''​|''​BASH_LINENO''​|''​BASH_SOURCE''​|''​BASH_VERSINFO''​| |''​BASH''​|''​BASH_ARGC''​|''​BASH_ARGV''​|''​BASH_LINENO''​|''​BASH_SOURCE''​|''​BASH_VERSINFO''​|
 |''​BASH_VERSION''​|''​COLUMNS''​|''​DIRSTACK''​|''​DISPLAY''​|''​EDITOR''​|''​EUID''​| |''​BASH_VERSION''​|''​COLUMNS''​|''​DIRSTACK''​|''​DISPLAY''​|''​EDITOR''​|''​EUID''​|
Line 88: Line 88:
 |''​PS2''​|''​PS4''​|''​PS3''​|''​PWD''​|''​SHELL''​|''​SHELLOPTS''​| |''​PS2''​|''​PS4''​|''​PS3''​|''​PWD''​|''​SHELL''​|''​SHELLOPTS''​|
 |''​SHLVL''​|''​TERM''​|''​UID''​|''​USER''​|''​USERNAME''​|''​XAUTHORITY''​| |''​SHLVL''​|''​TERM''​|''​UID''​|''​USER''​|''​USERNAME''​|''​XAUTHORITY''​|
-This list is fairly ​incomplete, but it might help you to understand: ​**The safest way really ​is to only use lowercase variable names.**+This list is incomplete**The safest way is to use all-lowercase variable names.**
  
  
Line 94: Line 94:
  
 ===== Exit codes ===== ===== Exit codes =====
-Every program you start terminates with a so-called ​exit code and reports it to the operating system. This exit code can be utilized by Bash. You can show it, you can react on it, you can control ​the script'​s ​flow with it. +Every program you start terminates with an exit code and reports it to the operating system. This exit code can be utilized by Bash. You can show it, you can act on it, you can control script flow with it. 
-The code is a number between 0 and 255, where the part from 126 to 255 is reserved ​to be used by the shell directly or for special purposes, like reporting a termination by a signal: +The code is a number between 0 and 255. Values ​from 126 to 255 are reserved ​for use by the shell directlyor for special purposes, like reporting a termination by a signal: 
-  * **126**: the requested command (file) can't be executed ​(but was found)+  * **126**: the requested command (file) ​was found, but can't be executed
   * **127**: command (file) not found   * **127**: command (file) not found
   * **128**: according to ABS it's used to report an invalid argument to the exit builtin, but I wasn't able to verify that in the source code of Bash (see code 255)   * **128**: according to ABS it's used to report an invalid argument to the exit builtin, but I wasn't able to verify that in the source code of Bash (see code 255)
Line 103: Line 103:
  
 The lower codes 0 to 125 are not reserved and may be used for whatever the program likes to report. The lower codes 0 to 125 are not reserved and may be used for whatever the program likes to report.
-A value of **0 means successful** termination,​ a value **not 0 means unsuccessful** termination. This behavior (== 0, != 0) is also what Bash reacts ​on in some code flow control statements.+A value of 0 means **successful** termination,​ a value not 0 means **unsuccessful** termination. This behavior (== 0, != 0) is also what Bash reacts ​to in some flow control statements.
  
 An example of using the exit code of the program ''​grep''​ to check if a specific user is present in /​etc/​passwd:​ An example of using the exit code of the program ''​grep''​ to check if a specific user is present in /​etc/​passwd:​
Line 113: Line 113:
 fi fi
 </​code>​ </​code>​
-A common command ​to use for decisions ​is the command ​"''​test''"​ or its equivalent "''​[''"​. But note that, when using the test-command ​with the command ​name "''​[''", ​**the braces ​are not part of the shell syntax, ​they are the test command!** :!:+ 
 +A common ​decision making ​command is "''​test''"​ or its equivalent "''​[''"​. But note that, when calling ​test with the name "''​[''",​ the square brackets  ​are not part of the shell syntax, the left bracket **is** the test command! 
 <code bash> <code bash>
 if [ "​$mystring"​ = "Hello world" ]; then if [ "​$mystring"​ = "Hello world" ]; then
Line 123: Line 125:
 Read more about [[commands:​classictest | the test command]] Read more about [[commands:​classictest | the test command]]
  
-A common ​way of checking the exit code is using the "''​||''"​ or "''&&''"​ operators, when you do something special only when the command ​has either failed ​or succeeded:+A common exit code check method uses the "''​||''"​ or "''&&''"​ operators. This lets you execute a command ​based on whether ​or not the previous command completed successfully:
 <code bash> <code bash>
 grep ^root: /etc/passwd >/​dev/​null || echo "root was not found - check the pub at the corner."​ grep ^root: /etc/passwd >/​dev/​null || echo "root was not found - check the pub at the corner."​
Line 129: Line 131:
 </​code>​ </​code>​
  
-Please, when your script exits on errors, provide a "​FALSE"​ exit code, that others can check the script execution.+Please, when your script exits on errors, provide a "​FALSE"​ exit code, so others can check the script execution.
  
 ===== Comments ===== ===== Comments =====
-In a bigger ​script, it's wise to comment the code. Also for debugging ​purposes ​or tests. Comments are introduced ​with # (hashmark) and go from that to the end of the line:+In a larger, or complex ​script, it's wise to comment the code. Comments can help with debugging or tests. Comments are stat with the character ​(hashmark) and continue ​to the end of the line:
 <code bash> <code bash>
 #!/bin/bash #!/bin/bash
Line 138: Line 140:
 echo "Be liberal in what you accept, and conservative in what you send" # say something echo "Be liberal in what you accept, and conservative in what you send" # say something
 </​code>​ </​code>​
-The first thing was already explained, it's the so-called shebang, for the shell, **only a comment**. The second one is a comment from the beginning of the line, where the third comment starts after a valid command. All three comments are in valid syntax.+The first thing was already explained, it's the so-called shebang, for the shell, **only a comment**. The second one is a comment from the beginning of the line, the third comment starts after a valid command. All three syntactically correct.
  
  
Line 145: Line 147:
  
  
-==== Mass commenting ==== +==== Block commenting ==== 
-To temporarily disable complete blocks of code you normally ​would have to preceede ​every line of that block with a # (hashmark) to make it being a comment. ​Now, there's a little trick, using the pseudo command '':''​ (colon) and input redirection. The '':''​ does nothing, it's a pseudo command, so it also does not care about its standard input. In the following code (you don't have to understand the code, just look what I do with the stuff), you want to test only the things that don't harm (mail, logging) ​but actually don't do anything to the system (dump database, shutdown):+To temporarily disable complete blocks of code you would normally ​have to prefix ​every line of that block with a # (hashmark) to make it a comment. ​There's a little trick, using the pseudo command '':''​ (colon) and input redirection. The '':''​ does nothing, it's a pseudo command, so it does not care about standard input. In the following code example, you want to test mail and logging, but not dump the database, ​or execute a shutdown:
 <code bash> <code bash>
 #!/bin/bash #!/bin/bash
Line 162: Line 164:
 ##### The ignored codeblock ends here ##### The ignored codeblock ends here
 </​code>​ </​code>​
-What happened, againWell, the '':''​ pseudo command was given some input by redirection (a here-document) - the pseudo command didn't care about it, effectively, ​this complete ​block is ignored ​now. +What happened? ​The '':''​ pseudo command was given some input by redirection (a here-document) - the pseudo command didn't care about it, effectively, ​the entire ​block was ignored.
-One could say, **the whole block is a comment**. For completeness:​ To make the here-document possible, the shell might generate a temporary file unter ''/​tmp''​ or similar.+
  
 The here-document-tag was quoted here **to avoid substitutions** in the "​commented"​ text! Check [[syntax:​redirection#​tag_heredoc | redirection with here-documents]] for more The here-document-tag was quoted here **to avoid substitutions** in the "​commented"​ text! Check [[syntax:​redirection#​tag_heredoc | redirection with here-documents]] for more
Line 171: Line 172:
  
  
-In Bash, the scope of the used variables is generally //global//. That means, ​if a variable is set in the "main program"​ or a "​function" ​does **not** matter, the variable is defined everywhere.+In Bash, the scope of user variables is generally //global//. That means, ​it does **not** matter whether ​a variable is set in the "main program"​ or in a "​function",​ the variable is defined everywhere.
  
-Compare the following //​equivalent//​ code snips+Compare the following //​equivalent//​ code snippets
-<​code>​+<​code ​bash>
 myvariable=test myvariable=test
 echo $myvariable echo $myvariable
 </​code>​ </​code>​
  
-<​code>​+<​code ​bash>
 myfunction() { myfunction() {
   myvariable=test   myvariable=test
Line 188: Line 189:
 </​code>​ </​code>​
  
-In both cases, the variable ''​myvariable''​ is set and accessible from everywhere in that script, functions and "main program"​.+In both cases, the variable ''​myvariable''​ is set and accessible from everywhere in that script, ​both in functions and in the "main program"​.
  
-**__Attention:​__** When you set variables in a child process, for example a //​subshell//,​ they will be set there, but you will **never** have access to them. One way to create a subshell is the pipe. It's all mentioned in a small article about [[scripting:​processtree | Bash in the processtree]]!+**__Attention:​__** When you set variables in a child process, for example a //​subshell//,​ they will be set there, but you will **never** have access to them outside of that subshell. One way to create a subshell is the pipe. It's all mentioned in a small article about [[scripting:​processtree | Bash in the processtree]]!
  
 ==== Local variables ==== ==== Local variables ====
  
-Bash provides ways to make variable scope //local// to a funtion+Bash provides ways to make variable'​s ​scope //local// to a function
-  * using the ''​local''​ keyword +  * Using the ''​local''​ keyword, or 
-  * using ''​declare''​ (will //​detect// ​that it was called from within a function ​an make the variable(s) local) +  * Using ''​declare''​ (which will //​detect// ​when it was called from within a function ​and make the variable(s) local). 
-<​code>​+<​code ​bash>
 myfunc() { myfunc() {
 local var=VALUE local var=VALUE
Line 208: Line 209:
 </​code>​ </​code>​
  
-The //local// (or declaring a variable using the ''​declare''​ command) ​keyword ​tags a variable to be treated //​completely local and separate// inside the function where it was declared:+The //​local// ​keyword ​(or declaring a variable using the ''​declare''​ command) tags a variable to be treated //​completely local and separate// inside the function where it was declared:
  
-<​code>​+<​code ​bash>
 foo=external foo=external
  
Line 232: Line 233:
 ==== Environment variables ==== ==== Environment variables ====
  
-The environment space is not directly related to the topic about scope, but it's worth to mention.+The environment space is not directly related to the topic about scope, but it's worth mentioning.
  
-Every UNIX(r) process has a so-called //​environment//​. ​Beside other stuff (unimportant for us), variables are saved there, so-called //​environment variables//​. ​Now when a child process is created (in Bash e.g. by simply executing another program, say ''​ls''​ to list files), the whole environment //including the environment variables// is copied to the new process. Reading that from the other side means: **Only variables that are part of the environment are available in the child process.**+Every UNIX(r) process has a so-called //​environment//​. ​Other itemsin addition to variablesare saved there, ​the so-called //​environment variables//​. ​When a child process is created (in Bash e.g. by simply executing another program, say ''​ls''​ to list files), the whole environment //including the environment variables// is copied to the new process. Reading that from the other side means: **Only variables that are part of the environment are available in the child process.**
  
 A variable can be tagged to be part of the environment using the ''​export''​ command: A variable can be tagged to be part of the environment using the ''​export''​ command:
-<​code>​+<​code ​bash>
 # create a new variable and set it: # create a new variable and set it:
 # -> This is a normal shell variable, not an environment variable! # -> This is a normal shell variable, not an environment variable!
Line 247: Line 248:
 </​code>​ </​code>​
  
-Remember that the //​exported//​ variable is a **copy** ​when accessed in the child process. There is no chance ​to "copy it back to the parent"​, it's a //​one-way//​! Also see the article about [[scripting:​processtree | Bash in the process tree]]! +Remember that the //​exported//​ variable is a **copy**. There is no provision ​to "copy it back to the parent." ​See the article about [[scripting:​processtree | Bash in the process tree]]!