Getting started with Linux -Part 3

• Regular Expression

  In regular expressions (regex) in Linux, a pattern is a sequence of characters that specifies a search or match operation. Patterns are constructed using metacharacters and literal characters.

  Regular expressions (regex) in Linux are patterns used to match strings or text in a file or input stream. They are used in many Linux commands, such as grep, sed, awk, and find, among others.

  Here are some common regular expression operators and their meanings in Linux:

  1. Matching a single character:

     • The dot (.) character matches any single character except a newline. For example, the pattern "h.t" matches "hat," "hot," and "hit."

  2. Matching a character set:

     • Square brackets [] can be used to match a single character from a set of characters. For example, the pattern "[aeiou]" matches any vowel.

     • The caret (^) inside square brackets negates the set, i.e., matches any character not in the set. For example, the pattern "[^aeiou]" matches any non-vowel.

  3. Matching repetitions:

     • The asterisk (*) matches zero or more occurrences of the preceding character or group. For example, the pattern "ba*t" matches "bt," "bat," "baat," "baaat," and so on.

     • The plus sign (+) matches one or more occurrences of the preceding character or group. For example, the pattern "ba+t" matches "bat," "baat," "baaat," and so on, but not "bt."

     • The question mark (?) matches zero or one occurrence of the preceding character or group. For example, the pattern "ba?t" matches "bt" and "bat," but not "baat."

  4. Anchors:

     • The caret (^) matches the beginning of a line. For example, the pattern "^Hello" matches all lines that begin with "Hello."

     • The dollar sign ($) matches the end of a line. For example, the pattern "world$" matches all lines that end with "world."

These are just a few examples of the many regular expression operators available in Linux. Understanding and using regular expressions can be a powerful tool for manipulating and searching text in Linux.

• Use of Regex in linux commands

1. Grep

   grep is a powerful Linux command used to search for patterns in files or input streams. It stands for "global regular expression print."

   The basic syntax of grep command is:

    grep [options] pattern [file(s)]  # pattern can be a string
   

   where pattern is the regular expression to search for, and file(s) are the file or files to search in. If no file is specified, grep will read from standard input (stdin).

   Here are some common options used with grep:

   Options:

   -c : This prints only a count of the lines that match a pattern

   -h : Display the matched lines, but do not display the filenames.

   -i : Ignores, case for matching

   -l : Displays list of a filenames only.

   -n : Display the matched lines and their line numbers.

   -v : This prints out all the lines that do not matches the pattern

   -e exp : Specifies expression with this option. Can use multiple times.

   -f file : Takes patterns from file, one per line.

   -E : Treats pattern as an extended regular expression (ERE)

   -w : Match whole word

   -o : Print only the matched parts of a matching line, with each such part on a separate output line.

   -A n : Prints searched line and n lines after the result.

   -B n : Prints searched line and n line before the result.

   -C n : Prints searched line and n lines after before the result.

   Here are some examples of using grep command:

   1. Search for a pattern in a file:

       grep pattern file.txt
      

      This will search for pattern in the file file.txt.

   2. Search for a pattern in multiple files:

       grep pattern file1.txt file2.txt
      

      This will search for pattern in both file1.txt and file2.txt.

   3. Search for a pattern in all files in a directory:

       grep -r pattern /path/to/directory/
      

      This will search for pattern in all files in the directory /path/to/directory/ and its subdirectories.

   4. Search for a pattern, ignoring case:

       grep -i pattern file.txt
      

      This will search for pattern in file.txt, ignoring case distinctions.

   5. Display the filenames of files that contain the pattern:

       grep -l pattern *
      

      This will search for pattern in all files in the current directory and display only the filenames of files that contain the pattern.

2. Find

   In Linux, the find command is used to search for files and directories that meet certain criteria. Regular expressions (regex) can be used with the find command to search for files and directories that match a specific pattern.

   Here are some examples of using regex with the find command:

   1. Search for files with a specific extension:

       find /path/to/directory/ -name "*.txt"
      

      This command will search for all files with the .txt extension in the directory /path/to/directory/ and its subdirectories.

   2. Search for files with a specific name pattern:

       find /path/to/directory/ -name "file*.txt"
      

      This command will search for all files with names that start with file and end with .txt in the directory /path/to/directory/ and its subdirectories.

   3. Search for files modified within a certain time frame:

       find /path/to/directory/ -type f -mtime -7
      

      This command will search for all files in the directory /path/to/directory/ and its subdirectories that have been modified within the last 7 days.

   4. Search for directories with a specific name pattern:

       find /path/to/directory/ -type d -regex ".*/dir[0-9]+"
      

      This command will search for all directories in the directory /path/to/directory/ and its subdirectories that have names that match the pattern dir[0-9]+.

   5. Search for files and directories based on size:

       find /path/to/directory/ -size +10M
      

      This command will search for all files in the directory /path/to/directory/ and its subdirectories that are larger than 10 megabytes.

3. SED

   In Linux, sed is a powerful command-line utility that can perform text transformations on input text. sed can be used with regular expressions (regex) to match and manipulate patterns in the input text.

   Here are some examples of using sed with regex in Linux:

   1. Substitute a string in a file:

       sed 's/oldstring/newstring/g' file.txt
      

      This command will replace all occurrences of "oldstring" with "newstring" in the file "file.txt".

   2. Delete lines matching a pattern:

       sed '/pattern/d' file.txt
      

      This command will delete all lines that contain the pattern "pattern" in the file "file.txt".

   3. Print lines matching a pattern:

       sed -n '/pattern/p' file.txt
      

      This command will print all lines that contain the pattern "pattern" in the file "file.txt".

   4. Replace the nth occurrence of a pattern in a line:

       sed 's/pattern/newstring/3' file.txt
      

      This command will replace the third occurrence of "pattern" with "newstring" in each line of the file "file.txt".

   5. Use a regular expression to match a pattern:

       sed '/^regex/p' file.txt
      

      This command will print all lines that start with the regular expression "regex" in the file "file.txt".

4. AWK

   In Linux, awk is another command-line utility that is useful for text processing and manipulation. awk can also be used with regular expressions (regex) to match and manipulate patterns in the input text.

   Here are some examples of using awk with regex in Linux:

   1. Print lines matching a pattern:

       awk '/pattern/' file.txt
      

      This command will print all lines that contain the pattern "pattern" in the file "file.txt".

   2. Print fields matching a pattern:

       awk '/pattern/{print $1}' file.txt
      

      This command will print the first field of all lines that contain the pattern "pattern" in the file "file.txt".

   3. Substitute a string in a file:

       awk '{gsub(/oldstring/, "newstring");print}' file.txt
      

      This command will replace all occurrences of "oldstring" with "newstring" in the file "file.txt".

   4. Print lines that match a regular expression:

       awk '/^regex/' file.txt
      

      This command will print all lines that start with the regular expression "regex" in the file "file.txt".

   5. Use regular expression to match a pattern and print specific fields:

       awk '/pattern/{print $1, $3}' file.txt
      

      This command will print the first and third fields of all lines that contain the pattern "pattern" in the file "file.txt".

5. Word count

   In Linux, you can use the grep and wc commands together to count the number of occurrences of a pattern in a file using regular expressions (regex).

   Here's an example of how to count the number of occurrences of a word "example" in a file "file.txt" using regex:

    grep -o '\<example\>' file.txt | wc -l
   

   Explanation:

   • grep -o '\<example\>' file.txt searches for the exact word "example" in the file "file.txt" using the regex pattern \< and \>. The -o option is used to output only the matched parts of a line.

   • The output of grep is then piped to wc -l, which counts the number of lines in the output. Since each match is on a separate line due to the -o option, this gives us the number of occurrences of "example" in the file.

Note that the \< and \> characters are word boundaries in regex, meaning that they match the beginning and end of a word respectively. This ensures that we count only the exact word "example" and not any other words that contain it as a substring.

1. Head

   In Linux, the head command is used to display the first few lines of a file. It can also be used with regular expressions (regex) to display lines that match a pattern.

   Here's an example of how to use head with regex to display the first 5 lines that contain the pattern "example" in a file "file.txt":

    grep 'example' file.txt | head -n 5
   

   Explanation:

   • grep 'example' file.txt searches for all lines in the file "file.txt" that contain the pattern "example".

   • The output of grep is then piped to head -n 5, which displays only the first 5 lines of the output.

Note that in this example, we are using grep to search for the pattern "example" in the file. head is used only to limit the output to the first 5 lines. You can use any regex pattern with grep to search for lines that match a pattern and then pipe the output to head to limit the number of lines displayed.

1. Tail

   In Linux, the tail command is used to display the last few lines of a file. It can also be used with regular expressions (regex) to display lines that match a pattern.

   Here's an example of how to use tail with regex to display the last 5 lines that contain the pattern "example" in a file "file.txt":

    grep 'example' file.txt | tail -n 5
   

   Explanation:

   • grep 'example' file.txt searches for all lines in the file "file.txt" that contain the pattern "example".

   • The output of grep is then piped to tail -n 5, which displays only the last 5 lines of the output.

Note that in this example, we are using grep to search for the pattern "example" in the file. tail is used only to limit the output to the last 5 lines. You can use any regex pattern with grep to search for lines that match a pattern and then pipe the output to tail to limit the number of lines displayed.

• Archive file in linux

  An Archive file is a file that is composed of one or more files along with metadata.

  Archive files are used to collect multiple data files together into a single file for easier portability and storage, or simply to compress files to use less storage space.

  In Linux, you can use the tar command to create and manage archive files.

    tar [options] [archive-file] [file or directory to be archived]
  

  Options:
  -c : Creates Archive
  -x : Extract the archive
  -f : creates archive with given filename
  -t : displays or lists files in archived file
  -u : archives and adds to an existing archive file
  -v : Displays Verbose Information
  -A : Concatenates the archive files
  -z : zip, tells tar command that creates tar file using gzip
  -j : filter archive tar file using tbzip
  -W : Verify a archive file
  -r : update or add file or directory in already existed .tar file

  Here are some examples of how to create and manage archive files in Linux:

  1. Create a tar archive file:

      tar -cvf archive.tar file1 file2 dir1
     

     This command creates a tar archive file named "archive.tar" that includes "file1", "file2", and the directory "dir1".

  2. Extract a tar archive file:

      tar -xvf archive.tar
     

     This command extracts the contents of the tar archive file "archive.tar" to the current directory.

  3. Extract a tar archive file to a specific directory:

      tar -xvf archive.tar -C /path/to/directory
     

     This command extracts the contents of the tar archive file "archive.tar" to the directory "/path/to/directory".

  4. List the contents of a tar archive file:

      tar -tvf archive.tar
     

     This command lists the contents of the tar archive file "archive.tar".

  5. Add files to an existing tar archive file:

      tar -rvf archive.tar file3 file4
     

     This command adds "file3" and "file4" to the existing tar archive file "archive.tar".

  6. Extract a specific file from a tar archive file:

      tar -xvf archive.tar file1
     

     This command extracts "file1" from the tar archive file "archive.tar".

To be continue.....