UNIX pipes and filters

Pipes and filters

You can connect two commands together so that the output of one program can be used as input to the next program. Two or more commands are connected in this way to form a pipeline.

To form a pipe, use a vertical line (or (|) Separate the two commands.

If a program treats the output of another program as input data, then performs some operations on the input data and writes the results to the standard output, it is called a filter.

Grep command

The grep program searches for one or more files in a fixed mode. Its syntax is:

$grep pattern file(s)

The name "grep" comes from the ed (UNIX Line Editor) command, g/re/p search with regular expressions and printing all the rows that contain it." ”

Regular expressions are plain text (for example, a word) / special characters that are used for pattern matching.

The simplest grep use is to match patterns that consist of one word. I t can be used in pipelines, so only those input lines that contain a given string are sent to the standard output. If you don't specify the file name that grep reads, it reads the standard input, which is how all filters work:

$ls -l | grep "Aug"
-rw-rw-rw-   1 john  doc     11008 Aug  6 14:10 ch02
-rw-rw-rw-   1 john  doc      8515 Aug  6 15:30 ch07
-rw-rw-r--   1 john  doc      2488 Aug 15 10:51 intro
-rw-rw-r--   1 carol doc      1605 Aug 23 07:35 macros
$

Here are a variety of optional parameters that you can use in the grep command:

Next, let's use a regular expression that lets the grep command find a line that contains the "carol" letter, followed by zero or more letters, which means ".", followed by the "Aug" character.

Here's how to use the -i parameter, which means you're not sensitive to letter case:

$ls -l | grep -i "carol.*aug"
-rw-rw-r--   1 carol doc      1605 Aug 23 07:35 macros
$

Sort command

The sort command sorts line text in alphabetical or numerical order. The following example is sorting the text in the food file:

$sort food
Afghani Cuisine
Bangkok Wok
Big Apple Deli
Isle of Java
Mandalay
Sushi and Sashimi
Sweet Tooth
Tio Pepe's Peppers
$

The sort command is sorted alphabetically by default. There are many optional parameters to control sorting:

Two or more commands can form a pipeline. Take the grep command mentioned earlier, for example, where we can sort the August file further by the size of the file.

The following pipeline contains the ls, grep, and sort commands:

$ls -l | grep "Aug" | sort +4n
-rw-rw-r--  1 carol doc      1605 Aug 23 07:35 macros
-rw-rw-r--  1 john  doc      2488 Aug 15 10:51 intro
-rw-rw-rw-  1 john  doc      8515 Aug  6 15:30 ch07
-rw-rw-rw-  1 john  doc     11008 Aug  6 14:10 ch02
$

The above pipeline sorts the files in the August directory by the size of the files and prints them to the terminal screen. The sort +4n skips four fields (fields separated by spaces) and then sorts the rows numerically.

Introduction to the pg and more commands

Output that is too long is usually compressed on your screen, but if you use the more or pg command as a filter, you won't stop until you know that the screen is full of text.

Suppose you have a long list of directories. To make it easy to read, we'll sort it and process the output of the pipeline by using the more command:

$ls -l | grep "Aug" | sort +4n | more
-rw-rw-r--  1 carol doc      1605 Aug 23 07:35 macros
-rw-rw-r--  1 john  doc      2488 Aug 15 10:51 intro
-rw-rw-rw-  1 john  doc      8515 Aug  6 15:30 ch07
-rw-rw-r--  1 john  doc     14827 Aug  9 12:40 ch03
    .
    .
    .
-rw-rw-rw-  1 john  doc     16867 Aug  6 15:56 ch05
--More--(74%)

The screen will be filled with text data, which is in file size order. At the bottom of the screen is a more command that you can tap in to let the screen scroll through more data.

When the screen is complete, you can then use any commands about the more program that are said in the discussion section.