How to be "awk"ward.

[dos2unix]

Using AWK in Linux: A Comprehensive Guide​

AWK is a powerful and versatile tool for text processing and data manipulation in Linux. It allows you to perform a wide range of operations on text files, making it an essential tool for system administrators, developers, and data analysts. In this article, we'll explore some of the most useful features of AWK, along with practical examples.

1. Printing Specific Columns​

One of the most common uses of AWK is to print specific columns from a text file. For example, to print the first and second columns from a space-delimited text file, you can use the following command:

 awk '{print $1, $2}' file.txt

If your file is comma-delimited, you can specify the field separator using the -F option:

 awk -F, '{print $1, $2}' file.csv

2. Pattern Matching​

AWK allows you to print lines that match a specific pattern. For example, to print lines containing the word "error":

 awk '/error/' file.txt

3. Field Manipulation​

You can perform operations on specific fields. For example, to add the values in the first and second columns:

 awk '{print $1 + $2}' file.txt

4. Conditional Statements​

AWK supports if-else statements, allowing you to perform actions based on conditions. For example, to print lines where the value in the second column is greater than 50:

 awk '$2 > 50 {print $0}' file.txt

5. Built-in Functions​

AWK has several built-in functions for string and numeric operations. Here are a couple of examples:

• To convert the first column to uppercase:

 awk '{print toupper($1)}' file.txt

• To convert all text in a file to lower case:

 awk '{print tolower($0)}' file.txt

• To convert the first character of each column to uppercase:

 awk '{for(i=1;i<=NF;i++) $i=toupper(substr($i,1,1)) tolower(substr($i,2))}1' file.txt

6. BEGIN and END Blocks​

You can use the BEGIN and END blocks to perform actions before processing any lines and after processing all lines. For example, to print a header and footer:

 awk 'BEGIN {print "Header"} {print $0} END {print "Footer"}' file.txt

7. Field Separators​

You can change the field separator to process files with different delimiters. For example, to process a CSV file:

 awk -F, '{print $1, $2}' file.csv

8. Summing Columns​

You can sum the values in a column. For example, to sum the values in the second column:

 awk '{sum += $2} END {print sum}' file.txt

---

These examples demonstrate the versatility and power of AWK for text processing in Linux. Whether you're extracting specific columns, performing calculations, or manipulating text, AWK provides a robust set of tools to help you accomplish your tasks efficiently.

 

[lizardfromoz]

I recently started using this command along with a few others. Pretty nice.

An example to extract your network interface from the nmcli output:

nmcli | grep interface | awk '{print $2}' | tail -n+2

 

[Alexzee]

Nice tutorial dos2unix!
Can awk be using in a script that was written in python?

 

[Vimmer]

Perl is honestly a lot better, even though using sed and awk in conjunction can have a lot of the same benefits, awk is great for manipulating output in bash scripts.

 

[Vimmer]

Nice tutorial dos2unix!
Can awk be using in a script that was written in python?

Py prolly has some module/library for that purpose

 

[dos2unix]

In python, I would probably use the subprocess to run awk externally

    import subprocess

    # Example: Print the first and third columns of a file
    command = "awk '{print $1, $3}' input.txt"
    try:
        output = subprocess.check_output(command, shell=True, text=True)
        print(output)
    except subprocess.CalledProcessError as e:
        print(f"Error executing awk command: {e}")

 

[Alexzee]

Py prolly has some module/library for that purpose

Thanks.
I'll find that in my python studies.

 

[Alexzee]

In python, I would probably use the subprocess to run awk externally

    import subprocess

    # Example: Print the first and third columns of a file
    command = "awk '{print $1, $3}' input.txt"
    try:
        output = subprocess.check_output(command, shell=True, text=True)
        print(output)
    except subprocess.CalledProcessError as e:
        print(f"Error executing awk command: {e}")

I'm not sure I got this, still learning.

 

[Vimmer]

I'm not sure I got this, still learning.

The module/library like we talked about is "subprocess", and the awk string is assigned to a variable, then, theres an attempt to run it using subprocess and the variable , and if it doesn't theres an error message.

 

[Alexzee]

The module/library like we talked about is "subprocess", and the awk string is assigned to a variable, then, theres an attempt to run it using subprocess and the variable , and if it doesn't theres an error message.

Ok, so in this case the place holder that stores the value is this?

 "awk '{print $1, $3}' input.txt"

 

[Vimmer]

Ok, so in this case the place holder that stores the value is this?

 "awk '{print $1, $3}' input.txt"

No thats the value, the placeholder is "command"

 

[Alexzee]

No thats the value, the placeholder is "command"

Got it, thanks!

 

[CptKrf]

Perl is honestly a lot better, even though using sed and awk in conjunction can have a lot of the same benefits, awk is great for manipulating output in bash scripts.

Absolutely agree. Perl is my goto (no pun intended) script of choice. Learned it back in the turn of the century and still prefer it. Interestingly, I consider myself fairly expert in the language, but with AI coding now, I am sometimes astounded at what can be done with Perl that is not in the books. It certainly won't replace C or C++ for down in the hardware bit twiddling or speed, but for anything that involves text, no competition.

That being said, I seldom use Awk, but I run the wheels off of Gawk almost every day.