uniq linux command

Linux is full of powerful terminal tools that are designed to do separate functions. These linux tools are so great, they are like individual lego blocks. When you fit these set if lego blocks together, it creates a new set of linux tool providing a new way to achieve your goal without going deeper on coding and programming to do the same similar job.

Here's another linux tool called uniq. Uniq linux command function to omit or report repeated lines.

Shown below are the different usage for uniq commands.

Supposed we have a testfile.txt that contains the following text data.

# cat testfile.txt
~~~~~~~~~~~~~~~~
abc
abc
abc
efg
xyz
xyz
~~~~~~~~~~~~~~~~

Here is how to print unique lines of text file.

# uniq testfile.txt
~~~~~~~~~~~~~~~~~~~~~~
abc
efg
xyz
~~~~~~~~~~~~~~~~~~~~~~


How to print and count the number of occurrence of each line from text file?

# uniq -c testfile.txt
~~~~~~~~~~~~~~~~~~~~~~
3 abc
1 efg
2 xyz
~~~~~~~~~~~~~~~~~~~~~~


How to print all repeated or duplicated occurrence of each line from text file?

# uniq -d testfile.txt
~~~~~~~~~~~~~~~~~~~~~~
abc
xyz
~~~~~~~~~~~~~~~~~~~~~~


How to print all unique occurrence of each line from text file?

# uniq -u testfile.txt
~~~~~~~~~~~~~~~~~~~~~~
efg
~~~~~~~~~~~~~~~~~~~~~~

Now, let modify testfile.txt to look like so
~~~~~~~~~~
abcd
abc
abc
efg
xyz
xyz
~~~~~~~~~~

Now, here's how to print uniq occurrences found on the first N characters of line from text file?
Assuming we would only compare the first 2 characters of each line and print unique occurrences.

# uniq -w 2 -u testfile.txt
~~~~~~~~~~
efg
~~~~~~~~~~


How to compare the first 2 characters of each line and print only the duplicated occurrences.

# uniq -w 2 -d testfile.txt
~~~~~~~~~~
abcd
xyz
~~~~~~~~~~


How to avoid comparing the N character of each line from text file using uniq linux command?

Assuming we want to avoid comparison of the first 2 characters of each line from text file and print only the unique occurrences with its corresponding number of occurrences.

# uniq -s 2 -u -c testfile.txt
~~~~~~~~~~~~~~~~~~
1 abcd
1 efg
~~~~~~~~~~~~~~~~~~


And another more, here is how to print only the duplicated lines of occurrences with comparison point starting after the 2nd character of a each lines using uniq linux command?

# uniq -s 2 -d -c testfile.txt
~~~~~~~~~~~~~~~~~~
2 abc
2 xyz
~~~~~~~~~~~~~~~~~~

Redirection of standard output is always possible with these linux commands.

Finished!