uniq 是文字處理利器，專門用於**過濾相鄰重複行**。關鍵特性：**只能偵測連續重複的行**，因此實務上必須搭配 sort 先排序。本文詳細說明所有參數、實戰範例與常見陷阱。

核心觀念：uniq + sort = 完整去重

# 錯誤：未排序，非相鄰重複行無法去除
uniq file.txt

# 正確：先排序再去重
sort file.txt | uniq
# 或直接
sort -u file.txt

參數完整對照表

參數	長參數	功能
-c	--count	每行前加上出現次數
-d	--repeated	僅輸出重複的行 (每組一行)
-D	--all-repeated	輸出所有重複行 (保留重複行數)，不可與 -c 合用
-f N	--skip-fields=N	忽略前 N 個欄位 (以空白分隔)
-i	--ignore-case	比較時忽略大小寫
-s N	--skip-chars=N	忽略每行前 N 個字元
-u	--unique	僅輸出不重複的行 (去重後完整顯示)
-w N	--check-chars=N	只比較每行前 N 個字元
-z	--zero-terminated	以 NUL 字元而非換行符作為行結束 (配合 sort -z)

實戰範例 (測試檔 uniqtest)

this is a test
this is a test
this is a test
i am tank
i love tank
i love tank
this is a test
whom have a try
WhoM have a try
you have a try
i want to abroad
those are good men
we are good men

1. 計數去重 (需先排序)

sort uniqtest | uniq -c
# 輸出:
#  1 WhoM have a try
#  1 i am tank
#  2 i love tank
#  1 i want to abroad
#  4 this is a test
#  1 those are good men
#  1 we are good men
#  1 whom have a try
#  1 you have a try

2. 僅顯示重複行

sort uniqtest | uniq -d
# i love tank
# this is a test

3. 顯示所有重複行 (含重複次數)

sort uniqtest | uniq -D
# i love tank
# i love tank
# this is a test
# this is a test
# this is a test
# this is a test

4. 忽略第一欄位比較

sort uniqtest | uniq -f 1 -c
# 因忽略第一欄，"those are good men" 與 "we are good men" 視為相同 → 計數 2

5. 忽略大小寫

sort uniqtest | uniq -i -c
# "whom have a try" 與 "WhoM have a try" 合併計數 2

6. 忽略前 4 字元

sort uniqtest | uniq -s 4 -c
# "whom have a try" (忽略 whom) 與 "you have a try" (忽略 you ) 相同 → 計數 3

7. 僅輸出唯一行

sort uniqtest | uniq -u
# i am tank
# i want to abroad
# those are good men
# we are good men

8. 只比較前 2 字元

sort uniqtest | uniq -w 2 -c
# "i am tank" 與 "i love tank" 前 2 字元 "i " 相同 → 合併計數

進階技巧

• 大檔案去重：sort -u largefile > uniqfile (記憶體不足時加 -T /tmp 指定暫存目錄)。
• 保留原始順序去重：awk '!seen[$0]++' file.txt (不需排序，記憶體允許下最快)。
• CSV 特定欄位去重：sort -t, -k2,2 -u file.csv (按第 2 欄去重)。