資料庫重複資料清理三大經典模式,按資料量、版本、複雜度選型。
DECLARE @max INT, @id INT;
DECLARE cur CURSOR LOCAL FOR
SELECT 主鍵, COUNT(*) FROM 表名 GROUP BY 主鍵 HAVING COUNT(*) > 1;
OPEN cur;
FETCH NEXT FROM cur INTO @id, @max;
WHILE @@FETCH_STATUS = 0
BEGIN
SET @max = @max - 1; -- 保留 1 筆
SET ROWCOUNT @max; -- 限制刪除筆數
DELETE FROM 表名 WHERE 主鍵 = @id;
FETCH NEXT FROM cur INTO @id, @max;
END
CLOSE cur; DEALLOCATE cur;
SET ROWCOUNT 0; -- 重置全域限制
優點:分批刪除避免長鎖表、不需額外空間。
缺點:游標慢、SET ROWCOUNT 影響後續語句、SQL 2012+ 棄用 ROWCOUNT。
-- 1. 建立無重複臨時表
SELECT DISTINCT * INTO #Tmp FROM TableName;
-- 2. 清空原表
TRUNCATE TABLE TableName; -- 或 DELETE (若有觸發器)
-- 3. 寫回
INSERT INTO TableName SELECT * FROM #Tmp;
DROP TABLE #Tmp;
適用:完全重複 (所有欄位相同)、中小表。
根治:事後**加上唯一索引/主鍵**防止再犯。
-- 假設重複判斷欄位為 Name, Address
SELECT IDENTITY(INT,1,1) AS autoID, * INTO #Tmp FROM TableName;
SELECT MIN(autoID) AS autoID INTO #Tmp2 FROM #Tmp GROUP BY Name, Address;
SELECT * FROM #Tmp WHERE autoID IN (SELECT autoID FROM #Tmp2);
-- 最後 SELECT 即為去重結果,實際刪除需配合 MERGE 或 JOIN DELETE
;WITH cte AS (
SELECT *,
ROW_NUMBER() OVER (PARTITION BY Name, Address ORDER BY ID) AS rn
FROM TableName
)
DELETE FROM cte WHERE rn > 1;
優勢:單條 SQL、可讀性高、效能最優、支援任意保留策略 (最早/最新/最大 ID)。
| 場景 | 推薦方法 | 版本需求 |
|---|---|---|
| 完全重複、小表 | DISTINCT + 臨時表 | All |
| 關鍵欄位重複、大表、需分批 | 游標 + ROWCOUNT | 2000+ |
| 關鍵欄位重複、追求效能/可維護 | ROW_NUMBER() CTE | 2005+ |
| 生產環境標準化 | ROW_NUMBER() + 交易控制 | 2005+ |
UNIQUE INDEX / PRIMARY KEY