萬盛學電腦網

您现在的位置: 萬盛學電腦網 >> 程式設計 >  >> 資料庫 >> MySQL 教學

MySQL 資料匯入實戰:mysqlimport、LOAD DATA INFILE 進階用法與 CSV/TXT 處理

工具對比:mysqlimport vs LOAD DATA INFILE

特性mysqlimportLOAD DATA INFILE
執行位置命令列(Shell)MySQL Client 內部
需指定資料庫是可不指定(先 USE db)
檔案權限Server 需讀取權限Server 需讀取權限;LOCAL 關鍵字改由 Client 讀取
適用場景批次腳本、自動化備份互動式、複雜轉換、預處理

基礎匯入範例

mysqlimport 匯入 TXT(Tab 分隔)

mysqlimport -uroot -p123456 test /tmp/mytbl.txt
# 慣例:檔名去副檔名即為表名(mytbl)

mysqlimport 匯入 CSV

mysqlimport -uroot -p --local \
  --lines-terminated-by="\r\n" \
  --fields-terminated-by="," \
  --fields-enclosed-by="\"" \
  test /tmp/mytbl.csv

LOAD DATA INFILE 匯入 TXT

LOAD DATA INFILE '/tmp/mytbl.txt' INTO TABLE mytbl;

LOAD DATA INFILE 匯入 CSV

LOAD DATA INFILE '/tmp/mytbl.csv' INTO TABLE mytbl
FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\r\n';

進階場景處理

1. 主鍵衝突策略

  • REPLACE:以新資料覆蓋現有主鍵記錄
    LOAD DATA INFILE '/tmp/mytbl.txt' REPLACE INTO TABLE mytbl ...
  • IGNORE:跳過主鍵衝突的行,保留舊資料
    LOAD DATA INFILE '/tmp/mytbl.csv' IGNORE INTO TABLE mytbl ...

2. 跳過標題行

LOAD DATA INFILE '/tmp/mytbl.txt' INTO TABLE mytbl IGNORE 1 LINES;

3. 複雜資料預處理與轉換(使用用戶變數)

來源檔 data.txt 格式:Date Time Name Weight State
目標表 tbl(dt, last_name, first_name, weight_kg, st_abbrev),需查閱 states 表轉換州名縮寫。

LOAD DATA INFILE '/tmp/data.txt' INTO TABLE tbl
IGNORE 1 LINES
(@date, @time, @name, @weight_lb, @state)
SET dt = CONCAT(@date, ' ', @time),
    first_name = SUBSTRING_INDEX(@name, ' ', 1),
    last_name = SUBSTRING_INDEX(@name, ' ', -1),
    weight_kg = @weight_lb * 0.454,
    st_abbrev = (SELECT shortname FROM states WHERE name = @state);

執行結果:

+---------------------+-----------+------------+-----------+-----------+
| dt                  | last_name | first_name | weight_kg | st_abbrev |
+---------------------+-----------+------------+-----------+-----------+
| 2006-09-01 12:00:00 | Wills     | Bill       | 90.80     | NV        |
| 2006-09-02 09:00:00 | Deft      | Jeff       | 68.10     | OK        |
| 2006-09-04 03:00:00 | Hobbs     | Bob        | 102.15    | UT        |
| 2006-09-07 08:00:00 | Banks     | Hank       | 79.45     | TX        |
+---------------------+-----------+------------+-----------+-----------+

4. Windows 本地檔案匯入至 Linux MySQL

LOAD DATA LOCAL INFILE 'C:/mytbl.txt' INTO TABLE mytbl
FIELDS TERMINATED BY '\t' LINES TERMINATED BY '\r\n';

關鍵在於 LOCAL 關鍵字,由 Client 讀取檔案再傳給 Server,繞過 Server 檔案權限限制。

效能提示

  • 大量資料匯入前:SET SESSION BULK_INSERT_BUFFER_SIZE=256M; SET SESSION MYISAM_SORT_BUFFER_SIZE=256M;(MyISAM)
  • InnoDB 建議:SET autocommit=0; 匯入後再 COMMIT; 或使用 --single-transaction 邏輯備份同理。
  • 官方宣稱 LOAD DATA 比 INSERT 快 20 倍。
copyright © 萬盛學電腦網 all rights reserved