背景:機房遷移導致的編碼不兼容
- A 機房:代碼統一 UTF-8,數據庫 UTF-8
- B 機房:歷史原因要求 GBK 編碼(數據庫 GBK、頁面 GBK)
- 兩機房網絡不通,需離線轉換後物理遷移
編碼轉換原理
# 核心邏輯:讀取 → 解碼為 Unicode → 編碼為目標編碼 → 寫回
content = open(file, 'rb').read()
unicode_str = content.decode(src_encoding)
new_content = unicode_str.encode(dst_encoding)
open(file, 'wb').write(new_content)
完整遷移腳本
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import os, sys
def convert_file(filepath, from_enc, to_enc):
try:
with open(filepath, 'rb') as f:
content = f.read()
# 嘗試解碼
text = content.decode(from_enc)
# 重新編碼
new_content = text.encode(to_enc)
with open(filepath, 'wb') as f:
f.write(new_content)
return True
except Exception as e:
print(f"[FAIL] {filepath}: {e}")
return False
def walk_convert(root_dir, from_enc, to_enc):
for dirpath, _, filenames in os.walk(root_dir):
for fn in filenames:
full = os.path.join(dirpath, fn)
# 可選:只處理特定後綴
if not fn.endswith(('.py', '.html', '.js', '.css', '.txt', '.json', '.xml')):
continue
if convert_file(full, from_enc, to_enc):
print(f"[OK] {full}")
def validate_args():
if len(sys.argv) != 4:
print("Usage: python convert_encoding.py ")
print("Example: python convert_encoding.py ./src UTF-8 GBK")
sys.exit(1)
_, dirpath, from_enc, to_enc = sys.argv
if not os.path.isdir(dirpath):
print(f"Error: {dirpath} not a directory")
sys.exit(1)
if from_enc.upper() == to_enc.upper():
print("Error: source and target encoding are same")
sys.exit(1)
return dirpath, from_enc, to_enc
if __name__ == '__main__':
dirpath, from_enc, to_enc = validate_args()
walk_convert(dirpath, from_enc, to_enc)
print("Conversion complete.")
編碼知識補充
- ASCII ⊂ UTF-8:純 ASCII 文件同時是合法 UTF-8
- ANSI = 系統默認編碼:中文 Windows 下 ANSI = GBK
- Linux 查看文件編碼:
file -i filename 或 enca filename
- 特殊字符(如智能引號、表情符)可能在 GBK 中無法表示,轉換會報錯或丟失
執行示例
python convert_encoding.py /path/to/project UTF-8 GBK