Sphinx 內置 ngram 分詞對中文支援有限,結合 SCWS(Simple Chinese Word Segmentation)實現精準中文分詞,構建生產級中文搜索引擎。本文涵蓋編譯安裝、詞庫配置、PHP 擴展整合、測試驗證全流程。
tar zxvf sphinx-2.2.5-release.tar.gz
cd sphinx-2.2.5-release
./configure --prefix=/usr/local/sphinx2 --with-mysql=/usr/local/mysql
make && make install
cd api/libsphinxclient
./configure --prefix=/usr/local/sphinx2/libsphinxclient
make && make install
tar zxvf sphinx-1.3.1.tgz
cd sphinx-1.3.1
phpize
./configure --with-sphinx=/usr/local/sphinx2/libsphinxclient --with-php-config=/usr/bin/php-config
make && make install
tar xvjf scws-1.2.2.tar.bz2
mkdir /usr/local/scws
cd scws-1.2.2
./configure --prefix=/usr/local/scws
make && make install
cd scws-1.2.2/phpext
phpize
./configure --with-php-config=/usr/bin/php-config
make && make install
tar xvjf scws-dict-chs-utf8.tar.bz2 -C /usr/local/scws/etc/
chown php-fpm-user:php-fpm-group /usr/local/scws/etc/dict.utf8.xdb
# 必須讓 php-fpm 運行用戶擁有詞庫讀取權限,否則報 Warning: Failed to add the dict file
查看 php-fpm 用戶:ps aux | grep php-fpm
source myorder { ... } -- 同 179670
index myorder {
source = myorder
path = /usr/local/sphinx2/var/data/myorder
docinfo = extern
min_word_len = 1
html_strip = 1
charset_table = U+FF10..U+FF19->0..9, 0..9, U+FF41..U+FF5A->a..z, ... # 完整中日韓字符映射
ngram_len = 1
ngram_chars = U+4E00..U+9FBF, U+3400..U+4DBF, ... # CJK 統一表意文字範圍
}
注意:Sphinx 2.2+ 移除 charset_type、sql_query_info 配置項,啟動會警告,需刪除。
[sphinx]
extension = sphinx.so
[scws]
extension = scws.so
scws.default.charset = utf-8
scws.default.fpath = /usr/local/scws/etc
# 1. 構建索引
/usr/local/sphinx2/bin/indexer --config /usr/local/sphinx2/etc/sphinx.conf --all
# 2. 啟動 searchd
/usr/local/sphinx2/bin/searchd --config /usr/local/sphinx2/etc/sphinx.conf
# 3. 重啟 php-fpm
/etc/init.d/php-fpm restart
mysql -h 127.0.0.1 -P 9306
SELECT * FROM myorder WHERE MATCH('張');
+------+--------+------------+----------+-------------+----------------+
| id | weight | user_id | username | create_time | product_name |
+------+--------+------------+----------+-------------+----------------+
| 9 | 1304 | 1311895262 | 張三 | 1406823894 | tank is 坦克 |
...
set_charset('utf-8');
$so->add_dict(ini_get('scws.default.fpath') . '/dict.utf8.xdb'); // 主詞庫
$so->set_rule(ini_get('scws.default.fpath') . '/rules.utf8.ini'); // 模糊規則
$so->set_ignore(true); // 過濾標點
$so->set_multi(false); // 不做復式分割
$so->set_duality(false); // 不做二元聚合
$so->send_text($key);
$words_array = $so->get_result();
$so->close();
// 組裝查詢詞:詞與詞用 | 隔開,擴展召回
$words = "";
foreach ($words_array as $v) { $words .= '|(' . $v['word'] . ')'; }
$words = trim($words, '|');
// 可選:加入原詞精確匹配
// $words = '('.$key.')'.$words;
echo "輸入:$key
分詞:$words
";
// === Sphinx 查詢 ===
$sc = new SphinxClient();
$sc->SetServer('127.0.0.1', 9312); // 原生 API 端口
$sc->SetMatchMode(SPH_MATCH_EXTENDED);
$sc->SetArrayResult(true);
$res = $sc->Query($words, $index);
print_r($res);
$e_time = microtime(true);
echo "耗時:" . ($e_time - $b_time) . " 秒
";
Array
(
[matches] => Array
(
[0] => Array
(
[id] => 9
[weight] => 2500
[attrs] => Array
(
[user_id] => 1311895262
[username] => 張三
[create_time] => 1406823894
[product_name] => tank is 坦克
)
)
)
[total] => 1
[total_found] => 1
[time] => 0
[words] => Array
(
[張] => Array([docs]=>4, [hits]=>4)
[三] => Array([docs]=>1, [hits]=>1)
)
)
weight:BM25 相關度分數,越高越相關attrs:sql_attr_* 定義的屬性,供業務邏輯直接使用,無需回表words:查詢詞拆解統計,docs=包含該詞文檔數,hits=詞總出現次數$so->add_dict('./custom.txt', SCWS_XDICT_TXT) 載入文本詞庫,優先級高於主詞庫set_multi(true) 開啟復式分割(短詞|二元|主要單字|所有單字),提高召回率INSERT/REPLACE/DELETE 索引,無需重建