萬盛學電腦網

您现在的位置: 萬盛學電腦網 >> 程式設計 >  >> 網路程式設計 >> 程式語言綜合

Python 從零構建輕量級基礎設施監控系統:Flask + Highcharts + Agent 實戰

專案背景

現有開源監控(Prometheus+Grafana、Zabbix、Datadog)功能強大但資源佔用高、學習曲線陡。對於中小規模、追求輕量、可控的團隊,用 Python + Flask + Highcharts 兩小時構建一套「採集-Agent → 存儲-MySQL → 展示-Web」最小可行監控系統,既能滿足核心指標可視化,又便於二次開發擴展。

架構總覽


+-----------+     HTTP POST JSON      +-----------+     SQL      +----------+
|  Agent    | ----------------------> |  Flask    | ---------->  |  MySQL   |
| (采集端)  |   {host, mem, load, ts} |  (Web/API)|              |  (存儲)  |
+-----------+                         +-----------+              +----------+
                                               |
                                               | GET /data (JSONP)
                                               v
                                        +--------------+
                                        |  Highcharts  |
                                        |  (瀏覽器)    |
                                        +--------------+

數據庫設計


CREATE DATABASE falcon CHARSET utf8mb4;
USE falcon;

CREATE TABLE `stat` (
  `id`        BIGINT UNSIGNED NOT NULL AUTO_INCREMENT,
  `host`      VARCHAR(255) NOT NULL,
  `mem_free`  BIGINT NOT NULL,
  `mem_usage` BIGINT NOT NULL,
  `mem_total` BIGINT NOT NULL,
  `load_avg`  VARCHAR(128) NOT NULL,
  `ts`        BIGINT NOT NULL COMMENT 'Unix ms timestamp',
  PRIMARY KEY (`id`),
  KEY `idx_host_ts` (`host`, `ts`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

Web 服務

目錄結構


web/
├── app.py
└── templates/
    └── index.html

app.py 核心代碼


import json, os, MySQLdb
from flask import Flask, request, render_template, jsonify

app = Flask(__name__)

# 資料庫連接池(生產建議用 SQLAlchemy + pool_pre_ping)
db = MySQLdb.connect(
    user=os.getenv("DB_USER", "monitor"),
    passwd=os.getenv("DB_PASS", "monitor123"),
    db=os.getenv("DB_NAME", "falcon"),
    charset="utf8mb4",
    autocommit=True
)
cursor = db.cursor()

@app.route("/", methods=["GET", "POST"])
def index():
    if request.method == "POST":
        data = request.get_json(force=True)
        sql = """INSERT INTO `stat` (`host`,`mem_free`,`mem_usage`,`mem_total`,`load_avg`,`ts`)
                 VALUES (%s,%s,%s,%s,%s,%s)"""
        try:
            cursor.execute(sql, (
                data["host"], data["mem_free"], data["mem_usage"],
                data["mem_total"], data["load_avg"], data["ts"]
            ))
            return "OK", 200
        except MySQLdb.IntegrityError:
            return "Duplicate", 409
    return render_template("index.html")

@app.route("/data")
def get_data():
    host = request.args.get("host", "")
    sql = "SELECT `ts`,`mem_usage` FROM `stat`"
    params = []
    if host:
        sql += " WHERE `host`=%s"
        params.append(host)
    sql += " ORDER BY `ts` ASC"
    cursor.execute(sql, params)
    rows = [[row[0], row[1]] for row in cursor.fetchall()]  # [timestamp, value]
    callback = request.args.get("callback")
    if callback:
        return f"{callback}({json.dumps(rows)});", 200, {"Content-Type": "application/javascript"}
    return jsonify(rows)

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=8888, debug=False)

templates/index.html Highcharts StockChart






Infra Monitor






Agent 采集端


# agent.py
import json, time, socket, psutil, requests

API_URL = "https://monitor-server:8888/"
HOSTNAME = socket.gethostname()
INTERVAL = 60  # seconds

def collect():
    mem = psutil.virtual_memory()
    load = psutil.getloadavg()[0] if hasattr(psutil, "getloadavg") else 0
    return {
        "host": HOSTNAME,
        "mem_free": mem.available,
        "mem_usage": mem.used,
        "mem_total": mem.total,
        "load_avg": f"{load:.2f}",
        "ts": int(time.time() * 1000)
    }

def push(payload):
    try:
        r = requests.post(API_URL, json=payload, timeout=5)
        print(f"[{time.ctime()}] Push {r.status_code}: {r.text}")
    except Exception as e:
        print(f"[{time.ctime()}] Push failed: {e}")

if __name__ == "__main__":
    print(f"Starting agent on {HOSTNAME}, posting to {API_URL}")
    while True:
        push(collect())
        time.sleep(INTERVAL)

依賴:pip install psutil requests。部署為 Systemd 服務或 Docker 容器即可。

部署與運維建議

  1. 資料庫索引:(host, ts) 覆蓋查詢,避免全表掃描。
  2. 數據保留:定時任務清理 30 天前數據,或使用分區表。
  3. 高可用:Flask 多進程 + Nginx 反向代理;MySQL 主從/集群。
  4. 擴展指標:CPU、磁碟、網絡、容器指標只需擴展 stat 表欄位與 Agent 采集邏輯。
  5. 告警:增加規則引擎,閾值超標發送 Webhook/郵件/鉤子。

總結

本文展示了從數據庫建模、Web API、前端圖表、Agent 採集的完整閉環,代碼總量 < 300 行,無重型依賴。適合作為監控系統原型、教學項目或輕量級生產環境起點。下一步可引入 Prometheus 客戶端庫,平滑遷移至行業標準生態。

copyright © 萬盛學電腦網 all rights reserved