DolphinDB HTTP API接入：RESTful数据推送

目录

• • 摘要
  • [一、HTTP API概述](#一、HTTP API概述)
  • • [1.1 什么是HTTP API](#1.1 什么是HTTP API)
    • [1.2 HTTP API特点](#1.2 HTTP API特点)
    • [1.3 适用场景](#1.3 适用场景)
  • [二、DolphinDB HTTP API](#二、DolphinDB HTTP API)
  • • [2.1 API端点](#2.1 API端点)
    • [2.2 数据推送接口](#2.2 数据推送接口)
    • [2.3 查询接口](#2.3 查询接口)
  • 三、数据推送
  • • [3.1 单条数据推送](#3.1 单条数据推送)
    • [3.2 批量数据推送](#3.2 批量数据推送)
    • [3.3 数据格式](#3.3 数据格式)
  • 四、API认证
  • • [4.1 基本认证](#4.1 基本认证)
    • [4.2 Token认证](#4.2 Token认证)
    • [4.3 API密钥](#4.3 API密钥)
  • 五、错误处理
  • • [5.1 错误响应](#5.1 错误响应)
    • [5.2 重试机制](#5.2 重试机制)
  • 六、性能优化
  • • [6.1 批量写入](#6.1 批量写入)
    • [6.2 连接池](#6.2 连接池)
    • [6.3 异步写入](#6.3 异步写入)
  • 七、实战案例
  • • [7.1 设备数据上报API](#7.1 设备数据上报API)
  • 八、总结
  • 参考资料

摘要

本文深入讲解DolphinDB HTTP API接入技术。从HTTP接口设计到数据推送，从批量写入到API认证，从错误处理到性能优化，全面介绍HTTP数据接入的核心方法。通过丰富的代码示例，帮助读者掌握RESTful数据推送的核心技能。

---

一、HTTP API概述

1.1 什么是HTTP API

HTTP API是通过HTTP协议进行数据交互的接口：
#mermaid-svg-2ShN5XrTpW36s29F{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-2ShN5XrTpW36s29F .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-2ShN5XrTpW36s29F .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-2ShN5XrTpW36s29F .error-icon{fill:#552222;}#mermaid-svg-2ShN5XrTpW36s29F .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-2ShN5XrTpW36s29F .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-2ShN5XrTpW36s29F .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-2ShN5XrTpW36s29F .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-2ShN5XrTpW36s29F .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-2ShN5XrTpW36s29F .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-2ShN5XrTpW36s29F .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-2ShN5XrTpW36s29F .marker{fill:#333333;stroke:#333333;}#mermaid-svg-2ShN5XrTpW36s29F .marker.cross{stroke:#333333;}#mermaid-svg-2ShN5XrTpW36s29F svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-2ShN5XrTpW36s29F p{margin:0;}#mermaid-svg-2ShN5XrTpW36s29F .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-2ShN5XrTpW36s29F .cluster-label text{fill:#333;}#mermaid-svg-2ShN5XrTpW36s29F .cluster-label span{color:#333;}#mermaid-svg-2ShN5XrTpW36s29F .cluster-label span p{background-color:transparent;}#mermaid-svg-2ShN5XrTpW36s29F .label text,#mermaid-svg-2ShN5XrTpW36s29F span{fill:#333;color:#333;}#mermaid-svg-2ShN5XrTpW36s29F .node rect,#mermaid-svg-2ShN5XrTpW36s29F .node circle,#mermaid-svg-2ShN5XrTpW36s29F .node ellipse,#mermaid-svg-2ShN5XrTpW36s29F .node polygon,#mermaid-svg-2ShN5XrTpW36s29F .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-2ShN5XrTpW36s29F .rough-node .label text,#mermaid-svg-2ShN5XrTpW36s29F .node .label text,#mermaid-svg-2ShN5XrTpW36s29F .image-shape .label,#mermaid-svg-2ShN5XrTpW36s29F .icon-shape .label{text-anchor:middle;}#mermaid-svg-2ShN5XrTpW36s29F .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-2ShN5XrTpW36s29F .rough-node .label,#mermaid-svg-2ShN5XrTpW36s29F .node .label,#mermaid-svg-2ShN5XrTpW36s29F .image-shape .label,#mermaid-svg-2ShN5XrTpW36s29F .icon-shape .label{text-align:center;}#mermaid-svg-2ShN5XrTpW36s29F .node.clickable{cursor:pointer;}#mermaid-svg-2ShN5XrTpW36s29F .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-2ShN5XrTpW36s29F .arrowheadPath{fill:#333333;}#mermaid-svg-2ShN5XrTpW36s29F .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-2ShN5XrTpW36s29F .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-2ShN5XrTpW36s29F .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-2ShN5XrTpW36s29F .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-2ShN5XrTpW36s29F .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-2ShN5XrTpW36s29F .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-2ShN5XrTpW36s29F .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-2ShN5XrTpW36s29F .cluster text{fill:#333;}#mermaid-svg-2ShN5XrTpW36s29F .cluster span{color:#333;}#mermaid-svg-2ShN5XrTpW36s29F div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-2ShN5XrTpW36s29F .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-2ShN5XrTpW36s29F rect.text{fill:none;stroke-width:0;}#mermaid-svg-2ShN5XrTpW36s29F .icon-shape,#mermaid-svg-2ShN5XrTpW36s29F .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-2ShN5XrTpW36s29F .icon-shape p,#mermaid-svg-2ShN5XrTpW36s29F .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-2ShN5XrTpW36s29F .icon-shape .label rect,#mermaid-svg-2ShN5XrTpW36s29F .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-2ShN5XrTpW36s29F .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-2ShN5XrTpW36s29F .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-2ShN5XrTpW36s29F :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} HTTP API架构
数据源
HTTP POST
DolphinDB API
数据存储
客户端
HTTP GET
查询结果

1.2 HTTP API特点

特点	说明
通用性	任何语言支持
简单易用	RESTful风格
无状态	每次请求独立
可扩展	易于扩展

1.3 适用场景

场景	说明
数据推送	设备数据上报
数据查询	数据查询接口
系统集成	第三方系统集成
Web应用	前端数据展示

---

二、DolphinDB HTTP API

2.1 API端点

// DolphinDB HTTP API端点
// http://host:port/api

// 示例
// http://localhost:8848/api

2.2 数据推送接口

// 创建流表接收数据
share streamTable(1:0, 
    `device_id`timestamp`temperature`humidity,
    [SYMBOL, TIMESTAMP, DOUBLE, DOUBLE]) as http_stream

// HTTP POST推送数据
// URL: http://localhost:8848/api/write
// Body: {"table": "http_stream", "data": [{"device_id": "D001", "timestamp": "2024-01-01 00:00:00", "temperature": 25.5, "humidity": 50.0}]}

2.3 查询接口

// HTTP GET查询数据
// URL: http://localhost:8848/api/query?sql=select * from http_stream limit 10

// 返回JSON格式结果

---

三、数据推送

3.1 单条数据推送

// Python示例：推送单条数据
import requests
import json

url = "http://localhost:8848/api/write"
data = {
    "table": "http_stream",
    "data": [{
        "device_id": "D001",
        "timestamp": "2024-01-01T00:00:00",
        "temperature": 25.5,
        "humidity": 50.0
    }]
}

response = requests.post(url, json=data)
print(response.json())

3.2 批量数据推送

// Python示例：批量推送数据
import requests
import json
from datetime import datetime

url = "http://localhost:8848/api/write"

// 生成批量数据
data = {
    "table": "http_stream",
    "data": [
        {
            "device_id": f"D{i:03d}",
            "timestamp": datetime.now().isoformat(),
            "temperature": 20 + i * 0.1,
            "humidity": 40 + i * 0.5
        }
        for i in range(100)
    ]
}

response = requests.post(url, json=data)
print(f"写入 {len(data['data'])} 条数据")

3.3 数据格式

// JSON数据格式
{
    "table": "表名",
    "data": [
        {
            "column1": "value1",
            "column2": 123,
            "column3": 45.6
        }
    ]
}

// CSV格式
// device_id,timestamp,temperature,humidity
// D001,2024-01-01T00:00:00,25.5,50.0

---

四、API认证

4.1 基本认证

// HTTP Basic认证
import requests
from requests.auth import HTTPBasicAuth

url = "http://localhost:8848/api/write"
auth = HTTPBasicAuth("admin", "123456")

response = requests.post(url, json=data, auth=auth)

4.2 Token认证

// Token认证
import requests

url = "http://localhost:8848/api/write"
headers = {
    "Authorization": "Bearer your_token_here"
}

response = requests.post(url, json=data, headers=headers)

4.3 API密钥

// API Key认证
import requests

url = "http://localhost:8848/api/write"
headers = {
    "X-API-Key": "your_api_key_here"
}

response = requests.post(url, json=data, headers=headers)

---

五、错误处理

5.1 错误响应

// 错误响应格式
{
    "success": false,
    "error": {
        "code": "INVALID_DATA",
        "message": "数据格式错误"
    }
}

// 处理错误
def handleResponse(response):
    if response.status_code == 200:
        result = response.json()
        if result.get("success"):
            return result["data"]
        else:
            raise Exception(result["error"]["message"])
    else:
        raise Exception(f"HTTP错误: {response.status_code}")

5.2 重试机制

// 重试机制
import requests
import time

def postWithRetry(url, data, maxRetries=3):
    for i in range(maxRetries):
        try:
            response = requests.post(url, json=data, timeout=10)
            if response.status_code == 200:
                return response.json()
        except Exception as e:
            print(f"重试 {i+1}/{maxRetries}: {e}")
            time.sleep(1)
    raise Exception("请求失败")

---

六、性能优化

6.1 批量写入

// 批量写入优化
// 推荐：每次写入1000-10000条

def batchWrite(url, data, batchSize=1000):
    for i in range(0, len(data), batchSize):
        batch = data[i:i+batchSize]
        response = requests.post(url, json={"table": "http_stream", "data": batch})
        print(f"写入批次 {i//batchSize + 1}: {len(batch)} 条")

6.2 连接池

// 使用连接池
import requests
from requests.adapters import HTTPAdapter

session = requests.Session()
adapter = HTTPAdapter(pool_connections=10, pool_maxsize=100)
session.mount('http://', adapter)

// 使用session发送请求
response = session.post(url, json=data)

6.3 异步写入

// 异步写入
import asyncio
import aiohttp

async def asyncWrite(url, data):
    async with aiohttp.ClientSession() as session:
        async with session.post(url, json=data) as response:
            return await response.json()

// 批量异步写入
async def batchAsyncWrite(url, dataList):
    tasks = [asyncWrite(url, data) for data in dataList]
    results = await asyncio.gather(*tasks)
    return results

---

七、实战案例

7.1 设备数据上报API

// ========== 设备数据上报API ==========

// 1. 创建流表
share streamTable(100000:0, 
    `device_id`timestamp`temperature`humidity`pressure,
    [SYMBOL, TIMESTAMP, DOUBLE, DOUBLE, DOUBLE]) as device_stream

// 2. 启用持久化
enableTablePersistence(device_stream, true, true, 1000000)

// 3. 创建分布式表
db = database("dfs://http_db", VALUE, 1..1000)
schema = table(1:0, `device_id`timestamp`temperature`humidity`pressure,
               [SYMBOL, TIMESTAMP, DOUBLE, DOUBLE, DOUBLE])
db.createPartitionedTable(schema, `device_data, `device_id)

// 4. 订阅写入
subscribeTable(, "device_stream", "persist", -1,
    def(msg) {
        loadTable("dfs://http_db", "device_data").append!(msg)
    }, 10000, 5000)

// 5. API端点说明
// POST http://localhost:8848/api/write
// Body: {"table": "device_stream", "data": [...]}

print("设备数据上报API已就绪")

---

八、总结

本文详细介绍了DolphinDB HTTP API接入：

1. HTTP API原理：RESTful风格、通用接口
2. 数据推送：单条推送、批量推送、数据格式
3. API认证：基本认证、Token认证、API密钥
4. 错误处理：错误响应、重试机制
5. 性能优化：批量写入、连接池、异步写入

思考题：

1. HTTP API有什么优势？
2. 如何设计高性能的数据推送接口？
3. 如何保证数据推送的可靠性？

---

参考资料

• DolphinDB HTTP API
• RESTful API设计

---