框架與平台
LlamaIndex
使用 OpenAI 相容整合將 TokenLab 與 LlamaIndex 進行整合
概覽
對於 TokenLab 而言，更穩健的 LlamaIndex 設定方式是使用 OpenAI 相容整合，而非內建的 OpenAI 類別。
目前的 LlamaIndex 文件明確建議針對第三方 OpenAI 相容端點使用 OpenAILike，因為內建的 OpenAI 類別會從官方模型名稱中推斷元數據 (metadata)。
換句話說：請將 OpenAILike 視為此處支援的 TokenLab 路徑，而非內建的 OpenAI 類別。
安裝
pip install llama-index-core \
llama-index-readers-file \
llama-index-llms-openai-like \
llama-index-embeddings-openai-like基本設定
from llama_index.core import Settings
from llama_index.llms.openai_like import OpenAILike
from llama_index.embeddings.openai_like import OpenAILikeEmbedding
llm = OpenAILike(
model="gpt-5.6-terra",
api_base="https://api.tokenlab.sh/v1",
api_key="sk-your-tokenlab-key",
is_chat_model=True,
)
embed_model = OpenAILikeEmbedding(
model_name="text-embedding-3-small",
api_base="https://api.tokenlab.sh/v1",
api_key="sk-your-tokenlab-key",
)
Settings.llm = llm
Settings.embed_model = embed_model基本用法
response = llm.complete("Explain TokenLab in one sentence.")
print(response.text)最小化 OpenAILike LLM
from llama_index.llms.openai_like import OpenAILike
llm = OpenAILike(
model="claude-sonnet-5",
api_base="https://api.tokenlab.sh/v1",
api_key="sk-your-tokenlab-key",
context_window=200000,
is_chat_model=True,
is_function_calling_model=True,
)聊天 (Chat)
from llama_index.core.llms import ChatMessage
messages = [
ChatMessage(role="system", content="You are a helpful assistant."),
ChatMessage(role="user", content="What is the capital of France?")
]
response = llm.chat(messages)
print(response.message.content)串流 (Streaming)
for chunk in llm.stream_complete("Write a short poem about AI."):
print(chunk.delta, end="", flush=True)嵌入 (Embeddings)
vector = embed_model.get_text_embedding("Hello, world!")
print(vector[:5])使用文件進行 RAG
from llama_index.core import SimpleDirectoryReader, VectorStoreIndex
documents = SimpleDirectoryReader("./data").load_data()
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()
response = query_engine.query("What is in my documents?")
print(response)聊天引擎 (Chat Engine)
chat_engine = index.as_chat_engine(chat_mode="condense_question")
response = chat_engine.chat("What is TokenLab?")
print(response)
response = chat_engine.chat("How many models does it support?")
print(response)非同步用法 (Async Usage)
import asyncio
async def main():
response = await llm.acomplete("Hello!")
print(response.text)
asyncio.run(main())