Framework & platform
LlamaIndex
Integrasikan TokenLab dengan LlamaIndex menggunakan integrasi yang kompatibel dengan OpenAI
Gambaran Umum
Gunakan LlamaIndex OpenAILike untuk TokenLab. Ini menerima Base URL kustom dan tidak menyimpulkan metadata model dari nama model OpenAI.
Instalasi
pip install llama-index-core \
llama-index-readers-file \
llama-index-llms-openai-like \
llama-index-embeddings-openai-likeKonfigurasi Dasar
from llama_index.core import Settings
from llama_index.llms.openai_like import OpenAILike
from llama_index.embeddings.openai_like import OpenAILikeEmbedding
llm = OpenAILike(
model="gpt-5.6-terra",
api_base="https://api.tokenlab.sh/v1",
api_key="sk-your-tokenlab-key",
is_chat_model=True,
)
embed_model = OpenAILikeEmbedding(
model_name="text-embedding-3-small",
api_base="https://api.tokenlab.sh/v1",
api_key="sk-your-tokenlab-key",
)
Settings.llm = llm
Settings.embed_model = embed_modelPenggunaan Dasar
response = llm.complete("Explain TokenLab in one sentence.")
print(response.text)LLM OpenAILike Minimal
from llama_index.llms.openai_like import OpenAILike
llm = OpenAILike(
model="claude-sonnet-5",
api_base="https://api.tokenlab.sh/v1",
api_key="sk-your-tokenlab-key",
context_window=200000,
is_chat_model=True,
is_function_calling_model=True,
)Chat
from llama_index.core.llms import ChatMessage
messages = [
ChatMessage(role="system", content="You are a helpful assistant."),
ChatMessage(role="user", content="What is the capital of France?")
]
response = llm.chat(messages)
print(response.message.content)Streaming
for chunk in llm.stream_complete("Write a short poem about AI."):
print(chunk.delta, end="", flush=True)Embeddings
vector = embed_model.get_text_embedding("Hello, world!")
print(vector[:5])RAG dengan Dokumen
from llama_index.core import SimpleDirectoryReader, VectorStoreIndex
documents = SimpleDirectoryReader("./data").load_data()
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()
response = query_engine.query("What is in my documents?")
print(response)Chat Engine
chat_engine = index.as_chat_engine(chat_mode="condense_question")
response = chat_engine.chat("What is TokenLab?")
print(response)
response = chat_engine.chat("How many models does it support?")
print(response)Penggunaan Asinkron
import asyncio
async def main():
response = await llm.acomplete("Hello!")
print(response.text)
asyncio.run(main())