OpenRouter
OpenRouter
OpenRouter를 LlamaIndex에서 사용해 다양한 LLM에 표준화된 API로 접근하는 방법을 정리한 문서예요. OpenRouter는 최적의 가격으로 여러 LLM에 접근할 수 있는 표준화 API를 제공하므로, 하나의 인터페이스로 다양한 모델을 비교하며 쓸 수 있어요.
출처: 문서
본문
OpenRouter는 최적의 가격으로 여러 LLM에 접근할 수 있는 표준화된 API를 제공해요. 자세한 내용은 홈페이지에서 확인할 수 있어요.
Colab에서 이 노트북을 여는 경우 LlamaIndex 🦙 설치가 필요할 거예요.
%pip install llama-index-llms-openrouter
!pip install llama-index
from llama_index.llms.openrouter import OpenRouter
from llama_index.core.llms import ChatMessage
chat을 ChatMessage 목록으로 호출하기
환경 변수 OPENROUTER_API_KEY를 설정하거나 클래스 생성자에서 api_key를 설정해야 해요.
# import os
# os.environ['OPENROUTER_API_KEY'] = '<your-api-key>'
llm = OpenRouter(
api_key="<your-api-key>",
max_tokens=256,
context_window=4096,
model="gryphe/mythomax-l2-13b",
)
message = ChatMessage(role="user", content="Tell me a joke")
resp = llm.chat([message])
print(resp)
assistant: Why did the tomato turn red? Because it saw the salad dressing!
Streaming (스트리밍)
message = ChatMessage(role="user", content="Tell me a story in 250 words")
resp = llm.stream_chat([message])
for r in resp:
print(r.delta, end="")
Once upon a time, there was a young girl named Maria who lived in a small village surrounded by lush green forests. Maria was a kind and gentle soul, loved by everyone in the village. She spent most of her days exploring the forests, discovering new species of plants and animals, and helping the villagers with their daily chores.
One day, while Maria was out on a walk, she stumbled upon a hidden path she had never seen before. The path was overgrown with weeds and vines, but something about it called to her. She decided to follow it, and it led her deeper and deeper into the forest.
As she walked, the trees grew taller and the air grew colder. Maria began to feel a sense of unease, but she was determined to see where the path led. Finally, she came to a clearing, and in the center of it stood an enormous tree, its trunk as wide as a house.
Maria approached the tree and saw that it was covered in strange symbols. She reached out to touch one of the symbols, and suddenly, the tree began to glow. The glow grew brighter and brighter, until Maria
프롬프트로 complete 호출하기
resp = llm.complete("Tell me a joke")
print(resp)
Sure, here's a joke for you:
Why couldn't the bicycle stand up by itself?
Because it was two-tired!
I hope that brought a smile to your face!
resp = llm.stream_complete("Tell me a story in 250 words")
for r in resp:
print(r.delta, end="")
Once upon a time, there was a young girl named Maria. She lived in a small village surrounded by lush green forests and sparkling rivers. Maria was a kind and gentle soul, loved by everyone in the village. She spent her days helping her parents with their farm work and exploring the surrounding nature.
One day, while wandering in the forest, Maria stumbled upon a hidden path she had never seen before. She decided to follow it, and it led her to a beautiful meadow filled with wildflowers. In the center of the meadow, she found a small pond, where she saw her own reflection in the water.
As she gazed into the pond, Maria saw a figure approaching her. It was a wise old woman, who introduced herself as the guardian of the meadow. The old woman told Maria that she had been chosen to receive a special gift, one that would bring her great joy and happiness.
The old woman then presented Maria with a small, delicate flower. She told her that this flower had the power to heal any wound, both physical and emotional. Maria was amazed and grateful, and she promised to use the flower wisely.
Model Configuration (모델 설정)
# View options at https://openrouter.ai/models
# This example uses Mistral's MoE, Mixtral:
llm = OpenRouter(model="mistralai/mixtral-8x7b-instruct")
resp = llm.complete("Write a story about a dragon who can code in Rust")
print(resp)