⚡ LLM Inference APIs - Latency

Streaming Off-Gemma 7B

Chat (Deep Infra - gemma-7b-it)Chat (Fireworks AI - gemma-7b-it)Chat (Groq - gemma-7b-it) 🏆Chat (NVIDIA AI - gemma-7b)

Streaming Off-Mixtral 8x7B

Chat (Anyscale - mixtral-8x7b-instruct-v0.1)Chat (Deep Infra - mixtral-8x7b-instruct-v0.1)Chat (Fireworks AI - mixtral-8x7b-instruct)Chat (Groq - mixtral-8x7b-32768) 🏆Chat (Lepton AI - mixtral-8x7b)Chat (Mistral AI - open-mixtral-8x7b)Chat (OctoAI - mixtral-8x7b-instruct)Chat (Perplexity AI - mixtral-8x7b-instruct)Chat (Together AI - mixtral-8x7b-instruct-v0.1)

Streaming Off-Llama 2 70B

Chat (Anyscale - llama-2-70b-chat-hf)Chat (Deep Infra - llama-2-70b-chat-hf)Chat (Fireworks AI - llama-v2-70b-chat) 🏆Chat (Groq - llama2-70b-4096)Chat (Lepton AI - llama2-70b-4096)Chat (NVIDIA AI - llama2-70b)Chat (OctoAI - llama-2-70b-chat)

Streaming Off-Llama 3 70B

Chat (Anyscale - llama-3-70b-chat-hf)Chat (Deep Infra - meta-llama/Meta-Llama-3-70B-Instruct)Chat (Fireworks AI - llama-v3-70b-instruct)Chat (Groq - llama3-70b-8192)Chat (Lepton AI - llama3-70b)Chat (NVIDIA AI - llama3-70b)Chat (OctoAI - meta-llama-3-70b-instruct)

Streaming Off-Llama 3.1 405B

Chat (Fireworks AI - llama-v3p1-405b-instruct)Chat (NVIDIA AI - llama-3.1-405b-instruct)Chat (OctoAI - meta-llama-3.1-405b-instruct)Chat (Together AI - Meta-Llama-3.1-405B-Instruct-Turbo)

Streaming On-Gemma 7B

Chat (Deep Infra - gemma-7b-it)Chat (Fireworks AI - gemma-7b-it)Chat (Groq - gemma-7b-it) 🏆Chat (NVIDIA AI - gemma-7b)Chat (OctoAI - gemma-7b-it)

Streaming On-Mixtral 8x7B

Chat (Anyscale - mixtral-8x7b-instruct-v0.1)Chat (Deep Infra - mixtral-8x7b-instruct-v0.1)Chat (Fireworks AI - mixtral-8x7b-instruct)Chat (Groq - mixtral-8x7b-32768) 🏆Chat (Lepton AI - mixtral-8x7b)Chat (Mistral AI - open-mixtral-8x7b)Chat (OctoAI - mixtral-8x7b-instruct)Chat (Perplexity AI - mixtral-8x7b-instruct)Chat (Together AI - mixtral-8x7b-instruct-v0.1)

Streaming On-Llama 2 70B

Chat (Anyscale - llama-2-70b-chat-hf)Chat (Deep Infra - llama-2-70b-chat-hf)Chat (Fireworks AI - llama-v2-70b-chat) 🏆Chat (Groq - llama2-70b-4096)Chat (Lepton AI - llama2-70b-4096)Chat (NVIDIA AI - llama2-70b)Chat (OctoAI - llama-2-70b-chat)

Streaming On-Llama 3 70B

Chat (Anyscale - llama-3-70b-chat-hf)Chat (Deep Infra - meta-llama/Meta-Llama-3-70B-Instruct)Chat (Fireworks AI - llama-v3-70b-instruct)Chat (Groq - llama3-70b-8192)Chat (Lepton AI - llama3-70b)Chat (NVIDIA AI - llama3-70b)Chat (OctoAI - meta-llama-3-70b-instruct)

Streaming On-Llama 3.1 405B

Chat (Fireworks AI - llama-v3p1-405b-instruct)Chat (NVIDIA AI - llama-3.1-405b-instruct)Chat (OctoAI - meta-llama-3.1-405b-instruct)Chat (Together AI - Meta-Llama-3.1-405B-Instruct-Turbo)

📌 AI21 Labs API

🚀 Get Started (Inference)

Answer (Single Document)Answer (RAG Engine)Chat

🚀 Get Started (Fine-Tuning)

Get Datasets List Custom Model Training Get Custom Models List Custom Model Complete

Answer (Single Document)Answer (RAG Engine)

j2-ultra jamba-instruct-preview

Complete (Instructional Prompt) - j2-ultra Complete (Instructional Prompt) - j2-mid Complete (Instructional Prompt) - j2-light

Custom Model Training Get Custom Models Get Custom Model Custom Model Inference Update Default Epoch

Get Datasets Get Dataset Delete Dataset

Grammatical Error Correction

Grammatical Error Correction

Summarize Summarize by Segment Summarize Conversation

📌 Amazon Bedrock API

🚀 Get Started

stability.stable-diffusion-xl-v1

amazon.titan-image-generator-v1

anthropic.claude-instant-v1

cohere.embed-multilingual-v3

meta.llama3-1-405b-instruct-v1:0

mistral.mixtral-8x7b-instruct-v0:1

stability.stable-diffusion-xl-v1

Chat (amazon.titan-tg1-large)

Use Cases-Embedding

Embedding (amazon.titan-embed-text-v1)

Use Cases-Image Generation

Image Generation (stability.stable-diffusion-xl)

Use Cases-Sentiment Analysis

Sentiment Analysis (amazon.titan-tg1-large)

Use Cases-SQL Generation

SQL Generation (amazon.titan-tg1-large)

Use Cases-Summarization

Summarization (amazon.titan-tg1-large)

Use Cases-Text Generation

Text Generation (amazon.titan-tg1-large)

Use Cases-Translation

Translation (amazon.titan-tg1-large)

📌 Anthropic API

🚀 Get Started

Messages (claude-3-5-sonnet-20240620)

claude-3-haiku-20240307

Complete (Legacy)

claude-instant-1.2

📌 Anyscale API

🚀 Get Started (Inference)

Chat (meta-llama/Llama-2-70b-chat-hf)Chat (meta-llama/Llama-3-70b-chat-hf)

🚀 Get Started (Fine-Tuning)-llama-2-13b-chat-hf

Create fine-tuning job Get fine-tuning job Chat (fine-tuned model)

🚀 Get Started (Fine-Tuning)-llama-2-70b-chat-hf

Create fine-tuning job Get fine-tuning job Chat (fine-tuned model)

🚀 Get Started (Fine-Tuning)-mistral-7B-Instruct-v0.1

Create fine-tuning job Get fine-tuning job Chat (fine-tuned model)

🚀 Get Started (Fine-Tuning)-mixtral-8x7B-Instruct-v0.1

Create fine-tuning job Get fine-tuning job Chat (fine-tuned model)

🚀 Get Started (Fine-Tuning)

Upload file (validation)List files

mistralai/Mixtral-8x22B-Instruct-v0.1 meta-llama/Llama-3-70b-chat-hf

Upload file Get files Get file Get file contents Delete file

Create fine-tuning job Get fine-tuning jobs Get fine-tuning job Cancel fine-tuning job

📌 Cohere API

🚀 Get Started

Chat (command)Summarize (command)Embed (embed-english-v3.0)Classify (embed-english-v3.0)Rerank (rerank-english-v2.0)

Summarize (command)

embed-multilingual-v2.0

Classify (embed)

embed-multilingual-v2.0

Rerank (rerank)

rerank-multilingual-v2.0

Create Connector Get Connectors Get Connector Update Connector Delete Connector Authorize Connector

Create Dataset Get Datasets Get Datasets Usage Get Dataset Delete Dataset

Create Embed Job Get Embed Jobs Get Embed Job Cancel Embed Job

Create fine-tuned model Get fine-tuned models Get fine-tuned model Update fine-tuned model Delete fine-tuned model Retrieves the chronology of statuses the fine-tuned model has been through Retrieves metrics measured during the training of a fine-tuned model

📌 Deep Infra API

🚀 Get Started

Chat (google/gemma-7b-it)

Phind/Phind-CodeLlama-34B-v2

google/gemma-7b-it

thenlper/gte-large

EleutherAI/gpt-j-6B EleutherAI/gpt-neo-1.3B EleutherAI/gpt-neo-125M EleutherAI/pythia-12b EleutherAI/pythia-2.8b Salesforce/codegen-16B-mono bigcode/starcoder gpt2 meta-llama/Meta-Llama-3-8B-Instruct meta-llama/Meta-Llama-3-70B-Instruct

📌 Deepgram API

🚀 Get Started

Speech-to-Text (remote file)

📌 Fireworks AI API

🚀 Get Started

Chat (llama-v3p1-405b-instruct)

mixtral-8x22b-instruct-preview

chinese-llama-2-lora-7b

nomic-ai/nomic-embed-text-v1.5

Image Generation

stable-diffusion-xl-1024-v1-0 SSD-1B playground-v2-1024px-aesthetic japanese-stable-diffusion-xl

📌 Google Vertex AI API

🚀 Get Started

imagegeneration texttospeech speechtotext

gemini-1.5-pro (Function)

gemini-1.5-flash

gemini-1.5-flash (Multimodal)

gemini-1.0-pro (Function)

gemini-1.0-pro-vision

gemini-1.0-pro-vision (Multimodal)

bison-text-bison

bison-chat-bison

bison-code-bison

bison-codechat-bison

gecko-code-gecko

gecko-textembedding-gecko

textembedding-gecko

multimodalembedding

multimodalembedding

imagegeneration

imagegeneration

🚀 Get Started

Chat (mixtral-8x7b-32768)

mixtral-8x7b-32768

📌 Hugging Face API

🚀 Get Started

Text Generation (google/gemma-7b-it)

Text Generation

mistralai/Mixtral-8x7B-Instruct-v0.1

facebook/fastspeech2-en-ljspeech

openai/whisper-large-v3

stabilityai/stable-diffusion-xl-base-1.0

Salesforce/blip-image-captioning-large

facebook/musicgen-stereo-large

Object Detection

facebook/detr-resnet-50

facebook/bart-large-cnn

Table Question Answering

microsoft/tapex-large-finetuned-wtq

📌 IBM watsonx.ai API

🚀 Get Started

identity/token Question Answering (llama-3-405b-instruct)

IBM / Mistral AI - merlinite-7b

Sentiment Classification (google/flan-ul2)

Code Generation (ibm/granite-34b-code-instruct)

Embeddings (ibm/slate-125m-english-rtrvr)

Fact Extraction (google/flan-ul2)

Thank You Note Generation (google/flan-ul2)

Question Answering

Finance Q&A (ibm/granite-13b-instruct-v2)

Earnings Call Summary (google/flan-ul2)

English to Spanish (ibm/granite-20b-multilingual)

📌 Lamini API

🚀 Get Started

Completions (meta-llama/Llama-2-7b-chat-hf)

microsoft/phi-2

sentence-transformers/all-MiniLM-L6-v2

Create Train Job Get Train Jobs Get Train Job Status Get Train Job Eval Cancel Train Job

📌 Lepton AI API

🚀 Get Started

Chat (llama2-7b)

llama2-7b llama2-13b llama2-70b llama3-70b mixtral-8x7b

Image Generation

📌 Mistral AI API

🚀 Get Started

Chat (open-mixtral-8x22b)

open-mistral-nemo-2407

📌 NVIDIA AI API

🚀 Get Started

Chat (nvidia/nemotron-4-340b-instruct)

snowflake/arctic

Image Generation

stabilityai/sdxl-turbo

Video Generation

stabilityai/stable-video-diffusion

📌 OctoAI API

🚀 Get Started

Chat (meta-llama-3.1-405b-instruct)

Image Generation

📌 Ollama API (localhost)

Generate - qwen2

📌 OpenAI API

🚀 Get Started (Inference)

Chat (gpt-3.5-turbo)

🚀 Get Started (Fine-Tuning)

Upload file (validation)Get files Create fine-tuning job (gpt-3.5-turbo-0125)Get fine-tuning job Chat (fine-tuned model)

Assistants (asynchronous stateful)

Create assistant List assistants Get assistant Modify assistant Delete assistant Create thread Get thread Modify thread Delete thread Create message List messages Get message Delete message Create run List runs Get run Modify run Submit tool output to run List run steps Get run step

Chat (synchronous stateless)

Completions (legacy)

gpt-3.5-turbo-instruct

tts-1-hd (voice: shimmer)

Audio-Transcription

Audio-Translation

Create batch Get batches Get batch Cancel batch

text-embedding-3-small

Create fine-tuning job Get fine-tuning jobs Get fine-tuning job Get fine-tuning job events Get fine-tuning job checkpoints Cancel fine-tuning job

Upload file Get files Get file Get file content Delete file

Images-Image Generation

Images-Image Edit

Images-Image Variation

Get models Get model

Create vector store List vector stores Get vector store Modify vector store Delete vector store Create vector store file List vector store files Get vector store file Delete vector store file Create vector store file batch Get vector store file batch List vector store file batch files

📌 Perplexity AI API

🚀 Get Started

Chat (llama-3-70b-instruct)

sonar-medium-online

📌 Replicate API

meta/meta-llama-3-8b-instruct mistralai/mixtral-8x7b-instruct-v0.1 meta/meta-llama-3-70b-instruct meta/meta-llama-3.1-405b-instruct mistralai/mixtral-8x7b-instruct-v0.1

📌 Stability AI API

🚀 Get Started

Image Generation (sd3)Image Generation (core)Image Generation (stable-diffusion-xl-1024-v1-0)

v1-generation-{engine_id}-text-to-image

stable-diffusion-xl-1024-v1-0

v1-generation-{engine_id}-image-to-image

stable-diffusion-xl-1024-v1-0

v1-engines-list

v2alpha-image-to-video-result-{id}

image-to-video/result

v2alpha-image-to-video

v2beta-image-to-video-result-{id}

image-to-video/result

v2beta-image-to-video

v2beta-stable-image-generate-core

Stable Image Core

v2beta-stable-image-generate-sd3

Stable Diffusion 3.0

📌 Together AI API

🚀 Get Started

Chat (meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo)

WizardLM/WizardLM-13B-V1.2

WhereIsAI/UAE-Large-V1

⚡ LLM Inference APIs - Latency

Streaming Off-Gemma 7B

Chat (Deep Infra - gemma-7b-it)Chat (Fireworks AI - gemma-7b-it)Chat (Groq - gemma-7b-it) 🏆Chat (NVIDIA AI - gemma-7b)

Streaming Off-Mixtral 8x7B

Chat (Anyscale - mixtral-8x7b-instruct-v0.1)Chat (Deep Infra - mixtral-8x7b-instruct-v0.1)Chat (Fireworks AI - mixtral-8x7b-instruct)Chat (Groq - mixtral-8x7b-32768) 🏆Chat (Lepton AI - mixtral-8x7b)Chat (Mistral AI - open-mixtral-8x7b)Chat (OctoAI - mixtral-8x7b-instruct)Chat (Perplexity AI - mixtral-8x7b-instruct)Chat (Together AI - mixtral-8x7b-instruct-v0.1)

Streaming Off-Llama 2 70B

Chat (Anyscale - llama-2-70b-chat-hf)Chat (Deep Infra - llama-2-70b-chat-hf)Chat (Fireworks AI - llama-v2-70b-chat) 🏆Chat (Groq - llama2-70b-4096)Chat (Lepton AI - llama2-70b-4096)Chat (NVIDIA AI - llama2-70b)Chat (OctoAI - llama-2-70b-chat)

Streaming Off-Llama 3 70B

Chat (Anyscale - llama-3-70b-chat-hf)Chat (Deep Infra - meta-llama/Meta-Llama-3-70B-Instruct)Chat (Fireworks AI - llama-v3-70b-instruct)Chat (Groq - llama3-70b-8192)Chat (Lepton AI - llama3-70b)Chat (NVIDIA AI - llama3-70b)Chat (OctoAI - meta-llama-3-70b-instruct)

Streaming Off-Llama 3.1 405B

Chat (Fireworks AI - llama-v3p1-405b-instruct)Chat (NVIDIA AI - llama-3.1-405b-instruct)Chat (OctoAI - meta-llama-3.1-405b-instruct)Chat (Together AI - Meta-Llama-3.1-405B-Instruct-Turbo)

Streaming On-Gemma 7B

Chat (Deep Infra - gemma-7b-it)Chat (Fireworks AI - gemma-7b-it)Chat (Groq - gemma-7b-it) 🏆Chat (NVIDIA AI - gemma-7b)Chat (OctoAI - gemma-7b-it)

Streaming On-Mixtral 8x7B

Chat (Anyscale - mixtral-8x7b-instruct-v0.1)Chat (Deep Infra - mixtral-8x7b-instruct-v0.1)Chat (Fireworks AI - mixtral-8x7b-instruct)Chat (Groq - mixtral-8x7b-32768) 🏆Chat (Lepton AI - mixtral-8x7b)Chat (Mistral AI - open-mixtral-8x7b)Chat (OctoAI - mixtral-8x7b-instruct)Chat (Perplexity AI - mixtral-8x7b-instruct)Chat (Together AI - mixtral-8x7b-instruct-v0.1)

Streaming On-Llama 2 70B

Chat (Anyscale - llama-2-70b-chat-hf)Chat (Deep Infra - llama-2-70b-chat-hf)Chat (Fireworks AI - llama-v2-70b-chat) 🏆Chat (Groq - llama2-70b-4096)Chat (Lepton AI - llama2-70b-4096)Chat (NVIDIA AI - llama2-70b)Chat (OctoAI - llama-2-70b-chat)

Streaming On-Llama 3 70B

Chat (Anyscale - llama-3-70b-chat-hf)Chat (Deep Infra - meta-llama/Meta-Llama-3-70B-Instruct)Chat (Fireworks AI - llama-v3-70b-instruct)Chat (Groq - llama3-70b-8192)Chat (Lepton AI - llama3-70b)Chat (NVIDIA AI - llama3-70b)Chat (OctoAI - meta-llama-3-70b-instruct)

Streaming On-Llama 3.1 405B

Chat (Fireworks AI - llama-v3p1-405b-instruct)Chat (NVIDIA AI - llama-3.1-405b-instruct)Chat (OctoAI - meta-llama-3.1-405b-instruct)Chat (Together AI - Meta-Llama-3.1-405B-Instruct-Turbo)

📌 AI21 Labs API

🚀 Get Started (Inference)

Answer (Single Document)Answer (RAG Engine)Chat

🚀 Get Started (Fine-Tuning)

Get Datasets List Custom Model Training Get Custom Models List Custom Model Complete

Answer (Single Document)Answer (RAG Engine)

j2-ultra jamba-instruct-preview

Complete (Instructional Prompt) - j2-ultra Complete (Instructional Prompt) - j2-mid Complete (Instructional Prompt) - j2-light

Custom Model Training Get Custom Models Get Custom Model Custom Model Inference Update Default Epoch

Get Datasets Get Dataset Delete Dataset

Grammatical Error Correction

Grammatical Error Correction

Summarize Summarize by Segment Summarize Conversation

📌 Amazon Bedrock API

🚀 Get Started

stability.stable-diffusion-xl-v1

amazon.titan-image-generator-v1

anthropic.claude-instant-v1

cohere.embed-multilingual-v3

meta.llama3-1-405b-instruct-v1:0

mistral.mixtral-8x7b-instruct-v0:1

stability.stable-diffusion-xl-v1

Chat (amazon.titan-tg1-large)

Use Cases-Embedding

Embedding (amazon.titan-embed-text-v1)

Use Cases-Image Generation

Image Generation (stability.stable-diffusion-xl)

Use Cases-Sentiment Analysis

Sentiment Analysis (amazon.titan-tg1-large)

Use Cases-SQL Generation

SQL Generation (amazon.titan-tg1-large)

Use Cases-Summarization

Summarization (amazon.titan-tg1-large)

Use Cases-Text Generation

Text Generation (amazon.titan-tg1-large)

Use Cases-Translation

Translation (amazon.titan-tg1-large)

📌 Anthropic API

🚀 Get Started

Messages (claude-3-5-sonnet-20240620)

claude-3-haiku-20240307

Complete (Legacy)

claude-instant-1.2

📌 Anyscale API

🚀 Get Started (Inference)

Chat (meta-llama/Llama-2-70b-chat-hf)Chat (meta-llama/Llama-3-70b-chat-hf)

🚀 Get Started (Fine-Tuning)-llama-2-13b-chat-hf

Create fine-tuning job Get fine-tuning job Chat (fine-tuned model)

🚀 Get Started (Fine-Tuning)-llama-2-70b-chat-hf

Create fine-tuning job Get fine-tuning job Chat (fine-tuned model)

🚀 Get Started (Fine-Tuning)-mistral-7B-Instruct-v0.1

Create fine-tuning job Get fine-tuning job Chat (fine-tuned model)

🚀 Get Started (Fine-Tuning)-mixtral-8x7B-Instruct-v0.1

Create fine-tuning job Get fine-tuning job Chat (fine-tuned model)

🚀 Get Started (Fine-Tuning)

Upload file (validation)List files

mistralai/Mixtral-8x22B-Instruct-v0.1 meta-llama/Llama-3-70b-chat-hf

Upload file Get files Get file Get file contents Delete file

Create fine-tuning job Get fine-tuning jobs Get fine-tuning job Cancel fine-tuning job

📌 Cohere API

🚀 Get Started

Chat (command)Summarize (command)Embed (embed-english-v3.0)Classify (embed-english-v3.0)Rerank (rerank-english-v2.0)

Summarize (command)

embed-multilingual-v2.0

Classify (embed)

embed-multilingual-v2.0

Rerank (rerank)

rerank-multilingual-v2.0

Create Connector Get Connectors Get Connector Update Connector Delete Connector Authorize Connector

Create Dataset Get Datasets Get Datasets Usage Get Dataset Delete Dataset

Create Embed Job Get Embed Jobs Get Embed Job Cancel Embed Job

Create fine-tuned model Get fine-tuned models Get fine-tuned model Update fine-tuned model Delete fine-tuned model Retrieves the chronology of statuses the fine-tuned model has been through Retrieves metrics measured during the training of a fine-tuned model

📌 Deep Infra API

🚀 Get Started

Chat (google/gemma-7b-it)

Phind/Phind-CodeLlama-34B-v2

google/gemma-7b-it

thenlper/gte-large

EleutherAI/gpt-j-6B EleutherAI/gpt-neo-1.3B EleutherAI/gpt-neo-125M EleutherAI/pythia-12b EleutherAI/pythia-2.8b Salesforce/codegen-16B-mono bigcode/starcoder gpt2 meta-llama/Meta-Llama-3-8B-Instruct meta-llama/Meta-Llama-3-70B-Instruct

📌 Deepgram API

🚀 Get Started

Speech-to-Text (remote file)

📌 Fireworks AI API

🚀 Get Started

Chat (llama-v3p1-405b-instruct)

mixtral-8x22b-instruct-preview

chinese-llama-2-lora-7b

nomic-ai/nomic-embed-text-v1.5

Image Generation

stable-diffusion-xl-1024-v1-0 SSD-1B playground-v2-1024px-aesthetic japanese-stable-diffusion-xl

📌 Google Vertex AI API

🚀 Get Started

imagegeneration texttospeech speechtotext

gemini-1.5-pro (Function)

gemini-1.5-flash

gemini-1.5-flash (Multimodal)

gemini-1.0-pro (Function)

gemini-1.0-pro-vision

gemini-1.0-pro-vision (Multimodal)

bison-text-bison

bison-chat-bison

bison-code-bison

bison-codechat-bison

gecko-code-gecko

gecko-textembedding-gecko

textembedding-gecko

multimodalembedding

multimodalembedding

imagegeneration

imagegeneration

🚀 Get Started

Chat (mixtral-8x7b-32768)

mixtral-8x7b-32768

📌 Hugging Face API

🚀 Get Started

Text Generation (google/gemma-7b-it)

Text Generation

mistralai/Mixtral-8x7B-Instruct-v0.1

facebook/fastspeech2-en-ljspeech

openai/whisper-large-v3

stabilityai/stable-diffusion-xl-base-1.0

Salesforce/blip-image-captioning-large

facebook/musicgen-stereo-large

Object Detection

facebook/detr-resnet-50

facebook/bart-large-cnn

Table Question Answering

microsoft/tapex-large-finetuned-wtq

📌 IBM watsonx.ai API

🚀 Get Started

identity/token Question Answering (llama-3-405b-instruct)

IBM / Mistral AI - merlinite-7b

Sentiment Classification (google/flan-ul2)

Code Generation (ibm/granite-34b-code-instruct)

Embeddings (ibm/slate-125m-english-rtrvr)

Fact Extraction (google/flan-ul2)

Thank You Note Generation (google/flan-ul2)

Question Answering

Finance Q&A (ibm/granite-13b-instruct-v2)

Earnings Call Summary (google/flan-ul2)

English to Spanish (ibm/granite-20b-multilingual)

📌 Lamini API

🚀 Get Started

Completions (meta-llama/Llama-2-7b-chat-hf)

microsoft/phi-2

sentence-transformers/all-MiniLM-L6-v2

Create Train Job Get Train Jobs Get Train Job Status Get Train Job Eval Cancel Train Job

📌 Lepton AI API

🚀 Get Started

Chat (llama2-7b)

llama2-7b llama2-13b llama2-70b llama3-70b mixtral-8x7b

Image Generation

📌 Mistral AI API

🚀 Get Started

Chat (open-mixtral-8x22b)

open-mistral-nemo-2407

📌 NVIDIA AI API

🚀 Get Started

Chat (nvidia/nemotron-4-340b-instruct)

snowflake/arctic

Image Generation

stabilityai/sdxl-turbo

Video Generation

stabilityai/stable-video-diffusion

📌 OctoAI API

🚀 Get Started

Chat (meta-llama-3.1-405b-instruct)

Image Generation

📌 Ollama API (localhost)

Generate - qwen2

📌 OpenAI API

🚀 Get Started (Inference)

Chat (gpt-3.5-turbo)

🚀 Get Started (Fine-Tuning)

Upload file (validation)Get files Create fine-tuning job (gpt-3.5-turbo-0125)Get fine-tuning job Chat (fine-tuned model)

Assistants (asynchronous stateful)

Create assistant List assistants Get assistant Modify assistant Delete assistant Create thread Get thread Modify thread Delete thread Create message List messages Get message Delete message Create run List runs Get run Modify run Submit tool output to run List run steps Get run step

Chat (synchronous stateless)

Completions (legacy)

gpt-3.5-turbo-instruct

tts-1-hd (voice: shimmer)

Audio-Transcription

Audio-Translation

Create batch Get batches Get batch Cancel batch

text-embedding-3-small

Create fine-tuning job Get fine-tuning jobs Get fine-tuning job Get fine-tuning job events Get fine-tuning job checkpoints Cancel fine-tuning job

Upload file Get files Get file Get file content Delete file

Images-Image Generation

Images-Image Edit

Images-Image Variation

Get models Get model

Create vector store List vector stores Get vector store Modify vector store Delete vector store Create vector store file List vector store files Get vector store file Delete vector store file Create vector store file batch Get vector store file batch List vector store file batch files

📌 Perplexity AI API

🚀 Get Started

Chat (llama-3-70b-instruct)

sonar-medium-online

📌 Replicate API

meta/meta-llama-3-8b-instruct mistralai/mixtral-8x7b-instruct-v0.1 meta/meta-llama-3-70b-instruct meta/meta-llama-3.1-405b-instruct mistralai/mixtral-8x7b-instruct-v0.1

📌 Stability AI API

🚀 Get Started

Image Generation (sd3)Image Generation (core)Image Generation (stable-diffusion-xl-1024-v1-0)

v1-generation-{engine_id}-text-to-image

stable-diffusion-xl-1024-v1-0

v1-generation-{engine_id}-image-to-image

stable-diffusion-xl-1024-v1-0

v1-engines-list

v2alpha-image-to-video-result-{id}

image-to-video/result

v2alpha-image-to-video

v2beta-image-to-video-result-{id}

image-to-video/result

v2beta-image-to-video

v2beta-stable-image-generate-core

Stable Image Core

v2beta-stable-image-generate-sd3

Stable Diffusion 3.0

📌 Together AI API

🚀 Get Started

Chat (meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo)

WizardLM/WizardLM-13B-V1.2

WhereIsAI/UAE-Large-V1

Home 📌 Anyscale APIChat

Chat

Number of APIs: 2

meta-llama/Llama-3-70b-chat-hf POST https://api.endpoints.anyscale.com/v1/chat/completions
mistralai/Mixtral-8x22B-Instruct-v0.1 POST {{baseUrl}}/chat/completions

mistralai/Mixtral-8x22B-Instruct-v0.1