// CONCEPT

LLM Models

http4k AI provides a unified, type-safe and functional API for interacting with various Large Language Models (LLMs) and AI services. Rather than dealing with vendor-specific APIs, you get consistent functional interfaces that work across different providers. As with all things http4k, the focus is on simplicity, composability, and functional programming principles, rather than complex abstractions and magic.

Core Interfaces#

Chat#

fun interface Chat {
    operator fun invoke(request: ChatRequest): LLMResult<ChatResponse>
}

Standard synchronous chat completions for request-response interactions.

StreamingChat#

fun interface StreamingChat {
    operator fun invoke(request: ChatRequest): LLMResult<Sequence<ChatResponse>>
}

Real-time streaming responses for applications requiring incremental text generation.

ImageGeneration#

fun interface ImageGeneration {
    operator fun invoke(request: ImageRequest): LLMResult<ImageResponse>
}

AI-powered image generation from text prompts.

Decide#

fun interface Decide {
    operator fun invoke(request: DecideRequest): LLMResult<DecideResponse>
}

Structured decisions evaluated against a piece of state, rather than free text generation. A DecideRequest carries the state (text, or a list of Content such as images), a map of named Questions and the model to use. Each question is answered with a matching Answer:

QuestionAsks forAnswer
Choiceone of a set of named optionsthe choice, per-option probabilities, confidence
Scorea position on an ordered list of Levelsthe score, per-level probabilities, confidence
YesNothe truth of a single statementa probability

A provider may also answer any question with Answer.Refusal. Implementations are available for OpenAI and TypeSafe.

Result-Based Error Handling#

All operations return LLMResult<T> - a Result4k type that forces explicit error handling:

typealias LLMResult<T> = Result4k<T, LLMError>

LLM Error Types:

  • Http - HTTP-level failures
  • Timeout - Request timeouts
  • NotFound - Missing resources
  • Internal - System errors
  • Custom - Provider-specific errors
scarf