class LittleGhost::Providers::OpenAICompatible

OpenAICompatible brings OpenAI-style Responses or Chat Completions endpoints into LittleGhost. Agents receive the same streaming events whether the endpoint is OpenAI, a hosted model service, or an application gateway.

provider = LittleGhost::Providers::OpenAICompatible.new(
  api_key: ENV.fetch("MODEL_API_KEY"),
  model: "example-model",
  base_url: "https://models.example.test/v1/"
)

The client translates ModelRequest values to the selected wire API and translates responses back to StreamEvent objects.

Retries and streaming output

Transient HTTP and stream failures retry with limited exponential backoff. A :model_retry event reports each retry and whether text had already been emitted. Partial text may repeat after a retry, so consumers that assemble streams must use that event to discard or replace superseded output.

Constants

DEFAULT_BASE_URL: The OpenAI API endpoint used when base_url is omitted.

Attributes

api [R]

Provider model identifier and selected OpenAI-compatible wire API.

model [R]

Provider model identifier and selected OpenAI-compatible wire API.

Public Class Methods

new (api_key:, model:, base_url: DEFAULT_BASE_URL, api: :responses, headers: {}, open_timeout: 10, read_timeout: 120, allow_insecure_http: false, max_response_bytes: HTTPTransport::DEFAULT_MAX_RESPONSE_BYTES, max_retries: 2, max_retry_delay: MAX_RETRY_DELAY, transport: nil, sleeper: nil, on_retry: ->(*) {})

Source

# File lib/little_ghost/providers/openai_compatible.rb, line 83
def initialize(
  api_key:,
  model:,
  base_url: DEFAULT_BASE_URL,
  api: :responses,
  headers: {},
  open_timeout: 10,
  read_timeout: 120,
  allow_insecure_http: false,
  max_response_bytes: HTTPTransport::DEFAULT_MAX_RESPONSE_BYTES,
  max_retries: 2,
  max_retry_delay: MAX_RETRY_DELAY,
  transport: nil,
  sleeper: nil,
  on_retry: ->(*) {}
)
  @api_key = api_key
  @model = model
  @api = api.to_sym
  raise ConfigurationError, "api must be :responses or :chat_completions" unless %i[responses chat_completions].include?(@api)

  @headers = headers.transform_keys(&:to_s).freeze
  @max_retries = Integer(max_retries)
  @max_retry_delay = Integer(max_retry_delay)
  @transport = transport || HTTPTransport.new(
    base_url:,
    open_timeout:,
    read_timeout:,
    allow_insecure_http:,
    max_response_bytes:
  )
  @sleeper = sleeper
  @on_retry = on_retry
end

Configures an OpenAI-compatible client.

api is :responses or :chat_completions. headers adds trusted endpoint-specific headers. max_retries controls retries before the original error is raised, and on_retry receives the attempt, error, and delay. Pass a custom transport for alternate HTTP execution.

Public Instance Methods

capabilities (metadata: {})

Source

# File lib/little_ghost/providers/openai_compatible.rb, line 160
def capabilities(metadata: {})
  ModelCapabilities.legacy
end

Returns the legacy capability contract expected from compatible APIs. Subclasses can override this when the endpoint advertises precise support.

stream (request) { |event| ... }

Source

# File lib/little_ghost/providers/openai_compatible.rb, line 123
def stream(request)
  return enum_for(__method__, request) unless block_given?

  attempts = 0

  begin
    partial_text = false
    request.cancellation_token.raise_if_cancelled!
    stream_once(request) do |event|
      partial_text ||= event.type == :text_delta && !event.data[:text].to_s.empty?
      yield event
    end
  rescue HTTPError, StreamError => error
    if context_window_overflow?(error)
      raise ContextWindowOverflowError, "The model context window was exceeded"
    end
    raise if !error.retryable? || attempts >= @max_retries

    attempts += 1
    request.cancellation_token.raise_if_cancelled!
    delay = capped_retry_delay(request, retry_delay(attempts))
    @on_retry.call(attempts, error, delay)
    yield StreamEvent.build(
      :model_retry,
      attempt: attempts,
      delay:,
      error_class: error.class.name,
      partial_text:,
      **retry_error_metadata(error)
    )
    wait_before_retry(request, delay)
    retry
  end
end

Streams LittleGhost StreamEvent objects for request.

Without a block, returns an Enumerator. Context-window errors normalize to ContextWindowOverflowError, and malformed tool calls normalize to MalformedToolCallError.