In this episode, we look at running a self hosted Large Language Model (LLM) and consuming it with a Rails application. We will use a background to make API requests to the LLM and then stream the responses in real-time to the browser.

Drifting Ruby Screencasts

Dave Kimura

Streaming LLM Responses

MAR 3, 202424 MIN
Drifting Ruby Screencasts

Streaming LLM Responses

MAR 3, 202424 MIN

Description

In this episode, we look at running a self hosted Large Language Model (LLM) and consuming it with a Rails application. We will use a background to make API requests to the LLM and then stream the responses in real-time to the browser.