Laravel OpenAI Streaming (SSE) returns broken words instead of full sentences.

Laravel OpenAI Streaming (SSE) returns broken words instead of full sentences.

To address the issue of Laravel OpenAI Streaming (Server-Sent Events, SSE) returning broken words instead of full sentences, we need to ensure that the data being sent from the server to the client is properly encoded and formatted.

First, let's make sure that we have installed and configured the necessary packages for Laravel and OpenAI. You can use the following packages:

1. Laravel Echo: A Laravel wrapper for Pusher Channels and Event Broadcasting.
2. Laravel Sanctum: A package for implementing API authentication.
3. Laravel OpenAI: A package for interacting with OpenAI API.

Once you have installed and configured these packages, you can create a new route and controller method to handle the OpenAI streaming request. Here's an example of how to set up the route and controller method:

// routes/api.php
Route::get('/generate-text/{prompt}', 'OpenAiController@generateText');

// app/Http/Controllers/OpenAiController.php
use App\Http\Controllers\Controller;
use Illuminate\Support\Facades\Broadcast;
use Illuminate\Support\Facades\Event;
use Illuminate\Support\Facades\Log;
use OpenAI;

class OpenAiController extends Controller
{
    public function generateText($prompt)
    {
        $openai = new OpenAI();

        Broadcast::channel('openai.chat.' . auth()->id(), function ($user) {
            return true;
        });

        Event::listen('openai.message', function ($message) {
            broadcast(new OpenAiMessage($message));
        });

        $response = $openai->completion()->create([
            'model' => 'text-davinci-003',
            'prompt' => $prompt,
            'max_length' => 100,
            'stream' => true,
        ]);

        $stream = function () use ($response) {
            while (!empty($line = stream_get_line($response->getBody(), 1024))) {
                $data = json_decode($line, true);

                if (isset($data['id'])) {
                    Event::dispatch(new OpenAiMessage($data['choices'][0]['text']));
                }
            }
        };

        dispatch($stream);

        return response()->json([
            'status' => 'success',
        ]);
    }
}

In the above example, we create a new route /generate-text/{prompt} that calls the generateText method in the OpenAiController. We also set up the Laravel Echo broadcasting channel and event listener to broadcast the generated text to the client.

Now, let's focus on the issue of broken words. The problem is likely due to the fact that the data being sent from the server to the client is not properly encoded or formatted. To fix this, we need to make sure that the data is sent as UTF-8 encoded JSON strings.

We can modify the generateText method to ensure that the data is properly encoded and formatted as follows:

public function generateText($prompt)
{
    $openai = new OpenAI();

    Broadcast::channel('openai.chat.' . auth()->id(), function ($user) {
        return true;
    });

    Event::listen('openai.message', function ($message) {
        broadcast(new OpenAiMessage($message));
    });

    $response = $openai->completion()->create([
        'model' => 'text-davinci-003',
        'prompt' => $prompt,
        'max_length' => 100,
        'stream' => true,
    ]);

    $stream = function () use ($response) {
        while (!empty($line = stream_get_line($response->getBody(), 1024))) {
            $data = json_decode($line, true);

            if (isset($data['id'])) {
                $message = $data['choices'][0]['text'];
                $message = mb_convert_encoding($message, 'UTF-8', mb_internal_encoding()); // Convert to UTF-8 encoding
                Event::dispatch(new OpenAiMessage($message));
            }
        }
    };

    dispatch($stream);

    return response()->json([
        'status' => 'success',
    ]);
}

In the modified generateText method, we add the mb_convert_encoding function to convert the data to UTF-8 encoding before broadcasting it to the client. This should ensure that the data is properly encoded and formatted, and the client receives full sentences instead of broken words.

With these modifications, the Laravel OpenAI Streaming should now return full sentences instead of broken words.