Skip to main content

LLM Batch Inference

Brief​

A customer has a dataset of employee profiles and wants to generate a summary of each profile. We can use the LLM functionality on the platform to read each column in the dataset and generate a summary of each employee profile.

Tutorial​

  1. Go to the Workflows - My Workflows page and create a new workflow.
create new workflow

  1. Set the project name and description.

  2. Select the ‘+’ icon in the middle of the screen and create a new Dataset node. Under ‘Selection Mode’ select Upload a new Dataset and name and upload the dataset.

  3. Once the dataset has uploaded, click the ‘+’ again and click ‘Use a model’.

  4. Set this model action type to be ‘LLM’ and design a prompt based on the example prompt syntax.

<|user|> In 1 sentence summarise the following employee's career: {first_name} {last_name} worked at {present_company} and previously worked at {company_history} with bio {bio} <|end|> <|assistant|>

  1. Click the ‘+’ arrow to create another dataset node. Leave the selection mode to ‘Automatically create dataset on run’ and call it ‘Bulk Prompt Output’.

  2. Drag the node edges to join them so that the detailed synthetic linkedIn data feeds into the LLM model, which then feeds into the bulk prompt output dataset.

  3. Click ‘save’ in the top menu, then return to the Workflows page with the ‘exit button’.

  4. Click ‘Actions’ then ‘Run’ next to the workflow name. You can click ‘Actions - Log’ to check the status of the run.