Embeddings

Brief

A customer from an Australian Investment firm was dissatisfied with the existing tooling available to allow them to search through Australian companies. Their company research team was working with spreadsheets in Excel, which let them sort by a few high level metadata fields, but did not allow them to drill down with their search, or to search with more complex natural language concepts. They had access to a ‘company description’ text field but could not search it.

The TrueState platform allowed them to create a dashboard to display their company data that allowed an in-browser natural language similarity search, and incorporated a retrieval-augmented-generation (RAG) chatbot that they could use to chat with an intelligent bot that understands their dataset. They were able to distribute this dashboard to their team of non technical users who had shared access to this dashboard and dataset.

Tutorial

1. Go to the Workflows - My Workflows page and create a new workflow
   

2. Set the project name and description
   

3. Select the ‘+’ icon in the middle of the screen and create a new Dataset node
   

4. Under ‘Selection Mode’ select Upload a new Dataset and name and upload the dataset
   

5. Once the dataset has uploaded, click the ‘+’ again and click ‘Use a model’
   

6. Set this model action type to be ‘Embedding model’ and set the column name to the text field that we want to search
   

7. Click the ‘+’ arrow to create another dataset node. Leave the selection mode to ‘Automatically create dataset on run’ and call it ‘output dataset’
   

8. Drag the node edges to join them so that the company data feeds into the embedding model, which then feeds into the output dataset
   

9. Click ‘save’ in the top menu, then return to the Workflows page with the ‘exit button’
   

10. Click ‘Actions’ then ‘Run’ next to the workflow name. You can click ‘Actions - Log’ to check the status of the run
    

11. When the Run has completed, navigate to ‘Apps - Dashboards’ and click ‘Create New’
    

12. Name the dashboard and select ‘output dataset’ and submit
    

13. On the Dashboard, click ‘Add Element’ to add a table and select relevant columns. Tick ‘Enable similarity score column’ and click submit
    

14. The dashboard will now have a searchable dataset. Filters can be added and modified, and the text search can be set by typing in the ‘Embedding Search’ box
    

15. To add the Chatbot element, click ‘Add Element’ and select Type ‘Chatbot Interface’
    

16. The Chatbot is now available on the dashboard