Generative Networks Blog

LLMs: A Real Use Case

LLMs: A Real Use Case

TLDR: We worked on implementing LLMs, Vector DB, and RAG for elMejorTrato.com. We concluded that:

  • Working with Spanish language has its tricks
  • ChatGPT API is too costly
  • LangChain and agents are sometimes an overkill
  • With SentenceBert and a VectorDB, marvelous things can be done
  • Quantized open-source LLMs give very good results

We leave here the base code open-source dockerized for you to use in your projects

Creating a Prototype

Large Language Models, which gained popularity since ChatGPT was made available to the general public, have sparked excitement about their potential future uses. There are many examples of LLMs being used to simulate General Artificial Intelligence, which truly seem futuristic. Here we will focus on using LLMs for a specific case on the elMejorTrato.com website.

LLMs introduced two incredible novelties for software developers:

  1. The ability to interpret natural language.
  2. The ability to perform vector searches on text, audio, and images.

A specific example from elMejorTrato.com was that instead of entering a number, users had the option to enter the loan amount they wished to search for as text, such as "I need $10,000 to expand my house". The solution was to replace a text box with a select box. Today, with the natural language interpretation capabilities provided by LLMs, interactions can be better captured, providing users with better service.

A second example is the ability to answer users' questions that have already been answered for other users using vector searches on text using embeddings generated by the LLM.

To test these new solutions, we created a prototype from a fork of this very good template of docker with the following adaptations:

  • Prompt so that llama2 works correctly in Spanish.
  • Bert embedding model in Spanish from the University of Chile.
  • Weaviate as a database to perform hybrid searches (vector, text) and also to be able to use it locally or in the cloud.
  • Simplification of the code to make it easier to understand and adapt to your project.