AUTOMATED ADVERSARIAL PERSONA CREATION

Abstract

Techniques for optimizing the performance of an LLM-based chatbot are disclosed. A service builds an adversarial prompt representative of an adverse user that will be implemented by an LLM. The service builds a persona prompt for an LLM-based chatbot. The service feeds the adversarial prompt to the LLM, resulting in the LLM implementing the adverse user. The service also feeds the persona prompt to the LLM-based chatbot. The service facilitates an interaction between the LLM-based chatbot and the adverse user. The service causes the adverse user to assign a grade to a performance of the LLM-based chatbot. The service provides, as input, a grading definition and the grade of the LLM-based chatbot to an objective function tasked with determining whether the chatbot's performance was satisfactory. If not satisfactory, an optimization process is triggered.

CPC Classifications