MULTI-MODAL MULTI-TASK FOUNDATIONAL MODELS FOR MEDICAL IMAGE MANIPULATION AND INFORMATION RETRIEVAL

Abstract

Systems and methods for automatically performing one or more actions on one or more medical applications are provided. Text-based instructions are received. The text-based instructions are encoded into text features using a machine learning based text encoder network. One or more instructions for performing by one or more medical applications are determined using a policy module based on the text features. The one or more instructions are performed by the one or more medical applications to generate a response to the text-based instructions. The response to the text-based instructions is output.

CPC Classifications