Runtime Architecture for Interfacing with Agents to Automate Multimodal Interface Workflows

Abstract

A system for client-side implementation of an interface automation language at runtime includes agent specification logic and runtime interpretation logic. The agent specification logic, running on client-side, is configured construct an agent specification, and to make the agent specification available for server-side translation into an intermediate representation, wherein the agent specification is configured to automate a multimodal interface workflow. The runtime interpretation logic, running on client-side, is configured to receive the intermediate representation, detect one or more agent functions in the intermediate representation, generate one or more agent calls based on the agent functions, issue the agent calls to an agent and, in response, receive at least one runtime actuation function from the agent, and translate the runtime actuation function into at least one runtime actuation command, wherein the runtime actuation command triggers at least one machine-actuated action as a runtime synthetic action that automates the multimodal interface workflow.

CPC Classifications