Guardrails in generative AI workflows via orchestration

With the increase in generative AI language model use and server implementations, there is a growing concern of how models can divulge information or how they can generate inappropriate content. This concern is leading to the development of technologies to “guardrail” user interactions with models, from screening user prompts for inappropriate asks, to checking model outputs to restrict unwanted responses. Examples include but are not limited to models presenting biases and model reveal of personal or proprietary business information.

This talk highlights an open-sourced guardrails orchestrator component which is designed to orchestrate calls and responses to generative model servers and detector servers, helping users easily apply guardrails to their generative AI workflows. We also share our defined taxonomy of detectors that provides a more intuitive organization of capabilities. This component can be easily supported on various AI platforms and is designed to work with popular model runtimes. Users can bring their own implementations of any generation model and detector servers. The orchestrator component can also help guardrail streaming content, enabling a more interactive user experience where users can receive detections along with streaming model-generated content.

We will discuss how we considered several important factors like performance, scalability, extensibility, and maintainability in the architectural design to make the solution production-ready. We will also share examples of use cases enabled, from those with simple lightweight detectors to more sophisticated LLM detectors, and how we adapted for multiple modalities.

Join us in exploring how you can incorporate diverse guardrails in your generative model workflow.