It additionally gives visitors routing and fee limiting for native and third-party massive language fashions (LLM) to take care of service availability and efficiency and management prices. F5 acknowledged. Semantic caching drives sooner response time and reduces operational prices by eradicating duplicate duties from LLMs, in keeping with the seller.
The AI Gateway can examine, establish, and block inbound assaults equivalent to immediate injection, insecure output dealing with, mannequin denial-of-service, delicate info disclosure, and mannequin theft. “For outbound responses, AI Gateway identifies and scrubs PII knowledge and prevents hallucinations. Software program improvement kits (SDKs) allow further enforcement of operational guidelines and compliance necessities for each prompts and responses to additional align to operational wants,” F5 acknowledged.
“Extra capabilities equivalent to reporting of a wide selection of metrics by way of OpenTelemetry, cautious consideration to audit log necessities, semantic caching, rate-limiting, and content-based mannequin routing guarantee help for all three AI supply and safety necessities: observe, shield, and speed up,” MacVittie wrote.
The AI Gateway might be built-in with F5’s NGINX software safety suite and BIG-IP software supply platforms providing prospects legacy integration and entry.
