Creating AI LLM Elements
LLM elements are the core components that enable unified integration of various AI large language model services within the JitAi platform. They provide foundational model capabilities for AI Agents, AI Assistants, and AI Knowledge Bases.
The platform natively supports mainstream LLM vendors including Anthropic, OpenAI, Gemini, Deepseek, Alibaba Cloud Bailian, and SiliconFlow, while also providing OpenAI-compatible interfaces to support any LLM services compatible with the OpenAI protocol (including privately deployed ones). Flexible API key management and intelligent retry mechanisms ensure reliable and stable model invocations. Developers can also extend new LLM Type elements to support any open-source or closed-source LLM services from different vendors with different protocols.
Mainstream LLM Vendor Support List
- Anthropic
- Alibaba Cloud Bailian
- Deepseek
- Gemini
- OpenAI
- SiliconFlow
- OpenAI Compatible
Creating LLM vendor elements
Click the + button next to the search box in the element directory tree, select AI LLMs from the popup menu, then choose the appropriate LLM vendor based on your requirements. After making your selection, the following dialog will appear.
In the dialog, enter the name, then visit the corresponding model vendor's configuration page to obtain an API Key (if you're unsure of the address, click Get API Key in the bottom left corner), enter the API Key here, and click the Save button to complete the creation.
API Keys can be configured using environment variables by clicking the button in the top right corner to prevent exposure.
The API URL typically doesn't require modification, as default configurations for each vendor are pre-filled.
Once created, the element will appear in the left element tree.
Retry and backup API key mechanism
To prevent model call failures caused by single key rate limits, excessive requests, and other constraints, we provide retry and multi-key mechanisms. The system performs a maximum of 3 retries with exponential backoff: 1 second, 2 seconds, and 4 seconds respectively. When multiple API Keys are configured, each retry rotates through backup API Keys. This ensures the system doesn't repeatedly attempt calls on the same failed key, but intelligently switches to backup keys to improve success rates. With only one API Key, the system will attempt 4 times total (1 initial call + 3 retries).
Click Enable Backup Keys → +Backup Key in the interface to add a key input field, enter your backup key, then click Save.
Private LLM integration
For enhanced information security, developers may deploy private models, which the platform fully supports through seamless integration.
Private models connect using OpenAI Compatible elements. Using Ollama as an example, the default address is http://localhost:11434/v1. If an API Key is required, enter it accordingly. Note that private models require enabling the Enable Custom Model configuration and specifying the complete model name, such as: qwen3:0.6b, nomic-embed-text.
Any vendor supporting OpenAI-compatible interfaces can utilize OpenAI Compatible elements for connection. The figure below demonstrates Volcano Engine's Doubao as a reference implementation.