Skip to main content
ABBYY Vantage uses external large language models (LLMs) to extract, classify, and understand document data from natural-language instructions. Vantage connects to LLM providers such as OpenAI ChatGPT, Microsoft Foundry, and Google AI Studio (Gemini), rather than hosting models itself.
LLM features are currently available in Document Skills through ABBYY Advanced Designer only. LLM connections are configured at the tenant level, under ADMIN → Configuration → Connections.

​
How LLMs work in Vantage

Vantage uses LLMs through two building blocks:
  • LLM connection - Tenant-level credentials that link Vantage to an LLM provider. A connection is shared across your tenant and can be reused by any skill.
  • Prompt-based extraction - An Advanced Designer activity that sends a document, plus your natural-language prompt, to the connected LLM and maps the response to your output fields.
A typical setup runs end to end as follows.
1

Configure an LLM connection

A tenant administrator adds a connection to a provider under ADMIN → Configuration → Connections. For the steps, see Configure LLM connections.
2

Add a prompt-based activity

In Advanced Designer, add a Prompt-based activity to a Document Skill and select the LLM connection it should use. For the steps, see Extract data with prompt-based activities.
3

Publish and run

When you publish the skill, the selected connection becomes the skill’s default in the Vantage Web Portal, under Skill Catalog → [skill] → Parameters. A tenant administrator can override it per environment without republishing. For more information, see Document skill parameters.

​
When to use LLMs

Prompt-based LLM extraction needs no training data and adapts quickly to new fields and layouts. Each extraction calls an external API, so it costs more and runs slower than a trained model, and results can vary slightly between runs.

​
Data handling and security

When a skill uses an LLM connection, the document content is sent to the external provider you connect to and is processed on that provider’s infrastructure, subject to the provider’s terms and data policies. Choose a provider that meets your data-residency and compliance needs. Microsoft Foundry supports Azure-based data residency and compliance controls. Protect the credentials stored in each connection:
  • Never share API keys or include them in a connection’s description.
  • Rotate credentials regularly.
  • Use separate connections for development and production.
  • Grant each key the least privilege it needs.

​
Cost and token usage

LLM usage is billed by the provider, based on the number of tokens processed. Cost scales with document length, prompt complexity, the document format sent to the model, and the number of fields you extract. Monitor usage in your provider’s dashboard, and use smaller, faster models for simple tasks.

​
Supported providers and models

Vantage supports several LLM providers, each with its own connection fields and models. For the current list, see Supported LLM providers and models.

​
What’s next

Configure LLM connections

Set up a tenant-level connection to an LLM provider.

Extract data with prompt-based activities

Build a prompt-based extraction activity in Advanced Designer.

Supported LLM providers and models

Compare providers, connection fields, and example models.