- You want to accumulate documents processed by Manual Review Operators and use these documents to train a skill. Since Vantage allows accumulating documents processed by a Classification or Document skill, you can use this feature by invoking the skill from a Process skill with a Manual Review activity. Documents processed by Manual Review Operators will be added to the document set of the corresponding Classification or Document skill. New documents can either be used for automatic skill training or can just be collected to be reviewed manually by the skill designer. For more information, see Online Learning.
- You want to switch between different Classification and Document skills (rather than between skill versions).
Selecting a document accumulation strategy for Online Learning
Depending on your objectives and the development stage, you should select a document accumulation strategy. You have to decide which of the following you want to do:- Enable or disable automatic skill training when new documents are acquired (Online learning mode).
- Collect all documents or only documents that had issues, such as rule errors or uncertain fields (Review Documents mode).
| Development phase | Recommended strategy | Online learning mode (Classification/Document skill settings) | Review Documents mode (Manual Review activity settings of a Process skill) |
|---|---|---|---|
| Early skill development | Enlarge the document set by imitating document processing and collecting all documents processed by the skill rather than only documents with uncertain fields and rule errors. Several users can combine their efforts to review documents simultaneously and thus add a large number of labeled documents to the document set. Having collected the documents, you can continue skill development. You may need to adjust the skill before training, so automatic training should be disabled. | Collect only | All (default value) |
| Automatic fine-tuning of a skill with normal performance | Collect documents that had issues and were corrected during manual review and use automatic training. The new trained model is tested before replacing the old model and is discarded if it performs worse than the previous one, meaning that automatic training will not decrease the skill’s accuracy. | Collect and learn (default value) | With rule errors, with uncertain fields, or of unknown type |
| Late development | For a well-adjusted skill with an extensive document set, you may want to collect documents that had issues. Since the skill works well already, it’s not recommended to retrain it automatically. Instead of enabling automatic skill training, you should occasionally review the collected documents manually. | Collect only | With rule errors, with uncertain fields, or of unknown type |
Setting up document accumulation
To set up a skill according to the selected strategy:- Open a Classification or Document skill in Vantage.
-
Open skill settings and check if the Online learning mode corresponds to the selected strategy if you want to accumulate documents processed by the skill.
Note: This property is only available in skill settings in Vantage. - Publish the skill to make the changes available for other users.
-
Create and set up a Process skill. It should contain the following:
- A Classify or Extract activity that will reference the skill you wish to train. Enable Online learning when adding the skill to the activity if you want to accumulate documents processed by the skill.
- A Manual Review activity. Verify that the Review Documents mode corresponds to the selected strategy if you want to accumulate documents processed by the skill.
- If necessary, configure an additional input source and/or an additional destination for the result files. For more information, see Input activity and Output activity.
- Publish the Process skill. It will get a version number and become available for testing and production. The documents will be processed by a Classification or Document skill referenced by the Process skill and will be accumulated in accordance with the settings. Users can get access to the Process skill via API, connectors, or via a shared folder/email (if configured in step 5).
Editing a skill after documents have been accumulated
To adjust a skill using collected the documents, follow these steps:- Open the skill for which you collected documents. A new draft version of this skill will be created. As the Process skill will still reference the published version, editing the skill won’t affect users.
Note: All documents collected from this point on will only be added to the document set of the published version. They will not appear in the draft version you have just created. - Review the documents, make any necessary adjustments, and train the skill.
- Publish the skill. This will create a new published version with an incremented number.
Important! The version referenced by the Process skill isn’t updated automatically. Hence, at this point, the Process skill still references the previous version of the Classification/Document skill. - Update the version of this skill referenced by the Process skill:
a. Click up arrow next to the Process skill version number in the Skill Catalog.
b. In the dialog box that opens, select the latest version of a referenced skill.
c. Click Save.
d. Publish the Process skill. The Process skill version will be updated, and those who use the Process skill will get access to the new version of the referenced skill.
Rolling back to previous versions and testing different strategies
If you aren’t satisfied with the new version performance, you can delete the latest Process skill version. To delete the latest Process skill version, follow these steps:- In the Skill Catalog, click the Process skill version number.
- Click the Delete icon next to the latest version in the list. The latest version will be permanently deleted. Users will be switched to the previous version of the Process skill that references the previous version of the Classification/Document skill you’re developing.
- Open the Process skill.
- Click the Skill settings icon to the right of the skill name.
- Select any available version of the referenced skill and click Save.
- Publish the Process skill. Users will be switched to the new version of the Process skill that references the selected (not necessarily the next to last one) version of the Classification/Document skill you’re developing.
Important! Any changes made to a skill become available to the user only after you publish the skill.