Scaling

FlexiCapture can process from several hundred to millions of pages per day, and support up to several thousands of operators. With guidelines from this paper, it is easy to estimate the System load in advance and select both the appropriate architecture and hardware for the servers. The System scales up by:

• Increasing the number of scanning clients, verification clients, and Processing Stations
• Increasing the power of machines for the Application, Processing, Licensing, and Database servers, and the FileStorage, using several machines for these roles

The numbers below help to assess or select a preliminary configuration of the FlexiCapture server component. System monitoring and bottleneck detection helps you ascertain that the hardware used is not sufficient for the target performance and that it is time to scale up.

​

Demo

Demo is a typical configuration for demonstrations or pilot projects, and is not recommended for production-scale projects. All the System components are installed on a virtual machine or deployed on a PC. MS SQL Express may be used as a database server and installed on the same machine with FlexiCapture servers. Instead of using separate FileStorage, files can be stored directly in the database. Operators and processing stations can be installed on the same machine.

​

Medium

Medium is a typical configuration for commercial projects, because it is scalable: each server component is installed on a dedicated machine.

​

Deployment guidelines

The Application Server should be installed on a dedicated machine, because it employs a scaling-up approach that is different from the Database, Processing, and Licensing servers. The Application Server and the Processing and Licensing Servers use different clustering technologies:

• The Application Server is a web service in IIS; its scaling and reliability are achieved by clustering that uses Microsoft Network Load Balancing technology. All cluster nodes are peers running in active-active mode and can be switched off at any time.
• The Processing Server and Licensing Server are Windows services; their reliability is achieved by creating an active-passive cluster based on Microsoft Failover Cluster technology.

Microsoft clearly prohibits the use of these technologies together on the same computer. If reliability is all you need, cluster the Application Server within IIS, which supports clustering by Microsoft Failover Cluster as well. Licensing and Processing servers can be installed on the same machine. We recommend installing the Database Server on a dedicated machine. It is very resource-consuming and if you do combine it with certain other FlexiCapture Servers, restrict its use of the CPU and RAM and locate the database files on a physically separate HDD, so as not to affect the performance of the neighboring server. For small loads and better performance, you can use fast HDDs on the Application Server machine as a FileStorage: for example, 15,000 RPM or faster SATA2 disks, arranged in at least RAID1 for redundancy, or RAID10 for better performance as well. At later stages of the project, however, if the volume of pages to process increases, this configuration will likely result in a bottleneck, especially for processing grayscale or color images, and the problem is that it can’t be scaled up on the fly – it will require the System to go down and other HDDs to be attached. Use external storage such as NAS or SAN, to which the Application Server has read-write access at 1 Gb/s over LAN, SCSI, Fibre Channel, or a similar interface. This will enable a smooth scaling up of the FileStorage. A typical FlexiCapture network configuration in an enterprise environment: Note that it is recommended to connect the Application Server directly to the FileStorage and Database Server for fast and reliable communication.

​

Application Server

Requirements:

• CPU: 8 physical cores, 2.4 GHz or faster
• 16 GB RAM
• HDD: 100 GB
• 2 NICs, 1 Gb/s: one to connect to the LAN and one to connect to the Database Server
• FileStorage: if a SAN is used, connect it using SCSI, Fibre Channel, or InfiniBand
• OS: Windows 2012 or later

A web service and the hub of all FlexiCapture communications, the Application Server is responsible for both:

• Transferring large binary bodies
• Responding quickly to small SOAP/JSON service requests

Critical resources are:

1. A fast network interface for connecting to clients.
2. A fast, stable connection to the FileStorage and Database Server.
3. A multi-core CPU with high speed:
   • The higher the speed, the faster each request is handled
   • The more physical cores there are, the more requests are handled at the same time
4. Sufficient RAM, at least 2 GB per physical core.

To make the most of the CPU, configure the FlexiCapture Web Services application pool with twice as many IIS Worker Processes as the number of physical cores. For example, 16 IIS Worker Processes for an 8-core processor. If any of these resources causes a bottleneck, scale up the Application Server:

• With Microsoft Network Load Balancing technology, which clusters multiple computers with the Application Server role. For detailed instructions, see Failover configuration.
• At the hardware level, by connecting a different range of clients to different machines with the Application Server role. For example, you can use one machine to serve all automatic processing and another to expose it to external clients.

In any case, all machines with the Application Server role should be equally connected to the same Database and FileStorage.

​

Processing Server and Licensing Server

Requirements:

• 4-core CPU, 2.4 GHz or faster
• 8 GB RAM
• HDD: 100 GB
• NIC: 1 Gb/s for connecting to the LAN
• OS: Windows 2012 or later

A stable network connection is essential for these servers. Otherwise, document processing will stop. To ensure redundancy, use Microsoft Failover Cluster. For detailed instructions, see Failover configuration. The Licensing Server handles copies of licenses for all concurrent clients in its memory. Keep this in mind if you are going to use a large number of scanning and verification operators simultaneously. We also recommend using the 64-bit version in projects involving a large number of concurrent clients. Our tests have shown that 2 GB of RAM is enough to handle licenses for up to 1000 clients. Consider using more than one Licensing Server to serve more concurrent clients simultaneously.

​

Database Server

Requirements:

• For MS SQL Server: MS SQL Server 2014 or higher, Standard or Enterprise Edition.
• For Oracle: Oracle 12c Enterprise Edition.

Hardware for MS SQL Server:

• CPU: 8 physical cores, 3.4 GHz or faster
• 16 GB RAM or more
• HDD: 400 GB
• OS: Windows 2012 or later

Hardware for Oracle: Oracle Exadata Database Machine X2-2, Quarter Rack. ABBYY FlexiCapture supports MS SQL Server and Oracle installed on any platform. Both database servers keep their own records on optimal settings, scaling, and fault tolerance. For MS SQL Server, we recommend:

• More RAM if possible on the Database Server machine, to host the greater part of the database files in RAM and access it faster
• A fast HDD for quick access to the database part hosted on the disk (we recommend using an SSD for this purpose)
• Avoiding database modes with transaction delays (mirroring, and so on)
• Choosing Simple as the database recovery model
• Storing the database and its log on separate disks
• Regular index updates for frequently changing tables (Document, Page, Batch, Task, and EventLog). Otherwise, the size of an index may become larger than the size of the data in the table.

​

FileStorage

Requirements:

• NAS or SAN, connected via LAN, SCSI, Fibre Channel, or InfiniBand
• Read-write speed: 100 MB/s*
• Capacity: 5 TB*

* Read-write speed and capacity depend on your workload; see FileStorage sizing.

​

FileStorage sizing

Read-write speed and capacity requirements depend on two factors. Factor 1: the average and peak pages processed per day (24 hours) and per hour, and their color mode. As mentioned in the Performance metrics section, you can estimate the input flow in bytes per second using typical file sizes for pages scanned in color, grayscale, and black-and-white. Images make up the majority of data transferred within the System. By analyzing the processing workflow, define two values:

• The number R of stages where page images are downloaded from the Application Server
• The number W of stages where page images are uploaded to the Application Server

The read-write speed requirements can be calculated as follows:

• Required write speed = W × input flow in bytes per second.
• Required read speed = R × input flow in bytes per second.

Example. A customer needs to process 10,000 grayscale pages per hour. The processing workflow includes the following stages:

1. A Processing Station downloads images from a hot folder, pre-recognizes them, and uploads them to the Application Server (W=1, R=0).
2. Another Processing Station retrieves these images from the Application Server, performs recognition, and the OCR results arrive at the Application Server (W=1, R=1).
3. A verification operator downloads images and recognized data for checking and sends the verified data back to the Server (W=1, R=2).
4. Finally, a Processing Station downloads images and verified data to send them to the customer’s backend system (W=1, R=3).

Assuming that the file size of an average A4 grayscale scan is 3 MB:

• Input flow = 10,000 grayscale page images/hour = 2.8 grayscale images/s = 8.4 MB/s.
• Required write speed = 1 × 8.4 MB/s = 8.4 MB/s.
• Required read speed = 3 × 8.4 MB/s = 25.2 MB/s.

To benchmark the performance of the hard disk, you can use the CrystalDiskMark tool, distributed under the MIT license. Factor 2: the amount of time that documents are stored in the System. Example. A customer needs to process 100,000 grayscale images in 24 hours. Under the Service-Level Agreement, the processing time is 2 days per document. Processed documents are stored for 2 weeks because of additional checks in the customer’s ERP system; in the event of any discrepancies, documents are edited in FlexiCapture and uploaded to the ERP system again. Thus, images are to be stored for 2 + 14 = 16 days, and the System will accumulate 16 × 100,000 grayscale images × 3 MB (average file size for an A4 grayscale image) = 4.8 TB of data.

​

Large

The Large configuration is required when you deal with a significant volume (more than 300,000) of color pages. Its range is up to 3 million black-and-white pages or up to 1 million color pages in 24 hours. All that is mentioned above about the Medium configuration remains valid for the Large configuration. The difference here is that you have to follow all optimization recommendations and pay special attention to each part of the system – to calculate the load and to choose hardware that is sufficiently powerful, yet not too expensive. Among other things, test the Internet connection and the backend connector to ensure they can operate at the target performance level. From the very beginning, consider using a 10 Gb/s network and a powerful FileStorage. Possible network architecture for the Large configuration is shown below. Instead of providing typical system requirements for Large configurations, we recommend looking at the configurations that were tested, and their performance, as provided in this document.

​

xLarge

To achieve even better performance, combine several independent FlexiCapture installations under one Administration and Monitoring point. This is referred to as the xLarge configuration and is beyond the scope of this document.