Storage

FineParser Enterprise uses three volumes:

Volume	Access mode	Holds
In-process	ReadWriteMany	Uploads and partial results while jobs run. Shared by ingest and every worker.
Output	ReadWriteMany	Finished results, as `{jobId}/{name}`. Shared by ingest and every worker.
Orchestrator state	ReadWriteOnce	The job queue.

The in-process and output volumes must be writable by user and group 65534, which every FineParser container runs as. FineParser never deletes finished results. Results stay on the output volume after they are downloaded and after the job’s status expires. Remove them with your storage’s own lifecycle rules.

Configuring the volumes

The chart creates a claim for each volume, or uses one you created with existingClaim. The in-process volume is an EFS file system and the output volume is an S3 bucket:

fineparser-values.yaml

storage:
  inProcess:
    efs:
      fileSystemId: fs-0123456789abcdef0
  output:
    s3:
      bucket: my-fineparser-output
  s3:
    region: us-east-2

See EFS file system and S3 output bucket, including how to use claims you created.

Orchestrator state

The orchestrator requests a ReadWriteOnce volume of orchestrator.storage.size (2Gi) from the cluster’s default StorageClass. Check that the cluster has one with kubectl get storageclass; it is marked (default).

fineparser-values.yaml

storageClass:
  name: <block-storage-class>   # empty uses the default class

The state volume is kept when the release is uninstalled. See Upgrades and uninstalling.

Ingest scratch space

Each ingest pod buffers uploads in node-local scratch space capped at ingest.scratchSize. A request that fills it gets the pod evicted. See Sizing uploads.

Getting Started

Basics

Concepts

Best Practices

AWS

Reference

Configuring the volumes

Orchestrator state

Ingest scratch space

​Configuring the volumes

​Orchestrator state

​Ingest scratch space

Configuring the volumes

Orchestrator state

Ingest scratch space