Processing

When an upload completes, the file is available at once and the service starts processing it in the background. Processing never trusts what the uploader declared: it reads the bytes. This page lists the steps, what each one records, and how your code learns that a file is ready. It is for backend developers and evaluators.

The steps

Each version goes through these steps in order. Each records its outcome in the file’s processing object, so you can follow it with GET P/files/:file_id.

Step	What it does	Outcomes
sniff	Detects the type from the first bytes and compares it with the declared one, under the application’s type_mismatch_policy	done; a mismatch under reject refuses the file
policy	Checks the detected type against allowed_types (or the ticket’s upload policy)	done; a type outside the list refuses the file
hash	Computes the SHA-256 of the whole file; compares it with a declared content_sha256	done; a different checksum refuses the file. duplicate_of names an earlier file with the same bytes, for your information
scan	Scans the bytes for known malware, under the application’s scan policy	clean; infected quarantines the file; skipped when scanning is off or the file is too large to scan; failed when the scan could not finish
metadata	Reads into media: dimensions, orientation and format of images, with the capture date (captured_at) and the camera (camera) from the camera data; duration, dimensions and codec of videos; duration of audio; page count, title and author of PDFs. Never a location	done, skipped for other types
sanitize	Rewrites images without their camera metadata (EXIF, XMP, IPTC), orientation applied, when the application’s image_metadata is strip (the default)	done, or skipped when there was nothing to strip
renditions	Makes the renditions: thumbnail, preview and large of images, thumbnail and preview of a PDF’s first page, poster, thumbnail, mp4_720p and mp4_1080p of videos. Office documents get none	done when at least one is ready, skipped when the type has none
text	Extracts the text of documents for content search	done, skipped for types without text, failed when the bytes could not be read
publish	Writes the public copy of a public file	done or skipped
notify	Sends file.ready (or file.failed)	done

Every step value is pending, done, skipped or failed (and for scan, clean or infected); every value is on Processing status. Completion answers the file with every step pending. Files larger than 2 GiB get no renditions and no media metadata; those steps report skipped.

When processing refuses a file

A type mismatch under reject, a type outside the policy or a checksum mismatch are final. The service deletes the bytes, releases the quota, moves the file to the trash with trashed_by: "system" (so the owner can see why in the console), writes an audit row file.rejected, and sends file.failed with the step and the reason. Minting its URL then answers file_trashed.

Setting	Values
type_mismatch_policy	reject (default), trust_detected (keep the file with the detected type), trust_declared (keep the declared type)
allowed_types	Checked twice: on the declared type at the ticket, on the detected type here

A step that fails for another reason is handled by what it is. Bytes a step cannot decode (a broken image, a damaged document) record failed for that step or rendition, and processing goes on: the file is ready without it. A temporary error is retried, up to 5 attempts; if it still fails, the step records failed, processing stops there and the service sends file.failed. The file stays available and the other steps’ results remain. A scan that fails follows the application’s scan_failed_policy (Malware scanning). POST P/files/:file_id/renditions runs the renditions again; GET P/files/:file_id/renditions lists them with their status and error.

Knowing when a file is ready

Way	When to use it
The file.ready event	The owner, the uploader and the ticket’s notify_user_ids receive it through the Notification Service, with file_id, external_ref, version_no and the names of the ready renditions. See Events
Webhooks	A signed call to your backend for every event, with no user involved
files.files.waitUntilReady(fileId)	Your backend polls until no step is pending; it rejects when the file was refused, quarantined or trashed, or when a step failed and processing stopped there
GET P/files/:file_id	Read processing and renditions yourself

You rarely need to wait. Show the original until the preview rendition exists (images in the formats browsers display, videos and PDFs always), and swap it when file.ready arrives or on the next render.

Under the application’s block_until_clean scan policy (the default), a file is not served before its scan is clean: its URLs answer 409 scan_pending until then. Show it as processing and swap it in on file.ready.