Ingress
Ingress is how data enters a workflow: uploads on File Input, HTTP triggers on REST Input, or catalog sync through Data Connector nodes backed by PyAirbyte. Parsed tabular data is stored securely in your workspace (see Storage and processing). Connector credentials are workspace secrets: the node’s secret fields hold {{secret.NAME}} references, and the values resolve from Vault when a run executes the node.
Connector credentials and OAuth
Section titled “Connector credentials and OAuth”Each field a connector marks as secret, including the OAuth client secret, accepts only a workspace secret: use Choose secret or Create secret inside the field. When you choose Connect on a connector that signs in with OAuth, you name the connection; the provider’s tokens are saved as workspace secrets for that connection name, so nodes that use the same name share one connected account. Replacing a secret’s value makes the next run execute the nodes that use it.
Upstream connector catalog
Section titled “Upstream connector catalog”Ingress mechanisms
Section titled “Ingress mechanisms”Ingest data from major databases, SaaS, files, APIs, streams, and custodian feeds into governed workflow datasets.
| Mechanism | Description |
|---|---|
| File Input | Upload or drag-and-drop. Tabular, JSON, XML, FIX, SWIFT/MT940, and OFX/QFX files parse to row datasets on the workflow engine; PDF and office documents route through Docling, a shared platform service. See Supported formats for the full extension list. |
| REST Input | External systems POST JSON to trigger a run and populate the input node output port. |
| REST API node | Fetch JSON from HTTPS APIs when a run executes the node, from a request you write (Manual) or an operation picked from the API’s OpenAPI specification (OpenAPI), with typed output columns and workspace secrets for sign-in. |
| Data Connector (Airbyte) | Airbyte-catalog sources via PyAirbyte: Notion, Postgres, S3, and hundreds more. No separate Airbyte platform server. |
PyAirbyte execution path
Section titled “PyAirbyte execution path”Docker/Java connectors spawn short-lived Kubernetes Jobs in the workspace namespace; Python/YAML connectors run in the sidecar.
- Python and YAML connectors run inside the workflow engine pod
- Docker and Java connectors run in isolated short-lived jobs with scanned, pinned images
- Connectors are baked into release images at build time (no runtime package installs on production hosts)
Related documentation
Section titled “Related documentation”- Supported formats - File Input extensions, FIX, SWIFT, and office formats
- REST API node - Manual and OpenAPI modes, authentication, typed output, and limits
- Storage and processing - where ingested data is stored and how runs persist
- Egress - export and downstream connectors
- Workspace secrets - connector credentials and OAuth connections
- Data - section overview