SFTP Stream Read

Read and parse structured data files from an SFTP server. Supports CSV, JSON, JSON Lines, XML, TSV, and YAML formats with automatic format detection. GZIP-compressed files are transparently decompressed.

Requires an SFTP connection.

Configuration

Field	Description	Notes
SFTP Connection	The SFTP connection to use	Required. Select from your configured SFTP connections.
Source Directory	The remote directory to read from	Supports templates. A directory browser is provided. If left empty, defaults to the connection's root directory.
File Pattern	A glob pattern to match filenames	Supports templates. Examples: *.csv, data_*.json, report-2026-*.tsv
File Selection Mode	Which files to read when multiple match the pattern	See table below. Defaults to Single File.
Data Format	The format used to parse file contents	See table below. Defaults to Auto-detect.
Decompress GZIP files	Automatically decompress .gz / .gzip files before parsing	Enabled by default.
Error Handling	How to handle errors during file reading or parsing	See table below. Defaults to Fail on Error.

File Selection Modes

Mode	Behavior
Single File	Reads the first matching file
Multiple Files	Reads all files matching the pattern and combines their data
Latest File	Reads only the most recently modified matching file

Data Formats

Format	Description
Auto-detect	Determines the format from the file extension (e.g., .csv, .json, .xml). Falls back to plain text if unrecognized.
CSV	Comma-separated values. The first row is used as column headers.
JSON	A JSON array or object. Arrays produce one item per element.
JSON Lines	Newline-delimited JSON. Each line is parsed as a separate item.
XML	XML document. Parsed into structured items.
TSV	Tab-separated values. The first row is used as column headers.

Error Handling Options

Option	Behavior
Fail on Error	The node fails immediately if any file cannot be read or parsed
Skip Invalid	Files that fail to read or parse are skipped; valid files are still processed
Collect Errors	All files are attempted; errors are collected and included in the output alongside successfully parsed data

Input

The node accepts an optional file_path input. If the Source Directory and File Pattern fields are left empty, the node will attempt to extract a file path from upstream data (looking for file_path, path, or file fields).

Output

When all files are read successfully, the node outputs a flat list of parsed data items. Each item includes metadata fields:

Field	Description
_source_file	Full path of the file the item came from
_source_name	Filename only
_source_size	File size in bytes
_source_modified	Last modified timestamp (UTC, ISO 8601)
_source_permissions	File permission string

When using Collect Errors error handling and some files fail, the output is an object with:

Field	Description
items	Array of successfully parsed items
total_count	Number of successfully parsed items
files_read	Number of files attempted
errors	Array of error objects with file and error fields

TIP

Use Multiple Files mode with a pattern like data_*.csv to combine data from many files into a single stream for downstream processing.

TIP

When reading compressed files, the format is detected from the name after removing the .gz extension. For example, report.csv.gz is detected as CSV.