Actions and Editing

Flow Actions

Clone

Cloning a Flow will open the Flow creation page with the same configurations pre-filled. This will not automatically create the new Flow.

Pause

Pausing a Flow will immediately stop processing data, and any in-progress sync will stop writing to the table. Pausing the Flow will retain a checkpoint of the last processed data from the source, so you can later resume from the same position.

When a Flow is paused, all table services running on the destination table will automatically be paused. When you resume the Flow, the table services will resume.

Resume

Resuming a paused Flow will begin processing data from the last processed checkpoint in the source. The Flow will fail if it attempts to process data that is no longer available in the source (for example, due to Kafka retention period).

Delete

Deleting a Flow will remove the Flow from your project. The Flow will immediately stop processing data, and any in-progress sync will stop writing to the table.

The destination table will NOT be automatically deleted from storage or the Onehouse console. All table services running on the destination table will automatically be paused.

Reusing the same BASE_PATH for a new Flow

Deleting a Flow marks the destination table as Stopped in the Onehouse console — it does not delete the table itself, and the table continues to occupy its BASE_PATH. If you try to create a new Flow on the same user-configured BASE_PATH, the API rejects with A table with base path '...' already exists. Please choose a different base path.

To reuse the path, also delete the table from the Data tab (Lake → Database → Table → Delete) before recreating the Flow. See Deleting tables.

Clean & Restart

Performing a Clean & Restart on a Flow will archive the destination table and restart processing from the earliest available data in the source. Flows whose destination table uses a custom base path are an exception — see Flows with a custom base path.

Clean & Restart is not for credential rotation

Clean & Restart is destructive — it archives the current table version and re-ingests from earliest. To pick up routine source configuration changes (rotated credentials, new broker hosts, etc.) without rebuilding the table, see Update a Source instead.

Table versioning

To ensure you do not lose historical data or break downstream queries, Onehouse versions every table by storing the data within a subfolder in the format:

s3://<tablePath>/<tableName>/v<versionNumber>/

The Onehouse console displays the latest version for each table as DFS Path on the table details page. When you Clean & Restart a table, the Flow creates a new version with an empty table and stops writing data to the current version. If the table is synced to catalog(s), you can continue querying the current table version while the Clean & Restart operation is in progress. As soon as the first commit is made to the new table version, Onehouse updates the catalog(s) to point to the new version so subsequent queries read the new version.

Flows with a custom base path

Clean & Restart depends on the table versioning described above — it needs to write the new, empty table into a fresh v<n> subfolder. When you supply a custom base path while creating a Flow, you also choose whether that path gets version support. The choice is made once, at creation, and cannot be changed later.

Version support	Where the table is written	Clean & Restart
On (the console default)	<custom base path>/v1/, then v2/, v3/, …	Supported. Each Clean & Restart writes the new, empty table to the next v<n> subfolder under your custom base path, exactly as for Onehouse-managed paths.
Off	Directly at <custom base path>/, with no v<n> subfolder	Not supported. Delete and recreate the Flow instead (steps below).

In the console, the Enable version support checkbox appears under Custom Base Path once you enter a path, and is checked by default. In SQL, set 'table.configured.base.path.versioned' = 'true' next to table.configured.base.path in CREATE FLOW; without it, a SQL-created Flow writes directly to the custom base path.

With version support on, the table details page shows the active version (for example s3://your-bucket/custom-path/v1) as DFS Path, and catalogs point at that version. Turn version support off only when downstream consumers must read the exact path you chose with no v<n> subfolder underneath it.

For a Flow created without version support, Clean & Restart is rejected with:

Flow/s: <flow name> use user configured base path without versioning. Clean and restart is supported for user configured base path only when versioning is enabled

To rewrite such a Flow's custom base path from the earliest available source data, delete and recreate the Flow by following the steps below:

1. Delete the Flow (Flow → Delete). This stops ingestion but leaves the destination table in place.
2. Delete the destination table from the Data tab (Lake → Database → Table → Delete). This releases the base path for reuse and, for Flow-created tables, deletes the underlying data. See Deleting tables.
3. Confirm the base path is empty in object storage, and remove anything left under it. If you may need the historical data later, copy it to a backup location first — this step is not reversible. A leftover Hudi table under the path will be picked up by the new Flow instead of a clean table being created.
4. Recreate the Flow with the same custom base path and the rest of the original configuration.

Use Clone so you don't have to re-enter the configuration

Clone opens the Flow creation page pre-filled with the existing configuration — it does not create a Flow until you submit it. Open the clone in a new browser tab, then complete steps 1–3 in the original tab before submitting the clone.

If you submit the clone while the old Flow or its table still exists, creation fails with The specified base path '...' conflicts with an existing table's base path '...'. Tables must not have overlapping or sibling base paths.

Editing Flows

After creating a Flow, you are able to edit the following configurations:

• Name
• Data Source (with limitations described below)
• Sync Frequency
• Pipeline Quarantine
• Transformations
• Data Quality Validations
• Catalogs

Change the Data Source

Currently, you can change the data source of an existing flow to any Kafka source.

This enables use cases such as bootstrapping historical data from an object storage bucket, then continuing ingestion from an active Kafka stream.

warning

Note that if you Clean & Restart the Flow after editing the source, the re-created Flow will only capture data from its current source.