docs / destinations / filesystem_azure

AB

Azure Blob Storage

Parquet files in your Azure container, authorised by a connection string.

Every destination receives the same layout; see the tables Ingest creates.

Setup

Parquet files in an Azure storage container you own, under the prefix in Base URI (az://your-container/warehouse).

A connection string

  1. In the Azure portal, open the storage account that holds the container.
  2. Security + networking → Access keys → Show keys, and copy a Connection string: the whole DefaultEndpointsProtocol=…;AccountName=…;AccountKey=… value.
  3. Paste it into Storage connection string on the destination. It is stored as a secret in your own namespace.

An access-key connection string carries the whole storage account. If you would rather scope Ingest to one container, a SAS connection string for that container (Shared access signature → blob service, read/write/delete/list, your expiry) goes in the same field; remember it expires on the date you chose, which then reads as a destination that worked and stopped.

Then test it

Test connection creates a scratch schema and a table, writes a few rows through the same load path a real run uses, reads them back to check the count is exactly right, round-trips pipeline state, and drops the lot. An authorisation failure here is the connection string or an expired SAS; a 404 on the container means the Base URI names a container that does not exist yet; create it first.

Staging

needs_staging: none. Parts are written straight to your prefix.

Writes

Plain files cannot merge: a resource with a merge disposition is excluded before deploy, with the reason shown. Pick the Iceberg option for connectors that need upserts.

Conformance

Every adapter passes the same suite before it ships: create a table with every canonical type, append 100k rows, atomic replace, merge (latest dedup_sort wins; child tables delete-then-insert by root id), schema evolution, NULL-key rejection, package replay without duplicates, state and schema round-trip, row-count read-back, identifier edge cases, double-underscored table names (__quarantine, child tables), and rollback on a mid-load failure.