Skip to main content
Cloud storage imports read objects matching a path prefix and combine the results into a dataset. Choose the prefix carefully: files beneath it should use the intended format and compatible columns.

​
Connection and permission requirements

AWS documents the permissions required by S3 operations. Apply source-side policies to limit access to the data intended for Dawn.
The current Azure connector uses an account key. This grants broader access than a read-only credential for one container. A container or prefix entered in Dawn selects data; it does not restrict the account key. Review Microsoft’s account-key guidance with your administrator.

​
Create the connection

Open Connections → New connection, select the storage source, enter its fields, and create or select the credential. For Azure, an Account URL looks like https://exampleaccount.blob.core.windows.net. Choose a descriptive connection name and save it. Keep the credential value in the credential form.

​
Configure the import

Specify the path and file type in the import step. Example paths are:
The connectors list objects by prefix. Even a path that looks like one filename is treated as a prefix, so verify that it does not also match backup files or similarly named objects. Directory markers are skipped; do not rely on the connector to filter a mixed directory by file extension. Select CSV or Parquet to match the actual files. Do not mix these formats under the selected prefix. For CSVs, use consistent headers and encoding across the files.

​
Verify the result

After the run, compare the row count with the expected combined files. Check duplicate records, the date range, and any columns that were missing from some files. Combining files does not establish that the records are unique. For recurring imports, decide how late files, replacements, and previously loaded records should be handled in your pipeline.

​
Troubleshooting

  • Access denied while listing: check bucket/container listing access, not just access to one object.
  • Access denied while downloading: check object permissions and any encryption-key permissions.
  • No data: check the bucket, container, prefix, and letter case.
  • Parsing error: look for an unexpected file or a mismatch between the selected format and actual contents.
  • Unexpected duplicates: check overlapping exports and repeated snapshots under the prefix.
For rotation and revocation, follow permissions and credentials.