Interactive Import

Interactive SQL Import parses your SQL query and transforms it into a set of views, dimensions, and data entities.

📘

Note

  • we require that the SQL query has at least 1 data entity. If you simply select a set of columns from tables, then you will see an error when you click "Import" that "There is no data entity in your imported SQL."
  • for BigQuery we require tables to be referenced usingproject.dataset.table notation e.g.sandbox-demo-db.thelook_ecommerce.order_items.
  • for BigQuery we only support standard SQL (not Legacy SQL)
  • for BigQuery we require that you add a _TABLE_SUFFIXto your WHERE clause when querying a table with a wildcard. For more on querying wildcard tables with _TABLE_SUFFIX, please see this BigQuery documentation.

Draft Submission

After a SQL query is imported, a draft is generated. You can update the draft by adding a name and description for each view and data entity imported.

In order to submit a draft for review all validation issues (like the name of an entity needing to be in snake case format) have to be resolved. Validation warnings (like the existence of a similar view) will not prevent draft submission; however, we do recommend investigating the cause of validation warnings.

Once a draft is ready, it can be submitted for review. Reviewers can see details for all the views, dimensions, and data entities that will be created (or updated).

📘

Permissions

Only users with the Builder or Admin role can review drafts

If a reviewer approves a draft, then it is validated one final time before it is added to production. If validation fails at this stage, then the draft is sent back to the user who owns the draft to resolve the validation issues; otherwise the new entities are added to production where others can discover and consume them 🎉

🚧

Validation Issues

At this time, only certain validation issues can be addressed by users. If there is a validation issue related to SQL syntax, then you will need to delete the draft and re-import. To quickly re-import, copy your original SQL from the draft before deleting it. You can paste this SQL into the interactive import page, make the necessary updates, and then re-import. Improvements to this flow are coming soon!

If a reviewer rejects a draft, then the draft is sent back to the user who submitted it.

Semantic Validation

Imported SQL queries are compared against existing views and data entities in production. Single Origin can detect deduplicate views and data entities upon import so that you reuse as much as possible. For more information, please see Semantic Validation.