Interactive Import

Interactive SQL Import parses your SQL query and transforms it into a set of views, dimensions, and data entities.

Navigate to the Interactive Import flow by selecting Manage in the navigation bar and then clicking Imports. Click the "New Import" button in the top right, and select "Interactive Import."

From here, simply:

• add your query to the SQL box
• run your query to validate that it is correct
• click Import

Once you click Import, we parse your query into a set of "Pending" views, dimensions, and data entities (see below). You can navigate to the "Pending" tab in your Catalog to "Publish" the entities. After publication, you can view the lineage of entities and query them.

📘

Note

• we require that the SQL query has at least one aggregation, which will become a data entity. For example, SELECT sum(impressions) FROM events_table. If you simply select a set of columns from tables, then you will see an error when you click "Import" that "There is no data entity in your imported SQL."
• for BigQuery we require tables to be referenced usingproject.dataset.table notation e.g.sandbox-demo-db.thelook_ecommerce.order_items.
• for BigQuery we only support standard SQL (not Legacy SQL)
• for BigQuery we require that you add a _TABLE_SUFFIXto your WHERE clause when querying a table with a wildcard. For more on querying wildcard tables with _TABLE_SUFFIX, please see this BigQuery documentation.

Pending Entities

After a SQL query is imported, a "Pending" entity is created. You can find the entities in the "Pending" tab of your Catalog. If you select a pending entity, you can update it by adding a name, description, etc.

To publish an entity, all validation issues (like the name of an entity needing to be in snake case format) have to be resolved. Validation warnings (like the existence of a similar view) will not prevent publication; however, we do recommend investigating the cause of validation warnings.

Once an entity is published, it is added to the catalog, where others can view its lineage and query it.

Semantic Validation

Imported SQL queries are compared against existing views and data entities in production. If entities from your import have semantic equivalents already published, then no pending entities will be created.

This is because Single Origin detects deduplicate views and data entities upon import so that you reuse as much as possible. For more information, please see Semantic Validation.