Registered Queries

Overview

Registered Queries are a data team service that gives GitLab team members a low-overhead path to get a reusable SQL query into the data warehouse as a documented, tested view. Instead of maintaining ad-hoc queries locally or requesting a full data model, you can register a query and have it reliably available in the warehouse, typically within 24 hours of submitting a request.

See the Enterprise Data Warehouse page for how Registered Queries fit into the broader warehouse architecture.

Where Registered Queries Fit

Registered Queries are one of three layers in PROD where reusable logic can live. Each layer trades speed for rigor.

A good rule of thumb is to start in the registry and promote when the query earns it.

Prerequisites

Before submitting a Registered Query request, the target schema for your function must already exist. Creating a new function schema and setting up its permissions falls outside of this workflow and must be arranged separately by opening a Standard Data Team Issue. If your team does not yet have a schema, use the general schema as the default.

Example schemas:

The authoritative list is the query_registry block in dbt_project.yml. If you provide a schema that does not exist, the flow will stop and list the valid options in the issue.

Benefits

• Reliability: Registered queries are monitored by the data team. If something upstream breaks, the data team will know and be able to fix it. Treating a query as a known model also makes the data team less likely to inadvertently break it with internal changes.
• Testability: Being a model allows basic tests to be applied, providing an early indicator when data changes in unexpected ways.
• Scalability: If other team members need the same query, having it as a documented view in the warehouse is the easiest way to share it.
• Transparency and lineage: Logic lives in version control with MR history, and lineage tools such as Atlan can trace it back to its upstream sources.
• AI and tool readiness: Descriptions and column documentation are persisted to Snowflake as object comments, so AI agents, Snowflake Cortex, and other BI tools can discover and understand the view.
• Portability: Logic in the warehouse does not have to be rebuilt when moving between reporting tools.

Common Use Cases

BI Tool Custom SQL

Custom SQL embedded in a BI tool is invisible to the rest of the data stack. Registering it provides:

• One governed view in place of near identical Custom SQL copied across several workbooks.
• Visibility for Analytics Engineers, so upstream changes account for the dependency instead of breaking the workbook.
• Lineage in Atlan, which traces dbt views far more reliably than Custom SQL.
• Reviewed, version controlled logic changes instead of silent edits in Tableau Desktop.
• Logic that can be reused by AI reporting and other BI tools without being rewritten.
• Consistent SAFE handling through secure views in the restricted_safe schema.

Streamlit Apps

SQL in a Streamlit app is usually embedded in Python code, which makes it hard to find, review, or reuse. Registering it provides:

• App code that reduces to a simple SELECT against a documented view.
• Logic fixes delivered through an MR without redeploying the app.
• Queries that are inspectable and reusable by other apps and tools.
• Clear department ownership through CODEOWNERS.

The app’s owner role needs SELECT access on the registry schema. See Access.

When to Use This Service

Use Registered Queries when:

• You have a query you run repeatedly and want it to be reliable and available to others.
• A view provides sufficient performance for your use case.
• You want the data team to monitor the query for upstream breakage.
• You want basic data quality tests applied to the results.

When Not to Use This Service

• Performance needs beyond a view: If query performance requires a materialized table, engage other data team services such as creating workspace or modeled tables.
• Building other models on top of it: Registered queries cannot be referenced (ref()) by other dbt models. If you need to build downstream models, the full data model development process should be used instead.
• Company wide certified metrics: Logic that defines an official company metric belongs in PROD.COMMON through the full development process.

How to Request

1. Open an issue using the Registered Query issue template.
2. Include the following in the issue:
   • The SQL query text you want registered.
   • Whether the results contain SAFE data (required — this determines which schema the query is placed in).
   • The GitLab username of the maintainer who will perform the final review and merge (required).
   • Optionally, the schema key for your function (defaults to general).
   • Optionally, the GitLab username of a reviewer (defaults to the issue creator).
   • Optionally, whether you want to be listed as a code owner (you will then be required to approve any future changes).
   • Optionally, a column or list of columns to use for uniqueness testing.
3. Assign the issue to a data team developer.
4. When the data team notifies you in the issue that the MR is ready, review and approve it.

What Happens After You Request

Once the issue is assigned, the Duo flow will:

1. Parse and validate your Config and SQL. If a required field is missing or invalid, it comments on the issue and stops.
2. Check existing registered queries for significant overlap. If a close match exists, it comments with links and stops so you can reuse it or clarify the difference.
3. Create a branch and write or update the rq_<name>.sql file, schema.yml entry, and documentation.
4. Lint with SQLFluff and run dbt parse and dbt compile, fixing errors automatically for up to three attempts.
5. Self review against the SQL Style Guide and Snowflake guidelines, fixing style issues and flagging performance considerations as MR comments.
6. Open the MR and assign your reviewer and maintainer.
7. Comment on the issue with the MR link.

After the maintainer merges, the deploy_query_registry CI job rebuilds registered queries in PROD, making your query available.

Self Service MR

Team members with Developer access to the gitlab-data/analytics project can open the MR directly using the dbt Registered Query Changes MR template. Follow the same file layout, naming, and documentation conventions described on this page. The same review and approval rules apply.

Configuration Options

Required

Optional

Review and Approval

• The [Query Registry] section of CODEOWNERS assigns the query_registry directory to members of the data team.
• If a code owner is specified in the issue, the flow adds a file level CODEOWNERS entry for that query.
• A maintainer approval is required before merge.

Access

Registered queries are read with the same Snowflake roles used for other PROD data. Views in restricted_safe schemas are secure views and require SAFE access. Service accounts, Tableau connections, and Streamlit app owner roles must have SELECT on the relevant registry schema.

Constraints

• All registered queries are materialized as views. This allows for continuous deployment without interrupting production dbt runs.
• All files use an rq_ prefix (e.g. rq_my_query.sql). This aids in audits and signals to users the potential limitations of these objects.
• Registered queries are placed in an independent schema separate from the rest of the warehouse. The schema used depends on whether the query is SAFE or not. A general schema is provided as a default for teams that do not yet have a dedicated workspace.
• Registered queries cannot be referenced (ref()) by other dbt models. They exist in a private dbt group specifically to prevent this.
• Queries that have not been used for more than 90 days are subject to removal during quarterly audits.
• Expected turnaround for new requests is approximately 24 hours.

Repository Structure

Registered query files are organized under a query_registry directory by function and SAFE status:

query_registry/
  <function>/
    restricted_safe/
      rq_<name>.sql
    rq_<name>.sql

SAFE queries are placed under restricted_safe/ within the appropriate function directory. Non-SAFE queries sit directly in the function directory.

View page source - Edit this page - please contribute.