dbt’s Model Groups & Access for Dummies

Welcome to another edition of “Faith takes you on the ride while she learns how to do her job.” Today, we learn about a handful of dbt’s model governance features.

Working for a high profile company like dbt Labs means I have to stay abreast of the product’s updates and functionalities, and learn how to actually use them. This is a pretty daunting task, because dbt ships a LOT. But generally, I am trying to be more of a “next right thing” type of person instead of a “holy shit I can’t believe how much I have to learn to be good at this job” sort of person.

The next right thing these last few weeks has been learning how to use dbt’s model governance features so that I could teach them to one of my training groups.

First off though—why bother with any of these model governance features? Who gives a shit, why can’t I just export my Looker dashboard to a csv and mess around with it from there? Well, you could definitely do that. You could also encourage everyone in your org to do just the same and then boom, you have self service data culture! Right guys? Guys?

You know I’m being facetious at this point. “Self-service data” is poorly defined buzzphrase anyways, and a free-for-all at your company’s data is probably going to put you in frustrating situations like I described in my semantic layer for dummies post. Realistically, there’s a number of reasons why you might want to restrict access (or ability to build on top of) some of your dbt models.

Why might I want to restrict model access?

dbt encourages a modular, iterative workflow in data. If this type of workflow is your vibe, then you are probably used to building things like staging models that take care of data cleanup tasks (renaming columns, cleaning up text, concatenating fields, this kind of thing). You’re also used to doing stuff like having intermediate models whose transformations and calculations get used as a building block elsewhere in your project.

If someone is new to your data ecosystem (or maybe they just don’t work in it much!), they might not really understand the difference between int_orders and fct_orders. Sure, sure, the difference may seem obvious to you, but can we please mature past assuming shit that is obvious to you is obvious to other people? If it’s your goal to get folks in your org to interact with your data on their own more often, it’s not a bad idea to give them guardrails. That can look like preventing them from ref’ing intermediate models when they should be ref’ing the final fact table, or whatever mature model you are okay with someone building on top of.

Model access is not the same as database permissions. dbt can facilitate database permissions with the grants config if you want help with that. Managing model access is more about putting guardrails around what people are allowed to build off of either within or between dbt projects.

How can I restrict model access?

Wanna take a wild guess? Why yes, you read my mind, it is in fact going to be YAML!!! Our BFF!!!

Look, I don’t like YAML much myself. But the good news is, you truly never have to write YAML from scratch, and frankly, you never should. It’s a rookie mistake to try. And, getting model groups (which enables model ownership & access controls) set up is really not so bad. It comes in just 3 easy steps.

1. Declare groups
2. Apply groups
3. Modify access if you need to

It’s wild that I’ve already spent nearly 1300 words and I only got through model groups and model access. The important things to remember are:

• Groups are collections of nodes in a DAG.
• Groups can be assigned an owner, and you can specify the owner’s name, email, slack, and github.
• Assigning models to groups is done very similarly to materializations—either do it in your dbt-project.yml, a model’s YAML, or a mode’s SQL file.
• After you’ve assigned models to a group, you can change their access depending on what you want. The default is protected, which means all models in the same project can ref the protected model.

Tune in for the next dummies post coming soon. If you also stick around for my non-dbt related writing, I appreciate you. It doesn’t always feel right to promo those posts on LinkedIn, so I am grateful for those of you that read them anyways.

Subscribe, tell your friends, you know the drill. Thanks for reading.