Manage metadata of Dataform repositories with Dataplex Universal Catalog
Stay organized with collections Save and categorize content based on your preferences.

This document describes using Dataplex Universal Catalog to view metadata of Dataform repositories and annotate them with aspects.

Dataplex Universal Catalog lets you store and manage your metadata. Dataform repositories are available in Dataplex Universal Catalog by default, without any additional configuration. Dataplex Universal Catalog automatically retrieves the following metadata from Dataform repositories:

Data asset name
Data asset parent
Data asset location
Data asset type
Corresponding Google Cloud project
Third-party repository source
Service account

Before you begin

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

Go to project selector
Enable the Dataplex API.
Enable the API
Ensure that you have the necessary Identity and Access Management (IAM) permissions.

Required roles

To manage Dataform repositories in Dataplex Universal Catalog, you need the predefined Dataform roles that give you access to Dataform resources, and required Dataplex Universal Catalog roles.

Manage metadata in Dataplex Universal Catalog

Dataplex Universal Catalog logs Dataform repositories as entries with the following entry values:

System entry group: The system entry group for Dataform repositories is @dataform. To view details of repository entries in Dataplex Universal Catalog, you need to view the dataform system entry group. For instructions about how to view a list of all entries in an entry group, see View details of an entry group in the Dataplex Universal Catalog documentation.
System entry type: The system entry type for Dataform repositories is dataform-repository. To view details of Dataform repositories, you need to view the dataform-repository system entry type. For instructions about how to view details of a selected entry type, see View details of an entry type in the Dataplex Universal Catalog documentation. For instructions about how to view details of a selected entry, see View details of an entry in the Dataplex Universal Catalog documentation.

Search for assets

For instructions about how to search for data assets in Dataplex Universal Catalog, see Search for data assets in Dataplex Universal Catalog in the Dataplex Universal Catalog documentation.

Annotate repositories

To provide additional context to Dataform repositories in Dataplex Universal Catalog, you can annotate Dataform entries with aspects.

The system aspect type for Dataform repositories is dataform-repository.

For instructions about how to annotate entries with aspects, see Manage aspects and enrich metadata in the Dataplex Universal Catalog documentation.

Limitations

Managing Dataform repositories in Dataplex Universal Catalog is available with the following limitations:

You cannot view or manage Dataform file-level assets in Dataplex Universal Catalog—for example, tables, or views.

Locations

You can use Dataplex Universal Catalog to manage Dataform repositories from all Dataform locations.

Quotas

Managing Dataform repositories in Dataplex Universal Catalog is subject to Dataplex Universal Catalog quotas and limits.

Pricing

Managing Dataform repositories in Dataplex Universal Catalog is subject to Dataplex Universal Catalog pricing.

What's next

To learn more about Dataplex Universal Catalog, see Dataplex Universal Catalog overview in the Dataplex Universal Catalog documentation.