Skip to content

Catalog-Based UDTF Registration

Introduction

Catalog-based UDTF registration registers User-Defined Table Functions (UDTFs) and Views in Unity Catalog, providing permanent, discoverable, and governable access to CDF data. This approach is ideal for:

  • Production Deployments: Permanent registration with Unity Catalog governance
  • Data Discovery: Views are indexed and searchable in the Databricks UI
  • Access Control: Unity Catalog permissions (GRANT/REVOKE) for fine-grained access control
  • Enterprise Security: Credentials stored securely in Databricks Secret Manager
  • Team Collaboration: Shared, discoverable data assets across teams

Unity Catalog provides a three-level hierarchy: catalog.schema.object where UDTFs and Views are registered as objects within schemas, which are organized within catalogs.

Overview

This documentation covers the complete workflow for using catalog-based UDTFs and Views:

  1. Quickstart: Get from zero to registered UDTFs and Views in one notebook (catalog-based)
  2. Prerequisites: System requirements and access permissions
  3. Secret Manager: Set up secure credential storage
  4. Registration: Register UDTFs and Views in Unity Catalog
  5. Views: Understand Views and their benefits
  6. Querying: Query Views and UDTFs directly
  7. Filtering: Filter data using WHERE clauses with predicate pushdown
  8. Joining: Join data from different Views based on external_id and space
  9. Time Series: Work with template-generated time series UDTFs in Unity Catalog (same template-based generation as Data Model UDTFs)
  10. SQL-Native Time Series (Alpha): SQL-native time series UDTF with pushdown hints (experimental)
  11. Governance: Set up Unity Catalog permissions
  12. Troubleshooting: Common issues and solutions

Quickstart

  • Catalog-Based Quickstart: End-to-end flow based on the init quickstart notebook (install → generate → Secret Manager → register UDTFs and Views)

Examples