Skip to content

Catalog-Based UDTF Registration

Introduction

Catalog-based UDTF registration registers User-Defined Table Functions (UDTFs) and Views in Unity Catalog, providing permanent, discoverable, and governable access to CDF data. This approach is ideal for:

  • Production Deployments: Permanent registration with Unity Catalog governance
  • Data Discovery: Views are indexed and searchable in the Databricks UI
  • Access Control: Unity Catalog permissions (GRANT/REVOKE) for fine-grained access control
  • Enterprise Security: Credentials stored securely in Databricks Secret Manager
  • Team Collaboration: Shared, discoverable data assets across teams

Unity Catalog provides a three-level hierarchy: catalog.schema.object where UDTFs and Views are registered as objects within schemas, which are organized within catalogs.

Overview

This documentation covers the complete workflow for using catalog-based UDTFs and Views:

  1. Deployment concepts: Base URL, TOML, and how to verify deployment — read first
  2. Quickstart: Run this — zero to registered UDTFs and Views (same flow as quickstart.ipynb on GitHub)
  3. Prerequisites: System requirements and access permissions
  4. Secret Manager: Set up secure credential storage
  5. Registration: Register UDTFs and Views in Unity Catalog
  6. Views: Understand Views and their benefits
  7. Querying: Query Views and UDTFs directly
  8. Filtering: Filter data using WHERE clauses with predicate pushdown
  9. Joining: Join data from different Views based on external_id and space
  10. Time Series: Work with template-generated time series UDTFs in Unity Catalog (same template-based generation as Data Model UDTFs)
  11. SQL-Native Time Series (Alpha): SQL-native time series UDTF with pushdown hints (experimental)
  12. Governance: Set up Unity Catalog permissions
  13. Troubleshooting: Common issues and solutions

Quickstart

Examples