🎉 What’s new

We’re excited to unveil significant enhancements to the Databricks connector, elevating the crawler and miner capabilities to drive greater flexibility, governance, and trust across your Databricks landscape.

✨ Let’s dig deeper

• Primary and Foreign Key (PK & FK) Support: We now capture and display PK/FK information from Databricks, helping users identify the unique keys of assets.
• Materialized View Support: We now ingest and catalog Materialized Views created via native Databricks SQL. Users can visualize MV lineage and gain clear insights into their dependencies and upstream sources.
• 
  Enriched DBX Lineage Process: Users can now view enriched Databricks lineage with detailed entity types (e.g., Pipelines, Notebooks) and unique entity IDs, providing clarity and traceability in their data flows.
• Enabling Cloned Catalog Support for Databricks Miners: We now allows configuring alternative catalog for Databricks Miners when access to the default System tables is restricted. This provides controlled access to lineage metadata, ensuring that users only see information relevant to their business entity.
• Improved Preflight Checks: Enhanced preflight validations provide a smoother Databricks onboarding experience in Atlan.
  

👏 Give it a shot

Learn more and let us know what you think!

• How to set up Databricks 
• What does Atlan crawl from Databricks?
• Create cloned views of system tables
• Troubleshooting Databricks connectivity