a month ago

Real time connectors setup & monitoring experience improvements

🚀 What's new

Listeners, or real time connectors don't depend on workflows execution nor workflow scheduling to catalog assets and create lineage. Listeners listen to source and when a run is finished/executed Atlan receive events, these events are then processed by Atlan to create relevant assets & lineage.

For this reason, we have renamed connector that listen in real time to listeners (Airflow & Spark). We have also removed any workflow dependencies to avoid confusions.

Finally we have made debugging listeners connection easier where you will find a new tab in listener profile in workflow center where you can view the events that Atlan received for a specific connection.

a month ago

Flexible Asset Scope in Policy Rules

🎉 What’s new

You can now define asset scope in policy rules using either:

Match All (AND condition): Assets must meet all selected criteria.
Match Any (OR condition): Assets can meet any of the selected criteria.

✨ Let's dig deeper

This enhancement eliminates the need for duplicating policies when asset selection requires OR logic—making policy management faster, simpler, and more flexible.

Save time: No more creating multiple policies for similar use cases.
Simplify configurations: Handle complex asset selection within a single policy.
Increase flexibility: Curate the exact asset scope you need with Match Any or Match All.

👏 Give it a shot

All users using the policy center, currently in public preview, can check out this feature! Join the conversation and help shape the future of policy center.

a month ago

Customize metadata visibility with asset metadata preferences

🎉 What's new

Introducing preferences to control the visibility of asset metadata on your assets in Atlan. This allows you to customize the metadata shown to specific user personas, thus making the experience more contextual to what the user is interested in.

✨ Let's dig deeper

When you open any persona today, you can head over to the "Preferences" tab in the persona. You will see a new "Asset metadata" section on the preferences.

This section allows you to customize visibility from the following metadata-

Certificate
Description
Domains
Linked Policies
Owners
Readme
Tags
Terms

Note- This impacts the assets page and the glossary (if persona switcher is turned on in labs for the glossary). This does NOT impact "all assets" experience i.e. if "all assets" is turned on in labs and is chosen on the persona selector, then the specific preferences for any persona will NOT be enforced.

👏 Give it a shot

Start customizing your users' metadata viewing experience today with asset metadata preferences. And while you try this, let us know what you think!

a month ago

View badges on data products

🎉 What's new

Introducing badges on data products to translate your custom metadata to digestible, color-coded signals on data products. This will help you add traffic-light markers, status markers etc. to better represent your data products for your consumers.

✨ Let's dig deeper

When you create a new badge off a custom metadata that in turn has been configured on "data products", you will see the badge show up on the data product profile.

You can configure the badge colors and conditions which will reflect on the data products.

Please note that any previously configured badges will start showing up as well.

👏 Give it a shot

Help your consumers find and digest data products better with badges on products. And while you try this, let us know what you think!

a month ago

View asset dependencies for cyclic lineage

🚀 What’s New

Cyclic lineage can be tricky, but we've got you covered! You can now view both downstream and upstream dependencies in cyclic relationships. This gives you full visibility into end-to-end lineage, ensuring better understanding.

a month ago

Lineage graph readability is made easier for you

🚀 WHAT’S NEW

You can now see full asset names in the lineage graph! Previously, long names were trimmed with "...", but we’ve improved this for better visibility. Browsing graphs is now easier, but also, exporting a graph & sharing it for collaboration would be possible after this release.

a month ago

Lineage quick dependency access via sidebar

🚀 What’s New

We’ve made it easier to see if an asset has upstream or downstream dependencies at a glance. In the Lineage tab on Atlan’s sidebar, assets with long dependency lists now show two options: "Load Upstream" and "Load Downstream". Selecting one retrieves more dependencies and updates the relevant tabs with the latest information.

Note: The number of displayed dependencies is based on a paginated request. To view all dependencies, keep clicking "Load More".

a month ago

Apache Airflow DAGs are now enriched with more metadata

🎉 What’s new

Atlan automatically retrieves and enriches Airflow DAGs with descriptions, source owners, and tags—if included in the OpenLineage event. This ensures your metadata stays up to date with source DAGs, helping you better govern, utilize, and understand your data estate.

Learn more:

a month ago

Discover an improved and dynamic product lineage

🎉 What’s new

Product lineage just got a 10x upgrade! Product lineage will now be generated even if the data products belong to different domains or have been modified after they are created or even archived. This release includes other improvements as well, making the depiction of dependencies between your data products automated and powerful.

✨ Let’s dig deeper

First, let's briefly revisit how product lineage works. If asset A is an output port in a data product A and a regular asset (not marked as an output port) is in data product B, then product lineage will be generated between data products A and B.

Data product lineage is now updated in the following conditions:

Data products belong to the same or different domains.
The asset selection in a data product is modified forming new links via output ports.
Assets are marked as output ports OR previous output ports are unmarked.
Data products are archived, thus removing them from product lineage.
A data products asset list is updated when new assets satisfy the filter conditions (this is for data products created using filter rules).

Product lineage is now generated in an async manner. This means there is a time gap from the last action that can update lineage (like marking an asset as an output port) to the changes being reflected in the lineage. This time gap is around 30 minutes to an hour. Most importantly, product lineage will not be generated when any participating product has more than 30,000 assets.

Any past product lineage links that were not present due to products being in different domains will now fully reflect the updated lineage, and without requiring any action from you.

👏 Give it a shot

Visit your favorite data product today and explore a more powerful product lineage experience! Create new data products to build out and experience the new product lineage. If you do not have data products enabled, you can head over to Labs (as an admin user) and turn on the products module!

Learn more: What is business lineage?

And while you try it out, let us know what you think!

2 months ago

Securely extract metadata from within your org

🎉 What’s new

The Atlan Secure Agent is a lightweight application that is deployed within an enterprise and facilitates metadata extraction. The Secure Agent acts as a gateway and connects to source systems and extracts metadata. This architecture avoids inbound connectivity to these systems as well as ensures secrets are managed internally.

✨ Let’s dig deeper

The Secure Agent is designed for secure, scalable, and efficient metadata extraction.

Currently supported connectors

Oracle, MS SQL Server, PostgreSQL, Salesforce
More connectors getting supported soon!

Security-first architecture

Runs entirely within the organization's infrastructure, preventing secrets from leaving its boundary.
Uses outbound, encrypted communication to interact with Atlan SaaS.
Supports logging and monitoring and integrates with external monitoring systems for auditing and compliance.

Scalable metadata extraction

A single deployment of the Agent can connect to multiple source systems.
Supports multiple concurrent metadata extraction jobs.

Flexible deployment

Deploys on cloud-based Kubernetes environments (such as Amazon EKS, Azure AKS, and Google GKE) as well as on-premises clusters.
Scales dynamically based on workload demands.

Automated operations

Continuously monitors system health and sends heartbeats to Atlan.
Captures and uploads execution logs for troubleshooting and auditing.

👏 Give it a shot

Switch on the Labs toggle to enable Agent functionality on the tenant.

Go through the Agent installation and configuration docs and setup a workflow in Agent execution mode

Feel free to reach out to your Atlan representative to know more!