In case you’ve been on the web not too long ago, you most likely noticed how OpenAI’s ChatGPT is taking the world by storm. From writing books and designing rooms to debugging code and explaining knowledge ideas, plainly automation and AI can now reply any query you ask.

Compared, metadata can really feel like residing within the Stone Age. Historically, it’s a tiresome, guide course of that wants loads of human intervention from knowledge leaders, stewards, engineers, and analysts alike. In our newest webinar, a ballot confirmed that 60% of attendees spend over 3 hours per week on guide knowledge duties.

At Atlan, one in every of our objectives is to automate wherever doable. How can we decrease the mundane and maximize the influence knowledge groups can have on the enterprise?

In final week’s Atlan Activate, our quarterly product webinar, we launched new automation options to superpower your knowledge. Listed below are the highest 5 options that may aid you map context throughout your knowledge property and restrict the repetitive guide work that slows knowledge groups down.

TL;DR: 5 new options in Atlan it’s best to learn about

• Metadata Playbooks for rule-based actions: Like Zapier for knowledge, that is the primary low-code/no-code metadata automation for knowledge groups.
• Atlan + AWS EventBridge event-based actions: Create production-grade, event-driven automations for the world of metadata, equivalent to alerts when possession modifications or auto-tagged classifications.
• Profiling and Reputation Insights: Use new column-level profiling, recognition, and utilization metrics to evaluate knowledge’s high quality, discover essentially the most extensively used queries, establish high customers, and extra.
• Atlan to GitHub integration: Carry metadata proper to GitHub to reduce danger and enhance transparency earlier than any modifications are made to your knowledge.
• Trident AI powered by GPT-3: Say goodbye to guide documentation with more and more clever automated descriptions, enterprise phrases, READMEs, and extra.

Metadata Playbooks: Introducing Zapier to your knowledge property

Create rule-based bulk automations at scale to robotically deprecate unused knowledge property, assign possession, report failures or bulletins, and extra.

One of many widespread questions we get from knowledge groups is, “How can we automate our metadata?” Whereas different groups like advertising or gross sales can do action-based automation at scale with instruments like Zapier or Salesforce, knowledge groups don’t have the bandwidth to code customized automations for every numerous use case. Why can’t there be a Zapier for metadata?

That’s the reason we’ve got developed the primary low-code/no-code metadata automation for knowledge groups. With Atlan’s Playbooks, customers can now create rule-based automations at scale. These can drive limitless use circumstances throughout your group. Listed below are just a few examples:

• Deprecate property: Mark any property that haven’t been queried within the final 30 days as deprecated.
• Add/change possession: For Salesforce property which might be lacking an proprietor, add RevOps because the proprietor.
• Report failed property: Publish a listing on Slack of each desk and dashboard with a “failed” Airflow standing.
• Defend delicate knowledge: Connect GDPR customized metadata on all property tagged as “PII”.
• Flag upstream alerts: Notify downstream homeowners when an upstream asset is tagged with a warning or announcement.

Atlan + AWS EventBridge: Construct customized, event-based metadata automations

Create production-grade, event-driven automations for the world of metadata, equivalent to alerts when possession modifications or auto-tagged classifications.

Low-code/no-code, rule-based automations like Atlan playbooks are nice, however some knowledge groups don’t wish to be restricted by guidelines. They wish to construct their very own automations for repetitive actions.

Different groups, from knowledge observability to incident administration, are constructing automations that may set off actions primarily based on occasions. For instance, many entry administration groups use Okta to watch system log occasions for suspicious exercise and automate actions to mitigate dangers. Buyer engagement groups use Salesforce to create occasions that enrich assist circumstances with order knowledge from prospects. However what about metadata administration? There’s nothing accessible to create these sort of event-driven use circumstances for metadata… till now.

We’re excited to announce that knowledge groups can now use Atlan to create production-grade, event-driven automations for the world of metadata. This leverages our integration between Atlan and AWS EventBridge, an AWS service that creates an occasion and lets customers devour and construct use circumstances on it, to create Atlan metadata occasions into an EventBridge account.

Listed below are some examples of how this would possibly work in Atlan:

• Possession alerts: Get notifications in Slack when there’s a change within the possession of an asset.
• Propagation and classification: If somebody marks a subject as PII in an upstream knowledge supply, robotically create a masking coverage to vary all associated fields downstream.
• And numerous extra use circumstances, equivalent to notifications round schema modifications on property in Atlan for knowledge engineering, login/logout occasions for safety groups, or triggering a Fivetran enrichment occasion to kick off an Atlan workflow.

Leverage Profiling and Reputation Insights to construct context and belief

Use new column-level profiling, recognition, and utilization metrics to evaluate knowledge’s high quality, discover essentially the most extensively used queries, establish high customers, and extra.

With the ability to profile your knowledge is highly effective. It builds provides the context and insights that folks want to grasp, use, and belief in knowledge, irrespective of whether or not they’re an information analyst or enterprise person.

That’s why a brand new Profiling function is now accessible inside Atlan. With a spread of recent metrics, customers all over the place can construct belief within the knowledge they’re consuming.

• Enterprise metrics: Common, most, minimal, imply, row rely, sum, and extra.
• Superior metrics: Duplicates, frequencies, lacking knowledge, uniqueness, customary deviation, and extra.

We’ve additionally added new Utilization and Reputation options to assist knowledge groups handle their property. Corporations can use these new metrics to save lots of prices on cloud knowledge warehouse spending, establish essentially the most used property, discover folks with essentially the most context on an information asset, and extra.

• Asset question logs: The variety of instances an asset has been queried (asset recognition), when it was final up to date, and so on.
• Person question logs: High customers, most up-to-date customers, who final queried an asset, and so on.
• Value and efficiency optimization logs: Which queries are slowest, which queries are costliest, and so on.

Atlan + GitHub: Allow knowledge contracts by bringing metadata into your knowledge creation course of

Carry energetic metadata proper to GitHub to reduce danger and enhance transparency earlier than any modifications are made to your knowledge.

An enormous dialogue these days is transferring knowledge “to the left” — that’s, transferring necessary knowledge processes and checks nearer to when property are created, moderately than when they’re distributed. This concept is a part of the knowledge contracts debate, which has highlighted how necessary it’s to enhance reliability and value between the information producers and customers.

Knowledge contracts can play a vital position in knowledge pipeline execution, validation of information varieties, versioning, and extra. As an illustration, if an information engineer makes modifications to a dbt mannequin, they may unknowingly be affecting tens or a whole bunch of individuals, tables, or dashboards. How can we decrease this danger whereas additionally offering transparency for the information engineer and knowledge client?

By bringing the facility of energetic metadata from Atlan to GitHub, knowledge engineers can now entry all of the context they should decrease the chance for knowledge customers.

Right here’s an instance of what this seems like: say that you simply’re an information engineer, and also you’ve created a pull request. When the GitHub motion runs, it should robotically create a listing of all downstream property that will likely be affected by this request — earlier than you make the change. From there, you possibly can attain out to customers upfront, or analysis and take a look at the property to see how they may be affected. Because the saying goes, “prevention is healthier than remedy”.

Automate metadata with Trident AI powered by GPT-3

Say goodbye to guide documentation with more and more clever automated descriptions, enterprise phrases, READMEs, and extra.

An current function in Atlan, Trident makes metadata enrichment enjoyable and straightforward by offering ideas for brand new descriptions, homeowners, phrases, and classifications. Whereas this has been extraordinarily profitable — in reality, one-third of all description updates on Atlan up to now had been made utilizing Trident — our prospects have been asking for extra. Extra AI and ML, that’s.

Introducing Trident AI, powered by GPT-3 — the powers of Trident mixed with the intelligence of GPT-3. Presently beneath growth, this function will assist use circumstances like:

• Creating descriptions for column names
• Creating descriptions for widespread enterprise phrases
• Writing a README for a enterprise time period

Earlier than, Trident would offer a suggestion for these things, which you might then apply for those who favored it. Now, with Trident AI, for those who don’t just like the suggestion, merely ask Trident AI for a change. Much like ChatGPT, Trident AI will present a extra sturdy advice primarily based on the facility of AI and GPT-3.

💡 If you wish to study extra, try the complete recording of Atlan Activate: Supercharged Automations to Map Your Complete Knowledge Property. You can too subscribe to our Product Updates e-newsletter for all the newest information.

💡 Prepared to begin utilizing these new options? Attain out to our Gross sales crew or your Buyer Success Supervisor to learn the way.