Data Sink

Learn about data sink.

What is a Data Sink?

A Data Sink in GlassFlow is the destination where processed data is sent after passing through the transformation function in your pipelines. This can be a dashboard application displaying real-time analytics, a machine learning platform consuming enriched data for predictions, a reporting tool generating periodic reports based on processed data, or storage where you want to store, analyze, or visualize the processed data.

Ways to Connect to Data Sinks

Python SDK

Using Python SDK to create custom data sinks to specify exactly how and where the data should be sent after processing. Python SDK lets you send data to any data sink from the GlassFlow pipelines.

• Example Use Case: You need to send processed data to a data sink that does not have a community connector or GlassFlow built-in integration.

Community Connectors

Community connectors for data sinks are integrations developed and shared by the GlassFlow community. These connectors are free to use and have open-source code hosted on the GlassFlow GitHub repo.

• Example Use Case: Exporting data to a community-supported data warehouse or BI tool.

Managed Connectors

Managed Connectors are fully managed services and provide seamless, out-of-the-box connectivity to common data sinks. These integrations are fully supported and maintained by the GlassFlow team. These integrations make it easier to connect to destinations without infrastructure setup.

• Example Use Case: Exporting data to Amazon S3, Google BigQuery, or SQL Database.