Data Source

Learn about data source.

What is a Data Source?

Data Source is any origin from which data is ingested into GlassFlow pipelines. Data Sources can be databases, streaming services, or real-time applications that generate events that need to be processed.

Ways to Connect to Data Sources

Python SDK

Use Python SDK to create a custom data source connector for your specific data source. Python SDK lets you send data from any data source to the GlassFlow pipelines in Python code.

• Example Use Case: You need to integrate with a data source that does not have a community connector or built-in GlassFlow integration.

Community Connectors

Community connectors are shared integrations developed and maintained by the GlassFlow community. These connectors are free to use and have open-source code hosted on the GlassFlow GitHub repo.

• Example Use Case: Connecting to a widely used data source with an existing community connector or code for the implementation.

Managed Connectors

Managed Connectors are fully managed services and provide seamless, out-of-the-box connectivity to common data sources. These integrations are fully supported and maintained by the GlassFlow team. These integrations make it easier to connect to data sources without infrastructure setup.

• Example Use Case: Integrating well-known services like Amazon S3, Google PubSub, or Azure Event Grid.