Glossary

This glossary provides definitions for key terms and concepts related to GlassFlow.

A

• API: The entry point for all requests to the GlassFlow platform via REST API, responsible for creating a GlassFlow account, authentication, and managing data pipelines.

B

• Batch Processing: The processing of data in large, discrete batches, typically at scheduled intervals.

C

• CLI (Command Line Interface): A tool provided by GlassFlow for users to interact with the platform through command line commands, enabling pipeline management and other operations.

• Custom Transformation Logic: User-defined Python code that specifies how data should be transformed as it moves through a pipeline.

D

• Data Pipeline: A configured sequence of operations in GlassFlow that processes and routes data from sources, through transformations, to destinations.

• Data Transformation: The process of converting data from one format or structure into another within a pipeline.

• Destination: The endpoint in a data pipeline where processed data is sent, such as a database or a cloud storage service.

E

• Event-Driven Architecture: A software architecture paradigm where operations are triggered by events or changes in state rather than by linear workflows.

F

• Function: In the context of GlassFlow, a Python script containing a handle a function that defines custom data transformation logic.

H

• Handle Function: The mandatory function within a transformation script that GlassFlow invokes to process data.

I

• Integration: The connection and communication between GlassFlow and external services or tools, enhancing the platform's capabilities.

L

• Logging: The recording of events and operations within GlassFlow, useful for monitoring and debugging pipelines.

M

• Message broker: In the context of GlassFlow, a message broker facilitates the efficient routing, processing, and delivery of data streams between sources and destinations within data pipelines, supporting various messaging patterns such as publish/subscribe, request/reply, and queuing.

N

• NATS Jetstream: An advanced, scalable messaging system integrated with GlassFlow for efficient data streaming and event handling.

O

• Organization: Represents a collective group or entity that encompasses multiple users, spaces, and pipelines. It serves as the primary account structure for managing access, projects, and resources within the GlassFlow platform, facilitating collaboration and resource sharing among team members.

P

• Pipeline Configuration: The process of defining the settings and operations of a data pipeline in GlassFlow, including its sources, transformations, and destinations.

• PostgreSQL: An open-source relational database system integrated with GlassFlow for data storage and management.

R

• Real-Time Data Processing: The analysis and processing of data immediately as it is received, enabling instant insights and actions.

S

• SDK (Software Development Kit): A set of tools and libraries provided by GlassFlow for developing applications and services that interact with the platform in Python.

• Serverless Architecture: A cloud computing execution model where the cloud provider dynamically manages the allocation of machine resources, scaling automatically to match the demands of the application.

• Source: The origin point in a data pipeline from which data is ingested, such as a web API, a database, or a message broker.

• Space: A workspace or environment in GlassFlow where related pipelines are organized and managed.

T

• Transformation Function: See Function.

Y

• YAML (YAML Ain't Markup Language): A human-readable data serialization standard used for configuring GlassFlow pipelines.