Real-time price recommendation

A practical example of creating a pipeline for real-time recommendation system using AI.

You will build a real-time price recommendation data pipeline to process car price data. The pipeline enriches data with predicted future prices using AI for the new vehicle registered in the primary PostgreSQL database.

Real-time car price recommendation pipeline with GlassFlow

Link to the GitHub project repository

We'll use the GlassFlow CLI to create a new space and configure the data pipeline.

Prerequisites

Make sure that you have the following before proceeding with the installation:

• You created a GlassFlow account.

• You installed GlassFlow CLI and logged into your account via the CLI.

• You have an OpenAI API account.

Installation

1. Clone the glassflow-examples repository to your local machine:

   git clone https://github.com/glassflow/glassflow-examples.git

2. Navigate to the project directory:

   cd use-cases/predict-car-price

3. Create a new virtual environment:

   python -m venv .venv && source .venv/bin/activate

4. Install the required dependencies:

   pip install -r requirements.txt

Steps to run the GlassFlow pipeline

1. Get a new OpenAI API Key

Create an API key to use OpenAI API endpoints.

2. Set OpenAI API Key

Open the transform.py file and replace {REPLACE_WITH_YOUR_OPENAI_API_KEY} with your API key.

3. Create a Space via CLI

Open a terminal and create a new space called examples to organize multiple pipelines:

glassflow space create examples

After creating the space successfully, you will get a SpaceID in the terminal.

4. Create a Pipeline via CLI

Use the GlassFlow CLI to create a new data pipeline inside the space.

glassflow pipeline create predict-car-price --space-id={space_id} --function=transform.py --requirements=openai

With --requirements=openai parameter to the CLI command, we specify which external libraries to use in the transformation function (Similarly how you define Python project dependencies in the requirements.txt file). You can also import other Python dependencies in the transformation function. See supported libraries with GlassFlow.

This command initializes the pipeline with the name predict-car-price in the examples space and specifies the transformation function transform.py. After running the command, it returns a new Pipeline ID with its Access Token.

5. Create an environment configuration file

Add a .env file in the project directory and add the following configuration variables:

PIPELINE_ID=your_pipeline_id
SPACE_ID=your_space_id
PIPELINE_ACCESS_TOKEN=your_pipeline_access_token

Replace your_pipeline_id, your_space_id, and your_pipeline_access_token with appropriate values obtained from your GlassFlow account.

6. Run data producer

Run the producer.py script to start publishing data:

python producer.py

7. Run data consumer

Run the consumer.py to retrieve transformed data from the pipeline:

python consumer.py