Forecasting and Planning

Overview

Choose this use-case if you wish to develop a model that analyzes historical demand data for a product or service to predict future demand patterns. Given a dataset of historical demand data for your good or service and an optional secondary dataset of all relevant item attributes such as product information, promotional history, or other external factors, you can generate a model that generates an accurate and comprehensive demand forecast.

To follow this tutorial, ensure you have followed these steps:

1. Log in to the Abacus developer platform
2. Create a new project of type "Forecasting and Planning"
3. Provide a name, and click on Skip to Project Dashboard.

If you are having trouble creating a new project, follow this guide

Example use cases would be:

• Retail inventory management and stock optimization
• Supply chain planning and warehouse management
• Budget planning for procurement departments

We leverage both open source as well as internally developed algorithms:

• Tree Based (XGBoost, LGBM, CatBoost)
• Deep Learning (LSTM, Tranformer Based)
• Statistical (ARIMA, SARIMA, ETS)

Abacus will automatically create lag features, moving averages, and other relevant time-series features to enhance the forecasting model's accuracy. The user is free to also create his own lag features, but that is not a requirement and not recommended.

Video Tutorial

Steps to Train a Model

Step 1: Ingest data into the platform

Once you are ready to upload data, follow these steps:

1. Click on Datasets within the project page on the left side panel.
2. Click on Create Dataset on the top right corner.
3. IMPORTANT: Provide a name and choose the Forecasting Timeseries Data Feature Group type

Step 2: Configuring Feature Groups

Once the dataset has finished inspecting, navigate to the Feature Group area from the left panel. You should be able to see a Feature Group that has the same name as the dataset name you provided.

1. Navigate to Feature group → Features
2. Select the appropriate feature group type (see Required Feature Groups)

To learn more about how feature group mapping works, visit our Feature Group mapping guide for detailed configuration instructions.

Step 3: Model Training

You are now ready to train your model. Create a new model by navigating to the models page:

Select your training Feature Group, and start the training process.

After you click on train:

• Select your training feature group(s) — Choose the datasets you want to use for training
• Configure training options — We recommend keeping the default values, but you can customize settings based on your needs. Click the (?) icon next to any option to see what it does
• Monitor training progress — Your model will begin training automatically. You can track its status in real-time

For use cases with ground truth data, you'll be able to evaluate model performance through the Metrics page, accessible directly from your model's dashboard.

Step 4: Deploying your model

Once the model is ready, click on the Deploy button within the model's page.

• Offline Deployment Mode: Select this option when you want the model to be available for batch processing.
• Online Deployment Mode: Select this option when you want the model to be available to use via the API

Here are some of the important options you will be able to see within the Deployment page of a model:

• Prediction API: Clicking on this button will provide you with a sample API request to call this model externally.
• Switch Version/Algorithm: This option allows you to change the model version or algorithm used for predictions.
• Auto Deployment: Whether model retrains will also auto-deploy the newest version of the model.

To learn more about Deployments, visit our Deployments Guide

Your model is now ready to use!

Batch Predictions

Batch predictions is the process of using a deployed model to make predictions on a batch of input data. It's how you get predictions out of all Machine Learning Models.

The process is as follows, and is applicable for all machine learning project types:

1. Upload data, ensuring that it is in the same format as the model training data
2. Navigate to Batch Predictions --> Create New Batch Prediction from the left side panel

3. Follow the wizard
   • Select the deployment (This is the model that will be used to make predictions)
   • Select the input Feature Group to the batch prediction
   • Select the output Feature Group for the batch prediction

Abacus will now use the deployed model you selected to create predictions. The output Feature Group will have columns with the predicted values. If you re-run the same batch prediction, a new version of Feature Group will be created with the new output.

To learn more about Batch Predictions, visit our Batch Predictions Guide

Batch Predictions on Time Series Data

When making batch predictions on time series data, you need to provide future values for any features that are marked as FUTURE. These are features that you know in advance for future time periods. For example, it could be promotional events.

Let's walk through an example. Suppose you trained a forecasting model using the following historical data:

Training Data (Historical Time Series)

ITEM_ID	DATE	SALES	INVENTORY	N_VISITORS	PRICE	PROMOTIONAL_EVENT
AAA	1/4/22	14	3	31	4	0
AAA	1/5/22	15	4	2	5	1
AAA	1/6/22	1	1	1	1	1

In this example:

• SALES is the target variable (what you want to predict)
• INVENTORY and N_VISITORS are historical features only
• PRICE and PROMOTIONAL_EVENT are marked as future features (known in advance)

To make predictions for future dates (1/7/22 through 1/11/22), you need to provide the input data in the following format:

Input Data for Batch Prediction

ITEM_ID	DATE	SALES	INVENTORY	N_VISITORS	PRICE	PROMOTIONAL_EVENT
AAA	1/4/22	14	3	31	4	0
AAA	1/5/22	15	4	2	5	1
AAA	1/6/22	1	1	1	1	1
AAA	1/7/22				3	0
AAA	1/8/22				14	1
AAA	1/9/22				15	1
AAA	1/10/22				12	0
AAA	1/11/22				12	1

Notice that:

• We pass both the historical data (actuals), as well as future data.
• SALES is left empty (this is what the model will predict)
• INVENTORY and N_VISITORS are left empty (historical features are not needed for future predictions)
• PRICE and PROMOTIONAL_EVENT must be provided with their known future values

After running the batch prediction, the output Feature Group will contain the predicted SALES values:

Output Data from Batch Prediction

ITEM_ID	DATE	prediction_p10	prediction_p50	prediction_p90
AAA	1/7/22	12.5	16.2	21.8
AAA	1/8/22	14.2	18.5	24.3
AAA	1/9/22	13.8	17.8	23.1
AAA	1/10/22	11.9	15.3	19.7
AAA	1/11/22	11.5	14.9	19.2

Key Points:

• Only provide values for features marked as "FUTURE" or "known in advance" during model training
• Leave the target variable (SALES) empty - this is what you want to predict
• Historical-only features should be left empty for future dates

In the scenario where no features as marked as FUTURE, you would only need to provide the historical data, and then choose predict on future:

ITEM_ID	DATE	SALES	INVENTORY	N_VISITORS	PRICE	PROMOTIONAL_EVENT
AAA	1/4/22	14	3	31	4	0
AAA	1/5/22	15	4	2	5	1
AAA	1/6/22	1	1	1	1	1

Required Feature Group Types

To train a model under this use case, you will need to create feature groups of the following type(s):

Feature Group Type	API Configuration Name	Required	Description
Forecasting Timeseries Data	HISTORICAL_DEMAND	True	Dataset containing historical demand data with timestamps, showing demand patterns over time for products or services.
Item Attributes	ITEM_ATTRIBUTES	False	Optional dataset containing product information, attributes, and metadata to enhance forecast accuracy.

Note: Once you upload the datasets under each Feature Group Type that comply with their respective required schemas, you will need to create Machine learning (ML) features that would be used to train your ML model(s). We use the term, "Feature Group" for a group of ML features (dataset columns) under a specific Feature Group Type. Our system support extensible schemas that enables you to provide any number of additional columns/features that you think are relevant to that Feature Group Type.

Feature Group: Forecasting Timeseries Data

This dataset contains historical demand data showing demand patterns over time.

Feature Mapping	Feature Type	Required	Description
ITEM_ID		Y	Unique identifier for each product or service
DATE		Y	Time period for the demand observation
DEMAND		Y	Demand quantity or value for the time period
FUTURE		N	Any features that are known in advance
[ADDITIONAL DEMAND ATTRIBUTES]		N	Any additional attributes related to demand measurements

Feature Group: Item Attributes

Optional dataset containing product information to enhance forecasting.

Feature Mapping	Feature Type	Required	Description
ITEM_ID		Y	Unique identifier for each product or service
[ITEM ATTRIBUTES]		N	Any relevant item attributes such as category, brand, price, seasonality type, product lifecycle stage, etc.

Model Evaluation

Once you train a model, you would be able to evaluate the performance of the model from the "Model Metrics" page.

Predictions

For any deployed model within the abacus platform, you can leverage an API to call it externally. The steps to do this are:

1. Deploy the Model
2. Navigate to the Deployments Page
3. Click on the Predictions API button on the left side

That will give you the exact API endpoints and tokens you need to use the deployment.

The Relevant API References for this use case are:

• Prediction API