Abacus Usage Metrics

Abacus Usage Metrics provide a comprehensive view of chatbot interactions, capturing various aspects such as conversation details, user feedback, and performance metrics.

Steps to Set Up Abacus Usage Metrics

Step 1: Navigate to the Dataset Creation Page

1. Create a new project and select the use case, then go to the "Datasets" tab and click "Create Dataset".

2. Click on "Create New".

3. Name the dataset, select the data type 'Tabular Data', and click "Continue".

Step 2: Add New Application Connector

1. Click on the Add New Application Connector button.
2. Select the tile for Abacus Usage Metrics.

Step 3: Provide Connector Details

1. Provide a name for the connector.
2. Click on the Save button.

Step 4: Configure the Connector

1. Click on the tile with the name given in Step 3.
2. Select the toggle options if desired: - Show entire conversation history: Includes all messages and responses in the conversation. - Show all feedback: Includes all feedback reported for the conversations.
3. Click on the Add Dataset button.

Using Abacus Usage Metrics

Once the dataset is created, you will have a Feature Group that you can perform transformations on using SQL or Python.

Utilize the Plots under the EDA (Exploratory Data Analysis) section for visualizations. Below are suggestions on how to glean insights from the usage data.

Analyze Conversation Patterns

1. Use the num_messages feature to analyze the length of conversations.
2. Examine conversation_history and first_message to understand the content and flow of conversations.

Track User Feedback

1. Utilize feedback-related features such as feedback_categories, last_feedback, last_feedback_type, and all_feedback to analyze user satisfaction.
2. Identify areas for improvement based on feedback trends.

Monitor Topic Popularity

1. Analyze topic-specific NUMERICAL features to track how often certain topics or documents are referenced.
2. Use this information to optimize content and improve chatbot responses.

Evaluate Performance Metrics

1. Use conversation_created_at and conversation_updated_at to analyze response times and conversation durations.
2. Identify patterns and areas for performance improvement.

Conduct Multi-dimensional Analysis

1. Leverage features such as project_id, deployment_id, and chatbot_name to analyze data across different chatbots, deployments, and projects.
2. Compare performance and usage metrics across various configurations.