Predict API

Once your model is trained, you must deploy the model on Abacus.AI platform to generate predictions. You can use the prediction dashboard to generate the predictions from the trained model. In this section the underlying prediction API and all other additional prediction API methods are discussed for the use case in consideration:

Method
getChatResponse POST
Copy POST

Return a chat response which continues the conversation based on the input messages and search results.

Arguments:

REQUIRED KEY TYPE DESCRIPTION
Yes deploymentToken str The deployment token to authenticate access to created deployments. This token is only authorized to predict on deployments in this project, so it is safe to embed this model inside of an application or website.
Yes deploymentId str The unique identifier to a deployment created under the project.
Yes messages list A list of chronologically ordered messages, starting with a user message and alternating sources. A message is a dict with attributes: is_user (bool): Whether the message is from the user. text (str): The message's text.
No llmName str Name of the specific LLM backend to use to power the chat experience
No numCompletionTokens int Default for maximum number of tokens for chat answers
No systemMessage str The generative LLM system message
No temperature float The generative LLM temperature
No filterKeyValues dict A dictionary mapping column names to a list of values to restrict the retrieved search results.
No searchScoreCutoff float Cutoff for the document retriever score. Matching search results below this score will be ignored.
No chatConfig dict A dictionary specifying the query chat config override.
Note: The arguments for the API methods follow camelCase but for Python SDK underscore_case is followed.

Response:

KEY TYPE DESCRIPTION
success Boolean true if the call succeeded, false if there was an error
NlpChatResponse
KEY TYPE Description
deploymentConversationId str The unique identifier of the deployment conversation.
messages list The conversation messages in the chat.

Exceptions:

TYPE WHEN
DataNotFoundError

deploymentId is not found.
Language:

python

Request
client.getChatResponse(
    deployment_token='<deployment_token>',
    deployment_id='<deployment_id>',
    messages=[{'is_user': True, 'text': 'Recommend a funny horror movie'}]
)

Response
{
    'messages': [
        {
            'is_user': true,
            'text': 'Recommend a funny horror movie'
        },
        {
            'is_user': false,
            'text': 'My Big Fat Independent Movie is a spoof along the lines of \'Scary Movie\' and 'Not Another Teen Movie.\' It includes parodies of some
            of the indie film world's most renowned movies such as 'Memento, \' \'Pulp Fiction, \' \'Magnolia, \' \'My Big Fat Greek Wedding, \' \'Amelie, \' \'Run Lola Ru
            n, \' \'El Mariachi,\' \'The Good Girl,\' \'Pi,\' and \'Swingers.\''
        }
    ],
    'search_results': [
        {
            'msg_id': 0,
            'results': [
                {
                    'answer': 'A low-budget horror anthology made of 5 short films, each belonging to a different horror sub-genre. These segments are intro
                    duced by a ghoulish host of the sort that used to present late-night horror movies on TV. The five films are Pig, The Closet, Fall Apart, Meat Man, and Watc
                    her.',
                    'context': {
                        'id': '73408',
                        'imdb_id': 'tt1305112',
                        'title': 'Drive-In Horrorshow'
                    },
                    'score': 0.5060906410217285
                },
                {
                    'answer': 'Documentary about horror fans',
                    'context': {
                        'id': '234652',
                        'imdb_id': 'tt2430084',
                        'title': 'I Heart Monster Movies'
                    },
                    'score': 0.47290533781051636
                },
                {
                    'answer': 'Ten stories from horror's top directors. Ghosts, ghouls, monsters, and the devil delight in terrorizing unsuspecting resident
                    s of a suburban neighborhood on Halloween night. This creepy anthology combines classic Halloween tales with the stuff of nightmares.',
                    'context': {
                        'id': '306964',
                        'imdb_id': 'tt4163020',
                        'title': 'Tales of Halloween'
                    },
                    'score': 0.464328408241272
                }
            ]
        }
    ]
}
© Copyright 2025