Skip to main content

SharePoint Connector Documentation

If you have already set up a connector, skip to How to Use the SharePoint Connector.

Connect your SharePoint to Abacus.AI​

To integrate SharePoint with Abacus.AI, you need to set up the connector and provide the necessary permissions.

Setting Up the SharePoint Connector​

Flow 1: App Based Setup​

  1. Before setting up the SharePoint Connector, you need to provide admin consent or register the Abacus.AI application.

    Go to the Admin Consent URL and grant consent or register the application.

  2. Access Connected Services Dashboard:

  3. Add New Connector:

    • Click on the "Add New Connector" option, select "SharePoint" from the list, and choose "App Based" as the authentication type.

  4. Enable Realtime Synced Datasets (Optional):

    • Toggle the "Enable Realtime Synced Datasets" option to Yes if you want to sync datasets in real time.

  5. Enable Multi-Site Selection (Optional):

    • If you want to pull data from multiple SharePoint sites, toggle the "Pull from Multiple Sites" option to Yes.
    • Enter the SharePoint domain in the "SharePoint Domain" field. The domain must end with sharepoint.com.
    • Use the + button to add multiple site names or enter them as comma-separated values in the "Sites" field.

    Example:

    • If your SharePoint domain is https://example.sharepoint.com, and you want to pull data from two sites, enter: 
      site1, site2
SharePoint Multi-site

  1. Granting Permissions to the Client ID

    After setting up the connector, you will see the client ID. You need to provide permission for your site to this client ID:

    1. Go to https://subdomain.sharepoint.com/sites/name-of-your-site/_layouts/15/appinv.aspx.
    2. Paste the client ID in place of App Id then click on the lookup button to fetch the information for the client ID.
    3. In the App Domain field, type abacus.ai.
    4. In the Permission Request XML field, put the following XML code and click on create:
        <AppPermissionRequests AllowAppOnlyPolicy="true">
          <AppPermissionRequest Scope="http://sharepoint/content/sitecollection" Right="Read"/>
        </AppPermissionRequests>

  2. A new page will load, click Trust It to grant the permission.

    NOTE: If you are not able to lookup the client id or you get the following error message when you try to verify your SharePoint connector,

    *Invalid Connector Configuration: Failed to verify SharePoint connector due to bad SharePoint connection. SharePoint response: {"error":"invalid_request","error_description":"Token type is not allowed."}

    You need to ask your administrator to configure settings in your SharePoint by running the following commands in a Windows OS/VM:

        Install-Module -Name Microsoft.Online.SharePoint.PowerShell
        Import-Module Microsoft.Online.SharePoint.PowerShell
        Connect-SPOService -Url https://<orgName>-admin.sharepoint.com
        Set-SPOTenant -DisableCustomAppAuthentication $false
        Set-SPOTenant -SiteOwnerManageLegacyServicePrincipalEnabled $true

Please replace <orgName> with your actual organization name in the SharePoint admin URL.

  1. Verify Connector Status:
    • Your connector should be set up and confirm that it has an "ACTIVE" status.

Flow 2: Username and password Based Setup​

  1. Before you begin, create a new SharePoint user and give it access only to the data that you want to pull from SharePoint.

    This step ensures that the connector has the necessary permissions to access the specific folder you want to pull data from.

    Note: You will need to create a password for the SharePoint user and not use the temporary password provided for the service account upon initial creation.

  2. Access Connected Services Dashboard:

  3. Add New Connector:

    • Click on the "Add New Connector" option, select "SharePoint" from the list, enter your site URL.

  4. Select "Username + Password" from the 'Authentication Type' dropdown. Provide the new user's Microsoft email and password, then click Create.

Sharepoint User Auth

  1. Verify Connector Status:

    • Your connector should be set up and confirm that it has an "ACTIVE" status.

  2. No Additional Permissions Required: With this flow, you do not need to grant additional permissions to the client ID.

Flow 3: OAuth Based Setup​

  1. Before setting up the SharePoint Connector with OAuth, ensure that you have the necessary permissions to generate an OAuth token for accessing your SharePoint site.

  2. Access Connected Services Dashboard:

  3. Add New Connector:

    • Click on the "Add New Connector" option, select "SharePoint" from the list, enter your site URL and click on tick mark beside the site URL.

  4. Select "OAuth" from the 'Authentication Type' dropdown.

  5. Click on Connect Sharepoint button to create and save the connector.

Flow 4: Personal Service Principal Setup with certificates​

  1. Select Authentication Type:

    • Go to the Abacus.AI Connected Services Dashboard. You can click your profile in the top right and then click "Manage Connectors".
    • Click on the "Add New Connector" option, select "SharePoint" from the list, and choose "Personal Service Principal" as the authentication type.

  2. Enter Client ID:

    • Provide the Client ID for your Personal Service Principal.

  3. Optional Configurations:

    • If you want to pull data from multiple SharePoint sites, toggle the "Pull from Multiple Sites" option to Yes.
    • Note: The "Enable Realtime Synced Datasets" option is only available for the App Based flow and is not applicable here.

  4. Save and Verify:

    • Click "Save". A popup box will appear with a certificate. Download this certificate as it will be required for the next steps.

  5. Grant Permissions in Microsoft:

  6. Upload the Certificate:

    • Follow the instructions provided in the popup box to upload the certificate and complete the verification process.

  7. Verify Connector Status:

    • Once the permissions are granted and the certificate is uploaded, the connector should be set up and display an "ACTIVE" status.

Flow 5: Custom OAuth​

  1. Access Connected Services Dashboard:

  2. Add New Connector:

    • Click on the "Add New Connector" option, select "SharePoint" from the list, enter your site URL and click on tick mark beside the site URL.

  3. Select "Custom OAuth" from the 'Authentication Type' dropdown.

  4. Use your own OAuth app and grant it access to read data.

  • Copy the app’s Client ID and Client Secret, then enter them into the Client ID and Client Secret fields. Note: Please refer to microsft documentaiton to create the app and add necessary scopes to app.
  1. Click on Connect Sharepoint button to create and save the connector.

User connector flow​

  1. Turn Config Only toogle ON:
  • Go to the Abacus.AI Connected Services Dashboard. You can click your profile in the top right and then click "Manage Connectors".
  • Click on the "Add New Connector" option, select "SharePoint" from the list, and turn "Config Only" toggle on.
Config Only
  1. Select Authentication Type:
  • Select "Basic" if you want to use Abacus.AI app for Sharepoint user connector.
  • Select "Custom Auth" if you want to use your own app with Sharepoint User connector.
Config Only Authentication
  1. Enter credentials:
  • Enter Sharepoint site URL
  • Enter client ID and secret if you have selected "Custom Auth".

Note: Following permissions are requird for user connector. Please note that all the permissions are delegated permissions:

Graph API scopes

  • Directory.Read.All: Read directory data
  • email View users: email address
  • Files.Read.All: Read all files that user can access
  • offline_access: Maintain access to data you have given it access to
  • openid: Sign users in
  • profile: View users' basic profile
  • Sites.Read.All: Read items in all site collections
  • Sites.Selected: Access selected Sites, on behalf of the signed-in user
  • User.Read: Sign in and read user profile

SharePoint Scopes:

  • AllSites.Read: Read items in all site collections
  • MyFiles.Read: Read user files
  • Sites.Search.All: Run search queries as a user
  • Sites.Selected: Access selected site collections
  • User.Read.All :Read user profiles

Navigate to Sharepoint user connector public documentation to create a user connector.

How to Use the SharePoint Connector​

Once the SharePoint connector is set up, you can fetch data to train models in Abacus.AI.

  1. Create a new project and select the use case, then go to the "Datasets" tab and click "Create Dataset".
Create Dataset
  1. Click on "Create New".
Create New Dataset
  1. Name the dataset, select the data type 'List of documents', and click "Continue".
Name Dataset

  1. Choose "Read from External Service" and select your SharePoint connector under "Connected Application Connectors".

  2. Enter the Site URL for the SharePoint site you want to use, and specify the folder path if you have set up the connector to pull from a specific folder.

  3. After the dataset is uploaded, configure the schema mapping and proceed to train models with the data.

Train Model with Data

Supported Data Types​

  • Supported Data Types: The SharePoint connector supports ingesting documents and files. Documents and files are treated as a list of documents, suitable for unstructured data use in ChatLLM projects.

Using SharePoint Data with LLMs​

  • ChatLLM Use Case: SharePoint documents, such as PDFs, Word files, and other unstructured data, are ideal for conversational AI use cases. These documents are ingested as a list of documents and processed into a Document Retriever for retrieval-augmented generation (RAG).

Feature Group Types​

  • List of Documents: Used for unstructured data, such as uploaded documents, in ChatLLM projects. These feature groups require processing into a Document Retriever for retrieval-augmented generation (RAG).

By combining both feature group types, you can handle both unstructured and structured data effectively in a single project.

Permissions and Access​

When setting up the connector, the following application level permissions are requested: (Please grant admin consent to the AbacusAI_Sharepoint_Connector application to grant application-level permissions)

  • Files.Read.All: Read files in all site collections
  • Sites.Read.All: Read items in all site collections
  • Sites.Selected: Access selected site collections
  • Directory.Read.All: Read directory data (Optional: Required for permission aware sharepoint connector)
  • GroupMember.Read.All: Read all group memberships (Optional: Required for permission aware sharepoint connector)
  • User.Read.All: Read all users' full profiles (Optional: Required for permission aware sharepoint connector)
  • User.ReadBasic.All: Read all users' basic profiles (Optional: Required for permission aware sharepoint connector)

We also request for following delegated permisisons:

  • Files.Read.All: Read all files that user can access
  • Sites.Read.All: Read items in all site collections
  • User.Read: Sign in and read user profile
  • Sites.Selected: Access selected Sites, on behalf of the signed-in user
  • AllSites.Read: Read items in all site collections
  • MyFiles.Read: Read user files
  • Directory.Read.All: Read directory data (Optional: Required for permission aware sharepoint connector)

These permissions allow the connector to pull necessary data for analysis and model training.

Troubleshooting and FAQ for the SharePoint connector​

Can I use multiple SharePoint connectors in Abacus.AI?

Yes, you can set up multiple SharePoint connectors in Abacus.AI, each with different configurations and permissions.

What's the right feature mapping schema that will allow documents to use the URLs of the original file location on the source page in the external chat portal?

To get the URLs to the original location of a file on the source page in the chatweb, you should set the URL as "Document source" instead of the file_path which is set as a document source by default.