Abacus.AI allows you to attach datasets to your project from your own AWS S3 buckets.
- Go to the Abacus.AI Connected Services Dashboard.
- Click on the "Add New Connector" option and select "AWS S3" option. Enter your AWS S3 bucket address. Note that you do not need to include the sub folders address into the S3 URI during the bucket verification process even if your data resides within one of the sub-folders. For example, you must use the URI, "s3://your-bucket-aws-us-east-xyz" instead of "s3://your-bucket-aws-us-east-xyz/sub1/sub2".
- Click on the "Add Bucket" button.
- (Optional) Click on the "Write Permission" slider button to enable it if you want Abacus.AI to have write access to your bucket.
- Copy the bucket policy JSON to your clipboard.
- Go to your AWS S3 bucket and select the "Permissions" tab. Next, click on the "Bucket Policy" button and paste the copied JSON from your clipboard into the "Bucket policy editor" text box. Finally, click on the "Save" button.
- Go to the Abacus.AI Connected Services Dashboard.
- Find your S3 bucket listing from the list.
- Click on the "Verify" button.
- Your AWS S3 Bucket is now verified.
- If the bucket doesn't verify ("Verify" button still appears), please check if the Role Policy was saved successfully on the AWS S3 bucket you're trying to verify.
- Follow all the steps under "Connect your AWS S3 Bucket Using a Bucket Policy" to connect the bucket to Abacus.AI. But if your bucket uses a KMS encryption key, you must also grant Abacus.AI access to the key to decrypt the files.
- From the list of s3 buckets, select the KMS encrypted bucket and click on the "Properties" tab.
- Scroll down to the "Default encryption" section and click on AWS KMS key ARN link.
- Under the Key Policy section, click on "Edit". Now, go back to the connected services instruction dialogue box, scroll down, and copy the dictionary object under "Attach the Key Policy to your KMS Key" and paste it within the Key policy section as shown in the snapshot below:
- Your bucket is now ready to be read by Abacus.AI. You can follow the steps under "Verify Abacus.AI Bucket Policy for your AWS S3 Bucket" section above if you don't see a check mark for the verified option against your bucket.
To ensure secure access, you need to whitelist specific IP addresses for Abacus.AI based on the region you are using. Please reach out to support@abacus.ai to get the list of IPs to whitelist for your specific region.
If you need help bringing your data to Abacus.AI from your connected bucket(s), feel free to refer to the Fetch Data from File Connectors document.
