Connecting with AWS storage

See how you can add your AWS credentials to connect your own AWS S3 buckets to your datasets on Mindkosh.

To connect your AWS account with Mindkosh and add data, you first need to add your AWS credentials.

When connecting your own Cloud service account with Mindkosh, please note that data egress charges may be applicable on your Cloud storage account. Please check with your cloud provider for more details.

Create an IAM user on AWS

Create an IAM user on AWS, and attach a policy with read access rights over the bucket and location you intend to use. When asked to choose the AWS credential type. select Access key - Programmatic access. We do not require AWS console access.

An example policy you can attach to the new user is given below. Be sure to replace example-bucket with your own bucket name.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetBucketLocation"
            ],
            "Resource": [
                "arn:aws:s3:::<example-bucket>"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::<example-bucket>/*"
            ]
        }
    ]
}

Remember to save the API Access keys

Once you have created a new user, you will be prompted to save the Access keys you just created. Make sure to click on the "Download .csv" button. If you close the tab, and reopen it, you will not be able to view the Secret access key required to access data from your bucket.

Set CORS policy

In order to enable the browser to directly fetch the files from your cloud storage, you will need to set the right CORS policy on the bucket, so the browser does not block loading the files. Here is how you can do it for a bucket on AWS S3.

1. Open the bucket on the AWS console.

2. Switch to the Permissions tab.

3. Scroll down to the CORS section and enter the following policy.

[
    {
        "AllowedHeaders": [
            "Access-Control-Allow-Origin"
        ],
        "AllowedMethods": [
            "GET"
        ],
        "AllowedOrigins": [
            "https://app.mindkosh.com",
        ],
        "ExposeHeaders": [
            "Access-Control-Allow-Origin"
        ]
    }
]

Add credentials to Mindkosh

Only organization admins can add cloud storage credentials on Mindkosh

With the permissions setup, we are now ready to add the credentials to Mindkosh. To add cloud storage credentials, go to the Organization page from the left sidebar on Mindkosh.

Click on the Manage Keys button in the AWS S3 section, and enter the following keys in the appropriate boxes:

1. Access key ID

2. Secret Access key

Once the credentials have been setup, you can create a dataset to add data from your storage. Checkout the steps mentioned here to do this.