Table of Contents

Amazon S3

Ramya Priya Updated by Ramya Priya

Connecting with Amazon S3

This documentation provides a step-by-step guide on how to connect to Amazon S3 from Tellius.

Prerequisites 

  • An Amazon Web Services (AWS) account
  • AWS Access Key ID and Secret Access Key
  • Tellius credentials

How to get AWS access key ID and secret access key

  1. Navigate to the IAM console at https://console.aws.amazon.com/iam/.
  2. From the navigation menu, select Users.
  3. Click on your specific IAM username (avoid clicking the checkbox).
  4. Under the Security credentials tab, opt to Create access key.
  5. Click Show to reveal your new access key. They should look like this:
    • Access key ID: <ACCESS_KEY_ID>
    • Secret access key: <SECRET_ACCESS_KEY>
  7. To save the key pair, click Download .csv file and keep the file in a secure place.
If a secret access key is misplaced, it cannot be retrieved, and a new key pair must be created.

How to connect with S3 from Tellius

Key-based access

To connect with Amazon S3 for the first time,

  1. Navigate to Data → Connect → Create new → Amazon S3.
  2. Provide the following details:
  • AWS Access Key - Enter the access key ID that was generated that was generated in the previous step.
  • AWS Secret Key - Enter the secret access key that was generated in the previous step.
  • S3 Bucket - Provide the Amazon S3 bucket name (globally unique name).
  • AWS Region - Choose the region from the dropdown. Locations may differ depending on the type of Cloud.
  • Datasource Name - Provide the name of the datasource
  1. Click on Save and Browse Host to proceed. The connection details will be saved.

To connect with Amazon S3 using an already existing datasource,

  1. Enable Use validated datasource connection details to use the details of the connections already established.
  2. A new field Select Datasource gets added.
  3. Click the dropdown to select from the list of already available connections.
  4. Based on the datasource selected, the fields will be auto-populated.
  5. Click on Browse Host to proceed.

Note: The dataset name can contain only letters, numbers, and underscores. It cannot begin with numbers or underscores or contain special characters, symbols, or spaces.

Role-based access

Role-based access control (RBAC) in AWS for Amazon S3 allows you to configure Tellius to securely access an S3 bucket without the need for access or secret keys. This approach is recommended and provides a secure way to access AWS resources.

To enable RBAC, you first need to identify the role associated with the nodes where Tellius is deployed.

EKS

If you are using EKS, the worker nodes of your EKS cluster will have an associated role. You can access this information using the EKS page on the AWS Console.

Standalone

If you have a standalone setup where Tellius is directly deployed on AWS EC2 instances, follow the steps outlined here to attach an IAM role to an EC2 instance:

Once you have identified the role associated with the Tellius deployment nodes, follow these steps to grant access to an S3 bucket:

  1. Sign in to the AWS Management Console and open the IAM console.
  2. In the navigation pane, select Policies and then click on the Create policy button.
  3. On the Create policy page, select the JSON tab to enter the policy document in JSON format.
  4. Enter the following JSON policy document, which provides read-only access to the S3 bucket:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": "arn:aws:s3:::YOUR_BUCKET_NAME"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:*"
            ],
            "Resource": [
                "arn:aws:s3:::YOUR_BUCKET_NAME",
                "arn:aws:s3:::YOUR_BUCKET_NAME/*"
            ]
        }
    ]
}

  1. Replace "YOUR_BUCKET_NAME" with the actual name of the S3 bucket to which you want to grant access. This policy allows the IAM role to perform "Get" and "List" actions on objects within the specified bucket.
  2. Click on the Review policy button to proceed.
  3. Provide a name and description for the policy, and review the policy summary.
  4. Click on the Create policy button to create the IAM policy.
  5. Once the policy is created, you can attach it to the IAM role associated with Tellius. To do this, navigate to the IAM Roles page, select the desired role, click on the Attach policies button, search for and select the policy you just created, and then click the Attach policy button.

Enhanced role-based access with detailed policy permissions

To provide a more granular level of control and understandability over the S3 access, here's an extended sample policy with an explanation for each of its attributes:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "s3:GetBucketLocation",
            "Resource": "arn:aws:s3:::*"
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": "s3:ListAllMyBuckets",
            "Resource": "arn:aws:s3:::*"
        },
        {
            "Sid": "VisualEditor2",
            "Effect": "Allow",
            "Action": [
                "s3:Get*",
                "s3:List*"
            ],
            "Resource": [
                "arn:aws:s3:::<BUCKET_NAME>/*",
                "arn:aws:s3:::<BUCKET_NAME>"
            ]
        }
    ]
}

Explanation

  • VisualEditor0 Statement:
    • Purpose: Grants permission to retrieve the location of the S3 bucket.
    • Effect: Decision attribute. Specifies whether the policy statement results in an allow or an explicit deny for the specified actions
    • Action: "s3:GetBucketLocation". Returns the region the bucket resides in.
    • Resource: Targets all S3 buckets.
  • VisualEditor1 Statement:
    • Purpose: Allows the listing of all S3 buckets.
    • Action: "s3:ListAllMyBuckets". Lists all buckets owned by the authenticated sender of the request.
    • Resource: Targets all S3 buckets.
  • VisualEditor2 Statement:
    • Purpose: Provides permission to get (read) and list (view) all objects and their metadata within a specified S3 bucket.
    • Action: "s3:Get*" and "s3:List*". These wildcard actions grant permissions for all Get and List related actions respectively.
    • Resource: Specifies which S3 bucket the actions apply to. Replace <BUCKET_NAME> with the actual name of your S3 bucket.
ARN stands for Amazon Resource Name. It is a unique identifier assigned to resources within the Amazon Web Services (AWS) ecosystem. An ARN provides a standardized format for identifying and referencing AWS resources across different services and accounts.

ARNs are structured as follows:

arn:aws:service:region:account-id:resource-type/resource-name

arn: This is the prefix that indicates it is an ARN.
aws:service: This specifies the AWS service that the resource belongs to, such as s3 for Amazon S3, ec2 for Amazon EC2, or lambda for AWS Lambda.
region: This indicates the AWS region where the resource is located, such as us-east-1 or eu-west-2.
account-id: This is the unique identifier for the AWS account that owns the resource.
resource-type/resource-name: These components vary depending on the service and resource being referenced. They identify the specific resource within the service.

By following these steps, you can configure Tellius to securely access the designated S3 bucket using the appropriate role-based access control.

When implementing this policy, ensure that the networking components, such as firewalls or VPC settings, do not block the connections required by these permissions. This is vital to ensure seamless access to the specified S3 resources.

Did we help you?

JDBC connector for PrestoDB

Time-to-Live (TTL) and Caching

Contact