Configure your persistent storage for your Kubernetes via AWS EBS Volumes

Configure AWS EBS Volumes for Kubernetes


This post covers how to setup a SSG site on AWS via S3, CloudFront, Route53, and ACM.

## S3 bucket

### Create the Bucket

First create the bucket

![Create Bucket](next-hosting/bucket-settings.png)

### Tags

Optional, but you can set tags. For tracking traffic per domain, it's a good idea to add a `domain` tag:

![Bucket Tags](next-hosting/tags.png)

### Policy

Second, edit the policy to be public

```json
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowPublicReadAccess",
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::yourdomain.com/*"
        }
    ]
}
```

## ACM Certificates

Before setting up CloudFront, we need SSL Certificates. You can obtain a free SSL certificate from the console [here](https://console.aws.amazon.com/acm/home?region=us-east-1#/wizard/)

### Create Domain

Enter your domain, and include `*.yourdomain.com` for wildcards, or any specific subdomains you'd like to use. 

![Create Domain](acm-certs/create-domain.png)

### DNS Validation

It's highly suggested to use Route53, and check DNS Validation for the validation method.

![Domain Name Validation](acm-certs/dns-validation.png)

Create and then choose "Create record in Route 53" to begin the DNS validation.

![Domain Name Validation](acm-certs/dns-validation-step.png)

Now just wait and the verification should be complete within a few minutes, potentially right away!

## CloudFront

### Create Distributions

Start by creating a distribution. You can create a distribution from the console [here](https://console.aws.amazon.com/cloudfront/home?region=us-east-1)

1. Choose Web
2. For "Origin Domain Name" set `yourdomain.com.s3.amazonaws.com`
3. Check "Redirect HTTP to HTTPS"
4. Decide your "Price Class"
5. Under "Alternate Domain Names (CNAMEs)" enter your domains each on separate lines (see below image)
6. Check "Custom SSL Certificate"
7. "Default Root Object" set to `index.html`

![Distribution Settings](cloudfront/distribution-settings.png)

Then click create!

### Set up Error Pages

While distribution is deploying, you can setup the error pages. Click on the distribution.

We will set this up once each for 403 and 404 errors:

1. Choose 403 or 404 for "HTTP Error Code"
2. Choose "Customize Error Response"
3. Set "Response Path Path" `/404.html`
4. Choose 200 for "HTTP Response Code"

![Error page](cloudfront/error-pages.png)

## Your Next.js Package

Update your `package.json`'s `scripts` object to contain the deploy info:

```json
  "scripts": {
    "dev": "next dev",
    "build": "next build",
    "start": "next start",
    "export": "next build && next export",
    "deploy": "AWS_PROFILE=myprofile1 aws s3 sync out/ s3://yourdomain.com",
    "invalidate": "AWS_PROFILE=myprofile1 aws cloudfront create-invalidation --distribution-id EEEEEEEKJCTPFA --paths \"/*\"",
    "paths": "AWS_PROFILE=myprofile1 ./deploy/prepare.sh",
    "deploy:all": "npm run export && npm run deploy && npm run paths"
  }
```

### CloudFront routing fix

Do make sure you can route to `/some-route` and under the hood, cloudfront will be able to find the `/some-route.html`, you need to setup a script.

Make sure to have this script in your `deploy/prepare.sh` (and also `chmod +x` it)

```sh
#!/bin/bash

export S3_BUCKET=yourdomain.com

(cd out &&
  find . -type f -name '*.html' | while read HTMLFILE; do
    HTMLFILESHORT=${HTMLFILE:2}
    # HTMLFILE_WITHOUT_INDEX=${HTMLFILESHORT::${#HTMLFILESHORT}-11}

    HTMLFILE_WITHOUT_INDEX=${HTMLFILESHORT//index.html/}
    HTMLFILE_WITHOUT_HTML=${HTMLFILE_WITHOUT_INDEX//.html/}


    # cp /about/index.html to /about
    aws s3 cp s3://$S3_BUCKET/${HTMLFILESHORT} s3://$S3_BUCKET/$HTMLFILE_WITHOUT_HTML
    echo aws s3 cp s3://$S3_BUCKET/${HTMLFILESHORT} s3://$S3_BUCKET/$HTMLFILE_WITHOUT_HTML

    if [ $? -ne 0 ]; then
      echo "***** Failed renaming build to $S3_BUCKET (html)"
      exit 1
    fi
  done)
```

## Route53

### DNS Records

Setup the root record first

1. Create an "A Record" (leave name blank for the root record)
2. Choose yes for "Alias"
3. Find your CloudFront distribution and link to it

Then setup the www and cname

1. Choose "CNAME"
2. Enter "www" for the name
3. Enter "yourdomain.com" for the Value

When finished you should have these two records at a minimum:

```
yourdomain.com.
A
ALIAS d12mkko9k4qnnn.cloudfront.net. 
No
-

www.yourdomain.com.
CNAME
yourdomain.com
```

## AWS Profiles

Make use of AWS Profiles

Inside of `~/.aws/credentials`

```
[default]
aws_access_key_id = REDACTED
aws_secret_access_key = REDACTED

[myprofile1]
aws_access_key_id = REDACTED
aws_secret_access_key = REDACTED
[myprofile2]
aws_access_key_id = REDACTED
aws_secret_access_key = REDACTED
```

Inside of `~/.aws/config`

```
[default]
region = us-east-1
[profile myprofile1]
output = json
region = us-east-1
[profile myprofile2]
output = json
region = us-east-1
```

### Sources

[https://medium.com/@omgwtfmarc/deploying-create-react-app-to-s3-or-cloudfront-48dae4ce0af](https://medium.com/@omgwtfmarc/deploying-create-react-app-to-s3-or-cloudfront-48dae4ce0af)

[https://medium.com/@serverlessguru/deploy-reactjs-app-with-s3-static-hosting-f640cb49d7e6](https://medium.com/@serverlessguru/deploy-reactjs-app-with-s3-static-hosting-f640cb49d7e6)

[https://medium.com/fredwong-it/aws-cloudfront-url-return-accessdenied-in-front-of-s3-react-app-a4bad7d3e4f2](https://medium.com/fredwong-it/aws-cloudfront-url-return-accessdenied-in-front-of-s3-react-app-a4bad7d3e4f2)

[https://github.com/jariz/gatsby-plugin-s3/blob/master/recipes/with-cloudfront.md](https://github.com/jariz/gatsby-plugin-s3/blob/master/recipes/with-cloudfront.md)


Setting up static sites on AWS S3 and CloudFront

Deploy Next.js to AWS on S3 and CloudFront


## Standard 

```yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: standard
provisioner: kubernetes.io/aws-ebs
reclaimPolicy: Delete
allowVolumeExpansion: true
parameters:
  type: gp2
```

## IOPS Storage

```yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: iops
provisioner: kubernetes.io/aws-ebs
reclaimPolicy: Delete
allowVolumeExpansion: true
parameters:
  type: io1
```

## Cold Storage

```yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: cold
provisioner: kubernetes.io/aws-ebs
reclaimPolicy: Delete
allowVolumeExpansion: true
parameters:
  type: sc1
```

## Example usage

```yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  namespace: myapp
  name: myapp-data-dir-claim
spec:
  volumeMode: Filesystem
  storageClassName: iops
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 4Gi
```


This assumes that you are using Traefik as a load balancer and handles your SSL traffic automatically. I'll be following up with a post and will link to it for how to set up Traefik.

# add prerequisite files

First, create a folder `./generate`. Inside create a file `generate-registry-config.sh`:

```sh
#!/bin/bash

export SHA=$(head -c 16 /dev/urandom | shasum | cut -d " " -f 1)
export USERNAME=user$(head -c 16 /dev/urandom | shasum | cut -d " " -f 1)

echo $USERNAME > registry-creds.txt
echo $SHA >> registry-creds.txt

docker run --entrypoint htpasswd registry:2.7.0 -Bbn $USERNAME $SHA > ./htpasswd

# https://github.com/helm/charts/tree/master/stable/docker-registry

helm template webinc-reg stable/docker-registry \
  --namespace default \
  --set persistence.enabled=true \
  --set persistence.accessMode=ReadWriteOnce \
  --set persistence.storageClass=standard \
  --set persistence.size=24Gi \
  --set service.type=NodePort \
  --set secrets.htpasswd=$(cat ./htpasswd) \
  > registry-source-from-helm.yaml
```

# Add stable to helm repos

```
helm repo add stable https://charts.helm.sh/stable
help repo update
```

# setup registry

inside of `./generate` 

1. run `./generate-registry-config.sh`, this will generate `registry-source-from-helm.yaml`

2. I prefer to copy/paste everything out into the different pieces out, pvc, service, deployment, config...

3. Make sure you have storage configured. Read up on that [here](/post/ebs-volume-configuration)

4. now deploy the individual files via `kubectl`

## ingress

Add ingress for traefik

```yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: docker-yourdomain-com-ingress
  namespace: webinc
  annotations:
    kubernetes.io/ingress.class: traefik
spec:
  rules:
    - host: docker.yourdomain.com
      http:
        paths:
          - path: /
            backend:
              serviceName: webinc-reg-docker-registry
              servicePort: registry

```

# authentication, etc.

Delete and store the registry-creds.txt ASAP, it's a secret. But get is secured first.

test it out: https://docker.yourdomain.com/v2/_catalog

cool, works.

Now, how to get this going...

## creating secrets

Caveat: you may have to do this for multiple namespaces... 

```
kubectl delete secret -n webinc regcred
kubectl create secret -n webinc \
 docker-registry regcred \
    --docker-server=$REGISTRY_DOMAIN \
    --docker-username=$REGISTRY_USER \
    --docker-password=$REGISTRY_PASSWORD
```

## logging in

```
docker login docker.yourdomain.com
```

## pushing an image up

You may need to tag first, but once you do:

```
docker push docker.yourdomain.com/username/yourimage:latest
```

## kubernetes configs

Add this to any containers

```yaml
 containers:
   - name: new-private-container
   ...
 imagePullSecrets:
   - name: regcred
```

### Sources

[https://github.com/alexellis/k8s-tls-registry](https://github.com/alexellis/k8s-tls-registry)

Build a secured private docker registry with persistent storage on your kubernetes cluster