neonvm: add support for mounting service account tokens

[conradludgate]

Service account tokens can be created by kubernetes and automatically mounted to the pod using a projected volume: https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#launch-a-pod-using-service-account-token-projection.

In EKS and AKS, there is a separate mechanism that watches pods for their service accounts, and adds a projection volumemount if that service account has an IAM policy in the annotations. https://docs.aws.amazon.com/eks/latest/userguide/associate-service-account-role.html#_step_2_create_and_associate_iam_role. However, it's also possible to manually configure this volume.

Using the sync mechanism, we can then mirror those into the VM.

[github-actions]

No changes to the coverage.

HTML Report

Click to open

[conradludgate]

Hmm. This doesn't yet work for accessing the aws-iam-token volume mount which is automatically created for us. I'll have to think more for a potential solution. Ideally we'd have some way to encode that "this volumemount already exists, but please watch it anyway". Maybe a "none" disk?

Additionally, if the VM wants access to the kubernetes API (why??) then we would need to somehow get the local kubernetes API endpoint into the VM - I imagine we don't need this

---

EDIT (2025-04-28)

If I remember correctly, I tested creating a pod with the aws-iam-token service account volume manually, which did work as expected.

Example volumes:

  volumes:
    - name: aws-iam-token
      projected:
        sources:
          - serviceAccountToken:
              audience: sts.amazonaws.com
              expirationSeconds: 86400
              path: token
        defaultMode: 420

  volumes:
    - name: azure-identity-token
      projected:
        sources:
          - serviceAccountToken:
              audience: api://AzureADTokenExchange
              expirationSeconds: 3600
              path: azure-identity-token
        defaultMode: 420

What we still miss out on though are the environment variables that describe the AWS or Azure role - however these should be fixed per region, so can be hardcoded by the control-plane or in a config-map.

[mikhail-sakhnov]

@conradludgate may I ask to provide more context for the PR like what the issue is it related to and what do we want to achieve? We struggled a bit with this one during team sync, because apparently none of us has any context.

[conradludgate]

@conradludgate may I ask to provide more context for the PR like what the issue is it related to and what do we want to achieve? We struggled a bit with this one during team sync, because apparently none of us has any context.

As far as I understand, the migration assistant project will require access to S3. Their current access setup is not sufficient as they are limited to 1 hour, and it doesn't work on Azure (cc @NanoBjorn for corrections).

This project runs pg_dump/pg_restore within compute VMs provisioned by neonvm.

They would rather switch to a service account model, where their migration assistant VMs have a service account assigned, and this service account grants them access to assume an AWS role. Kubernetes does this with service account tokens, mounted into the file system using a projected volume. These tokens are not static, they are JWT based and will eventually expire. Before expiration, kubernetes will update the projected volume.

Exactly the same problem as with TLS keys (#1213), the VM must be able to observe changes to this file in order to refresh the AWS or Azure permissions. This is achieved by adding ProjectedVolume support to the synchronisation routine that is currently supported for both secrets and config maps.