27

So, I'm trying to get Nexus running based off of this image in Kubernetes, but it's failing with:

mkdir: cannot create directory '../sonatype-work/nexus3/log': Permission denied
mkdir: cannot create directory '../sonatype-work/nexus3/tmp': Permission denied
Java HotSpot(TM) 64-Bit Server VM warning: Cannot open file ../sonatype-work/nexus3/log/jvm.log due to No such file or directory

From the documentation it says that the process runs with UID 200 and the volume must be mounted with those permissions:

A persistent directory, /nexus-data, is used for configuration,
logs, and storage. This directory needs to be writable by the Nexus
process, which runs as UID 200.

I've tried to search through the documentation to find a way to mount the volume with those permissions, however, I couldn't find any way to do it.

Does anyone know whether you can specify in the configuration for either the PVC/PV or Deployment what UID to mount the volume with? If so, how?

Wytrzymały Wiktor
  • 731
  • 3
  • 10
srkiNZ84
  • 531
  • 1
  • 6
  • 10
  • For reference, [Kubernetes issue in progress](https://github.com/kubernetes/kubernetes/issues/2630) – tisc0 Apr 02 '20 at 23:32

3 Answers3

41

There is no way to set the UID using the definition of Pod, but Kubernetes saves the UID of sourced volume.

So, you can set the UID by InitContainer, which launches before the main container, just add it to the containers path of the Deployment:

initContainers:
- name: volume-mount-hack
  image: busybox
  command: ["sh", "-c", "chown -R 200:200 /nexus"]
  volumeMounts:
  - name: <your nexus volume>
    mountPath: /nexus
030
  • 5,731
  • 12
  • 61
  • 107
Anton Kostenko
  • 652
  • 6
  • 5
  • Works great. Thanks for this hack. Using it with Oracle DB image. – Thomas Hofmann May 15 '18 at 12:49
  • 2
    This does not help with ConfigMaps and Secrets, though. – Torsten Bronger Aug 11 '18 at 22:49
  • This worked for me as well (also with chmod). I hope someone (or Kubernetes) implement a less hacky method. – leeman24 Mar 15 '19 at 18:47
  • I'm using something like `command: ["sh", "-c", "chmod 777 /nexus && chown 200:200 /nexus"]` to ensure the folder is writable. – Martin Tapp Jun 18 '19 at 14:47
  • But the question is, how do we know the UID of the process in the main container? It could be anything, other than 200, as well, right? – Nawaz Mar 12 '20 at 21:12
  • Would be great to find a hack without having to run any [init]container as root. I tend to prefer the below solution (@chemi0213) using fsGroup, which is PSP compliant, but doesn't help for the use case where you have to set, for example, read only permissions on files (often required by software like pgadmin). Thanks in advance for sharing any insight. – tisc0 Apr 02 '20 at 22:13
  • https://github.com/kubernetes/kubernetes/issues/2630 – tisc0 Apr 02 '20 at 23:33
11

Like Anton said, although we can't set the UID using the definition of Pod. Here comes another workaround for this topic.

Please refer to the Kubernetes official document Configure a Security Context for a Pod or Container

The pod definition I used:

apiVersion: v1
kind: Pod
metadata:
  name: nexus3
  labels:
    app: nexus3
spec:
  securityContext:
    fsGroup: 200
  volumes:
  - name: nexus-data-vol
    emptyDir: {}
  containers:
  - name: nexus3-container
    image: sonatype/nexus3
    volumeMounts:
    - name: nexus-data-vol
      mountPath: /nexus-data

The Service definition:

apiVersion: v1
kind: Service
metadata:
  name: nexus3-service
spec:
  type: NodePort
  ports:
  - port: 8081
    nodePort: 30390
    protocol: TCP
    targetPort: 8081
  selector:
    app: nexus3

And then create pod and service without any permission denied or other errors:

# kubectl create -f nexus3.yaml
# kubectl create -f nexus3-svc.yaml

Try to login the Nexus3 container and check the owner/permission of /nexus-data:

# kubectl exec -it nexus3 -- sh
sh-4.2$ ls -ld /nexus-data/
drwxrwsrwx 16 root nexus 4096 Mar 13 09:00 /nexus-data/
sh-4.2$

As you can see, the directory belongs to root:nexus, and you can also check the files in the directory:

sh-4.2$ cd /nexus-data/
sh-4.2$ ls -l
total 72
drwxr-sr-x   3 nexus nexus  4096 Mar 13 09:00 blobs
drwxr-sr-x 269 nexus nexus 12288 Mar 13 08:59 cache
drwxr-sr-x   8 nexus nexus  4096 Mar 13 09:00 db
drwxr-sr-x   3 nexus nexus  4096 Mar 13 09:00 elasticsearch
drwxr-sr-x   3 nexus nexus  4096 Mar 13 08:59 etc
drwxr-sr-x   2 nexus nexus  4096 Mar 13 08:59 generated-bundles
drwxr-sr-x   2 nexus nexus  4096 Mar 13 08:59 instances
drwxr-sr-x   3 nexus nexus  4096 Mar 13 08:59 javaprefs
drwxr-sr-x   2 nexus nexus  4096 Mar 13 08:59 kar
drwxr-sr-x   3 nexus nexus  4096 Mar 13 08:59 keystores
-rw-r--r--   1 nexus nexus     8 Mar 13 08:59 lock
drwxr-sr-x   2 nexus nexus  4096 Mar 13 09:00 log
drwxr-sr-x   2 nexus nexus  4096 Mar 13 08:59 orient
-rw-r--r--   1 nexus nexus     5 Mar 13 08:59 port
drwxr-sr-x   2 nexus nexus  4096 Mar 13 08:59 restore-from-backup
drwxr-sr-x   7 nexus nexus  4096 Mar 13 09:00 tmp
sh-4.2$ touch test-file
sh-4.2$ ls -l test-file
-rw-r--r-- 1 nexus nexus 0 Mar 13 09:13 test-file
sh-4.2$ mkdir test-dir
sh-4.2$ ls -l test-dir
total 0
sh-4.2$ ls -ld test-dir
drwxr-sr-x 2 nexus nexus 4096 Mar 13 09:13 test-dir

That is the power of SetGID :)

Now Let's check the service is working or not. I use minikube to running a kubernetes cluster:

chris@XPS-13-9350 ~ $ minikube service nexus3-service --url
http://192.168.39.95:30390
chris@XPS-13-9350 ~ $ curl -u admin:admin123 http://192.168.39.95:30390/service/metrics/ping
pong

The service is working as expected.

chemi0213
  • 111
  • 1
  • 3
  • I don't think that emptyDir is anywhere close to a proper solution. Reading kube docs: https://kubernetes.io/docs/concepts/storage/volumes/#emptydir "emptyDir volume is first created when a Pod is assigned to a node, and exists as long as that Pod is running on that node. (...) When a Pod is removed from a node for any reason, the data in the emptyDir is deleted permanently." In short - it is volatile storage. – splatch Sep 09 '22 at 13:17
1

Regarding Torsten Bronger's comment, when you configure ConfigMaps and Secrets in the volumes array in the pod spec, you can specify the permissions to allow the access you want using the defaultMode property, so while you can't set group and user ownership, you can allow processes in the pod to read files in those mounts. Writing to a secret or config map does not really make sense and the default permission mode is 755 anyway so reading shouldn't be an issue for any user.

David Dooling
  • 111
  • 2