I am trying to set up a drupal site using docker in a Vagrant VM and can't seem to get Certbot to work

Question

So I am trying to set up a drupal site using docker in a vagrant VM using bento/ubuntu-20.04.

I went through this tutorial: https://www.digitalocean.com/community/tutorials/how-to-install-drupal-with-docker-compose . It seemed to work (I could navigate to the url I was using http 80 before I tried to set up https) but trying to implement the certbot setup it didn’t work. I don’t think it is the certbot directly because when I do: docker-compose exec webserver ls -lsh /etc/letsencrypt/live/server.otherrealm.org there seems to be a valid certificate. However, when I try to view the site using https/443, it does not work (gives ‘The connection has timed out’). I can’t see anything wrong with my configuration but I’ve been staring at it for the past day and I need someone else to take a fresh look at it. THANKS!! Let me know if there is anything that is not clear.

nginx-conf/nginx.conf

server {
    listen 80;
    listen [::]:80;
    server_name server.otherrealm.org;

    location ~ /.well-known/acme-challenge {
        allow all;
        root /var/www/html;
    }

    location / {
        rewrite ^ https://$host$request_uri? permanent;
    }
}
server {
    listen 443 ssl;
    listen [::]:443 ssl;
    server_name server.otherrealm.org;

    index index.php index.html index.htm;

    root /var/www/html;

    server_tokens off;

    ssl_certificate /etc/letsencrypt/live/server.otherrealm.org/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/server.otherrealm.org/privkey.pem;

    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-XSS-Protection "1; mode=block" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header Referrer-Policy "no-referrer-when-downgrade" always;
    add_header Content-Security-Policy "default-src * data: 'unsafe-eval' 'unsafe-inline'" always;

    location / {
        try_files $uri $uri/ /index.php$is_args$args;
    }

    rewrite ^/core/authorize.php/core/authorize.php(.*)$ /core/authorize.php$1;

    location ~ \.php$ {
        try_files $uri =404;
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_pass drupal:9000;
        fastcgi_index index.php;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param PATH_INFO $fastcgi_path_info;
    }

    location ~ /\.ht {
        deny all;
    }

    location = /favicon.ico {
        log_not_found off; access_log off;
    }
    location = /robots.txt {
        log_not_found off; access_log off; allow all;
    }
    location ~* \.(css|gif|ico|jpeg|jpg|js|png)$ {
        expires max;
        log_not_found off;
    }
}

docker-compose.yml

version: '3.8'
services:
  mysql:
    image: mysql:latest
    container_name: mysql
    restart: unless-stopped
    env_file: .env
    volumes:
      - db-data:/var/lib/mysql
    networks:
      - internal
  drupal:
    image: drupal:fpm-alpine
    container_name: drupal
    depends_on:
      - mysql
    restart: unless-stopped
    networks:
      - internal
      - external
    volumes:
      - drupal-data:/var/www/html
  webserver:
    image: nginx:latest
    container_name: webserver
    depends_on:
      - drupal
    restart: unless-stopped
    ports:
      - 80:80
      - 443:443
    volumes:
      - drupal-data:/var/www/html
      - ./nginx-conf:/etc/nginx/conf.d
      - certbot-etc:/etc/letsencrypt
    networks:
      - external
  certbot:
    depends_on:
      - webserver
    image: certbot/dns-google
    container_name: certbot
    volumes:
      - certbot-etc:/etc/letsencrypt
      - drupal-data:/var/www/html
    command: certonly -d [sub.example.com] --webroot --webroot-path=/var/www/html --email [someone@example.com] --agree-tos --no-eff-email --force-renewal 

networks:
  external:
    driver: bridge
  internal:
    driver: bridge

volumes:
  drupal-data:
  db-data:
  certbot-etc:

vagrant@manager:/vagrant$ docker-compose ps

 Name                 Command               State                     Ports
----------------------------------------------------------------------------------------------
certbot     certbot certonly --webroot ...   Exit 0
drupal      docker-php-entrypoint php-fpm    Up       9000/tcp
mysql       docker-entrypoint.sh mysqld      Up       3306/tcp, 33060/tcp
webserver   /docker-entrypoint.sh ngin ...   Up       0.0.0.0:443->443/tcp, 0.0.0.0:80->80/tcp

vagrant@manager:/vagrant$ docker-compose logs certbot (Of course, now it is saying ...too many certificates already issued for exact set of domains... but that is because I tried it too many times ‍♂️)

Attaching to certbot
certbot      | Saving debug log to /var/log/letsencrypt/letsencrypt.log
certbot      | Plugins selected: Authenticator webroot, Installer None
certbot      | Renewing an existing certificate
certbot      | IMPORTANT NOTES:
certbot      |  - Congratulations! Your certificate and chain have been saved at:
certbot      |    /etc/letsencrypt/live/[server.example.com]/fullchain.pem
certbot      |    Your key file has been saved at:
certbot      |    /etc/letsencrypt/live/[server.example.com]/privkey.pem
certbot      |    Your cert will expire on 2021-01-24. To obtain a new or tweaked
certbot      |    version of this certificate in the future, simply run certbot
certbot      |    again. To non-interactively renew *all* of your certificates, run
certbot      |    "certbot renew"
certbot      |  - If you like Certbot, please consider supporting our work by:
certbot      |
certbot      |    Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
certbot      |    Donating to EFF:                    https://eff.org/donate-le
certbot      |
certbot      | Saving debug log to /var/log/letsencrypt/letsencrypt.log
certbot      | Plugins selected: Authenticator webroot, Installer None
certbot      | Renewing an existing certificate
certbot      | IMPORTANT NOTES:
certbot      |  - Congratulations! Your certificate and chain have been saved at:
certbot      |    /etc/letsencrypt/live/[server.example.com]/fullchain.pem
certbot      |    Your key file has been saved at:
certbot      |    /etc/letsencrypt/live/[server.example.com]/privkey.pem
certbot      |    Your cert will expire on 2021-01-24. To obtain a new or tweaked
certbot      |    version of this certificate in the future, simply run certbot
certbot      |    again. To non-interactively renew *all* of your certificates, run
certbot      |    "certbot renew"
certbot      |  - If you like Certbot, please consider supporting our work by:
certbot      |
certbot      |    Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
certbot      |    Donating to EFF:                    https://eff.org/donate-le
certbot      |

vagrant@manager:/vagrant$ docker-compose logs webserver

Attaching to webserver
webserver    | /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform 
configuration
webserver    | /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
webserver    | /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
webserver    | 10-listen-on-ipv6-by-default.sh: error: /etc/nginx/conf.d/default.conf is not a file or does not exist
webserver    | /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh  
webserver    | /docker-entrypoint.sh: Configuration complete; ready for start up
webserver    | 10.0.2.2 - - [26/Oct/2020:23:16:28 +0000] "GET / HTTP/1.0" 301 169 "-" "masscan/1.0 (https://github.com/robertdavidgraham/masscan)" "-"