VSFTPD not behaving in command line, but fine in GUI (Filezilla)

Question

I'm trying to use VSFTP 3.0.3-3ubuntu2 on an Ubuntu 16:04.1 install. I have done this so that an application on another server can run an automated script to pull files from the FTP server triggered by a cron job.

I can connect to the FTP server easily using Filezilla and can browse the ftp directory structure, and both download & upload files in both active and passive mode, so I know that VSTFP is installed and behaving at least in some form, as expected. However, when I try to connect to the FTP server using command line, I can't browse the file structure at all, I can't even do a simple dir or ls command, so naturally the software running on my other server isn't able to pull the required files since it's in essence, running a script to connect to the FTP server in the background.

Can anybody at all suggest what might be the problem here?

Here's what I get when I log into to my FTP server and try to type a few basic commands:

FTP from DOS (Windows 10 command line)

230 Login successful

ftp> ls 
550 Permission denied

ftp> dir
550 Permission denied.
425 Use PORT or PASV first

ftp> quote passv
227 Entering Pasive Mode (172,31,45,155,173,61)

ftp> ls
550 Permission denied

ftp> dir
425 Failed to establish a connection

mget filename.csv
Permission denied
200 Switching to ASCII mode.
Cannot find list of remote files

FTP from Linux Command Line:

230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> ls
550 Permission denied.
ftp: bind: Address already in use
ftp> dir
550 Permission denied.
ftp> quote pasv
227 Entering Passive Mode (172,31,45,155,165,40).
ftp> ls
550 Permission denied.
ftp> dir
550 Permission denied.

I have also tried enabling passive mode via command line but am getting the same "permission denied" error messages. Although for a "dir" command I get a slightly different response in Passive - 425 Failed to establish connection.

I originally set-up the server as per the tutorial from: https://www.digitalocean.com/community/tutorials/how-to-set-up-vsftpd-for-a-user-s-directory-on-ubuntu-16-04

I have enabled vsftpd logging, but it is only showing connection successful notifications in relation to trying to login via command line. I can't see any clues as to why I cannot run command via the command line. If I login via Filezilla and pull a few files, I can see those downloads being logged correctly.

My VSFTP config file is as follows, note that the bottom line, I have added in the "commands allowed" line in an attempt to get VSFTP to listen to me, but clearly is hasn't worked. Also, I did set up sftp originally, however, have since commented out some of the lines as the software on the other server I'm using to get the files is set to only use FTP.

# Example config file /etc/vsftpd.conf
#
# The default compiled in settings are fairly paranoid. This sample file
# loosens things up a bit, to make the ftp daemon more usable.
# Please see vsftpd.conf.5 for all compiled in defaults.
#
# READ THIS: This example file is NOT an exhaustive list of vsftpd options.
# Please read the vsftpd.conf.5 manual page to get a full idea of vsftpd's
# capabilities.
#
#
# Run standalone?  vsftpd can run either from an inetd or as a standalone
# daemon started from an initscript.
listen=NO
#
# This directive enables listening on IPv6 sockets. By default, listening
# on the IPv6 "any" address (::) will accept connections from both IPv6
# and IPv4 clients. It is not necessary to listen on *both* IPv4 and IPv6
# sockets. If you want that (perhaps because you want to listen on specific
# addresses) then you must run two copies of vsftpd with two configuration
# files.
listen_ipv6=YES
#
# Allow anonymous FTP? (Disabled by default).
anonymous_enable=NO
#
# Uncomment this to allow local users to log in.
local_enable=YES
#
# Uncomment this to enable any form of FTP write command.
write_enable=YES
#
# Default umask for local users is 077. You may wish to change this to 022,
# if your users expect that (022 is used by most other ftpd's)
local_umask=022
#
# Uncomment this to allow the anonymous FTP user to upload files. This only
# has an effect if the above global write enable is activated. Also, you will
# obviously need to create a directory writable by the FTP user.
#anon_upload_enable=YES
#
# Uncomment this if you want the anonymous FTP user to be able to create
# new directories.
#anon_mkdir_write_enable=YES
#
# Activate directory messages - messages given to remote users when they
# go into a certain directory.
dirmessage_enable=YES
#
# If enabled, vsftpd will display directory listings with the time
# in  your  local  time  zone.  The default is to display GMT. The
# times returned by the MDTM FTP command are also affected by this
# option.
use_localtime=YES
#
# Activate logging of uploads/downloads.
xferlog_enable=YES
#
# Make sure PORT transfer connections originate from port 20 (ftp-data).
connect_from_port_20=YES
#
# If you want, you can arrange for uploaded anonymous files to be owned by
# a different user. Note! Using "root" for uploaded files is not
# recommended!
#chown_uploads=YES
#chown_username=whoever
#
# You may override where the log file goes if you like. The default is shown
# below.
#xferlog_file=/var/log/vsftpd.log
#
# If you want, you can have your log file in standard ftpd xferlog format.
# Note that the default log file location is /var/log/xferlog in this case.
#xferlog_std_format=YES
#
# You may change the default value for timing out an idle session.
#idle_session_timeout=600
#
# You may change the default value for timing out a data connection.
#data_connection_timeout=120
#
# It is recommended that you define on your system a unique user which the
# ftp server can use as a totally isolated and unprivileged user.
#nopriv_user=ftpsecure
#
# Enable this and the server will recognise asynchronous ABOR requests. Not
# recommended for security (the code is non-trivial). Not enabling it,
# however, may confuse older FTP clients.
#async_abor_enable=YES
#
# By default the server will pretend to allow ASCII mode but in fact ignore
# the request. Turn on the below options to have the server actually do ASCII
# mangling on files when in ASCII mode.
# Beware that on some FTP servers, ASCII support allows a denial of service
# attack (DoS) via the command "SIZE /big/file" in ASCII mode. vsftpd
# predicted this attack and has always been safe, reporting the size of the
# raw file.
# ASCII mangling is a horrible feature of the protocol.
#ascii_upload_enable=YES
#ascii_download_enable=YES
#
# You may fully customise the login banner string:
# ftpd_banner= FTP for supplier product feeds ONLY
#
# You may specify a file of disallowed anonymous e-mail addresses. Apparently
# useful for combatting certain DoS attacks.
#deny_email_enable=YES
# (default follows)
#banned_email_file=/etc/vsftpd.banned_emails
#
# You may restrict local users to their home directories.  See the FAQ for
# the possible risks in this before using chroot_local_user or
# chroot_list_enable below.
# chroot_local_user=YES
#
# You may specify an explicit list of local users to chroot() to their home
# directory. If chroot_local_user is YES, then this list becomes a list of
# users to NOT chroot().
# (Warning! chroot'ing can be very dangerous. If using chroot, make sure that
# the user does not have write access to the top level directory within the
# chroot)

chroot_local_user=YES
allow_writeable_chroot=YES

# chroot_list_enable=YES
# (default follows)
# chroot_list_file=/etc/vsftpd.chroot_list

pasv_enable=Yes
pasv_min_port=40000
pasv_max_port=50000
pasv_address=xxx.xxx.xxx.xxx

# Added as per https://www.digitalocean.com/community/tutorials/how-to-set-up-vsftpd-for-a-user-s-directory-on-ubuntu-16-04
user_sub_token=$USER
local_root=/home/$USER/ftp

userlist_enable=YES
userlist_file=/etc/vsftpd.userlist
userlist_deny=NO

# You may activate the "-R" option to the builtin ls. This is disabled by
# default to avoid remote users being able to cause excessive I/O on large
# sites. However, some broken FTP clients such as "ncftp" and "mirror" assume
# the presence of the "-R" option, so there is a strong case for enabling it.
#ls_recurse_enable=YES
#
# Customization
#
# Some of vsftpd's settings don't fit the filesystem layout by
# default.
#
# This option should be the name of a directory which is empty.  Also, the
# directory should not be writable by the ftp user. This directory is used
# as a secure chroot() jail at times vsftpd does not require filesystem
# access.
secure_chroot_dir=/var/run/vsftpd/empty
#
# This string is the name of the PAM service vsftpd will use.
pam_service_name=vsftpd
#
# This option specifies the location of the RSA certificate to use for SSL
# encrypted connections.
# rsa_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
# rsa_private_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
ssl_enable=NO

rsa_cert_file=/etc/ssl/private/vsftpd.pem
rsa_private_key_file=/etc/ssl/private/vsftpd.pem

#allow_anon_ssl=NO
#force_local_data_ssl=YES
#force_local_logins_ssl=YES

#ssl_tlsv1=YES
#ssl_sslv2=NO
#ssl_sslv3=NO

#require_ssl_reuse=NO
#ssl_ciphers=HIGH

# Uncomment this to indicate that vsftpd use a utf8 filesystem.
#utf8_filesystem=YES

cmds_allowed=dir,get,ls,put,cd,mkdir,rm,rmdir,PUT,PWD,CWD,SYST,FEAT,STOR,LIST,MKD,DELE,RMD,GET,EPSV,PASV,RETR,TYPE,NOOP,EXIT,QUIT

Log from Command line login and dir list attempts...

Mon Jan  9 16:41:05 2017 [pid 11271] FTP response: Client "::ffff:80.229.82.100", "220 (vsFTPd 3.0.3)"
Mon Jan  9 16:41:06 2017 [pid 11271] FTP command: Client "::ffff:80.229.82.100", "OPTS UTF8 ON"
Mon Jan  9 16:41:06 2017 [pid 11271] FTP response: Client "::ffff:80.229.82.100", "200 Always in UTF8 mode."
Mon Jan  9 16:41:09 2017 [pid 11271] FTP command: Client "::ffff:80.229.82.100", "USER pmcftp"
Mon Jan  9 16:41:09 2017 [pid 11271] [pmcftp] FTP response: Client "::ffff:80.229.82.100", "331 Please specify the password."
Mon Jan  9 16:41:16 2017 [pid 11271] [pmcftp] FTP command: Client "::ffff:80.229.82.100", "PASS <password>"
Mon Jan  9 16:41:16 2017 [pid 11270] [pmcftp] OK LOGIN: Client "::ffff:80.229.82.100"
Mon Jan  9 16:41:17 2017 [pid 11275] [pmcftp] FTP response: Client "::ffff:80.229.82.100", "230 Login successful."
Mon Jan  9 16:41:27 2017 [pid 11275] [pmcftp] FTP command: Client "::ffff:80.229.82.100", "NLST"
Mon Jan  9 16:41:27 2017 [pid 11275] [pmcftp] FTP response: Client "::ffff:80.229.82.100", "550 Permission denied."
Mon Jan  9 16:41:29 2017 [pid 11275] [pmcftp] FTP command: Client "::ffff:80.229.82.100", "LIST"
Mon Jan  9 16:41:29 2017 [pid 11275] [pmcftp] FTP response: Client "::ffff:80.229.82.100", "425 Use PORT or PASV first."
Mon Jan  9 16:42:02 2017 [pid 11275] [pmcftp] FTP command: Client "::ffff:80.229.82.100", "PASV"
Mon Jan  9 16:42:02 2017 [pid 11275] [pmcftp] FTP response: Client "::ffff:80.229.82.100", "227 Entering Passive Mode (0,0,0,0,194,105)."
Mon Jan  9 16:42:04 2017 [pid 11275] [pmcftp] FTP command: Client "::ffff:80.229.82.100", "NLST"
Mon Jan  9 16:42:04 2017 [pid 11275] [pmcftp] FTP response: Client "::ffff:80.229.82.100", "550 Permission denied."
Mon Jan  9 16:42:05 2017 [pid 11275] [pmcftp] FTP command: Client "::ffff:80.229.82.100", "LIST"
Mon Jan  9 16:43:05 2017 [pid 11275] [pmcftp] FTP response: Client "::ffff:80.229.82.100", "425 Failed to establish connection."

Log from Filezilla login and download n.b In trying to put a file to the server I get a critical transfer error. Whilst this is a problem, it's not my priority here.

Mon Jan  9 16:48:05 2017 [pid 11275] [pmcftp] FTP response: Client "::ffff:80.229.82.100", "421 Timeout."
Mon Jan  9 16:48:16 2017 [pid 11460] CONNECT: Client "::ffff:80.229.82.100"
Mon Jan  9 16:48:16 2017 [pid 11460] FTP response: Client "::ffff:80.229.82.100", "220 (vsFTPd 3.0.3)"
Mon Jan  9 16:48:16 2017 [pid 11460] FTP command: Client "::ffff:80.229.82.100", "AUTH TLS"
Mon Jan  9 16:48:16 2017 [pid 11460] FTP response: Client "::ffff:80.229.82.100", "530 Please login with USER and PASS."
Mon Jan  9 16:48:16 2017 [pid 11460] FTP command: Client "::ffff:80.229.82.100", "AUTH SSL"
Mon Jan  9 16:48:16 2017 [pid 11460] FTP response: Client "::ffff:80.229.82.100", "530 Please login with USER and PASS."
Mon Jan  9 16:48:16 2017 [pid 11460] FTP command: Client "::ffff:80.229.82.100", "USER pmcftp"
Mon Jan  9 16:48:16 2017 [pid 11460] [pmcftp] FTP response: Client "::ffff:80.229.82.100", "331 Please specify the password."
Mon Jan  9 16:48:16 2017 [pid 11460] [pmcftp] FTP command: Client "::ffff:80.229.82.100", "PASS <password>"
Mon Jan  9 16:48:16 2017 [pid 11459] [pmcftp] OK LOGIN: Client "::ffff:80.229.82.100"
Mon Jan  9 16:48:16 2017 [pid 11464] [pmcftp] FTP response: Client "::ffff:80.229.82.100", "230 Login successful."
Mon Jan  9 16:48:16 2017 [pid 11464] [pmcftp] FTP command: Client "::ffff:80.229.82.100", "PWD"
Mon Jan  9 16:48:16 2017 [pid 11464] [pmcftp] FTP response: Client "::ffff:80.229.82.100", "257 "/" is the current directory"
Mon Jan  9 16:48:46 2017 [pid 11514] CONNECT: Client "::ffff:80.229.82.100"
Mon Jan  9 16:48:46 2017 [pid 11514] FTP response: Client "::ffff:80.229.82.100", "220 (vsFTPd 3.0.3)"
Mon Jan  9 16:48:46 2017 [pid 11514] FTP command: Client "::ffff:80.229.82.100", "AUTH TLS"
Mon Jan  9 16:48:46 2017 [pid 11514] FTP response: Client "::ffff:80.229.82.100", "530 Please login with USER and PASS."
Mon Jan  9 16:48:46 2017 [pid 11514] FTP command: Client "::ffff:80.229.82.100", "AUTH SSL"
Mon Jan  9 16:48:46 2017 [pid 11514] FTP response: Client "::ffff:80.229.82.100", "530 Please login with USER and PASS."
Mon Jan  9 16:48:46 2017 [pid 11514] FTP command: Client "::ffff:80.229.82.100", "USER pmcftp"
Mon Jan  9 16:48:46 2017 [pid 11514] [pmcftp] FTP response: Client "::ffff:80.229.82.100", "331 Please specify the password."
Mon Jan  9 16:48:46 2017 [pid 11514] [pmcftp] FTP command: Client "::ffff:80.229.82.100", "PASS <password>"
Mon Jan  9 16:48:46 2017 [pid 11513] [pmcftp] OK LOGIN: Client "::ffff:80.229.82.100"
Mon Jan  9 16:48:46 2017 [pid 11518] [pmcftp] FTP response: Client "::ffff:80.229.82.100", "230 Login successful."
Mon Jan  9 16:48:46 2017 [pid 11518] [pmcftp] FTP command: Client "::ffff:80.229.82.100", "CWD /nimans"
Mon Jan  9 16:48:46 2017 [pid 11518] [pmcftp] FTP response: Client "::ffff:80.229.82.100", "250 Directory successfully changed."
Mon Jan  9 16:48:48 2017 [pid 11518] [pmcftp] FTP command: Client "::ffff:80.229.82.100", "TYPE I"
Mon Jan  9 16:48:48 2017 [pid 11518] [pmcftp] FTP response: Client "::ffff:80.229.82.100", "200 Switching to Binary mode."
Mon Jan  9 16:48:48 2017 [pid 11518] [pmcftp] FTP command: Client "::ffff:80.229.82.100", "PASV"
Mon Jan  9 16:48:48 2017 [pid 11518] [pmcftp] FTP response: Client "::ffff:80.229.82.100", "227 Entering Passive Mode (0,0,0,0,167,164)."
Mon Jan  9 16:48:48 2017 [pid 11518] [pmcftp] FTP command: Client "::ffff:80.229.82.100", "RETR F126841.CSV"
Mon Jan  9 16:48:48 2017 [pid 11518] [pmcftp] FTP response: Client "::ffff:80.229.82.100", "150 Opening BINARY mode data connection for F126841.CSV (1434704 bytes)."
Mon Jan  9 16:48:49 2017 [pid 11518] [pmcftp] OK DOWNLOAD: Client "::ffff:80.229.82.100", "/nimans/F126841.CSV", 1434704 bytes, 1153.59Kbyte/sec
Mon Jan  9 16:48:49 2017 [pid 11518] [pmcftp] FTP response: Client "::ffff:80.229.82.100", "226 Transfer complete."
Mon Jan  9 16:49:13 2017 [pid 11518] [pmcftp] FTP command: Client "::ffff:80.229.82.100", "TYPE A"
Mon Jan  9 16:49:13 2017 [pid 11518] [pmcftp] FTP response: Client "::ffff:80.229.82.100", "200 Switching to ASCII mode."
Mon Jan  9 16:49:13 2017 [pid 11518] [pmcftp] FTP command: Client "::ffff:80.229.82.100", "PASV"
Mon Jan  9 16:49:13 2017 [pid 11518] [pmcftp] FTP response: Client "::ffff:80.229.82.100", "227 Entering Passive Mode (0,0,0,0,165,201)."
Mon Jan  9 16:49:13 2017 [pid 11518] [pmcftp] FTP command: Client "::ffff:80.229.82.100", "STOR .htaccess"
Mon Jan  9 16:49:13 2017 [pid 11518] [pmcftp] FTP response: Client "::ffff:80.229.82.100", "553 Could not create file."
Mon Jan  9 16:49:13 2017 [pid 11518] [pmcftp] FAIL UPLOAD: Client "::ffff:80.229.82.100", "/nimans/.htaccess", 0.00Kbyte/sec

Answer 1

Okay, this isn't a definitive answer, but it has fixed my problem and I'm hoping others might be able to expand on this.

It was to do with the passive_address value I had set in the vsftpd.conf

It seemed that adding the CIDR /32 switch to the end of the IP address would allow me to log in via command line, but then locked me out of logging in via Filezilla.

pasv_address=xxx.xxx.xxx.xxx/32

Here's the weird thing...

As an experiment, I commented out all of the passive mode settings to see if I could login via both methods (Filezilla and command line FTP) which I could. However, the connection seemed unstable, very slow and would time-out after a short period of time.

#pasv_enable=Yes
#pasv_min_port=40000
#pasv_max_port=50000
#pasv_address=xxx.xxx.xxx.xxx/32

When I un-commented all of these again, for convenience, I needed to log in via Filezilla so I omitted the /32 CIDR off the ip address and was able to login with no problems. As an experiment, and for consistency's sake without making changes or re-adding the /32, I thought I'd try to log back into my FTP with command line again and it worked!!

I have no idea why. If anyone can expand, it would be helpful, but I'll leave this here for anyone else having the same problem. Maybe it will help.