0
I have 1.2 million files in a folder at a remote location. I am connecting to the remote location through VPN. The path to the files location is as follows:
hatari@192.168.1.192:/home/hatari/xmlfiles/
I have a folder in my local machine at /home/mycomp/xmls
I am trying to use find to copy all the files from remote location to my local folder. I used the following code:
find hatari@192.168.1.192:/home/hatari/xmlfiles/ -maxdepth 1 -type f |head -10 |xargs cp -t "/home/mycomp/xmls/"
This gives me an error "find hatari@192.168.1.192:/home/hatari/xmlfiles/": No such file or directory
I tried creating a tar file of the folder in remote location and copying to local machine. Since the size of the tar.gz is around 14G I could not complete the copy operation.
I also tried splitting up the tar file into 1G each and cat ting locally, but it ended up with some tar errors.
How do I copy million files from remote location accessed through VPN to local machine.
Apricot
Posted 2019-08-12T05:43:21.993
Reputation: 101
VPN is a layer that is not relevant here, I guess.
hatari@192.168.1.192:/home/hatari/xmlfiles/looks like a pathscpwould understand;findhas no clue. If you're trying to copy files one by one,scpmay be your solution, albeit it can be slow with so many files. Tarring them is a very sane idea. You said "I tried creating a tar file in remote location". I understand you have access to the console of the remote server (like with SSH). Please confirm ([edit] the question). What were these "tar errors" exactly? – Kamil Maciorowski – 2019-08-12T06:44:09.247