SFTP file from Mainframe to Linux

0

1

I want to download a file from a mainframe server to a Linux machine using an SFTP command and want to preserve EOL. It seems that the actual file does not have EOL characters, whereas the SFTP process adds EOL characters while downloading it to the Linux server.

Is there any option to download the file in binary mode using an SFTP command?

I tried to download data from the mainframe server using FTP and SFTP. FTP with binary mode preserves the actual content of original file without any modification, whereas FTP with ASCII mode and SFTP commands are modifying the content of the file by adding EOL and other special characters.

I used the following SFTP command to download the file from the mainframe server:

[ABC@XXXXXX.XXX.XXX.XXX.com: /var/fedex/srs/rcd/data/Dev/serial/Src/Private/src_intl_rating/main/ABC ]$ sftp XYZ@XXX.XXX.XXX.com
* * * * * * * * * * * * * * * * * * * * * * * * * * *
*  NOTE: YOU ARE CONNECTING VIA IBM'S SSH PORTED    *
*        TOOLS.                                     *
*          DEV2.                                    *
*        THIS IS IN TEST MODE AND MAY NOT FUNCTION  *
*        AS EXPECTED.                               *
* * * * * * * * * * * * * * * * * * * * * * * * * * *

XYZ@XXX.XXX.XXX.com's password:
Connected to XXX.XXX.XXX.com.
sftp> binary
Invalid command.
sftp> get //SYS2542.RTSB.RCD130.OUT.FEAB091
Fetching //SYS2542.RTSB.RCD130.OUT.FEAB091 to SYS2542.RTSB.RCD130.OUT.FEAB091
//SYS2542.RTSB.RCD130.OUT.FEAB091                                                                                                                                                                                                                  99% 2620KB   2.0MB/s   00:00 ETA
sftp> bye
[ABC@XXXXXX.XXX.XXX.XXX.com: /var/fedex/srs/rcd/data/Dev/serial/Src/Private/src_intl_rating/main/ABC ]$ ls -ltr
total 22360
-rw-r--r-- 1 ABC rcd 2682504 Jul 26 03:49 FTP.SYS2542.RTSB.RCD130.OUT.FEAB091.ASCII
-rw-r--r-- 1 ABC rcd 2680128 Jul 26 03:50 FTP.SYS2542.RTSB.RCD130.OUT.FEAB091.BINARY
-rw-r--r-- 1 ABC rcd 2682504 Jul 26 15:21 SYS2542.RTSB.RCD130.OUT.FEAB091
[ABC@XXXXXX.XXX.XXX.XXX.com: /var/fedex/srs/rcd/data/Dev/serial/Src/Private/src_intl_rating/main/ABC ]$ mv SYS2542.RTSB.RCD130.OUT.FEAB091 SFTP.SYS2542.RTSB.RCD130.OUT.FEAB091
[ABC@XXXXXX.XXX.XXX.XXX.com: /var/fedex/srs/rcd/data/Dev/serial/Src/Private/src_intl_rating/main/ABC ]$ ls -ltr
total 22360
-rw-r--r-- 1 ABC rcd 2682504 Jul 26 03:49 FTP.SYS2542.RTSB.RCD130.OUT.FEAB091.ASCII
-rw-r--r-- 1 ABC rcd 2680128 Jul 26 03:50 FTP.SYS2542.RTSB.RCD130.OUT.FEAB091.BINARY
-rw-r--r-- 1 ABC rcd 2682504 Jul 26 15:21 SFTP.SYS2542.RTSB.RCD130.OUT.FEAB091
[ABC@XXXXXX.XXX.XXX.XXX.com: /var/fedex/srs/rcd/data/Dev/serial/Src/Private/src_intl_rating/main/ABC ]$

Dilip Panda

Posted 2019-07-26T15:43:22.410

Reputation: 1

"does not have EOL character" - What does that even mean? Is it a binary file? – Martin Prikryl – 2019-07-26T16:08:29.873

New line character. The original file does not have that character which was generated from a mainframe process. Somehow SFTP is adding this, due to which our downstream processes are failing. – Dilip Panda – 2019-07-26T16:11:43.473

I know what EOL is. So even if your question said "does not have New line character", my question is still the same: What does that even mean? Is it a binary file? Or a text file that has a single line only (so has no new line character)? Or something else? – Martin Prikryl – 2019-07-26T16:23:08.337

I am not sure if it is a binary file or not? Since the file was generated by Mainframe process, there are no EOL character is there in the file. I wanted to get the same file with SFTP command which i got using FTP command with Binary transfer mode. My Downstream apps are working fine on the file downloaded using FTP Binary method. – Dilip Panda – 2019-07-26T16:56:40.433

Is there any way you could use scp instead of sftp? scp always transfers files in "binary mode" (it doesn't even have an "ASCII mode"). scp works over the SFTP protocol. – Cliff Armstrong – 2019-07-27T06:30:32.247

@CliffArmstrong 1) OpenSSH sftp also always transfers files in "binary mode" (doesn't even have an "ASCII mode"). 2) OpenSSH scp does not work over the SFTP protocol. – I do not mean that your suggestion to use scp could not help – after all this looks like a bug/feature of the z/OS SFTP server – see also a answer by @Hogstrom to OP's duplicate question on SO – but your claims are not correct.

– Martin Prikryl – 2019-07-27T06:37:04.950

Very well, I stand corrected regarding scp<>sftp. However, what you described happening is most definitely not binary mode. Regardless of whether the server of the client is causing the issue... binary mode, by definition, does not change the content of the file as part of the transfer. And the entire point of ASCII mode was to enable swapping EOL character formats across operating systems. So forgive me for thinking you were somehow ending up with an ASCII mode file transfer based on the description of your issue. – Cliff Armstrong – 2019-07-27T07:01:18.120

Answers

0

The z/OS port of OpenSSH included with z/OS 2.2, z/OS 2.3, and z/OS 2.4 (GA 30-Sep-2019) provides ascii and binary commands. There is a note...

The ascii subcommand is only valid for file transfers between UNIX platforms. It is not valid for file transfers between Windows and UNIX platforms.

...regarding a restriction on the use of the ascii command. Separately, binary is noted as the default.

It's possible you are using an older OpenSSH port that does not support the binary command, as I see in your session log that you get an error when using it. I do not know when the binary command was introduced, perhaps it's been there in the z/OS port all along.

It's also possible you are using the Dovetailed Technologies Co:Z sftp server, which uses a different command set. One way to test this would be to issue the command ls /+ during your sftp session.

The option you would be looking for is mode and the default is binary. But the default can be changed by an administrator. If the default is text and you want it to be binary you can change it by issuing the command ls /+mode=binary during your sftp session.

cschneid

Posted 2019-07-26T15:43:22.410

Reputation: 101

Thanks. The option for binary mode(ls /+mode=binary) of transfer with SFTP worked for me. I am now able to match with the byte size i downloaded using FTP Binary mode. I will get in touch with the admin to set default transfer mode to "Binary". @cschneid – Dilip Panda – 2019-07-29T00:34:42.120

I believe OP is running sftp command/client on Linux, so the z/OS commands ascii/binary won't work. The hack with ls would though. – Martin Prikryl – 2019-07-30T06:25:40.880

Asked: 2019-07-26T15:43:22.410

Viewed: 348 times

Active: 2019-07-27T21:37:43.133