Cygwin loses drive letter colon of Windows style paths in variables

1

I ran into a strange-looking issue while trying to get a build script (from 3rd party) to work on my setup with Win7 and Cygwin (latest version). The issue can maybe best described with an example bash script snippet:

foo="/cygdrive/c/svn/Projects/Client Config/Android/Repack/foo/out"
cygpath -w "$foo/play-services-tasks/classes.jar"
bar=`cygpath -w "$foo/play-services-tasks/classes.jar"`
echo $bar

Running that results the following (pay attention to the colon after drive letter):

C:\svn\Projects\Client Config\Android\Repack\foo\out\play-services-tasks\classes.jar
C \svn\Projects\Client Config\Android\Repack\foo\out\play-services-tasks\classes.jar

So, the path (the output of cygpath) is proper before putting it in a variable, but the variable doesn't anymore contain the colon after the drive letter. That, in turn, makes another script/tool fail when it tries to iterate over space separated paths in a variable. And unfortunately, that tool expects to have the Windows style paths.

Needless to say, I am baffled...

The issue appeared when the build script changed and it may well lack some cygwin compatibility hacks, some of which I already managed to apply, but this one's really nasty. It might be possible to work around the changed parts somehow but I'd rather figure what's the reason for my finding and how to deal with that directly.

zagrimsan

Posted 2017-01-26T11:51:25.003

Reputation: 995

Answers

1

The issue appeared when the build script changed and it was lacking some cygwin compatibility hacks, some of which I already had managed to apply before hitting this one.

I finally managed to figure out the real culprit, which was that there was a function in the build script that (while iterating over the PATH variable) set environmental variable IFS=: and didn't reset it afterwards. As bash uses that variable to split strings to fields, and it does variable expansion also when using unquoted variables. So, the fix was to do SAVEIFS="$IFS" before re-setting IFS and then afterwards set it back by IFS=$SAVEIFS. Alternatively, as Gordon Davisson kindly explained, I could have ensured that the variables were used always double-quoted (e.g. echo "$bar" instead of echo $bar).

Problem solved, not really related to Cygwin in any other way than colons being used for different purposes in Windows and Cygwin and thus making it sometimes necessary to touch the internal field separator variable. For that sole reason I thought I should leave this here, in case someone else runs into a similar issue. There is also plenty of good stuff on all the aspects of shell expansion on USE.

zagrimsan

Posted 2017-01-26T11:51:25.003

Reputation: 995

try bar=$(cygpath -w "$foo/play-services-tasks/classes.jar") – matzeri – 2017-01-26T14:36:44.410

It's not happening when the variable is assigned, but when it's used. When the shell expands a variable without double-quotes (e.g. echo $bar), the value is split into "words" based on IFS (and then any "words" that contain wildcards get expanded to a list of matching filenames. Setting IFS back to normal is a good idea, but you should also double-quote variable references (e.g. echo "$bar") to prevent unexpected parsing like this. – Gordon Davisson – 2017-01-26T14:46:11.980

Shell variable expansion is and has not been my strongest area of knowledge, have to admit that, but it's good to be learning... @GordonDavisson the variable was later on "split" using eval set -- $bar, I wonder if there is any way to prevent the shell from expanding things with that? – zagrimsan – 2017-01-27T04:30:06.360

@zagrimsan Don't use eval, it's a real bug magnet unless you know exactly what you're doing. Wouldn't set -- "$bar" work in this case? (And then be sure to reference the argument with double-quotes, as in "$1".) – Gordon Davisson – 2017-01-27T04:45:37.230

Well, the build script isn't my creation... But even with IFS=: it would seem that using double-quotes everywhere, i.e. eval set -- "$dxlibs"; echo "$@"; would prevent the expansion from happening. Thanks, I'll try to remember this the next time I run into something like this (and remember not to cause this in my own scripts). – zagrimsan – 2017-01-27T07:14:04.973

@zagrimsan: eval set -- "$dxlibs" won't do word splitting, but will expand wildcards. Is there any reason for eval to be there? – Gordon Davisson – 2017-01-28T03:01:14.833

@GordonDavisson to be frank, I had to look up what even set -- is supposed to do - as I said, the script isn't of my creation. So, it might well be that eval is quite useless there (can't test it now as I'm not in the office). Even though Windows and bash share the same wildcards and thus it is unlikely that a valid library path would contain such and cause issues with wildcard expansion, I think your point is good, it's always good to eliminate excess complexity. – zagrimsan – 2017-01-28T05:40:05.017

More detail about temporary IFS overrides and how to do it safely can be found in this answer, which on the face of it isn't related so it wouldn't have shown up in a search. But I had exactly the same problem as the OP here, and the linked answer is how I solved it.

– Ti Strga – 2018-04-24T19:45:58.087

Asked: 2017-01-26T11:51:25.003

Viewed: 518 times

Active: 2017-01-27T07:32:58.723