Hadoop 3.2.1 namenode formatting not working

1

I am having [what sounds like] the same issue to Hadoop 3.2.1 namenode formatting

Per request to ask this as a new question, and also provide more details:

This happens on a Desktop, Windows 10, 64bit, all Windows Updates applied.

  1. Followed all instructions per https://dev.to/awwsmm/installing-and-running-hadoop-and-spark-on-windows-33kc. Except: a) Hadoop 3.2.1 (and after blocked on this issue, tried Hadoop 3.1.0 and Hadoop 2.8.0) b) apache-hadoop-3.1.0-winutils-master.zip for winutils.exe and hadoop.dll patch (was unable to find a later version?) c) In previous test, used C:\Hadoop\hadoop-3.2.1\data[namenode,datanode] folders, which led to same result as shown below. The change to D:\Hadoop\hadoop-3.2.1\data.. (shown below) is to a D:drive with more available space.

  2. [FROM cmd prompt, Run as Administrator], confirm Java and hdfs are available 

    C:\WINDOWS\system32>hdfs -version java version "1.8.0_221" Java(TM) SE Runtime Environment (build 1.8.0_221-b11) Java HotSpot(TM) 64-Bit Server VM (build 25.221-b11, mixed mode)

C:\WINDOWS\system32>java -version java version "1.8.0_221" Java(TM) SE Runtime Environment (build 1.8.0_221-b11) Java HotSpot(TM) 64-Bit Server VM (build 25.221-b11, mixed mode)

  3. Run namenode -format C:\WINDOWS\system32>hdfs namenode -format

    2019-10-13 12:01:58,470 INFO namenode.NameNode: STARTUP_MSG:  
C:\WINDOWS\system32>hdfs namenode -format
2019-10-13 12:01:58,470 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = DESKTOP-354IRPS/10.0.0.23
STARTUP_MSG:   args = [-format]
STARTUP_MSG:   version = 3.2.1
STARTUP_MSG:   classpath = C:\Hadoop\hadoop-3.2.1\etc\hadoop;C:\Hadoop\hadoop-3.2.1\share\hadoop\common;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\accessors-smart-1.2.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\animal-sniffer-annotations-1.17.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\asm-5.0.4.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\audience-annotations-0.5.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\avro-1.7.7.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\checker-qual-2.5.2.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\commons-beanutils-1.9.3.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\commons-cli-1.2.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\commons-codec-1.11.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\commons-collections-3.2.2.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\commons-compress-1.18.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\commons-configuration2-2.1.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\commons-io-2.5.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\commons-lang3-3.7.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\commons-logging-1.1.3.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\commons-math3-3.1.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\commons-net-3.6.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\commons-text-1.4.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\curator-client-2.13.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\curator-framework-2.13.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\curator-recipes-2.13.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\dnsjava-2.1.7.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\error_prone_annotations-2.2.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\failureaccess-1.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\gson-2.2.4.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\guava-27.0-jre.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\hadoop-annotations-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\hadoop-auth-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\htrace-core4-4.1.0-incubating.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\httpclient-4.5.6.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\httpcore-4.4.10.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\j2objc-annotations-1.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jackson-annotations-2.9.8.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jackson-core-2.9.8.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jackson-core-asl-1.9.13.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jackson-databind-2.9.8.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jackson-jaxrs-1.9.13.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jackson-mapper-asl-1.9.13.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jackson-xc-1.9.13.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\javax.servlet-api-3.1.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jaxb-api-2.2.11.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jaxb-impl-2.2.3-1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jcip-annotations-1.0-1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jersey-core-1.19.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jersey-json-1.19.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jersey-server-1.19.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jersey-servlet-1.19.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jettison-1.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jetty-http-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jetty-io-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jetty-security-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jetty-server-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jetty-servlet-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jetty-util-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jetty-webapp-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jetty-xml-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jsch-0.1.54.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\json-smart-2.3.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jsp-api-2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jsr305-3.0.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jsr311-api-1.1.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jul-to-slf4j-1.7.25.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\kerb-admin-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\kerb-client-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\kerb-common-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\kerb-core-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\kerb-crypto-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\kerb-identity-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\kerb-server-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\kerb-simplekdc-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\kerb-util-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\kerby-asn1-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\kerby-config-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\kerby-pkix-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\kerby-util-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\kerby-xdr-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\log4j-1.2.17.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\metrics-core-3.2.4.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\netty-3.10.5.Final.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\nimbus-jose-jwt-4.41.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\paranamer-2.3.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\protobuf-java-2.5.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\re2j-1.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\slf4j-api-1.7.25.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\slf4j-log4j12-1.7.25.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\snappy-java-1.0.5.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\stax2-api-3.1.4.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\token-provider-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\woodstox-core-5.0.3.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\zookeeper-3.4.13.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\hadoop-common-3.2.1-tests.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\hadoop-common-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\hadoop-kms-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\hadoop-nfs-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\accessors-smart-1.2.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\animal-sniffer-annotations-1.17.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\asm-5.0.4.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\audience-annotations-0.5.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\avro-1.7.7.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\checker-qual-2.5.2.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\commons-beanutils-1.9.3.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\commons-cli-1.2.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\commons-codec-1.11.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\commons-collections-3.2.2.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\commons-compress-1.18.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\commons-configuration2-2.1.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\commons-daemon-1.0.13.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\commons-io-2.5.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\commons-lang3-3.7.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\commons-logging-1.1.3.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\commons-math3-3.1.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\commons-net-3.6.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\commons-text-1.4.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\curator-client-2.13.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\curator-framework-2.13.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\curator-recipes-2.13.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\dnsjava-2.1.7.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\error_prone_annotations-2.2.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\failureaccess-1.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\gson-2.2.4.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\guava-27.0-jre.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\hadoop-annotations-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\hadoop-auth-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\htrace-core4-4.1.0-incubating.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\httpclient-4.5.6.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\httpcore-4.4.10.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\j2objc-annotations-1.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jackson-annotations-2.9.8.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jackson-core-2.9.8.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jackson-core-asl-1.9.13.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jackson-databind-2.9.8.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jackson-jaxrs-1.9.13.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jackson-mapper-asl-1.9.13.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jackson-xc-1.9.13.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\javax.servlet-api-3.1.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jaxb-api-2.2.11.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jaxb-impl-2.2.3-1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jcip-annotations-1.0-1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jersey-core-1.19.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jersey-json-1.19.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jersey-server-1.19.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jersey-servlet-1.19.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jettison-1.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jetty-http-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jetty-io-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jetty-security-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jetty-server-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jetty-servlet-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jetty-util-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jetty-util-ajax-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jetty-webapp-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jetty-xml-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jsch-0.1.54.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\json-simple-1.1.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\json-smart-2.3.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jsr305-3.0.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jsr311-api-1.1.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\kerb-admin-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\kerb-client-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\kerb-common-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\kerb-core-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\kerb-crypto-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\kerb-identity-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\kerb-server-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\kerb-simplekdc-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\kerb-util-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\kerby-asn1-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\kerby-config-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\kerby-pkix-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\kerby-util-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\kerby-xdr-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\leveldbjni-all-1.8.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\log4j-1.2.17.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\netty-3.10.5.Final.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\netty-all-4.0.52.Final.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\nimbus-jose-jwt-4.41.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\okhttp-2.7.5.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\okio-1.6.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\paranamer-2.3.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\protobuf-java-2.5.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\re2j-1.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\snappy-java-1.0.5.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\stax2-api-3.1.4.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\token-provider-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\woodstox-core-5.0.3.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\zookeeper-3.4.13.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\hadoop-hdfs-3.2.1-tests.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\hadoop-hdfs-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\hadoop-hdfs-client-3.2.1-tests.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\hadoop-hdfs-client-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\hadoop-hdfs-httpfs-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\hadoop-hdfs-native-client-3.2.1-tests.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\hadoop-hdfs-native-client-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\hadoop-hdfs-nfs-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\hadoop-hdfs-rbf-3.2.1-tests.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\hadoop-hdfs-rbf-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\aopalliance-1.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\bcpkix-jdk15on-1.60.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\bcprov-jdk15on-1.60.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\ehcache-3.3.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\fst-2.50.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\geronimo-jcache_1.0_spec-1.0-alpha-1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\guice-4.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\guice-servlet-4.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\HikariCP-java7-2.4.12.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\jackson-jaxrs-base-2.9.8.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\jackson-jaxrs-json-provider-2.9.8.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\jackson-module-jaxb-annotations-2.9.8.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\java-util-1.9.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\javax.inject-1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\jersey-client-1.19.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\jersey-guice-1.19.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\json-io-2.5.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\metrics-core-3.2.4.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\mssql-jdbc-6.2.1.jre7.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\objenesis-1.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\snakeyaml-1.16.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\swagger-annotations-1.5.4.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-api-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-applications-distributedshell-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-applications-unmanaged-am-launcher-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-client-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-common-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-registry-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-server-applicationhistoryservice-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-server-common-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-server-nodemanager-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-server-resourcemanager-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-server-router-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-server-sharedcachemanager-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-server-tests-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-server-timeline-pluginstorage-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-server-timelineservice-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-server-web-proxy-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-services-api-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-services-core-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-submarine-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\mapreduce\lib\hamcrest-core-1.3.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\mapreduce\lib\junit-4.11.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\mapreduce\hadoop-mapreduce-client-app-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\mapreduce\hadoop-mapreduce-client-common-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\mapreduce\hadoop-mapreduce-client-core-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\mapreduce\hadoop-mapreduce-client-hs-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\mapreduce\hadoop-mapreduce-client-hs-plugins-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\mapreduce\hadoop-mapreduce-client-jobclient-3.2.1-tests.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\mapreduce\hadoop-mapreduce-client-jobclient-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\mapreduce\hadoop-mapreduce-client-nativetask-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\mapreduce\hadoop-mapreduce-client-shuffle-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\mapreduce\hadoop-mapreduce-client-uploader-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\mapreduce\hadoop-mapreduce-examples-3.2.1.jar
STARTUP_MSG:   build = https://gitbox.apache.org/repos/asf/hadoop.git -r b3cbbb467e22ea829b3808f4b7b01d07e0bf3842; compiled by 'rohithsharmaks' on 2019-09-10T15:56Z
STARTUP_MSG:   java = 1.8.0_221
************************************************************/
2019-10-13 12:01:58,567 INFO namenode.NameNode: createNameNode [-format]
Formatting using clusterid: CID-e07d7b19-b692-418b-a48d-d1b4cfabb51f
2019-10-13 12:01:59,508 INFO namenode.FSEditLog: Edit logging is async:true
2019-10-13 12:01:59,524 INFO namenode.FSNamesystem: KeyProvider: null
2019-10-13 12:01:59,525 INFO namenode.FSNamesystem: fsLock is fair: true
2019-10-13 12:01:59,527 INFO namenode.FSNamesystem: Detailed lock hold time metrics enabled: false
2019-10-13 12:01:59,535 INFO namenode.FSNamesystem: fsOwner             = stowl (auth:SIMPLE)
2019-10-13 12:01:59,535 INFO namenode.FSNamesystem: supergroup          = supergroup
2019-10-13 12:01:59,535 INFO namenode.FSNamesystem: isPermissionEnabled = true
2019-10-13 12:01:59,540 INFO namenode.FSNamesystem: HA Enabled: false
2019-10-13 12:01:59,595 INFO common.Util: dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO profiling
2019-10-13 12:01:59,611 INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit: configured=1000, counted=60, effected=1000
2019-10-13 12:01:59,611 INFO blockmanagement.DatanodeManager: dfs.namenode.datanode.registration.ip-hostname-check=true
2019-10-13 12:01:59,624 INFO blockmanagement.BlockManager: dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
2019-10-13 12:01:59,624 INFO blockmanagement.BlockManager: The block deletion will start around 2019 Oct 13 12:01:59
2019-10-13 12:01:59,628 INFO util.GSet: Computing capacity for map BlocksMap
2019-10-13 12:01:59,632 INFO util.GSet: VM type       = 64-bit
2019-10-13 12:01:59,634 INFO util.GSet: 2.0% max memory 889 MB = 17.8 MB
2019-10-13 12:01:59,637 INFO util.GSet: capacity      = 2^21 = 2097152 entries
2019-10-13 12:01:59,650 INFO blockmanagement.BlockManager: Storage policy satisfier is disabled
2019-10-13 12:01:59,650 INFO blockmanagement.BlockManager: dfs.block.access.token.enable = false
2019-10-13 12:01:59,658 INFO Configuration.deprecation: No unit for dfs.namenode.safemode.extension(30000) assuming MILLISECONDS
2019-10-13 12:01:59,659 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.threshold-pct = 0.9990000128746033
2019-10-13 12:01:59,663 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.min.datanodes = 0
2019-10-13 12:01:59,667 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.extension = 30000
2019-10-13 12:01:59,671 INFO blockmanagement.BlockManager: defaultReplication         = 1
2019-10-13 12:01:59,671 INFO blockmanagement.BlockManager: maxReplication             = 512
2019-10-13 12:01:59,671 INFO blockmanagement.BlockManager: minReplication             = 1
2019-10-13 12:01:59,672 INFO blockmanagement.BlockManager: maxReplicationStreams      = 2
2019-10-13 12:01:59,672 INFO blockmanagement.BlockManager: redundancyRecheckInterval  = 3000ms
2019-10-13 12:01:59,673 INFO blockmanagement.BlockManager: encryptDataTransfer        = false
2019-10-13 12:01:59,673 INFO blockmanagement.BlockManager: maxNumBlocksToLog          = 1000
2019-10-13 12:01:59,725 INFO namenode.FSDirectory: GLOBAL serial map: bits=29 maxEntries=536870911
2019-10-13 12:01:59,726 INFO namenode.FSDirectory: USER serial map: bits=24 maxEntries=16777215
2019-10-13 12:01:59,727 INFO namenode.FSDirectory: GROUP serial map: bits=24 maxEntries=16777215
2019-10-13 12:01:59,732 INFO namenode.FSDirectory: XATTR serial map: bits=24 maxEntries=16777215
2019-10-13 12:01:59,755 INFO util.GSet: Computing capacity for map INodeMap
2019-10-13 12:01:59,755 INFO util.GSet: VM type       = 64-bit
2019-10-13 12:01:59,757 INFO util.GSet: 1.0% max memory 889 MB = 8.9 MB
2019-10-13 12:01:59,763 INFO util.GSet: capacity      = 2^20 = 1048576 entries
2019-10-13 12:01:59,765 INFO namenode.FSDirectory: ACLs enabled? false
2019-10-13 12:01:59,766 INFO namenode.FSDirectory: POSIX ACL inheritance enabled? true
2019-10-13 12:01:59,766 INFO namenode.FSDirectory: XAttrs enabled? true
2019-10-13 12:01:59,771 INFO namenode.NameNode: Caching file names occurring more than 10 times
2019-10-13 12:01:59,781 INFO snapshot.SnapshotManager: Loaded config captureOpenFiles: false, skipCaptureAccessTimeOnlyChange: false, snapshotDiffAllowSnapRootDescendant: true, maxSnapshotLimit: 65536
2019-10-13 12:01:59,783 INFO snapshot.SnapshotManager: SkipList is disabled
2019-10-13 12:01:59,790 INFO util.GSet: Computing capacity for map cachedBlocks
2019-10-13 12:01:59,790 INFO util.GSet: VM type       = 64-bit
2019-10-13 12:01:59,791 INFO util.GSet: 0.25% max memory 889 MB = 2.2 MB
2019-10-13 12:01:59,797 INFO util.GSet: capacity      = 2^18 = 262144 entries
2019-10-13 12:01:59,814 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.window.num.buckets = 10
2019-10-13 12:01:59,815 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.num.users = 10
2019-10-13 12:01:59,817 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.windows.minutes = 1,5,25
2019-10-13 12:01:59,829 INFO namenode.FSNamesystem: Retry cache on namenode is enabled
2019-10-13 12:01:59,829 INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
2019-10-13 12:01:59,834 INFO util.GSet: Computing capacity for map NameNodeRetryCache
2019-10-13 12:01:59,835 INFO util.GSet: VM type       = 64-bit
2019-10-13 12:01:59,835 INFO util.GSet: 0.029999999329447746% max memory 889 MB = 273.1 KB
2019-10-13 12:01:59,836 INFO util.GSet: capacity      = 2^15 = 32768 entries
Re-format filesystem in Storage Directory root= D:\Hadoop\data\namenode; location= null ? (Y or N) y
2019-10-13 12:02:02,200 INFO namenode.FSImage: Allocated new BlockPoolId: BP-1688834052-10.0.0.23-1570986122192
2019-10-13 12:02:02,200 INFO common.Storage: Will remove files: []
2019-10-13 12:02:02,284 ERROR namenode.NameNode: Failed to start namenode.
java.lang.UnsupportedOperationException
        at java.nio.file.Files.setPosixFilePermissions(Unknown Source)
        at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.clearDirectory(Storage.java:452)
        at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:591)
        at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:613)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:188)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1206)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1649)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1759)
2019-10-13 12:02:02,297 INFO util.ExitUtil: Exiting with status 1: java.lang.UnsupportedOperationException
2019-10-13 12:02:02,306 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at DESKTOP-354IRPS/10.0.0.23
************************************************************/

SteveT

Posted 2019-10-13T20:49:37.183

Reputation: 11

Answers

1

This is a bug, I have struggled with it for a couple of days now and then found this: https://issues.apache.org/jira/browse/HDFS-14890

Setting permissions on name directory fails on non posix compliant filesystems and affects Hadoop 3.2.1 on Windows, it has been resolved in versions 3.3.0 and 3.2.2 - neither of which has been published yet, so you would need to install a different version of Hadoop

André du Plessis

Posted 2019-10-13T20:49:37.183

Reputation: 11

Welcome to Super User! While this may answer the question, it would be preferable to include the essential parts of the answer here, and provide the link for reference.

– MMM – 2019-11-29T16:12:18.033

-1

This is a bug. If you really want to install 3.2.1 now on Windows, please refer to my following post:

Basically I checkout 3.2.1 branch and manually fixed the code of file permission and then rebuild Hadoop locally on my Windows 10 machine.

Install Hadoop 3.2.1 on Windows 10

Compile and Build Hadoop 3.2.1 on Windows 10

Raymond Tang

Posted 2019-10-13T20:49:37.183

Reputation: 1

Welcome to Super User! Please quote the essential parts of the answer from the reference link(s), as the answer can become invalid if the linked page(s) change. – DavidPostill – 2020-01-24T20:31:11.037

Asked: 2019-10-13T20:49:37.183

Viewed: 379 times

Active: 2020-01-24T04:03:46.353