Good morning, I am trying to install Kylin-3.1.1 on a remote linux server, I made sure it had all software requirments and I installed already the following programms: apache-hive-3.1.2-bin , kylin-3.1.1-bin-hadoop3 , kafka_2.12-2.5.0 , hadoop-3.3.0 , and hbase-2.3.4. But something is going wrong. I have checked the environment with $KYLIN_HOME/bin/check-env.sh and no error apperaed, the checking was completed succesfully.
But whenever I try to start kylin by $KYLIN_HOME/bin/kylin.sh start it gives me the following error:
2021-02-15 07:35:22,542 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18] zookeeper.ZooKeeper:109 : Client environment:java.library.path=/home/cuquerella/hadoop-3.3.0/lib/native
2021-02-15 07:35:22,542 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18] zookeeper.ZooKeeper:109 : Client environment:java.io.tmpdir=/tmp
2021-02-15 07:35:22,542 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18] zookeeper.ZooKeeper:109 : Client environment:java.compiler=<NA>
2021-02-15 07:35:22,542 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18] zookeeper.ZooKeeper:109 : Client environment:os.name=Linux
2021-02-15 07:35:22,545 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18] zookeeper.ZooKeeper:109 : Client environment:os.arch=amd64
2021-02-15 07:35:22,546 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18] zookeeper.ZooKeeper:109 : Client environment:os.version=5.4.0-65-generic
2021-02-15 07:35:22,546 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18] zookeeper.ZooKeeper:109 : Client environment:user.name=cuquerella
2021-02-15 07:35:22,546 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18] zookeeper.ZooKeeper:109 : Client environment:user.home=/home/cuquerella
2021-02-15 07:35:22,546 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18] zookeeper.ZooKeeper:109 : Client environment:user.dir=/home/cuquerella/kylin-3.1.1-bin-hadoop3
2021-02-15 07:35:22,546 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18] zookeeper.ZooKeeper:109 : Client environment:os.memory.free=793MB
2021-02-15 07:35:22,546 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18] zookeeper.ZooKeeper:109 : Client environment:os.memory.max=4062MB
2021-02-15 07:35:22,546 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18] zookeeper.ZooKeeper:109 : Client environment:os.memory.total=990MB
2021-02-15 07:35:22,555 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18] zookeeper.ZooKeeper:868 : Initiating client connection, connectString=localhost:2181 sessionTimeout=90000 watcher=org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$14/900792976@1dda2073
2021-02-15 07:35:22,567 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18] common.X509Util:79 : Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation
2021-02-15 07:35:22,583 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18] zookeeper.ClientCnxnSocket:237 : jute.maxbuffer value is 4194304 Bytes
2021-02-15 07:35:22,598 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18] zookeeper.ClientCnxn:1653 : zookeeper.request.timeout value is 0. feature enabled=
2021-02-15 07:35:22,615 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18-SendThread(localhost:2181)] zookeeper.ClientCnxn:1112 : Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2021-02-15 07:35:22,625 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18-SendThread(localhost:2181)] zookeeper.ClientCnxn:959 : Socket connection established, initiating session, client: /127.0.0.1:46350, server: localhost/127.0.0.1:2181
2021-02-15 07:35:22,637 INFO [ReadOnlyZKClient-localhost:2181@0x5bf22f18-SendThread(localhost:2181)] zookeeper.ClientCnxn:1394 : Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x10000255ea6000b, negotiated timeout = 90000
Exception in thread "main" java.lang.IllegalArgumentException: Failed to find metadata store by url: kylin_metadata@hbase
at org.apache.kylin.common.persistence.ResourceStore.createResourceStore(ResourceStore.java:101)
at org.apache.kylin.common.persistence.ResourceStore.getStore(ResourceStore.java:113)
at org.apache.kylin.rest.service.AclTableMigrationTool.checkIfNeedMigrate(AclTableMigrationTool.java:99)
at org.apache.kylin.tool.AclTableMigrationCLI.main(AclTableMigrationCLI.java:43)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.kylin.common.persistence.ResourceStore.createResourceStore(ResourceStore.java:94)
... 3 more
Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=2, exceptions:
2021-02-15T07:35:23.302Z, RpcRetryingCaller{globalStartTime=1613374523078, pause=100, maxAttempts=2}, org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=2, exceptions:
2021-02-15T07:35:23.159Z, RpcRetryingCaller{globalStartTime=1613374523130, pause=100, maxAttempts=2}, java.io.IOException: No meta znode available
2021-02-15T07:35:23.263Z, RpcRetryingCaller{globalStartTime=1613374523130, pause=100, maxAttempts=2}, java.io.IOException: No meta znode available
2021-02-15T07:35:23.506Z, RpcRetryingCaller{globalStartTime=1613374523078, pause=100, maxAttempts=2}, org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=2, exceptions:
2021-02-15T07:35:23.404Z, RpcRetryingCaller{globalStartTime=1613374523402, pause=100, maxAttempts=2}, java.io.IOException: No meta znode available
2021-02-15T07:35:23.506Z, RpcRetryingCaller{globalStartTime=1613374523402, pause=100, maxAttempts=2}, java.io.IOException: No meta znode available
at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:145)
at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3014)
at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3006)
at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:464)
at org.apache.kylin.storage.hbase.HBaseConnection.tableExists(HBaseConnection.java:291)
at org.apache.kylin.storage.hbase.HBaseConnection.createHTableIfNeeded(HBaseConnection.java:316)
at org.apache.kylin.storage.hbase.HBaseResourceStore.createHTableIfNeeded(HBaseResourceStore.java:119)
at org.apache.kylin.storage.hbase.HBaseResourceStore.<init>(HBaseResourceStore.java:89)
... 8 more
Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=2, exceptions:
2021-02-15T07:35:23.404Z, RpcRetryingCaller{globalStartTime=1613374523402, pause=100, maxAttempts=2}, java.io.IOException: No meta znode available
2021-02-15T07:35:23.506Z, RpcRetryingCaller{globalStartTime=1613374523402, pause=100, maxAttempts=2}, java.io.IOException: No meta znode available
at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:145)
at org.apache.hadoop.hbase.client.HTable.get(HTable.java:383)
at org.apache.hadoop.hbase.client.HTable.get(HTable.java:357)
at org.apache.hadoop.hbase.MetaTableAccessor.getTableState(MetaTableAccessor.java:1166)
at org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:463)
at org.apache.hadoop.hbase.client.HBaseAdmin$6.rpcCall(HBaseAdmin.java:467)
at org.apache.hadoop.hbase.client.HBaseAdmin$6.rpcCall(HBaseAdmin.java:464)
at org.apache.hadoop.hbase.client.RpcRetryingCallable.call(RpcRetryingCallable.java:58)
at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:107)
... 15 more
Caused by: java.io.IOException: No meta znode available
at org.apache.hadoop.hbase.client.ZKConnectionRegistry.getMetaRegionLocation(ZKConnectionRegistry.java:139)
at org.apache.hadoop.hbase.client.ZKConnectionRegistry.lambda$getMetaRegionLocations$5(ZKConnectionRegistry.java:205)
at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68)
at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
at org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$ZKTask$1.exec(ReadOnlyZKClient.java:174)
at org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:342)
at java.lang.Thread.run(Thread.java:748)
2021-02-15 07:35:23,578 INFO [close-hbase-conn] hbase.HBaseConnection:138 : Closing HBase connections...
ERROR: Unknown error. Please check full log.
If I run it with a sudo command the answer is the following:
/home/cuquerella/kylin-3.1.1-bin-hadoop3/bin/set-java-home.sh: line 32: [: : integer expression expected
Retrieving hadoop conf dir...
/home/cuquerella/kylin-3.1.1-bin-hadoop3/bin/find-hadoop-conf-dir.sh: line 33: hbase: command not found
...................................................[PASS]
KYLIN_HOME is set to /home/cuquerella/kylin-3.1.1-bin-hadoop3
Checking HBase
...................................................[FAIL]
Please make sure the user has the privilege to run hbase shell
I tend to think it has to do with some problem when connecting to HBase cause whenever I ran a list it gives me the following error:
hbase(main):001:0> list
TABLE
ERROR: org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet
at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2921)
at org.apache.hadoop.hbase.master.MasterRpcServices.isMasterRunning(MasterRpcServices.java:1182)
at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:393)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318)
When I search in log of HBase I get the following outcome:
$ less hbase-2.3.4/logs/hbase-cuquerella-master-tfg.log
2021-02-13 18:12:42,107 ERROR [master/tfg:16000:becomeActiveMaster] master.HMaster: Failed to become active master
java.io.IOException: cannot get log writer
at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createAsyncWriter(AsyncFSWALProvider.java:126)
at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:651)
at org.apache.hadoop.hbase.regionserver.wal.AsyncFSWAL.createWriterInstance(AsyncFSWAL.java:128)
at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:797)
at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:531)
at org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.init(AbstractFSWAL.java:472)
at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:158)
at org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:63)
at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:288)
at org.apache.hadoop.hbase.master.region.MasterRegion.createWAL(MasterRegion.java:193)
at org.apache.hadoop.hbase.master.region.MasterRegion.open(MasterRegion.java:266)
at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:309)
at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104)
at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:948)
I would greatly appreciate if anybody could shed some light upon the problem. I am sorry for any mistakes I have done while asking this question cause I am a real beginner at this.
