Presto hive connector S3 access denied

0

I have some hive external tables on my S3 buckets. To access the hive external tables from presto, I made presto hive catalogs for each S3 buckets as they have different aws_access_key and aws_secret_key under ${PRESTO_HOME}/etc/catalog.

hive_A.properties for s3 bucket A

connector.name=hive-hadoop2
hive.s3.use-instance-credentials=false
hive.metastore.uri=thrift://namenode:9083
hive.s3.aws-access-key=xxxxxxxxx1
hive.s3.aws-secret-key=xxxxxxxxx1
hive.non-managed-table-writes-enabled=true
hive.allow-drop-table=true

hive_B.properties for s3 bucket B

connector.name=hive-hadoop2
hive.s3.use-instance-credentials=false
hive.metastore.uri=thrift://namenode:9083
hive.s3.aws-access-key=yyyyyyyyy2
hive.s3.aws-secret-key=yyyyyyyyy2
hive.non-managed-table-writes-enabled=true
hive.allow-drop-table=true

After Presto has launched, I sent a presto query to one of the external tables in s3 bucket A using hive_B.properties. So presto definitely gave me an access denied error like this.

com.facebook.presto.spi.PrestoException: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 56B93979F0BC5209; S3 Extended Request ID: Q3kQ0TifWVnYpU3h09XPvGcsLgCrAWoT/RLrjPlmNV8/FAVup5MoB0GHeV3yUs/+IgW6aDVJqXI=)
    at com.facebook.presto.hive.BackgroundHiveSplitLoader$HiveSplitLoaderTask.process(BackgroundHiveSplitLoader.java:194)
    at com.facebook.presto.hive.util.ResumableTasks.safeProcessTask(ResumableTasks.java:47)
    at com.facebook.presto.hive.util.ResumableTasks.access$000(ResumableTasks.java:20)
    at com.facebook.presto.hive.util.ResumableTasks$1.run(ResumableTasks.java:35)
    at io.airlift.concurrent.BoundedExecutor.drainQueue(BoundedExecutor.java:78)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 56B93979F0BC5209; S3 Extended Request ID: Q3kQ0TifWVnYpU3h09XPvGcsLgCrAWoT/RLrjPlmNV8/FAVup5MoB0GHeV3yUs/+IgW6aDVJqXI=)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1695)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1350)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1101)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:758)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:732)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:714)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:674)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:656)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:520)
    at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4443)
    at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4390)
    at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4384)
    at com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:844)
    at com.facebook.presto.hive.s3.PrestoS3FileSystem.listPrefix(PrestoS3FileSystem.java:490)
    at com.facebook.presto.hive.s3.PrestoS3FileSystem.access$000(PrestoS3FileSystem.java:146)
    at com.facebook.presto.hive.s3.PrestoS3FileSystem$1.<init>(PrestoS3FileSystem.java:278)
    at com.facebook.presto.hive.s3.PrestoS3FileSystem.listLocatedStatus(PrestoS3FileSystem.java:276)
    at org.apache.hadoop.fs.FilterFileSystem.listLocatedStatus(FilterFileSystem.java:263)
    at com.facebook.presto.hive.HadoopDirectoryLister.list(HadoopDirectoryLister.java:30)
    at com.facebook.presto.hive.util.HiveFileIterator$FileStatusIterator.<init>(HiveFileIterator.java:130)
    at com.facebook.presto.hive.util.HiveFileIterator$FileStatusIterator.<init>(HiveFileIterator.java:118)
    at com.facebook.presto.hive.util.HiveFileIterator.getLocatedFileStatusRemoteIterator(HiveFileIterator.java:107)
    at com.facebook.presto.hive.util.HiveFileIterator.computeNext(HiveFileIterator.java:100)
    at com.facebook.presto.hive.util.HiveFileIterator.computeNext(HiveFileIterator.java:37)
    at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
    at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
    at java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1811)
    at java.util.stream.StreamSpliterators$WrappingSpliterator.lambda$initPartialTraversalState$0(StreamSpliterators.java:294)
    at java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.fillBuffer(StreamSpliterators.java:206)
    at java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.doAdvance(StreamSpliterators.java:161)
    at java.util.stream.StreamSpliterators$WrappingSpliterator.tryAdvance(StreamSpliterators.java:300)
    at java.util.Spliterators$1Adapter.hasNext(Spliterators.java:681)
    at com.facebook.presto.hive.BackgroundHiveSplitLoader.loadSplits(BackgroundHiveSplitLoader.java:261)
    at com.facebook.presto.hive.BackgroundHiveSplitLoader.access$300(BackgroundHiveSplitLoader.java:93)
    at com.facebook.presto.hive.BackgroundHiveSplitLoader$HiveSplitLoaderTask.process(BackgroundHiveSplitLoader.java:187)

So I tried the query with correct hive_A.propreties but Presto kept giving me the access denied error. When I restarted presto and sent the query again with correct hive_A.properties just before I sent the wrong one, I have no problem with getting the result.

Does anyone know how to fix using incorrect hive connector issue which is giving s3 access denied error without restarting presto and running the correct hive connector query first before running incorrect one?

user2782719

Posted 2019-05-09T08:18:11.583

Reputation: 1

No answers