Code: val df = spark.read.csv("s3a://sample_bucket//csvFiles/file.csv");
Error: 22/06/23 20:02:57 WARN impl.MetricsConfig: Cannot locate configuration: tried hadoop-metrics2-s3a-file-system.properties,hadoop-metrics2.properties org.apache.spark.sql.AnalysisException: Path does not exist: s3a://sample_bucket//csvFiles/file.csv; at org.apache.spark.sql.execution.datasources.DataSource$$anonfun$org$apache$spark$sql$execution$datasources$DataSource$$checkAndGlobPathIfNecessary$1.apply(DataSource.scala:557) at org.apache.spark.sql.execution.datasources.DataSource$$anonfun$org$apache$spark$sql$execution$datasources$DataSource$$checkAndGlobPathIfNecessary$1.apply(DataSource.scala:544) at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241) at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241) at scala.collection.immutable.List.foreach(List.scala:392) at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241) at scala.collection.immutable.List.flatMap(List.scala:355) at org.apache.spark.sql.execution.datasources.DataSource.org$apache$spark$sql$execution$datasources$DataSource$$checkAndGlobPathIfNecessary(DataSource.scala:544) at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:358) at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:223) at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:211) at org.apache.spark.sql.DataFrameReader.csv(DataFrameReader.scala:615) at org.apache.spark.sql.DataFrameReader.csv(DataFrameReader.scala:467) ... 49 elided
When I used the above code, got this error. The file path is taken from the UI directly, and I don't know why it is showing path does not exist error.
Please help me out. Thanks in advance.
