[How To :]Copy Merge + S3 File System

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

[How To :]Copy Merge + S3 File System

Divya Gehlot

val srcPath =  s3n://bucket_name_1
val dstPath =s3n://bucket_name_2 
val config = new Configuration()
val fs = FileSystem.get(URI.create(srcPath), config)
FileUtil.copyMerge(fs, new Path(srcPath), fs, new Path(dstPath), false, config, null)

I am trying to use Copymerge .. Its not throwing error but its not merging the files from one bucket to another .

Would really appreciate the help.