In this tutorial, I am going to show you how to use AWS SDK for .NET to do some basic file operations on an S3 bucket. AWS provides both low-level and high-level APIs. First, we are going to see how to use low-level APIs and then we will perform the same operations using high-level APIs.
In order to run the following code, you need to install AWSSDK.S3 Nuget package.
I have created a S3 bucket named: my-bucket-name-123 and I have created a folder named my-folder inside the bucket,
Low-Level APIs
The low-level APIs are mapped closely to the underlying REST API, here we use a Request object to provide request information and AWS responds with Response object.
Understanding S3 Path (when using low-level API)
First, we need to understand that there is no concept of folder in S3, everything is an object. If I want to create a folder called sub-folder, I need to append the folder name with a / to let AWS know that what I want is a folder not a file... so I need to create,
my-s3-bucket-name-123/my-folder/sub-folder/
If I don't include the trailing slash AWS will create an object called sub-folder instead of a folder.
Initializeing AmazonS3Client
This is how we initialize S3 client, which we are going to use for the remaining examples,
- string bucketName = "my-bucket-name-123";
- string awsAccessKey = "AKI............";
- string awsSecretKey = "+8Bo..................................";
- IAmazonS3 client = new AmazonS3Client(_awsAccessKey, _awsSecretKey, RegionEndpoint.APSoutheast2);
Creating a folder
Here we are going to create a folder called sub-folder inside my-folder
- string folderPath = "my-folder/sub-folder/";
- PutObjectRequest request = new PutObjectRequest()
- {
- BucketName = _bucketName,
- Key = folderPath // <-- in S3 key represents a path
- };
- PutObjectResponse response = client.PutObject(request);
Note1If you forget the trailing slash in the path (i.e. "my-folder/sub-folder") it would create an object called sub-folder.
Note2If you include a slash at the beginning of the path (i.e. "/my-folder/sub-folder/") it will create a folder with name as an empty string and put the remaining folders inside it.
Copying file into folder
The following code would copy test.txt inside sub-folder,
- FileInfo file = new FileInfo(@"c:\test.txt");
- string path = "my-folder/sub-folder/test.txt";
- PutObjectRequest request = new PutObjectRequest()
- {
- InputStream = file.OpenRead(),
- BucketName = _bucketName,
- Key = path // <-- in S3 key represents a path
- };
- PutObjectResponse response = client.PutObject(request);
Listing content of a folder
The following code would list the content of sub-folder,
- ListObjectsRequest request = new ListObjectsRequest
- {
- BucketName = _bucketName,
- Prefix = "my-folder/sub-folder/"
- };
- ListObjectsResponse response = client.ListObjects(request);
- foreach (S3Object obj in response.S3Objects)
- {
- Console.WriteLine(obj.Key);
- }
- // result:
- // my-folder/sub-folder/
- // my-folder/sub-folder/test.txt
Deleting file/folder
In the following code first we delete test.txt and then sub-folder,
- // delete test.txt file
- string filePath = "my-folder/sub-folder/test.txt";
- var deleteFileRequest = new DeleteObjectRequest
- {
- BucketName = _bucketName,
- Key = filePath
- };
- DeleteObjectResponse fileDeleteResponse = client.DeleteObject(deleteFileRequest);
- // delete sub-folder
- string folderPath = "my-folder/sub-folder/";
- var deleteFolderRequest = new DeleteObjectRequest
- {
- BucketName = _bucketName,
- Key = folderPath
- };
- DeleteObjectResponse folderDeleteResponse = client.DeleteObject(deleteFolderRequest);
High-level APIs
High-level APIs are designed to mimic the semanic of File I/O operations. They are very similar to working with FileInfo and Directory.
Understanding S3 path (when using high-level APIs)
When using high-level APIs, we need to use windows' styles paths, so use backslash (NOT slash) in your path,
"my-folder\sub-folder\test.txt"
Also note that, similar to low-level APIs we need a trailing backslash to indicate a folder, for example "my-folder\sub-folder\" indicates that sub-folder is a folder where as "my-folder\sub-folder" indicates that sub-folder is an object inside my-folder.
Initializeing AmazonS3Client
Use the same code as low-level APIs (above) to initilize AmazonS3Client.
Creating a folder
Here we are going to create a folder called high-level-folder and create another folder called my-folder indide it.
- string path = @"high-level-folder";
- S3DirectoryInfo di = new S3DirectoryInfo(client, _bucketName, path);
- if (!di.Exists)
- {
- di.Create();
- di.CreateSubdirectory("sub-folder");
- }
Copying file into folder
The following code would copy test.txt inside sub-folder,
- FileInfo localFile = new FileInfo(@"c:\test.txt");
- string path = @"high-level-folder\sub-folder\test.txt";
- S3FileInfo s3File = new S3FileInfo(client, _bucketName, path);
- if (!s3File.Exists)
- {
- using (var s3Stream = s3File.Create()) // <-- create file in S3
- {
- localFile.OpenRead().CopyTo(s3Stream); // <-- copy the content to S3
- }
- }
Listing content of a folder
The following code would list the content of a sub-folder,
- string path = @"high-level-folder\sub-folder\";
- S3DirectoryInfo di = new S3DirectoryInfo(client, _bucketName, path);
- IS3FileSystemInfo[] files = di.GetFileSystemInfos();
- foreach (S3FileInfo file in files)
- {
- Console.WriteLine($"{file.Name}");
- }
- // result:
- // test.txt
Deleting file/folder
In the following code first we delete test.txt and then sub-folder,
- // delete test.txt file
- string filePath = @"high-level-folder\sub-folder\test.txt";
- S3FileInfo s3File = new S3FileInfo(client, _bucketName, filePath);
- if (s3File.Exists)
- {
- s3File.Delete();
- }
- // delete sub-folder
- string folderPath = @"high-level-folder\sub-folder\";
- S3DirectoryInfo directory = new S3DirectoryInfo(client, _bucketName, folderPath);
- if (directory.Exists)
- {
- directory.Delete();
- }
Wrapping up
Apart from low-level and high-level APIs, AWS also provides TransferUtility, which runs on top of low-level API, which you can use for getting and puting objects. Have a look at AWS Documentation for some comparison between these options.
