Working with S3 folders using the .Net AWS SDK

If you’ve been using S3 client in the AWS SDK for .Net you might have noticed that there are no methods that let you interact with the folders in a bucket. As it turns out, S3 does not support folders in the conventional sense*, everything is still a key value pair, but tools such as Cloud Berry or indeed the Amazon web console simply uses ‘/’ characters in the key to indicate a folder structure.

This might seem odd at first but when you think about it, there are no folder structure on your hard drive either, it’s a logical structure the OS provides for you to make it easier for us mere mortals to work with.

Back to the topic at hand, what this means is that:

• if you add an object with key myfolder/ to S3, it’ll be seen as a folder
• if you add an object with key myfolder/myfile.txt to S3, it’ll be seen as a file myfile.txt inside a myfolder folder, if the folder object doesn’t exist already it’ll be added automatically
• when you make a ListObjects call both myfolder/ and myfolder/myfile.txt will be included in the result

Creating folders

To create a folder, you just need to add an object which ends with ‘/’, like this:

public void CreateFolder(string bucket, string folder)
{
    var key = string.Format(@"{0}/", folder);
    var request = new PutObjectRequest().WithBucketName(bucket).WithKey(key);
    request.InputStream = new MemoryStream();
    _client.PutObject(request);
}

Here is a thread on the Amazon forum which covers this technique.

Listing contents of a folder

With the ListObjects method on the S3 client you can provide a prefix requirement, and to get the list of objects in a particular folder simply add the path of the folder (e.g. topfolder/middlefolder/) in the request:

var request = new ListObjectsRequest().WithBucketName(bucket).WithPrefix(folder);

If you are only interested in the objects (including folders) that are in the top level of your folder/bucket then you’d need to do some filtering on the S3 objects returned in the response, something along the line of:

// get the objects at the TOP LEVEL, i.e. not inside any folders
var objects = response.S3Objects.Where(o => !o.Key.Contains(@"/"));

// get the folders at the TOP LEVEL only
var folders = response.S3Objects.Except(objects)
                      .Where(o => o.Key.Last() == '/' &&
                                  o.Key.IndexOf(@"/") == o.Key.LastIndexOf(@"/"));