Best practices for using Cloud Files

What is Cloud Files used for?

You can use Cloud Files in many ways. It is strongest when
it is functioning as unlimited object storage on the cloud, or when
you use it as a website accelerator with a Content Delivery Network
(CDN). This article presents these use-case scenarios and provides some
recommendations for getting the most out of Cloud Files.

Object storage

At its core, Cloud Files is an object storage solution and is not designed for
high input/output operations per second (IOPS). Instead, Cloud Files is
designed for consistent reliability of data. The primary function of Cloud
Files is to ensure that your data is available when you ask for it. This works
best with relatively static files, as opposed to files that are frequently
updated. As a result, it is impractical to run a database out of Cloud Files.
You can't expect to write to the same object 20 times per second. Cloud Files
was designed so that when you write to an object in Cloud Files, that object
is there each and every time you call for it.

Organize content for web acceleration

When you use Cloud Files for web acceleration, a basic organizational
structure separates your content into different containers based on
object type, such as images, Cascading Style Sheets (CSS), JavaScript,
videos, uploaded content, and so on. This structure enables you to quickly
locate objects when you need them.

Container management

This section describes how to use containers to group your data in Cloud
Files.

Use multiple containers

If you have an extremely large number of objects, we recommend storing
them in multiple containers. You can have up to 500,000 containers per
account in Cloud Files.

Note: When you write large numbers of objects to a single container,
the limit of 100 object write requests per second per container might reduce
overall performance.

How to label your containers

When you organize your containers for Cloud Files, we recommend that you label
the container based on the type of storage (perhaps based on the segment of
the application accessing it). We also recommend that you attach an
incremental number to the label name to plan ahead in case the number of
objects becomes too high. For example, a container name might look something
like personnel-00000.

A basic setup for web acceleration organizes content in separate containers
such as images, CSS, JavaScript, videos, uploaded content, and so on. This
structure enables you to quickly locate objects when you need them.

Keep a local database of your container structure

If you have a large number of files, we also recommend that you
keep a local copy of the container structure and listing so that you don't
have to wait on the container to list all of the objects, 10,000 at a time.

You can do this in a local database, which significantly reduces the chance
of a naming conflict. With this approach, if there is an update to an object,
you know in which container the object is located. This enables a simple
update to the object without having to list all of the containers, which
significantly reduces the time required to complete updates.

This tip is most appropriate for customers who are using Cloud Files for an
object storage solution because these solutions are frequently accessed
programmatically, and also grow organically over time. This tip also applies
to any site that allows for additional content, such as an uploads section,
that might quickly grow beyond the website administrator's expectations.

Consider future growth in object count and usage. Any system that has
many millions of objects and very high object churn (such as creation and
deletion) should have a fragmentation or sharing system to minimize any
performance impacts.

You can keep track of the object count locally, and verify it against the
object count in the container by performing a HEAD operation on the
container and checking the Object Count.

Pathing

Containers in Cloud Files do not nest, and all objects in a
single container are subject to the same limitations. Some
applications fake a folder structure by adding the path to
the beginning of the object name, which works for pathing in the CDN
URLs as well. This structure enables virtual pathing if needed.
For object storage, this enables better subdivision of slow-growth,
closely-grouped data. As a result, you're unlikely to need to divide it
again later. For website acceleration, this enables pathing that
displays in the browser, as shown in the following example:

https://c123456.r02.cf3.rackcdn.com/ducks/funny/duckling.jpg

In this example, the name of the object is ducks/funny/duckling.jpg.

Remove containers and container data

You must delete all of the objects in a container before you can delete the
container itself. Multiple containers allow for better threading of the
deletion scripts. Because content grows regardless of the use case, it's
best to plan ahead for this situation.

Set the Time to Live

In addition to the performance benefit of quicker listings, you're also able
to set the Time to Live (TTL) value for containers that contain objects on the
CDN. You set the TTL on the Cloud Files container, and all of the objects
inherit the container-level setting. A particular object in a container cannot
have a TTL that is different than the other objects in the same container.
As a result, you might want to set longer TTL values on content that is
unlikely to change often.

For website acceleration, you might use a longer TTL (at the container level)
for files that are updated infrequently, such as your CSS, JavaScript, and
videos. At the same time, you can set a shorter TTL for rapidly changing
objects such as user uploads. Using an appropriate TTL for the object type in
your container improves performance. The longer the TTL, the more
consistent the performance because it renews the CDN cache from your Cloud
Files less frequently.