download free 30 days trial version buy bucket explorer
Documentation  Download  Purchase  Support  FAQs   Forum   ScreenShots & Demos

Amazon S3 Metadata (HTTP Header) - Introduction

HTTP Headers: You can specify metadata for Amazon S3 Objects (Files), which are Name- Value pairs, which can be sent along with Amazon S3 PUT Request, similar to other standard HTTP headers. Once you upload the S3 Object, you cannot update the Object metadata on Amazon S3. The only way to modify the Object Metadata is to make a copy of the Object and set the Metadata.

When any file is shared to and from the web server, it uses some or other protocol, and these protocols also transmit some common information in header section of that particular file. This information provides basic and important details about the file such as the file type, date or creation/modification, etc.

The attached information can be of two types; protocol defined and user defined . Protocol defined information is attached automatically as defined in specifications of that protocol. This information helps the files to be recognized under that protocol domain. User defined Metadata on the other hand is specified by the creator of the file. This information helps the receiver to understand the file details without the need to open it. We can understand this metadata by the example of “ Content-Type ” which is system defined.

Content-Type ” has different values as per file type. It shows the type of file which is stored on the server.


Sr. No Value Description
1 text/plain simple text messages
2 text/html Html content
3 image/jpg images
4 audio/mp3 Audio
5 video/mp4 Video
6 application/mswork Application

Amazon S3 also offers both aforementioned Meta Data types. Each Amazon S3 Object has a set of key-value pairs with which it is associated. There are two kinds of metadata: system metadata and user metadata.

There are two types of Metadata:

  1. System metadata : System metadata is used and is sometimes processed by Amazon S3. It depends on the API (REST or SOAP) being used.
  2. User metadata : User metadata has nothing to do with Amazon's functionality. User metadata or custom headers are specified by you.  Metadata added by you can be of any length but must conform to US-ASCII when using REST and UTF-8 when using SOAP or browser-based uploads through POST . Amazon simply stores it and passes it back when you ask for it.

For both REST and SOAP requests to Amazon S3, user metadata size is limited to 2k bytes for the total length of all values and keys.

Metadata Interoperability via REST and SOAP:

When user metadata keys are stored using REST, it must begin with "x-amz-meta-" which distinguish them as custom HTTP headers. When this metadata is retrieved via SOAP, the x-amz-meta- prefix is removed. Similarly, metadata stored via SOAP will have x-amz-meta- added as a prefix when it is retrieved via REST or HTTP, except the Content-Type header.

When metadata is retrieved through REST API, Amazon S3 combines headers that have the same name (ignoring case) into a comma-delimited list. If some metadata contains unprintable characters, it is not returned. Instead, the "x-amz-missing-meta" header is returned with a value of the number of the unprintable metadata entries.

Few of the custom header samples are given below:

  • Expires suggested expiration time
    Syntax: "Expires" ":"HTTP-date"
    Examples: Expires: Thu, 31 st Dec 2009 16:00:00 GMT
  • From author(s) or person(s) who is taking responsibility for the message
    Syntax: "From" ":"mailbox"
    Example: From: info@chambal.com

Content-MD5
The base64 encoded 128-bit MD5 digest of the message (without the headers) according to RFC 1864. This header can be used as a message integrity check to verify if the data is the same data that was originally sent. Although it is optional, we recommend using the Content-MD5 mechanism as an end-to-end integrity check. 

Etag
The entity tag is a hash of the object. The Etag only reflects changes to the contents of an object, not its metadata. The Etag is determined when an object is created. For objects created by the PUT Object operation and the POST Object operation, the Etag is quoted, 32-digit hexadecimal string representing the MD5 digest of the object data. For other objects, the Etag may or may not be an MD5 digest of the object data. If the Etag is not an MD5 digest of the object data, it will contain one or more non-hexadecimal characters and/or will consist of less than 32 or more than 32 hexadecimal digits.

Bucket Explorer allows you to manage and customize all Amazon S3 HTTP Headers (Metadata). You can just click and type to specify the updated metadata values and Bucket Explorer internally can make a copy of the S3 Object for you and do all the work for you to modify the Object Metadata. This makes it easy for users to add Header on S3 Objects and upload Objects with Custom Headers and setting defaults.

 

You can perform the following HTTP Header operations to manage the S3 Object Metadata:

  • Set file name in Metadata: You can set file name in Metadata using S3 PUT Object request . Check the link to get more details about Set Filename in Metadata using Amazon REST API and using Bucket Explorer.
  • Set cache control header: You can set cache control header for S3 Object using PUT Object. Check the link to get more details about Cache Control Header and S3 Object using Amazon REST API and using Bucket Explorer.
  • Update MIME type for new Files Uploaded on Amazon S3: You can update MIME type (Content Type) of the files at the time of uploading. Check the link to get more details about MIME Type using Bucket Explorer.
  • Update User Metadata or Custom Headers on Amazon S3 Files: You can update user metadata or custom headers on Amazon S3 files. Check the link to get more details about Update Custom Metadata using Bucket Explorer.
  • Set, Update and Remove Default Metadata or Custom Header on Amazon S3 Bucket: You can set/edit and remove default metadata or custom header on Amazon S3 Bucket. Check the link to get more details about Bucket Default Metadata using Bucket Explorer.
  • Upload files on Amazon S3 with Custom Headers: You can upload files on Amazon S3 with custom header. Check the link to get more details about Upload Files with Custom Metadata using Bucket Explorer.
  • Manage HTTP Headers for Amazon S3 Objects: You can manage and customize all Amazon S3 HTTP Headers. Check the link to get more details about Manage HTTP Headers for Amazon S3 Objects using Bucket Explorer.