s3cmd and Walrus for your private object storage
What is s3cmd?
s3cmd is the command line tool to access files in the Amazon S3 object storage. It is written in Python.
What is Walrus?
Walrus is the object storage of Eucalyptus.
Few more terms
Objects : All files which are stored in the object storage are known as objects.
Key : The filename of an object is known as key.
Bucket : It is the name of the place where people store the files, it behaves like directory with some limitations. Buckets must have unique names.
Install the latest s3cmd from Eucalyptus fork
Install latest version of s3cmd using git in a virtual environment.
$ virtualenv s3
$ source s3/bin/activate
$ git clone https://github.com/eucalyptus/s3cmd
$ cd s3cmd
$ python setup.py install
Configuration file
The default configuration filename is .s3cfg
Following is a minimal example.
[default]
access_key = MRKEEDJHJKQRLMV3K2FB94
secret_key = 3Jy1VKDZmVpwdsffdf8d2PsmfhVojfAW8RmFO2FD
host_base = 192.168.1.100:8773
host_bucket = 192.168.1.100:8773
service_path = /services/Walrus
access_key and secret_key are from your eucarc file. You also know the ip of the host.
To list buckets
$ s3cmd ls
2013-12-24 16:27 s3://foolto
2013-10-19 13:56 s3://mybucket
2013-10-24 15:35 s3://official
2013-12-03 14:47 s3://snapset-5599d8c1-296f-480f-b2be-1e2b03933e42
To list contents of a bucket
$ s3cmd ls s3://foolto
DIR s3://foolto//t2/
2013-12-24 16:51 52680 s3://foolto/lscpu
To create a new bucket
$ s3cmd mb s3://testbucket
Bucket 's3://testbucket/' created
Uploading a file to the bucket
$ s3cmd put .s3cfg s3://testbucket/sample-configuration.txt
.s3cfg -> s3://testbucket/sample-configuration.txt [1 of 1]
213 of 213 100% in 0s 668.66 B/s done
In this example we uploaded the configuration file to the testbucket. We can list the bucket content once more.
$ s3cmd ls s3://testbucket/
2013-12-26 04:57 213 s3://testbucket/sample-configuration.txt
Downloading an object from the bucket
$ s3cmd get s3://testbucket/sample-configuration.txt
s3://testbucket/sample-configuration.txt -> ./sample-configuration.txt [1 of 1]
213 of 213 100% in 0s 1315.45 B/s done
Delete an object in the bucket
$ s3cmd del s3://testbucket/sample-configuration.txt
File s3://testbucket/sample-configuration.txt deleted
You can learn more about s3cmd from here. If you want to learn about more configuration values available, you can check this link.