Upload files and directories with swift in OpenStack

Author:

First, you need to install

swift command line utility

and here is how to do it: Install OpenStack swift client only
In general, you will need:

  1. username (–os-username) – Username
  2. password (–os-password) – Password
  3. authentication url (–os-auth-url) – The URL address, which authorize your requests, it generates a security token for your operations. Always use https!
  4. tenant name (–os-tenant-name) – Tenant is like a project.

All of the above information should be available from your OpenStack administrator.
For the examples we assume there is a container “mytest” (it’s like a main directory from the root). You cannot upload files in the root, because this is the place for containers only i.e. directories. You must always upload files under container (i.e. directory aka folder).

To upload a single file with swift cli execute:

myuser@myserver:~$ swift --os-username myuser --os-tenant-name mytenant --os-password mypass --os-auth-url https://auth-url.example.com/v2.0/ upload mytest ./file1.log 
file1.log

To upload multiple files with swift cli execute:

myuser@myserver:~$ swift --os-username myuser --os-tenant-name mytenant --os-password mypass --os-auth-url https://auth-url.example.com/v2.0/ upload mytest ./newfiles-1.log ./newfiles-2.log ./newfiles-3.log 
newfiles-2.log
newfiles-3.log
newfiles-1.log
myuser@myserver:~$ swift --os-username myuser --os-tenant-name mytenant --os-password mypass --os-auth-url https://auth-url.example.com/v2.0/ upload mytest upload mytest ./newfiles-*.log
newfiles-1.log
newfiles-2.log
newfiles-6.log
newfiles-3.log
newfiles-4.log
newfiles-5.log

Adding multiple files name or shell expand strings will upload the files in the OpenStack account.

To upload a directory and its content recursively

myuser@myserver:~$ swift --os-username myuser --os-tenant-name mytenant --os-password mypass --os-auth-url https://auth-url.example.com/v2.0/ upload mytest ./more-files
more-files/more-files4.log
more-files/files/file2.log
more-files/files/file4.log
more-files/more-files2.log
more-files/more-files1.log
more-files/files/file3.log
more-files/more-files3.log

As you can see if a directory is used with upload command all of the files and sub-directories will be uploaded recursively and the order is unknown.

List uploaded files

myuser@myserver:~$ swift --os-username myuser --os-tenant-name mytenant --os-password mypass --os-auth-url https://auth-url.example.com/v2.0/ list mytest
file1.log
files/file2.log
files/file3.log
files/file4.log
more-files/files/file2.log
more-files/files/file3.log
more-files/files/file4.log
more-files/more-files1.log
more-files/more-files2.log
more-files/more-files3.log
more-files/more-files4.log
newfiles-1.log
newfiles-2.log
newfiles-3.log
newfiles-4.log
newfiles-5.log
newfiles-6.log

Be careful here if you have millions of files under “mytest” it will try to list them all. Of course, probably there is a limit and it will output only a fixed number of files.

Environment variables

You can export the 4 required variables (OS_AUTH_URL, OS_USERNAME, OS_PASSWORD, OS_TENANT_NAME) in the shell to use shorter syntax. Be careful with the security and the password exported in the shell!

myuser@myserver:~$ export OS_AUTH_URL="https://auth-url.example.com/v2.0/"
myuser@myserver:~$ export OS_USERNAME="myuser"
myuser@myserver:~$ export OS_PASSWORD="mypass"
myuser@myserver:~$ export OS_TENANT_NAME="mytenant"
myuser@myserver:~$ swift list
mytest
mytest2
myuser@myserver:~$ swift upload mytest2 files/
files/newfiles-4.log
files/newfiles-3.log
files/newfiles-2.log
files/newfiles-6.log
files/newfiles-1.log
files/file4.log
files/newfiles-5.log
files/file2.log
files/file3.log
myuser@myserver:~$ swift list mytest2
files/file2.log
files/file3.log
files/file4.log
files/newfiles-1.log
files/newfiles-2.log
files/newfiles-3.log
files/newfiles-4.log
files/newfiles-5.log
files/newfiles-6.log

Create a container

As said the container is like a directory in the first level. You cannot upload files in the first level, only containers are allowed there i.e. directories. Create a container (aka directory) with the “post” swift command. To create “mytest” for the examples above you should do:

myuser@myserver:~$ swift --os-username myuser --os-tenant-name mytenant --os-password mypass --os-auth-url https://auth-url.example.com/v2.0/ post mytest
myuser@myserver:~$ swift --os-username myuser --os-tenant-name mytenant --os-password mypass --os-auth-url https://auth-url.example.com/v2.0/ list
mytest
myuser@myserver:~$ swift --os-username myuser --os-tenant-name mytenant --os-password mypass --os-auth-url https://auth-url.example.com/v2.0/ post mytest2
myuser@myserver:~$ swift --os-username myuser --os-tenant-name mytenant --os-password mypass --os-auth-url https://auth-url.example.com/v2.0/ list
mytest
mytest2

Leave a Reply

Your email address will not be published. Required fields are marked *