Rclone

Rclone is a command-line utility for synchronizing files and directories between a local machine and cloud storage.

Installing Rclone

You can download Rclone from the official website and follow the installation guides, also available on the official site. 

For Windows, simply download and run rclone.exe via the command line.

Configuring Rclone

To set up Rclone to work with our S3 object storage:

1. Create the rclone.conf file in the .config/rclone directory in the root of your home directory
2. Add the following content:

[hostman]
type = s3
provider = Other
env_auth = false
access_key_id = <Access_Key>
secret_access_key = <Secret_Access_Key>
endpoint = https://s3.hostman.com

Alternatively, you can use the configuration wizard, which will create a similar file. To launch the configuration wizard, run the command:

rclone config

Below is an example of setting up using the configuration wizard.

• No remotes found - make a new one

Enter n and then the name hostman.

No remotes found - make a new one
n) New remote
s) Set configuration password
q) Quit config
n/s/q> n
name> hostman

• Option Storage

Select Amazon S3 Compliant Storage Providers.

Storage> 4

• Option provider

Select Any other S3 compatible provider.

provider> 32

• Option env_auth

Enter false.

env_auth> false

• Option access_key_id

Enter the Access Key from your bucket settings. For example:

access_key_id> 6RNN1MM9UZOUU5XWVQUE

• Option secret_access_key

Enter the Secret Access Key from your bucket settings. For example:

secret_access_key> jX8DgkESG2uqTtQYAuT25FZhctrbz2mhyBrwUgKK

• Option region

Enter us-2.

region> us-2

• Option endpoint

Enter https://s3.hostman.com

endpoint> https://s3.hostman.com

• Option location_constraint

Leave blank and press Enter.

location_constraint>

• Option acl

Leave blank and press Enter.

acl>

• Edit advanced config?

Enter n.

y/n> n

• Configuration complete

Check the details and press y.

Options:
- type: s3
- provider: Other
- access_key_id: 6RNN1MM9UZOUU5XWVQUE
- secret_access_key: jX8DgkESG2uqTtQYAuT25FZhctrbz2mhyBrwUgKK
- region: us-2
- endpoint: https://s3.hostman.com

Keep this "hostman" remote?
y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote
y/e/d> y

• Current remotes

Enter q to exit.

Current remotes
Name                 Type
====                 ====
hostman2             s3

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q

Commands for Working with Storage

For a complete list of supported commands, refer to the Rclone documentation. 

The storage name specified in the rclone.conf configuration file should be used as the host name. In our example earlier, we configured a storage named hostman.

Viewing the List of Buckets

rclone lsd <host_name>:

For example:

rclone lsd hostman:

       -1 2021-12-10 11:34:51     -1 1d65c254-17ab-4b22-b262-1111fc006937
       -1 2022-01-13 10:05:47     -1 3b24945b-d373-4713-89ec-3dece346c7b3
       -1 2022-01-14 12:31:36     -1 bbf30ad7-8872-40fc-8d41-54537d894b7f

Getting a List of Files in a Bucket

rclone ls <host_name>:<bucket_name>

For example:

rclone ls hostman:3b24945b-d373-4713-89ec-3dece346c7b3

    13 test.txt
    13 test1.txt

Uploading a File to a Bucket

rclone -P copy <file_name> <host_name>:<bucket_name>

For example:

rclone -P copy test5.txt hostman:3b24945b-d373-4713-89ec-3dece346c7b3

Deleting a File in a Bucket

rclone delete <host_name>:<bucket_name>/<file_name>

For example:

rclone delete hostman:3b24945b-d373-4713-89ec-3dece346c7b3/test5.txt

Copying Files between Buckets

rclone copyto -vP --log-file=rclone_<source_bucket_name>.log --transfers=256 <source_host_name>:<source_bucket_name> <destination_host_name>:<destination_bucket_name>

Synchronizing Files Between a Bucket and a Local Directory

rclone -P sync <host_name>:<bucket_name> /path/to/directory

For example:

rclone -P sync hostman:3b24945b-d373-4713-89ec-3dece346c7b3 /root/test

For reverse synchronization, simply reverse the arguments.