You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 11 Next »

For the instruction here, you need Dropbox UC Enterprise account. Storage | University of Canterbury

Getting an Access Token

You need  `rclone` on NeSI/Hypocentre (or whatever Linux box). On NeSI, you can access it with module command on Mahuika (doesn't work on Maui)

module add rclone

You will be accessing Dropbox via SSH terminal, not via Webbrowser as you do normally, so you need to configure it and get an access code.

Follow the instruction in this page : Dropbox (rclone.org)  - The documentation can be a little bit outdated. Follow the default values until you are asked if you wish to use "autoconfig" - Say No

This will tell you to execute a command `rclone authorize "dropbox"` on a computer that has both web browser and rclone.

Install rclone on your computer, and open a command prompt (on Windows) or a terminal (on Mac/Linux), enter `rclone authorize "dropbox"`. This will open a web browser and show you an authorization screen. Select Dropbox (I think it is meant to be Enterprise).

The next page of your webbrowser will say something like "Success" (Sorry, no screenshot). This page itself has no important information. Go back to the terminal (or command prompt screen), and find your code. It says Paste the following into your remote machine......End paste. Copy that.


and go back to the terminal where you want to set up rclone. Paste the code. 



Quit the config. You should have a config file here.

$HOME/.config/rclone/rclone.conf

which looks like


[remote]
type = dropbox
client_id =
client_secret =
token = {"access_token":"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX","token_type":"bearer","expiry":"0001-01-01T00:00:00Z"}


Try

rclone lsd remote:


If you can see a output similar to below, you now have the access to Dropbox. Congratulations.

(python3_mahuika) baes@mahuika01: ~$ rclone lsd remote:
          -1 2020-10-20 01:00:56        -1 backup
          -1 2020-10-20 01:00:56        -1 emp_ds

Some useful commands.

List all the files in your dropbox

rclone ls remote:

To copy a local directory to a dropbox directory called backup

rclone copy v21p1 remote:Cybershake/v21p1 -vv

Meaning, everything under the directory v21p1 (note: v21p1 itself is excluded) will be copied to Cybershake/v21p1. It doesn't auto-makedir, so you should make a folder named "v21p1" under Cybershake first.

  • -vv gives a detailed log and progress.
  • You can assume as long as a file is listed on Dropbox folder, its upload is "complete" - it should be an exact copy of the original.
  • If you terminate the upload command, and restart, it will skip the files that have been uploaded already.
  • The upload is NOT fast. 50Gb takes about 5 hours. It's best to run it on the login node, but in "screen". For more info about screen, see Linux Screen Tutorial With Examples – POFTUT
  • Dropbox has undocumented maximum file size of 340Gb. If you r file is larger than that, split them into multiple parts before upload

split -b 340G $TARNAME "${TARNAME}.part"


To generate a link to file or folder to share

 rclone link remote:path [flags]



For more info,  check out https://rclone.org/dropbox/ 

  • No labels