Äîêóìåíò âçÿò èç êýøà ïîèñêîâîé ìàøèíû. Àäðåñ
îðèãèíàëüíîãî äîêóìåíòà
: http://www.atnf.csiro.au/vlbi/dokuwiki/doku.php/correlator/distribution
Äàòà èçìåíåíèÿ: Unknown Äàòà èíäåêñèðîâàíèÿ: Mon Apr 11 01:27:51 2016 Êîäèðîâêà: IBM-866 Ïîèñêîâûå ñëîâà: ngc 4676 |
Note: Data are now distributed via the Pawsey Data portal. See archiving notes. Below is only for reference in case an alternative is required.
The ftp server cira.ivec.org is no longer in use. Below are instructions for transfer of data to AARNet's CloudStor+ for distribution.
(Note there are various ways to do this. The instructions below use davfs2 to access your CloudStor+ area via the command line on cuppa. )
If you haven't already done so, you need to set a password for webdav access to CloudStor+ (this is different from your institutional login). To do this, log in to CloudStor+ on your web browser, click on your email address in the top right-hand corner of the page and go to your "Personal" page.
On cuppa21 (logged in to your own account):
You can make your own directory for distributing data to PIs (or geodetic analysts) on cuppa21 under:
/data/corr/corrdat/DistributeData/
e.g. /data/corr/corrdat/DistributeData/hayley/ownCloud
In your home area, create a directory called .davfs2
and within that, a file secrets
with the line (inserting your credentials):
https://cloudstor.aarnet.edu.au/plus/remote.php/webdav/ <username> <password>
Make sure this file is accessible only to you, with chmod 600 ~/.davfs2/secrets
Contact someone with superuser access to add you to the davfs2 group:
sudo usermod -aG davfs2 <user>
and to add the following line to /etc/fstab
(to mount in your area on cuppa21):
https://cloudstor.aarnet.edu.au/plus/remote.php/webdav/ /data/corr/corrdat/DistributeData/hayley/ownCloud davfs user,rw,noauto 0 0
davfs2 caches files for transfer and by default will use ~/.davfs2/cache
for this. This is a potentially fatal problem when trying to transfer large files as the /home partition on cuppa has limited space. The file being copied will be truncated when the /home partition is full (not to mention other problems that causes). To avoid this, create a directory in the data area on cuppa21 (outside of your ownCloud area), e.g.:
mkdir /data/corr/corrdat/DistributeData/hayley/davfs2-cache
Remove any existing ~/.davfs2/cache
, and create a soft link to the new area:
ln -s /data/corr/corrdat/DistributeData/hayley/davfs2-cache ~/.davfs2/cache
You should be able to mount your CloudStor+ area with (e.g.) mount /data/corr/corrdat/DistributeData/hayley/ownCloud
You can add the following to your .bashrc (or .mybashrc) if you want to mount automatically on login to cuppa21 (after a reboot etc.)
if [ -d /data/corr/corrdat/DistributeData/hayley/ownCloud ]; then mount /data/corr/corrdat/DistributeData/hayley/ownCloud fi
On cuppa (logged in to your own account on the correlator output data node):
To copy FITS files etc. from the correlator output area to a folder on CloudStor+ (can also just use scp or rsync if preferred):
gloPut7T.sh -m VX021A.FITS /data/corr/corrdat/vx021a hbignall@cuppa21 /data/corr/corrdat/DistributeData/hayley/ownCloud/VLBI/vx021a
To copy the pipeline outputs:
gloPut7T.sh /data/corr/pipe/vx021a/out hbignall@cuppa21 /data/corr/corrdat/DistributeData/hayley/ownCloud/VLBI/vx021a/pipeline-output
On your local web browser:
Go to your files on CloudStor+. Hover the mouse over the experiment folder to see the option òÀÜShareòÀÝ - select òÀÜShare with linkòÀÝ and send the link to the PI in the data release email.
Clicking on òÀÜDownloadòÀÝ for a folder will result in a zip file for download.
As an alternative to the web interface, the entire experiment folder may be retrieved with: wget -O {name}.zip òÀÜ{shared link}&downloadòÀÝ. However note: there may be a (2GB?) file size limit for zip file creation, so this may not be viable for most experiments (suspected due to getting òÀÜHTTP request sent, awaiting responseòÀæ 504 Gateway Time-outòÀÝ when trying to get a folder of ~2GB, whereas the wget option worked for a smaller folder.)
If your CloudStor+ area is getting close to full (100GB free storage limit), you can move older directories out to .. on cuppa21 and permanently remove from your òÀÜDeleted filesòÀÝ on CloudStor+. (Keeping them on cuppa21, provided there is space, means you can readily restore if the PI asks for access again.)