Help for SOI Data Export Requests

Selecting the Data Sets

The top section of the request form contains a list of the SOI data sets matching your previous selections, with their internal dataset names and sizes. You must select at least one of these if you want any data exported. If there is only one dataset, it may be pre-selected.

An electronic mail address is required so that you may be notified when your data export request has been fulfilled, or in case of any problems encountered in exporting the data. Your SolarMail id is acceptable, as is an account name on the Stanford SSSC systems; otherwise please provide your full email address.

Options for Data Export

SOI data sets are available for export via the following mechanisms: Each of the export mechanisms and associated requirements are detailed below.

Copy to Local Disk

The easiest and fastest way to obtain data is to provide the name of a directory on a file system mounted either locally or via nfs on tarax, the SOI data server. This requires that you have an account on the Stanford SSSC systems; accounts are automatically provided to all SOI team members.

The directory name you provide must be that of a directory that is writeable and executable by the daemon process. This can be assured by giving the directory world permissions (umask 777) or by placing the directory in group SOI with standard permissions (umask 775). Before executing your request, the server will verify that the directory is writeable and has sufficient space to hold the requested data sets; if not, you will be notified promptly.

Copy to Remote Disk

If you do not have an account at the SSSC, or if you would like to have the data transferred to your machine at another site, you can have the data automatically copied into the directory you specify on the target machine. This requires that you give the daemon process on tarax permission to access your machine as yourself. No password is required, but you must place the following line in a file named .rhosts in your home directory (it is assumed that this is a unix machine):
tarax.Stanford.EDU daemon
Some systems require that the .rhosts file be unreadable by anyone but you (umask 600); it is a good idea in any case. You must of course provide your username on the machine, the hostname with its full Internet domain, and the name of a directory on that machine writeable by you and with sufficient free space to hold the data requested.

The server will promptly verify that you have provided it access to your account, but no verification of directory writeability or free space is performed in this case.

ftp to Remote Site

As an alternative to the rcp option described above, you can arrange to have the requested data transferred via ftp to the machine and directory specified. This requires that you provide the hostname, a user name to perform the transfer under (anonymous is the default), the name of a directory writeable by the user designated, and a password. In the case of anonymous ftp the password is usually a dummy, but the choice of directories accessible to the server is limited. If you provide a password of a real account, please note that the password will be echoed when you fill out the form and sent in the clear.

No verification will be performed prior to actually attempting the ftp. If the transfer fails the error notification may not be completely clear.

Stage for ftp from Remote Site

If you cannot or do not wish to have the data exporter place requested data on your own disk, you may place the data on a local cache disk accessible to anonymous ftp from outside. The data files and directories from a single request can be optionally tar'd together and compressed or zipped for ease of transfer. The data files or directories from a given request will be placed under the directory ftp://sun.stanford.edu/data/user@host/prog.lev.ser.ser#.tar (if tar'd), or ftp://sun.stanford.edu/data/user@host/prog/lev/ser.ser# where prog, lev, ser, and ser# refer to the program, level, and series name and series number of the dataset. If multiple datasets are requested and tar'd together the filename of the tar file will be constructed from the first dataset in the request.

You will automatically be notified of the location by email when the data are placed in the cache. Because of limited cache size, exported data will be purged after two days. There is a 3.9 GB limit to the size of individual requests for data staging to the anonymous ftp site.

Tapes

Datasets too large to be transferred to disk automatically over the network can be exported on tape media which will be mailed to you. Three different tape formats are supported: Exabyte 8mm tapes (2.5 or 5 GB), DAT 4mm tapes, and large Ampex 19mm tapes (270 GB). The request form must specify which format is desired and you must provide your mailing address.

Offline data requests are not fulfilled automatically. They are forwarded to an operator who will verify suitability. Requests from SOI team members will be fulfilled, although in the case of very large requests there may be some charge for media cost. Other people requesting data on tape may be asked to provide scientific or educational justification, and a fee to cover media and copying costs may be charged, subject to negotiation. In general, hard-copy export should be the mode of last resort. We are trying to make it as easy as possible for you to get data onto your disk directly.


Page last revised Tuesday, 11-Jan-2000 13:34:04 PST
Please address comments and questions to Rick Bogart, SOI Data Scientist


SOI Data - SOI Home