...
| Note |
|---|
Please note that after six months, data is moved from spinning disk to tape. Users are requested to download their data as soon as possible after observations have been done. At the moment, the only data products that are restaged from tape are the flags and visibilities. No image products are restored. Users should be prepared to image the data themselves. |
The SARAO archive can be found on https://archive.sarao.ac.za/.
Once you have logged in, you will be taken to a screen that lists all observations that you have access to. If you are not on any active proposals, you will only see non-proprietary data consisting of observations past their proprietary periods.
...
Most users would want their data in the CASA Measurement Set Format (MS). MeerKAT data is stored in a native format known as MeerKAT Visibility Format (MVF) which is a distributed structure. The in-house package katdal (KAT data access layer) is used to retrieve the data and the script mvftoms.py is used to convert to MS format. Note that this process can take several days, depending on the size of the dataset and the load on our transfer servers. Once the file is ready, you will receive an email with a download link.
| Note |
|---|
The direct download link created for your measurement set is valid for a month. Please contact the SARAO helpdesk in advance should you need this period extendedensure to download the dataset link taking into account the link expiry date received automatically. |
Since the last archive update of 9 March 2022, users will be taken directly to the configuration options for measurement set configuration. The estimated dataset size is indicated in the top left, next to the ‘Create Direct Download’ button.
Note that currently the old MS file will need to be cleared manually by the archive support team MS file link automatically generated has a validity period of 60 days that is send directly to your registered credentials. Once the link-validity period has expired, then only one is able to re-request another set of MS parameters as part of the SARAO data life-cycle policy. So choose wisely before you can generate a dataset with new parameters, so choose wisely. If you run into problems, please contact the helpdesk for assistance.
...
| Note |
|---|
A note to observers of strong spectral lines: The SDP flagger is very good at finding narrow lines and flagging them, particularly Galactic HI and maser lines. In this case you should not apply the ingest_rfi flags. It is strongly advised that you keep cam and data_lost flags since these are due to system issues such as antennas losing target lock or correlator faults. |
...
| Note |
|---|
Katdal has a few quirks when it can’t access data. You may end up with a dataset consisting of zeros only if the data is not available e.g. it is still on tape. We are currently working on improving error handling and logging. If your tarred MS is unexpectedly small, this may have happened to you. Please check on the status of the dataset and request restaging if it has been archived. There have recently been instances where downloaded measurement sets had scans fully flagged, yet the calibration report shows good solutions on these scans and no flags. We believe this occurs when the connection to the archive and the transfer servers times out due to network glitches. In this case, please refresh your browser and wait for the interface link to be recovered. Alternatively, contact the helpdesk to reset the file flags and have the conversion redonewith details to your observation. |
LTO 7 tape
Large projects can arrange with SARAO to have their data shipped on tape, at their own expense.
...