User Supplied Equipment (USE)

User Supplied Equipment (USE)

Introduction

MeerKAT supports a number of guest instruments/backends, which can be made available to general MeerKAT users subject to conditions which are outlined in relevant Calls for proposals.

In addition, some USEs operate commensally with all observations, and are enabled by default. These enable detections of rapid transients, and the Breakthrough Listen SETI program.

Signal flow for the MeerKAT correlator beamformer and backend systems

 

PTUSE

The Pulsar Timing User-Supplied Equipment (PTUSE) allows observers to use beamformed data from the 64-dish MeerKAT telescope to acquire data in three different modes for radio pulsar studies. Users can obtain data in fold-mode, filterbank (or search) mode and baseband mode, and some combinations thereof. Observations with sub-arrays are also supported, as are observations from different frequency bands, and from up to four coherent beams within the primary beam. In addition to pulsars, any time-variable source can in principle be observed in one of the pulsar search modes, such as RRATs, magnetars, giant pulse emitters or repeating FRBs.

It is not essential to include a MeerTime co-investigator in your proposal unless you want to try a new untested mode or record baseband data, although you might wish to check with an experienced MeerTime (https://www.meertime.org ) observer that your experiment will be routine before submitting your proposal.

Sub-arrays

The 64-dish MeerKAT telescope correlator can form tied array beams from different numbers of dishes, with the most common used for pulsar applications being the maximum 64 or 2x32. Due to limited dish availability, typical numbers are between 59-64 dishes for the 64-dish mode and 29-32 dishes for the two-sub-array modes.

It is possible to observe in three different subarrays - for example 32 dishes in one subarray and 2 x 16 antenna subarrays. It is possible to have three subarrays observing simultaneously in U-, L- and S-band.

Users should keep in mind that if the source position is uncertain, it might be worth reducing the diameter of the array so that the source will appear well inside the width of the tied beam. MeerKAT observations of globular clusters often drop (zero-weight) the outlying antennas (> 1000m from the core) so that most of the pulsars appear in the tied beam.

Frequency Bands

The most common frequency bands for pulsar observation are L-band (856-1712 MHz) and UHF (544-1088 MHz). At L-band the top and bottom 48 channels (of 1024) are usually omitted from analysis due to band roll-off giving an effective bandwidth of 775.75 MHz. To avoid aliasing and inter-channel bleeding, the usual 1024 channel mode at L-band downweights the channels at their edges as described in Bailes et al. (2020).

S-band has been recently commissioned. S-band has five potential bands from 1750-3500 MHz, each of which is 875 MHz wide.

Sensitivity

If the full array is available, the system temperature at L-band is approximately 18K and the gain near 2.8 K/Jy although this varies across the band. At UHF it is approximately 10% worse, but the sky background temperature is much higher, scaling as typically

. Since pulsars are steep spectrum objects with typical spectral indices of -1 to -3, the UHF band provides excellent opportunities if the pulsar is not scattered.

Modes

Fold mode

This is the most often used and best tested mode. Every 8s pulsars are coherently dedispersed with a nominal DM, folded at the apparent topocentric period of the pulsar and the data binned with 1024 bins, 1024 frequency channels and four Stokes parameters. If a calibrator has been observed (strongly recommended), the data are polarisation calibrated automatically and to a high degree of accuracy at L-band and UHF frequencies. Absolute flux calibration is not possible without assuming the radiometer equation is applicable, and the radio frequency interference has been excised. Observers specify an integration time, and a single psrfits PSRCHIVE-compliant archive is produced in psrfits format. The first and last integration are unreliable and normally discarded as the time lag between PTUSE start and stop times is unreliable at the few second level. 

We strongly recommend observing PSR J1909-3744 or another MeerKAT pulsar timing array pulsar for 256s to confirm the time-stamping of the observation is reliable. Other suitable pulsars (depending upon the RA of your targets) are: PSRs J0125-2327, J0437-4715, J1017-7156 or J1600-3053. Each of these pulsars should deliver a sub-microsecond accuracy TOA for comparison with long-running programs, such as the MeerTime/MeerKAT PTA.

Observers need to supply the coordinates of the source and a tempo2-compliant ephemeris by email to SARAO, as well as the desired integration time and frequency resolution. These are currently held in a single repository, and coordination with MeerTime will be required if you want to change a folding ephemeris. 

Observations appropriately labelled will be automatically transferred to the OzSTAR supercomputer at Swinburne and cleaned for RFI by the Meerpipe pipeline if desired. Obtaining an account on OzSTAR will enable raw data transfer to your own computer - email Matthew Bailes to obtain an account. Clock correction files are maintained by Dr Ryan Shannon, and he can be emailed for the latest files that take typically 1 month to be updated with the clock correction files. MeerTime is routinely achieving sub-100 ns timing on PSR J1909-3744.

If there is more than 1 source in the primary beam, a tied beam can be placed on up to four pulsars that can be observed simultaneously. Due to hardware memory limitations, the maximum spacing between pulsars is only ~15 arc minutes at 20cm at L-band.

 

If more than one tied array beam is required SARAO should be consulted to ensure the observing mode is feasible.

Some observers may wish to use more than 1024 frequency channels. Internally the PTUSE machines use dspsr to coherently dedisperse the data, and higher resolutions can be obtained. 8192 channels is reliable, 16384 is not recommended except at UHF.

To obtain data with higher numbers of channels, SARAO should be advised well ahead of time to check if the mode will function.

Rarely, observers use the 4096 channel mode of the beamformer, in which case PTUSE defaults to 4096 channels at the expense of time resolution. There is interchannel leakage of power which can lead to artefacts in the pulse profiles.

Filterbank or Search Mode

The 4 PTUSE machines can each produce coherently dedispersed filterbank data that has 1024 frequency channels, an integer number of time samples added together and up to four Stokes parameters. It is routine to take filterbank data on one PTUSE machine and folded data on another. It is also possible to, in real time, coherently dedisperse the data and do “on-the-fly-fscrunching” to a smaller number of frequency channels to avoid data overload. This is useful when employing the 4096-channel mode from the beamformer. Extremely high DM/frequency^3 ratios may fail and should be tested before use. 

The search mode data all record 8 bits per Stokes parameter, and the data are integrated typically for an integer number of time samples at the native time resolution of the filterbank. At UHF the fastest time resolution is therefore in multiples of 1024/(544e6) ~ 1.8823 microseconds, at L-band multiples of 1024/(856e6) ~ 1.196 microseconds and at S-band multiples of 1024/(875e6) ~ 1.1703 microseconds. However, attempts to record data at this rate will fail, because the disk drives can’t sustain the output data rate. The maximum data rate we recommend is 420 MB/s.

Screenshot 2024-05-31 at 15.14.51.png
The data rates for L-band 8-bit data for 1024 frequency channels. Using 4096 channels the data rates are four times this.

 

Screenshot 2024-05-31 at 15.13.39.png
The data rates for U-band 8-bit data for 1024 frequency channels. Using 4096 channels the data rates are four times this.

 

Screenshot 2024-05-31 at 15.15.25.png
The data rates for S-band 8-bit data for 1024 frequency channels. Using 4096 channels the data rates are four times this.

 

Observers specify a desired time sampling rate, and this is rounded to the nearest integer number of samples.  It is not recommended to attempt sampling rates less than 9

s. Search mode at S-band is untested at present but would probably work if the time sampling is at least 10
s. Slower sampling rates should be routine.

Baseband Mode

The PTUSE machines can, for a limited time, record the tied beam baseband data directly to disk. Unlike the other modes, observers will quickly fill the disks if their observations last for too long. Once on disk the data can be read by tools like dspsr and used for a multitude of studies but the PTUSE machines are required for other experiments and the data will need to be transferred off soon after the observation ceases. The raw data rate at L-band is 856x4=3.424 GB/s and the absolute maximum time the observation can last is the capacity of the disk divided by this rate, or about 7300/3.424~2000 seconds. The top and bottom 48 channels can be omitted to extend the integration time.

At UHF the experiment could last about 3300s or almost 1 hour per PTUSE machine. Anyone wishing to use this mode should be prepared to liaise closely with SARAO and Swinburne staff to ensure there is a pathway to both record and move the data from the machine once it is recorded. It may be necessary to enlist the support of MPIfR and the University of Manchester who have LTO-9 tape drives on site. It would be necessary for the observers to supply LTO-9 tapes for data transfer. Unfortunately the same switch that connects the MPIfR tape robot is used by MeerKAT and transfers are limited to 50 MB/s. For a 7 TB observation this means almost two days to transfer the data. It may be possible to enlist the help of Swinburne to process the data on the PTUSE machines when observations are not making use of them, but the PTUSE machines are meant to remain robust data acquisition machines, not general user machines and their use for baseband processing is discouraged.

In principle an observer could use all 4 PTUSE machines in a round robin/sequential fashion to record baseband observations up to a few hours in length. The baseband data are divided into 8 second files that can be read by dspsr. Observers should be aware that the baseband data have artefacts imprinted upon them by the polyphase filterbank that produces the channelised data.

Radio Frequency Interference

The UHF band is usually remarkably clean, with the exception of a small region a few 10s of MHz wide. The 20cm band is punctuated by satellite transmissions, but often ~80% of it is good for science. Observers are encouraged to read Bailes et al. (2020) PASA Vol 37, 28B for a complete description of the instrument and its performance.

Observers of slow pulsars should be aware that during lightning storms the instrument can be badly affected by lightning storms anywhere within 100s of km of the site. If the data are folded, there can be of order 1 – a few broadband bursts of RFI per 8s integration that can corrupt the observations and are very difficult or impossible to excise. Although these can occur during MSP observations, they don’t appear to be as big an issue.

Acknowledgement statements

Publications making use of PTUSE should include the following statement in addition to the standard MeerKAT acknowledgement:

Observations made use of the Pulsar Timing User Supplied Equipment (PTUSE) servers at MeerKAT which were funded by the MeerTime Collaboration members ASTRON, AUT, CSIRO, ICRAR-Curtin, MPIfR, INAF, NRAO, Swinburne University of Technology, the University of Oxford, UBC and the University of Manchester. The system design and integration was led by Swinburne University of Technology and Auckland University of Technology in collaboration with SARAO and supported by the ARC Centre of Excellence for Gravitational Wave Discovery (OzGrav) under grant CE170100004.

FBFUSE/APSUSE

The Filterbanking Beamformer User Supplied Equipment (FBFUSE) and Accelerated Pulsar Search User Supplied Equipment (APSUSE), developed by the Max-Planck-Institut für Radioastronomie (MPIfR), provide MeerKAT users the means to perform wide-field-of-view pulsar searches on coherently beamformed total power observations. The instruments consist of two GPU-accelerated computing clusters, with FBFUSE providing total power beamforming and APSUSE providing data recording and analysis.

Operating FBFUSE/APSUSE requires the support of an experienced user. Proposers interested in using the instrument are required to consult the MPIfR (FBFAPSUSE@mpifr-bonn.mpg.de) for a technical evaluation and must include at least one experienced user as a Co-I on their proposal.

Beamforming capabilities

FBFUSE supports up to 768 independently steerable beams. Users typically configure the beams as tilings of a given shape and size. Here a hexpack algorithm will attempt to fill a user-defined bounding box at a given overlap between beams (determined by the fractional gain response of the beam at some distance from its boresight). It is possible to configure both independently steered beams and multiple tilings together in a single observation. For detail on the beamforming and beam positioning methods used in FBFUSE we direct the reader to Chen et al. (2021).

Recording capabilities

When configured, the APSUSE instrument will attempt to record all beams produced by FBFUSE up to a maximum ingest rate of ~12 GB/s. Users may trade between beams, bandwidth and time resolution while remaining within this cap. The following configurations are currently supported:

Receiver

Maximum Nbeams

Time resolution

(

s)

Frequency Resolution

(kHz)

Nchannels

Receiver

Maximum Nbeams

Time resolution

(

s)

Frequency Resolution

(kHz)

Nchannels

L-band

768

153

418

2048

480

76

418

2048

288

76

209

4096

S-band

768

150

427

2048

480

75

427

2048

288

75

214

4096

UHF

480

120

266

2048

276

60

266

2048

276

120

133

4096