Download - The FHD/εppsilon Epoch of Reionization Power Spectrum Pipeline … · 2019. 7. 24. · FHD/εppsilonEoRPSPipeline 3 Type Variables Definition B F Faradayrotation O Sourcepositionoffsets

Transcript
Page 1: The FHD/εppsilon Epoch of Reionization Power Spectrum Pipeline … · 2019. 7. 24. · FHD/εppsilonEoRPSPipeline 3 Type Variables Definition B F Faradayrotation O Sourcepositionoffsets

Publications of the Astronomical Society of Australia (PASA)doi: 10.1017/pas.2019.xxx.

The FHD/εppsilon Epoch of Reionization PowerSpectrum Pipeline

N. Barry1,2∗, A. P. Beardsley3, R. Byrne4, B. Hazelton4,5, M. F. Morales4,6, J. C. Pober7, and I.Sullivan81School of Physics, The University of Melbourne, Parkville, VIC 3010, Australia2ARC Centre of Excellence for All Sky Astrophysics in 3 Dimensions (ASTRO 3D)3School of Earth and Space Exploration, Arizona State University, Tempe, AZ 85287, USA4Department of Physics, University of Washington, Seattle, WA 98195, USA5eScience Institute, University of Washington, Seattle, WA 98195, USA6Dark Universe Science Center, University of Washington, Seattle, 98195, USA7Department of Physics, Brown University, Providence, RI 02906, USA8Department of Astronomy, University of Washington, Seattle, WA 98195, USA

AbstractEpoch of Reionization data analysis requires unprecedented levels of accuracy in radio interferometerpipelines. We have developed an imaging power spectrum analysis to meet these requirements andgenerate robust 21 cm EoR measurements. In this work, we build a signal path framework to mathemat-ically describe each step in the analysis, from data reduction in the FHD package to power spectrumgeneration in the εppsilon package. In particular, we focus on the distinguishing characteristics ofFHD/εppsilon: highly accurate spectral calibration, extensive data verification products, and end-to-enderror propagation. We present our key data analysis products in detail to facilitate understanding of theprominent systematics in image-based power spectrum analyses. As a verification to our analysis, wealso highlight a full-pipeline analysis simulation to demonstrate signal preservation and lack of signalloss. This careful treatment ensures that the FHD/εppsilon power spectrum pipeline can reduce radiointerferometric data to produce credible 21 cm EoR measurements.

Keywords: cosmology: dark ages, reionization, first stars – techniques: interferometric – methods: dataanalysis – instrumentation: interferometers

1 INTRODUCTION

Structure measurements of the Epoch of Reionization(EoR) have the potential to revolutionize our understand-ing of the early universe. Many radio interferometers arepursuing these detections, including the MWA (Tingayet al., 2013; Bowman et al., 2013; Wayth et al., 2018),PAPER (Parsons et al., 2010), LOFAR (Yatawatta et al.,2013; van Haarlem et al., 2013), HERA (Pober et al.,2014; DeBoer et al., 2017), and the SKA (Mellema et al.,2013; Koopmans et al., 2014). The sheer amount of datato be processed even for the most basic of interferometricanalyses requires sophisticated pipelines.Furthermore, these analyses must be very accurate;

the EoR signal is many orders of magnitude fainter thanthe foregrounds. Systematics from inaccurate calibra-tion, spatial/frequency transform artifacts, and othersources of spectral contamination will preclude an EoR

∗Contact: [email protected]

measurement. Understanding and correcting for thesesystematics has been the main focus of improvementsfor MWA Phase I EoR analyses, including the Fast Holo-graphic Deconvolution (FHD)/εppsilon pipeline, sincetheir description in Jacobs et al. (2016).FHD/εppsilon power spectrum analyses have been

prevalent in the literature, including data reduction forMWA Phase I (Beardsley et al., 2016), MWA PhaseII (Li et al., 2018), and for PAPER (Kerrigan et al.,2018), with HERA analyses planned. It is also flexibleenough to be used for pure simulation, particularly ininvestigating calibration effects (Barry et al., 2016; Byrneet al., 2019). Combined with its focus on spectrallyaccurate calibration, as well as end-to-end data-matchedmodel simulations and error propagation, FHD/εppsilonis well-suited for EoR measurements and studies.In this work, we describe the FHD/εppsilon power

spectrum pipeline in full with a consistent mathemat-ical framework, highlighting recent improvements and

1

arX

iv:1

901.

0298

0v2

[as

tro-

ph.I

M]

23

Jul 2

019

Page 2: The FHD/εppsilon Epoch of Reionization Power Spectrum Pipeline … · 2019. 7. 24. · FHD/εppsilonEoRPSPipeline 3 Type Variables Definition B F Faradayrotation O Sourcepositionoffsets

2 N. Barry et al.

key features. We trace the signal path from its originthrough the main components of our data reductionanalysis. Our final power spectrum products, includ-ing uncertainty estimates, are detailed extensively. Weperform many diagnostics with various types of powerspectra, which provide confirmations for our improve-ments to the pipeline.As further verification of the FHD/εppsilon power

spectrum pipeline, we present proof of signal preserva-tion. Confidence in EoR upper limits relies on our abilityto avoid absorption of the signal itself. A full end-to-endanalysis simulation within our pipeline proves that wedo not suffer from signal loss, thereby adding credibilityto our EoR upper limits.First, we detail the signal path framework in §2 to

provide a mathematical foundation. In §3, we build ananalytical description of FHD, focusing on the recentimprovements in calibration. §4 outlines the data prod-ucts from FHD and our choice of integration methodol-ogy. Propagating errors and creating power spectra inεppsilon is described in §5. Finally, we illustrate all ofour power spectra data products in §6, including proofof signal preservation.

2 THE SIGNAL PATH FRAMEWORK

The sky signal is modified during the journey from whenit was emitted to when it was recorded. In order touncover the EoR, the true sky must be separated fromall intervening effects, including those introduced via theinstrument. We describe all known signal modificationsto build a consistent, mathematical framework.There are three main types of signal modification:

those that occur before, during, and after interactionwith the antenna elements, as shown in Figure 1. Weadhere to notation from Hamaker et al. (1996) wheneverpossible in our brief catalog of interactions.

B: Before antenna• Faraday rotation from interaction with the

ionosphere, F.• Source position offsets O due to variation in

ionospheric thickness.• Unmodelled signals caused by radio frequency

interference (RFI), U.S: At antenna

• Parallactic rotation P between the rotatingbasis of the sky and the basis of the antennaelements.

• Antenna correlations from cross-talk betweenantennas, X.

• Antenna element response, C, usually referredto as the beam1.

1Hamaker et al. (1996) use C to primarily account for a rotatingfeed, however we incorporate the antenna response as well. We also

F O

U

XC P

K

A

DR T

Figure 1. The signal path through the instrument. There arethree categories of signal modification: before antenna, at antenna(coloured light blue), and after antenna. Each modification matrix(green) is detailed in the text and Table 1.

• Errors in the expected nominal configurationand beam model, D.

E: After antenna• Electronic gain amplitude and phase R from a

typical response of each antenna.• Gain amplitude changes from temperature ef-

fects T on amplifiers.• Gain amplitude and phase oscillations K due to

cable reflections, both at the end of the cablesand at locations where the cable is kinked.

• Frequency correlations A caused by aliasing inpolyphase filter banks or other channelisers.

Each contribution can be modelled as a matrix whichdepends on [f, t, P,AB]: frequency, time, instrumentalpolarisation, and antenna cross-correlations. The ex-pected contributions along the signal path are thusB = UOF, S = DCXP, and E = AKTR. The visibilitymeasurement equation takes the form

vmeas ∼ E S B Itrue + noise, (1)

expect and model polarisation correlation in the Jones matrices(§3.2), whereas Hamaker et al. (1996) does not.

Page 3: The FHD/εppsilon Epoch of Reionization Power Spectrum Pipeline … · 2019. 7. 24. · FHD/εppsilonEoRPSPipeline 3 Type Variables Definition B F Faradayrotation O Sourcepositionoffsets

FHD/εppsilon EoR PS Pipeline 3

Type Variables Definition

BF Faraday rotationO Source position offsetsU Unmodelled RFI

S

P Parallactic rotationX Cross-talk antenna correlationsC Antenna element responseD Errors in nominal configuration

E

R Gain amplitude and phaseT Temperature changesK Cable reflectionsA Frequency correlations

vItrue True skyvmeas Measured visibilitiesnoise Thermal noise

Table 1 Brief definitions of the variables used within thesignal path framework, organized by type. There are fourtypes, B) interactions that occur before the antenna ele-ments, S) interactions that occur at the antenna elements,E) interactions that occur after the antenna elements, andv) visibility-related variables.

where vmeas are the measured visibilities, Itrue is the truesky, B are contributions that occur between emission andthe ground, S are contributions from the antenna con-figuration, and E are contributions from the electronicresponse. We have included all known contributions andmodifications to the signal, including the total thermalnoise. However, there may be unknown contributions,hence we describe Equation 1 as an approximation. Allcomponents are summarised in Table 1 for reference.The visibilities can be condensed into vectors since

the measurements are naturally independent over the[f, t, P,AB] dimensions. However, the modification ma-trices can introduce correlations across the dimensions,and thus cannot be reduced without assumptions.One of the goals of an imaging EoR analysis is to

reconstruct the true sky visibilities, vtrue, given themeasured sky visibilities, vmeas. This is achieved throughthe process of calibration. We will use our generalizedframework described in this subsection to detail theassumptions and methodology for the FHD/εppsilonpipeline.

3 FAST HOLOGRAPHICDECONVOLUTION

FHD2 is an open-source radio analysis package thatproduces calibrated sky maps from measured visibilities.Initially built to implement an efficient deconvolutionalgorithm (Sullivan et al., 2012), its purpose is now to

2https://github.com/EoRImaging/FHD

serve as an vital step in EoR power spectrum analysis3.FHD is a tool to analyse radio interferometric data,

and has a series of main functions at its core: 1) creatingmodel visibilities from sky catalogues for calibration andsubtraction, 2) gridding calibrated data, and 3) makingimages for analysis and integration.Using the signal path framework described in §2, we

will describe the steps in building a gridding kernel(§3.2), forming model visibilities (§3.3), calibrating datavisibilities (§3.4), and producing images (§3.5).

3.1 Pre-pipeline flagging

Before any analysis can begin, the data must be RFI-flagged. Radio frequency interference (RFI), particularlyfrom FM radio and digital TV, can contaminate the data.However, RFI has characteristic signatures in time andfrequency which allow it to be systematically removedby trained packages.We use the package aoflagger4 to RFI-flag the

data (Offringa et al., 2015). This removes bright line-like emission, but has difficulty removing faint, broademission like TV. We completely remove any observa-tions that have signatures of TV. Therefore, we removecontributions from U by avoidance.As demonstrated in Offringa et al. (2019), averaging

over flagged channels can cause bias. However, this doesnot affect the nominal FHD/εppsilon pipeline because itavoids inverse-variance weighting of the visibilities. Anyfuture incorporation of inverse-variance weighting willneed to take this into account.

3.2 Generating the beam

The measurement collecting area of an antenna element,C, is commonly referred to as the primary beam. Adeep knowledge of the beam is critical for precision mea-surements with widefield interferometers (Pober et al.,2016). Since our visibility measurements are correlationsof the voltage response between elements, we must un-derstand the footprint of each element’s voltage responseto reconstruct images (Morales & Matejek, 2009).We build the antenna element response from finely

interpolated beam images. This happens in the in-strument’s coherency domain, or instrumental polar-isation. We assume each element has two physical com-ponents p and q which are orthogonal, so each visibil-ity correlation between elements a and b will have aP = {papb, paqb, qapb, qaqb} response in the coherencydomain (Hamaker et al., 1996). This is calculated fromthe two elements’ polarised response patterns. For ex-

3Deconvolution in FHD is generally only used in building skymodels for calibration and subtraction.

4https://sourceforge.net/p/aoflagger/wiki/Home/

Page 4: The FHD/εppsilon Epoch of Reionization Power Spectrum Pipeline … · 2019. 7. 24. · FHD/εppsilonEoRPSPipeline 3 Type Variables Definition B F Faradayrotation O Sourcepositionoffsets

4 N. Barry et al.

ample,

Cpaqb = Cqapb =((Jx,paJ∗x,pb + Jy,paJ∗y,pb

)(Jx,qaJ∗x,qb + Jy,qaJ∗y,qb

)) 12

, (2)

where Cpaqb is the beam response for a p componentin element a and a q component in element b, J is thevector field for an element (also known as the Jones ma-trix), and the subscript x, pa is the contribution of the pcomponent in element a for the coordinate x (and like-wise for the other subscripts) (Sutinjo et al., 2015). Eachmatrix is a function of spatial coordinates and frequency,and each operation is done element-by-element.The Jones matrices describe the transformation

needed to account for P, or parallactic rotation. Dueto the wide field-of-view and the lack of moving partsfor most EoR instruments, this is a natural requirement.Known inter-dipole mutual coupling, a form of X, isalso captured in the Jones matrices. We generate CPfor a pair of elements, which can be applied to all otheridentical element pairs.We then take the various beams in direction cosine

space {l,m} and Fourier transform them to get beamsin {u, v}. Since we plan on using the beam as a griddingkernel later on, the beam must vary as smoothly aspossible. This is achievable by hyperresolving beyond theusual uv-grid resolution of 1

2λ, down to typically 17000λ.

We create this beam once to build a highly-resolvedreference table.FHD does have the flexibility to generate unique

beams for each element given individual element meta-data, however this quickly increases computing resources.Instead, we can build a coarse beam per baseline withphase offsets in image space to account for pixel centeroffsets (Line, 2017), which is built on-demand ratherthan saved as a reference table. This corrects for oneform of D, or individual element variation and error.When we generate model visibilities, we want to be

as instrumentally accurate as possible. This necessitatesa kernel which represents the instrumental responseto the best of our knowledge. However, we can choosea separate, modified kernel for uv-plane generation inpower spectrum analysis, similar to a Tapered GriddedEstimator (Choudhuri et al., 2014, 2016). As long as theproper normalisations are taken into account in powerestimation, a modified gridding kernel acts as a weightingof the instrumental response. This image-space weightingwill correlate pixels; we investigate contributions frompixel correlations in the power spectrum in §6.1.

3.3 Creating model visibilities

We must calibrate our input visibilities. Due to our widefield-of-view and accuracy requirements, FHD simulatesall reliable sources out to typically 1% beam level inthe primary lobe and the sidelobes to build a nearlycomplete theoretical sky. For the MWA Phase I, the 1%beam level includes the first sidelobe out to a field-of-view of approximately 100°. By comparing these modelvisibilities to the data visibilities, we can estimate theinstrument’s contribution.

Generating accurate model visibilities is an importantstep in calibration. Therefore, the estimate of the skymust be as complete as possible, including source posi-tions, morphology, and brightness. For example, we canuse GLEAM (Hurley-Walker et al., 2017), an extragalac-tic catalogue with large coverage and high completeness,to model sources for observations in the Southern sky.For a typical MWA field off the galactic plane, we modelover 50,000 known sources using GLEAM. As for a typi-cal PAPER observation, we model over 10,000 knownsources above 1 Jy (Kerrigan et al., 2018).

Not all sources are unpolarised. There are cases wherea source can be linearly or circularly polarized. Onlyten out of the thousands of sources seen in a typicalMWA field off the galactic plane are known to be reliablypolarised (Riseley et al., 2018). Therefore, we assumethere are no polarised sources in making model visibili-ties. With this assumption, we avoid complications fromF, or Faraday rotation in the polarisation componentsdue to the ionosphere. Implicitly, we also assume theionosphere is not structured, therefore F does not affectunpolarised sources.We disregard source position offsets O, a contamina-

tion resulting from ionospheric distortions. We minimizethis contribution by excluding data which is significantlymodified by ionospheric weather. We determine the qual-ity of the observation using various metrics describedin Beardsley et al. (2016), which can be further sup-plemented by ionospheric data products (Jordan et al.,2017; Trott et al., 2018).

Not all sources are unresolved. We also can optionallyinclude extended sources by modelling their contributionas a series of unresolved point sources (Carroll, 2016).This can also be done for creating models of diffusesynchrotron emission (Beardsley, 2015). However, thediffuse emission is significantly polarised (Lenc et al.,2016) and difficult to include.

With these assumptions, we now have a reliable skymodel that we can use to generate model visibilities.For each point source in our catalogue, we perform adiscrete Fourier transform using the RA/Dec floating-point location and the Stokes I brightness. This resultsin a discretised, model uv-plane for each source withoutinstrumental effects; typically at 1

2 λ resolution. All uv-planes from all sources are summed to create a model

Page 5: The FHD/εppsilon Epoch of Reionization Power Spectrum Pipeline … · 2019. 7. 24. · FHD/εppsilonEoRPSPipeline 3 Type Variables Definition B F Faradayrotation O Sourcepositionoffsets

FHD/εppsilon EoR PS Pipeline 5

uv-plane of the sky, and this process is repeated foreach observation to minimize w-terms associated withthe instantaneous measurement plane (see §4 for morediscussion).

Once a model uv-plane is created with all source con-tributions, we simulate what the instrument actuallymeasures. The hyperresolved uv-beam from §3.2 is thesensitivity of the cross-correlation of two elements. Wecalculate the uv-locations of each cross-correlated visibil-ity, and multiply the model uv-plane with the uv-beamsampling function. The sum of the sensitivity multipliedby the model at the sampled points is the estimatedmeasured value for that cross-correlated visibility.

These visibilities represent our best estimate of whatthe instrument should have measured, disregarding anysource position offsets O, polarised Faraday rotation F,and diffuse emission. Any deviations from these modelvisibilities (whether instrumental or not) will manifestas errors in the comparison between the data and modelduring calibration.

Our model visibilities can be represented in the signalpath framework as

m ∼ S Itrue, (3)

where m are the estimated model visibilities, Itrue is thetrue sky, and S are contributions from the plane of themeasurement. We have not included any modificationsin the signal path from before the instrument, B, andhave instead chosen to use an avoidance technique foraffected data.

3.4 Calibration

We are left with one type of modification to the signalthat has not been accounted for by the model visibilitiesor by avoidance: E, the electronic response. This is whatwe classify as our calibration.

At this point in the analysis, we have a measurementequation that looks like

vmeas ∼ E m + noise. (4)

We assume the electronic response E varies slowly withtime, and thus does not change significantly over an ob-servation (e.g. 2minutes for the MWA). Due to ourmodel-based assumptions, we do not have any non-celestial time correlations, antenna correlations, or un-known polarisation correlations. The electronic responseis thus simply a time-independent gain G per observationwhich is independent per element and polarisation.

We begin by rewriting Equation 4 using these as-sumptions. The measured cross-correlated visibilitiesare a function of frequency, time, and polarisation.Individual elements are grouped into the sets A ={a0, a1, a2, . . . , an} and B = {b0, b1, b2, . . . , bn}, where a

and b iterate through antenna pairs. A visibility is mea-sured for each polarisation P = {papb, paqb, qapb, qaqb},where p and q iterate through the two orthogonal instru-mental polarisation components for each element a andb.The resulting relation between the measured visibili-

ties and the model visibilities is

vab,pq([fo, t]) ∼Ga,p(fo, fi)G∗b,q(fo, fi)mab,pq([fi, t])

+ nab,pq([fo, t]), (5)

where vab,pq([fo, t]) are the measured visibilities andmab,pq([fi, t]) are the model visibilities. Both are fre-quency and time vectors [f, t] of the visibilities over allA and B element pairs and over all P polarisation prod-ucts. Ga,p(fo, fi) is a frequency matrix of gains giveninput frequencies fi which affect multiple output fre-quencies fo for instrumental polarisations p for elementsin A (and likewise for q and B). Thermal noise n isindependent for each visibility. All variables used in thissection are summarised in Table 2 for reference.Our notation has been specifically chosen. Naturally

discrete variables (element pairs and polarisation prod-ucts) are described in the subscripts. Naturally contin-uous variables (frequency and time) are function argu-ments. We group frequency and time into a set [f, t] inthe visibilities to create vectors. Since frequency andtime are independent, this notation is more compact.In contrast, the gain matrices are not independent infrequency. A full matrix must be used to accuratelycapture frequency correlation due to A, which includesaliasing from common electronics like polyphase filterbanks or bandpasses.To reduce Equation 5 significantly, we make the as-

sumption that the frequencies are independent in G.This forces the frequency correlation contribution A = I,giving

Ga,p(fo, fi) ∼ diag(ga,p(f)). (6)

The instrumental gains g are now an independent vectorof frequencies for elements in A per instrumental polari-sations p (and likewise for q and B). We flag frequencychannels which are most affected by aliasing to makethis assumption viable. The effect of this flagging is de-pendent on the instrument; for the MWA, flagging every1.28MHz to remove aliasing from the polyphase filterbanks introduces harmonic contamination in the powerspectrum. Thus, enforcing this assumption usually hasconsequences in the power spectrum space.

We can now fully vectorise the variables in Equation 5:

vab,pq([f, t]) ∼diag(ga,p(f)) diag(g∗b,q(f))mab,pq([f, t])

+ nab,pq([f, t]). (7)

Page 6: The FHD/εppsilon Epoch of Reionization Power Spectrum Pipeline … · 2019. 7. 24. · FHD/εppsilonEoRPSPipeline 3 Type Variables Definition B F Faradayrotation O Sourcepositionoffsets

6 N. Barry et al.

Type Variables Definition

Observation& Scale

Parameters

f Measured frequencies of the observation.p, q Orthogonal instrumental polarisations of the elements in the array.a, b Elements in the array.t Time steps within an observation.

Sets

A = {a0, a1, a2, ...an} All elements of the array. A and B can be iterated separately to formcross-correlated element pairs.B = {b0, b1, b2, ...bn}

P = {papb, ...qaqb} Polarisations in the coherency domain between two elements.C = {cn, ...c0,

φn, ...φ0}Coefficients of a low-order amplitude and phase polynomial across thefrequency band.

L = {l0, l1, l2, ...ln} Sets of elements associated with cable lengths and types.

D = {c, τ, φ} Amplitude, mode, and phase of a cable reflection fit across the frequen-cy band.

T = {ρ0, ρ1, ρ2, ...ρn} Observation timing sets of physical time separations, such as pointings.[f, t] A combined set of all frequencies and times.

Groups,Matrices,& Vectors

αLAn element grouping, where parameters are per element group ratherthan per element.

θTA grouping of observation times, where parameters are per timinggroup rather than per observation time.

Ga,p(fo, fi)The full gain matrix for each element in group A per p where inputand output frequencies are correlated.

ga,p(f) A vectorised approximation of the gains G for each element in thegroup A per P over frequency.

mab,pq([f, t])A vector of the simulated model visibilities from a model sky withfrequency-dependent beam effects for each element pair ab andpolarisation product pq over the set [f, t].

nab,pq([f, t])A vector of the thermal noise for each element pair ab and polarisationproduct pq over the set [f, t].

vab,pq([f, t])A vector of the uncalibrated data visibilities for each element pair aband polarisation product pq over the set [f, t].

aa,p(f) The calculated auto-gain for each element in the group A per p overfrequency.

ηa,p(f) Scaling relation of the discrepancy between the measured sky and thecalibration model in the cross-correlations.

aa,p(f) The calculated auto-gain for each element in the group A per p overfrequency which has been scaled to match the cross-correlations.

FunctionsP( ) A polynomial fit as a function of frequency of the input.

R〈 〉 A resistant mean of the input vector over element set L (and option-ally a time group θT ). Outliers beyond 2σ are excluded in the average.

DiscreteFourier

Transforms

k The Nyquist frequency index of the Fourier dual of frequency.

κThe hyperfine sub-Nyquist frequency index of the Fourier dual offrequency, with resolution at 1/20th of k.

n The index of the frequency.N The total number of frequency channels.

τκ

The Fourier dual of frequency: a timing delay in the detection of thewaveform between one element and another. The κ index indicates itruns over the hyperfine index.

Table 2 Definitions of the variables used for calibration, organised by type.

Page 7: The FHD/εppsilon Epoch of Reionization Power Spectrum Pipeline … · 2019. 7. 24. · FHD/εppsilonEoRPSPipeline 3 Type Variables Definition B F Faradayrotation O Sourcepositionoffsets

FHD/εppsilon EoR PS Pipeline 7

The gains calculated from solving Equation 7 will en-code differences between the model visibilities and thetrue visibilities. This is a systematic; the gains will becontaminated with a non-instrumental contribution. Wecan reduce this effect with long-baseline arrays by ex-cluding short baselines from calibration since we lacka diffuse-emission model. However, this can potentiallyintroduce systematic biases at low λ (Patil et al., 2016).For the remainder of §3.4, we describe our attempts toremove the systematics encoded within the gains due toan imperfect model.

3.4.1 Per Frequency SolutionsEquation 7 can be used to solve for the instrumentalgains for all frequencies and polarisations independently.This allows the use of Alternating Direction Implicit(ADI) methods for fast and efficient solving of O(N2)(Mitchell et al., 2008; Salvini & Wijnholds, 2014). Dueto this independence, parallelisation can also be applied.Noise is ignored during the ADI for simplicity; the least-squares framework is the maximum likelihood estimatefor a Gaussian distribution. However, if any noise has anon-Gaussian distribution, it will affect the instrumentalgains.

We begin solving Equation 7 by estimating an initialsolution for g∗b,q(f) to force the gains into a region witha local minimum. Reliable choices are the average gainexpected across all elements or scaled auto-correlations.

With an input for g∗b,q(f), Equation 7 can then becomea linear least-squares problem.

χ2a,p([f, t]) =

∑b

∣∣∣vab,pq([f, t])−diag(ga,p(f)) diag(g∗b,q(f)) mab,pq([f, t])

∣∣∣2, (8)

where ga,p(f) is found given a minimisation of χ2a,p([f, t])

for each element a and instrumental polarisation p. Alltime steps are used to find the temporally constantgains over the observation. For computation efficiency,we have also assumed P = {papb, qaqb} (e.gXX and Y Yin linear polarisation) since these contributions are mostsignificant. Optionally, a full polarisation treatment canbe used given polarised calibration sources.

The current estimation of g∗b,q(f) is then updated withknowledge from ga,p(f) by adding together the currentand new estimation and dividing by 2. By updatingin partial steps, a smooth convergence is ensured. Thelinear least-squares process is then repeated with anupdated g∗b,q(f) until convergence is reached5.

3.4.2 BandpassThe resulting gains from the least-squares iteration pro-cess are fully independent in frequency, element, observa-

5We have found that allowing the first 10 iterations to onlyupdate the phase of g∗b,q(f) helps to converge faster.

tion, and polarisation. This is not a completely accuraterepresentation of the gains. It was necessary to makethis assumption for the efficient solving technique in§3.4.1, but we can incorporate our prior knowledge ofthe nature of the instrument and its spectral structureex post facto.

For example, we did not account for noise contribu-tions during the per-frequency ADI fit; this adds spuriousdeviations from the gain’s true value with mean of zero.Historically, we accounted for these effects by creating aglobal bandpass,∣∣gp(f ;α)

∣∣ =⟨∣∣ga,p(f)

∣∣⟩α, (9)

where the normalised amplitude average is taken over allelements α as a function of frequency to create a globalbandpass

∣∣gp(f ;α)∣∣ independent of elements. Figure 2

shows an example of the global bandpass alongside thenoisy per-frequency inputs for the MWA.This methodology drastically reduces noise contribu-

tions to the bandpass when there are many elements,and it reduces spectral structure contributions due toimperfections in the model (Barry et al., 2016). How-ever, this averaging implicitly assumes that all elementsare identical. We must use other schemes to capturemore instrumental parameters while maintaining spec-tral smoothness in the bandpass.The level of accuracy required in the calibration to

feasibly detect the EoR is 1 part in 105 as a functionof frequency (Barry et al., 2016). As seen in Figure 2,instruments can be very spectrally complicated. There-fore, more sophisticated calibration procedures must beused.Usually, these schemes are instrument-specific. For

example, in the MWA, sets of elements experience thesame attenuation as a function of frequency due tocable types, cable lengths, and whitening filters. Wegroup these elements into different cable length setsL = {l0, l1, l2, ...ln}. We get∣∣gl,p(f ;αl)

∣∣ = R⟨∣∣ga∈L,p(f)

∣∣ , 2σ⟩ , (10)

where R is the resistant mean function6 calculated overeach element set αL for each polarisation and frequency.We choose the resistant mean because outlier contribu-tions are more reliably reduced than median calculations.The variable change from a to α indicates one parameterper group of elements.

If more observations are available, we follow a similaraveraging process over time. If the instrument is stablein time, a normalised bandpass per antenna should benearly identical from one time to the next excludingnoise contributions and potential Van Vleck quantisa-tion corrections (Vleck & Middleton, 1966). Gains from

6The resistant mean function calculates the distribution of theamplitudes of a similar element set αl for each frequency andpolarisation and then calculates the mean of that distributionafter Gaussian 2σ outliers have been excluded.

Page 8: The FHD/εppsilon Epoch of Reionization Power Spectrum Pipeline … · 2019. 7. 24. · FHD/εppsilonEoRPSPipeline 3 Type Variables Definition B F Faradayrotation O Sourcepositionoffsets

8 N. Barry et al.

170 175 180 185 190 195

0.8

0.9

1.0

1.1

1.2

Global bandpass for zenith observation 8/23/2013

170 175 180 185 190 195

Frequency (MHz)

0.8

0.9

1.0

1.1

1.2

No

rma

lized

ca

libra

tio

n a

mplitu

de

YYXX

XX

YY

Figure 2. The MWA global bandpass for the zenith observationof August 23, 2013 for polarisations pp = XX (blue) and qq =Y Y (red). All the per-frequency antenna solutions used in theglobal bandpass average are shown in the background (grey).This historical approach greatly decreased expected noise on thesolutions.

different LSTs will have different spectral structure fromunmodelled sources, and thus an average will removeeven more of this effect.We create a time set T = {ρ0, ρ1, ρ2, ...ρn} where

times are grouped by physical time separations basedon the instrument. For example, we group the MWAobservations by pointings7 because they sample differentbeam errors, and thus averaging between pointings wouldremove this instrumental feature. Whenever possible, weuse∣∣gl,p(f ;αl,θρ)

∣∣ = R⟨∣∣ga∈L,p(f)

∣∣ , t ∈ T, 2σ⟩ , (11)

where θρ runs over observations within physical timeseparations in the set T and over as many days as ap-plicable. The variable change from t to θ indicates oneparameter per group of times.

3.4.3 Low-order polynomialsAn overall amplitude due to temperature dependenceT in the amplifiers must still be accounted for withinthe gains. These differ from day to day and element toelement, therefore they cannot be included in the averagebandpass. For well-behaved amplifiers, this varies slightlyas a function of frequency and is easily characterised witha low-order polynomial for each element and observation.In addition to fitting polynomials to the amplitude

as a function of frequency, we must also account forthe phase. There is an inherent per-frequency degen-eracy that cannot be accounted for by sky-based cali-bration. Therefore, we reference the phase to a specificelement to remove this degeneracy, which makes thecollective phases smooth as a function of frequency forwell-behaved instruments. We have found that using

7A pointing defines a group of observations with the sameelectronic delay. As the sky rotates throughout the night, differentelectronic delays are used to roughly point the instrument to thesame location in the sky.

a per-element polynomial fit as the calibration phasesolution has been a reliable estimate.For the amplitude, we fit

cnfn + ...+ c1f + c0 = P

( |ga,p(f)||gp(f ;α)|

), (12)

where the bandpass contribution, |gp(f ;α)|, is removedbefore the fit and cn are the resulting coefficients. Anybandpass contribution can be used; |gp(f ;α)| is just anexample. For the phase, we fit

φnfn + ...+ φ1f + φ0 = P

(arg ga,p(f)

), (13)

where the polynomial fit is done over the phase ofthe residual and φn are the resulting coefficients. Dueto phase jumps between −π and π, special care istaken to ensure the function is continuous across theπ boundary8. We can create a set of these coefficients,C = {cn, ...c1, c0, φn, ...φ1, φ0}, for easy reference.In all cases, we choose the lowest-order polynomials

possible for these fits on the amplitude and phase. Forexample, the MWA is described by a 2nd-order poly-nomial fit in amplitude and a linear fit in phase. Asa general rule, we do not fit polynomials which havemodes present in the EoR window.

3.4.4 Cable reflectionsReflections due to a mismatched impedance must alsobe accounted for within the gains (contribution K inthe signal path framework). Even though instrumentalhardware is designed to meet engineering specifications,residual reflection signals are still orders of magnitudeabove the EoR and are different for each element sig-nal path. Averaging the gains across elements and timein §3.4.2 artificially erased cable reflections from thesolutions, thus we must specifically incorporate them.Currently, we only fit the cable reflection for elementswith cable lengths that can contaminate prime loca-tions within the EoR window in power spectrum space(Beardsley et al., 2016; Ewall-Wice et al., 2016).

We find the theoretical location of the mode usingthe nominal cable length and the specified light traveltime of the cable. We then perform a hyperfine discreteFourier transform of the gain around the theoreticalmode

ga,p(τκ) =M−1∑m=0

(ga,p(fm)|gp(f ;α)|

− (cnfnm + ...+ c0) ei(φnfnm+...+φ0)

)e−2πiκmM , (14)

8We “unwrap” the phase by creating a new continuous planefrom the Riemann sheets. We then solve, and “rewrap.” If the phasevaries quickly, there can be ambiguity in which Riemann sheet toplace the phase, but this is not an issue with most instruments.

Page 9: The FHD/εppsilon Epoch of Reionization Power Spectrum Pipeline … · 2019. 7. 24. · FHD/εppsilonEoRPSPipeline 3 Type Variables Definition B F Faradayrotation O Sourcepositionoffsets

FHD/εppsilon EoR PS Pipeline 9

where τκ is the delay, [m,M ] ∈ Z, and κ is the hyper-fine index component. Typically, we set the range of κto be [kτo − 1

20k, kτo + 120k], where kτo is the index of

the theoretical mode and k is the index in the rangeof [0,M − 1] (Beardsley, 2015). Again, any bandpasscontribution can be used; |gp(f ;α)| is just an example.The maximum |ga,p(τκ)| around kτo is chosen as the

experimental cable reflection. The associated ampli-tude c, phase φ, and mode τ are then calculated togenerate the experimental cable reflection contributionce−2πiτf+iφ to the gain for each observation. Optionally,these coefficients can be averaged over sets of observa-tions and times, depending on the instrument. We cancreate a set of these coefficients, D = {c, τ, φ}, for easyreference.

3.4.5 Auto-correlation bandpassThe fine-frequency bandpass is a huge potential sourceof error in the power spectrum (Barry et al., 2016).As such, we have tried averaging along various axes toreduce implicit assumptions while maintaining spectralsmoothness in §3.4.2. However, there always remains anassumption of stability along any axis that we averageover, and this may not be valid enough for the level ofrequired spectral accuracy.We can instead use the auto-correlations to bypass

these assumptions. An auto-correlation should only con-tain information about the total power on the sky, instru-mental effects, and noise. No information about structureon the sky is encoded, so spectral effects from unmod-elled sources do not contribute. However, there are twomain issues with using auto-correlations as the bandpass:improper scaling and correlated noise.

• The scale of the auto-correlations are dominated bythe largest modes on the sky. Currently, we do nothave reliable models of these largest modes. Eithera large-scale calibration model must be obtained,or the scaling must be forced to match the cross-correlations.

• Thermal noise is correlated in an auto-correlation,and will contribute directly to the solutions. Trun-cation effects during digitisation and channelisationcan artificially correlate visibilities. This will re-sult in bit noise, which will also contribute to theauto-correlations.

Whether or not auto-correlations can effectively beused for the bandpass depends on the importance oftheir errors in power spectrum space and our ability tomitigate these errors. We solve for the auto-correlationgains via the relation,

aa,p(f) =

√〈vaa,pp([f, t])〉t〈maa,pp([f, t])〉t

, (15)

where aa,p(f) is the auto-gain for element a and polari-sation p as a function of frequency.

To correct for the scaling error and for the correlatedthermal noise floor, we define a cross-correlation scalingrelation,

ηa,p(f) =P(|ga,p(f ;αl,θρ, [C], [D])|

)P (aa,p(f)) , (16)

where P is a polynomial fit to the gains, usually limitedto just a linear fit. This captures the discrepancy betweenthe measured sky and the calibration model in the spaceof one cross-correlation gain. We then rescale the auto-gains to match the cross-correlations,

aa,p(f) = ηa,p(f) aa,p(f), (17)

where aa,p(f) is the scaled auto-gain for element a andpolarisation p as a function of frequency.

We have reduced scaling errors and correlated thermalnoise on the auto-gain solutions. However, we have notcorrected for bit noise, or a bias caused by bit trunca-tion at any point along the signal path. These errorsare instrument-specific and will persist into power spec-trum space. If the bits in the digital system are notartificially correlated due to bit truncation, then theauto-correlation can yield a better fine-frequency band-pass.

3.4.6 Final calibration solutionsOur final calibration solution using only cross-correlations is

ga,p(f ;αl,θρ, [C], [D]) =∣∣gl,p(f ;αl,θρ)

∣∣︸ ︷︷ ︸bandpass §3.4.2(

(cnfn + ...+ c0)ei(φnfn+...+φ0)︸ ︷︷ ︸

low-order polynomials §3.4.3

+ ce−2πiτf+iφ)︸ ︷︷ ︸reflection §3.4.4

,

(18)

for elements with fitted cable reflections, and

ga,p(f ;αl,θρ, [C]) =∣∣gl,p(f ;αl,θρ)∣∣︸ ︷︷ ︸

bandpass §3.4.2

(cnfn + ...+ c0)ei(φnfn+...+φ0)︸ ︷︷ ︸

low-order polynomials §3.4.3

, (19)

for all other elements. The bandpass amplitude solu-tion is generated over a set of elements with the samecable/attenuation properties (αl) and includes manyobservations within a timing set (θρ) covering manydays. The same bandpass is applied to all elements ofthe appropriate type and all observing times from therespective time set. In contrast, the polynomials and thecable reflection are fit independently for each observa-tion and per element. We divide the data visibilities bythe applicable form of gag∗b to form our final, calibratedvisibilities.

Page 10: The FHD/εppsilon Epoch of Reionization Power Spectrum Pipeline … · 2019. 7. 24. · FHD/εppsilonEoRPSPipeline 3 Type Variables Definition B F Faradayrotation O Sourcepositionoffsets

10 N. Barry et al.

Alternatively, our final calibration solution harnessingthe auto-correlations is

ga,p(f ;αl,θρ, [C], [D], a) =aa,p(f)︸ ︷︷ ︸

auto-gain §3.4.5

ei arg ga,p(f ;αl,θρ,[C],[D])︸ ︷︷ ︸phase from Eq. 18

, (20)

where the amplitude is described by the scaled auto-gains and the phase is described by the cross-gains.

For EoR science, we want to reduce spectral structureas much as possible but still capture instrumental pa-rameters. Our auto-gain calibration solution performsthe best in the most sensitive, foreground-free regions.Therefore, we currently use Equation 20 in creating EoRupper limits with the MWA. This is an active area ofresearch within the EoR community; reaching the levelof accuracy required to detect the EoR is ongoing.

3.5 Imaging

The final stage of FHD transforms calibrated data visi-bilities into a space where they can be combined acrossobservations. To begin this process, we perform an op-eration called gridding on the visibilities.

We take each complex visibility value and multiply itby the corresponding uv-beam in the coherency domaincalculated in §3.2 through a process called Optimal Map-Making. In essence, this takes visibility values integratedby the instrument and estimates their original spatialuv-response given our knowledge of the beam. Theseare approximations of the instrument power responsefor each baseline to a set of regular gridding points (e.g.Myers et al. (2003); Bhatnagar et al. (2008); Morales& Matejek (2009); Sullivan et al. (2012); Dillon et al.(2014); Shaw et al. (2014); Zheng et al. (2017)).

We separately perform gridding to individual uv-planes for calibrated data visibilities, model visibilities,and residual visibilities generated from their difference.By gridding each separate data product, we can make avariety of diagnostic images. For example, images gen-erated from residual visibilities help to ascertain thelevel of foreground removal in image space, and thus areimportant for quality assurance.We also grid visibilities of value 1 with the beam

gridding kernel to create natural uv-space weights. Thisgenerates a sampling map which describes how muchof a measurement went into each pixel. In addition,we separately grid with the beam-squared kernel tocreate a variance map. The variance map relates tothe uncertainty for each uv-pixel, which will be a vitalcomponent for end-to-end error propagation in εppsilon.We then have three types of uv-plane products: the

sampling map, the variance map, and the data uv-planes.From these, we can create weighted-data uv-planes usingthe sampling map and data uv-planes. All these products

are transformed via 2D FFTs to image space in slantorthographic projection9.At this point, the various images are made for two

different purposes: 1) the sampling map, variance map,and data image planes are for power spectrum packagesand 2) the weighted-data image plane is for diagnosticimages per observation. The result of calibrated dataand residual snapshot images for a zenith observationwith Phase I of the MWA is shown in Figure 3.

Our slant orthographic images are in a basis thatchanges with LST. Therefore, we perform a bilinearinterpolation to HEALPix pixel centers10 (Górski et al.,2005), which are the same for all LSTs. We interpolatethe calibrated data, model, sampling map, and variancemap to HEALPix pixel centers separately for use inεppsilon. If we want to combine multiple observations,we will need to do a weighted average. Therefore, we keepthe numerator (data) separate from the denominator(sampling map) for this purpose.

3.6 Interleaved cubes

The images that will be used for power spectrum analysisare split by interleaved time steps, grouped by even indexand odd index. The even–odd distinction is arbitrary;what really matters is that they are interleaved. Whilethis doubles the number of Fourier transforms to perform,it allows for crucial error analysis.

The sky should not vary much over sufficiently smallintegration intervals. Any significant variation can beattributed to either RFI (which has been accountedfor in §3.1) or thermal noise on the observation. Bysubtracting the even–odd groupings and enforcing con-sistent flagging, we should be left with the thermal noisecontribution to the observation.

We carry even–odd interleaved cubes throughout thepower spectrum analysis and check at various stagesagainst noise calculations. It is a robust way to ensurethat error propagation and normalisation has occurredcorrectly, and it allows us to build more diagnostic dataproducts while still building the cross-power spectrum(see §5.4).

Thus, there are at least eight images per polarisationproduct output from FHD for use in a power spectrumanalysis: the calibrated data, model, sampling map, andvariance map for each even–odd set.

4 INTEGRATION

In order to reduce noise to reach the signal-to-noiserequired to detect the EoR, we must now integrate.This requires integration of thousands of observations

9Slant orthographic projection: a flat projection of the sky thatis slanted to be parallel with the measurement plane.

10HEALPix: the Hierarchical Equal Area isoLatitude Pixeliza-tion of a sphere.

Page 11: The FHD/εppsilon Epoch of Reionization Power Spectrum Pipeline … · 2019. 7. 24. · FHD/εppsilonEoRPSPipeline 3 Type Variables Definition B F Faradayrotation O Sourcepositionoffsets

FHD/εppsilon EoR PS Pipeline 11

Figure 3. An example of images output from FHD: calibrated data (top) and residual (bottom) Stokes I images from the MWA for thezenith observation of August 23, 2013. There is significant reduction of sources and point spread functions in the residual. However, thediffuse synchrotron emission can be seen in the residual because it was not in the subtraction model.

Page 12: The FHD/εppsilon Epoch of Reionization Power Spectrum Pipeline … · 2019. 7. 24. · FHD/εppsilonEoRPSPipeline 3 Type Variables Definition B F Faradayrotation O Sourcepositionoffsets

12 N. Barry et al.

(Beardsley et al., 2013).FHD outputs a variety of data cubes that we must

integrate together before input into our next package,εppsilon (described in §5). These RA/Dec frequencyslices are numerators and denominators which createmeaningful maps in uv-space. The various cubes are:

Calibrated data cubes: HEALPix images for eachfrequency of the unweighted calibrated data.

Model data cubes: HEALPix images for each fre-quency of the unweighted generated model.

Sampling map cubes: HEALPix images for eachfrequency of the sampling estimate.

Variance map cubes: HEALPix images for each fre-quency of the variance estimate.

These cube types are split by instrumental polarisationand by an interleaved time sampling set of even and oddtime indices (detailed in §3.6). Each even–odd polari-sation grouping is added across observations, pixel bypixel, using the HEALPix coordinate system. Averagingin image space, rather than in uv-space, is a crucial as-pect of the analysis. Either approach can theoretically beused, but the computational requirements vary greatly.

The measurement plane is not parallel to the tangentplane of the sky for non-zenith measurements. The in-strument’s measurement plane appears tilted comparedto the wave’s propagation direction; this tilt causes ameasurement delay. Since each measurement plane willmeasure different phases of the wave propagation, dif-ferent uv-planes cannot be coadded. This is a classicdecoherence problem in Fourier space.There are two methodologies to account for this de-

coherence. The first is to project the measurement uv-plane to be parallel to the tangent plane of the sky. Thisis called w-projection since it projects {u, v, w}-spaceto {u, v, w = 0}-space (Cornwell et al., 2008). Unfor-tunately, the projection requires the propagation of aFresnel pattern for every visibility to reconstruct thewave on the w = 0 plane. This requires intense computa-tional overhead, but has been successfully implementedby other packages (Trott et al., 2016).

The second methodology is to integrate in image space.We create an image for each observation by perform-ing a 2D spatial FFT of the uv-plane, which results ina slant orthographic projection of the sky. This inher-ently assumes the array is coplanar; the integration timemust be relatively small and altitude variations mustbe insignificant or absorbed into unique kernel genera-tion (§3.2). For the MWA Phase I, a mean w-offset of0.15λ in the imaged modes is small enough to not be adominating systematic for current analyses.We can then easily interpolate from the slant ortho-

graphic projection to the HEALPix projection (Ordet al., 2010), which is a constant basis. We do not needto propagate waves in image space in order for coher-ent integration, thus it is computationally efficient in

comparison.

5 ERROR PROPAGATED POWERSPECTRUM WITH INTERLEAVEDOBSERVED NOISE

Error Propagated Power Spectrum with Interleaved Ob-served Noise, or εppsilon11, is an open source powerspectrum analysis package designed to take integratedimages and create various types of diagnostic and limitpower spectra. It was created as a way to propagateerrors into power spectrum space, rather than have esti-mated errors. Other image-based power spectrum anal-yses exist for radio interferometric data (Paciga et al.,2011; Shaw et al., 2014; Patil et al., 2014; Shaw et al.,2015; Dillon et al., 2015; Ewall-Wice et al., 2016; Patilet al., 2017), however, the end-to-end error propagationof εppsilon and CHIPS (Trott et al., 2016) is uncommon.The main functions of εppsilon are to transform in-

tegrated images into {u, v, f}-space (§5.1), calculateobserved noise using even–odd interleaving (§5.2), trans-form frequency to k-space (§5.3), and average k-spacevoxels together for diagnostic and limit power spectra(§5.4). Detailing these processes will build the ground-work needed to describe 2D power spectra, 1D powerspectra, and their respective uncertainty estimates in§6.

5.1 From integrated images to uv-space

The input products of εppsilon are integrated imagecubes per frequency. While this space was necessary forintegration, the error bars in image space are compli-cated; every pixel is covariant with every other pixel.Our measurements were inherently taken in uv-space,and that is where our uncertainties on the measurementare easiest to propagate.

We then Fourier transform the integrated image backinto uv-space using a direct Fourier transform betweenthe curved HEALPix sky and the flat, regularly spaceduv-plane. Since this happens once per observation inte-gration set, rather than once per observation, the slowerdirect Fourier transform calculation is computationallyfeasible. The modified gridding kernel described in Sec-tion 3.2 acts as a window in the HEALPix image. Inorder to propagate our uncertainties correctly, this mustbe done during the generation of the kernel.After transforming into uv-space, we calculate the

resulting {kx, ky}-values for each pixel. The uv-space isrelated to the wavenumber space by the simple trans-forms (Morales & Hewitt, 2004)

kx = u2πDM (z) ky = v2π

DM (z) , (21)

11https://github.com/EoRImaging/eppsilon

Page 13: The FHD/εppsilon Epoch of Reionization Power Spectrum Pipeline … · 2019. 7. 24. · FHD/εppsilonEoRPSPipeline 3 Type Variables Definition B F Faradayrotation O Sourcepositionoffsets

FHD/εppsilon EoR PS Pipeline 13

where DM (z) is the transverse comoving distance de-pendent on redshift (Hogg, 1999).In addition, we also perform the 3D pixel-by-pixel

subtraction of the integrated model from the integrateddata to create residual cubes. We could instead per-form this after we have transformed f to kz to save oncomputation time, but it is helpful to make diagnosticplots of the residual cube in {u, v, f}-space. By creatingdiagnostics at every important step in the analysis, wecan better understand our systematics.

5.2 Mean and noise calculation

We now have twenty various uv-products: calibrated,model, residual, sampling map, and variance map data,for each polarisation product pp and qq, and for eachinterleaved even–odd time sample set. We have keptthe numerators and denominators (data and weights)separate up until this point so that we can performvariance-weighted sums and differences.

First, we weight the calibrated data, model, and resid-ual by the sampling map, thereby upweighting well-measured modes and downweighting poorly measuredmodes. Second, we weight the variance map by thesquare of the sampling map to scale our uncertaintyestimates with our choice of weighting scheme. We thenperform sums and differences using our sampling-map-weighted uncertainty estimates as the weights,

µ =xeσ2e

+ xoσ2o

1σ2e

+ 1σ2o

n =xeσ2e− xo

σ2o

1σ2e

+ 1σ2o

, (22)

where µ is the mean, n is the noise, {e, o} are the inter-leaved even–odd sets, x is the sampling-map-weighteddata (calibrated data, model, or residual) for a giveneven–odd set, and σ2 is the sampling-map-weighted vari-ance map for a given even–odd set. The calculated meanand noise are the maximum likelihood estimates for aweighted Gaussian probability distribution, hence usingthe variables µ and n instead of µ and n.Finally, we also calculate our uncertainty estimates

using the maximum likelihood estimation:

ε2 = 11σ2e

+ 1σ2o

, (23)

where {e, o} are the interleaved even–odd sets and σ2

is the sampling-map-weighted variance map for a giveneven–odd set.

These analytical uncertainty estimates are propagatedthrough the analysis to create our uncertainty on thecross power spectrum. We assume that there are nocross correlations in the noise; each noise pixel in uv-space is assumed to be independent. Various techniques,including spatial/frequency windowing, increases pixel-to-pixel correlation. We investigate the strength of the

correlations by comparing the analytic noise propagationto the observed noise in the power spectrum in §6.1.

5.3 Transforming frequency to k-space

We now have a variety of cubes in {kx, ky, f}-space. Inorder to go to power spectrum space, we must perform aspectral transform in the frequency direction to go fromf to kz. Due to the nature of the data, this includesseveral steps.

While the binned data are regularly spaced as a func-tion of frequency, the sampling distribution is not con-stant. We have flagged channels due to RFI in §3.1 anddue to channaliser aliasing in §3.4. The sampling of theuv-plane is also inherently non-regular as a function offrequency due to baseline length evolving with frequency.As a result, the sin and cos basis functions of the Fouriertransform are not orthogonal to the noise distributionon our sampling; the noise is not independent in thesin and cos basis. However, we can find a basis whichis orthogonal.We use the Lomb-Scargle periodogram to find this

basis (Lomb, 1976; Scargle, 1982). A rotation phase isfound for each spectral mode that creates orthogonal cos-like and sin-like eigenfunctions in the noise distribution.This periodogram effectively erases the phase of thedata, though this is suitable given that our final goalis to create a power spectrum (a naturally phase-lessproduct).The spectral transformation of a relatively small, fi-

nite set of data will cause leakage. We mitigate thisby multiplying the data by a Blackman-Harris windowfunction in frequency. This will decrease the leakage byabout 70 dB at the first sidelobe, but at the cost of halfthe effective bandwidth.The uneven sampling as a function of frequency can

also cause leakage from the bright foregrounds intohigher kz. To reduce leakage from the uneven sam-pling and from the finite spectral transform window,we remove the mean of the data before applying thewindow function. To preserve the DC term in the powerspectrum, we add this value back to the zeroth modeafter our spectral transform. This preserves power whileimproving dynamic range.We do not expect this mean subtraction and reinser-

tion to affect our measured EoR power spectrum becausethe vast majority of the DC-mode power is from fore-grounds. In addition, the zeroth kz-mode is not includedin our EoR estimates. All our signal loss simulations in-clude this technique to verify that there is no unexpectedeffect on higher kz-modes.The periodogram dual, η, is related to wavenumber

space through

kz ≈2πH0f21E(z)c(1 + z)2 η, (24)

Page 14: The FHD/εppsilon Epoch of Reionization Power Spectrum Pipeline … · 2019. 7. 24. · FHD/εppsilonEoRPSPipeline 3 Type Variables Definition B F Faradayrotation O Sourcepositionoffsets

14 N. Barry et al.

where c is the speed of light, z is redshift, H0 is the Hub-ble constant in the present epoch, f21 is the frequency ofthe 21 cm emission line, E(z) describes how H0 evolvesas a function of redshift, and kz is the wavenumber alongthe line-of-sight. This is an approximation; it is valid forall reasonable parameters and avoids a frequency kernel(Morales & Hewitt, 2004).

We have chosen our power estimator to be along kzrather than along the time delay of the electric field prop-agation between elements kτ . This subsequently deter-mines how the foreground contaminates our final powerspectrum (Morales et al., 2018). Given our choice ofbasis, we are constructing an imaging (or reconstructedsky) power spectrum.

5.4 Power spectrum products

We now have the power spectra estimates for the mean,the noise, and the uncertainty estimates as a functionof {kx, ky, kz}.We construct our power spectra by subtracting the

power of the even–odd difference from the power of theeven–odd summation, and dividing by 4:

p = pµ − pn4 = µ2 − n2

4 =xeσ2e

xoσ2o

1σ2e

+ 1σ2o

. (25)

This gives us the same result as the cross power betweenthe even and odd cubes, and the additional productsalso allow us to build more diagnostics and to carry thenoise throughout the pipeline.We would like to perform averages over these cubes

to generate the best possible limits and to generatediagnostics. To do so, we must assume that the EoR isspatially homogeneous and isotropic, which allows forspherical averaging in Fourier space (Morales & Hewitt,2004). Much like the creation of even–odd sum anddifference cubes, we perform the weighted average ofpixels. The only difference is that the variances σ2 areno longer Gaussian, but rather Erlang variances whichcan be propagated from the original variances.

We can now create various power spectrum productsin 2D and 1D with our 3D power spectrum cube. Un-certainty estimates, measured noise contributions, andexpectation values that we have generated in εppsilonwill also be used in our diagnostics.

6 POWER SPECTRUM DIAGNOSTICS

We create a variety of diagnostic power spectrum plotsusing various averaging schemes. These are essential tounderstanding contributions to the power spectrum, andare vital to assessing changes to the analysis. In thissection, we use MWA Phase I data as a specific example,however all of these diagnostic plots are part of anyanalysis with FHD/εppsilon.

Intrinsic Foregrounds

Figure 4. A schematic representation of a 2D power spectrum.Intrinsic foregrounds dominate low k‖ (modes along the line-of-sight) for all k⊥ (modes perpendicular to the line-of-sight) dueto their relatively smooth spectral structure. Chromaticity ofthe instrument mixes foreground modes up into the foregroundwedge. The primary-field-of-view line and the horizon line arecontamination limits dependent on how far off-axis sources are onthe sky. Foreground-free measurement modes are expected to bein the EoR window.

6.1 2D power spectrum

The most useful diagnostic we have is the 2D powerspectrum. We average our {kx, ky, kz}-power measure-ments along only the angular wavenumbers {kx, ky} incylindrical shells. The resulting power spectrum is afunction of modes perpendicular to the line-of-sight (k⊥)and modes parallel to the line-of-sight (k‖) shown inFigure 4. The {k‖, k⊥}-axes have been converted into{τ, λ} on the right and top axes—delay in nanosecondsand baseline length in wavelengths, respectively.Wavenumber space is crucial for statistical measure-

ments due to the spectral characteristics of the fore-grounds. Diffuse synchrotron emission and bright radiosources, while distributed across the sky, vary smoothlyin frequency (e.g. Matteo et al. (2002); Oh & Mack(2003)). Only small k‖-values are theoretically contam-inated by bright, spectrally smooth astrophysical fore-grounds. Since the foreground power is restricted to onlya few low k‖-modes, larger k‖-values tend to be free ofthese intrinsic foregrounds in wavenumber space.However, interferometers are naturally chromatic.

This chromaticity distributes foreground power into a

Page 15: The FHD/εppsilon Epoch of Reionization Power Spectrum Pipeline … · 2019. 7. 24. · FHD/εppsilonEoRPSPipeline 3 Type Variables Definition B F Faradayrotation O Sourcepositionoffsets

FHD/εppsilon EoR PS Pipeline 15

distinctive foreground wedge due to the mode-mixingof power from small k‖-values into larger k‖-values asillustrated in Figure 4 (Datta et al., 2010; Morales et al.,2012; Vedantham et al., 2012; Parsons et al., 2012; Trottet al., 2012; Hazelton et al., 2013; Thyagarajan et al.,2013; Pober et al., 2013; Liu et al., 2014). The primaryfield-of-view line and the horizon line are the expectedcontamination limits caused by measured sources in theprimary field-of-view and the sidelobes, respectively. Theremaining region, called the EoR window, is expectedto be contaminant-free. Because the power of the EoRsignal decreases with increasing k||, the most sensitivemeasurements are expected to be in the lower, left-handcorner of the EoR window.

The intrinsic foregrounds, the foreground wedge, theprimary field-of-view line, the horizon line, and the re-sulting EoR window all have characteristic shapes in 2Dpower spectrum space. Thus, it is a very useful diagnos-tic space for identifying contamination in real data. Wegenerate a 2D power spectrum as a function of k⊥ and k‖for each of the calibrated data, model, and residual datasets in polarisations pp = XX and qq = Y Y . Shown inFigure 5 is an example 2D power spectrum panel froma MWA integration of August 23, 2013.We use this 2D power spectrum panel to help deter-

mine if expected contamination occurred in expectedregions. In addition to the features shown in Figure 4,there is harmonic k‖ contamination in the EoR windowwhich is constant in k⊥. This is caused from regular flag-ging of aliased frequency channels due to the polyphasefilter banks in the MWA hardware, which creates har-monics in k-space.

In order to verify our error propagation, we also calcu-late observed and expected 2D noise power spectra. Theexpected noise, observed noise, propagated error andnoise ratio 2D power spectra are shown in Figure 6. Theobserved noise is the power spectrum of the maximumlikelihood noise in Equation 22, so it is the realisationof the noise in the data. The expected noise and errorare the expectation value and the square root of thevariance, respectively, of the analytically propagated er-ror distributions. We calculate these by propagating theinitial Gaussian distributions through the full analysis:

Var[N ] = 1n∑i=0

14σ4i

E[N ] =

n∑i=0

12σ2i

n∑i=0

14σ4i

, (26)

where Var[N ] is the variance on the noise, E[N ] is theexpected noise, n is the number of pixels in the average,and σ2 is the original Gaussian variance.

The final plot in Figure 6 is the ratio of our observednoise to our expected noise. This investigates our as-sumption of uncorrelated pixels in uv-space during noisepropagation. There will be fluctuations given different

noise realisations on the observed noise, but the ratioshould fluctuate around one. In practice, this is approx-imately true; the mean of the noise ratio from 10 to50λ is ∼0.9. The propagated noise is therefore slightlyhigher, indicating that we overestimate our noise due toassuming a lack of correlation.

6.2 1D power spectrum

Averaging to a 1D power spectrum harnesses as muchdata as possible, thereby making the limits with thelowest noise. However, the 1D power spectrum also hasthe ability to be an excellent secondary diagnostic afterthe 2D power spectrum. While characteristic locationsof contamination are easier to distinguish on 2D plots,1D diagnostics are more able to distinguish subtleties.

The most simplistic 1D power spectrum is an averagein spherical shells of all voxels in the 3D power spectrumcube which surpass a low-weight cutoff (see Appendix Afor more details). This includes all areas of contaminationexplored in §6.1 which can obscure low-power regions.Therefore, a typical secondary diagnostic is a 1D powerspectrum generated only from pixels which fall within10 and 50λ in k⊥, shown in Figure 7. This will includeintrinsic foregrounds and some of the foreground wedge,but will avoid contaminated k⊥-modes with poor uv-coverage for the MWA. The exact k⊥-region for thisdiagnostic will depend on the instrument.The typical characteristic contamination shapes are

present in Figure 7, like the intrinsic foregrounds, fore-ground wedge, and flagging harmonics, all of which areexpected given the 2D power spectra. However, a newcontamination feature occurs at about 0.7 hMpc−1. Thiscontamination is from a cable reflection in the elementswith 150m cables. It was significantly reduced due tothe calibration technique in §3.4.4, but spectral struc-ture from unmodelled sources limit the precision (Barryet al., 2016). Much like with the aliased channel flag-ging, spectrally repetitive signals will appear as a brightcontamination along k⊥, translating to a near-constant1D contribution if high in k‖.

This highlights the potential of utilizing multiple typesof power spectra. The 2D power spectrum is a powerfuldiagnostic due to characteristic locations of contamina-tion. However, it is useful to also use 1D diagnostic powerspectra to more quantitatively measure contamination,which helps in discerning smaller contributions.

The cable reflections and flagged aliased channels leadto bright contamination due to their modulation as afunction of frequency. This is an important revelation; wemust minimise all forms of spectrally repetitive signalsduring power spectrum analysis. If a spectral mode isintroduced in the instrument or in the processing, thatmode cannot be used to detect the EoR.

Page 16: The FHD/εppsilon Epoch of Reionization Power Spectrum Pipeline … · 2019. 7. 24. · FHD/εppsilonEoRPSPipeline 3 Type Variables Definition B F Faradayrotation O Sourcepositionoffsets

16 N. Barry et al.

Figure 5. The 2D power spectra for the calibrated data, model, and residual for polarisations XX and Y Y of an integration of 64MWA observations (∼2 hrs of data) from August 23, 2013. The characteristic locations of contamination are very similar to Figure 4,with the addition of contamination at k‖ harmonics due to flagged frequencies with channeliser aliasing. Voxels that are negative due tothermal noise are dark purple-blue.

Page 17: The FHD/εppsilon Epoch of Reionization Power Spectrum Pipeline … · 2019. 7. 24. · FHD/εppsilonEoRPSPipeline 3 Type Variables Definition B F Faradayrotation O Sourcepositionoffsets

FHD/εppsilon EoR PS Pipeline 17

Figure 6. The 2D power spectra for expected noise, observed noise, error bars, and noise ratio of an integration of 64 observations fromAugust 23, 2013 in instrumental XX for the MWA. The observed noise (NO) and expected analytically propagated noise (NE) have aratio near 1, indicating our assumptions are satisfactory. The error bars are related to the observed noise via Equation 26.

Page 18: The FHD/εppsilon Epoch of Reionization Power Spectrum Pipeline … · 2019. 7. 24. · FHD/εppsilonEoRPSPipeline 3 Type Variables Definition B F Faradayrotation O Sourcepositionoffsets

18 N. Barry et al.

1D Diagnostic Power Spectrum

0.01 0.10 1.00

k (h Mpc−1)

102

104

106

108

Δ2(m

K2)

calibrated data yymodel yyresidual yy1σ thermal noiseEoR signal

Figure 7. 1D power spectra as a function of k for calibrated data (black), model (blue), residual (red), 2σ uncertainties (shaded grey),theoretical EoR (for comparison, green), and the thermal noise (dashed purple) of an integration of 64 MWA observations from August23, 2013 for instrumental Y Y . The 2D power spectrum highlights the bins that went into the 1D averaging, which we can modify toexclude the foreground wedge when making limits. A cable-reflection contamination feature at 0.7 h Mpc−1 is more obvious in this 1Dpower spectrum, which highlights the importance of using 1D space as a secondary diagnostic.

6.3 2D difference power spectrum

Often, we would like to compare a new data analysistechnique to a standard to assess potential improvements.We can do this with a side-by-side comparison of 2Dpower spectra, but are limited by the large dynamicrange of the color bar. Instead, we create 2D differencepower spectra.

We take a 3D bin-by-bin difference between two{kx, ky, kz}-cubes and then average to generate a 2Ddifference power spectrum. The reference or standardis subtracted from the new run, and the 2D differencepower spectrum varies positive and negative dependingon power levels. We choose a red–blue log-symmetriccolor bar to indicate sign, where red indicates an increasein power relative to the reference and blue indicates adecrease in power.

Figure 8 shows an example of a 2D difference powerspectra. The new data analysis run is the left panel, thereference is the middle panel, and the 2D difference is theright panel. For this example, we have chosen a new dataanalysis run that had excess power in the window and adecrease in power in the foreground wedge compared tothe reference. In general, we make 2D difference powerspectra for dirty, model, and residual, for both XX andY Y polarisations to match the six-panel plot in Figure 5.

7 SIGNAL LOSS SIMULATION

The FHD/εppsilon pipeline can be used as an in situsimulation to test for signal loss. This validates our finalpower spectra results, allowing for confidence in our EoRupper limits.

FHD is an instrument simulator by design; we createmodel visibilities from all reliable point sources and re-alistic beam kernels. We can use these model visibilitiesas the base of an input simulation of the instrument andsky. A statistical Gaussian EoR is Fourier-transformedto uv-space, encoded with instrumental effects via theuv-beam, and added to point source visibilities to cre-ate data that we can test for signal loss. These in situsimulation visibilities are input into the FHD/εppsilonpipeline. They are treated like real data, and are subjectto all real data analyses.We perform four in situ simulations to validate our

pipeline:

• Compare input EoR signal to output EoR signalwith no additional foregrounds or calibration. Thissimple test demonstrates consistency in power spec-trum normalisation and signal preservation.

• Allow all sources to be used in the calibration andsubtraction model and compare the output powerto the input EoR signal. By recovering the inputEoR signal, we demonstrate that the addition offoregrounds and calibration does not result in signalloss.

Page 19: The FHD/εppsilon Epoch of Reionization Power Spectrum Pipeline … · 2019. 7. 24. · FHD/εppsilonEoRPSPipeline 3 Type Variables Definition B F Faradayrotation O Sourcepositionoffsets

FHD/εppsilon EoR PS Pipeline 19

Figure 8. The subtraction of a residual 2D power spectrum (left) and a reference residual 2D power spectrum (middle) to create adifference 2D power spectrum (right). Red indicates a relative excess of power, and blue indicates a relative depression of power. Thisdiagnostic is helpful in determining differences in plots that inherently cover twelve orders of magnitude.

• Use an imperfect model for calibration and subtrac-tion and compare the output power to the inputEoR signal. Particularly, we calibrate with a globalbandpass (Equation 9) in the comparison. The cal-ibration errors caused by spectral structure fromunmodelled sources cause contamination in the EoRwindow (see Barry et al. (2016)), but there is noevidence of signal loss.

• Use an imperfect model for subtraction but a per-fect model for calibration, and compare the outputpower in the EoR window to the input EoR signal.We can recover the EoR signal in some of the EoRwindow, demonstrating that we can theoreticallydetect the EoR even with unsubtracted foregrounds.

Figure 9 shows the output power from each test along-side the underlying, input EoR signal. For these simu-lations, we use the MWA Phase I instrument withoutchanneliser effects. They perform as expected, recoveringthe EoR signal when possible. However, at high k-modes,the foreground simulation without calibration errors iscontaminated due to a gridding resolution systematic(Beardsley et al., 2016).

If regular flagging of aliased channeliser effects areincluded, more modes are contaminated in the EoRwindow, as seen in the model data of Figure 7. This isa consequence of the lack of inverse-variance weightingduring the spectral transform. Implementing a morecomplex weighting scheme and/or modifying the actualinstrument will be necessary to recover those modes inthe future.

It is important for an EoR power spectrum analysis tocharacterise potential signal loss in order for EoR limits

In−situ simulation

0.1 1.0k (h Mpc−1)

100

101

102

103

104

105

106

107

Δ2(m

K2)

input EoREoR in pipelinerecovered EoRforegrounds, EoRcal errors, foregrounds, EoR

Figure 9. The signal loss simulations of the FHD/εppsilonpipeline. We recover the input EoR signal (purple) for most kmodes if: 1) an EoR signal is propagated through the pipeline with-out foregrounds or calibration (orange, dashed), 2) all foregroundsare used for calibration and are perfectly subtracted (blue), and3) all foregrounds are used for calibration but are not perfectlysubtracted (green). We do not recover the EoR if there are cal-ibration errors (red), however these errors are not indicative ofsignal loss.

to be valid (Cheng et al., 2018). These tests demonstratethat we are not subject to signal loss within our pipeline.None of our analysis methodologies, from FHD all theway through εppsilon, are artificially removing the EoRsignal. In addition, these simulations can be expandedto include other effects in the future, such as channeliserstructure, diffuse emission, and beam errors.These signal loss simulations can only test relative

forms of signal loss. For example, this cannot test thevalidity of the initial encoded instrument effects on theinput (i.e the in situ simulation or EoR visibilities), norcan it test the accuracy of the sky temperature used in

Page 20: The FHD/εppsilon Epoch of Reionization Power Spectrum Pipeline … · 2019. 7. 24. · FHD/εppsilonEoRPSPipeline 3 Type Variables Definition B F Faradayrotation O Sourcepositionoffsets

20 N. Barry et al.

normalisation.

8 OVERVIEW

We have built an analysis framework that takes largequantities of measured visibilities and generates im-ages, power spectra, and other diagnostics. Four modu-larised components exist: pre-analysis flagging and aver-aging, calibration and imaging, integration, and error-propagated power spectrum calculations. The accuracyof each component is crucial due to the level of preci-sion needed in an EoR experiment. In particular, theanalysis handled by FHD and εppsilon is complicatedand multifaceted, necessitating constant refinement anddevelopment to ensure accuracy, precision, and repro-ducibility.Many data products are produced with our analy-

sis pipeline. We make instrumental polarisation imagesfor each observation for calibrated, model, and residualdata. As for power spectra, we make 2D and 1D repre-sentations for the calibrated, model, and residual datafrom integrated cubes along with propagated noise andpropagated uncertainty estimates. These form the basicoutputs within the FHD/εppsilon pipeline.

All examples provided have been from MWA Phase Idata, however FHD/εppsilon is more generally applica-ble. It has been used to publish MWA Phase II data (Liet al., 2018) and PAPER data (Kerrigan et al., 2018),with plans to analyse HERA data. Flexibility within thepipeline allows for constant cross-validation and a widevariety of use cases.

It is crucial to have a fully verifiable pipeline, and thishas been the major motivation behind FHD/εppsilon.All of our diagnostic outputs, error propagation prod-ucts, and signal loss simulations are a part of a signalpreservation narrative. By establishing confidence in ourdata analysis, we can pursue and publish credible EoRupper limits and measurements.

The FHD/εppsilon pipeline is readily available online,and developed fully in public. Anyone can view ourprogress and critically analyse our methodologies. This isa resource for the community; FHD/εppsilon is an open-source example of the necessary precision techniquesrequired for EoR power spectrum analysis.

9 ACKNOWLEDGEMENTS

This research was supported by the Australian ResearchCouncil Centre of Excellence for All Sky Astrophysicsin 3 Dimensions (ASTRO 3D), through project numberCE170100013. This work has been funded by National Sci-ence Foundation grants AST-1410484, AST-1506024, AST-1613040, AST-1613855, and AST-1643011. APB is supportedby an NSF Astronomy and Astrophysics Postdoctoral Fel-lowship under #1701440.

Figure 10. Results of our end-to-end simulations throughFHD/εppsilon of a flat power spectrum signal. The blue pointsgive the ratio of the reconstructed 1D power spectra using thecalculated sparse normalisation to the input power level (whichis flat in k). The x axis is a measure of baseline density–it givesthe average baseline weight gridded to each uv-pixel in the simu-lation. In constructing limits and 1D power spectra, we only useregions of the uv-plane where the minimum weight (as a functionof frequency) is greater than or equal to 1.

A POWER SPECTRUM VOLUMENORMALIZATION

The data products handed from FHD to εppsilon arereconstructions of the apparent sky—a 1 Jy source at thehalf-power point of the beam will appear as a 1

2 Jy source.A volume factor is needed to convert the power spec-trum of the apparent image into a properly normalisedcosmological power spectrum. The normalisation factorneeded for an area of the uv-plane estimated from anisolated baseline differs from the normalisation factor foran area estimated from many overlapping baselines. For-mally, this is related to the covariance of the overlappingvisibilities, as described in Liu et al. (2014). Carryingthe full covariance through the pipeline is computation-ally intractable, so we separately normalise regions withdense and poor uv-coverage.We demonstrate this effect with end-to-end simula-

tions in FHD/εppsilon of a flat power spectrum signal(constant power as a function of k) and randomly lo-cated baselines with increasing density in the uv-plane.For each simulation, a baseline density is selected andbaselines are randomly placed in a uv-plane. The corre-sponding visibilities are simulated for a stochastic signalwith a flat power in k, and then gridded to calculate a1D power spectrum. The reconstructed 1D power spec-tra are flat in k, but the normalisation relative to thecalculated sparse normalisation depends on baseline den-sity, as shown in Figure 10. In the limit of very sparsebaselines, there are almost no overlaps and the sparsenormalisation is correct, but as the density increasesthe normalisation decreases quickly and asymptotes tohalf the sparse normalisation. This factor of 2 in thedenominator can be understood as a doubling of theeffective area of the uv-plane that contributes to thegridded value at any location in the gridded uv-plane.

Page 21: The FHD/εppsilon Epoch of Reionization Power Spectrum Pipeline … · 2019. 7. 24. · FHD/εppsilonEoRPSPipeline 3 Type Variables Definition B F Faradayrotation O Sourcepositionoffsets

FHD/εppsilon EoR PS Pipeline 21

Formally, there is a transition region where the normal-isation is difficult to calculate without fully propagatingthe covariance. However, virtually all our sensitivitycomes from regions of the uv-plane with very denseoverlap. We do not discard the low-density regions inour 2D power spectra, because 2D spectra are primarilydiagnostic in nature and are useful in helping us to iden-tify systematics. However, for EoR upper limits and 1Dpower spectra we discard low density regions to avoiduncertainty in the normalization. Discarding the sparseareas of the uv-plane does not significantly decrease oursensitivity and alleviates the need to fully propagate thecovariance matrix.

REFERENCES

Barry N., Hazelton B., Sullivan I., Morales M. F., PoberJ. C., 2016, Monthly Notices of the Royal Astronomi-cal Society, 461, 3135

Beardsley A. P., 2015, PhD thesis, University of Wash-ington

Beardsley A. P., et al., 2013, Monthly Notices of theRoyal Astronomical Society: Letters, 429, L5

Beardsley A. P., et al., 2016, The Astrophysical Journal,833, 102

Bhatnagar S., Cornwell T. J., Golap K., Uson J. M.,2008, Astronomy and Astrophysics, 487, 419

Bowman J. D., et al., 2013, PASA - Publications of theAstronomical Society of Australia, 30

Byrne R., et al., 2019, The Astrophysical Journal, 875,70

Carroll P. A., 2016, PhD thesis, University of Washing-ton

Cheng C., et al., 2018, The Astrophysical Journal, 868,26

Choudhuri S., Bharadwaj S., Ghosh A., Ali S. S., 2014,Monthly Notices of the Royal Astronomical Society,445, 4351

Choudhuri S., Bharadwaj S., Roy N., Ghosh A., AliS. S., 2016, Monthly Notices of the Royal AstronomicalSociety, 459, 151

Cornwell T. J., Golap K., Bhatnagar S., 2008, IEEEJournal of Selected Topics in Signal Processing, 2, 647

Datta A., Bowman J. D., Carilli C. L., 2010, The Astro-physical Journal, 724, 526

DeBoer D. R., et al., 2017, Publications of the Astro-nomical Society of the Pacific, 129, 045001

Dillon J. S., et al., 2014, Physical Review D, 89, 023002Dillon J. S., et al., 2015, Physical Review D, 91, 123011Ewall-Wice A., et al., 2016, Monthly Notices of the RoyalAstronomical Society, 460, 4320

Górski K. M., Hivon E., Banday A. J., Wandelt B. D.,Hansen F. K., Reinecke M., Bartelmann M., 2005,The Astrophysical Journal, 622, 759

van Haarlem M. P., et al., 2013, Astronomy & Astro-physics, 556, A2

Hamaker J. P., Bregman J. D., Sault R. J., 1996, As-tronomy & Astrophysics Supplement Series, 117, 137

Hazelton B. J., Morales M. F., Sullivan I. S., 2013, As-trophysical Journal, 770, 156

Hogg D. W., 1999, arXiv:astro-ph/9905116Hurley-Walker N., et al., 2017, Monthly Notices of theRoyal Astronomical Society, 464, 1146

Jacobs D. C., et al., 2016, The Astrophysical Journal,825, 114

Jordan C. H., et al., 2017, Monthly Notices of the RoyalAstronomical Society, 471, 3974

Kerrigan J. R., et al., 2018, The Astrophysical Journal,864, 131

Koopmans L., et al., 2014, in Bourke T. L., et al., eds,Advancing Astrophysics with the Square KilometreArray (AASKA14). SISSA

Lenc E., et al., 2016, The Astrophysical Journal, 830, 38Li W., et al., 2018, The Astrophysical Journal, 863, 170Line J., 2017, PhD thesis, University of Mel-

bourne, http://minerva-access.unimelb.edu.au/handle/11343/191127

Liu A., Parsons A. R., Trott C. M., 2014, PhysicalReview D, 90, 023018

Lomb N. R., 1976, Astrophysics and Space Science, 39,447

Matteo T. D., Perna R., Abel T., Rees M. J., 2002, TheAstrophysical Journal, 564, 576

Mellema G., et al., 2013, Experimental Astronomy, 36,235

Mitchell D. A., Greenhill L. J., Wayth R. B., Sault R. J.,Lonsdale C. J., Cappallo R. J., Morales M. F., OrdS. M., 2008, IEEE Journal of Selected Topics in SignalProcessing, 2, 707

Morales M. F., Hewitt J., 2004, The Astrophysical Jour-nal, 615, 7

Morales M. F., Matejek M., 2009, Monthly Notices ofthe Royal Astronomical Society, 400, 1814

Morales M. F., Hazelton B., Sullivan I., Beardsley A.,2012, The Astrophysical Journal, 752, 137

Morales M. F., Beardsley A., Pober J., Barry N., Hazel-ton B., Jacobs D., Sullivan I., 2018, Monthly Noticesof the Royal Astronomical Society

Myers S. T., et al., 2003, The Astrophysical Journal,591, 575

Offringa A. R., et al., 2015, Publications of the Astro-nomical Society of Australia, 32, e008

Offringa A. R., Mertens F., Koopmans L. V. E., 2019,Monthly Notices of the Royal Astronomical Society,484, 2866

Oh S. P., Mack K. J., 2003, Monthly Notices of theRoyal Astronomical Society, 346, 871

Ord S. M., et al., 2010, Publications of the Astronomical

Page 22: The FHD/εppsilon Epoch of Reionization Power Spectrum Pipeline … · 2019. 7. 24. · FHD/εppsilonEoRPSPipeline 3 Type Variables Definition B F Faradayrotation O Sourcepositionoffsets

22 N. Barry et al.

Society of the Pacific, 122, 1353Paciga G., et al., 2011, Monthly Notices of the RoyalAstronomical Society, 413, 1174

Parsons A. R., et al., 2010, The Astronomical Journal,139, 1468

Parsons A. R., Pober J. C., Aguirre J. E., Carilli C. L.,Jacobs D. C., Moore D. F., 2012, The AstrophysicalJournal, 756, 165

Patil A. H., et al., 2014, Monthly Notices of the RoyalAstronomical Society, 443, 1113

Patil A. H., et al., 2016, Monthly Notices of the RoyalAstronomical Society, 463, 4317

Patil A. H., et al., 2017, The Astrophysical Journal, 838,65

Pober J. C., et al., 2013, The Astrophysical JournalLetters, 768, L36

Pober J. C., et al., 2014, The Astrophysical Journal, 782,66

Pober J. C., et al., 2016, The Astrophysical Journal, 819,8

Riseley C. J., et al., 2018, Publications of the Astronom-ical Society of Australia, 35

Salvini S., Wijnholds S. J., 2014, Astronomy & Astro-physics, 571, A97

Scargle J. D., 1982, The Astrophysical Journal, 263, 835Shaw J. R., Sigurdson K., Pen U.-L., Stebbins A., SitwellM., 2014, The Astrophysical Journal, 781, 57

Shaw J. R., Sigurdson K., Sitwell M., Stebbins A., PenU.-L., 2015, Physical Review D, 91, 083514

Sullivan I. S., et al., 2012, The Astrophysical Journal,759, 17

Sutinjo A., O’Sullivan J., Lenc E., Wayth R. B., PadhiS., Hall P., Tingay S. J., 2015, Radio Science, 50,2014RS005517

Thyagarajan N., et al., 2013, The Astrophysical Journal,776, 6

Tingay S. J., et al., 2013, Publications of the Astronom-ical Society of Australia, 30, e007

Trott C. M., Wayth R. B., Tingay S. J., 2012, TheAstrophysical Journal, 757, 101

Trott C. M., et al., 2016, Astrophysical Journal, 818,139

Trott C. M., et al., 2018, The Astrophysical Journal,867, 15

Vedantham H., Shankar N. U., Subrahmanyan R., 2012,The Astrophysical Journal, 745, 176

Vleck J. H. V., Middleton D., 1966, Proceedings of theIEEE, 54, 2

Wayth R. B., et al., 2018, Publications of the Astronom-ical Society of Australia, 35

Yatawatta S., et al., 2013, Astronomy & Astrophysics,550, A136

Zheng H., et al., 2017, Monthly Notices of the RoyalAstronomical Society, 465, 2901