wsa logo
2023-06-01
UHS DR2 public

WSA Home
Start Here
Data Overview
Known Issues
the Surveys
* Coverage Maps
Schema browser
Data access
Login
Archive Listing
GetImage
ColourImage
MultiGetImage
Region
Menu query
Freeform SQL
CrossID
Analysis services
SQL Cookbook
Q&A
Glossary
Release History
non-Survey
Gallery
Publications
Monitor
Downtime
Links
wsa logo bottom
IFA     ROE
Home | Overview | Browser | Access | Login | Cookbook | nonSurvey 
 
  WSA logo

Quality Error Bit Flags

This page details the quality error bit flags assigned during post processing. These flags appear as the ppErrBits attributes in the detection and source tables. The flags can be used to refine object samples extracted from the archive. Examples of this usage are given in the SQL cookbook.

Bit flag design

ppErrBits is a 32-bit (4-byte) integer in every detection table (e.g. lasDetection) that contains the quality error information for each source detection encoded as a bit flag, according to the prescription given in the table below. So the bit flag will give a 'yes' or 'no' answer to up to 32 different quality issues for every source detection. The ppErrBits attribute from the detection table is propagated into the relevant passband ppErrBits in the corresponding source table during source merging. For example the ppErrBits flag in lasDetection for Y-band observations is propagated into yppErrBits in lasSource.

The quality issues are listed in order of severity, such that a selection of sources can be easily filtered according to the level of quality the user desires. These quality issues are divided amongst the four bytes such that the least significant byte represents information about the source that is most probably harmless (such as it is a deblended source), the most significant byte contains bits that highlight some kind of severe warning about the quality of the source, and the two remaining bytes in the middle contain various warnings that could just possibly imply the source is spurious.

For example, to select only those sources with absolutely no quality issues the user can filter on ppErrBits = 0, and to include sources with only minor quality issues the user can filter as ppErrBits < 256 (i.e. where only the first byte contains 1s, so that is a range up to 2^8). Alternatively, the user could use hexadecimal bit masks if they prefer, which will also allow them to ignore the list of priorities and pick out specific quality issues that they are interested in.

Bit flag table

BytesQuality CategoryBitsDetection Quality IssueBit MaskDecimal Threshold
Byte 0 (LSB) Information Bit 0 0x00 00 00 011
Bit 1 0x00 00 00 022
Bit 2Close to a bright source (not yet implemented)0x00 00 00 044
Bit 3 0x00 00 00 088
Bit 4Deblended0x00 00 00 1016
Bit 5 0x00 00 00 2032
Bit 6Bad pixel(s) in default aperture0x00 00 00 4064
Bit 7 0x00 00 00 80128
Byte 1 Warning Bit 8 0x00 00 01 00256
Bit 9 0x00 00 02 00512
Bit 10 0x00 00 04 001 024
Bit 11 0x00 00 08 002 048
Bit 12 0x00 00 10 004 096
Bit 13 0x00 00 20 008 192
Bit 14 0x00 00 40 0016 384
Bit 15Source in poor flat field region0x00 00 80 0032 768
Byte 2 Important Warning Bit 16Close to saturated0x00 01 00 0065 536
Bit 17Photometric calibration probably subject to systematic error0x00 02 00 00131 072
Bit 18 0x00 04 00 00262 144
Bit 19Possible cross-talk artefact/contamination0x00 08 00 00524 288
Bit 20Possible diffraction spike artefact/contamination (not yet implemented)0x00 10 00 001 048 576
Bit 21 0x00 20 00 002 097 152
Bit 22Lies within a dither offset of the stacked frame boundary0x00 40 00 004 194 304
Bit 23 0x00 80 00 008 388 608
Byte 3 (MSB) Severe Warning Bit 24 0x01 00 00 0016 777 216
Bit 25 0x02 00 00 0033 554 432
Bit 26 0x04 00 00 0067 108 864
Bit 27 0x08 00 00 00134 217 728
Bit 28 0x10 00 00 00268 435 456
Bit 29 0x20 00 00 00536 870 912
Bit 30used in UHS to indicate failed quantitative QC0x40 00 00 001 073 741 824
Bit 31used in UHS to indicate failed eyeball QC 0x80 00 00 002 147 483 648

Quality issues

At the time of DR2 only quality flags that could be generated, relatively easily, from pre-existing information in the database were implemented. The agreed list of five different quality issues implemented for DR2 are listed below:

Note on the UDS: For DR2 the UDS sources are unchanged from DR1, due to no observations of this field during the 06A semester, and so do not have any ppErrBits flags. The UDS sources already have the default SExtractor bit flags in the errBits attribute (see the glossary entry for this attribute for more details). From DR3 the UDS has a cross-talk flag. The boundary flag will be calculated only if deemed worthwhile and a worthy algorithm is deduced. The remaining bit flags described below will only be included if they are already contained within the SExtractor bit flag attribute, errBits.

Source image is deblended

This is determined from the aprof2 to aprof8 detection attributes, all of which should be -1 if the source image is deblended.

Bad pixel in default aperture

These sources have at least one bad pixel in the default aperture, so contain missing information. The total number of bad pixels in the default aperture is supplied in the errBits attribute (for all surveys other than the UDS), which can be used to refine the criterion.

Source in poor flat field region

The flag that marks if an object is within a dither step of the edge is not ideal and the reason is that the flat-field goes awry near the edges of detectors 1 and 2. There is also an edge region on detector 2 near 2048, 220 where a bite is taken out, where the flat-field is bad (though is better at longer wavelengths). This flag covers a 5" buffer around all of detectors 1 and 2 i.e. flag if the object is within dither step + 5" of the edge. In addition, any object whose coordinates are within 300 (0.4") pixels of coordinate 2270 220 (with suitable modification for microstepped images) are flagged, this is not applied to deeps because it's less of a problem and harder to define.

Saturated source image

Source image contains at least one pixel that is close to being saturated, defined as having a count (pixel height + sky level) > 40,000 ADU for catalogues from nightly stacks. For deep multi-night stacks, the flag has only been applied since DR8 and is set to a count > 0.9 * avSaturLevel (the average saturation level of the detector) .

Photometric calibration probably subject to systematic error

This flag is applied to GPS detections from DR7 onwards that come from multiframes with a poor photometric calibration due to heavy and highly variable extinction in Galactic centre fields. A detector level criterion is applied to all frames in that multiframe: nightZPerrcat < 0.05 AND photZPcat - 1.505 - nightZPcat < -0.1

Possible cross-talk artefact

Cross-talk flagging was applied to the DR2 release (LAS, GCS, DXS) but did not affect the seaming. The GPS cannot ever be cross-talk flagged with the current algorithm parameters as its fields are just simply too crowded.

Source may possibly be a cross-talk artefact, or at least be contaminated by it. All sources within a pre-defined radius (for practical purposes, a box) of a suspected cross-talk artefact are flagged.

The cross-talk artefacts are illustrated and described in this CASU document. There are artefacts either side of a bright source, in one dimension, all of the way to the detector quadrant boundary (at most 7 orders, though with dithering you can produce 8+ in a stacked image product). The orientation of the line of artefacts rotates between each detector quadrant in the way illustrated in the CASU document.

The flagging was implemented by assuming that all point sources brighter than 14th magnitude in the J-band of the 2MASS catalogue will produce cross-talk in WFCAM detector fields. The size of the artefact was empirically calculated as a function of the magnitude of the cross-talk producing source, and all sources lying within these radii are flagged for artefacts up to the highest order (i.e. to the edge of a detector or quadrant boundary). The possibility of cross-talk artefacts appearing in two dimensions for stars that lie within dither offsets of a quadrant boundary in stacked images is also considered.

Source lies within a dither offset of the frame boundary

Source lies within a jitter + microstep offset of the stacked frame boundary with an additional safety margin of twice the default aperture radius to include all cases where the source image is incomplete. This is an important warning because all of these sources should certainly contain missing information - i.e. there may be partial sources and/or the source may not be present in all component images of the stack.

For deep stacks we also consider any positional offsets in creating the final deep stack image from intermediate stacks.

Possible future flags

Source close to bright star

Bright star halos generate many spurious sources.

Diffraction spike

Spurious sources occur in lines along diffraction spikes.



Home | Overview | Browser | Access | Login | Cookbook | nonSurvey
Listing | Region | MenuQuery | FreeSQL
Links | Credits

WFAU, Institute for Astronomy,
Royal Observatory, Blackford Hill
Edinburgh, EH9 3HJ, UK

wsa-support@roe.ac.uk
31/7/2017