The Core Data File

Filename conventions

The core file is named as core_faam_<date>_<version>_<revision>_<fltnum>[_<freq>].nc, where:

  • date indicates the date on which the flight took place, formatted as YYYYmmdd, where YYYY is the four-digit year, mm is the two-digit month, and dd is the two-digit day.

  • version indicates the version of the core file, formatted as vnnn, where nnn is a three-digit integer. The most recent version is 5 (v005), which this document applies to. It is important to note that older versions do not strictly follow the conventions outlined in this document.

  • revision indicates the revision number of the data, formatted as rn, where n is an integer. Larger values of n indicate a more recent release. Where more than one data revision is available, the file with the largest revision number should be used.

  • fltnum indicates the flight number of the flight, formatted as xnnn where x is a lowercase letter, and nnn a three digit integer.

  • freq indicates the frequency of the dataset, formatted as nhz, where n is an integer. If not present, the file is at ‘full’ frequency, with the frequency differing between variables. Typically ‘full’ and 1hz files are provided.

Dimensions

All variables in the core dataset are timeseries with the same start and end times, and with frequencies of 1 Hz or greater. There is only one time dimension, Time, with a length equal to the duration of the dataset in seconds. Variables which are recorded at 1 Hz will have Time as their only dimension. Variables which are recorded at higher than 1 Hz will be stored as a two-dimensional array, with Time as the first dimension and spsNN as the second. Here, the NN repesents the frequency in Hz. For example, a 4 Hz variable would have dimensions Time and sps04, and would be stored as an \(n\times4\), where n is the length of the dataset in seconds.

The dimensions which may be included in the full temporal resolution core data file are:

  • Time - An unlimited dimension whose length corresponds to the length of the dataset, in seconds.

  • sps02 - 2 samples per second. A dimension of length 2.

  • sps04 - 4 samples per second. A dimension of length 4.

  • sps10 - 10 samples per second. A dimension of length 10.

  • sps20 - 20 samples per second. A dimension of length 20.

  • sps32 - 32 samples per second. A dimension of length 32.

  • sps64 - 64 samples per second. A dimension of length 64.

Global Attributes

Required Global Attributes

  • Conventions - The conventions followed by this data

  • acknowledgement - The acknowledgement for use of this data. Should read: Airborne data was obtained using the BAe-146-301 Atmospheric Research Aircraft [ARA] flown by Airtask Ltd and managed by FAAM Airborne Laboratory, jointly operated by UKRI and the University of Leeds for FAAM data.

  • creator_address - The address of the FAAM offices. Should read: Building 146, Cranfield University, College Road, Cranfield, Bedford. MK43 0AL. UK. for FAAM data.

  • creator_email - The email address of the data creator, or a generic FAAM email.

  • creator_institution - The institute responsible for the creation of the data. Should read: FAAM Airborne Laboratory for FAAM data

  • creator_name - The name of the person or institute responsible for this data

  • creator_type - The type of creator. Should be one of person, institution, position

  • date - Date of the flight

  • date_created - ISO:8601 date/time file created, ideally yyyy-mm-ddTHH:MM:SSZ. Equivalent to revision_date.

  • flight_date - Date of the flight

  • flight_number - The flight number

  • geospatial_bounds - A well-known text representation of the flight envelope

  • geospatial_bounds_crs - Coordinate reference system for geospatial attributes.

  • geospatial_lat_max - Maximum latitude of the flight.

  • geospatial_lat_min - Minimum latitude of the flight.

  • geospatial_lat_units - The units for geospatial_lat_* attributes

  • geospatial_lon_max - Maximum longitude of the flight

  • geospatial_lon_min - Minimum longitude of the flight

  • geospatial_lon_units - The units for geospatial_lon_* attributes

  • geospatial_vertical_max - The maximum altitude of the flight

  • geospatial_vertical_min - The minimum altitude of the flight

  • geospatial_vertical_units - Vertical units associated with geospatial_vertical_* attributes

  • geospatial_vertical_positive - Positive direction of geospatial_vertical_* attributes

  • id - filename without extension, needs to be globally unique w/ naming_authority

  • institution - The institute responsible for the creation of this data. Should read: FAAM Airborne Laboratory for FAAM data.

  • keywords - A comma separated list of keywords from the GCMD

  • keywords_vocabulary - Controlled vocabulary for keywords. Should read Global Change Master Directory (GCMD)

  • license - The license for this data. Should read: UK Government Open Licence agreement - http://www.nationalarchives.gov.uk/doc/open-government-licence

  • metadata_link - A link to a release of a controlled vocabulary of metadata attributes

  • naming_authority - The authority responsible for naming this data. Should read uk.ac.faam

  • platform - Platform name of the FAAM aircraft. Should read: FAAM BAe-146-301 Atmospheric Research Aircraft

  • platform_type - Platform type: aircraft

  • project - The project a flight was for. Expected to be of the form <project_acronym> - <project_name>

  • publisher_email - The email address of the data publisher

  • publisher_institution - The institution responsible for publishing data. Should read: CEDA

  • publisher_type - Type of entity responsible for publishing data. Should be institution

  • publisher_url - The URL of the data publisher. Should be https://www.ceda.ac.uk

  • references - A list of references relevant to the file

  • revision_date - The date this revision created. Analogous to date_created

  • revision_number - The revision number of the dataset. Starts at 0, increments by 1 for each revision

  • source - A description of the source of the data. An instrument where this makes sense, otherwise a description of how data obtained

  • standard_name_vocabulary - A vocabulary of standard names. Expected to be a CF standard name table, e.g. CF Standard Name Table v78

  • summary - A brief summary of the data

  • time_coverage_duration - The duration of the data, in ISO8601 duration format

  • time_coverage_start - The start time of the data in the file, in ISO8601 format

  • time_coverage_end - The end time of the data in the file, in ISO8601 format

  • title - A brief title for the dataset

  • uuid - V3 UUID: MD5 hash of id+date_created

Optional Global Attribtues

  • calibration_date - Calibration date, where this applies to all file contents

  • calibration_information - Calibration information, where this applies all file contents

  • calibration_url - Permanent URI or DOI linking to calibration info, where this applies to all file contents

  • comment - Comment about data

  • constants_file - Name of external file providing input to processing software

  • creator_url - Institutional URL or researcher ORCID URL

  • deployment_mode - Mode of deployment. Should always be “air”

  • external_variables - Blank separated list of variables used from an external file

  • history - Logs modifications to file since production

  • instrument - Where all data id from a single instrument. Currently freeform, controlled instrument vocab to follow

  • instrument_description - Where all data are from a single instrument. Textual description.

  • instrument_location - Where all data are from a single instrument. Location of instrument on aircraft

  • instrument_manufacturer - Where all data are from a single instrument. Instrument manufacturer

  • instrument_model - Where all data are from a single instrument. Instrument model

  • instrument_serial_number - Where all data are from a single instrument. Instrument serial number

  • instrument_software - Where all data are from a single instrument. Name of software deployed on instrument

  • instrument_software_version - Where all data are from a single instrument. Version of software deployed on instrument

  • notes - Notes on this data file. For more detailed info than comment

  • processing_software_commit - Commit hash for processing software in vcs

  • processing_software_doi - DOI of the processing doftware release, if available

  • processing_software_url - URL pointing to processing software source code, if available

  • processing_software_version - Version of processing software

  • project_acronym - Project acronym

  • project_name - Full project name

  • project_principal_investigator - Name(s) of project PIs. Comma separated if more than one

  • project_principal_investigator_email - Email address(es) of project PIs. Comma separated if more than one

  • project_principal_investigator_url - ORCID URL(s) of project PIs. Comma separated if more than one

  • revision_comment - Brief description of changes between revisions

  • source_files - List of source files used in the processing of this data product

  • time_coverage_resolution - Resolution of data, where this is uniform across the file

  • processing_level - Processing level of data, where it is uniform for all data in the file

Variable Attributes

Required Variable Attributes

  • _FillValue - The prefill/missing data value

  • coverage_content_type - ISO 19115-1 code. One of image, thematicClassification, physicalMeasurement, auxiliaryInformation, qualityInformation, referenceInformation, modelResult, coordinate

  • frequency - The frequency of the data. Where this is >1, will typically correspond to an spsNN dimension

  • long_name - A longer, descriptive name for the variable

  • units - Valid UDUNITS unit. Where a standard_name is given, must be equivalent to canonical units

Optional Variable Attributes

  • axis - Axis for coordinate variables. Should be one of X, Y, Z, T

  • actual_range - A length 2 array, of the same datatype as the variable, giving the maximum and minimum valid values

  • add_offset - Offset for packing. Default is 0

  • ancillary_variables - Optional in no ancillary variables, otherwise required. Ancillary variables e.g. flags, uncertainties

  • calendar - Required for the time coordinate variable. Should be one of standard, gregorian

  • calibration_date - Calibration date, where this applies to variable

  • calibration_information - Calibration information, where this applies variable

  • calibration_url - Permanent URI or DOI linking to calibration info, where this applies to variable

  • comment - Comment about data

  • coordinates - Blank separated list of coordinate variables

  • flag_masks - Allowed flag mask values. Required for bitmask style flags

  • flag_meanings - Blank separated list of flag meanings. Required for flag variables

  • flag_values - Allowed flag values. Required for classic flags

  • instrument_description - Textual description of instrument.

  • instrument_location - Where variable is derived from a single instrument. Location of instrument on aircraft

  • instrument_manufacturer - Where variable is derived from a single instrument. Instrument manufacturer

  • instrument_model - Where variable is derived from a single instrument. Instrument model number

  • instrument_serial_number - Where variable is derived from a single instrument. Instrument serial number

  • instrument_software - Where variable is derived from a single instrument. Name of software deploed on instrument

  • instrument_software_version - Where all data in the group are from a single instrument. Version of software deployed on instrument

  • sensor_manufacturer - Similar to instrument_manufacturer, but where the item is more accurately described as a sensor

  • sensor_model - Similar to instrument_model, but where the item is more accurately described as a sensor

  • sensor_serial_number - Where variable is derived from a single sensor, similar to instrument_serial_number. Sensor serial number.

  • sensor_type - The type of sensor fitted, where different sensor types may be used to produce the same measurement. A canonical example of this is the type of temperature sensor in the Rosemount housings.

  • positive - Applies to vertical coordinate. Should be “up” if included

  • scale_factor - Scale for packing. Default is 1

  • standard_name - See CF standard names list. Should be used whenever possible

  • valid_max - Recommended where feasible. Where both valid_min and valid_max make sense, valid_range is preferred.

  • valid_min - Recommended where feasible. Where both valid_min and valid_max make sense, valid_range is preferred.

  • valid_range - A length 2 array of the same datatype as the variable, giving minimum and maximum valid values

  • processing_level - Processing level of variable data

Variables

Below is a list of all output variables which can be created during postprocessing, split into variables which will always be written to file if available, and those which are generally only used internally, but which can be written to file if requested. Note that some variables may be duplicated here. This indicates that the same output variable may be produced from two or more processing modules, depending on the fit of the aircraft. These will be mutually exclusive, so that variables will be unique in a dataset.

Written if Available

Default Variables

Name

Long Name

Standard Name

Processing Module

ACLD_GIN

Acceleration along the aircraft vertical axis from POS AV 410 GPS-aided Inertial Navigation unit (positive down)

Gin

ACLF_GIN

Acceleration along the aircraft longitudinal axis from POS AV 410 GPS-aided Inertial Navigation unit (positive forward)

Gin

ACLS_GIN

Acceleration along the aircraft transverse axis from POS AV 410 GPS-aided Inertial Navigation unit (positive starboard)

Gin

ALT_GIN

Altitude from POS AV 410 GPS-aided Inertial Navigation unit

altitude

Gin

AOA

Angle of attack from the turbulence probe (positive, flow upwards wrt a/c axes)

TurbulenceProbe

AOSS

Angle of sideslip from the turbulence probe (positive, flow from left)

TurbulenceProbe

BSC_BLUU

Uncorrected blue back scattering coefficient from TSI 3563 Nephelometer

Nephelometer

BSC_GRNU

Uncorrected green back scattering coefficient from TSI 3563 Nephelometer

Nephelometer

BSC_REDU

Uncorrected red back scattering coefficient from TSI 3563 Nephelometer

Nephelometer

BTHEIM_U

Uncorrected brightness temperature from the Heimann radiometer

Heimann

CAB_PRES

Cabin Pressure

CabinPressure

CAB_TEMP

Cabin temperature at the core consoles

CabinTemp

CO_AERO

Mole fraction of Carbon Monoxide in air from the AERO AL5002 instrument

mole_fraction_of_carbon_monoxide_in_air

AL52CO

CPC_CNTS

Condensation Particle Counts measured by the TSI 3786

CPC

EXX_JCI

Raw data from the Fwd Core Console JCI static monitor, static signal

ElectricFieldJci140

EXX_ZEUS

Electric field measured by the Zeus instrument

ElectricFieldZeus

GSPD_GIN

Groundspeed from POS AV 410 GPS-aided Inertial Navigation unit

platform_speed_wrt_ground

Gin

HDGR_GIN

Rate-of-change of heading from POS AV 410 GPS-aided Inertial Navigation unit

platform_yaw_rate

Gin

HDG_GIN

Heading from POS AV 410 GPS-aided Inertial Navigation unit

platform_yaw_angle

Gin

HGT_RADR

Radar height from the aircraft radar altimeter

height

RadAlt

IAS_RVSM

Indicated air speed from the aircraft RVSM (air data) system

AirSpeed

IR_DN_C

Corrected downward longwave irradiance

KippZonenPyrgeometer

IR_UP_C

Corrected upward longwave irradiance

KippZonenPyrgeometer

LAT_GIN

Latitude from POS AV 410 GPS-aided Inertial Navigation unit

latitude

Gin

LON_GIN

Longitude from POS AV 410 GPS-aided Inertial Navigation unit

longitude

Gin

NEPH_PR

Internal sample pressure of the Nephelometer

Nephelometer

NEPH_RH

Relative humidity from TSI 3563 Nephelometer

Nephelometer

NEPH_T

Internal sample temperature of the Nephelometer

Nephelometer

NV_LWC1_C

Corrected liquid water content from the Nevzorov probe (1st collector)

mass_concentration_of_liquid_water_in_air

Nevzorov

NV_LWC1_COL_P

LWC1 collector power

Nevzorov

NV_LWC2_C

Corrected liquid water content from the Nevzorov probe (2nd collector)

mass_concentration_of_liquid_water_in_air

Nevzorov

NV_LWC2_COL_P

LWC2 collector power

Nevzorov

NV_LWC_C

Corrected liquid water content from the Nevzorov probe

mass_concentration_of_liquid_water_in_air

Nevzorov

NV_LWC_COL_P

LWC collector power

Nevzorov

NV_LWC_REF_P

LWC reference power

Nevzorov

NV_REF_P

Reference power

Nevzorov

NV_TWC_C

Corrected total condensed water content from the Nevzorov probe

Nevzorov

NV_TWC_COL_P

TWC collector power

Nevzorov

NV_TWC_REF_P

TWC reference power

Nevzorov

O3_2BTECH

Mole fraction of Ozone in air from the 2BTech photometer

mole_fraction_of_ozone_in_air

TwoBOzone

O3_TECO

Mole fraction of Ozone in air from the TECO 49 instrument

mole_fraction_of_ozone_in_air

TeiOzone

P0_S10

Calibrated differential pressure between centre (P0) port and S10 static

TPress

P10_STAT

Static pressure from S10 fuselage ports

air_pressure

S10Pressure

P9_STAT

Static pressure from S9 fuselage ports

air_pressure

S9Pressure

PALT_RVS

Pressure altitude from the aircraft RVSM (air data) system

barometric_altitude

Rvsm

PA_TURB

Calibrated differential pressure between turbulence probe vertical ports

TPress

PB_TURB

Calibrated differential pressure between turbulence probe horizontal ports

TPress

PITR_GIN

Rate-of-change of pitch angle from POS AV 410 GPS-aided Inertial Navigation unit

platform_pitch_rate

Gin

PSAP_FLO

PSAP Flow

PSAP

PSAP_LIN

Uncorrected absorbtion coefficient at 565nm, linear, from PSAP

PSAP

PSAP_LOG

Uncorrected absorption coefficient at 565nm, log, from PSAP

PSAP

PSAP_TRA

PSAP Transmittance

PSAP

PSP_TURB

Pitot-static pressure from centre-port measurements corrrected for AoA and AoSS

TurbulenceProbe

PS_RVSM

Static pressure from the aircraft RVSM (air data) system

air_pressure

Rvsm

PTCH_GIN

Pitch angle from POS AV 410 GPS-aided Inertial Navigation unit

platform_pitch_angle

Gin

Q_RVSM

Pitot static pressure inverted from RVSM (air data) system indicated airspeed

Rvsm

RED_DN_C

Corrected downward short wave irradiance, red dome

BBRFlux

RED_UP_C

Corrected upward short wave irradiance, red dome

BBRFlux

RH_ICE

Relative humidity wrt ice water, derived from corrected WVSS-II VMR and Rosemount temperatures

relative_humidity

WVSS2RH

RH_ICE_CU

Combined uncertainty estimate for RH_ICE

WVSS2RH

RH_LIQ

Relative humidity wrt liquid water, derived from corrected WVSS-II VMR and Rosemount temperatures

relative_humidity

WVSS2RH

RH_LIQ_CU

Combined uncertainty estimate for RH_LIQ

WVSS2RH

ROLL_GIN

Roll angle from POS AV 410 GPS-aided Inertial Navigation unit

platform_roll_angle

Gin

ROLR_GIN

Rate-of-change of roll angle from POS AV 410 GPS-aided Inertial Navigation unit

platform_roll_rate

Gin

SEA_LWC_021

Liquid water content from the SEA WCM-2000 probe, element 021

mass_concentration_of_liquid_water_in_air

SeaProbe

SEA_LWC_083

Liquid water content from the SEA WCM-2000 probe, element 083

mass_concentration_of_liquid_water_in_air

SeaProbe

SEA_TWC_021

Total water content from the SEA WCM-2000 probe, element 021

SeaProbe

SEA_TWC_083

Total water content from the SEA WCM-2000 probe, element 083

SeaProbe

SO2_TECO

Mole fraction of Sulphur Dioxide in air from TECO 43 instrument

mole_fraction_of_sulfur_dioxide_in_air

TecoSO2

SO2_TECO

Mole fraction of Sulphur Dioxide in air from TECO 43 instrument

mole_fraction_of_sulfur_dioxide_in_air

TecoSO2V2

SOL_AZIM

Solar azimuth derived from aircraft position and time

solar_azimuth_angle

SolarAngles

SOL_ZEN

Solar zenith derived from aircraft position and time

solar_zenith_angle

SolarAngles

STATUS_GIN

Solution status from POS AV 410 GPS-aided Inertial Navigation unit

Gin

SW_DN_C

Corrected downward short wave irradiance, clear dome

downwelling_shortwave_flux_in_air

BBRFlux

SW_UP_C

Corrected upward short wave irradiance, clear dome

upwelling_shortwave_flux_in_air

BBRFlux

TAS

True airspeed (dry-air) from turbulence probe

platform_speed_wrt_air

TurbulenceProbe

TAS_RVSM

True air speed from the aircraft RVSM (air data) system and deiced temperature

platform_speed_wrt_air

AirSpeed

TAT_DI_R

True air temperature from the Rosemount deiced temperature sensor

air_temperature

PRTTemperatures

TAT_DI_R

True air temperature from the Rosemount deiced temperature sensor

air_temperature

ThermistorV1Temperatures

TAT_DI_R_CU

Combined uncertainty estimate for TAT_DI_R

PRTTemperatureUncertainties

TAT_ND_R

True air temperature from the Rosemount non-deiced temperature sensor

air_temperature

PRTTemperatures

TAT_ND_R

True air temperature from the Rosemount non-deiced temperature sensor

air_temperature

ThermistorV1Temperatures

TAT_ND_R_CU

Combined uncertainty estimate for TAT_ND_R

PRTTemperatureUncertainties

TDEWCR2C

Corrected dew point temperature measured by the Buck CR2 Hygrometer

dew_point_temperature

BuckCR2

TDEW_CR2

Mirror Temperature measured by the Buck CR2 Hygrometer

dew_point_temperature

BuckCR2

TDEW_C_U

Combined uncertainty estimate for Buck CR2 Mirror Temperature

BuckCR2

TDEW_GE

Dew Point from the General Eastern instrument

dew_point_temperature

GeneralEastern

TRCK_GIN

Aircraft track angle from POS AV 410 GPS-aided Inertial Navigation unit

platform_course

Gin

TSC_BLUU

Uncorrected blue total scattering coefficient from TSI 3563 Nephelometer

Nephelometer

TSC_GRNU

Uncorrected green total scattering coefficient from TSI 3563 Nephelometer

Nephelometer

TSC_REDU

Uncorrected red total scattering coefficient from TSI 3563 Nephelometer

Nephelometer

U_C

Eastward wind component from turbulence probe and GIN

eastward_wind

TurbulenceProbe

U_NOTURB

Eastward wind component derived from aircraft instruments and GIN

eastward_wind

GINWinds

VELD_GIN

Aircraft velocity down from POS AV 410 GPS-aided Inertial Navigation unit

Gin

VELE_GIN

Aircraft velocity east from POS AV 410 GPS-aided Inertial Navigation unit

Gin

VELN_GIN

Aircraft velocity north from POS AV 410 GPS-aided Inertial Navigation unit

Gin

VMR_CR2

Water vapour volume mixing ratio measured by the Buck CR2

BuckCR2

VMR_C_U

Combined uncertainty estimate for water vapour volume mixing ratio measured by the Buck CR2

BuckCR2

V_C

Northward wind component from turbulence probe and GIN

northward_wind

TurbulenceProbe

V_NOTURB

Northward wind component derived from aircraft instruments and GIN

northward_wind

GINWinds

WAND_GIN

Wander angle from POS AV 410 GPS-aided Inertial Navigation unit

Gin

WOW_IND

Weight on wheels indicator

WeightOnWheels

WVSS2F_PRESS_U

Pressure inside {ins} sample cell {interp}

WVSS2A

WVSS2F_VMR_C

Calibrated volume mixing ratio from WVSS2F

WVSS2Calibrated

WVSS2F_VMR_C_CU

Uncertainty estimate for calibrated volume mixing ratio from WVSS2F

WVSS2Calibrated

WVSS2F_VMR_U

Water Vapour Measurement from {ins} {interp}

WVSS2A

WVSS2R_PRESS_U

Pressure inside {ins} sample cell {interp}

WVSS2B

WVSS2R_VMR_U

Water Vapour Measurement from {ins} {interp}

WVSS2B

W_C

Vertical wind component from turbulence probe and GIN

upward_air_velocity

TurbulenceProbe

Used internally

Default Variables

Name

Long Name

Standard Name

Processing Module

ETA_DI

Variable recovery factor for deiced Rosemount housing

RecoveryFactor

ETA_DI_CU

Uncertainty in recovery factor for non-deiced Rosemount housing

RecoveryFactor

ETA_ND

Variable recovery factor for non-deiced Rosemount housing

RecoveryFactor

ETA_ND_CU

Uncertainty in recovery factor for deiced Rosemount housing

RecoveryFactor

IAT_DI_R

Indicated air temperature from the Rosemount deiced temperature sensor

PRTTemperatures

IAT_DI_R

Indicated air temperature from the Rosemount deiced temperature sensor

ThermistorV1Temperatures

IAT_DI_R_CU

Combined uncertainty estimate for IAT_DI_R

PRTTemperatureUncertainties

IAT_ND_R

Indicated air temperature from the Rosemount non-deiced temperature sensor

PRTTemperatures

IAT_ND_R

Indicated air temperature from the Rosemount non-deiced temperature sensor

ThermistorV1Temperatures

IAT_ND_R_CU

Combined uncertainty estimate for IAT_ND_R

PRTTemperatureUncertainties

LIRS

LWR I/R SIGNAL

BBRSols

LIRT

LWR I/R TEMP

BBRSols

LIRZ

LWR I/R ZERO

BBRSols

LP1S

LWR VIS CLR SIG

BBRSols

LP1T

LWR VIS CLR TEMP

BBRSols

LP1Z

LWR VIS CLR ZERO

BBRSols

LP2S

LWR VIS RED SIG

BBRSols

LP2T

LWR VIS RED TEMP

BBRSols

LP2Z

LWR VIS RED ZERO

BBRSols

MACH

Moist air Mach derived from WVSS-II(F) and RVSM

WetMach

MACH

Dry air Mach derived from and RVSM

DryMach

MACH_CU

Uncertainty estimate for moist-air Mach

WetMach

MACH_CU

Uncertainty estimate for MACH

DryMach

NV_CLEAR_AIR_MASK

Clear air mask based on Nevzorov Total Water power variance

Nevzorov

P9_STAT_U

Static pressure from S9 fuselage ports, uncorrected

air_pressure

S9Pressure

SH_GAMMA

Ratio of specific heats at constant pressure and constant pressure

WetMach

SH_GAMMA

Ratio of specific heats at constant pressure and constant pressure

DryMach

SH_GAMMA_CU

Uncertainty estimate for SH_GAMMA

WetMach

SH_GAMMA_CU

Uncertainty estimate for SH_GAMMA

DryMach

SO2_TECO_ZERO

TECO 43 SO2 interpolated zero

TecoSO2V2

SREG

Signal Register

SignalRegister

UIRS

UPP I/R SIGNAL

BBRSols

UIRT

UPP I/R TEMP

BBRSols

UIRZ

UPP I/R ZERO

BBRSols

UP1S

UPP VIS CLR SIG

BBRSols

UP1T

UPP VIS CLR TEMP

BBRSols

UP1Z

UPP VIS CLR ZERO

BBRSols

UP2S

UPP VIS RED SIG

BBRSols

UP2T

UPP VIS RED TEMP

BBRSols

UP2Z

UPP VIS RED ZERO

BBRSols

Flags

Every variable has an associated flag variable named <variable_name>_<postfix>, and referenced in the variable’s ancillary_variables attribute, which provides some quality or situational information about the data in the variable. There are two different flagging strategies used in the core data file; the first is a value-based quality flag, referred to as a value-based or classic flag, and the second is a packed boolean representation. Flagging is largely automatic, other than the flagged_in_qc flag meaning, which indicates that in the opinion of the person performing the quality control, that data should be treated with caution. It is up to the user to decide whether or not to use data which has been flagged. If in doubt, users should contact FAAM for advice.

Value-based Flags

Value-based flags represent the quality of the corresponding data variable, with a flag value of 0 representing data which are presumed to be of sufficient quality. Larger values of the flag generally correspond to lower quality data, though this isn’t always the case.

For example, consider a variable array with values

1 2 6 5 4 3 6 5 4 3 2 1 4 5 6 7 7 6 5 3 2

and its corresponding flag with values

0 0 0 0 1 1 0 1 1 1 2 2 1 0 0 0 0 0 0 0 0

To understand the meaning of these flag values, we can look at the flag_values and flag_meanings attributes of the flag variable, which may look like

  • flag_meanings: data_good minor_data_quality_issue major_data_quality_issue

  • flag_values: 0b 1b 2b

We can see that there are three different flag meanings and three different flag values, and can deduce that a flag value of 0 indicates the data are considered good, a flag value of 1 indicates a minor data quality issue, and a flag value of 2 indicates a major data quality issue. A user may choose, for example, to eliminate data with a major quality issue. To do this, they would simply mask/nan the variable whereever the flag variable has a value of 2, leaving the variable array as

1 2 6 5 4 3 6 5 4 3 - - 4 5 6 7 7 6 5 3 2.

Value-based flags will have the same dimensions as their associated variable, and the following variable attributes:

  • _FillValue: -128b

  • standard_name: ‘status_flag’ if the associated variable has no standard name, otherwise ‘<variable_standard_name> status_flag’

  • long_name: Flag for <variable_name>

  • flag_values: An array of the values that the flag variable can take. Typically runs from 0 to <length of flag_meanings> - 1.

  • flag_meanings: A space separated string of the meanings of each of the values in flag_values.

Bitmask flags

While the value-based flags map the values of an array to a single meaning, bitmask flags allow the representation of a boolean array for every flag_meaning. This is done by mapping each flag meaning to an increasing power of 2, which allows the representation of every possible state of every meaning using values from 1 to \(2^{\text{num. flags}-1}\). A value of 0 indicates that no flags are set, and is set as a fill value. In order to a bitmask flag it must first be unpacked. This adds to the complexity of using the flag, but makes flags much more powerful, so most variables in the FAAM core data product use bitmask flags.

For example, consider a variable array with values

1 2 6 5 4 3 6 5 4 3 2 1 4 5 6 7 7 6 5 3 2

and its corresponding flag with values

1 1 3 3 2 2 4 4 4 4 6 6 6 6 8 8 5 5 3 3 1

To understand the meaning of these flag values, we can look at the flag_masks and flag_meanings attributes of the flag variable, which may look like

  • flag_meanings: aircraft_on_ground flow_out_of_range temp_out_of_range data_out_of_bounds

  • flag_masks: 1b 2b 4b 8b

There are four meanings, with each associated with a value of \(2^n\) with \(n\) taking the four values 0, 1, 2, 3. In the FAAM core data, flag values are guaranteed to be increasing powers of 2, thus the flag array can be unpacked simply by progressively right-bitshifting the flag array, and taking the result modulo 2. In python, this can be achieved with the following code:

# Note that we don't need to worry about using the flag_masks attribute, as it
# is guaranteed to be powers of 2 from 1 to 2^(n-1)
unpacked = {}
for i, meaning in enumerate(flag_var.flag_meanings.split()):
    unpacked[meaning] = (flag_data >> i) % 2

this would leave us with the following in unpacked:

{
    aircraft_on_ground: array([1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1]),
    flow_out_of_range: array([0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 0]),
    temp_out_of_range: array([0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 0, 0, 0]),
    data_out_of_bounds: array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0])
}

Bitmask flags will have the same dimensions as their associated variable, and the following variable attributes:

  • _FillValue: 0b

  • standard_name: ‘status_flag’ if the associated variable has no standard name, otherwise ‘<variable_standard_name> status_flag’

  • long_name: Flag for <variable_name>

  • valid_range: The valid range of values in the flag variable array. Should be 1b, 2^(<number of flag_meanings>) - 1

  • flag_masks: An array of the values that the flag variable can take, which will runs from 1 to 2^(<number of flag_meanings> - 1).

  • flag_meanings: A space separated string of the meanings of each of the values in flag_values.