DBCP Community Practiceshttps://repository.oceanbestpractices.org/handle/11329/11262024-03-28T18:28:29Z2024-03-28T18:28:29ZDBCP drifting buoys DAC data processing chain, Version 1.0.https://repository.oceanbestpractices.org/handle/11329/11002019-11-05T13:58:48Z2018-01-01T00:00:00ZDBCP drifting buoys DAC data processing chain, Version 1.0.
The Coriolis DAC drifting buoys data processing chain decodes, processes, formats and performs quality control on drifters data.
The attached manual describes how to install, configure and use the Coriolis Python decoder used to process Drifting buoys data and metadata.
The main function of the decoder is to format the drifter data (drifter metadata, drifter measurements, drifter technical data) into a unique Drifter NetCDF CF file.
The decoder applies Real Time Quality Control (RTQC) tests on the Drifting buoys data file.
It manages both Argos or Iridium messages.
The drifter decoder is used through the IridiumSbdDrifterDecoder.py or the ArgosDrifterDecoder.py programs. It generates an JSON report of the processing done and is designed to be used in a Data Assembly Center (DAC) real time flux.
2018-01-01T00:00:00ZNetCDF file format checker for Argo floats, Copernicus In Situ TAC, EGO gliders, OceanSITES, Version 1.15 with new rule files.https://repository.oceanbestpractices.org/handle/11329/10952019-11-05T13:57:30Z2019-01-01T00:00:00ZNetCDF file format checker for Argo floats, Copernicus In Situ TAC, EGO gliders, OceanSITES, Version 1.15 with new rule files.
This java tool checks the format of a NetCDF file and produces a report of conformity with:
Argo floats NetCDF formats
EGO gliders NetCDF formats
OceanSITES NetCDF formats, with Copernicus implementation (valid rules :A_CF*.xml & B_CF*.xml)
OCO NetCDF formats (coastal operational oceanography)
DBCP drifting buoys
The NetCDF format checker is flexible, you may add your own format rules in the RULES directory. Each file format is specified in an XML rules file.
2019-01-01T00:00:00ZSea surface salinity quality control processes for potential use on data buoy observations. Version 1.3.https://repository.oceanbestpractices.org/handle/11329/8682021-11-01T12:45:12Z2011-01-01T00:00:00ZSea surface salinity quality control processes for potential use on data buoy observations. Version 1.3.
This document will aim to provide processes and approaches to real-time and delayed mode quality control of Sea Surface Salinity data, for review by the DBCP community. This document aims to bring together the best practice and suggested approaches to quality controlling salinity data from various programmes and for several different types of observing platforms. Once the DBCP has ascertained which processes apply to drifting and moored buoys, the appropriate tests and procedures can be added to DBCP Technical documents which already exist.
2011-01-01T00:00:00ZData Buoy Cooperation Panel
10.3 - Guidelines to
Oceanographic
Instruments.[DBCP 34 Session].https://repository.oceanbestpractices.org/handle/11329/8402019-11-05T14:03:44Z2018-01-01T00:00:00ZData Buoy Cooperation Panel
10.3 - Guidelines to
Oceanographic
Instruments.[DBCP 34 Session].
The Guidelines for Oceanographic Instrument standards and Methods of Observation aids in instrumentation and measurement techniques used to make ocean observations. DBCP 31 session agreed that DBCP guidelines for instrument standards ought to be developed. The Panel agreed that the following should initially be undertaken: (i) checking existing materials (e.g. WMO No. 8, Guide to Meteorological Instruments and Methods of Observation), (ii) agreeing on the scope of the guidelines document, and the methodology for producing it, and then (iii) proposing a work-plan, and who should contribute. The guidelines document could include instrument classes, and information on traceability requirements, while certification could simply be undertaken at the DBCP level through a committee to be established for that purpose. The Panel requested R. Venkatesan (India) to lead such developments, with assistance from Luca Centuriani, David Meldrum, and the Secretariat in the view to make a proposal (possibly a draft guidelines document) at the next Panel Session . The Panel requested R. Venkatesan to lead such developments, with assistance from Luca Centuriani, David Meldrum, and the Secretariat in the view to make a proposal (possibly a draft guidelines document) at the next Panel Session
During DBCP 32 a draft guidelines was presented addressing the points suggested such as checking existing materials (e.g. WMO No. 8, Guide to Meteorological Instruments and Methods of Observation),, defining the scope of the guidelines document, and the methodology for producing it and clarity on the work plan was getting emerged (Action DBCP 33)
Further DBCP Session 33 discussed and recommended Rec 8.7/2 to pursue this work and: DBCP guidelines for oceanographic instruments (revised draft) was presented and urged members to review and finalize before DBCP 34 Action 9.2/3
In oceanography, much is still to be learned through observation and instrumentation is often the limiting factor to observation. It is essential to develop skill in the utilization of oceanographic instrumentation and to specify or invent new instrumentation that will aid observation. Better techniques of measurement lead to greater and more accurate understanding of the natural system. The short term benefit of such understanding is the ability to predict the response of the ocean and the long term benefit is a change in our own behaviour as a society that leads to opportunities for human use and individual appreciation to society’s benefit.
The ocean observations have immense societal value through various climate and weather applications, such as forecasts of droughts, tropical cyclones and associated storm surges, and projections of decadal to multidecadal climate variability and change. These observations provide vital information in the management of ocean ecosystems and human adaptation activities in response to climate variability and change. But there is no concrete guide for Oceanographic Observations and Instruments, such as CIMO guide for the Meteorological Instruments, which is already well established by World Meteorological Organization (WMO). Thus, standardization of methodologies and instrumentation in oceanographic observation has remained a key concern in the oceanographic community. Industries of oceanographic sensors from many countries have attempted to develop standardized ocean observation techniques and new technology in instrumentation and these attempts are sparse and not wide spread.
The oceanographic instruments were developed by small companies and have been taken over by big companies nowadays. The standardization of protocols used to control and configure the instruments, and to retrieve their data are minimal. Hence an attempt is made to detail the specifications of instruments, practices and procedures for data collection, quality control of data and standardization of instruments. The aim of this guide is to fill the void that has occurred from few decades in ocean observation instrumentation and standardized methodologies based on the experience and knowledge gained by the scientific community.
The process of standardisation required in various stages from Instrument to data reception is described below
1. Instrument
2. Integration of Instruments to Observing Platform & Real time Data Transmission
3. Data and Met Data format : Standardization efforts in the marine research community have largely focused on this standard formats for data and meta-data to ensure interoperability between data producers andconsumers
Instrument to Data
1. Instrument standard
• Instrument Protocol: RS232 and RS485 serial are the dominant physical layer protocols and has been increasingly displaced by Ethernet nowadays; the syntax and command sets for the instruments are exclusively designed by the manufacturer.
• Some Instruments return data in a readable format, typically as ASCII-encoded decimal numbers separated by commas or tab characters. However, there are many other formats, with varying degrees of complexity, compatible to the characteristics of each instrument. These may include fields defined by fixed or delimited formats, and binary encoding of various integer and floating-point number types. Different data fields within a packet may be encoded differently.
• A single instrument may also produce multiple packet formats within the same data stream. For example, an instrument may report “housekeeping” data in separate packets, and at different rates, from its primary measurement data. The contents of a data field could even indicate the length or a format of subsequent data within the same packet.
2. Integration of Instruments to the Observing Platform and Data Transmission:
• The oceanographic instruments are often integrated into an observing system or a sensor network, which provides software infrastructure for many useful functions, such as instrument data acquisition, data logging, and data transfer to other locations via wired or wireless telemetry links. Most observational systems use generic or standard protocols for these functions.
• A driver software that translates between specific instrument and generic system protocols must be written for each kind of instrument.
• The driver must be configured properly as soon as the instrument is installed onto a communication port on the observing system.
• A definition of the raw instrument protocol exchanged between the instrument and the data acquisition system is essential
Real time data transmission from Observing Platform
• After instruments are connected to an observing system’s network, the real - time remote access to instrument data via the Internet is provided. Few instruments provide communications in a standard command protocol format—thus observatory or shore-based software is required to transform the instrument data format to a standard form.
• Standardization is required to minimize the need for software development and manual configuration steps, thereby reducing system complexity, development and operational cost.
3 Data and Met Data:
• Standardization efforts in the marine research community have largely focused on the standard formats for data and meta-data to ensure interoperability between data producers and consumers.
• In terms of metadata formats, most viable standards are based on XML, and use the ISO19115 schema for describing geographic information, with some extensions to cover characteristics of marine data.
• The International Oceanographic Data and Information Exchange (IODE) of the Intergovernmental Oceanographic Commission (IOC) of UNESCO promote XML and ISO19115 for metadata encoding, with the World Meteorological Organization (WMO) Core Metadata Profile.
This definition may be noted for reference throughout this document
• Sensor – that measures a desired parameter
• Instrument – a sensor or collection of sensors
• Components – Data loggers, communications and positioning equipment, power supplies, ancillary cables and connectors, mounting hardware, etc.
• Platform – physical structure on which components are deployed in the field (e.g., ship, mooring, drifters, floats, gliders etc.)
Existing standard
• Many navigational marine instruments implement NMEA 0183 or NMEA 2000, but the standard’s restrictions to ASCII formats and a 4800 baud serial data bus have limited its application to “scientific” instruments.
• Marine instrumentation most commonly uses serial links, so IEEE 1451.2 would apply. IEEE 1451.2 is fully compatible with an RS232 instrument, using the communication and measurement services described in IEEE 1451.0. Different protocols are addressed by different branches of the standard, for example,1451.2 for RS232, I C, and SPI; 1451.4 for analog sensors, 1451.6 for controller area network (CAN), etc. IEEE 1451 uses the term transducer interface module (TIM) to refer to a sensor or actuator, and a network capable application processor (NCAP) to mean a controller interfacing to one or more TIMs. Data from the non-IEEE 1451 instruments are processed to input data into an IEEE 1451.0 server. This server publishes data using the HTTP 1451 standard.
• The IEEE 1451 Smart Sensor Interface Standard provides a common communications architecture with sensors over different communication protocols at the physical level. This standard has not yet been widely used, especially in marine sensors, and the lack of software tools for implementation limits its adoption. However, it has some capabilities that may be useful in marine networks.
• Standard
• In the measurement context considered here, the word “standard” is used with two meanings. Firstly, it refers to a calibrationstandard – a method which is used to provide traceability back to a common benchmark. Secondly, it may refer to aspecificationstandard – a written procedure describing the method for undertaking a measurement. Here we propose specification standard, as calibration standards are comparatively better well established.
Specification standards
• Specification standards are documents describing procedures to be followed when undertaking measurements. The highest of such standards are international standards produced under the auspices of organisations such as ISO (International Organization for Standardization) and IEC (International Electrotechnical Commission).ISO produces standards which cover physical measurements; with regard to underwater acoustics including environmental noise or noise radiated from specific sources such as ships. IEC produces standards that cover electrical measurements, including the calibration of instruments such as hydrophones. ISO and IEC standards are typically adopted as national standards within member countries
The need of standardisation of Instrument to Observing platform
Standardizing the installation and operating processes can dramatically reduce costs, as well as the risk of failures due to manual errors.
Standardization also facilitates easier maintenance and replacement of observatory instruments, and traceability of the data they generate.
Standardization will ensure manufacturers a well-tested technology.
2018-01-01T00:00:00ZInternational Tsunameter Partnership of Data Buoy Cooperation Panel Tsunameter Equipment Performance Standards and Guidelines.https://repository.oceanbestpractices.org/handle/11329/3332021-11-01T12:46:01Z2016-01-01T00:00:00ZInternational Tsunameter Partnership of Data Buoy Cooperation Panel Tsunameter Equipment Performance Standards and Guidelines.
This document sets out functional, performance and other operational characteristics for deep ocean tsunami detection stations that will meet the requirements of local, regional and ocean-wide tsunami warning systems. Compliance with these guidelines and with their related quality assurance processes will enable warning centres, equipment purchasers, operators and non-warning-centre data users to have confidence in a tsunameter’s performance, data quality and interoperability, regardless of the specific equipment’s design or source of supply
2016-01-01T00:00:00ZDBCP Quality Control Guidelines: written August 2000, updated 2009.https://repository.oceanbestpractices.org/handle/11329/1242021-11-01T12:46:45Z2009-01-01T00:00:00ZDBCP Quality Control Guidelines: written August 2000, updated 2009.
Quality control procedures, jointly developed and implemented by the DBCP, GTS Data Processing centres and the operators buoys, currently ensure that surface observations are validated in real-time before insertion on to the GTS (see DBCP Technical Document No. 2). Sub-surface (e.g., from the TAO array) data are further controlled by NOAA / NDBC. Several other bodies (ECMWF, national weather and oceanographic agencies, GDC, ISDM, etc.) contribute to an active off-line assessment of data quality. A well-defined (see Annex A) feedback mechanism ensures that any interventions arising from this off-line quality control (e.g., modifications to individual sensor transfer functions) are implemented into the real-time data processing chain in a coordinated and audit-able fashion. Some history of the mechanism is given below.
GTS Buoys: quality control; guidelines
2009-01-01T00:00:00ZGuide to Moored Buoys and Other Ocean Data Acquisition Systems.Meindl, A.https://repository.oceanbestpractices.org/handle/11329/812021-11-01T12:49:08Z1996-01-01T00:00:00ZGuide to Moored Buoys and Other Ocean Data Acquisition Systems.
Meindl, A.
The DBCP at its 2nd session (Geneva October 1986) noted that there was a clear requirement for a technical document on the subject of moored buoys which would both provide essential information for countries wishing to initiate a programme, as well as act as a means for sharing experiences amongst countries already active in the field. This guide provides a guide to moored buoys and other data acquisition systems.
Moored buoy observation; guides
1996-01-01T00:00:00ZReference Guide to the GTS Sub-system of the Argos Processing System. Revision 1.6.Data Buoy Cooperation Panelhttps://repository.oceanbestpractices.org/handle/11329/742021-11-01T12:49:49Z2005-01-01T00:00:00ZReference Guide to the GTS Sub-system of the Argos Processing System. Revision 1.6.
Data Buoy Cooperation Panel
A Reference Guide to the Argos GTS Processing Sub-system was prepared and issued at that time (DBCP Technical Document No. 2) to assist Principal Investigators (PIs) running Argos programmes and wishing their data to be distributed on the GTS; PIs and manufacturers intending to design Argos platforms and messages for GTS distribution; and GTS users who receive data from the Argos centres. This guide has recently been updated to reflect various changes that have been made in the last few years and to clarify certain issues. The guide should be read in conjunction with the Guide to Data Collection and Services Using Service Argos (DBCP Technical Document No. 3), which provides details of the structure of the sub-system, and provides background on the system's various applications.
GTS distribution. GTS codes, quality control; guides
2005-01-01T00:00:00Z