CTD Database 2015 includes additional data-quality code columns to annotate bad or questionable data so they may be excluded during queries. The Microsoft Access CTD database is updated using combined, bottle-corrected CTD + bottle data csvs for upcast and downcasts. Both the Access database mdb file plus the downcast & upcast CTD.csvs are available. Refer to the CTD Database Index and the CTD.csv with QC Index pages for columns and descriptions. Please note that CTD data requires reprocessing to add these new columns and fill in the data codes so the database will be gradually updated with older cruises.
There are two versions of the CTD final data for each cruise (YY is year, MM is month, SS is ship code):
|Year||Winter Cruise||Spring Cruise||Summer Cruise||Autumn Cruise|
|2015||1501NH CTD+BTL csvs
1501NH CTD complete
|2014: CTD+BTL csvs||1402SH CTD+BTL csvs
1402SH CTD complete
|1404OS CTD+BTL csvs
1404OS CTD Complete
|1407NH CTD+BTL csvs
1407NH CTD Complete
|1411NH CTD+BTL csvs
1411NH CTD Complete
|2013||1307NH CTD+BTL csvs
1307NH CTD Complete
|1311NH CTD+BTL csvs
1311NH CTD Complete
CTD Database Development Notes
Please note as of mid-2015, a revision of the CTD database is currently underway. With additional data quality code columns added to the CTD+bottle csvs offered as final processed data, new files are being created by re-processing the CTD archives. Measurement values should remain unchanged but the format is being updated.
Older information about database development is described below:
Starting with CalCOFI 0907M2, the BTLvsCTD program (developed by SIO-CalCOFI) generates several database-friendly csvs. With Seasave Data Processing software v7.19, sensor calibration coefficients are embedded as xml data in the .hdr file. BTLvsCTD.exe parses the .hdr for all stations into three csvs: hdr.csv, xmlcoeff.csv, & span.csv.
Starting with 0907, the first cruise to use dual SBE43 O2 sensors so the data tables will have these columns by default. Earlier cruises, using single O2 sensors will auto-fill the tables, leaving the second O2 columns blank.
Since sensor installation may vary from cruise to cruise and sensors may occupy different voltage channels, the voltage references v1 v2 v3...
are replaced by TransV FluorV AltV... whenever possible.This requirement required a revision of BTLvsCTD.exe dated 9 Nov 2009 since I had originally maintained the voltage # in respect to sensor. But this creates a database import problem; v0_Trans, v1_Fluor will not integrate with v0_Fluor, v1_Trans so both are renamed TransV FluorV, removing the specific voltage # reference.
Currently, relabeling of the voltage channels is offline since it creates data reprocessing issues. You will have to refer to the hdr file to determine which sensor occupies the 8 voltage channels (V0-V7). We try to keep the channels consistent unless deploying test sensors for performance assessment. Like the RINKO Oxygen Optode on 1210NH.
BTLvsCTD.exe's main module also generates bottle-corrected upcast and downcast csvs, merging all the individual station up and down csvs into two files. A dbcoeff.csv file is generated tabulating the cruise-ave bottle vs CTD coefficients as well as the station regression coefficients (each station's CTD data are regressed against bottle samples from that station, reducing the station-to-station variability witnessed using the cruise-ave coefficients). If bottle data are not available then only cruise-ave coefficients are used to correct the CTD data.
BTLvsCTD.exe has a module to automatically relabel the v0 v1 v2... to the sensor voltage in all .asc files. (I plan on using the hdr to automate this but currently the table in BTLvsCTD tabulating voltages vs sensor is used.)
BTLvsCTD.exe has a file renaming module that renames all dYYMM###.asc & .hdr, uYYMM###.asc & .hdr files to web-format YYMM_LLLLSSSS_###d.asc & .hdr, YYMM_LLLLSSSS_###u.asc & u.hdr.
The CalCOFI CTD Database format is currently being revised (Sept 2015). Data quality columns for each sensors are being added to the database tables to flag questionable or bad data. Until this version is available, the original CTD database links will be available below.
The CTD database is constructed or updated from each cruises' CTD cast files after bottle-corrections have been applied. Two tables - upcast & downcast - are in the database along with bottle data used to calibrate the sensors. Note that typically three columns are available for each sensor: Seasave-only processed sensor data - no bottle correction applied; cruise-corrected ("CruiseCorr") = bottle-corrected sensor data using coefficients derived from regressing all bottle data to sensor data; station-corrected ("StaCorr") = bottle-corrected sensor data using coefficients derived from regressing only the current station's bottle data to sensor data. Station-corrected sensor data is considered the best when 20 or more bottle samples are available to calibrate the sensor measurements. Station-correcting helps eliminate the station-to-station variability in certain sensor, particularly the ISUS when a new lamp is installed and experiencing "burn-in" drift.
|ID||Number/Long Integer||Database assigned index|
|Event_Num||Number/Integer||CTD-at-depth Event Number|
|Cast_ID||Text||Cruise_OrderOccupied_Up or Down|
|Date_Time_UTC||Date/Time||CTD-At-Depth GPS UTC|
|Date_Time_PST||Date/Time||CTD-At-Depth Computer PST|
|Lat_Dec||Number/Double||GPS Decimal Latitude (always N)|
|Lon_Dec||Number/Double||GPS Decimal Longitude (always W)|
|Sta_ID||Text||CalCOFI Line Station|
|Depth||Number/Double||Depth in meters|
|Pressure||Number/Double||Pressure in decibars|
|Temp1||Number/Double||CTD Sensor 1 Temperature deg C|
|Temp2||Number/Double||CTD Sensor 2 Temperature deg C|
|TempAve||Number/Double||Ave CTD Sensor 1+2 Temperature deg C|
|Salt1||Number/Double||CTD Temp 1+Conductivity 1 Salinity PSU|
|Salt1_Corr||Number/Double||CTD Temp 1+Conductivity 1 Salinity PSU Bottle Offset Corrected|
|Salt2||Number/Double||CTD Temp 2+Conductivity 2 Salinity PSU|
|Salt2_Corr||Number/Double||CTD Temp 2+Conductivity 2 Salinity PSU Bottle Offset Corrected|
|SaltAve_Corr||Number/Double||CTD Primary+Secondary Salinity Bottle-corrected Average|
|Ox1||Number/Double||CTD Primary Oxygen mL/L|
|Ox1_CruiseCorr||Number/Double||CTD Primary Oxygen mL/L, regression corrected using all O2 bottles|
|Ox1_StaCorr||Number/Double||CTD Primary Oxygen mL/L, regression corrected using matching station O2 bottles|
|Ox2||Number/Double||CTD Secondary Oxygen mL/L|
|Ox2_CruiseCorr||Number/Double||CTD Secondary Oxygen mL/L, regression corrected using all O2 bottles|
|Ox2_StaCorr||Number/Double||CTD Secondary Oxygen mL/L, regression corrected using matching station O2 bottles|
|OxAve_StaCorr||Number/Double||CTD Station-corrected Primary+Secondary Oxygen Average mL/L|
|EstChl_CruiseCorr||Number/Double||Fluorometer Estimated Chl, regression calc'd using all cruise chlorophyll-a samples|
|EstChl_StaCorr||Number/Double||Fluorometer Estimated Chl, regression calc'd using matching sta chlorophyll-a samples|
|ISUSV||Number/Double||ISUS Nitrate Sensor Voltage|
|EstNO3_CruiseCorr||Number/Double||ISUS Estimated NO3, regression calc'd using all cruise NO3 samples|
|EstNO3_StaCorr||Number/Double||ISUS Estimated NO3, regression calc'd using matching sta NO3 samples|
|SigThetaTS1||Number/Double||Sigma-theta, primary sensors, Seabird algorithm|
|SigThetaTS2||Number/Double||Sigma-theta, secondary sensors, Seabird algorithm|
|BAT||Number/Double||Transmissometer Beam Attenuation Coefficient, Seabird algorithm|
|XMiss||Number/Double||Transmissometer %Transmission, Seabird algorithm|
|SPAR||Number/Double||Surface Photoradiometer Irradiance (unavailable on NOAA vessels)|
|PAR||Number/Double||CTD Insitu Photoradiometer Irradiance, Seabird algorithm|
|PoT1||Number/Double||Primary Potential Temperature, Seabird algorithm|
|PoT2||Number/Double||Secondary Potential Temperature , Seabird algorithm|
|DynHt||Number/Double||Dynamic Height, Seabird algorithm|
|SVA||Number/Double||Specific Volume Anomaly, Seabird algorithm|
|OxSat1||Number/Double||Primary O2 Saturation, Seabird algorithm|
|OxSat2||Number/Double||Secondary O2 Saturation, Seabird algorithm|
|BTL_Temp||Number/Single||Bottle Temperature (should match CTD Primary Temperature)|