Environmental data have historically been captured through manual sampling, monitoring, and measurement activities. As environmental professionals, most of us have taken groundwater samples, lowered water level meters down wells, or have taken manual pH readings in the field. Manual data collection still serves a significant scientific purpose in our environmental workflows, but it’s now accompanied and complemented by automated time-series data collection from sensors, loggers, and open data platforms. Managing discrete and time-series data in the same central EQuIS database improves the accuracy and reliability of our statistical analyses, pattern recognition abilities, and relationships between continuous and discrete values like flow rates and Total Maximum Daily Load (TMDL).
Time-series data add a new layer of insight to the environmental workflow, as well as additional complexity in the way they are managed. These data are produced in a variety of electronic formats and can contain thousands of measurements per day. Processing and managing these large datasets requires a different table structure than the standard Sample/Test/Result hierarchy. To avoid the computational cost of latency, EQuIS leverages a historian-like data structure and clustered index design to minimize database input/output and maximize query performance.
Aggregation agents are used to correct data in real time and render meaningful decision-support metrics such as hourly, daily, and weekly averages. Raw data are captured for verification and can be stored perpetually or cleared out at a pre-defined frequency. Users leverage open data web services for further verification of data. For example, a pressure reading taken in the field can be compared to a reading from NOAA’s NCDC web service and flagged if the delta exceeds the acceptable range.
Different than discrete data management, time-series metadata like instrument vendors and serial numbers are stored in separate tables to reduce the number of fields per table and improve processing speeds. The value of our time-series data is dependent upon the tools we use to interrogate and visualize it, as well as how quickly we can gain access to the data. EQuIS Live is continually optimized to reduce latency, maximize performance, and provide meaningful integrations with other environmental data stored in EQuIS.
Time-series and discrete sampling data are managed in a central EQuIS repository. As new data are loaded by the laboratories or telemetered to the database, they can be analyzed together or separately using multiple reports or third-party visualization platforms. For example, discharge monitoring reports use discrete sampling results, TMDLs, continuous flow measurements, and meteorological data to ensure industrial discharges are within regulatory limits.
By centrally managing these discrete and high frequency variables, a single report quickly pulls data from different tables in EQuIS, calculates the required values, and produces a presentation-quality deliverable. Similarly, you can interrogate discrete environmental metadata like lithology in conjunction with time-series water quality and groundwater elevation measurements.
Using time-lapse features in EQuIS EnviroInsite combined with high frequency chemistry data from EQuIS Live, watch your plume regress (hopefully), expand, or migrate as the water table fluctuates. These valuable insights help us gain a deeper scientific understanding of site conditions.
Furthermore, improvements in statistical accuracy and pattern recognition realized by leveraging both data types shorten our response time to compliance infractions and ultimately save money. Whether analyzing site conditions through a scientific or financial lens, there is a strong business case for managing these data together. Shortened response times, mitigation of risk to our sites, streamlined cleanup activities, and higher quality decision support are all realized through integration of our discrete and time-series data.
“Own the workflow, make better decisions” EQuIS™ is Earth’s most widely used environmental data management workflow and efficiently manages the environmental sample data for thousands of organizations. Please email firstname.lastname@example.org for more information.