Adsa Logo White Adsa Title White

Automated real-time integration of data from multiple sensors and nonsensor systems for prediction of dairy cow and herd status and performance.

M. M. Pérez

Events

06-24-2020

Abstract:

417
Automated real-time integration of data from multiple sensors and nonsensor systems for prediction of dairy cow and herd status and performance.
M. M. Pérez*1, G. Rubambiza2, B. Barker3, H. Weatherspoon2, J. O. Giordano1. 1Department of Animal Science, Cornell University Ithaca, NY, 2Department of Computer Science, Cornell University Ithaca, NY, 3Cornell Center for Advanced Computing, Cornell University Ithaca, NY.

Our goal was to build a software infrastructure to integrate data from multiple heterogeneous cow wearable and non-wearable sensors, herd management software, and climate sensors for subsequent data analytics. To address variation in exposed interfaces and data update frequencies to our aggregator, a Python client was prototyped to periodically check for unprocessed files stored on a farm PC. The unprocessed files are forwarded to a cloud-based aggregator. Files are structured to be processed into Google's Protocol Buffer (protobuf) objects. Protobufs allow for minimalistic encoding/decoding of data limiting the size of probable sparse messages such as cow health status and sensor data. Protobufs read and modularize the structures of Excel (XLSX and CSV) files into a singular structure used by the aggregator. In preliminary experiments using protobufs, our client achieved nearly an order of magnitude reduction in the size of data transferred to the aggregator (e.g., 14KB xlsx file to 1500 byte protobuf message). To streamline communication between the data aggregator and analytics services for prediction with improved modularity and reduced data sparsity, incoming data is stored into a NoSQL Cassandra database with a distinct table for each data source. This framework allows querying data using a cow number and timestamp. To transfer data to an analytics pipeline, RESTful queries are implemented to the data aggregator, which serves up CSV files generated from the database. Thus far, the data aggregator receives and integrates data from the herd management software, in-line milk sensors (yield and components), walk-in scale, a physical activity and resting behavior leg-mounted sensor, a rumination and eating behavior neck-mounted sensor, a reticulo-rumen temperature sensor, and climate conditions sensors in and outside barns. In summary, we built a software system that automatically integrates in real-time heterogeneous data from diverse sources at a dairy farm. Aggregated data is transferred to data analytic tools for prediction of cow and herd status and performance.

Keywords: data integration, sensor, prediction.