An Approach Based On Big Data And Machine Learning For Optimizing The Management Of Agricultural Production Risks

Colin Hay and Tony Payne developed the “great uncertainty” concept in order to synthesize certain aspects in relation to the dynamics of economic processes in the current period. They highlight three key elements that mark the structural changes of the moment and that generate incertitude: financial crisis, global economic power change and environmental threats, which bring about significant structural changes in the economy. At the same time, a technological paradigm change can be noticed at global level, characterized by an unprecedented growth of the “informational power” (computing power, storage power, high analysis capacity) oriented towards the creation of “anti-fragile” economic and administrative structures capable to develop rigorous analyses to outline the tendencies in this area dominated by uncertainty. The dual solutions in relation to hedging the agricultural commodities supplied by the OTC markets through the weather derived products and those supplied by the insurance companies based on weather indices are well known. We are facing a situation in which the hedging solutions based on OTC market products suffer from a deficit of image following the 2007/2008 crisis, and the solutions provided by the insurers raise the final price by up to 10% depending on the insurance market of each country, playing an important role on the reinsurance market as well. A unification variant of these markets is presented, providing for operational and production hedging for the farmers acting in a global agricultural industry worth 3 trillion USD, for which 150 billion observations is processed in connection with the soil types, summing up 200 TB of data each month. Weather events are simulated and risk is calculated on 4 × 4 km 2 areas. These data are combined with business data to support farmers’ management decisions. All these functionalities are supported by a package of technologies based on: NoSQL databases, including HBase, Hive, MySQL, Excel, by software service suppliers in cloud computing context such as Amazon S3, as well as applications delivered under SaaS regime. A recent report of the Global Institute Mc Kinsey refers to machine learning as an innovation stimulating technology. Being classified as a “silent technology”, this can be applied in modeling the weather phenomena to obtain the hedging level necessary to the production context. A composite index such as the Selyaninov index used in a technological context of machine learning type can bring about certain advantages in production risk evaluation. The paper investigates the capacity of certain technologies from the "Big Data” category to add value in the development of certain risk markets in order to obtain a proper hedging.