Mandi Project - Technology

Technology - How it Works

Our goal is to use state-of-the-art Automatic Speech Recognition (ASR) technology to provide the latest agricultural commodity prices and weather information around-the-clock, over telephone. Thus preventing the buyers from exploiting farmers and procuring the produce at lower than the standard AGMARK set rates.

All that the farmer has to do is, he/she dials a number from his/her phone, and tells the name of the commodity for which he/she needs the price in a particular district or makes a request for weather information by simply speaking it out. The system then recognizes the request and plays out the latest price/weather information from a database that is updated on a daily basis.

The Mandi Project has two vital components: the front-end User Interface (UI) and the back-end ASR engine that work in tandem.

Front End : User Interface

Mandi system is programmed to have the following call flow to understand what the farmer needs.

It prompts the farmer to speak out his/her district name
Then it passes on the audio of the spoken district name to ASR engine, which decodes the audio and returns the corresponding text
The system reconfirms the text by playing it back to the farmer
Likewise, it prompts asking if one needs commodity/weather information and reconfirms
Finally, it retrieves the relevant latest information from the database and plays out to them

Back End : Automatic Speech Recognition Engine

The most challenging aspect of the project is the ASR, as it demands expertise in multiple engineering areas including speech processing, statistical signal processing, machine learning, mathematics, programming, scripting and many more. The brain of the system is the ASR system, built using open source Speech Recognition tool-kit Kaldi, comprises of Hidden Markov Models (HMMs) and Gaussian Mixture Model (GMMs) built using Mel Frequency Cepstral Coefficients (MFCC) extracted from audio data.

We provide this service in nine different Indian languages with a separate set of CDHMM module for each language is being built. The languages are Assamese, Bengali, Hindi, Kannada, Oriya, Marathi, Tamil, Telugu and Gujarati. Deep Neural Network (DNN) based model have been deployed for Hindi(UP) and Tamil.

Systems built cater to farmers in twelve Indian states: Assam, West Bengal, Uttar Pradesh, Bihar, Jharkhand, Karnataka, Odisha, Maharashtra, Tamil Nadu, Andhra Pradesh, Telangana and Gujarat.