Thursday 4 May 2017

Drought Analytics of Karnataka Using MODIS Satellite Level 2 Data(2001-2015) using ArcPy python package and Tableau Public


 Humans have been changing the face of the earth since they emerged out out of Africa. Most of the recent changes has been happening since 1750s. Be it industrial revolution, be it green revolution or white revolution ; all has happened in this short period of time. However post 1950s the change has accelerated in such as way that its being called as Anthropocene : The Age of Humans.

The Age of Humans also brought with the complication of guiding the whole mankind through the problem that came along with Anthropocene. Drought is one of the biggest challenge the mankind has, is and will be facing. There are lot of index to define different type of droughts; this would be a topic for some other day. In the present article i am going to come out with simple index on drought and how to extract this index using MODIS Satellite Level 2 data for entire India using ArcGIS tools and visualize data using Tableau Public.

In previous article i had explained point extraction of data on MODIS level 3 data. Here we would be concentrating on MODIS Level 2 data. Thanks to our work at NIIT University's GIS Lab, we are able to extract and process MODIS Level 2 data for entire India. I am also happy to announce that NIIT University GIS Lab has now capabilities to work on Level 2 data of 4 MODIS products, in large scale running in 100s of GBs. Existing index of Drought is a simple ratio of NDVI and LST(In Kelvin). It does have its own flaws but its simplest and easiest way to conduct preliminary analysis on huge Spatio-temporal data. As this project has other stack holder I would abstain from disclosing the full methodology. I would rather discuss this in brief.

Method:

As previously said we have capability to extract all years data of MODIS Level 2 satellite data for whole India(South Asia) or any part of the world. Resolution of this product varies from 250meters to 500 meters and 1km. NDVI product that has been derived for whole India, has resolution of 500 meters. LST which is in Kelvin has 1 km resolution. This data is 8 days(LST) and NDVI (16 days) composite data. We have codes to convert 8 days LST to 16 days LST and NDVI 500m data to 1 Km resolution. Hence end product will be 1km resolution and 16 days composite data. Drought Index will be calculated pixel wise. Has its index, we intend to know how many number of pixel are present in the defined range of the Index. Drought Index has both Minimum and Maximum, all other data will vary between these. Minimum would be -9 and maximum would be 36. This index is just number but this is power full number that could give a lot of insight in the drought.

If index is less which means NDVI is less and LST is high, and if index is more means NDVI is high and LST is less. Every other things would vary between these interpretation. We have created numerical class for putting all values of pixels index less that 0 in one class and then defining 7 class with the gap of 5. In all 8 classes Class 0 to Class 7. This is still not standard, more research with more parameters could be added to come out with brand new index. Here we are interested in having capability to extract data and analysis district wise using raster data, rather than on Index itself. We have also developed scripts that can take shape file(GIS) of whole state or nation or region containing 1 level of subdivision and extract the raster data based on the subdivision. This extracted data would be in the form of CSV file. The data extracted is just simple conversion of attribute table of each raster file that has been clipped based of shape file given. This is done using ArcPy a python package provided by ESRI. CSV file is fed into Tableau software of both GIS and non-GIS visualization. I am glad to tell readers that this integration of Arcpy and Tableau software for analyzing Raster data(MODIS) is uniquely developed by us in NIIT GIS lab. This method uses processing power of ArcPy at ArcGIS server and visualization power of Tableau.

 Link 1
Dry Areas(here u can interact with data by changing years and months)

Link 1: Please Click above Link to see interactive graphs
Our data processing is done for whole India. Just extraction of data from drought raster file took more than 24 hr of processing for entire India. I can tell that i have district wise data for all district of India in this Drought Index. In coming day LST for all districts would be generated. Here in Tableau public I have presented data for my state ie Karnataka state in Republic Of India. Same can be done for any part of the world. Tableau public provides unique opportunity to share data publicly. Given above figure (links) are off Tableau public . These are interactive graphs users can change the selection given in the Tableau. Tableau public is free. Tableau also comes with GIS feature which i have tried to showcase. Lot of interpretation can be drawn out of these graphs. Figure 1(Link 1) will give the spatio-temporal variation of 15 years of Drought Index for all district of Karnataka, for all months and all years. User could filter the visualization. Figure 2(link2) gives plotting. Figure 3 (Link3 ) gives GIS visulaiztaion of Dry spell for entire data. Dry spell data is for post-Rabi season. Northern part of Karnataka state is prominently effected by drought like situations. Reader, I would encourage you to use filters to see how this Index changes year wise and month wise.

Link 2: DroughtAnalytics of Karnataka Using MODIS Satellite Data 2001-2015

There could be lot of discussion on just visualization but typing constrain. But i promise reader that I would come back with more such interesting ESDA(Earth Science Data Analytics) works. Next time it would be LST for districts of karnataka or could be at sub-division level or could be at panchayat level. I have scripts which could work till Panchayat level, even i would be eager to work on this but only processing capability is blocking the move at Panchayat level. Anyway till next time happy learning , Happy ESDA.

Link 3 Here reader can select the graph based upon District and Month. Interpretation of the graph will give rain fall pattern of each districts of Karnataka
Readers please do contact me if you which to derive data for your own area. I would more than happy to help you. mail me at shivaprakash.ssy@gmail.com or contact me via LinkedIn. I would also entourage readers to download Tableau Workbook of these analytics and you can work on my data. We would also provide assistance for those who want to extract data for their region.