RHoK 4.0 - Oxford/Projects/Tropical Flood Prediction

From Random Hacks of Kindness
Jump to: navigation, search

Contents

[edit] Introduction

Wouldn't it be great if everyone living in flood risk areas received timely and accurate flood forecasts?


This project aims to bring this wish a little closer to reality.

[edit] =Update

18 Dec 2011. There's now a live demo of the completed parts of this project at [1]

[edit] What are we doing?

Bringing together in an easily accessible form records of past floods and the weather that preceded the flood. This will allow others to design and trial flood forecast models.

The data - we're bringing together data from

Though we're ensuring that what we do could work with other data sources. How about crowdsourcing flood observations.

Flood reports may be available for some countries, e.g.

http://ns1.mrcmekong.org/flood_report/2007/table_of_content.htm


There are examples of access to the reanalysis data at http://mike.saunby.net

[edit] Who is this for?

Right now we are aiming at researchers, particularly those in flood prone countries who might not normally be able to access large datasets (NetCDF anyone?).


[edit] Other RHOK projects

http://www.rhok.org/problems/data-scraping-dartmouth-flood-data

http://www.rhok.org/problems/rainfall-related-urban-risks-s%C3%A3o-paulo-city

[edit] APIs, etc

  • Pydap

[edit] Our API

To access sub-daily data for plotting or CSV output use

gvis20cr?lat=LAT&lng=LNG&yr0=YYYY&yr1=YYYY&mo1=MM&mo2=MM&fi=PARAM&tqx=GVIS

where

  • PARAM is one of air.2m prate runoff soilm
  • GVIS is most likely out:csv - but see link above for Google Chart Tools Datasource API
  • LAT is decimal degrees north, e.g. 13.28 (negative for south)
  • LNG is decimal degrees east, e.g. 105.88 (negative for west)

NB. only use yr0 == yr1 or yr1 == (yr0 + 1) if not you'll just get the start and end years, not all year in between. For very long runs of data ue monthly mean values.

e.g 

  http://saunby.net/cgi-bin/py/gviz20cr.py?lat=13.28&lng=105.88&fi=prate&tqx=out:csv&yr0=2006&yr1=2007&mo0=10&mo1=9

To access monthly mean values for plotting or CSV output use

gvis20cr?lat=LAT&lng=LNG&yr0=YYYY&yr1=YYYY&mo1=MM&mo2=MM&q=QUANT&tqx=GVIS

  • QUANT is one of prate_mon_mean ncep_prate_sfc_mon_mean air_2m_mon_mean ncep_air_2m_mon_mean


N.B. Other data is available. See the source code for more details.


Other info

This query http://www.esrl.noaa.gov/psd/thredds/dodsC/Datasets20thC_ReanV2/gaussian/monolevel/air.2m.1992.nc.ascii?lat[0:1:93],lon[0:1:191]

yields

Dataset { 

   Float32 lat[lat = 94];
   Float32 lon[lon = 192];

} Datasets20thC_ReanV2/gaussian/monolevel/air.2m.1992.nc;

lat[94] 88.542, 86.6531, 84.7532, 82.8508, 80.9473, 79.0435, 77.1394, 75.2351, 73.3307, 71.4262, 69.5217, 67.6171, 65.7125, 63.8079, 61.9033, 59.9986, 58.0939, 56.1893, 54.2846, 52.3799, 50.4752, 48.5705, 46.6658, 44.7611, 42.8564, 40.9517, 39.047, 37.1422, 35.2375, 33.3328, 31.4281, 29.5234, 27.6186, 25.7139, 23.8092, 21.9044, 19.9997, 18.095, 16.1902, 14.2855, 12.3808, 10.47604, 8.57131, 6.66657, 4.76184, 2.8571, 0.952368, -0.952368, -2.8571, -4.76184, -6.66657, -8.57131, -10.47604, -12.3808, -14.2855, -16.1902, -18.095, -19.9997, -21.9044, -23.8092, -25.7139, -27.6186, -29.5234, -31.4281, -33.3328, -35.2375, -37.1422, -39.047, -40.9517, -42.8564, -44.7611, -46.6658, -48.5705, -50.4752, -52.3799, -54.2846, -56.1893, -58.0939, -59.9986, -61.9033, -63.8079, -65.7125, -67.6171, -69.5217, -71.4262, -73.3307, -75.2351, -77.1394, -79.0435, -80.9473, -82.8508, -84.7532, -86.6531, -88.542

lon[192] 0.0, 1.875, 3.75, 5.625, 7.5, 9.375, 11.25, 13.125, 15.0, 16.875, 18.75, 20.625, 22.5, 24.375, 26.25, 28.125, 30.0, 31.875, 33.75, 35.625, 37.5, 39.375, 41.25, 43.125, 45.0, 46.875, 48.75, 50.625, 52.5, 54.375, 56.25, 58.125, 60.0, 61.875, 63.75, 65.625, 67.5, 69.375, 71.25, 73.125, 75.0, 76.875, 78.75, 80.625, 82.5, 84.375, 86.25, 88.125, 90.0, 91.875, 93.75, 95.625, 97.5, 99.375, 101.25, 103.125, 105.0, 106.875, 108.75, 110.625, 112.5, 114.375, 116.25, 118.125, 120.0, 121.875, 123.75, 125.625, 127.5, 129.375, 131.25, 133.125, 135.0, 136.875, 138.75, 140.625, 142.5, 144.375, 146.25, 148.125, 150.0, 151.875, 153.75, 155.625, 157.5, 159.375, 161.25, 163.125, 165.0, 166.875, 168.75, 170.625, 172.5, 174.375, 176.25, 178.125, 180.0, 181.875, 183.75, 185.625, 187.5, 189.375, 191.25, 193.125, 195.0, 196.875, 198.75, 200.625, 202.5, 204.375, 206.25, 208.125, 210.0, 211.875, 213.75, 215.625, 217.5, 219.375, 221.25, 223.125, 225.0, 226.875, 228.75, 230.625, 232.5, 234.375, 236.25, 238.125, 240.0, 241.875, 243.75, 245.625, 247.5, 249.375, 251.25, 253.125, 255.0, 256.875, 258.75, 260.625, 262.5, 264.375, 266.25, 268.125, 270.0, 271.875, 273.75, 275.625, 277.5, 279.375, 281.25, 283.125, 285.0, 286.875, 288.75, 290.625, 292.5, 294.375, 296.25, 298.125, 300.0, 301.875, 303.75, 305.625, 307.5, 309.375, 311.25, 313.125, 315.0, 316.875, 318.75, 320.625, 322.5, 324.375, 326.25, 328.125, 330.0, 331.875, 333.75, 335.625, 337.5, 339.375, 341.25, 343.125, 345.0, 346.875, 348.75, 350.625, 352.5, 354.375, 356.25, 358.125

http://www.flickr.com/photos/56379236@N02/6448231749/

[edit] Related projects (not RHOK)

[edit] Features List/Wish list

[edit] Targyet User Types

[edit] Version 1

(Mainly developing world as have undeveloped local meteorological services)

  • NGOs
  • Academics
  • Gov & local authorities. Infrastructure, health ministries, emergency services.
  • Infrastructure companies: mobile, electricity etc. (prepared for damage)

[edit] Later versions

  • Local radio (TV?). General presenters & weather forecasters
  • General public in flood risk areas
  • Researcher/aid worker in the field/traveling globally

[edit] Basic included in version 1

  • Three application modes of increasing zoom/granularity
    • Global
    • 2 deg square location & floods history view (default 30 year full history)
      • Includes lists of floods visible
    • Flood specific (default 18 months history data prior to flood)
  • Maps
    • Pan, zoom, scroll
  • Graph of flood data + reanalysis climate data
    • Clickable graph flood events
    • Reanalysis data lines in different colours
    • Tick box controls to determine which reanalysis data is on the map and which is sis
  • Data sets
    • Reanalysis data
      • "Globally & temporarily complete" (heat map style)
      • Based on model fitting historical constraints
      • Choose most relevant feilds
      • Rainfall
      • Temp
      • Soil moisture
      • Run-off
      •  ?
    • Flood observatory event data
      • Place name
      • Lat & lon
      • Severity
      • Lives lost
      • Date range
  • Download data
    • Limits determined by what's visible on screen (time frame, which 2 deg square, data sets chosen)
    • .csv


[edit] Stretch inclusions in version 1

  • Optimise the defaults
  • Map
  • In flood specific view - allow people to pan and click on nearby squares to see reanalysis data upstream whilst still maintaining temporal focus on the original flood.
    • As click on other squares, the reanalysis data changes to the new squares, whilst the flood event data is that from the original square plus the new square combined.
  • Clickable graph flood events
  • Rich content relating to flood
    • Wikipedia search (1/2 results)
    • Google search (2/3 results)
    •  ? Google News (can search specific period)
    •  ? Twitter/Facebook for live/recent events
  • Recent large floods on world map on vanilla landing page (last two years?)
  • Place-name search on page one takes you to relevant location 2 deg square view
    • Using geonames gazetteer
    • Fuzzy search

[edit] Future Incremental improvements

  • Shape files for recent floods
    • Using topography
    • Link to Google Earth
  • More data sets
    • Tide & coastal flooding
    • Wind data (pushing sea onland)
    • Terrain height
  • Contact real target users, and ask them what's useful
  • Semantically rich flood (or area) specific content
    • Using dbpedia, like Wikipedia blue plaques projects - but environmental stuff nearby (Tim Davies/Chris Gutteridge)
  • Live floods
    • Predicting end of floods happening
  • Location awareness zoom on landing page
  • High level flood list content pane on the landing page
  • Mobile version (necessary given large 2 deg granularity of data?)
    • (see crowdsourcing data)
  • More granular relevant data as heat map in background of maps
  • Personalisation
    • My favorite location?
    • My favorite floods???
    • Privacy issues
  • Offline version for carrying with you in places with
  • Cloud download of data to Google docs/dropbox

[edit] Major future ideas

[edit] Crowdsourcing real time data on floods

    • Web app to source data - "Flood alert button"
      • Identifying people's location with smartphone GPS or some other interesting stuff
    • Taking photos of events
    • Can be used to verify/train algoritms?
    • Integration with emergency response systems (e.g. Sahana? Ushahidi?)
    • Integration with Twitter

[edit] Larger/custom/catchment areas analysis & more granular reanalysis data

    • Integrate catchment basin area/flood source locations/river data set
    • Work with hydrographers to understand what we should be doing!
    • Move to a higher granularity more recent data set - leave behind the 2 deg square paradigm
    • Give reanalysis data averaged/totaled over an area rather than just for one 2 deg square
    • Custom area = the span of the map
    • Important because makes it much easier to look over a whole catchment area
    • Giorgi regions (climate zones)

[edit] Predictions of flood provided by system

    • Cutting edge global analysis of the data to provide real-time flood warnings
    • Integrate good hydrological/climatological research in the area, connect with the researchers
    • Web competition to deliver the best flood prediction algoritm? (Sponsored by Google.org?) A la innocentre or Kaggle
    • Q - Statistical validity of doing prediction based on model founded reanalysis data??
    • Integration with forecast data!
    • Go forward 100 years - how will increase in **climate change**
    • Predicting end of floods happening for live floods

An example plot of the rainfall rate and other relevant parameters in 2007 from Cambodia, taken from the re-analysis dataset. There was a flood from 10-24 August 2007. We can see that the period leading up to the flood being characterised by a dry spell with very low rainfall and soil moisture. On 6-7 August, the huge spike in rainfall caused saturation with a leveling of soil moisture, resulting in the flood two days later.

Image here: http://farm8.staticflickr.com/7168/6452694851_36c520c947.jpg

However the characteristics of the weather leading up to floods vary greatly, so a simple generalisation is not possible.

[edit] Alerts

    • Small SMS warning alerts for the public on the ground
    • Email alerts for more in depth actors (e.g. NGOs, media, government)

[edit] API

    • Stream live data - so researchers can create their own alerts based on their own prediction analysis
    • Access historic data sets in one place a la particular parameters
Retrieved from "http://wiki.rhok.org/RHoK_4.0_-_Oxford/Projects/Tropical_Flood_Prediction"
Personal tools