tag:blogger.com,1999:blog-24183990869318735332024-03-13T10:13:11.265-07:00fishvicea space that will most likely focus on fisheries data and exploration using Reinar hjörleifssonhttp://www.blogger.com/profile/03811120168583854238noreply@blogger.comBlogger8125tag:blogger.com,1999:blog-2418399086931873533.post-60417123163419308502012-07-01T10:10:00.001-07:002014-07-29T05:01:57.655-07:00FAO statistical areas in Google Earth<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-41224938-1', 'auto');
ga('send', 'pageview');
</script>
Some <a href="http://fishvice.blogspot.com/2012/05/ices-statistical-areas-viewed-in-google.html" target="_blank">time</a> ago I did a blog describing how to get <a href="http://www.ices.dk/" target="_blank">ICES</a> and <a href="http://www.nafo.int/" target="_blank">NAFO</a> statistical areas, originally as shapefiles into Google Earth readable format (<a href="http://dl.dropbox.com/u/40576923/ices_areas.kml" target="_blank">ICES</a>, <a href="http://dl.dropbox.com/u/40576923/nafo_areas.kml" target="_blank">NAFO</a>). These areas are the primary fisheries areas upon which nominal fisheries catch statistics have been reported in the past (since 1905 when it comes to ICES).<br />
A colleague of mine asked me if I was aware of an analogous format for the global <a href="http://www.fao.org/fishery/area/search/en" target="_blank">FAO</a> fisheries statistical areas. After some search I found indeed that FAO provides these statistical areas in <a href="http://www.fao.org/geonetwork/srv/en/google.kml?uuid=ac02a460-da52-11dc-9d70-0017f293bd28&layers=subdivisions_erase" target="_blank">a Google Earth readable format</a>. But a little cumbersome for my taste. Luckily, the <a href="http://www.fao.org/geonetwork/srv/en/resources.get?id=31627&fname=fa_.zip&access=private" target="_blank">shapefiles</a> are easily available at the FAO site. So here is my take on things:<br />
<br />
<div style="overflow: auto;">
<div class="geshifilter">
<pre class="r geshifilter-R" style="font-family: monospace;"><span style="color: #666666; font-style: italic;">################################################################################</span>
<span style="color: #666666; font-style: italic;"># FAO statistical areas in Google Earth format</span>
<a href="http://inside-r.org/r-doc/base/require"><span style="color: #003399; font-weight: bold;">require</span></a><span style="color: #009900;">(</span><a href="http://inside-r.org/packages/cran/rgdal">rgdal</a><span style="color: #009900;">)</span>
<a href="http://inside-r.org/r-doc/base/require"><span style="color: #003399; font-weight: bold;">require</span></a><span style="color: #009900;">(</span>plotKML<span style="color: #009900;">)</span>
<a href="http://inside-r.org/r-doc/base/setwd"><span style="color: #003399; font-weight: bold;">setwd</span></a><span style="color: #009900;">(</span><span style="color: blue;">"~/Dropbox/blogspotScripts/20120701FAOareas/"</span><span style="color: #009900;">)</span> <span style="color: #666666; font-style: italic;"># your 'native' directory</span>
<a href="http://inside-r.org/r-doc/base/url"><span style="color: #003399; font-weight: bold;">url</span></a> <- <span style="color: blue;">'http://www.fao.org/geonetwork/srv/en/resources.get?id=31627&fname=fa_.zip&access=private'</span>
<a href="http://inside-r.org/r-doc/utils/download.file"><span style="color: #003399; font-weight: bold;">download.file</span></a><span style="color: #009900;">(</span><a href="http://inside-r.org/r-doc/base/url"><span style="color: #003399; font-weight: bold;">url</span></a><span style="color: #339933;">,</span>destfile=<span style="color: blue;">"~/Downloads/tmp.zip"</span><span style="color: #009900;">)</span> <span style="color: #666666; font-style: italic;"># your 'native' directory</span>
<a href="http://inside-r.org/r-doc/utils/unzip"><span style="color: #003399; font-weight: bold;">unzip</span></a><span style="color: #009900;">(</span><span style="color: blue;">"~/Downloads/tmp.zip"</span><span style="color: #009900;">)</span> <span style="color: #666666; font-style: italic;"># your 'native' directory</span>
fao <- readOGR<span style="color: #009900;">(</span><span style="color: blue;">"."</span><span style="color: #339933;">,</span><span style="color: blue;">"fa_"</span><span style="color: #009900;">)</span>
<a href="http://inside-r.org/r-doc/base/setwd"><span style="color: #003399; font-weight: bold;">setwd</span></a><span style="color: #009900;">(</span><span style="color: blue;">"~/Dropbox/Public/"</span><span style="color: #009900;">)</span> <span style="color: #666666; font-style: italic;"># your 'native' directory</span>
<a href="http://inside-r.org/r-doc/graphics/plot"><span style="color: #003399; font-weight: bold;">plot</span></a><span style="color: #009900;">(</span>fao<span style="color: #009900;">)</span> <span style="color: #666666; font-style: italic;"># a peek preview in R</span>
<span style="color: #666666; font-style: italic;"># here one could simply do, via function from the plotKML package</span>
<a href="http://inside-r.org/packages/cran/kml">kml</a><span style="color: #009900;">(</span>fao<span style="color: #339933;">,</span><a href="http://inside-r.org/r-doc/base/file"><span style="color: #003399; font-weight: bold;">file</span></a>=<span style="color: blue;">"tmpFAO.kml"</span><span style="color: #009900;">)</span> <span style="color: #666666; font-style: italic;"># open your tmp.kml in GE to have a view</span>
<span style="color: #666666; font-style: italic;"># but that stuff is no good</span>
<span style="color: #666666; font-style: italic;"># to put some order to things, etc.</span>
<a href="http://inside-r.org/r-doc/base/names"><span style="color: #003399; font-weight: bold;">names</span></a><span style="color: #009900;">(</span>fao<span style="color: #009900;">)</span> <- <a href="http://inside-r.org/r-doc/base/tolower"><span style="color: #003399; font-weight: bold;">tolower</span></a><span style="color: #009900;">(</span><a href="http://inside-r.org/r-doc/base/names"><span style="color: #003399; font-weight: bold;">names</span></a><span style="color: #009900;">(</span>fao<span style="color: #009900;">)</span><span style="color: #009900;">)</span>
dat <- <a href="http://inside-r.org/r-doc/base/as.data.frame"><span style="color: #003399; font-weight: bold;">as.data.frame</span></a><span style="color: #009900;">(</span>fao<span style="color: #009900;">)</span>
i <- <a href="http://inside-r.org/r-doc/base/data.frame"><span style="color: #003399; font-weight: bold;">data.frame</span></a><span style="color: #009900;">(</span>str_locate<span style="color: #009900;">(</span>fao$f_subarea<span style="color: #339933;">,</span><span style="color: blue;">"_"</span><span style="color: #009900;">)</span><span style="color: #009900;">)</span>
i <- !<a href="http://inside-r.org/r-doc/base/is.na"><span style="color: #003399; font-weight: bold;">is.na</span></a><span style="color: #009900;">(</span>i<span style="color: #009900;">[</span><span style="color: #339933;">,</span><span style="color: #cc66cc;">1</span><span style="color: #009900;">]</span><span style="color: #009900;">)</span>
fao$f_subarea<span style="color: #009900;">[</span>i<span style="color: #009900;">]</span> <- str_sub<span style="color: #009900;">(</span>fao$f_subarea<span style="color: #009900;">[</span>i<span style="color: #009900;">]</span><span style="color: #339933;">,</span><span style="color: #cc66cc;">2</span><span style="color: #009900;">)</span>
fao$f_division<span style="color: #009900;">[</span>i<span style="color: #009900;">]</span> <- str_sub<span style="color: #009900;">(</span>fao$f_division<span style="color: #009900;">[</span>i<span style="color: #009900;">]</span><span style="color: #339933;">,</span><span style="color: #cc66cc;">2</span><span style="color: #009900;">)</span>
fao$f_subdivis<span style="color: #009900;">[</span>i<span style="color: #009900;">]</span> <- str_sub<span style="color: #009900;">(</span>fao$f_subdivis<span style="color: #009900;">[</span>i<span style="color: #009900;">]</span><span style="color: #339933;">,</span><span style="color: #cc66cc;">2</span><span style="color: #009900;">)</span>
fao$name <- <a href="http://inside-r.org/r-doc/base/ifelse"><span style="color: #003399; font-weight: bold;">ifelse</span></a><span style="color: #009900;">(</span>!<a href="http://inside-r.org/r-doc/base/is.na"><span style="color: #003399; font-weight: bold;">is.na</span></a><span style="color: #009900;">(</span>fao$f_subunit<span style="color: #009900;">)</span><span style="color: #339933;">,</span>fao$f_subunit<span style="color: #339933;">,</span>
<a href="http://inside-r.org/r-doc/base/ifelse"><span style="color: #003399; font-weight: bold;">ifelse</span></a><span style="color: #009900;">(</span>!<a href="http://inside-r.org/r-doc/base/is.na"><span style="color: #003399; font-weight: bold;">is.na</span></a><span style="color: #009900;">(</span>fao$f_subdivis<span style="color: #009900;">)</span><span style="color: #339933;">,</span>fao$f_subdivis<span style="color: #339933;">,</span>
<a href="http://inside-r.org/r-doc/base/ifelse"><span style="color: #003399; font-weight: bold;">ifelse</span></a><span style="color: #009900;">(</span>!<a href="http://inside-r.org/r-doc/base/is.na"><span style="color: #003399; font-weight: bold;">is.na</span></a><span style="color: #009900;">(</span>fao$f_division<span style="color: #009900;">)</span><span style="color: #339933;">,</span>fao$f_division<span style="color: #339933;">,</span>
<a href="http://inside-r.org/r-doc/base/ifelse"><span style="color: #003399; font-weight: bold;">ifelse</span></a><span style="color: #009900;">(</span>!<a href="http://inside-r.org/r-doc/base/is.na"><span style="color: #003399; font-weight: bold;">is.na</span></a><span style="color: #009900;">(</span>fao$f_subarea<span style="color: #009900;">)</span><span style="color: #339933;">,</span>fao$f_subarea<span style="color: #339933;">,</span>
fao$f_area<span style="color: #009900;">)</span><span style="color: #009900;">)</span><span style="color: #009900;">)</span><span style="color: #009900;">)</span>
fao <- <a href="http://inside-r.org/r-doc/methods/as"><span style="color: #003399; font-weight: bold;">as</span></a><span style="color: #009900;">(</span>fao<span style="color: #339933;">,</span><span style="color: blue;">"SpatialLinesDataFrame"</span><span style="color: #009900;">)</span>
fao$id <- <a href="http://inside-r.org/r-doc/base/as.numeric"><span style="color: #003399; font-weight: bold;">as.numeric</span></a><span style="color: #009900;">(</span>fao$f_area<span style="color: #009900;">)</span>
<a href="http://inside-r.org/r-doc/MASS/area"><span style="color: #003399; font-weight: bold;">area</span></a> <- <a href="http://inside-r.org/r-doc/base/sort"><span style="color: #003399; font-weight: bold;">sort</span></a><span style="color: #009900;">(</span><a href="http://inside-r.org/r-doc/base/unique"><span style="color: #003399; font-weight: bold;">unique</span></a><span style="color: #009900;">(</span>fao$f_area<span style="color: #009900;">)</span><span style="color: #009900;">)</span> <span style="color: #666666; font-style: italic;"># put the stuff in order of the major areas</span>
<span style="color: #666666; font-style: italic;"># again some function of the plotKML packages put in use</span>
kml_open<span style="color: #009900;">(</span><span style="color: blue;">"fao.kml"</span><span style="color: #009900;">)</span>
<span style="color: #666666; font-style: italic;"># filling up the russian doll</span>
<span style="color: black; font-weight: bold;">for</span> <span style="color: #009900;">(</span>i <span style="color: black; font-weight: bold;">in</span> <span style="color: #cc66cc;">1</span>:<a href="http://inside-r.org/r-doc/base/length"><span style="color: #003399; font-weight: bold;">length</span></a><span style="color: #009900;">(</span><a href="http://inside-r.org/r-doc/MASS/area"><span style="color: #003399; font-weight: bold;">area</span></a><span style="color: #009900;">)</span><span style="color: #009900;">)</span> <span style="color: #009900;">{</span>
tmp <- fao<span style="color: #009900;">[</span>fao$f_area %in% <a href="http://inside-r.org/r-doc/MASS/area"><span style="color: #003399; font-weight: bold;">area</span></a><span style="color: #009900;">[</span>i<span style="color: #009900;">]</span><span style="color: #339933;">,</span><span style="color: #009900;">]</span>
kml_layer.SpatialLines<span style="color: #009900;">(</span>tmp<span style="color: #339933;">,</span><a href="http://inside-r.org/r-doc/base/labels"><span style="color: #003399; font-weight: bold;">labels</span></a>=tmp$name<span style="color: #339933;">,</span>folder.name = <a href="http://inside-r.org/r-doc/MASS/area"><span style="color: #003399; font-weight: bold;">area</span></a><span style="color: #009900;">[</span>i<span style="color: #009900;">]</span><span style="color: #009900;">)</span>
<span style="color: #009900;">}</span>
kml_close<span style="color: #009900;">(</span><span style="color: blue;">"fao.kml"</span><span style="color: #009900;">)</span>
kml_compress<span style="color: #009900;">(</span><span style="color: blue;">"fao.kml"</span><span style="color: #009900;">)</span> <span style="color: #666666; font-style: italic;"># create a compressed (KMZ) file</span></pre>
</div>
</div>
<a href="http://www.inside-r.org/pretty-r" title="Created by Pretty R at inside-R.org">Created by Pretty R at inside-R.org</a><br />
<br />
Did not quite get want I really wanted (plotKML is really in its infancy, just released to cran last week. But a very promising package). So played a little around (non reproducible code) to end up, preliminary with <a href="https://dl.dropbox.com/u/40576923/FAO.kmz" target="_blank">this</a>, the following being a snapshot:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjrKoHdclQCIxC_vSx4b2jwoIwYSiDXWLjCn9OoxwGSZOfScbVNKf7tJ3lesgDv0_gruv1522l-VXW-Fzz-_jHRz_bfEV142DCP_GkbS1cHqJrLZzC3wdKdY7x_FZvWF_arWx3gLRixd4LQ/s1600/FAOinGE" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjrKoHdclQCIxC_vSx4b2jwoIwYSiDXWLjCn9OoxwGSZOfScbVNKf7tJ3lesgDv0_gruv1522l-VXW-Fzz-_jHRz_bfEV142DCP_GkbS1cHqJrLZzC3wdKdY7x_FZvWF_arWx3gLRixd4LQ/s320/FAOinGE" height="198" width="320" /></a></div>
<br />einar hjörleifssonhttp://www.blogger.com/profile/03811120168583854238noreply@blogger.com0tag:blogger.com,1999:blog-2418399086931873533.post-4311555742026973582012-05-29T16:45:00.000-07:002014-07-29T05:00:55.205-07:00Google Earth and ocean depth contours<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-41224938-1', 'auto');
ga('send', 'pageview');
</script>
Been playing around in R for a while and then a bit in the GIS environment - within R of course. Do not know very much about GIS, but know what I want: Looking at GIS fisheries data across various scales (macro to global). Now we have myriads of websites that provide a view, on a 'data' on a scale. But herein lies the limitation, it is piecemeal information. Now what GIS software medium actually operates on all scales? Who else to call than Google Earth?<br />
Here is an snapshot of the depth contours for from 0 to 1500 m by 100 meter intervals for the whole of the North Atlantic. Original data is from <a href="http://www.gebco.net/" target="_blank">General Bathymetric Chart of the Oceans</a>. All that was needed was some lines of code in R to convert point values to contour and then generate the kml-file. And adding on top of that the <a href="http://geo.ices.dk/" target="_blank">ICES statistical rectangles</a> as a layer, coloured in red.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi1p6Sqi8Sqslt7GmxEZoRuSEhOrxqqcha_xLnxy2u3kjMRh-kexBML7RdFa7_LORN1cyevsLh1OrgJAKrDOYafK-vtrrXzwto6X1CCeZcY0lwFojRXTMsYmCtxiKA-dFxKQfO-m3IWvPAH/s1600/preview.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="198" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi1p6Sqi8Sqslt7GmxEZoRuSEhOrxqqcha_xLnxy2u3kjMRh-kexBML7RdFa7_LORN1cyevsLh1OrgJAKrDOYafK-vtrrXzwto6X1CCeZcY0lwFojRXTMsYmCtxiKA-dFxKQfO-m3IWvPAH/s320/preview.png" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Zooming in or out in Google Earth environment is a breeze. An example on the macroscale, showing the Porupine bank is shown below.</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhwACdBqWhDNsbzZyh02HAlDWCwCW-LcboSVW1yrwzaCIwQklEuqyjwPXHULnr-RDbZ74cIsZ_HeST-myfyd5v65-JTMhsu7HDLSpBRQPw4qi424GQ6dWkNpdfIkA56MCTPRIUWicpRGSXj/s1600/porcupineBank.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="198" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhwACdBqWhDNsbzZyh02HAlDWCwCW-LcboSVW1yrwzaCIwQklEuqyjwPXHULnr-RDbZ74cIsZ_HeST-myfyd5v65-JTMhsu7HDLSpBRQPw4qi424GQ6dWkNpdfIkA56MCTPRIUWicpRGSXj/s320/porcupineBank.png" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
Wandering if one should not use the Google Earth as the base platform to publish all fisheries and management information. Both the global as well as the local.<br />
<br />einar hjörleifssonhttp://www.blogger.com/profile/03811120168583854238noreply@blogger.com4tag:blogger.com,1999:blog-2418399086931873533.post-54445720141849045232012-05-28T07:36:00.001-07:002014-07-29T05:01:06.403-07:00ICES and NAFO statistical areas viewed in Google Earth<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-41224938-1', 'auto');
ga('send', 'pageview');
</script>
On the <a href="http://geo.ices.dk/" target="_blank">ICES Spatial Facility webpage</a> one can obtain a view of the <a href="http://geo.ices.dk/viewer.php?add_layers=ices_ref:ices_areas" target="_blank">ICES statistical areas</a>. Now since the projection used is more than a bit misleading I was wandering how these statistical areas would look like in Google Earth. Thankfully the shapefiles associated with these rectangles are available to <a href="http://geo.ices.dk/download.php?dataset=ices_ref:ices_areas" target="_blank">download</a> from the web page. What follows is less than a bare bone code to convert the shape files into kml files. Except that a bit of a thinning of the spatial codes was needed because the original shapefiles are very detailed for some of the coastal areas resulting in very nonsmoothed zooming etc. in Google Earth. While doing this I also included the <a href="http://www.nafo.int/" target="_blank">NAFO areas</a>. Since I also wanted to see the EEZ on top of this I include a code to convert EEZ shapefiles to kml. The EEZ data were <a href="http://www.vliz.be/vmdcdata/marbound/download_file.php?fn=v6.1_20110512_LR" target="_blank">downloaded</a> from the <a href="http://www.vliz.be/vmdcdata/marbound/index.php" target="_blank">VLIZ Maritime Boundaries Geodatabase</a>.<br />
<br />
<div>
Before running the code below, one needs to download the necessary data onto the local machine and extract the data from the zipped file into a working directory (not needed for NAFO areas - is included in the code below). Once this has been done the following code was used to convert the data into Google Earth format:<br />
<br /></div>
<div>
<div style="overflow: auto;">
<div class="geshifilter">
<pre class="r geshifilter-R" style="font-family: monospace;"><a href="http://inside-r.org/r-doc/base/setwd"><span style="color: #003399; font-weight: bold;">setwd</span></a><span style="color: #009900;">(</span><span style="color: blue;">"yourWorkingDirectory"</span><span style="color: #009900;">)</span>
<a href="http://inside-r.org/r-doc/base/library"><span style="color: #003399; font-weight: bold;">library</span></a><span style="color: #009900;">(</span><a href="http://inside-r.org/packages/cran/maptools">maptools</a><span style="color: #009900;">)</span>
<span style="color: #666666; font-style: italic;"># gpclibPermit() # may be needed</span>
<a href="http://inside-r.org/r-doc/base/library"><span style="color: #003399; font-weight: bold;">library</span></a><span style="color: #009900;">(</span><a href="http://inside-r.org/packages/cran/rgdal">rgdal</a><span style="color: #009900;">)</span>
<span style="color: #666666; font-style: italic;"># ices areas</span>
ices_areas = readOGR<span style="color: #009900;">(</span><span style="color: blue;">'ices_areas.shp'</span><span style="color: #339933;">,</span><span style="color: blue;">'ices_areas'</span><span style="color: #009900;">)</span>
ices_areas <- thinnedSpatialPoly<span style="color: #009900;">(</span>ices_areas<span style="color: #339933;">,</span><a href="http://inside-r.org/packages/cran/tolerance">tolerance</a>=<span style="color: #cc66cc;">0.04</span><span style="color: #009900;">)</span> <span style="color: #666666; font-style: italic;"># takes some time</span>
writeOGR<span style="color: #009900;">(</span>ices_areas<span style="color: #339933;">,</span><span style="color: blue;">'ices_areas.kml'</span><span style="color: #339933;">,</span><span style="color: blue;">'ices_areas'</span><span style="color: #339933;">,</span>driver=<span style="color: blue;">'KML'</span><span style="color: #009900;">)</span>
<span style="color: #666666; font-style: italic;"># world eez</span>
eez <- readOGR<span style="color: #009900;">(</span><span style="color: blue;">'World_EEZ_v6_1_simpliefiedcoastlines_20110512.shp'</span><span style="color: #339933;">,</span>
<span style="color: blue;">'World_EEZ_v6_1_simpliefiedcoastlines_20110512'</span><span style="color: #009900;">)</span>
writeOGR<span style="color: #009900;">(</span>eez<span style="color: #339933;">,</span> <span style="color: blue;">"eez.kml"</span><span style="color: #339933;">,</span> <span style="color: blue;">"eez"</span><span style="color: #339933;">,</span> driver=<span style="color: blue;">"KML"</span><span style="color: #009900;">)</span>
<span style="color: #666666; font-style: italic;"># nafo areas</span>
<a href="http://inside-r.org/r-doc/utils/download.file"><span style="color: #003399; font-weight: bold;">download.file</span></a><span style="color: #009900;">(</span><span style="color: blue;">'http://www.nafo.int/about/overview/gis/Divisions.zip'</span><span style="color: #339933;">,</span>
destfile=<span style="color: blue;">'NAFOdivisions.zip'</span><span style="color: #009900;">)</span>
<a href="http://inside-r.org/r-doc/utils/unzip"><span style="color: #003399; font-weight: bold;">unzip</span></a><span style="color: #009900;">(</span><span style="color: blue;">'NAFOdivisions'</span><span style="color: #009900;">)</span>
nafo_areas <- readOGR<span style="color: #009900;">(</span><span style="color: blue;">'</span><span style="color: blue;">NAFOdivisions.shp</span><span style="color: blue;">'</span><span style="color: #339933;">,</span><span style="color: blue;">'</span><span style="color: blue;">NAFOdivisions</span><span style="color: blue;">'</span><span style="color: #009900;">)</span>
writeOGR<span style="color: #009900;">(</span>nafo_areas<span style="color: #339933;">,</span><span style="color: blue;">'nafo_areas.kml'</span><span style="color: #339933;">,</span><span style="color: blue;">'nafo_areas'</span><span style="color: #339933;">,</span>driver=<span style="color: blue;">'KML'</span><span style="color: #009900;">)</span></pre>
</div>
</div>
<a href="http://www.inside-r.org/pretty-r" title="Created by Pretty R at inside-R.org">Created by Pretty R at inside-R.org</a><br />
<br />
Now one only needs to open the kml files (ices_areas.kml, nafo_area.kml and eez.kml, located in the working directory) in Google Earth. If there are problems with downloading or running the scripts I provide the <a href="http://dl.dropbox.com/u/40576923/ices_areas.kml">ices_areas.kml</a>, <a href="http://dl.dropbox.com/u/40576923/nafo_areas.kml" target="_blank">nafo_areas.kml</a> and <a href="http://dl.dropbox.com/u/40576923/eez.kml">eez.kml</a> via my Public Dropbox space (there is also a depth contour in the E-Greenland, Iceland Faero Is. region, see <a href="http://dl.dropbox.com/u/40576923/nwwgDepth.kml" target="_blank">nwwgDepth.kml</a>)<br />
As said, this is very much a bare bone attempt but at least describes the essence of getting a GE view of the ICES and NAFO statistical areas. Zooming in and out is a breeze and by clicking on an area some statistics show up. A snapshot of the broad view as presented in the projection on the ices page vs. the one provided in Google Earth are shown for comparisons. The global EEZ are a bonus :-)<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh-RFY6u9z51iWJTzyQIuB1KTlae_tK7819kMPdG5BldelLxYmvJtrqjhHygf7c976ySZng8tTugBJ5ZsHtffDY1tsAapHNOQE0iC8_wQw_qQgjTwokaj1kc09lpIU9OgMKyrZbOkTAxqwK/s1600/ices_areas2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="232" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh-RFY6u9z51iWJTzyQIuB1KTlae_tK7819kMPdG5BldelLxYmvJtrqjhHygf7c976ySZng8tTugBJ5ZsHtffDY1tsAapHNOQE0iC8_wQw_qQgjTwokaj1kc09lpIU9OgMKyrZbOkTAxqwK/s320/ices_areas2.png" width="320" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgT31pTuLrEwVRnZdyH-aZNd5oUMjCvuP-1oxv-pMiEFD1QhLTkDgMoKo8oiDCk1eXDf72Z_M4frD_DydgYzAEUuIVscioebHu9xBr74E2h38Egel4NzOg09zxY9TrRXlFWPrmUNyzDkKC8/s1600/ices_nafo_areas.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="198" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgT31pTuLrEwVRnZdyH-aZNd5oUMjCvuP-1oxv-pMiEFD1QhLTkDgMoKo8oiDCk1eXDf72Z_M4frD_DydgYzAEUuIVscioebHu9xBr74E2h38Egel4NzOg09zxY9TrRXlFWPrmUNyzDkKC8/s320/ices_nafo_areas.png" width="320" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br /></div>einar hjörleifssonhttp://www.blogger.com/profile/03811120168583854238noreply@blogger.com0tag:blogger.com,1999:blog-2418399086931873533.post-21002162960265700112012-04-13T14:15:00.003-07:002012-04-13T14:15:43.800-07:00ICES maturity: datamining, MS Access, R, plyr, reshape2 and ggplot2ok, the title is a bit bloated<br />
<br />
but a very short code:<br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">download.file("http://ices.dk/datacentre/StdGraphDB/FishStockDB.mdb",</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> "FishStockDB.mdb")</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">require(Hmisc) # need also mdbtools, in Fedora do >yum install mdbtools</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">FSDB <- mdb.get("FishStockDB.mdb")</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">require(plyr)</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">dat <- ddply(FSDB$Yieldrecruit[,c("FishStock","Age","Mat","F")],</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> c("FishStock"),transform,sel=F/max(F))</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">require(reshape2)</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">dat <- melt(dat[,c("FishStock","Age","Mat","sel")],id.vars=c("FishStock","Age"))</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">dat$variable <- ifelse(dat$variable %in% "Mat","Maturity","Selection")</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">require(ggplot2)</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">ggplot(dat,aes(Age,value,colour=variable)) + </span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> geom_line() + </span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> facet_wrap(~ FishStock) +</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> xlim(0,10)</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">ggsave("ICESmaturity.png")</span><br />
is behind this picture.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEibODZHwITh7Orr5t94aPii0X2aTfqyPmAf43ao5w73D5pxiR-Kyny4Se9hyphenhyphendM3yas9-wDQFADdP7Tw5VTz-IvrWBKORnm7Gb87M_29g1EmnWzlDfU8ME9pfZkQd_JKT555D4I2g8hxoz4x/s1600/ICESmaturity.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="271" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEibODZHwITh7Orr5t94aPii0X2aTfqyPmAf43ao5w73D5pxiR-Kyny4Se9hyphenhyphendM3yas9-wDQFADdP7Tw5VTz-IvrWBKORnm7Gb87M_29g1EmnWzlDfU8ME9pfZkQd_JKT555D4I2g8hxoz4x/s320/ICESmaturity.png" width="320" /></a></div>
<br />
and it demonstrates how to:<br />
download data<br />
access MS Access mdb files via R<br />
use plyr<br />
use reshape2<br />
use ggplot2<br />
<br />
the resulting graph poses on the other hand, myriads of fisheries related questions. be they biological and/or management. like the question if there is a pattern between the sync/out-of-sync in the fisheries selection pattern and the biological pattern (here maturity) and the state of the stock/fisheries? leave that for somebody else to determine.<br />
<br />einar hjörleifssonhttp://www.blogger.com/profile/03811120168583854238noreply@blogger.com0tag:blogger.com,1999:blog-2418399086931873533.post-34403348225250310182012-04-03T11:07:00.001-07:002012-04-05T04:32:17.766-07:00CLIWOC (British, Spanish and Dutch shipping 1750-1855): Getting the data into Ron the <a href="http://spatialanalysis.co.uk/" target="_blank">Spatial analysis blog</a> a nice visualisation of the major shipping route of the British, Dutch and Spanish fleet in 1750-1850 was <a href="http://spatialanalysis.co.uk/2012/03/mapped-british-shipping-1750-1800/" target="_blank">presented</a> recently. based on the Climatological Databases for the World's Oceans (<span style="background-color: white;"><a href="http://www.ucm.es/info/cliwoc/" target="_blank"><span class="J-JK9eJ-PJVNOc" style="background-attachment: scroll; background-clip: initial; background-image: none; background-origin: initial; background-position: 0% 0%; background-repeat: repeat repeat;">CLIWOC</span></a>)</span>. another even nicer visualisation of the same data was presented by <a href="http://unconsenting.wordpress.com/" target="_blank"><span class="J-JK9eJ-PJVNOc" style="background-attachment: scroll; background-clip: initial; background-color: white; background-image: none; background-origin: initial; background-position: 0% 0%; background-repeat: repeat repeat;">unconsenting</span></a>.
in neither case any code was provided, in particular with regards to
just getting the data into a readable format. so i did some digging into
the the <a href="http://www.ucm.es/info/cliwoc/" target="_blank"><span class="J-JK9eJ-PJVNOc" style="background-attachment: scroll; background-clip: initial; background-color: white; background-image: none; background-origin: initial; background-position: 0% 0%; background-repeat: repeat repeat;">CLIWOC</span></a> home-page, with just that particularity in mind.<br />
being
a Linux Fedora user i started limiting my, albeit quick trial, with
what was supposed to be a <span class="J-JK9eJ-PJVNOc" style="background-attachment: scroll; background-clip: initial; background-color: white; background-image: none; background-origin: initial; background-position: 0% 0%; background-repeat: repeat repeat;">linux</span>/<span class="J-JK9eJ-PJVNOc" style="background-attachment: scroll; background-clip: initial; background-color: white; background-image: none; background-origin: initial; background-position: 0% 0%; background-repeat: repeat repeat;">unix</span> readable format on the <a href="http://www.ucm.es/info/cliwoc/cliwoc15.htm" target="_blank"><span class="J-JK9eJ-PJVNOc" style="background-attachment: scroll; background-clip: initial; background-color: white; background-image: none; background-origin: initial; background-position: 0% 0%; background-repeat: repeat repeat;">CLWVOC</span> main database release page</a>. this being <a href="http://www.knmi.nl/cliwoc/download/CLIWOC15.Z" target="_blank">CLIWOC15.Z</a>
file. that trial was not very successful. managed to read in the data
into R after getting rid of some "unwanted" characters that R did not
"like". however, the fields (columns supposed to be semicolon
delimited) was more or less messed up. and the number of records were
also less than specified on the page. most importantly the trial was
based on non-reproducible code within the R-environment.<br />
so, reluctantly i decided to have a go at the MS Access databases<a href="http://www.knmi.nl/cliwoc/download/CLIWOC15_2000.zip" target="_blank"> CLIWOC15_2000.zip</a> source, albeit not having high hopes that i could find a solution that would work withing the Linux environment. but after some search on the web on how to read mdb format directly into R environment within Linux i stumbled across this <a href="http://r.789695.n4.nabble.com/problems-accessing-MS-Access-2003-database-with-RODBC-td2065467.html" target="_blank">post</a>. In particular: "Use <span class="bold highlight search-highlight">mdb</span>.get() from <span class="bold highlight search-highlight">Hmisc</span> package to import entire tables from the
database into dataframes." just what the doctor ordered. now i had the Hmisc library already installed. but I did not have the success with the mdb.get() function. reading the help file on mdb.get (?mdb.get) one "gets":<span style="font-size: small;"><span style="font-family: inherit;"> </span><span class="Apple-style-span" style="border-collapse: separate; color: black; font-family: inherit; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px;"><span class="Apple-style-span">"Uses the<span class="Apple-converted-space"> </span><code>mdbtools</code><span class="Apple-converted-space"> </span>package executables<span class="Apple-converted-space"> </span><code>mdb-tables</code>,<span class="Apple-converted-space"> </span><code>mdb-schema</code>, and<span class="Apple-converted-space"> </span><code>mdb-export</code>. in Debian/Ubuntu Linux run<span class="Apple-converted-space"> </span><code>apt get install mdbtools</code>." being a Fedora user the equivalent command to install the mdbtools is:</span></span></span><br />
<span style="font-size: small;"><span class="Apple-style-span" style="border-collapse: separate; color: black; font-family: inherit; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px;"><span class="Apple-style-span">yum install mdbtools</span></span></span><br />
<span style="font-size: small;"><span class="Apple-style-span" style="border-collapse: separate; color: black; font-family: inherit; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px;"><span class="Apple-style-span">with that I was ready to go. with the following code i managed to achieve my objective of getting the CLIWOC MS Access data into R environment within the Linux framework (as well as do some very crude initial ggplot2):</span></span></span><br />
<br />
<div style="overflow: auto;">
<div class="geshifilter">
<pre class="r geshifilter-R" style="font-family: monospace;"><a href="http://inside-r.org/r-doc/base/require"><span style="color: #003399; font-weight: bold;">require</span></a><span style="color: #009900;">(</span><a href="http://inside-r.org/packages/cran/Hmisc">Hmisc</a><span style="color: #009900;">)</span> <span style="color: #666666; font-style: italic;"># need also mdbtools, in Fedora do >yum install mdbtools</span>
path <- <span style="color: blue;">"yourworkingdirectory"</span>
URL <- <span style="color: blue;">"http://www.knmi.nl/"</span>
PATH <- <span style="color: blue;">"cliwoc/download/"</span>
FILE <- <span style="color: blue;">"CLIWOC15_2000.zip"</span>
<a href="http://inside-r.org/r-doc/utils/download.file"><span style="color: #003399; font-weight: bold;">download.file</span></a><span style="color: #009900;">(</span><a href="http://inside-r.org/r-doc/base/paste"><span style="color: #003399; font-weight: bold;">paste</span></a><span style="color: #009900;">(</span>URL<span style="color: #339933;">,</span>PATH<span style="color: #339933;">,</span>FILE<span style="color: #339933;">,</span>sep=<span style="color: blue;">""</span><span style="color: #009900;">)</span><span style="color: #339933;">,</span>
<a href="http://inside-r.org/r-doc/base/paste"><span style="color: #003399; font-weight: bold;">paste</span></a><span style="color: #009900;">(</span>path<span style="color: #339933;">,</span><span style="color: blue;">"CLIWOC15_2000.zip"</span><span style="color: #339933;">,</span>sep=<span style="color: blue;">""</span><span style="color: #009900;">)</span><span style="color: #009900;">)</span>
<a href="http://inside-r.org/r-doc/base/dir"><span style="color: #003399; font-weight: bold;">dir</span></a> <- <a href="http://inside-r.org/r-doc/utils/unzip"><span style="color: #003399; font-weight: bold;">unzip</span></a><span style="color: #009900;">(</span><a href="http://inside-r.org/r-doc/base/paste"><span style="color: #003399; font-weight: bold;">paste</span></a><span style="color: #009900;">(</span>path<span style="color: #339933;">,</span><span style="color: blue;">"CLIWOC15_2000.zip"</span><span style="color: #339933;">,</span>sep=<span style="color: blue;">""</span><span style="color: #009900;">)</span><span style="color: #009900;">)</span>
<a href="http://inside-r.org/r-doc/base/file"><span style="color: #003399; font-weight: bold;">file</span></a> <- <a href="http://inside-r.org/r-doc/base/substr"><span style="color: #003399; font-weight: bold;">substr</span></a><span style="color: #009900;">(</span><a href="http://inside-r.org/r-doc/base/dir"><span style="color: #003399; font-weight: bold;">dir</span></a><span style="color: #339933;">,</span><span style="color: #cc66cc;">3</span><span style="color: #339933;">,</span><a href="http://inside-r.org/r-doc/base/nchar"><span style="color: #003399; font-weight: bold;">nchar</span></a><span style="color: #009900;">(</span><a href="http://inside-r.org/r-doc/base/dir"><span style="color: #003399; font-weight: bold;">dir</span></a><span style="color: #009900;">)</span><span style="color: #009900;">)</span>
dat <- mdb.get<span style="color: #009900;">(</span><a href="http://inside-r.org/r-doc/base/file"><span style="color: #003399; font-weight: bold;">file</span></a><span style="color: #009900;">)</span>
tmp <- dat$CLIWOC15<span style="color: #009900;">[</span><span style="color: #339933;">,</span><a href="http://inside-r.org/r-doc/base/c"><span style="color: #003399; font-weight: bold;">c</span></a><span style="color: #009900;">(</span><span style="color: blue;">"Lon3"</span><span style="color: #339933;">,</span><span style="color: blue;">"Lat3"</span><span style="color: #009900;">)</span><span style="color: #009900;">]</span>
<a href="http://inside-r.org/r-doc/base/require"><span style="color: #003399; font-weight: bold;">require</span></a><span style="color: #009900;">(</span><a href="http://inside-r.org/packages/cran/ggplot2">ggplot2</a><span style="color: #009900;">)</span>
<a href="http://inside-r.org/packages/cran/ggplot">ggplot</a><span style="color: #009900;">(</span>tmp<span style="color: #339933;">,</span>aes<span style="color: #009900;">(</span>Lon3<span style="color: #339933;">,</span>Lat3<span style="color: #009900;">)</span><span style="color: #009900;">)</span> + geom_point<span style="color: #009900;">(</span>alpha=<span style="color: #cc66cc;">0.01</span><span style="color: #339933;">,</span>size=<span style="color: #cc66cc;">1</span><span style="color: #009900;">)</span> + coord_map<span style="color: #009900;">(</span><span style="color: #009900;">)</span> +
ylim<span style="color: #009900;">(</span>-<span style="color: #cc66cc;">90</span><span style="color: #339933;">,</span><span style="color: #cc66cc;">90</span><span style="color: #009900;">)</span><span style="color: blue;">
</span></pre>
<pre class="r geshifilter-R" style="font-family: monospace;"><span style="color: blue;"> </span></pre>
</div>
</div>
<a href="http://www.inside-r.org/pretty-r" title="Created by Pretty R at inside-R.org">Created by Pretty R at inside-R.org</a><br />
<br />
<br />einar hjörleifssonhttp://www.blogger.com/profile/03811120168583854238noreply@blogger.com0tag:blogger.com,1999:blog-2418399086931873533.post-46647593410012834302012-01-05T13:01:00.000-08:002012-01-05T13:11:27.930-08:00getting ICES 1903-1949 catch statistics into R<br />
in the last post i showed how to get the nominal north east atlantic landings data from 1950 onwards into a usable format in R. this time around a script that shows how to import zipped excel files from 1903 to 1949 is provided. finalizing with a ggplot of the total reported landings over the period.<br />
<br />
again the data is available at the web site <a href="http://www.ices.dk/fish/CATChSTATISTICS.asp" target="_blank">(http://www.ices.dk/fish/CATChSTATISTICS.asp</a>) as excel files. the twist is that the files are stored in a zipped format and there are separate excel files for each country. and within each country file there is a separate worksheet for each of the years. within each worksheet the data are again stored in a wide format, the species recorded in the landings is by rows and the area fished is by columns. the number of rows (species recorded) and the number of columns (area fished) differs among different worksheet. and then there are some other nasties here and there in the worksheets, something that took a little trial and error to circumvent in the final script provided.<br />
<br />
this is what the final work looked like:<br />
################################################################################<br />
# stuff needed<br />
require(gdata)<br />
require(reshape2)<br />
require(stringr)<br />
require(plyr)<br />
<br />
################################################################################<br />
# the source<br />
path <- "YOURDIRECTORY"<br />
URL <- "http://www.ices.dk/"<br />
PATH <- "fish/statlant/"<br />
<br />
<br />
<br />
################################################################################<br />
# Downloading and extraction of 1903 to 1949 data<br />
FILE <- "ICES1903-49.zip"<br />
download.file(paste(URL,PATH,FILE,sep=""),<br />
paste(path,"ICES19030-49.zip",sep=""))<br />
dir <- unzip(paste(path,"ICES19030-49.zip",sep=""))<br />
<br />
files <- sort(dir[grep("xls",dir)])<br />
files <- substr(files,3,nchar(files))<br />
cntr <- substr(files,1,3)<br />
<br />
dat <- NULL<br />
for (i in 1:length(cntr)) {<br />
print(cntr[i])<br />
for (j in c(2:48)) {<br />
print(j+1901)<br />
if (i == 1 & j == 44) { # problem with header character<br />
tmp <- read.xls(files[i],sheet=j,strip.white=T,stringsAsFactors=F,<br />
header=F,skip=1)<br />
names(tmp) <- c("species","alpha","IVc","IV","VIIa,f","VIId,e","UNK","Total")<br />
} else {<br />
tmp <- read.xls(files[i],sheet=j,strip.white=T,stringsAsFactors=F,blank.lines.skip = TRUE)<br />
}<br />
if(i == 19 & j == 25) tmp <- tmp[1,1:2] # Russia 1926 (empty spaces)<br />
#print(tmp)<br />
if(ncol(tmp)>1 & nrow(tmp)> 1) {<br />
rowTotal <- grep("Total",tmp[,1]) # find the location of row Total<br />
colTotal <- grep("Total",names(tmp)) # find the location of column Total<br />
if(i == 6 & j == 22) {rowTotal <- 22 ; colTotal <- 8} # France 1923<br />
if(i == 12 & j == 7) {rowTotal <- 18 ; colTotal <- 7} # Ireland 1908<br />
tmp <- tmp[1:(rowTotal-1),1:(colTotal-1)] # exclude anything below and to<br />
# the right of Total, including the Total<br />
names(tmp)[1:2] <- c("species","alfa") # the species acronym column is in most cases left blank<br />
# in some cases it is however names something odd<br />
tmp <- melt(tmp,id.var=c("species","alfa")) # turn into long format (kind of like inverse of "Pivot table"<br />
tmp$variable <- as.character(tmp$variable)<br />
tmp$value <- str_replace_all(tmp$value,",","")<br />
tmp$value <- ifelse(tmp$value %in% "-",0,tmp$value)<br />
tmp$value <- ifelse(tmp$value %in% ".",NA,tmp$value)<br />
tmp$value <- ifelse(tmp$value %in% "<0.5",0.25,tmp$value)<br />
tmp$value <- ifelse(tmp$value %in% "",NA,tmp$value)<br />
tmp$value <- as.numeric(tmp$value)<br />
tmp$year <- rep(j+1901,nrow(tmp))<br />
tmp$cUN2 <- cntr[i]<br />
dat <- rbind(dat,tmp)<br />
} <br />
} # end loop through sheets (years/worksheet)<br />
} # end loop through countries (xls file)<br />
require(ggplot2)<br />
ggplot(dat,aes(factor(year),weight=value/1e6))+ geom_bar() + xlab("") +<br />
ylab("Nominal landings [million tonnes]")<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-rxQR7JOI1m0/TwYN-HN_4HI/AAAAAAAAAF4/paEStpnptEY/s1600/ggplotNominalLandings.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="215" src="http://4.bp.blogspot.com/-rxQR7JOI1m0/TwYN-HN_4HI/AAAAAAAAAF4/paEStpnptEY/s320/ggplotNominalLandings.png" width="320" /></a></div>
<br />
<br />
the focus here has been on just getting the data into R. more work is needed, in particular when it comes to combining the data from 1903-1949 and 1950-2010. hope i will have some time to work and report on that later.<br />
<br />
<br />
<br />
<br />einar hjörleifssonhttp://www.blogger.com/profile/03811120168583854238noreply@blogger.com0tag:blogger.com,1999:blog-2418399086931873533.post-68135792525480368302011-12-26T05:42:00.000-08:002011-12-26T07:16:38.303-08:00getting ICES 1950-2010 catch statistics into Rthe International Council for the Exploration of the Sea (ICES) has been compiling nominal annual landings statistics by country and division for the north east atlantic since 1904. the data is available at <a href="http://www.ices.dk/fish/CATChSTATISTICS.asp" target="_blank">http://www.ices.dk/fish/CATChSTATISTICS.asp</a>.<br />
<br />
the data from 1950 onwards is either available in the archaic <a href="http://www.fao.org/fishery/statistics/software/fishstat/en" target="_blank">FishStat Plus</a> format or in an MS Excel format. in both cases the format of the data are in a wide format, where the country, species and division is listed in rows and the year is in column. what follows is a script in R that shows how the data can be extracted directly from the website and then put in a long format, more suitable for further exploration.<br />
<br />
<div style="font-family: "Courier New",Courier,monospace;">
# required library<br />
require(gdata)<br />
require(reshape2)<br />
require(stringr)<br />
options(stringsAsFactors=F)<br />
<br />
# downloading and extract<br />
temp <- tempfile()<br />
download.file("http://ices.dk/fish/statlant/ICES1950-2010.zip",temp)<br />
dat <- read.xls(paste(unzip(temp, "ICES_1950-2010.xls"),sep=""),<br />
sheet=1,strip.white=T)<br />
unlink(temp)<br />
<br />
# some cleaning<br />
names(dat) <- tolower(names(dat))<br />
dat <- melt(dat,c("country","species","division"))<br />
dat$variable <- as.character(dat$variable)<br />
dat$variable <- as.integer(str_sub(dat$variable,2))<br />
dat$value <- str_replace_all(dat$value,",","")<br />
dat$value <- ifelse(dat$value %in% "-",0,dat$value)<br />
dat$value <- ifelse(dat$value %in% ".",NA,dat$value)<br />
dat$value <- ifelse(dat$value %in% "<0.5",0.25,dat$value)<br />
dat$value <- ifelse(dat$value %in% "",NA,dat$value)<br />
dat$value <- as.numeric(dat$value)<br />
names(dat) <- c("country","species","descr","year","landings")<br />
dat <- dat[!is.na(dat$landings),]<br />
dat <- dat[dat$landings > 0,]<br />
ices.1950.2010 <- dat<br />
rm(dat,temp)</div>
<br />
<br />
the data frame ices.1950.2010 now contains 5 variables, i.e. the country, the species, descr (the area fished), the year and the landings in tonnes. some initial scoping of the data, like the total nominal landings by year can now be simply plotted by e.g.<br />
<div style="font-family: "Courier New",Courier,monospace;">
# a simple plot<br />
require(ggplot2)<br />
ggplot(ices.1950.2010,aes(year,weight=landings/1e6)) + <br />
geom_bar(binwidth=1) +<br />
ylab("Nominal landings (million metric tonnes)")</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-yd4L4FyzCMQ/TviOmQdZKpI/AAAAAAAAAFg/8SOdth7uWwk/s1600/nominalLandings.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="225" src="http://3.bp.blogspot.com/-yd4L4FyzCMQ/TviOmQdZKpI/AAAAAAAAAFg/8SOdth7uWwk/s320/nominalLandings.png" width="320" /></a></div>
<br />
Further exploration of the data, e.g. landings by particular species and/or country in the format provided is a breeze. However landings by particular are (division, area and subdivision) require further manipulation on the field/variable "descr". more on that later.<br />
<br />
einareinar hjörleifssonhttp://www.blogger.com/profile/03811120168583854238noreply@blogger.com0tag:blogger.com,1999:blog-2418399086931873533.post-61494317829641095072011-12-26T05:19:00.000-08:002011-12-26T05:21:47.531-08:00... in the beginningthis is an evolving space. the most likely focus will be on fisheries data.<br />
lets see how this stuff goes.einar hjörleifssonhttp://www.blogger.com/profile/03811120168583854238noreply@blogger.com0Granaskjól, Capital Region, Iceland64.145667660593844 -21.9701671600341864.144802160593841 -21.972634660034181 64.146533160593847 -21.967699660034178