<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Hi Ben,
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">If you calculate the NDVI for the multiple years, you can aggregate them using the Maximum Value Composite aggregation method that essentially picks the pixel
with highest NDVI value. Cloud cover affects NDVI by decreasing its value (as clouds have high red reflectance).
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">It is always good to assess the final composite using the cloud masks to identify if there are areas of persistent cloud cover where the NDVI would not be representative
of the cover. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">There are several factors that affect this oldie-but-goody technique but it is still widely used for aggregation of NDVI images.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">https://www.tandfonline.com/doi/pdf/10.1080/01431168608948945<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Flor<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> RS-ecology <rs-ecology-bounces@lists.uni-wuerzburg.de>
<b>On Behalf Of </b>Ben Carlson<br>
<b>Sent:</b> Wednesday, December 5, 2018 1:22 PM<br>
<b>To:</b> rs-ecology@lists.uni-wuerzburg.de<br>
<b>Subject:</b> Re: [RS-ecology] Best practices for cloud-masking using L8 SR<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">Hi Ruben,<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Thanks so much for your reply. I'm not doing time-series analysis, but instead looking at animal's fine-grained habitat use using movement data. I'd like to avoid interpolation, because my landscape is a mosaic of different habitats and
transitions between habitats can be sharp, so I'm worried that interpolation is not appropriate. I'm just hoping that I can get enough clear pixels and then I'll remove any telemetry locations where there is no NDVI data. Due to the high cloud cover in my
study area, right now I'm settling for a monthly (for each year) composite. I'm really only interested in the summer months (~May to September). I plan to use this monthly time-series variable as an input into a resource selection function.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I'm really interested to know if there are any best practices around using the qa bits to detect clouds (or if there are better ways of masking clouds). I looked around your rsMove code (very impressive!), but could not find if you are
doing any cloud masking. For example, when you do interpolation, how do you decide that a particular pixel is not good and should instead be interpolated?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Thanks a lot for the help!<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Ben<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Tue, Dec 4, 2018 at 12:47 PM Ruben Soares Lapa Remelgado <<a href="mailto:ruben.remelgado@uni-wuerzburg.de">ruben.remelgado@uni-wuerzburg.de</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<p>Hi Ben,<br>
<br>
Depends on what kind of analysis you are doing. If you are doing some kind of time series analysis, I would suggest you interpolate the missing values (taking in consideration the acquisition dates) and then smooth the output using e.g. a running mean. Even
if you get a mask from USGS, that is still a generic estimation and will contain artifacts. Might also be useful to filter out very small values (a typical empirical threshold is -0.3) if you are not interested in water bodies. Still, note that not every cloud/shadow
has a low NDVI.<br>
<br>
If you are interested, I implemented a function for RS data interpolation. You can install rsMove and use the intime() function. You should install it from my gitHub (see
<a href="https://github.com/RRemelgado/rsMove" target="_blank">here</a>) because I still haven't submitted an update to CRAN. intime() It's a c++ that performs a linear, time sensitive interpolation. This means that for, each observation, it will check what
are the closest time steps in relation to the date(s) you want to have NDVI values for (in the past and in the future) and used them to interpolate the missing value, assuming that the time different (in days) is smaller than a predefined temporal buffer.
A temporal buffer is required because, if the gaps are too big, you will likely over-generalize the NDVI curve and thus miss a lot of important, seasonal changes. The function is also called by imgInt() - which applies it to raster objects - but I'm still
working on it. So I suggest you use getValues() to extract the data as a matrix before applying the function. You can then set it back with setValues() and the original raster stack. It will build a new stack with the interpolated values.<br>
<br>
If you really want to build mosaics, I can propose you another function from the same package called rsComposite(). It builds composites for the nearest date (if you want something for a specific year) or composites that are phenology dependent (if your landscape
doesn't change much over the years , it uses multi-year data to build a composite around a specific day of the year). After, you can maybe use a spatial smoothing (e.g. using the median with the focal() function of the raster package) to deal with outliers.<br>
<br>
Best,<br>
Ruben<o:p></o:p></p>
<div>
<p class="MsoNormal">On 04/12/2018 18:19, Ben Carlson wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal">Hello, <o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I'm working with landsat 8 collection 1 surface collection data. I'd like to mask out clouds, with the goal of generating NDVI so that I can examine how animal movements respond to this variable.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">In the product guide for L8 SR data (page 22), it is recommended that "clear" conditions correspond to pixel_qa values of 322, 386, 834, 898, and 1346.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><a href="https://landsat.usgs.gov/sites/default/files/documents/lasrc_product_guide.pdf" target="_blank">https://landsat.usgs.gov/sites/default/files/documents/lasrc_product_guide.pdf</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I've implemented this recommendation as a cloud mask and based on visual inspection it looks fine. However, I'm wondering if there are other, better recommendations based on my intended use of the data.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Note: In case this affects the answer, I'm also mosaicing multiple landsat scenes in order to fill in blank areas due to clouds.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Thank you!<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Ben<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<pre>_______________________________________________<o:p></o:p></pre>
<pre>RS-ecology mailing list<o:p></o:p></pre>
<pre><a href="mailto:RS-ecology@lists.uni-wuerzburg.de" target="_blank">RS-ecology@lists.uni-wuerzburg.de</a><o:p></o:p></pre>
<pre><a href="https://lists.uni-wuerzburg.de/mailman/listinfo/rs-ecology" target="_blank">https://lists.uni-wuerzburg.de/mailman/listinfo/rs-ecology</a><o:p></o:p></pre>
</blockquote>
<pre>-- <o:p></o:p></pre>
<pre>Ruben Remelgado, M.Sc.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Department of Remote Sensing<o:p></o:p></pre>
<pre>Institute of Geography and Geology<o:p></o:p></pre>
<pre>University of Wuerzburg<o:p></o:p></pre>
<pre>Oswald-Kuelpe-Weg 86<o:p></o:p></pre>
<pre>97074 Wuerzburg, Germany<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Phone - +49 (0) 931 31-83562<o:p></o:p></pre>
<pre>Email - <a href="mailto:ruben.remelgado@uni-wuerzburg.de" target="_blank">ruben.remelgado@uni-wuerzburg.de</a><o:p></o:p></pre>
<pre>Url - <a href="https://www.geographie.uni-wuerzburg.de/fernerkundung/personen/remelgado_ruben_msc/" target="_blank">https://www.geographie.uni-wuerzburg.de/fernerkundung/personen/remelgado_ruben_msc/</a><o:p></o:p></pre>
</div>
</blockquote>
</div>
</div>
</body>
</html>