<div dir="ltr">Hi all, <div><br></div><div>I don't know if anybody has suggested this, but Google Earth Engine has some good algorithm for that! </div><div><br></div><div>Sincerely, </div><div><br clear="all"><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr">Natalia L Quinteros Casaverde<div>Ph.D Student </div><div>Ecology, Evolutionary Biology and Behavior Subprogram </div><div>Biology Program.<br></div><div>City College of New York & </div><div><span style="font-size:12.8px">Graduate Center</span><br></div><div>CUNY</div><div><br></div><div><br></div></div></div></div></div></div></div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr">On Wed, Dec 5, 2018 at 6:08 PM Ruben Soares Lapa Remelgado <<a href="mailto:ruben.remelgado@uni-wuerzburg.de">ruben.remelgado@uni-wuerzburg.de</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<p>Hi Ben,<br>
<br>
I get your point. That is similar to what I did in the past (see <a href="https://zslpublications.onlinelibrary.wiley.com/doi/full/10.1002/rse2.70" target="_blank">here</a>).
In answer to your question: I used the Landsat cloud information
without further processing. I interpreted the QA layer and made a
mask from the "best quality" pixels. Then, I built phenological
composites since, in my case, the habitat requirements of my
target species were heavily influenced by vegetation phenology. I
was also dealing with an area heavily affected by clouds (i.e. SW
Germany) and found that while there were some artifacts, in its
majority, the output was reasonable since the Landsat QA layer
captured most of the clouds, most of the time. In my study, I
aimed for the closest date to the reference dates (i.e. for Start
of Season, Mid of Season, End of Season). My goal was to have an
output as faithful to the local phenology as possible. However, if
you are worried about clouds - and following up on Florencias
suggestion - you can try to refine this approach by choosing the
pixels with the highest NDVI within the temporal window (i.e.
target date -/+ a given time buffer) instead of the pixel with the
closest acquisition date. The rsComposite() function of rsMove is
the same algorithm I used in my study. If my solution fits your
expectations, I can help you adjust the code or add an option in
rsMove (so that you don't need to compile the code yourself).<br>
<br>
Best,<br>
Ruben</p>
<div class="m_6502248774575520568moz-cite-prefix">On 05/12/2018 19:22, Ben Carlson wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi Ruben,
<div><br>
</div>
<div>Thanks so much for your reply. I'm not doing time-series
analysis, but instead looking at animal's fine-grained habitat
use using movement data. I'd like to avoid interpolation,
because my landscape is a mosaic of different habitats and
transitions between habitats can be sharp, so I'm worried that
interpolation is not appropriate. I'm just hoping that I can
get enough clear pixels and then I'll remove any telemetry
locations where there is no NDVI data. Due to the high cloud
cover in my study area, right now I'm settling for a monthly
(for each year) composite. I'm really only interested in the
summer months (~May to September). I plan to use this monthly
time-series variable as an input into a resource selection
function.</div>
<div><br>
</div>
<div>I'm really interested to know if there are any best
practices around using the qa bits to detect clouds (or if
there are better ways of masking clouds). I looked around your
rsMove code (very impressive!), but could not find if you are
doing any cloud masking. For example, when you do
interpolation, how do you decide that a particular pixel is
not good and should instead be interpolated?</div>
<div><br>
</div>
<div>Thanks a lot for the help!</div>
<div><br>
</div>
<div>Ben</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr">On Tue, Dec 4, 2018 at 12:47 PM Ruben Soares Lapa
Remelgado <<a href="mailto:ruben.remelgado@uni-wuerzburg.de" target="_blank">ruben.remelgado@uni-wuerzburg.de</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<p>Hi Ben,<br>
<br>
Depends on what kind of analysis you are doing. If you are
doing some kind of time series analysis, I would suggest
you interpolate the missing values (taking in
consideration the acquisition dates) and then smooth the
output using e.g. a running mean. Even if you get a mask
from USGS, that is still a generic estimation and will
contain artifacts. Might also be useful to filter out very
small values (a typical empirical threshold is -0.3) if
you are not interested in water bodies. Still, note that
not every cloud/shadow has a low NDVI.<br>
<br>
If you are interested, I implemented a function for RS
data interpolation. You can install rsMove and use the
intime() function. You should install it from my gitHub
(see <a href="https://github.com/RRemelgado/rsMove" target="_blank">here</a>) because
I still haven't submitted an update to CRAN. intime() It's
a c++ that performs a linear, time sensitive
interpolation. This means that for, each observation, it
will check what are the closest time steps in relation to
the date(s) you want to have NDVI values for (in the past
and in the future) and used them to interpolate the
missing value, assuming that the time different (in days)
is smaller than a predefined temporal buffer. A temporal
buffer is required because, if the gaps are too big, you
will likely over-generalize the NDVI curve and thus miss a
lot of important, seasonal changes. The function is also
called by imgInt() - which applies it to raster objects -
but I'm still working on it. So I suggest you use
getValues() to extract the data as a matrix before
applying the function. You can then set it back with
setValues() and the original raster stack. It will build a
new stack with the interpolated values.<br>
<br>
If you really want to build mosaics, I can propose you
another function from the same package called
rsComposite(). It builds composites for the nearest date
(if you want something for a specific year) or composites
that are phenology dependent (if your landscape doesn't
change much over the years , it uses multi-year data to
build a composite around a specific day of the year).
After, you can maybe use a spatial smoothing (e.g. using
the median with the focal() function of the raster
package) to deal with outliers.<br>
<br>
Best,<br>
Ruben</p>
<div class="m_6502248774575520568m_-8529814442834977966moz-cite-prefix">On
04/12/2018 18:19, Ben Carlson wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">Hello,
<div><br>
</div>
<div>I'm working with landsat 8 collection 1 surface
collection data. I'd like to mask out clouds, with
the goal of generating NDVI so that I can examine
how animal movements respond to this variable.</div>
<div><br>
</div>
<div>In the product guide for L8 SR data (page 22),
it is recommended that "clear" conditions
correspond to pixel_qa values of 322, 386, 834,
898, and 1346.</div>
<div><br>
</div>
<div><a href="https://landsat.usgs.gov/sites/default/files/documents/lasrc_product_guide.pdf" target="_blank">https://landsat.usgs.gov/sites/default/files/documents/lasrc_product_guide.pdf</a><br>
</div>
<div><br>
</div>
<div>I've implemented this recommendation as a cloud
mask and based on visual inspection it looks fine.
However, I'm wondering if there are other, better
recommendations based on my intended use of the
data.</div>
<div><br>
</div>
<div>Note: In case this affects the answer, I'm also
mosaicing multiple landsat scenes in order to fill
in blank areas due to clouds.</div>
<div><br>
</div>
<div>Thank you!</div>
<div><br>
</div>
<div>Ben</div>
<div><br>
</div>
<div><br>
</div>
</div>
</div>
</div>
<br>
<fieldset class="m_6502248774575520568m_-8529814442834977966mimeAttachmentHeader"></fieldset>
<pre class="m_6502248774575520568m_-8529814442834977966moz-quote-pre">_______________________________________________
RS-ecology mailing list
<a class="m_6502248774575520568m_-8529814442834977966moz-txt-link-abbreviated" href="mailto:RS-ecology@lists.uni-wuerzburg.de" target="_blank">RS-ecology@lists.uni-wuerzburg.de</a>
<a class="m_6502248774575520568m_-8529814442834977966moz-txt-link-freetext" href="https://lists.uni-wuerzburg.de/mailman/listinfo/rs-ecology" target="_blank">https://lists.uni-wuerzburg.de/mailman/listinfo/rs-ecology</a>
</pre>
</blockquote>
<pre class="m_6502248774575520568m_-8529814442834977966moz-signature" cols="72">--
Ruben Remelgado, M.Sc.
Department of Remote Sensing
Institute of Geography and Geology
University of Wuerzburg
Oswald-Kuelpe-Weg 86
97074 Wuerzburg, Germany
Phone - +49 (0) 931 31-83562
Email - <a class="m_6502248774575520568m_-8529814442834977966moz-txt-link-abbreviated" href="mailto:ruben.remelgado@uni-wuerzburg.de" target="_blank">ruben.remelgado@uni-wuerzburg.de</a>
Url - <a class="m_6502248774575520568m_-8529814442834977966moz-txt-link-freetext" href="https://www.geographie.uni-wuerzburg.de/fernerkundung/personen/remelgado_ruben_msc/" target="_blank">https://www.geographie.uni-wuerzburg.de/fernerkundung/personen/remelgado_ruben_msc/</a></pre>
</div>
</blockquote>
</div>
<br>
<fieldset class="m_6502248774575520568mimeAttachmentHeader"></fieldset>
<pre class="m_6502248774575520568moz-quote-pre">_______________________________________________
RS-ecology mailing list
<a class="m_6502248774575520568moz-txt-link-abbreviated" href="mailto:RS-ecology@lists.uni-wuerzburg.de" target="_blank">RS-ecology@lists.uni-wuerzburg.de</a>
<a class="m_6502248774575520568moz-txt-link-freetext" href="https://lists.uni-wuerzburg.de/mailman/listinfo/rs-ecology" target="_blank">https://lists.uni-wuerzburg.de/mailman/listinfo/rs-ecology</a>
</pre>
</blockquote>
<pre class="m_6502248774575520568moz-signature" cols="72">--
Ruben Remelgado, M.Sc.
Department of Remote Sensing
Institute of Geography and Geology
University of Wuerzburg
Oswald-Kuelpe-Weg 86
97074 Wuerzburg, Germany
Phone - +49 (0) 931 31-83562
Email - <a class="m_6502248774575520568moz-txt-link-abbreviated" href="mailto:ruben.remelgado@uni-wuerzburg.de" target="_blank">ruben.remelgado@uni-wuerzburg.de</a>
Url - <a class="m_6502248774575520568moz-txt-link-freetext" href="https://www.geographie.uni-wuerzburg.de/fernerkundung/personen/remelgado_ruben_msc/" target="_blank">https://www.geographie.uni-wuerzburg.de/fernerkundung/personen/remelgado_ruben_msc/</a></pre>
</div>
_______________________________________________<br>
RS-ecology mailing list<br>
<a href="mailto:RS-ecology@lists.uni-wuerzburg.de" target="_blank">RS-ecology@lists.uni-wuerzburg.de</a><br>
<a href="https://lists.uni-wuerzburg.de/mailman/listinfo/rs-ecology" rel="noreferrer" target="_blank">https://lists.uni-wuerzburg.de/mailman/listinfo/rs-ecology</a><br>
</blockquote></div>