[RS-ecology] Best practices for cloud-masking using L8 SR

Thu Dec 6 16:17:24 CET 2018

Hi all,

I don't know if anybody has suggested this, but Google Earth Engine has
some good algorithm for that!

Sincerely,

Natalia L Quinteros Casaverde
Ph.D Student
Ecology, Evolutionary Biology and Behavior Subprogram
Biology Program.
City College of New York &
Graduate Center
CUNY

On Wed, Dec 5, 2018 at 6:08 PM Ruben Soares Lapa Remelgado <
ruben.remelgado at uni-wuerzburg.de> wrote:

> Hi Ben,
>
> I get your point. That is similar to what I did in the past (see here
> <https://zslpublications.onlinelibrary.wiley.com/doi/full/10.1002/rse2.70>).
> In answer to your question: I used the Landsat cloud information without
> further processing. I interpreted the QA layer and made a mask from the
> "best quality" pixels. Then, I built phenological composites since, in my
> case, the habitat requirements of my target species were heavily influenced
> by vegetation phenology. I was also dealing with an area heavily affected
> by clouds (i.e. SW Germany) and found that while there were some artifacts,
> in its majority, the output was reasonable since the Landsat QA layer
> captured most of the clouds, most of the time. In my study, I aimed for the
> closest date to the reference dates (i.e. for Start of Season, Mid of
> Season, End of Season). My goal was to have an output as faithful to the
> local phenology as possible. However, if you are worried about clouds - and
> following up on Florencias suggestion - you can try to refine this approach
> by choosing the pixels with the highest NDVI within the temporal window
> (i.e. target date -/+ a given time buffer) instead of the pixel with the
> closest acquisition date. The rsComposite() function of rsMove is the same
> algorithm I used  in my study. If my solution fits your expectations, I can
> help you adjust the code or add an option in rsMove (so that you don't need
> to compile the code yourself).
>
> Best,
> Ruben
> On 05/12/2018 19:22, Ben Carlson wrote:
>
> Hi Ruben,
>
> Thanks so much for your reply. I'm not doing time-series analysis, but
> instead looking at animal's fine-grained habitat use using movement data.
> I'd like to avoid interpolation, because my landscape is a mosaic of
> different habitats and transitions between habitats can be sharp, so I'm
> worried that interpolation is not appropriate. I'm just hoping that I can
> get enough clear pixels and then I'll remove any telemetry locations where
> there is no NDVI data. Due to the high cloud cover in my study area, right
> now I'm settling for a monthly (for each year) composite. I'm really only
> interested in the summer months (~May to September). I plan to use this
> monthly time-series variable as an input into a resource selection function.
>
> I'm really interested to know if there are any best practices around using
> the qa bits to detect clouds (or if there are better ways of masking
> clouds). I looked around your rsMove code (very impressive!), but could not
> find if you are doing any cloud masking. For example, when you do
> interpolation, how do you decide that a particular pixel is not good and
> should instead be interpolated?
>
> Thanks a lot for the help!
>
> Ben
>
> On Tue, Dec 4, 2018 at 12:47 PM Ruben Soares Lapa Remelgado <
> ruben.remelgado at uni-wuerzburg.de> wrote:
>
>> Hi Ben,
>>
>> Depends on what kind of analysis you are doing. If you are doing some
>> kind of time series analysis, I would suggest you interpolate the missing
>> values (taking in consideration the acquisition dates) and then smooth the
>> output using e.g. a running mean. Even if you get a mask from USGS, that is
>> still a generic estimation and will contain artifacts. Might also be useful
>> to filter out very small values (a typical empirical threshold is -0.3) if
>> you are not interested in water bodies. Still, note that not every
>> cloud/shadow has a low NDVI.
>>
>> If you are interested, I implemented a function for RS data
>> interpolation. You can install rsMove and use the intime() function. You
>> should install it from my gitHub (see here
>> <https://github.com/RRemelgado/rsMove>) because I still haven't
>> submitted an update to CRAN. intime() It's a c++ that performs a linear,
>> time sensitive interpolation. This means that for, each observation, it
>> will check what are the closest time steps in relation to the date(s) you
>> want to have NDVI values for (in the past and in the future) and used them
>> to interpolate the missing value, assuming that the time different (in
>> days) is smaller  than a predefined temporal buffer. A temporal buffer is
>> required because, if the gaps are too big, you will likely over-generalize
>> the NDVI curve and thus miss a lot of important, seasonal changes. The
>> function is also called by imgInt() - which applies it to raster objects -
>> but I'm still working on it. So I suggest you use getValues() to extract
>> the data as a matrix before applying the function. You can then set it back
>> with setValues() and the original raster stack. It will build a new stack
>> with the interpolated values.
>>
>> If you really want to build mosaics, I can propose you another function
>> from the same package called rsComposite(). It builds composites for the
>> nearest date (if you want something for a specific year) or composites that
>> are phenology dependent (if your landscape doesn't change much over the
>> years , it uses multi-year data to build a composite around a specific day
>> of the year). After, you can maybe use a spatial smoothing (e.g. using the
>> median with the focal() function of the raster package) to deal with
>> outliers.
>>
>> Best,
>> Ruben
>> On 04/12/2018 18:19, Ben Carlson wrote:
>>
>> Hello,
>>
>> I'm working with landsat 8 collection 1 surface collection data. I'd like
>> to mask out clouds, with the goal of generating NDVI so that I can examine
>> how animal movements respond to this variable.
>>
>> In the product guide for L8 SR data (page 22), it is recommended that
>> "clear" conditions correspond to pixel_qa values of 322, 386, 834, 898, and
>> 1346.
>>
>>
>> https://landsat.usgs.gov/sites/default/files/documents/lasrc_product_guide.pdf
>>
>> I've implemented this recommendation as a cloud mask and based on visual
>> inspection it looks fine. However, I'm wondering if there are other, better
>> recommendations based on my intended use of the data.
>>
>> Note: In case this affects the answer, I'm also mosaicing multiple
>> landsat scenes in order to fill in blank areas due to clouds.
>>
>> Thank you!
>>
>> Ben
>>
>>
>>
>> _______________________________________________
>> RS-ecology mailing listRS-ecology at lists.uni-wuerzburg.dehttps://lists.uni-wuerzburg.de/mailman/listinfo/rs-ecology
>>
>> --
>> Ruben Remelgado, M.Sc.
>>
>> Department of Remote Sensing
>> Institute of Geography and Geology
>> University of Wuerzburg
>> Oswald-Kuelpe-Weg 86
>> 97074 Wuerzburg, Germany
>>
>> Phone - +49 (0) 931 31-83562
>> Email - ruben.remelgado at uni-wuerzburg.de
>> Url - https://www.geographie.uni-wuerzburg.de/fernerkundung/personen/remelgado_ruben_msc/
>>
>>
> _______________________________________________
> RS-ecology mailing listRS-ecology at lists.uni-wuerzburg.dehttps://lists.uni-wuerzburg.de/mailman/listinfo/rs-ecology
>
> --
> Ruben Remelgado, M.Sc.
>
> Department of Remote Sensing
> Institute of Geography and Geology
> University of Wuerzburg
> Oswald-Kuelpe-Weg 86
> 97074 Wuerzburg, Germany
>
> Phone - +49 (0) 931 31-83562
> Email - ruben.remelgado at uni-wuerzburg.de
> Url - https://www.geographie.uni-wuerzburg.de/fernerkundung/personen/remelgado_ruben_msc/
>
> _______________________________________________
> RS-ecology mailing list
> RS-ecology at lists.uni-wuerzburg.de
> https://lists.uni-wuerzburg.de/mailman/listinfo/rs-ecology
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.uni-wuerzburg.de/pipermail/rs-ecology/attachments/20181206/5d2fa11b/attachment.html>