[RS-ecology] Best practices for cloud-masking using L8 SR

Ruben Soares Lapa Remelgado ruben.remelgado at uni-wuerzburg.de
Thu Dec 6 00:08:50 CET 2018 

Hi Ben,

I get your point. That is similar to what I did in the past (see here 
<https://zslpublications.onlinelibrary.wiley.com/doi/full/10.1002/rse2.70>). 
In answer to your question: I used the Landsat cloud information without 
further processing. I interpreted the QA layer and made a mask from the 
"best quality" pixels. Then, I built phenological composites since, in 
my case, the habitat requirements of my target species were heavily 
influenced by vegetation phenology. I was also dealing with an area 
heavily affected by clouds (i.e. SW Germany) and found that while there 
were some artifacts, in its majority, the output was reasonable since 
the Landsat QA layer captured most of the clouds, most of the time. In 
my study, I aimed for the closest date to the reference dates (i.e. for 
Start of Season, Mid of Season, End of Season). My goal was to have an 
output as faithful to the local phenology as possible. However, if you 
are worried about clouds - and following up on Florencias suggestion - 
you can try to refine this approach by choosing the pixels with the 
highest NDVI within the temporal window (i.e. target date -/+ a given 
time buffer) instead of the pixel with the closest acquisition date. The 
rsComposite() function of rsMove is the same algorithm I used  in my 
study. If my solution fits your expectations, I can help you adjust the 
code or add an option in rsMove (so that you don't need to compile the 
code yourself).

Best,
Ruben

On 05/12/2018 19:22, Ben Carlson wrote:
> Hi Ruben,
>
> Thanks so much for your reply. I'm not doing time-series analysis, but 
> instead looking at animal's fine-grained habitat use using movement 
> data. I'd like to avoid interpolation, because my landscape is a 
> mosaic of different habitats and transitions between habitats can be 
> sharp, so I'm worried that interpolation is not appropriate. I'm just 
> hoping that I can get enough clear pixels and then I'll remove any 
> telemetry locations where there is no NDVI data. Due to the high cloud 
> cover in my study area, right now I'm settling for a monthly (for each 
> year) composite. I'm really only interested in the summer months (~May 
> to September). I plan to use this monthly time-series variable as an 
> input into a resource selection function.
>
> I'm really interested to know if there are any best practices around 
> using the qa bits to detect clouds (or if there are better ways of 
> masking clouds). I looked around your rsMove code (very impressive!), 
> but could not find if you are doing any cloud masking. For example, 
> when you do interpolation, how do you decide that a particular pixel 
> is not good and should instead be interpolated?
>
> Thanks a lot for the help!
>
> Ben
>
> On Tue, Dec 4, 2018 at 12:47 PM Ruben Soares Lapa Remelgado 
> <ruben.remelgado at uni-wuerzburg.de 
> <mailto:ruben.remelgado at uni-wuerzburg.de>> wrote:
>
>     Hi Ben,
>
>     Depends on what kind of analysis you are doing. If you are doing
>     some kind of time series analysis, I would suggest you interpolate
>     the missing values (taking in consideration the acquisition dates)
>     and then smooth the output using e.g. a running mean. Even if you
>     get a mask from USGS, that is still a generic estimation and will
>     contain artifacts. Might also be useful to filter out very small
>     values (a typical empirical threshold is -0.3) if you are not
>     interested in water bodies. Still, note that not every
>     cloud/shadow has a low NDVI.
>
>     If you are interested, I implemented a function for RS data
>     interpolation. You can install rsMove and use the intime()
>     function. You should install it from my gitHub (see here
>     <https://github.com/RRemelgado/rsMove>) because I still haven't
>     submitted an update to CRAN. intime() It's a c++ that performs a
>     linear, time sensitive interpolation. This means that for, each
>     observation, it will check what are the closest time steps in
>     relation to the date(s) you want to have NDVI values for (in the
>     past and in the future) and used them to interpolate the missing
>     value, assuming that the time different (in days) is smaller  than
>     a predefined temporal buffer. A temporal buffer is required
>     because, if the gaps are too big, you will likely over-generalize
>     the NDVI curve and thus miss a lot of important, seasonal changes.
>     The function is also called by imgInt() - which applies it to
>     raster objects - but I'm still working on it. So I suggest you use
>     getValues() to extract the data as a matrix before applying the
>     function. You can then set it back with setValues() and the
>     original raster stack. It will build a new stack with the
>     interpolated values.
>
>     If you really want to build mosaics, I can propose you another
>     function from the same package called rsComposite(). It builds
>     composites for the nearest date (if you want something for a
>     specific year) or composites that are phenology dependent (if your
>     landscape doesn't change much over the years , it uses multi-year
>     data to build a composite around a specific day of the year).
>     After, you can maybe use a spatial smoothing (e.g. using the
>     median with the focal() function of the raster package) to deal
>     with outliers.
>
>     Best,
>     Ruben
>
>     On 04/12/2018 18:19, Ben Carlson wrote:
>>     Hello,
>>
>>     I'm working with landsat 8 collection 1 surface collection data.
>>     I'd like to mask out clouds, with the goal of generating NDVI so
>>     that I can examine how animal movements respond to this variable.
>>
>>     In the product guide for L8 SR data (page 22), it is recommended
>>     that "clear" conditions correspond to pixel_qa values of 322,
>>     386, 834, 898, and 1346.
>>
>>     https://landsat.usgs.gov/sites/default/files/documents/lasrc_product_guide.pdf
>>
>>     I've implemented this recommendation as a cloud mask and based on
>>     visual inspection it looks fine. However, I'm wondering if there
>>     are other, better recommendations based on my intended use of the
>>     data.
>>
>>     Note: In case this affects the answer, I'm also mosaicing
>>     multiple landsat scenes in order to fill in blank areas due to
>>     clouds.
>>
>>     Thank you!
>>
>>     Ben
>>
>>
>>
>>     _______________________________________________
>>     RS-ecology mailing list
>>     RS-ecology at lists.uni-wuerzburg.de  <mailto:RS-ecology at lists.uni-wuerzburg.de>
>>     https://lists.uni-wuerzburg.de/mailman/listinfo/rs-ecology
>
>     -- 
>     Ruben Remelgado, M.Sc.
>
>     Department of Remote Sensing
>     Institute of Geography and Geology
>     University of Wuerzburg
>     Oswald-Kuelpe-Weg 86
>     97074 Wuerzburg, Germany
>
>     Phone - +49 (0) 931 31-83562
>     Email -ruben.remelgado at uni-wuerzburg.de  <mailto:ruben.remelgado at uni-wuerzburg.de>
>     Url -https://www.geographie.uni-wuerzburg.de/fernerkundung/personen/remelgado_ruben_msc/
>
>
> _______________________________________________
> RS-ecology mailing list
> RS-ecology at lists.uni-wuerzburg.de
> https://lists.uni-wuerzburg.de/mailman/listinfo/rs-ecology

-- 
Ruben Remelgado, M.Sc.

Department of Remote Sensing
Institute of Geography and Geology
University of Wuerzburg
Oswald-Kuelpe-Weg 86
97074 Wuerzburg, Germany

Phone - +49 (0) 931 31-83562
Email - ruben.remelgado at uni-wuerzburg.de
Url - https://www.geographie.uni-wuerzburg.de/fernerkundung/personen/remelgado_ruben_msc/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.uni-wuerzburg.de/pipermail/rs-ecology/attachments/20181206/d2784fc4/attachment.html>

More information about the RS-ecology mailing list