[RS-ecology] Best practices for cloud-masking using L8 SR

Wed Dec 5 19:37:56 CET 2018

Hi Ben,
If you calculate the NDVI for the multiple years, you can aggregate them using the Maximum Value Composite aggregation method that essentially picks the pixel with highest NDVI value.  Cloud cover affects NDVI by decreasing its value (as clouds have high red reflectance).
It is always good to assess the final composite using the cloud masks to identify if there are areas of persistent cloud cover where the NDVI would not be representative of the cover.
There are several factors that affect this oldie-but-goody technique but it is still widely used for aggregation of NDVI images.
https://www.tandfonline.com/doi/pdf/10.1080/01431168608948945
Flor

From: RS-ecology <rs-ecology-bounces at lists.uni-wuerzburg.de> On Behalf Of Ben Carlson
Sent: Wednesday, December 5, 2018 1:22 PM
To: rs-ecology at lists.uni-wuerzburg.de
Subject: Re: [RS-ecology] Best practices for cloud-masking using L8 SR

Hi Ruben,

Thanks so much for your reply. I'm not doing time-series analysis, but instead looking at animal's fine-grained habitat use using movement data. I'd like to avoid interpolation, because my landscape is a mosaic of different habitats and transitions between habitats can be sharp, so I'm worried that interpolation is not appropriate. I'm just hoping that I can get enough clear pixels and then I'll remove any telemetry locations where there is no NDVI data. Due to the high cloud cover in my study area, right now I'm settling for a monthly (for each year) composite. I'm really only interested in the summer months (~May to September). I plan to use this monthly time-series variable as an input into a resource selection function.

I'm really interested to know if there are any best practices around using the qa bits to detect clouds (or if there are better ways of masking clouds). I looked around your rsMove code (very impressive!), but could not find if you are doing any cloud masking. For example, when you do interpolation, how do you decide that a particular pixel is not good and should instead be interpolated?

Thanks a lot for the help!

Ben

On Tue, Dec 4, 2018 at 12:47 PM Ruben Soares Lapa Remelgado <ruben.remelgado at uni-wuerzburg.de<mailto:ruben.remelgado at uni-wuerzburg.de>> wrote:

Hi Ben,

Depends on what kind of analysis you are doing. If you are doing some kind of time series analysis, I would suggest you interpolate the missing values (taking in consideration the acquisition dates) and then smooth the output using e.g. a running mean. Even if you get a mask from USGS, that is still a generic estimation and will contain artifacts. Might also be useful to filter out very small values (a typical empirical threshold is -0.3) if you are not interested in water bodies. Still, note that not every cloud/shadow has a low NDVI.

If you are interested, I implemented a function for RS data interpolation. You can install rsMove and use the intime() function. You should install it from my gitHub (see here<https://github.com/RRemelgado/rsMove>) because I still haven't submitted an update to CRAN. intime() It's a c++ that performs a linear, time sensitive interpolation. This means that for, each observation, it will check what are the closest time steps in relation to the date(s) you want to have NDVI values for (in the past and in the future) and used them to interpolate the missing value, assuming that the time different (in days) is smaller  than a predefined temporal buffer. A temporal buffer is required because, if the gaps are too big, you will likely over-generalize the NDVI curve and thus miss a lot of important, seasonal changes. The function is also called by imgInt() - which applies it to raster objects - but I'm still working on it. So I suggest you use getValues() to extract the data as a matrix before applying the function. You can then set it back with setValues() and the original raster stack. It will build a new stack with the interpolated values.

If you really want to build mosaics, I can propose you another function from the same package called rsComposite(). It builds composites for the nearest date (if you want something for a specific year) or composites that are phenology dependent (if your landscape doesn't change much over the years , it uses multi-year data to build a composite around a specific day of the year). After, you can maybe use a spatial smoothing (e.g. using the median with the focal() function of the raster package) to deal with outliers.

Best,
Ruben
On 04/12/2018 18:19, Ben Carlson wrote:
Hello,

I'm working with landsat 8 collection 1 surface collection data. I'd like to mask out clouds, with the goal of generating NDVI so that I can examine how animal movements respond to this variable.

In the product guide for L8 SR data (page 22), it is recommended that "clear" conditions correspond to pixel_qa values of 322, 386, 834, 898, and 1346.

https://landsat.usgs.gov/sites/default/files/documents/lasrc_product_guide.pdf

I've implemented this recommendation as a cloud mask and based on visual inspection it looks fine. However, I'm wondering if there are other, better recommendations based on my intended use of the data.

Note: In case this affects the answer, I'm also mosaicing multiple landsat scenes in order to fill in blank areas due to clouds.

Thank you!

Ben

_______________________________________________

RS-ecology mailing list

RS-ecology at lists.uni-wuerzburg.de<mailto:RS-ecology at lists.uni-wuerzburg.de>

https://lists.uni-wuerzburg.de/mailman/listinfo/rs-ecology

--

Ruben Remelgado, M.Sc.

Department of Remote Sensing

Institute of Geography and Geology

University of Wuerzburg

Oswald-Kuelpe-Weg 86

97074 Wuerzburg, Germany

Phone - +49 (0) 931 31-83562

Email - ruben.remelgado at uni-wuerzburg.de<mailto:ruben.remelgado at uni-wuerzburg.de>

Url - https://www.geographie.uni-wuerzburg.de/fernerkundung/personen/remelgado_ruben_msc/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.uni-wuerzburg.de/pipermail/rs-ecology/attachments/20181205/4ee467b1/attachment-0001.html>