Sentinel 2 first steps

Although optical imagery is easier to process and accommodate into one’s workflow, I admit I have had some other interests lately and never managed to fully explore Sentinel 2 as it deserves. With all my attention turned towards S1 and S3, their middle brother somehow escaped the ruffle. Not anymore.

For my research thesis, I am supposed to use both radar and optical imagery, hence I need to distribute equal time to check the developments in the field, to understand the sensors, their potential contribution and ultimately, how can I turn the data into useful information. At 10/20 m spatial resolution, Sentinel 2 is not ideal, but it is closer to what I am expecting from my satellite data, because of its multispectral capabilities in the IR region (you can check out info about the S2 bands here and here). Better than the great old Landsat though.

These being said, this is not the first time I tango with S2 data, did it many times before, but it is the first time I document it and share my thoughts. I’ll walk you through a couple of easy peasy ways to acquire, pre-process and obtain an analysis-ready product in an open source environment, so bear with me.

Ceahlau

#1 Sometimes old is gold

Call me old-fashioned, but currently, I am just experimenting to understand if the S2 data will be of use for my project. I don’t have so many scenes to investigate and I prefer to actually run through the whole workflow in order to understand what am I dealing with. Hence, I’ve decided it is enough for me to download a few scenes and use SNAP (the open source software ESA has developed for its flagship satellite program).

Why SNAP? Well, apart from the obvious statement above, SNAP natively recognizes the S2 format, has loads of dedicated processing options integrated, it is quite simple and intuitive and it is free. Free to download, free to multiply, free to use, free to enhance with more custom-made tools. Aaand it uses Python and GDAL integration for heavy load processing. Of course, there are other GIS applications that will help you get the desired result, even though they don’t rely on the same processing algorithms.

As I said, this tutorial is based on open source products so there will be no ArcGIS, ENVI or ERDAS covered, although you can also follow roughly the same steps in one of those applications, with variations or help from processing wizards on the way. 

Now, for the first step, please upgrade to SNAP 6.0, which is now available as a stable version and comes with the atmospheric correction plug-in Sen2Corr already integrated (?). Ok, you’ll need to perform a couple of steps before actually using Sen2Cor, but fortunately, user ABraun (thank you!) has detailed all the necessary information here. Did I mention you need to be on the STEP forum? No? Well, you need to be there for the latest info about SNAP and Sentinel data usage. All in all, this is quite an important step, so make sure you don’t skip it unless you have already done that.

We need some S2 data. My go-to method is to look for it on the SciHub. Some useful information about logging in and downloading imagery from there I have covered in this article. For this exercise, I did not select the S2 platform but specified I wanted to be ordered by sensing date (with no date inserted) and to be an L1C product with [0 TO 10] cloud coverage. There are not a lot of results for my area, therefore I had the liberty to do this query as light as possible, but you can add as many constraints as you want. I picked a cloudless scene from the 5th of November, over Piatra Neamt (my hometown), that covers both the mountainous and the Moldavian plain areas. For diversity, you know.

scihub.PNG

After downloading, which may take a while depending on the size of your scene, open the product in SNAP (File-> Open Product-> zip archive or the XML if it is unzipped). You may want to create a quick RGB to inspect your image. Click right on your product, choose Open RGB Image Window and in the new panel leave the suggested combination of bands. Take a look at your new image, move your lookup window, identify some places or features you aim for. For example, I am planning to use much of my scene and I have already spotted some points of interest: the towns of Piatra Neamt, Suceava and Botosani, the Ceahlau Massif and the Izvorul Muntelui reservoir.

L1C products already come with some atmospheric adjustments from pre-processing steps, but a TOA (Top of Atmosphere) correction has never hurt someone, so we’ll put sen2corr to work. Here is also where you may want to return to the STEP forum thread we used before because although you may have installed the sen2corr plug-in, there are still some steps to perform before using it. Another important thing is not to alter the name of your product. Sen2Cor will return an error to that.

Next step is Select Optical -> Thematic Land Processing-> Sen2Cor. Assuming you have installed the Sen2Corr bundle and created the paths as instructed, we can now run the algorithm (you only have to do this once), you need to choose the right parameters. For the I/O just select your original product. In Processing Parameters choose the right type of aerosol (for me is RURAL), and the Mid Lat (November is rather WINTER). I have left everything else as default (they are automatically detected based on the information contained in the product auxiliary data). Pay attention to the resolution. If you want to retain your 10 m resolution on relevant bands, then select 10. Keep in mind that the corrections will be applied to the 10 m bands, but the algorithm first performs the 20 m processing and upsamples 60 m bands to this resolution. There is a lot more info about how to rightly choose your parameters you will find herehere and here. I encourage you to allow some time to read the user manuals. When you are satisfied with the settings hit run and wait, it will take a while. All the magic behind is thoroughly explained in the documents.

sen2cor.PNG

To be honest, my image was kind of perfect, I can barely see the difference after the TOA correction. A close-up pixel inspection reveals some changed values for the same pixels though.

Diferente.PNG
Left: processed using Sen2Cor. Right: the original image.

You should know Sen2Cor creates a new product so if you want to compare results, make sure to save the corrected one and open the original product. Now, for the next step, we will resample everything to 10 m. That is Raster-> Geometric Options-> Resampling. For reference use a 10 m band (B02- B04), and for the method Bilinear. For your purpose, you should first establish how much manipulation will be done afterward and choose the right parameters. You can find some answers here,  here and here.

resample

Next, I advise you to inspect your product and see if it fits your requirements. Mine does,  therefore I proceed to subsetting. Subsetting is not allowed without the previous step. Raster-> Subset in the menu. My scene is not regular, being a bit chipped in the lower right corner. For display purposes, I’ll choose a rectangular shape, but I’ll keep much of my scene intact as I want to cover my key elements. For scientific purposes, I’ll make individual subsets for each of my POIs. Right now, it is just an example, therefore I’ll define the pixels for a rectangular even shape. For pixel values, zoom in, and keep an eye on the X and Y (Lat/Long) values in the down bar (Pixel Info tab). Watch as the footprint is changing and adjust to your needs directly in the preview. If you want to perform a band subset also, choose the bands in the immediate tab. I’ll keep them all here. Save your subset product.

subset

Now, the most common thing to do is to create an RGB composite. You have done this before, but this time is for analysis. The chosen bands depend mostly on your purpose and what are willing to get from the data. It might be useful to make a parallel to Landsat 8 band combinations as the two missions are pretty close in terms of the number of wavelength intervals covered and the number of bands. Here and here I’ve included some useful info. Moreover, SNAP already has some band combinations presets, which are all very suggestively named.

I have decided to do a natural color, a shortwave infrared, and false color combination. Right-click product-> Open RGB Image Window-> Choose profile and store as a band if you need it.

 

bands.PNG
Left: False color infrared (R-B8, G-B4, B-B3). Middle: Natural Colors (R-B4, G-B3, B-B2). Right: Shortwave Infrared (R-B12, G-B8, B-B4)

 

Conclusions:

PRO 1:  SNAP comes with all the necessary tools for opening and manipulating Sentinel 2 imagery, as the Sentinel 2 toolbox and the Sen2Cor plugin have been designed and implemented specifically for this type of data.

PRO 2: The entire workflow is not difficult and even a novice can pull it in a short amount of time. 

PRO 3: The processing chain is customizable and there are loads of parameters to choose from. 

PRO 4: The results look very good and are easy to integrate after the initial steps. 

PRO 5: You will find loads of support on the STEP forum and plenty of material on ESA’s web pages. 

CON 1: Although dedicated, SNAP still has a lot of buggy material attached. There is an entire community out there to help you but sometimes is pretty frustrating to wait for bugs to be reported and solved. 

CON 2: The Sen2Cor plugin installation is a bit fuzzy and it might seem complicated to a newcomer. Also, the example of the whole setup works only with the current version of SNAP (6.0). 

CON 3: Working in SNAP’s GUI means downloading the entire scene, which is not so handy when you’ve got limited space and network resources. 

CON 4: SNAP requires a lot of processing resources and sometimes your success might be entirely dependent on your computer configuration. 

CON 5: Working with SNAP and figuring out the entire workflow might be tricky for a new user and require some prior training. (Not necessarily a CON)


PROS and CONS are based on the presented workflow only and do not take into account future analysis a user might want to perform, although some of them apply to any forthcoming actions.

#2 The advanced easy way

Making this blog post ridiculously long, I’ve decided to present you….an entire workflow in QGIS. As an experienced GIS user or remote sensing enthusiast, or someone who has barely any idea about Earth Observation or geospatial technologies, the chances to run into working with QGIS are pretty high. QGIS is an open source GIS platform, that has grown so much in the last years and it is now a viable alternative to proprietary software. Besides the great spectrum of applications, which I am not going to detail here, but you can read about on its official website, and numerous other blogs (or maybe this one here uh, I don’t know, Google has so many choices) QGIS also has attracted a great number of geospatial developers ready to put together a new useful plugin.

scp.PNG
The old SCP plug-in GUI. This has been redesigned for the 6th version, but it  works only in QGIS 3.0

This is the story of SCP (Semi-Automatic Classification Plug-in), conceived by Luca Congedo . For a great deal of info and support, make sure you follow the Facebook page and join the official group or read about it on Luca’s blog, From GIS to Remote Sensing. The plugin was launched a few years ago and has grown beautifully ever since and with the 6th version which was released this January and is compatible with QGIS 3.0 (QGIS 2.99) only, many have changed, including the interface (became more friendly) and there are a couple of interesting new features integrated now. I’m pretty sure I’ll cover this wonderful tool in other blog posts, but for now, all you need to know to get started with it can be found on Luca’s blog.

As one of the most interesting and useful tools in QGIS, this is not designed only for Sentinel 2 processing, but also for Sentinel 3, Landsat, ASTER, and MODIS. I’ll walk you through downloading, preprocessing and the creation of a band combination, just as I did in the SNAP workflow, just to show you how is to process S2 in QGIS. We will use the 5.11 version. New SCP is not so different either, maybe even friendlier, but I have encountered some problems while using it. Maybe is QGIS’s fault, maybe not.

First, install QGIS 2.18 (if you have not already). You’ll find the installer in the highlighted link (QGIS 2.18 Las Palmas). I will not detail, but everything is very straightforward, with next-next instructions. Make sure you choose the right installer for your OS version. I’m on Windows, but you may not be. Also, keep in mind QGIS works best on Linux and somehow Windows, with MacOS requiring a separate GDAL dependency framework installation (for obvious integration reasons). After install, open your QGIS 2.18 and find the Plugins tab in the main bar. Click it and go to Manage and Install Plugins. In the All tab’s search bar, look for Semi-Automatic Classification Plugin. a detailed description will appear on the right panel. Click the Install button and wait. Similarly, install Open Layers plugin.

After installation, a new tab will be displayed in the main bar- SCP and a new bar and SC Dock will be pinned on the GUI’s sides. The Dock is actually quite useful, as you can launch the plug-in from there, by just scrolling through the tabs. You’ll also find links to documentation, tutorials, and the community.

This slideshow requires JavaScript.

Go to the Web tab in the main bar, select the Open Layers plugin and bring the OpenStreetMap basemap. Zoom in to your area and adjust the bounding box. Open SCP from the SCP tab and select the Download tab. In the Search Area boxes, you will fill in with coordinates from the map. Select the “+” icon and left click on the place you consider it to be the upper corner of your search bounding box. Then right-click on the place you consider it to be the lower corner of your search bounding box. The coordinates are automatically filled for each of the two points and a yellow bounding box will be displayed on the map.

Make sure product is correctly switched to Sentinel 2, fill in the desired dates and adjust the maximum cloud coverage (remember, it was no date, and 10 % max coverage). In Download options, check only Sentinel 2 bands (to deactivate everything else, push the associated yellow card button). You can also set the maximum number of results and filter them as you wish. Click the Loupe button for Find! This may take a while, so you can make yourself a coffee, take a nap…

This slideshow requires JavaScript.

 

Once it’s done, a full list will be shown in the Product list and you’ll be able to preview each highlighted scene, check the specifications in the table, remove those unwanted and order the remaining as you wish. After choosing your scenes, turn the Download checkboxes on the bottom, and select the ones you like. I’ll let Only if preview in layers and Load bands in QGIS active. Hit Run (the green wheel button) and specify a folder for your saved product.

After a while, the product will be downloaded at the specified path. Go to Preprocessing, select the path for the directory containing the S2 bands and the metadata file. Apply DOS1 atmospheric correction for turning your radiance to reflectance, set a value for NoData (0) and specify that you want to create a new Band set. Tell the plugin where to save the new product and ….RUN! In the end  all the bands will be shown in your map window.

dos1.PNG

Skip the multitude of other options, and go to the Band Set tab. You will see that the bands in the Band list are unchecked. Keep them so, as you already have them all in the Band set definition. This is only for adding a new band into the second panel. Now, in the Band set definition, highlight all the bands from B5 to B12 and click on the “-” button. We don’t need them for a natural color combination. Sort (Highlight and up/down arrow) the bands according to the RGB order 1st- R, 2nd- G, 3rd- B. That is B4, B3, B2. Quick wavelength should be set on Sentinel 2.

bandcombo.PNG

There are 4 options on the bottom: a) Create virtual raster of band set – use if you are unsure of the result and don’t want to save, b)  Create raster of band set – useif you want to save, c) Build band overviews – creates raster pyramids for faster visualization, d) Band calc expressions – use only if you have defined any prior Band calc in the dedicated tab. I chose b) and c). There is folder location to be set and RUN to be mmm…well…run!

result

 

Your result will be displayed in the map view. It is nice, isn’t it? In a similar fashion, you can do other band combinations. Just choose the necessary bands from the Band list and add them to your Band set, order them and voila!

I’ve added a quick comparison between SNAP’s sen2corr result and the one you get in SCP, after pre-processing. Which one is better? Looking forward to hearing from you in the comments section!

Conclusions:

PRO 1: QGIS and the plugins are entirely free, easy to install, use and master in no time.

PRIO 2: You can download, pre-process and create band combinations in a single software and without surfing between different windows, platforms, etc. 

PRO 3: The plugin is masterfully executed and has a lot of useful options apart from those presented here, making it suitable for a full-length processing workflow. 

Pro 4: Has a lot of customizable options, it is very well scientifically documented and it is kept up-to-date constantly.

CON 1: The GUI for the 5.11 version is a bit crowded and may seem somehow unfriendly for the first time. This is set in the 6th version, though. 

CON 2: It does take a lot of time to search, download, process…It may depend on your network and computer capabilities.

CON 3: Saves a lot of new products into your computer, it is not storage friendly. 

CON 4: Setting the coordinates for your bounding box is not very intuitive. It may take you a bit to catch the rule. 


#3  No fuss way

Whoohoo, this article has monstrous proportions by now. Ok. 3rd way and we’re done. For this option, we will use a service provided by Sinergise’s Sentinel Hub. This is a new startup that uses the free imagery provided through Copernicus’ flagship satellite program and delivers it in a more user-friendly way. Their applications are not entirely free, but this example is based on a free QGIS plugin.

Step 1. Open QGIS (2.18 for this) and go to the Plugins tab. Search for the Sentinel Hub plugin in the All tab, select it and install it.

senitnel hub.PNG

Easy, but here comes the tricky part. After install, a new button will be added to the toolbars (a green S on a brown background) and the new plugin will be available from the Web tab (as shown below).

 scihub2.PNG

Now it should ask for a Sentinel Hub ID and you should see a large row of xs. First of all, you’ll need a free account. Don’t panic, go to this link, check Create New Account and fill in some credentials. You’ll receive them after of minutes and then you should be able to log in.

credentials.PNG

You will be redirected to an internal platform where a New configuration for a WMS instance was created. Copy and paste the code into the Sentinel Hub Instance ID box in the QGIS plugin. Optionally, click Edit in the Configurations platform and customize the layers (Name, Source, Time Range, Cloud coverage, Mosaic order, Atmospheric correction -DOS1), add, duplicate and reconfigure them, and create your WMS as you wish. The Open in Playground button lets you visualize all the effects you’ve applied to your layers. Use it before saving.

This slideshow requires JavaScript.

Next, open the Open Layers -> Open Street Map in QGIS and zoom in to your desired area. Select the EPSG, the layer type (band combination) and the date (the calendar is dynamic). Drag the cloud coverage bar. Hit Create new WMS layer. The layer will be updated in the map view if you are moving around the map, only if you push the update button!

This slideshow requires JavaScript.

You can only download regular image formats (jpg, png, tiff or raw) at their maximum resolution. Use your desired extent and specify the download folder. I chose to save it as a TIFF (as raw failed) and play a bit with the image in Adobe LR/PS afterward. An in detail tutorial can also be found here.

 

download.PNG
Get your image in the Download tab.

 

Conclusions: 

PRO 1: This is by far the fastest way to bring an already processed Sentinel 2 product right into your project. 

PRO 2: The process is pretty straightforward and although does require some odd access to their platform, the workflow is very easy and fast. 

PRO 3The Sentinel Hub configuration platform allows users to perform a lot of WMS customization and bring the layers exactly as they desire. Moreover, further controls are included in the plug-in also. 

CON 1: Well, the obvious, you can only use this as a WMS Service, and download the product as a regular image. Good for a basemap, first-hand analysis, and a nice wallpaper. Not more. 

CON 2: You are restricted to the band combinations provided by Sentinel Hub. No creativity here. 

CON 3: Only a limited amount of images are returned, you cannot choose the best scene as you wish. 

CON 4: Your account is not everlasting. It is more of a trial version and this info comes only at the end of the period. You need to pay for more. My first one expired 3 days ago and I had to create a new one. 


All in all, I believe my go-to way of pre-processing Sentinel 2 data is still SNAP, as it uses the most advanced algorithm. Results can be pushed into the SCP plugin afterward. This is also my second option because of the direct download, batch processing and plethora of tools provided. For nice wallpapers, I’d go for the Sentinel Hub though.

There are certainly many more options to handle S2 data (Amazon Web Services, QGIS workflows, ArcGIS, other open source software, Python or GDAL, plugins, RUS environment) and I will cover them in future posts, but for the sake of your time and the length of this post, I will stop here. I am also planning to do some classification tutorials on S2 data, so hang around for those as well. As for a final word, I invite you to read this nice fresh article about Copernicus and the open source environment. which is a great and unexpected summary, suitable for this blog post.

Cheers,

Cristina

Advertisements