Performance assessment of state-of-the-art and novel methods for remote compliance monitoring of sulphur emissions from shipping

Beecken, Jörg; Weigelt, Andreas; Griesel, Simone; Mellqvist, Johan; Conde, Alexander Vladimir; van Dinther, Daniëlle; Duyzer, Jan; Knudsen, Jon; Knudsen, Bettina; Ntziachristos, Leonidas

doi:https://doi.org/10.5194/amt-2023-93

Preprints

https://doi.org/10.5194/amt-2023-93

Preprints

12 Jun 2023

| 12 Jun 2023

Status: a revised version of this preprint was accepted for the journal AMT and is expected to appear here in due course.

Performance assessment of state-of-the-art and novel methods for remote compliance monitoring of sulphur emissions from shipping

Jörg Beecken, Andreas Weigelt, Simone Griesel, Johan Mellqvist, Alexander Vladimir Conde, Daniëlle van Dinther, Jan Duyzer, Jon Knudsen, Bettina Knudsen, and Leonidas Ntziachristos

Abstract. Sulphur emissions to the air from sea-going and inland vessels were measured simultaneously by eight different, state-of-the-art and novel, monitoring systems during a six-week campaign at the Elbe River, at about 10 km distance to the port of Hamburg, Germany. Both, stationary, and airborne systems on unmanned aerial vehicles (UAV) were operated by four participating partners in a side-by-side measurement setup to observe the same emission sources under similar conditions. A novel laser spectrometer with significantly better precision specifications as compared to the other instruments was used for the first time for emission monitoring regarding MARPOL Annex VI regulations.

The comparison took place in the Northern European Sulphur Emission Control Area (SECA) where the allowed Fuel Sulphur Content (FSC) is limited to 0.10 %S_m/m. In total, 966 plumes that originated from 436 different vessels were analysed in this study. At the same time, fuel samples, obtained from 34 different vessels, and bunker delivery notes (BDN) from five frequently monitored vessels were used as references to assess the uncertainties of the different systems. Most measurement systems tended to underestimate the FSC found from fuel samples and BDNs. A possible relation was seen to high relative humidities above 80 %. The lowest systematic deviations were observed for the airborne systems and the novel laser spectrometer. The latter showed the lowest total uncertainty of 0.05 %S_m/m (confidence level: 95 %) compared to other stationary sniffer systems whose total uncertainties range from 0.08 to 0.09 %S_m/m. The two UAV-borne systems showed total uncertainties of 0.07 and 0.09 %S_m/m, respectively. Overall, it was found that non-compliant vessels with an actual FSC of the combusted fuel above 0.15 to 0.19 %S_m/m can be detected by the compared systems with 95 % confidence.

Received: 26 Apr 2023 – Discussion started: 12 Jun 2023

Download & links

Jörg Beecken et al.

Status: closed

RC1:
'Comment on amt-2023-93', Anonymous Referee #1, 27 Jun 2023
General comments
Beecken et al. describes the results of a measurement campaign undertaken to evaluate the performance of different instruments for the monitoring of pollutants emitted by ships. The manuscript focuses on SO_x emissions and compares the results of the different instruments and measurement principles. The importance of proper calibration is highlighted. Generally, the manuscript is well written and fits into the scope of AMT. It can be accepted for publication after some minor clarifications.
Sect 2: Is the evaluation process the same for all instruments? Where the same evaluation programs used or where there at least common settings (for example for peak detection and ship assignment to a plume)? A brief summary of the steps of the evaluation process might be beneficial.

Sect 2.1: Here the calculation of the fuel sulphur content is explained. I feel there should be a brief description of how the baseline is defined for SO₂ and CO₂. Are the baselines defined in the same way for all instruments? Are there any additional sources next to ships that could cause enhancements in SO₂ and CO₂ at the same time?

Sect 2.5: Perhaps this section should be moved to the beginning of section 2, before the detailed description of the instruments, data evaluation and description of the uncertainties.

Sect 3.2: Some of the instruments are specifically used by the BSH to monitor ship emissions. Was the negative bias already known and is this usually corrected for? Is it relevant for the identification of non-compliant ships?

Specific comments
Line 146: How are these sweet spots identified, how long does it take to find a good position and how long does the UAV need to be in this position for an accurate measurement? Are the results significantly different for measurements outside the sweet spot?

Line 177: Any reason why the units are replaced after 100 hours or 1 year after their production? Is regular calibration not good enough?

Line 253: The stationary instruments were set up right next to each other and should always see the same plumes, but it seems that 25% of the detected plumes were only detected by one of the instruments. Any explanations on this?

Line 258: Is the deviation a known issue for SO₂ calibration gas and is this regularly tested for?

Line 260: What kind of correction was applied and how were possible affected data identified?

Line 320: Was there a specific reason the UAVs measured in different distances?

Technical suggestions
Add NOx = NO + NO2 where it first appeared in the introduction.
Citation: https://doi.org/10.5194/amt-2023-93-RC1
- AC1: 'Reply on RC1', Jörg Beecken, 08 Sep 2023
  
  The comment was uploaded in the form of a supplement: https://amt.copernicus.org/preprints/amt-2023-93/amt-2023-93-AC1-supplement.pdf
  
  Citation: https://doi.org/10.5194/amt-2023-93-AC1
RC2:
'Comment on amt-2023-93', Ward Van Roy, 01 Jul 2023

General comments:

The manuscript is well written and includes valuable information. The research questions are well defined and the methods are well worked and properly referenced. The results are clearly presented and will enhance the scientific knowledge in the field. Some references to the international regulations (IMO, MEPC) could be added and legal wording can be improved. Make sure to use always the latest regulations or consolidated versions in the references to international regulations and conventions. When using legal references avoid mentioning the pages and avoid duplicating the year (at the end) as this has no added value (e.g. 11.5.1999, p. 13–18 (EN), 1999. this can better be mentioned as: adopted on 11 May 1999.).

Reference and comparison to previous studies and literature is somehow lacking. The number of references is relatively short. In the discussion some considerations could have been made on NOx regulations. As all remote monitoring groups measured NOx as well, it would be interesting to see the intercomparison of the NOx measurement results.

The importance of the results for operational compliance monitoring/enforcement could be more elaborated. How did the research contribute to the enforcement practices in the field? What are the lessons learned for the compliance monitoring organizations. Will reporting thresholds be lowered … ?

The evaluation is now limited to the analysis of the absolute differences between the measurements and the BDN/Fuel samples. As this will work fine for the range 0-0.1% FSC. This is less useful for high measurements e.g. non-compliant vessels or measurements outside the ECA. An elaboration of the proportional bias beside the absolute bias would therefore be interesting. If the authors consider that this is outside the scope of this research, the authors should mention this.
Detailed comments:
15. I Would suggest to use “seagoing” (withthout “-“) vessels or use “ocean going vessels”

18. suggest rephrasing “observe the same emission sources under similar conditions” as measurements were conducted to measure the emission factors, so I would suggest to mention that.

21. Would suggest to stick to the official term “North Sea Sulphur Emission Control Area”

24 “reference” without “s”

24. Specify “most”

26. would suggest to split up “the lowest systematic deviation was observed for the airborne system.. having a deviation of … %”. “..The lowest the total uncertainty was observed for the laser …”

30. Maybe make a reference to the formerly established thresholds at 95% CI (e.g. 0.2%FSC with 95% CI) to demonstrate the advancement.

32. The correct reference to MARPOL is: “International Convention for the Prevention of Pollution from Ships”

Add reference:

International Convention for the Prevention of Pollution from Ships 1973, as modified by the Protocol of 1978 relating thereto (adopted 17 February 1978 (MARPOL), in force 2 October 1983) 1340 UNTS 61, as amended

35. Suggestion to add Regulation nrs (Reg. 13 and Reg. 14) here.

39. Suggestion to add guidelines on scrubbers:

2021 Guidelines for Exhaust Gas Cleaning Systems, Resolution MEPC.340(77), Adopted on 26 November 2021

41. replace “implemented” by “entered into force” as this are 2 different things. Add a reference:

Protocol of 1997 to amend the International Convention for the Prevention of Pollution from Ships of 2 November 1973, as modified by the Protocol of 17 February 1978, London, 1997, into force on 19 May 2005 and amended

43. Make reference to the revised Sulphur Directive of 2016:

Directive (EU) 2016/802 of the European Parliament and of the Council of 11 May 2016 relating to a reduction in the sulphur content of certain liquid fuels, (codification), OJ L 132/58, 21.05.2016

45. Use the correct legal wording “..North Sea SECA came in effect on 22 November 2007 …” “..Baltic Sea SECA came in effect on 19 May 2006 ..” + add reference

List of Special Areas, Emission Control Areas and Particularly Sensitive Sea Areas, Circular MEPC.1/Circ.778/Rev.3 of 2 July 2008

52. Or quick sampling methods (e.g. XRF)

70. Specify “near shore” as airplanes can also operate near shore as proven by the BE measurements inside the Westerschelde and as close to the ports as 200m. Suggestion to skip this sentence as it implies that airplanes cannot operate near shore.

73. Use SCIPPER abbreviation after the written out acronym instead of before.

148. Provide the manufacturer and model of the EC sensors of the UAS systems like was done for the stationary systems.

149. Ambient air quality laboratory use more stringent calibration methods, why are these not applied here? Not only the sensors need to be calibrated but the VMRs inside the cylinders need to be checked too. Without this step the calibrations risk to be erroneous.

224. However kn is also correct the use of kts is more common in the maritime sector

257. Cfr comment 149, issue with unreliable VMRs of ordered gasses is a bit lost here, suggestion to move this to 149. However the data correction for the measurements that were conducted using the sensors calibrated using the wrong VMR reference ratios should be elaborated here.

264. Please elaborate on possibility that FSC could potentially change between measurement and sampling. My opinion is that for most cases no substantial change is expected, but it can not ruled out, certainly when vessels have been doing fuel change over procedures. A potential may to exclude this potential error, was to limit the analyses to vessels with only 1 fuel type on board.

290. The graph is interesting, however, not that relevant for operational use. What is important is not the reported uncertainty, but the maximum allowable difference. For measurements under the non-compliance threshold the reported uncertainty and maximum allowable difference can be the same, however as soon as the non-compliance reporting threshold is reached, the reported uncertainty is irrelevant for operational reasons, but its rather the difference between the FSC measurement and the FSC limit which must be examined, to avoid type II errors where compliant vessels are assigned as non-compliant.

Please also refer to the comparison analysis made by airborne measurement and EGCS/fuel samples

https://www.mdpi.com/2073-4433/14/4/623

300. Reference could be made to the negative bias correction by Van Roy et al 2022 b which observed similar negative biases.

357. The over NO correction could also be a potential cause for the negative bias. Not that Van Roy et al. b use lower NO correction factors compared to Chalmers.

330. Note that Van Roy et al. evaluated RH and T impact (https://doi.org/10.3390/atmos14040623), see supplementary material.

420. Round robin test are first mentioned here, they should have been mentioned before in the discussion on how to correct negative biases. Suggestion to add reference: https://www.mdpi.com/2073-4433/14/6/969

Citation: https://doi.org/10.5194/amt-2023-93-RC2
- AC2: 'Reply on RC2', Jörg Beecken, 08 Sep 2023
  
  The comment was uploaded in the form of a supplement: https://amt.copernicus.org/preprints/amt-2023-93/amt-2023-93-AC2-supplement.pdf
  
  Citation: https://doi.org/10.5194/amt-2023-93-AC2

Status: closed

RC1:
'Comment on amt-2023-93', Anonymous Referee #1, 27 Jun 2023
General comments
Beecken et al. describes the results of a measurement campaign undertaken to evaluate the performance of different instruments for the monitoring of pollutants emitted by ships. The manuscript focuses on SO_x emissions and compares the results of the different instruments and measurement principles. The importance of proper calibration is highlighted. Generally, the manuscript is well written and fits into the scope of AMT. It can be accepted for publication after some minor clarifications.
Sect 2: Is the evaluation process the same for all instruments? Where the same evaluation programs used or where there at least common settings (for example for peak detection and ship assignment to a plume)? A brief summary of the steps of the evaluation process might be beneficial.

Sect 2.1: Here the calculation of the fuel sulphur content is explained. I feel there should be a brief description of how the baseline is defined for SO₂ and CO₂. Are the baselines defined in the same way for all instruments? Are there any additional sources next to ships that could cause enhancements in SO₂ and CO₂ at the same time?

Sect 2.5: Perhaps this section should be moved to the beginning of section 2, before the detailed description of the instruments, data evaluation and description of the uncertainties.

Sect 3.2: Some of the instruments are specifically used by the BSH to monitor ship emissions. Was the negative bias already known and is this usually corrected for? Is it relevant for the identification of non-compliant ships?

Specific comments
Line 146: How are these sweet spots identified, how long does it take to find a good position and how long does the UAV need to be in this position for an accurate measurement? Are the results significantly different for measurements outside the sweet spot?

Line 177: Any reason why the units are replaced after 100 hours or 1 year after their production? Is regular calibration not good enough?

Line 253: The stationary instruments were set up right next to each other and should always see the same plumes, but it seems that 25% of the detected plumes were only detected by one of the instruments. Any explanations on this?

Line 258: Is the deviation a known issue for SO₂ calibration gas and is this regularly tested for?

Line 260: What kind of correction was applied and how were possible affected data identified?

Line 320: Was there a specific reason the UAVs measured in different distances?

Technical suggestions
Add NOx = NO + NO2 where it first appeared in the introduction.
Citation: https://doi.org/10.5194/amt-2023-93-RC1
- AC1: 'Reply on RC1', Jörg Beecken, 08 Sep 2023
  
  The comment was uploaded in the form of a supplement: https://amt.copernicus.org/preprints/amt-2023-93/amt-2023-93-AC1-supplement.pdf
  
  Citation: https://doi.org/10.5194/amt-2023-93-AC1
RC2:
'Comment on amt-2023-93', Ward Van Roy, 01 Jul 2023

General comments:

The manuscript is well written and includes valuable information. The research questions are well defined and the methods are well worked and properly referenced. The results are clearly presented and will enhance the scientific knowledge in the field. Some references to the international regulations (IMO, MEPC) could be added and legal wording can be improved. Make sure to use always the latest regulations or consolidated versions in the references to international regulations and conventions. When using legal references avoid mentioning the pages and avoid duplicating the year (at the end) as this has no added value (e.g. 11.5.1999, p. 13–18 (EN), 1999. this can better be mentioned as: adopted on 11 May 1999.).

Reference and comparison to previous studies and literature is somehow lacking. The number of references is relatively short. In the discussion some considerations could have been made on NOx regulations. As all remote monitoring groups measured NOx as well, it would be interesting to see the intercomparison of the NOx measurement results.

The importance of the results for operational compliance monitoring/enforcement could be more elaborated. How did the research contribute to the enforcement practices in the field? What are the lessons learned for the compliance monitoring organizations. Will reporting thresholds be lowered … ?

The evaluation is now limited to the analysis of the absolute differences between the measurements and the BDN/Fuel samples. As this will work fine for the range 0-0.1% FSC. This is less useful for high measurements e.g. non-compliant vessels or measurements outside the ECA. An elaboration of the proportional bias beside the absolute bias would therefore be interesting. If the authors consider that this is outside the scope of this research, the authors should mention this.
Detailed comments:
15. I Would suggest to use “seagoing” (withthout “-“) vessels or use “ocean going vessels”

18. suggest rephrasing “observe the same emission sources under similar conditions” as measurements were conducted to measure the emission factors, so I would suggest to mention that.

21. Would suggest to stick to the official term “North Sea Sulphur Emission Control Area”

24 “reference” without “s”

24. Specify “most”

26. would suggest to split up “the lowest systematic deviation was observed for the airborne system.. having a deviation of … %”. “..The lowest the total uncertainty was observed for the laser …”

30. Maybe make a reference to the formerly established thresholds at 95% CI (e.g. 0.2%FSC with 95% CI) to demonstrate the advancement.

32. The correct reference to MARPOL is: “International Convention for the Prevention of Pollution from Ships”

Add reference:

International Convention for the Prevention of Pollution from Ships 1973, as modified by the Protocol of 1978 relating thereto (adopted 17 February 1978 (MARPOL), in force 2 October 1983) 1340 UNTS 61, as amended

35. Suggestion to add Regulation nrs (Reg. 13 and Reg. 14) here.

39. Suggestion to add guidelines on scrubbers:

2021 Guidelines for Exhaust Gas Cleaning Systems, Resolution MEPC.340(77), Adopted on 26 November 2021

41. replace “implemented” by “entered into force” as this are 2 different things. Add a reference:

Protocol of 1997 to amend the International Convention for the Prevention of Pollution from Ships of 2 November 1973, as modified by the Protocol of 17 February 1978, London, 1997, into force on 19 May 2005 and amended

43. Make reference to the revised Sulphur Directive of 2016:

Directive (EU) 2016/802 of the European Parliament and of the Council of 11 May 2016 relating to a reduction in the sulphur content of certain liquid fuels, (codification), OJ L 132/58, 21.05.2016

45. Use the correct legal wording “..North Sea SECA came in effect on 22 November 2007 …” “..Baltic Sea SECA came in effect on 19 May 2006 ..” + add reference

List of Special Areas, Emission Control Areas and Particularly Sensitive Sea Areas, Circular MEPC.1/Circ.778/Rev.3 of 2 July 2008

52. Or quick sampling methods (e.g. XRF)

70. Specify “near shore” as airplanes can also operate near shore as proven by the BE measurements inside the Westerschelde and as close to the ports as 200m. Suggestion to skip this sentence as it implies that airplanes cannot operate near shore.

73. Use SCIPPER abbreviation after the written out acronym instead of before.

148. Provide the manufacturer and model of the EC sensors of the UAS systems like was done for the stationary systems.

149. Ambient air quality laboratory use more stringent calibration methods, why are these not applied here? Not only the sensors need to be calibrated but the VMRs inside the cylinders need to be checked too. Without this step the calibrations risk to be erroneous.

224. However kn is also correct the use of kts is more common in the maritime sector

257. Cfr comment 149, issue with unreliable VMRs of ordered gasses is a bit lost here, suggestion to move this to 149. However the data correction for the measurements that were conducted using the sensors calibrated using the wrong VMR reference ratios should be elaborated here.

264. Please elaborate on possibility that FSC could potentially change between measurement and sampling. My opinion is that for most cases no substantial change is expected, but it can not ruled out, certainly when vessels have been doing fuel change over procedures. A potential may to exclude this potential error, was to limit the analyses to vessels with only 1 fuel type on board.

290. The graph is interesting, however, not that relevant for operational use. What is important is not the reported uncertainty, but the maximum allowable difference. For measurements under the non-compliance threshold the reported uncertainty and maximum allowable difference can be the same, however as soon as the non-compliance reporting threshold is reached, the reported uncertainty is irrelevant for operational reasons, but its rather the difference between the FSC measurement and the FSC limit which must be examined, to avoid type II errors where compliant vessels are assigned as non-compliant.

Please also refer to the comparison analysis made by airborne measurement and EGCS/fuel samples

https://www.mdpi.com/2073-4433/14/4/623

300. Reference could be made to the negative bias correction by Van Roy et al 2022 b which observed similar negative biases.

357. The over NO correction could also be a potential cause for the negative bias. Not that Van Roy et al. b use lower NO correction factors compared to Chalmers.

330. Note that Van Roy et al. evaluated RH and T impact (https://doi.org/10.3390/atmos14040623), see supplementary material.

420. Round robin test are first mentioned here, they should have been mentioned before in the discussion on how to correct negative biases. Suggestion to add reference: https://www.mdpi.com/2073-4433/14/6/969

Citation: https://doi.org/10.5194/amt-2023-93-RC2
- AC2: 'Reply on RC2', Jörg Beecken, 08 Sep 2023
  
  The comment was uploaded in the form of a supplement: https://amt.copernicus.org/preprints/amt-2023-93/amt-2023-93-AC2-supplement.pdf
  
  Citation: https://doi.org/10.5194/amt-2023-93-AC2

Jörg Beecken et al.

Viewed

Total article views: 381 (including HTML, PDF, and XML)

HTML	PDF	XML	Total	BibTeX	EndNote
265	96	20	381	14	13

HTML: 265
PDF: 96
XML: 20
Total: 381
BibTeX: 14
EndNote: 13

Views and downloads (calculated since 12 Jun 2023)

Month	HTML	PDF	XML	Total
Jun 2023	138	32	5	175
Jul 2023	51	23	6	80
Aug 2023	17	8	1	26
Sep 2023	37	20	8	65
Oct 2023	16	9	0	25
Nov 2023	6	4	0	10

Cumulative views and downloads (calculated since 12 Jun 2023)

Month	HTML	PDF	XML	Total
Jun 2023	138	32	5	175
Jul 2023	51	23	6	80
Aug 2023	17	8	1	26
Sep 2023	37	20	8	65
Oct 2023	16	9	0	25
Nov 2023	6	4	0	10

Viewed (geographical distribution)

Total article views: 375 (including HTML, PDF, and XML) Thereof 375 with geography defined and 0 with unknown origin.

Country	#	Views	%

Latest update: 07 Nov 2023

Short summary

Air pollution from shipping is a debated topic in science and politics. We compare differenent monitoring systems used currently in different European countries for the enforcement of emission limits regarding air pollution from ships according to regulation. The system performances were individually assessed in the field by comparison to true values. Non-compliant vessels with actual fuel sulphur contents >0.15–0.19 % can be detected by the compared systems with 95 % confidence.


Total:	0
HTML:	0
PDF:	0
XML:	0