AMT Atmospheric Measurement Techniques AMT Atmos. Meas. Tech. 1867-8548 Copernicus Publications Göttingen, Germany 10.5194/amt-6-2851-2013 Semi-autonomous sounding selection for OCO-2 Mandrake L. 1 Frankenberg C. https://orcid.org/0000-0002-0546-5857 1 O'Dell C. W. 2 Osterman G. 1 Wennberg P. https://orcid.org/0000-0002-6126-3854 3 Wunch D. https://orcid.org/0000-0002-4924-0377 3 Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA, USA Colorado State University, Fort Collins, CO, USA California Institute of Technology, Pasadena, CA, USA 25 10 2013 6 10 2851 2864 Copyright: © 2013 L. Mandrake et al. 2013 This work is licensed under the Creative Commons Attribution 3.0 Unported License. To view a copy of this licence, visit https://creativecommons.org/licenses/by/3.0/ This article is available from https://amt.copernicus.org/articles/6/2851/2013/amt-6-2851-2013.html The full text article is available as a PDF file from https://amt.copernicus.org/articles/6/2851/2013/amt-6-2851-2013.pdf

Many modern instruments generate more data than may be fully processed in a timely manner. For some atmospheric sounders, much of the raw data cannot be processed into meaningful observations due to suboptimal viewing conditions, such as the presence of clouds. Conventional solutions are quick, empirical-threshold filters hand-created by domain experts to weed out unlikely or unreasonable observations, coupled with randomized down sampling when the data volume is still too high. In this paper, we describe a method for the construction of a subsampling and ordering solution that maximizes the likelihood that a requested data subset will be usefully processed. The method can be used for any metadata-rich source and implicitly discerns informative vs. non-informative data features while still permitting user feedback into the final features selected for filter implementation. We demonstrate the method by creating a selector for the spectra of the Japanese GOSAT satellite designed to measure column averaged mixing ratios of greenhouse gases including carbon dioxide (CO<sub>2</sub>). This is done within the Atmospheric CO<sub>2</sub> Measurements from Space (ACOS) NASA project with the intention of eventual use during the early Orbiting Carbon Observatory-2 (OCO-2) mission. OCO-2 will have a 1.5 orders of magnitude larger data volume than ACOS, requiring intelligent pre-filtration.

 References Boesch, H., Backer, D., Connor, B., Crips, D., and Miller, C.: Global Characterization of CO<sub>2</sub> Column Retrievals from Shortwave-Infrared Satellite Observations of thet Orbiting Carbon Observatory-2 Mission, Remote Sens., 3, 270–304, 2011. Conner, B., Boesch, H., Toon, G., Sen, B., Miller, C., and Crisp, D.: Orbiting Carbon Observatory: Inverse method and prospective error analysis, J. Geophys. Res.-Atmos., 113, D05305, <a href="http://dx.doi.org/10.1029/2006JD008336">https://doi.org/10.1029/2006JD008336</a>, 2008. Crisp, D., Atlas, R. M., Breon, F.-M., Brown, L.R., Burrows, J. P., Ciais, P., Connor, B. J., Doney, S. C., Fung, I. Y., Jacob, D. J., Miller, C. E., O'Brien, D., Pawson, S., Randerson, J. T., Rayner, P., Salawitch, R. J., Sander, S. P., Sen, B., Stephens, G. L., Tans, P. P., Toon, G. C., Wennberg, P. O., Wofsy, S. C., Yung, Y. L., Kuang, Z., Chudasama, B., Sprague, G., Weiss, B., Pollock, R., Kenyon, D., and Schroll, S.: The Orbiting Carbon Observatory (OCO) mission, Adv. Space Res., 34, 700–709, 2004. Crisp, D., Fisher, B. M., O'Dell, C., Frankenberg, C., Basilio, R., Bösch, H., Brown, L. R., Castano, R., Connor, B., Deutscher, N. M., Eldering, A., Griffith, D., Gunson, M., Kuze, A., Mandrake, L., McDuffie, J., Messerschmidt, J., Miller, C. E., Morino, I., Natraj, V., Notholt, J., O'Brien, D. M., Oyafuso, F., Polonsky, I., Robinson, J., Salawitch, R., Sherlock, V., Smyth, M., Suto, H., Taylor, T. E., Thompson, D. R., Wennberg, P. O., Wunch, D., and Yung, Y. L.: The ACOS CO<sub>2</sub> retrieval algorithm – Part II: Global X$_CO_2$ data characterization, Atmos. Meas. Tech., 5, 687–707, <a href="http://dx.doi.org/10.5194/amt-5-687-2012">https://doi.org/10.5194/amt-5-687-2012</a>, 2012. Frankenberg, C., Platt, U., and Wagner, T.: Iterative maximum a posteriori (IMAP)-DOAS for retrieval of strongly absorbing trace gases: Model studies for CH<sub>4</sub> and CO<sub>2</sub> retrieval from near infrared spectra of SCIAMACHY onboard ENVISAT, Atmos. Chem. Phys., 5, 9–22, <a href="http://dx.doi.org/10.5194/acp-5-9-2005">https://doi.org/10.5194/acp-5-9-2005</a>, 2005. Guerlet, S., Butz, A., Schepers, D., Basu, S., Hasekamp, O. P., Kuze, A., Yokota, T., Blavier, J.-F., Deutscher, N. M., Griffith, D., W., T., Hase, F., Kyro, E., Mornio, I., Sherlock, V., Sussman, R., Galli, A., and Aben, I.: Impact of aerosol and thin cirrus on retrieving and validating X$_CO_2$ from GOSAT shortwave infrared measurements, J. Geophys. Res.-Atmos., 118, 4887–4905, <a href="http://dx.doi.org/10.1002/jgrd.50332">https://doi.org/10.1002/jgrd.50332</a>, 2013. Guyon, I. and Elisseeff, A.: An Introduction to Variable and Feature Selection, J. Machine Learn. Res., 3, 1157–1182, 2003. Jin, Y. and Sendhoff, B.: Pareto-based multi-objective machine learning: An overview and case studies, IEEE Trans. Systems, Man, and Cybernetics, Part C: Applic. Rev., 38, 397–415, 2008. O'Brien, D. M., Pollock, R., Polonsky, I., and Rogers, M.: Identification and Correction of Residual Image in the O<sub>2</sub> A-Band of the Orbiting Carbon Observatory, IEEE Trans. Geosci. Remote Sens., 49, 2426–2437, 2011. O'Dell, C. W., Connor, B., Bösch, H., O'Brien, D., Frankenberg, C., Castano, R., Christi, M., Eldering, D., Fisher, B., Gunson, M., McDuffie, J., Miller, C. E., Natraj, V., Oyafuso, F., Polonsky, I., Smyth, M., Taylor, T., Toon, G. C., Wennberg, P. O., and Wunch, D.: The ACOS CO<sub>2</sub> retrieval algorithm – Part 1: Description and validation against synthetic observations, Atmos. Meas. Tech., 5, 99–121, <a href="http://dx.doi.org/10.5194/amt-5-99-2012">https://doi.org/10.5194/amt-5-99-2012</a>, 2012. Periaux, J. and Galan, M.: Genetic Algorithms in Engineering and Computer Science, John Wiley & Son Ltd, 1995. Reuter, M., Bovensmann, H., Buchwitz, M., Burrows, J. P., Connor, B. J., Deutscher, N. M., Griffith, D. W. T., Heymann, J., Keppel-Aleks, G., Messerschmidt, J., Notholt, J., Petri, C., Robinson, J., Schneising, O., Sherlock, V., Velazco, V., Warneke, T., Wennberg, P. O., and Wunch, D.: Retrieval of atmospheric CO<sub>2</sub> with enhanced accuracy and precision from SCIA- MACHY: Validation with FTS measurements and comparison with model results, J. Geophys. Res., 116, D04301, <a href="http://dx.doi.org/10.1029/2010JD015047">https://doi.org/10.1029/2010JD015047</a>, 2011. Taylor, T. E., O'Dell, C. W., O'Brien, D. M., Kikuchi, N., Yokota, T., Nakajima, T. Y., Ishida, H., Crisp, D., and Nakajima, T.: Comparison of Cloud-Screening Methods Applied to GOSAT Near-Infrared Spectra, IEEE Trans. Geosci. Remote Sens., 50, 295–309, 2012. Wunch, D., Wennberg, P. O., Toon, G. C., Connor, B. J., Fisher, B., Osterman, G. B., Frankenberg, C., Mandrake, L., O'Dell, C., Ahonen, P., Biraud, S. C., Castano, R., Cressie, N., Crisp, D., Deutscher, N. M., Eldering, A., Fisher, M. L., Griffith, D. W. T., Gunson, M., Heikkinen, P., Keppel-Aleks, G., Kyrö, E., Lindenmaier, R., Macatangay, R., Mendonca, J., Messerschmidt, J., Miller, C. E., Morino, I., Notholt, J., Oyafuso, F. A., Rettinger, M., Robinson, J., Roehl, C. M., Salawitch, R. J., Sherlock, V., Strong, K., Sussmann, R., Tanaka, T., Thompson, D. R., Uchino, O., Warneke, T., and Wofsy, S. C.: A method for evaluating bias in global measurements of CO<sub>2</sub> total columns from space, Atmos. Chem. Phys., 11, 12317–12337, <a href="http://dx.doi.org/10.5194/acp-11-12317-2011">https://doi.org/10.5194/acp-11-12317-2011</a>, 2011. Yokota, T., Oguma, H., Morino, I., and Inoue, G.: A nadir looking SWIR FTS to monitor CO<sub>2</sub> column density for Japanese GOSAT project, Proc. Twenty-fourth Int. Sympo. on Space Technol. and Sci., JSASS and Organizing Comm. of the 24th ISTS, 887–889, 2004.