The Chang'E 1 and 2 Microwave Radiometer Processed Data are now available in the Planetary Data System (PDS) archives. This bundle contains Planetary Data System 4 (PDS4) versions of calibrated (L2C) data products along with new products derived from those data. This substantial recalibration and recovery effort--- performed my Million Concepts in collaboration with Matt Siegler and Jianqing Feng (PSI)--- helped support the discovery of a major granitic batholic on the Moon. The bundle documentation is well-written and worth a read.
A paper on the Planetary Data Reader (pdr)
has been published in the
Journal of Open Source Software (JOSS).
Chase Million (CEO) attended the "Software for the NASA Science Mission Directorate Workshop 2024" at NASA HQ in Washington, DC. The workshop "aim[ed] to explore the current opportunities and challenges for the various categories and lifecycle stages of software that are relevant for activities funded by the NASA Science Mission Directorate (SMD). This includes increasing the visibility of software being developed and igniting wider collaborations on those challenges between centers, missions, data repositories, and sciences." Several Million Concepts staff members gave multiple lightning talk presentations at the workshop, on the topics of the Planetary Data Reader (S. Curtis), the hostess library for remote resource management and execution (M. St. Clair), the marslab ecosystem tools for spectral image data processing (C. Million), and the Knowing the Sky open science training materials (S. Brown).
Dr. Michael St. Clair (CTO) has released quickseries, a library of Python functions that
perform fast vectorized power series approximations of mathematical functions. It can provide performance
improvements ranging from ~3x (simple functions, no fiddling around with parameters) to ~100x (complicated
functions, some parameter tuning). It is available on conda-forge, and documentation and source code are
available on Github.
Chase Million (CEO), Dr. Michael St. Clair (CTO), and Dr. Sierra Brown (RSE, Staff Scientist) attended the 2024 PyCon US conference in Pittsburgh PA. Along with connecting with colleagues and old friends, including from pyOpenSci and OpenTeams, they hosted an "open space" on the topic of space: a space open space.
Dr. Sierra Brown (Staff Scientist and RSE) has been selected as a member of the first cohort of NASA's Space Tech Catalyst Prize. This initiative intends to "expand engagement with underrepresented and diverse individuals in the space technology sector. [...] Winning individuals and organizations demonstrate the best collaboration practices with diverse researchers, technologists, and entrepreneurs. The champions also bring effective strategies that contribute to NASA’s ongoing efforts to develop a representative space technology landscape, while enhancing its ability to find creative solutions to technical challenges."
Million Concepts has released a new Python library named
hostess that provides lightweight, Pythonic interfaces
to distributed resources, including AWS. We have been developing and using hostess as internal tooling for over
two years, and it dramatically improves our time-to-value for tasks ranging from web app deployment to processing
petabytes of data. A non-exhaustive list of things that hostess makes easy to do right in your Python script or
Jupyter Notebook:
hostess is available on conda-forge. Tutorial Notebooks and source code are
available on Github, and
documentation is available on ReadTheDocs.
The Planetary Data Reader (pdr) has been accepted as an affiliate package to PlanetaryPy! The PlanetaryPy project
aims to create a centralized library of general use tools for planetary science, analogous to what astropy has
done in Astronomy. pdr is a Python tool that is capable of reading the vast majority of the petabytes of
diverse, archival planetary science data spanning half a century—from Apollo, Voyager, and Viking to Mars 2020
and DART—into standard Python workflows.
pdr is available on conda-forge and Github.
Dr. Sierra Brown was chosen to participate in a NASA UNBOUND workshop titled "Broadening the use of & access to NASA data and tools for tribal food, water, and energy priorities."
"Spectral Variability of Rocks and Soils on the Jezero rater Floor: A Summary of Multispectral Observations from Perseverances's Mastcam-Z Instrument" (Rice, et al.) has been published in JGR Planets. This paper presents analysis of multispectral data from Mastcam-Z from the first 380 sols of the Perseverance mission, including the definition of eight spectral classes of rocks. This work relied extensively on the marslab family of multispectral analysis tools developed by Million Concepts.
Chase Million (CEO) has been added to the advisory council for pyOpenSci, a non-profit organization dedicated to enhancing the practice and quality of open science.
"The GFCAT: A Catalog of Ultraviolet Variables Observed by GALEX with Subminute Resolution" (Million, et al.) has been published in the The Astrophysical Journal (ApJ). This paper describes a catalog of 1426 ultraviolet variable sources, including stellar flares, eclipsing binaries, δ Scuti and RR Lyrae variables, and active galactic nuclei. Many of these sources have never previously been identified as variable.
“Remote Detection of a Lunar Granitic Batholith at Compton-Belkovich.” (Siegler, et al.) has been published in Nature. This paper reports strong evidence for a massive granitic system on the Moon—far larger than previously believed possible—using data derived from the Chang'e-2 microwave radiometer. Dr. Michael St. Clair and Chase Million are coauthors on this work; they contributed substantially to data reduction methods and software in support of this investigation.
The entire Million Concepts development staff attended the 2023 Planetary Data Workshop (PDW) in Flagstaff, AZ.
Dr. Sierra Brown gave a talk on the Planetary Data Reader (PDR) project named
"Transforming Research Workflows with
the Planetary Data Reader: Accessing PDS Data in Python with Ease." Dr. Michael St. Clair presented a poster
named "Streamlining PDS3 to PDS4 Conversion:
Leveraging the Planetary Data Reader for Improved Efficiency: a Case Study with Viking Data" and gave a talk
named "Automated Spectral Image Processing
Techniques in the marslab Family of Applications."
Following the workshop, Million Concepts held a company retreat, which included a tour of the USGS Astrogeology archives and a day trip to the Grand Canyon.
Million Concepts has been selected to create an astronomy and planetary science focused ScienceCore curriculum for NASA’s Transform to Open Science Training (TOPST) Initiative . Sierra Brown (Staff Scientist/RSE) will lead the project entitled “Knowing the Sky: Building Open Science Skills through Native Knowledge Practices”. This effort will create a narrative-driven, interactive Jupyter Book that will serve as a self-paced course in open science data, tools, and techniques targeted towards learners ranging from interested laypeople to experienced scientists. The narrative components of the course will lead learners through engaging hands-on investigations of Native knowledge concepts relating to astronomy and planetary science.
We have released "antiscope," a Python-language library that provides structures for irrealis programming. Irrealis programming permits evaluation of code in which some objects are undefined or incompletely defined. The program will run as if those objects were well-defined. (You could say that these objects are in a subjunctive mood, or that they are counterfactual, or that they are defined using possible-worlds semantics.)
Antiscope is a powerful new programming paradigm disguised as a joke. Its source code can be found here. An introductory / description document with many worked examples can be found here.
Chase Million gave a webinar on “Effective Strategies for Writing Proposal Work Plans for Research Software” to the Computational Infrastructure for Geodynamics (CIG) group.
A paper leveraging Million Concepts work on the GALEX data archive has been published in Monthly Notices of the Royal Astronomical Society, testing the accuracy of ultraviolet flare rate models by comparing them to observations from TESS. “Extending optical flare models to the UV: results form comparing of TESS and GALEX flare observations for M Dwarfs.” Jackman, et al. MNRAS
Million Concepts will be participating in the Volatiles Investigating Polar Exploration Rover (VIPER) mission to explore permanently shadowed regions of the Lunar south pole. MC will assist with production of the ground data processing pipeline for VIPER’s Visible Imaging System (VIS) under the direction of Ross Beyer (SETI).
Million Concepts has just completed a contract with the USGS to update the archived data from the Viking Orbiter 1 and 2 Visual Imaging Subsystems (VIS) to Planetary Data System v4 (PDS4) standards. The resulting bundle contains accessible, PDS4-compliant versions of all observational data in the PDS Imaging Node's VO holdings---much of which was previously incompatible with any modern software. We have delivered it to the PDS, where it is in review prior to inclusion in the archive. See the bundle documentation here for an overview of the process.
Rebekah Albach (RSE and Staff Scientist) and Katherine Szymborski (Intern) both presented posters at the 241st meeting of the American Astronomical Societ (AAS) in Seattle, WA. Szymborski’s poster, “Flares on the M-dwarf AF Psc as seen by TESS,” gave an overview of analysis of data from short-cadence observations of AF Psc observed as part of a Guest Investigator program (PI: Million). Abach presented “Multi-tiered Approach to Point and Extended Source Extraction in GALEX UV Images,” describing work in progress on generating improved photometric extractions for the GALEX Legacy Catalog effort.
The Mastcam-Z team has just published a major summary analysis of the data taken by Mastcam-Z during the Perseverance rover's first (Earth calendar) year on Mars. We are pleased to have created many of the tools that made these analyses possible. “Geological, multispectral, and meteorological imaging results from the Mars 2020 Perseverance rover in Jezero crater.” Bell, et al. Science Advances
Analysis of the Chang'e-2 data has yielded some surprising results, including a massive ancient batholith on the far side of the Moon we believe indicates the presence of much larger and more widespread lunar volcanic structures than previously believed. A paper is in process at Nature, and a preprint is available here: “Remote Detection of a Lunar Granitic Batholith at Compton-Belkovich.” Siegler, et al.
We have recently completed a project with the Space Telescope Science Institute (STScI). They contracted us to perform fundamental research into cloud-based data access and cost optimization in support of the FORNAX project (an initiative for improved data provision and increased interoperability between astronomical data archives). A summary of some of our findings is available here.
Chase Million delivered a presentation on “Effective strategies for writing proposal work plans for research software” for the Exascale Computing Project (ECP) IDEAS Productivity project. This is a culmination of his work as a Better Scientific Software (BSSw) fellow, and an expansion and refinement of his earlier talk on the same topic.
Our MultiDEx multispectral analysis software---and the marslab suite
that backs it---has enabled the production of an authoritative study of MSL Mastcam's
decade of multispectral observations:
“Spectral diversity of rocks and soils in Mastcam observations along the Curiosity
rover’s traverse in Gale crater, Mars.” Rice, et al. JGR Planets
Chase Million gave a presentation at the Planetary Science Informatics and Data Analytics (PSIDA) conference at the European Space Astronomy (ESA) Centre in Madrid, Spain. The talk covered the wide range of software for planetary science produced by Million Concepts, featuring most prominently the Planetary Data Reader.
“A practical guide to research software project estimation,” by Chase Million, is now available. This summary / tutorial document describes a powerful and effective approach to estimating the required work effort and costs of research efforts with significant software components. This is a primary deliverable of Million’s Better Scientific Software (BSSw) fellowship.
Million Concepts staff presented several posters at the 53rd Lunar and
Planetary Science Conference (LPSC) in The Woodlands, TX. These were:
•
“How We’ll Know We Can read All the Data in the PDS: A Testing Methodology
for the Planetary Data Reader!!!” Kaufman (now Brown), et al.
•
“Approaches to Production of Intermediate Data Products for Characterizing
Systemic Anomalies in the Chang’e-2 Microwave Radiometer Data.” St. Clair, et al.
•
“Software tools for rapid analysis of Mastcam-Z multispectral data.” Million, et al.
A paper tittled "New Time-Resolved, Multi-Band Flares In The GJ 65 System With gPhoton" has been accepted for publication the Astrophysical Journal and uploaded to arXiv. Coauthored by Chase Million and part of the GFCAT and gPhoton projects on which he is PI, this paper describes measurements of flares on the closest and brightest active flare star(s) observed by GALEX, which therefore represent the best opportunities to observe time-resolved flares at low energies in ultraviolet bands within the GALEX data corpus.
Chase Million was interviewed for the Research Software Engineer Stories podcast about his career path including the founding and growth of Million Concepts.
pdr (the Planetary Data Reader). It includes
improved performance, broadened compatibility, a host of convenience features, and a new test
framework. Try:data = pdr.read(planetary_data_file.img); data.dump_browse()pdr soon.
Check out the new version on GitHub.
lhorizonOur ephemeris package lhorizon
has been published in JOSS.
lhorizon helps you find things in the Solar System. It is built around
a thick Python wrapper for the JPL Horizons service. It parses Horizons' responses
into standard Python objects and smoothly incorporates them into bulk calculation and
transformation workflows. It is designed for ease of use, speed, and flexibility. We believe
that it is simpler and more performant for many use cases than existing alternatives. For
a quick overview of functionality, you can
browse the GitHub repository
or go directly
to these example Jupyter Notebooks on Binder.
marslab demodustgogglesmarslabWe have released an alpha version of a general-purpose
scientific library called marslab. We wrote
marslab primarily to help fill gaps in our
Python-based workflows for multispectral imaging data collected
by Mars rovers, but it contains utilities that might be useful
in a much wider range of contexts, including:
marslab in the near future.A TESS Guest Investigator project led by Chase Million has been selected for inclusion in Cycle 4. "Using AF Psc To Test Flare Energy Partitions With TESS And Swift" will observe the active flare star AF Psc in TESS 20-second cadence mode with several hours of coordinated simultaneous observations in Swift's U-band event mode and XRT. We expect to observe at least one large flare across multiple electromagnetic regimes at high time resolution, providing a rare measurement of flare energy partitions and temporal evolution. Fast quasi-periodic pulsations (QPP) have previously been detected on flares in UV GALEX observations of AF Psc [Doyle et al. 2018] and in TESS observations of other stars [Million et al. 2021], so there is a possibility of also capturing the first multi-band, fast cadence observation of a QPP on a star other than the Sun. Scott Fleming (Space Telescope Science Institute), Isaiah Tristan (University of Colorado), and Adam Kowalski (University of Colorado) are collaborators on the effort.
pdr) PDARTNASA's Planetary Data Archiving, Restoration, and Tools
(PDART) program has selected Million Concepts' proposal to
fully develop the Planetary Data
Reader (pdr). This Python-based software tool
will provide an easy, well-documented, thoroughly-tested,
single-point solution for reading the enormous quantity -- over
two petabytes -- of observational planetary science data
currently stored in the Planetary Data System. The project PI
is Chase Million. Michael St. Clair (Million Concepts) and
Michael Aye (LASP) are co-investigators. Jordan Padams (PDS ENG
Node, JPL) is a collaborator.
Michael St. Clair led a poster presentation at the 52nd
Lunar and Planetary Science Conference, describing the
process and technology around our recent
conversion of the Chandrayaan-1 M3 from PDS3 to PDS4
format.
You
can read the poster here.
Chase Million led a poster presentation at the Cool
Stars 20.5 conference, presenting the first discovery of
fast (< 5 minute) quasi-periodic pulsations in the TESS
20-second mode data. These measurements were made on
observations of nearby M Dwarfs that were requested as part of
a GI proposal, also led by Million.
You can read the
poster here.
picks: occasional imaging utilitiesWe have released a small collection of web-based utilities that may be useful to people attempting to work with imaging data like those provided by the Mars 2020 Raw Image site. These include the capability to split and combine three-channel images and to decompose RGGB bayer pattern images into their component filters.
We have released a beta version of pdr, the
Planetary Data Reader. pdr has the ambitious goal
of making every file and product in the complex, diverse
ecology of planetary data instantly ready for analysis in
standard Python workflows. It already supports an overwhelming
majority of the data hosted in the Planetary Data System (PDS)
archives. It
is available on Github. Try pdr.read(filename)
and see what happens. We'd love your comments, requests, and
bug reports.
Under contract from the United States Geological Survey, Million Concepts has produced Planetary Data System 4 (PDS4) bundles containing new versions of:
Our new versions of these crucial lunar data are highly usable and designed to be easily-maintainable for many more decades.
The data are still in peer review at the PDS. See our project page for links to documentation, software, and videos that show the glitchy delights of the Clementine EDR.
Chase Million and Michael St. Clair have joined the Mars 2020 mission as Collaborators on the Science Team. They will develop software to assist with the scientific analysis of rover data under the direction of Mastcam-Z Co-I Melissa Rice (Western Washington University).
NASA's Astrophysics Data Analysis Program (ADAP) has
selected Million Concepts' proposed GLCat (GALEX Legacy
Catalog) project for funding, with our founder Chase Million as
PI and in collaboration with Luciana Bianchi (JHU). GLCat will
be a massive catalog of all unique extended and point sources
detected by the orbital ultraviolet (UV) Galaxy Evolution
Explorer (GALEX) telescope during its decade-long mission.
GLCat will be the most extensive UV sky catalog ever produced,
correcting critical issues with earlier GALEX catalogs and
including several million new seconds of imaging coverage that
have not previously been released in a usable format. We expect
it to become an enabling component for scientific
investigations in multiple sub-disciplines of astronomy for
years to come, and also be a key resource in the design and
planning of future ultraviolet astronomy missions. GLCat will
build on and vastly extend the scope of our existing GFCat (GALEX Flare
Catalog) and gPhoton projects,
enable the discovery of UV phenomena
visible only at very high time resolution.
Read excerpts from
our proposal for more information.
Chase Million has been selected as a 2021 Better Scientific Software (BSSw) Fellow. As a fellowship project, he will develop tools and techniques for generating realistic and useful software project estimates that account for the unique circumstances of scientific research. This project is based on workflows already in use within Million Concepts and follows from a workshop on the same topic that Chase presented at the first OpenPlanetary Virtual Conference (OPvCon) in June 2020.
Million Concepts will be working with PI Matthew Siegler (Planetary Science Institute), Jianqing Feng (Planetary Science Institute), and Paul Hayne (CU Boulder) on a research project recently selected for funding by the Lunar Data Analysis program. The project, "Thermal and Dielectric Properties From the Chang'e-2 Microwave Radiometer," focuses on recalibrating the CE-2 data set and finding synergies between these corrected data and data from other instruments (such as the Lunar Reconnaissance Orbiter's Diviner radiometer). This new fused data set will enable us to produce more accurate maps of the dielectric qualities and subsurface composition of the Moon than have ever previously been released. This is our third research collaboration with Dr. Siegler, and we are pleased to be continuing to lend our analysis expertise to his groundbreaking work on lunar geology. Read our proposal for more detail on the project.
As output from the GFCat project, Chase Million presented a catalog of GALEX photometric measurements reprocessed from their original 25-minute visit depths to 2-minute cadence. This catalog enables the systematic search of fast stellar variability in the ultraviolet across the entire GALEX corpus. A video of the talk can be found here. A Jupyter Notebook with example workflows is here.
Chase Million organized the first OpenPlanetary Virtual Conference (OPvCon), and also presented a two-hour workshop on Research Software Project Management.
The Apollo 15/17 Heat Flow Experiment Concatenated Data archive are now available at the PDS Geoscience Node. Read more about this effort here.
Chase Million presented an initial sample of 400 GALEX light curves with significant (>6-sigma) time-domain variability on timescales less than approx. 30 minutes. The poster can be viewed here.
Chase Million and Michael St. Clair have joined the Mars Science Laboratory mission as Collaborators on the Science Team. They will develop software to assist with the scientific analysis of rover multispectral data under the direction of Melissa Rice (Western Washington University).
Michael St. Clair presented at the 4th Planetary Data Workshop on improvements to the Apollo 15 and 17 Heat Flow Experiment (HFE) data archive.
As a NASA iTech Finalist, Chase Million attended the Global Entrepreneurship Summit in The Hague, the Netherlands. See our NASA iTech presentation here.