3. MAROON-X DRP: Using the CLI tools

3.1. Complete Reduction Workflow Using CLI

This guide walks through the end-to-end reduction of a single MAROON-X science exposure using the DRAGONS command-line tools (reduce, dataselect, caldb). The example reduces an exposure of object HD 3651 taken on 2025-07-17. The final output is a fully reduced, wavelength-calibrated spectrum with science quality.

The same workflow expressed through the Python API is in MAROON-X DRP: Using the Reduce class API. Both tutorials reduce the same files and produce the same outputs; they differ only in interface.

3.1.1. Overview of Reduction Steps

The full reduction proceeds in nine steps, executed in order:

Debundle - split raw GOA bundles into per-arm FITS files
Master Darks - combine raw darks per exposure time and arm
Dark Coefficients - fit dark scaling vs. exposure time per arm
Master Flats - build per-arm master flats from DFFFD + DDDDF inputs
Wavelength Calibration - fit the dynamic etalon wavelength solution
Synthetic Darks - interpolate a dark matched to each science exposure
Science Reduction - extract and wavelength-calibrate science spectra
Barycentric Correction - apply the BERV shift to the reduced spectra
Export Bundle - combine BLUE and RED arms into a single output FITS

Note

MAROON-X has two independent arms (BLUE and RED). Every calibration and science step is run separately per arm.

Important

MAROON-X is currently an external instrument package, so all DRAGONS CLI tools require --adpkg maroonx_instruments (and --drpkg maroonxdr for reduce) to recognise the file tags and load the recipes. These flags will no longer be needed once MAROON-X is integrated into DRAGONS.

3.2. Prerequisites

Before beginning the reduction, ensure you have:

A working maroonxdr installation in a Python virtual environment, with the DRAGONS framework available on the same environment. See the installation guide of the User Manual for details.
All raw FITS files for the example (darks, flats, wavecal, science) downloaded from the Gemini Observatory Archive (GOA) into a single working directory. We call this directory science_dir/ throughout the tutorial.

3.2.1. Directory Structure

The tutorial assumes a single working directory holding every raw bundle:

science_dir/
└── N*.fits                   # Raw GOA bundle files

As the reduction proceeds, reduce writes its outputs into the working directory and registers calibrations under a calibrations/ subtree managed by caldb:

science_dir/
├── N*.fits                          # Raw GOA bundle files
├── *_b_*.fits, *_r_*.fits           # Debundled per-arm files and
│                                    # downstream reduction products
├── calibrations/
│   ├── processed_dark/              # Master darks (per exptime, per arm)
│   ├── processed_dark_coeff/        # Dark scaling coefficients (per arm)
│   ├── processed_flat/              # Master flats (per arm)
│   └── processed_wavecal/           # Dynamic etalon wavelength solutions
└── reduce_*.log                     # Reduction log files

Note

Calibration files produced by reduce are written twice: once in science_dir/ and once in the matching calibrations/<caltype>/ subdirectory. The copy under calibrations/ is what caldb indexes and serves to later steps.

Note

This tutorial assumes caldb is configured and initialised. See Calibration Database Setup for a one-time setup walkthrough.

3.2.2. Dataset

The worked example reduces a single 300 s exposure of HD 3651 in the SOOOE fiber pattern, taken on 2025-07-17, against calibrations taken on 2025-07-01, 2025-07-17, and 2025-07-21.

All files can be downloaded from the Gemini Observatory Archive (GOA). The table below lists the raw GOA bundle ranges; each N…M*.fits bundle contains one BLUE and one RED exposure that splitBundle (Step 1) separates into individual files.

Type	GOA bundle range	Notes
Darks	`N20250717M6066` - `N20250721M6059`	DDDDE pattern, BLUE and RED, at exposure times 60, 120, 300, 600, 900, 1200, 1800 s
Flats	`N20250701M6126` - `N20250701M6271`	DFFFD and DDDDF illumination patterns, BLUE and RED
Wavecal	`N20250717M5948`	DEEEE etalon frame, BLUE and RED
Science	`N20250717M5299`	HD 3651, 300 s, SOOOE pattern, BLUE and RED

The dark exposure times cover the range needed by the dark-coefficient fit (Step 3), which interpolates a synthetic dark matched to the science exposure time.

3.3. Reduction Steps

3.3.1. Step 1: Debundle Raw Data

Purpose: split each raw GOA bundle into one BLUE and one RED FITS file.

MAROON-X data arrives from GOA as bundle files (tagged BUNDLE) that contain both arms in a single FITS. The default processBundle recipe picks up every BUNDLE-tagged file and writes a per-arm FITS for each one. For example, the science bundle N20250717M5299.fits yields 20250717T144308Z_SOOOE_b_0300.fits and 20250717T144308Z_SOOOE_r_0004.fits.

Select every bundle in the working directory and run reduce:

# Build the bundle list
dataselect --adpkg maroonx_instruments --tags BUNDLE -o bundles.lis *.fits

# Debundle (no --recipe needed: processBundle is the default for BUNDLE)
reduce --adpkg maroonx_instruments --drpkg maroonxdr @bundles.lis

Note

The @ prefix tells reduce to read filenames from the given list file, one per line.

After this step science_dir/ contains the original N*.fits bundles plus the per-arm debundled files used by every subsequent step.

3.3.2. Step 2: Master Darks

Purpose: combine the raw darks at each exposure time into a master dark, per arm.

For every (exposure time, arm) pair, dataselect finds the matching raw darks (DDDDE pattern, tagged RAW,DARK) and reduce runs the default makeProcessedDark recipe. The dataset has 7 exposure times × 2 arms, so 14 master darks total.

For a single combination of exposure time and arm, the process is:

# Process darks for a specific exposure time and arm, 300s and BLUE
dataselect --adpkg maroonx_instruments --tags RAW,DARK,300s,BLUE \
    -o darks_300s_BLUE.lis *.fits
reduce --adpkg maroonx_instruments --drpkg maroonxdr @darks_300s_BLUE.lis

Run the full loop directly in the shell:

# Process darks for each exposure time and arm
for exptime in 60s 120s 300s 600s 900s 1200s 1800s; do
    for arm in BLUE RED; do

        # Select darks for this combination and save to list
        dataselect --adpkg maroonx_instruments --tags RAW,DARK,$exptime,$arm \
            -o darks_${exptime}_${arm}.lis *.fits

        # Reduce the list
        reduce --adpkg maroonx_instruments --drpkg maroonxdr \
            @darks_${exptime}_${arm}.lis
    done
done

Each output is a single master dark file named after the first raw frame in the stack with a _dark suffix - for example 20250721T170049Z_DDDDE_b_0300_dark.fits. The 14 master darks land in calibrations/processed_dark/ (and as duplicates in science_dir/).

Verify processed darks:

# List all processed darks
dataselect --adpkg maroonx_instruments --tags PROCESSED,DARK *.fits

Note

Processed calibration files are saved twice: in the current working directory science_dir/ and in the matching calibrations/<caltype>/ subdirectory. The copy under calibrations/ is the one caldb indexes and serves to later steps.

3.3.3. Step 3: Dark Coefficients

Purpose: fit, per arm, the dark scaling as a function of exposure time so a synthetic dark can later be interpolated for any science exposure.

The makeDarkCoefficients recipe takes every master dark for one arm (one per exposure time) and produces a single dark-coefficient file. The input selector takes all PROCESSED,DARK files for the arm and excludes the DARK_COEFF and DARK_SYNTH tags so previous coefficient or synthetic-dark outputs in the same directory are not re-ingested.

Run once per arm:

for arm in BLUE RED; do

    # Select all master darks for this arm, excluding any previous
    # coefficient or synthetic-dark outputs
    dataselect --adpkg maroonx_instruments \
        --tags PROCESSED,DARK,$arm \
        --xtags DARK_COEFF,DARK_SYNTH \
        -o dark_coeffs_${arm}.lis *.fits

    # Fit the dark-coefficient model
    reduce --adpkg maroonx_instruments --drpkg maroonxdr \
        --recipe makeDarkCoefficients @dark_coeffs_${arm}.lis
done

Each call writes one *_darkCoefficients.fits file (named after the first input master dark) into science_dir/, and a copy under the processed_dark_coeff caltype in calibrations/processed_dark_coeff/. The output carries the DARK_COEFF tag, which is how later steps locate it.

Verify dark-coefficient files:

# Select dark coefficient files
dataselect --adpkg maroonx_instruments --tags DARK_COEFF *.fits

3.3.4. Step 4: Master Flats

Purpose: build a master flat per arm.

MAROON-X flats come in two illumination patterns: DFFFD (flat on fibers 2-4) and DDDDF (flat on fiber 5 only). The dataset for this tutorial contains both patterns and uses the makeProcessedFlatDFFFF recipe, which combines the two into a single DFFFF master flat per arm.

Run the reduction per arm:

for arm in BLUE RED; do

    # Select every raw flat for this arm (both DFFFD and DDDDF)
    dataselect --adpkg maroonx_instruments --tags RAW,FLAT,$arm \
        -o flats_${arm}.lis *.fits

    # Build the master flat
    reduce --adpkg maroonx_instruments --drpkg maroonxdr \
        --recipe makeProcessedFlatDFFFF @flats_${arm}.lis
done

Each call writes one *_flat.fits master flat into science_dir/ and a copy under the processed_flat caltype in calibrations/processed_flat/. The output carries the PROCESSED,FLAT tag set.

Note

The default recipe makeProcessedFlat combines DFFFD and FDDDF patterns into an FFFFF master flat. When the dataset has DDDDF instead of FDDDF (as in this tutorial), use the makeProcessedFlatDFFFF variant explicitly via --recipe.

Verify processed flats:

# List all processed flats
dataselect --adpkg maroonx_instruments --tags PROCESSED,FLAT *.fits

3.3.5. Step 5: Wavelength Calibration

Purpose: fit, per arm, a dynamic wavelength solution from the etalon calibration frame.

The Fabry-Perot etalon is the primary wavelength calibrator for MAROON-X. Anchored to an initial ThAr solution, the etalon provides the refined wavelength and drift solution that the science reduction (Step 7) applies to extracted spectra.

The makeDynamicWavecal recipe takes the raw etalon frame (DEEEE pattern, tagged RAW,ETALON) for one arm, identifies the peaks in every order, and fits the per-pixel wavelength solution. The getPeaksAndPolynomials:multithreading=True override parallelizes the fit and is recommended on multi-core machines.

Run once per arm:

for arm in BLUE RED; do

    # Select the raw etalon wavecal frame for this arm
    dataselect --adpkg maroonx_instruments --tags RAW,ETALON,$arm \
        -o wavecal_${arm}.lis *.fits

    # Fit the dynamic wavelength solution
    reduce --adpkg maroonx_instruments --drpkg maroonxdr \
        --recipe makeDynamicWavecal \
        -p getPeaksAndPolynomials:multithreading=True \
        @wavecal_${arm}.lis
done

Each call writes one *_wavecal.fits file into science_dir/ and a copy under the processed_wavecal caltype in calibrations/processed_wavecal/. The output carries the PROCESSED,WAVECAL tag set.

Verify wavelength calibrations:

# List processed wavecal files
dataselect --adpkg maroonx_instruments --tags PROCESSED,WAVECAL *.fits

3.3.6. Step 6: Synthetic Darks

Purpose: interpolate, per science frame, a synthetic dark matched to its exact exposure time using the dark coefficients fit in Step 3.

The makeSyntheticDark recipe takes the raw science frames and uses the DARK_COEFF calibration to build a dark frame at the science exposure time. The synthetic darks are stored under the processed_dark caltype with the DARK_SYNTH tag so Step 7 can find them as ordinary darks.

Run once per arm:

for arm in BLUE RED; do

    # Select raw science frames for this arm
    dataselect --adpkg maroonx_instruments --tags RAW,SCI,$arm \
        -o sci_${arm}.lis *.fits

    # Build the synthetic dark for each science frame
    reduce --adpkg maroonx_instruments --drpkg maroonxdr \
        --recipe makeSyntheticDark @sci_${arm}.lis
done

3.3.7. Step 7: Science Data Reduction

Purpose: extract and wavelength-calibrate the science spectra, applying the master flat, the dynamic wavelength solution, and the synthetic dark matched to the science exposure time.

The default recipe for RAW,SCI files runs the full echelle reduction: dark and flat correction, straylight removal, stripe extraction, optimal extraction, wavelength assignment, and fiber combination. caldb resolves each input calibration automatically from the work done in Steps 2-6.

The three -p overrides below are the recommended defaults for the current pipeline: extractStripes:straylight_removal_fibers=[5] applies the straylight correction to the calibration fiber, getPeaksAndPolynomials runs the per-order fits in parallel, and combineFibers:max_clips=20 sets the outlier rejection threshold when combining the science fibers.

Run once per arm:

for arm in BLUE RED; do

    # Select raw science frames for this arm
    dataselect --adpkg maroonx_instruments --tags RAW,SCI,$arm \
        -o sci_${arm}.lis *.fits

    # Full echelle reduction
    reduce --adpkg maroonx_instruments --drpkg maroonxdr \
        -p 'extractStripes:straylight_removal_fibers=[5]' \
        -p 'getPeaksAndPolynomials:multithreading=True' \
        -p 'combineFibers:max_clips=20' \
        @sci_${arm}.lis
done

Each call writes a *_reduced.fits file per input science frame into science_dir/. The reduced products carry the PROCESSED_SCIENCE tag and are the input to the barycentric-correction step (Step 8).

Verify reduced science frames:

# List all reduced science frames
dataselect --adpkg maroonx_instruments --tags PROCESSED_SCIENCE *.fits

3.3.8. Step 8: Barycentric Correction

Purpose: shift the reduced spectra into the solar-system barycentric rest frame.

The applyBarycentricCorrection recipe reads the time, sky-position, and observatory descriptors from each reduced file, queries SIMBAD to resolve the target’s coordinates, computes the barycentric Earth radial velocity (BERV), and writes the corrected spectrum to *_barycor.fits.

The barycentricCorrection primitive accepts two related parameters:

target_name — substring match against the FITS OBJECT header. When set, only files whose OBJECT contains this string are processed; when None, every input is processed using its own OBJECT value for the SIMBAD lookup.
simbad_target_name — explicit SIMBAD-resolvable name. Use when the FITS OBJECT value (e.g. HD3651) is not directly resolvable and you need to override the lookup string (e.g. HD 3651).

For this tutorial the FITS header has OBJECT='HD3651' (no space), which is already SIMBAD-resolvable, so only target_name is needed.

Run once per arm:

for arm in BLUE RED; do

    # Select the reduced science file for this arm
    dataselect --adpkg maroonx_instruments --tags PROCESSED_SCIENCE,$arm \
        -o barycor_${arm}.lis *_reduced.fits

    # Apply the barycentric correction
    reduce --adpkg maroonx_instruments --drpkg maroonxdr \
        --recipe applyBarycentricCorrection \
        -p barycentricCorrection:target_name=HD3651 \
        @barycor_${arm}.lis
done

Each call writes one *_barycor.fits file into science_dir/. The output carries the BARYCOR and PROCESSED_SCIENCE tags.

Verify barycor-corrected files:

# List all barycor-corrected files
dataselect --adpkg maroonx_instruments --tags PROCESSED,BARYCOR *.fits

3.3.9. Step 9: Export Reduced Science Bundle

Purpose: combine the BLUE and RED barycor-corrected spectra of each science observation into a single multi-extension FITS.

The exportReducedBundle recipe groups its inputs by the original GOA bundle name (ARCHNAME), then merges each (BLUE, RED) pair into one output. Both arms for a given observation must be passed in the same reduce call - the recipe raises an error if either arm is missing for any ARCHNAME.

Select every barycor output and run the recipe once:

# Select all barycor-corrected files (both arms)
dataselect --adpkg maroonx_instruments --tags PROCESSED,BARYCOR \
    -o bundle.lis *_barycor.fits

# Combine BLUE + RED into the final bundle
reduce --adpkg maroonx_instruments --drpkg maroonxdr \
    --recipe exportReducedBundle @bundle.lis

The output is a single <ARCHNAME>_reduced.fits per observation in science_dir/ - for this tutorial, N20250717M5299_reduced.fits. This is the science-ready product.

3.4. Advanced CLI Usage

This section covers a few CLI tools and patterns that go beyond the linear walk-through above. They are useful when something goes wrong, or when you want to inspect the pipeline state interactively.

3.4.1. Inspecting files: `typewalk`

typewalk prints the AstroData tag set of one or more FITS files. It is the first thing to reach for when dataselect is not picking up the files you expect:

# Show all tags on a single file
typewalk --adpkg maroonx_instruments N20250717M5299.fits

# Walk a whole directory and print tags per file
typewalk --adpkg maroonx_instruments .

Tag combinations such as RAW, SCI, BLUE, 300s, PROCESSED_SCIENCE, BARYCOR are exactly the strings dataselect will match against.

3.4.2. Discovering recipes and parameters: `showrecipes` / `showpars`

The DRAGONS framework provides two discovery commands that answer “what recipes can run on this file?” and “what parameters does this primitive take?”:

# List every recipe applicable to this file's tag set
showrecipes --adpkg maroonx_instruments --drpkg maroonxdr \
    20250717T144308Z_SOOOE_b_0300.fits

# Show the default parameters for a primitive (and its docstring)
showpars --adpkg maroonx_instruments --drpkg maroonxdr \
    20250717T144308Z_SOOOE_b_0300.fits combineFibers

Use these to discover the recipe names you can pass to reduce --recipe and the parameter names you can pass to -p primitive:param=value.

3.4.3. Overriding primitive parameters

The -p primitive:param=value syntax used throughout this tutorial sets a single parameter on a single primitive for the current reduce call. Several -p flags can be chained:

reduce --adpkg maroonx_instruments --drpkg maroonxdr \
    -p extractStripes:straylight_removal_fibers=[5] \
    -p getPeaksAndPolynomials:multithreading=True \
    -p combineFibers:max_clips=20 \
    @sci_BLUE.lis

Discover available parameters with showpars (above).

3.4.4. Managing the calibration database: `caldb`

reduce registers processed calibrations with caldb automatically, so the linear walk-through above never needs to call caldb directly. For troubleshooting - “this run picked the wrong flat”, “the wavecal from yesterday is shadowing today’s” - caldb has a small set of commands:

caldb list                            # show every calibration registered
caldb add calibrations/processed_flat/<file>.fits
caldb remove <file>.fits

For initial configuration and database initialisation, see Calibration Database Setup.

3.4.5. Logging

Two flags control reduce logging:

# Set log verbosity (mode: quiet | standard | debug)
reduce --adpkg maroonx_instruments --drpkg maroonxdr \
    --logmode debug @files.lis

# Send the log to a specific file (default: reduce.log)
reduce --adpkg maroonx_instruments --drpkg maroonxdr \
    --logfile my_reduction.log @files.lis

3. MAROON-X DRP: Using the CLI tools

3.1. Complete Reduction Workflow Using CLI

3.1.1. Overview of Reduction Steps

3.2. Prerequisites

3.2.1. Directory Structure

3.2.2. Dataset

3.3. Reduction Steps

3.3.1. Step 1: Debundle Raw Data

3.3.2. Step 2: Master Darks

3.3.3. Step 3: Dark Coefficients

3.3.4. Step 4: Master Flats

3.3.5. Step 5: Wavelength Calibration

3.3.6. Step 6: Synthetic Darks

3.3.7. Step 7: Science Data Reduction

3.3.8. Step 8: Barycentric Correction

3.3.9. Step 9: Export Reduced Science Bundle

3.4. Advanced CLI Usage

3.4.1. Inspecting files: typewalk

3.4.2. Discovering recipes and parameters: showrecipes / showpars

3.4.3. Overriding primitive parameters

3.4.4. Managing the calibration database: caldb

3.4.5. Logging

3.4.1. Inspecting files: `typewalk`

3.4.2. Discovering recipes and parameters: `showrecipes` / `showpars`

3.4.4. Managing the calibration database: `caldb`