1. Setup and Installation

1.1. Introduction

This guide walks you through setting up the MAROON-X data reduction pipeline development environment. The MAROON-X pipeline (maroonxdr) is built on top of the DRAGONS framework and provides recipes and primitives for reducing MAROON-X echelle spectroscopy data.

The setup process uses nox (a Python task automation tool) to create a complete development environment with all necessary dependencies, including the DRAGONS framework, scientific libraries, and the MAROON-X instrument packages.

1.2. Prerequisites

Before beginning, ensure you have:

  • Python 3.12 installed on your system

  • Git for cloning repositories

  • Poetry for dependency management (will be installed automatically by nox if missing)

Note

The MAROON-X pipeline requires Python 3.12 specifically. Using other Python versions may cause compatibility issues.

1.3. Installation Steps

1.3.1. Step 1: Clone the MAROONXDR Repository

First, clone the MAROON-X DRAGONS repository:

cd /path/to/your/projects
git clone https://github.com/GeminiDRSoftware/MAROONXDR.git
cd MAROONXDR

The pipeline also needs a set of static instrument reference files that are not tracked in git: bad pixel masks (BPM), stripe ID traces (SID), and static wavelength solutions (WLS). They are distributed with each release as lookups_files.zip on the GitHub releases page.

Download the archive and extract it into the package lookup directory:

unzip lookups_files.zip -d maroonxdr/maroonx/lookups/

This places the FITS files under lookups/BPM/, lookups/SID/ and lookups/WLS/, where the pipeline expects them. The packages are installed in editable mode later in this guide, so the extracted files are picked up automatically.

1.3.2. Step 2: Install Nox

Nox is the task automation tool that manages the development environment setup. Install it using pip:

# Install nox globally or in your user directory
pip install --user nox

# Or install globally (may require sudo)
sudo apt install nox

# Verify installation
nox --version

1.3.3. Step 3: List Available Nox Sessions

View all available nox sessions to understand what tasks are available:

# List all nox sessions
nox -l

noxfile.py defines several sessions; the most relevant to a new developer are:

- devenv -> Create a development environment.
- devconda -> Create a conda development environment.
- unit_tests -> Run unit tests.
- regression_tests -> Run DRAGONS-style regression tests with inputs/refs comparison.
- docs -> Build documentation using Sphinx.

The devenv session is what we’ll use to create the development environment.

1.3.4. Step 4: Create the Development Environment

Run the devenv nox session to automatically set up the complete development environment:

# Create development environment (this may take a few minutes)
nox -s devenv

Important

This command performs the following steps:

  1. Creates a Python 3.12 virtual environment at venv/

  2. Clones the DRAGONS framework into DRAGONS/ directory (if not already present)

  3. Installs DRAGONS in development mode along with FitsStorage from GitHub

  4. Installs all Python dependencies from pyproject.toml (main, dev, docs, test groups)

  5. Installs pytest_dragons from GitHub

  6. Installs maroonxdr and maroonx_instruments in editable mode

  7. Sets DRAGONS_TEST environment variable in the activate script

Once the setup completes, activate the virtual environment:

source venv/bin/activate

Your shell prompt should now show (mx_dev) indicating the environment is active.

Note

The devenv session automatically writes DRAGONS_TEST into the activate script, pointing to a mx_test/ directory next to the repository. The synthetic unit tests run without any data there; to also run the regression tests, unpack the shared test data archive into that directory. The Tests chapter of the Programmer Manual describes how to obtain the archive and the expected layout (the Programmer Manual is built locally, see Building the Manuals Locally).

1.3.5. Step 5: Verify Installation

Verify that the installation was successful:

# Check that maroonxdr is installed
python -c "import maroonxdr; print('maroonxdr installed.')"

# Check that maroonx_instruments is installed
python -c "import maroonx_instruments; print('maroonx_instruments installed.')"

1.4. Calibration Database Setup

The DRAGONS calibration database (caldb) keeps track of every processed calibration produced during a reduction and serves it back to later steps on demand. The reduction tutorials assume caldb is configured and initialised before you start. This is a one-time setup per working directory.

1.4.1. Create the configuration file

DRAGONS reads its configuration from ~/.dragons/dragonsrc. Create the directory and file:

mkdir -p ~/.dragons
touch ~/.dragons/dragonsrc

Edit ~/.dragons/dragonsrc so it contains a single [calibs] section pointing at the calibration database file for the tutorial. We recommend keeping the database inside the same working directory you will reduce in, so the tutorial state is self-contained:

[calibs]
databases = /absolute/path/to/science_dir/cal_manager.db get store

The trailing get store flags tell DRAGONS to retrieve calibrations from this database during reductions and to store every newly produced calibration into it automatically. Both flags are required for the tutorials to work as written - without store, every processed calibration would have to be registered manually with caldb add.

Note

Use an absolute path for the database file.

1.4.2. Initialise the database

Once the config file is in place, create the (empty) database file with:

caldb init

If a database already exists at the configured path, pass -w (--wipe) to start over.

1.4.3. Verify the configuration

To confirm DRAGONS is reading the file you just wrote and to see which database it will use, run:

caldb config

The output should show ~/.dragons/dragonsrc as the configuration file and the absolute path of your cal_manager.db as the configured database.

For full details on the calibration service, including remote databases and multi-database setups, see the DRAGONS Recipe System User Manual.

1.5. Understanding the Development Environment

1.5.1. Directory Structure After Setup

After running nox -s devenv, your directory structure will look like this:

MAROONXDR/
├── DRAGONS/                      # DRAGONS framework (cloned by nox)
│   ├── geminidr/                 # Core DRAGONS primitives
│   ├── gempy/                    # DRAGONS utilities
│   └── astrodata/                # AstroData framework
├── maroonxdr/                    # MAROON-X pipeline implementation
│   └── maroonx/                  # Primitives, recipes, tests
├── maroonx_instruments/          # MAROON-X instrument definitions
│   └── maroonx/                  # AstroData class, tags, descriptors
├── venv/                         # Virtual environment (created by nox)
├── doc/                          # Documentation
├── noxfile.py                    # Nox configuration
├── pyproject.toml                # Package metadata and dependencies
└── README.md                     # Repository README

To activate the working environment, run:

source venv/bin/activate

You should see (mx_dev) in your shell prompt, indicating the environment is active.

1.6. Building the Manuals Locally

Only this Tutorial is published to Read the Docs. The User Manual and Programmer Manual live in the repository under doc/usermanuals/MAROONXDR_UserManual/ and doc/progmanuals/MAROONXDR_ProgManual/ and are built on demand with two nox sessions.

With mx_dev activated:

# HTML build (default)
nox -s usermanual
nox -s progmanual

# PDF build (requires latexmk and a TeX Live install)
nox -s usermanual -- --pdf
nox -s progmanual -- --pdf

The User Manual’s primitive and recipe reference pages are generated from the live docstrings and parameter defaults. Pass --regen to rebuild those fragments from the current code before building:

# Regenerate the primitive/recipe reference, then build HTML
nox -s usermanual -- --regen

Use it after editing a primitive or recipe docstring so the reference reflects your changes. Without --regen the build uses old fragments. The flag can be combined with --pdf.

Each session prints a clickable file:// link to the rendered output when it finishes. Most terminals turn it into a hyperlink you can open with Ctrl+Click (Cmd+Click on macOS).

The Tutorial itself can also be rebuilt locally with nox -s tutorial (or nox -s tutorial -- --pdf), useful when you’re editing tutorial pages and want to preview them before pushing.

1.7. Alternative: Conda Environment

If you prefer using Conda instead of virtualenv, use the devconda session:

# Create conda development environment
nox -s devconda

# Activate the conda environment
conda activate mx_devconda

1.8. Next Steps

Once your development environment is set up, you can proceed to:

Both tutorials will guide you through reducing MAROON-X data from raw frames to science-ready spectra.