Note

Download the full notebook for this tutorial here

Tutorial: Finding and Cleaning Bad Pixels with spaceKLIP

---

In this notebook, you will learn how to find and correct for bad pixels in high-contrast imaging data from NIRCam and MIRI using the spaceKLIP pipeline. We will walk through each of the find and clean methods available in the pipeline, which are designed to detect outlier pixels and mitigate their impact on downstream analysis.

By the end of this notebook, you will have gained hands-on experience applying the supported bad pixel identification and correction techniques, and you will be prepared to incorporate these steps into your own reductions using the spaceKLIP pipeline.

Table of Contents

• 1. Imports

• 2. Helper Functions

• 3. Setup Directories & Download the Data

• 4. Identify Bad Pixels: DQ Array

• 5. Identify Bad Pixels: TIMEINTS

• 6. Identify Bad Pixels: Sigma Clipping

• 7. Identify Bad Pixels: Custom Mask

• 8. Clean Methods

• 9. Conclusions

---

Imports

# Generate interactive plots?
interactive = True

try:
    import plotly.io as pio
    pio.renderers.default = "sphinx_gallery"
except ImportError:
    interactive = False
    print("Plotly is not installed. "
          "Falling back to static Matplotlib plots.")

import os
import glob
import requests
import numpy as np

import spaceKLIP
from spaceKLIP.plotting import compare_find_methods, compare_clean_methods

from astropy.io import fits

---

Helper Functions

This function will download the demo data from Box.

def download_box_file(box_url,
                      base_dir='.',
                      filename=None):
    """
    Download a file from Box.

    Parameters
    ----------
    box_url : str
        Box shared link
    base_dir : str, optional
        Folder to save the file.
    filename : str or None, optional
        Custom filename.
    """
    # Ensure the directory exists.
    os.makedirs(base_dir, exist_ok=True)

    # Download from Box.
    r = requests.get(box_url + '?raw=1')
    r.raise_for_status()

    # Determine filename.
    if filename is None:
        cd = r.headers.get('content-disposition')
        if cd:
            filename = cd.split('filename=')[-1].strip(' "')
        else:
            filename = 'downloaded_file.fits'

    # Save file.
    filepath = os.path.join(base_dir, filename)
    with open(filepath, 'wb') as f:
        f.write(r.content)

    print(f"Downloaded {filename} to {base_dir}")
    return filepath

This function automates the execution of spaceKLIP bad-pixel find and clean methods across multiple parameter configurations.

def run_bad_pixel_methods(data_root,
                          param_sets,
                          set_dq_zero=True):
    """
    Run spaceKLIP bad-pixel "find" or "clean" methods for given parameter set(s).

    Parameters
    ----------
    data_root : str
        Path to the directory containing the input FITS files.
    method : str
        Name of the spaceKLIP bad pixel find or clean method to run.

        Supported "find" options include:
        - 'dqarr'
        - 'timeints'
        - 'sigclip'
        - 'custom'

        Supported "clean" options include:
        - 'timemed'
        - 'localmed'
        - 'medfilt'
        - 'interp2d'
        - 'astrofix'

    param_sets : list of dict
        List of parameter dictionaries, where each dictionary defines a single
        run configuration for the chosen method.
    set_dq_zero : bool, optional
        If True, resets the DQ array before running the method so that only
        newly identified bad pixels are included. If False, existing DQ flags
        are preserved and new flags are added. Default is True.

    Returns
    -------
    files : list of list of str
        List of file lists, where each sublist contains the output FITS files
        for a given parameter set.
    kwargs_list : list of dict
        List of parameter dictionaries corresponding to each run.
    """

    files = []  # Store files with bad pixels identified/cleaned.
    kwargs_list = []  # Store the parameters used in each run.

    # Define supported methods.
    find_methods = {'dqarr', 'timeints', 'sigclip', 'custom'}
    clean_methods = {'timemed', 'localmed', 'medfilt', 'interp2d', 'astrofix'}

    # Loop through each parameter set.
    for run_idx, params in enumerate(param_sets):
        method = params['method']
        kwargs_name = f"{method}_kwargs"
        if kwargs_name not in params:
            raise ValueError(
                f"Missing kwargs for method '{method}' in param_sets[{run_idx}]"
            )
        kwargs = params[kwargs_name]
        print(f"------------------ {method.upper()} RUN {run_idx + 1} ------------------")

        # Input/output directories.
        input_dir = os.path.join(data_root, params.get('input_dir', ''))
        subdir_name = f"{method}_run{run_idx + 1}"
        output_dir = os.path.join(data_root, subdir_name)
        os.makedirs(output_dir, exist_ok=True)

        print(f"Reading from: {input_dir}")
        print(f"Writing to: {output_dir}")

        # Initialize SpaceKLIP tools.
        database = spaceKLIP.database.create_database(input_dir=input_dir, file_type='calints.fits', output_dir=data_root, verbose=False)
        imageTools = spaceKLIP.imagetools.ImageTools(database=database)

        # Run method.
        if method in find_methods:
            imageTools.find_bad_pixels(method=method, set_dq_zero=set_dq_zero, subdir=subdir_name, **{kwargs_name: kwargs})
        elif method in clean_methods:
            imageTools.clean_bad_pixels(method=method, subdir=subdir_name, **{kwargs_name: kwargs})
        else:
            raise ValueError(f"Unknown method: {method}")

        print()

        # Collect results.
        run_files = sorted(f for f in glob.glob(os.path.join(output_dir, "*.fits")) if "_calints.fits" in f)
        files.append(run_files)
        kwargs_list.append(params)

    return files, kwargs_list

---

Setup Directories & Download the Data

Throughout this tutorial, we will use NIRCam and MIRI coronagraphic datasets from the JWST ERS program on Direct Observations of Exoplanetary Systems (Program 1386), with a focus on the exoplanet HIP 65426 b. These datasets are consistent with those used in the reduction tutorial notebooks:

• NIRCam Reduction Tutorial: https://spaceklip.readthedocs.io/en/stable/tutorials/tutorial_NIRCam_reductions.html#

• MIRI Reduction Tutorial: https://spaceklip.readthedocs.io/en/stable/tutorials/tutorial_MIRI_reductions.html

# Box file links.
nircam_files = {'jw01386003001_0310a_00001_nrcalong_calints.fits': 'https://stsci.box.com/shared/static/kfolqrm370eio2ilu04jjxc9ouely58v.fits'}
miri_files = {'jw01386030001_02101_00001_mirimage_calints.fits': 'https://stsci.box.com/shared/static/lad7l6qcd3tiglwd4mbfikaklyb7550o.fits',  # MIRI background.
              'jw01386008001_04101_00001_mirimage_calints.fits': 'https://stsci.box.com/shared/static/5o1qy0vf0px8kr6gu2i4wbfuaw14rau6.fits'}

# Define directories.
data_root = './bad_pixel_testing/'
data_root_nircam = os.path.join(data_root, 'nircam/')
data_root_miri = os.path.join(data_root, 'miri/')

# Download NIRCam files.
for fname, url in nircam_files.items():
    download_box_file(url, base_dir=data_root_nircam, filename=fname)

# Download MIRI files.
for fname, url in miri_files.items():
    download_box_file(url, base_dir=data_root_miri, filename=fname)

Downloaded jw01386003001_0310a_00001_nrcalong_calints.fits to ./bad_pixel_testing/nircam/
Downloaded jw01386030001_02101_00001_mirimage_calints.fits to ./bad_pixel_testing/miri/
Downloaded jw01386008001_04101_00001_mirimage_calints.fits to ./bad_pixel_testing/miri/

---

Identify Bad Pixels: DQ Array

The dqarr method is typically the initial diagnostic for identifying bad pixels in the spaceKLIP pipeline. The data quality (DQ) array from JWST data products is informed by the bad pixel mask reference file in the Calibration Reference Data System (CRDS), which identifies known bad (DO_NOT_USE) pixels. The JWST pipeline automatically NaNs these bad pixels in the data, as illustrated in the example below.

However, because the bad pixel masks are not perfect and can become outdated, some ‘hot’ pixels and their elevated neighbors may remain unflagged. To handle these cases, the dqarr method has an option to expand a bad pixel to include neighboring pixels that are elevated relative to the local background. This behavior can be controlled using the following parameters in dqarr_kwargs:

• flag_neighbors (bool, optional) : If True, the 4-connected neighbors (up, down, left, right) of each DO_NOT_USE pixel are evaluated and flagged if they are elevated relative to the local background, which is estimated from the surrounding diagonal pixels.

• sigma (float, optional) : Threshold used to flag neighboring pixels whose values exceed diag_med + sigma * diag_std.

Note: spaceKLIP’s find_bad_pixels function provides the set_dq_zero parameter to start with a clean DQ array. By default, this is True, which clears all existing flags and allows each find method to independently identify bad pixels.

Below, we define a set of parameters for the dqarr method to loop through.

# Define the parameter sets to run for "dqarr".
dqarr_param_sets = [

    {'method': 'dqarr', 'dqarr_kwargs': {'flag_neighbors': False, 'sigma': None}},  # Run 0: Base Run.
    {'method': 'dqarr', 'dqarr_kwargs': {'flag_neighbors': True, 'sigma': 10}},  # Run 1: Expand flagging on neighbors.

]

First, we process the NIRCam data using the parameter sets defined above. A zoom region is specified to focus on a representative region of pixels that highlights the functionality of the flag_neighbors option best in this data.

Refer to the run_bad_pixel_methods function defined at the top to see how this step was run through the pipeline.

Note: In all of these plots, we by default, show the last integration. Also all of these plots are interactive, allowing you to zoom in or out to the full image using the controls in the top-right corner of each plot. A given run can be selected via the drop down menu.

# Run NIRCam examples with "dqarr".
dqarr_nircam_files, dqarr_kwargs_list = run_bad_pixel_methods(data_root_nircam, dqarr_param_sets, set_dq_zero=False)

# Plot NIRCam examples after running "dqarr".
compare_find_methods(files=[run_files[0] for run_files in dqarr_nircam_files],
                     kwargs_list=dqarr_kwargs_list,
                     zoom_region=(200, 215, 196, 212),
                     interactive=interactive)

------------------ DQARR RUN 1 ------------------
Reading from: ./bad_pixel_testing/nircam/
Writing to: ./bad_pixel_testing/nircam/dqarr_run1

2026-06-03 13:30:54,245 - CRDS - INFO -  Calibration SW Found: jwst 2.0.1 (/opt/anaconda3/envs/spaceklip_may26/lib/python3.11/site-packages/jwst-2.0.1.dist-info)

[spaceKLIP.imagetools:INFO] --> Concatenation JWST_NIRCAM_NRCALONG_F444W_MASKRND_MASK335R_SUB320A335R
[spaceKLIP.imagetools:INFO]   --> Method dqarr: jw01386003001_0310a_00001_nrcalong_calints.fits

------------------ DQARR RUN 2 ------------------
Reading from: ./bad_pixel_testing/nircam/
Writing to: ./bad_pixel_testing/nircam/dqarr_run2
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_NIRCAM_NRCALONG_F444W_MASKRND_MASK335R_SUB320A335R
[spaceKLIP.imagetools:INFO]   --> Method dqarr: jw01386003001_0310a_00001_nrcalong_calints.fits
[spaceKLIP.imagetools:INFO]   --> Total neighbor pixels flagged: 201

Next, we process the MIRI data with the parameter sets defined above. Because this MIRI dataset is less affected by cross-shaped bad pixels, we do not focus on or zoom in on any specific example below. Instead, we present this as an example for users to consider whether or not to enable this step for MIRI data.

# Run MIRI examples with "dqarr".
dqarr_miri_files, dqarr_kwargs_list = run_bad_pixel_methods(data_root_miri, dqarr_param_sets, set_dq_zero=False)

# Plot MIRI examples after running "dqarr".
compare_find_methods(files=[run_files[0] for run_files in dqarr_miri_files],
                     kwargs_list=dqarr_kwargs_list,
                     interactive=interactive)

------------------ DQARR RUN 1 ------------------
Reading from: ./bad_pixel_testing/miri/
Writing to: ./bad_pixel_testing/miri/dqarr_run1
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_MIRI_MIRIMAGE_F1550C_NONE_4QPM_1550_MASK1550
[spaceKLIP.imagetools:INFO]   --> Method dqarr: jw01386008001_04101_00001_mirimage_calints.fits
[spaceKLIP.imagetools:INFO]   --> Method dqarr: jw01386030001_02101_00001_mirimage_calints.fits

------------------ DQARR RUN 2 ------------------
Reading from: ./bad_pixel_testing/miri/
Writing to: ./bad_pixel_testing/miri/dqarr_run2
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_MIRI_MIRIMAGE_F1550C_NONE_4QPM_1550_MASK1550
[spaceKLIP.imagetools:INFO]   --> Method dqarr: jw01386008001_04101_00001_mirimage_calints.fits
[spaceKLIP.imagetools:INFO]   --> Total neighbor pixels flagged: 1006
[spaceKLIP.imagetools:INFO]   --> Method dqarr: jw01386030001_02101_00001_mirimage_calints.fits
[spaceKLIP.imagetools:INFO]   --> Total neighbor pixels flagged: 937

In all example cases, the dqarr step alone does not identify all bad pixels. We therefore present additional supported detection methods below.

---

Identify Bad Pixels: TIMEINTS

The timeints method identifies bad pixels by analyzing their behavior across integrations (i.e., how they change over time). For each pixel, it computes a median and median absolute deviation (MAD) across integrations, then flags values that deviate significantly from the expected signal. The timeints can be effective at better flagging transient artifacts such as cosmic rays. Two modes are available for identifying these outliers:

• per_pixel (default): Computes the variation across integrations independently for each pixel.

• group_pixels: Groups pixels by similar flux, computes the variation across integrations for each pixel, and compares each pixel’s variation to that of its corresponding flux group. Pixels that cannot be grouped (e.g., negative pixels) fall back to the per-pixel method.

Optional Parameters include:

• sigma (float, optional): Sigma-clipping threshold used to flag outliers in time. Default: 10

• method (str, optional): Detection mode to use. Default: ‘per_pixel’.

  • per_pixel: independent temporal outlier detection

  • group_pixels: flux-group-based comparison

• n_groups (int, optional): Number of flux groups used when method=’group_pixels’. Default: 25.

# Define the parameter sets to run for "timeints".

timeints_param_sets = [

    {'method': 'timeints', 'timeints_kwargs': {'mode': 'per_pixel', 'sigma': 5}},
    {'method': 'timeints', 'timeints_kwargs': {'mode': 'per_pixel', 'sigma': 3}},
    {'method': 'timeints', 'timeints_kwargs': {'mode': 'group_pixels', 'sigma': 3, 'n_groups': 25}},
    {'method': 'timeints', 'timeints_kwargs': {'mode': 'group_pixels', 'sigma': 3, 'n_groups': 10}},

]

Refer to the run_bad_pixel_methods function defined at the top to see how this step was run through the pipeline.

# Run NIRCam examples with "timeints".
timeints_nircam_files, timeints_kwargs_list = run_bad_pixel_methods(data_root_nircam, timeints_param_sets)

# Plot NIRCam examples after running "timeints".
compare_find_methods(files=[run_files[0] for run_files in timeints_nircam_files],
                     kwargs_list=timeints_kwargs_list,
                     interactive=interactive)

------------------ TIMEINTS RUN 1 ------------------
Reading from: ./bad_pixel_testing/nircam/
Writing to: ./bad_pixel_testing/nircam/timeints_run1
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_NIRCAM_NRCALONG_F444W_MASKRND_MASK335R_SUB320A335R
[spaceKLIP.imagetools:INFO]   --> Method timeints: jw01386003001_0310a_00001_nrcalong_calints.fits
[spaceKLIP.imagetools:INFO]   --> Method timeints: identified 0 additional bad pixel(s) -- 0.00%


------------------ TIMEINTS RUN 2 ------------------
Reading from: ./bad_pixel_testing/nircam/
Writing to: ./bad_pixel_testing/nircam/timeints_run2
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_NIRCAM_NRCALONG_F444W_MASKRND_MASK335R_SUB320A335R
[spaceKLIP.imagetools:INFO]   --> Method timeints: jw01386003001_0310a_00001_nrcalong_calints.fits
[spaceKLIP.imagetools:INFO]   --> Method timeints: identified 0 additional bad pixel(s) -- 0.00%


------------------ TIMEINTS RUN 3 ------------------
Reading from: ./bad_pixel_testing/nircam/
Writing to: ./bad_pixel_testing/nircam/timeints_run3
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_NIRCAM_NRCALONG_F444W_MASKRND_MASK335R_SUB320A335R
[spaceKLIP.imagetools:INFO]   --> Method timeints: jw01386003001_0310a_00001_nrcalong_calints.fits
[spaceKLIP.imagetools:INFO]   --> Method timeints: identified 452 additional bad pixel(s) -- 0.22%


------------------ TIMEINTS RUN 4 ------------------
Reading from: ./bad_pixel_testing/nircam/
Writing to: ./bad_pixel_testing/nircam/timeints_run4
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_NIRCAM_NRCALONG_F444W_MASKRND_MASK335R_SUB320A335R
[spaceKLIP.imagetools:INFO]   --> Method timeints: jw01386003001_0310a_00001_nrcalong_calints.fits
[spaceKLIP.imagetools:INFO]   --> Method timeints: identified 354 additional bad pixel(s) -- 0.17%

Note that the per_pixel mode may perform poorly when there are too few integrations (like this NIRCam example). It can also over-flag pixels if the sigma threshold is too high, sometimes even in the center of the PSF. In such cases, the group_pixels mode may provide better results.

# Run MIRI examples with "timeints".
timeints_miri_files, timeints_kwargs_list = run_bad_pixel_methods(data_root_miri, timeints_param_sets)

# Plot MIRI examples after running "timeints".
compare_find_methods(files=[run_files[0] for run_files in timeints_miri_files],
                     kwargs_list=timeints_kwargs_list,
                     interactive=interactive)

------------------ TIMEINTS RUN 1 ------------------
Reading from: ./bad_pixel_testing/miri/
Writing to: ./bad_pixel_testing/miri/timeints_run1
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_MIRI_MIRIMAGE_F1550C_NONE_4QPM_1550_MASK1550
[spaceKLIP.imagetools:INFO]   --> Method timeints: jw01386008001_04101_00001_mirimage_calints.fits
[spaceKLIP.imagetools:INFO]   --> Method timeints: identified 5813 additional bad pixel(s) -- 0.21%

[spaceKLIP.imagetools:INFO]   --> Method timeints: jw01386030001_02101_00001_mirimage_calints.fits
[spaceKLIP.imagetools:INFO]   --> Method timeints: identified 4455 additional bad pixel(s) -- 0.16%


------------------ TIMEINTS RUN 2 ------------------
Reading from: ./bad_pixel_testing/miri/
Writing to: ./bad_pixel_testing/miri/timeints_run2
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_MIRI_MIRIMAGE_F1550C_NONE_4QPM_1550_MASK1550
[spaceKLIP.imagetools:INFO]   --> Method timeints: jw01386008001_04101_00001_mirimage_calints.fits
[spaceKLIP.imagetools:INFO]   --> Method timeints: identified 47229 additional bad pixel(s) -- 1.74%

[spaceKLIP.imagetools:INFO]   --> Method timeints: jw01386030001_02101_00001_mirimage_calints.fits
[spaceKLIP.imagetools:INFO]   --> Method timeints: identified 42027 additional bad pixel(s) -- 1.55%


------------------ TIMEINTS RUN 3 ------------------
Reading from: ./bad_pixel_testing/miri/
Writing to: ./bad_pixel_testing/miri/timeints_run3
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_MIRI_MIRIMAGE_F1550C_NONE_4QPM_1550_MASK1550
[spaceKLIP.imagetools:INFO]   --> Method timeints: jw01386008001_04101_00001_mirimage_calints.fits
[spaceKLIP.imagetools:INFO]   --> Method timeints: identified 33818 additional bad pixel(s) -- 1.25%

[spaceKLIP.imagetools:INFO]   --> Method timeints: jw01386030001_02101_00001_mirimage_calints.fits
[spaceKLIP.imagetools:INFO]   --> Method timeints: identified 31245 additional bad pixel(s) -- 1.15%


------------------ TIMEINTS RUN 4 ------------------
Reading from: ./bad_pixel_testing/miri/
Writing to: ./bad_pixel_testing/miri/timeints_run4
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_MIRI_MIRIMAGE_F1550C_NONE_4QPM_1550_MASK1550
[spaceKLIP.imagetools:INFO]   --> Method timeints: jw01386008001_04101_00001_mirimage_calints.fits
[spaceKLIP.imagetools:INFO]   --> Method timeints: identified 9737 additional bad pixel(s) -- 0.36%

[spaceKLIP.imagetools:INFO]   --> Method timeints: jw01386030001_02101_00001_mirimage_calints.fits
[spaceKLIP.imagetools:INFO]   --> Method timeints: identified 8521 additional bad pixel(s) -- 0.31%

Compared to per_pixel, group_pixels generally reduces false positives in bright or high-variance regions, though it can still miss bad pixels whose variability is similar to others within the same flux group; increasing n_groups can help mitigate this by using narrower flux bins.

---

Identify Bad Pixels: Sigma Clipping

The sigclip method applies an iterative sigma-clipping algorithm to each integration, comparing each pixel to its local neighborhood to detect spatial outliers that deviate significantly from the surrounding background. Both positive and negative outliers can be flagged, and optional weighting by the pixel uncertainties can be applied. This method is particularly useful for finding missed/new hot pixels.

The behavior of sigclip can be customized using the sigclip_kwargs dictionary. Key options include:

• sigma (float, optional) : Sigma threshold for flagging positive outliers. Default is 5.

• neg_sigma (float, optional) : Sigma threshold for flagging negative outliers. Default is 1.

• shift_x / shift_y (list of int, optional) : Pixels in x- and y-directions used for local neighborhood comparisons. Defaults are [-1, 0, 1].

  In practice, the default neighborhood can be too small, causing overly aggressive flagging. Expanding it (e.g., [-2, -1, 0, 1, 2] in both x and y) generally yields more reliable results.

• diagonal_only (bool, optional) : If True, only diagonal neighbors are used for median calculations. Default is False.

• max_cluster_size (int, optional) : Maximum size of bad pixel clusters to be flagged. If None, no limit is applied.

• cluster_dilate_radius (int, optional) : Radius used when dilating clusters of bad pixels before applying cluster size filtering. Default is 6 pixels.

  Together, the parameters above help prevent overflagging in NIRCam data, where background PSFs can appear in the same image as the science target. Standard sigma clipping can mistakenly identify these PSFs as large clusters of outliers, even though they are not bad pixels. These parameters can be tuned to the specific dataset to limit flagging in these regions, as illustrated in the example below.

• threshold_metric : str, optional Metric to compute local variability: either ‘std’ (standard deviation) or ‘mad’ (median absolute deviation). Default is ‘std’.

• method (str, optional) : Sigma-clipping strategy. Options are:

  • local : Flags pixels that deviate from the median of their neighboring pixels.

  • local_weighted : Same as ‘local’, but includes pixel uncertainties when computing the threshold. This mode can help prevent overflagging in the neutral density square regions of NIRCam data.

• mask_psf (bool, optional) : Restrict bad pixel flagging inside the PSF? Default is False.

• crpix1 / crpix2 (float, optional) : Coordinates of the PSF center used if mask_psf is True. By default will pull from the file headers.

  Depending on the setup, sigma clipping can sometimes overflag pixels near the center of the PSF. For NIRCam data, there is an option to prevent flagging in the PSF core. This option does not affect MIRI data.

Below, we explore several setups for the sigma clipping method to demonstrate how each parameter affects the flagged pixels.

# Base sigclip kwargs.
base_sigclip_kwargs = {
    'sigma': 3,
    'neg_sigma': 3,
    'shift_x': [-2, -1, 0, 1, 2],
    'shift_y': [-2, -1, 0, 1, 2],
    'mode': 'local',
    'threshold_metric': 'std',
    'mask_psf': False,
    'cluster_dilate_radius': 6,
    'max_cluster_size': None,
    'diagonal_only': False,
    'crpix1': None,
    'crpix2': None
}

# Define the parameter sets to run for "sigclip".
sigclip_param_sets = [

    {'method': 'sigclip', 'sigclip_kwargs': base_sigclip_kwargs},  # Run 1: Base Run
    {'method': 'sigclip', 'sigclip_kwargs': {**base_sigclip_kwargs, 'sigma': 5, 'neg_sigma': 5}},  # Run 2: Sigma thresholds
    {'method': 'sigclip', 'sigclip_kwargs': {**base_sigclip_kwargs, 'shift_x': [-1, 0, 1], 'shift_y': [-1, 0, 1]}},  # Run 3: Neighborhood size
    {'method': 'sigclip', 'sigclip_kwargs': {**base_sigclip_kwargs, 'threshold_metric': 'mad'}},  # Run 4: Threshold metric
    {'method': 'sigclip', 'sigclip_kwargs': {**base_sigclip_kwargs, 'mode': 'local_weighted', 'threshold_metric': 'mad'}},  # Run 5: Method comparison
    {'method': 'sigclip', 'sigclip_kwargs': {**base_sigclip_kwargs, 'mode': 'local_weighted', 'threshold_metric': 'mad', 'mask_psf': True}},  # Run 6: PSF masking
    {'method': 'sigclip', 'sigclip_kwargs': {**base_sigclip_kwargs, 'mode': 'local_weighted', 'threshold_metric': 'mad', 'mask_psf': True, 'cluster_dilate_radius': 6, 'max_cluster_size': 9}},  # Run 7: Clustering OFF vs. ON
    {'method': 'sigclip', 'sigclip_kwargs': {**base_sigclip_kwargs, 'mode': 'local_weighted', 'threshold_metric': 'mad', 'mask_psf': True, 'cluster_dilate_radius': 6, 'max_cluster_size': 9, 'diagonal_only': True}},  # Run 8: Only diagonal neighbors

]

Refer to the run_bad_pixel_methods function defined at the top to see how this step was run through the pipeline.

# Run NIRCam examples with "sigclip".
sigclip_nircam_files, sigclip_kwargs_list = run_bad_pixel_methods(data_root_nircam, sigclip_param_sets)

# Plot NIRCam examples after running "sigclip".
compare_find_methods(files=[run_files[0] for run_files in sigclip_nircam_files],
                     kwargs_list=sigclip_kwargs_list,
                     interactive=interactive)

------------------ SIGCLIP RUN 1 ------------------
Reading from: ./bad_pixel_testing/nircam/
Writing to: ./bad_pixel_testing/nircam/sigclip_run1
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_NIRCAM_NRCALONG_F444W_MASKRND_MASK335R_SUB320A335R
[spaceKLIP.imagetools:INFO]   --> Method sigclip: jw01386003001_0310a_00001_nrcalong_calints.fits
Frame 2/2, iteration 5
[spaceKLIP.imagetools:INFO]   --> Method sigclip: identified 2563 additional bad pixel(s) -- 1.25%

------------------ SIGCLIP RUN 2 ------------------
Reading from: ./bad_pixel_testing/nircam/
Writing to: ./bad_pixel_testing/nircam/sigclip_run2
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_NIRCAM_NRCALONG_F444W_MASKRND_MASK335R_SUB320A335R
[spaceKLIP.imagetools:INFO]   --> Method sigclip: jw01386003001_0310a_00001_nrcalong_calints.fits
Frame 2/2, iteration 4
[spaceKLIP.imagetools:INFO]   --> Method sigclip: identified 44 additional bad pixel(s) -- 0.02%

------------------ SIGCLIP RUN 3 ------------------
Reading from: ./bad_pixel_testing/nircam/
Writing to: ./bad_pixel_testing/nircam/sigclip_run3
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_NIRCAM_NRCALONG_F444W_MASKRND_MASK335R_SUB320A335R
[spaceKLIP.imagetools:INFO]   --> Method sigclip: jw01386003001_0310a_00001_nrcalong_calints.fits
Frame 2/2, iteration 4
[spaceKLIP.imagetools:INFO]   --> Method sigclip: identified 4863 additional bad pixel(s) -- 2.37%

------------------ SIGCLIP RUN 4 ------------------
Reading from: ./bad_pixel_testing/nircam/
Writing to: ./bad_pixel_testing/nircam/sigclip_run4
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_NIRCAM_NRCALONG_F444W_MASKRND_MASK335R_SUB320A335R
[spaceKLIP.imagetools:INFO]   --> Method sigclip: jw01386003001_0310a_00001_nrcalong_calints.fits
Frame 2/2, iteration 10
[spaceKLIP.imagetools:INFO]   --> Method sigclip: identified 9029 additional bad pixel(s) -- 4.41%

------------------ SIGCLIP RUN 5 ------------------
Reading from: ./bad_pixel_testing/nircam/
Writing to: ./bad_pixel_testing/nircam/sigclip_run5
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_NIRCAM_NRCALONG_F444W_MASKRND_MASK335R_SUB320A335R
[spaceKLIP.imagetools:INFO]   --> Method sigclip: jw01386003001_0310a_00001_nrcalong_calints.fits
Frame 2/2, iteration 8
[spaceKLIP.imagetools:INFO]   --> Method sigclip: identified 6467 additional bad pixel(s) -- 3.16%

------------------ SIGCLIP RUN 6 ------------------
Reading from: ./bad_pixel_testing/nircam/
Writing to: ./bad_pixel_testing/nircam/sigclip_run6
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_NIRCAM_NRCALONG_F444W_MASKRND_MASK335R_SUB320A335R
[spaceKLIP.imagetools:INFO]   --> Method sigclip: jw01386003001_0310a_00001_nrcalong_calints.fits
Frame 2/2, iteration 6
[spaceKLIP.imagetools:INFO]   --> Method sigclip: identified 710 additional bad pixel(s) -- 0.35%

------------------ SIGCLIP RUN 7 ------------------
Reading from: ./bad_pixel_testing/nircam/
Writing to: ./bad_pixel_testing/nircam/sigclip_run7
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_NIRCAM_NRCALONG_F444W_MASKRND_MASK335R_SUB320A335R
[spaceKLIP.imagetools:INFO]   --> Method sigclip: jw01386003001_0310a_00001_nrcalong_calints.fits
Frame 2/2, iteration 3
[spaceKLIP.imagetools:INFO]   --> Method sigclip: identified 101 additional bad pixel(s) -- 0.05%

------------------ SIGCLIP RUN 8 ------------------
Reading from: ./bad_pixel_testing/nircam/
Writing to: ./bad_pixel_testing/nircam/sigclip_run8
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_NIRCAM_NRCALONG_F444W_MASKRND_MASK335R_SUB320A335R
[spaceKLIP.imagetools:INFO]   --> Method sigclip: jw01386003001_0310a_00001_nrcalong_calints.fits
Frame 2/2, iteration 3
[spaceKLIP.imagetools:INFO]   --> Method sigclip: identified 64 additional bad pixel(s) -- 0.03%

# Run MIRI examples with "sigclip".
sigclip_miri_files, sigclip_kwargs_list = run_bad_pixel_methods(data_root_miri, sigclip_param_sets)

# Plot MIRI examples after running "sigclip".
compare_find_methods(files=[run_files[0] for run_files in sigclip_miri_files],
                     kwargs_list=sigclip_kwargs_list,
                     interactive=interactive)

------------------ SIGCLIP RUN 1 ------------------
Reading from: ./bad_pixel_testing/miri/
Writing to: ./bad_pixel_testing/miri/sigclip_run1
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_MIRI_MIRIMAGE_F1550C_NONE_4QPM_1550_MASK1550
[spaceKLIP.imagetools:INFO]   --> Method sigclip: jw01386008001_04101_00001_mirimage_calints.fits
Frame 60/60, iteration 3
[spaceKLIP.imagetools:INFO]   --> Method sigclip: identified 15588 additional bad pixel(s) -- 0.58%
[spaceKLIP.imagetools:INFO]   --> Method sigclip: jw01386030001_02101_00001_mirimage_calints.fits
Frame 60/60, iteration 4
[spaceKLIP.imagetools:INFO]   --> Method sigclip: identified 15325 additional bad pixel(s) -- 0.57%

------------------ SIGCLIP RUN 2 ------------------
Reading from: ./bad_pixel_testing/miri/
Writing to: ./bad_pixel_testing/miri/sigclip_run2
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_MIRI_MIRIMAGE_F1550C_NONE_4QPM_1550_MASK1550
[spaceKLIP.imagetools:INFO]   --> Method sigclip: jw01386008001_04101_00001_mirimage_calints.fits
Frame 60/60, iteration 3
[spaceKLIP.imagetools:INFO]   --> Method sigclip: identified 3930 additional bad pixel(s) -- 0.15%
[spaceKLIP.imagetools:INFO]   --> Method sigclip: jw01386030001_02101_00001_mirimage_calints.fits
Frame 60/60, iteration 3
[spaceKLIP.imagetools:INFO]   --> Method sigclip: identified 3894 additional bad pixel(s) -- 0.14%

------------------ SIGCLIP RUN 3 ------------------
Reading from: ./bad_pixel_testing/miri/
Writing to: ./bad_pixel_testing/miri/sigclip_run3
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_MIRI_MIRIMAGE_F1550C_NONE_4QPM_1550_MASK1550
[spaceKLIP.imagetools:INFO]   --> Method sigclip: jw01386008001_04101_00001_mirimage_calints.fits
Frame 60/60, iteration 4
[spaceKLIP.imagetools:INFO]   --> Method sigclip: identified 48337 additional bad pixel(s) -- 1.78%
[spaceKLIP.imagetools:INFO]   --> Method sigclip: jw01386030001_02101_00001_mirimage_calints.fits
Frame 60/60, iteration 4
[spaceKLIP.imagetools:INFO]   --> Method sigclip: identified 49197 additional bad pixel(s) -- 1.82%

------------------ SIGCLIP RUN 4 ------------------
Reading from: ./bad_pixel_testing/miri/
Writing to: ./bad_pixel_testing/miri/sigclip_run4
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_MIRI_MIRIMAGE_F1550C_NONE_4QPM_1550_MASK1550
[spaceKLIP.imagetools:INFO]   --> Method sigclip: jw01386008001_04101_00001_mirimage_calints.fits
Frame 60/60, iteration 80
[spaceKLIP.imagetools:INFO]   --> Method sigclip: identified 94117 additional bad pixel(s) -- 3.47%
[spaceKLIP.imagetools:INFO]   --> Method sigclip: jw01386030001_02101_00001_mirimage_calints.fits
Frame 60/60, iteration 80
[spaceKLIP.imagetools:INFO]   --> Method sigclip: identified 94412 additional bad pixel(s) -- 3.49%

------------------ SIGCLIP RUN 5 ------------------
Reading from: ./bad_pixel_testing/miri/
Writing to: ./bad_pixel_testing/miri/sigclip_run5
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_MIRI_MIRIMAGE_F1550C_NONE_4QPM_1550_MASK1550
[spaceKLIP.imagetools:INFO]   --> Method sigclip: jw01386008001_04101_00001_mirimage_calints.fits
Frame 60/60, iteration 2
[spaceKLIP.imagetools:INFO]   --> Method sigclip: identified 0 additional bad pixel(s) -- 0.00%
[spaceKLIP.imagetools:INFO]   --> Method sigclip: jw01386030001_02101_00001_mirimage_calints.fits
Frame 60/60, iteration 2
[spaceKLIP.imagetools:INFO]   --> Method sigclip: identified 1 additional bad pixel(s) -- 0.00%

------------------ SIGCLIP RUN 6 ------------------
Reading from: ./bad_pixel_testing/miri/
Writing to: ./bad_pixel_testing/miri/sigclip_run6
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_MIRI_MIRIMAGE_F1550C_NONE_4QPM_1550_MASK1550
[spaceKLIP.imagetools:INFO]   --> Method sigclip: jw01386008001_04101_00001_mirimage_calints.fits
Frame 60/60, iteration 2
[spaceKLIP.imagetools:INFO]   --> Method sigclip: identified 0 additional bad pixel(s) -- 0.00%
[spaceKLIP.imagetools:INFO]   --> Method sigclip: jw01386030001_02101_00001_mirimage_calints.fits
Frame 60/60, iteration 2
[spaceKLIP.imagetools:INFO]   --> Method sigclip: identified 1 additional bad pixel(s) -- 0.00%

------------------ SIGCLIP RUN 7 ------------------
Reading from: ./bad_pixel_testing/miri/
Writing to: ./bad_pixel_testing/miri/sigclip_run7
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_MIRI_MIRIMAGE_F1550C_NONE_4QPM_1550_MASK1550
[spaceKLIP.imagetools:INFO]   --> Method sigclip: jw01386008001_04101_00001_mirimage_calints.fits
Frame 60/60, iteration 2
[spaceKLIP.imagetools:INFO]   --> Method sigclip: identified 0 additional bad pixel(s) -- 0.00%
[spaceKLIP.imagetools:INFO]   --> Method sigclip: jw01386030001_02101_00001_mirimage_calints.fits
Frame 60/60, iteration 2
[spaceKLIP.imagetools:INFO]   --> Method sigclip: identified 1 additional bad pixel(s) -- 0.00%

------------------ SIGCLIP RUN 8 ------------------
Reading from: ./bad_pixel_testing/miri/
Writing to: ./bad_pixel_testing/miri/sigclip_run8
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_MIRI_MIRIMAGE_F1550C_NONE_4QPM_1550_MASK1550
[spaceKLIP.imagetools:INFO]   --> Method sigclip: jw01386008001_04101_00001_mirimage_calints.fits
Frame 60/60, iteration 2
[spaceKLIP.imagetools:INFO]   --> Method sigclip: identified 0 additional bad pixel(s) -- 0.00%
[spaceKLIP.imagetools:INFO]   --> Method sigclip: jw01386030001_02101_00001_mirimage_calints.fits
Frame 60/60, iteration 2
[spaceKLIP.imagetools:INFO]   --> Method sigclip: identified 1 additional bad pixel(s) -- 0.00%

• Run 1 vs. 2: Lower sigma thresholds identify more candidate bad pixels in both the NIRCam and MIRI data.

• Run 2 vs. 3: Increasing the neighborhood size used for comparison provides a more stable estimate of the local pixel behavior in both the NIRCam and MIRI data, reducing overly aggressive flagging.

• Run 3 vs. 4: Changing the threshold metric from standard deviation to MAD reduces overflagging in the background regions, but leads to significant overflagging in the PSF regions for both MIRI and NIRCam, as well as within the neutral density squares and around background sources in the NIRCam data. Additional parameters introduced in the following runs can better handle these regions.

• Run 4 vs. 5: Using the local_weighted mode improves the overflagging behavior in both the MIRI and NIRCam data, though the NIRCam data still shows overflagging around the PSFs.

• Run 5 vs. 6: Enabling mask_psf removes the overflagging in the central PSF region for the NIRCam data, though this parameter is not applicable to the MIRI data.

• Run 6 vs. 7 vs. 8: Enabling the clustering parameters helps reduce overflagging in the PSF centers of the background sources within the field. No real differences for MIRI.

---

Identify Bad Pixels: Custom Mask

The custom method lets you manually mark bad pixels that other methods might miss. This is particularly useful for incorporating prior knowledge of the bad pixels on the detector or results from external analyses.

The binary mask you provide is applied directly: pixels marked as bad (DO_NOT_USE) are added to the existing DQ mask.

# Make database for the custom mask kwargs dict.
database = spaceKLIP.database.create_database(input_dir=data_root_nircam,
                                              file_type='calints.fits',
                                              output_dir=data_root_nircam,
                                              verbose=False)

# Define coordinates for F444W bad pixels.
f444w_coords = [
    [209, 205], [210, 205], [209, 206], [210, 206],
    [210, 207], [211, 206], [212, 206], [188, 73],
    [189, 73], [188, 74], [280, 153], [279, 153],
    [280, 154], [279, 154], [279, 155], [280, 156],
    [24, 216], [25, 216]
]

# Initialize custom masks dictionary.
custom_mask = {}

for i in database.obs.keys():
    key = str(i)
    data = fits.getdata(database.obs[i][0]['FITSFILE'], ext=1)  # Read SCI extension.
    nints, ny, nx = data.shape if data.ndim == 3 else (1, *data.shape)
    custom_mask[key] = np.zeros((nints, ny, nx), dtype=bool)

    # Insert custom bad pixels.
    if 'F444W' in key:
        for x, y in f444w_coords:
            # Safety bounds check.
            if 0 <= x < nx and 0 <= y < ny:
                custom_mask[key][:, y, x] = True

# Run NIRCam examples with "custom" mask.
custom_params = [{'method': 'custom', 'custom_kwargs': custom_mask}]
custom_nircam_files, custom_kwargs_list = run_bad_pixel_methods(data_root_nircam, custom_params)

# Plot NIRCam examples after running "custom" mask.
compare_find_methods(files=[run_files[0] for run_files in custom_nircam_files],
                     kwargs_list=custom_kwargs_list,
                     interactive=interactive)

------------------ CUSTOM RUN 1 ------------------
Reading from: ./bad_pixel_testing/nircam/
Writing to: ./bad_pixel_testing/nircam/custom_run1
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_NIRCAM_NRCALONG_F444W_MASKRND_MASK335R_SUB320A335R
[spaceKLIP.imagetools:INFO]   --> Method custom: jw01386003001_0310a_00001_nrcalong_calints.fits
[spaceKLIP.imagetools:INFO]   --> Method custom: flagged 36 additional bad pixel(s) -- 0.02%

For MIRI, one effective way to detect bad pixels is to run a simple sigma-clipping (sigclip) detection on a background image (it’s always recommended to take a background with MIRI coronagraphy). The bad pixels detected in the background can then be used to flag corresponding pixels in the science images.

Below, we demonstrate this using the MIRI background and science example. We zoom in on a specific bad pixel identified in the background image and can clearly see that it is also bad in the science image.

# Make database for the custom mask kwargs dict.
database = spaceKLIP.database.create_database(input_dir=data_root_miri,
                                              file_type='calints.fits',
                                              output_dir=data_root_miri,
                                              verbose=False)


# Grab the bad pixel mask from the sigclip run on the background image.
miri_bg_file = sigclip_miri_files[1][1]  # Sigclip run 2.
data = fits.getdata(miri_bg_file, ext=1)
nints, ny, nx = data.shape if data.ndim == 3 else (1, *data.shape)

# Initialize custom mask.
custom_mask = np.zeros((nints, ny, nx), dtype=bool)

# Locate the bad pixels in the background image.
with fits.open(miri_bg_file) as hdul:
    dq = hdul["DQ"].data
    consistent = np.all(dq == 1, axis=0)
    y, x = np.where(consistent)
    custom_mask[:, y, x] = True

# Run MIRI examples with "custom" mask.
custom_mask_dict = {key: custom_mask.copy() for key in database.obs.keys()}
custom_params = [{'method': 'custom', 'custom_kwargs': custom_mask_dict}]
custom_miri_files, custom_kwargs_list = run_bad_pixel_methods(data_root_miri, custom_params)

# Plot MIRI examples after running "custom" mask.
compare_find_methods(files=[run_files[0] for run_files in custom_miri_files],
                     kwargs_list=custom_kwargs_list,
                     interactive=interactive)

------------------ CUSTOM RUN 1 ------------------
Reading from: ./bad_pixel_testing/miri/
Writing to: ./bad_pixel_testing/miri/custom_run1
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_MIRI_MIRIMAGE_F1550C_NONE_4QPM_1550_MASK1550
[spaceKLIP.imagetools:INFO]   --> Method custom: jw01386008001_04101_00001_mirimage_calints.fits
[spaceKLIP.imagetools:INFO]   --> Method custom: flagged 960 additional bad pixel(s) -- 0.04%
[spaceKLIP.imagetools:INFO]   --> Method custom: jw01386030001_02101_00001_mirimage_calints.fits
[spaceKLIP.imagetools:INFO]   --> Method custom: flagged 960 additional bad pixel(s) -- 0.04%

Now we will move on to the supported options to clean these bad pixels.

---

Clean Methods

Now that bad pixels have been identified and the DQ array updated, the next step is to clean the data. The spaceKLIP pipeline provides several methods for replacing bad pixels, each with configurable parameters:

• timemed: Replaces pixels that are only flagged as bad in some frames with the median value computed from the corresponding good frames across time. This method is typically used after the timeints bad pixel identification step (shown in the next section).

  There are no configurable parameters for this step. However, when working with datasets that have few integrations, this method may perform poorly (shown below). If a pixel is flagged as bad in most or all frames, there may be insufficient valid data to compute a reliable median, which can result in NaNs remaining in the output.

• localmed: Replaces bad pixels using the median of neighboring good pixels in the spatial domain.

  • shift_x (list of int, optional): Pixel offsets in the x-direction used to compute the median. Default: [-1, 0, 1].

  • shift_y (list of int, optional): Pixel offsets in the y-direction used to compute the median. Default: [-1, 0, 1].

  In practice, the default neighborhood can be too small, and the replacement can be strongly impacted by unflagged elevated neighboring pixels. Expanding it (e.g., [-2, -1, 0, 1, 2] in both x and y) generally yields more reliable results.

• medfilt: Applies a median filter to the image and replaces bad pixels with the filtered values.

  • size (int, optional): Kernel size of the median filter which controls how many neighboring pixels are used for the median replacement. Default: 4.

• interp2d: Replaces bad pixels through interpolation using neighboring pixel values.

  • size (int, optional): Kernel size used for the interpolation window. Default: 4

• astrofix: An astronomical image correction algorithm based on Gaussian Process Regression. It learns an optimal interpolation kernel directly from the data for each image, enabling significantly improved reconstruction compared to standard median replacement or fixed-kernel interpolation methods (astrofix).

  • sig_clip (float, optional): Pixels below median + sig_clip × MAD are excluded from training. Default: 10

  • max_clip (float, optional): Pixels above max(image) / max_clip are excluded from training. Default: 5

  • sig_data (float, optional): Assumed measurement noise level (uniform). The kernel depends on the ratio relative to this value. Default: 1

  • width (int, optional): Size of the local window (width × width) used for interpolation. Default: 9

  • init_guess (array-like, optional): Initial guess for the training process. By default, the 0th element gives the initial guess of a, and the 1st element gives the initial guess of h. If the size of init_guess is 3, the training optimizes h_x and h_y separately instead of using h for all directions. In that case, the 1st element gives the initial guess of h_x, and the 2nd element gives the initial guess of h_y. Default: [1,1].

  The sig_clip and max_clip parameters can be tuned directly to the dataset being analyzed to improve the training and performance of the pixel replacement process.

Below, we select one of the runs from above as an example for cleaning.

# Choose a "dqarr" run to start from.
#find_subdir = "dqarr_run2"  # With the expanded neighbors.
find_subdir = "sigclip_run2"

# Define the parameter sets to run for cleaning.
clean_param_sets = [

    {'input_dir': find_subdir, 'method': 'timemed', 'timemed_kwargs': {}},
    {'input_dir': find_subdir, 'method': 'localmed', 'localmed_kwargs': {'shift_x': [-2, -1, 0, 1, 2], 'shift_y': [-2, -1, 0, 1, 2]}},
    {'input_dir': find_subdir, 'method': 'medfilt', 'medfilt_kwargs': {'size': 5}},
    {'input_dir': find_subdir, 'method': 'interp2d', 'interp2d_kwargs': {'size': 5}},
    {'input_dir': find_subdir, 'method': 'astrofix', 'astrofix_kwargs': {'sig_clip': 5, 'max_clip': 2, 'width': 3}},

]

Refer to the functions run_bad_pixel_methods and compare_find_clean defined at the top to see how this step was run and plotted.

# Run NIRCam examples through cleaning methods.
nircam_clean_files, clean_kwargs_list = run_bad_pixel_methods(data_root_nircam, clean_param_sets)

# Plot cleaned NIRCam examples.
compare_clean_methods(files=[run_files[0] for run_files in nircam_clean_files],
                      kwargs_list=clean_kwargs_list, interactive=interactive,
                      #zoom_region=(200, 215, 196, 212),
                      zoom_region=(220, 240, 60, 80),
                      )

------------------ TIMEMED RUN 1 ------------------
Reading from: ./bad_pixel_testing/nircam/sigclip_run2
Writing to: ./bad_pixel_testing/nircam/timemed_run1
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_NIRCAM_NRCALONG_F444W_MASKRND_MASK335R_SUB320A335R
[spaceKLIP.imagetools:INFO]   --> Method timemed: jw01386003001_0310a_00001_nrcalong_calints.fits
[spaceKLIP.imagetools:INFO]   --> Method timemed: fixing 36 bad pixel(s) -- 0.02%

------------------ LOCALMED RUN 2 ------------------
Reading from: ./bad_pixel_testing/nircam/sigclip_run2
Writing to: ./bad_pixel_testing/nircam/localmed_run2
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_NIRCAM_NRCALONG_F444W_MASKRND_MASK335R_SUB320A335R
[spaceKLIP.imagetools:INFO]   --> Method localmed: jw01386003001_0310a_00001_nrcalong_calints.fits
[spaceKLIP.imagetools:INFO]   --> Method localmed: fixing 3180 bad pixel(s) -- 1.55%

------------------ MEDFILT RUN 3 ------------------
Reading from: ./bad_pixel_testing/nircam/sigclip_run2
Writing to: ./bad_pixel_testing/nircam/medfilt_run3
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_NIRCAM_NRCALONG_F444W_MASKRND_MASK335R_SUB320A335R
[spaceKLIP.imagetools:INFO]   --> Method medfilt: jw01386003001_0310a_00001_nrcalong_calints.fits
[spaceKLIP.imagetools:INFO]   --> Method medfilt: fixing 3180 bad pixel(s) -- 1.55%

------------------ INTERP2D RUN 4 ------------------
Reading from: ./bad_pixel_testing/nircam/sigclip_run2
Writing to: ./bad_pixel_testing/nircam/interp2d_run4
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_NIRCAM_NRCALONG_F444W_MASKRND_MASK335R_SUB320A335R
[spaceKLIP.imagetools:INFO]   --> Method interp2d: jw01386003001_0310a_00001_nrcalong_calints.fits
[spaceKLIP.imagetools:INFO]   --> Method interp2d: fixing 3180 bad pixel(s) -- 1.55%

------------------ ASTROFIX RUN 5 ------------------
Reading from: ./bad_pixel_testing/nircam/sigclip_run2
Writing to: ./bad_pixel_testing/nircam/astrofix_run5
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_NIRCAM_NRCALONG_F444W_MASKRND_MASK335R_SUB320A335R
[spaceKLIP.imagetools:INFO]   --> Method astrofix: jw01386003001_0310a_00001_nrcalong_calints.fits
[spaceKLIP.imagetools:INFO]   --> Method astrofix: fixing 3180 bad pixel(s) -- 1.55%

In the example region displayed above, the bad pixel at the center of the background PSF is best corrected by the interp2d and astrofix methods, while the other methods replace that pixel poorly. The timemed method also performs poorly in cases with a small number of integrations (pixels remain NaN).

Now we can clean the MIRI data.

# Run MIRI examples through cleaning methods.
miri_clean_files, clean_kwargs_list = run_bad_pixel_methods(data_root_miri, clean_param_sets)

# Plot cleaned MIRI examples.
compare_clean_methods(files=[run_files[0] for run_files in miri_clean_files],
                      kwargs_list=clean_kwargs_list,
                      interactive=interactive,
                      zoom_region=(90, 120, 90, 120))

------------------ TIMEMED RUN 1 ------------------
Reading from: ./bad_pixel_testing/miri/sigclip_run2
Writing to: ./bad_pixel_testing/miri/timemed_run1
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_MIRI_MIRIMAGE_F1550C_NONE_4QPM_1550_MASK1550
[spaceKLIP.imagetools:INFO]   --> Method timemed: jw01386008001_04101_00001_mirimage_calints.fits
[spaceKLIP.imagetools:INFO]   --> Method timemed: fixing 2971 bad pixel(s) -- 0.11%
[spaceKLIP.imagetools:INFO]   --> Method timemed: jw01386030001_02101_00001_mirimage_calints.fits
[spaceKLIP.imagetools:INFO]   --> Method timemed: fixing 2938 bad pixel(s) -- 0.11%

------------------ LOCALMED RUN 2 ------------------
Reading from: ./bad_pixel_testing/miri/sigclip_run2
Writing to: ./bad_pixel_testing/miri/localmed_run2
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_MIRI_MIRIMAGE_F1550C_NONE_4QPM_1550_MASK1550
[spaceKLIP.imagetools:INFO]   --> Method localmed: jw01386008001_04101_00001_mirimage_calints.fits
[spaceKLIP.imagetools:INFO]   --> Method localmed: fixing 26311 bad pixel(s) -- 0.97%
[spaceKLIP.imagetools:INFO]   --> Method localmed: jw01386030001_02101_00001_mirimage_calints.fits
[spaceKLIP.imagetools:INFO]   --> Method localmed: fixing 26278 bad pixel(s) -- 0.97%

------------------ MEDFILT RUN 3 ------------------
Reading from: ./bad_pixel_testing/miri/sigclip_run2
Writing to: ./bad_pixel_testing/miri/medfilt_run3
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_MIRI_MIRIMAGE_F1550C_NONE_4QPM_1550_MASK1550
[spaceKLIP.imagetools:INFO]   --> Method medfilt: jw01386008001_04101_00001_mirimage_calints.fits
[spaceKLIP.imagetools:INFO]   --> Method medfilt: fixing 26311 bad pixel(s) -- 0.97%
[spaceKLIP.imagetools:INFO]   --> Method medfilt: jw01386030001_02101_00001_mirimage_calints.fits
[spaceKLIP.imagetools:INFO]   --> Method medfilt: fixing 26278 bad pixel(s) -- 0.97%

------------------ INTERP2D RUN 4 ------------------
Reading from: ./bad_pixel_testing/miri/sigclip_run2
Writing to: ./bad_pixel_testing/miri/interp2d_run4
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_MIRI_MIRIMAGE_F1550C_NONE_4QPM_1550_MASK1550
[spaceKLIP.imagetools:INFO]   --> Method interp2d: jw01386008001_04101_00001_mirimage_calints.fits
[spaceKLIP.imagetools:INFO]   --> Method interp2d: fixing 26311 bad pixel(s) -- 0.97%
[spaceKLIP.imagetools:INFO]   --> Method interp2d: jw01386030001_02101_00001_mirimage_calints.fits
[spaceKLIP.imagetools:INFO]   --> Method interp2d: fixing 26278 bad pixel(s) -- 0.97%

------------------ ASTROFIX RUN 5 ------------------
Reading from: ./bad_pixel_testing/miri/sigclip_run2
Writing to: ./bad_pixel_testing/miri/astrofix_run5
[spaceKLIP.imagetools:INFO] --> Concatenation JWST_MIRI_MIRIMAGE_F1550C_NONE_4QPM_1550_MASK1550
[spaceKLIP.imagetools:INFO]   --> Method astrofix: jw01386008001_04101_00001_mirimage_calints.fits
[spaceKLIP.imagetools:INFO]   --> Method astrofix: fixing 26311 bad pixel(s) -- 0.97%
[spaceKLIP.imagetools:INFO]   --> Method astrofix: jw01386030001_02101_00001_mirimage_calints.fits
[spaceKLIP.imagetools:INFO]   --> Method astrofix: fixing 26278 bad pixel(s) -- 0.97%

Similarly to the NIRCam example, the clearest comparison of the replacement methods is at the center of the MIRI PSF, where the differences in how each method handles the bad pixel are most apparent.

---

Conclusions

Typically, these methods are run in the following order: dqarr → clean, timeints → clean, sigclip → clean, and custom → clean. This sequence is similar to the workflow shown in the reduction notebooks linked at the top of this notebook.