> For the complete documentation index, see [llms.txt](https://docs.nimbusimage.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.nimbusimage.com/documentation/analyzing-image-data-with-objects-connections-and-properties/tools-for-making-objects.md).

# Tools for making objects

The way to create and edit objects is through the use of **tools**. Tools are defined by the user and are customized (via tags) to flexible organization of the results without a lot of clicking. There are two general categories of tools, manual and automated.

<figure><img src="https://4239675806-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FX54NriBmLyFl6hdSGC9K%2Fuploads%2Fgit-blob-05f8ca2795e937f22993dc98e3640a7d12b2720c%2Fadd-tool-dialog.png?alt=media" alt=""><figcaption><p>The "Add new tool" dialog, where tools are organized by category (manual object tools, Segment Anything, connections, line scans, and more)</p></figcaption></figure>

**Manual tools.** Manual tools are often based on a primitive type, like a point, line, or blob, and is customized with tags and other features. For instance, you might make a tool for circling cells in your image. You would choose a manual blob creation tool, name it "Nucleus" and add the tag \[nucleus]. You could also add a hotkey. That tool gets added to your interface and you can use it to circle cells at any time.

**Automated tools.** NimbusImage has been designed to allow the use of automated algorithms for finding things like cells and points in your images, often using the latest deep learning methods. For instance, you can set up an automated tool to use Cellpose to find cells within your images. These tools often have specific parameters that you can use to obtain optimal results.

## AI-suggested tools

When you open a freshly created collection that doesn't have any tools yet, NimbusImage can suggest a starting set of tools for you. It looks at your image and its channel names and proposes relevant tools — for example, Cellpose-SAM for nuclei, a manual blob tool for cells, or Piscis for spots — in a floating panel. Review the suggestions and click to add the ones you want. You can always add, remove, or reconfigure tools yourself afterward.

<figure><img src="https://4239675806-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FX54NriBmLyFl6hdSGC9K%2Fuploads%2Fgit-blob-a2fc1f8184aafe3348f9aaf276f8214c5c8246a8%2Fai-suggested-tools.png?alt=media" alt=""><figcaption><p>The "Suggested tools" panel proposes tools based on your image — here, H&#x26;E Deconvolution and a Blob tool for a histology slide</p></figcaption></figure>

## Organizing your tools

Hover over a tool in the Tools palette to reveal three controls:

* **Pin**: Moves the tool into a **Pinned** section at the top of the palette, above *Annotation tools* and *Analysis tools*. Click again to unpin it. Pin the handful of tools you use most so they're always within reach.
* **Edit**: Opens the tool's settings.
* **Drag grip**: Drag to reorder tools within their section. Each section is ordered independently, so a tool can't be dragged into a different section.

Pins and ordering are saved with the collection's configuration, so they apply to every dataset in the collection and for everyone who opens it — just like the tools themselves.

When an automated tool's settings menu is open, clicking anywhere outside it (or pressing Escape) closes it. Panning the image does not close the menu, and clicking a different automated tool switches the menu to that tool.

{% hint style="info" %}
You can also find and activate any tool — or start adding a new one — from the [command palette](/documentation/viewing-your-data.md#command-palette) with ⌘K / Ctrl+K.
{% endhint %}

## Manual blob/point/line/rectangle tools

Manual blob/point/line/rectangle tools are the most basic tools in NimbusImage. Set it up by clicking on "Add New Tool" and choosing "Manual Blob", "Manual Point", "Manual Line", or "Manual Rectangle". Set the tag, and then you can use it to create objects in your image. You can use the same tag for multiple tools. For instance, you can use an automated cell finding tool and then add more cells using the manual tool, and they will be all treated the same for downstream analysis.

## Segment-Anything semi-automated object finding, AKA "God Mode"

Segment-Anything is a new method for finding objects in images. It is a semi-automated tool that allows you to find objects in your images by either clicking on them or drawing a bounding box around them. To use it, click on "Add New Tool" and choose "ViT-B" under "Segment Anything Model".

{% hint style="info" %}
If you use Segment Anything Model (SAM) in your research, please cite the [relevant papers](/citations.md#segment-anything-model-sam).
{% endhint %}

Options include:

* **Simplification**: The simplification parameter controls how much the segmentation is smoothed out. Smoother annotations are faster to run computations on and navigate.
* **Turbo mode**: The turbo mode allows you to rapidly segment without having to manually "accept" each segment.

Segment Anything works by first encoding the image to enable it to find the objects. This will take a few seconds to compute when you move to a new area. Every time you move the image, you will need to re-encode the image; if you want to avoid this, click the "lock" icon at the bottom left of the image to prevent the image from moving around.

Once encoded, just move your mouse over the image and it will outline what the object would be if you were to shift-click. Shift-click to make the object! Sometimes, it will give you better results if you define a box around the object you want to find. Just shift-click and drag to make a box, and it will segment that region.

{% hint style="info" %}
Segment Anything "sees" what you see in the image. Adjust the contrast and zoom to make objects visible and of a size that is around 5-15% of the size of the image for best results.
{% endhint %}

## Segment similar objects (experimental)

The "Segment similar objects" tool lets you mark a few example objects and then automatically find and segment other, similar objects across the current view. It's designed for cases where objects share a consistent appearance but would be tedious to click one by one.

{% hint style="warning" %}
This tool is **experimental** and currently requires **Google Chrome with WebGPU** support. It operates on the current viewport only.
{% endhint %}

<figure><img src="https://4239675806-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FX54NriBmLyFl6hdSGC9K%2Fuploads%2Fgit-blob-5b4bd093fa04534d4b1f616e31f4c0e0579c6bcb%2Fsegment-similar-objects.png?alt=media" alt=""><figcaption><p>The Segment similar objects panel, with controls for how examples are selected and how matches are found</p></figcaption></figure>

### How to use

1. **Add the "Segment similar objects" tool** from the tool menu. A control panel appears at the bottom right of the viewer.
2. **Choose how to select examples** under **Select by**:
   * **Click (SAM)**: Shift-click an example object and Segment Anything outlines it.
   * **Box (SAM)**: Shift-drag a box around an example object.
   * **Circle**: Circle an example object by hand.
3. **Choose how to find matches** under **Find with**:
   * **SAM**: Scores candidate regions by how similar they are to your examples in the Segment Anything model's feature space.
   * **Classifier**: Trains a lightweight random-forest classifier, in your browser, on the example objects.
   * **SAM→Classifier**: Runs SAM first, then trains the classifier on everything SAM found, combining both approaches.
4. **Mark a few examples.** Shift-click (or shift-drag/circle) the objects you want to match; shift-right-click marks background to exclude. The panel tracks how many examples and putative objects you currently have.
5. **Review the putative results.** Matches appear as outlines across the current view.
6. **Accept** to commit the results in bulk. Accepted objects are deduplicated against any annotations that already exist.

{% hint style="info" %}
Because both propagation methods draw on the same set of examples, you can mix and match how you select examples with how they're applied — for instance, select with SAM clicks but propagate with the classifier.
{% endhint %}

Plain dragging pans the image as usual; hold Shift to select, matching the behavior of the Segment Anything tool.

## Edit objects

You can edit objects by using the Annotation Edits -> Blob edit tool. With that tool, you can just drag on your object and it will "slice" it into a new object. Whatever line you draw will define a new outline for that segment of the object. That allows you to both remove and add areas to the objects without having to use separate brush and eraser tools.

## Automated object finding and connection

We have a number of automated tools for finding and connecting objects (cells, spots, etc.) that take advantage of the latest deep learning methods.

### Cellpose-SAM for versatile cell segmentation

Cellpose-SAM combines the power of Cellpose with the Segment Anything Model (SAM) to offer a versatile approach to cell segmentation. It simplifies the input process by allowing users to specify up to three input channels, making it adaptable for various segmentation scenarios, from nuclei-only to full RGB images.

{% hint style="info" %}
If you use Cellpose-SAM in your research, please ensure you cite the relevant papers for both [Cellpose](/citations.md#cellpose) and the [Segment Anything Model (SAM)](/citations.md#segment-anything-model-sam).
{% endhint %}

#### How Cellpose-SAM works

Cellpose-SAM processes images by taking input from up to three configurable channel "slots":

* **Single Channel Input:**
  * To segment nuclei, provide the nuclear stain channel to Slot 1.
  * To segment cell boundaries, provide the cytoplasm/membrane channel to Slot 1.
* **Dual Channel Input:** For improved cell boundary segmentation, you can provide the cytoplasm/membrane channel to Slot 1 and the nuclear channel to Slot 2.
* **Triple Channel Input (RGB):** To segment cells in an RGB image, map the Red, Green, and Blue channels to Slots 1, 2, and 3 respectively.

The model then uses this information to perform segmentation.

#### Available models

Currently, the primary model available is:

* **cellpose-sam**: This is the base model and should be suitable for most general-purpose cell segmentation tasks.

#### Key parameters

* **Model**: Selects the segmentation model. Defaults to `cellpose-sam`.
* **Channel for Slot 1**: **Required.** The primary channel for segmentation. Select the source channel for the model's first input. If multiple are selected, only the first will be used.
* **Channel for Slot 2**: (Optional) The secondary channel, often used for nuclear information when segmenting cytoplasm. If multiple are selected, only the first will be used.
* **Channel for Slot 3**: (Optional) The tertiary channel, typically used for the blue channel in RGB images. If multiple are selected, only the first will be used.
* **Diameter**: The object size the image is rescaled to before segmentation (10-200 pixels, default 30). **Leave this at 30 for almost all images** — that is the identity value, meaning the image is segmented at its native resolution, which is how Cellpose-SAM was trained. The image is resized by `30 / Diameter`, so a smaller value enlarges the image and a larger one shrinks it. Change it only if your objects are far outside the size range the model handles well.
* **Smoothing**: Controls the simplification of the generated polygons (0-10, default 0.7). Higher values create smoother outlines. A value of 0.7 is a good default.
* **Padding**: Expands (positive values) or contracts (negative values) the final polygons in pixels (-20 to 20 pixels, default 0).
* **Tile Size**: The size of image tiles (in pixels) for processing (0-2048, default 1024). Larger tiles require more memory.
* **Tile Overlap**: The fractional overlap between adjacent tiles (0-1, default 0.1). Ensure this overlap is larger than your largest cells (e.g., for 1024px tiles with 0.1 overlap, objects should be <102px).

#### Best practices

1. **Channel Selection**:
   * For nuclei: Use your nuclear stain in Slot 1.
   * For cell boundaries: Use your cytoplasm/membrane channel in Slot 1. Consider adding a nuclear channel to Slot 2 for refinement.
   * For RGB images: Map R, G, B to Slots 1, 2, and 3.
2. **Diameter Setting**: Start with a diameter around 30 pixels. Adjust if necessary, but exact precision is less critical than with standard Cellpose.
3. **Review Results**: Always visually inspect segmentation and adjust parameters if needed.
4. **Post-process**: Utilize NimbusImage's manual editing tools to correct any segmentation errors.

### Cellpose for automated cell finding

Cellpose is a deep learning-based tool for automatically finding and segmenting cells in microscopy images. It has been trained on a large collection of diverse cell images, making it highly effective for many cell types without requiring additional training. Cellpose is a powerful starting point for cell segmentation, allowing you to quickly generate cell outlines that can be refined with NimbusImage's interactive tools.

{% hint style="info" %}
If you use Cellpose in your research, please cite the [relevant papers](/citations.md#cellpose).
{% endhint %}

{% hint style="info" %}
Note that NimbusImage also allows for retraining of the Cellpose models to (often greatly) enhance performance for your specific dataset.
{% endhint %}

#### Available models

NimbusImage includes several pre-trained Cellpose models:

* **cyto3**: The most accurate for general cell segmentation (recommended default)
* **cyto2**: An older model for cell segmentation
* **cyto**: The original cell segmentation model
* **nuclei**: Specifically trained for nuclear segmentation

#### Key parameters

* **Primary Channel**: The main channel to use for segmentation
  * For cytoplasm segmentation (Cyto3): use your cytoplasm/membrane channel
  * For nuclear segmentation (Nuclei): use your nuclear stain channel (e.g., DAPI)
* **Secondary Channel** (optional): A secondary channel to improve segmentation
  * For cytoplasm segmentation (Cyto3): you can add your nuclear channel
  * For nuclear segmentation (Nuclei): leave this blank (adding a secondary channel may decrease performance)
* **Diameter**: The approximate diameter of cells in pixels
  * **This parameter is crucial for good results**
  * Set this as close as possible to the actual cell diameter in your images
* **Smoothing**: Controls how much the cell outlines are simplified (0-10)
  * Higher values create smoother outlines with fewer vertices
  * Default of 0.7 works well for most images
  * Increase for smoother boundaries, decrease for more precise outlines
  * Very precise outlines can decrease performance, especially if you have a large number of objects
* **Padding**: Expand or contract the final cell outlines in pixels
  * Positive values expand the outlines (useful if cells appear too small)
  * Negative values contract the outlines (useful if segmentation is too generous)
  * Default of 0 means no adjustment

#### Advanced parameters for large images

* **Tile Size**: The size of each image tile in pixels
  * Default of 1024 works well for most images
  * Decrease if you encounter errors
  * Larger tiles will require more memory and can cause Cellpose to crash
* **Tile Overlap**: The fraction of overlap between adjacent tiles
  * Default of 0.1 (10% overlap) works well in most cases
  * **Important**: Make sure your overlap is larger than your largest cells
  * For example, for 1024 pixel tiles with 0.1 overlap, the largest cell should be less than 102 pixels across

#### Best practices

1. **Start with the right model**: Use cyto3 for general cell segmentation or nuclei for nuclear segmentation, or use your own custom retrained model
2. **Set the diameter carefully**: This is the most important parameter for accurate results
3. **Check results visually**: Always inspect the segmentation and refine parameters if needed
4. **Use the secondary channel** when segmenting cytoplasm if you have a good nuclear stain
5. **Post-process as needed**: Use NimbusImage's manual editing tools to correct any segmentation errors

### Cellpose Training

(Currently for Cellpose 3.0 only; Cellpose-SAM training coming soon!) NimbusImage offers the ability to train custom Cellpose models directly within the platform using your own annotated data. This feature allows you to create specialized segmentation models tailored to your specific cell types or imaging conditions. Custom models will often provide greatly improved performance for your specific dataset, even if your cells look pretty "normal", and especially if they don't!

#### Training data preparation

To train a custom model, you'll need:

1. Representative images with cells you want to segment
2. Accurate cell annotations (outlines) created manually or corrected from automated results
3. Optional: Defined regions of interest for training

#### Key parameters

* **Base Model**: The pre-trained model to use as a starting point
  * **cyto3**: Recommended for most cell types
  * **nuclei**: Use when training a nuclear segmentation model
  * **Custom models**: Previously trained models will also appear here
* **Nuclear Model?**: Check this box if you're training a model for nuclear segmentation
  * When checked, the model will be optimized for nuclear detection
* **Output Model Name**: Name for your custom model
  * This model will be saved and appear in the Cellpose worker's model list
* **Primary Channel**: The main channel for segmentation
  * For cytoplasm models: use your cytoplasm/membrane channel
  * For nuclear models: use your nuclear stain channel
* **Secondary Channel**: The secondary channel to improve segmentation
  * For cytoplasm models: your nuclear channel
  * For nuclear models: leave blank or set to -1
* **Training Tag**: The tag applied to annotated cells used for training
  * All cells with this tag will be used to train the model
  * Use this to select high-quality annotations
* **Training Region**: Optional tag (recommended) to define specific regions for training
  * If selected, only annotations within these regions will be used
  * Useful for limiting training to representative areas of your image

#### Training parameters

Unless you have a specific reason to change these, leave them at the default values.

* **Learning Rate**: Controls how quickly the model adapts during training
  * Default: 0.01
  * Lower values (0.001) create more stable but slower training
  * Higher values can speed up training but might decrease stability
* **Epochs**: Number of training iterations
  * Default: 1000
  * More epochs generally improve results but take longer
  * Consider increasing for complex cell types
* **Weight Decay**: Regularization parameter to prevent overfitting
  * Default: 0.0001
  * Higher values create simpler models that may generalize better

#### Training workflow

1. Create high-quality annotations of your cells
   * Tag these annotations with a consistent label (e.g., "training\_cells")
   * Ensure annotations are accurate and representative
   * **Don't forget that what you do NOT annotate will also be used for training!**
2. Optional: Define training regions
   * Create rectangles or polygons around areas with good examples
   * Tag these regions (e.g., "training\_region")
   * **Choosing specific regions can be very helpful in focusing the retraining on specific areas of your image**
3. Add a Cellpose Training tool from the toolset menu
   * Configure all parameters as described above
   * Run the training process
4. Use your custom model
   * After training completes, your model will appear in the Cellpose worker's model list
   * Select it when using the regular Cellpose tool for segmentation

#### Tips for successful training

1. **Provide diverse examples**: Include cells of different sizes, shapes, and intensities
2. **Quality over quantity**: A smaller set of perfect annotations is better than many poor ones. A few good images, carefully annotated, will usually do the trick.
3. **Choose representative regions**: Select areas with good imaging quality and typical cells. Be sure to include examples of cells that the default algorithm might have trouble with.
4. **Be patient**: Training can take several minutes to complete

#### Advanced usage

Your custom Cellpose models are saved in a `.cellpose` directory at the root of your Private or Public folder. You can:

* Manually add models trained offline to this directory
* Share models between team members by copying model files
* Use these models in the standard Cellpose Python package

The integration between NimbusImage and Cellpose makes it easy to create specialized models for your specific research needs without requiring deep learning expertise.

### Piscis for automated spot finding

Piscis is a deep learning-based tool for automatically finding and segmenting spots in microscopy images. It has been trained on a large collection of diverse spot images, making it highly effective for many spot types without requiring additional training. Piscis is a powerful starting point for spot segmentation, allowing you to quickly generate spot outlines that can be refined with NimbusImage's interactive tools. It can be retrained on your own data as well, which can often give you great results if the default models don't work well for your data.

{% hint style="info" %}
If you use Piscis in your research, please cite the [relevant paper](/citations.md#piscis).
{% endhint %}

> **Key tip**: If you're getting too many or too few spots, try different models first before adjusting other parameters. The built-in models vary in sensitivity, and selecting the right one is usually more effective than tweaking threshold values.

#### How Piscis works

Piscis uses a specialized neural network designed specifically for detecting small punctate structures in fluorescence microscopy images, such as:

* RNA molecules in FISH experiments
* Protein clusters
* Vesicles
* Small organelles
* Synaptic puncta

The model can work in both 2D (single slice) and 3D (Z-stack) modes, making it flexible for different experimental setups.

#### Key parameters

* **Model**: Select which pre-trained Piscis model to use
  * Different models have varying levels of sensitivity
  * Try several models if you're not getting optimal results
  * Custom trained models will also appear in this list
* **Mode**: Choose between "Current Z" or "Z-Stack"
  * **Current Z**: Process each Z-slice independently (2D mode)
  * **Z-Stack**: Process the entire Z-stack as a 3D volume (better for connecting spots across Z)
* **Scale**: Adjust for the size of spots (0-5)
  * Default value of 1 works well for most applications
  * Increase to detect larger spots
  * Decrease to detect smaller spots
  * Affects how the model interprets spot size relative to the training data
* **Threshold**: Confidence threshold for spot detection (0-9)
  * Default value of 1.0 works for most cases
  * Note that this parameter has a relatively minor effect compared to changing models
  * Higher values create stricter detection (fewer spots)
  * Lower values create more lenient detection (more spots)
* **Skip Frames Without**: Optional tag to skip processing frames
  * If specified, only process frames containing objects with the given tag
  * Useful for optimizing processing time by focusing on relevant frames

#### Advanced features

* **Batch processing**: Process multiple positions, Z-slices, or time points
  * Use the "Batch XY", "Batch Z", and "Batch Time" fields to specify ranges
  * Format example: "1-3, 5-8" processes positions 1, 2, 3, 5, 6, 7, 8
  * Enter `all` to process every available position, Z-slice, or time point, without having to know how many there are
  * Note: If using "Z-Stack" mode, the "Batch Z" field is ignored

#### Best practices

1. **Start with the default model**: Try the default "20230905" model first
2. **Try different models**: If detection isn't optimal, switching models is more effective than adjusting thresholds
3. **Use Z-Stack mode** for 3D data where spots may span multiple Z-slices
4. **Validate results visually**: Always check the detection results manually
5. **Consider retraining**: For specialized applications, retraining on your own data can significantly improve results

#### Example workflow

1. Add a new Piscis tool from the toolset menu
2. Select your channel containing spots
3. Choose a model (start with the default)
4. Select either "Current Z" or "Z-Stack" mode
5. Run the model
6. Review the results
7. If needed, try a different model or adjust parameters and run again

#### Training custom models

If the pre-built models don't work well for your specific spot detection needs, consider training a custom model using the Piscis Train tool (documented separately). Custom-trained models will appear in the model selection dropdown once trained.

Piscis is particularly effective for RNA FISH and similar applications where automatic detection of small punctate structures is needed, saving significant time compared to manual annotation.

### Piscis Training

The Piscis Training tool allows you to create custom spot detection models tailored to your specific microscopy data. By training on your own annotated examples, you can significantly improve spot detection performance for challenging or unique datasets.

> **Key tip**: You don't need extensive training data to get good results! Even a few carefully annotated regions can produce a highly effective custom model.

#### Why train a custom model?

Consider training a custom Piscis model when:

* Default models consistently detect too many or too few spots
* Your spots have unusual characteristics (size, intensity, shape)
* You're working with specialized microscopy techniques
* Background noise or artifacts are causing detection issues

#### Training data preparation

To train an effective custom model:

1. **Create point annotations** for the spots you want to detect
   * Tag them consistently (e.g., "training\_spots")
   * Be thorough within your selected regions - mark all visible spots
2. **Define training regions** where your annotations are complete
   * Use rectangles or polygons to outline areas with fully annotated spots
   * Tag these regions consistently (e.g., "training\_region")
   * Focus on high-quality regions rather than quantity

#### Key parameters

* **Initial Model Name**: The base model to start training from
  * Starting from an existing model (like "20230905") speeds up training
  * The base model provides initial weights that get refined with your data
* **New Model Name**: What to call your custom model
  * Default is a timestamp (e.g., "20250305\_143022")
  * Consider using a descriptive name for your experiment or spot type
* **Annotation Tag**: The tag used on your spot annotations
  * Only points with this tag will be used for training
* **Region Tag**: The tag used on your training regions
  * Only annotations within these regions will be used for training
  * Crucial for ensuring the model learns from fully annotated areas

#### Training parameters

* **Learning Rate**: Controls how quickly the model adapts (default: 0.2)
  * Higher values can lead to faster convergence but may be less stable
  * Lower values produce more stable but slower training
* **Weight Decay**: Regularization parameter to prevent overfitting (default: 0.0001)
  * Higher values create simpler models that may generalize better
* **Epochs**: Number of training iterations (default: 40)
  * More epochs generally improve results but take longer
  * The default is usually sufficient for good performance
* **Random Seed**: Controls randomness in training for reproducibility (default: 42)
  * Change this value if you want to try different random initializations

#### Training workflow

1. **Create point annotations**
   * Manually mark spots in representative areas of your image
   * Apply a consistent tag (e.g., "training\_points")
2. **Define training regions**
   * Draw rectangles or polygons around fully annotated areas
   * Apply a consistent tag (e.g., "training\_region")
3. **Configure the Piscis Train tool**
   * Select a base model
   * Configure training parameters (or use defaults)
   * Specify your annotation and region tags
4. **Run the training process**
   * This will take several minutes depending on your data
   * The model will be saved with your specified name
5. **Use your custom model**
   * Your trained model will automatically appear in the Piscis tool's model list
   * Select it when using Piscis for spot detection

#### Best practices

1. **Quality over quantity**: A few well-annotated regions are better than many poorly annotated ones
2. **Include challenging examples**: Include difficult spots in your training data
3. **Use representative regions**: Select areas that reflect the variability in your dataset
4. **Start small**: Begin with a small training set and evaluate before expanding
5. **Iterative improvement**: Train, test, correct annotations, and train again for best results

#### Testing your model

After training, test your model by:

1. Using the standard Piscis tool with your new model
2. Comparing results against manual annotations in regions not used for training
3. Adjusting the Scale parameter slightly if needed for fine-tuning

#### Model storage

Your custom Piscis models are saved in a `.piscis` directory at the root of your Private or Public folder. Once training completes, the model files are automatically stored in this location and will appear in the Piscis tool's model dropdown menu for future use. You can:

* Manually add models trained offline to this directory
* Share models between team members by copying model files

Custom-trained Piscis models often dramatically improve spot detection accuracy for specialized applications, saving significant time in your image analysis workflow.

### CondensateNet for automated condensate segmentation

CondensateNet is a deep learning-based tool for automatically detecting and segmenting biomolecular condensates in brightfield microscopy images. It uses a Feature Pyramid Network (FPN) architecture with an EfficientNet encoder to identify condensates and create polygon annotations for each one, which you can then analyze using NimbusImage's object properties and tagging features.

{% hint style="info" %}
If you use CondensateNet in your research, please cite the [relevant paper](/citations.md#condensatenet).
{% endhint %}

#### How CondensateNet works

CondensateNet processes brightfield images through a three-stage pipeline:

1. **Preprocessing**: Normalizes the input image
2. **Inference**: Runs the FPN model to produce probability masks and flow fields
3. **Post-processing**: Converts predictions to instance segmentation using watershed-based methods, outputting polygon annotations for each detected condensate

#### Key parameters

* **Probability Threshold**: Minimum confidence score for a detection to be included (0–1, default: 0.15). Lower values detect more condensates, including faint ones; higher values detect only high-confidence condensates.
* **Min Size**: Minimum condensate size in pixels (default: 15). Objects smaller than this are discarded.
* **Max Size**: Maximum condensate size in pixels (default: 600). Objects larger than this are discarded.
* **Smoothing**: Polygon simplification tolerance (default: 0.3). Higher values produce simpler polygon outlines with fewer vertices.
* **Padding**: Expand or shrink detected polygons in pixels (default: 0). Positive values dilate outlines; negative values erode them.

#### Advanced parameters for large images

* **Tile Size**: Size of tiles for processing large images in pixels (default: 1024). Smaller tiles use less memory but may miss large objects.
* **Tile Overlap**: Fraction of overlap between adjacent tiles (0–1, default: 0.1). Objects spanning tile boundaries are stitched together using the overlap region.
* **Important**: Ensure your objects are smaller than the overlap region. For example, with a tile size of 1024 and overlap of 0.1, objects should be less than approximately 102 pixels in their longest dimension for correct stitching.

#### Batch processing

* Use the **Batch XY**, **Batch Z**, and **Batch Time** fields to specify ranges for processing multiple positions, Z-slices, or time points (format: "1-3, 5-8", or `all` for every available coordinate). Defaults to the current tile.

#### Best practices

1. **Adjust the probability threshold** as your primary tuning parameter — start with the default of 0.15 and increase if you see too many false positives or decrease if you're missing condensates
2. **Use Min/Max Size** to filter out noise (small detections) or non-condensate objects (large detections)
3. **Use larger tile sizes** (1024 recommended) for consistent detection — smaller tiles can cause detection differences due to per-tile intensity normalization
4. **Review results visually**: Always inspect the segmentation and refine parameters if needed
5. **Post-process as needed**: Use NimbusImage's manual editing tools to correct any segmentation errors

### StarDist for nucleus segmentation

StarDist finds nuclei (and other round, compact objects) by predicting each object as a **star-convex polygon** — a shape where every point on the outline can be reached by a straight line from the object's center. That assumption suits nuclei very well, and it makes StarDist good at separating touching or crowded nuclei. It is a good choice for a DAPI or Hoechst channel. For cells with irregular or concave outlines, Cellpose-SAM is usually a better fit.

{% hint style="info" %}
If you use StarDist in your research, please cite the [relevant papers](/citations.md#stardist).
{% endhint %}

#### How to use

1. **Navigate to the position you want to segment.** StarDist processes the XY position, Z slice, and time point you are currently viewing.
2. **Add the tool** by clicking "ADD NEW TOOL" and choosing "Stardist" from the Stardist category.
3. **Pick the channel** containing your nuclei and choose a model.
4. **Run the tool.** Each detected object is added as a polygon with the tool's tag.

#### Key parameters

* **Model**: The pretrained model to use. `2D_versatile_fluo` (default) is trained on fluorescent nuclei; `2D_versatile_he` is trained on H\&E-stained histology.
* **Channel**: The image channel to segment (required). StarDist works on one channel at a time.
* **Probability Threshold**: How confident the model must be to report an object (range: 0–1, default: 0.5). Lower it to pick up dim or faint nuclei; raise it if you're getting spurious detections.
* **NMS Threshold**: How much two candidate objects may overlap before the weaker one is discarded (range: 0–1, default: 0.4). Lower values remove more overlapping detections; raise it if neighboring nuclei are being merged or dropped in dense regions.
* **Padding**: Expands (positive values) or shrinks (negative values) every outline by this many pixels (range: −20 to 20, default: 0).
* **Smoothing**: Simplifies the outlines by removing vertices (range: 0–10, default: 1). Higher values give simpler polygons that are faster to display and compute on.

#### Best practices

* **Start with the defaults** and adjust Probability Threshold first; it has the biggest effect on what gets found.
* **Check crowded regions.** If touching nuclei come out merged, try raising the NMS Threshold slightly.
* **Be mindful of very large images.** StarDist analyzes the whole image at the current position in a single pass, so very large images need a lot of GPU memory.

### SAM2 tools for segmentation, refinement, and tracking

NimbusImage includes a family of automated tools built on Meta's Segment Anything Model 2 (SAM2). You'll find them under the **SAM2** category in the "Add new tool" dialog (the SAM1-based few-shot tool is under **SAM**). Unlike the interactive [Segment-Anything "God Mode"](#segment-anything-semi-automated-object-finding-aka-god-mode) and [Segment similar objects](#segment-similar-objects-experimental) tools, which run in your browser on the current view, these tools run as server jobs and can process many XY positions, Z-slices, and time points at once. They create blob (polygon) objects with the tags you set for the tool.

{% hint style="info" %}
If you use the SAM2 tools in your research, please cite [SAM 2](/citations.md#segment-anything-model-2-sam-2). For the SAM few-shot segmentation tool, cite the original [Segment Anything Model (SAM)](/citations.md#segment-anything-model-sam).
{% endhint %}

{% hint style="info" %}
All of these tools "see" the image as it is displayed: they combine your channels using the current layer contrast and color settings. Adjust your layers so the objects you care about are clearly visible before running them.
{% endhint %}

Several settings are shared across these tools:

* **Batch XY / Batch Z / Batch Time**: The positions, Z-slices, and time points to process (format: "1-3, 5-8", or `all`). Leave blank to use only the current one.
* **Model** (SAM2 tools): The SAM2.1 model size: `sam2.1_hiera_tiny`, `sam2.1_hiera_small`, `sam2.1_hiera_base_plus`, or `sam2.1_hiera_large`. Larger models may give slightly better results but are slower and use more memory, which can matter when processing many objects at once.
* **Smoothing**: How much the resulting outlines are simplified (0-3). Higher values give smoother outlines with fewer vertices.
* **Padding**: Expands (positive) or shrinks (negative) the resulting outlines, in pixels (-20 to 20, default 0).

### SAM2 automatic mask generator

The **SAM2 automatic mask generator** finds and outlines *everything* SAM2 can identify in the image, without any examples or prompts. It's a quick way to get a first pass of objects in an image, which you can then filter or clean up.

#### Key parameters

* **Model**: The SAM2 model size (default: `sam2.1_hiera_small.pt`)
* **Smoothing**: Outline simplification (default: 0.3)
* **Points per side**: How densely SAM2 samples the image when looking for objects (16-128, default: 32). Higher values find more and smaller objects but take longer.

#### Best practices

Because it segments everything, expect to get background regions and other objects you don't want along with your objects of interest. If you only want objects that look like a few examples you've drawn, use few-shot segmentation instead. Each object is a single outline; holes inside objects are not preserved.

### SAM and SAM2 few-shot segmentation

The few-shot segmentation tools find objects that resemble a handful of examples you've outlined. Outline roughly 5-20 representative objects, give them a tag, and the tool searches the batch range for similar objects. No model training is needed.

Under the hood, the tool generates every candidate object it can find (like the automatic mask generator), compares each candidate's appearance to your examples, and keeps only the candidates that are similar enough.

There are two versions:

* **SAM2 few-shot segmentation** (under **SAM2**): Uses SAM2. Faster, and lets you choose among SAM2 model sizes (default: `sam2.1_hiera_base_plus.pt`).
* **SAM few-shot segmentation** (under **SAM**): Uses the original, larger SAM ViT-H model (`sam_vit_h_4b8939`). In our tests on microscopy images, it found more of the target objects than the SAM2 version, but it is considerably slower. Choose it when segmentation quality matters more than speed.

#### How to use

1. Outline 5-20 example objects with a manual blob tool and give them a common tag (e.g., "training\_cells"). Choose examples that represent the range of appearances you want to find.
2. Add **SAM2 few-shot segmentation** or **SAM few-shot segmentation** from the "Add new tool" dialog.
3. Set **Training Tag** to the tag of your examples, and set the batch range you want to segment.
4. Run the tool and review the results. Adjust **Similarity Threshold** and the mask area limits as needed, and run again.

#### Key parameters

* **Training Tag**: **Required.** The tag of your example objects. Examples can be anywhere in the dataset.
* **Similarity Threshold**: How closely a candidate must resemble your examples to be kept (0-1, default: 0.5). Increase (e.g., 0.6-0.8) if you get too many false positives; decrease (e.g., 0.3-0.4) if objects are being missed.
* **Target Occupancy**: How much context around each object the model looks at, as the fraction of the view the object fills (0.05-0.8, default: 0.2). Try 0.1-0.15 for very small objects and 0.3-0.4 for large ones.
* **Points per side**: How densely to search for candidate objects (default: 128; range 16-256 for SAM2 and 16-128 for SAM). Increase to find more small objects; decrease for faster processing.
* **Min Mask Area** / **Max Mask Area**: Candidates smaller or larger than these areas (in pixels) are ignored (defaults: 30 and 1000). Set **Max Mask Area** to 0 for no upper limit. A good starting point is about half the area of your smallest example for the minimum and about twice the area of your largest example for the maximum. **Be sure to raise Max Mask Area if your objects are larger than 1000 pixels.**
* **Smoothing**: Outline simplification (default: 0.3)

{% hint style="info" %}
Looking for something interactive? The [Segment similar objects](#segment-similar-objects-experimental) tool does a similar job in your browser on the current view, with immediate feedback. The few-shot tools are better when you want to process many positions, slices, or time points in one go.
{% endhint %}

### SAM2 Refiner

The **SAM2 Refiner** cleans up existing objects. For each tagged object, it uses the object's bounding box as a prompt to SAM2 and creates a new outline that better follows the object's edges in the image. This is useful for tightening up rough hand-drawn outlines or objects from another segmentation tool.

#### How to use

1. Add **SAM2 Refiner** from the "Add new tool" dialog.
2. Set **Tag of objects to refine** to the tag of the objects you want to clean up, and set the batch range to cover them.
3. Set the tool's output tags. The refined objects get the tool's tags, not the tags of the originals.
4. Run the tool and review the results.

#### Key parameters

* **Tag of objects to refine**: The tag of the objects to refine. Only objects within the batch range are refined. If left empty, all blob objects in the batch range are refined.
* **Delete original annotations**: Delete the original objects after the refined ones are created (default: off).
* **Model**: The SAM2 model size (default: `sam2.1_hiera_small.pt`)
* **Padding**: Outline expansion or shrinkage in pixels (default: 0)
* **Smoothing**: Outline simplification (default: 0.7)

{% hint style="warning" %}
The refiner creates new objects rather than editing the originals in place, so connections and other references to the original objects are not carried over to the refined ones.
{% endhint %}

### SAM2 propagator and SAM2 video

These two tools take objects you've outlined in one frame and carry them through time or Z, creating a matching object in each subsequent frame (or slice). Both can connect the objects across frames so that each object forms a track you can browse and analyze with [connections](/documentation/analyzing-image-data-with-objects-connections-and-properties/tools-for-connecting-objects.md) and review in [Time lapse mode](/documentation/time-lapse-mode.md).

* **SAM2 propagator** works one frame at a time: it uses the bounding boxes of the objects in the current frame as prompts to segment the next frame, then repeats from there. It works well when objects change in appearance over time (e.g., cell differentiation or condensate dynamics), and uses less memory on long sequences.
* **SAM2 video** uses SAM2's video mode, which considers the whole sequence together and keeps a memory of each object across frames. This tends to give more consistent tracking, but long sequences can require a lot of GPU memory. If a long sequence fails, try the propagator instead.

#### How to use

1. Outline the objects you want to follow in your starting frame and give them a tag. With SAM2 video, you can also provide starting objects in different frames; each one is tracked separately.
2. Add **SAM2 propagator** or **SAM2 video** from the "Add new tool" dialog.
3. Set the tag of your starting objects, choose whether to follow them across **Time** or **Z**, and choose the direction (**Forward** or **Backward**).
4. Set **Batch Time** (or **Batch Z**, when working across Z) to the full range of frames to follow the objects through, including your starting frame, e.g., "1-50" or `all`. With the default (blank), only the current frame is processed and nothing is propagated.
5. Run the tool and review the results.

#### Key parameters

* **Tag of objects to propagate** (propagator) / **Tag of objects to track** (video): The tag of your starting objects.
* **Propagate across** / **Track across**: Whether to follow objects through **Time** or **Z** (default: Time).
* **Propagation direction** / **Track direction**: **Forward** to go from earlier to later frames, or **Backward** to go from later to earlier (default: Forward).
* **Connect sequentially**: Connect each object to its counterpart in the next frame (default: on). Connections run from earlier to later frames regardless of direction, and are tagged `SAM2_PROPAGATED` (propagator) or `SAM2_VIDEO` (video).
* **Resegment propagation objects** (propagator only): Before propagating, re-segment your starting objects with SAM2 so the whole track is drawn consistently (default: on). This adds new SAM2-drawn objects in the starting frame alongside your originals.
* **Model**: The SAM2 model size (default: `sam2.1_hiera_small.pt`)
* **Padding**: Outline expansion or shrinkage in pixels (default: 0)
* **Smoothing**: Outline simplification (default: 0.7 for the propagator, 0.3 for video)

{% hint style="info" %}
To find objects first and then follow them, you can combine tools: for example, segment the first time point with Cellpose-SAM or few-shot segmentation, then propagate those objects with the SAM2 propagator or SAM2 video.
{% endhint %}