deeptrack.sources.folder Module#

Data sources from images organized in a directory structure.

This module provides the ImageFolder class, which enables structured access to images stored in a hierarchical folder layout, such as:

root/train/cat/image1.jpg root/train/dog/image2.jpg root/test/bird/image3.jpg

The class supports automatic labeling based on directory names, integration with DeepTrack2 data pipelines, and flexible splitting of datasets by folder.

Key Features#

  • Attribute Access

    Provides access to common attributes such as image paths, label indices, and category names. Each entry is returned as a SourceItem with fields path, label, and label_name.

  • Automatic Labeling

    Converts directory names into integer labels, supporting direct use in training pipelines or models that expect categorical inputs.

  • Flexible Dataset Splitting

    Supports splitting datasets based on the top-level folder structure. This enables separating data into training, validation, and test sets using directory naming conventions.

Module Structure#

Classes:

  • ImageFolder: Source of image paths and labels from a structured folder.

    Wraps a directory of image files into a DeepTrack Source, supporting standard methods such as iteration, indexing, and filtering.

Attributes:

  • known_extensions: list[str]

    List of recognized file extensions used when scanning directories for valid image files: [“png”, “jpg”, “jpeg”, “tif”, “tiff”, “bmp”, “gif”]

Examples#

Create a dummy dataset structure with train/test subfolders

>>> import os
>>> import shutil

Temporary root directory:

>>> root = "tmp_data"

Remove existing directory if needed:

>>> if os.path.exists(root):
...     shutil.rmtree(root)

Define splits and classes:

>>> splits = ["train", "test"]
>>> classes = ["cat", "dog", "bird"]

Create directories and dummy files:

>>> for split in splits:
...     for cls in classes:
...         folder_path = os.path.join(root, split, cls)
...         os.makedirs(folder_path)
...         for i in range(2):
...             file_path = os.path.join(folder_path, f"image_{i}.jpg")
...             with open(file_path, "w") as f:
...                 f.write("dummy")

Load a split of the dataset, specifically, the training set:

>>> from deeptrack.sources import ImageFolder
>>>
>>> train_data = ImageFolder(os.path.join(root, "train"))

>>> len(train_data)
6

>>> train_data.classes
['bird', 'cat', 'dog']

>>> train_data.path()
'tmp_data/train/bird/image_0.jpg'

Access a source item

>>> item = train_data[0]

>>> item["path"]
'tmp_data/train/bird/image_0.jpg'

>>> item["label"]
0

>>> item["label_name"]
'bird'

Convert between label names and indices

>>> train_data.name_to_label("cat")
1

>>> train_data.label_to_name(0)
'bird'

Split the dataset across top-level folders

>>> all_data = ImageFolder(root)
>>> train, test = all_data.split("train", "test")

>>> print(f"Train size: {len(train)}")
Train size: 6

>>> print(f"Test size: {len(test)}")
Test size: 6

Print paths in each split

Train files:

>>> for item in train:
...     print(item["path"])
tmp_data/train/bird/image_0.jpg
tmp_data/train/bird/image_1.jpg
tmp_data/train/cat/image_0.jpg
tmp_data/train/cat/image_1.jpg
tmp_data/train/dog/image_0.jpg
tmp_data/train/dog/image_1.jpg

Test files:

>>> for item in test:
...     print(item["path"])
tmp_data/test/bird/image_0.jpg
tmp_data/test/bird/image_1.jpg
tmp_data/test/cat/image_0.jpg
tmp_data/test/cat/image_1.jpg
tmp_data/test/dog/image_0.jpg
tmp_data/test/dog/image_1.jpg

Classes#

ImageFolder(root)

Data source for images organized in a directory structure.