Refactor Validation and Create RollCall module by codedbyishika · Pull Request #56 · Imageomics/cautious-robot

codedbyishika · 2026-06-03T20:44:48Z

Move following functions from main.py to Validation.py + Change AGENTS.md:
validate_csv_extension
~~validate_output_directory~~ This function was removed, as it should have been in #43.
validate_filename_uniqueness
handle_missing_filenames
setup_expected_columns

RollCall Module

Added above functionalities and check_existing_images to the rollcall module which prechecks all the conditions before downloading.

Non Interactive Missing File Handling

Implemented missing file check as non interactive error handing

Duplicate Checksums

Implemented checks for checking duplicates

Summary

Implemented summary of where images & downsized images will be downloaded

Changed main according to new csv handle
Updated tests to the new features

Move following functions: validate_csv_extension validate_output_directory validate_filename_uniqueness handle_missing_filenames setup_expected_columns

egrace479

Found a holdover from prior implementation method.

Co-authored-by: Elizabeth Campolongo <38985481+egrace479@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

- Change tests - Include check_existing_images

This reverts commit ff930b0.

egrace479

I haven't had a chance to check the test changes, but some efficiency can be added and I have some questions in implementation of new features that rely on consideration of the motivation. Mainly the new duplicate check that should have an option to ignore for the purposes of still downloading, but then what is the intended response to duplicates; if the ignore flag is passed, should it be skipped or at least provide the duplicates in a usable format so the user can do something with the information?

egrace479 · 2026-07-09T23:14:45Z

+        roll_call.check_duplicate_checksums(
+            data_df=source_df,
+            hash_col=args.verifier_col.lower(),
+            ignore_duplicates=getattr(args, "ignore_duplicates", False)


Where is this defined?

egrace479 · 2026-07-09T23:16:40Z

    # Validate and handle existing output directory
    img_dir = args.output_dir
-    source_df, filtered_df = check_existing_images(csv_path, img_dir, source_df, filename_col, subfolders)
+    source_df, filtered_df = roll_call.check_existing_images(csv_path, img_dir, source_df, filename_col, subfolders)


Is the idea of checking for duplicate hashes in the source DataFrame first to alert the user to their existence even if they have most images already?

egrace479 · 2026-07-09T23:20:29Z

    if isinstance(args.side_length, int):
        downsample_dest_path = img_dir + "_downsized"


Suggested change

if downsample_dir:

Since this is already defined above, no need to do extra checks and redefine. Unfortunately, this will be a couple comments since the formatting isn't holding for a block comment on these unedited lines. (it should be replacing the if instance... and downsample_dest_path = ...

egrace479 · 2026-07-09T23:21:20Z

                       error_log_filepath=error_log_filepath,
                       filename=filename_col,
                       subfolders=subfolders,
                       downsample_path=downsample_dest_path,


Suggested change

downsample_path=downsample_dir,

egrace479 · 2026-07-09T23:21:37Z

                       file_url=url_col,
                       wait=args.wait_time,
                       retry=args.max_retries)
        print(f"Images downloaded from {csv_path} to {img_dir}, with downsampled images in {downsample_dest_path}.")


Suggested change

print(f"Images downloaded from {csv_path} to {img_dir}, with downsampled images in {downsample_dir}.")

egrace479 · 2026-07-09T23:29:30Z

+            url_col (str): Name of the column containing image URLs.
+
+        Returns:
+            pd.DataFrame | None: Rows with missing filenames when present; otherwise None.


Suggested change

pd.DataFrame | None: Rows with missing filenames when present; otherwise None.

missing (pd.DataFrame) | None: DataFrame of entries with URLs but missing filenames or None.

egrace479 · 2026-07-09T23:31:12Z

+            args (argparse.Namespace): Parsed command-line arguments.
+
+        Returns:
+            tuple[dict, str | None]: Column mapping to use for filename, URL, and optional subfolder handling.


Suggested change

tuple[dict, str | None]: Column mapping to use for filename, URL, and optional subfolder handling.

(expected_cols, subfolders) tuple[dict, str | None]: Column mapping to use for filename, URL, and optional subfolder handling.

egrace479 · 2026-07-10T21:36:09Z

+        # If duplicates exist, preview or save
+        self._preview_or_save("duplicate_checksums", dupes)
+
+        if ignore_duplicates:


If this is going to run regardless of the ignore, it seems like it would make sense to print or save duplicates so the user is aware and can do something with that information.

egrace479 · 2026-07-10T21:41:10Z


 ### Core Modules
 - `__main__.py` - Main entry point with CLI argument parsing and workflow orchestration
+- `roll_call.py` - CSV validation and input handling functions


Suggested change

- `roll_call.py` - CSV validation and input handling functions

- `roll_call.py` - Pre-download checks and processing.

Not sure that processing is exactly the right term, but staging doesn't quite fit the bill either.

codedbyishika added 2 commits June 3, 2026 16:43

Refractor Validation.py

4868e30

Move following functions: validate_csv_extension validate_output_directory validate_filename_uniqueness handle_missing_filenames setup_expected_columns

Remove unused imports

784c5a4

codedbyishika requested a review from egrace479 June 4, 2026 17:04

egrace479 requested changes Jun 4, 2026

View reviewed changes

Comment thread src/cautiousrobot/validation.py Outdated

egrace479 changed the title ~~Refractor Validation.py~~ Refactor Validation.py Jun 4, 2026

Delete validate_output_directory

5de93ee

Co-authored-by: Elizabeth Campolongo <38985481+egrace479@users.noreply.github.com>

egrace479 requested a review from Copilot June 4, 2026 17:30

Copilot started reviewing on behalf of egrace479 June 4, 2026 17:30 View session

This comment was marked as outdated.

Sign in to view

egrace479 reviewed Jun 4, 2026

View reviewed changes

Comment thread src/cautiousrobot/validation.py Outdated

Remove unused import

bb336a6

egrace479 requested review from Copilot and egrace479 June 4, 2026 17:37

Copilot started reviewing on behalf of egrace479 June 4, 2026 17:38 View session

This comment was marked as outdated.

Sign in to view

egrace479 requested a review from Copilot June 4, 2026 17:59

Copilot started reviewing on behalf of egrace479 June 4, 2026 17:59 View session

This comment was marked as outdated.

Sign in to view

codedbyishika self-assigned this Jun 4, 2026

egrace479 requested a review from Copilot June 4, 2026 19:09

Copilot started reviewing on behalf of egrace479 June 4, 2026 19:09 View session

Copilot AI reviewed Jun 4, 2026

View reviewed changes

Comment thread src/cautiousrobot/validation.py Outdated

Comment thread src/cautiousrobot/validation.py Outdated

egrace479 and others added 3 commits June 4, 2026 15:23

small formatting fix

930e33d

Create test_validation.py

a67fe33

Fix imports

48e72d6

egrace479 mentioned this pull request Jun 8, 2026

Create roll_call module #61

Open

2 tasks

codedbyishika linked an issue Jun 12, 2026 that may be closed by this pull request

Extract the pre-download verification elements into roll_call module. #63

Open

codedbyishika added 3 commits June 12, 2026 12:37

Create RollCall Module

c2d1b38

- Change tests - Include check_existing_images

Change Imports

ff930b0

Revert "Change Imports"

d14d7ac

This reverts commit ff930b0.

codedbyishika added 4 commits June 12, 2026 15:02

Change Tests and Imports

9d46b5f

Name change

6e91c0d

Update tests

8fe0e6d

Update interactive to missing filenames error handling

15cd820

codedbyishika linked an issue Jun 17, 2026 that may be closed by this pull request

Add dry-run functionality #66

Open

codedbyishika changed the title ~~Refactor Validation.py~~ Refactor RollCall.py Jun 17, 2026

Small error fix

af134f8

egrace479 reviewed Jun 17, 2026

View reviewed changes

Comment thread src/cautiousrobot/roll_call.py Outdated

codedbyishika added 3 commits June 19, 2026 13:22

Update for csv to initialize when dataframe is missing

1271c26

duplicate checksums

c905318

summary of where images & downsized images will be downloaded

2f34879

codedbyishika requested a review from egrace479 June 19, 2026 19:07

codedbyishika changed the title ~~Refactor RollCall.py~~ Refactor Validation and Create RollCall module Jun 22, 2026

Include comments and docstrings

6c0b5d4

egrace479 requested changes Jul 10, 2026

View reviewed changes

		if isinstance(args.side_length, int):
		downsample_dest_path = img_dir + "_downsized"


	print(f"Images downloaded from {csv_path} to {img_dir}, with downsampled images in {downsample_dir}.")

	pd.DataFrame \| None: Rows with missing filenames when present; otherwise None.
	missing (pd.DataFrame) \| None: DataFrame of entries with URLs but missing filenames or None.

	tuple[dict, str \| None]: Column mapping to use for filename, URL, and optional subfolder handling.
	(expected_cols, subfolders) tuple[dict, str \| None]: Column mapping to use for filename, URL, and optional subfolder handling.

	- `roll_call.py` - CSV validation and input handling functions
	- `roll_call.py` - Pre-download checks and processing.

Uh oh!

Conversation

codedbyishika commented Jun 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

RollCall Module

Non Interactive Missing File Handling

Duplicate Checksums

Summary

Uh oh!

egrace479 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

egrace479 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codedbyishika commented Jun 3, 2026 •

edited

Loading