Feat/progressive inbox scan #72

2025-09-24T03:06:51-04:00

Vamsiindugu commented

2025-09-24 03:06:51 -04:00

(Migrated from github.com)

Summary

This PR resolves the performance issue where the extension would freeze when scanning very large inboxes (~500k emails).

It replaces the "all-at-once" email fetching mechanism with a progressive loading system that processes emails in batches.

This provides a much better user experience by preventing the UI from freezing and giving the user real-time feedback and control over the scanning process.

✨ Key Changes

Batched Processing: The BrowserEmailService now fetches emails in configurable batches to avoid blocking the main thread.
Progress UI: A new FetchProgressBar component has been added, which displays:
- Current page vs. total pages
- Number of processed emails
- A percentage completion bar
Cancellation Support: Users can now cancel an in-progress scan via a "Cancel" button.
State Management: The AppProvider now manages the state for fetch progress and cancellation.
Incremental Aggregation: Sender data is aggregated incrementally, improving memory efficiency.

✅ Verification

All 37 existing Playwright UI tests were executed and passed successfully, confirming that these changes do not break existing functionality.

# Summary This PR resolves the performance issue where the extension would freeze when scanning very large inboxes (~500k emails). It replaces the "all-at-once" email fetching mechanism with a progressive loading system that processes emails in batches. This provides a much better user experience by preventing the UI from freezing and giving the user real-time feedback and control over the scanning process. ## ✨ Key Changes - **Batched Processing:** The `BrowserEmailService` now fetches emails in configurable batches to avoid blocking the main thread. - **Progress UI:** A new `FetchProgressBar` component has been added, which displays: - Current page vs. total pages - Number of processed emails - A percentage completion bar - **Cancellation Support:** Users can now cancel an in-progress scan via a "Cancel" button. - **State Management:** The `AppProvider` now manages the state for fetch progress and cancellation. - **Incremental Aggregation:** Sender data is aggregated incrementally, improving memory efficiency. ## ✅ Verification All 37 existing Playwright UI tests were executed and passed successfully, confirming that these changes do not break existing functionality.

👍 3

copilot-pull-request-reviewer[bot] (Migrated from github.com) reviewed

2025-10-28 22:05:52 -04:00

copilot-pull-request-reviewer[bot] (Migrated from github.com) left a comment

Pull Request Overview

This PR adds progressive loading functionality to display real-time progress when fetching email senders from Gmail, improving user experience during long-running operations.

Key changes:

Introduces a visual progress bar UI with cancellation support
Implements batch processing with progress reporting in the email fetching logic
Adds comprehensive Playwright tests for progressive loading behavior

Reviewed Changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
`test/ui/sidebar/progressiveLoading.spec.ts`	New test suite with 10 tests covering progress bar display, cancellation, transitions, and edge cases
`test/ui/sidebar/helpers.ts`	Added helper functions for waiting on progress completion and retrieving progress state
`src/presentation/providers/app_provider.tsx`	Updated context to manage fetch progress state and cancellation, integrated progress callbacks with MockEmailRepo
`src/presentation/apps/sidebar/components/sendersContainer.tsx`	Modified rendering logic to show progress bar when `fetchProgress` is available
`src/presentation/apps/sidebar/components/fetchProgress.tsx`	New component rendering the progress bar UI with stats and cancel button
`src/presentation/apps/sidebar/components/fetchProgress.css`	Styling for the progress bar component
`src/data/services/browser_email_service.ts`	Refactored to support batch processing, progress callbacks, and cancellation via AbortSignal
`src/data/repositories/mocks/mock_email_repo.ts`	Added progress simulation with configurable progressive loading and cancellation
`src/data/repositories/browser_email_repo.ts`	Added progress callback support and cancel method, listens for progress messages from content script
`src/data/content_scripts/content.ts`	Refactored to support cancellation with AbortController and send progress updates to side panel

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

## Pull Request Overview This PR adds progressive loading functionality to display real-time progress when fetching email senders from Gmail, improving user experience during long-running operations. Key changes: - Introduces a visual progress bar UI with cancellation support - Implements batch processing with progress reporting in the email fetching logic - Adds comprehensive Playwright tests for progressive loading behavior ### Reviewed Changes Copilot reviewed 10 out of 10 changed files in this pull request and generated 6 comments. <details> <summary>Show a summary per file</summary> | File | Description | | ---- | ----------- | | `test/ui/sidebar/progressiveLoading.spec.ts` | New test suite with 10 tests covering progress bar display, cancellation, transitions, and edge cases | | `test/ui/sidebar/helpers.ts` | Added helper functions for waiting on progress completion and retrieving progress state | | `src/presentation/providers/app_provider.tsx` | Updated context to manage fetch progress state and cancellation, integrated progress callbacks with MockEmailRepo | | `src/presentation/apps/sidebar/components/sendersContainer.tsx` | Modified rendering logic to show progress bar when `fetchProgress` is available | | `src/presentation/apps/sidebar/components/fetchProgress.tsx` | New component rendering the progress bar UI with stats and cancel button | | `src/presentation/apps/sidebar/components/fetchProgress.css` | Styling for the progress bar component | | `src/data/services/browser_email_service.ts` | Refactored to support batch processing, progress callbacks, and cancellation via AbortSignal | | `src/data/repositories/mocks/mock_email_repo.ts` | Added progress simulation with configurable progressive loading and cancellation | | `src/data/repositories/browser_email_repo.ts` | Added progress callback support and cancel method, listens for progress messages from content script | | `src/data/content_scripts/content.ts` | Refactored to support cancellation with AbortController and send progress updates to side panel | </details> --- 💡 <a href="/InboxWhiz/gmail-declutter-extension/new/main/.github?filename=*.instructions.md" class="Link--inTextBlock" target="_blank" rel="noopener noreferrer">Add Copilot custom instructions</a> for smarter, more guided reviews. <a href="https://docs.github.com/en/copilot/customizing-copilot/adding-repository-custom-instructions-for-github-copilot" class="Link--inTextBlock" target="_blank" rel="noopener noreferrer">Learn how to get started</a>.

src/data/repositories/browser_email_repo.ts

					
				@ -1,19 +1,33 @@

				import { Sender } from "../../domain/entities/sender";

The setProgressCallback and cancelFetch methods are added to BrowserEmailRepo and MockEmailRepo but are not part of the EmailRepo interface. This creates an inconsistent API where consumers need to check instance types or cast to use these methods. Consider adding these methods to the EmailRepo interface for a consistent contract across implementations.

The `setProgressCallback` and `cancelFetch` methods are added to `BrowserEmailRepo` and `MockEmailRepo` but are not part of the `EmailRepo` interface. This creates an inconsistent API where consumers need to check instance types or cast to use these methods. Consider adding these methods to the `EmailRepo` interface for a consistent contract across implementations.

src/data/services/browser_email_service.ts

The comment says 'Process 10 pages at a time', but the batch processing logic appears to process pages sequentially within each batch (lines 252-278). The batches only add a 100ms delay between them. Consider clarifying whether this is truly batch processing or just sequential processing with periodic delays.

      batchSize = 10, // Process pages sequentially in batches of 10, with a delay between batches

The comment says 'Process 10 pages at a time', but the batch processing logic appears to process pages sequentially within each batch (lines 252-278). The batches only add a 100ms delay between them. Consider clarifying whether this is truly batch processing or just sequential processing with periodic delays. ```suggestion batchSize = 10, // Process pages sequentially in batches of 10, with a delay between batches ```

src/presentation/apps/sidebar/components/fetchProgress.tsx

					
				@ -0,0 +1,42 @@

				import React from "react";

The FetchProgress interface is duplicated across multiple files (fetchProgress.tsx, browser_email_service.ts, and mock_email_repo.ts). Consider defining it once in a shared location, such as in the domain layer or a shared types file, and importing it where needed to maintain a single source of truth.

import { FetchProgress } from "../../../domain/types";

The `FetchProgress` interface is duplicated across multiple files (fetchProgress.tsx, browser_email_service.ts, and mock_email_repo.ts). Consider defining it once in a shared location, such as in the domain layer or a shared types file, and importing it where needed to maintain a single source of truth. ```suggestion import { FetchProgress } from "../../../domain/types"; ```

src/presentation/providers/app_provider.tsx

					
				@ -50,15 +54,25 @@ export const AppProvider: React.FC<{ children: React.ReactNode }> = ({

				  >({});

The progress callback is only set up for MockEmailRepo but not for BrowserEmailRepo. This means the progress bar won't be displayed in production. The setProgressCallback should also be called on the BrowserEmailRepo instance.

    const browserRepo = new BrowserEmailRepo();
    // Set up progress callback for production
    browserRepo.setProgressCallback((progress) => {
      setFetchProgress(progress);
    });
    return browserRepo;

The progress callback is only set up for `MockEmailRepo` but not for `BrowserEmailRepo`. This means the progress bar won't be displayed in production. The `setProgressCallback` should also be called on the `BrowserEmailRepo` instance. ```suggestion const browserRepo = new BrowserEmailRepo(); // Set up progress callback for production browserRepo.setProgressCallback((progress) => { setFetchProgress(progress); }); return browserRepo; ```

src/presentation/providers/app_provider.tsx

					
				@ -88,11 +103,19 @@ export const AppProvider: React.FC<{ children: React.ReactNode }> = ({

				        }

The cancelFetch function only calls cancelFetch() on MockEmailRepo instances, but not on BrowserEmailRepo. This means cancellation won't work in production. The condition should check for both repository types or call the method unconditionally if it's added to the interface.

    emailRepo.cancelFetch();

The `cancelFetch` function only calls `cancelFetch()` on `MockEmailRepo` instances, but not on `BrowserEmailRepo`. This means cancellation won't work in production. The condition should check for both repository types or call the method unconditionally if it's added to the interface. ```suggestion emailRepo.cancelFetch(); ```

anna-st-40 (Migrated from github.com) reviewed

2025-10-28 22:12:11 -04:00

src/presentation/providers/app_provider.tsx

					
				@ -50,15 +54,25 @@ export const AppProvider: React.FC<{ children: React.ReactNode }> = ({

				  >({});

Accurate - I noticed the new UI changes do not show up when the extension is loaded up with a real Gmail inbox.

anna-st-40 (Migrated from github.com) requested changes

2025-10-28 22:15:35 -04:00

anna-st-40 (Migrated from github.com) left a comment

Please consider the comments left by Copilot

anna-st-40 (Migrated from github.com) approved these changes

2025-10-29 16:25:12 -04:00

Sign in to join this conversation.

No reviewers

anna-st-40

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

InboxWhiz/gmail-declutter-extension!72

				`@ -1,19 +1,33 @@`
				`import { Sender } from "../../domain/entities/sender";`

				`@ -50,15 +54,25 @@ export const AppProvider: React.FC<{ children: React.ReactNode }> = ({`
				`>({});`

				`@ -88,11 +103,19 @@ export const AppProvider: React.FC<{ children: React.ReactNode }> = ({`
				`}`

Rows
Columns