API Reference

This section provides detailed documentation for all the classes and methods in Natural PDF.

Core Classes

natural_pdf

Natural PDF - A more intuitive interface for working with PDFs.

Classes

natural_pdf.ConfigSection

A configuration section that holds key-value option pairs.

Source code in natural_pdf/__init__.py

class ConfigSection:
    """A configuration section that holds key-value option pairs."""

    def __init__(self, **defaults):
        self.__dict__.update(defaults)

    def __repr__(self):
        items = [f"{k}={v!r}" for k, v in self.__dict__.items()]
        return f"{self.__class__.__name__}({', '.join(items)})"

natural_pdf.Flow

Bases: Visualizable, SelectorHostMixin

Defines a logical flow or sequence of physical Page or Region objects.

A Flow represents a continuous logical document structure that spans across multiple pages or regions, enabling operations on content that flows across boundaries. This is essential for handling multi-page tables, articles that span columns, or any content that requires reading order across segments.

Flows specify arrangement (vertical/horizontal) and alignment rules to create a unified coordinate system for element extraction and text processing. They enable natural-pdf to treat fragmented content as a single continuous area for analysis and extraction operations.

The Flow system is particularly useful for: - Multi-page tables that break across page boundaries - Multi-column articles with complex reading order - Forms that span multiple pages - Any content requiring logical continuation across segments

Attributes:

Name	Type	Description
segments	List[Region]	List of Page or Region objects in flow order.
arrangement	Literal['vertical', 'horizontal']	Primary flow direction ('vertical' or 'horizontal').
alignment	Literal['start', 'center', 'end', 'top', 'left', 'bottom', 'right']	Cross-axis alignment for segments of different sizes.
segment_gap	float	Virtual gap between segments in PDF points.

Example

Multi-page table flow:

pdf = npdf.PDF("multi_page_table.pdf")

# Create flow for table spanning pages 2-4
table_flow = Flow(
    segments=[pdf.pages[1], pdf.pages[2], pdf.pages[3]],
    arrangement='vertical',
    alignment='left',
    segment_gap=10.0
)

# Extract table as if it were continuous
table_data = table_flow.extract_table()
text_content = table_flow.get_text()

Multi-column article flow:

page = pdf.pages[0]
left_column = page.region(0, 0, 300, page.height)
right_column = page.region(320, 0, page.width, page.height)

# Create horizontal flow for columns
article_flow = Flow(
    segments=[left_column, right_column],
    arrangement='horizontal',
    alignment='top'
)

# Read in proper order
article_text = article_flow.get_text()

Note

Flows create virtual coordinate systems that map element positions across segments, enabling spatial navigation and element selection to work seamlessly across boundaries.

Source code in natural_pdf/flows/flow.py

class Flow(Visualizable, SelectorHostMixin):
    """Defines a logical flow or sequence of physical Page or Region objects.

    A Flow represents a continuous logical document structure that spans across
    multiple pages or regions, enabling operations on content that flows across
    boundaries. This is essential for handling multi-page tables, articles that
    span columns, or any content that requires reading order across segments.

    Flows specify arrangement (vertical/horizontal) and alignment rules to create
    a unified coordinate system for element extraction and text processing. They
    enable natural-pdf to treat fragmented content as a single continuous area
    for analysis and extraction operations.

    The Flow system is particularly useful for:
    - Multi-page tables that break across page boundaries
    - Multi-column articles with complex reading order
    - Forms that span multiple pages
    - Any content requiring logical continuation across segments

    Attributes:
        segments: List of Page or Region objects in flow order.
        arrangement: Primary flow direction ('vertical' or 'horizontal').
        alignment: Cross-axis alignment for segments of different sizes.
        segment_gap: Virtual gap between segments in PDF points.

    Example:
        Multi-page table flow:
        ```python
        pdf = npdf.PDF("multi_page_table.pdf")

        # Create flow for table spanning pages 2-4
        table_flow = Flow(
            segments=[pdf.pages[1], pdf.pages[2], pdf.pages[3]],
            arrangement='vertical',
            alignment='left',
            segment_gap=10.0
        )

        # Extract table as if it were continuous
        table_data = table_flow.extract_table()
        text_content = table_flow.get_text()
        ```

        Multi-column article flow:
        ```python
        page = pdf.pages[0]
        left_column = page.region(0, 0, 300, page.height)
        right_column = page.region(320, 0, page.width, page.height)

        # Create horizontal flow for columns
        article_flow = Flow(
            segments=[left_column, right_column],
            arrangement='horizontal',
            alignment='top'
        )

        # Read in proper order
        article_text = article_flow.get_text()
        ```

    Note:
        Flows create virtual coordinate systems that map element positions across
        segments, enabling spatial navigation and element selection to work
        seamlessly across boundaries.
    """

    def __init__(
        self,
        segments: Union[Sequence[SupportsSections], "PageCollection"],
        arrangement: Literal["vertical", "horizontal"],
        alignment: Literal["start", "center", "end", "top", "left", "bottom", "right"] = "start",
        segment_gap: float = 0.0,
    ):
        """
        Initializes a Flow object.

        Args:
            segments: An ordered sequence of objects implementing SupportsSections (e.g., Page,
                      Region) that constitute the flow, or a PageCollection containing pages.
            arrangement: The primary direction of the flow.
                         - "vertical": Segments are stacked top-to-bottom.
                         - "horizontal": Segments are arranged left-to-right.
            alignment: How segments are aligned on their cross-axis if they have
                       differing dimensions. For a "vertical" arrangement:
                       - "left" (or "start"): Align left edges.
                       - "center": Align centers.
                       - "right" (or "end"): Align right edges.
                       For a "horizontal" arrangement:
                       - "top" (or "start"): Align top edges.
                       - "center": Align centers.
                       - "bottom" (or "end"): Align bottom edges.
            segment_gap: The virtual gap (in PDF points) between segments.
        """
        # Handle PageCollection input
        from natural_pdf.core.page_collection import PageCollection as _PageCollection

        if isinstance(segments, _PageCollection):
            segment_list: List[SupportsSections] = list(segments.pages)
        else:
            segment_list = list(segments)

        if not segment_list:
            raise ValueError("Flow segments cannot be empty.")
        if arrangement not in ["vertical", "horizontal"]:
            raise ValueError("Arrangement must be 'vertical' or 'horizontal'.")

        self.segments: List["PhysicalRegion"] = self._normalize_segments(segment_list)
        self.arrangement: Literal["vertical", "horizontal"] = arrangement
        self.alignment: Literal["start", "center", "end", "top", "left", "bottom", "right"] = (
            alignment
        )
        self.segment_gap: float = segment_gap
        self._analysis_region_cache: Optional["FlowRegion"] = None

        self._validate_alignment()

        # TODO: Pre-calculate segment offsets for faster lookups if needed

    def _normalize_segments(self, segments: Sequence[SupportsSections]) -> List["PhysicalRegion"]:
        """Materialize all segments as Region objects for uniform processing."""
        normalized: List["PhysicalRegion"] = []
        from natural_pdf.elements.region import Region as ElementsRegion

        for index, segment in enumerate(segments):
            if not isinstance(segment, SupportsSections):
                raise TypeError(
                    f"Segment {index} must implement SupportsSections; found {type(segment)}."
                )

            region_candidate = segment.to_region()
            if not isinstance(region_candidate, ElementsRegion):
                raise TypeError(
                    f"Segment {index} returned unsupported region type {type(region_candidate)}."
                )
            normalized.append(region_candidate)
        return normalized

    def _validate_alignment(self) -> None:
        """Validates the alignment based on the arrangement."""
        valid_alignments = {
            "vertical": ["start", "center", "end", "left", "right"],
            "horizontal": ["start", "center", "end", "top", "bottom"],
        }
        if self.alignment not in valid_alignments[self.arrangement]:
            raise ValueError(
                f"Invalid alignment '{self.alignment}' for '{self.arrangement}' arrangement. "
                f"Valid options are: {valid_alignments[self.arrangement]}"
            )

    def _analysis_region(self) -> "FlowRegion":
        """
        Create (and cache) a FlowRegion representing the entire flow for analysis tasks.
        """

        if self._analysis_region_cache is None:
            from natural_pdf.flows.region import FlowRegion

            self._analysis_region_cache = FlowRegion(
                flow=self,
                constituent_regions=list(self.segments),
            )
        return self._analysis_region_cache

    def _get_highlighter(self):
        """Get the highlighting service from the first segment."""
        if not self.segments:
            raise RuntimeError("Flow has no segments to get highlighter from")

        return resolve_highlighter(self.segments[0])

    def show(
        self,
        *,
        resolution: Optional[float] = None,
        width: Optional[int] = None,
        color: Optional[Union[str, Tuple[int, int, int]]] = None,
        labels: bool = True,
        label_format: Optional[str] = None,
        highlights: Optional[Union[List[Dict[str, Any]], bool]] = None,
        legend_position: str = "right",
        annotate: Optional[Union[str, List[str]]] = None,
        layout: Optional[Literal["stack", "grid", "single"]] = None,
        stack_direction: Literal["vertical", "horizontal"] = "vertical",
        gap: int = 5,
        columns: Optional[int] = 6,
        crop: Union[bool, int, str, "PhysicalRegion", Literal["wide"]] = False,
        crop_bbox: Optional[Tuple[float, float, float, float]] = None,
        in_context: bool = False,
        separator_color: Optional[Tuple[int, int, int]] = None,
        separator_thickness: int = 2,
        **kwargs,
    ) -> Optional["PIL_Image"]:
        """Generate a preview image with highlights.

        If in_context=True, shows segments as cropped images stacked together
        with separators between segments.

        Args:
            resolution: DPI for rendering (default from global settings)
            width: Target width in pixels (overrides resolution)
            color: Default highlight color
            labels: Whether to show labels for highlights
            label_format: Format string for labels
            highlights: Additional highlight groups to show
            layout: How to arrange multiple pages/regions
            stack_direction: Direction for stack layout
            gap: Pixels between stacked images
            columns: Number of columns for grid layout
            crop: Whether to crop
            crop_bbox: Explicit crop bounds
            in_context: If True, use special Flow visualization with separators
            separator_color: RGB color for separator lines (default: red)
            separator_thickness: Thickness of separator lines
            **kwargs: Additional parameters passed to rendering

        Returns:
            PIL Image object or None if nothing to render
        """
        resolved_resolution = self._resolve_image_resolution(resolution)

        if in_context:
            # Use the special in_context visualization
            return self._show_in_context(
                resolution=resolved_resolution,
                width=width,
                stack_direction=stack_direction,
                stack_gap=gap,
                separator_color=separator_color or (255, 0, 0),
                separator_thickness=separator_thickness,
                **kwargs,
            )

        # Otherwise use the standard show method
        return super().show(
            resolution=resolution,
            width=width,
            color=color,
            labels=labels,
            label_format=label_format,
            highlights=highlights,
            legend_position=legend_position,
            annotate=annotate,
            layout=layout,
            stack_direction=stack_direction,
            gap=gap,
            columns=columns,
            crop=crop,
            crop_bbox=crop_bbox,
            **kwargs,
        )

    # ------------------------------------------------------------------
    # Analysis helpers (delegated to a FlowRegion spanning all segments)
    # ------------------------------------------------------------------

    def apply_ocr(self, *args: Any, **kwargs: Any) -> "Flow":
        """
        Apply OCR across every segment in the flow.
        """

        self._analysis_region().apply_ocr(*args, **kwargs)
        return self

    def extract_ocr_elements(self, *args: Any, **kwargs: Any) -> List[Any]:
        """
        Extract OCR-derived text elements from all segments.
        """

        return self._analysis_region().extract_ocr_elements(*args, **kwargs)

    def remove_ocr_elements(self) -> int:
        """
        Remove OCR elements that were previously added to constituent pages.
        """

        return self._analysis_region().remove_ocr_elements()

    def clear_text_layer(self) -> Tuple[int, int]:
        """
        Clear the underlying text layers (words/chars) for every segment page.
        """

        return self._analysis_region().clear_text_layer()

    def create_text_elements_from_ocr(
        self,
        ocr_results: Any,
        scale_x: Optional[float] = None,
        scale_y: Optional[float] = None,
    ) -> List[Any]:
        """
        Utility for constructing text elements from OCR output.
        """

        return self._analysis_region().create_text_elements_from_ocr(
            ocr_results,
            scale_x=scale_x,
            scale_y=scale_y,
        )

    def ask(
        self,
        question: QuestionInput,
        min_confidence: float = 0.1,
        model: Optional[str] = None,
        debug: bool = False,
        **kwargs: Any,
    ) -> Any:
        """
        Run document QA across the flow by delegating to a FlowRegion.
        """

        return self._analysis_region().ask(
            question,
            min_confidence=min_confidence,
            model=model,
            debug=debug,
            **kwargs,
        )

    def find(
        self,
        selector: Optional[str] = None,
        *,
        text: Optional[Union[str, Sequence[str]]] = None,
        apply_exclusions: bool = True,
        regex: bool = False,
        case: bool = True,
        text_tolerance: Optional[Dict[str, Any]] = None,
        auto_text_tolerance: Optional[Dict[str, Any]] = None,
        reading_order: bool = True,
        engine: Optional[str] = None,
    ) -> Optional["FlowElement"]:
        """
        Finds the first element within the flow that matches the given selector or text criteria.

        Elements found are wrapped as FlowElement objects, anchored to this Flow.

        Args:
            selector: CSS-like selector string.
            text: Optional text shortcut (equivalent to ``text:contains(...)``).
            apply_exclusions: Whether to respect exclusion zones on the original pages/regions.
            regex: Whether the text search uses regex.
            case: Whether the text search is case-sensitive.
            text_tolerance: Optional dict of text tolerance overrides.
            auto_text_tolerance: Optional overrides controlling automatic tolerance logic.
            reading_order: Whether to sort matches in reading order when applicable (default: True).
            engine: Optional selector engine name registered via the selector provider.

        Returns:
            A FlowElement if a match is found, otherwise None.
        """
        results = self.find_all(
            selector=selector,
            text=text,
            apply_exclusions=apply_exclusions,
            regex=regex,
            case=case,
            text_tolerance=text_tolerance,
            auto_text_tolerance=auto_text_tolerance,
            reading_order=reading_order,
            engine=engine,
        )
        return results.first if results else None

    def find_all(
        self,
        selector: Optional[str] = None,
        *,
        text: Optional[Union[str, Sequence[str]]] = None,
        apply_exclusions: bool = True,
        regex: bool = False,
        case: bool = True,
        text_tolerance: Optional[Dict[str, Any]] = None,
        auto_text_tolerance: Optional[Dict[str, Any]] = None,
        reading_order: bool = True,
        engine: Optional[str] = None,
    ) -> "FlowElementCollection":
        """
        Finds all elements within the flow that match the given selector or text criteria.

        This method efficiently groups segments by their parent pages, searches at the page level,
        then filters results appropriately for each segment. This ensures elements that intersect
        with flow segments (but aren't fully contained) are still found.

        Elements found are wrapped as FlowElement objects, anchored to this Flow,
        and returned in a FlowElementCollection.

        Args:
            engine: Optional selector engine name forwarded to page-level queries.
        """
        from .collections import FlowElementCollection
        from .element import FlowElement

        # Step 1: Group segments by their parent pages (like in analyze_layout)
        segments_by_page = {}  # Dict[Page, List[Segment]]

        for i, segment in enumerate(self.segments):
            if hasattr(segment, "page") and hasattr(segment.page, "find_all"):
                page_obj = segment.page
                segment_type = "region"
            elif (
                hasattr(segment, "find_all")
                and hasattr(segment, "width")
                and hasattr(segment, "height")
                and not hasattr(segment, "page")
            ):
                page_obj = segment
                segment_type = "page"
            else:
                raise TypeError(f"Segment {i+1} does not support find_all: {segment!r}")

            if page_obj not in segments_by_page:
                segments_by_page[page_obj] = []
            segments_by_page[page_obj].append((segment, segment_type))

        if not segments_by_page:
            raise ValueError("No segments with searchable pages found")

        # Step 2: Search each unique page only once
        all_flow_elements: List["FlowElement"] = []

        for page_obj, page_segments in segments_by_page.items():
            # Find all matching elements on this page
            page_matches = page_obj.find_all(
                selector=selector,
                text=text,
                apply_exclusions=apply_exclusions,
                regex=regex,
                case=case,
                text_tolerance=text_tolerance,
                auto_text_tolerance=auto_text_tolerance,
                reading_order=reading_order,
                engine=engine,
            )

            if not page_matches:
                continue

            # Step 3: For each segment on this page, collect relevant elements
            for segment, segment_type in page_segments:
                if segment_type == "page":
                    # Full page segment: include all elements
                    for phys_elem in page_matches.elements:
                        all_flow_elements.append(FlowElement(physical_object=phys_elem, flow=self))

                elif segment_type == "region":
                    # Region segment: filter to only intersecting elements
                    for phys_elem in page_matches.elements:
                        try:
                            # Check if element intersects with this flow segment
                            if segment.intersects(phys_elem):
                                all_flow_elements.append(
                                    FlowElement(physical_object=phys_elem, flow=self)
                                )
                        except Exception as intersect_error:
                            logger.debug(
                                f"Error checking intersection for element: {intersect_error}"
                            )
                            # Include the element anyway if intersection check fails
                            all_flow_elements.append(
                                FlowElement(physical_object=phys_elem, flow=self)
                            )

        # Step 4: Remove duplicates (can happen if multiple segments intersect the same element)
        unique_flow_elements = []
        seen_element_ids = set()

        for flow_elem in all_flow_elements:
            # Create a unique identifier for the underlying physical element
            phys_elem = flow_elem.physical_object
            elem_id = (
                (
                    getattr(phys_elem.page, "index", id(phys_elem.page))
                    if hasattr(phys_elem, "page")
                    else id(phys_elem)
                ),
                phys_elem.bbox if hasattr(phys_elem, "bbox") else id(phys_elem),
            )

            if elem_id not in seen_element_ids:
                unique_flow_elements.append(flow_elem)
                seen_element_ids.add(elem_id)

        return FlowElementCollection(unique_flow_elements)

    def __repr__(self) -> str:
        return (
            f"<Flow segments={len(self.segments)}, "
            f"arrangement='{self.arrangement}', alignment='{self.alignment}', gap={self.segment_gap}>"
        )

    @overload
    def extract_table(
        self,
        method: Optional[str] = None,
        table_settings: Optional[dict] = None,
        use_ocr: bool = False,
        ocr_config: Optional[dict] = None,
        text_options: Optional[dict] = None,
        cell_extraction_func: Optional[Any] = None,
        show_progress: bool = False,
        content_filter: Optional[Any] = None,
        stitch_rows: Optional[Callable[[List[Optional[str]]], bool]] = None,
        merge_headers: Optional[bool] = None,
        structure_engine: Optional[str] = None,
    ) -> TableResult: ...

    @overload
    def extract_table(
        self,
        method: Optional[str] = None,
        table_settings: Optional[dict] = None,
        use_ocr: bool = False,
        ocr_config: Optional[dict] = None,
        text_options: Optional[dict] = None,
        cell_extraction_func: Optional[Any] = None,
        show_progress: bool = False,
        content_filter: Optional[Any] = None,
        stitch_rows: Optional[
            Callable[
                [List[Optional[str]], List[Optional[str]], int, Union["Page", "PhysicalRegion"]],
                bool,
            ]
        ] = None,
        merge_headers: Optional[bool] = None,
        structure_engine: Optional[str] = None,
    ) -> TableResult: ...

    def extract_table(
        self,
        method: Optional[str] = None,
        table_settings: Optional[dict] = None,
        use_ocr: bool = False,
        ocr_config: Optional[dict] = None,
        text_options: Optional[dict] = None,
        cell_extraction_func: Optional[Any] = None,
        show_progress: bool = False,
        content_filter: Optional[Any] = None,
        stitch_rows: Optional[Callable[..., bool]] = None,
        merge_headers: Optional[bool] = None,
        structure_engine: Optional[str] = None,
    ) -> TableResult:
        """
        Extract table data from all segments in the flow, combining results sequentially.

        This method extracts table data from each segment in flow order and combines
        the results into a single logical table. This is particularly useful for
        multi-page tables or tables that span across columns.

        Args:
            method: Method to use: 'tatr', 'pdfplumber', 'text', 'stream', 'lattice', or None (auto-detect).
            table_settings: Settings for pdfplumber table extraction.
            use_ocr: Whether to use OCR for text extraction (currently only applicable with 'tatr' method).
            ocr_config: OCR configuration parameters.
            text_options: Dictionary of options for the 'text' method.
            cell_extraction_func: Optional callable function that takes a cell Region object
                                  and returns its string content. For 'text' method only.
            show_progress: If True, display a progress bar during cell text extraction for the 'text' method.
            content_filter: Optional content filter to apply during cell text extraction.
            merge_headers: Whether to merge tables by removing repeated headers from subsequent
                segments. If None (default), auto-detects by checking if the first row
                of each segment matches the first row of the first segment. If segments have
                inconsistent header patterns (some repeat, others don't), raises ValueError.
                Useful for multi-page tables where headers repeat on each page.
            stitch_rows: Optional callable to determine when rows should be merged across
                         segment boundaries. Applied AFTER header removal if merge_headers
                         is enabled. Two overloaded signatures are supported:

                         • func(current_row) -> bool
                           Called only on the first row of each segment (after the first).
                           Return True to merge this first row with the last row from
                           the previous segment.

                         • func(prev_row, current_row, row_index, segment) -> bool
                           Called for every row. Return True to merge current_row with
                           the previous row in the aggregated results.

                         When True is returned, rows are concatenated cell-by-cell.
                         This is useful for handling table rows split across page
                         boundaries or segments. If None, rows are never merged.
            structure_engine: Optional structure detection engine forwarded to each segment's
                ``extract_table`` implementation before falling back to traditional engines.

        Returns:
            TableResult object containing the aggregated table data from all segments.

        Example:
            Multi-page table extraction:
            ```python
            pdf = npdf.PDF("multi_page_table.pdf")

            # Create flow for table spanning pages 2-4
            table_flow = Flow(
                segments=[pdf.pages[1], pdf.pages[2], pdf.pages[3]],
                arrangement='vertical'
            )

            # Extract table as if it were continuous
            table_data = table_flow.extract_table()
            df = table_data.df  # Convert to pandas DataFrame

            # Custom row stitching - single parameter (simple case)
            table_data = table_flow.extract_table(
                stitch_rows=lambda row: row and not (row[0] or "").strip()
            )

            # Custom row stitching - full parameters (advanced case)
            table_data = table_flow.extract_table(
                stitch_rows=lambda prev, curr, idx, seg: idx == 0 and curr and not (curr[0] or "").strip()
            )
            ```
        """
        logger.info(
            f"Extracting table from Flow with {len(self.segments)} segments (method: {method or 'auto'})"
        )

        if not self.segments:
            raise ValueError("Flow has no segments; cannot extract table")

        # Resolve predicate and determine its signature
        predicate: Optional[Callable] = None
        predicate_type: str = "none"

        if callable(stitch_rows):
            import inspect

            sig = inspect.signature(stitch_rows)
            param_count = len(sig.parameters)

            if param_count == 1:
                predicate = stitch_rows
                predicate_type = "single_param"
            elif param_count == 4:
                predicate = stitch_rows
                predicate_type = "full_params"
            else:
                raise TypeError(
                    f"stitch_rows function must accept 1 or 4 parameters, got {param_count}"
                )

        def _default_merge(
            prev_row: List[Optional[str]], cur_row: List[Optional[str]]
        ) -> List[Optional[str]]:
            from itertools import zip_longest

            merged: List[Optional[str]] = []
            for p, c in zip_longest(prev_row, cur_row, fillvalue=""):
                if (p or "").strip() and (c or "").strip():
                    merged.append(f"{p} {c}".strip())
                else:
                    merged.append((p or "") + (c or ""))
            return merged

        aggregated_rows: List[List[Optional[str]]] = []
        processed_segments = 0
        header_row: Optional[List[Optional[str]]] = None
        merge_headers_enabled = False
        headers_warned = False  # Track if we've already warned about dropping headers
        segment_has_repeated_header = []  # Track which segments have repeated headers

        for seg_idx, segment in enumerate(self.segments):
            try:
                logger.debug(f"  Extracting table from segment {seg_idx+1}/{len(self.segments)}")

                segment_result = segment.extract_table(
                    method=method,
                    table_settings=table_settings.copy() if table_settings else None,
                    use_ocr=use_ocr,
                    ocr_config=ocr_config,
                    text_options=text_options.copy() if text_options else None,
                    cell_extraction_func=cell_extraction_func,
                    show_progress=show_progress,
                    content_filter=content_filter,
                    structure_engine=structure_engine,
                )

                if not segment_result:
                    continue

                if hasattr(segment_result, "_rows"):
                    segment_rows = list(segment_result._rows)
                else:
                    segment_rows = list(segment_result)

                if not segment_rows:
                    logger.debug(f"    No table data found in segment {seg_idx+1}")
                    continue

                # Handle header detection and merging for multi-page tables
                if seg_idx == 0:
                    # First segment: capture potential header row
                    if segment_rows:
                        header_row = segment_rows[0]
                        # Determine if we should merge headers
                        if merge_headers is None:
                            # Auto-detect: we'll check all subsequent segments
                            merge_headers_enabled = False  # Will be determined later
                        else:
                            merge_headers_enabled = merge_headers
                        # Track that first segment exists (for consistency checking)
                        segment_has_repeated_header.append(False)  # First segment doesn't "repeat"
                elif seg_idx == 1 and merge_headers is None:
                    # Auto-detection: check if first row of second segment matches header
                    has_header = segment_rows and header_row and segment_rows[0] == header_row
                    segment_has_repeated_header.append(has_header)

                    if has_header:
                        merge_headers_enabled = True
                        # Remove the detected repeated header from this segment
                        segment_rows = segment_rows[1:]
                        logger.debug(
                            f"    Auto-detected repeated header in segment {seg_idx+1}, removed"
                        )
                        if not headers_warned:
                            warnings.warn(
                                "Detected repeated headers in multi-page table. Merging by removing "
                                "repeated headers from subsequent pages.",
                                UserWarning,
                                stacklevel=2,
                            )
                            headers_warned = True
                    else:
                        merge_headers_enabled = False
                        logger.debug(f"    No repeated header detected in segment {seg_idx+1}")
                elif seg_idx > 1:
                    # Check consistency: all segments should have same pattern
                    has_header = segment_rows and header_row and segment_rows[0] == header_row
                    segment_has_repeated_header.append(has_header)

                    # Remove header if merging is enabled and header is present
                    if merge_headers_enabled and has_header:
                        segment_rows = segment_rows[1:]
                        logger.debug(f"    Removed repeated header from segment {seg_idx+1}")
                elif seg_idx > 0 and merge_headers_enabled:
                    # Explicit merge_headers=True: remove headers from subsequent segments
                    if segment_rows and header_row and segment_rows[0] == header_row:
                        segment_rows = segment_rows[1:]
                        logger.debug(f"    Removed repeated header from segment {seg_idx+1}")
                        if not headers_warned:
                            warnings.warn(
                                "Removing repeated headers from multi-page table during merge.",
                                UserWarning,
                                stacklevel=2,
                            )
                            headers_warned = True

                for row_idx, row in enumerate(segment_rows):
                    should_merge = False

                    if predicate is not None and aggregated_rows:
                        if predicate_type == "single_param":
                            # For single param: only call on first row of segment (row_idx == 0)
                            # and pass the current row
                            if row_idx == 0:
                                should_merge = predicate(row)
                        elif predicate_type == "full_params":
                            # For full params: call with all arguments
                            should_merge = predicate(aggregated_rows[-1], row, row_idx, segment)

                    if should_merge:
                        aggregated_rows[-1] = _default_merge(aggregated_rows[-1], row)
                    else:
                        aggregated_rows.append(row)

                processed_segments += 1
                logger.debug(
                    f"    Added {len(segment_rows)} rows (post-merge) from segment {seg_idx+1}"
                )

            except Exception as e:
                logger.error(f"Error extracting table from segment {seg_idx+1}: {e}", exc_info=True)
                continue

        # Check for inconsistent header patterns after processing all segments
        if merge_headers is None and len(segment_has_repeated_header) > 2:
            # During auto-detection, check for consistency across all segments
            expected_pattern = segment_has_repeated_header[1]  # Pattern from second segment
            for seg_idx, has_header in enumerate(segment_has_repeated_header[2:], 2):
                if has_header != expected_pattern:
                    # Inconsistent pattern detected
                    segments_with_headers = [
                        i for i, has_h in enumerate(segment_has_repeated_header[1:], 1) if has_h
                    ]
                    segments_without_headers = [
                        i for i, has_h in enumerate(segment_has_repeated_header[1:], 1) if not has_h
                    ]
                    raise ValueError(
                        f"Inconsistent header pattern in multi-page table: "
                        f"segments {segments_with_headers} have repeated headers, "
                        f"but segments {segments_without_headers} do not. "
                        f"All segments must have the same header pattern for reliable merging."
                    )

        logger.info(
            f"Flow table extraction complete: {len(aggregated_rows)} total rows from {processed_segments}/{len(self.segments)} segments"
        )
        return TableResult(aggregated_rows)

    def extract_tables(
        self,
        method: Optional[str] = None,
        table_settings: Optional[dict] = None,
        **kwargs,
    ) -> List[List[List[Optional[str]]]]:
        """Extract every table across all segments in reading order.

        Args:
            method: Optional table engine name. Mirrors :meth:`Flow.extract_table`.
            table_settings: Base pdfplumber settings (copied per segment).
            **kwargs: Additional keyword arguments forwarded to each segment's
                :meth:`extract_tables` implementation (e.g., ``content_filter``).

        Returns:
            List of tables, where each table is represented as a list of rows.
            The order matches the order of ``self.segments``.
        """

        if not self.segments:
            return []

        base_settings = table_settings.copy() if table_settings else None
        all_tables: List[List[List[Optional[str]]]] = []

        for segment in self.segments:
            segment_settings = base_settings.copy() if base_settings else None
            tables = cast(
                List[List[List[Optional[str]]]],
                segment.extract_tables(
                    method=method,
                    table_settings=segment_settings,
                    **kwargs,
                ),
            )
            if tables:
                all_tables.extend(tables)

        return all_tables

    def analyze_layout(
        self,
        engine: Optional[str] = None,
        options: Optional[Any] = None,
        confidence: Optional[float] = None,
        classes: Optional[List[str]] = None,
        exclude_classes: Optional[List[str]] = None,
        device: Optional[str] = None,
        existing: str = "replace",
        model_name: Optional[str] = None,
        client: Optional[Any] = None,
    ) -> "PhysicalElementCollection":
        """
        Analyze layout across all segments in the flow.

        This method efficiently groups segments by their parent pages, runs layout analysis
        only once per unique page, then filters results appropriately for each segment.
        This avoids redundant analysis when multiple flow segments come from the same page.

        Args:
            engine: Name of the layout engine (e.g., 'yolo', 'tatr'). Uses manager's default if None.
            options: Specific LayoutOptions object for advanced configuration.
            confidence: Minimum confidence threshold.
            classes: Specific classes to detect.
            exclude_classes: Classes to exclude.
            device: Device for inference.
            existing: How to handle existing detected regions: 'replace' (default) or 'append'.
            model_name: Optional model name for the engine.
            client: Optional client for API-based engines.

        Returns:
            ElementCollection containing all detected Region objects from all segments.

        Example:
            Multi-page layout analysis:
            ```python
            pdf = npdf.PDF("document.pdf")

            # Create flow for first 3 pages
            page_flow = Flow(
                segments=pdf.pages[:3],
                arrangement='vertical'
            )

            # Analyze layout across all pages (efficiently)
            all_regions = page_flow.analyze_layout(engine='yolo')

            # Find all tables across the flow
            tables = all_regions.filter('region[type=table]')
            ```
        """
        from natural_pdf.elements.element_collection import ElementCollection

        logger.info(
            f"Analyzing layout across Flow with {len(self.segments)} segments (engine: {engine or 'default'})"
        )

        if not self.segments:
            raise ValueError("Flow has no segments; cannot analyze layout")

        # Step 1: Group segments by their parent pages to avoid redundant analysis
        segments_by_page = {}  # Dict[Page, List[Segment]]

        for i, segment in enumerate(self.segments):
            if hasattr(segment, "analyze_layout"):
                page_obj = segment
                segment_type = "page"
            elif hasattr(segment, "page") and hasattr(segment.page, "analyze_layout"):
                page_obj = segment.page
                segment_type = "region"
            else:
                raise TypeError(f"Segment {i+1} does not support layout analysis: {segment!r}")

            if page_obj not in segments_by_page:
                segments_by_page[page_obj] = []
            segments_by_page[page_obj].append((segment, segment_type))

        if not segments_by_page:
            raise ValueError("No segments with analyzable pages found")

        logger.debug(
            f"  Grouped {len(self.segments)} segments into {len(segments_by_page)} unique pages"
        )

        # Step 2: Analyze each unique page only once
        all_detected_regions: List["PhysicalRegion"] = []
        processed_pages = 0

        for page_obj, page_segments in segments_by_page.items():
            try:
                logger.debug(
                    f"  Analyzing layout for page {getattr(page_obj, 'number', '?')} with {len(page_segments)} segments"
                )

                # Run layout analysis once for this page
                page_results = page_obj.analyze_layout(
                    engine=engine,
                    options=options,
                    confidence=confidence,
                    classes=classes,
                    exclude_classes=exclude_classes,
                    device=device,
                    existing=existing,
                    model_name=model_name,
                    client=client,
                )

                # Extract regions from results
                if hasattr(page_results, "elements"):
                    # It's an ElementCollection
                    page_regions = page_results.elements
                elif isinstance(page_results, list):
                    # It's a list of regions
                    page_regions = page_results
                else:
                    raise TypeError(
                        f"Page {getattr(page_obj, 'number', '?')} returned unexpected layout analysis result type: {type(page_results)}"
                    )

                if not page_regions:
                    logger.debug(
                        f"    No layout regions found on page {getattr(page_obj, 'number', '?')}"
                    )
                    continue

                # Step 3: For each segment on this page, collect relevant regions
                segments_processed_on_page = 0
                for segment, segment_type in page_segments:
                    if segment_type == "page":
                        # Full page segment: include all detected regions
                        all_detected_regions.extend(page_regions)
                        segments_processed_on_page += 1
                        logger.debug(f"    Added {len(page_regions)} regions for full-page segment")

                    elif segment_type == "region":
                        # Region segment: filter to only intersecting regions
                        intersecting_regions = []
                        for region in page_regions:
                            try:
                                if segment.intersects(region):
                                    intersecting_regions.append(region)
                            except Exception as intersect_error:
                                logger.debug(
                                    f"Error checking intersection for region: {intersect_error}"
                                )
                                # Include the region anyway if intersection check fails
                                intersecting_regions.append(region)

                        all_detected_regions.extend(intersecting_regions)
                        segments_processed_on_page += 1
                        logger.debug(
                            f"    Added {len(intersecting_regions)} intersecting regions for region segment {segment.bbox}"
                        )

                processed_pages += 1
                logger.debug(
                    f"    Processed {segments_processed_on_page} segments on page {getattr(page_obj, 'number', '?')}"
                )

            except Exception as e:
                logger.error(
                    f"Error analyzing layout for page {getattr(page_obj, 'number', '?')}: {e}",
                    exc_info=True,
                )
                continue

        # Step 4: Remove duplicates (can happen if multiple segments intersect the same region)
        unique_regions = []
        seen_region_ids = set()

        for region in all_detected_regions:
            # Create a unique identifier for this region (page + bbox)
            region_id = (
                getattr(region.page, "index", id(region.page)),
                region.bbox if hasattr(region, "bbox") else id(region),
            )

            if region_id not in seen_region_ids:
                unique_regions.append(region)
                seen_region_ids.add(region_id)

        dedupe_removed = len(all_detected_regions) - len(unique_regions)
        if dedupe_removed > 0:
            logger.debug(f"  Removed {dedupe_removed} duplicate regions")

        logger.info(
            f"Flow layout analysis complete: {len(unique_regions)} unique regions from {processed_pages} pages"
        )
        return ElementCollection(unique_regions)

    def _get_render_specs(
        self,
        mode: Literal["show", "render"] = "show",
        color: Optional[Union[str, Tuple[int, int, int]]] = None,
        highlights: Optional[List[Dict[str, Any]]] = None,
        crop: Union[bool, Literal["content"]] = False,
        crop_bbox: Optional[Tuple[float, float, float, float]] = None,
        label_prefix: Optional[str] = "FlowSegment",
        **kwargs,
    ) -> List[RenderSpec]:
        """Get render specifications for this flow.

        Args:
            mode: Rendering mode - 'show' includes highlights, 'render' is clean
            color: Color for highlighting segments in show mode
            highlights: Additional highlight groups to show
            crop: Whether to crop to segments
            crop_bbox: Explicit crop bounds
            label_prefix: Prefix for segment labels
            **kwargs: Additional parameters

        Returns:
            List of RenderSpec objects, one per page with segments
        """
        if not self.segments:
            return []

        # Group segments by their physical pages
        segments_by_page = {}  # Dict[Page, List[PhysicalRegion]]

        for i, segment in enumerate(self.segments):
            # Get the page for this segment
            if hasattr(segment, "page") and segment.page is not None:
                # It's a Region, use its page
                page_obj = segment.page
                if page_obj not in segments_by_page:
                    segments_by_page[page_obj] = []
                segments_by_page[page_obj].append(segment)
            elif (
                hasattr(segment, "index")
                and hasattr(segment, "width")
                and hasattr(segment, "height")
            ):
                # It's a full Page object, create a full-page region for it
                page_obj = segment
                full_page_region = segment.region(0, 0, segment.width, segment.height)
                if page_obj not in segments_by_page:
                    segments_by_page[page_obj] = []
                segments_by_page[page_obj].append(full_page_region)
            else:
                logger.warning(f"Segment {i+1} has no identifiable page, skipping")
                continue

        if not segments_by_page:
            return []

        # Create RenderSpec for each page
        specs = []

        # Sort pages by index for consistent output order
        sorted_pages = sorted(
            segments_by_page.keys(),
            key=lambda p: p.index if hasattr(p, "index") else getattr(p, "page_number", 0),
        )

        for page_idx, page_obj in enumerate(sorted_pages):
            segments_on_this_page = segments_by_page[page_obj]
            if not segments_on_this_page:
                raise RuntimeError(
                    f"No segments recorded for page {getattr(page_obj, 'number', '?')}"
                )

            spec = RenderSpec(page=page_obj)

            # Handle cropping
            if crop_bbox:
                spec.crop_bbox = crop_bbox
            elif crop == "content" or crop is True:
                # Calculate bounds of segments on this page
                x_coords = []
                y_coords = []
                for segment in segments_on_this_page:
                    if hasattr(segment, "bbox") and segment.bbox:
                        x0, y0, x1, y1 = segment.bbox
                        x_coords.extend([x0, x1])
                        y_coords.extend([y0, y1])

                if x_coords and y_coords:
                    spec.crop_bbox = (min(x_coords), min(y_coords), max(x_coords), max(y_coords))

            # Add highlights in show mode
            if mode == "show":
                # Highlight segments
                for i, segment in enumerate(segments_on_this_page):
                    segment_label = None
                    if label_prefix:
                        # Create label for this segment
                        global_segment_idx = None
                        try:
                            # Find the global index of this segment in the original flow
                            global_segment_idx = self.segments.index(segment)
                        except ValueError:
                            # If it's a generated full-page region, find its source page
                            seg_page = getattr(segment, "page", None)
                            if seg_page is not None:
                                for idx, orig_segment in enumerate(self.segments):
                                    if getattr(orig_segment, "page", None) is seg_page:
                                        global_segment_idx = idx
                                        break

                        if global_segment_idx is not None:
                            segment_label = f"{label_prefix}_{global_segment_idx + 1}"
                        else:
                            segment_label = f"{label_prefix}_p{page_idx + 1}s{i + 1}"

                    spec.add_highlight(
                        bbox=segment.bbox,
                        polygon=segment.polygon if segment.has_polygon else None,
                        color=color or "blue",
                        label=segment_label,
                    )

                # Add additional highlight groups if provided
                if highlights:
                    for group in highlights:
                        group_elements = group.get("elements", [])
                        group_color = group.get("color", color)
                        group_label = group.get("label")

                        for elem in group_elements:
                            # Only add if element is on this page
                            if hasattr(elem, "page") and elem.page == page_obj:
                                spec.add_highlight(
                                    element=elem, color=group_color, label=group_label
                                )

            specs.append(spec)

        return specs

    def _show_in_context(
        self,
        resolution: float,
        width: Optional[int] = None,
        stack_direction: str = "vertical",
        stack_gap: int = 5,
        stack_background_color: Tuple[int, int, int] = (255, 255, 255),
        separator_color: Tuple[int, int, int] = (255, 0, 0),
        separator_thickness: int = 2,
        **kwargs,
    ) -> Optional["PIL_Image"]:
        """
        Show segments as cropped images stacked together with separators between segments.

        Args:
            resolution: Resolution in DPI for rendering segment images
            width: Optional width for segment images
            stack_direction: Direction to stack segments ('vertical' or 'horizontal')
            stack_gap: Gap in pixels between segments
            stack_background_color: RGB background color for the final image
            separator_color: RGB color for separator lines between segments
            separator_thickness: Thickness in pixels of separator lines
            **kwargs: Additional arguments passed to segment rendering

        Returns:
            PIL Image with all segments stacked together
        """
        from PIL import Image, ImageDraw

        segment_images = []
        segment_pages = []

        # Determine stacking direction
        final_stack_direction = stack_direction
        if stack_direction == "auto":
            final_stack_direction = self.arrangement

        # Get cropped images for each segment
        for i, segment in enumerate(self.segments):
            # Get the page reference for this segment
            if hasattr(segment, "page") and segment.page is not None:
                segment_page = segment.page
                # Get cropped image of the segment
                # Use render() for clean image without highlights
                segment_image = segment.render(
                    resolution=resolution,
                    crop=True,
                    width=width,
                    **kwargs,
                )

            elif (
                hasattr(segment, "index")
                and hasattr(segment, "width")
                and hasattr(segment, "height")
            ):
                # It's a full Page object
                segment_page = segment
                # Use render() for clean image without highlights
                segment_image = segment.render(resolution=resolution, width=width, **kwargs)
            else:
                raise ValueError(
                    f"Segment {i+1} has no identifiable page. Segment type: {type(segment)}, attributes: {dir(segment)}"
                )

            if segment_image is None:
                raise RuntimeError(f"Segment {i+1} render() returned None")

            segment_images.append(segment_image)
            segment_pages.append(segment_page)

        # Check if we have any valid images
        if not segment_images:
            logger.error("No valid segment images could be rendered")
            return None

        # We should have at least one segment image by now (or an exception would have been raised)
        if len(segment_images) == 1:
            return segment_images[0]

        # Calculate dimensions for the final stacked image
        if final_stack_direction == "vertical":
            # Stack vertically
            final_width = max(img.width for img in segment_images)

            # Calculate total height including gaps and separators
            total_height = sum(img.height for img in segment_images)
            total_height += (len(segment_images) - 1) * stack_gap

            # Add separator thickness between all segments
            num_separators = len(segment_images) - 1 if len(segment_images) > 1 else 0
            total_height += num_separators * separator_thickness

            # Create the final image
            final_image = Image.new("RGB", (final_width, total_height), stack_background_color)
            draw = ImageDraw.Draw(final_image)

            current_y = 0

            for i, img in enumerate(segment_images):
                # Add separator line before each segment (except the first one)
                if i > 0:
                    # Draw separator line
                    draw.rectangle(
                        [(0, current_y), (final_width, current_y + separator_thickness)],
                        fill=separator_color,
                    )
                    current_y += separator_thickness

                # Paste the segment image
                paste_x = (final_width - img.width) // 2  # Center horizontally
                final_image.paste(img, (paste_x, current_y))
                current_y += img.height

                # Add gap after segment (except for the last one)
                if i < len(segment_images) - 1:
                    current_y += stack_gap

            return final_image

        elif final_stack_direction == "horizontal":
            # Stack horizontally
            final_height = max(img.height for img in segment_images)

            # Calculate total width including gaps and separators
            total_width = sum(img.width for img in segment_images)
            total_width += (len(segment_images) - 1) * stack_gap

            # Add separator thickness between all segments
            num_separators = len(segment_images) - 1 if len(segment_images) > 1 else 0
            total_width += num_separators * separator_thickness

            # Create the final image
            final_image = Image.new("RGB", (total_width, final_height), stack_background_color)
            draw = ImageDraw.Draw(final_image)

            current_x = 0

            for i, img in enumerate(segment_images):
                # Add separator line before each segment (except the first one)
                if i > 0:
                    # Draw separator line
                    draw.rectangle(
                        [(current_x, 0), (current_x + separator_thickness, final_height)],
                        fill=separator_color,
                    )
                    current_x += separator_thickness

                # Paste the segment image
                paste_y = (final_height - img.height) // 2  # Center vertically
                final_image.paste(img, (current_x, paste_y))
                current_x += img.width

                # Add gap after segment (except for the last one)
                if i < len(segment_images) - 1:
                    current_x += stack_gap

            return final_image

        else:
            raise ValueError(
                f"Invalid stack_direction '{final_stack_direction}' for in_context. Must be 'vertical' or 'horizontal'."
            )

    # --- Helper methods for coordinate transformations and segment iteration ---
    # These will be crucial for FlowElement's directional methods.

    def get_segment_bounding_box_in_flow(
        self, segment_index: int
    ) -> Optional[tuple[float, float, float, float]]:
        """
        Calculates the conceptual bounding box of a segment within the flow's coordinate system.
        This considers arrangement, alignment, and segment gaps.
        (This is a placeholder for more complex logic if a true virtual coordinate system is needed)
        For now, it might just return the physical segment's bbox if gaps are 0 and alignment is simple.
        """
        if segment_index < 0 or segment_index >= len(self.segments):
            return None

        # This is a simplified version. A full implementation would calculate offsets.
        # For now, we assume FlowElement directional logic handles segment traversal and uses physical coords.
        # If we were to *draw* the flow or get a FlowRegion bbox that spans gaps, this would be critical.
        # physical_segment = self.segments[segment_index]
        # return physical_segment.bbox
        raise NotImplementedError(
            "Calculating a segment's bbox *within the flow's virtual coordinate system* is not yet fully implemented."
        )

    def get_element_flow_coordinates(
        self, physical_element: "PhysicalElement"
    ) -> Optional[tuple[float, float, float, float]]:
        """
        Translates a physical element's coordinates into the flow's virtual coordinate system.
        (Placeholder - very complex if segment_gap > 0 or complex alignments)
        """
        # For now, elements operate in their own physical coordinates. This method would be needed
        # if FlowRegion.bbox or other operations needed to present a unified coordinate space.
        # As per our discussion, elements *within* a FlowRegion retain original physical coordinates.
        # So, this might not be strictly necessary for the current design's core functionality.
        raise NotImplementedError(
            "Translating element coordinates to a unified flow coordinate system is not yet implemented."
        )

    def get_sections(
        self,
        start_elements=None,
        end_elements=None,
        new_section_on_page_break: bool = False,
        include_boundaries: str = "both",
        orientation: str = "vertical",
    ) -> "PhysicalElementCollection":
        """
        Extract logical sections from the Flow based on *start* and *end* boundary
        elements, mirroring the behaviour of PDF/PageCollection.get_sections().

        This implementation is a thin wrapper that converts the Flow into a
        temporary PageCollection (constructed from the unique pages that the
        Flow spans) and then delegates the heavy‐lifting to that existing
        implementation.  Any FlowElement / FlowElementCollection inputs are
        automatically unwrapped to their underlying physical elements so that
        PageCollection can work with them directly.

        Args:
            start_elements: Elements or selector string that mark the start of
                sections (optional).
            end_elements: Elements or selector string that mark the end of
                sections (optional).
            new_section_on_page_break: Whether to start a new section at page
                boundaries (default: False).
            include_boundaries: How to include boundary elements: 'start',
                'end', 'both', or 'none' (default: 'both').
            orientation: 'vertical' (default) or 'horizontal' - determines section direction.

        Returns:
            ElementCollection of Region/FlowRegion objects representing the
            extracted sections.
        """
        # ------------------------------------------------------------------
        # Unwrap FlowElement(-Collection) inputs and selector strings so we
        # can reason about them generically.
        # ------------------------------------------------------------------
        from natural_pdf.flows.collections import FlowElementCollection
        from natural_pdf.flows.element import FlowElement

        def _unwrap(obj):
            """Convert Flow-specific wrappers to their underlying physical objects.

            Keeps selector strings as-is; converts FlowElement to its physical
            element; converts FlowElementCollection to list of physical
            elements; passes through ElementCollection by taking .elements.
            """

            if obj is None or isinstance(obj, str):
                return obj

            if isinstance(obj, FlowElement):
                return obj.physical_object

            if isinstance(obj, FlowElementCollection):
                return [fe.physical_object for fe in obj.flow_elements]

            if hasattr(obj, "elements"):
                return obj.elements

            if isinstance(obj, (list, tuple, set)):
                out = []
                for item in obj:
                    if isinstance(item, FlowElement):
                        out.append(item.physical_object)
                    else:
                        out.append(item)
                return out

            return obj  # Fallback – unknown type

        start_elements_unwrapped = _unwrap(start_elements)
        end_elements_unwrapped = _unwrap(end_elements)

        # ------------------------------------------------------------------
        # For Flow, we need to handle sections that may span segments
        # We'll process all segments together, not independently
        # ------------------------------------------------------------------
        from natural_pdf.elements.element_collection import ElementCollection
        from natural_pdf.elements.region import Region
        from natural_pdf.flows.region import FlowRegion

        # Helper to check if element is in segment
        def _element_in_segment(elem, segment):
            # Simple bbox check
            return (
                elem.page == segment.page
                and elem.top >= segment.top
                and elem.bottom <= segment.bottom
                and elem.x0 >= segment.x0
                and elem.x1 <= segment.x1
            )

        # Collect all boundary elements with their segment info
        all_starts = []
        all_ends = []

        for seg_idx, segment in enumerate(self.segments):
            # Find starts in this segment
            if isinstance(start_elements_unwrapped, str):
                seg_starts = segment.find_all(start_elements_unwrapped).elements
            elif start_elements_unwrapped:
                if isinstance(start_elements_unwrapped, Iterable):
                    candidates = list(start_elements_unwrapped)
                else:
                    candidates = [start_elements_unwrapped]
                seg_starts = [e for e in candidates if _element_in_segment(e, segment)]
            else:
                seg_starts = []

            for elem in seg_starts:
                all_starts.append((elem, seg_idx, segment))

            # Find ends in this segment
            if isinstance(end_elements_unwrapped, str):
                seg_ends = segment.find_all(end_elements_unwrapped).elements
            elif end_elements_unwrapped:
                if isinstance(end_elements_unwrapped, Iterable):
                    candidates_end = list(end_elements_unwrapped)
                else:
                    candidates_end = [end_elements_unwrapped]
                seg_ends = [e for e in candidates_end if _element_in_segment(e, segment)]
            else:
                seg_ends = []

            for elem in seg_ends:
                all_ends.append((elem, seg_idx, segment))

        # Sort by segment index, then position
        all_starts.sort(key=lambda x: (x[1], x[0].top, x[0].x0))
        all_ends.sort(key=lambda x: (x[1], x[0].top, x[0].x0))

        # When only end elements are supplied we synthesise implicit start
        # markers so we can reuse the "start-only" logic below.  This mirrors
        # the historical behaviour of PageCollection.get_sections which
        # created implicit starts at the top of the document and immediately
        # after each end boundary.
        if not all_starts and all_ends:

            def _create_implicit(segment, position):
                thickness = 0.1
                if orientation == "vertical":
                    top = max(segment.top, min(position, segment.bottom))
                    bottom = min(segment.bottom, top + thickness)
                    if bottom <= top:
                        bottom = min(segment.bottom, top + thickness)
                        if bottom <= top:
                            bottom = top
                    implicit_region = Region(segment.page, (segment.x0, top, segment.x1, bottom))
                else:
                    left = max(segment.x0, min(position, segment.x1))
                    right = min(segment.x1, left + thickness)
                    if right <= left:
                        right = min(segment.x1, left + thickness)
                        if right <= left:
                            right = left
                    implicit_region = Region(
                        segment.page, (left, segment.top, right, segment.bottom)
                    )

                implicit_region.metadata["is_implicit_start"] = True
                return implicit_region

            synthetic_starts: List[Tuple[Region, int, Region]] = []

            first_segment = self.segments[0]
            initial_position = first_segment.top if orientation == "vertical" else first_segment.x0
            synthetic_starts.append(
                (
                    _create_implicit(first_segment, initial_position),
                    0,
                    first_segment,
                )
            )

            seen_end_ids = set()
            for end_elem, end_seg_idx, _ in all_ends:
                if id(end_elem) in seen_end_ids:
                    continue
                seen_end_ids.add(id(end_elem))

                position = end_elem.bottom if orientation == "vertical" else end_elem.x1
                if include_boundaries in ["start", "none"]:
                    position = end_elem.top if orientation == "vertical" else end_elem.x0

                segment = self.segments[end_seg_idx]
                synthetic_starts.append(
                    (
                        _create_implicit(segment, position),
                        end_seg_idx,
                        segment,
                    )
                )

            all_starts = synthetic_starts
            all_ends = []

        # If no boundary elements found, return empty collection
        if not all_starts and not all_ends:
            return ElementCollection([])

        sections = []

        # Case 1: Only start elements provided
        if all_starts and not all_ends:
            for i in range(len(all_starts)):
                start_elem, start_seg_idx, start_seg = all_starts[i]

                # Find end (next start or end of flow)
                if i + 1 < len(all_starts):
                    # Section ends at next start
                    end_elem, end_seg_idx, end_seg = all_starts[i + 1]

                    if start_seg_idx == end_seg_idx:
                        # Same segment - create regular Region
                        section = start_seg.get_section_between(
                            start_elem, end_elem, include_boundaries, orientation
                        )
                        if section:
                            sections.append(section)
                    else:
                        # Cross-segment - create FlowRegion
                        regions = []

                        # First segment: from start to bottom
                        if include_boundaries in ["both", "start"]:
                            top = start_elem.top
                        else:
                            top = start_elem.bottom
                        regions.append(
                            Region(
                                start_seg.page, (start_seg.x0, top, start_seg.x1, start_seg.bottom)
                            )
                        )

                        # Middle segments (full)
                        for idx in range(start_seg_idx + 1, end_seg_idx):
                            regions.append(self.segments[idx])

                        # Last segment: from top to end element
                        if include_boundaries in ["both", "end"]:
                            bottom = end_elem.bottom
                        else:
                            bottom = end_elem.top
                        regions.append(
                            Region(end_seg.page, (end_seg.x0, end_seg.top, end_seg.x1, bottom))
                        )

                        # Create FlowRegion
                        flow_element = FlowElement(physical_object=start_elem, flow=self)
                        flow_region = FlowRegion(
                            flow=self,
                            constituent_regions=regions,
                            source_flow_element=flow_element,
                            boundary_element_found=end_elem,
                        )
                        flow_region.start_element = start_elem
                        flow_region.end_element = end_elem
                        flow_region._boundary_exclusions = include_boundaries
                        sections.append(flow_region)
                else:
                    # Last section - goes to end of flow
                    if start_seg_idx == len(self.segments) - 1:
                        # Within last segment
                        section = start_seg.get_section_between(
                            start_elem, None, include_boundaries, orientation
                        )
                        if section:
                            sections.append(section)
                    else:
                        # Spans to end
                        regions = []

                        # First segment: from start to bottom
                        if include_boundaries in ["both", "start"]:
                            top = start_elem.top
                        else:
                            top = start_elem.bottom
                        regions.append(
                            Region(
                                start_seg.page, (start_seg.x0, top, start_seg.x1, start_seg.bottom)
                            )
                        )

                        # Remaining segments (full)
                        for idx in range(start_seg_idx + 1, len(self.segments)):
                            regions.append(self.segments[idx])

                        # Create FlowRegion
                        flow_element = FlowElement(physical_object=start_elem, flow=self)
                        flow_region = FlowRegion(
                            flow=self,
                            constituent_regions=regions,
                            source_flow_element=flow_element,
                            boundary_element_found=None,
                        )
                        flow_region.start_element = start_elem
                        flow_region._boundary_exclusions = include_boundaries
                        sections.append(flow_region)

        # Case 2: Both start and end elements
        elif all_starts and all_ends:
            # Match starts with ends
            used_ends = set()

            for start_elem, start_seg_idx, start_seg in all_starts:
                # Find matching end
                best_end = None

                for end_elem, end_seg_idx, end_seg in all_ends:
                    if id(end_elem) in used_ends:
                        continue

                    # End must come after start
                    if end_seg_idx > start_seg_idx or (
                        end_seg_idx == start_seg_idx and end_elem.top >= start_elem.bottom
                    ):
                        best_end = (end_elem, end_seg_idx, end_seg)
                        break

                if best_end:
                    end_elem, end_seg_idx, end_seg = best_end
                    used_ends.add(id(end_elem))

                    if start_seg_idx == end_seg_idx:
                        # Same segment
                        section = start_seg.get_section_between(
                            start_elem, end_elem, include_boundaries, orientation
                        )
                        if section:
                            sections.append(section)
                    else:
                        # Cross-segment FlowRegion
                        regions = []

                        # First segment
                        if include_boundaries in ["both", "start"]:
                            top = start_elem.top
                        else:
                            top = start_elem.bottom
                        regions.append(
                            Region(
                                start_seg.page, (start_seg.x0, top, start_seg.x1, start_seg.bottom)
                            )
                        )

                        # Middle segments
                        for idx in range(start_seg_idx + 1, end_seg_idx):
                            regions.append(self.segments[idx])

                        # Last segment
                        if include_boundaries in ["both", "end"]:
                            bottom = end_elem.bottom
                        else:
                            bottom = end_elem.top
                        regions.append(
                            Region(end_seg.page, (end_seg.x0, end_seg.top, end_seg.x1, bottom))
                        )

                        # Create FlowRegion
                        flow_element = FlowElement(physical_object=start_elem, flow=self)
                        flow_region = FlowRegion(
                            flow=self,
                            constituent_regions=regions,
                            source_flow_element=flow_element,
                            boundary_element_found=end_elem,
                        )
                        flow_region.start_element = start_elem
                        flow_region.end_element = end_elem
                        flow_region._boundary_exclusions = include_boundaries
                        sections.append(flow_region)

        # Case 3 is handled by synthesising implicit start elements above.
        elif not all_starts and all_ends:
            pass

        return ElementCollection(sections)

    def highlights(self, show: bool = False):
        """
        Create a highlight context for accumulating highlights.

        This allows for clean syntax to show multiple highlight groups:

        Example:
            with flow.highlights() as h:
                h.add(flow.find_all('table'), label='tables', color='blue')
                h.add(flow.find_all('text:bold'), label='bold text', color='red')
                h.show()

        Or with automatic display:
            with flow.highlights(show=True) as h:
                h.add(flow.find_all('table'), label='tables')
                h.add(flow.find_all('text:bold'), label='bold')
                # Automatically shows when exiting the context

        Args:
            show: If True, automatically show highlights when exiting context

        Returns:
            HighlightContext for accumulating highlights
        """
        from natural_pdf.core.highlighting_service import HighlightContext

        return HighlightContext(self, show_on_exit=show)

Functions

natural_pdf.Flow.__init__(segments, arrangement, alignment='start', segment_gap=0.0)

Initializes a Flow object.

Parameters:

Name	Type	Description	Default
segments	Union[Sequence[SupportsSections], PageCollection]	An ordered sequence of objects implementing SupportsSections (e.g., Page, Region) that constitute the flow, or a PageCollection containing pages.	required
arrangement	Literal['vertical', 'horizontal']	The primary direction of the flow. - "vertical": Segments are stacked top-to-bottom. - "horizontal": Segments are arranged left-to-right.	required
alignment	Literal['start', 'center', 'end', 'top', 'left', 'bottom', 'right']	How segments are aligned on their cross-axis if they have differing dimensions. For a "vertical" arrangement: - "left" (or "start"): Align left edges. - "center": Align centers. - "right" (or "end"): Align right edges. For a "horizontal" arrangement: - "top" (or "start"): Align top edges. - "center": Align centers. - "bottom" (or "end"): Align bottom edges.	'start'
segment_gap	float	The virtual gap (in PDF points) between segments.	0.0

Source code in natural_pdf/flows/flow.py

def __init__(
    self,
    segments: Union[Sequence[SupportsSections], "PageCollection"],
    arrangement: Literal["vertical", "horizontal"],
    alignment: Literal["start", "center", "end", "top", "left", "bottom", "right"] = "start",
    segment_gap: float = 0.0,
):
    """
    Initializes a Flow object.

    Args:
        segments: An ordered sequence of objects implementing SupportsSections (e.g., Page,
                  Region) that constitute the flow, or a PageCollection containing pages.
        arrangement: The primary direction of the flow.
                     - "vertical": Segments are stacked top-to-bottom.
                     - "horizontal": Segments are arranged left-to-right.
        alignment: How segments are aligned on their cross-axis if they have
                   differing dimensions. For a "vertical" arrangement:
                   - "left" (or "start"): Align left edges.
                   - "center": Align centers.
                   - "right" (or "end"): Align right edges.
                   For a "horizontal" arrangement:
                   - "top" (or "start"): Align top edges.
                   - "center": Align centers.
                   - "bottom" (or "end"): Align bottom edges.
        segment_gap: The virtual gap (in PDF points) between segments.
    """
    # Handle PageCollection input
    from natural_pdf.core.page_collection import PageCollection as _PageCollection

    if isinstance(segments, _PageCollection):
        segment_list: List[SupportsSections] = list(segments.pages)
    else:
        segment_list = list(segments)

    if not segment_list:
        raise ValueError("Flow segments cannot be empty.")
    if arrangement not in ["vertical", "horizontal"]:
        raise ValueError("Arrangement must be 'vertical' or 'horizontal'.")

    self.segments: List["PhysicalRegion"] = self._normalize_segments(segment_list)
    self.arrangement: Literal["vertical", "horizontal"] = arrangement
    self.alignment: Literal["start", "center", "end", "top", "left", "bottom", "right"] = (
        alignment
    )
    self.segment_gap: float = segment_gap
    self._analysis_region_cache: Optional["FlowRegion"] = None

    self._validate_alignment()

natural_pdf.Flow.analyze_layout(engine=None, options=None, confidence=None, classes=None, exclude_classes=None, device=None, existing='replace', model_name=None, client=None)

Analyze layout across all segments in the flow.

This method efficiently groups segments by their parent pages, runs layout analysis only once per unique page, then filters results appropriately for each segment. This avoids redundant analysis when multiple flow segments come from the same page.

Parameters:

Name	Type	Description	Default
engine	Optional[str]	Name of the layout engine (e.g., 'yolo', 'tatr'). Uses manager's default if None.	None
options	Optional[Any]	Specific LayoutOptions object for advanced configuration.	None
confidence	Optional[float]	Minimum confidence threshold.	None
classes	Optional[List[str]]	Specific classes to detect.	None
exclude_classes	Optional[List[str]]	Classes to exclude.	None
device	Optional[str]	Device for inference.	None
existing	str	How to handle existing detected regions: 'replace' (default) or 'append'.	'replace'
model_name	Optional[str]	Optional model name for the engine.	None
client	Optional[Any]	Optional client for API-based engines.	None

Returns:

Type	Description
ElementCollection	ElementCollection containing all detected Region objects from all segments.

Example

Multi-page layout analysis:

pdf = npdf.PDF("document.pdf")

# Create flow for first 3 pages
page_flow = Flow(
    segments=pdf.pages[:3],
    arrangement='vertical'
)

# Analyze layout across all pages (efficiently)
all_regions = page_flow.analyze_layout(engine='yolo')

# Find all tables across the flow
tables = all_regions.filter('region[type=table]')

Source code in natural_pdf/flows/flow.py

def analyze_layout(
    self,
    engine: Optional[str] = None,
    options: Optional[Any] = None,
    confidence: Optional[float] = None,
    classes: Optional[List[str]] = None,
    exclude_classes: Optional[List[str]] = None,
    device: Optional[str] = None,
    existing: str = "replace",
    model_name: Optional[str] = None,
    client: Optional[Any] = None,
) -> "PhysicalElementCollection":
    """
    Analyze layout across all segments in the flow.

    This method efficiently groups segments by their parent pages, runs layout analysis
    only once per unique page, then filters results appropriately for each segment.
    This avoids redundant analysis when multiple flow segments come from the same page.

    Args:
        engine: Name of the layout engine (e.g., 'yolo', 'tatr'). Uses manager's default if None.
        options: Specific LayoutOptions object for advanced configuration.
        confidence: Minimum confidence threshold.
        classes: Specific classes to detect.
        exclude_classes: Classes to exclude.
        device: Device for inference.
        existing: How to handle existing detected regions: 'replace' (default) or 'append'.
        model_name: Optional model name for the engine.
        client: Optional client for API-based engines.

    Returns:
        ElementCollection containing all detected Region objects from all segments.

    Example:
        Multi-page layout analysis:
        ```python
        pdf = npdf.PDF("document.pdf")

        # Create flow for first 3 pages
        page_flow = Flow(
            segments=pdf.pages[:3],
            arrangement='vertical'
        )

        # Analyze layout across all pages (efficiently)
        all_regions = page_flow.analyze_layout(engine='yolo')

        # Find all tables across the flow
        tables = all_regions.filter('region[type=table]')
        ```
    """
    from natural_pdf.elements.element_collection import ElementCollection

    logger.info(
        f"Analyzing layout across Flow with {len(self.segments)} segments (engine: {engine or 'default'})"
    )

    if not self.segments:
        raise ValueError("Flow has no segments; cannot analyze layout")

    # Step 1: Group segments by their parent pages to avoid redundant analysis
    segments_by_page = {}  # Dict[Page, List[Segment]]

    for i, segment in enumerate(self.segments):
        if hasattr(segment, "analyze_layout"):
            page_obj = segment
            segment_type = "page"
        elif hasattr(segment, "page") and hasattr(segment.page, "analyze_layout"):
            page_obj = segment.page
            segment_type = "region"
        else:
            raise TypeError(f"Segment {i+1} does not support layout analysis: {segment!r}")

        if page_obj not in segments_by_page:
            segments_by_page[page_obj] = []
        segments_by_page[page_obj].append((segment, segment_type))

    if not segments_by_page:
        raise ValueError("No segments with analyzable pages found")

    logger.debug(
        f"  Grouped {len(self.segments)} segments into {len(segments_by_page)} unique pages"
    )

    # Step 2: Analyze each unique page only once
    all_detected_regions: List["PhysicalRegion"] = []
    processed_pages = 0

    for page_obj, page_segments in segments_by_page.items():
        try:
            logger.debug(
                f"  Analyzing layout for page {getattr(page_obj, 'number', '?')} with {len(page_segments)} segments"
            )

            # Run layout analysis once for this page
            page_results = page_obj.analyze_layout(
                engine=engine,
                options=options,
                confidence=confidence,
                classes=classes,
                exclude_classes=exclude_classes,
                device=device,
                existing=existing,
                model_name=model_name,
                client=client,
            )

            # Extract regions from results
            if hasattr(page_results, "elements"):
                # It's an ElementCollection
                page_regions = page_results.elements
            elif isinstance(page_results, list):
                # It's a list of regions
                page_regions = page_results
            else:
                raise TypeError(
                    f"Page {getattr(page_obj, 'number', '?')} returned unexpected layout analysis result type: {type(page_results)}"
                )

            if not page_regions:
                logger.debug(
                    f"    No layout regions found on page {getattr(page_obj, 'number', '?')}"
                )
                continue

            # Step 3: For each segment on this page, collect relevant regions
            segments_processed_on_page = 0
            for segment, segment_type in page_segments:
                if segment_type == "page":
                    # Full page segment: include all detected regions
                    all_detected_regions.extend(page_regions)
                    segments_processed_on_page += 1
                    logger.debug(f"    Added {len(page_regions)} regions for full-page segment")

                elif segment_type == "region":
                    # Region segment: filter to only intersecting regions
                    intersecting_regions = []
                    for region in page_regions:
                        try:
                            if segment.intersects(region):
                                intersecting_regions.append(region)
                        except Exception as intersect_error:
                            logger.debug(
                                f"Error checking intersection for region: {intersect_error}"
                            )
                            # Include the region anyway if intersection check fails
                            intersecting_regions.append(region)

                    all_detected_regions.extend(intersecting_regions)
                    segments_processed_on_page += 1
                    logger.debug(
                        f"    Added {len(intersecting_regions)} intersecting regions for region segment {segment.bbox}"
                    )

            processed_pages += 1
            logger.debug(
                f"    Processed {segments_processed_on_page} segments on page {getattr(page_obj, 'number', '?')}"
            )

        except Exception as e:
            logger.error(
                f"Error analyzing layout for page {getattr(page_obj, 'number', '?')}: {e}",
                exc_info=True,
            )
            continue

    # Step 4: Remove duplicates (can happen if multiple segments intersect the same region)
    unique_regions = []
    seen_region_ids = set()

    for region in all_detected_regions:
        # Create a unique identifier for this region (page + bbox)
        region_id = (
            getattr(region.page, "index", id(region.page)),
            region.bbox if hasattr(region, "bbox") else id(region),
        )

        if region_id not in seen_region_ids:
            unique_regions.append(region)
            seen_region_ids.add(region_id)

    dedupe_removed = len(all_detected_regions) - len(unique_regions)
    if dedupe_removed > 0:
        logger.debug(f"  Removed {dedupe_removed} duplicate regions")

    logger.info(
        f"Flow layout analysis complete: {len(unique_regions)} unique regions from {processed_pages} pages"
    )
    return ElementCollection(unique_regions)

natural_pdf.Flow.apply_ocr(*args, **kwargs)

Apply OCR across every segment in the flow.

Source code in natural_pdf/flows/flow.py

def apply_ocr(self, *args: Any, **kwargs: Any) -> "Flow":
    """
    Apply OCR across every segment in the flow.
    """

    self._analysis_region().apply_ocr(*args, **kwargs)
    return self

natural_pdf.Flow.ask(question, min_confidence=0.1, model=None, debug=False, **kwargs)

Run document QA across the flow by delegating to a FlowRegion.

Source code in natural_pdf/flows/flow.py

def ask(
    self,
    question: QuestionInput,
    min_confidence: float = 0.1,
    model: Optional[str] = None,
    debug: bool = False,
    **kwargs: Any,
) -> Any:
    """
    Run document QA across the flow by delegating to a FlowRegion.
    """

    return self._analysis_region().ask(
        question,
        min_confidence=min_confidence,
        model=model,
        debug=debug,
        **kwargs,
    )

natural_pdf.Flow.clear_text_layer()

Clear the underlying text layers (words/chars) for every segment page.

Source code in natural_pdf/flows/flow.py

def clear_text_layer(self) -> Tuple[int, int]:
    """
    Clear the underlying text layers (words/chars) for every segment page.
    """

    return self._analysis_region().clear_text_layer()

natural_pdf.Flow.create_text_elements_from_ocr(ocr_results, scale_x=None, scale_y=None)

Utility for constructing text elements from OCR output.

Source code in natural_pdf/flows/flow.py

def create_text_elements_from_ocr(
    self,
    ocr_results: Any,
    scale_x: Optional[float] = None,
    scale_y: Optional[float] = None,
) -> List[Any]:
    """
    Utility for constructing text elements from OCR output.
    """

    return self._analysis_region().create_text_elements_from_ocr(
        ocr_results,
        scale_x=scale_x,
        scale_y=scale_y,
    )

natural_pdf.Flow.extract_ocr_elements(*args, **kwargs)

Extract OCR-derived text elements from all segments.

Source code in natural_pdf/flows/flow.py

def extract_ocr_elements(self, *args: Any, **kwargs: Any) -> List[Any]:
    """
    Extract OCR-derived text elements from all segments.
    """

    return self._analysis_region().extract_ocr_elements(*args, **kwargs)

natural_pdf.Flow.extract_table(method=None, table_settings=None, use_ocr=False, ocr_config=None, text_options=None, cell_extraction_func=None, show_progress=False, content_filter=None, stitch_rows=None, merge_headers=None, structure_engine=None)

extract_table(method: Optional[str] = None, table_settings: Optional[dict] = None, use_ocr: bool = False, ocr_config: Optional[dict] = None, text_options: Optional[dict] = None, cell_extraction_func: Optional[Any] = None, show_progress: bool = False, content_filter: Optional[Any] = None, stitch_rows: Optional[Callable[[List[Optional[str]]], bool]] = None, merge_headers: Optional[bool] = None, structure_engine: Optional[str] = None) -> TableResult

extract_table(method: Optional[str] = None, table_settings: Optional[dict] = None, use_ocr: bool = False, ocr_config: Optional[dict] = None, text_options: Optional[dict] = None, cell_extraction_func: Optional[Any] = None, show_progress: bool = False, content_filter: Optional[Any] = None, stitch_rows: Optional[Callable[[List[Optional[str]], List[Optional[str]], int, Union[Page, PhysicalRegion]], bool]] = None, merge_headers: Optional[bool] = None, structure_engine: Optional[str] = None) -> TableResult

Extract table data from all segments in the flow, combining results sequentially.

This method extracts table data from each segment in flow order and combines the results into a single logical table. This is particularly useful for multi-page tables or tables that span across columns.

Parameters:

Name	Type	Description	Default
method	Optional[str]	Method to use: 'tatr', 'pdfplumber', 'text', 'stream', 'lattice', or None (auto-detect).	None
table_settings	Optional[dict]	Settings for pdfplumber table extraction.	None
use_ocr	bool	Whether to use OCR for text extraction (currently only applicable with 'tatr' method).	False
ocr_config	Optional[dict]	OCR configuration parameters.	None
text_options	Optional[dict]	Dictionary of options for the 'text' method.	None
cell_extraction_func	Optional[Any]	Optional callable function that takes a cell Region object and returns its string content. For 'text' method only.	None
show_progress	bool	If True, display a progress bar during cell text extraction for the 'text' method.	False
content_filter	Optional[Any]	Optional content filter to apply during cell text extraction.	None
merge_headers	Optional[bool]	Whether to merge tables by removing repeated headers from subsequent segments. If None (default), auto-detects by checking if the first row of each segment matches the first row of the first segment. If segments have inconsistent header patterns (some repeat, others don't), raises ValueError. Useful for multi-page tables where headers repeat on each page.	None
stitch_rows	Optional[Callable[..., bool]]	Optional callable to determine when rows should be merged across segment boundaries. Applied AFTER header removal if merge_headers is enabled. Two overloaded signatures are supported: • func(current_row) -> bool Called only on the first row of each segment (after the first). Return True to merge this first row with the last row from the previous segment. • func(prev_row, current_row, row_index, segment) -> bool Called for every row. Return True to merge current_row with the previous row in the aggregated results. When True is returned, rows are concatenated cell-by-cell. This is useful for handling table rows split across page boundaries or segments. If None, rows are never merged.	None
structure_engine	Optional[str]	Optional structure detection engine forwarded to each segment's extract_table implementation before falling back to traditional engines.	None

Returns:

Type	Description
TableResult	TableResult object containing the aggregated table data from all segments.

Example

Multi-page table extraction:

pdf = npdf.PDF("multi_page_table.pdf")

# Create flow for table spanning pages 2-4
table_flow = Flow(
    segments=[pdf.pages[1], pdf.pages[2], pdf.pages[3]],
    arrangement='vertical'
)

# Extract table as if it were continuous
table_data = table_flow.extract_table()
df = table_data.df  # Convert to pandas DataFrame

# Custom row stitching - single parameter (simple case)
table_data = table_flow.extract_table(
    stitch_rows=lambda row: row and not (row[0] or "").strip()
)

# Custom row stitching - full parameters (advanced case)
table_data = table_flow.extract_table(
    stitch_rows=lambda prev, curr, idx, seg: idx == 0 and curr and not (curr[0] or "").strip()
)

Source code in natural_pdf/flows/flow.py

def extract_table(
    self,
    method: Optional[str] = None,
    table_settings: Optional[dict] = None,
    use_ocr: bool = False,
    ocr_config: Optional[dict] = None,
    text_options: Optional[dict] = None,
    cell_extraction_func: Optional[Any] = None,
    show_progress: bool = False,
    content_filter: Optional[Any] = None,
    stitch_rows: Optional[Callable[..., bool]] = None,
    merge_headers: Optional[bool] = None,
    structure_engine: Optional[str] = None,
) -> TableResult:
    """
    Extract table data from all segments in the flow, combining results sequentially.

    This method extracts table data from each segment in flow order and combines
    the results into a single logical table. This is particularly useful for
    multi-page tables or tables that span across columns.

    Args:
        method: Method to use: 'tatr', 'pdfplumber', 'text', 'stream', 'lattice', or None (auto-detect).
        table_settings: Settings for pdfplumber table extraction.
        use_ocr: Whether to use OCR for text extraction (currently only applicable with 'tatr' method).
        ocr_config: OCR configuration parameters.
        text_options: Dictionary of options for the 'text' method.
        cell_extraction_func: Optional callable function that takes a cell Region object
                              and returns its string content. For 'text' method only.
        show_progress: If True, display a progress bar during cell text extraction for the 'text' method.
        content_filter: Optional content filter to apply during cell text extraction.
        merge_headers: Whether to merge tables by removing repeated headers from subsequent
            segments. If None (default), auto-detects by checking if the first row
            of each segment matches the first row of the first segment. If segments have
            inconsistent header patterns (some repeat, others don't), raises ValueError.
            Useful for multi-page tables where headers repeat on each page.
        stitch_rows: Optional callable to determine when rows should be merged across
                     segment boundaries. Applied AFTER header removal if merge_headers
                     is enabled. Two overloaded signatures are supported:

                     • func(current_row) -> bool
                       Called only on the first row of each segment (after the first).
                       Return True to merge this first row with the last row from
                       the previous segment.

                     • func(prev_row, current_row, row_index, segment) -> bool
                       Called for every row. Return True to merge current_row with
                       the previous row in the aggregated results.

                     When True is returned, rows are concatenated cell-by-cell.
                     This is useful for handling table rows split across page
                     boundaries or segments. If None, rows are never merged.
        structure_engine: Optional structure detection engine forwarded to each segment's
            ``extract_table`` implementation before falling back to traditional engines.

    Returns:
        TableResult object containing the aggregated table data from all segments.

    Example:
        Multi-page table extraction:
        ```python
        pdf = npdf.PDF("multi_page_table.pdf")

        # Create flow for table spanning pages 2-4
        table_flow = Flow(
            segments=[pdf.pages[1], pdf.pages[2], pdf.pages[3]],
            arrangement='vertical'
        )

        # Extract table as if it were continuous
        table_data = table_flow.extract_table()
        df = table_data.df  # Convert to pandas DataFrame

        # Custom row stitching - single parameter (simple case)
        table_data = table_flow.extract_table(
            stitch_rows=lambda row: row and not (row[0] or "").strip()
        )

        # Custom row stitching - full parameters (advanced case)
        table_data = table_flow.extract_table(
            stitch_rows=lambda prev, curr, idx, seg: idx == 0 and curr and not (curr[0] or "").strip()
        )
        ```
    """
    logger.info(
        f"Extracting table from Flow with {len(self.segments)} segments (method: {method or 'auto'})"
    )

    if not self.segments:
        raise ValueError("Flow has no segments; cannot extract table")

    # Resolve predicate and determine its signature
    predicate: Optional[Callable] = None
    predicate_type: str = "none"

    if callable(stitch_rows):
        import inspect

        sig = inspect.signature(stitch_rows)
        param_count = len(sig.parameters)

        if param_count == 1:
            predicate = stitch_rows
            predicate_type = "single_param"
        elif param_count == 4:
            predicate = stitch_rows
            predicate_type = "full_params"
        else:
            raise TypeError(
                f"stitch_rows function must accept 1 or 4 parameters, got {param_count}"
            )

    def _default_merge(
        prev_row: List[Optional[str]], cur_row: List[Optional[str]]
    ) -> List[Optional[str]]:
        from itertools import zip_longest

        merged: List[Optional[str]] = []
        for p, c in zip_longest(prev_row, cur_row, fillvalue=""):
            if (p or "").strip() and (c or "").strip():
                merged.append(f"{p} {c}".strip())
            else:
                merged.append((p or "") + (c or ""))
        return merged

    aggregated_rows: List[List[Optional[str]]] = []
    processed_segments = 0
    header_row: Optional[List[Optional[str]]] = None
    merge_headers_enabled = False
    headers_warned = False  # Track if we've already warned about dropping headers
    segment_has_repeated_header = []  # Track which segments have repeated headers

    for seg_idx, segment in enumerate(self.segments):
        try:
            logger.debug(f"  Extracting table from segment {seg_idx+1}/{len(self.segments)}")

            segment_result = segment.extract_table(
                method=method,
                table_settings=table_settings.copy() if table_settings else None,
                use_ocr=use_ocr,
                ocr_config=ocr_config,
                text_options=text_options.copy() if text_options else None,
                cell_extraction_func=cell_extraction_func,
                show_progress=show_progress,
                content_filter=content_filter,
                structure_engine=structure_engine,
            )

            if not segment_result:
                continue

            if hasattr(segment_result, "_rows"):
                segment_rows = list(segment_result._rows)
            else:
                segment_rows = list(segment_result)

            if not segment_rows:
                logger.debug(f"    No table data found in segment {seg_idx+1}")
                continue

            # Handle header detection and merging for multi-page tables
            if seg_idx == 0:
                # First segment: capture potential header row
                if segment_rows:
                    header_row = segment_rows[0]
                    # Determine if we should merge headers
                    if merge_headers is None:
                        # Auto-detect: we'll check all subsequent segments
                        merge_headers_enabled = False  # Will be determined later
                    else:
                        merge_headers_enabled = merge_headers
                    # Track that first segment exists (for consistency checking)
                    segment_has_repeated_header.append(False)  # First segment doesn't "repeat"
            elif seg_idx == 1 and merge_headers is None:
                # Auto-detection: check if first row of second segment matches header
                has_header = segment_rows and header_row and segment_rows[0] == header_row
                segment_has_repeated_header.append(has_header)

                if has_header:
                    merge_headers_enabled = True
                    # Remove the detected repeated header from this segment
                    segment_rows = segment_rows[1:]
                    logger.debug(
                        f"    Auto-detected repeated header in segment {seg_idx+1}, removed"
                    )
                    if not headers_warned:
                        warnings.warn(
                            "Detected repeated headers in multi-page table. Merging by removing "
                            "repeated headers from subsequent pages.",
                            UserWarning,
                            stacklevel=2,
                        )
                        headers_warned = True
                else:
                    merge_headers_enabled = False
                    logger.debug(f"    No repeated header detected in segment {seg_idx+1}")
            elif seg_idx > 1:
                # Check consistency: all segments should have same pattern
                has_header = segment_rows and header_row and segment_rows[0] == header_row
                segment_has_repeated_header.append(has_header)

                # Remove header if merging is enabled and header is present
                if merge_headers_enabled and has_header:
                    segment_rows = segment_rows[1:]
                    logger.debug(f"    Removed repeated header from segment {seg_idx+1}")
            elif seg_idx > 0 and merge_headers_enabled:
                # Explicit merge_headers=True: remove headers from subsequent segments
                if segment_rows and header_row and segment_rows[0] == header_row:
                    segment_rows = segment_rows[1:]
                    logger.debug(f"    Removed repeated header from segment {seg_idx+1}")
                    if not headers_warned:
                        warnings.warn(
                            "Removing repeated headers from multi-page table during merge.",
                            UserWarning,
                            stacklevel=2,
                        )
                        headers_warned = True

            for row_idx, row in enumerate(segment_rows):
                should_merge = False

                if predicate is not None and aggregated_rows:
                    if predicate_type == "single_param":
                        # For single param: only call on first row of segment (row_idx == 0)
                        # and pass the current row
                        if row_idx == 0:
                            should_merge = predicate(row)
                    elif predicate_type == "full_params":
                        # For full params: call with all arguments
                        should_merge = predicate(aggregated_rows[-1], row, row_idx, segment)

                if should_merge:
                    aggregated_rows[-1] = _default_merge(aggregated_rows[-1], row)
                else:
                    aggregated_rows.append(row)

            processed_segments += 1
            logger.debug(
                f"    Added {len(segment_rows)} rows (post-merge) from segment {seg_idx+1}"
            )

        except Exception as e:
            logger.error(f"Error extracting table from segment {seg_idx+1}: {e}", exc_info=True)
            continue

    # Check for inconsistent header patterns after processing all segments
    if merge_headers is None and len(segment_has_repeated_header) > 2:
        # During auto-detection, check for consistency across all segments
        expected_pattern = segment_has_repeated_header[1]  # Pattern from second segment
        for seg_idx, has_header in enumerate(segment_has_repeated_header[2:], 2):
            if has_header != expected_pattern:
                # Inconsistent pattern detected
                segments_with_headers = [
                    i for i, has_h in enumerate(segment_has_repeated_header[1:], 1) if has_h
                ]
                segments_without_headers = [
                    i for i, has_h in enumerate(segment_has_repeated_header[1:], 1) if not has_h
                ]
                raise ValueError(
                    f"Inconsistent header pattern in multi-page table: "
                    f"segments {segments_with_headers} have repeated headers, "
                    f"but segments {segments_without_headers} do not. "
                    f"All segments must have the same header pattern for reliable merging."
                )

    logger.info(
        f"Flow table extraction complete: {len(aggregated_rows)} total rows from {processed_segments}/{len(self.segments)} segments"
    )
    return TableResult(aggregated_rows)

natural_pdf.Flow.extract_tables(method=None, table_settings=None, **kwargs)

Extract every table across all segments in reading order.

Parameters:

Name	Type	Description	Default
method	Optional[str]	Optional table engine name. Mirrors :meth:Flow.extract_table.	None
table_settings	Optional[dict]	Base pdfplumber settings (copied per segment).	None
**kwargs		Additional keyword arguments forwarded to each segment's :meth:extract_tables implementation (e.g., content_filter).	{}

Returns:

Type	Description
List[List[List[Optional[str]]]]	List of tables, where each table is represented as a list of rows.
List[List[List[Optional[str]]]]	The order matches the order of self.segments.

Source code in natural_pdf/flows/flow.py

def extract_tables(
    self,
    method: Optional[str] = None,
    table_settings: Optional[dict] = None,
    **kwargs,
) -> List[List[List[Optional[str]]]]:
    """Extract every table across all segments in reading order.

    Args:
        method: Optional table engine name. Mirrors :meth:`Flow.extract_table`.
        table_settings: Base pdfplumber settings (copied per segment).
        **kwargs: Additional keyword arguments forwarded to each segment's
            :meth:`extract_tables` implementation (e.g., ``content_filter``).

    Returns:
        List of tables, where each table is represented as a list of rows.
        The order matches the order of ``self.segments``.
    """

    if not self.segments:
        return []

    base_settings = table_settings.copy() if table_settings else None
    all_tables: List[List[List[Optional[str]]]] = []

    for segment in self.segments:
        segment_settings = base_settings.copy() if base_settings else None
        tables = cast(
            List[List[List[Optional[str]]]],
            segment.extract_tables(
                method=method,
                table_settings=segment_settings,
                **kwargs,
            ),
        )
        if tables:
            all_tables.extend(tables)

    return all_tables

natural_pdf.Flow.find(selector=None, *, text=None, apply_exclusions=True, regex=False, case=True, text_tolerance=None, auto_text_tolerance=None, reading_order=True, engine=None)

Finds the first element within the flow that matches the given selector or text criteria.

Elements found are wrapped as FlowElement objects, anchored to this Flow.

Parameters:

Name	Type	Description	Default
selector	Optional[str]	CSS-like selector string.	None
text	Optional[Union[str, Sequence[str]]]	Optional text shortcut (equivalent to text:contains(...)).	None
apply_exclusions	bool	Whether to respect exclusion zones on the original pages/regions.	True
regex	bool	Whether the text search uses regex.	False
case	bool	Whether the text search is case-sensitive.	True
text_tolerance	Optional[Dict[str, Any]]	Optional dict of text tolerance overrides.	None
auto_text_tolerance	Optional[Dict[str, Any]]	Optional overrides controlling automatic tolerance logic.	None
reading_order	bool	Whether to sort matches in reading order when applicable (default: True).	True
engine	Optional[str]	Optional selector engine name registered via the selector provider.	None

Returns:

Type	Description
Optional[FlowElement]	A FlowElement if a match is found, otherwise None.

Source code in natural_pdf/flows/flow.py

def find(
    self,
    selector: Optional[str] = None,
    *,
    text: Optional[Union[str, Sequence[str]]] = None,
    apply_exclusions: bool = True,
    regex: bool = False,
    case: bool = True,
    text_tolerance: Optional[Dict[str, Any]] = None,
    auto_text_tolerance: Optional[Dict[str, Any]] = None,
    reading_order: bool = True,
    engine: Optional[str] = None,
) -> Optional["FlowElement"]:
    """
    Finds the first element within the flow that matches the given selector or text criteria.

    Elements found are wrapped as FlowElement objects, anchored to this Flow.

    Args:
        selector: CSS-like selector string.
        text: Optional text shortcut (equivalent to ``text:contains(...)``).
        apply_exclusions: Whether to respect exclusion zones on the original pages/regions.
        regex: Whether the text search uses regex.
        case: Whether the text search is case-sensitive.
        text_tolerance: Optional dict of text tolerance overrides.
        auto_text_tolerance: Optional overrides controlling automatic tolerance logic.
        reading_order: Whether to sort matches in reading order when applicable (default: True).
        engine: Optional selector engine name registered via the selector provider.

    Returns:
        A FlowElement if a match is found, otherwise None.
    """
    results = self.find_all(
        selector=selector,
        text=text,
        apply_exclusions=apply_exclusions,
        regex=regex,
        case=case,
        text_tolerance=text_tolerance,
        auto_text_tolerance=auto_text_tolerance,
        reading_order=reading_order,
        engine=engine,
    )
    return results.first if results else None

natural_pdf.Flow.find_all(selector=None, *, text=None, apply_exclusions=True, regex=False, case=True, text_tolerance=None, auto_text_tolerance=None, reading_order=True, engine=None)

Finds all elements within the flow that match the given selector or text criteria.

This method efficiently groups segments by their parent pages, searches at the page level, then filters results appropriately for each segment. This ensures elements that intersect with flow segments (but aren't fully contained) are still found.

Elements found are wrapped as FlowElement objects, anchored to this Flow, and returned in a FlowElementCollection.

Parameters:

Name	Type	Description	Default
engine	Optional[str]	Optional selector engine name forwarded to page-level queries.	None

Source code in natural_pdf/flows/flow.py

def find_all(
    self,
    selector: Optional[str] = None,
    *,
    text: Optional[Union[str, Sequence[str]]] = None,
    apply_exclusions: bool = True,
    regex: bool = False,
    case: bool = True,
    text_tolerance: Optional[Dict[str, Any]] = None,
    auto_text_tolerance: Optional[Dict[str, Any]] = None,
    reading_order: bool = True,
    engine: Optional[str] = None,
) -> "FlowElementCollection":
    """
    Finds all elements within the flow that match the given selector or text criteria.

    This method efficiently groups segments by their parent pages, searches at the page level,
    then filters results appropriately for each segment. This ensures elements that intersect
    with flow segments (but aren't fully contained) are still found.

    Elements found are wrapped as FlowElement objects, anchored to this Flow,
    and returned in a FlowElementCollection.

    Args:
        engine: Optional selector engine name forwarded to page-level queries.
    """
    from .collections import FlowElementCollection
    from .element import FlowElement

    # Step 1: Group segments by their parent pages (like in analyze_layout)
    segments_by_page = {}  # Dict[Page, List[Segment]]

    for i, segment in enumerate(self.segments):
        if hasattr(segment, "page") and hasattr(segment.page, "find_all"):
            page_obj = segment.page
            segment_type = "region"
        elif (
            hasattr(segment, "find_all")
            and hasattr(segment, "width")
            and hasattr(segment, "height")
            and not hasattr(segment, "page")
        ):
            page_obj = segment
            segment_type = "page"
        else:
            raise TypeError(f"Segment {i+1} does not support find_all: {segment!r}")

        if page_obj not in segments_by_page:
            segments_by_page[page_obj] = []
        segments_by_page[page_obj].append((segment, segment_type))

    if not segments_by_page:
        raise ValueError("No segments with searchable pages found")

    # Step 2: Search each unique page only once
    all_flow_elements: List["FlowElement"] = []

    for page_obj, page_segments in segments_by_page.items():
        # Find all matching elements on this page
        page_matches = page_obj.find_all(
            selector=selector,
            text=text,
            apply_exclusions=apply_exclusions,
            regex=regex,
            case=case,
            text_tolerance=text_tolerance,
            auto_text_tolerance=auto_text_tolerance,
            reading_order=reading_order,
            engine=engine,
        )

        if not page_matches:
            continue

        # Step 3: For each segment on this page, collect relevant elements
        for segment, segment_type in page_segments:
            if segment_type == "page":
                # Full page segment: include all elements
                for phys_elem in page_matches.elements:
                    all_flow_elements.append(FlowElement(physical_object=phys_elem, flow=self))

            elif segment_type == "region":
                # Region segment: filter to only intersecting elements
                for phys_elem in page_matches.elements:
                    try:
                        # Check if element intersects with this flow segment
                        if segment.intersects(phys_elem):
                            all_flow_elements.append(
                                FlowElement(physical_object=phys_elem, flow=self)
                            )
                    except Exception as intersect_error:
                        logger.debug(
                            f"Error checking intersection for element: {intersect_error}"
                        )
                        # Include the element anyway if intersection check fails
                        all_flow_elements.append(
                            FlowElement(physical_object=phys_elem, flow=self)
                        )

    # Step 4: Remove duplicates (can happen if multiple segments intersect the same element)
    unique_flow_elements = []
    seen_element_ids = set()

    for flow_elem in all_flow_elements:
        # Create a unique identifier for the underlying physical element
        phys_elem = flow_elem.physical_object
        elem_id = (
            (
                getattr(phys_elem.page, "index", id(phys_elem.page))
                if hasattr(phys_elem, "page")
                else id(phys_elem)
            ),
            phys_elem.bbox if hasattr(phys_elem, "bbox") else id(phys_elem),
        )

        if elem_id not in seen_element_ids:
            unique_flow_elements.append(flow_elem)
            seen_element_ids.add(elem_id)

    return FlowElementCollection(unique_flow_elements)

natural_pdf.Flow.get_element_flow_coordinates(physical_element)

Translates a physical element's coordinates into the flow's virtual coordinate system. (Placeholder - very complex if segment_gap > 0 or complex alignments)

Source code in natural_pdf/flows/flow.py

def get_element_flow_coordinates(
    self, physical_element: "PhysicalElement"
) -> Optional[tuple[float, float, float, float]]:
    """
    Translates a physical element's coordinates into the flow's virtual coordinate system.
    (Placeholder - very complex if segment_gap > 0 or complex alignments)
    """
    # For now, elements operate in their own physical coordinates. This method would be needed
    # if FlowRegion.bbox or other operations needed to present a unified coordinate space.
    # As per our discussion, elements *within* a FlowRegion retain original physical coordinates.
    # So, this might not be strictly necessary for the current design's core functionality.
    raise NotImplementedError(
        "Translating element coordinates to a unified flow coordinate system is not yet implemented."
    )

natural_pdf.Flow.get_sections(start_elements=None, end_elements=None, new_section_on_page_break=False, include_boundaries='both', orientation='vertical')

Extract logical sections from the Flow based on start and end boundary elements, mirroring the behaviour of PDF/PageCollection.get_sections().

This implementation is a thin wrapper that converts the Flow into a temporary PageCollection (constructed from the unique pages that the Flow spans) and then delegates the heavy‐lifting to that existing implementation. Any FlowElement / FlowElementCollection inputs are automatically unwrapped to their underlying physical elements so that PageCollection can work with them directly.

Parameters:

Name	Type	Description	Default
start_elements		Elements or selector string that mark the start of sections (optional).	None
end_elements		Elements or selector string that mark the end of sections (optional).	None
new_section_on_page_break	bool	Whether to start a new section at page boundaries (default: False).	False
include_boundaries	str	How to include boundary elements: 'start', 'end', 'both', or 'none' (default: 'both').	'both'
orientation	str	'vertical' (default) or 'horizontal' - determines section direction.	'vertical'

Returns:

Type	Description
ElementCollection	ElementCollection of Region/FlowRegion objects representing the
ElementCollection	extracted sections.

Source code in natural_pdf/flows/flow.py

def get_sections(
    self,
    start_elements=None,
    end_elements=None,
    new_section_on_page_break: bool = False,
    include_boundaries: str = "both",
    orientation: str = "vertical",
) -> "PhysicalElementCollection":
    """
    Extract logical sections from the Flow based on *start* and *end* boundary
    elements, mirroring the behaviour of PDF/PageCollection.get_sections().

    This implementation is a thin wrapper that converts the Flow into a
    temporary PageCollection (constructed from the unique pages that the
    Flow spans) and then delegates the heavy‐lifting to that existing
    implementation.  Any FlowElement / FlowElementCollection inputs are
    automatically unwrapped to their underlying physical elements so that
    PageCollection can work with them directly.

    Args:
        start_elements: Elements or selector string that mark the start of
            sections (optional).
        end_elements: Elements or selector string that mark the end of
            sections (optional).
        new_section_on_page_break: Whether to start a new section at page
            boundaries (default: False).
        include_boundaries: How to include boundary elements: 'start',
            'end', 'both', or 'none' (default: 'both').
        orientation: 'vertical' (default) or 'horizontal' - determines section direction.

    Returns:
        ElementCollection of Region/FlowRegion objects representing the
        extracted sections.
    """
    # ------------------------------------------------------------------
    # Unwrap FlowElement(-Collection) inputs and selector strings so we
    # can reason about them generically.
    # ------------------------------------------------------------------
    from natural_pdf.flows.collections import FlowElementCollection
    from natural_pdf.flows.element import FlowElement

    def _unwrap(obj):
        """Convert Flow-specific wrappers to their underlying physical objects.

        Keeps selector strings as-is; converts FlowElement to its physical
        element; converts FlowElementCollection to list of physical
        elements; passes through ElementCollection by taking .elements.
        """

        if obj is None or isinstance(obj, str):
            return obj

        if isinstance(obj, FlowElement):
            return obj.physical_object

        if isinstance(obj, FlowElementCollection):
            return [fe.physical_object for fe in obj.flow_elements]

        if hasattr(obj, "elements"):
            return obj.elements

        if isinstance(obj, (list, tuple, set)):
            out = []
            for item in obj:
                if isinstance(item, FlowElement):
                    out.append(item.physical_object)
                else:
                    out.append(item)
            return out

        return obj  # Fallback – unknown type

    start_elements_unwrapped = _unwrap(start_elements)
    end_elements_unwrapped = _unwrap(end_elements)

    # ------------------------------------------------------------------
    # For Flow, we need to handle sections that may span segments
    # We'll process all segments together, not independently
    # ------------------------------------------------------------------
    from natural_pdf.elements.element_collection import ElementCollection
    from natural_pdf.elements.region import Region
    from natural_pdf.flows.region import FlowRegion

    # Helper to check if element is in segment
    def _element_in_segment(elem, segment):
        # Simple bbox check
        return (
            elem.page == segment.page
            and elem.top >= segment.top
            and elem.bottom <= segment.bottom
            and elem.x0 >= segment.x0
            and elem.x1 <= segment.x1
        )

    # Collect all boundary elements with their segment info
    all_starts = []
    all_ends = []

    for seg_idx, segment in enumerate(self.segments):
        # Find starts in this segment
        if isinstance(start_elements_unwrapped, str):
            seg_starts = segment.find_all(start_elements_unwrapped).elements
        elif start_elements_unwrapped:
            if isinstance(start_elements_unwrapped, Iterable):
                candidates = list(start_elements_unwrapped)
            else:
                candidates = [start_elements_unwrapped]
            seg_starts = [e for e in candidates if _element_in_segment(e, segment)]
        else:
            seg_starts = []

        for elem in seg_starts:
            all_starts.append((elem, seg_idx, segment))

        # Find ends in this segment
        if isinstance(end_elements_unwrapped, str):
            seg_ends = segment.find_all(end_elements_unwrapped).elements
        elif end_elements_unwrapped:
            if isinstance(end_elements_unwrapped, Iterable):
                candidates_end = list(end_elements_unwrapped)
            else:
                candidates_end = [end_elements_unwrapped]
            seg_ends = [e for e in candidates_end if _element_in_segment(e, segment)]
        else:
            seg_ends = []

        for elem in seg_ends:
            all_ends.append((elem, seg_idx, segment))

    # Sort by segment index, then position
    all_starts.sort(key=lambda x: (x[1], x[0].top, x[0].x0))
    all_ends.sort(key=lambda x: (x[1], x[0].top, x[0].x0))

    # When only end elements are supplied we synthesise implicit start
    # markers so we can reuse the "start-only" logic below.  This mirrors
    # the historical behaviour of PageCollection.get_sections which
    # created implicit starts at the top of the document and immediately
    # after each end boundary.
    if not all_starts and all_ends:

        def _create_implicit(segment, position):
            thickness = 0.1
            if orientation == "vertical":
                top = max(segment.top, min(position, segment.bottom))
                bottom = min(segment.bottom, top + thickness)
                if bottom <= top:
                    bottom = min(segment.bottom, top + thickness)
                    if bottom <= top:
                        bottom = top
                implicit_region = Region(segment.page, (segment.x0, top, segment.x1, bottom))
            else:
                left = max(segment.x0, min(position, segment.x1))
                right = min(segment.x1, left + thickness)
                if right <= left:
                    right = min(segment.x1, left + thickness)
                    if right <= left:
                        right = left
                implicit_region = Region(
                    segment.page, (left, segment.top, right, segment.bottom)
                )

            implicit_region.metadata["is_implicit_start"] = True
            return implicit_region

        synthetic_starts: List[Tuple[Region, int, Region]] = []

        first_segment = self.segments[0]
        initial_position = first_segment.top if orientation == "vertical" else first_segment.x0
        synthetic_starts.append(
            (
                _create_implicit(first_segment, initial_position),
                0,
                first_segment,
            )
        )

        seen_end_ids = set()
        for end_elem, end_seg_idx, _ in all_ends:
            if id(end_elem) in seen_end_ids:
                continue
            seen_end_ids.add(id(end_elem))

            position = end_elem.bottom if orientation == "vertical" else end_elem.x1
            if include_boundaries in ["start", "none"]:
                position = end_elem.top if orientation == "vertical" else end_elem.x0

            segment = self.segments[end_seg_idx]
            synthetic_starts.append(
                (
                    _create_implicit(segment, position),
                    end_seg_idx,
                    segment,
                )
            )

        all_starts = synthetic_starts
        all_ends = []

    # If no boundary elements found, return empty collection
    if not all_starts and not all_ends:
        return ElementCollection([])

    sections = []

    # Case 1: Only start elements provided
    if all_starts and not all_ends:
        for i in range(len(all_starts)):
            start_elem, start_seg_idx, start_seg = all_starts[i]

            # Find end (next start or end of flow)
            if i + 1 < len(all_starts):
                # Section ends at next start
                end_elem, end_seg_idx, end_seg = all_starts[i + 1]

                if start_seg_idx == end_seg_idx:
                    # Same segment - create regular Region
                    section = start_seg.get_section_between(
                        start_elem, end_elem, include_boundaries, orientation
                    )
                    if section:
                        sections.append(section)
                else:
                    # Cross-segment - create FlowRegion
                    regions = []

                    # First segment: from start to bottom
                    if include_boundaries in ["both", "start"]:
                        top = start_elem.top
                    else:
                        top = start_elem.bottom
                    regions.append(
                        Region(
                            start_seg.page, (start_seg.x0, top, start_seg.x1, start_seg.bottom)
                        )
                    )

                    # Middle segments (full)
                    for idx in range(start_seg_idx + 1, end_seg_idx):
                        regions.append(self.segments[idx])

                    # Last segment: from top to end element
                    if include_boundaries in ["both", "end"]:
                        bottom = end_elem.bottom
                    else:
                        bottom = end_elem.top
                    regions.append(
                        Region(end_seg.page, (end_seg.x0, end_seg.top, end_seg.x1, bottom))
                    )

                    # Create FlowRegion
                    flow_element = FlowElement(physical_object=start_elem, flow=self)
                    flow_region = FlowRegion(
                        flow=self,
                        constituent_regions=regions,
                        source_flow_element=flow_element,
                        boundary_element_found=end_elem,
                    )
                    flow_region.start_element = start_elem
                    flow_region.end_element = end_elem
                    flow_region._boundary_exclusions = include_boundaries
                    sections.append(flow_region)
            else:
                # Last section - goes to end of flow
                if start_seg_idx == len(self.segments) - 1:
                    # Within last segment
                    section = start_seg.get_section_between(
                        start_elem, None, include_boundaries, orientation
                    )
                    if section:
                        sections.append(section)
                else:
                    # Spans to end
                    regions = []

                    # First segment: from start to bottom
                    if include_boundaries in ["both", "start"]:
                        top = start_elem.top
                    else:
                        top = start_elem.bottom
                    regions.append(
                        Region(
                            start_seg.page, (start_seg.x0, top, start_seg.x1, start_seg.bottom)
                        )
                    )

                    # Remaining segments (full)
                    for idx in range(start_seg_idx + 1, len(self.segments)):
                        regions.append(self.segments[idx])

                    # Create FlowRegion
                    flow_element = FlowElement(physical_object=start_elem, flow=self)
                    flow_region = FlowRegion(
                        flow=self,
                        constituent_regions=regions,
                        source_flow_element=flow_element,
                        boundary_element_found=None,
                    )
                    flow_region.start_element = start_elem
                    flow_region._boundary_exclusions = include_boundaries
                    sections.append(flow_region)

    # Case 2: Both start and end elements
    elif all_starts and all_ends:
        # Match starts with ends
        used_ends = set()

        for start_elem, start_seg_idx, start_seg in all_starts:
            # Find matching end
            best_end = None

            for end_elem, end_seg_idx, end_seg in all_ends:
                if id(end_elem) in used_ends:
                    continue

                # End must come after start
                if end_seg_idx > start_seg_idx or (
                    end_seg_idx == start_seg_idx and end_elem.top >= start_elem.bottom
                ):
                    best_end = (end_elem, end_seg_idx, end_seg)
                    break

            if best_end:
                end_elem, end_seg_idx, end_seg = best_end
                used_ends.add(id(end_elem))

                if start_seg_idx == end_seg_idx:
                    # Same segment
                    section = start_seg.get_section_between(
                        start_elem, end_elem, include_boundaries, orientation
                    )
                    if section:
                        sections.append(section)
                else:
                    # Cross-segment FlowRegion
                    regions = []

                    # First segment
                    if include_boundaries in ["both", "start"]:
                        top = start_elem.top
                    else:
                        top = start_elem.bottom
                    regions.append(
                        Region(
                            start_seg.page, (start_seg.x0, top, start_seg.x1, start_seg.bottom)
                        )
                    )

                    # Middle segments
                    for idx in range(start_seg_idx + 1, end_seg_idx):
                        regions.append(self.segments[idx])

                    # Last segment
                    if include_boundaries in ["both", "end"]:
                        bottom = end_elem.bottom
                    else:
                        bottom = end_elem.top
                    regions.append(
                        Region(end_seg.page, (end_seg.x0, end_seg.top, end_seg.x1, bottom))
                    )

                    # Create FlowRegion
                    flow_element = FlowElement(physical_object=start_elem, flow=self)
                    flow_region = FlowRegion(
                        flow=self,
                        constituent_regions=regions,
                        source_flow_element=flow_element,
                        boundary_element_found=end_elem,
                    )
                    flow_region.start_element = start_elem
                    flow_region.end_element = end_elem
                    flow_region._boundary_exclusions = include_boundaries
                    sections.append(flow_region)

    # Case 3 is handled by synthesising implicit start elements above.
    elif not all_starts and all_ends:
        pass

    return ElementCollection(sections)

natural_pdf.Flow.get_segment_bounding_box_in_flow(segment_index)

Calculates the conceptual bounding box of a segment within the flow's coordinate system. This considers arrangement, alignment, and segment gaps. (This is a placeholder for more complex logic if a true virtual coordinate system is needed) For now, it might just return the physical segment's bbox if gaps are 0 and alignment is simple.

Source code in natural_pdf/flows/flow.py

def get_segment_bounding_box_in_flow(
    self, segment_index: int
) -> Optional[tuple[float, float, float, float]]:
    """
    Calculates the conceptual bounding box of a segment within the flow's coordinate system.
    This considers arrangement, alignment, and segment gaps.
    (This is a placeholder for more complex logic if a true virtual coordinate system is needed)
    For now, it might just return the physical segment's bbox if gaps are 0 and alignment is simple.
    """
    if segment_index < 0 or segment_index >= len(self.segments):
        return None

    # This is a simplified version. A full implementation would calculate offsets.
    # For now, we assume FlowElement directional logic handles segment traversal and uses physical coords.
    # If we were to *draw* the flow or get a FlowRegion bbox that spans gaps, this would be critical.
    # physical_segment = self.segments[segment_index]
    # return physical_segment.bbox
    raise NotImplementedError(
        "Calculating a segment's bbox *within the flow's virtual coordinate system* is not yet fully implemented."
    )

natural_pdf.Flow.highlights(show=False)

Create a highlight context for accumulating highlights.

This allows for clean syntax to show multiple highlight groups:

Example

with flow.highlights() as h: h.add(flow.find_all('table'), label='tables', color='blue') h.add(flow.find_all('text:bold'), label='bold text', color='red') h.show()

Or with automatic display

with flow.highlights(show=True) as h: h.add(flow.find_all('table'), label='tables') h.add(flow.find_all('text:bold'), label='bold') # Automatically shows when exiting the context

Parameters:

Name	Type	Description	Default
show	bool	If True, automatically show highlights when exiting context	False

Returns:

Type	Description
	HighlightContext for accumulating highlights

Source code in natural_pdf/flows/flow.py

def highlights(self, show: bool = False):
    """
    Create a highlight context for accumulating highlights.

    This allows for clean syntax to show multiple highlight groups:

    Example:
        with flow.highlights() as h:
            h.add(flow.find_all('table'), label='tables', color='blue')
            h.add(flow.find_all('text:bold'), label='bold text', color='red')
            h.show()

    Or with automatic display:
        with flow.highlights(show=True) as h:
            h.add(flow.find_all('table'), label='tables')
            h.add(flow.find_all('text:bold'), label='bold')
            # Automatically shows when exiting the context

    Args:
        show: If True, automatically show highlights when exiting context

    Returns:
        HighlightContext for accumulating highlights
    """
    from natural_pdf.core.highlighting_service import HighlightContext

    return HighlightContext(self, show_on_exit=show)

natural_pdf.Flow.remove_ocr_elements()

Remove OCR elements that were previously added to constituent pages.

Source code in natural_pdf/flows/flow.py

def remove_ocr_elements(self) -> int:
    """
    Remove OCR elements that were previously added to constituent pages.
    """

    return self._analysis_region().remove_ocr_elements()

natural_pdf.Flow.show(*, resolution=None, width=None, color=None, labels=True, label_format=None, highlights=None, legend_position='right', annotate=None, layout=None, stack_direction='vertical', gap=5, columns=6, crop=False, crop_bbox=None, in_context=False, separator_color=None, separator_thickness=2, **kwargs)

Generate a preview image with highlights.

If in_context=True, shows segments as cropped images stacked together with separators between segments.

Parameters:

Name	Type	Description	Default
resolution	Optional[float]	DPI for rendering (default from global settings)	None
width	Optional[int]	Target width in pixels (overrides resolution)	None
color	Optional[Union[str, Tuple[int, int, int]]]	Default highlight color	None
labels	bool	Whether to show labels for highlights	True
label_format	Optional[str]	Format string for labels	None
highlights	Optional[Union[List[Dict[str, Any]], bool]]	Additional highlight groups to show	None
layout	Optional[Literal['stack', 'grid', 'single']]	How to arrange multiple pages/regions	None
stack_direction	Literal['vertical', 'horizontal']	Direction for stack layout	'vertical'
gap	int	Pixels between stacked images	5
columns	Optional[int]	Number of columns for grid layout	6
crop	Union[bool, int, str, Region, Literal['wide']]	Whether to crop	False
crop_bbox	Optional[Tuple[float, float, float, float]]	Explicit crop bounds	None
in_context	bool	If True, use special Flow visualization with separators	False
separator_color	Optional[Tuple[int, int, int]]	RGB color for separator lines (default: red)	None
separator_thickness	int	Thickness of separator lines	2
**kwargs		Additional parameters passed to rendering	{}

Returns:

Type	Description
Optional[Image]	PIL Image object or None if nothing to render

Source code in natural_pdf/flows/flow.py

def show(
    self,
    *,
    resolution: Optional[float] = None,
    width: Optional[int] = None,
    color: Optional[Union[str, Tuple[int, int, int]]] = None,
    labels: bool = True,
    label_format: Optional[str] = None,
    highlights: Optional[Union[List[Dict[str, Any]], bool]] = None,
    legend_position: str = "right",
    annotate: Optional[Union[str, List[str]]] = None,
    layout: Optional[Literal["stack", "grid", "single"]] = None,
    stack_direction: Literal["vertical", "horizontal"] = "vertical",
    gap: int = 5,
    columns: Optional[int] = 6,
    crop: Union[bool, int, str, "PhysicalRegion", Literal["wide"]] = False,
    crop_bbox: Optional[Tuple[float, float, float, float]] = None,
    in_context: bool = False,
    separator_color: Optional[Tuple[int, int, int]] = None,
    separator_thickness: int = 2,
    **kwargs,
) -> Optional["PIL_Image"]:
    """Generate a preview image with highlights.

    If in_context=True, shows segments as cropped images stacked together
    with separators between segments.

    Args:
        resolution: DPI for rendering (default from global settings)
        width: Target width in pixels (overrides resolution)
        color: Default highlight color
        labels: Whether to show labels for highlights
        label_format: Format string for labels
        highlights: Additional highlight groups to show
        layout: How to arrange multiple pages/regions
        stack_direction: Direction for stack layout
        gap: Pixels between stacked images
        columns: Number of columns for grid layout
        crop: Whether to crop
        crop_bbox: Explicit crop bounds
        in_context: If True, use special Flow visualization with separators
        separator_color: RGB color for separator lines (default: red)
        separator_thickness: Thickness of separator lines
        **kwargs: Additional parameters passed to rendering

    Returns:
        PIL Image object or None if nothing to render
    """
    resolved_resolution = self._resolve_image_resolution(resolution)

    if in_context:
        # Use the special in_context visualization
        return self._show_in_context(
            resolution=resolved_resolution,
            width=width,
            stack_direction=stack_direction,
            stack_gap=gap,
            separator_color=separator_color or (255, 0, 0),
            separator_thickness=separator_thickness,
            **kwargs,
        )

    # Otherwise use the standard show method
    return super().show(
        resolution=resolution,
        width=width,
        color=color,
        labels=labels,
        label_format=label_format,
        highlights=highlights,
        legend_position=legend_position,
        annotate=annotate,
        layout=layout,
        stack_direction=stack_direction,
        gap=gap,
        columns=columns,
        crop=crop,
        crop_bbox=crop_bbox,
        **kwargs,
    )

natural_pdf.FlowRegion

Bases: Visualizable, MultiRegionAnalysisMixin, ContextResolverMixin

Represents a selected area within a Flow, potentially composed of multiple physical Region objects (constituent_regions) that might span across different original pages or disjoint physical regions defined in the Flow.

A FlowRegion is the result of a directional operation (e.g., .below(), .above()) on a FlowElement.

Source code in natural_pdf/flows/region.py

class FlowRegion(Visualizable, MultiRegionAnalysisMixin, ContextResolverMixin):
    """
    Represents a selected area within a Flow, potentially composed of multiple
    physical Region objects (constituent_regions) that might span across
    different original pages or disjoint physical regions defined in the Flow.

    A FlowRegion is the result of a directional operation (e.g., .below(), .above())
    on a FlowElement.
    """

    def __init__(
        self,
        flow: "Flow",
        constituent_regions: List["PhysicalRegion"],
        source_flow_element: Optional["FlowElement"] = None,
        boundary_element_found: Optional[Union["PhysicalElement", "PhysicalRegion"]] = None,
    ):
        """
        Initializes a FlowRegion.

        Args:
            flow: The Flow instance this region belongs to.
            constituent_regions: A list of physical natural_pdf.elements.region.Region
                                 objects that make up this FlowRegion.
            source_flow_element: The FlowElement that created this FlowRegion.
            boundary_element_found: The physical element that stopped an 'until' search,
                                    if applicable.
        """
        self.flow: "Flow" = flow
        self.constituent_regions: List["PhysicalRegion"] = constituent_regions
        self.source_flow_element: Optional["FlowElement"] = source_flow_element
        self.boundary_element_found: Optional[Union["PhysicalElement", "PhysicalRegion"]] = (
            boundary_element_found
        )

        self.start_element: Optional[Union["PhysicalElement", "PhysicalRegion"]] = None
        self.end_element: Optional[Union["PhysicalElement", "PhysicalRegion"]] = None
        self._boundary_exclusions: Optional[str] = None

        # Add attributes for grid building, similar to Region
        self.source: Optional[str] = None
        self.region_type: Optional[str] = None
        self.metadata: Dict[str, Any] = {}

        # Cache for expensive operations
        self._cached_text: Optional[str] = None
        self._cached_elements: Optional["ElementCollection"] = None  # Stringized
        self._cached_bbox: Optional[Tuple[float, float, float, float]] = None
        self._exclusions: List[ExclusionSpec] = []
        self._multi_page_page_warned: bool = False

    def _qa_context_page_number(self) -> int:
        pages = self.pages
        return pages[0].number if pages else -1

    def _qa_source_elements(self) -> ElementCollection:
        return ElementCollection([])

    def _qa_normalize_result(self, result: Any) -> Union[Dict[str, Any], List[Dict[str, Any]]]:
        from natural_pdf.elements.region import Region

        return Region._normalize_qa_output(result)

    def _qa_blank_result(self, question: QuestionInput) -> Union[QAResult, List[QAResult]]:
        return super()._qa_blank_result(question)

    def _qa_target_region(self):
        if not self.constituent_regions:
            raise RuntimeError("FlowRegion has no constituent regions for QA.")

        if len(self.constituent_regions) > 1:
            logger.info(
                "FlowRegion spans multiple regions; Document QA will evaluate the first region only."
            )

        return self.constituent_regions[0]

    def _ocr_element_manager(self):
        if not self.constituent_regions:
            raise RuntimeError("FlowRegion has no regions for OCR operations")
        return self.constituent_regions[0].page._element_mgr

    def remove_ocr_elements(self) -> int:
        removed = 0
        for region in self.constituent_regions:
            removed += region.remove_ocr_elements()
        return removed

    def clear_text_layer(self) -> Tuple[int, int]:
        total_chars = 0
        total_words = 0
        seen_pages: Set[int] = set()
        for region in self.constituent_regions:
            page = region.page
            marker = id(page)
            if marker in seen_pages:
                continue
            seen_pages.add(marker)
            cleared_chars, cleared_words = page.clear_text_layer()
            total_chars += cleared_chars
            total_words += cleared_words
        return total_chars, total_words

    def create_text_elements_from_ocr(
        self, ocr_results: Any, scale_x: Optional[float] = None, scale_y: Optional[float] = None
    ) -> List[Any]:
        if not self.constituent_regions:
            return []
        return self.constituent_regions[0].page.create_text_elements_from_ocr(
            ocr_results, scale_x=scale_x, scale_y=scale_y
        )

    def _iter_ocr_regions(self) -> Iterable[Any]:
        return tuple(self.constituent_regions)

    def _exclusion_element_manager(self):
        if not self.constituent_regions:
            raise RuntimeError("FlowRegion has no constituent regions for exclusions")
        return self.constituent_regions[0].page._element_mgr

    def _element_to_region(
        self, element: Any, label: Optional[str] = None
    ) -> Optional["PhysicalRegion"]:
        bbox = extract_bbox(element)
        if not bbox:
            return None

        page = getattr(element, "page", None)
        if page is None and self.constituent_regions:
            page = self.constituent_regions[0].page
        if page is None:
            return None

        from natural_pdf.elements.region import (
            Region as PhysicalRegion,  # Local import to avoid cycles
        )

        clamp_bbox = bbox
        matching_regions = [
            region for region in self.constituent_regions if getattr(region, "page", None) is page
        ]
        if matching_regions:
            min_x0 = min(region.x0 for region in matching_regions)
            min_top = min(region.top for region in matching_regions)
            max_x1 = max(region.x1 for region in matching_regions)
            max_bottom = max(region.bottom for region in matching_regions)

            clamp_bbox = (
                max(min_x0, bbox[0]),
                max(min_top, bbox[1]),
                min(max_x1, bbox[2]),
                min(max_bottom, bbox[3]),
            )

            if clamp_bbox[0] >= clamp_bbox[2] or clamp_bbox[1] >= clamp_bbox[3]:
                return None

        return PhysicalRegion(page, clamp_bbox, label=label)

    def _invalidate_exclusion_cache(self) -> None:
        self._cached_text = None
        self._cached_elements = None

    def _iter_exclusion_regions(self) -> Iterable[Any]:
        return tuple(self.constituent_regions)

    @property
    def pages(self) -> Tuple["Page", ...]:
        """Return the distinct pages covered by this flow region."""
        seen: Set[int] = set()
        ordered_pages: List["Page"] = []
        for region in self.constituent_regions:
            page = getattr(region, "page", None)
            if page is None:
                continue
            marker = id(page)
            if marker not in seen:
                seen.add(marker)
                ordered_pages.append(page)
        return tuple(ordered_pages)

    @property
    def page(self) -> "Page":
        """Return the primary page for this region (first page when multi-page)."""
        pages = self.pages
        if len(pages) == 0:
            raise AttributeError("FlowRegion has no associated pages")
        if len(pages) > 1 and not self._multi_page_page_warned:
            warnings.warn(
                "FlowRegion spans multiple pages; returning the first page for .page access. "
                "Use .pages to inspect all pages.",
                UserWarning,
                stacklevel=2,
            )
            self._multi_page_page_warned = True
        return pages[0]

    def get_highlighter(self) -> "HighlightingService":
        """Resolve a highlighting service from the constituent regions."""
        if not self.constituent_regions:
            raise RuntimeError("FlowRegion has no constituent regions to get highlighter from")
        return cast("HighlightingService", resolve_highlighter(self.constituent_regions, self.flow))

    def _get_highlighter(self):
        """Compatibility hook for Visualizable mixin."""
        return self.get_highlighter()

    def _context_resolution_roots(self) -> Iterable[Any]:
        roots: List[Any] = [self]
        roots.extend(self.constituent_regions)
        if self.flow is not None:
            roots.append(self.flow)
        return roots

    def _get_render_specs(
        self,
        mode: Literal["show", "render"] = "show",
        color: Optional[Union[str, Tuple[int, int, int]]] = None,
        highlights: Optional[List[Dict[str, Any]]] = None,
        crop: Union[bool, Literal["content"]] = False,
        crop_bbox: Optional[Tuple[float, float, float, float]] = None,
        **kwargs,
    ) -> List[RenderSpec]:
        """Get render specifications for this flow region.

        Args:
            mode: Rendering mode - 'show' includes highlights, 'render' is clean
            color: Color for highlighting this region in show mode
            highlights: Additional highlight groups to show
            crop: Whether to crop to constituent regions
            crop_bbox: Explicit crop bounds
            **kwargs: Additional parameters

        Returns:
            List of RenderSpec objects, one per page with constituent regions
        """
        if not self.constituent_regions:
            return []

        label_prefix = kwargs.pop("label_prefix", None)

        regions_by_page = {}
        for region in self.constituent_regions:
            page = getattr(region, "page", None)
            if page is None:
                continue
            regions_by_page.setdefault(page, []).append(region)

        if not regions_by_page:
            return []

        specs = []
        for page, page_regions in regions_by_page.items():
            spec = RenderSpec(page=page)

            def union_bbox() -> Optional[Tuple[float, float, float, float]]:
                x_coords: List[float] = []
                y_coords: List[float] = []
                for region in page_regions:
                    bbox = getattr(region, "bbox", None)
                    if bbox:
                        x0, y0, x1, y1 = bbox
                        x_coords.extend([x0, x1])
                        y_coords.extend([y0, y1])
                if x_coords and y_coords:
                    return (min(x_coords), min(y_coords), max(x_coords), max(y_coords))
                return None

            spec.crop_bbox = resolve_crop_bbox(
                width=page.width,
                height=page.height,
                crop=crop,
                crop_bbox=crop_bbox,
                content_bbox_fn=union_bbox,
            )

            # Add highlights in show mode
            if mode == "show":
                # Highlight constituent regions
                for i, region in enumerate(page_regions):
                    # Label each part if multiple regions
                    label = None
                    if len(self.constituent_regions) > 1:
                        try:
                            global_idx = self.constituent_regions.index(region)
                        except ValueError:
                            global_idx = i
                        if label_prefix:
                            label = f"{label_prefix}_{global_idx + 1}"
                        else:
                            label = f"FlowPart_{global_idx + 1}"
                    else:
                        label = label_prefix or "FlowRegion"

                    spec.add_highlight(
                        bbox=region.bbox,
                        polygon=region.polygon if region.has_polygon else None,
                        color=color or "fuchsia",
                        label=label,
                    )

                # Add additional highlight groups if provided
                if highlights:
                    for group in highlights:
                        group_elements = group.get("elements", [])
                        group_color = group.get("color", color)
                        group_label = group.get("label")

                        for elem in group_elements:
                            # Only add if element is on this page
                            if hasattr(elem, "page") and elem.page == page:
                                spec.add_highlight(
                                    element=elem, color=group_color, label=group_label
                                )

            specs.append(spec)

        return specs

    def __getattr__(self, name: str) -> Any:
        """
        Dynamically proxy attribute access to the source FlowElement for safe attributes only.
        Spatial methods (above, below, left, right) are explicitly implemented to prevent
        silent failures and incorrect behavior.
        """
        if name in self.__dict__:
            return self.__dict__[name]

        # List of methods that should NOT be proxied - they need proper FlowRegion implementation
        spatial_methods = {"above", "below", "left", "right", "to_region"}

        if name in spatial_methods:
            raise AttributeError(
                f"'{self.__class__.__name__}' object has no attribute '{name}'. "
                f"This method requires proper FlowRegion implementation to handle spatial relationships correctly."
            )

        # Only proxy safe attributes and methods
        if self.source_flow_element is not None:
            try:
                attr = getattr(self.source_flow_element, name)
                # Only proxy non-callable attributes and explicitly safe methods
                if not callable(attr) or name in {"page", "document"}:  # Add safe methods as needed
                    return attr
                else:
                    raise AttributeError(
                        f"Method '{name}' cannot be safely proxied from FlowElement to FlowRegion. "
                        f"It may need explicit implementation."
                    )
            except AttributeError:
                pass

        raise AttributeError(f"'{self.__class__.__name__}' object has no attribute '{name}'")

    @property
    def bbox(self) -> Optional[Tuple[float, float, float, float]]:
        """
        The bounding box that encloses all constituent regions.
        Calculated dynamically and cached.
        """
        if self._cached_bbox is not None:
            return self._cached_bbox
        if not self.constituent_regions:
            return None

        # Use merge_bboxes from pdfplumber.utils.geometry to merge bboxes
        # Extract bbox tuples from regions first
        region_bboxes = [
            region.bbox for region in self.constituent_regions if hasattr(region, "bbox")
        ]
        if not region_bboxes:
            return None

        self._cached_bbox = merge_bboxes(region_bboxes)
        return self._cached_bbox

    def _require_bbox(self) -> Tuple[float, float, float, float]:
        bbox = self.bbox
        if bbox is None:
            raise ValueError("FlowRegion has no bounding box; ensure it has constituent regions")
        return bbox

    @property
    def x0(self) -> float:
        return self._require_bbox()[0]

    @property
    def top(self) -> float:
        return self._require_bbox()[1]

    @property
    def x1(self) -> float:
        return self._require_bbox()[2]

    @property
    def bottom(self) -> float:
        return self._require_bbox()[3]

    @property
    def width(self) -> Optional[float]:
        bbox = self.bbox
        if not bbox:
            return None
        return bbox[2] - bbox[0]

    @property
    def height(self) -> Optional[float]:
        bbox = self.bbox
        if not bbox:
            return None
        return bbox[3] - bbox[1]

    @property
    def has_polygon(self) -> bool:
        return False

    @property
    def polygon(self) -> List[Tuple[float, float]]:
        bbox = self.bbox
        if not bbox:
            return []
        x0, y0, x1, y1 = bbox
        return [(x0, y0), (x1, y0), (x1, y1), (x0, y1)]

    def extract_text(self, apply_exclusions: bool = True, **kwargs) -> str:
        """Concatenate text from constituent regions while preserving flow order."""
        if self._cached_text is not None and apply_exclusions:
            return self._cached_text

        if not self.constituent_regions:
            return ""

        from natural_pdf.elements.element_collection import ElementCollection

        elements = self.elements(apply_exclusions=apply_exclusions)
        # ElementCollection.extract_text handles ordering and layout-specific kwargs.
        extracted = elements.extract_text(**kwargs)

        if not extracted:
            return ""

        if apply_exclusions:
            self._cached_text = extracted
        return extracted

    def elements(self, apply_exclusions: bool = True) -> "ElementCollection":  # Stringized return
        """
        Collects all unique physical elements from all constituent physical regions.

        Args:
            apply_exclusions: Whether to respect PDF exclusion zones within each
                              constituent physical region when gathering elements.

        Returns:
            An ElementCollection containing all unique elements.
        """
        from natural_pdf.elements.element_collection import ElementCollection

        if self._cached_elements is not None and apply_exclusions:  # Simple cache check
            return self._cached_elements

        if not self.constituent_regions:
            return ElementCollection([])

        all_physical_elements: List["PhysicalElement"] = []  # Stringized item type
        seen_elements = (
            set()
        )  # To ensure uniqueness if elements are shared or duplicated by region definitions

        for region in self.constituent_regions:
            # Region.get_elements() returns a list, not ElementCollection
            elements_in_region: List["PhysicalElement"] = region.get_elements(
                apply_exclusions=apply_exclusions
            )
            for elem in elements_in_region:
                if elem not in seen_elements:  # Check for uniqueness based on object identity
                    all_physical_elements.append(elem)
                    seen_elements.add(elem)

        # Basic reading order sort based on original page and coordinates.
        def get_sort_key(phys_elem: "PhysicalElement"):  # Stringized param type
            page_idx = -1
            if hasattr(phys_elem, "page") and hasattr(phys_elem.page, "index"):
                page_idx = phys_elem.page.index
            return (page_idx, phys_elem.top, phys_elem.x0)

        try:
            sorted_physical_elements = sorted(all_physical_elements, key=get_sort_key)
        except AttributeError as exc:
            raise AttributeError(
                "Could not sort elements in FlowRegion; ensure elements expose page, top, and x0"
            ) from exc

        result_collection = ElementCollection(sorted_physical_elements)
        if apply_exclusions:
            self._cached_elements = result_collection
        return result_collection

    def find(
        self,
        selector: Optional[str] = None,
        *,
        text: Optional[Union[str, Sequence[str]]] = None,
        overlap: str = "full",
        apply_exclusions: bool = True,
        regex: bool = False,
        case: bool = True,
        text_tolerance: Optional[Dict[str, Any]] = None,
        auto_text_tolerance: Optional[Dict[str, Any]] = None,
        reading_order: bool = True,
        engine: Optional[str] = None,
    ) -> Optional["PhysicalElement"]:
        """Find the first matching element in flow order.

        Args:
            engine: Optional selector engine name forwarded to each constituent region.
        """

        if not self.constituent_regions:
            return None

        for region in self.constituent_regions:
            result = region.find(
                selector=selector,
                text=text,
                overlap=overlap,
                apply_exclusions=apply_exclusions,
                regex=regex,
                case=case,
                text_tolerance=text_tolerance,
                auto_text_tolerance=auto_text_tolerance,
                reading_order=reading_order,
                engine=engine,
            )
            if result is not None:
                return result
        return None

    def find_all(
        self,
        selector: Optional[str] = None,
        *,
        text: Optional[Union[str, Sequence[str]]] = None,
        overlap: str = "full",
        apply_exclusions: bool = True,
        regex: bool = False,
        case: bool = True,
        text_tolerance: Optional[Dict[str, Any]] = None,
        auto_text_tolerance: Optional[Dict[str, Any]] = None,
        reading_order: bool = True,
        engine: Optional[str] = None,
    ) -> "ElementCollection":
        """Find all matching elements across constituent regions.

        Args:
            engine: Optional selector engine name forwarded to each constituent region.
        """

        from natural_pdf.elements.element_collection import ElementCollection

        combined: List["PhysicalElement"] = []
        for region in self.constituent_regions:
            collection = region.find_all(
                selector=selector,
                text=text,
                overlap=overlap,
                apply_exclusions=apply_exclusions,
                regex=regex,
                case=case,
                text_tolerance=text_tolerance,
                auto_text_tolerance=auto_text_tolerance,
                reading_order=reading_order,
                engine=engine,
            )
            if collection:
                combined.extend(collection.elements)

        unique: List["PhysicalElement"] = []
        seen: Set["PhysicalElement"] = set()
        for el in combined:
            if el not in seen:
                unique.append(el)
                seen.add(el)

        return ElementCollection(unique)

    def highlight(
        self, label: Optional[str] = None, color: Optional[Union[Tuple, str]] = None, **kwargs
    ) -> Optional["PIL_Image"]:
        """
        Highlights all constituent physical regions on their respective pages.

        Args:
            label: A base label for the highlights. Each constituent region might get an indexed label.
            color: Color for the highlight.
            **kwargs: Additional arguments for the underlying highlight method.

        Returns:
            Image generated by the underlying highlight call, or None if no highlights were added.
        """
        if not self.constituent_regions:
            return None

        base_label = label if label else "FlowRegionPart"
        for i, region in enumerate(self.constituent_regions):
            current_label = (
                f"{base_label}_{i+1}" if len(self.constituent_regions) > 1 else base_label
            )
            region.highlight(label=current_label, color=color, **kwargs)
        return None

    def highlights(self, show: bool = False) -> "HighlightContext":
        """
        Create a highlight context for accumulating highlights.

        This allows for clean syntax to show multiple highlight groups:

        Example:
            with flow_region.highlights() as h:
                h.add(flow_region.find_all('table'), label='tables', color='blue')
                h.add(flow_region.find_all('text:bold'), label='bold text', color='red')
                h.show()

        Or with automatic display:
            with flow_region.highlights(show=True) as h:
                h.add(flow_region.find_all('table'), label='tables')
                h.add(flow_region.find_all('text:bold'), label='bold')
                # Automatically shows when exiting the context

        Args:
            show: If True, automatically show highlights when exiting context

        Returns:
            HighlightContext for accumulating highlights
        """
        from natural_pdf.core.highlighting_service import HighlightContext

        return HighlightContext(self, show_on_exit=show)

    def to_images(
        self,
        resolution: float = 150,
        **kwargs,
    ) -> List["PIL_Image"]:
        """
        Generates and returns a list of cropped PIL Images,
        one for each constituent physical region of this FlowRegion.
        """
        if not self.constituent_regions:
            logger.info("FlowRegion.to_images() called on an empty FlowRegion.")
            return []

        cropped_images: List["PIL_Image"] = []
        for region_part in self.constituent_regions:
            # Use render() for clean image without highlights
            img = region_part.render(resolution=resolution, crop=True, **kwargs)
            if img:
                cropped_images.append(img)

        return cropped_images

    def __repr__(self) -> str:
        return (
            f"<FlowRegion constituents={len(self.constituent_regions)}, flow={self.flow}, "
            f"source_bbox={self.source_flow_element.bbox if self.source_flow_element else 'N/A'}>"
        )

    @overload
    def expand(self, amount: float, *, apply_exclusions: bool = True) -> "FlowRegion": ...

    @overload
    def expand(
        self,
        *,
        left: Union[float, bool, str] = 0,
        right: Union[float, bool, str] = 0,
        top: Union[float, bool, str] = 0,
        bottom: Union[float, bool, str] = 0,
        width_factor: float = 1.0,
        height_factor: float = 1.0,
        apply_exclusions: bool = True,
    ) -> "FlowRegion": ...

    def expand(
        self,
        amount: Optional[float] = None,
        *,
        left: Union[float, bool, str] = 0,
        right: Union[float, bool, str] = 0,
        top: Union[float, bool, str] = 0,
        bottom: Union[float, bool, str] = 0,
        width_factor: float = 1.0,
        height_factor: float = 1.0,
        apply_exclusions: bool = True,
    ) -> "FlowRegion":
        """
        Create a new FlowRegion with all constituent regions expanded.

        Args:
            left: Amount to expand left edge (positive value expands leftwards)
            right: Amount to expand right edge (positive value expands rightwards)
            top: Amount to expand top edge (positive value expands upwards)
            bottom: Amount to expand bottom edge (positive value expands downwards)
            width_factor: Factor to multiply width by (applied after absolute expansion)
            height_factor: Factor to multiply height by (applied after absolute expansion)

        Returns:
            New FlowRegion with expanded constituent regions
        """
        if not self.constituent_regions:
            return self._spawn_from_regions([])

        if amount is not None:
            left = right = top = bottom = amount

        expanded_regions = []
        for idx, region in enumerate(self.constituent_regions):
            # Determine which adjustments to apply based on flow arrangement
            apply_left = left
            apply_right = right
            apply_top = top
            apply_bottom = bottom

            if self.flow.arrangement == "vertical":
                # In a vertical flow, only the *first* region should react to `top`
                # and only the *last* region should react to `bottom`.  This keeps
                # the virtual contiguous area intact while allowing users to nudge
                # the flow boundaries.
                if idx != 0:
                    apply_top = 0
                if idx != len(self.constituent_regions) - 1:
                    apply_bottom = 0
                # left/right apply to every region (same column width change)
            else:  # horizontal flow
                # In a horizontal flow, only the first region reacts to `left`
                # and only the last region reacts to `right`.
                if idx != 0:
                    apply_left = 0
                if idx != len(self.constituent_regions) - 1:
                    apply_right = 0
                # top/bottom apply to every region in horizontal flows

            # Skip no-op expansion to avoid extra Region objects
            needs_expansion = (
                any(v not in (0, 1.0) for v in (apply_left, apply_right, apply_top, apply_bottom))
                or width_factor != 1.0
                or height_factor != 1.0
            )

            expanded_region = (
                region.expand(
                    left=apply_left,
                    right=apply_right,
                    top=apply_top,
                    bottom=apply_bottom,
                    width_factor=width_factor,
                    height_factor=height_factor,
                    apply_exclusions=apply_exclusions,
                )
                if needs_expansion
                else region
            )
            expanded_regions.append(expanded_region)

        # Create new FlowRegion with expanded constituent regions
        return self._spawn_from_regions(expanded_regions)

    def _spawn_from_regions(self, regions: List["PhysicalRegion"]) -> "FlowRegion":
        """Create a FlowRegion clone with the supplied constituent regions."""
        new_flow_region = FlowRegion(
            flow=self.flow,
            constituent_regions=list(regions),
            source_flow_element=self.source_flow_element,
            boundary_element_found=self.boundary_element_found,
        )
        new_flow_region.source = self.source
        new_flow_region.region_type = self.region_type
        if isinstance(self.metadata, dict):
            new_flow_region.metadata = self.metadata.copy()
        else:
            new_flow_region.metadata = self.metadata
        new_flow_region._cached_text = None
        new_flow_region._cached_elements = None
        new_flow_region._cached_bbox = None
        return new_flow_region

    # Directional methods are provided by MultiRegionDirectionalMixin.

    # (remaining directional helpers inherited)

    # Table extraction helpers (delegates to underlying physical regions)

    def extract_table(
        self,
        method: Optional[str] = None,
        table_settings: Optional[dict] = None,
        use_ocr: bool = False,
        ocr_config: Optional[dict] = None,
        text_options: Optional[Dict] = None,
        cell_extraction_func: Optional[Callable[["PhysicalRegion"], Optional[str]]] = None,
        show_progress: bool = False,
        content_filter: Optional[Union[str, Callable[[str], bool], List[str]]] = None,
        apply_exclusions: bool = True,
        verticals: Optional[List[float]] = None,
        horizontals: Optional[List[float]] = None,
        # Optional row-level merge predicate. If provided, it decides whether
        # the current row (first row of a segment/page) should be merged with
        # the previous one (to handle multi-page spill-overs).
        stitch_rows: Optional[
            Callable[[List[Optional[str]], List[Optional[str]], int, "PhysicalRegion"], bool]
        ] = None,
        merge_headers: Optional[bool] = None,
        structure_engine: Optional[str] = None,
        **kwargs,
    ) -> TableResult:
        """Extracts a single logical table from the FlowRegion.

        This is a convenience wrapper that iterates through the constituent
        physical regions **in flow order**, calls their ``extract_table``
        method, and concatenates the resulting rows.  It mirrors the public
        interface of :pymeth:`natural_pdf.elements.region.Region.extract_table`.

        Args:
            method, table_settings, use_ocr, ocr_config, text_options, cell_extraction_func, show_progress:
                Same as in :pymeth:`Region.extract_table` and are forwarded as-is
                to each physical region.
            content_filter: Optional content filter applied via the underlying Region extraction.
            apply_exclusions: Whether exclusions should be applied inside each physical region.
            verticals, horizontals: Explicit guide coordinates forwarded to each Region.
            merge_headers: Whether to merge tables by removing repeated headers from subsequent
                pages/segments. If None (default), auto-detects by checking if the first row
                of each segment matches the first row of the first segment. If segments have
                inconsistent header patterns (some repeat, others don't), raises ValueError.
                Useful for multi-page tables where headers repeat on each page.
            structure_engine: Optional structure detection engine forwarded to constituent regions.
            **kwargs: Additional keyword arguments forwarded to the underlying
                ``Region.extract_table`` implementation.

        Returns:
            A TableResult object containing the aggregated table data.  Rows returned from
            consecutive constituent regions are appended in document order.  If
            no tables are detected in any region, an empty TableResult is returned.

        stitch_rows parameter:
            Controls whether the first rows of subsequent segments/regions should be merged
            into the previous row (to handle spill-over across page breaks).
            Applied AFTER header removal if merge_headers is enabled.

            • None (default) – no merging (behaviour identical to previous versions).
            • Callable – custom predicate taking
                   (prev_row, cur_row, row_idx_in_segment, segment_object) → bool.
               Return True to merge `cur_row` into `prev_row` (default column-wise merge is used).
        """

        if table_settings is None:
            table_settings = {}
        if text_options is None:
            text_options = {}

        if not self.constituent_regions:
            return TableResult([])

        # Resolve stitch_rows predicate -------------------------------------------------------
        predicate: Optional[
            Callable[[List[Optional[str]], List[Optional[str]], int, "PhysicalRegion"], bool]
        ] = (stitch_rows if callable(stitch_rows) else None)

        def _default_merge(
            prev_row: List[Optional[str]], cur_row: List[Optional[str]]
        ) -> List[Optional[str]]:
            """Column-wise merge – concatenates non-empty strings with a space."""
            from itertools import zip_longest

            merged: List[Optional[str]] = []
            for p, c in zip_longest(prev_row, cur_row, fillvalue=""):
                if (p or "").strip() and (c or "").strip():
                    merged.append(f"{p} {c}".strip())
                else:
                    merged.append((p or "") + (c or ""))
            return merged

        aggregated_rows: List[List[Optional[str]]] = []
        header_row: Optional[List[Optional[str]]] = None
        merge_headers_enabled = False
        headers_warned = False  # Track if we've already warned about dropping headers
        segment_has_repeated_header = []  # Track which segments have repeated headers

        for region_idx, region in enumerate(self.constituent_regions):
            region_result = region.extract_table(
                method=method,
                table_settings=table_settings.copy(),  # Avoid side-effects
                use_ocr=use_ocr,
                ocr_config=ocr_config,
                text_options=text_options.copy(),
                cell_extraction_func=cell_extraction_func,
                show_progress=show_progress,
                content_filter=content_filter,
                apply_exclusions=apply_exclusions,
                verticals=verticals,
                horizontals=horizontals,
                structure_engine=structure_engine,
                **kwargs,
            )

            # Convert result to list of rows
            if not region_result:
                continue

            segment_rows = (
                list(region_result)
                if isinstance(region_result, TableResult)
                else list(region_result)
            )

            # Handle header detection and merging for multi-page tables
            if region_idx == 0:
                # First segment: capture potential header row
                if segment_rows:
                    header_row = segment_rows[0]
                    # Determine if we should merge headers
                    if merge_headers is None:
                        # Auto-detect: we'll check all subsequent segments
                        merge_headers_enabled = False  # Will be determined later
                    else:
                        merge_headers_enabled = merge_headers
                    # Track that first segment exists (for consistency checking)
                    segment_has_repeated_header.append(False)  # First segment doesn't "repeat"
            elif region_idx == 1 and merge_headers is None:
                # Auto-detection: check if first row of second segment matches header
                has_header = segment_rows and header_row and segment_rows[0] == header_row
                segment_has_repeated_header.append(has_header)

                if has_header:
                    merge_headers_enabled = True
                    # Remove the detected repeated header from this segment
                    segment_rows = segment_rows[1:]
                    if not headers_warned:
                        warnings.warn(
                            "Detected repeated headers in multi-page table. Merging by removing "
                            "repeated headers from subsequent pages.",
                            UserWarning,
                            stacklevel=2,
                        )
                        headers_warned = True
                else:
                    merge_headers_enabled = False
            elif region_idx > 1:
                # Check consistency: all segments should have same pattern
                has_header = segment_rows and header_row and segment_rows[0] == header_row
                segment_has_repeated_header.append(has_header)

                # Remove header if merging is enabled and header is present
                if merge_headers_enabled and has_header:
                    segment_rows = segment_rows[1:]
            elif region_idx > 0 and merge_headers_enabled:
                # Explicit merge_headers=True: remove headers from subsequent segments
                if segment_rows and header_row and segment_rows[0] == header_row:
                    segment_rows = segment_rows[1:]
                    if not headers_warned:
                        warnings.warn(
                            "Removing repeated headers from multi-page table during merge.",
                            UserWarning,
                            stacklevel=2,
                        )
                        headers_warned = True

            # Process remaining rows with stitch_rows logic
            for row_idx, row in enumerate(segment_rows):
                if (
                    predicate is not None
                    and aggregated_rows
                    and predicate(aggregated_rows[-1], row, row_idx, region)
                ):
                    # Merge with previous row
                    aggregated_rows[-1] = _default_merge(aggregated_rows[-1], row)
                else:
                    aggregated_rows.append(row)

        # Check for inconsistent header patterns after processing all segments
        if merge_headers is None and len(segment_has_repeated_header) > 2:
            # During auto-detection, check for consistency across all segments
            expected_pattern = segment_has_repeated_header[1]  # Pattern from second segment
            for seg_idx, has_header in enumerate(segment_has_repeated_header[2:], 2):
                if has_header != expected_pattern:
                    # Inconsistent pattern detected
                    segments_with_headers = [
                        i for i, has_h in enumerate(segment_has_repeated_header[1:], 1) if has_h
                    ]
                    segments_without_headers = [
                        i for i, has_h in enumerate(segment_has_repeated_header[1:], 1) if not has_h
                    ]
                    raise ValueError(
                        f"Inconsistent header pattern in multi-page table: "
                        f"segments {segments_with_headers} have repeated headers, "
                        f"but segments {segments_without_headers} do not. "
                        f"All segments must have the same header pattern for reliable merging."
                    )

        return TableResult(aggregated_rows)

    def extract_tables(
        self,
        method: Optional[str] = None,
        table_settings: Optional[dict] = None,
        **kwargs,
    ) -> List[List[List[Optional[str]]]]:
        """Extract **all** tables from the FlowRegion.

        This simply chains :pymeth:`Region.extract_tables` over each physical
        region and concatenates their results, preserving flow order.

        Args:
            method, table_settings: Forwarded to underlying ``Region.extract_tables``.
            **kwargs: Additional keyword arguments forwarded.

        Returns:
            A list where each item is a full table (list of rows).  The order of
            tables follows the order of the constituent regions in the flow.
        """

        if table_settings is None:
            table_settings = {}

        if not self.constituent_regions:
            return []

        all_tables: List[List[List[Optional[str]]]] = []

        for region in self.constituent_regions:
            region_tables = cast(
                List[List[List[Optional[str]]]],
                region.extract_tables(
                    method=method,
                    table_settings=table_settings.copy(),
                    **kwargs,
                ),
            )
            # ``region_tables`` is a list (possibly empty).
            if region_tables:
                all_tables.extend(region_tables)

        return all_tables

    def get_sections(
        self,
        start_elements=None,
        end_elements=None,
        new_section_on_page_break: bool = False,
        include_boundaries: str = "both",
        orientation: str = "vertical",
    ) -> "ElementCollection":
        """
        Extract logical sections from this FlowRegion based on start/end boundary elements.

        This delegates to the parent Flow's get_sections() method, but only operates
        on the segments that are part of this FlowRegion.

        Args:
            start_elements: Elements or selector string that mark the start of sections
            end_elements: Elements or selector string that mark the end of sections
            new_section_on_page_break: Whether to start a new section at page boundaries
            include_boundaries: How to include boundary elements: 'start', 'end', 'both', or 'none'
            orientation: 'vertical' (default) or 'horizontal' - determines section direction

        Returns:
            ElementCollection of FlowRegion objects representing the extracted sections

        Example:
            # Split a multi-page table region by headers
            table_region = flow.find("text:contains('Table 4')").below(until="text:contains('Table 5')")
            sections = table_region.get_sections(start_elements="text:bold")
        """
        # Create a temporary Flow with just our constituent regions as segments
        from natural_pdf.flows.flow import Flow

        temp_flow = Flow(
            segments=self.constituent_regions,
            arrangement=self.flow.arrangement,
            alignment=self.flow.alignment,
            segment_gap=self.flow.segment_gap,
        )

        # Delegate to Flow's get_sections implementation
        return temp_flow.get_sections(
            start_elements=start_elements,
            end_elements=end_elements,
            new_section_on_page_break=new_section_on_page_break,
            include_boundaries=include_boundaries,
            orientation=orientation,
        )

    def split(
        self, by: Optional[str] = None, page_breaks: bool = True, **kwargs
    ) -> "ElementCollection":
        """
        Split this FlowRegion into sections.

        This is a convenience method that wraps get_sections() with common splitting patterns.

        Args:
            by: Selector string for elements that mark section boundaries (e.g., "text:bold")
            page_breaks: Whether to also split at page boundaries (default: True)
            **kwargs: Additional arguments passed to get_sections()

        Returns:
            ElementCollection of FlowRegion objects representing the sections

        Example:
            # Split by bold headers
            sections = flow_region.split(by="text:bold")

            # Split only by specific text pattern, ignoring page breaks
            sections = flow_region.split(
                by="text:contains('Section')",
                page_breaks=False
            )
        """
        return self.get_sections(start_elements=by, new_section_on_page_break=page_breaks, **kwargs)

    @property
    def normalized_type(self) -> Optional[str]:
        """
        Return the normalized type for selector compatibility.
        This allows FlowRegion to be found by selectors like 'table'.
        """
        if self.region_type:
            # Convert region_type to normalized format (replace spaces with underscores, lowercase)
            return self.region_type.lower().replace(" ", "_")
        return None

    @property
    def type(self) -> Optional[str]:
        """
        Return the type attribute for selector compatibility.
        This is an alias for normalized_type.
        """
        return self.normalized_type

    def get_highlight_specs(self) -> List[Dict[str, Any]]:
        """
        Get highlight specifications for all constituent regions.

        This implements the highlighting protocol for FlowRegions, returning
        specs for each constituent region so they can be highlighted on their
        respective pages.

        Returns:
            List of highlight specification dictionaries, one for each
            constituent region.
        """
        specs = []

        for region in self.constituent_regions:
            if not hasattr(region, "page") or region.page is None:
                continue

            if not hasattr(region, "bbox") or region.bbox is None:
                continue

            spec = {
                "page": region.page,
                "page_index": region.page.index if hasattr(region.page, "index") else 0,
                "bbox": region.bbox,
                "element": region,  # Reference to the constituent region
            }

            # Add polygon if available
            if hasattr(region, "polygon") and hasattr(region, "has_polygon") and region.has_polygon:
                spec["polygon"] = region.polygon

            specs.append(spec)

        return specs

Attributes

natural_pdf.FlowRegion.bbox property

The bounding box that encloses all constituent regions. Calculated dynamically and cached.

natural_pdf.FlowRegion.normalized_type property

Return the normalized type for selector compatibility. This allows FlowRegion to be found by selectors like 'table'.

natural_pdf.FlowRegion.page property

Return the primary page for this region (first page when multi-page).

natural_pdf.FlowRegion.pages property

Return the distinct pages covered by this flow region.

natural_pdf.FlowRegion.type property

Return the type attribute for selector compatibility. This is an alias for normalized_type.

Functions

natural_pdf.FlowRegion.__getattr__(name)

Dynamically proxy attribute access to the source FlowElement for safe attributes only. Spatial methods (above, below, left, right) are explicitly implemented to prevent silent failures and incorrect behavior.

Source code in natural_pdf/flows/region.py

def __getattr__(self, name: str) -> Any:
    """
    Dynamically proxy attribute access to the source FlowElement for safe attributes only.
    Spatial methods (above, below, left, right) are explicitly implemented to prevent
    silent failures and incorrect behavior.
    """
    if name in self.__dict__:
        return self.__dict__[name]

    # List of methods that should NOT be proxied - they need proper FlowRegion implementation
    spatial_methods = {"above", "below", "left", "right", "to_region"}

    if name in spatial_methods:
        raise AttributeError(
            f"'{self.__class__.__name__}' object has no attribute '{name}'. "
            f"This method requires proper FlowRegion implementation to handle spatial relationships correctly."
        )

    # Only proxy safe attributes and methods
    if self.source_flow_element is not None:
        try:
            attr = getattr(self.source_flow_element, name)
            # Only proxy non-callable attributes and explicitly safe methods
            if not callable(attr) or name in {"page", "document"}:  # Add safe methods as needed
                return attr
            else:
                raise AttributeError(
                    f"Method '{name}' cannot be safely proxied from FlowElement to FlowRegion. "
                    f"It may need explicit implementation."
                )
        except AttributeError:
            pass

    raise AttributeError(f"'{self.__class__.__name__}' object has no attribute '{name}'")

natural_pdf.FlowRegion.__init__(flow, constituent_regions, source_flow_element=None, boundary_element_found=None)

Initializes a FlowRegion.

Parameters:

Name	Type	Description	Default
flow	'Flow'	The Flow instance this region belongs to.	required
constituent_regions	List['PhysicalRegion']	A list of physical natural_pdf.elements.region.Region objects that make up this FlowRegion.	required
source_flow_element	Optional['FlowElement']	The FlowElement that created this FlowRegion.	None
boundary_element_found	Optional[Union['PhysicalElement', 'PhysicalRegion']]	The physical element that stopped an 'until' search, if applicable.	None

Source code in natural_pdf/flows/region.py

def __init__(
    self,
    flow: "Flow",
    constituent_regions: List["PhysicalRegion"],
    source_flow_element: Optional["FlowElement"] = None,
    boundary_element_found: Optional[Union["PhysicalElement", "PhysicalRegion"]] = None,
):
    """
    Initializes a FlowRegion.

    Args:
        flow: The Flow instance this region belongs to.
        constituent_regions: A list of physical natural_pdf.elements.region.Region
                             objects that make up this FlowRegion.
        source_flow_element: The FlowElement that created this FlowRegion.
        boundary_element_found: The physical element that stopped an 'until' search,
                                if applicable.
    """
    self.flow: "Flow" = flow
    self.constituent_regions: List["PhysicalRegion"] = constituent_regions
    self.source_flow_element: Optional["FlowElement"] = source_flow_element
    self.boundary_element_found: Optional[Union["PhysicalElement", "PhysicalRegion"]] = (
        boundary_element_found
    )

    self.start_element: Optional[Union["PhysicalElement", "PhysicalRegion"]] = None
    self.end_element: Optional[Union["PhysicalElement", "PhysicalRegion"]] = None
    self._boundary_exclusions: Optional[str] = None

    # Add attributes for grid building, similar to Region
    self.source: Optional[str] = None
    self.region_type: Optional[str] = None
    self.metadata: Dict[str, Any] = {}

    # Cache for expensive operations
    self._cached_text: Optional[str] = None
    self._cached_elements: Optional["ElementCollection"] = None  # Stringized
    self._cached_bbox: Optional[Tuple[float, float, float, float]] = None
    self._exclusions: List[ExclusionSpec] = []
    self._multi_page_page_warned: bool = False

natural_pdf.FlowRegion.elements(apply_exclusions=True)

Collects all unique physical elements from all constituent physical regions.

Parameters:

Name	Type	Description	Default
apply_exclusions	bool	Whether to respect PDF exclusion zones within each constituent physical region when gathering elements.	True

Returns:

Type	Description
'ElementCollection'	An ElementCollection containing all unique elements.

Source code in natural_pdf/flows/region.py

def elements(self, apply_exclusions: bool = True) -> "ElementCollection":  # Stringized return
    """
    Collects all unique physical elements from all constituent physical regions.

    Args:
        apply_exclusions: Whether to respect PDF exclusion zones within each
                          constituent physical region when gathering elements.

    Returns:
        An ElementCollection containing all unique elements.
    """
    from natural_pdf.elements.element_collection import ElementCollection

    if self._cached_elements is not None and apply_exclusions:  # Simple cache check
        return self._cached_elements

    if not self.constituent_regions:
        return ElementCollection([])

    all_physical_elements: List["PhysicalElement"] = []  # Stringized item type
    seen_elements = (
        set()
    )  # To ensure uniqueness if elements are shared or duplicated by region definitions

    for region in self.constituent_regions:
        # Region.get_elements() returns a list, not ElementCollection
        elements_in_region: List["PhysicalElement"] = region.get_elements(
            apply_exclusions=apply_exclusions
        )
        for elem in elements_in_region:
            if elem not in seen_elements:  # Check for uniqueness based on object identity
                all_physical_elements.append(elem)
                seen_elements.add(elem)

    # Basic reading order sort based on original page and coordinates.
    def get_sort_key(phys_elem: "PhysicalElement"):  # Stringized param type
        page_idx = -1
        if hasattr(phys_elem, "page") and hasattr(phys_elem.page, "index"):
            page_idx = phys_elem.page.index
        return (page_idx, phys_elem.top, phys_elem.x0)

    try:
        sorted_physical_elements = sorted(all_physical_elements, key=get_sort_key)
    except AttributeError as exc:
        raise AttributeError(
            "Could not sort elements in FlowRegion; ensure elements expose page, top, and x0"
        ) from exc

    result_collection = ElementCollection(sorted_physical_elements)
    if apply_exclusions:
        self._cached_elements = result_collection
    return result_collection

natural_pdf.FlowRegion.expand(amount=None, *, left=0, right=0, top=0, bottom=0, width_factor=1.0, height_factor=1.0, apply_exclusions=True)

expand(amount: float, *, apply_exclusions: bool = True) -> 'FlowRegion'

expand(*, left: Union[float, bool, str] = 0, right: Union[float, bool, str] = 0, top: Union[float, bool, str] = 0, bottom: Union[float, bool, str] = 0, width_factor: float = 1.0, height_factor: float = 1.0, apply_exclusions: bool = True) -> 'FlowRegion'

Create a new FlowRegion with all constituent regions expanded.

Parameters:

Name	Type	Description	Default
left	Union[float, bool, str]	Amount to expand left edge (positive value expands leftwards)	0
right	Union[float, bool, str]	Amount to expand right edge (positive value expands rightwards)	0
top	Union[float, bool, str]	Amount to expand top edge (positive value expands upwards)	0
bottom	Union[float, bool, str]	Amount to expand bottom edge (positive value expands downwards)	0
width_factor	float	Factor to multiply width by (applied after absolute expansion)	1.0
height_factor	float	Factor to multiply height by (applied after absolute expansion)	1.0

Returns:

Type	Description
'FlowRegion'	New FlowRegion with expanded constituent regions

Source code in natural_pdf/flows/region.py

def expand(
    self,
    amount: Optional[float] = None,
    *,
    left: Union[float, bool, str] = 0,
    right: Union[float, bool, str] = 0,
    top: Union[float, bool, str] = 0,
    bottom: Union[float, bool, str] = 0,
    width_factor: float = 1.0,
    height_factor: float = 1.0,
    apply_exclusions: bool = True,
) -> "FlowRegion":
    """
    Create a new FlowRegion with all constituent regions expanded.

    Args:
        left: Amount to expand left edge (positive value expands leftwards)
        right: Amount to expand right edge (positive value expands rightwards)
        top: Amount to expand top edge (positive value expands upwards)
        bottom: Amount to expand bottom edge (positive value expands downwards)
        width_factor: Factor to multiply width by (applied after absolute expansion)
        height_factor: Factor to multiply height by (applied after absolute expansion)

    Returns:
        New FlowRegion with expanded constituent regions
    """
    if not self.constituent_regions:
        return self._spawn_from_regions([])

    if amount is not None:
        left = right = top = bottom = amount

    expanded_regions = []
    for idx, region in enumerate(self.constituent_regions):
        # Determine which adjustments to apply based on flow arrangement
        apply_left = left
        apply_right = right
        apply_top = top
        apply_bottom = bottom

        if self.flow.arrangement == "vertical":
            # In a vertical flow, only the *first* region should react to `top`
            # and only the *last* region should react to `bottom`.  This keeps
            # the virtual contiguous area intact while allowing users to nudge
            # the flow boundaries.
            if idx != 0:
                apply_top = 0
            if idx != len(self.constituent_regions) - 1:
                apply_bottom = 0
            # left/right apply to every region (same column width change)
        else:  # horizontal flow
            # In a horizontal flow, only the first region reacts to `left`
            # and only the last region reacts to `right`.
            if idx != 0:
                apply_left = 0
            if idx != len(self.constituent_regions) - 1:
                apply_right = 0
            # top/bottom apply to every region in horizontal flows

        # Skip no-op expansion to avoid extra Region objects
        needs_expansion = (
            any(v not in (0, 1.0) for v in (apply_left, apply_right, apply_top, apply_bottom))
            or width_factor != 1.0
            or height_factor != 1.0
        )

        expanded_region = (
            region.expand(
                left=apply_left,
                right=apply_right,
                top=apply_top,
                bottom=apply_bottom,
                width_factor=width_factor,
                height_factor=height_factor,
                apply_exclusions=apply_exclusions,
            )
            if needs_expansion
            else region
        )
        expanded_regions.append(expanded_region)

    # Create new FlowRegion with expanded constituent regions
    return self._spawn_from_regions(expanded_regions)

natural_pdf.FlowRegion.extract_table(method=None, table_settings=None, use_ocr=False, ocr_config=None, text_options=None, cell_extraction_func=None, show_progress=False, content_filter=None, apply_exclusions=True, verticals=None, horizontals=None, stitch_rows=None, merge_headers=None, structure_engine=None, **kwargs)

Extracts a single logical table from the FlowRegion.

This is a convenience wrapper that iterates through the constituent physical regions in flow order, calls their extract_table method, and concatenates the resulting rows. It mirrors the public interface of :pymeth:natural_pdf.elements.region.Region.extract_table.

Parameters:

Name	Type	Description	Default
method, table_settings, use_ocr, ocr_config, text_options, cell_extraction_func, show_progress		Same as in :pymeth:Region.extract_table and are forwarded as-is to each physical region.	required
content_filter	Optional[Union[str, Callable[[str], bool], List[str]]]	Optional content filter applied via the underlying Region extraction.	None
apply_exclusions	bool	Whether exclusions should be applied inside each physical region.	True
verticals, horizontals		Explicit guide coordinates forwarded to each Region.	required
merge_headers	Optional[bool]	Whether to merge tables by removing repeated headers from subsequent pages/segments. If None (default), auto-detects by checking if the first row of each segment matches the first row of the first segment. If segments have inconsistent header patterns (some repeat, others don't), raises ValueError. Useful for multi-page tables where headers repeat on each page.	None
structure_engine	Optional[str]	Optional structure detection engine forwarded to constituent regions.	None
**kwargs		Additional keyword arguments forwarded to the underlying Region.extract_table implementation.	{}

Returns:

Type	Description
TableResult	A TableResult object containing the aggregated table data. Rows returned from
TableResult	consecutive constituent regions are appended in document order. If
TableResult	no tables are detected in any region, an empty TableResult is returned.

stitch_rows parameter

Controls whether the first rows of subsequent segments/regions should be merged into the previous row (to handle spill-over across page breaks). Applied AFTER header removal if merge_headers is enabled.

• None (default) – no merging (behaviour identical to previous versions). • Callable – custom predicate taking (prev_row, cur_row, row_idx_in_segment, segment_object) → bool. Return True to merge cur_row into prev_row (default column-wise merge is used).

Source code in natural_pdf/flows/region.py

def extract_table(
    self,
    method: Optional[str] = None,
    table_settings: Optional[dict] = None,
    use_ocr: bool = False,
    ocr_config: Optional[dict] = None,
    text_options: Optional[Dict] = None,
    cell_extraction_func: Optional[Callable[["PhysicalRegion"], Optional[str]]] = None,
    show_progress: bool = False,
    content_filter: Optional[Union[str, Callable[[str], bool], List[str]]] = None,
    apply_exclusions: bool = True,
    verticals: Optional[List[float]] = None,
    horizontals: Optional[List[float]] = None,
    # Optional row-level merge predicate. If provided, it decides whether
    # the current row (first row of a segment/page) should be merged with
    # the previous one (to handle multi-page spill-overs).
    stitch_rows: Optional[
        Callable[[List[Optional[str]], List[Optional[str]], int, "PhysicalRegion"], bool]
    ] = None,
    merge_headers: Optional[bool] = None,
    structure_engine: Optional[str] = None,
    **kwargs,
) -> TableResult:
    """Extracts a single logical table from the FlowRegion.

    This is a convenience wrapper that iterates through the constituent
    physical regions **in flow order**, calls their ``extract_table``
    method, and concatenates the resulting rows.  It mirrors the public
    interface of :pymeth:`natural_pdf.elements.region.Region.extract_table`.

    Args:
        method, table_settings, use_ocr, ocr_config, text_options, cell_extraction_func, show_progress:
            Same as in :pymeth:`Region.extract_table` and are forwarded as-is
            to each physical region.
        content_filter: Optional content filter applied via the underlying Region extraction.
        apply_exclusions: Whether exclusions should be applied inside each physical region.
        verticals, horizontals: Explicit guide coordinates forwarded to each Region.
        merge_headers: Whether to merge tables by removing repeated headers from subsequent
            pages/segments. If None (default), auto-detects by checking if the first row
            of each segment matches the first row of the first segment. If segments have
            inconsistent header patterns (some repeat, others don't), raises ValueError.
            Useful for multi-page tables where headers repeat on each page.
        structure_engine: Optional structure detection engine forwarded to constituent regions.
        **kwargs: Additional keyword arguments forwarded to the underlying
            ``Region.extract_table`` implementation.

    Returns:
        A TableResult object containing the aggregated table data.  Rows returned from
        consecutive constituent regions are appended in document order.  If
        no tables are detected in any region, an empty TableResult is returned.

    stitch_rows parameter:
        Controls whether the first rows of subsequent segments/regions should be merged
        into the previous row (to handle spill-over across page breaks).
        Applied AFTER header removal if merge_headers is enabled.

        • None (default) – no merging (behaviour identical to previous versions).
        • Callable – custom predicate taking
               (prev_row, cur_row, row_idx_in_segment, segment_object) → bool.
           Return True to merge `cur_row` into `prev_row` (default column-wise merge is used).
    """

    if table_settings is None:
        table_settings = {}
    if text_options is None:
        text_options = {}

    if not self.constituent_regions:
        return TableResult([])

    # Resolve stitch_rows predicate -------------------------------------------------------
    predicate: Optional[
        Callable[[List[Optional[str]], List[Optional[str]], int, "PhysicalRegion"], bool]
    ] = (stitch_rows if callable(stitch_rows) else None)

    def _default_merge(
        prev_row: List[Optional[str]], cur_row: List[Optional[str]]
    ) -> List[Optional[str]]:
        """Column-wise merge – concatenates non-empty strings with a space."""
        from itertools import zip_longest

        merged: List[Optional[str]] = []
        for p, c in zip_longest(prev_row, cur_row, fillvalue=""):
            if (p or "").strip() and (c or "").strip():
                merged.append(f"{p} {c}".strip())
            else:
                merged.append((p or "") + (c or ""))
        return merged

    aggregated_rows: List[List[Optional[str]]] = []
    header_row: Optional[List[Optional[str]]] = None
    merge_headers_enabled = False
    headers_warned = False  # Track if we've already warned about dropping headers
    segment_has_repeated_header = []  # Track which segments have repeated headers

    for region_idx, region in enumerate(self.constituent_regions):
        region_result = region.extract_table(
            method=method,
            table_settings=table_settings.copy(),  # Avoid side-effects
            use_ocr=use_ocr,
            ocr_config=ocr_config,
            text_options=text_options.copy(),
            cell_extraction_func=cell_extraction_func,
            show_progress=show_progress,
            content_filter=content_filter,
            apply_exclusions=apply_exclusions,
            verticals=verticals,
            horizontals=horizontals,
            structure_engine=structure_engine,
            **kwargs,
        )

        # Convert result to list of rows
        if not region_result:
            continue

        segment_rows = (
            list(region_result)
            if isinstance(region_result, TableResult)
            else list(region_result)
        )

        # Handle header detection and merging for multi-page tables
        if region_idx == 0:
            # First segment: capture potential header row
            if segment_rows:
                header_row = segment_rows[0]
                # Determine if we should merge headers
                if merge_headers is None:
                    # Auto-detect: we'll check all subsequent segments
                    merge_headers_enabled = False  # Will be determined later
                else:
                    merge_headers_enabled = merge_headers
                # Track that first segment exists (for consistency checking)
                segment_has_repeated_header.append(False)  # First segment doesn't "repeat"
        elif region_idx == 1 and merge_headers is None:
            # Auto-detection: check if first row of second segment matches header
            has_header = segment_rows and header_row and segment_rows[0] == header_row
            segment_has_repeated_header.append(has_header)

            if has_header:
                merge_headers_enabled = True
                # Remove the detected repeated header from this segment
                segment_rows = segment_rows[1:]
                if not headers_warned:
                    warnings.warn(
                        "Detected repeated headers in multi-page table. Merging by removing "
                        "repeated headers from subsequent pages.",
                        UserWarning,
                        stacklevel=2,
                    )
                    headers_warned = True
            else:
                merge_headers_enabled = False
        elif region_idx > 1:
            # Check consistency: all segments should have same pattern
            has_header = segment_rows and header_row and segment_rows[0] == header_row
            segment_has_repeated_header.append(has_header)

            # Remove header if merging is enabled and header is present
            if merge_headers_enabled and has_header:
                segment_rows = segment_rows[1:]
        elif region_idx > 0 and merge_headers_enabled:
            # Explicit merge_headers=True: remove headers from subsequent segments
            if segment_rows and header_row and segment_rows[0] == header_row:
                segment_rows = segment_rows[1:]
                if not headers_warned:
                    warnings.warn(
                        "Removing repeated headers from multi-page table during merge.",
                        UserWarning,
                        stacklevel=2,
                    )
                    headers_warned = True

        # Process remaining rows with stitch_rows logic
        for row_idx, row in enumerate(segment_rows):
            if (
                predicate is not None
                and aggregated_rows
                and predicate(aggregated_rows[-1], row, row_idx, region)
            ):
                # Merge with previous row
                aggregated_rows[-1] = _default_merge(aggregated_rows[-1], row)
            else:
                aggregated_rows.append(row)

    # Check for inconsistent header patterns after processing all segments
    if merge_headers is None and len(segment_has_repeated_header) > 2:
        # During auto-detection, check for consistency across all segments
        expected_pattern = segment_has_repeated_header[1]  # Pattern from second segment
        for seg_idx, has_header in enumerate(segment_has_repeated_header[2:], 2):
            if has_header != expected_pattern:
                # Inconsistent pattern detected
                segments_with_headers = [
                    i for i, has_h in enumerate(segment_has_repeated_header[1:], 1) if has_h
                ]
                segments_without_headers = [
                    i for i, has_h in enumerate(segment_has_repeated_header[1:], 1) if not has_h
                ]
                raise ValueError(
                    f"Inconsistent header pattern in multi-page table: "
                    f"segments {segments_with_headers} have repeated headers, "
                    f"but segments {segments_without_headers} do not. "
                    f"All segments must have the same header pattern for reliable merging."
                )

    return TableResult(aggregated_rows)

natural_pdf.FlowRegion.extract_tables(method=None, table_settings=None, **kwargs)

Extract all tables from the FlowRegion.

This simply chains :pymeth:Region.extract_tables over each physical region and concatenates their results, preserving flow order.

Parameters:

Name	Type	Description	Default
method, table_settings		Forwarded to underlying Region.extract_tables.	required
**kwargs		Additional keyword arguments forwarded.	{}

Returns:

Type	Description
List[List[List[Optional[str]]]]	A list where each item is a full table (list of rows). The order of
List[List[List[Optional[str]]]]	tables follows the order of the constituent regions in the flow.

Source code in natural_pdf/flows/region.py

def extract_tables(
    self,
    method: Optional[str] = None,
    table_settings: Optional[dict] = None,
    **kwargs,
) -> List[List[List[Optional[str]]]]:
    """Extract **all** tables from the FlowRegion.

    This simply chains :pymeth:`Region.extract_tables` over each physical
    region and concatenates their results, preserving flow order.

    Args:
        method, table_settings: Forwarded to underlying ``Region.extract_tables``.
        **kwargs: Additional keyword arguments forwarded.

    Returns:
        A list where each item is a full table (list of rows).  The order of
        tables follows the order of the constituent regions in the flow.
    """

    if table_settings is None:
        table_settings = {}

    if not self.constituent_regions:
        return []

    all_tables: List[List[List[Optional[str]]]] = []

    for region in self.constituent_regions:
        region_tables = cast(
            List[List[List[Optional[str]]]],
            region.extract_tables(
                method=method,
                table_settings=table_settings.copy(),
                **kwargs,
            ),
        )
        # ``region_tables`` is a list (possibly empty).
        if region_tables:
            all_tables.extend(region_tables)

    return all_tables

natural_pdf.FlowRegion.extract_text(apply_exclusions=True, **kwargs)

Concatenate text from constituent regions while preserving flow order.

Source code in natural_pdf/flows/region.py

def extract_text(self, apply_exclusions: bool = True, **kwargs) -> str:
    """Concatenate text from constituent regions while preserving flow order."""
    if self._cached_text is not None and apply_exclusions:
        return self._cached_text

    if not self.constituent_regions:
        return ""

    from natural_pdf.elements.element_collection import ElementCollection

    elements = self.elements(apply_exclusions=apply_exclusions)
    # ElementCollection.extract_text handles ordering and layout-specific kwargs.
    extracted = elements.extract_text(**kwargs)

    if not extracted:
        return ""

    if apply_exclusions:
        self._cached_text = extracted
    return extracted

natural_pdf.FlowRegion.find(selector=None, *, text=None, overlap='full', apply_exclusions=True, regex=False, case=True, text_tolerance=None, auto_text_tolerance=None, reading_order=True, engine=None)

Find the first matching element in flow order.

Parameters:

Name	Type	Description	Default
engine	Optional[str]	Optional selector engine name forwarded to each constituent region.	None

Source code in natural_pdf/flows/region.py

def find(
    self,
    selector: Optional[str] = None,
    *,
    text: Optional[Union[str, Sequence[str]]] = None,
    overlap: str = "full",
    apply_exclusions: bool = True,
    regex: bool = False,
    case: bool = True,
    text_tolerance: Optional[Dict[str, Any]] = None,
    auto_text_tolerance: Optional[Dict[str, Any]] = None,
    reading_order: bool = True,
    engine: Optional[str] = None,
) -> Optional["PhysicalElement"]:
    """Find the first matching element in flow order.

    Args:
        engine: Optional selector engine name forwarded to each constituent region.
    """

    if not self.constituent_regions:
        return None

    for region in self.constituent_regions:
        result = region.find(
            selector=selector,
            text=text,
            overlap=overlap,
            apply_exclusions=apply_exclusions,
            regex=regex,
            case=case,
            text_tolerance=text_tolerance,
            auto_text_tolerance=auto_text_tolerance,
            reading_order=reading_order,
            engine=engine,
        )
        if result is not None:
            return result
    return None

natural_pdf.FlowRegion.find_all(selector=None, *, text=None, overlap='full', apply_exclusions=True, regex=False, case=True, text_tolerance=None, auto_text_tolerance=None, reading_order=True, engine=None)

Find all matching elements across constituent regions.

Parameters:

Name	Type	Description	Default
engine	Optional[str]	Optional selector engine name forwarded to each constituent region.	None

Source code in natural_pdf/flows/region.py

def find_all(
    self,
    selector: Optional[str] = None,
    *,
    text: Optional[Union[str, Sequence[str]]] = None,
    overlap: str = "full",
    apply_exclusions: bool = True,
    regex: bool = False,
    case: bool = True,
    text_tolerance: Optional[Dict[str, Any]] = None,
    auto_text_tolerance: Optional[Dict[str, Any]] = None,
    reading_order: bool = True,
    engine: Optional[str] = None,
) -> "ElementCollection":
    """Find all matching elements across constituent regions.

    Args:
        engine: Optional selector engine name forwarded to each constituent region.
    """

    from natural_pdf.elements.element_collection import ElementCollection

    combined: List["PhysicalElement"] = []
    for region in self.constituent_regions:
        collection = region.find_all(
            selector=selector,
            text=text,
            overlap=overlap,
            apply_exclusions=apply_exclusions,
            regex=regex,
            case=case,
            text_tolerance=text_tolerance,
            auto_text_tolerance=auto_text_tolerance,
            reading_order=reading_order,
            engine=engine,
        )
        if collection:
            combined.extend(collection.elements)

    unique: List["PhysicalElement"] = []
    seen: Set["PhysicalElement"] = set()
    for el in combined:
        if el not in seen:
            unique.append(el)
            seen.add(el)

    return ElementCollection(unique)

natural_pdf.FlowRegion.get_highlight_specs()

Get highlight specifications for all constituent regions.

This implements the highlighting protocol for FlowRegions, returning specs for each constituent region so they can be highlighted on their respective pages.

Returns:

Type	Description
List[Dict[str, Any]]	List of highlight specification dictionaries, one for each
List[Dict[str, Any]]	constituent region.

Source code in natural_pdf/flows/region.py

def get_highlight_specs(self) -> List[Dict[str, Any]]:
    """
    Get highlight specifications for all constituent regions.

    This implements the highlighting protocol for FlowRegions, returning
    specs for each constituent region so they can be highlighted on their
    respective pages.

    Returns:
        List of highlight specification dictionaries, one for each
        constituent region.
    """
    specs = []

    for region in self.constituent_regions:
        if not hasattr(region, "page") or region.page is None:
            continue

        if not hasattr(region, "bbox") or region.bbox is None:
            continue

        spec = {
            "page": region.page,
            "page_index": region.page.index if hasattr(region.page, "index") else 0,
            "bbox": region.bbox,
            "element": region,  # Reference to the constituent region
        }

        # Add polygon if available
        if hasattr(region, "polygon") and hasattr(region, "has_polygon") and region.has_polygon:
            spec["polygon"] = region.polygon

        specs.append(spec)

    return specs

natural_pdf.FlowRegion.get_highlighter()

Resolve a highlighting service from the constituent regions.

Source code in natural_pdf/flows/region.py

def get_highlighter(self) -> "HighlightingService":
    """Resolve a highlighting service from the constituent regions."""
    if not self.constituent_regions:
        raise RuntimeError("FlowRegion has no constituent regions to get highlighter from")
    return cast("HighlightingService", resolve_highlighter(self.constituent_regions, self.flow))

natural_pdf.FlowRegion.get_sections(start_elements=None, end_elements=None, new_section_on_page_break=False, include_boundaries='both', orientation='vertical')

Extract logical sections from this FlowRegion based on start/end boundary elements.

This delegates to the parent Flow's get_sections() method, but only operates on the segments that are part of this FlowRegion.

Parameters:

Name	Type	Description	Default
start_elements		Elements or selector string that mark the start of sections	None
end_elements		Elements or selector string that mark the end of sections	None
new_section_on_page_break	bool	Whether to start a new section at page boundaries	False
include_boundaries	str	How to include boundary elements: 'start', 'end', 'both', or 'none'	'both'
orientation	str	'vertical' (default) or 'horizontal' - determines section direction	'vertical'

Returns:

Type	Description
'ElementCollection'	ElementCollection of FlowRegion objects representing the extracted sections

Example

Split a multi-page table region by headers

table_region = flow.find("text:contains('Table 4')").below(until="text:contains('Table 5')") sections = table_region.get_sections(start_elements="text:bold")

Source code in natural_pdf/flows/region.py

def get_sections(
    self,
    start_elements=None,
    end_elements=None,
    new_section_on_page_break: bool = False,
    include_boundaries: str = "both",
    orientation: str = "vertical",
) -> "ElementCollection":
    """
    Extract logical sections from this FlowRegion based on start/end boundary elements.

    This delegates to the parent Flow's get_sections() method, but only operates
    on the segments that are part of this FlowRegion.

    Args:
        start_elements: Elements or selector string that mark the start of sections
        end_elements: Elements or selector string that mark the end of sections
        new_section_on_page_break: Whether to start a new section at page boundaries
        include_boundaries: How to include boundary elements: 'start', 'end', 'both', or 'none'
        orientation: 'vertical' (default) or 'horizontal' - determines section direction

    Returns:
        ElementCollection of FlowRegion objects representing the extracted sections

    Example:
        # Split a multi-page table region by headers
        table_region = flow.find("text:contains('Table 4')").below(until="text:contains('Table 5')")
        sections = table_region.get_sections(start_elements="text:bold")
    """
    # Create a temporary Flow with just our constituent regions as segments
    from natural_pdf.flows.flow import Flow

    temp_flow = Flow(
        segments=self.constituent_regions,
        arrangement=self.flow.arrangement,
        alignment=self.flow.alignment,
        segment_gap=self.flow.segment_gap,
    )

    # Delegate to Flow's get_sections implementation
    return temp_flow.get_sections(
        start_elements=start_elements,
        end_elements=end_elements,
        new_section_on_page_break=new_section_on_page_break,
        include_boundaries=include_boundaries,
        orientation=orientation,
    )

natural_pdf.FlowRegion.highlight(label=None, color=None, **kwargs)

Highlights all constituent physical regions on their respective pages.

Parameters:

Name	Type	Description	Default
label	Optional[str]	A base label for the highlights. Each constituent region might get an indexed label.	None
color	Optional[Union[Tuple, str]]	Color for the highlight.	None
**kwargs		Additional arguments for the underlying highlight method.	{}

Returns:

Type	Description
Optional['PIL_Image']	Image generated by the underlying highlight call, or None if no highlights were added.

Source code in natural_pdf/flows/region.py

def highlight(
    self, label: Optional[str] = None, color: Optional[Union[Tuple, str]] = None, **kwargs
) -> Optional["PIL_Image"]:
    """
    Highlights all constituent physical regions on their respective pages.

    Args:
        label: A base label for the highlights. Each constituent region might get an indexed label.
        color: Color for the highlight.
        **kwargs: Additional arguments for the underlying highlight method.

    Returns:
        Image generated by the underlying highlight call, or None if no highlights were added.
    """
    if not self.constituent_regions:
        return None

    base_label = label if label else "FlowRegionPart"
    for i, region in enumerate(self.constituent_regions):
        current_label = (
            f"{base_label}_{i+1}" if len(self.constituent_regions) > 1 else base_label
        )
        region.highlight(label=current_label, color=color, **kwargs)
    return None

natural_pdf.FlowRegion.highlights(show=False)

Create a highlight context for accumulating highlights.

This allows for clean syntax to show multiple highlight groups:

Example

with flow_region.highlights() as h: h.add(flow_region.find_all('table'), label='tables', color='blue') h.add(flow_region.find_all('text:bold'), label='bold text', color='red') h.show()

Or with automatic display

with flow_region.highlights(show=True) as h: h.add(flow_region.find_all('table'), label='tables') h.add(flow_region.find_all('text:bold'), label='bold') # Automatically shows when exiting the context

Parameters:

Name	Type	Description	Default
show	bool	If True, automatically show highlights when exiting context	False

Returns:

Type	Description
'HighlightContext'	HighlightContext for accumulating highlights

Source code in natural_pdf/flows/region.py

def highlights(self, show: bool = False) -> "HighlightContext":
    """
    Create a highlight context for accumulating highlights.

    This allows for clean syntax to show multiple highlight groups:

    Example:
        with flow_region.highlights() as h:
            h.add(flow_region.find_all('table'), label='tables', color='blue')
            h.add(flow_region.find_all('text:bold'), label='bold text', color='red')
            h.show()

    Or with automatic display:
        with flow_region.highlights(show=True) as h:
            h.add(flow_region.find_all('table'), label='tables')
            h.add(flow_region.find_all('text:bold'), label='bold')
            # Automatically shows when exiting the context

    Args:
        show: If True, automatically show highlights when exiting context

    Returns:
        HighlightContext for accumulating highlights
    """
    from natural_pdf.core.highlighting_service import HighlightContext

    return HighlightContext(self, show_on_exit=show)

natural_pdf.FlowRegion.split(by=None, page_breaks=True, **kwargs)

Split this FlowRegion into sections.

This is a convenience method that wraps get_sections() with common splitting patterns.

Parameters:

Name	Type	Description	Default
by	Optional[str]	Selector string for elements that mark section boundaries (e.g., "text:bold")	None
page_breaks	bool	Whether to also split at page boundaries (default: True)	True
**kwargs		Additional arguments passed to get_sections()	{}

Returns:

Type	Description
'ElementCollection'	ElementCollection of FlowRegion objects representing the sections

Example

Split by bold headers

sections = flow_region.split(by="text:bold")

Split only by specific text pattern, ignoring page breaks

sections = flow_region.split( by="text:contains('Section')", page_breaks=False )

Source code in natural_pdf/flows/region.py

def split(
    self, by: Optional[str] = None, page_breaks: bool = True, **kwargs
) -> "ElementCollection":
    """
    Split this FlowRegion into sections.

    This is a convenience method that wraps get_sections() with common splitting patterns.

    Args:
        by: Selector string for elements that mark section boundaries (e.g., "text:bold")
        page_breaks: Whether to also split at page boundaries (default: True)
        **kwargs: Additional arguments passed to get_sections()

    Returns:
        ElementCollection of FlowRegion objects representing the sections

    Example:
        # Split by bold headers
        sections = flow_region.split(by="text:bold")

        # Split only by specific text pattern, ignoring page breaks
        sections = flow_region.split(
            by="text:contains('Section')",
            page_breaks=False
        )
    """
    return self.get_sections(start_elements=by, new_section_on_page_break=page_breaks, **kwargs)

natural_pdf.FlowRegion.to_images(resolution=150, **kwargs)

Generates and returns a list of cropped PIL Images, one for each constituent physical region of this FlowRegion.

Source code in natural_pdf/flows/region.py

def to_images(
    self,
    resolution: float = 150,
    **kwargs,
) -> List["PIL_Image"]:
    """
    Generates and returns a list of cropped PIL Images,
    one for each constituent physical region of this FlowRegion.
    """
    if not self.constituent_regions:
        logger.info("FlowRegion.to_images() called on an empty FlowRegion.")
        return []

    cropped_images: List["PIL_Image"] = []
    for region_part in self.constituent_regions:
        # Use render() for clean image without highlights
        img = region_part.render(resolution=resolution, crop=True, **kwargs)
        if img:
            cropped_images.append(img)

    return cropped_images

natural_pdf.Guides

Manages vertical and horizontal guide lines for table extraction and layout analysis.

Guides are collections of coordinates that can be used to define table boundaries, column positions, or general layout structures. They can be created through various detection methods or manually specified.

Attributes:

Name	Type	Description
verticals		List of x-coordinates for vertical guide lines
horizontals		List of y-coordinates for horizontal guide lines
context		Optional Page/Region that these guides relate to
bounds	Optional[Bounds]	Optional bounding box (x0, y0, x1, y1) for relative coordinate conversion
snap_behavior		How to handle failed snapping operations ('warn', 'ignore', 'raise')

Source code in natural_pdf/analyzers/guides/base.py

class Guides:
    """
    Manages vertical and horizontal guide lines for table extraction and layout analysis.

    Guides are collections of coordinates that can be used to define table boundaries,
    column positions, or general layout structures. They can be created through various
    detection methods or manually specified.

    Attributes:
        verticals: List of x-coordinates for vertical guide lines
        horizontals: List of y-coordinates for horizontal guide lines
        context: Optional Page/Region that these guides relate to
        bounds: Optional bounding box (x0, y0, x1, y1) for relative coordinate conversion
        snap_behavior: How to handle failed snapping operations ('warn', 'ignore', 'raise')
    """

    def __init__(
        self,
        verticals: Optional[Union[Iterable[float], GuidesContext]] = None,
        horizontals: Optional[Iterable[float]] = None,
        context: Optional[GuidesContext] = None,
        bounds: Optional[Tuple[float, float, float, float]] = None,
        relative: bool = False,
        snap_behavior: Literal["raise", "warn", "ignore"] = "warn",
    ):
        """
        Initialize a Guides object.

        Args:
            verticals: Iterable of x-coordinates for vertical guides, or a context object shorthand
            horizontals: Iterable of y-coordinates for horizontal guides
            context: Object providing spatial context (page, region, flow, etc.)
            bounds: Bounding box (x0, top, x1, bottom) if context not provided
            relative: Whether coordinates are relative (0-1) or absolute
            snap_behavior: How to handle snapping conflicts ('raise', 'warn', or 'ignore')
        """
        context_obj = context
        vertical_seed: Iterable[float] = ()

        # Handle Guides(page) or Guides(flow_region) shorthand
        if (
            verticals is not None
            and horizontals is None
            and context_obj is None
            and _is_guides_context(verticals)
        ):
            context_obj = cast(GuidesContext, verticals)
        elif verticals is not None:
            vertical_seed = cast(Iterable[float], verticals)

        self.context = context_obj
        coerced_bounds = _bounds_from_object(bounds) if bounds is not None else None
        self.bounds: Optional[Bounds] = coerced_bounds
        self.relative = relative
        self.snap_behavior = snap_behavior
        # Backwards compatibility alias for legacy options
        self.on_no_snap = snap_behavior

        # Check if we're dealing with a FlowRegion
        self.is_flow_region = _is_flow_region(context_obj)

        # If FlowRegion, we'll store guides per constituent region
        if self.is_flow_region:
            self._flow_guides: Dict["Region", Tuple[List[float], List[float]]] = {}
            # For unified view across all regions
            self._unified_vertical: List[Tuple[float, "Region"]] = []
            self._unified_horizontal: List[Tuple[float, "Region"]] = []
            # Cache for sorted unique coordinates
            self._vertical_cache: Optional[List[float]] = None
            self._horizontal_cache: Optional[List[float]] = None

        # Initialize with GuidesList instances
        horizontal_seed: Iterable[float] = horizontals if horizontals is not None else ()
        self._vertical = GuidesList(
            self,
            "vertical",
            sorted([float(x) for x in vertical_seed]),
        )
        self._horizontal = GuidesList(
            self,
            "horizontal",
            sorted([float(y) for y in horizontal_seed]),
        )

        # Determine bounds from context if needed
        if self.bounds is None and self.context is not None:
            self.bounds = _bounds_from_object(self.context)

        # Convert relative to absolute if needed
        if self.relative and self.bounds is not None:
            x0, top, x1, bottom = self.bounds
            width = x1 - x0
            height = bottom - top

            self._vertical.data = [x0 + float(v) * width for v in self._vertical]
            self._horizontal.data = [top + float(h) * height for h in self._horizontal]
            self.relative = False

    def _flow_context(self) -> FlowRegion:
        if not _is_flow_region(self.context):
            raise AttributeError("Flow context is not available for these guides")
        return cast(FlowRegion, self.context)

    def _flow_constituent_regions(self) -> Sequence["Region"]:
        return _constituent_regions(self._flow_context())

    @property
    def vertical(self) -> GuidesList:
        """Get vertical guide coordinates."""
        if self.is_flow_region and self._vertical_cache is not None:
            # Return cached unified view
            self._vertical.data = self._vertical_cache
        elif self.is_flow_region and self._unified_vertical:
            # Build unified view from flow guides
            all_verticals = []
            for coord, region in self._unified_vertical:
                all_verticals.append(coord)
            # Remove duplicates and sort
            self._vertical_cache = sorted(list(set(all_verticals)))
            self._vertical.data = self._vertical_cache
        return self._vertical

    @vertical.setter
    def vertical(self, value: Union[List[float], "Guides", None]):
        """Set vertical guides from a list of coordinates or another Guides object."""
        if self.is_flow_region:
            # Invalidate cache when setting new values
            self._vertical_cache = None

        if value is None:
            self._vertical.data = []
        elif isinstance(value, Guides):
            # Extract vertical coordinates from another Guides object
            self._vertical.data = sorted([float(x) for x in value.vertical])
        elif isinstance(value, str):
            # Explicitly reject strings to avoid confusing iteration over characters
            raise TypeError(
                f"vertical cannot be a string, got '{value}'. Use a list of coordinates or Guides object."
            )
        elif hasattr(value, "__iter__"):
            # Handle list/tuple of coordinates
            try:
                self._vertical.data = sorted([float(x) for x in value])
            except (ValueError, TypeError) as e:
                raise TypeError(f"vertical must contain numeric values, got {value}: {e}")
        else:
            raise TypeError(f"vertical must be a list, Guides object, or None, got {type(value)}")

    @property
    def horizontal(self) -> GuidesList:
        """Get horizontal guide coordinates."""
        if self.is_flow_region and self._horizontal_cache is not None:
            # Return cached unified view
            self._horizontal.data = self._horizontal_cache
        elif self.is_flow_region and self._unified_horizontal:
            # Build unified view from flow guides
            all_horizontals = []
            for coord, region in self._unified_horizontal:
                all_horizontals.append(coord)
            # Remove duplicates and sort
            self._horizontal_cache = sorted(list(set(all_horizontals)))
            self._horizontal.data = self._horizontal_cache
        return self._horizontal

    @horizontal.setter
    def horizontal(self, value: Union[List[float], "Guides", None]):
        """Set horizontal guides from a list of coordinates or another Guides object."""
        if self.is_flow_region:
            # Invalidate cache when setting new values
            self._horizontal_cache = None

        if value is None:
            self._horizontal.data = []
        elif isinstance(value, Guides):
            # Extract horizontal coordinates from another Guides object
            self._horizontal.data = sorted([float(y) for y in value.horizontal])
        elif isinstance(value, str):
            # Explicitly reject strings
            raise TypeError(
                f"horizontal cannot be a string, got '{value}'. Use a list of coordinates or Guides object."
            )
        elif hasattr(value, "__iter__"):
            # Handle list/tuple of coordinates
            try:
                self._horizontal.data = sorted([float(y) for y in value])
            except (ValueError, TypeError) as e:
                raise TypeError(f"horizontal must contain numeric values, got {value}: {e}")
        else:
            raise TypeError(f"horizontal must be a list, Guides object, or None, got {type(value)}")

    def _get_context_bounds(self) -> Tuple[float, float, float, float]:
        """Return bounding box for the current context, ensuring it exists."""
        if self.context is None:
            raise ValueError("No context available for bounds computation")
        return _require_bounds(self.context, context="guide context")

    # -------------------------------------------------------------------------
    # Factory Methods
    # -------------------------------------------------------------------------

    @classmethod
    def divide(
        cls,
        obj: Union["Page", "Region", Tuple[float, float, float, float]],
        n: Optional[int] = None,
        cols: Optional[int] = None,
        rows: Optional[int] = None,
        axis: Literal["vertical", "horizontal", "both"] = "both",
    ) -> "Guides":
        """
        Create guides by evenly dividing an object.

        Args:
            obj: Object to divide (Page, Region, or bbox tuple)
            n: Number of divisions (creates n+1 guides). Used if cols/rows not specified.
            cols: Number of columns (creates cols+1 vertical guides)
            rows: Number of rows (creates rows+1 horizontal guides)
            axis: Which axis to divide along

        Returns:
            New Guides object with evenly spaced lines

        Examples:
            # Divide into 3 columns
            guides = Guides.divide(page, cols=3)

            # Divide into 5 rows
            guides = Guides.divide(region, rows=5)

            # Divide both axes
            guides = Guides.divide(page, cols=3, rows=5)
        """
        # Extract bounds from object
        if isinstance(obj, tuple) and len(obj) == 4:
            bounds = _ensure_bounds_tuple(obj)
            context = None
        else:
            context = obj
            bounds = _require_bounds(obj, context="object to divide")

        x0, y0, x1, y1 = bounds
        verticals = []
        horizontals = []

        # Handle vertical guides
        if axis in ("vertical", "both"):
            n_vertical = cols + 1 if cols is not None else (n + 1 if n is not None else 0)
            if n_vertical > 0:
                for i in range(n_vertical):
                    x = x0 + (x1 - x0) * i / (n_vertical - 1)
                    verticals.append(float(x))

        # Handle horizontal guides
        if axis in ("horizontal", "both"):
            n_horizontal = rows + 1 if rows is not None else (n + 1 if n is not None else 0)
            if n_horizontal > 0:
                for i in range(n_horizontal):
                    y = y0 + (y1 - y0) * i / (n_horizontal - 1)
                    horizontals.append(float(y))

        return cls(verticals=verticals, horizontals=horizontals, context=context, bounds=bounds)

    @classmethod
    def from_lines(
        cls,
        obj: GuidesContext,
        axis: Literal["vertical", "horizontal", "both"] = "both",
        threshold: Union[float, str] = "auto",
        source_label: Optional[str] = None,
        max_lines_h: Optional[int] = None,
        max_lines_v: Optional[int] = None,
        outer: bool = False,
        detection_method: str = "auto",
        resolution: int = 192,
        **detect_kwargs,
    ) -> "Guides":
        """
        Create guides from detected line elements.

        Args:
            obj: Page, Region, or FlowRegion to detect lines from
            axis: Which orientations to detect
            threshold: Detection threshold ('auto' or float 0.0-1.0) - used for pixel detection
            source_label: Filter for line source (vector method) or label for detected lines (pixel method)
            max_lines_h: Maximum number of horizontal lines to keep
            max_lines_v: Maximum number of vertical lines to keep
            outer: Whether to add outer boundary guides
            detection_method: 'vector', 'pixels' (default), or 'auto' for hybrid detection.
            resolution: DPI for pixel-based detection (default: 192)
            **detect_kwargs: Additional parameters for pixel-based detection:
                - min_gap_h: Minimum gap between horizontal lines (pixels)
                - min_gap_v: Minimum gap between vertical lines (pixels)
                - binarization_method: 'adaptive' or 'otsu'
                - morph_op_h/v: Morphological operations ('open', 'close', 'none')
                - smoothing_sigma_h/v: Gaussian smoothing sigma
                - method: 'projection' (default) or 'lsd' (requires opencv)

        Returns:
            New Guides object with detected line positions
        """
        if axis == "both":
            vertical_guides = cls.from_lines(
                obj,
                axis="vertical",
                threshold=threshold,
                source_label=source_label,
                max_lines_h=max_lines_h,
                max_lines_v=max_lines_v,
                outer=outer,
                detection_method=detection_method,
                resolution=resolution,
                **detect_kwargs,
            )
            horizontal_guides = cls.from_lines(
                obj,
                axis="horizontal",
                threshold=threshold,
                source_label=source_label,
                max_lines_h=max_lines_h,
                max_lines_v=max_lines_v,
                outer=outer,
                detection_method=detection_method,
                resolution=resolution,
                **detect_kwargs,
            )
            bounds = _bounds_from_object(obj)
            return cls(
                verticals=list(vertical_guides.vertical),
                horizontals=list(horizontal_guides.horizontal),
                context=obj,
                bounds=bounds,
            )

        options = {
            "threshold": threshold,
            "source_label": source_label,
            "max_lines_h": max_lines_h if axis == "horizontal" else None,
            "max_lines_v": max_lines_v if axis == "vertical" else None,
            "outer": outer,
            "detection_method": detection_method,
            "resolution": resolution,
            **detect_kwargs,
        }

        if _is_flow_region(obj):
            guides = cls(context=obj)
            adapter = FlowGuideAdapter(guides)
            axis_values: Dict[Any, Sequence[float]] = {}
            for region in adapter.regions:
                result = run_guides_detect(
                    axis=axis,
                    method="lines",
                    context=region,
                    options=options,
                )
                axis_values[region] = [float(value) for value in result.coordinates]

            adapter.update_axis_from_regions(axis, axis_values, append=False)
            return guides

        result = run_guides_detect(
            axis=axis,
            method="lines",
            context=obj,
            options=options,
        )

        coords = [float(value) for value in result.coordinates]
        bounds = _bounds_from_object(obj)
        if axis == "vertical":
            return cls(verticals=coords, horizontals=[], context=obj, bounds=bounds)
        return cls(verticals=[], horizontals=coords, context=obj, bounds=bounds)

    @classmethod
    def from_content(
        cls,
        obj: GuidesContext,
        axis: Literal["vertical", "horizontal"] = "vertical",
        markers: Union[str, List[str], "ElementCollection", None] = None,
        align: Union[
            Literal["left", "right", "center", "between"], Literal["top", "bottom"]
        ] = "left",
        outer: OuterBoundaryMode = True,
        tolerance: float = 5,
        apply_exclusions: bool = True,
    ) -> "Guides":
        """
        Create guides based on text content positions.

        Args:
            obj: Page, Region, or FlowRegion to search for content
            axis: Whether to create vertical or horizontal guides
            markers: Content to search for. Can be:
                - str: single selector (e.g., 'text:contains("Name")') or literal text
                - List[str]: list of selectors or literal text strings
                - ElementCollection: collection of elements to extract text from
                - None: no markers
            align: Where to place guides relative to found text:
                - For vertical guides: 'left', 'right', 'center', 'between'
                - For horizontal guides: 'top', 'bottom', 'center', 'between'
            outer: Whether to add guides at the boundaries
            tolerance: Maximum distance to search for text
            apply_exclusions: Whether to apply exclusion zones when searching for text

        Returns:
            New Guides object aligned to text content
        """
        if axis == "horizontal":
            if align == "top":
                align = "left"
            elif align == "bottom":
                align = "right"

        options = {
            "markers": markers,
            "align": align,
            "outer": outer,
            "tolerance": tolerance,
            "apply_exclusions": apply_exclusions,
        }

        if _is_flow_region(obj):
            guides = cls(context=obj)
            adapter = FlowGuideAdapter(guides)
            axis_values: Dict[Any, Sequence[float]] = {}
            for region in adapter.regions:
                result = run_guides_detect(
                    axis=axis,
                    method="content",
                    context=region,
                    options=options,
                )
                axis_values[region] = [float(value) for value in result.coordinates]

            adapter.update_axis_from_regions(axis, axis_values, append=False)
            return guides

        result = run_guides_detect(
            axis=axis,
            method="content",
            context=obj,
            options=options,
        )

        coords = [float(value) for value in result.coordinates]
        bounds = _bounds_from_object(obj)
        if axis == "vertical":
            return cls(verticals=coords, context=obj, bounds=bounds)
        return cls(horizontals=coords, context=obj, bounds=bounds)

    @classmethod
    def from_whitespace(
        cls,
        obj: GuidesContext,
        axis: Literal["vertical", "horizontal", "both"] = "both",
        min_gap: float = 10,
    ) -> "Guides":
        """Create guides by detecting whitespace gaps (divide + snap placeholder)."""
        if axis == "both":
            vertical_guides = cls.from_whitespace(obj, axis="vertical", min_gap=min_gap)
            horizontal_guides = cls.from_whitespace(obj, axis="horizontal", min_gap=min_gap)
            bounds = _bounds_from_object(obj)
            return cls(
                verticals=list(vertical_guides.vertical),
                horizontals=list(horizontal_guides.horizontal),
                context=obj,
                bounds=bounds,
            )

        options = {"min_gap": min_gap}

        if _is_flow_region(obj):
            guides = cls(context=obj)
            adapter = FlowGuideAdapter(guides)
            axis_values: Dict[Any, Sequence[float]] = {}
            for region in adapter.regions:
                result = run_guides_detect(
                    axis=axis,
                    method="whitespace",
                    context=region,
                    options=options,
                )
                axis_values[region] = [float(value) for value in result.coordinates]

            adapter.update_axis_from_regions(axis, axis_values, append=False)
            return guides

        result = run_guides_detect(
            axis=axis,
            method="whitespace",
            context=obj,
            options=options,
        )

        coords = [float(value) for value in result.coordinates]
        bounds = _bounds_from_object(obj)
        if axis == "vertical":
            return cls(verticals=coords, horizontals=[], context=obj, bounds=bounds)
        return cls(verticals=[], horizontals=coords, context=obj, bounds=bounds)

    @classmethod
    def from_headers(
        cls,
        obj: GuidesContext,
        axis: Literal["vertical", "horizontal"] = "vertical",
        headers: Union["ElementCollection", Sequence[Any], None] = None,
        method: Literal["min_crossings", "seam_carving"] = "min_crossings",
        min_width: Optional[float] = None,
        max_width: Optional[float] = None,
        margin: float = 0.5,
        row_stabilization: bool = True,
        num_samples: int = 400,
    ) -> "Guides":
        """Create vertical guides by analyzing header elements."""

        if axis != "vertical":
            raise ValueError("from_headers() only works for vertical guides (columns)")

        options = {
            "headers": headers,
            "method": method,
            "min_width": min_width,
            "max_width": max_width,
            "margin": margin,
            "row_stabilization": row_stabilization,
            "num_samples": num_samples,
        }

        if _is_flow_region(obj):
            guides = cls(context=obj)
            adapter = FlowGuideAdapter(guides)
            region_values: Dict[Any, Sequence[float]] = {}

            for region in adapter.regions:
                result = run_guides_detect(
                    axis="vertical",
                    method="headers",
                    context=region,
                    options=options,
                )
                region_values[region] = [float(value) for value in result.coordinates]

            adapter.update_axis_from_regions("vertical", region_values, append=False)
            return guides

        result = run_guides_detect(
            axis="vertical",
            method="headers",
            context=obj,
            options=options,
        )

        coords = [float(value) for value in result.coordinates]
        bounds = _bounds_from_object(obj)
        return cls(verticals=coords, horizontals=[], context=obj, bounds=bounds)

    @classmethod
    def from_stripes(
        cls,
        obj: GuidesContext,
        axis: Literal["vertical", "horizontal"] = "horizontal",
        stripes: Optional[Union["ElementCollection", Sequence[Any]]] = None,
        color: Optional[str] = None,
    ) -> "Guides":
        """Create guides from zebra stripes or colored bands."""

        axis_lower = axis.lower()
        if axis_lower not in {"vertical", "horizontal"}:
            raise ValueError("axis must be 'vertical' or 'horizontal'")
        axis = cast(Literal["vertical", "horizontal"], axis_lower)

        options = {"stripes": stripes, "color": color}

        if _is_flow_region(obj):
            guides = cls(context=obj)
            adapter = FlowGuideAdapter(guides)
            region_values: Dict[Any, Sequence[float]] = {}

            for region in adapter.regions:
                result = run_guides_detect(
                    axis=axis, method="stripes", context=region, options=options
                )
                region_values[region] = [float(value) for value in result.coordinates]

            adapter.update_axis_from_regions(axis, region_values, append=True)
            return guides

        result = run_guides_detect(
            axis=axis,
            method="stripes",
            context=obj,
            options=options,
        )

        coords = [float(value) for value in result.coordinates]
        bounds = _bounds_from_object(obj)
        if axis == "vertical":
            return cls(verticals=coords, horizontals=[], context=obj, bounds=bounds)
        return cls(verticals=[], horizontals=coords, context=obj, bounds=bounds)

    @classmethod
    def new(cls, context: Optional[Union["Page", "Region"]] = None) -> "Guides":
        """
        Create a new empty Guides object, optionally with a context.

        This provides a clean way to start building guides through chaining:
        guides = Guides.new(page).add_content(axis='vertical', markers=[...])

        Args:
            context: Optional Page or Region to use as default context for operations

        Returns:
            New empty Guides object
        """
        return cls(verticals=[], horizontals=[], context=context)

    # -------------------------------------------------------------------------
    # Manipulation Methods
    # -------------------------------------------------------------------------

    def snap_to_whitespace(
        self,
        axis: str = "vertical",
        min_gap: float = 10.0,
        detection_method: str = "pixels",  # 'pixels' or 'text'
        threshold: Union[
            float, str
        ] = "auto",  # threshold for what counts as a trough (0.0-1.0) or 'auto'
        on_no_snap: str = "warn",
    ) -> "Guides":
        """
        Snap guides to nearby whitespace gaps (troughs) using optimal assignment.
        Modifies this Guides object in place.

        Args:
            axis: Direction to snap ('vertical' or 'horizontal')
            min_gap: Minimum gap size to consider as a valid trough
            detection_method: Method for detecting troughs:
                            'pixels' - use pixel-based density analysis (default)
                            'text' - use text element spacing analysis
            threshold: Threshold for what counts as a trough:
                      - float (0.0-1.0): areas with this fraction or less of max density count as troughs
                      - 'auto': automatically find threshold that creates enough troughs for guides
                      (only applies when detection_method='pixels')
            on_no_snap: Action when snapping fails ('warn', 'ignore', 'raise')

        Returns:
            Self for method chaining.
        """
        if not self.context:
            logger.warning("No context available for whitespace detection")
            return self

        detection_mode = detection_method.lower()
        if detection_mode not in {"pixels", "text"}:
            raise ValueError("detection_method must be 'pixels' or 'text'")

        text_elements = collect_text_elements(self.context)

        def _compute_gaps(axis: str, guide_positions: Sequence[float]) -> List[Tuple[float, float]]:
            if detection_mode == "pixels":
                if axis == "vertical":
                    return find_vertical_whitespace_gaps(
                        self.bounds,
                        text_elements,
                        min_gap,
                        threshold,
                        guide_positions=guide_positions,
                    )
                return find_horizontal_whitespace_gaps(
                    self.bounds,
                    text_elements,
                    min_gap,
                    threshold,
                    guide_positions=guide_positions,
                )
            if axis == "vertical":
                return find_vertical_element_gaps(self.bounds, text_elements, min_gap)
            return find_horizontal_element_gaps(self.bounds, text_elements, min_gap)

        # Handle FlowRegion case - collect all text elements across regions
        if self.is_flow_region:
            if not text_elements:
                logger.warning(
                    "No text elements found across flow regions for whitespace detection"
                )
                return self

            if axis == "vertical":
                gaps = _compute_gaps("vertical", self.vertical.data)
                all_guides = []
                guide_to_region_map = {}
                for coord, region in self._unified_vertical:
                    all_guides.append(coord)
                    guide_to_region_map.setdefault(coord, []).append(region)

                if gaps and all_guides:
                    original_guides = all_guides.copy()
                    self._snap_guides_to_gaps(all_guides, gaps, axis)

                    self._unified_vertical = []
                    for i, new_coord in enumerate(all_guides):
                        original_coord = original_guides[i]
                        regions = guide_to_region_map.get(original_coord, [])
                        for region in regions:
                            self._unified_vertical.append((new_coord, region))

                    for region in self._flow_guides:
                        region_verticals = [
                            coord for coord, r in self._unified_vertical if r == region
                        ]
                        self._flow_guides[region] = (
                            sorted(list(set(region_verticals))),
                            self._flow_guides[region][1],
                        )

                    self._vertical_cache = None

            elif axis == "horizontal":
                gaps = _compute_gaps("horizontal", self.horizontal.data)
                all_guides = []
                guide_to_region_map = {}
                for coord, region in self._unified_horizontal:
                    all_guides.append(coord)
                    guide_to_region_map.setdefault(coord, []).append(region)

                if gaps and all_guides:
                    original_guides = all_guides.copy()
                    self._snap_guides_to_gaps(all_guides, gaps, axis)

                    self._unified_horizontal = []
                    for i, new_coord in enumerate(all_guides):
                        original_coord = original_guides[i]
                        regions = guide_to_region_map.get(original_coord, [])
                        for region in regions:
                            self._unified_horizontal.append((new_coord, region))

                    for region in self._flow_guides:
                        region_horizontals = [
                            coord for coord, r in self._unified_horizontal if r == region
                        ]
                        self._flow_guides[region] = (
                            self._flow_guides[region][0],
                            sorted(list(set(region_horizontals))),
                        )

                    self._horizontal_cache = None

            else:
                raise ValueError("axis must be 'vertical' or 'horizontal'")

            return self

        if not text_elements:
            logger.warning("No text elements found for whitespace detection")
            return self

        if axis == "vertical":
            gaps = _compute_gaps("vertical", self.vertical.data)
            if gaps:
                self._snap_guides_to_gaps(self.vertical.data, gaps, axis)
        elif axis == "horizontal":
            gaps = _compute_gaps("horizontal", self.horizontal.data)
            if gaps:
                self._snap_guides_to_gaps(self.horizontal.data, gaps, axis)
        else:
            raise ValueError("axis must be 'vertical' or 'horizontal'")

        # Ensure all coordinates are Python floats (not numpy types)
        self.vertical.data[:] = [float(x) for x in self.vertical.data]
        self.horizontal.data[:] = [float(y) for y in self.horizontal.data]

        return self

    def shift(
        self, index: int, offset: float, axis: Literal["vertical", "horizontal"] = "vertical"
    ) -> "Guides":
        """
        Move a specific guide by a offset amount.

        Args:
            index: Index of the guide to move
            offset: Amount to move (positive = right/down)
            axis: Which guide list to modify

        Returns:
            Self for method chaining
        """
        if axis == "vertical":
            if 0 <= index < len(self.vertical):
                self.vertical[index] += offset
                self.vertical = sorted(self.vertical)
            else:
                logger.warning(f"Vertical guide index {index} out of range")
        else:
            if 0 <= index < len(self.horizontal):
                self.horizontal[index] += offset
                self.horizontal = sorted(self.horizontal)
            else:
                logger.warning(f"Horizontal guide index {index} out of range")

        return self

    def add_vertical(self, x: float) -> "Guides":
        """Add a vertical guide at the specified x-coordinate."""
        self.vertical.append(x)
        self.vertical = sorted(self.vertical)
        return self

    def add_horizontal(self, y: float) -> "Guides":
        """Add a horizontal guide at the specified y-coordinate."""
        self.horizontal.append(y)
        self.horizontal = sorted(self.horizontal)
        return self

    def remove_vertical(self, index: int) -> "Guides":
        """Remove a vertical guide by index."""
        if 0 <= index < len(self.vertical):
            self.vertical.pop(index)
        return self

    def remove_horizontal(self, index: int) -> "Guides":
        """Remove a horizontal guide by index."""
        if 0 <= index < len(self.horizontal):
            self.horizontal.pop(index)
        return self

    # -------------------------------------------------------------------------
    # Region extraction properties
    # -------------------------------------------------------------------------

    @property
    def columns(self):
        """Access columns by index like guides.columns[0]."""
        return _ColumnAccessor(self)

    @property
    def rows(self):
        """Access rows by index like guides.rows[0]."""
        return _RowAccessor(self)

    @property
    def cells(self):
        """Access cells by index like guides.cells[row][col] or guides.cells[row, col]."""
        return _CellAccessor(self)

    # -------------------------------------------------------------------------
    # Region extraction methods (alternative API)
    # -------------------------------------------------------------------------

    def column(self, index: int, obj: Optional[Union["Page", "Region"]] = None) -> "Region":
        """
        Get a column region from the guides.

        Args:
            index: Column index (0-based)
            obj: Page or Region to create the column on (uses self.context if None)

        Returns:
            Region representing the specified column

        Raises:
            IndexError: If column index is out of range
        """
        target = obj or self.context
        if target is None:
            raise ValueError("No context available for region creation")

        vertical_guides = list(self.vertical.data)
        if not vertical_guides or index < 0 or index >= len(vertical_guides) - 1:
            raise IndexError(
                f"Column index {index} out of range (have {len(vertical_guides)-1} columns)"
            )

        # Get bounds from context
        _, y0, _, y1 = self._get_context_bounds()

        # Get column boundaries
        x0 = vertical_guides[index]
        x1 = vertical_guides[index + 1]

        # Create region using absolute coordinates
        if hasattr(target, "create_region"):
            if isinstance(target, Region):
                return target.create_region(x0, y0, x1, y1, relative=False)
            return target.create_region(x0, y0, x1, y1)

        try:
            page = _resolve_single_page(target)
        except (TypeError, ValueError) as exc:
            raise TypeError(f"Cannot create region on {type(target)}") from exc

        return page.create_region(x0, y0, x1, y1)

    def row(self, index: int, obj: Optional[Union["Page", "Region"]] = None) -> "Region":
        """
        Get a row region from the guides.

        Args:
            index: Row index (0-based)
            obj: Page or Region to create the row on (uses self.context if None)

        Returns:
            Region representing the specified row

        Raises:
            IndexError: If row index is out of range
        """
        target = obj or self.context
        if target is None:
            raise ValueError("No context available for region creation")

        horizontal_guides = list(self.horizontal.data)
        if not horizontal_guides or index < 0 or index >= len(horizontal_guides) - 1:
            raise IndexError(
                f"Row index {index} out of range (have {len(horizontal_guides)-1} rows)"
            )

        # Get bounds from context
        x0, _, x1, _ = self._get_context_bounds()

        # Get row boundaries
        y0 = horizontal_guides[index]
        y1 = horizontal_guides[index + 1]

        # Create region using absolute coordinates
        if hasattr(target, "create_region"):
            if isinstance(target, Region):
                return target.create_region(x0, y0, x1, y1, relative=False)
            return target.create_region(x0, y0, x1, y1)

        try:
            page = _resolve_single_page(target)
        except (TypeError, ValueError) as exc:
            raise TypeError(f"Cannot create region on {type(target)}") from exc

        return page.create_region(x0, y0, x1, y1)

    def cell(self, row: int, col: int, obj: Optional[Union["Page", "Region"]] = None) -> "Region":
        """
        Get a cell region from the guides.

        Args:
            row: Row index (0-based)
            col: Column index (0-based)
            obj: Page or Region to create the cell on (uses self.context if None)

        Returns:
            Region representing the specified cell

        Raises:
            IndexError: If row or column index is out of range
        """
        target = obj or self.context
        if target is None:
            raise ValueError("No context available for region creation")

        vertical_guides = list(self.vertical.data)
        horizontal_guides = list(self.horizontal.data)
        if not vertical_guides or col < 0 or col >= len(vertical_guides) - 1:
            raise IndexError(
                f"Column index {col} out of range (have {len(vertical_guides)-1} columns)"
            )
        if not horizontal_guides or row < 0 or row >= len(horizontal_guides) - 1:
            raise IndexError(f"Row index {row} out of range (have {len(horizontal_guides)-1} rows)")

        # Get cell boundaries
        x0 = vertical_guides[col]
        x1 = vertical_guides[col + 1]
        y0 = horizontal_guides[row]
        y1 = horizontal_guides[row + 1]

        # Create region using absolute coordinates
        if hasattr(target, "create_region"):
            if isinstance(target, Region):
                return target.create_region(x0, y0, x1, y1, relative=False)
            return target.create_region(x0, y0, x1, y1)

        try:
            page = _resolve_single_page(target)
        except (TypeError, ValueError) as exc:
            raise TypeError(f"Cannot create region on {type(target)}") from exc

        return page.create_region(x0, y0, x1, y1)

    def left_of(self, guide_index: int, obj: Optional[Union["Page", "Region"]] = None) -> "Region":
        """
        Get a region to the left of a vertical guide.

        Args:
            guide_index: Vertical guide index
            obj: Page or Region to create the region on (uses self.context if None)

        Returns:
            Region to the left of the specified guide
        """
        target = obj or self.context
        if target is None:
            raise ValueError("No context available for region creation")

        if not self.vertical or guide_index < 0 or guide_index >= len(self.vertical):
            raise IndexError(f"Guide index {guide_index} out of range")

        # Get bounds from context
        bounds = self._get_context_bounds()
        if not bounds:
            raise ValueError("Could not determine bounds")
        x0, y0, _, y1 = bounds

        # Create region from left edge to guide
        x1 = self.vertical[guide_index]

        if hasattr(target, "region"):
            return target.region(x0, y0, x1, y1)
        else:
            raise TypeError(f"Cannot create region on {type(target)}")

    def right_of(self, guide_index: int, obj: Optional[Union["Page", "Region"]] = None) -> "Region":
        """
        Get a region to the right of a vertical guide.

        Args:
            guide_index: Vertical guide index
            obj: Page or Region to create the region on (uses self.context if None)

        Returns:
            Region to the right of the specified guide
        """
        target = obj or self.context
        if target is None:
            raise ValueError("No context available for region creation")

        if not self.vertical or guide_index < 0 or guide_index >= len(self.vertical):
            raise IndexError(f"Guide index {guide_index} out of range")

        # Get bounds from context
        bounds = self._get_context_bounds()
        if not bounds:
            raise ValueError("Could not determine bounds")
        _, y0, x1, y1 = bounds

        # Create region from guide to right edge
        x0 = self.vertical[guide_index]

        if hasattr(target, "region"):
            return target.region(x0, y0, x1, y1)
        else:
            raise TypeError(f"Cannot create region on {type(target)}")

    def above(self, guide_index: int, obj: Optional[Union["Page", "Region"]] = None) -> "Region":
        """
        Get a region above a horizontal guide.

        Args:
            guide_index: Horizontal guide index
            obj: Page or Region to create the region on (uses self.context if None)

        Returns:
            Region above the specified guide
        """
        target = obj or self.context
        if target is None:
            raise ValueError("No context available for region creation")

        if not self.horizontal or guide_index < 0 or guide_index >= len(self.horizontal):
            raise IndexError(f"Guide index {guide_index} out of range")

        # Get bounds from context
        bounds = self._get_context_bounds()
        if not bounds:
            raise ValueError("Could not determine bounds")
        x0, y0, x1, _ = bounds

        # Create region from top edge to guide
        y1 = self.horizontal[guide_index]

        if hasattr(target, "region"):
            return target.region(x0, y0, x1, y1)
        else:
            raise TypeError(f"Cannot create region on {type(target)}")

    def below(self, guide_index: int, obj: Optional[Union["Page", "Region"]] = None) -> "Region":
        """
        Get a region below a horizontal guide.

        Args:
            guide_index: Horizontal guide index
            obj: Page or Region to create the region on (uses self.context if None)

        Returns:
            Region below the specified guide
        """
        target = obj or self.context
        if target is None:
            raise ValueError("No context available for region creation")

        if not self.horizontal or guide_index < 0 or guide_index >= len(self.horizontal):
            raise IndexError(f"Guide index {guide_index} out of range")

        # Get bounds from context
        bounds = self._get_context_bounds()
        if not bounds:
            raise ValueError("Could not determine bounds")
        x0, _, x1, y1 = bounds

        # Create region from guide to bottom edge
        y0 = self.horizontal[guide_index]

        if hasattr(target, "region"):
            return target.region(x0, y0, x1, y1)
        else:
            raise TypeError(f"Cannot create region on {type(target)}")

    def between_vertical(
        self, start_index: int, end_index: int, obj: Optional[Union["Page", "Region"]] = None
    ) -> "Region":
        """
        Get a region between two vertical guides.

        Args:
            start_index: Starting vertical guide index
            end_index: Ending vertical guide index
            obj: Page or Region to create the region on (uses self.context if None)

        Returns:
            Region between the specified guides
        """
        target = obj or self.context
        if target is None:
            raise ValueError("No context available for region creation")

        if not self.vertical:
            raise ValueError("No vertical guides available")
        if start_index < 0 or start_index >= len(self.vertical):
            raise IndexError(f"Start index {start_index} out of range")
        if end_index < 0 or end_index >= len(self.vertical):
            raise IndexError(f"End index {end_index} out of range")
        if start_index >= end_index:
            raise ValueError("Start index must be less than end index")

        # Get bounds from context
        bounds = self._get_context_bounds()
        if not bounds:
            raise ValueError("Could not determine bounds")
        _, y0, _, y1 = bounds

        # Get horizontal boundaries
        x0 = self.vertical[start_index]
        x1 = self.vertical[end_index]

        if hasattr(target, "region"):
            return target.region(x0, y0, x1, y1)
        else:
            raise TypeError(f"Cannot create region on {type(target)}")

    def between_horizontal(
        self, start_index: int, end_index: int, obj: Optional[Union["Page", "Region"]] = None
    ) -> "Region":
        """
        Get a region between two horizontal guides.

        Args:
            start_index: Starting horizontal guide index
            end_index: Ending horizontal guide index
            obj: Page or Region to create the region on (uses self.context if None)

        Returns:
            Region between the specified guides
        """
        target = obj or self.context
        if target is None:
            raise ValueError("No context available for region creation")

        if not self.horizontal:
            raise ValueError("No horizontal guides available")
        if start_index < 0 or start_index >= len(self.horizontal):
            raise IndexError(f"Start index {start_index} out of range")
        if end_index < 0 or end_index >= len(self.horizontal):
            raise IndexError(f"End index {end_index} out of range")
        if start_index >= end_index:
            raise ValueError("Start index must be less than end index")

        # Get bounds from context
        bounds = self._get_context_bounds()
        if not bounds:
            raise ValueError("Could not determine bounds")
        x0, _, x1, _ = bounds

        # Get vertical boundaries
        y0 = self.horizontal[start_index]
        y1 = self.horizontal[end_index]

        if hasattr(target, "region"):
            return target.region(x0, y0, x1, y1)
        else:
            raise TypeError(f"Cannot create region on {type(target)}")

    # -------------------------------------------------------------------------
    # Operations
    # -------------------------------------------------------------------------

    def __add__(self, other: "Guides") -> "Guides":
        """
        Combine two guide sets.

        Returns:
            New Guides object with combined coordinates
        """
        # Combine and deduplicate coordinates, ensuring Python floats
        combined_verticals = sorted([float(x) for x in set(self.vertical + other.vertical)])
        combined_horizontals = sorted([float(y) for y in set(self.horizontal + other.horizontal)])

        # Handle FlowRegion context merging
        new_context = self.context or other.context

        # If both are flow regions, we might need a more complex merge,
        # but for now, just picking one context is sufficient.

        # Create the new Guides object
        new_guides = Guides(
            verticals=combined_verticals,
            horizontals=combined_horizontals,
            context=new_context,
            bounds=self.bounds or other.bounds,
        )

        # If the new context is a FlowRegion, we need to rebuild the flow-related state
        if new_guides.is_flow_region:
            # Re-initialize flow guides from both sources
            # This is a simplification; a true merge would be more complex.
            # For now, we combine the flow_guides dictionaries.
            if hasattr(self, "_flow_guides"):
                new_guides._flow_guides.update(self._flow_guides)
            if hasattr(other, "_flow_guides"):
                new_guides._flow_guides.update(other._flow_guides)

            # Re-initialize unified views
            if hasattr(self, "_unified_vertical"):
                new_guides._unified_vertical.extend(self._unified_vertical)
            if hasattr(other, "_unified_vertical"):
                new_guides._unified_vertical.extend(other._unified_vertical)

            if hasattr(self, "_unified_horizontal"):
                new_guides._unified_horizontal.extend(self._unified_horizontal)
            if hasattr(other, "_unified_horizontal"):
                new_guides._unified_horizontal.extend(other._unified_horizontal)

            # Invalidate caches to force rebuild
            new_guides._vertical_cache = None
            new_guides._horizontal_cache = None

        return new_guides

    def show(self, on=None, **kwargs):
        """
        Display the guides overlaid on a page or region.

        Args:
            on: Page, Region, PIL Image, or string to display guides on.
                If None, uses self.context (the object guides were created from).
                If string 'page', uses the page from self.context.
            **kwargs: Additional arguments passed to to_image() if applicable.

        Returns:
            PIL Image with guides drawn on it.
        """
        # Handle FlowRegion case
        if self.is_flow_region and (on is None or on == self.context):
            if not self._flow_guides:
                raise ValueError("No guides to show for FlowRegion")

            # Get stacking parameters from kwargs or use defaults
            stack_direction = kwargs.get("stack_direction", "vertical")
            stack_gap = kwargs.get("stack_gap", 5)
            stack_background_color = kwargs.get("stack_background_color", (255, 255, 255))

            # First, render all constituent regions without guides to get base images
            base_images = []
            region_infos = []  # Store region info for guide coordinate mapping

            for region in list(self._flow_constituent_regions()):
                render_fn = getattr(region, "render", None)
                if render_fn is None:
                    raise AttributeError(f"Region {region} does not support rendering")
                img = render_fn(
                    resolution=kwargs.get("resolution", 150),
                    width=kwargs.get("width", None),
                    crop=True,
                )
                if img:
                    base_images.append(img)

                    scale_x = img.width / region.width
                    scale_y = img.height / region.height

                    region_infos.append(
                        {
                            "region": region,
                            "img_width": img.width,
                            "img_height": img.height,
                            "scale_x": scale_x,
                            "scale_y": scale_y,
                            "pdf_x0": region.x0,
                            "pdf_top": region.top,
                            "pdf_x1": region.x1,
                            "pdf_bottom": region.bottom,
                        }
                    )

            if not base_images:
                raise ValueError("Failed to render any images for FlowRegion")

            # Calculate final canvas size based on stacking direction
            if stack_direction == "vertical":
                final_width = max(img.width for img in base_images)
                final_height = (
                    sum(img.height for img in base_images) + (len(base_images) - 1) * stack_gap
                )
            else:  # horizontal
                final_width = (
                    sum(img.width for img in base_images) + (len(base_images) - 1) * stack_gap
                )
                final_height = max(img.height for img in base_images)

            # Create unified canvas
            canvas = Image.new("RGB", (final_width, final_height), stack_background_color)
            draw = ImageDraw.Draw(canvas)

            # Paste base images and track positions
            region_positions = []  # (region_info, paste_x, paste_y)

            if stack_direction == "vertical":
                current_y = 0
                for i, (img, info) in enumerate(zip(base_images, region_infos)):
                    paste_x = (final_width - img.width) // 2  # Center horizontally
                    canvas.paste(img, (paste_x, current_y))
                    region_positions.append((info, paste_x, current_y))
                    current_y += img.height + stack_gap
            else:  # horizontal
                current_x = 0
                for i, (img, info) in enumerate(zip(base_images, region_infos)):
                    paste_y = (final_height - img.height) // 2  # Center vertically
                    canvas.paste(img, (current_x, paste_y))
                    region_positions.append((info, current_x, paste_y))
                    current_x += img.width + stack_gap

            # Now draw guides on the unified canvas
            # Draw vertical guides (blue) - these extend through the full canvas height
            for v_coord in self.vertical:
                # Find which region(s) this guide intersects
                for info, paste_x, paste_y in region_positions:
                    if info["pdf_x0"] <= v_coord <= info["pdf_x1"]:
                        # This guide is within this region's x-bounds
                        # Convert PDF coordinate to pixel coordinate relative to the region
                        adjusted_x = v_coord - info["pdf_x0"]
                        pixel_x = adjusted_x * info["scale_x"] + paste_x

                        # Draw full-height line on canvas (not clipped to region)
                        if 0 <= pixel_x <= final_width:
                            x_pixel = int(pixel_x)
                            draw.line(
                                [(x_pixel, 0), (x_pixel, final_height - 1)],
                                fill=(0, 0, 255, 200),
                                width=2,
                            )
                        break  # Only draw once per guide

            # Draw horizontal guides (red) - these extend through the full canvas width
            for h_coord in self.horizontal:
                # Find which region(s) this guide intersects
                for info, paste_x, paste_y in region_positions:
                    if info["pdf_top"] <= h_coord <= info["pdf_bottom"]:
                        # This guide is within this region's y-bounds
                        # Convert PDF coordinate to pixel coordinate relative to the region
                        adjusted_y = h_coord - info["pdf_top"]
                        pixel_y = adjusted_y * info["scale_y"] + paste_y

                        # Draw full-width line on canvas (not clipped to region)
                        if 0 <= pixel_y <= final_height:
                            y_pixel = int(pixel_y)
                            draw.line(
                                [(0, y_pixel), (final_width - 1, y_pixel)],
                                fill=(255, 0, 0, 200),
                                width=2,
                            )
                        break  # Only draw once per guide

            return canvas

        # Original single-region logic follows...
        # Determine what to display guides on
        target = on if on is not None else self.context

        # Handle string shortcuts
        if isinstance(target, str):
            if target == "page":
                if self.context is None:
                    raise ValueError("Cannot resolve 'page' without a guides context")
                try:
                    target = _resolve_single_page(self.context)
                except (TypeError, ValueError) as exc:
                    raise ValueError(
                        "Cannot resolve 'page' from the current guides context"
                    ) from exc
            else:
                raise ValueError(f"Unknown string target: {target}. Only 'page' is supported.")

        if target is None:
            raise ValueError("No target specified and no context available for guides display")

        # Prepare kwargs for image generation
        image_kwargs = {}

        # Extract only the parameters that the new render() method accepts
        if "resolution" in kwargs:
            image_kwargs["resolution"] = kwargs["resolution"]
        if "width" in kwargs:
            image_kwargs["width"] = kwargs["width"]
        if "crop" in kwargs:
            image_kwargs["crop"] = kwargs["crop"]

        # If target is a region-like object, crop to just that region
        try:
            _resolve_single_page(target)
            target_is_single_page = True
        except (TypeError, ValueError):
            target_is_single_page = False

        if hasattr(target, "bbox") and target_is_single_page:
            image_kwargs["crop"] = True

        # Get base image
        if hasattr(target, "render"):
            rendered = cast(Any, target).render(**image_kwargs)
            if rendered is None:
                raise ValueError("Failed to generate base image")
            img = cast(Image.Image, rendered)
        elif hasattr(target, "mode") and hasattr(target, "size"):
            # It's already a PIL Image
            img = cast(Image.Image, target)
        else:
            raise ValueError(f"Object {target} does not support render() and is not a PIL Image")

        # Create a copy to draw on
        img = cast(Image.Image, img.copy())
        draw = ImageDraw.Draw(img)

        # Determine scale factor for coordinate conversion
        if _has_size(target) and not (hasattr(target, "mode") and hasattr(target, "size")):
            # target is a PDF object (Page/Region) with PDF coordinates
            size_target = cast(_SupportsSize, target)
            scale_x = img.width / size_target.width
            scale_y = img.height / size_target.height

            # If we're showing guides on a region, we need to adjust coordinates
            # to be relative to the region's origin
            if hasattr(target, "bbox") and target_is_single_page:
                # This is a Region - adjust guide coordinates to be relative to region
                region_like = cast(Any, target)
                region_x0 = float(getattr(region_like, "x0", 0.0))
                region_top = float(getattr(region_like, "top", 0.0))
            else:
                # This is a Page - no adjustment needed
                region_x0, region_top = 0, 0
        else:
            # target is already an image, no scaling needed
            scale_x = 1.0
            scale_y = 1.0
            region_x0, region_top = 0, 0

        # Draw vertical guides (blue)
        for x_coord in self.vertical:
            # Adjust coordinate if we're showing on a region
            adjusted_x = x_coord - region_x0
            pixel_x = adjusted_x * scale_x
            # Ensure guides at the edge are still visible by clamping to valid range
            if 0 <= pixel_x <= img.width - 1:
                x_pixel = int(min(pixel_x, img.width - 1))
                draw.line([(x_pixel, 0), (x_pixel, img.height - 1)], fill=(0, 0, 255, 200), width=2)

        # Draw horizontal guides (red)
        for y_coord in self.horizontal:
            # Adjust coordinate if we're showing on a region
            adjusted_y = y_coord - region_top
            pixel_y = adjusted_y * scale_y
            # Ensure guides at the edge are still visible by clamping to valid range
            if 0 <= pixel_y <= img.height - 1:
                y_pixel = int(min(pixel_y, img.height - 1))
                draw.line([(0, y_pixel), (img.width - 1, y_pixel)], fill=(255, 0, 0, 200), width=2)

        return img

    # -------------------------------------------------------------------------
    # Utility Methods
    # -------------------------------------------------------------------------

    def get_cells(self) -> List[Tuple[float, float, float, float]]:
        """
        Get all cell bounding boxes from guide intersections.

        Returns:
            List of (x0, y0, x1, y1) tuples for each cell
        """
        cells = []

        # Create cells from guide intersections
        for i in range(len(self.vertical) - 1):
            for j in range(len(self.horizontal) - 1):
                x0 = self.vertical[i]
                x1 = self.vertical[i + 1]
                y0 = self.horizontal[j]
                y1 = self.horizontal[j + 1]
                cells.append((x0, y0, x1, y1))

        return cells

    def to_dict(self) -> Dict[str, Any]:
        """
        Convert to dictionary format suitable for pdfplumber table_settings.

        Returns:
            Dictionary with explicit_vertical_lines and explicit_horizontal_lines
        """
        return {
            "explicit_vertical_lines": self.vertical,
            "explicit_horizontal_lines": self.horizontal,
        }

    def to_relative(self) -> "Guides":
        """
        Convert absolute coordinates to relative (0-1) coordinates.

        Returns:
            New Guides object with relative coordinates
        """
        if self.relative:
            return self  # Already relative

        if not self.bounds:
            raise ValueError("Cannot convert to relative without bounds")

        x0, y0, x1, y1 = self.bounds
        width = x1 - x0
        height = y1 - y0

        rel_verticals = [(x - x0) / width for x in self.vertical]
        rel_horizontals = [(y - y0) / height for y in self.horizontal]

        return Guides(
            verticals=rel_verticals,
            horizontals=rel_horizontals,
            context=self.context,
            bounds=(0, 0, 1, 1),
            relative=True,
        )

    def to_absolute(self, bounds: Tuple[float, float, float, float]) -> "Guides":
        """
        Convert relative coordinates to absolute coordinates.

        Args:
            bounds: Target bounding box (x0, y0, x1, y1)

        Returns:
            New Guides object with absolute coordinates
        """
        if not self.relative:
            return self  # Already absolute

        x0, y0, x1, y1 = bounds
        width = x1 - x0
        height = y1 - y0

        abs_verticals = [x0 + x * width for x in self.vertical]
        abs_horizontals = [y0 + y * height for y in self.horizontal]

        return Guides(
            verticals=abs_verticals,
            horizontals=abs_horizontals,
            context=self.context,
            bounds=bounds,
            relative=False,
        )

    @property
    def n_rows(self) -> int:
        """Number of rows defined by horizontal guides."""
        return max(0, len(self.horizontal) - 1)

    @property
    def n_cols(self) -> int:
        """Number of columns defined by vertical guides."""
        return max(0, len(self.vertical) - 1)

    def _handle_snap_failure(self, message: str):
        """Handle cases where snapping cannot be performed."""
        behavior = getattr(self, "snap_behavior", getattr(self, "on_no_snap", "warn"))
        if behavior == "warn":
            logger.warning(message)
        elif behavior == "raise":
            raise ValueError(message)
        # 'ignore' case: do nothing

    def _optimal_guide_assignment(
        self, guides: List[float], trough_ranges: List[Tuple[float, float]]
    ) -> Dict[int, int]:
        """
        Assign guides to trough ranges using the user's desired logic:
        - Guides already in a trough stay put
        - Only guides NOT in any trough get moved to available troughs
        - Prefer closest assignment for guides that need to move
        """
        if not guides or not trough_ranges:
            return {}

        assignments = {}

        # Step 1: Identify which guides are already in troughs
        guides_in_troughs = set()
        for i, guide_pos in enumerate(guides):
            for trough_start, trough_end in trough_ranges:
                if trough_start <= guide_pos <= trough_end:
                    guides_in_troughs.add(i)
                    logger.debug(
                        f"Guide {i} (pos {guide_pos:.1f}) is already in trough ({trough_start:.1f}-{trough_end:.1f}), keeping in place"
                    )
                    break

        # Step 2: Identify which troughs are already occupied
        occupied_troughs = set()
        for i in guides_in_troughs:
            guide_pos = guides[i]
            for j, (trough_start, trough_end) in enumerate(trough_ranges):
                if trough_start <= guide_pos <= trough_end:
                    occupied_troughs.add(j)
                    break

        # Step 3: Find guides that need reassignment (not in any trough)
        guides_to_move = []
        for i, guide_pos in enumerate(guides):
            if i not in guides_in_troughs:
                guides_to_move.append(i)
                logger.debug(
                    f"Guide {i} (pos {guide_pos:.1f}) is NOT in any trough, needs reassignment"
                )

        # Step 4: Find available troughs (not occupied by existing guides)
        available_troughs = []
        for j, (trough_start, trough_end) in enumerate(trough_ranges):
            if j not in occupied_troughs:
                available_troughs.append(j)
                logger.debug(f"Trough {j} ({trough_start:.1f}-{trough_end:.1f}) is available")

        # Step 5: Assign guides to move to closest available troughs
        if guides_to_move and available_troughs:
            # Calculate distances for all combinations
            distances = []
            for guide_idx in guides_to_move:
                guide_pos = guides[guide_idx]
                for trough_idx in available_troughs:
                    trough_start, trough_end = trough_ranges[trough_idx]
                    trough_center = (trough_start + trough_end) / 2
                    distance = abs(guide_pos - trough_center)
                    distances.append((distance, guide_idx, trough_idx))

            # Sort by distance and assign greedily
            distances.sort()
            used_troughs = set()

            for distance, guide_idx, trough_idx in distances:
                if guide_idx not in assignments and trough_idx not in used_troughs:
                    assignments[guide_idx] = trough_idx
                    used_troughs.add(trough_idx)
                    logger.debug(
                        f"Assigned guide {guide_idx} (pos {guides[guide_idx]:.1f}) to trough {trough_idx} (distance: {distance:.1f})"
                    )

        logger.debug(f"Final assignments: {assignments}")
        return assignments

    def _snap_guides_to_gaps(self, guides: List[float], gaps: List[Tuple[float, float]], axis: str):
        """
        Snap guides to nearby gaps using optimal assignment.
        Only moves guides that are NOT already in a trough.
        """
        if not guides or not gaps:
            return

        logger.debug(f"Snapping {len(guides)} {axis} guides to {len(gaps)} trough ranges")
        for i, (start, end) in enumerate(gaps):
            center = (start + end) / 2
            logger.debug(f"  Trough {i}: {start:.1f} to {end:.1f} (center: {center:.1f})")

        # Get optimal assignments
        assignments = self._optimal_guide_assignment(guides, gaps)

        # Apply assignments (modify guides list in-place)
        for guide_idx, trough_idx in assignments.items():
            trough_start, trough_end = gaps[trough_idx]
            new_pos = (trough_start + trough_end) / 2  # Move to trough center
            old_pos = guides[guide_idx]
            guides[guide_idx] = new_pos
            logger.info(f"Snapped {axis} guide from {old_pos:.1f} to {new_pos:.1f}")

    def build_grid(
        self,
        target: Optional[GuidesContext] = None,
        source: str = "guides",
        cell_padding: float = 0.5,
        include_outer_boundaries: bool = False,
        *,
        multi_page: Literal["auto", True, False] = "auto",
    ) -> Dict[str, Any]:
        """
        Create table structure (table, rows, columns, cells) from guide coordinates.

        Args:
            target: Page or Region to create regions on (uses self.context if None)
            source: Source label for created regions (for identification)
            cell_padding: Internal padding for cell regions in points
            include_outer_boundaries: Whether to add boundaries at edges if missing
            multi_page: Controls multi-region table creation for FlowRegions.
                - "auto": (default) Creates a unified grid if there are multiple regions or guides span pages.
                - True: Forces creation of a unified multi-region grid.
                - False: Creates separate grids for each region.

        Returns:
            Dictionary with 'counts' and 'regions' created.
        """
        # Dispatch to appropriate implementation based on context and flags
        if self.is_flow_region:
            # Check if we should create a unified multi-region grid
            has_multiple_regions = len(self._flow_constituent_regions()) > 1
            spans_pages = self._spans_pages()

            # Create unified grid if:
            # - multi_page is explicitly True, OR
            # - multi_page is "auto" AND (spans pages OR has multiple regions)
            if multi_page is True or (
                multi_page == "auto" and (spans_pages or has_multiple_regions)
            ):
                return self._build_grid_multi_page(
                    source=source,
                    cell_padding=cell_padding,
                    include_outer_boundaries=include_outer_boundaries,
                )
            else:
                # Single region FlowRegion or multi_page=False: create separate tables per region
                total_counts = {"table": 0, "rows": 0, "columns": 0, "cells": 0}
                all_regions = {"table": [], "rows": [], "columns": [], "cells": []}

                for region in self._flow_constituent_regions():
                    if region in self._flow_guides:
                        verticals, horizontals = self._flow_guides[region]

                        region_guides = Guides(
                            verticals=verticals, horizontals=horizontals, context=region
                        )

                        result = region_guides._build_grid_single_page(
                            target=region,
                            source=source,
                            cell_padding=cell_padding,
                            include_outer_boundaries=include_outer_boundaries,
                        )

                        for key in total_counts:
                            total_counts[key] += result["counts"][key]

                        if result["regions"]["table"]:
                            all_regions["table"].append(result["regions"]["table"])
                        all_regions["rows"].extend(result["regions"]["rows"])
                        all_regions["columns"].extend(result["regions"]["columns"])
                        all_regions["cells"].extend(result["regions"]["cells"])

                logger.info(
                    f"Created {total_counts['table']} tables, {total_counts['rows']} rows, "
                    f"{total_counts['columns']} columns, and {total_counts['cells']} cells "
                    f"from guides across {len(self._flow_guides)} regions"
                )

                return {"counts": total_counts, "regions": all_regions}

        # Fallback for single page/region
        return self._build_grid_single_page(
            target=target,
            source=source,
            cell_padding=cell_padding,
            include_outer_boundaries=include_outer_boundaries,
        )

    def _build_grid_multi_page(
        self,
        source: str,
        cell_padding: float,
        include_outer_boundaries: bool,
    ) -> Dict[str, Any]:
        """
        Builds a single, coherent grid across multiple regions of a FlowRegion.

        Creates physical Region objects for each constituent region with _fragment
        region types (e.g., table_column_fragment), then stitches them into logical
        FlowRegion objects. Both are registered with pages, but the fragment types
        allow easy differentiation:
        - find_all('table_column') returns only logical columns
        - find_all('table_column_fragment') returns only physical fragments
        """
        from natural_pdf.flows.region import FlowRegion

        if not self.is_flow_region:
            raise ValueError("Multi-page grid building requires a FlowRegion with a valid Flow.")
        flow_context = self._flow_context()
        if not getattr(flow_context, "flow", None):
            raise ValueError("Multi-page grid building requires a FlowRegion with a valid Flow.")

        orientation = self._get_flow_orientation()
        adapter = FlowGuideAdapter(self)
        region_grids = adapter.build_region_grids(source=source, cell_padding=cell_padding)

        if not region_grids:
            return {
                "counts": {"table": 0, "rows": 0, "columns": 0, "cells": 0},
                "regions": {"table": None, "rows": [], "columns": [], "cells": []},
            }

        flow_region = self._flow_context()
        flow = flow_region.flow

        physical_tables: List[Any] = [grid.table for grid in region_grids if grid.table is not None]
        flattened_tables = adapter._flatten_region_likes(physical_tables)
        multi_page_table = FlowRegion(
            flow=flow, constituent_regions=flattened_tables, source_flow_element=None
        )
        multi_page_table.source = source
        multi_page_table.region_type = "table"
        multi_page_table.metadata.update(
            {"is_multi_page": True, "num_rows": self.n_rows, "num_cols": self.n_cols}
        )

        final_rows, final_cols, final_cells = adapter.stitch_region_results(
            region_grids, orientation, source
        )

        constituent_pages = collect_constituent_pages(self._flow_constituent_regions())
        register_regions_with_pages(
            constituent_pages,
            multi_page_table,
            final_rows,
            final_cols,
            final_cells,
            log=logger,
        )

        final_counts = {
            "table": 1,
            "rows": len(final_rows),
            "columns": len(final_cols),
            "cells": len(final_cells),
        }
        final_regions = {
            "table": multi_page_table,
            "rows": final_rows,
            "columns": final_cols,
            "cells": final_cells,
        }

        logger.info(
            f"Created 1 multi-page table, {final_counts['rows']} logical rows, "
            f"{final_counts['columns']} logical columns from guides and registered with all constituent pages"
        )

        return {"counts": final_counts, "regions": final_regions}

    def _build_grid_single_page(
        self,
        target: Optional[GuidesContext] = None,
        source: str = "guides",
        cell_padding: float = 0.5,
        include_outer_boundaries: bool = False,
    ) -> Dict[str, Any]:
        """
        Private method to create table structure on a single page or region.
        (Refactored from the original public build_grid method).
        """
        # This method now only handles a single page/region context.
        # Looping for FlowRegions is handled by the public `build_grid` method.

        # Original single-region logic follows...
        target_obj = target or self.context
        if not target_obj:
            raise ValueError("No target object available. Provide target parameter or context.")
        if _is_flow_region(target_obj):
            raise ValueError(
                "FlowRegion targets require multi-page handling – call build_grid with multi_page=True."
            )

        bounds = _require_bounds(target_obj, context="grid target")
        origin_x, origin_y, max_x, max_y = bounds
        context_width = max_x - origin_x
        context_height = max_y - origin_y

        from natural_pdf.core.page import Page

        page: Page
        if isinstance(target_obj, Page):
            page = target_obj
        elif isinstance(target_obj, Region):
            if target_obj.page is None:
                raise ValueError("Region is not associated with a page")
            page = target_obj.page
        else:
            try:
                page = _resolve_single_page(target_obj)
            except (TypeError, ValueError) as exc:
                raise ValueError(
                    f"Target object {target_obj} does not expose a single page reference"
                ) from exc
            if not isinstance(page, Page):
                raise ValueError("Resolved page does not implement the Page interface")

        if not hasattr(page, "add_element") or not hasattr(page, "remove_element"):
            raise ValueError("Target page does not expose element registration helpers")

        # Setup boundaries
        row_boundaries = [float(v) for v in self.horizontal]
        col_boundaries = [float(v) for v in self.vertical]

        # Add outer boundaries if requested and missing
        if include_outer_boundaries:
            if not row_boundaries or row_boundaries[0] > origin_y:
                row_boundaries.insert(0, origin_y)
            if not row_boundaries or row_boundaries[-1] < origin_y + context_height:
                row_boundaries.append(origin_y + context_height)

            if not col_boundaries or col_boundaries[0] > origin_x:
                col_boundaries.insert(0, origin_x)
            if not col_boundaries or col_boundaries[-1] < origin_x + context_width:
                col_boundaries.append(origin_x + context_width)

        # Remove duplicates and sort
        row_boundaries = sorted(list(set(row_boundaries)))
        col_boundaries = sorted(list(set(col_boundaries)))

        def _collect_regions_from_page_obj(page_obj: Any) -> List[Any]:
            iterator = getattr(page_obj, "iter_regions", None)
            regions: Optional[Iterable[Any]]
            if callable(iterator):
                regions = cast(Optional[Iterable[Any]], iterator())
            else:
                regions = cast(Optional[Iterable[Any]], iterator)

            if regions is None:
                return []
            if not isinstance(regions, IterableABC):
                return []
            return list(regions)

        # ------------------------------------------------------------------
        # Clean-up: remove any previously created grid regions (table, rows,
        # columns, cells) that were generated by the same `source` label and
        # overlap the area we are about to populate.  This prevents the page's
        # `ElementManager` from accumulating stale/duplicate regions when the
        # user rebuilds the grid multiple times.
        # ------------------------------------------------------------------
        if row_boundaries and col_boundaries:
            grid_bbox = (
                col_boundaries[0],  # x0
                row_boundaries[0],  # top
                col_boundaries[-1],  # x1
                row_boundaries[-1],  # bottom
            )

            def _bbox_overlap(b1, b2):
                """Return True if two (x0, top, x1, bottom) bboxes overlap."""
                return not (
                    b1[2] <= b2[0]  # b1 right ≤ b2 left
                    or b1[0] >= b2[2]  # b1 left ≥ b2 right
                    or b1[3] <= b2[1]  # b1 bottom ≤ b2 top
                    or b1[1] >= b2[3]  # b1 top ≥ b2 bottom
                )

            existing_regions = _collect_regions_from_page_obj(page)
            regions_to_remove = []
            for r in existing_regions:
                if getattr(r, "source", None) != source:
                    continue
                if getattr(r, "region_type", None) not in {
                    "table",
                    "table_row",
                    "table_column",
                    "table_cell",
                }:
                    continue
                if not hasattr(r, "bbox"):
                    continue
                if _bbox_overlap(r.bbox, grid_bbox):
                    regions_to_remove.append(r)

            for r in regions_to_remove:
                page.remove_element(r, element_type="regions")

            if regions_to_remove:
                logger.debug(
                    f"Removed {len(regions_to_remove)} existing grid region(s) prior to rebuild"
                )

        logger.debug(
            f"Building grid with {len(row_boundaries)} row and {len(col_boundaries)} col boundaries"
        )

        # Track creation counts and regions
        counts = {"table": 0, "rows": 0, "columns": 0, "cells": 0}
        created_regions = {"table": None, "rows": [], "columns": [], "cells": []}

        # Create overall table region
        if len(row_boundaries) >= 2 and len(col_boundaries) >= 2:
            table_region = page.create_region(
                col_boundaries[0], row_boundaries[0], col_boundaries[-1], row_boundaries[-1]
            )
            table_region.source = source
            table_region.region_type = "table"
            table_region.normalized_type = "table"
            table_region.metadata.update(
                {
                    "source_guides": True,
                    "num_rows": len(row_boundaries) - 1,
                    "num_cols": len(col_boundaries) - 1,
                    "boundaries": {"rows": row_boundaries, "cols": col_boundaries},
                }
            )
            page.add_region(table_region, source=source)
            counts["table"] = 1
            created_regions["table"] = table_region

        # Create row regions
        if len(row_boundaries) >= 2 and len(col_boundaries) >= 2:
            for i in range(len(row_boundaries) - 1):
                row_region = page.create_region(
                    col_boundaries[0], row_boundaries[i], col_boundaries[-1], row_boundaries[i + 1]
                )
                row_region.source = source
                row_region.region_type = "table_row"
                row_region.normalized_type = "table_row"
                row_region.metadata.update({"row_index": i, "source_guides": True})
                page.add_region(row_region, source=source)
                counts["rows"] += 1
                created_regions["rows"].append(row_region)

        # Create column regions
        if len(col_boundaries) >= 2 and len(row_boundaries) >= 2:
            for j in range(len(col_boundaries) - 1):
                col_region = page.create_region(
                    col_boundaries[j], row_boundaries[0], col_boundaries[j + 1], row_boundaries[-1]
                )
                col_region.source = source
                col_region.region_type = "table_column"
                col_region.normalized_type = "table_column"
                col_region.metadata.update({"col_index": j, "source_guides": True})
                page.add_region(col_region, source=source)
                counts["columns"] += 1
                created_regions["columns"].append(col_region)

        # Create cell regions
        if len(row_boundaries) >= 2 and len(col_boundaries) >= 2:
            for i in range(len(row_boundaries) - 1):
                for j in range(len(col_boundaries) - 1):
                    # Apply padding
                    cell_x0 = col_boundaries[j] + cell_padding
                    cell_top = row_boundaries[i] + cell_padding
                    cell_x1 = col_boundaries[j + 1] - cell_padding
                    cell_bottom = row_boundaries[i + 1] - cell_padding

                    # Skip invalid cells
                    if cell_x1 <= cell_x0 or cell_bottom <= cell_top:
                        continue

                    cell_region = page.create_region(cell_x0, cell_top, cell_x1, cell_bottom)
                    cell_region.source = source
                    cell_region.region_type = "table_cell"
                    cell_region.normalized_type = "table_cell"
                    cell_region.metadata.update(
                        {
                            "row_index": i,
                            "col_index": j,
                            "source_guides": True,
                            "original_boundaries": {
                                "left": col_boundaries[j],
                                "top": row_boundaries[i],
                                "right": col_boundaries[j + 1],
                                "bottom": row_boundaries[i + 1],
                            },
                        }
                    )
                    page.add_region(cell_region, source=source)
                    counts["cells"] += 1
                    created_regions["cells"].append(cell_region)

        logger.info(
            f"Created {counts['table']} table, {counts['rows']} rows, "
            f"{counts['columns']} columns, and {counts['cells']} cells from guides"
        )

        return {"counts": counts, "regions": created_regions}

    def __repr__(self) -> str:
        """String representation of the guides."""
        return (
            f"Guides(verticals={len(self.vertical)}, "
            f"horizontals={len(self.horizontal)}, "
            f"cells={len(self.get_cells())})"
        )

    def _spans_pages(self) -> bool:
        """Check if any guides are defined across multiple pages in a FlowRegion."""
        if not self.is_flow_region:
            return False

        # Check vertical guides
        v_guide_pages = {}
        for coord, region in self._unified_vertical:
            v_guide_pages.setdefault(coord, set()).add(region.page.page_number)

        for pages in v_guide_pages.values():
            if len(pages) > 1:
                return True

        # Check horizontal guides
        h_guide_pages = {}
        for coord, region in self._unified_horizontal:
            h_guide_pages.setdefault(coord, set()).add(region.page.page_number)

        for pages in h_guide_pages.values():
            if len(pages) > 1:
                return True

        return False

    # -------------------------------------------------------------------------
    # Instance methods for fluent chaining (avoid name conflicts with class methods)
    # -------------------------------------------------------------------------

    def add_content(
        self,
        axis: Literal["vertical", "horizontal"] = "vertical",
        markers: Union[str, List[str], "ElementCollection", None] = None,
        obj: Optional[Union["Page", "Region"]] = None,
        align: Literal["left", "right", "center", "between"] = "left",
        outer: OuterBoundaryMode = True,
        tolerance: float = 5,
        apply_exclusions: bool = True,
    ) -> "Guides":
        """
        Instance method: Add guides from content, allowing chaining.
        This allows: Guides.new(page).add_content(axis='vertical', markers=[...])

        Args:
            axis: Which axis to create guides for
            markers: Content to search for. Can be:
                - str: single selector or literal text
                - List[str]: list of selectors or literal text strings
                - ElementCollection: collection of elements to extract text from
                - None: no markers
            obj: Page or Region to search (uses self.context if None)
            align: How to align guides relative to found elements
            outer: Whether to add outer boundary guides. Can be:
                - bool: True/False to add/not add both
                - "first": To add boundary before the first element
                - "last": To add boundary before the last element
            tolerance: Tolerance for snapping to element edges
            apply_exclusions: Whether to apply exclusion zones when searching for text

        Returns:
            Self for method chaining
        """
        # Use provided object or fall back to stored context
        target_obj = obj or self.context
        if target_obj is None:
            raise ValueError("No object provided and no context available")

        # Create new guides using the class method
        new_guides = Guides.from_content(
            obj=target_obj,
            axis=axis,
            markers=markers,
            align=align,
            outer=outer,
            tolerance=tolerance,
            apply_exclusions=apply_exclusions,
        )

        # Add the appropriate coordinates to this object
        if axis == "vertical":
            self.vertical = list(set(self.vertical + new_guides.vertical))
        else:
            self.horizontal = list(set(self.horizontal + new_guides.horizontal))

        return self

    def add_lines(
        self,
        axis: Literal["vertical", "horizontal", "both"] = "both",
        obj: Optional[Union["Page", "Region"]] = None,
        threshold: Union[float, str] = "auto",
        source_label: Optional[str] = None,
        max_lines_h: Optional[int] = None,
        max_lines_v: Optional[int] = None,
        outer: bool = False,
        detection_method: str = "vector",
        resolution: int = 192,
        **detect_kwargs,
    ) -> "Guides":
        """
        Instance method: Add guides from lines, allowing chaining.
        This allows: Guides.new(page).add_lines(axis='horizontal')

        Args:
            axis: Which axis to detect lines for
            obj: Page or Region to search (uses self.context if None)
            threshold: Line detection threshold ('auto' or float 0.0-1.0)
            source_label: Filter lines by source label (vector) or label for detected lines (pixels)
            max_lines_h: Maximum horizontal lines to use
            max_lines_v: Maximum vertical lines to use
            outer: Whether to add outer boundary guides
            detection_method: 'vector', 'pixels', or 'auto' (default). 'auto' uses vector line
                information when available and falls back to pixel detection otherwise.
            resolution: DPI for pixel-based detection (default: 192)
            **detect_kwargs: Additional parameters for pixel detection (see from_lines)

        Returns:
            Self for method chaining
        """
        # Use provided object or fall back to stored context
        target_obj = obj or self.context
        if target_obj is None:
            raise ValueError("No object provided and no context available")

        # Create new guides using the class method
        new_guides = Guides.from_lines(
            obj=target_obj,
            axis=axis,
            threshold=threshold,
            source_label=source_label,
            max_lines_h=max_lines_h,
            max_lines_v=max_lines_v,
            outer=outer,
            detection_method=detection_method,
            resolution=resolution,
            **detect_kwargs,
        )

        # Add the appropriate coordinates to this object
        if axis in ("vertical", "both"):
            self.vertical = list(set(self.vertical + new_guides.vertical))
        if axis in ("horizontal", "both"):
            self.horizontal = list(set(self.horizontal + new_guides.horizontal))

        return self

    def add_whitespace(
        self,
        axis: Literal["vertical", "horizontal", "both"] = "both",
        obj: Optional[Union["Page", "Region"]] = None,
        min_gap: float = 10,
    ) -> "Guides":
        """
        Instance method: Add guides from whitespace, allowing chaining.
        This allows: Guides.new(page).add_whitespace(axis='both')

        Args:
            axis: Which axis to create guides for
            obj: Page or Region to search (uses self.context if None)
            min_gap: Minimum gap size to consider

        Returns:
            Self for method chaining
        """
        # Use provided object or fall back to stored context
        target_obj = obj or self.context
        if target_obj is None:
            raise ValueError("No object provided and no context available")

        # Create new guides using the class method
        new_guides = Guides.from_whitespace(obj=target_obj, axis=axis, min_gap=min_gap)

        # Add the appropriate coordinates to this object
        if axis in ("vertical", "both"):
            self.vertical = list(set(self.vertical + new_guides.vertical))
        if axis in ("horizontal", "both"):
            self.horizontal = list(set(self.horizontal + new_guides.horizontal))

        return self

    def extract_table(
        self,
        target: Optional[
            Union[
                "Page",
                "Region",
                "PageCollection",
                "ElementCollection",
                List[Union["Page", "Region"]],
            ]
        ] = None,
        source: str = "guides_temp",
        cell_padding: float = 0.5,
        include_outer_boundaries: bool = False,
        method: Optional[str] = None,
        table_settings: Optional[dict] = None,
        use_ocr: bool = False,
        ocr_config: Optional[dict] = None,
        text_options: Optional[Dict] = None,
        cell_extraction_func: Optional[Callable[["Region"], Optional[str]]] = None,
        show_progress: bool = False,
        content_filter: Optional[Union[str, Callable[[str], bool], List[str]]] = None,
        apply_exclusions: bool = True,
        *,
        multi_page: Literal["auto", True, False] = "auto",
        header: Union[str, List[str], None] = "first",
        skip_repeating_headers: Optional[bool] = None,
        structure_engine: Optional[str] = None,
    ) -> "TableResult":
        """
        Extract table data directly from guides without leaving temporary regions.

        This method:
        1. Creates table structure using build_grid()
        2. Extracts table data from the created table region
        3. Cleans up all temporary regions
        4. Returns the TableResult

        When passed a collection (PageCollection, ElementCollection, or list), this method
        will extract tables from each element and combine them into a single result.

        Args:
            target: Page, Region, or collection of Pages/Regions to extract from (uses self.context if None)
            source: Source label for temporary regions (will be cleaned up)
            cell_padding: Internal padding for cell regions in points
            include_outer_boundaries: Whether to add boundaries at edges if missing
            method: Table extraction method ('tatr', 'pdfplumber', 'text', etc.)
            table_settings: Settings for pdfplumber table extraction
            use_ocr: Whether to use OCR for text extraction
            ocr_config: OCR configuration parameters
            text_options: Dictionary of options for the 'text' method
            cell_extraction_func: Optional callable for custom cell text extraction
            show_progress: Controls progress bar for text method
            content_filter: Content filtering function or patterns
            apply_exclusions: Whether to apply exclusion regions during text extraction (default: True)
            multi_page: Controls multi-region table creation for FlowRegions
            header: How to handle headers when extracting from collections:
                - "first": Use first row of first element as headers (default)
                - "all": Expect headers on each element, use from first element
                - None: No headers, use numeric indices
                - List[str]: Custom column names
            skip_repeating_headers: Whether to remove duplicate header rows when extracting from collections.
                Defaults to True when header is "first" or "all", False otherwise.
            structure_engine: Optional structure detection engine name passed to the underlying
                region extraction to leverage provider-backed table structure results.

        Returns:
            TableResult: Extracted table data

        Raises:
            ValueError: If no table region is created from the guides

        Example:
            ```python
            from natural_pdf.analyzers import Guides

            # Single page extraction
            guides = Guides.from_lines(page, source_label="detected")
            table_data = guides.extract_table()
            df = table_data.to_df()

            # Multiple page extraction
            guides = Guides(pages[0])
            guides.vertical.from_content(['Column 1', 'Column 2'])
            table_result = guides.extract_table(pages, header=['Col1', 'Col2'])
            df = table_result.to_df()

            # Region collection extraction
            regions = pdf.find_all('region[type=table]')
            guides = Guides(regions[0])
            guides.vertical.from_lines(n=3)
            table_result = guides.extract_table(regions)
            ```
        """
        from natural_pdf.core.page_collection import PageCollection

        target_obj = target if target is not None else self.context
        if target_obj is None:
            raise ValueError("No target object available. Provide target parameter or context.")

        # Check if target is a collection - if so, delegate to _extract_table_from_collection
        if isinstance(target_obj, (PageCollection, ElementCollection, list)):
            # For collections, pass through most parameters as-is
            return self._extract_table_from_collection(
                elements=target_obj,
                header=header,
                skip_repeating_headers=skip_repeating_headers,
                method=method,
                table_settings=table_settings,
                use_ocr=use_ocr,
                ocr_config=ocr_config,
                text_options=text_options,
                cell_extraction_func=cell_extraction_func,
                show_progress=show_progress,
                content_filter=content_filter,
                apply_exclusions=apply_exclusions,
                structure_engine=structure_engine,
            )

        from natural_pdf.core.page import Page

        # Get the page for cleanup later
        if hasattr(target_obj, "x0") and hasattr(target_obj, "top"):  # Region
            page_obj = getattr(target_obj, "_page", None)
            if page_obj is None:
                raise ValueError("Region is not associated with a page")
            page = cast(Page, page_obj)
        elif hasattr(target_obj, "add_element"):  # Page-like
            page = cast(Page, target_obj)
        else:
            raise ValueError(f"Target object {target_obj} is not a Page or Region")

        # Check if we have guides in only one dimension
        has_verticals = len(self.vertical) > 0
        has_horizontals = len(self.horizontal) > 0

        # If we have guides in only one dimension, use direct extraction with explicit lines
        if (has_verticals and not has_horizontals) or (has_horizontals and not has_verticals):
            logger.debug(
                f"Partial guides detected - using direct extraction (v={has_verticals}, h={has_horizontals})"
            )

            # Extract directly from the target using explicit lines
            if hasattr(target_obj, "extract_table"):
                return target_obj.extract_table(
                    method=method,  # Let auto-detection work when None
                    table_settings=table_settings,
                    use_ocr=use_ocr,
                    ocr_config=ocr_config,
                    text_options=text_options,
                    cell_extraction_func=cell_extraction_func,
                    show_progress=show_progress,
                    content_filter=content_filter,
                    verticals=list(self.vertical) if has_verticals else None,
                    horizontals=list(self.horizontal) if has_horizontals else None,
                    structure_engine=structure_engine,
                )
            else:
                raise ValueError(f"Target object {type(target_obj)} does not support extract_table")

        # Both dimensions have guides - use normal grid-based extraction
        try:
            # Step 1: Build grid structure (creates temporary regions)
            grid_result = self.build_grid(
                target=target_obj,
                source=source,
                cell_padding=cell_padding,
                include_outer_boundaries=include_outer_boundaries,
                multi_page=multi_page,
            )

            # Step 2: Get the table region and extract table data
            table_region = grid_result["regions"]["table"]
            if table_region is None:
                raise ValueError(
                    "No table region was created from the guides. Check that you have both vertical and horizontal guides."
                )

            # Handle multi-page case where table_region might be a list
            if isinstance(table_region, list):
                if not table_region:
                    raise ValueError("No table regions were created from the guides.")
                # Use the first table region for extraction
                table_region = table_region[0]

            table_result = table_region.extract_table(
                method=method,
                table_settings=table_settings,
                use_ocr=use_ocr,
                ocr_config=ocr_config,
                text_options=text_options,
                cell_extraction_func=cell_extraction_func,
                show_progress=show_progress,
                content_filter=content_filter,
                apply_exclusions=apply_exclusions,
                structure_engine=structure_engine,
            )
            self._assign_headers_from_rows(table_result, header)
            return table_result

        finally:
            # Step 4: Clean up all temporary regions created by build_grid
            # This ensures no regions are left behind regardless of success/failure
            iter_regions = getattr(page, "iter_regions", None)
            raw_regions: Optional[Iterable[Any]]
            if callable(iter_regions):
                raw_regions = cast(Optional[Iterable[Any]], iter_regions())
            else:
                raw_regions = cast(Optional[Iterable[Any]], iter_regions)

            if raw_regions is None or not isinstance(raw_regions, IterableABC):
                existing_regions = []
            else:
                existing_regions = list(raw_regions)

            regions_to_remove = [
                r
                for r in existing_regions
                if getattr(r, "source", None) == source
                and getattr(r, "region_type", None)
                in {"table", "table_row", "table_column", "table_cell"}
            ]

            for region in regions_to_remove:
                page.remove_element(region, element_type="regions")

            if regions_to_remove:
                logger.debug("Cleaned up %d temporary regions", len(regions_to_remove))

    def _extract_table_from_collection(
        self,
        elements: Union["PageCollection", "ElementCollection", List[Union["Page", "Region"]]],
        header: Union[str, List[str], None] = "first",
        skip_repeating_headers: Optional[bool] = None,
        method: Optional[str] = None,
        table_settings: Optional[dict] = None,
        use_ocr: bool = False,
        ocr_config: Optional[dict] = None,
        text_options: Optional[Dict] = None,
        cell_extraction_func: Optional[Callable[["Region"], Optional[str]]] = None,
        show_progress: bool = True,
        content_filter: Optional[Union[str, Callable[[str], bool], List[str]]] = None,
        apply_exclusions: bool = True,
        structure_engine: Optional[str] = None,
    ) -> "TableResult":
        """
        Extract tables from multiple pages or regions using this guide pattern.

        This method applies the guide to each element, extracts tables, and combines
        them into a single TableResult. Dynamic guides (using lambdas) are evaluated
        for each element.

        Args:
            elements: PageCollection, ElementCollection, or list of Pages/Regions to extract from
            header: How to handle headers:
                - "first": Use first row of first element as headers (default)
                - "all": Expect headers on each element, use from first element
                - None: No headers, use numeric indices
                - List[str]: Custom column names
            skip_repeating_headers: Whether to remove duplicate header rows.
                Defaults to True when header is "first" or "all", False otherwise.
            method: Table extraction method (passed to extract_table)
            table_settings: Settings for pdfplumber table extraction
            use_ocr: Whether to use OCR for text extraction
            ocr_config: OCR configuration parameters
            text_options: Dictionary of options for the 'text' method
            cell_extraction_func: Optional callable for custom cell text extraction
            show_progress: Show progress bar for multi-element extraction (default: True)
            content_filter: Content filtering function or patterns
            apply_exclusions: Whether to apply exclusion regions during extraction
            structure_engine: Optional structure engine forwarded to each element extraction.

        Returns:
            TableResult: Combined table data from all elements

        Example:
            ```python
            # Create guide with static vertical, dynamic horizontal
            guide = Guides(regions[0])
            guide.vertical.from_content(columns, outer="last")
            guide.horizontal.from_content(lambda r: r.find_all('text:starts-with(NF-)'))

            # Extract from all regions
            table_result = guide._extract_table_from_collection(regions, header=columns)
            df = table_result.to_df()
            ```
        """
        from natural_pdf.core.page_collection import PageCollection
        from natural_pdf.tables.result import TableResult

        # Convert to list if it's a collection
        if isinstance(elements, (PageCollection, ElementCollection)):
            element_list = list(elements)
        else:
            element_list = elements

        if not element_list:
            return TableResult([])

        # Determine header handling
        if skip_repeating_headers is None:
            skip_repeating_headers = header in ["first", "all"] or isinstance(header, list)

        all_rows = []
        header_row = None

        # Configure progress bar
        iterator = element_list
        if show_progress and len(element_list) > 1:
            from tqdm.auto import tqdm

            iterator = tqdm(element_list, desc="Extracting tables from elements", unit="element")

        for i, element in enumerate(iterator):
            # Create a new Guides object for this element
            element_guide = Guides(element)

            # Copy vertical guides (usually static)
            if hasattr(self.vertical, "_callable") and self.vertical._callable is not None:
                # If vertical is dynamic (lambda), evaluate it
                element_guide.vertical.from_content(self.vertical._callable(element))
            else:
                # Copy static vertical positions
                element_guide.vertical.data = self.vertical.data.copy()

            # Handle horizontal guides
            if hasattr(self.horizontal, "_callable") and self.horizontal._callable is not None:
                # If horizontal is dynamic (lambda), evaluate it
                element_guide.horizontal.from_content(self.horizontal._callable(element))
            else:
                # Copy static horizontal positions
                element_guide.horizontal.data = self.horizontal.data.copy()

            # Extract table from this element
            table_result = element_guide.extract_table(
                method=method,
                table_settings=table_settings,
                use_ocr=use_ocr,
                ocr_config=ocr_config,
                text_options=text_options,
                cell_extraction_func=cell_extraction_func,
                show_progress=False,  # Don't show nested progress
                content_filter=content_filter,
                apply_exclusions=apply_exclusions,
                structure_engine=structure_engine,
            )

            # Convert to list of rows
            rows = list(table_result)

            # Handle headers based on strategy
            if i == 0:  # First element
                if header == "first" or header == "all":
                    # Use first row as header
                    if rows:
                        header_row = rows[0]
                        rows = rows[1:]  # Remove header from data
                elif isinstance(header, list):
                    # Custom headers provided
                    header_row = header
            else:  # Subsequent elements
                if header == "all" and skip_repeating_headers and rows:
                    # Expect and remove header row
                    if rows and header_row and rows[0] == header_row:
                        rows = rows[1:]
                    elif rows:
                        # Still remove first row if it looks like a header
                        rows = rows[1:]

            # Add rows to combined result
            all_rows.extend(rows)

        # Create final TableResult
        if isinstance(header, list):
            final_result = TableResult(all_rows)
            final_result.headers = header
        elif header_row is not None:
            final_result = TableResult([header_row] + all_rows)
            final_result.headers = header_row
        else:
            final_result = TableResult(all_rows)
        self._assign_headers_from_rows(final_result, header)
        return final_result

    @staticmethod
    def _assign_headers_from_rows(
        table_result: TableResult, header: Union[str, int, List[str], None]
    ) -> None:
        """Normalize headers on a TableResult and drop empty leading rows."""

        def _row_is_blank(row: Sequence[Any]) -> bool:
            return all(cell is None or (isinstance(cell, str) and not cell.strip()) for cell in row)

        rows = getattr(table_result, "_rows", None)
        if not isinstance(rows, list) or not rows or header in (None, False):
            return

        if isinstance(header, list):
            table_result.headers = header
            return

        # Remove leading empty rows that can appear due to outer boundaries
        while rows and _row_is_blank(rows[0]):
            rows.pop(0)

        if not rows:
            return

        header_row: Optional[List[Any]] = None
        if header == "first":
            header_row = list(rows[0])
        elif isinstance(header, int):
            idx = max(0, min(len(rows) - 1, header))
            header_row = list(rows[idx])

        if header_row:
            table_result.headers = header_row

    def _get_flow_orientation(self) -> Literal["vertical", "horizontal", "unknown"]:
        """Determines if a FlowRegion's constituent parts are arranged vertically or horizontally."""
        if not self.is_flow_region or len(self._flow_constituent_regions()) < 2:
            return "unknown"

        r1 = self._flow_constituent_regions()[0]
        r2 = self._flow_constituent_regions()[1]  # Compare first two regions

        if not r1.bbox or not r2.bbox:
            return "unknown"

        # Calculate non-overlapping distances.
        # This determines the primary direction of separation.
        x_dist = max(0, max(r1.x0, r2.x0) - min(r1.x1, r2.x1))
        y_dist = max(0, max(r1.top, r2.top) - min(r1.bottom, r2.bottom))

        if y_dist > x_dist:
            return "vertical"
        else:
            return "horizontal"

Attributes

natural_pdf.Guides.cells property

Access cells by index like guides.cells[row][col] or guides.cells[row, col].

natural_pdf.Guides.columns property

Access columns by index like guides.columns[0].

natural_pdf.Guides.horizontal property writable

Get horizontal guide coordinates.

natural_pdf.Guides.n_cols property

Number of columns defined by vertical guides.

natural_pdf.Guides.n_rows property

Number of rows defined by horizontal guides.

natural_pdf.Guides.rows property

Access rows by index like guides.rows[0].

natural_pdf.Guides.vertical property writable

Get vertical guide coordinates.

Functions

natural_pdf.Guides.__add__(other)

Combine two guide sets.

Returns:

Type	Description
Guides	New Guides object with combined coordinates

Source code in natural_pdf/analyzers/guides/base.py

def __add__(self, other: "Guides") -> "Guides":
    """
    Combine two guide sets.

    Returns:
        New Guides object with combined coordinates
    """
    # Combine and deduplicate coordinates, ensuring Python floats
    combined_verticals = sorted([float(x) for x in set(self.vertical + other.vertical)])
    combined_horizontals = sorted([float(y) for y in set(self.horizontal + other.horizontal)])

    # Handle FlowRegion context merging
    new_context = self.context or other.context

    # If both are flow regions, we might need a more complex merge,
    # but for now, just picking one context is sufficient.

    # Create the new Guides object
    new_guides = Guides(
        verticals=combined_verticals,
        horizontals=combined_horizontals,
        context=new_context,
        bounds=self.bounds or other.bounds,
    )

    # If the new context is a FlowRegion, we need to rebuild the flow-related state
    if new_guides.is_flow_region:
        # Re-initialize flow guides from both sources
        # This is a simplification; a true merge would be more complex.
        # For now, we combine the flow_guides dictionaries.
        if hasattr(self, "_flow_guides"):
            new_guides._flow_guides.update(self._flow_guides)
        if hasattr(other, "_flow_guides"):
            new_guides._flow_guides.update(other._flow_guides)

        # Re-initialize unified views
        if hasattr(self, "_unified_vertical"):
            new_guides._unified_vertical.extend(self._unified_vertical)
        if hasattr(other, "_unified_vertical"):
            new_guides._unified_vertical.extend(other._unified_vertical)

        if hasattr(self, "_unified_horizontal"):
            new_guides._unified_horizontal.extend(self._unified_horizontal)
        if hasattr(other, "_unified_horizontal"):
            new_guides._unified_horizontal.extend(other._unified_horizontal)

        # Invalidate caches to force rebuild
        new_guides._vertical_cache = None
        new_guides._horizontal_cache = None

    return new_guides

natural_pdf.Guides.__init__(verticals=None, horizontals=None, context=None, bounds=None, relative=False, snap_behavior='warn')

Initialize a Guides object.

Parameters:

Name	Type	Description	Default
verticals	Optional[Union[Iterable[float], GuidesContext]]	Iterable of x-coordinates for vertical guides, or a context object shorthand	None
horizontals	Optional[Iterable[float]]	Iterable of y-coordinates for horizontal guides	None
context	Optional[GuidesContext]	Object providing spatial context (page, region, flow, etc.)	None
bounds	Optional[Tuple[float, float, float, float]]	Bounding box (x0, top, x1, bottom) if context not provided	None
relative	bool	Whether coordinates are relative (0-1) or absolute	False
snap_behavior	Literal['raise', 'warn', 'ignore']	How to handle snapping conflicts ('raise', 'warn', or 'ignore')	'warn'

Source code in natural_pdf/analyzers/guides/base.py

def __init__(
    self,
    verticals: Optional[Union[Iterable[float], GuidesContext]] = None,
    horizontals: Optional[Iterable[float]] = None,
    context: Optional[GuidesContext] = None,
    bounds: Optional[Tuple[float, float, float, float]] = None,
    relative: bool = False,
    snap_behavior: Literal["raise", "warn", "ignore"] = "warn",
):
    """
    Initialize a Guides object.

    Args:
        verticals: Iterable of x-coordinates for vertical guides, or a context object shorthand
        horizontals: Iterable of y-coordinates for horizontal guides
        context: Object providing spatial context (page, region, flow, etc.)
        bounds: Bounding box (x0, top, x1, bottom) if context not provided
        relative: Whether coordinates are relative (0-1) or absolute
        snap_behavior: How to handle snapping conflicts ('raise', 'warn', or 'ignore')
    """
    context_obj = context
    vertical_seed: Iterable[float] = ()

    # Handle Guides(page) or Guides(flow_region) shorthand
    if (
        verticals is not None
        and horizontals is None
        and context_obj is None
        and _is_guides_context(verticals)
    ):
        context_obj = cast(GuidesContext, verticals)
    elif verticals is not None:
        vertical_seed = cast(Iterable[float], verticals)

    self.context = context_obj
    coerced_bounds = _bounds_from_object(bounds) if bounds is not None else None
    self.bounds: Optional[Bounds] = coerced_bounds
    self.relative = relative
    self.snap_behavior = snap_behavior
    # Backwards compatibility alias for legacy options
    self.on_no_snap = snap_behavior

    # Check if we're dealing with a FlowRegion
    self.is_flow_region = _is_flow_region(context_obj)

    # If FlowRegion, we'll store guides per constituent region
    if self.is_flow_region:
        self._flow_guides: Dict["Region", Tuple[List[float], List[float]]] = {}
        # For unified view across all regions
        self._unified_vertical: List[Tuple[float, "Region"]] = []
        self._unified_horizontal: List[Tuple[float, "Region"]] = []
        # Cache for sorted unique coordinates
        self._vertical_cache: Optional[List[float]] = None
        self._horizontal_cache: Optional[List[float]] = None

    # Initialize with GuidesList instances
    horizontal_seed: Iterable[float] = horizontals if horizontals is not None else ()
    self._vertical = GuidesList(
        self,
        "vertical",
        sorted([float(x) for x in vertical_seed]),
    )
    self._horizontal = GuidesList(
        self,
        "horizontal",
        sorted([float(y) for y in horizontal_seed]),
    )

    # Determine bounds from context if needed
    if self.bounds is None and self.context is not None:
        self.bounds = _bounds_from_object(self.context)

    # Convert relative to absolute if needed
    if self.relative and self.bounds is not None:
        x0, top, x1, bottom = self.bounds
        width = x1 - x0
        height = bottom - top

        self._vertical.data = [x0 + float(v) * width for v in self._vertical]
        self._horizontal.data = [top + float(h) * height for h in self._horizontal]
        self.relative = False

natural_pdf.Guides.__repr__()

String representation of the guides.

Source code in natural_pdf/analyzers/guides/base.py

def __repr__(self) -> str:
    """String representation of the guides."""
    return (
        f"Guides(verticals={len(self.vertical)}, "
        f"horizontals={len(self.horizontal)}, "
        f"cells={len(self.get_cells())})"
    )

natural_pdf.Guides.above(guide_index, obj=None)

Get a region above a horizontal guide.

Parameters:

Name	Type	Description	Default
guide_index	int	Horizontal guide index	required
obj	Optional[Union[Page, Region]]	Page or Region to create the region on (uses self.context if None)	None

Returns:

Type	Description
Region	Region above the specified guide

Source code in natural_pdf/analyzers/guides/base.py

def above(self, guide_index: int, obj: Optional[Union["Page", "Region"]] = None) -> "Region":
    """
    Get a region above a horizontal guide.

    Args:
        guide_index: Horizontal guide index
        obj: Page or Region to create the region on (uses self.context if None)

    Returns:
        Region above the specified guide
    """
    target = obj or self.context
    if target is None:
        raise ValueError("No context available for region creation")

    if not self.horizontal or guide_index < 0 or guide_index >= len(self.horizontal):
        raise IndexError(f"Guide index {guide_index} out of range")

    # Get bounds from context
    bounds = self._get_context_bounds()
    if not bounds:
        raise ValueError("Could not determine bounds")
    x0, y0, x1, _ = bounds

    # Create region from top edge to guide
    y1 = self.horizontal[guide_index]

    if hasattr(target, "region"):
        return target.region(x0, y0, x1, y1)
    else:
        raise TypeError(f"Cannot create region on {type(target)}")

natural_pdf.Guides.add_content(axis='vertical', markers=None, obj=None, align='left', outer=True, tolerance=5, apply_exclusions=True)

Instance method: Add guides from content, allowing chaining. This allows: Guides.new(page).add_content(axis='vertical', markers=[...])

Parameters:

Name	Type	Description	Default
axis	Literal['vertical', 'horizontal']	Which axis to create guides for	'vertical'
markers	Union[str, List[str], ElementCollection, None]	Content to search for. Can be: - str: single selector or literal text - List[str]: list of selectors or literal text strings - ElementCollection: collection of elements to extract text from - None: no markers	None
obj	Optional[Union[Page, Region]]	Page or Region to search (uses self.context if None)	None
align	Literal['left', 'right', 'center', 'between']	How to align guides relative to found elements	'left'
outer	OuterBoundaryMode	Whether to add outer boundary guides. Can be: - bool: True/False to add/not add both - "first": To add boundary before the first element - "last": To add boundary before the last element	True
tolerance	float	Tolerance for snapping to element edges	5
apply_exclusions	bool	Whether to apply exclusion zones when searching for text	True

Returns:

Type	Description
Guides	Self for method chaining

Source code in natural_pdf/analyzers/guides/base.py

def add_content(
    self,
    axis: Literal["vertical", "horizontal"] = "vertical",
    markers: Union[str, List[str], "ElementCollection", None] = None,
    obj: Optional[Union["Page", "Region"]] = None,
    align: Literal["left", "right", "center", "between"] = "left",
    outer: OuterBoundaryMode = True,
    tolerance: float = 5,
    apply_exclusions: bool = True,
) -> "Guides":
    """
    Instance method: Add guides from content, allowing chaining.
    This allows: Guides.new(page).add_content(axis='vertical', markers=[...])

    Args:
        axis: Which axis to create guides for
        markers: Content to search for. Can be:
            - str: single selector or literal text
            - List[str]: list of selectors or literal text strings
            - ElementCollection: collection of elements to extract text from
            - None: no markers
        obj: Page or Region to search (uses self.context if None)
        align: How to align guides relative to found elements
        outer: Whether to add outer boundary guides. Can be:
            - bool: True/False to add/not add both
            - "first": To add boundary before the first element
            - "last": To add boundary before the last element
        tolerance: Tolerance for snapping to element edges
        apply_exclusions: Whether to apply exclusion zones when searching for text

    Returns:
        Self for method chaining
    """
    # Use provided object or fall back to stored context
    target_obj = obj or self.context
    if target_obj is None:
        raise ValueError("No object provided and no context available")

    # Create new guides using the class method
    new_guides = Guides.from_content(
        obj=target_obj,
        axis=axis,
        markers=markers,
        align=align,
        outer=outer,
        tolerance=tolerance,
        apply_exclusions=apply_exclusions,
    )

    # Add the appropriate coordinates to this object
    if axis == "vertical":
        self.vertical = list(set(self.vertical + new_guides.vertical))
    else:
        self.horizontal = list(set(self.horizontal + new_guides.horizontal))

    return self

natural_pdf.Guides.add_horizontal(y)

Add a horizontal guide at the specified y-coordinate.

Source code in natural_pdf/analyzers/guides/base.py

def add_horizontal(self, y: float) -> "Guides":
    """Add a horizontal guide at the specified y-coordinate."""
    self.horizontal.append(y)
    self.horizontal = sorted(self.horizontal)
    return self

natural_pdf.Guides.add_lines(axis='both', obj=None, threshold='auto', source_label=None, max_lines_h=None, max_lines_v=None, outer=False, detection_method='vector', resolution=192, **detect_kwargs)

Instance method: Add guides from lines, allowing chaining. This allows: Guides.new(page).add_lines(axis='horizontal')

Parameters:

Name	Type	Description	Default
axis	Literal['vertical', 'horizontal', 'both']	Which axis to detect lines for	'both'
obj	Optional[Union[Page, Region]]	Page or Region to search (uses self.context if None)	None
threshold	Union[float, str]	Line detection threshold ('auto' or float 0.0-1.0)	'auto'
source_label	Optional[str]	Filter lines by source label (vector) or label for detected lines (pixels)	None
max_lines_h	Optional[int]	Maximum horizontal lines to use	None
max_lines_v	Optional[int]	Maximum vertical lines to use	None
outer	bool	Whether to add outer boundary guides	False
detection_method	str	'vector', 'pixels', or 'auto' (default). 'auto' uses vector line information when available and falls back to pixel detection otherwise.	'vector'
resolution	int	DPI for pixel-based detection (default: 192)	192
**detect_kwargs		Additional parameters for pixel detection (see from_lines)	{}

Returns:

Type	Description
Guides	Self for method chaining

Source code in natural_pdf/analyzers/guides/base.py

def add_lines(
    self,
    axis: Literal["vertical", "horizontal", "both"] = "both",
    obj: Optional[Union["Page", "Region"]] = None,
    threshold: Union[float, str] = "auto",
    source_label: Optional[str] = None,
    max_lines_h: Optional[int] = None,
    max_lines_v: Optional[int] = None,
    outer: bool = False,
    detection_method: str = "vector",
    resolution: int = 192,
    **detect_kwargs,
) -> "Guides":
    """
    Instance method: Add guides from lines, allowing chaining.
    This allows: Guides.new(page).add_lines(axis='horizontal')

    Args:
        axis: Which axis to detect lines for
        obj: Page or Region to search (uses self.context if None)
        threshold: Line detection threshold ('auto' or float 0.0-1.0)
        source_label: Filter lines by source label (vector) or label for detected lines (pixels)
        max_lines_h: Maximum horizontal lines to use
        max_lines_v: Maximum vertical lines to use
        outer: Whether to add outer boundary guides
        detection_method: 'vector', 'pixels', or 'auto' (default). 'auto' uses vector line
            information when available and falls back to pixel detection otherwise.
        resolution: DPI for pixel-based detection (default: 192)
        **detect_kwargs: Additional parameters for pixel detection (see from_lines)

    Returns:
        Self for method chaining
    """
    # Use provided object or fall back to stored context
    target_obj = obj or self.context
    if target_obj is None:
        raise ValueError("No object provided and no context available")

    # Create new guides using the class method
    new_guides = Guides.from_lines(
        obj=target_obj,
        axis=axis,
        threshold=threshold,
        source_label=source_label,
        max_lines_h=max_lines_h,
        max_lines_v=max_lines_v,
        outer=outer,
        detection_method=detection_method,
        resolution=resolution,
        **detect_kwargs,
    )

    # Add the appropriate coordinates to this object
    if axis in ("vertical", "both"):
        self.vertical = list(set(self.vertical + new_guides.vertical))
    if axis in ("horizontal", "both"):
        self.horizontal = list(set(self.horizontal + new_guides.horizontal))

    return self

natural_pdf.Guides.add_vertical(x)

Add a vertical guide at the specified x-coordinate.

Source code in natural_pdf/analyzers/guides/base.py

def add_vertical(self, x: float) -> "Guides":
    """Add a vertical guide at the specified x-coordinate."""
    self.vertical.append(x)
    self.vertical = sorted(self.vertical)
    return self

natural_pdf.Guides.add_whitespace(axis='both', obj=None, min_gap=10)

Instance method: Add guides from whitespace, allowing chaining. This allows: Guides.new(page).add_whitespace(axis='both')

Parameters:

Name	Type	Description	Default
axis	Literal['vertical', 'horizontal', 'both']	Which axis to create guides for	'both'
obj	Optional[Union[Page, Region]]	Page or Region to search (uses self.context if None)	None
min_gap	float	Minimum gap size to consider	10

Returns:

Type	Description
Guides	Self for method chaining

Source code in natural_pdf/analyzers/guides/base.py

def add_whitespace(
    self,
    axis: Literal["vertical", "horizontal", "both"] = "both",
    obj: Optional[Union["Page", "Region"]] = None,
    min_gap: float = 10,
) -> "Guides":
    """
    Instance method: Add guides from whitespace, allowing chaining.
    This allows: Guides.new(page).add_whitespace(axis='both')

    Args:
        axis: Which axis to create guides for
        obj: Page or Region to search (uses self.context if None)
        min_gap: Minimum gap size to consider

    Returns:
        Self for method chaining
    """
    # Use provided object or fall back to stored context
    target_obj = obj or self.context
    if target_obj is None:
        raise ValueError("No object provided and no context available")

    # Create new guides using the class method
    new_guides = Guides.from_whitespace(obj=target_obj, axis=axis, min_gap=min_gap)

    # Add the appropriate coordinates to this object
    if axis in ("vertical", "both"):
        self.vertical = list(set(self.vertical + new_guides.vertical))
    if axis in ("horizontal", "both"):
        self.horizontal = list(set(self.horizontal + new_guides.horizontal))

    return self

natural_pdf.Guides.below(guide_index, obj=None)

Get a region below a horizontal guide.

Parameters:

Name	Type	Description	Default
guide_index	int	Horizontal guide index	required
obj	Optional[Union[Page, Region]]	Page or Region to create the region on (uses self.context if None)	None

Returns:

Type	Description
Region	Region below the specified guide

Source code in natural_pdf/analyzers/guides/base.py

def below(self, guide_index: int, obj: Optional[Union["Page", "Region"]] = None) -> "Region":
    """
    Get a region below a horizontal guide.

    Args:
        guide_index: Horizontal guide index
        obj: Page or Region to create the region on (uses self.context if None)

    Returns:
        Region below the specified guide
    """
    target = obj or self.context
    if target is None:
        raise ValueError("No context available for region creation")

    if not self.horizontal or guide_index < 0 or guide_index >= len(self.horizontal):
        raise IndexError(f"Guide index {guide_index} out of range")

    # Get bounds from context
    bounds = self._get_context_bounds()
    if not bounds:
        raise ValueError("Could not determine bounds")
    x0, _, x1, y1 = bounds

    # Create region from guide to bottom edge
    y0 = self.horizontal[guide_index]

    if hasattr(target, "region"):
        return target.region(x0, y0, x1, y1)
    else:
        raise TypeError(f"Cannot create region on {type(target)}")

natural_pdf.Guides.between_horizontal(start_index, end_index, obj=None)

Get a region between two horizontal guides.

Parameters:

Name	Type	Description	Default
start_index	int	Starting horizontal guide index	required
end_index	int	Ending horizontal guide index	required
obj	Optional[Union[Page, Region]]	Page or Region to create the region on (uses self.context if None)	None

Returns:

Type	Description
Region	Region between the specified guides

Source code in natural_pdf/analyzers/guides/base.py

def between_horizontal(
    self, start_index: int, end_index: int, obj: Optional[Union["Page", "Region"]] = None
) -> "Region":
    """
    Get a region between two horizontal guides.

    Args:
        start_index: Starting horizontal guide index
        end_index: Ending horizontal guide index
        obj: Page or Region to create the region on (uses self.context if None)

    Returns:
        Region between the specified guides
    """
    target = obj or self.context
    if target is None:
        raise ValueError("No context available for region creation")

    if not self.horizontal:
        raise ValueError("No horizontal guides available")
    if start_index < 0 or start_index >= len(self.horizontal):
        raise IndexError(f"Start index {start_index} out of range")
    if end_index < 0 or end_index >= len(self.horizontal):
        raise IndexError(f"End index {end_index} out of range")
    if start_index >= end_index:
        raise ValueError("Start index must be less than end index")

    # Get bounds from context
    bounds = self._get_context_bounds()
    if not bounds:
        raise ValueError("Could not determine bounds")
    x0, _, x1, _ = bounds

    # Get vertical boundaries
    y0 = self.horizontal[start_index]
    y1 = self.horizontal[end_index]

    if hasattr(target, "region"):
        return target.region(x0, y0, x1, y1)
    else:
        raise TypeError(f"Cannot create region on {type(target)}")

natural_pdf.Guides.between_vertical(start_index, end_index, obj=None)

Get a region between two vertical guides.

Parameters:

Name	Type	Description	Default
start_index	int	Starting vertical guide index	required
end_index	int	Ending vertical guide index	required
obj	Optional[Union[Page, Region]]	Page or Region to create the region on (uses self.context if None)	None

Returns:

Type	Description
Region	Region between the specified guides

Source code in natural_pdf/analyzers/guides/base.py

def between_vertical(
    self, start_index: int, end_index: int, obj: Optional[Union["Page", "Region"]] = None
) -> "Region":
    """
    Get a region between two vertical guides.

    Args:
        start_index: Starting vertical guide index
        end_index: Ending vertical guide index
        obj: Page or Region to create the region on (uses self.context if None)

    Returns:
        Region between the specified guides
    """
    target = obj or self.context
    if target is None:
        raise ValueError("No context available for region creation")

    if not self.vertical:
        raise ValueError("No vertical guides available")
    if start_index < 0 or start_index >= len(self.vertical):
        raise IndexError(f"Start index {start_index} out of range")
    if end_index < 0 or end_index >= len(self.vertical):
        raise IndexError(f"End index {end_index} out of range")
    if start_index >= end_index:
        raise ValueError("Start index must be less than end index")

    # Get bounds from context
    bounds = self._get_context_bounds()
    if not bounds:
        raise ValueError("Could not determine bounds")
    _, y0, _, y1 = bounds

    # Get horizontal boundaries
    x0 = self.vertical[start_index]
    x1 = self.vertical[end_index]

    if hasattr(target, "region"):
        return target.region(x0, y0, x1, y1)
    else:
        raise TypeError(f"Cannot create region on {type(target)}")

natural_pdf.Guides.build_grid(target=None, source='guides', cell_padding=0.5, include_outer_boundaries=False, *, multi_page='auto')

Create table structure (table, rows, columns, cells) from guide coordinates.

Parameters:

Name	Type	Description	Default
target	Optional[GuidesContext]	Page or Region to create regions on (uses self.context if None)	None
source	str	Source label for created regions (for identification)	'guides'
cell_padding	float	Internal padding for cell regions in points	0.5
include_outer_boundaries	bool	Whether to add boundaries at edges if missing	False
multi_page	Literal['auto', True, False]	Controls multi-region table creation for FlowRegions. - "auto": (default) Creates a unified grid if there are multiple regions or guides span pages. - True: Forces creation of a unified multi-region grid. - False: Creates separate grids for each region.	'auto'

Returns:

Type	Description
Dict[str, Any]	Dictionary with 'counts' and 'regions' created.

Source code in natural_pdf/analyzers/guides/base.py

def build_grid(
    self,
    target: Optional[GuidesContext] = None,
    source: str = "guides",
    cell_padding: float = 0.5,
    include_outer_boundaries: bool = False,
    *,
    multi_page: Literal["auto", True, False] = "auto",
) -> Dict[str, Any]:
    """
    Create table structure (table, rows, columns, cells) from guide coordinates.

    Args:
        target: Page or Region to create regions on (uses self.context if None)
        source: Source label for created regions (for identification)
        cell_padding: Internal padding for cell regions in points
        include_outer_boundaries: Whether to add boundaries at edges if missing
        multi_page: Controls multi-region table creation for FlowRegions.
            - "auto": (default) Creates a unified grid if there are multiple regions or guides span pages.
            - True: Forces creation of a unified multi-region grid.
            - False: Creates separate grids for each region.

    Returns:
        Dictionary with 'counts' and 'regions' created.
    """
    # Dispatch to appropriate implementation based on context and flags
    if self.is_flow_region:
        # Check if we should create a unified multi-region grid
        has_multiple_regions = len(self._flow_constituent_regions()) > 1
        spans_pages = self._spans_pages()

        # Create unified grid if:
        # - multi_page is explicitly True, OR
        # - multi_page is "auto" AND (spans pages OR has multiple regions)
        if multi_page is True or (
            multi_page == "auto" and (spans_pages or has_multiple_regions)
        ):
            return self._build_grid_multi_page(
                source=source,
                cell_padding=cell_padding,
                include_outer_boundaries=include_outer_boundaries,
            )
        else:
            # Single region FlowRegion or multi_page=False: create separate tables per region
            total_counts = {"table": 0, "rows": 0, "columns": 0, "cells": 0}
            all_regions = {"table": [], "rows": [], "columns": [], "cells": []}

            for region in self._flow_constituent_regions():
                if region in self._flow_guides:
                    verticals, horizontals = self._flow_guides[region]

                    region_guides = Guides(
                        verticals=verticals, horizontals=horizontals, context=region
                    )

                    result = region_guides._build_grid_single_page(
                        target=region,
                        source=source,
                        cell_padding=cell_padding,
                        include_outer_boundaries=include_outer_boundaries,
                    )

                    for key in total_counts:
                        total_counts[key] += result["counts"][key]

                    if result["regions"]["table"]:
                        all_regions["table"].append(result["regions"]["table"])
                    all_regions["rows"].extend(result["regions"]["rows"])
                    all_regions["columns"].extend(result["regions"]["columns"])
                    all_regions["cells"].extend(result["regions"]["cells"])

            logger.info(
                f"Created {total_counts['table']} tables, {total_counts['rows']} rows, "
                f"{total_counts['columns']} columns, and {total_counts['cells']} cells "
                f"from guides across {len(self._flow_guides)} regions"
            )

            return {"counts": total_counts, "regions": all_regions}

    # Fallback for single page/region
    return self._build_grid_single_page(
        target=target,
        source=source,
        cell_padding=cell_padding,
        include_outer_boundaries=include_outer_boundaries,
    )

natural_pdf.Guides.cell(row, col, obj=None)

Get a cell region from the guides.

Parameters:

Name	Type	Description	Default
row	int	Row index (0-based)	required
col	int	Column index (0-based)	required
obj	Optional[Union[Page, Region]]	Page or Region to create the cell on (uses self.context if None)	None

Returns:

Type	Description
Region	Region representing the specified cell

Raises:

Type	Description
IndexError	If row or column index is out of range

Source code in natural_pdf/analyzers/guides/base.py

def cell(self, row: int, col: int, obj: Optional[Union["Page", "Region"]] = None) -> "Region":
    """
    Get a cell region from the guides.

    Args:
        row: Row index (0-based)
        col: Column index (0-based)
        obj: Page or Region to create the cell on (uses self.context if None)

    Returns:
        Region representing the specified cell

    Raises:
        IndexError: If row or column index is out of range
    """
    target = obj or self.context
    if target is None:
        raise ValueError("No context available for region creation")

    vertical_guides = list(self.vertical.data)
    horizontal_guides = list(self.horizontal.data)
    if not vertical_guides or col < 0 or col >= len(vertical_guides) - 1:
        raise IndexError(
            f"Column index {col} out of range (have {len(vertical_guides)-1} columns)"
        )
    if not horizontal_guides or row < 0 or row >= len(horizontal_guides) - 1:
        raise IndexError(f"Row index {row} out of range (have {len(horizontal_guides)-1} rows)")

    # Get cell boundaries
    x0 = vertical_guides[col]
    x1 = vertical_guides[col + 1]
    y0 = horizontal_guides[row]
    y1 = horizontal_guides[row + 1]

    # Create region using absolute coordinates
    if hasattr(target, "create_region"):
        if isinstance(target, Region):
            return target.create_region(x0, y0, x1, y1, relative=False)
        return target.create_region(x0, y0, x1, y1)

    try:
        page = _resolve_single_page(target)
    except (TypeError, ValueError) as exc:
        raise TypeError(f"Cannot create region on {type(target)}") from exc

    return page.create_region(x0, y0, x1, y1)

natural_pdf.Guides.column(index, obj=None)

Get a column region from the guides.

Parameters:

Name	Type	Description	Default
index	int	Column index (0-based)	required
obj	Optional[Union[Page, Region]]	Page or Region to create the column on (uses self.context if None)	None

Returns:

Type	Description
Region	Region representing the specified column

Raises:

Type	Description
IndexError	If column index is out of range

Source code in natural_pdf/analyzers/guides/base.py

def column(self, index: int, obj: Optional[Union["Page", "Region"]] = None) -> "Region":
    """
    Get a column region from the guides.

    Args:
        index: Column index (0-based)
        obj: Page or Region to create the column on (uses self.context if None)

    Returns:
        Region representing the specified column

    Raises:
        IndexError: If column index is out of range
    """
    target = obj or self.context
    if target is None:
        raise ValueError("No context available for region creation")

    vertical_guides = list(self.vertical.data)
    if not vertical_guides or index < 0 or index >= len(vertical_guides) - 1:
        raise IndexError(
            f"Column index {index} out of range (have {len(vertical_guides)-1} columns)"
        )

    # Get bounds from context
    _, y0, _, y1 = self._get_context_bounds()

    # Get column boundaries
    x0 = vertical_guides[index]
    x1 = vertical_guides[index + 1]

    # Create region using absolute coordinates
    if hasattr(target, "create_region"):
        if isinstance(target, Region):
            return target.create_region(x0, y0, x1, y1, relative=False)
        return target.create_region(x0, y0, x1, y1)

    try:
        page = _resolve_single_page(target)
    except (TypeError, ValueError) as exc:
        raise TypeError(f"Cannot create region on {type(target)}") from exc

    return page.create_region(x0, y0, x1, y1)

natural_pdf.Guides.divide(obj, n=None, cols=None, rows=None, axis='both') classmethod

Create guides by evenly dividing an object.

Parameters:

Name	Type	Description	Default
obj	Union[Page, Region, Tuple[float, float, float, float]]	Object to divide (Page, Region, or bbox tuple)	required
n	Optional[int]	Number of divisions (creates n+1 guides). Used if cols/rows not specified.	None
cols	Optional[int]	Number of columns (creates cols+1 vertical guides)	None
rows	Optional[int]	Number of rows (creates rows+1 horizontal guides)	None
axis	Literal['vertical', 'horizontal', 'both']	Which axis to divide along	'both'

Returns:

Type	Description
Guides	New Guides object with evenly spaced lines

Examples:

Divide into 3 columns

guides = Guides.divide(page, cols=3)

Divide into 5 rows

guides = Guides.divide(region, rows=5)

Divide both axes

guides = Guides.divide(page, cols=3, rows=5)

Source code in natural_pdf/analyzers/guides/base.py

@classmethod
def divide(
    cls,
    obj: Union["Page", "Region", Tuple[float, float, float, float]],
    n: Optional[int] = None,
    cols: Optional[int] = None,
    rows: Optional[int] = None,
    axis: Literal["vertical", "horizontal", "both"] = "both",
) -> "Guides":
    """
    Create guides by evenly dividing an object.

    Args:
        obj: Object to divide (Page, Region, or bbox tuple)
        n: Number of divisions (creates n+1 guides). Used if cols/rows not specified.
        cols: Number of columns (creates cols+1 vertical guides)
        rows: Number of rows (creates rows+1 horizontal guides)
        axis: Which axis to divide along

    Returns:
        New Guides object with evenly spaced lines

    Examples:
        # Divide into 3 columns
        guides = Guides.divide(page, cols=3)

        # Divide into 5 rows
        guides = Guides.divide(region, rows=5)

        # Divide both axes
        guides = Guides.divide(page, cols=3, rows=5)
    """
    # Extract bounds from object
    if isinstance(obj, tuple) and len(obj) == 4:
        bounds = _ensure_bounds_tuple(obj)
        context = None
    else:
        context = obj
        bounds = _require_bounds(obj, context="object to divide")

    x0, y0, x1, y1 = bounds
    verticals = []
    horizontals = []

    # Handle vertical guides
    if axis in ("vertical", "both"):
        n_vertical = cols + 1 if cols is not None else (n + 1 if n is not None else 0)
        if n_vertical > 0:
            for i in range(n_vertical):
                x = x0 + (x1 - x0) * i / (n_vertical - 1)
                verticals.append(float(x))

    # Handle horizontal guides
    if axis in ("horizontal", "both"):
        n_horizontal = rows + 1 if rows is not None else (n + 1 if n is not None else 0)
        if n_horizontal > 0:
            for i in range(n_horizontal):
                y = y0 + (y1 - y0) * i / (n_horizontal - 1)
                horizontals.append(float(y))

    return cls(verticals=verticals, horizontals=horizontals, context=context, bounds=bounds)

natural_pdf.Guides.extract_table(target=None, source='guides_temp', cell_padding=0.5, include_outer_boundaries=False, method=None, table_settings=None, use_ocr=False, ocr_config=None, text_options=None, cell_extraction_func=None, show_progress=False, content_filter=None, apply_exclusions=True, *, multi_page='auto', header='first', skip_repeating_headers=None, structure_engine=None)

Extract table data directly from guides without leaving temporary regions.

This method: 1. Creates table structure using build_grid() 2. Extracts table data from the created table region 3. Cleans up all temporary regions 4. Returns the TableResult

When passed a collection (PageCollection, ElementCollection, or list), this method will extract tables from each element and combine them into a single result.

Parameters:

Name	Type	Description	Default
target	Optional[Union[Page, Region, PageCollection, ElementCollection, List[Union[Page, Region]]]]	Page, Region, or collection of Pages/Regions to extract from (uses self.context if None)	None
source	str	Source label for temporary regions (will be cleaned up)	'guides_temp'
cell_padding	float	Internal padding for cell regions in points	0.5
include_outer_boundaries	bool	Whether to add boundaries at edges if missing	False
method	Optional[str]	Table extraction method ('tatr', 'pdfplumber', 'text', etc.)	None
table_settings	Optional[dict]	Settings for pdfplumber table extraction	None
use_ocr	bool	Whether to use OCR for text extraction	False
ocr_config	Optional[dict]	OCR configuration parameters	None
text_options	Optional[Dict]	Dictionary of options for the 'text' method	None
cell_extraction_func	Optional[Callable[[Region], Optional[str]]]	Optional callable for custom cell text extraction	None
show_progress	bool	Controls progress bar for text method	False
content_filter	Optional[Union[str, Callable[[str], bool], List[str]]]	Content filtering function or patterns	None
apply_exclusions	bool	Whether to apply exclusion regions during text extraction (default: True)	True
multi_page	Literal['auto', True, False]	Controls multi-region table creation for FlowRegions	'auto'
header	Union[str, List[str], None]	How to handle headers when extracting from collections: - "first": Use first row of first element as headers (default) - "all": Expect headers on each element, use from first element - None: No headers, use numeric indices - List[str]: Custom column names	'first'
skip_repeating_headers	Optional[bool]	Whether to remove duplicate header rows when extracting from collections. Defaults to True when header is "first" or "all", False otherwise.	None
structure_engine	Optional[str]	Optional structure detection engine name passed to the underlying region extraction to leverage provider-backed table structure results.	None

Returns:

Name	Type	Description
TableResult	TableResult	Extracted table data

Raises:

Type	Description
ValueError	If no table region is created from the guides

Example

from natural_pdf.analyzers import Guides

# Single page extraction
guides = Guides.from_lines(page, source_label="detected")
table_data = guides.extract_table()
df = table_data.to_df()

# Multiple page extraction
guides = Guides(pages[0])
guides.vertical.from_content(['Column 1', 'Column 2'])
table_result = guides.extract_table(pages, header=['Col1', 'Col2'])
df = table_result.to_df()

# Region collection extraction
regions = pdf.find_all('region[type=table]')
guides = Guides(regions[0])
guides.vertical.from_lines(n=3)
table_result = guides.extract_table(regions)

Source code in natural_pdf/analyzers/guides/base.py

def extract_table(
    self,
    target: Optional[
        Union[
            "Page",
            "Region",
            "PageCollection",
            "ElementCollection",
            List[Union["Page", "Region"]],
        ]
    ] = None,
    source: str = "guides_temp",
    cell_padding: float = 0.5,
    include_outer_boundaries: bool = False,
    method: Optional[str] = None,
    table_settings: Optional[dict] = None,
    use_ocr: bool = False,
    ocr_config: Optional[dict] = None,
    text_options: Optional[Dict] = None,
    cell_extraction_func: Optional[Callable[["Region"], Optional[str]]] = None,
    show_progress: bool = False,
    content_filter: Optional[Union[str, Callable[[str], bool], List[str]]] = None,
    apply_exclusions: bool = True,
    *,
    multi_page: Literal["auto", True, False] = "auto",
    header: Union[str, List[str], None] = "first",
    skip_repeating_headers: Optional[bool] = None,
    structure_engine: Optional[str] = None,
) -> "TableResult":
    """
    Extract table data directly from guides without leaving temporary regions.

    This method:
    1. Creates table structure using build_grid()
    2. Extracts table data from the created table region
    3. Cleans up all temporary regions
    4. Returns the TableResult

    When passed a collection (PageCollection, ElementCollection, or list), this method
    will extract tables from each element and combine them into a single result.

    Args:
        target: Page, Region, or collection of Pages/Regions to extract from (uses self.context if None)
        source: Source label for temporary regions (will be cleaned up)
        cell_padding: Internal padding for cell regions in points
        include_outer_boundaries: Whether to add boundaries at edges if missing
        method: Table extraction method ('tatr', 'pdfplumber', 'text', etc.)
        table_settings: Settings for pdfplumber table extraction
        use_ocr: Whether to use OCR for text extraction
        ocr_config: OCR configuration parameters
        text_options: Dictionary of options for the 'text' method
        cell_extraction_func: Optional callable for custom cell text extraction
        show_progress: Controls progress bar for text method
        content_filter: Content filtering function or patterns
        apply_exclusions: Whether to apply exclusion regions during text extraction (default: True)
        multi_page: Controls multi-region table creation for FlowRegions
        header: How to handle headers when extracting from collections:
            - "first": Use first row of first element as headers (default)
            - "all": Expect headers on each element, use from first element
            - None: No headers, use numeric indices
            - List[str]: Custom column names
        skip_repeating_headers: Whether to remove duplicate header rows when extracting from collections.
            Defaults to True when header is "first" or "all", False otherwise.
        structure_engine: Optional structure detection engine name passed to the underlying
            region extraction to leverage provider-backed table structure results.

    Returns:
        TableResult: Extracted table data

    Raises:
        ValueError: If no table region is created from the guides

    Example:
        ```python
        from natural_pdf.analyzers import Guides

        # Single page extraction
        guides = Guides.from_lines(page, source_label="detected")
        table_data = guides.extract_table()
        df = table_data.to_df()

        # Multiple page extraction
        guides = Guides(pages[0])
        guides.vertical.from_content(['Column 1', 'Column 2'])
        table_result = guides.extract_table(pages, header=['Col1', 'Col2'])
        df = table_result.to_df()

        # Region collection extraction
        regions = pdf.find_all('region[type=table]')
        guides = Guides(regions[0])
        guides.vertical.from_lines(n=3)
        table_result = guides.extract_table(regions)
        ```
    """
    from natural_pdf.core.page_collection import PageCollection

    target_obj = target if target is not None else self.context
    if target_obj is None:
        raise ValueError("No target object available. Provide target parameter or context.")

    # Check if target is a collection - if so, delegate to _extract_table_from_collection
    if isinstance(target_obj, (PageCollection, ElementCollection, list)):
        # For collections, pass through most parameters as-is
        return self._extract_table_from_collection(
            elements=target_obj,
            header=header,
            skip_repeating_headers=skip_repeating_headers,
            method=method,
            table_settings=table_settings,
            use_ocr=use_ocr,
            ocr_config=ocr_config,
            text_options=text_options,
            cell_extraction_func=cell_extraction_func,
            show_progress=show_progress,
            content_filter=content_filter,
            apply_exclusions=apply_exclusions,
            structure_engine=structure_engine,
        )

    from natural_pdf.core.page import Page

    # Get the page for cleanup later
    if hasattr(target_obj, "x0") and hasattr(target_obj, "top"):  # Region
        page_obj = getattr(target_obj, "_page", None)
        if page_obj is None:
            raise ValueError("Region is not associated with a page")
        page = cast(Page, page_obj)
    elif hasattr(target_obj, "add_element"):  # Page-like
        page = cast(Page, target_obj)
    else:
        raise ValueError(f"Target object {target_obj} is not a Page or Region")

    # Check if we have guides in only one dimension
    has_verticals = len(self.vertical) > 0
    has_horizontals = len(self.horizontal) > 0

    # If we have guides in only one dimension, use direct extraction with explicit lines
    if (has_verticals and not has_horizontals) or (has_horizontals and not has_verticals):
        logger.debug(
            f"Partial guides detected - using direct extraction (v={has_verticals}, h={has_horizontals})"
        )

        # Extract directly from the target using explicit lines
        if hasattr(target_obj, "extract_table"):
            return target_obj.extract_table(
                method=method,  # Let auto-detection work when None
                table_settings=table_settings,
                use_ocr=use_ocr,
                ocr_config=ocr_config,
                text_options=text_options,
                cell_extraction_func=cell_extraction_func,
                show_progress=show_progress,
                content_filter=content_filter,
                verticals=list(self.vertical) if has_verticals else None,
                horizontals=list(self.horizontal) if has_horizontals else None,
                structure_engine=structure_engine,
            )
        else:
            raise ValueError(f"Target object {type(target_obj)} does not support extract_table")

    # Both dimensions have guides - use normal grid-based extraction
    try:
        # Step 1: Build grid structure (creates temporary regions)
        grid_result = self.build_grid(
            target=target_obj,
            source=source,
            cell_padding=cell_padding,
            include_outer_boundaries=include_outer_boundaries,
            multi_page=multi_page,
        )

        # Step 2: Get the table region and extract table data
        table_region = grid_result["regions"]["table"]
        if table_region is None:
            raise ValueError(
                "No table region was created from the guides. Check that you have both vertical and horizontal guides."
            )

        # Handle multi-page case where table_region might be a list
        if isinstance(table_region, list):
            if not table_region:
                raise ValueError("No table regions were created from the guides.")
            # Use the first table region for extraction
            table_region = table_region[0]

        table_result = table_region.extract_table(
            method=method,
            table_settings=table_settings,
            use_ocr=use_ocr,
            ocr_config=ocr_config,
            text_options=text_options,
            cell_extraction_func=cell_extraction_func,
            show_progress=show_progress,
            content_filter=content_filter,
            apply_exclusions=apply_exclusions,
            structure_engine=structure_engine,
        )
        self._assign_headers_from_rows(table_result, header)
        return table_result

    finally:
        # Step 4: Clean up all temporary regions created by build_grid
        # This ensures no regions are left behind regardless of success/failure
        iter_regions = getattr(page, "iter_regions", None)
        raw_regions: Optional[Iterable[Any]]
        if callable(iter_regions):
            raw_regions = cast(Optional[Iterable[Any]], iter_regions())
        else:
            raw_regions = cast(Optional[Iterable[Any]], iter_regions)

        if raw_regions is None or not isinstance(raw_regions, IterableABC):
            existing_regions = []
        else:
            existing_regions = list(raw_regions)

        regions_to_remove = [
            r
            for r in existing_regions
            if getattr(r, "source", None) == source
            and getattr(r, "region_type", None)
            in {"table", "table_row", "table_column", "table_cell"}
        ]

        for region in regions_to_remove:
            page.remove_element(region, element_type="regions")

        if regions_to_remove:
            logger.debug("Cleaned up %d temporary regions", len(regions_to_remove))

natural_pdf.Guides.from_content(obj, axis='vertical', markers=None, align='left', outer=True, tolerance=5, apply_exclusions=True) classmethod

Create guides based on text content positions.

Parameters:

Name	Type	Description	Default
obj	GuidesContext	Page, Region, or FlowRegion to search for content	required
axis	Literal['vertical', 'horizontal']	Whether to create vertical or horizontal guides	'vertical'
markers	Union[str, List[str], ElementCollection, None]	Content to search for. Can be: - str: single selector (e.g., 'text:contains("Name")') or literal text - List[str]: list of selectors or literal text strings - ElementCollection: collection of elements to extract text from - None: no markers	None
align	Union[Literal['left', 'right', 'center', 'between'], Literal['top', 'bottom']]	Where to place guides relative to found text: - For vertical guides: 'left', 'right', 'center', 'between' - For horizontal guides: 'top', 'bottom', 'center', 'between'	'left'
outer	OuterBoundaryMode	Whether to add guides at the boundaries	True
tolerance	float	Maximum distance to search for text	5
apply_exclusions	bool	Whether to apply exclusion zones when searching for text	True

Returns:

Type	Description
Guides	New Guides object aligned to text content

Source code in natural_pdf/analyzers/guides/base.py

@classmethod
def from_content(
    cls,
    obj: GuidesContext,
    axis: Literal["vertical", "horizontal"] = "vertical",
    markers: Union[str, List[str], "ElementCollection", None] = None,
    align: Union[
        Literal["left", "right", "center", "between"], Literal["top", "bottom"]
    ] = "left",
    outer: OuterBoundaryMode = True,
    tolerance: float = 5,
    apply_exclusions: bool = True,
) -> "Guides":
    """
    Create guides based on text content positions.

    Args:
        obj: Page, Region, or FlowRegion to search for content
        axis: Whether to create vertical or horizontal guides
        markers: Content to search for. Can be:
            - str: single selector (e.g., 'text:contains("Name")') or literal text
            - List[str]: list of selectors or literal text strings
            - ElementCollection: collection of elements to extract text from
            - None: no markers
        align: Where to place guides relative to found text:
            - For vertical guides: 'left', 'right', 'center', 'between'
            - For horizontal guides: 'top', 'bottom', 'center', 'between'
        outer: Whether to add guides at the boundaries
        tolerance: Maximum distance to search for text
        apply_exclusions: Whether to apply exclusion zones when searching for text

    Returns:
        New Guides object aligned to text content
    """
    if axis == "horizontal":
        if align == "top":
            align = "left"
        elif align == "bottom":
            align = "right"

    options = {
        "markers": markers,
        "align": align,
        "outer": outer,
        "tolerance": tolerance,
        "apply_exclusions": apply_exclusions,
    }

    if _is_flow_region(obj):
        guides = cls(context=obj)
        adapter = FlowGuideAdapter(guides)
        axis_values: Dict[Any, Sequence[float]] = {}
        for region in adapter.regions:
            result = run_guides_detect(
                axis=axis,
                method="content",
                context=region,
                options=options,
            )
            axis_values[region] = [float(value) for value in result.coordinates]

        adapter.update_axis_from_regions(axis, axis_values, append=False)
        return guides

    result = run_guides_detect(
        axis=axis,
        method="content",
        context=obj,
        options=options,
    )

    coords = [float(value) for value in result.coordinates]
    bounds = _bounds_from_object(obj)
    if axis == "vertical":
        return cls(verticals=coords, context=obj, bounds=bounds)
    return cls(horizontals=coords, context=obj, bounds=bounds)

natural_pdf.Guides.from_headers(obj, axis='vertical', headers=None, method='min_crossings', min_width=None, max_width=None, margin=0.5, row_stabilization=True, num_samples=400) classmethod

Create vertical guides by analyzing header elements.

Source code in natural_pdf/analyzers/guides/base.py

@classmethod
def from_headers(
    cls,
    obj: GuidesContext,
    axis: Literal["vertical", "horizontal"] = "vertical",
    headers: Union["ElementCollection", Sequence[Any], None] = None,
    method: Literal["min_crossings", "seam_carving"] = "min_crossings",
    min_width: Optional[float] = None,
    max_width: Optional[float] = None,
    margin: float = 0.5,
    row_stabilization: bool = True,
    num_samples: int = 400,
) -> "Guides":
    """Create vertical guides by analyzing header elements."""

    if axis != "vertical":
        raise ValueError("from_headers() only works for vertical guides (columns)")

    options = {
        "headers": headers,
        "method": method,
        "min_width": min_width,
        "max_width": max_width,
        "margin": margin,
        "row_stabilization": row_stabilization,
        "num_samples": num_samples,
    }

    if _is_flow_region(obj):
        guides = cls(context=obj)
        adapter = FlowGuideAdapter(guides)
        region_values: Dict[Any, Sequence[float]] = {}

        for region in adapter.regions:
            result = run_guides_detect(
                axis="vertical",
                method="headers",
                context=region,
                options=options,
            )
            region_values[region] = [float(value) for value in result.coordinates]

        adapter.update_axis_from_regions("vertical", region_values, append=False)
        return guides

    result = run_guides_detect(
        axis="vertical",
        method="headers",
        context=obj,
        options=options,
    )

    coords = [float(value) for value in result.coordinates]
    bounds = _bounds_from_object(obj)
    return cls(verticals=coords, horizontals=[], context=obj, bounds=bounds)

natural_pdf.Guides.from_lines(obj, axis='both', threshold='auto', source_label=None, max_lines_h=None, max_lines_v=None, outer=False, detection_method='auto', resolution=192, **detect_kwargs) classmethod

Create guides from detected line elements.

Parameters:

Name	Type	Description	Default
obj	GuidesContext	Page, Region, or FlowRegion to detect lines from	required
axis	Literal['vertical', 'horizontal', 'both']	Which orientations to detect	'both'
threshold	Union[float, str]	Detection threshold ('auto' or float 0.0-1.0) - used for pixel detection	'auto'
source_label	Optional[str]	Filter for line source (vector method) or label for detected lines (pixel method)	None
max_lines_h	Optional[int]	Maximum number of horizontal lines to keep	None
max_lines_v	Optional[int]	Maximum number of vertical lines to keep	None
outer	bool	Whether to add outer boundary guides	False
detection_method	str	'vector', 'pixels' (default), or 'auto' for hybrid detection.	'auto'
resolution	int	DPI for pixel-based detection (default: 192)	192
**detect_kwargs		Additional parameters for pixel-based detection: - min_gap_h: Minimum gap between horizontal lines (pixels) - min_gap_v: Minimum gap between vertical lines (pixels) - binarization_method: 'adaptive' or 'otsu' - morph_op_h/v: Morphological operations ('open', 'close', 'none') - smoothing_sigma_h/v: Gaussian smoothing sigma - method: 'projection' (default) or 'lsd' (requires opencv)	{}

Returns:

Type	Description
Guides	New Guides object with detected line positions

Source code in natural_pdf/analyzers/guides/base.py

@classmethod
def from_lines(
    cls,
    obj: GuidesContext,
    axis: Literal["vertical", "horizontal", "both"] = "both",
    threshold: Union[float, str] = "auto",
    source_label: Optional[str] = None,
    max_lines_h: Optional[int] = None,
    max_lines_v: Optional[int] = None,
    outer: bool = False,
    detection_method: str = "auto",
    resolution: int = 192,
    **detect_kwargs,
) -> "Guides":
    """
    Create guides from detected line elements.

    Args:
        obj: Page, Region, or FlowRegion to detect lines from
        axis: Which orientations to detect
        threshold: Detection threshold ('auto' or float 0.0-1.0) - used for pixel detection
        source_label: Filter for line source (vector method) or label for detected lines (pixel method)
        max_lines_h: Maximum number of horizontal lines to keep
        max_lines_v: Maximum number of vertical lines to keep
        outer: Whether to add outer boundary guides
        detection_method: 'vector', 'pixels' (default), or 'auto' for hybrid detection.
        resolution: DPI for pixel-based detection (default: 192)
        **detect_kwargs: Additional parameters for pixel-based detection:
            - min_gap_h: Minimum gap between horizontal lines (pixels)
            - min_gap_v: Minimum gap between vertical lines (pixels)
            - binarization_method: 'adaptive' or 'otsu'
            - morph_op_h/v: Morphological operations ('open', 'close', 'none')
            - smoothing_sigma_h/v: Gaussian smoothing sigma
            - method: 'projection' (default) or 'lsd' (requires opencv)

    Returns:
        New Guides object with detected line positions
    """
    if axis == "both":
        vertical_guides = cls.from_lines(
            obj,
            axis="vertical",
            threshold=threshold,
            source_label=source_label,
            max_lines_h=max_lines_h,
            max_lines_v=max_lines_v,
            outer=outer,
            detection_method=detection_method,
            resolution=resolution,
            **detect_kwargs,
        )
        horizontal_guides = cls.from_lines(
            obj,
            axis="horizontal",
            threshold=threshold,
            source_label=source_label,
            max_lines_h=max_lines_h,
            max_lines_v=max_lines_v,
            outer=outer,
            detection_method=detection_method,
            resolution=resolution,
            **detect_kwargs,
        )
        bounds = _bounds_from_object(obj)
        return cls(
            verticals=list(vertical_guides.vertical),
            horizontals=list(horizontal_guides.horizontal),
            context=obj,
            bounds=bounds,
        )

    options = {
        "threshold": threshold,
        "source_label": source_label,
        "max_lines_h": max_lines_h if axis == "horizontal" else None,
        "max_lines_v": max_lines_v if axis == "vertical" else None,
        "outer": outer,
        "detection_method": detection_method,
        "resolution": resolution,
        **detect_kwargs,
    }

    if _is_flow_region(obj):
        guides = cls(context=obj)
        adapter = FlowGuideAdapter(guides)
        axis_values: Dict[Any, Sequence[float]] = {}
        for region in adapter.regions:
            result = run_guides_detect(
                axis=axis,
                method="lines",
                context=region,
                options=options,
            )
            axis_values[region] = [float(value) for value in result.coordinates]

        adapter.update_axis_from_regions(axis, axis_values, append=False)
        return guides

    result = run_guides_detect(
        axis=axis,
        method="lines",
        context=obj,
        options=options,
    )

    coords = [float(value) for value in result.coordinates]
    bounds = _bounds_from_object(obj)
    if axis == "vertical":
        return cls(verticals=coords, horizontals=[], context=obj, bounds=bounds)
    return cls(verticals=[], horizontals=coords, context=obj, bounds=bounds)

natural_pdf.Guides.from_stripes(obj, axis='horizontal', stripes=None, color=None) classmethod

Create guides from zebra stripes or colored bands.

Source code in natural_pdf/analyzers/guides/base.py

@classmethod
def from_stripes(
    cls,
    obj: GuidesContext,
    axis: Literal["vertical", "horizontal"] = "horizontal",
    stripes: Optional[Union["ElementCollection", Sequence[Any]]] = None,
    color: Optional[str] = None,
) -> "Guides":
    """Create guides from zebra stripes or colored bands."""

    axis_lower = axis.lower()
    if axis_lower not in {"vertical", "horizontal"}:
        raise ValueError("axis must be 'vertical' or 'horizontal'")
    axis = cast(Literal["vertical", "horizontal"], axis_lower)

    options = {"stripes": stripes, "color": color}

    if _is_flow_region(obj):
        guides = cls(context=obj)
        adapter = FlowGuideAdapter(guides)
        region_values: Dict[Any, Sequence[float]] = {}

        for region in adapter.regions:
            result = run_guides_detect(
                axis=axis, method="stripes", context=region, options=options
            )
            region_values[region] = [float(value) for value in result.coordinates]

        adapter.update_axis_from_regions(axis, region_values, append=True)
        return guides

    result = run_guides_detect(
        axis=axis,
        method="stripes",
        context=obj,
        options=options,
    )

    coords = [float(value) for value in result.coordinates]
    bounds = _bounds_from_object(obj)
    if axis == "vertical":
        return cls(verticals=coords, horizontals=[], context=obj, bounds=bounds)
    return cls(verticals=[], horizontals=coords, context=obj, bounds=bounds)

natural_pdf.Guides.from_whitespace(obj, axis='both', min_gap=10) classmethod

Create guides by detecting whitespace gaps (divide + snap placeholder).

Source code in natural_pdf/analyzers/guides/base.py

@classmethod
def from_whitespace(
    cls,
    obj: GuidesContext,
    axis: Literal["vertical", "horizontal", "both"] = "both",
    min_gap: float = 10,
) -> "Guides":
    """Create guides by detecting whitespace gaps (divide + snap placeholder)."""
    if axis == "both":
        vertical_guides = cls.from_whitespace(obj, axis="vertical", min_gap=min_gap)
        horizontal_guides = cls.from_whitespace(obj, axis="horizontal", min_gap=min_gap)
        bounds = _bounds_from_object(obj)
        return cls(
            verticals=list(vertical_guides.vertical),
            horizontals=list(horizontal_guides.horizontal),
            context=obj,
            bounds=bounds,
        )

    options = {"min_gap": min_gap}

    if _is_flow_region(obj):
        guides = cls(context=obj)
        adapter = FlowGuideAdapter(guides)
        axis_values: Dict[Any, Sequence[float]] = {}
        for region in adapter.regions:
            result = run_guides_detect(
                axis=axis,
                method="whitespace",
                context=region,
                options=options,
            )
            axis_values[region] = [float(value) for value in result.coordinates]

        adapter.update_axis_from_regions(axis, axis_values, append=False)
        return guides

    result = run_guides_detect(
        axis=axis,
        method="whitespace",
        context=obj,
        options=options,
    )

    coords = [float(value) for value in result.coordinates]
    bounds = _bounds_from_object(obj)
    if axis == "vertical":
        return cls(verticals=coords, horizontals=[], context=obj, bounds=bounds)
    return cls(verticals=[], horizontals=coords, context=obj, bounds=bounds)

natural_pdf.Guides.get_cells()

Get all cell bounding boxes from guide intersections.

Returns:

Type	Description
List[Tuple[float, float, float, float]]	List of (x0, y0, x1, y1) tuples for each cell

Source code in natural_pdf/analyzers/guides/base.py

def get_cells(self) -> List[Tuple[float, float, float, float]]:
    """
    Get all cell bounding boxes from guide intersections.

    Returns:
        List of (x0, y0, x1, y1) tuples for each cell
    """
    cells = []

    # Create cells from guide intersections
    for i in range(len(self.vertical) - 1):
        for j in range(len(self.horizontal) - 1):
            x0 = self.vertical[i]
            x1 = self.vertical[i + 1]
            y0 = self.horizontal[j]
            y1 = self.horizontal[j + 1]
            cells.append((x0, y0, x1, y1))

    return cells

natural_pdf.Guides.left_of(guide_index, obj=None)

Get a region to the left of a vertical guide.

Parameters:

Name	Type	Description	Default
guide_index	int	Vertical guide index	required
obj	Optional[Union[Page, Region]]	Page or Region to create the region on (uses self.context if None)	None

Returns:

Type	Description
Region	Region to the left of the specified guide

Source code in natural_pdf/analyzers/guides/base.py

def left_of(self, guide_index: int, obj: Optional[Union["Page", "Region"]] = None) -> "Region":
    """
    Get a region to the left of a vertical guide.

    Args:
        guide_index: Vertical guide index
        obj: Page or Region to create the region on (uses self.context if None)

    Returns:
        Region to the left of the specified guide
    """
    target = obj or self.context
    if target is None:
        raise ValueError("No context available for region creation")

    if not self.vertical or guide_index < 0 or guide_index >= len(self.vertical):
        raise IndexError(f"Guide index {guide_index} out of range")

    # Get bounds from context
    bounds = self._get_context_bounds()
    if not bounds:
        raise ValueError("Could not determine bounds")
    x0, y0, _, y1 = bounds

    # Create region from left edge to guide
    x1 = self.vertical[guide_index]

    if hasattr(target, "region"):
        return target.region(x0, y0, x1, y1)
    else:
        raise TypeError(f"Cannot create region on {type(target)}")

natural_pdf.Guides.new(context=None) classmethod

Create a new empty Guides object, optionally with a context.

This provides a clean way to start building guides through chaining: guides = Guides.new(page).add_content(axis='vertical', markers=[...])

Parameters:

Name	Type	Description	Default
context	Optional[Union[Page, Region]]	Optional Page or Region to use as default context for operations	None

Returns:

Type	Description
Guides	New empty Guides object

Source code in natural_pdf/analyzers/guides/base.py

@classmethod
def new(cls, context: Optional[Union["Page", "Region"]] = None) -> "Guides":
    """
    Create a new empty Guides object, optionally with a context.

    This provides a clean way to start building guides through chaining:
    guides = Guides.new(page).add_content(axis='vertical', markers=[...])

    Args:
        context: Optional Page or Region to use as default context for operations

    Returns:
        New empty Guides object
    """
    return cls(verticals=[], horizontals=[], context=context)

natural_pdf.Guides.remove_horizontal(index)

Remove a horizontal guide by index.

Source code in natural_pdf/analyzers/guides/base.py

def remove_horizontal(self, index: int) -> "Guides":
    """Remove a horizontal guide by index."""
    if 0 <= index < len(self.horizontal):
        self.horizontal.pop(index)
    return self

natural_pdf.Guides.remove_vertical(index)

Remove a vertical guide by index.

Source code in natural_pdf/analyzers/guides/base.py

def remove_vertical(self, index: int) -> "Guides":
    """Remove a vertical guide by index."""
    if 0 <= index < len(self.vertical):
        self.vertical.pop(index)
    return self

natural_pdf.Guides.right_of(guide_index, obj=None)

Get a region to the right of a vertical guide.

Parameters:

Name	Type	Description	Default
guide_index	int	Vertical guide index	required
obj	Optional[Union[Page, Region]]	Page or Region to create the region on (uses self.context if None)	None

Returns:

Type	Description
Region	Region to the right of the specified guide

Source code in natural_pdf/analyzers/guides/base.py

def right_of(self, guide_index: int, obj: Optional[Union["Page", "Region"]] = None) -> "Region":
    """
    Get a region to the right of a vertical guide.

    Args:
        guide_index: Vertical guide index
        obj: Page or Region to create the region on (uses self.context if None)

    Returns:
        Region to the right of the specified guide
    """
    target = obj or self.context
    if target is None:
        raise ValueError("No context available for region creation")

    if not self.vertical or guide_index < 0 or guide_index >= len(self.vertical):
        raise IndexError(f"Guide index {guide_index} out of range")

    # Get bounds from context
    bounds = self._get_context_bounds()
    if not bounds:
        raise ValueError("Could not determine bounds")
    _, y0, x1, y1 = bounds

    # Create region from guide to right edge
    x0 = self.vertical[guide_index]

    if hasattr(target, "region"):
        return target.region(x0, y0, x1, y1)
    else:
        raise TypeError(f"Cannot create region on {type(target)}")

natural_pdf.Guides.row(index, obj=None)

Get a row region from the guides.

Parameters:

Name	Type	Description	Default
index	int	Row index (0-based)	required
obj	Optional[Union[Page, Region]]	Page or Region to create the row on (uses self.context if None)	None

Returns:

Type	Description
Region	Region representing the specified row

Raises:

Type	Description
IndexError	If row index is out of range

Source code in natural_pdf/analyzers/guides/base.py

def row(self, index: int, obj: Optional[Union["Page", "Region"]] = None) -> "Region":
    """
    Get a row region from the guides.

    Args:
        index: Row index (0-based)
        obj: Page or Region to create the row on (uses self.context if None)

    Returns:
        Region representing the specified row

    Raises:
        IndexError: If row index is out of range
    """
    target = obj or self.context
    if target is None:
        raise ValueError("No context available for region creation")

    horizontal_guides = list(self.horizontal.data)
    if not horizontal_guides or index < 0 or index >= len(horizontal_guides) - 1:
        raise IndexError(
            f"Row index {index} out of range (have {len(horizontal_guides)-1} rows)"
        )

    # Get bounds from context
    x0, _, x1, _ = self._get_context_bounds()

    # Get row boundaries
    y0 = horizontal_guides[index]
    y1 = horizontal_guides[index + 1]

    # Create region using absolute coordinates
    if hasattr(target, "create_region"):
        if isinstance(target, Region):
            return target.create_region(x0, y0, x1, y1, relative=False)
        return target.create_region(x0, y0, x1, y1)

    try:
        page = _resolve_single_page(target)
    except (TypeError, ValueError) as exc:
        raise TypeError(f"Cannot create region on {type(target)}") from exc

    return page.create_region(x0, y0, x1, y1)

natural_pdf.Guides.shift(index, offset, axis='vertical')

Move a specific guide by a offset amount.

Parameters:

Name	Type	Description	Default
index	int	Index of the guide to move	required
offset	float	Amount to move (positive = right/down)	required
axis	Literal['vertical', 'horizontal']	Which guide list to modify	'vertical'

Returns:

Type	Description
Guides	Self for method chaining

Source code in natural_pdf/analyzers/guides/base.py

def shift(
    self, index: int, offset: float, axis: Literal["vertical", "horizontal"] = "vertical"
) -> "Guides":
    """
    Move a specific guide by a offset amount.

    Args:
        index: Index of the guide to move
        offset: Amount to move (positive = right/down)
        axis: Which guide list to modify

    Returns:
        Self for method chaining
    """
    if axis == "vertical":
        if 0 <= index < len(self.vertical):
            self.vertical[index] += offset
            self.vertical = sorted(self.vertical)
        else:
            logger.warning(f"Vertical guide index {index} out of range")
    else:
        if 0 <= index < len(self.horizontal):
            self.horizontal[index] += offset
            self.horizontal = sorted(self.horizontal)
        else:
            logger.warning(f"Horizontal guide index {index} out of range")

    return self

natural_pdf.Guides.show(on=None, **kwargs)

Display the guides overlaid on a page or region.

Parameters:

Name	Type	Description	Default
on		Page, Region, PIL Image, or string to display guides on. If None, uses self.context (the object guides were created from). If string 'page', uses the page from self.context.	None
**kwargs		Additional arguments passed to to_image() if applicable.	{}

Returns:

Type	Description
	PIL Image with guides drawn on it.

Source code in natural_pdf/analyzers/guides/base.py

def show(self, on=None, **kwargs):
    """
    Display the guides overlaid on a page or region.

    Args:
        on: Page, Region, PIL Image, or string to display guides on.
            If None, uses self.context (the object guides were created from).
            If string 'page', uses the page from self.context.
        **kwargs: Additional arguments passed to to_image() if applicable.

    Returns:
        PIL Image with guides drawn on it.
    """
    # Handle FlowRegion case
    if self.is_flow_region and (on is None or on == self.context):
        if not self._flow_guides:
            raise ValueError("No guides to show for FlowRegion")

        # Get stacking parameters from kwargs or use defaults
        stack_direction = kwargs.get("stack_direction", "vertical")
        stack_gap = kwargs.get("stack_gap", 5)
        stack_background_color = kwargs.get("stack_background_color", (255, 255, 255))

        # First, render all constituent regions without guides to get base images
        base_images = []
        region_infos = []  # Store region info for guide coordinate mapping

        for region in list(self._flow_constituent_regions()):
            render_fn = getattr(region, "render", None)
            if render_fn is None:
                raise AttributeError(f"Region {region} does not support rendering")
            img = render_fn(
                resolution=kwargs.get("resolution", 150),
                width=kwargs.get("width", None),
                crop=True,
            )
            if img:
                base_images.append(img)

                scale_x = img.width / region.width
                scale_y = img.height / region.height

                region_infos.append(
                    {
                        "region": region,
                        "img_width": img.width,
                        "img_height": img.height,
                        "scale_x": scale_x,
                        "scale_y": scale_y,
                        "pdf_x0": region.x0,
                        "pdf_top": region.top,
                        "pdf_x1": region.x1,
                        "pdf_bottom": region.bottom,
                    }
                )

        if not base_images:
            raise ValueError("Failed to render any images for FlowRegion")

        # Calculate final canvas size based on stacking direction
        if stack_direction == "vertical":
            final_width = max(img.width for img in base_images)
            final_height = (
                sum(img.height for img in base_images) + (len(base_images) - 1) * stack_gap
            )
        else:  # horizontal
            final_width = (
                sum(img.width for img in base_images) + (len(base_images) - 1) * stack_gap
            )
            final_height = max(img.height for img in base_images)

        # Create unified canvas
        canvas = Image.new("RGB", (final_width, final_height), stack_background_color)
        draw = ImageDraw.Draw(canvas)

        # Paste base images and track positions
        region_positions = []  # (region_info, paste_x, paste_y)

        if stack_direction == "vertical":
            current_y = 0
            for i, (img, info) in enumerate(zip(base_images, region_infos)):
                paste_x = (final_width - img.width) // 2  # Center horizontally
                canvas.paste(img, (paste_x, current_y))
                region_positions.append((info, paste_x, current_y))
                current_y += img.height + stack_gap
        else:  # horizontal
            current_x = 0
            for i, (img, info) in enumerate(zip(base_images, region_infos)):
                paste_y = (final_height - img.height) // 2  # Center vertically
                canvas.paste(img, (current_x, paste_y))
                region_positions.append((info, current_x, paste_y))
                current_x += img.width + stack_gap

        # Now draw guides on the unified canvas
        # Draw vertical guides (blue) - these extend through the full canvas height
        for v_coord in self.vertical:
            # Find which region(s) this guide intersects
            for info, paste_x, paste_y in region_positions:
                if info["pdf_x0"] <= v_coord <= info["pdf_x1"]:
                    # This guide is within this region's x-bounds
                    # Convert PDF coordinate to pixel coordinate relative to the region
                    adjusted_x = v_coord - info["pdf_x0"]
                    pixel_x = adjusted_x * info["scale_x"] + paste_x

                    # Draw full-height line on canvas (not clipped to region)
                    if 0 <= pixel_x <= final_width:
                        x_pixel = int(pixel_x)
                        draw.line(
                            [(x_pixel, 0), (x_pixel, final_height - 1)],
                            fill=(0, 0, 255, 200),
                            width=2,
                        )
                    break  # Only draw once per guide

        # Draw horizontal guides (red) - these extend through the full canvas width
        for h_coord in self.horizontal:
            # Find which region(s) this guide intersects
            for info, paste_x, paste_y in region_positions:
                if info["pdf_top"] <= h_coord <= info["pdf_bottom"]:
                    # This guide is within this region's y-bounds
                    # Convert PDF coordinate to pixel coordinate relative to the region
                    adjusted_y = h_coord - info["pdf_top"]
                    pixel_y = adjusted_y * info["scale_y"] + paste_y

                    # Draw full-width line on canvas (not clipped to region)
                    if 0 <= pixel_y <= final_height:
                        y_pixel = int(pixel_y)
                        draw.line(
                            [(0, y_pixel), (final_width - 1, y_pixel)],
                            fill=(255, 0, 0, 200),
                            width=2,
                        )
                    break  # Only draw once per guide

        return canvas

    # Original single-region logic follows...
    # Determine what to display guides on
    target = on if on is not None else self.context

    # Handle string shortcuts
    if isinstance(target, str):
        if target == "page":
            if self.context is None:
                raise ValueError("Cannot resolve 'page' without a guides context")
            try:
                target = _resolve_single_page(self.context)
            except (TypeError, ValueError) as exc:
                raise ValueError(
                    "Cannot resolve 'page' from the current guides context"
                ) from exc
        else:
            raise ValueError(f"Unknown string target: {target}. Only 'page' is supported.")

    if target is None:
        raise ValueError("No target specified and no context available for guides display")

    # Prepare kwargs for image generation
    image_kwargs = {}

    # Extract only the parameters that the new render() method accepts
    if "resolution" in kwargs:
        image_kwargs["resolution"] = kwargs["resolution"]
    if "width" in kwargs:
        image_kwargs["width"] = kwargs["width"]
    if "crop" in kwargs:
        image_kwargs["crop"] = kwargs["crop"]

    # If target is a region-like object, crop to just that region
    try:
        _resolve_single_page(target)
        target_is_single_page = True
    except (TypeError, ValueError):
        target_is_single_page = False

    if hasattr(target, "bbox") and target_is_single_page:
        image_kwargs["crop"] = True

    # Get base image
    if hasattr(target, "render"):
        rendered = cast(Any, target).render(**image_kwargs)
        if rendered is None:
            raise ValueError("Failed to generate base image")
        img = cast(Image.Image, rendered)
    elif hasattr(target, "mode") and hasattr(target, "size"):
        # It's already a PIL Image
        img = cast(Image.Image, target)
    else:
        raise ValueError(f"Object {target} does not support render() and is not a PIL Image")

    # Create a copy to draw on
    img = cast(Image.Image, img.copy())
    draw = ImageDraw.Draw(img)

    # Determine scale factor for coordinate conversion
    if _has_size(target) and not (hasattr(target, "mode") and hasattr(target, "size")):
        # target is a PDF object (Page/Region) with PDF coordinates
        size_target = cast(_SupportsSize, target)
        scale_x = img.width / size_target.width
        scale_y = img.height / size_target.height

        # If we're showing guides on a region, we need to adjust coordinates
        # to be relative to the region's origin
        if hasattr(target, "bbox") and target_is_single_page:
            # This is a Region - adjust guide coordinates to be relative to region
            region_like = cast(Any, target)
            region_x0 = float(getattr(region_like, "x0", 0.0))
            region_top = float(getattr(region_like, "top", 0.0))
        else:
            # This is a Page - no adjustment needed
            region_x0, region_top = 0, 0
    else:
        # target is already an image, no scaling needed
        scale_x = 1.0
        scale_y = 1.0
        region_x0, region_top = 0, 0

    # Draw vertical guides (blue)
    for x_coord in self.vertical:
        # Adjust coordinate if we're showing on a region
        adjusted_x = x_coord - region_x0
        pixel_x = adjusted_x * scale_x
        # Ensure guides at the edge are still visible by clamping to valid range
        if 0 <= pixel_x <= img.width - 1:
            x_pixel = int(min(pixel_x, img.width - 1))
            draw.line([(x_pixel, 0), (x_pixel, img.height - 1)], fill=(0, 0, 255, 200), width=2)

    # Draw horizontal guides (red)
    for y_coord in self.horizontal:
        # Adjust coordinate if we're showing on a region
        adjusted_y = y_coord - region_top
        pixel_y = adjusted_y * scale_y
        # Ensure guides at the edge are still visible by clamping to valid range
        if 0 <= pixel_y <= img.height - 1:
            y_pixel = int(min(pixel_y, img.height - 1))
            draw.line([(0, y_pixel), (img.width - 1, y_pixel)], fill=(255, 0, 0, 200), width=2)

    return img

natural_pdf.Guides.snap_to_whitespace(axis='vertical', min_gap=10.0, detection_method='pixels', threshold='auto', on_no_snap='warn')

Snap guides to nearby whitespace gaps (troughs) using optimal assignment. Modifies this Guides object in place.

Parameters:

Name	Type	Description	Default
axis	str	Direction to snap ('vertical' or 'horizontal')	'vertical'
min_gap	float	Minimum gap size to consider as a valid trough	10.0
detection_method	str	Method for detecting troughs: 'pixels' - use pixel-based density analysis (default) 'text' - use text element spacing analysis	'pixels'
threshold	Union[float, str]	Threshold for what counts as a trough: - float (0.0-1.0): areas with this fraction or less of max density count as troughs - 'auto': automatically find threshold that creates enough troughs for guides (only applies when detection_method='pixels')	'auto'
on_no_snap	str	Action when snapping fails ('warn', 'ignore', 'raise')	'warn'

Returns:

Type	Description
Guides	Self for method chaining.

Source code in natural_pdf/analyzers/guides/base.py

def snap_to_whitespace(
    self,
    axis: str = "vertical",
    min_gap: float = 10.0,
    detection_method: str = "pixels",  # 'pixels' or 'text'
    threshold: Union[
        float, str
    ] = "auto",  # threshold for what counts as a trough (0.0-1.0) or 'auto'
    on_no_snap: str = "warn",
) -> "Guides":
    """
    Snap guides to nearby whitespace gaps (troughs) using optimal assignment.
    Modifies this Guides object in place.

    Args:
        axis: Direction to snap ('vertical' or 'horizontal')
        min_gap: Minimum gap size to consider as a valid trough
        detection_method: Method for detecting troughs:
                        'pixels' - use pixel-based density analysis (default)
                        'text' - use text element spacing analysis
        threshold: Threshold for what counts as a trough:
                  - float (0.0-1.0): areas with this fraction or less of max density count as troughs
                  - 'auto': automatically find threshold that creates enough troughs for guides
                  (only applies when detection_method='pixels')
        on_no_snap: Action when snapping fails ('warn', 'ignore', 'raise')

    Returns:
        Self for method chaining.
    """
    if not self.context:
        logger.warning("No context available for whitespace detection")
        return self

    detection_mode = detection_method.lower()
    if detection_mode not in {"pixels", "text"}:
        raise ValueError("detection_method must be 'pixels' or 'text'")

    text_elements = collect_text_elements(self.context)

    def _compute_gaps(axis: str, guide_positions: Sequence[float]) -> List[Tuple[float, float]]:
        if detection_mode == "pixels":
            if axis == "vertical":
                return find_vertical_whitespace_gaps(
                    self.bounds,
                    text_elements,
                    min_gap,
                    threshold,
                    guide_positions=guide_positions,
                )
            return find_horizontal_whitespace_gaps(
                self.bounds,
                text_elements,
                min_gap,
                threshold,
                guide_positions=guide_positions,
            )
        if axis == "vertical":
            return find_vertical_element_gaps(self.bounds, text_elements, min_gap)
        return find_horizontal_element_gaps(self.bounds, text_elements, min_gap)

    # Handle FlowRegion case - collect all text elements across regions
    if self.is_flow_region:
        if not text_elements:
            logger.warning(
                "No text elements found across flow regions for whitespace detection"
            )
            return self

        if axis == "vertical":
            gaps = _compute_gaps("vertical", self.vertical.data)
            all_guides = []
            guide_to_region_map = {}
            for coord, region in self._unified_vertical:
                all_guides.append(coord)
                guide_to_region_map.setdefault(coord, []).append(region)

            if gaps and all_guides:
                original_guides = all_guides.copy()
                self._snap_guides_to_gaps(all_guides, gaps, axis)

                self._unified_vertical = []
                for i, new_coord in enumerate(all_guides):
                    original_coord = original_guides[i]
                    regions = guide_to_region_map.get(original_coord, [])
                    for region in regions:
                        self._unified_vertical.append((new_coord, region))

                for region in self._flow_guides:
                    region_verticals = [
                        coord for coord, r in self._unified_vertical if r == region
                    ]
                    self._flow_guides[region] = (
                        sorted(list(set(region_verticals))),
                        self._flow_guides[region][1],
                    )

                self._vertical_cache = None

        elif axis == "horizontal":
            gaps = _compute_gaps("horizontal", self.horizontal.data)
            all_guides = []
            guide_to_region_map = {}
            for coord, region in self._unified_horizontal:
                all_guides.append(coord)
                guide_to_region_map.setdefault(coord, []).append(region)

            if gaps and all_guides:
                original_guides = all_guides.copy()
                self._snap_guides_to_gaps(all_guides, gaps, axis)

                self._unified_horizontal = []
                for i, new_coord in enumerate(all_guides):
                    original_coord = original_guides[i]
                    regions = guide_to_region_map.get(original_coord, [])
                    for region in regions:
                        self._unified_horizontal.append((new_coord, region))

                for region in self._flow_guides:
                    region_horizontals = [
                        coord for coord, r in self._unified_horizontal if r == region
                    ]
                    self._flow_guides[region] = (
                        self._flow_guides[region][0],
                        sorted(list(set(region_horizontals))),
                    )

                self._horizontal_cache = None

        else:
            raise ValueError("axis must be 'vertical' or 'horizontal'")

        return self

    if not text_elements:
        logger.warning("No text elements found for whitespace detection")
        return self

    if axis == "vertical":
        gaps = _compute_gaps("vertical", self.vertical.data)
        if gaps:
            self._snap_guides_to_gaps(self.vertical.data, gaps, axis)
    elif axis == "horizontal":
        gaps = _compute_gaps("horizontal", self.horizontal.data)
        if gaps:
            self._snap_guides_to_gaps(self.horizontal.data, gaps, axis)
    else:
        raise ValueError("axis must be 'vertical' or 'horizontal'")

    # Ensure all coordinates are Python floats (not numpy types)
    self.vertical.data[:] = [float(x) for x in self.vertical.data]
    self.horizontal.data[:] = [float(y) for y in self.horizontal.data]

    return self

natural_pdf.Guides.to_absolute(bounds)

Convert relative coordinates to absolute coordinates.

Parameters:

Name	Type	Description	Default
bounds	Tuple[float, float, float, float]	Target bounding box (x0, y0, x1, y1)	required

Returns:

Type	Description
Guides	New Guides object with absolute coordinates

Source code in natural_pdf/analyzers/guides/base.py

def to_absolute(self, bounds: Tuple[float, float, float, float]) -> "Guides":
    """
    Convert relative coordinates to absolute coordinates.

    Args:
        bounds: Target bounding box (x0, y0, x1, y1)

    Returns:
        New Guides object with absolute coordinates
    """
    if not self.relative:
        return self  # Already absolute

    x0, y0, x1, y1 = bounds
    width = x1 - x0
    height = y1 - y0

    abs_verticals = [x0 + x * width for x in self.vertical]
    abs_horizontals = [y0 + y * height for y in self.horizontal]

    return Guides(
        verticals=abs_verticals,
        horizontals=abs_horizontals,
        context=self.context,
        bounds=bounds,
        relative=False,
    )

natural_pdf.Guides.to_dict()

Convert to dictionary format suitable for pdfplumber table_settings.

Returns:

Type	Description
Dict[str, Any]	Dictionary with explicit_vertical_lines and explicit_horizontal_lines

Source code in natural_pdf/analyzers/guides/base.py

def to_dict(self) -> Dict[str, Any]:
    """
    Convert to dictionary format suitable for pdfplumber table_settings.

    Returns:
        Dictionary with explicit_vertical_lines and explicit_horizontal_lines
    """
    return {
        "explicit_vertical_lines": self.vertical,
        "explicit_horizontal_lines": self.horizontal,
    }

natural_pdf.Guides.to_relative()

Convert absolute coordinates to relative (0-1) coordinates.

Returns:

Type	Description
Guides	New Guides object with relative coordinates

Source code in natural_pdf/analyzers/guides/base.py

def to_relative(self) -> "Guides":
    """
    Convert absolute coordinates to relative (0-1) coordinates.

    Returns:
        New Guides object with relative coordinates
    """
    if self.relative:
        return self  # Already relative

    if not self.bounds:
        raise ValueError("Cannot convert to relative without bounds")

    x0, y0, x1, y1 = self.bounds
    width = x1 - x0
    height = y1 - y0

    rel_verticals = [(x - x0) / width for x in self.vertical]
    rel_horizontals = [(y - y0) / height for y in self.horizontal]

    return Guides(
        verticals=rel_verticals,
        horizontals=rel_horizontals,
        context=self.context,
        bounds=(0, 0, 1, 1),
        relative=True,
    )

natural_pdf.Judge

Visual classifier for regions using simple image metrics.

Requires class labels to be specified. For binary classification, requires at least one example of each class before making decisions.

Examples:

Checkbox detection:

judge = Judge("checkboxes", labels=["unchecked", "checked"])
judge.add(empty_box, "unchecked")
judge.add(marked_box, "checked")

result = judge.decide(new_box)
if result.label == "checked":
    print("Box is checked!")

Signature detection:

judge = Judge("signatures", labels=["unsigned", "signed"])
judge.add(blank_area, "unsigned")
judge.add(signature_area, "signed")

result = judge.decide(new_region)
print(f"Classification: {result.label} (confidence: {result.score})")

Source code in natural_pdf/judge.py

class Judge:
    """
    Visual classifier for regions using simple image metrics.

    Requires class labels to be specified. For binary classification,
    requires at least one example of each class before making decisions.

    Examples:
        Checkbox detection:
        ```python
        judge = Judge("checkboxes", labels=["unchecked", "checked"])
        judge.add(empty_box, "unchecked")
        judge.add(marked_box, "checked")

        result = judge.decide(new_box)
        if result.label == "checked":
            print("Box is checked!")
        ```

        Signature detection:
        ```python
        judge = Judge("signatures", labels=["unsigned", "signed"])
        judge.add(blank_area, "unsigned")
        judge.add(signature_area, "signed")

        result = judge.decide(new_region)
        print(f"Classification: {result.label} (confidence: {result.score})")
        ```
    """

    def __init__(
        self,
        name: str,
        labels: List[str],
        base_dir: Optional[Union[str, Path]] = None,
        target_prior: Optional[float] = None,
    ):
        """
        Initialize a Judge for visual classification.

        Args:
            name: Name for this judge (used for folder name)
            labels: Class labels (required, typically 2 for binary classification)
            base_dir: Base directory for storage. Defaults to current directory
            target_prior: Target prior probability for the FIRST label in the labels list.
                         - 0.5 (default) = neutral, treats both classes equally
                         - >0.5 = favors labels[0]
                         - <0.5 = favors labels[1]
                         Example: Judge("cb", ["checked", "unchecked"], target_prior=0.6)
                         favors detecting "checked" checkboxes.
        """
        if not labels or len(labels) != 2:
            raise JudgeError("Judge requires exactly 2 class labels (binary classification only)")

        self.name = name
        self.labels = labels
        self.target_prior = float(target_prior) if target_prior is not None else 0.5

        # Set up directory structure
        self.base_dir: Path = Path(base_dir) if base_dir is not None else Path.cwd()
        self.root_dir: Path = self.base_dir / name
        self.root_dir.mkdir(exist_ok=True)

        # Create label directories
        for label in self.labels:
            (self.root_dir / label).mkdir(exist_ok=True)
        (self.root_dir / "unlabeled").mkdir(exist_ok=True)
        (self.root_dir / "_removed").mkdir(exist_ok=True)

        # Config file
        self.config_path: Path = self.root_dir / "judge.json"

        # Load existing config or initialize
        self.thresholds: Dict[str, Dict[str, Any]] = {}
        self.metrics_info: Dict[str, Dict[str, float]] = {}
        if self.config_path.exists():
            self._load_config()

    def add(self, region: SupportsRender, label: Optional[str] = None) -> None:
        """
        Add a region to the judge's dataset.

        Args:
            region: Region object to add
            label: Class label. If None, added to unlabeled for later teaching

        Raises:
            JudgeError: If label is not in allowed labels
        """
        if label is not None and label not in self.labels:
            raise JudgeError(f"Label '{label}' not in allowed labels: {self.labels}")

        # Render region to image
        try:
            img = region.render(crop=True)
            if img is None:
                raise JudgeError("Region.render returned no image.")
            if not isinstance(img, Image.Image):
                img = Image.fromarray(np.asarray(img))
        except Exception as e:
            raise JudgeError(f"Failed to render region: {e}")

        # Convert to RGB if needed
        if img.mode != "RGB":
            img = img.convert("RGB")

        # Generate hash from image content
        img_array = np.array(img)
        img_hash = hashlib.md5(img_array.tobytes()).hexdigest()[:12]

        # Determine target directory
        target_dir = self.root_dir / (label if label else "unlabeled")
        target_path = target_dir / f"{img_hash}.png"

        # Check if hash already exists anywhere
        existing_locations = []
        for check_label in self.labels + ["unlabeled", "_removed"]:
            check_path = self.root_dir / check_label / f"{img_hash}.png"
            if check_path.exists():
                existing_locations.append(check_label)

        if existing_locations:
            logger.warning(f"Duplicate image detected (hash: {img_hash})")
            logger.warning(f"Already exists in: {', '.join(existing_locations)}")
            print(f"⚠️  Duplicate image - already exists in: {', '.join(existing_locations)}")
            return

        # Save image
        img.save(target_path)
        logger.debug(f"Added image {img_hash} to {label if label else 'unlabeled'}")

    def teach(self, labels: Optional[List[str]] = None, review: bool = False) -> None:
        """
        Interactive teaching interface using IPython widgets.

        Args:
            labels: Labels to use for teaching. Defaults to self.labels
            review: If True, review already labeled images for re-classification
        """
        # Check for IPython environment
        try:
            import ipywidgets as widgets
            from IPython.display import clear_output, display
        except ImportError:
            raise JudgeError(
                "Teaching requires IPython and ipywidgets. Use 'pip install ipywidgets'"
            )

        labels = labels or self.labels

        # Get images to review
        if review:
            # Get all labeled images for review
            files_to_review = []
            for label in self.labels:
                label_dir = self.root_dir / label
                for img_path in sorted(label_dir.glob("*.png")):
                    files_to_review.append((img_path, label))

            if not files_to_review:
                print("No labeled images to review")
                return

            # Shuffle for review
            import random

            random.shuffle(files_to_review)
            review_files = [f[0] for f in files_to_review]
            original_labels = {str(f[0]): f[1] for f in files_to_review}
        else:
            # Get unlabeled images
            unlabeled_dir = self.root_dir / "unlabeled"
            review_files = sorted(unlabeled_dir.glob("*.png"))
            original_labels = {}

            if not review_files:
                print("No unlabeled images to teach")
                return

        # State for teaching
        self._teaching_state = {
            "current_index": 0,
            "labeled_count": 0,
            "removed_count": 0,
            "files": review_files,
            "labels": labels,
            "review_mode": review,
            "original_labels": original_labels,
        }

        # Create widgets
        image_widget = widgets.Image()
        status_label = widgets.Label()

        # Create buttons for labeling
        button_layout = widgets.Layout(width="auto", margin="5px")

        btn_prev = widgets.Button(description="↑ Previous", layout=button_layout)
        btn_class1 = widgets.Button(
            description=f"← {labels[0]}", layout=button_layout, button_style="primary"
        )
        btn_class2 = widgets.Button(
            description=f"→ {labels[1]}", layout=button_layout, button_style="success"
        )
        btn_skip = widgets.Button(description="↓ Skip", layout=button_layout)
        btn_remove = widgets.Button(
            description="✗ Remove", layout=button_layout, button_style="danger"
        )

        button_box = widgets.HBox([btn_prev, btn_class1, btn_class2, btn_skip, btn_remove])

        # Keyboard shortcuts info
        info_label = widgets.Label(
            value="Keys: ↑ prev | ← "
            + labels[0]
            + " | → "
            + labels[1]
            + " | ↓ skip | Delete remove"
        )

        def update_display():
            """Update the displayed image and status."""
            state = self._teaching_state
            if 0 <= state["current_index"] < len(state["files"]):
                img_path = state["files"][state["current_index"]]
                with open(img_path, "rb") as f:
                    image_widget.value = f.read()

                # Build status text
                status_text = f"Image {state['current_index'] + 1} of {len(state['files'])}"
                if state["review_mode"]:
                    current_label = state["original_labels"].get(str(img_path), "unknown")
                    status_text += f" (Currently: {current_label})"
                status_text += f" | Labeled: {state['labeled_count']}"
                if state["removed_count"] > 0:
                    status_text += f" | Removed: {state['removed_count']}"

                status_label.value = status_text

                # Update button states
                btn_prev.disabled = state["current_index"] == 0
            else:
                status_label.value = "Teaching complete!"
                # Hide the image widget instead of showing broken image
                image_widget.layout.display = "none"
                # Disable all buttons
                btn_prev.disabled = True
                btn_class1.disabled = True
                btn_class2.disabled = True
                btn_skip.disabled = True

                # Auto-retrain
                if state["labeled_count"] > 0 or state["removed_count"] > 0:
                    clear_output(wait=True)
                    print("Teaching complete!")
                    print(f"Labeled: {state['labeled_count']} images")
                    if state["removed_count"] > 0:
                        print(f"Removed: {state['removed_count']} images")

                    if state["labeled_count"] > 0:
                        print("\nRetraining with new examples...")
                        self._retrain()
                        print("✓ Training complete! Judge is ready to use.")
                else:
                    print("No changes made.")

        def move_file_to_class(class_index):
            """Move current file to specified class."""
            state = self._teaching_state
            if state["current_index"] >= len(state["files"]):
                return

            current_file = state["files"][state["current_index"]]
            target_dir = self.root_dir / labels[class_index]
            shutil.move(str(current_file), str(target_dir / current_file.name))
            state["labeled_count"] += 1
            state["current_index"] += 1
            update_display()

        # Button callbacks
        def on_prev(b):
            state = self._teaching_state
            if state["current_index"] > 0:
                state["current_index"] -= 1
                update_display()

        def on_class1(b):
            move_file_to_class(0)

        def on_class2(b):
            move_file_to_class(1)

        def on_skip(b):
            state = self._teaching_state
            state["current_index"] += 1
            update_display()

        def on_remove(b):
            state = self._teaching_state
            if state["current_index"] >= len(state["files"]):
                return

            current_file = state["files"][state["current_index"]]
            target_dir = self.root_dir / "_removed"
            shutil.move(str(current_file), str(target_dir / current_file.name))
            state["removed_count"] += 1
            state["current_index"] += 1
            update_display()

        # Connect buttons
        btn_prev.on_click(on_prev)
        btn_class1.on_click(on_class1)
        btn_class2.on_click(on_class2)
        btn_skip.on_click(on_skip)
        btn_remove.on_click(on_remove)

        # Create output widget for keyboard handling
        output = widgets.Output()

        # Keyboard event handler
        def on_key(event):
            """Handle keyboard events."""
            if event["type"] != "keydown":
                return

            key = event["key"]

            if key == "ArrowUp":
                on_prev(None)
            elif key == "ArrowLeft":
                on_class1(None)
            elif key == "ArrowRight":
                on_class2(None)
            elif key == "ArrowDown":
                on_skip(None)
            elif key in ["Delete", "Backspace"]:
                on_remove(None)

        # Display everything
        display(status_label)
        display(image_widget)
        display(button_box)
        display(info_label)
        display(output)

        # Show first image
        update_display()

        # Try to set up keyboard handling (may not work in all environments)
        try:
            from ipyevents import Event  # type: ignore[import-untyped]

            event_handler = Event(source=output, watched_events=["keydown"])
            event_handler.on_dom_event(on_key)
        except:
            # If ipyevents not available, just use buttons
            print("Note: Install ipyevents for keyboard shortcuts: pip install ipyevents")

    def decide(
        self, regions: Union[SupportsRender, Iterable[SupportsRender]]
    ) -> Union[Decision, List[Decision]]:
        """
        Classify one or more regions.

        Args:
            regions: Single region or list of regions to classify

        Returns:
            Decision or list of Decisions with label and score

        Raises:
            JudgeError: If not enough training examples
        """
        # Check if we have examples
        for label in self.labels:
            label_dir = self.root_dir / label
            if not any(label_dir.glob("*.png")):
                raise JudgeError(f"Need at least one example of class '{label}' before deciding")

        # Ensure thresholds are current
        if not self.thresholds:
            self._retrain()

        # Normalize to list of regions
        if isinstance(regions, IterableABC) and not isinstance(regions, (str, bytes)):
            region_list: List[SupportsRender] = list(regions)
            single_input = False
        else:
            region_list = [cast(SupportsRender, regions)]
            single_input = True

        results: List[Decision] = []
        for region in region_list:
            # Extract metrics
            metrics = self._extract_metrics(region)

            # Apply thresholds with soft voting
            votes: Dict[str, float] = {label: 0.0 for label in self.labels}
            total_weight = 0.0

            for metric_name, value in metrics.items():
                if metric_name in self.thresholds:
                    metric_info = self.thresholds[metric_name]
                    weight = metric_info["accuracy"]  # This is now Youden's J

                    # For binary classification
                    label1, label2 = self.labels
                    threshold1, direction1 = metric_info["thresholds"][label1]

                    # Get standard deviations for soft voting
                    stats = self.metrics_info.get(metric_name, {})
                    s1 = stats.get(f"std_{label1}", 0.0)
                    s2 = stats.get(f"std_{label2}", 0.0)
                    scale1 = s1 if s1 > 1e-6 else 1.0
                    scale2 = s2 if s2 > 1e-6 else 1.0

                    # Calculate signed margin (positive favors label1, negative favors label2)
                    if direction1 == "higher":
                        margin = (value - threshold1) / (scale1 if value >= threshold1 else scale2)
                    else:
                        margin = (threshold1 - value) / (scale1 if value <= threshold1 else scale2)

                    # Clip margin to avoid single metric dominating
                    margin = np.clip(margin, -6, 6)

                    # Soft votes using sigmoid
                    p1 = 1.0 / (1.0 + np.exp(-margin))
                    p2 = 1.0 - p1

                    votes[label1] += weight * p1
                    votes[label2] += weight * p2
                    total_weight += weight

            # Normalize votes
            if total_weight > 0:
                for label in votes:
                    votes[label] /= total_weight
            else:
                # Fallback: uniform votes so prior still works
                for label in votes:
                    votes[label] = 0.5
                total_weight = 1.0

            # Apply prior bias correction
            def _logit(p, eps=1e-6):
                p = max(eps, min(1 - eps, p))
                return np.log(p / (1 - p))

            def _sigmoid(x):
                if x >= 0:
                    z = np.exp(-x)
                    return 1.0 / (1.0 + z)
                else:
                    z = np.exp(x)
                    return z / (1.0 + z)

            # Estimate priors from training counts
            counts = self._get_training_counts()
            label1, label2 = self.labels
            n1 = counts.get(label1, 0)
            n2 = counts.get(label2, 0)
            total = max(1, n1 + n2)

            if n1 > 0 and n2 > 0:  # Only apply bias if we have examples of both classes
                emp_prior1 = n1 / total
                emp_prior2 = n2 / total

                # Target prior (0.5/0.5 neutralizes imbalance)
                target_prior1 = self.target_prior
                target_prior2 = 1.0 - self.target_prior

                # Calculate bias
                bias1 = _logit(target_prior1) - _logit(emp_prior1)
                bias2 = _logit(target_prior2) - _logit(emp_prior2)

                # Apply bias in logit space
                v1 = _sigmoid(_logit(votes[label1]) + bias1)
                v2 = _sigmoid(_logit(votes[label2]) + bias2)

                # Renormalize
                s = v1 + v2
                votes[label1] = v1 / s
                votes[label2] = v2 / s

            # Find best label
            best_label = max(votes.items(), key=lambda x: x[1])
            results.append(Decision(label=best_label[0], score=best_label[1]))

        return results[0] if single_input else results

    def pick(
        self,
        target_label: str,
        regions: Iterable[SupportsRender],
        labels: Optional[Sequence[str]] = None,
    ) -> PickResult:
        """
        Pick which region best matches the target label.

        Args:
            target_label: The class label to look for
            regions: List of regions to choose from
            labels: Optional human-friendly labels for each region

        Returns:
            PickResult with winning region, index, label (if provided), and score

        Raises:
            JudgeError: If target_label not in allowed labels
        """
        if target_label not in self.labels:
            raise JudgeError(f"Target label '{target_label}' not in allowed labels: {self.labels}")

        # Classify all regions
        region_list = list(regions)
        decisions_result = self.decide(region_list)
        decisions = decisions_result if isinstance(decisions_result, list) else [decisions_result]

        # Find best match for target label
        best_index = -1
        best_score = -1.0

        for i, decision in enumerate(decisions):
            if decision.label == target_label and decision.score > best_score:
                best_score = decision.score
                best_index = i

        if best_index == -1:
            # No region matched the target label
            raise JudgeError(f"No region classified as '{target_label}'")

        # Build result
        region = region_list[best_index]
        label_list = list(labels) if labels is not None else None
        label = (
            label_list[best_index]
            if label_list is not None and best_index < len(label_list)
            else None
        )

        return PickResult(region=region, index=best_index, label=label, score=best_score)

    def count(self, target_label: str, regions: Iterable[SupportsRender]) -> int:
        """
        Count how many regions match the target label.

        Args:
            target_label: The class label to count
            regions: List of regions to check

        Returns:
            Number of regions classified as target_label
        """
        decisions_result = self.decide(regions)
        decisions = decisions_result if isinstance(decisions_result, list) else [decisions_result]
        return sum(1 for decision in decisions if decision.label == target_label)

    def info(self) -> None:
        """
        Show configuration and training information for this Judge.
        """
        print(f"Judge: {self.name}")
        print(f"Labels: {self.labels}")
        if self.target_prior != 0.5:
            print(
                f"Target prior: {self.target_prior:.2f} (favors '{self.labels[0]}')"
                if self.target_prior > 0.5
                else f"Target prior: {self.target_prior:.2f} (favors '{self.labels[1]}')"
            )

        # Get training counts
        counts = self._get_training_counts()
        print("\nTraining examples:")
        for label in self.labels:
            count = counts.get(label, 0)
            print(f"  {label}: {count}")

        if counts.get("unlabeled", 0) > 0:
            print(f"  unlabeled: {counts['unlabeled']}")

        # Show actual imbalance
        labeled_counts = [counts.get(label, 0) for label in self.labels]
        if all(c > 0 for c in labeled_counts):
            max_count = max(labeled_counts)
            min_count = min(labeled_counts)
            if max_count != min_count:
                # Find which is which
                majority_label: Optional[str] = None
                minority_label: Optional[str] = None
                for i, label in enumerate(self.labels):
                    if counts.get(label, 0) == max_count:
                        majority_label = label
                    if counts.get(label, 0) == min_count:
                        minority_label = label

                ratio = max_count / min_count
                if majority_label is not None and minority_label is not None:
                    print(
                        f"\nClass imbalance: {majority_label}:{minority_label} = {max_count}:{min_count} ({ratio:.1f}:1)"
                    )
                    print("  Using Youden's J weights with soft voting and prior correction")

    def inspect(self, preview: bool = True) -> None:
        """
        Inspect all stored examples, showing their true labels and predicted labels/scores.
        Useful for debugging classification issues.

        Args:
            preview: If True (default), display images inline in HTML tables (requires IPython/Jupyter).
                     If False, use text-only output.
        """
        if not self.thresholds:
            print("No trained model yet. Add examples and the model will auto-train.")
            return

        if not preview:
            # Show basic info first
            self.info()
            print("-" * 80)

            print("\nThresholds learned:")
            for metric, info in self.thresholds.items():
                weight = info["accuracy"]  # This is now Youden's J
                selection_acc = info.get(
                    "selection_accuracy", info["accuracy"]
                )  # Fallback for old models
                print(f"  {metric}: weight={weight:.3f} (selection_accuracy={selection_acc:.3f})")
                for label, (threshold, direction) in info["thresholds"].items():
                    print(f"    {label}: {direction} than {threshold:.3f}")

                # Show metric distribution info if available
                if metric in self.metrics_info:
                    metric_stats = self.metrics_info[metric]
                    for label in self.labels:
                        mean_key = f"mean_{label}"
                        std_key = f"std_{label}"
                        if mean_key in metric_stats:
                            print(
                                f"    {label} distribution: mean={metric_stats[mean_key]:.3f}, std={metric_stats[std_key]:.3f}"
                            )

        HTML = None
        display = None

        if preview:
            # HTML preview mode
            try:
                from IPython.display import HTML as _HTML
                from IPython.display import display as _display

                HTML = _HTML
                display = _display
            except ImportError:
                print("Preview mode requires IPython/Jupyter. Falling back to text mode.")
                preview = False
        if preview and (HTML is None or display is None):
            preview = False

        if preview:
            # Build HTML tables for everything
            assert HTML is not None and display is not None
            html_parts = []
            html_parts.append("<style>")
            html_parts.append("table { border-collapse: collapse; margin: 20px 0; }")
            html_parts.append("th, td { border: 1px solid #ddd; padding: 8px; text-align: left; }")
            html_parts.append("th { background-color: #f2f2f2; font-weight: bold; }")
            html_parts.append("img { max-width: 60px; max-height: 60px; }")
            html_parts.append(".correct { color: green; }")
            html_parts.append(".incorrect { color: red; }")
            html_parts.append(".metrics { font-size: 0.9em; color: #666; }")
            html_parts.append("h3 { margin-top: 30px; }")
            html_parts.append(".imbalance-warning { background-color: #fff3cd; color: #856404; }")
            html_parts.append("</style>")

            # Configuration table
            html_parts.append("<h3>Judge Configuration</h3>")
            html_parts.append("<table>")
            html_parts.append("<tr><th>Property</th><th>Value</th></tr>")
            html_parts.append(f"<tr><td>Name</td><td>{self.name}</td></tr>")
            html_parts.append(f"<tr><td>Labels</td><td>{', '.join(self.labels)}</td></tr>")
            html_parts.append(f"<tr><td>Target Prior</td><td>{self.target_prior:.2f}")
            if self.target_prior != 0.5:
                html_parts.append(
                    f" (favors '{self.labels[0] if self.target_prior > 0.5 else self.labels[1]}')"
                )
            html_parts.append("</td></tr>")
            html_parts.append("</table>")

            # Training counts table
            counts = self._get_training_counts()
            html_parts.append("<h3>Training Examples</h3>")
            html_parts.append("<table>")
            html_parts.append("<tr><th>Class</th><th>Count</th></tr>")

            # Check for imbalance
            labeled_counts = [counts.get(label, 0) for label in self.labels]
            is_imbalanced = False
            ratio: Optional[float] = None
            if all(c > 0 for c in labeled_counts):
                max_count = max(labeled_counts)
                min_count = min(labeled_counts)
                if max_count != min_count:
                    ratio = max_count / min_count
                    is_imbalanced = ratio > 1.5

            for label in self.labels:
                count = counts.get(label, 0)
                row_class = ""
                if is_imbalanced:
                    if count == max(labeled_counts):
                        row_class = ' class="imbalance-warning"'
                html_parts.append(f"<tr{row_class}><td>{label}</td><td>{count}</td></tr>")

            if counts.get("unlabeled", 0) > 0:
                html_parts.append(f"<tr><td>unlabeled</td><td>{counts['unlabeled']}</td></tr>")

            html_parts.append("</table>")

            if is_imbalanced and ratio is not None:
                html_parts.append(
                    f"<p><em>Class imbalance detected ({ratio:.1f}:1). Using Youden's J weights with prior correction.</em></p>"
                )

            # Thresholds table
            html_parts.append("<h3>Learned Thresholds</h3>")
            html_parts.append("<table>")
            html_parts.append(
                "<tr><th>Metric</th><th>Weight (Youden's J)</th><th>Selection Accuracy</th><th>Threshold Details</th></tr>"
            )

            for metric, info in self.thresholds.items():
                weight = info["accuracy"]  # This is Youden's J
                selection_acc = info.get("selection_accuracy", weight)

                # Build threshold details
                details = []
                for label, (threshold, direction) in info["thresholds"].items():
                    details.append(f"<br>{label}: {direction} than {threshold:.3f}")

                # Add distribution info if available
                if metric in self.metrics_info:
                    metric_stats = self.metrics_info[metric]
                    details.append("<br><em>Distributions:</em>")
                    for label in self.labels:
                        mean_key = f"mean_{label}"
                        std_key = f"std_{label}"
                        if mean_key in metric_stats:
                            details.append(
                                f"<br>&nbsp;&nbsp;{label}: μ={metric_stats[mean_key]:.1f}, σ={metric_stats[std_key]:.1f}"
                            )

                html_parts.append("<tr>")
                html_parts.append(f"<td>{metric}</td>")
                html_parts.append(f"<td>{weight:.3f}</td>")
                html_parts.append(f"<td>{selection_acc:.3f}</td>")
                html_parts.append(f"<td>{''.join(details)}</td>")
                html_parts.append("</tr>")

            html_parts.append("</table>")

            all_correct = 0
            all_total = 0

            # First show labeled examples
            for true_label in self.labels:
                label_dir = self.root_dir / true_label
                examples = list(label_dir.glob("*.png"))

                if not examples:
                    continue

                html_parts.append(
                    f"<h3>Predictions: {true_label.upper()} ({len(examples)} total)</h3>"
                )
                html_parts.append("<table>")
                html_parts.append(
                    "<tr><th>Image</th><th>Status</th><th>Predicted</th><th>Score</th><th>Key Metrics</th></tr>"
                )

                correct = 0

                for img_path in sorted(examples)[:20]:  # Show max 20 per class in preview
                    # Load image
                    img = Image.open(img_path)
                    preview_region = PreviewRegion(img)

                    # Get prediction
                    decision = cast(Decision, self.decide(preview_region))
                    is_correct = decision.label == true_label
                    if is_correct:
                        correct += 1

                    # Extract metrics
                    metrics = self._extract_metrics(preview_region)

                    # Convert image to base64
                    buffered = io.BytesIO()
                    img.save(buffered, format="PNG")
                    img_str = base64.b64encode(buffered.getvalue()).decode()

                    # Build row
                    status_class = "correct" if is_correct else "incorrect"
                    status_symbol = "✓" if is_correct else "✗"

                    # Format key metrics
                    metric_strs = []
                    for metric, value in sorted(metrics.items()):
                        if metric in self.thresholds:
                            metric_strs.append(f"{metric}={value:.1f}")
                    metrics_html = "<br>".join(metric_strs[:3])

                    html_parts.append("<tr>")
                    html_parts.append(f'<td><img src="data:image/png;base64,{img_str}" /></td>')
                    html_parts.append(f'<td class="{status_class}">{status_symbol}</td>')
                    html_parts.append(f"<td>{decision.label}</td>")
                    html_parts.append(f"<td>{decision.score:.3f}</td>")
                    html_parts.append(f'<td class="metrics">{metrics_html}</td>')
                    html_parts.append("</tr>")

                html_parts.append("</table>")

                accuracy = correct / len(examples) if examples else 0
                all_correct += correct
                all_total += len(examples)

                if len(examples) > 20:
                    html_parts.append(f"<p><em>... and {len(examples) - 20} more</em></p>")
                html_parts.append(
                    f"<p>Accuracy for {true_label}: <strong>{accuracy:.1%}</strong> ({correct}/{len(examples)})</p>"
                )

            if all_total > 0:
                overall_accuracy = all_correct / all_total
                html_parts.append(
                    f"<h3>Overall accuracy: {overall_accuracy:.1%} ({all_correct}/{all_total})</h3>"
                )

            # Now show unlabeled examples with predictions
            unlabeled_dir = self.root_dir / "unlabeled"
            unlabeled_examples = list(unlabeled_dir.glob("*.png"))

            if unlabeled_examples:
                html_parts.append(
                    f"<h3>Predictions: UNLABELED ({len(unlabeled_examples)} total)</h3>"
                )
                html_parts.append("<table>")
                html_parts.append(
                    "<tr><th>Image</th><th>Predicted</th><th>Score</th><th>Key Metrics</th></tr>"
                )

                for img_path in sorted(unlabeled_examples)[:20]:  # Show max 20
                    # Load image
                    img = Image.open(img_path)
                    preview_region = PreviewRegion(img)

                    # Get prediction
                    decision = cast(Decision, self.decide(preview_region))

                    # Extract metrics
                    metrics = self._extract_metrics(preview_region)

                    # Convert image to base64
                    buffered = io.BytesIO()
                    img.save(buffered, format="PNG")
                    img_str = base64.b64encode(buffered.getvalue()).decode()

                    # Format key metrics
                    metric_strs = []
                    for metric, value in sorted(metrics.items()):
                        if metric in self.thresholds:
                            metric_strs.append(f"{metric}={value:.1f}")
                    metrics_html = "<br>".join(metric_strs[:3])

                    html_parts.append("<tr>")
                    html_parts.append(f'<td><img src="data:image/png;base64,{img_str}" /></td>')
                    html_parts.append(f"<td>{decision.label}</td>")
                    html_parts.append(f"<td>{decision.score:.3f}</td>")
                    html_parts.append(f'<td class="metrics">{metrics_html}</td>')
                    html_parts.append("</tr>")

                html_parts.append("</table>")

                if len(unlabeled_examples) > 20:
                    html_parts.append(
                        f"<p><em>... and {len(unlabeled_examples) - 20} more</em></p>"
                    )

            # Display HTML
            display(HTML("".join(html_parts)))

        else:
            # Text mode (original)
            print("\nPredictions on training data:")
            print("-" * 80)

            # Test each labeled example
            all_correct = 0
            all_total = 0

            for true_label in self.labels:
                label_dir = self.root_dir / true_label
                examples = list(label_dir.glob("*.png"))

                if not examples:
                    continue

                print(f"\n{true_label.upper()} examples ({len(examples)} total):")
                correct = 0

                for img_path in sorted(examples)[:10]:  # Show max 10 per class
                    # Load image and create mock region
                    img = Image.open(img_path)
                    preview_region = PreviewRegion(img)

                    # Get prediction
                    decision = cast(Decision, self.decide(preview_region))
                    is_correct = decision.label == true_label
                    if is_correct:
                        correct += 1

                    # Extract metrics for this example
                    metrics = self._extract_metrics(preview_region)

                    # Show result
                    status = "✓" if is_correct else "✗"
                    print(
                        f"  {status} {img_path.name}: predicted={decision.label} (score={decision.score:.3f})"
                    )

                    # Show key metric values
                    metric_strs = []
                    for metric, value in sorted(metrics.items()):
                        if metric in self.thresholds:
                            metric_strs.append(f"{metric}={value:.2f}")
                    if metric_strs:
                        print(f"     Metrics: {', '.join(metric_strs[:3])}")

                accuracy = correct / len(examples) if examples else 0
                all_correct += correct
                all_total += len(examples)

                if len(examples) > 10:
                    print(f"  ... and {len(examples) - 10} more")
                print(f"  Accuracy for {true_label}: {accuracy:.1%} ({correct}/{len(examples)})")

            if all_total > 0:
                overall_accuracy = all_correct / all_total
                print(f"\nOverall accuracy: {overall_accuracy:.1%} ({all_correct}/{all_total})")

            # Show unlabeled examples with predictions
            unlabeled_dir = self.root_dir / "unlabeled"
            unlabeled_examples = list(unlabeled_dir.glob("*.png"))

            if unlabeled_examples:
                print(f"\nUNLABELED examples ({len(unlabeled_examples)} total) - predictions:")

                for img_path in sorted(unlabeled_examples)[:10]:  # Show max 10
                    # Load image and create preview region
                    img = Image.open(img_path)
                    preview_region = PreviewRegion(img)

                    # Get prediction
                    decision = cast(Decision, self.decide(preview_region))

                    # Extract metrics
                    metrics = self._extract_metrics(preview_region)

                    print(
                        f"  {img_path.name}: predicted={decision.label} (score={decision.score:.3f})"
                    )

                    # Show key metric values
                    metric_strs = []
                    for metric, value in sorted(metrics.items()):
                        if metric in self.thresholds:
                            metric_strs.append(f"{metric}={value:.2f}")
                    if metric_strs:
                        print(f"     Metrics: {', '.join(metric_strs[:3])}")

                if len(unlabeled_examples) > 10:
                    print(f"  ... and {len(unlabeled_examples) - 10} more")

    def lookup(self, region: SupportsRender) -> Optional[Tuple[str, Image.Image]]:
        """
        Look up a region and return its hash and image if found in training data.

        Args:
            region: Region to look up

        Returns:
            Tuple of (hash, image) if found, None if not found
        """
        try:
            # Generate hash for the region
            img = region.render(crop=True)
            if img is None:
                raise JudgeError("Region.render returned no image")
            if not isinstance(img, Image.Image):
                img = Image.fromarray(np.asarray(img))
            if img.mode != "RGB":
                img = img.convert("RGB")
            img_array = np.array(img)
            img_hash = hashlib.md5(img_array.tobytes()).hexdigest()[:12]

            # Look for the image in all directories
            for subdir in ["checked", "unchecked", "unlabeled", "_removed"]:
                if subdir == "checked" or subdir == "unchecked":
                    # Only look in valid label directories
                    if subdir not in self.labels:
                        continue

                img_path = self.root_dir / subdir / f"{img_hash}.png"
                if img_path.exists():
                    stored_img = Image.open(img_path)
                    logger.debug(f"Found region in '{subdir}' with hash {img_hash}")
                    return (img_hash, stored_img)

            logger.debug(f"Region not found in training data (hash: {img_hash})")
            return None

        except Exception as e:
            logger.error(f"Failed to lookup region: {e}")
            return None

    def show(self, max_per_class: int = 10, size: Tuple[int, int] = (100, 100)) -> None:
        """
        Display a grid showing examples from each category.

        Args:
            max_per_class: Maximum number of examples to show per class
            size: Size of each image in pixels (width, height)
        """
        try:
            import ipywidgets as widgets  # type: ignore[import-untyped]
            from IPython.display import display
            from PIL import Image as PILImage
        except ImportError:
            print("Show requires IPython and ipywidgets")
            return

        # Collect images from each category
        categories = {}
        total_counts = {}
        for label in self.labels:
            label_dir = self.root_dir / label
            all_images = list(label_dir.glob("*.png"))
            total_counts[label] = len(all_images)
            images = sorted(all_images)[:max_per_class]
            if images:
                categories[label] = images

        # Add unlabeled if any
        unlabeled_dir = self.root_dir / "unlabeled"
        all_unlabeled = list(unlabeled_dir.glob("*.png"))
        total_counts["unlabeled"] = len(all_unlabeled)
        unlabeled = sorted(all_unlabeled)[:max_per_class]
        if unlabeled:
            categories["unlabeled"] = unlabeled

        if not categories:
            print("No images to show")
            return

        # Create grid layout
        rows = []

        # Check for class imbalance
        labeled_counts = {k: v for k, v in total_counts.items() if k != "unlabeled"}
        if labeled_counts and len(labeled_counts) >= 2:
            max_count = max(labeled_counts.values())
            min_count = min(labeled_counts.values())
            if min_count > 0 and max_count / min_count > 3:
                warning = widgets.HTML(
                    f'<div style="background: #fff3cd; padding: 10px; margin: 10px 0; border: 1px solid #ffeeba; border-radius: 4px;">'
                    f"<strong>⚠️ Class imbalance detected:</strong> {labeled_counts}<br>"
                    f"Consider adding more examples of the minority class for better accuracy."
                    f"</div>"
                )
                rows.append(warning)

        for category, image_paths in categories.items():
            # Category header showing total count
            shown = len(image_paths)
            total = total_counts[category]
            header_text = f"<h3>{category}"
            if shown < total:
                header_text += f" ({shown} of {total} shown)"
            else:
                header_text += f" ({total} total)"
            header_text += "</h3>"
            header = widgets.HTML(header_text)

            # Image row
            image_widgets = []
            for img_path in image_paths:
                # Load and resize image
                img = PILImage.open(img_path)
                img.thumbnail(size, PILImage.Resampling.LANCZOS)

                # Convert to bytes for display
                import io

                img_bytes = io.BytesIO()
                img.save(img_bytes, format="PNG")
                img_bytes.seek(0)

                # Create image widget
                img_widget = widgets.Image(value=img_bytes.read(), width=size[0], height=size[1])
                image_widgets.append(img_widget)

            # Create horizontal box for this category
            category_box = widgets.VBox([header, widgets.HBox(image_widgets)])
            rows.append(category_box)

        # Display all categories
        display(widgets.VBox(rows))

    def forget(self, region: Optional[SupportsRender] = None, delete: bool = False) -> None:
        """
        Clear training data, delete all files, or move a specific region to unlabeled.

        Args:
            region: If provided, move this specific region to unlabeled
            delete: If True, permanently delete all files
        """
        # Handle specific region case
        if region is not None:
            # Get hash of the region
            try:
                img = region.render(crop=True)
                if img is None:
                    raise ValueError("Region.render returned no image")
                if not isinstance(img, Image.Image):
                    img = Image.fromarray(np.asarray(img))
                if img.mode != "RGB":
                    img = img.convert("RGB")
                img_array = np.array(img)
                img_hash = hashlib.md5(img_array.tobytes()).hexdigest()[:12]
            except Exception as e:
                logger.error(f"Failed to hash region: {e}")
                return

            # Find and move the image
            moved = False
            for label in self.labels + ["_removed"]:
                source_path = self.root_dir / label / f"{img_hash}.png"
                if source_path.exists():
                    target_path = self.root_dir / "unlabeled" / f"{img_hash}.png"
                    shutil.move(str(source_path), str(target_path))
                    print(f"Moved region from '{label}' to 'unlabeled'")
                    moved = True
                    break

            if not moved:
                print("Region not found in training data")
            return

        # Handle delete or clear training
        if delete:
            # Delete entire directory
            if self.root_dir.exists():
                shutil.rmtree(self.root_dir)
                print(f"Deleted all data for judge '{self.name}'")
            else:
                print(f"No data found for judge '{self.name}'")

            # Reset internal state
            self.thresholds = {}  # type: Dict[str, Dict[str, Any]]
            self.metrics_info = {}  # type: Dict[str, Dict[str, float]]

            # Recreate directory structure
            self.root_dir.mkdir(exist_ok=True)
            for label in self.labels:
                (self.root_dir / label).mkdir(exist_ok=True)
            (self.root_dir / "unlabeled").mkdir(exist_ok=True)
            (self.root_dir / "_removed").mkdir(exist_ok=True)

        else:
            # Just clear training (move everything to unlabeled)
            moved_count = 0

            # Move all labeled images back to unlabeled
            unlabeled_dir = self.root_dir / "unlabeled"
            for label in self.labels:
                label_dir = self.root_dir / label
                if label_dir.exists():
                    for img_path in label_dir.glob("*.png"):
                        shutil.move(str(img_path), str(unlabeled_dir / img_path.name))
                        moved_count += 1

            # Clear thresholds
            self.thresholds = {}  # type: Dict[str, Dict[str, Any]]
            self.metrics_info = {}  # type: Dict[str, Dict[str, float]]

            # Remove saved config
            if self.config_path.exists():
                self.config_path.unlink()

            print(f"Moved {moved_count} labeled images back to unlabeled.")
            print("Training data cleared. Judge is now untrained.")

    def save(self, path: Optional[Union[str, Path]] = None) -> None:
        """
        Save the judge configuration (auto-retrains first).

        Args:
            path: Optional path to save to. Defaults to judge.json in root directory
        """
        # Retrain with current examples
        self._retrain()

        # Save config
        save_path = Path(path) if path else self.config_path

        config = {
            "name": self.name,
            "labels": self.labels,
            "target_prior": self.target_prior,
            "thresholds": self.thresholds,
            "metrics_info": self.metrics_info,
            "training_counts": self._get_training_counts(),
        }

        with open(save_path, "w") as f:
            json.dump(config, f, indent=2)

        logger.info(f"Saved judge to {save_path}")

    @classmethod
    def load(cls, path: Union[str, Path]) -> "Judge":
        """
        Load a judge from a saved configuration.

        Args:
            path: Path to the saved judge.json file or the judge directory

        Returns:
            Loaded Judge instance
        """
        path = Path(path)

        # If path is a directory, look for judge.json inside
        if path.is_dir():
            config_path = path / "judge.json"
            base_dir = path.parent
            name = path.name
        else:
            config_path = path
            base_dir = path.parent.parent if path.parent.name != "." else path.parent
            # Try to infer name from path
            name = None

        with open(config_path, "r") as f:
            config = json.load(f)

        # Use saved name if we couldn't infer it
        if name is None:
            name = config["name"]

        # Create judge with saved config
        judge = cls(
            name,
            labels=config["labels"],
            base_dir=base_dir,
            target_prior=config.get("target_prior", 0.5),
        )  # Default to 0.5 for old configs
        judge.thresholds = cast(Dict[str, Dict[str, Any]], config.get("thresholds", {}))
        judge.metrics_info = cast(Dict[str, Dict[str, float]], config.get("metrics_info", {}))

        return judge

    # Private methods

    def _extract_metrics(self, region: SupportsRender) -> Dict[str, float]:
        """Extract image metrics from a region."""
        try:
            img = region.render(crop=True)
            if img is None:
                raise JudgeError("Region.render returned no image")
            if not isinstance(img, Image.Image):
                img_array = np.asarray(img)
                img = Image.fromarray(img_array)

            # Convert to grayscale for analysis
            gray = np.array(img.convert("L"))

            metrics = {}

            # 1. Center darkness
            h, w = gray.shape
            cy, cx = h // 2, w // 2
            center_size = min(5, h // 4, w // 4)  # Adaptive center size
            center = gray[
                max(0, cy - center_size) : min(h, cy + center_size + 1),
                max(0, cx - center_size) : min(w, cx + center_size + 1),
            ]
            metrics["center_darkness"] = 255 - np.mean(center)

            # 2. Overall darkness (ink density)
            metrics["ink_density"] = 255 - np.mean(gray)

            # 3. Dark pixel ratio
            metrics["dark_pixel_ratio"] = np.sum(gray < 200) / gray.size

            # 4. Standard deviation (complexity)
            metrics["std_dev"] = np.std(gray)

            # 5. Edge vs center ratio
            edge_size = max(2, min(h // 10, w // 10))
            edge_mask = np.zeros_like(gray, dtype=bool)
            edge_mask[:edge_size, :] = True
            edge_mask[-edge_size:, :] = True
            edge_mask[:, :edge_size] = True
            edge_mask[:, -edge_size:] = True

            edge_mean = np.mean(gray[edge_mask]) if np.any(edge_mask) else 255
            center_mean = np.mean(center)
            metrics["edge_center_ratio"] = edge_mean / (center_mean + 1)

            # 6. Diagonal density (for X patterns)
            if h > 10 and w > 10:
                diag_mask = np.zeros_like(gray, dtype=bool)
                for i in range(min(h, w)):
                    if i < h and i < w:
                        diag_mask[i, i] = True
                        diag_mask[i, w - 1 - i] = True
                metrics["diagonal_density"] = 255 - np.mean(gray[diag_mask])
            else:
                metrics["diagonal_density"] = metrics["ink_density"]

            return metrics

        except Exception as e:
            raise JudgeError(f"Failed to extract metrics: {e}")

    def _retrain(self) -> None:
        """Retrain thresholds from current examples."""
        # Collect all examples
        examples: Dict[str, List[Dict[str, float]]] = {label: [] for label in self.labels}

        for label in self.labels:
            label_dir = self.root_dir / label
            for img_path in label_dir.glob("*.png"):
                img = Image.open(img_path)
                preview_region = PreviewRegion(img)
                metrics = self._extract_metrics(preview_region)
                examples[label].append(metrics)

        # Check we have examples
        for label, exs in examples.items():
            if not exs:
                logger.warning(f"No examples for class '{label}'")
                return

        # Check for class imbalance
        example_counts = {label: len(exs) for label, exs in examples.items()}
        max_count = max(example_counts.values())
        min_count = min(example_counts.values())

        imbalance_ratio = max_count / min_count if min_count > 0 else float("inf")
        is_imbalanced = imbalance_ratio > 1.5  # Consider imbalanced if more than 1.5x difference

        if is_imbalanced:
            logger.info(
                f"Class imbalance detected: {example_counts} (ratio {imbalance_ratio:.1f}:1)"
            )
            logger.info("Using balanced accuracy for threshold selection")

        # Find best thresholds for each metric
        self.thresholds = {}  # type: Dict[str, Dict[str, Any]]
        self.metrics_info = {}  # type: Dict[str, Dict[str, float]]
        metric_candidates: List[Dict[str, Any]] = []

        all_metrics = set()
        for exs in examples.values():
            for ex in exs:
                all_metrics.update(ex.keys())

        for metric in all_metrics:
            # Get all values for this metric
            values_by_label = {}
            for label, exs in examples.items():
                values_by_label[label] = [ex.get(metric, 0) for ex in exs]

            # Find threshold that best separates classes (for binary)
            if len(self.labels) == 2:
                label1, label2 = self.labels
                vals1 = values_by_label[label1]
                vals2 = values_by_label[label2]

                # Try different thresholds
                all_vals = vals1 + vals2
                best_threshold = None
                best_accuracy = 0
                best_direction = None

                for threshold in np.percentile(all_vals, [10, 20, 30, 40, 50, 60, 70, 80, 90]):
                    # Test both directions
                    for direction in ["higher", "lower"]:
                        if direction == "higher":
                            correct1 = sum(1 for v in vals1 if v > threshold)
                            correct2 = sum(1 for v in vals2 if v <= threshold)
                        else:
                            correct1 = sum(1 for v in vals1 if v < threshold)
                            correct2 = sum(1 for v in vals2 if v >= threshold)

                        # Always use balanced accuracy for threshold selection
                        # This finds fair thresholds regardless of class imbalance
                        acc1 = correct1 / len(vals1) if len(vals1) > 0 else 0
                        acc2 = correct2 / len(vals2) if len(vals2) > 0 else 0
                        accuracy = (acc1 + acc2) / 2

                        if accuracy > best_accuracy:
                            best_accuracy = accuracy
                            best_threshold = threshold
                            best_direction = direction

                if best_threshold is None or best_direction is None:
                    continue

                # Calculate Youden's J statistic for weight (TPR - FPR)
                if best_direction == "higher":
                    tp = sum(1 for v in vals1 if v > best_threshold)
                    fn = len(vals1) - tp
                    tn = sum(1 for v in vals2 if v <= best_threshold)
                    fp = len(vals2) - tn
                else:
                    tp = sum(1 for v in vals1 if v < best_threshold)
                    fn = len(vals1) - tp
                    tn = sum(1 for v in vals2 if v >= best_threshold)
                    fp = len(vals2) - tn

                tpr = tp / len(vals1) if len(vals1) > 0 else 0
                fpr = fp / len(vals2) if len(vals2) > 0 else 0
                youden_j = max(0.0, min(1.0, tpr - fpr))

                # Store all candidates
                metric_candidates.append(
                    {
                        "metric": metric,
                        "youden_j": youden_j,
                        "selection_accuracy": best_accuracy,
                        "threshold": best_threshold,
                        "direction": best_direction,
                        "label1": label1,
                        "label2": label2,
                        "stats": {
                            "mean_" + label1: np.mean(vals1),
                            "mean_" + label2: np.mean(vals2),
                            "std_" + label1: np.std(vals1),
                            "std_" + label2: np.std(vals2),
                        },
                    }
                )

        # Sort by selection accuracy
        metric_candidates.sort(key=lambda x: x["selection_accuracy"], reverse=True)

        # Use relaxed cutoff when imbalanced
        keep_cutoff = 0.55 if is_imbalanced else 0.60

        # Keep metrics that pass cutoff, or top 3 if none pass
        kept_metrics = [m for m in metric_candidates if m["selection_accuracy"] > keep_cutoff]
        if not kept_metrics and metric_candidates:
            # Keep top 3 metrics even if they don't pass cutoff
            kept_metrics = metric_candidates[:3]
            logger.warning(
                f"No metrics passed cutoff {keep_cutoff}, keeping top {len(kept_metrics)} metrics"
            )

        # Store selected metrics
        for candidate in kept_metrics:
            metric = candidate["metric"]
            label1 = candidate["label1"]
            label2 = candidate["label2"]
            self.thresholds[metric] = {
                "accuracy": candidate["youden_j"],  # Use Youden's J as weight
                "selection_accuracy": candidate["selection_accuracy"],
                "thresholds": {
                    label1: (candidate["threshold"], candidate["direction"]),
                    label2: (
                        candidate["threshold"],
                        "lower" if candidate["direction"] == "higher" else "higher",
                    ),
                },
            }
            self.metrics_info[metric] = candidate["stats"]

    def _load_config(self) -> None:
        """Load configuration from file."""
        try:
            with open(self.config_path, "r") as f:
                config = json.load(f)

            self.thresholds = config.get("thresholds", {})
            self.metrics_info = config.get("metrics_info", {})

            # Verify labels match
            if config.get("labels") != self.labels:
                logger.warning(
                    f"Saved labels {config.get('labels')} don't match current {self.labels}"
                )

        except Exception as e:
            logger.warning(f"Failed to load config: {e}")

    def _get_training_counts(self) -> Dict[str, int]:
        """Get count of examples per class."""
        counts: Dict[str, int] = {}
        for label in self.labels:
            label_dir = self.root_dir / label
            counts[label] = len(list(label_dir.glob("*.png")))
        counts["unlabeled"] = len(list((self.root_dir / "unlabeled").glob("*.png")))
        return counts

Functions

natural_pdf.Judge.__init__(name, labels, base_dir=None, target_prior=None)

Initialize a Judge for visual classification.

Parameters:

Name	Type	Description	Default
name	str	Name for this judge (used for folder name)	required
labels	List[str]	Class labels (required, typically 2 for binary classification)	required
base_dir	Optional[Union[str, Path]]	Base directory for storage. Defaults to current directory	None
target_prior	Optional[float]	Target prior probability for the FIRST label in the labels list. - 0.5 (default) = neutral, treats both classes equally - >0.5 = favors labels[0] - <0.5 = favors labels[1] Example: Judge("cb", ["checked", "unchecked"], target_prior=0.6) favors detecting "checked" checkboxes.	None

Source code in natural_pdf/judge.py

def __init__(
    self,
    name: str,
    labels: List[str],
    base_dir: Optional[Union[str, Path]] = None,
    target_prior: Optional[float] = None,
):
    """
    Initialize a Judge for visual classification.

    Args:
        name: Name for this judge (used for folder name)
        labels: Class labels (required, typically 2 for binary classification)
        base_dir: Base directory for storage. Defaults to current directory
        target_prior: Target prior probability for the FIRST label in the labels list.
                     - 0.5 (default) = neutral, treats both classes equally
                     - >0.5 = favors labels[0]
                     - <0.5 = favors labels[1]
                     Example: Judge("cb", ["checked", "unchecked"], target_prior=0.6)
                     favors detecting "checked" checkboxes.
    """
    if not labels or len(labels) != 2:
        raise JudgeError("Judge requires exactly 2 class labels (binary classification only)")

    self.name = name
    self.labels = labels
    self.target_prior = float(target_prior) if target_prior is not None else 0.5

    # Set up directory structure
    self.base_dir: Path = Path(base_dir) if base_dir is not None else Path.cwd()
    self.root_dir: Path = self.base_dir / name
    self.root_dir.mkdir(exist_ok=True)

    # Create label directories
    for label in self.labels:
        (self.root_dir / label).mkdir(exist_ok=True)
    (self.root_dir / "unlabeled").mkdir(exist_ok=True)
    (self.root_dir / "_removed").mkdir(exist_ok=True)

    # Config file
    self.config_path: Path = self.root_dir / "judge.json"

    # Load existing config or initialize
    self.thresholds: Dict[str, Dict[str, Any]] = {}
    self.metrics_info: Dict[str, Dict[str, float]] = {}
    if self.config_path.exists():
        self._load_config()

natural_pdf.Judge.add(region, label=None)

Add a region to the judge's dataset.

Parameters:

Name	Type	Description	Default
region	SupportsRender	Region object to add	required
label	Optional[str]	Class label. If None, added to unlabeled for later teaching	None

Raises:

Type	Description
JudgeError	If label is not in allowed labels

Source code in natural_pdf/judge.py

def add(self, region: SupportsRender, label: Optional[str] = None) -> None:
    """
    Add a region to the judge's dataset.

    Args:
        region: Region object to add
        label: Class label. If None, added to unlabeled for later teaching

    Raises:
        JudgeError: If label is not in allowed labels
    """
    if label is not None and label not in self.labels:
        raise JudgeError(f"Label '{label}' not in allowed labels: {self.labels}")

    # Render region to image
    try:
        img = region.render(crop=True)
        if img is None:
            raise JudgeError("Region.render returned no image.")
        if not isinstance(img, Image.Image):
            img = Image.fromarray(np.asarray(img))
    except Exception as e:
        raise JudgeError(f"Failed to render region: {e}")

    # Convert to RGB if needed
    if img.mode != "RGB":
        img = img.convert("RGB")

    # Generate hash from image content
    img_array = np.array(img)
    img_hash = hashlib.md5(img_array.tobytes()).hexdigest()[:12]

    # Determine target directory
    target_dir = self.root_dir / (label if label else "unlabeled")
    target_path = target_dir / f"{img_hash}.png"

    # Check if hash already exists anywhere
    existing_locations = []
    for check_label in self.labels + ["unlabeled", "_removed"]:
        check_path = self.root_dir / check_label / f"{img_hash}.png"
        if check_path.exists():
            existing_locations.append(check_label)

    if existing_locations:
        logger.warning(f"Duplicate image detected (hash: {img_hash})")
        logger.warning(f"Already exists in: {', '.join(existing_locations)}")
        print(f"⚠️  Duplicate image - already exists in: {', '.join(existing_locations)}")
        return

    # Save image
    img.save(target_path)
    logger.debug(f"Added image {img_hash} to {label if label else 'unlabeled'}")

natural_pdf.Judge.count(target_label, regions)

Count how many regions match the target label.

Parameters:

Name	Type	Description	Default
target_label	str	The class label to count	required
regions	Iterable[SupportsRender]	List of regions to check	required

Returns:

Type	Description
int	Number of regions classified as target_label

Source code in natural_pdf/judge.py

def count(self, target_label: str, regions: Iterable[SupportsRender]) -> int:
    """
    Count how many regions match the target label.

    Args:
        target_label: The class label to count
        regions: List of regions to check

    Returns:
        Number of regions classified as target_label
    """
    decisions_result = self.decide(regions)
    decisions = decisions_result if isinstance(decisions_result, list) else [decisions_result]
    return sum(1 for decision in decisions if decision.label == target_label)

natural_pdf.Judge.decide(regions)

Classify one or more regions.

Parameters:

Name	Type	Description	Default
regions	Union[SupportsRender, Iterable[SupportsRender]]	Single region or list of regions to classify	required

Returns:

Type	Description
Union[Decision, List[Decision]]	Decision or list of Decisions with label and score

Raises:

Type	Description
JudgeError	If not enough training examples

Source code in natural_pdf/judge.py

def decide(
    self, regions: Union[SupportsRender, Iterable[SupportsRender]]
) -> Union[Decision, List[Decision]]:
    """
    Classify one or more regions.

    Args:
        regions: Single region or list of regions to classify

    Returns:
        Decision or list of Decisions with label and score

    Raises:
        JudgeError: If not enough training examples
    """
    # Check if we have examples
    for label in self.labels:
        label_dir = self.root_dir / label
        if not any(label_dir.glob("*.png")):
            raise JudgeError(f"Need at least one example of class '{label}' before deciding")

    # Ensure thresholds are current
    if not self.thresholds:
        self._retrain()

    # Normalize to list of regions
    if isinstance(regions, IterableABC) and not isinstance(regions, (str, bytes)):
        region_list: List[SupportsRender] = list(regions)
        single_input = False
    else:
        region_list = [cast(SupportsRender, regions)]
        single_input = True

    results: List[Decision] = []
    for region in region_list:
        # Extract metrics
        metrics = self._extract_metrics(region)

        # Apply thresholds with soft voting
        votes: Dict[str, float] = {label: 0.0 for label in self.labels}
        total_weight = 0.0

        for metric_name, value in metrics.items():
            if metric_name in self.thresholds:
                metric_info = self.thresholds[metric_name]
                weight = metric_info["accuracy"]  # This is now Youden's J

                # For binary classification
                label1, label2 = self.labels
                threshold1, direction1 = metric_info["thresholds"][label1]

                # Get standard deviations for soft voting
                stats = self.metrics_info.get(metric_name, {})
                s1 = stats.get(f"std_{label1}", 0.0)
                s2 = stats.get(f"std_{label2}", 0.0)
                scale1 = s1 if s1 > 1e-6 else 1.0
                scale2 = s2 if s2 > 1e-6 else 1.0

                # Calculate signed margin (positive favors label1, negative favors label2)
                if direction1 == "higher":
                    margin = (value - threshold1) / (scale1 if value >= threshold1 else scale2)
                else:
                    margin = (threshold1 - value) / (scale1 if value <= threshold1 else scale2)

                # Clip margin to avoid single metric dominating
                margin = np.clip(margin, -6, 6)

                # Soft votes using sigmoid
                p1 = 1.0 / (1.0 + np.exp(-margin))
                p2 = 1.0 - p1

                votes[label1] += weight * p1
                votes[label2] += weight * p2
                total_weight += weight

        # Normalize votes
        if total_weight > 0:
            for label in votes:
                votes[label] /= total_weight
        else:
            # Fallback: uniform votes so prior still works
            for label in votes:
                votes[label] = 0.5
            total_weight = 1.0

        # Apply prior bias correction
        def _logit(p, eps=1e-6):
            p = max(eps, min(1 - eps, p))
            return np.log(p / (1 - p))

        def _sigmoid(x):
            if x >= 0:
                z = np.exp(-x)
                return 1.0 / (1.0 + z)
            else:
                z = np.exp(x)
                return z / (1.0 + z)

        # Estimate priors from training counts
        counts = self._get_training_counts()
        label1, label2 = self.labels
        n1 = counts.get(label1, 0)
        n2 = counts.get(label2, 0)
        total = max(1, n1 + n2)

        if n1 > 0 and n2 > 0:  # Only apply bias if we have examples of both classes
            emp_prior1 = n1 / total
            emp_prior2 = n2 / total

            # Target prior (0.5/0.5 neutralizes imbalance)
            target_prior1 = self.target_prior
            target_prior2 = 1.0 - self.target_prior

            # Calculate bias
            bias1 = _logit(target_prior1) - _logit(emp_prior1)
            bias2 = _logit(target_prior2) - _logit(emp_prior2)

            # Apply bias in logit space
            v1 = _sigmoid(_logit(votes[label1]) + bias1)
            v2 = _sigmoid(_logit(votes[label2]) + bias2)

            # Renormalize
            s = v1 + v2
            votes[label1] = v1 / s
            votes[label2] = v2 / s

        # Find best label
        best_label = max(votes.items(), key=lambda x: x[1])
        results.append(Decision(label=best_label[0], score=best_label[1]))

    return results[0] if single_input else results

natural_pdf.Judge.forget(region=None, delete=False)

Clear training data, delete all files, or move a specific region to unlabeled.

Parameters:

Name	Type	Description	Default
region	Optional[SupportsRender]	If provided, move this specific region to unlabeled	None
delete	bool	If True, permanently delete all files	False

Source code in natural_pdf/judge.py

def forget(self, region: Optional[SupportsRender] = None, delete: bool = False) -> None:
    """
    Clear training data, delete all files, or move a specific region to unlabeled.

    Args:
        region: If provided, move this specific region to unlabeled
        delete: If True, permanently delete all files
    """
    # Handle specific region case
    if region is not None:
        # Get hash of the region
        try:
            img = region.render(crop=True)
            if img is None:
                raise ValueError("Region.render returned no image")
            if not isinstance(img, Image.Image):
                img = Image.fromarray(np.asarray(img))
            if img.mode != "RGB":
                img = img.convert("RGB")
            img_array = np.array(img)
            img_hash = hashlib.md5(img_array.tobytes()).hexdigest()[:12]
        except Exception as e:
            logger.error(f"Failed to hash region: {e}")
            return

        # Find and move the image
        moved = False
        for label in self.labels + ["_removed"]:
            source_path = self.root_dir / label / f"{img_hash}.png"
            if source_path.exists():
                target_path = self.root_dir / "unlabeled" / f"{img_hash}.png"
                shutil.move(str(source_path), str(target_path))
                print(f"Moved region from '{label}' to 'unlabeled'")
                moved = True
                break

        if not moved:
            print("Region not found in training data")
        return

    # Handle delete or clear training
    if delete:
        # Delete entire directory
        if self.root_dir.exists():
            shutil.rmtree(self.root_dir)
            print(f"Deleted all data for judge '{self.name}'")
        else:
            print(f"No data found for judge '{self.name}'")

        # Reset internal state
        self.thresholds = {}  # type: Dict[str, Dict[str, Any]]
        self.metrics_info = {}  # type: Dict[str, Dict[str, float]]

        # Recreate directory structure
        self.root_dir.mkdir(exist_ok=True)
        for label in self.labels:
            (self.root_dir / label).mkdir(exist_ok=True)
        (self.root_dir / "unlabeled").mkdir(exist_ok=True)
        (self.root_dir / "_removed").mkdir(exist_ok=True)

    else:
        # Just clear training (move everything to unlabeled)
        moved_count = 0

        # Move all labeled images back to unlabeled
        unlabeled_dir = self.root_dir / "unlabeled"
        for label in self.labels:
            label_dir = self.root_dir / label
            if label_dir.exists():
                for img_path in label_dir.glob("*.png"):
                    shutil.move(str(img_path), str(unlabeled_dir / img_path.name))
                    moved_count += 1

        # Clear thresholds
        self.thresholds = {}  # type: Dict[str, Dict[str, Any]]
        self.metrics_info = {}  # type: Dict[str, Dict[str, float]]

        # Remove saved config
        if self.config_path.exists():
            self.config_path.unlink()

        print(f"Moved {moved_count} labeled images back to unlabeled.")
        print("Training data cleared. Judge is now untrained.")

natural_pdf.Judge.info()

Show configuration and training information for this Judge.

Source code in natural_pdf/judge.py

def info(self) -> None:
    """
    Show configuration and training information for this Judge.
    """
    print(f"Judge: {self.name}")
    print(f"Labels: {self.labels}")
    if self.target_prior != 0.5:
        print(
            f"Target prior: {self.target_prior:.2f} (favors '{self.labels[0]}')"
            if self.target_prior > 0.5
            else f"Target prior: {self.target_prior:.2f} (favors '{self.labels[1]}')"
        )

    # Get training counts
    counts = self._get_training_counts()
    print("\nTraining examples:")
    for label in self.labels:
        count = counts.get(label, 0)
        print(f"  {label}: {count}")

    if counts.get("unlabeled", 0) > 0:
        print(f"  unlabeled: {counts['unlabeled']}")

    # Show actual imbalance
    labeled_counts = [counts.get(label, 0) for label in self.labels]
    if all(c > 0 for c in labeled_counts):
        max_count = max(labeled_counts)
        min_count = min(labeled_counts)
        if max_count != min_count:
            # Find which is which
            majority_label: Optional[str] = None
            minority_label: Optional[str] = None
            for i, label in enumerate(self.labels):
                if counts.get(label, 0) == max_count:
                    majority_label = label
                if counts.get(label, 0) == min_count:
                    minority_label = label

            ratio = max_count / min_count
            if majority_label is not None and minority_label is not None:
                print(
                    f"\nClass imbalance: {majority_label}:{minority_label} = {max_count}:{min_count} ({ratio:.1f}:1)"
                )
                print("  Using Youden's J weights with soft voting and prior correction")

natural_pdf.Judge.inspect(preview=True)

Inspect all stored examples, showing their true labels and predicted labels/scores. Useful for debugging classification issues.

Parameters:

Name	Type	Description	Default
preview	bool	If True (default), display images inline in HTML tables (requires IPython/Jupyter). If False, use text-only output.	True

Source code in natural_pdf/judge.py

def inspect(self, preview: bool = True) -> None:
    """
    Inspect all stored examples, showing their true labels and predicted labels/scores.
    Useful for debugging classification issues.

    Args:
        preview: If True (default), display images inline in HTML tables (requires IPython/Jupyter).
                 If False, use text-only output.
    """
    if not self.thresholds:
        print("No trained model yet. Add examples and the model will auto-train.")
        return

    if not preview:
        # Show basic info first
        self.info()
        print("-" * 80)

        print("\nThresholds learned:")
        for metric, info in self.thresholds.items():
            weight = info["accuracy"]  # This is now Youden's J
            selection_acc = info.get(
                "selection_accuracy", info["accuracy"]
            )  # Fallback for old models
            print(f"  {metric}: weight={weight:.3f} (selection_accuracy={selection_acc:.3f})")
            for label, (threshold, direction) in info["thresholds"].items():
                print(f"    {label}: {direction} than {threshold:.3f}")

            # Show metric distribution info if available
            if metric in self.metrics_info:
                metric_stats = self.metrics_info[metric]
                for label in self.labels:
                    mean_key = f"mean_{label}"
                    std_key = f"std_{label}"
                    if mean_key in metric_stats:
                        print(
                            f"    {label} distribution: mean={metric_stats[mean_key]:.3f}, std={metric_stats[std_key]:.3f}"
                        )

    HTML = None
    display = None

    if preview:
        # HTML preview mode
        try:
            from IPython.display import HTML as _HTML
            from IPython.display import display as _display

            HTML = _HTML
            display = _display
        except ImportError:
            print("Preview mode requires IPython/Jupyter. Falling back to text mode.")
            preview = False
    if preview and (HTML is None or display is None):
        preview = False

    if preview:
        # Build HTML tables for everything
        assert HTML is not None and display is not None
        html_parts = []
        html_parts.append("<style>")
        html_parts.append("table { border-collapse: collapse; margin: 20px 0; }")
        html_parts.append("th, td { border: 1px solid #ddd; padding: 8px; text-align: left; }")
        html_parts.append("th { background-color: #f2f2f2; font-weight: bold; }")
        html_parts.append("img { max-width: 60px; max-height: 60px; }")
        html_parts.append(".correct { color: green; }")
        html_parts.append(".incorrect { color: red; }")
        html_parts.append(".metrics { font-size: 0.9em; color: #666; }")
        html_parts.append("h3 { margin-top: 30px; }")
        html_parts.append(".imbalance-warning { background-color: #fff3cd; color: #856404; }")
        html_parts.append("</style>")

        # Configuration table
        html_parts.append("<h3>Judge Configuration</h3>")
        html_parts.append("<table>")
        html_parts.append("<tr><th>Property</th><th>Value</th></tr>")
        html_parts.append(f"<tr><td>Name</td><td>{self.name}</td></tr>")
        html_parts.append(f"<tr><td>Labels</td><td>{', '.join(self.labels)}</td></tr>")
        html_parts.append(f"<tr><td>Target Prior</td><td>{self.target_prior:.2f}")
        if self.target_prior != 0.5:
            html_parts.append(
                f" (favors '{self.labels[0] if self.target_prior > 0.5 else self.labels[1]}')"
            )
        html_parts.append("</td></tr>")
        html_parts.append("</table>")

        # Training counts table
        counts = self._get_training_counts()
        html_parts.append("<h3>Training Examples</h3>")
        html_parts.append("<table>")
        html_parts.append("<tr><th>Class</th><th>Count</th></tr>")

        # Check for imbalance
        labeled_counts = [counts.get(label, 0) for label in self.labels]
        is_imbalanced = False
        ratio: Optional[float] = None
        if all(c > 0 for c in labeled_counts):
            max_count = max(labeled_counts)
            min_count = min(labeled_counts)
            if max_count != min_count:
                ratio = max_count / min_count
                is_imbalanced = ratio > 1.5

        for label in self.labels:
            count = counts.get(label, 0)
            row_class = ""
            if is_imbalanced:
                if count == max(labeled_counts):
                    row_class = ' class="imbalance-warning"'
            html_parts.append(f"<tr{row_class}><td>{label}</td><td>{count}</td></tr>")

        if counts.get("unlabeled", 0) > 0:
            html_parts.append(f"<tr><td>unlabeled</td><td>{counts['unlabeled']}</td></tr>")

        html_parts.append("</table>")

        if is_imbalanced and ratio is not None:
            html_parts.append(
                f"<p><em>Class imbalance detected ({ratio:.1f}:1). Using Youden's J weights with prior correction.</em></p>"
            )

        # Thresholds table
        html_parts.append("<h3>Learned Thresholds</h3>")
        html_parts.append("<table>")
        html_parts.append(
            "<tr><th>Metric</th><th>Weight (Youden's J)</th><th>Selection Accuracy</th><th>Threshold Details</th></tr>"
        )

        for metric, info in self.thresholds.items():
            weight = info["accuracy"]  # This is Youden's J
            selection_acc = info.get("selection_accuracy", weight)

            # Build threshold details
            details = []
            for label, (threshold, direction) in info["thresholds"].items():
                details.append(f"<br>{label}: {direction} than {threshold:.3f}")

            # Add distribution info if available
            if metric in self.metrics_info:
                metric_stats = self.metrics_info[metric]
                details.append("<br><em>Distributions:</em>")
                for label in self.labels:
                    mean_key = f"mean_{label}"
                    std_key = f"std_{label}"
                    if mean_key in metric_stats:
                        details.append(
                            f"<br>&nbsp;&nbsp;{label}: μ={metric_stats[mean_key]:.1f}, σ={metric_stats[std_key]:.1f}"
                        )

            html_parts.append("<tr>")
            html_parts.append(f"<td>{metric}</td>")
            html_parts.append(f"<td>{weight:.3f}</td>")
            html_parts.append(f"<td>{selection_acc:.3f}</td>")
            html_parts.append(f"<td>{''.join(details)}</td>")
            html_parts.append("</tr>")

        html_parts.append("</table>")

        all_correct = 0
        all_total = 0

        # First show labeled examples
        for true_label in self.labels:
            label_dir = self.root_dir / true_label
            examples = list(label_dir.glob("*.png"))

            if not examples:
                continue

            html_parts.append(
                f"<h3>Predictions: {true_label.upper()} ({len(examples)} total)</h3>"
            )
            html_parts.append("<table>")
            html_parts.append(
                "<tr><th>Image</th><th>Status</th><th>Predicted</th><th>Score</th><th>Key Metrics</th></tr>"
            )

            correct = 0

            for img_path in sorted(examples)[:20]:  # Show max 20 per class in preview
                # Load image
                img = Image.open(img_path)
                preview_region = PreviewRegion(img)

                # Get prediction
                decision = cast(Decision, self.decide(preview_region))
                is_correct = decision.label == true_label
                if is_correct:
                    correct += 1

                # Extract metrics
                metrics = self._extract_metrics(preview_region)

                # Convert image to base64
                buffered = io.BytesIO()
                img.save(buffered, format="PNG")
                img_str = base64.b64encode(buffered.getvalue()).decode()

                # Build row
                status_class = "correct" if is_correct else "incorrect"
                status_symbol = "✓" if is_correct else "✗"

                # Format key metrics
                metric_strs = []
                for metric, value in sorted(metrics.items()):
                    if metric in self.thresholds:
                        metric_strs.append(f"{metric}={value:.1f}")
                metrics_html = "<br>".join(metric_strs[:3])

                html_parts.append("<tr>")
                html_parts.append(f'<td><img src="data:image/png;base64,{img_str}" /></td>')
                html_parts.append(f'<td class="{status_class}">{status_symbol}</td>')
                html_parts.append(f"<td>{decision.label}</td>")
                html_parts.append(f"<td>{decision.score:.3f}</td>")
                html_parts.append(f'<td class="metrics">{metrics_html}</td>')
                html_parts.append("</tr>")

            html_parts.append("</table>")

            accuracy = correct / len(examples) if examples else 0
            all_correct += correct
            all_total += len(examples)

            if len(examples) > 20:
                html_parts.append(f"<p><em>... and {len(examples) - 20} more</em></p>")
            html_parts.append(
                f"<p>Accuracy for {true_label}: <strong>{accuracy:.1%}</strong> ({correct}/{len(examples)})</p>"
            )

        if all_total > 0:
            overall_accuracy = all_correct / all_total
            html_parts.append(
                f"<h3>Overall accuracy: {overall_accuracy:.1%} ({all_correct}/{all_total})</h3>"
            )

        # Now show unlabeled examples with predictions
        unlabeled_dir = self.root_dir / "unlabeled"
        unlabeled_examples = list(unlabeled_dir.glob("*.png"))

        if unlabeled_examples:
            html_parts.append(
                f"<h3>Predictions: UNLABELED ({len(unlabeled_examples)} total)</h3>"
            )
            html_parts.append("<table>")
            html_parts.append(
                "<tr><th>Image</th><th>Predicted</th><th>Score</th><th>Key Metrics</th></tr>"
            )

            for img_path in sorted(unlabeled_examples)[:20]:  # Show max 20
                # Load image
                img = Image.open(img_path)
                preview_region = PreviewRegion(img)

                # Get prediction
                decision = cast(Decision, self.decide(preview_region))

                # Extract metrics
                metrics = self._extract_metrics(preview_region)

                # Convert image to base64
                buffered = io.BytesIO()
                img.save(buffered, format="PNG")
                img_str = base64.b64encode(buffered.getvalue()).decode()

                # Format key metrics
                metric_strs = []
                for metric, value in sorted(metrics.items()):
                    if metric in self.thresholds:
                        metric_strs.append(f"{metric}={value:.1f}")
                metrics_html = "<br>".join(metric_strs[:3])

                html_parts.append("<tr>")
                html_parts.append(f'<td><img src="data:image/png;base64,{img_str}" /></td>')
                html_parts.append(f"<td>{decision.label}</td>")
                html_parts.append(f"<td>{decision.score:.3f}</td>")
                html_parts.append(f'<td class="metrics">{metrics_html}</td>')
                html_parts.append("</tr>")

            html_parts.append("</table>")

            if len(unlabeled_examples) > 20:
                html_parts.append(
                    f"<p><em>... and {len(unlabeled_examples) - 20} more</em></p>"
                )

        # Display HTML
        display(HTML("".join(html_parts)))

    else:
        # Text mode (original)
        print("\nPredictions on training data:")
        print("-" * 80)

        # Test each labeled example
        all_correct = 0
        all_total = 0

        for true_label in self.labels:
            label_dir = self.root_dir / true_label
            examples = list(label_dir.glob("*.png"))

            if not examples:
                continue

            print(f"\n{true_label.upper()} examples ({len(examples)} total):")
            correct = 0

            for img_path in sorted(examples)[:10]:  # Show max 10 per class
                # Load image and create mock region
                img = Image.open(img_path)
                preview_region = PreviewRegion(img)

                # Get prediction
                decision = cast(Decision, self.decide(preview_region))
                is_correct = decision.label == true_label
                if is_correct:
                    correct += 1

                # Extract metrics for this example
                metrics = self._extract_metrics(preview_region)

                # Show result
                status = "✓" if is_correct else "✗"
                print(
                    f"  {status} {img_path.name}: predicted={decision.label} (score={decision.score:.3f})"
                )

                # Show key metric values
                metric_strs = []
                for metric, value in sorted(metrics.items()):
                    if metric in self.thresholds:
                        metric_strs.append(f"{metric}={value:.2f}")
                if metric_strs:
                    print(f"     Metrics: {', '.join(metric_strs[:3])}")

            accuracy = correct / len(examples) if examples else 0
            all_correct += correct
            all_total += len(examples)

            if len(examples) > 10:
                print(f"  ... and {len(examples) - 10} more")
            print(f"  Accuracy for {true_label}: {accuracy:.1%} ({correct}/{len(examples)})")

        if all_total > 0:
            overall_accuracy = all_correct / all_total
            print(f"\nOverall accuracy: {overall_accuracy:.1%} ({all_correct}/{all_total})")

        # Show unlabeled examples with predictions
        unlabeled_dir = self.root_dir / "unlabeled"
        unlabeled_examples = list(unlabeled_dir.glob("*.png"))

        if unlabeled_examples:
            print(f"\nUNLABELED examples ({len(unlabeled_examples)} total) - predictions:")

            for img_path in sorted(unlabeled_examples)[:10]:  # Show max 10
                # Load image and create preview region
                img = Image.open(img_path)
                preview_region = PreviewRegion(img)

                # Get prediction
                decision = cast(Decision, self.decide(preview_region))

                # Extract metrics
                metrics = self._extract_metrics(preview_region)

                print(
                    f"  {img_path.name}: predicted={decision.label} (score={decision.score:.3f})"
                )

                # Show key metric values
                metric_strs = []
                for metric, value in sorted(metrics.items()):
                    if metric in self.thresholds:
                        metric_strs.append(f"{metric}={value:.2f}")
                if metric_strs:
                    print(f"     Metrics: {', '.join(metric_strs[:3])}")

            if len(unlabeled_examples) > 10:
                print(f"  ... and {len(unlabeled_examples) - 10} more")

natural_pdf.Judge.load(path) classmethod

Load a judge from a saved configuration.

Parameters:

Name	Type	Description	Default
path	Union[str, Path]	Path to the saved judge.json file or the judge directory	required

Returns:

Type	Description
Judge	Loaded Judge instance

Source code in natural_pdf/judge.py

@classmethod
def load(cls, path: Union[str, Path]) -> "Judge":
    """
    Load a judge from a saved configuration.

    Args:
        path: Path to the saved judge.json file or the judge directory

    Returns:
        Loaded Judge instance
    """
    path = Path(path)

    # If path is a directory, look for judge.json inside
    if path.is_dir():
        config_path = path / "judge.json"
        base_dir = path.parent
        name = path.name
    else:
        config_path = path
        base_dir = path.parent.parent if path.parent.name != "." else path.parent
        # Try to infer name from path
        name = None

    with open(config_path, "r") as f:
        config = json.load(f)

    # Use saved name if we couldn't infer it
    if name is None:
        name = config["name"]

    # Create judge with saved config
    judge = cls(
        name,
        labels=config["labels"],
        base_dir=base_dir,
        target_prior=config.get("target_prior", 0.5),
    )  # Default to 0.5 for old configs
    judge.thresholds = cast(Dict[str, Dict[str, Any]], config.get("thresholds", {}))
    judge.metrics_info = cast(Dict[str, Dict[str, float]], config.get("metrics_info", {}))

    return judge

natural_pdf.Judge.lookup(region)

Look up a region and return its hash and image if found in training data.

Parameters:

Name	Type	Description	Default
region	SupportsRender	Region to look up	required

Returns:

Type	Description
Optional[Tuple[str, Image]]	Tuple of (hash, image) if found, None if not found

Source code in natural_pdf/judge.py

def lookup(self, region: SupportsRender) -> Optional[Tuple[str, Image.Image]]:
    """
    Look up a region and return its hash and image if found in training data.

    Args:
        region: Region to look up

    Returns:
        Tuple of (hash, image) if found, None if not found
    """
    try:
        # Generate hash for the region
        img = region.render(crop=True)
        if img is None:
            raise JudgeError("Region.render returned no image")
        if not isinstance(img, Image.Image):
            img = Image.fromarray(np.asarray(img))
        if img.mode != "RGB":
            img = img.convert("RGB")
        img_array = np.array(img)
        img_hash = hashlib.md5(img_array.tobytes()).hexdigest()[:12]

        # Look for the image in all directories
        for subdir in ["checked", "unchecked", "unlabeled", "_removed"]:
            if subdir == "checked" or subdir == "unchecked":
                # Only look in valid label directories
                if subdir not in self.labels:
                    continue

            img_path = self.root_dir / subdir / f"{img_hash}.png"
            if img_path.exists():
                stored_img = Image.open(img_path)
                logger.debug(f"Found region in '{subdir}' with hash {img_hash}")
                return (img_hash, stored_img)

        logger.debug(f"Region not found in training data (hash: {img_hash})")
        return None

    except Exception as e:
        logger.error(f"Failed to lookup region: {e}")
        return None

natural_pdf.Judge.pick(target_label, regions, labels=None)

Pick which region best matches the target label.

Parameters:

Name	Type	Description	Default
target_label	str	The class label to look for	required
regions	Iterable[SupportsRender]	List of regions to choose from	required
labels	Optional[Sequence[str]]	Optional human-friendly labels for each region	None

Returns:

Type	Description
PickResult	PickResult with winning region, index, label (if provided), and score

Raises:

Type	Description
JudgeError	If target_label not in allowed labels

Source code in natural_pdf/judge.py

def pick(
    self,
    target_label: str,
    regions: Iterable[SupportsRender],
    labels: Optional[Sequence[str]] = None,
) -> PickResult:
    """
    Pick which region best matches the target label.

    Args:
        target_label: The class label to look for
        regions: List of regions to choose from
        labels: Optional human-friendly labels for each region

    Returns:
        PickResult with winning region, index, label (if provided), and score

    Raises:
        JudgeError: If target_label not in allowed labels
    """
    if target_label not in self.labels:
        raise JudgeError(f"Target label '{target_label}' not in allowed labels: {self.labels}")

    # Classify all regions
    region_list = list(regions)
    decisions_result = self.decide(region_list)
    decisions = decisions_result if isinstance(decisions_result, list) else [decisions_result]

    # Find best match for target label
    best_index = -1
    best_score = -1.0

    for i, decision in enumerate(decisions):
        if decision.label == target_label and decision.score > best_score:
            best_score = decision.score
            best_index = i

    if best_index == -1:
        # No region matched the target label
        raise JudgeError(f"No region classified as '{target_label}'")

    # Build result
    region = region_list[best_index]
    label_list = list(labels) if labels is not None else None
    label = (
        label_list[best_index]
        if label_list is not None and best_index < len(label_list)
        else None
    )

    return PickResult(region=region, index=best_index, label=label, score=best_score)

natural_pdf.Judge.save(path=None)

Save the judge configuration (auto-retrains first).

Parameters:

Name	Type	Description	Default
path	Optional[Union[str, Path]]	Optional path to save to. Defaults to judge.json in root directory	None

Source code in natural_pdf/judge.py

def save(self, path: Optional[Union[str, Path]] = None) -> None:
    """
    Save the judge configuration (auto-retrains first).

    Args:
        path: Optional path to save to. Defaults to judge.json in root directory
    """
    # Retrain with current examples
    self._retrain()

    # Save config
    save_path = Path(path) if path else self.config_path

    config = {
        "name": self.name,
        "labels": self.labels,
        "target_prior": self.target_prior,
        "thresholds": self.thresholds,
        "metrics_info": self.metrics_info,
        "training_counts": self._get_training_counts(),
    }

    with open(save_path, "w") as f:
        json.dump(config, f, indent=2)

    logger.info(f"Saved judge to {save_path}")

natural_pdf.Judge.show(max_per_class=10, size=(100, 100))

Display a grid showing examples from each category.

Parameters:

Name	Type	Description	Default
max_per_class	int	Maximum number of examples to show per class	10
size	Tuple[int, int]	Size of each image in pixels (width, height)	(100, 100)

Source code in natural_pdf/judge.py

def show(self, max_per_class: int = 10, size: Tuple[int, int] = (100, 100)) -> None:
    """
    Display a grid showing examples from each category.

    Args:
        max_per_class: Maximum number of examples to show per class
        size: Size of each image in pixels (width, height)
    """
    try:
        import ipywidgets as widgets  # type: ignore[import-untyped]
        from IPython.display import display
        from PIL import Image as PILImage
    except ImportError:
        print("Show requires IPython and ipywidgets")
        return

    # Collect images from each category
    categories = {}
    total_counts = {}
    for label in self.labels:
        label_dir = self.root_dir / label
        all_images = list(label_dir.glob("*.png"))
        total_counts[label] = len(all_images)
        images = sorted(all_images)[:max_per_class]
        if images:
            categories[label] = images

    # Add unlabeled if any
    unlabeled_dir = self.root_dir / "unlabeled"
    all_unlabeled = list(unlabeled_dir.glob("*.png"))
    total_counts["unlabeled"] = len(all_unlabeled)
    unlabeled = sorted(all_unlabeled)[:max_per_class]
    if unlabeled:
        categories["unlabeled"] = unlabeled

    if not categories:
        print("No images to show")
        return

    # Create grid layout
    rows = []

    # Check for class imbalance
    labeled_counts = {k: v for k, v in total_counts.items() if k != "unlabeled"}
    if labeled_counts and len(labeled_counts) >= 2:
        max_count = max(labeled_counts.values())
        min_count = min(labeled_counts.values())
        if min_count > 0 and max_count / min_count > 3:
            warning = widgets.HTML(
                f'<div style="background: #fff3cd; padding: 10px; margin: 10px 0; border: 1px solid #ffeeba; border-radius: 4px;">'
                f"<strong>⚠️ Class imbalance detected:</strong> {labeled_counts}<br>"
                f"Consider adding more examples of the minority class for better accuracy."
                f"</div>"
            )
            rows.append(warning)

    for category, image_paths in categories.items():
        # Category header showing total count
        shown = len(image_paths)
        total = total_counts[category]
        header_text = f"<h3>{category}"
        if shown < total:
            header_text += f" ({shown} of {total} shown)"
        else:
            header_text += f" ({total} total)"
        header_text += "</h3>"
        header = widgets.HTML(header_text)

        # Image row
        image_widgets = []
        for img_path in image_paths:
            # Load and resize image
            img = PILImage.open(img_path)
            img.thumbnail(size, PILImage.Resampling.LANCZOS)

            # Convert to bytes for display
            import io

            img_bytes = io.BytesIO()
            img.save(img_bytes, format="PNG")
            img_bytes.seek(0)

            # Create image widget
            img_widget = widgets.Image(value=img_bytes.read(), width=size[0], height=size[1])
            image_widgets.append(img_widget)

        # Create horizontal box for this category
        category_box = widgets.VBox([header, widgets.HBox(image_widgets)])
        rows.append(category_box)

    # Display all categories
    display(widgets.VBox(rows))

natural_pdf.Judge.teach(labels=None, review=False)

Interactive teaching interface using IPython widgets.

Parameters:

Name	Type	Description	Default
labels	Optional[List[str]]	Labels to use for teaching. Defaults to self.labels	None
review	bool	If True, review already labeled images for re-classification	False

Source code in natural_pdf/judge.py

def teach(self, labels: Optional[List[str]] = None, review: bool = False) -> None:
    """
    Interactive teaching interface using IPython widgets.

    Args:
        labels: Labels to use for teaching. Defaults to self.labels
        review: If True, review already labeled images for re-classification
    """
    # Check for IPython environment
    try:
        import ipywidgets as widgets
        from IPython.display import clear_output, display
    except ImportError:
        raise JudgeError(
            "Teaching requires IPython and ipywidgets. Use 'pip install ipywidgets'"
        )

    labels = labels or self.labels

    # Get images to review
    if review:
        # Get all labeled images for review
        files_to_review = []
        for label in self.labels:
            label_dir = self.root_dir / label
            for img_path in sorted(label_dir.glob("*.png")):
                files_to_review.append((img_path, label))

        if not files_to_review:
            print("No labeled images to review")
            return

        # Shuffle for review
        import random

        random.shuffle(files_to_review)
        review_files = [f[0] for f in files_to_review]
        original_labels = {str(f[0]): f[1] for f in files_to_review}
    else:
        # Get unlabeled images
        unlabeled_dir = self.root_dir / "unlabeled"
        review_files = sorted(unlabeled_dir.glob("*.png"))
        original_labels = {}

        if not review_files:
            print("No unlabeled images to teach")
            return

    # State for teaching
    self._teaching_state = {
        "current_index": 0,
        "labeled_count": 0,
        "removed_count": 0,
        "files": review_files,
        "labels": labels,
        "review_mode": review,
        "original_labels": original_labels,
    }

    # Create widgets
    image_widget = widgets.Image()
    status_label = widgets.Label()

    # Create buttons for labeling
    button_layout = widgets.Layout(width="auto", margin="5px")

    btn_prev = widgets.Button(description="↑ Previous", layout=button_layout)
    btn_class1 = widgets.Button(
        description=f"← {labels[0]}", layout=button_layout, button_style="primary"
    )
    btn_class2 = widgets.Button(
        description=f"→ {labels[1]}", layout=button_layout, button_style="success"
    )
    btn_skip = widgets.Button(description="↓ Skip", layout=button_layout)
    btn_remove = widgets.Button(
        description="✗ Remove", layout=button_layout, button_style="danger"
    )

    button_box = widgets.HBox([btn_prev, btn_class1, btn_class2, btn_skip, btn_remove])

    # Keyboard shortcuts info
    info_label = widgets.Label(
        value="Keys: ↑ prev | ← "
        + labels[0]
        + " | → "
        + labels[1]
        + " | ↓ skip | Delete remove"
    )

    def update_display():
        """Update the displayed image and status."""
        state = self._teaching_state
        if 0 <= state["current_index"] < len(state["files"]):
            img_path = state["files"][state["current_index"]]
            with open(img_path, "rb") as f:
                image_widget.value = f.read()

            # Build status text
            status_text = f"Image {state['current_index'] + 1} of {len(state['files'])}"
            if state["review_mode"]:
                current_label = state["original_labels"].get(str(img_path), "unknown")
                status_text += f" (Currently: {current_label})"
            status_text += f" | Labeled: {state['labeled_count']}"
            if state["removed_count"] > 0:
                status_text += f" | Removed: {state['removed_count']}"

            status_label.value = status_text

            # Update button states
            btn_prev.disabled = state["current_index"] == 0
        else:
            status_label.value = "Teaching complete!"
            # Hide the image widget instead of showing broken image
            image_widget.layout.display = "none"
            # Disable all buttons
            btn_prev.disabled = True
            btn_class1.disabled = True
            btn_class2.disabled = True
            btn_skip.disabled = True

            # Auto-retrain
            if state["labeled_count"] > 0 or state["removed_count"] > 0:
                clear_output(wait=True)
                print("Teaching complete!")
                print(f"Labeled: {state['labeled_count']} images")
                if state["removed_count"] > 0:
                    print(f"Removed: {state['removed_count']} images")

                if state["labeled_count"] > 0:
                    print("\nRetraining with new examples...")
                    self._retrain()
                    print("✓ Training complete! Judge is ready to use.")
            else:
                print("No changes made.")

    def move_file_to_class(class_index):
        """Move current file to specified class."""
        state = self._teaching_state
        if state["current_index"] >= len(state["files"]):
            return

        current_file = state["files"][state["current_index"]]
        target_dir = self.root_dir / labels[class_index]
        shutil.move(str(current_file), str(target_dir / current_file.name))
        state["labeled_count"] += 1
        state["current_index"] += 1
        update_display()

    # Button callbacks
    def on_prev(b):
        state = self._teaching_state
        if state["current_index"] > 0:
            state["current_index"] -= 1
            update_display()

    def on_class1(b):
        move_file_to_class(0)

    def on_class2(b):
        move_file_to_class(1)

    def on_skip(b):
        state = self._teaching_state
        state["current_index"] += 1
        update_display()

    def on_remove(b):
        state = self._teaching_state
        if state["current_index"] >= len(state["files"]):
            return

        current_file = state["files"][state["current_index"]]
        target_dir = self.root_dir / "_removed"
        shutil.move(str(current_file), str(target_dir / current_file.name))
        state["removed_count"] += 1
        state["current_index"] += 1
        update_display()

    # Connect buttons
    btn_prev.on_click(on_prev)
    btn_class1.on_click(on_class1)
    btn_class2.on_click(on_class2)
    btn_skip.on_click(on_skip)
    btn_remove.on_click(on_remove)

    # Create output widget for keyboard handling
    output = widgets.Output()

    # Keyboard event handler
    def on_key(event):
        """Handle keyboard events."""
        if event["type"] != "keydown":
            return

        key = event["key"]

        if key == "ArrowUp":
            on_prev(None)
        elif key == "ArrowLeft":
            on_class1(None)
        elif key == "ArrowRight":
            on_class2(None)
        elif key == "ArrowDown":
            on_skip(None)
        elif key in ["Delete", "Backspace"]:
            on_remove(None)

    # Display everything
    display(status_label)
    display(image_widget)
    display(button_box)
    display(info_label)
    display(output)

    # Show first image
    update_display()

    # Try to set up keyboard handling (may not work in all environments)
    try:
        from ipyevents import Event  # type: ignore[import-untyped]

        event_handler = Event(source=output, watched_events=["keydown"])
        event_handler.on_dom_event(on_key)
    except:
        # If ipyevents not available, just use buttons
        print("Note: Install ipyevents for keyboard shortcuts: pip install ipyevents")

natural_pdf.JudgeError

Bases: Exception

Raised when Judge operations fail.

Source code in natural_pdf/judge.py

class JudgeError(Exception):
    """Raised when Judge operations fail."""

    pass

natural_pdf.Options

Global options for natural-pdf, similar to pandas options.

Source code in natural_pdf/__init__.py

class Options:
    """Global options for natural-pdf, similar to pandas options."""

    def __init__(self):
        # Image rendering defaults
        self.image = ConfigSection(width=None, resolution=150)

        # OCR defaults
        self.ocr = ConfigSection(engine="easyocr", languages=["en"], min_confidence=0.5)

        # Text extraction defaults (empty for now)
        self.text = ConfigSection()

        # Layout and navigation defaults
        self.layout = ConfigSection(
            engine="tatr",
            directional_offset=0.01,  # Offset in points when using directional methods
            auto_multipage=False,  # Whether directional methods span pages by default
            directional_within=None,  # Region to constrain directional operations to
        )

        # Table extraction defaults
        self.tables = ConfigSection(engine="pdfplumber")

        # Selector defaults (None = native engine)
        self.selectors = ConfigSection(engine=None)

natural_pdf.PDF

Bases: TextMixin, ExtractionMixin, ExportMixin, ClassificationMixin, CheckboxDetectionMixin, VisualSearchMixin, Visualizable

Enhanced PDF wrapper built on top of pdfplumber.

This class provides a fluent interface for working with PDF documents, with improved selection, navigation, and extraction capabilities. It integrates OCR, layout analysis, and AI-powered data extraction features while maintaining compatibility with the underlying pdfplumber API.

The PDF class supports loading from files, URLs, or streams, and provides spatial navigation, element selection with CSS-like selectors, and advanced document processing workflows including multi-page content flows.

Attributes:

Name	Type	Description
pages	PageCollection	Lazy-loaded list of Page objects for document pages.
path		Resolved path to the PDF file or source identifier.
source_path		Original path, URL, or stream identifier provided during initialization.
highlighter	HighlightingService	Service for rendering highlighted visualizations of document content.

Example

Basic usage:

import natural_pdf as npdf

pdf = npdf.PDF("document.pdf")
page = pdf.pages[0]
text_elements = page.find_all('text:contains("Summary")')

Advanced usage with OCR:

pdf = npdf.PDF("scanned_document.pdf")
pdf.apply_ocr(engine="easyocr", resolution=144)
tables = pdf.pages[0].find_all('table')

Source code in natural_pdf/core/pdf.py

class PDF(
    TextMixin,
    ExtractionMixin,
    ExportMixin,
    ClassificationMixin,
    CheckboxDetectionMixin,
    VisualSearchMixin,
    Visualizable,
):
    """Enhanced PDF wrapper built on top of pdfplumber.

    This class provides a fluent interface for working with PDF documents,
    with improved selection, navigation, and extraction capabilities. It integrates
    OCR, layout analysis, and AI-powered data extraction features while maintaining
    compatibility with the underlying pdfplumber API.

    The PDF class supports loading from files, URLs, or streams, and provides
    spatial navigation, element selection with CSS-like selectors, and advanced
    document processing workflows including multi-page content flows.

    Attributes:
        pages: Lazy-loaded list of Page objects for document pages.
        path: Resolved path to the PDF file or source identifier.
        source_path: Original path, URL, or stream identifier provided during initialization.
        highlighter: Service for rendering highlighted visualizations of document content.

    Example:
        Basic usage:
        ```python
        import natural_pdf as npdf

        pdf = npdf.PDF("document.pdf")
        page = pdf.pages[0]
        text_elements = page.find_all('text:contains("Summary")')
        ```

        Advanced usage with OCR:
        ```python
        pdf = npdf.PDF("scanned_document.pdf")
        pdf.apply_ocr(engine="easyocr", resolution=144)
        tables = pdf.pages[0].find_all('table')
        ```
    """

    @classmethod
    def from_images(
        cls,
        images: Union["Image.Image", List["Image.Image"], str, List[str], Path, List[Path]],
        resolution: int = 300,
        apply_ocr: bool = True,
        ocr_engine: Optional[str] = None,
        **pdf_options,
    ) -> "PDF":
        """Create a PDF from image(s).

        Args:
            images: Single image, list of images, or path(s)/URL(s) to image files
            resolution: DPI for the PDF (default: 300, good for OCR and viewing)
            apply_ocr: Apply OCR to make searchable (default: True)
            ocr_engine: OCR engine to use (default: auto-detect)
            **pdf_options: Options passed to PDF constructor

        Returns:
            PDF object containing the images as pages

        Example:
            ```python
            # Simple scan to searchable PDF
            pdf = PDF.from_images("scan.jpg")

            # From URL
            pdf = PDF.from_images("https://example.com/image.png")

            # Multiple pages (mix of local and URLs)
            pdf = PDF.from_images(["page1.png", "https://example.com/page2.jpg"])

            # Without OCR
            pdf = PDF.from_images(images, apply_ocr=False)

            # With specific engine
            pdf = PDF.from_images(images, ocr_engine='surya')
            ```
        """
        import urllib.request

        from PIL import ImageOps

        def _open_image(source: Union[Image.Image, str, Path]) -> Image.Image:
            """Open an image from file path, URL, or return PIL Image as-is."""
            if isinstance(source, Image.Image):
                return source

            source_str = str(source)
            if source_str.startswith(("http://", "https://")):
                # Download from URL
                with urllib.request.urlopen(source_str) as response:
                    img_data = response.read()
                return Image.open(io.BytesIO(img_data))
            else:
                # Local file path
                return Image.open(source)

        # Normalize inputs to list of PIL Images
        if isinstance(images, (str, Path)):
            image_list = [_open_image(images)]
        elif isinstance(images, Image.Image):
            image_list = [images]
        elif isinstance(images, list):
            image_list = [_open_image(item) for item in images]
        else:
            raise TypeError(
                "images must be a path, PIL Image, or a list of paths/PIL Image instances."
            )

        # Process images
        processed_images: List[Image.Image] = []
        for img in image_list:
            # Fix EXIF rotation
            img = ImageOps.exif_transpose(img) or img

            # Convert RGBA to RGB (PDF doesn't handle transparency well)
            if img.mode == "RGBA":
                bg = Image.new("RGB", img.size, "white")
                bg.paste(img, mask=img.split()[3])
                img = bg
            elif img.mode not in ["RGB", "L", "1", "CMYK"]:
                img = img.convert("RGB")

            processed_images.append(img)

        # Create PDF at specified resolution
        # Use BytesIO to keep in memory
        pdf_buffer = io.BytesIO()
        processed_images[0].save(
            pdf_buffer,
            "PDF",
            save_all=True,
            append_images=processed_images[1:] if len(processed_images) > 1 else [],
            resolution=resolution,
        )
        pdf_buffer.seek(0)

        # Create PDF object
        pdf = cls(pdf_buffer, **pdf_options)

        # Store metadata about source
        pdf._from_images = True
        pdf._source_metadata = {
            "type": "images",
            "count": len(processed_images),
            "resolution": resolution,
        }

        # Apply OCR if requested
        if apply_ocr:
            pdf.apply_ocr(engine=ocr_engine, resolution=resolution)

        return pdf

    def __init__(
        self,
        path_or_url_or_stream,
        reading_order: bool = True,
        font_attrs: Optional[List[str]] = None,
        keep_spaces: bool = True,
        text_tolerance: Optional[dict] = None,
        auto_text_tolerance: bool = True,
        text_layer: bool = True,
    ):
        """Initialize the enhanced PDF object.

        Args:
            path_or_url_or_stream: Path to the PDF file (str/Path), a URL (str),
                or a file-like object (stream). URLs must start with 'http://' or 'https://'.
            reading_order: If True, use natural reading order for text extraction.
                Defaults to True.
            font_attrs: List of font attributes for grouping characters into words.
                Common attributes include ['fontname', 'size']. Defaults to None.
            keep_spaces: If True, include spaces in word elements during text extraction.
                Defaults to True.
            text_tolerance: PDFplumber-style tolerance settings for text grouping.
                Dictionary with keys like 'x_tolerance', 'y_tolerance'. Defaults to None.
            auto_text_tolerance: If True, automatically scale text tolerance based on
                font size and document characteristics. Defaults to True.
            text_layer: If True, preserve existing text layer from the PDF. If False,
                removes all existing text elements during initialization, useful for
                OCR-only workflows. Defaults to True.

        Raises:
            TypeError: If path_or_url_or_stream is not a valid type.
            IOError: If the PDF file cannot be opened or read.
            ValueError: If URL download fails.

        Example:
            ```python
            # From file path
            pdf = npdf.PDF("document.pdf")

            # From URL
            pdf = npdf.PDF("https://example.com/document.pdf")

            # From stream
            with open("document.pdf", "rb") as f:
                pdf = npdf.PDF(f)

            # With custom settings
            pdf = npdf.PDF("document.pdf",
                          reading_order=False,
                          text_layer=False,  # For OCR-only processing
                          font_attrs=['fontname', 'size', 'flags'])
            ```
        """
        self._original_path_or_stream = path_or_url_or_stream
        self._temp_file = None
        self._resolved_path = None
        self._is_stream = False
        self._text_layer = text_layer
        stream_to_open = None

        if hasattr(path_or_url_or_stream, "read"):  # Check if it's file-like
            logger.info("Initializing PDF from in-memory stream.")
            self._is_stream = True
            self._resolved_path = None  # No resolved file path for streams
            self.source_path = "<stream>"  # Identifier for source
            self.path = self.source_path  # Use source identifier as path for streams
            stream_to_open = path_or_url_or_stream
            try:
                if hasattr(path_or_url_or_stream, "read"):
                    # If caller provided an in-memory binary stream, capture bytes for potential re-export
                    current_pos = path_or_url_or_stream.tell()
                    path_or_url_or_stream.seek(0)
                    self._original_bytes = path_or_url_or_stream.read()
                    path_or_url_or_stream.seek(current_pos)
            except Exception:
                pass
        elif isinstance(path_or_url_or_stream, (str, Path)):
            path_or_url = str(path_or_url_or_stream)
            self.source_path = path_or_url  # Store original path/URL as source
            is_url = path_or_url.startswith("http://") or path_or_url.startswith("https://")

            if is_url:
                logger.info(f"Downloading PDF from URL: {path_or_url}")
                try:
                    ssl_context = None
                    try:
                        if certifi is not None:
                            ssl_context = ssl.create_default_context(cafile=certifi.where())
                        else:
                            ssl_context = ssl.create_default_context()
                    except Exception:
                        ssl_context = ssl.create_default_context()

                    with urllib.request.urlopen(path_or_url, context=ssl_context) as response:
                        data = response.read()
                    # Load directly into an in-memory buffer — no temp file needed
                    buffer = io.BytesIO(data)
                    buffer.seek(0)
                    self._temp_file = None  # No on-disk temp file
                    self._resolved_path = path_or_url  # For repr / get_id purposes
                    stream_to_open = buffer  # pdfplumber accepts file-like objects
                except Exception as e:
                    logger.error(f"Failed to download PDF from URL: {e}")
                    raise ValueError(f"Failed to download PDF from URL: {e}")
            else:
                self._resolved_path = str(Path(path_or_url).resolve())  # Resolve local paths
                stream_to_open = self._resolved_path
            self.path = self._resolved_path  # Use resolved path for file-based PDFs
        else:
            raise TypeError(
                f"Invalid input type: {type(path_or_url_or_stream)}. "
                f"Expected path (str/Path), URL (str), or file-like object."
            )

        logger.info(f"Opening PDF source: {self.source_path}")
        logger.debug(
            f"Parameters: reading_order={reading_order}, font_attrs={font_attrs}, keep_spaces={keep_spaces}"
        )

        try:
            self._pdf = pdfplumber.open(stream_to_open)
        except Exception as e:
            logger.error(f"Failed to open PDF: {e}", exc_info=True)
            self.close()  # Attempt cleanup if opening fails
            raise IOError(f"Failed to open PDF source: {self.source_path}") from e

        # Store configuration used for initialization
        self._reading_order = reading_order
        self._config = {"keep_spaces": keep_spaces}
        self._font_attrs = font_attrs

        # Deprecated managers remain for backwards compatibility but are no longer instantiated.
        self._layout_manager = None

        self.highlighter: HighlightingService = HighlightingService(self)
        self._manager_factories: ManagerFactories = {}
        self._managers: ManagerCache = {}

        # Lazily instantiate pages only when accessed
        self._pages: _LazyPageList = _LazyPageList(
            self, self._pdf, font_attrs=font_attrs, load_text=self._text_layer
        )

        self._element_cache: Dict[str, Any] = {}
        self._exclusions: List[ExclusionSpec] = []
        self._regions: RegionRegistry = []
        self._from_images: bool = False
        self._source_metadata: Optional[Dict[str, Any]] = None

        logger.info(f"PDF '{self.source_path}' initialized with {len(self._pages)} pages.")

        self._initialize_managers()
        self._initialize_highlighter()

        # Remove text layer if requested
        if not self._text_layer:
            logger.info("Removing text layer as requested (text_layer=False)")
            # Text layer is not loaded when text_layer=False, so no need to remove
            pass

        # Analysis results accessed via self.analyses property (see below)

        # --- Automatic cleanup when object is garbage-collected ---
        self._finalizer = weakref.finalize(
            self,
            PDF._finalize_cleanup,
            self._pdf,
            getattr(self, "_temp_file", None),
            getattr(self, "_is_stream", False),
        )

        # --- Text tolerance settings ------------------------------------
        # Users can pass pdfplumber-style keys (x_tolerance, x_tolerance_ratio,
        # y_tolerance, etc.) via *text_tolerance*.  We also keep a flag that
        # enables automatic tolerance scaling when explicit values are not
        # supplied.
        self._config["auto_text_tolerance"] = bool(auto_text_tolerance)
        if text_tolerance:
            # Only copy recognised primitives (numbers / None); ignore junk.
            allowed = {
                "x_tolerance",
                "x_tolerance_ratio",
                "y_tolerance",
                "y_tolerance_ratio",
                "keep_blank_chars",  # passthrough convenience
            }
            for k, v in text_tolerance.items():
                if k in allowed:
                    self._config[k] = v

    def _initialize_managers(self) -> None:
        """Set up manager factories for lazy instantiation."""
        # Store factories/classes for each manager key
        self._manager_factories = dict(DEFAULT_MANAGERS)
        self._managers = {}

    def get_manager(self, key: str) -> Any:
        """Retrieve a manager instance by its key, instantiating it lazily if needed.

        Managers are specialized components that handle specific functionality like
        classification, structured data extraction, or OCR processing. They are
        instantiated on-demand to minimize memory usage and startup time.

        Args:
            key: The manager key to retrieve. Common keys include 'classification'
                and 'structured_data'.

        Returns:
            The manager instance for the specified key.

        Raises:
            KeyError: If no manager is registered for the given key.
            RuntimeError: If the manager failed to initialize.

        Example:
            ```python
            pdf = npdf.PDF("document.pdf")
            classification_mgr = pdf.get_manager('classification')
            structured_data_mgr = pdf.get_manager('structured_data')
            ```
        """
        # Check if already instantiated
        if key in self._managers:
            manager_instance = self._managers[key]
            if manager_instance is None:
                raise RuntimeError(f"Manager '{key}' failed to initialize previously.")
            return manager_instance

        # Not instantiated yet: get factory/class
        if not hasattr(self, "_manager_factories") or key not in self._manager_factories:
            raise KeyError(
                f"No manager registered for key '{key}'. Available: {list(getattr(self, '_manager_factories', {}).keys())}"
            )
        factory_or_class = self._manager_factories[key]
        try:
            resolved = factory_or_class
            # If it's a callable that's not a class, call it to get the class/instance
            if not isinstance(resolved, type) and callable(resolved):
                resolved = resolved()
            # If it's a class, instantiate it
            if isinstance(resolved, type):
                instance = resolved()
            else:
                instance = resolved  # Already an instance
            self._managers[key] = instance
            return instance
        except Exception as e:
            logger.error(f"Failed to initialize manager for key '{key}': {e}")
            self._managers[key] = None
            raise RuntimeError(f"Manager '{key}' failed to initialize: {e}") from e

    def _initialize_highlighter(self):
        pass

    @property
    def metadata(self) -> Dict[str, Any]:
        """Access PDF metadata as a dictionary.

        Returns document metadata such as title, author, creation date, and other
        properties embedded in the PDF file. The exact keys available depend on
        what metadata was included when the PDF was created.

        Returns:
            Dictionary containing PDF metadata. Common keys include 'Title',
            'Author', 'Subject', 'Creator', 'Producer', 'CreationDate', and
            'ModDate'. May be empty if no metadata is available.

        Example:
            ```python
            pdf = npdf.PDF("document.pdf")
            print(pdf.metadata.get('Title', 'No title'))
            print(f"Created: {pdf.metadata.get('CreationDate')}")
            ```
        """
        if not hasattr(self, "_pdf") or self._pdf is None:
            return {}
        meta = getattr(self._pdf, "metadata", None)
        if meta is None:
            return {}
        if isinstance(meta, dict):
            return cast(Dict[str, Any], meta)
        # pdfplumber stores metadata as a custom object exposing dict-like interface.
        try:
            return dict(meta)
        except TypeError:
            logger.debug("Unable to coerce PDF metadata to dict; returning empty metadata.")
            return {}

    @property
    def pages(self) -> "PageCollection":
        """Access pages as a PageCollection object.

        Provides access to individual pages of the PDF document through a
        collection interface that supports indexing, slicing, and iteration.
        Pages are lazy-loaded to minimize memory usage.

        Returns:
            PageCollection object that provides list-like access to PDF pages.

        Raises:
            AttributeError: If PDF pages are not yet initialized.

        Example:
            ```python
            pdf = npdf.PDF("document.pdf")

            # Access individual pages
            first_page = pdf.pages[0]
            last_page = pdf.pages[-1]

            # Slice pages
            first_three = pdf.pages[0:3]

            # Iterate over pages
            for page in pdf.pages:
                print(f"Page {page.index} has {len(page.chars)} characters")
            ```
        """
        from natural_pdf.core.page_collection import PageCollection

        if not hasattr(self, "_pages"):
            raise AttributeError("PDF pages not yet initialized.")
        return PageCollection(cast(Sequence["Page"], self._pages))

    def clear_exclusions(self) -> "PDF":
        """Clear all exclusion functions from the PDF.

        Removes all previously added exclusion functions that were used to filter
        out unwanted content (like headers, footers, or administrative text) from
        text extraction and analysis operations.

        Returns:
            Self for method chaining.

        Raises:
            AttributeError: If PDF pages are not yet initialized.

        Example:
            ```python
            pdf = npdf.PDF("document.pdf")
            pdf.add_exclusion(lambda page: page.find('text:contains("CONFIDENTIAL")').above())

            # Later, remove all exclusions
            pdf.clear_exclusions()
            ```
        """
        if not hasattr(self, "_pages"):
            raise AttributeError("PDF pages not yet initialized.")

        self._exclusions = []

        # Clear exclusions only from already-created (cached) pages to avoid forcing page creation
        for i in range(len(self._pages)):
            cached_page = self._pages._cache[i]
            if cached_page is None:
                continue
            try:
                cached_page.clear_exclusions()
            except Exception as e:
                logger.warning(f"Failed to clear exclusions from existing page {i}: {e}")
        return self

    def add_exclusion(self, exclusion_func, label: Optional[str] = None) -> "PDF":
        """Add an exclusion function to the PDF.

        Exclusion functions define regions of each page that should be ignored during
        text extraction and analysis operations. This is useful for filtering out headers,
        footers, watermarks, or other administrative content that shouldn't be included
        in the main document processing.

        Args:
            exclusion_func: A function that takes a Page object and returns a Region
                to exclude from processing, or None if no exclusion should be applied
                to that page. The function is called once per page.
            label: Optional descriptive label for this exclusion rule, useful for
                debugging and identification.

        Returns:
            Self for method chaining.

        Raises:
            AttributeError: If PDF pages are not yet initialized.

        Example:
            ```python
            pdf = npdf.PDF("document.pdf")

            # Exclude headers (top 50 points of each page)
            pdf.add_exclusion(
                lambda page: page.region(0, 0, page.width, 50),
                label="header_exclusion"
            )

            # Exclude any text containing "CONFIDENTIAL"
            pdf.add_exclusion(
                lambda page: page.find('text:contains("CONFIDENTIAL")').above(include_source=True)
                if page.find('text:contains("CONFIDENTIAL")') else None,
                label="confidential_exclusion"
            )

            # Chain multiple exclusions
            pdf.add_exclusion(header_func).add_exclusion(footer_func)
            ```
        """
        if not hasattr(self, "_pages"):
            raise AttributeError("PDF pages not yet initialized.")

        # ------------------------------------------------------------------
        # Support selector strings and ElementCollection objects directly.
        # Store exclusion and apply only to already-created pages.
        # ------------------------------------------------------------------
        from natural_pdf.elements.element_collection import ElementCollection  # local import

        if isinstance(exclusion_func, str) or isinstance(exclusion_func, ElementCollection):
            # Store for bookkeeping and lazy application
            self._exclusions.append((exclusion_func, label))

            # Don't modify already-cached pages - they will get PDF-level exclusions
            # dynamically through _get_exclusion_regions()
            return self

        # Fallback to original callable / Region behaviour ------------------
        exclusion_data = (exclusion_func, label)
        self._exclusions.append(exclusion_data)

        # Don't modify already-cached pages - they will get PDF-level exclusions
        # dynamically through _get_exclusion_regions()

        return self

    def apply_ocr(
        self,
        engine: Optional[str] = None,
        languages: Optional[List[str]] = None,
        min_confidence: Optional[float] = None,
        device: Optional[str] = None,
        resolution: Optional[int] = None,
        apply_exclusions: bool = True,
        detect_only: bool = False,
        replace: bool = True,
        options: Optional[Any] = None,
        pages: Optional[Union[Iterable[int], range, slice]] = None,
    ) -> "PDF":
        """Apply OCR to specified pages of the PDF using batch processing.

        Performs optical character recognition on the specified pages, converting
        image-based text into searchable and extractable text elements. This method
        supports multiple OCR engines and provides batch processing for efficiency.

        Args:
            engine: Name of the OCR engine to use. Supported engines include
                'easyocr' (default), 'surya', 'paddle', and 'doctr'. If None,
                uses the global default from natural_pdf.options.ocr.engine.
            languages: List of language codes for OCR recognition (e.g., ['en', 'es']).
                If None, uses the global default from natural_pdf.options.ocr.languages.
            min_confidence: Minimum confidence threshold (0.0-1.0) for accepting
                OCR results. Text with lower confidence will be filtered out.
                If None, uses the global default.
            device: Device to run OCR on ('cpu', 'cuda', 'mps'). Engine-specific
                availability varies. If None, uses engine defaults.
            resolution: DPI resolution for rendering pages to images before OCR.
                Higher values improve accuracy but increase processing time and memory.
                Typical values: 150 (fast), 300 (balanced), 600 (high quality).
            apply_exclusions: If True, mask excluded regions before OCR to prevent
                processing of headers, footers, or other unwanted content.
            detect_only: If True, only detect text bounding boxes without performing
                character recognition. Useful for layout analysis workflows.
            replace: If True, replace any existing OCR elements on the pages.
                If False, append new OCR results to existing elements.
            options: Engine-specific options object (e.g., EasyOCROptions, SuryaOptions).
                Allows fine-tuning of engine behavior beyond common parameters.
            pages: Page indices to process. Can be:
                - None: Process all pages
                - slice: Process a range of pages (e.g., slice(0, 10))
                - Iterable[int]: Process specific page indices (e.g., [0, 2, 5])

        Returns:
            Self for method chaining.

        Raises:
            ValueError: If invalid page index is provided.
            TypeError: If pages parameter has invalid type.
            RuntimeError: If OCR engine is not available or fails.

        Example:
            ```python
            pdf = npdf.PDF("scanned_document.pdf")

            # Basic OCR on all pages
            pdf.apply_ocr()

            # High-quality OCR with specific settings
            pdf.apply_ocr(
                engine='easyocr',
                languages=['en', 'es'],
                resolution=300,
                min_confidence=0.8
            )

            # OCR specific pages only
            pdf.apply_ocr(pages=[0, 1, 2])  # First 3 pages
            pdf.apply_ocr(pages=slice(5, 10))  # Pages 5-9

            # Detection-only workflow for layout analysis
            pdf.apply_ocr(detect_only=True, resolution=150)
            ```

        Note:
            OCR processing can be time and memory intensive, especially at high
            resolutions. Consider using exclusions to mask unwanted regions and
            processing pages in batches for large documents.
        """
        normalized_options = normalize_ocr_options(options)
        engine_name = resolve_ocr_engine_name(
            context=self,
            requested=engine,
            options=normalized_options,
            scope="pdf",
        )
        resolved_languages = resolve_ocr_languages(self, languages, scope="pdf")
        resolved_min_confidence = resolve_ocr_min_confidence(self, min_confidence, scope="pdf")
        resolved_device = resolve_ocr_device(self, device, scope="pdf")

        target_pages = self._get_target_pages(pages)
        if not target_pages:
            logger.warning("No pages selected for OCR processing.")
            return self

        final_resolution = resolution or self._config.get("resolution", 150)
        logger.info(
            "Applying OCR to %d page(s) with engine '%s' at %s DPI.",
            len(target_pages),
            engine_name,
            final_resolution,
        )

        for page in tqdm(target_pages, desc="Applying OCR", leave=False):
            page.apply_ocr(
                engine=engine_name,
                options=normalized_options,
                languages=resolved_languages,
                min_confidence=resolved_min_confidence,
                device=resolved_device,
                resolution=final_resolution,
                detect_only=detect_only,
                apply_exclusions=apply_exclusions,
                replace=replace,
            )

        return self

    def add_region(
        self, region_func: Callable[["Page"], Optional["Region"]], name: Optional[str] = None
    ) -> "PDF":
        """
        Add a region function to the PDF.

        Args:
            region_func: A function that takes a Page and returns a Region, or None
            name: Optional name for the region

        Returns:
            Self for method chaining
        """
        if not hasattr(self, "_pages"):
            raise AttributeError("PDF pages not yet initialized.")

        region_data = (region_func, name)
        self._regions.append(region_data)

        # Apply only to already-created (cached) pages to avoid forcing page creation
        for i in range(len(self._pages)):
            cached_page = self._pages._cache[i]
            if cached_page is None:
                continue
            try:
                region_instance = region_func(cached_page)
                if region_instance and isinstance(region_instance, Region):
                    cached_page.add_region(region_instance, name=name, source="named")
                elif region_instance is not None:
                    logger.warning(
                        f"Region function did not return a valid Region for page {cached_page.number}"
                    )
            except Exception as e:
                logger.error(f"Error adding region for page {cached_page.number}: {e}")

        return self

    def find(
        self,
        selector: Optional[str] = None,
        *,
        text: Optional[Union[str, Sequence[str]]] = None,
        apply_exclusions: bool = True,
        regex: bool = False,
        case: bool = True,
        text_tolerance: Optional[Dict[str, Any]] = None,
        auto_text_tolerance: Optional[Dict[str, Any]] = None,
        reading_order: bool = True,
        **extra_kwargs: Any,
    ) -> Optional[Any]:
        """
        Find the first element matching the selector OR text content across all pages.

        Provide EITHER `selector` OR `text`, but not both.

        Args:
            selector: CSS-like selector string.
            text: Optional text shortcut (equivalent to ``text:contains(...)``).
            apply_exclusions: Whether to exclude elements in exclusion regions (default: True).
            regex: Whether to use regex for text search (`selector` or `text`) (default: False).
            case: Whether to do case-sensitive text search (`selector` or `text`) (default: True).
            text_tolerance: Optional dict of tolerance overrides applied temporarily.
            auto_text_tolerance: Optional overrides controlling automatic tolerance logic.
            reading_order: Whether to sort matches in reading order when applicable (default: True).

        Returns:
            Element object or None if not found.
        """
        if not hasattr(self, "_pages"):
            raise AttributeError("PDF pages not yet initialized.")

        if selector is not None and text is not None:
            raise ValueError("Provide either 'selector' or 'text', not both.")
        if selector is None and text is None:
            raise ValueError("Provide either 'selector' or 'text'.")

        # Construct selector if 'text' is provided
        effective_selector = ""
        if text is not None:
            effective_selector = build_text_contains_selector(text)
            logger.debug(
                f"Using text shortcut: find(text={text!r}) -> find('{effective_selector}')"
            )
        elif selector is not None:
            effective_selector = selector
        else:
            raise ValueError("Internal error: No selector or text provided.")

        _ = parse_selector(effective_selector)  # Parse once to fail fast on invalid selectors

        selector_kwargs: Dict[str, Any] = dict(extra_kwargs)
        selector_kwargs.update(
            {
                "apply_exclusions": apply_exclusions,
                "regex": regex,
                "case": case,
                "text_tolerance": text_tolerance,
                "auto_text_tolerance": auto_text_tolerance,
                "reading_order": reading_order,
            }
        )

        for page in self.pages:
            element = page.find(selector=effective_selector, **selector_kwargs)
            if element is not None:
                return element

        return None

    def find_all(
        self,
        selector: Optional[str] = None,
        *,
        text: Optional[Union[str, Sequence[str]]] = None,
        apply_exclusions: bool = True,
        regex: bool = False,
        case: bool = True,
        text_tolerance: Optional[Dict[str, Any]] = None,
        auto_text_tolerance: Optional[Dict[str, Any]] = None,
        reading_order: bool = True,
        **extra_kwargs: Any,
    ) -> "ElementCollection":
        """
        Find all elements matching the selector OR text content across all pages.

        Provide EITHER `selector` OR `text`, but not both.

        Args:
            selector: CSS-like selector string.
            text: Optional text shortcut (equivalent to ``text:contains(...)``).
            apply_exclusions: Whether to exclude elements in exclusion regions (default: True).
            regex: Whether to use regex for text search (`selector` or `text`) (default: False).
            case: Whether to do case-sensitive text search (`selector` or `text`) (default: True).
            text_tolerance: Optional dict of tolerance overrides applied temporarily.
            auto_text_tolerance: Optional overrides controlling automatic tolerance logic.
            reading_order: Whether to sort matches in reading order when applicable (default: True).

        Returns:
            ElementCollection with matching elements.
        """
        if not hasattr(self, "_pages"):
            raise AttributeError("PDF pages not yet initialized.")

        if selector is not None and text is not None:
            raise ValueError("Provide either 'selector' or 'text', not both.")
        if selector is None and text is None:
            raise ValueError("Provide either 'selector' or 'text'.")

        # Construct selector if 'text' is provided
        effective_selector = ""
        if text is not None:
            effective_selector = build_text_contains_selector(text)
            logger.debug(
                f"Using text shortcut: find_all(text={text!r}) -> find_all('{effective_selector}')"
            )
        elif selector is not None:
            effective_selector = selector
        else:
            raise ValueError("Internal error: No selector or text provided.")

        _ = parse_selector(effective_selector)  # Validate selector once

        selector_kwargs: Dict[str, Any] = dict(extra_kwargs)
        selector_kwargs.update(
            {
                "apply_exclusions": apply_exclusions,
                "regex": regex,
                "case": case,
                "text_tolerance": text_tolerance,
                "auto_text_tolerance": auto_text_tolerance,
                "reading_order": reading_order,
            }
        )

        all_elements: List = []
        for page in self.pages:
            page_elements = page.find_all(selector=effective_selector, **selector_kwargs)
            if page_elements:
                all_elements.extend(page_elements.elements)

        from natural_pdf.elements.element_collection import ElementCollection

        return ElementCollection(all_elements)

    def extract_text(
        self,
        selector: Optional[str] = None,
        preserve_whitespace: bool = True,
        preserve_line_breaks: bool = True,
        page_separator: Optional[str] = "\n",
        use_exclusions: bool = True,
        debug_exclusions: bool = False,
        *,
        layout: bool = True,
        x_density: Optional[float] = None,
        y_density: Optional[float] = None,
        x_tolerance: Optional[float] = None,
        y_tolerance: Optional[float] = None,
        line_dir: Optional[str] = None,
        char_dir: Optional[str] = None,
        strip_final: bool = False,
        strip_empty: bool = False,
    ) -> str:
        """
        Extract text from the entire document or matching elements.

        Args:
            selector: Optional selector to filter elements
            preserve_whitespace: Whether to keep blank characters
            preserve_line_breaks: When False, collapse newlines in each page's text.
            page_separator: String inserted between page texts when combining results.
            use_exclusions: Whether to apply exclusion regions
            debug_exclusions: Whether to output detailed debugging for exclusions
            preserve_whitespace: Whether to keep blank characters
            use_exclusions: Whether to apply exclusion regions
            debug_exclusions: Whether to output detailed debugging for exclusions
            layout: Whether to enable layout-aware spacing (default: True).
            x_density: Horizontal character density override.
            y_density: Vertical line density override.
            x_tolerance: Horizontal clustering tolerance.
            y_tolerance: Vertical clustering tolerance.
            line_dir: Line reading direction override.
            char_dir: Character reading direction override.
            strip_final: When True, strip trailing whitespace from the combined text.
            strip_empty: When True, drop empty lines from the output.

        Returns:
            Extracted text as string
        """
        if not hasattr(self, "_pages"):
            raise AttributeError("PDF pages not yet initialized.")

        if selector:
            elements = self.find_all(
                selector,
                apply_exclusions=use_exclusions,
                layout=layout,
                x_density=x_density,
                y_density=y_density,
                x_tolerance=x_tolerance,
                y_tolerance=y_tolerance,
                line_dir=line_dir,
                char_dir=char_dir,
                strip_final=strip_final,
                strip_empty=strip_empty,
            )
            return elements.extract_text(
                preserve_whitespace=preserve_whitespace,
                preserve_line_breaks=preserve_line_breaks,
                layout=layout,
                x_density=x_density,
                y_density=y_density,
                x_tolerance=x_tolerance,
                y_tolerance=y_tolerance,
                line_dir=line_dir,
                char_dir=char_dir,
                strip_final=strip_final,
                strip_empty=strip_empty,
            )

        if debug_exclusions:
            print(f"PDF: Extracting text with exclusions from {len(self.pages)} pages")
            print(f"PDF: Found {len(self._exclusions)} document-level exclusions")

        texts = []
        for page in self.pages:
            texts.append(
                page.extract_text(
                    preserve_whitespace=preserve_whitespace,
                    preserve_line_breaks=preserve_line_breaks,
                    use_exclusions=use_exclusions,
                    debug_exclusions=debug_exclusions,
                    layout=layout,
                    x_density=x_density,
                    y_density=y_density,
                    x_tolerance=x_tolerance,
                    y_tolerance=y_tolerance,
                    line_dir=line_dir,
                    char_dir=char_dir,
                    strip_final=strip_final,
                    strip_empty=strip_empty,
                )
            )

        if debug_exclusions:
            print(f"PDF: Combined {len(texts)} pages of text")

        separator = "" if page_separator is None else page_separator
        return separator.join(texts)

    def extract_tables(
        self,
        selector: Optional[str] = None,
        merge_across_pages: bool = False,
        method: Optional[str] = None,
        table_settings: Optional[dict] = None,
        check_tatr: bool = True,
    ) -> List[Any]:
        """
        Extract tables from the document or matching elements.

        Args:
            selector: Optional selector to filter tables (not yet implemented).
            merge_across_pages: Whether to merge tables that span across pages (not yet implemented).
            method: Extraction strategy to prefer. Mirrors ``Page.extract_tables``.
            table_settings: Per-method configuration forwarded to ``Page.extract_tables``.
            check_tatr: When True, visit TATR table regions before falling back to page extraction.

        Returns:
            List of extracted tables
        """
        if not hasattr(self, "_pages"):
            raise AttributeError("PDF pages not yet initialized.")

        logger.warning("PDF.extract_tables is not fully implemented yet.")
        all_tables = []

        for page in self.pages:
            if hasattr(page, "extract_tables"):
                all_tables.extend(
                    page.extract_tables(
                        method=method,
                        table_settings=table_settings,
                        check_tatr=check_tatr,
                    )
                )
            else:
                logger.debug(f"Page {page.number} does not have extract_tables method.")

        if selector:
            logger.warning("Filtering extracted tables by selector is not implemented.")

        if merge_across_pages:
            logger.warning("Merging tables across pages is not implemented.")

        return all_tables

    def get_sections(
        self,
        start_elements=None,
        end_elements=None,
        new_section_on_page_break=False,
        include_boundaries="both",
        orientation="vertical",
    ) -> "ElementCollection":
        """
        Extract sections from the entire PDF based on start/end elements.

        This method delegates to the PageCollection.get_sections() method,
        providing a convenient way to extract document sections across all pages.

        Args:
            start_elements: Elements or selector string that mark the start of sections (optional)
            end_elements: Elements or selector string that mark the end of sections (optional)
            new_section_on_page_break: Whether to start a new section at page boundaries (default: False)
            include_boundaries: How to include boundary elements: 'start', 'end', 'both', or 'none' (default: 'both')
            orientation: 'vertical' (default) or 'horizontal' - determines section direction

        Returns:
            ElementCollection of Region objects representing the extracted sections

        Example:
            Extract sections between headers:
            ```python
            pdf = npdf.PDF("document.pdf")

            # Get sections between headers
            sections = pdf.get_sections(
                start_elements='text[size>14]:bold',
                end_elements='text[size>14]:bold'
            )

            # Get sections that break at page boundaries
            sections = pdf.get_sections(
                start_elements='text:contains("Chapter")',
                new_section_on_page_break=True
            )
            ```

        Note:
            You can provide only start_elements, only end_elements, or both.
            - With only start_elements: sections go from each start to the next start (or end of document)
            - With only end_elements: sections go from beginning of document to each end
            - With both: sections go from each start to the corresponding end
        """
        if not hasattr(self, "_pages"):
            raise AttributeError("PDF pages not yet initialized.")

        return self.pages.get_sections(
            start_elements=start_elements,
            end_elements=end_elements,
            new_section_on_page_break=new_section_on_page_break,
            include_boundaries=include_boundaries,
            orientation=orientation,
        )

    def split(
        self,
        divider,
        *,
        include_boundaries: str = "start",
        orientation: str = "vertical",
        new_section_on_page_break: bool = False,
    ) -> "ElementCollection":
        """
        Divide the PDF into sections based on the provided divider elements.

        Args:
            divider: Elements or selector string that mark section boundaries
            include_boundaries: How to include boundary elements (default: 'start').
            orientation: 'vertical' or 'horizontal' (default: 'vertical').
            new_section_on_page_break: Whether to split at page boundaries (default: False).

        Returns:
            ElementCollection of Region objects representing the sections

        Example:
            # Split a PDF by chapter titles
            chapters = pdf.split("text[size>20]:contains('Chapter')")

            # Export each chapter to a separate file
            for i, chapter in enumerate(chapters):
                chapter_text = chapter.extract_text()
                with open(f"chapter_{i+1}.txt", "w") as f:
                    f.write(chapter_text)

            # Split by horizontal rules/lines
            sections = pdf.split("line[orientation=horizontal]")

            # Split only by page breaks (no divider elements)
            pages = pdf.split(None, new_section_on_page_break=True)
        """
        # Delegate to pages collection
        return self.pages.split(
            divider,
            include_boundaries=include_boundaries,
            orientation=orientation,
            new_section_on_page_break=new_section_on_page_break,
        )

    def save_searchable(self, output_path: Union[str, "Path"], dpi: int = 300):
        """
        DEPRECATED: Use save_pdf(..., ocr=True) instead.
        Saves the PDF with an OCR text layer, making content searchable.

        Requires optional dependencies. Install with: pip install \"natural-pdf[ocr-export]\"

        Args:
            output_path: Path to save the searchable PDF
            dpi: Resolution for rendering and OCR overlay.
        """
        logger.warning(
            "PDF.save_searchable() is deprecated. Use PDF.save_pdf(..., ocr=True) instead."
        )
        if create_searchable_pdf is None:
            raise ImportError(
                "Saving searchable PDF requires 'pikepdf'. "
                'Install with: pip install "natural-pdf[ocr-export]"'
            )
        output_path_str = str(output_path)
        # Call the exporter directly, passing self (the PDF instance)
        create_searchable_pdf(self, output_path_str, dpi=dpi)
        # Logger info is handled within the exporter now
        # logger.info(f"Searchable PDF saved to: {output_path_str}")

    def save_pdf(
        self,
        output_path: Union[str, Path],
        ocr: bool = False,
        original: bool = False,
        dpi: int = 300,
    ):
        """
        Saves the PDF object (all its pages) to a new file.

        Choose one saving mode:
        - `ocr=True`: Creates a new, image-based PDF using OCR results from all pages.
          Text generated during the natural-pdf session becomes searchable,
          but original vector content is lost. Requires 'ocr-export' extras.
        - `original=True`: Saves a copy of the original PDF file this object represents.
          Any OCR results or analyses from the natural-pdf session are NOT included.
          If the PDF was opened from an in-memory buffer, this mode may not be suitable.
          Requires 'ocr-export' extras.

        Args:
            output_path: Path to save the new PDF file.
            ocr: If True, save as a searchable, image-based PDF using OCR data.
            original: If True, save the original source PDF content.
            dpi: Resolution (dots per inch) used only when ocr=True.

        Raises:
            ValueError: If the PDF has no pages, if neither or both 'ocr'
                        and 'original' are True.
            ImportError: If required libraries are not installed for the chosen mode.
            RuntimeError: If an unexpected error occurs during saving.
        """
        if not self.pages:
            raise ValueError("Cannot save an empty PDF object.")

        if not (ocr ^ original):  # XOR: exactly one must be true
            raise ValueError("Exactly one of 'ocr' or 'original' must be True.")

        output_path_obj = Path(output_path)
        output_path_str = str(output_path_obj)

        if ocr:
            if create_searchable_pdf is None:
                raise ImportError(
                    "Saving with ocr=True requires the OCR export dependencies. "
                    'Install with: pip install "natural-pdf[ocr-export]"'
                )
            has_vector_elements = False
            for page in self.pages:
                rects = getattr(page, "rects", None)
                lines = getattr(page, "lines", None)
                curves = getattr(page, "curves", None)
                chars = getattr(page, "chars", None)
                words = getattr(page, "words", None)
                if (
                    (rects and len(rects) > 0)  # type: ignore[arg-type]
                    or (lines and len(lines) > 0)  # type: ignore[arg-type]
                    or (curves and len(curves) > 0)  # type: ignore[arg-type]
                    or (
                        chars
                        and any(
                            getattr(el, "source", None) != "ocr"
                            for el in cast(Iterable[Any], chars)
                        )
                    )
                    or (
                        words
                        and any(
                            getattr(el, "source", None) != "ocr"
                            for el in cast(Iterable[Any], words)
                        )
                    )
                ):
                    has_vector_elements = True
                    break
            if has_vector_elements:
                logger.warning(
                    "Warning: Saving with ocr=True creates an image-based PDF. "
                    "Original vector elements (rects, lines, non-OCR text/chars) "
                    "will not be preserved in the output file."
                )

            logger.info(f"Saving searchable PDF (OCR text layer) to: {output_path_str}")
            try:
                # Delegate to the searchable PDF exporter, passing self (PDF instance)
                create_searchable_pdf(self, output_path_str, dpi=dpi)
            except Exception as e:
                raise RuntimeError(f"Failed to create searchable PDF: {e}") from e

        elif original:
            if create_original_pdf is None:
                raise ImportError(
                    "Saving with original=True requires 'pikepdf'. "
                    'Install with: pip install "natural-pdf[ocr-export]"'
                )

            # Optional: Add warning about losing OCR data similar to PageCollection
            has_ocr_elements = False
            for page in self.pages:
                if hasattr(page, "find_all"):
                    ocr_text_elements = page.find_all("text[source=ocr]")
                    if ocr_text_elements:
                        has_ocr_elements = True
                        break
                elif hasattr(page, "words"):  # Fallback
                    if any(getattr(el, "source", None) == "ocr" for el in page.words):
                        has_ocr_elements = True
                        break
            if has_ocr_elements:
                logger.warning(
                    "Warning: Saving with original=True preserves original page content. "
                    "OCR text generated in this session will not be included in the saved file."
                )

            logger.info(f"Saving original PDF content to: {output_path_str}")
            try:
                # Delegate to the original PDF exporter, passing self (PDF instance)
                create_original_pdf(self, output_path_str)
            except Exception as e:
                # Re-raise exception from exporter
                raise e

    def _get_render_specs(
        self,
        mode: Literal["show", "render"] = "show",
        color: Optional[Union[str, Tuple[int, int, int]]] = None,
        highlights: Optional[List[Dict[str, Any]]] = None,
        crop: Union[bool, Literal["content"]] = False,
        crop_bbox: Optional[Tuple[float, float, float, float]] = None,
        **kwargs,
    ) -> List[RenderSpec]:
        """Get render specifications for this PDF.

        For PDF objects, this delegates to the pages collection to handle
        multi-page rendering.

        Args:
            mode: Rendering mode - 'show' includes highlights, 'render' is clean
            color: Color for highlighting pages in show mode
            highlights: Additional highlight groups to show
            crop: Whether to crop pages
            crop_bbox: Explicit crop bounds
            **kwargs: Additional parameters

        Returns:
            List of RenderSpec objects, one per page
        """
        # Delegate to pages collection
        return self.pages._get_render_specs(
            mode=mode, color=color, highlights=highlights, crop=crop, crop_bbox=crop_bbox, **kwargs
        )

    def _resolve_qa_pages(self, pages: Optional[Union[int, Iterable[int], range]]) -> List["Page"]:
        """
        Normalize the ``pages`` argument for QA operations into a list of Page objects.

        Args:
            pages: None for all pages, an integer page index, or an iterable of indices.

        Returns:
            List of resolved Page objects (may be empty if no valid indices were supplied).
        """
        if pages is None:
            return list(self.pages)

        total_pages = len(self.pages)

        if isinstance(pages, int):
            if 0 <= pages < total_pages:
                return [cast("Page", self.pages[pages])]
            raise IndexError(f"Page index {pages} out of range (0-{total_pages-1})")

        if isinstance(pages, range) or isinstance(pages, list) or isinstance(pages, tuple):
            resolved: List["Page"] = []
            for page_idx in pages:
                if isinstance(page_idx, int) and 0 <= page_idx < total_pages:
                    resolved.append(cast("Page", self.pages[page_idx]))
                else:
                    logger.warning(f"Page index {page_idx} out of range, skipping")
            return resolved

        raise ValueError(f"Invalid pages parameter: {pages}")

    def ask(
        self,
        question: str,
        *,
        mode: Literal["extractive", "generative"] = "extractive",
        pages: Optional[Union[int, Iterable[int], range]] = None,
        min_confidence: float = 0.1,
        model: Optional[str] = None,
        temperature: Optional[float] = None,
        top_p: Optional[float] = None,
        llm_client: Optional[Any] = None,
        prompt: Optional[str] = None,
    ) -> QAResult:
        """
        Ask a single question about the document content.

        Args:
            question: Question string to ask about the document
            mode: "extractive" to extract answer from document, "generative" to generate
            pages: Specific pages to query (default: all pages)
            min_confidence: Minimum confidence threshold for answers (extractive mode).
            model: Optional model name for the QA engine or LLM.
            temperature: Optional sampling temperature for LLM-backed QA.
            top_p: Optional nucleus sampling parameter for LLM-backed QA.
            llm_client: Client instance to use when ``mode="generative"``.
            prompt: Optional system prompt override for generative QA.

        Returns:
            :class:`QAResult` containing answer metadata. Confidence may be ``None`` in generative
            mode; ``source_elements`` may be empty if no span is available.
        """
        # Delegate to ask_batch and return the first result
        results = self.ask_batch(
            [question],
            mode=mode,
            pages=pages,
            min_confidence=min_confidence,
            model=model,
            temperature=temperature,
            top_p=top_p,
            llm_client=llm_client,
            prompt=prompt,
        )
        if results:
            return results[0]
        return QAResult(
            {
                "answer": None,
                "confidence": 0.0,
                "found": False,
                "page_num": None,
                "source_elements": [],
            }
        )

    def ask_batch(
        self,
        questions: List[str],
        *,
        mode: Literal["extractive", "generative"] = "extractive",
        pages: Optional[Union[int, Iterable[int], range]] = None,
        min_confidence: float = 0.1,
        model: Optional[str] = None,
        temperature: Optional[float] = None,
        top_p: Optional[float] = None,
        llm_client: Optional[Any] = None,
        prompt: Optional[str] = None,
    ) -> List[QAResult]:
        """
        Ask multiple questions about the document content using batch processing.

        This method processes multiple questions efficiently in a single batch,
        avoiding the multiprocessing resource accumulation that can occur with
        sequential individual question calls.

        Args:
            questions: List of question strings to ask about the document
            mode: "extractive" to extract answer from document, "generative" to generate
            pages: Specific pages to query (default: all pages)
            min_confidence: Minimum confidence threshold for extractive answers.
            model: Optional model name for the QA engine or LLM.
            temperature: Optional sampling temperature for LLM-backed QA.
            top_p: Optional nucleus sampling parameter for LLM-backed QA.
            llm_client: Client instance to use when ``mode="generative"``.
            prompt: Optional system prompt override for generative QA.

        Returns:
            List of :class:`QAResult` objects containing answer metadata. Confidence may be
            ``None`` in generative mode; ``source_elements`` may be empty if no span is available.
        """
        if not questions:
            return []

        if mode not in ("extractive", "generative"):
            raise ValueError("mode must be either 'extractive' or 'generative'")

        if not isinstance(questions, list) or not all(isinstance(q, str) for q in questions):
            raise TypeError("'questions' must be a list of strings")

        target_pages = self._resolve_qa_pages(pages)

        def _empty_result() -> QAResult:
            return QAResult(
                {
                    "answer": None,
                    "confidence": 0.0,
                    "found": False,
                    "page_num": None,
                    "source_elements": [],
                }
            )

        if not target_pages:
            logger.warning("No valid pages found for QA processing.")
            return [_empty_result() for _ in questions]

        if mode == "generative":
            return self._ask_batch_generative(
                questions,
                target_pages=target_pages,
                llm_client=llm_client,
                prompt=prompt,
                model=model,
                temperature=temperature,
                top_p=top_p,
            )

        from natural_pdf.qa import get_qa_engine

        qa_engine = get_qa_engine() if model is None else get_qa_engine(model_name=model)

        if temperature is not None:
            set_temp = getattr(qa_engine, "set_temperature", None)
            if callable(set_temp):
                try:
                    set_temp(temperature)
                except Exception:
                    logger.debug("QA engine rejected temperature override; ignoring.")

        if top_p is not None:
            set_top_p = getattr(qa_engine, "set_top_p", None)
            if callable(set_top_p):
                try:
                    set_top_p(top_p)
                except Exception:
                    logger.debug("QA engine rejected top_p override; ignoring.")

        logger.info(
            f"Processing {len(questions)} question(s) across {len(target_pages)} page(s) using batch QA..."
        )

        # Collect all page images and metadata for batch processing
        page_images: List[Image.Image] = []
        page_word_boxes: List[Any] = []
        page_metadata: List[Dict[str, Any]] = []

        for page in target_pages:
            # Get page image
            try:
                # Use render() for clean image without highlights
                page_image = page.render(resolution=150)
                if page_image is None:
                    logger.warning(f"Failed to render image for page {page.number}, skipping")
                    continue

                # Get text elements for word boxes
                elements = page.find_all("text")
                if not elements:
                    logger.warning(f"No text elements found on page {page.number}")
                    word_boxes = []
                else:
                    word_boxes = qa_engine._get_word_boxes_from_elements(
                        elements, offset_x=0, offset_y=0
                    )

                page_images.append(page_image)
                page_word_boxes.append(word_boxes)
                page_metadata.append({"page_number": page.number, "page_object": page})

            except Exception as e:
                logger.warning(f"Error processing page {page.number}: {e}")
                continue

        if not page_images:
            logger.warning("No page images could be processed for QA.")
            return [_empty_result() for _ in questions]

        # Process all questions against all pages in batch
        all_results: List[QAResult] = []

        for question_text in questions:
            question_results: List[QAResult] = []

            # Ask this question against each page (but in batch per page)
            for i, (page_image, word_boxes, page_meta) in enumerate(
                zip(page_images, page_word_boxes, page_metadata)
            ):
                try:
                    # Use the DocumentQA batch interface
                    raw_result = qa_engine.ask(
                        image=page_image,
                        question=question_text,
                        word_boxes=word_boxes,
                        min_confidence=min_confidence,
                    )

                    page_result: Optional[QAResult]
                    if isinstance(raw_result, QAResult):
                        page_result = raw_result
                    elif isinstance(raw_result, list) and raw_result:
                        page_result = raw_result[0]
                    else:
                        page_result = None

                    if page_result and page_result.found:
                        qa_result = QAResult(
                            {
                                "answer": page_result.answer,
                                "confidence": page_result.confidence,
                                "found": page_result.found,
                                "page_num": page_meta["page_number"],
                                "source_elements": getattr(page_result, "source_elements", []),
                                "start": getattr(page_result, "start", -1),
                                "end": getattr(page_result, "end", -1),
                            }
                        )
                        question_results.append(qa_result)

                except Exception as e:
                    logger.warning(
                        f"Error processing question '{question_text}' on page {page_meta['page_number']}: {e}"
                    )
                    continue

            # Sort results by confidence and take the best one for this question
            question_results.sort(key=lambda x: x.get("confidence", 0), reverse=True)

            if question_results:
                all_results.append(question_results[0])
            else:
                # No results found for this question
                all_results.append(_empty_result())

        return all_results

    def _ask_batch_generative(
        self,
        questions: List[str],
        *,
        target_pages: List["Page"],
        llm_client: Optional[Any],
        prompt: Optional[str],
        model: Optional[str],
        temperature: Optional[float],
        top_p: Optional[float],
    ) -> List[QAResult]:
        """Handle the ``mode='generative'`` logic for :meth:`ask_batch`."""

        def _empty_result() -> QAResult:
            return QAResult(
                {
                    "answer": None,
                    "confidence": 0.0,
                    "found": False,
                    "page_num": None,
                    "source_elements": [],
                }
            )

        if llm_client is None:
            raise ValueError("Generative QA requires 'llm_client' to be provided.")

        try:
            manager = self.get_manager("structured_data")
        except (KeyError, RuntimeError) as exc:
            raise RuntimeError(
                "StructuredDataManager is not available; cannot run generative QA."
            ) from exc

        if manager is None or not manager.is_available():
            raise RuntimeError("StructuredDataManager is not available for generative QA.")

        # Compile text content from selected pages
        page_sections: List[str] = []
        for page in target_pages:
            page_text = ""
            try:
                page_text = page.extract_text(layout=True) or ""
            except Exception as exc:  # noqa: BLE001
                logger.debug(
                    "Failed to extract layout text from page %s: %s",
                    getattr(page, "number", "?"),
                    exc,
                )

            if not page_text.strip():
                try:
                    page_text = page.extract_text(layout=False) or ""
                except Exception as exc:  # noqa: BLE001
                    logger.debug(
                        "Failed to extract text (layout=False) from page %s: %s",
                        getattr(page, "number", "?"),
                        exc,
                    )

            page_text = page_text.strip()
            if page_text:
                page_sections.append(f"Page {page.number}:\n{page_text}")

        combined_text = "\n\n".join(page_sections).strip()
        if not combined_text:
            logger.warning(
                "Generative QA could not extract text from the selected pages. "
                "Consider running apply_ocr() before using mode='generative'."
            )
            return [_empty_result() for _ in questions]

        max_chars = 20000
        if len(combined_text) > max_chars:
            logger.info(
                "Truncating combined page text for generative QA from %d to %d characters",
                len(combined_text),
                max_chars,
            )
            combined_text = combined_text[:max_chars]

        GenerativeQAResponse = create_model(
            "GenerativeQAResponse",
            answer=(str, Field("", description="Answer to the question; leave empty if unknown.")),
        )

        llm_kwargs: Dict[str, Any] = {}
        if temperature is not None:
            llm_kwargs["temperature"] = temperature
        if top_p is not None:
            llm_kwargs["top_p"] = top_p

        prompt_text = prompt or DEFAULT_GENERATIVE_QA_PROMPT
        results: List[QAResult] = []

        for question_text in questions:
            question_text = question_text.strip()
            if not question_text:
                results.append(_empty_result())
                continue

            payload = json.dumps(
                {
                    "question": question_text,
                    "document": combined_text,
                }
            )

            try:
                extraction_result = manager.extract(
                    content=payload,
                    schema=GenerativeQAResponse,
                    client=llm_client,
                    prompt=prompt_text,
                    using="text",
                    model=model,
                    **llm_kwargs,
                )
            except Exception as exc:  # noqa: BLE001
                logger.error("Generative QA failed for question %r: %s", question_text, exc)
                results.append(_empty_result())
                continue

            parsed = getattr(extraction_result, "data", None)
            if not extraction_result.success or parsed is None:
                logger.debug(
                    "Generative QA returned no parsed data for question %r (success=%s, error=%s)",
                    question_text,
                    getattr(extraction_result, "success", None),
                    getattr(extraction_result, "error_message", None),
                )
                results.append(_empty_result())
                continue

            answer = getattr(parsed, "answer", "")
            normalized_answer = answer.strip() if isinstance(answer, str) else ""

            found = bool(normalized_answer)

            if not found:
                results.append(_empty_result())
                continue

            results.append(
                QAResult(
                    {
                        "answer": normalized_answer,
                        "confidence": None,
                        "found": True,
                        "page_num": None,
                        "source_elements": [],
                    }
                )
            )

        return results

    def search_within_index(
        self,
        query: Union[str, Path, Image.Image, "Region"],
        search_service: "SearchServiceProtocol",
        options: Optional["SearchOptions"] = None,
    ) -> List[Dict[str, Any]]:
        """
        Finds relevant documents from this PDF within a search index.
        Finds relevant documents from this PDF within a search index.

        Args:
            query: The search query (text, image path, PIL Image, Region)
            search_service: A pre-configured SearchService instance
            options: Optional SearchOptions to configure the query
            query: The search query (text, image path, PIL Image, Region)
            search_service: A pre-configured SearchService instance
            options: Optional SearchOptions to configure the query

        Returns:
            A list of result dictionaries, sorted by relevance
            A list of result dictionaries, sorted by relevance

        Raises:
            ImportError: If search dependencies are not installed
            ValueError: If search_service is None
            TypeError: If search_service does not conform to the protocol
            FileNotFoundError: If the collection managed by the service does not exist
            RuntimeError: For other search failures
            ImportError: If search dependencies are not installed
            ValueError: If search_service is None
            TypeError: If search_service does not conform to the protocol
            FileNotFoundError: If the collection managed by the service does not exist
            RuntimeError: For other search failures
        """
        if not search_service:
            raise ValueError("A configured SearchServiceProtocol instance must be provided.")

        collection_name = getattr(search_service, "collection_name", "<Unknown Collection>")
        logger.info(
            f"Searching within index '{collection_name}' for content from PDF '{self.path}'"
        )

        service = search_service

        query_input = query
        effective_options = copy.deepcopy(options) if options is not None else TextSearchOptions()

        if isinstance(query, Region):
            logger.debug("Query is a Region object. Extracting text.")
            if not isinstance(effective_options, TextSearchOptions):
                logger.warning(
                    "Querying with Region image requires MultiModalSearchOptions. Falling back to text extraction."
                )
            query_input = query.extract_text()
            if not query_input or query_input.isspace():
                logger.error("Region has no extractable text for query.")
                return []

        # Add filter to scope search to THIS PDF
        # Add filter to scope search to THIS PDF
        pdf_scope_filter = {
            "field": "pdf_path",
            "operator": "eq",
            "value": self.path,
        }
        logger.debug(f"Applying filter to scope search to PDF: {pdf_scope_filter}")

        # Combine with existing filters in options (if any)
        if effective_options.filters:
            logger.debug("Combining PDF scope filter with existing filters")
            if (
                isinstance(effective_options.filters, dict)
                and effective_options.filters.get("operator") == "AND"
            ):
                effective_options.filters["conditions"].append(pdf_scope_filter)
            elif isinstance(effective_options.filters, list):
                effective_options.filters = {
                    "operator": "AND",
                    "conditions": effective_options.filters + [pdf_scope_filter],
                }
            elif isinstance(effective_options.filters, dict):
                effective_options.filters = {
                    "operator": "AND",
                    "conditions": [effective_options.filters, pdf_scope_filter],
                }
            else:
                logger.warning(
                    "Unsupported format for existing filters. Overwriting with PDF scope filter."
                )
                effective_options.filters = pdf_scope_filter
        else:
            effective_options.filters = pdf_scope_filter

        logger.debug(f"Final filters for service search: {effective_options.filters}")

        try:
            results = service.search(
                query=query_input,
                options=effective_options,
            )
            logger.info(f"SearchService returned {len(results)} results from PDF '{self.path}'")
            return results
        except FileNotFoundError as fnf:
            logger.error(f"Search failed: Collection not found. Error: {fnf}")
            raise
            logger.error(f"Search failed: Collection not found. Error: {fnf}")
            raise
        except Exception as e:
            logger.error(f"SearchService search failed: {e}")
            raise RuntimeError("Search within index failed. See logs for details.") from e
            logger.error(f"SearchService search failed: {e}")
            raise RuntimeError("Search within index failed. See logs for details.") from e

    def export_ocr_correction_task(
        self,
        output_zip_path: str,
        *,
        overwrite: bool = False,
        suggest=None,
        resolution: int = 300,
    ):
        """
        Exports OCR results from this PDF into a correction task package.
        Exports OCR results from this PDF into a correction task package.

        Args:
            output_zip_path: The path to save the output zip file.
            overwrite: When True, replace any existing archive at ``output_zip_path``.
            suggest: Optional callable that can provide OCR suggestions per region.
            resolution: DPI used when rendering page images for the package.
        """
        try:
            from natural_pdf.utils.packaging import create_correction_task_package

            create_correction_task_package(
                source=self,
                output_zip_path=output_zip_path,
                overwrite=overwrite,
                suggest=suggest,
                resolution=resolution,
            )
        except ImportError:
            logger.error(
                "Failed to import 'create_correction_task_package'. Packaging utility might be missing."
            )
            logger.error(
                "Failed to import 'create_correction_task_package'. Packaging utility might be missing."
            )
        except Exception as e:
            logger.error(f"Failed to export correction task: {e}")
            raise
            logger.error(f"Failed to export correction task: {e}")
            raise

    def update_text(
        self,
        transform: Callable[[Any], Optional[str]],
        *,
        selector: str = "text",
        apply_exclusions: bool = False,
        pages: Optional[Union[Iterable[int], range, slice]] = None,
        max_workers: Optional[int] = None,
        progress_callback: Optional[Callable[[], None]] = None,
    ) -> "PDF":
        """
        Applies corrections to text elements using a callback function.

        Args:
            transform: Function that takes an element and returns corrected text or None
            selector: Selector to apply corrections to (default: "text")
            apply_exclusions: Whether to honour exclusion regions while selecting text.
            pages: Optional page indices/slice to limit the scope of correction
            max_workers: Maximum number of threads to use for parallel execution
            progress_callback: Optional callback function for progress updates

        Returns:
            Self for method chaining
        """
        pages_to_update = self._get_target_pages(pages)

        if not pages_to_update:
            logger.warning("No pages selected for text update.")
            return self

        page_identities = [page.index for page in pages_to_update]
        logger.info(f"Starting text update for pages: {page_identities} with selector='{selector}'")

        for page in pages_to_update:
            try:
                page.update_text(
                    transform=transform,
                    selector=selector,
                    apply_exclusions=apply_exclusions,
                    max_workers=max_workers,
                    progress_callback=progress_callback,
                )
            except Exception as e:
                logger.error(f"Error during text update on page {page.index}: {e}")
                logger.error(f"Error during text update on page {page.index}: {e}")

        logger.info("Text update process finished.")
        return self

    def __len__(self) -> int:
        """Return the number of pages in the PDF."""
        if not hasattr(self, "_pages"):
            return 0
        return len(self._pages)

    def __getitem__(self, key) -> Union["Page", "PageCollection"]:
        """Access pages by index or slice."""
        if not hasattr(self, "_pages"):
            raise AttributeError("PDF pages not initialized yet.")

        if isinstance(key, slice):
            from natural_pdf.core.page_collection import PageCollection

            # Use the lazy page list's slicing which returns another _LazyPageList
            lazy_slice = self._pages[key]
            # Wrap in PageCollection for compatibility
            return PageCollection(lazy_slice)
        elif isinstance(key, int):
            if 0 <= key < len(self._pages):
                return self._pages[key]
            else:
                raise IndexError(f"Page index {key} out of range (0-{len(self._pages)-1}).")
        else:
            raise TypeError(f"Page indices must be integers or slices, not {type(key)}.")

    def close(self):
        """Close the underlying PDF file and clean up any temporary files."""
        if hasattr(self, "_pdf") and self._pdf is not None:
            try:
                self._pdf.close()
                logger.debug(f"Closed pdfplumber PDF object for {self.source_path}")
            except Exception as e:
                logger.warning(f"Error closing pdfplumber object: {e}")
            finally:
                self._pdf = None

        if hasattr(self, "_temp_file") and self._temp_file is not None:
            temp_file_path = None
            try:
                if hasattr(self._temp_file, "name") and self._temp_file.name:
                    temp_file_path = self._temp_file.name
                    # Only unlink if it exists and _is_stream is False (meaning WE created it)
                    if not self._is_stream and os.path.exists(temp_file_path):
                        os.unlink(temp_file_path)
                        logger.debug(f"Removed temporary PDF file: {temp_file_path}")
            except Exception as e:
                logger.warning(f"Failed to clean up temporary file '{temp_file_path}': {e}")

        # Cancels the weakref finalizer so we don't double-clean
        if hasattr(self, "_finalizer") and self._finalizer.alive:
            self._finalizer()

    def __enter__(self):
        """Context manager entry."""
        return self

    def __exit__(self, exc_type, exc_val, exc_tb):
        """Context manager exit."""
        self.close()

    def __repr__(self) -> str:
        """Return a string representation of the PDF object."""
        if not hasattr(self, "_pages"):
            page_count_str = "uninitialized"
        else:
            page_count_str = str(len(self._pages))

        source_info = getattr(self, "source_path", "unknown source")
        return f"<PDF source='{source_info}' pages={page_count_str}>"

    def get_id(self) -> str:
        """Get unique identifier for this PDF."""
        """Get unique identifier for this PDF."""
        return self.path

    # --- Deskew Method --- #

    def deskew(
        self,
        pages: Optional[Union[Iterable[int], range, slice]] = None,
        resolution: int = 300,
        angle: Optional[float] = None,
        detection_resolution: int = 72,
        force_overwrite: bool = False,
        **deskew_kwargs,
    ) -> "PDF":
        """
        Creates a new, in-memory PDF object containing deskewed versions of the
        specified pages from the original PDF.

        This method renders each selected page, detects and corrects skew using the 'deskew'
        library, and then combines the resulting images into a new PDF using 'img2pdf'.
        The new PDF object is returned directly.

        Important: The returned PDF is image-based. Any existing text, OCR results,
        annotations, or other elements from the original pages will *not* be carried over.

        Args:
            pages: Page indices/slice to include (0-based). If None, processes all pages.
            resolution: DPI resolution for rendering the output deskewed pages.
            angle: The specific angle (in degrees) to rotate by. If None, detects automatically.
            detection_resolution: DPI resolution used for skew detection if angles are not
                                  already cached on the page objects.
            force_overwrite: If False (default), raises a ValueError if any target page
                             already contains processed elements (text, OCR, regions) to
                             prevent accidental data loss. Set to True to proceed anyway.
            **deskew_kwargs: Additional keyword arguments forwarded to the deskew engine.
                             during automatic detection (e.g., `max_angle`, `num_peaks`).

        Returns:
            A new PDF object representing the deskewed document.

        Raises:
            ImportError: If 'deskew' or 'img2pdf' libraries are not installed.
            ValueError: If `force_overwrite` is False and target pages contain elements.
            FileNotFoundError: If the source PDF cannot be read (if file-based).
            IOError: If creating the in-memory PDF fails.
            RuntimeError: If rendering or deskewing individual pages fails.
        """
        if not DESKEW_AVAILABLE:
            raise ImportError(
                "Deskew/img2pdf libraries missing. Install with: pip install natural-pdf[deskew]"
            )

        target_pages = self._get_target_pages(pages)  # Use helper to resolve pages

        # --- Safety Check --- #
        if not force_overwrite:
            for page in target_pages:
                # Check if the element manager has been initialized and contains any elements
                if hasattr(page, "has_element_cache") and page.has_element_cache():
                    raise ValueError(
                        f"Page {page.number} contains existing elements (text, OCR, etc.). "
                        f"Deskewing creates an image-only PDF, discarding these elements. "
                        f"Set force_overwrite=True to proceed."
                    )

        # --- Process Pages --- #
        deskewed_images_bytes = []
        logger.info(f"Deskewing {len(target_pages)} pages (output resolution={resolution} DPI)...")

        for page in tqdm(target_pages, desc="Deskewing Pages", leave=False):
            try:
                # Use page.deskew to get the corrected PIL image
                # Pass down resolutions and kwargs
                deskewed_img = page.deskew(
                    resolution=resolution,
                    angle=angle,  # Let page.deskew handle detection/caching
                    detection_resolution=detection_resolution,
                    **deskew_kwargs,
                )

                if not deskewed_img:
                    logger.warning(
                        f"Page {page.number}: Failed to generate deskewed image, skipping."
                    )
                    continue

                # Convert image to bytes for img2pdf (use PNG for lossless quality)
                with io.BytesIO() as buf:
                    deskewed_img.save(buf, format="PNG")
                    deskewed_images_bytes.append(buf.getvalue())

            except Exception as e:
                logger.error(
                    f"Page {page.number}: Failed during deskewing process: {e}", exc_info=True
                )
                # Option: Raise a runtime error, or continue and skip the page?
                # Raising makes the whole operation fail if one page fails.
                raise RuntimeError(f"Failed to process page {page.number} during deskewing.") from e

        # --- Create PDF --- #
        if not deskewed_images_bytes:
            raise RuntimeError("No pages were successfully processed to create the deskewed PDF.")

        logger.info(f"Combining {len(deskewed_images_bytes)} deskewed images into in-memory PDF...")
        try:
            # Use img2pdf to combine image bytes into PDF bytes
            if img2pdf is None:
                raise RuntimeError(
                    "img2pdf library is not available despite DESKEW_AVAILABLE flag being set."
                )
            pdf_bytes = img2pdf.convert(deskewed_images_bytes)
            if pdf_bytes is None:
                raise RuntimeError("img2pdf.convert returned no data.")

            # Wrap bytes in a stream
            pdf_stream = io.BytesIO(pdf_bytes)

            # Create a new PDF object from the stream using original config
            logger.info("Creating new PDF object from deskewed stream...")
            new_pdf = PDF(
                pdf_stream,
                reading_order=self._reading_order,
                font_attrs=self._font_attrs,
                keep_spaces=self._config.get("keep_spaces", True),
                text_layer=self._text_layer,
            )
            return new_pdf
        except Exception as e:
            logger.error(f"Failed to create in-memory PDF using img2pdf/PDF init: {e}")
            raise IOError("Failed to create deskewed PDF object from image stream.") from e

    # --- End Deskew Method --- #

    # --- Classification Methods --- #

    def classify_pages(
        self,
        labels: List[str],
        model: Optional[str] = None,
        pages: Optional[Union[Iterable[int], range, slice]] = None,
        analysis_key: str = "classification",
        using: Optional[str] = None,
        min_confidence: float = 0.0,
        multi_label: bool = False,
        **kwargs,
    ) -> "PDF":
        """
        Classifies specified pages of the PDF.

        Args:
            labels: List of category names
            model: Model identifier ('text', 'vision', or specific HF ID)
            pages: Page indices, slice, or None for all pages
            analysis_key: Key to store results in page's analyses dict
            using: Processing mode ('text' or 'vision')
            **kwargs: Additional arguments for the ClassificationManager

        Returns:
            Self for method chaining
        """
        if not labels:
            raise ValueError("Labels list cannot be empty.")

        target_pages = self._get_target_pages(pages)

        if not target_pages:
            logger.warning("No pages selected for classification.")
            return self

        engine_name = kwargs.pop("classification_engine", None)
        engine_obj = get_classification_engine(self, engine_name)
        inferred_using = engine_obj.infer_using(model or engine_obj.default_model("text"), using)
        logger.info(
            f"Classifying {len(target_pages)} pages using model '{model or '(default)'}' (mode: {inferred_using})"
        )

        page_contents = []
        pages_to_classify = []
        logger.debug(f"Gathering content for {len(target_pages)} pages...")

        for page in target_pages:
            try:
                content = page._get_classification_content(model_type=inferred_using, **kwargs)
                page_contents.append(content)
                pages_to_classify.append(page)
            except ValueError as e:
                logger.warning(f"Skipping page {page.number}: Cannot get content - {e}")
            except Exception as e:
                logger.warning(f"Skipping page {page.number}: Error getting content - {e}")

        if not page_contents:
            logger.warning("No content could be gathered for batch classification.")
            return self

        logger.debug(f"Gathered content for {len(pages_to_classify)} pages.")

        try:
            batch_results = run_classification_batch(
                context=self,
                contents=page_contents,
                labels=labels,
                model_id=model or engine_obj.default_model(inferred_using),
                using=inferred_using,
                min_confidence=min_confidence,
                multi_label=multi_label,
                batch_size=8,
                progress_bar=True,
                engine_name=engine_name,
                **kwargs,
            )
        except Exception as e:
            logger.error(f"Batch classification failed: {e}")
            raise ClassificationError(f"Batch classification failed: {e}") from e

        if len(batch_results) != len(pages_to_classify):
            logger.error(
                f"Mismatch between number of results ({len(batch_results)}) and pages ({len(pages_to_classify)})"
            )
            return self

        logger.debug(
            f"Distributing {len(batch_results)} results to pages under key '{analysis_key}'..."
        )
        for page, result_obj in zip(pages_to_classify, batch_results):
            try:
                if not hasattr(page, "analyses") or page.analyses is None:
                    page.analyses = {}
                page.analyses[analysis_key] = result_obj
            except Exception as e:
                logger.warning(
                    f"Failed to store classification results for page {page.number}: {e}"
                )

        logger.info("Finished classifying PDF pages.")
        return self

    # --- End Classification Methods --- #

    # --- Extraction Support --- #
    def _get_extraction_content(self, using: str = "text", **kwargs) -> Any:
        """
        Retrieves the content for the entire PDF.

        Args:
            using: 'text' or 'vision'
            **kwargs: Additional arguments passed to extract_text or page.to_image

        Returns:
            str: Extracted text if using='text'
            List[PIL.Image.Image]: List of page images if using='vision'
            None: If content cannot be retrieved
        """
        if using == "text":
            try:
                layout = kwargs.pop("layout", True)
                return self.extract_text(layout=layout, **kwargs)
            except Exception as e:
                logger.error(f"Error extracting text from PDF: {e}")
                return None
        elif using == "vision":
            page_images = []
            logger.info(f"Rendering {len(self.pages)} pages to images...")

            resolution = kwargs.pop("resolution", 72)
            kwargs.pop("include_highlights", False)
            kwargs.pop("labels", False)

            try:
                for page in tqdm(self.pages, desc="Rendering Pages"):
                    # Use render() for clean images
                    img = page.render(
                        resolution=resolution,
                        **kwargs,
                    )
                    if img:
                        page_images.append(img)
                    else:
                        logger.warning(f"Failed to render page {page.number}, skipping.")
                if not page_images:
                    logger.error("Failed to render any pages.")
                    return None
                return page_images
            except Exception as e:
                logger.error(f"Error rendering pages: {e}")
                return None
        else:
            logger.error(f"Unsupported value for 'using': {using}")
            return None

    # --- End Extraction Support --- #

    def _gather_analysis_data(
        self,
        analysis_keys: List[str],
        include_content: bool,
        include_images: bool,
        image_dir: Optional[Path],
        image_format: str,
        image_resolution: int,
    ) -> List[Dict[str, Any]]:
        """
        Gather analysis data from all pages in the PDF.

        Args:
            analysis_keys: Keys in the analyses dictionary to export
            include_content: Whether to include extracted text
            include_images: Whether to export images
            image_dir: Directory to save images
            image_format: Format to save images
            image_resolution: Resolution for exported images

        Returns:
            List of dictionaries containing analysis data
        """
        if not hasattr(self, "_pages") or not self._pages:
            logger.warning(f"No pages found in PDF {self.path}")
            return []

        all_data = []
        image_dir_path = Path(image_dir) if image_dir is not None else None
        if include_images and image_dir_path is None:
            raise ValueError("'image_dir' must be provided when include_images=True.")

        for page in tqdm(self._pages, desc="Gathering page data", leave=False):
            # Basic page information
            page_data = {
                "pdf_path": self.path,
                "page_number": page.number,
                "page_index": page.index,
            }

            # Include extracted text if requested
            if include_content:
                try:
                    page_data["content"] = page.extract_text(preserve_whitespace=True)
                except Exception as e:
                    logger.error(f"Error extracting text from page {page.number}: {e}")
                    page_data["content"] = ""

            # Save image if requested
            if include_images and image_dir_path is not None:
                try:
                    # Create image filename
                    source_stem = Path(self.path).stem if isinstance(self.path, str) else "pdf"
                    image_filename = f"pdf_{source_stem}_page_{page.number}.{image_format}"
                    image_path = image_dir_path / image_filename

                    # Save image
                    page.save_image(
                        str(image_path), resolution=image_resolution, include_highlights=True
                    )

                    # Add relative path to data
                    page_data["image_path"] = str(
                        Path(image_path).relative_to(image_dir_path.parent)
                    )
                except Exception as e:
                    logger.error(f"Error saving image for page {page.number}: {e}")
                    page_data["image_path"] = None

            # Add analyses data
            for key in analysis_keys:
                if not hasattr(page, "analyses") or not page.analyses:
                    raise ValueError(f"Page {page.number} does not have analyses data")

                if key not in page.analyses:
                    raise KeyError(f"Analysis key '{key}' not found in page {page.number}")

                # Get the analysis result
                analysis_result = page.analyses[key]

                # If the result has a to_dict method, use it
                if hasattr(analysis_result, "to_dict"):
                    analysis_data = analysis_result.to_dict()
                else:
                    # Otherwise, use the result directly if it's dict-like
                    try:
                        analysis_data = dict(analysis_result)
                    except (TypeError, ValueError):
                        # Last resort: convert to string
                        analysis_data = {"raw_result": str(analysis_result)}

                # Add analysis data to page data with the key as prefix
                for k, v in analysis_data.items():
                    page_data[f"{key}.{k}"] = v

            all_data.append(page_data)

        return all_data

    def _get_target_pages(
        self, pages: Optional[Union[int, Iterable[int], range, slice]] = None
    ) -> List["Page"]:
        """
        Helper method to get a list of Page objects based on the input pages.

        Args:
            pages: Page index/int, iterable of indices, slice, or None for all pages

        Returns:
            List of Page objects
        """
        if pages is None:
            return list(self._pages)
        if isinstance(pages, int):
            total = len(self._pages)
            idx = pages if pages >= 0 else pages + total
            if not (0 <= idx < total):
                raise IndexError(f"Page index {pages} out of range (0-{total-1}).")
            return [cast("Page", self._pages[idx])]
        if isinstance(pages, slice):
            return list(self._pages[pages])
        if isinstance(pages, range):
            indices = list(pages)
        elif isinstance(pages, Iterable):
            try:
                indices = [int(i) for i in pages]
            except (TypeError, ValueError) as exc:
                raise TypeError("'pages' iterable must contain integer indices.") from exc
        else:
            raise TypeError(
                "'pages' must be None, an int, a slice, or an iterable of page indices."
            )

        resolved: List["Page"] = []
        total = len(self._pages)
        for idx in indices:
            actual_idx = idx if idx >= 0 else idx + total
            if not (0 <= actual_idx < total):
                raise IndexError(f"Page index {idx} out of range (0-{total-1}).")
            resolved.append(cast("Page", self._pages[actual_idx]))
        return resolved

    # --- Classification Mixin Implementation --- #

    def _get_classification_content(self, model_type: str, **kwargs) -> Union[str, Image.Image]:
        """
        Provides the content for classifying the entire PDF.

        Args:
            model_type: 'text' or 'vision'.
            **kwargs: Additional arguments (e.g., for text extraction or image rendering).

        Returns:
            Extracted text (str) or the first page's image (PIL.Image).

        Raises:
            ValueError: If model_type is 'vision' and PDF has != 1 page,
                      or if model_type is unsupported, or if content cannot be generated.
        """
        if model_type == "text":
            try:
                # Extract text from the whole document
                text = self.extract_text(**kwargs)  # Pass relevant kwargs
                if not text or text.isspace():
                    raise ValueError("PDF contains no extractable text for classification.")
                return text
            except Exception as e:
                logger.error(f"Error extracting text for PDF classification: {e}")
                raise ValueError("Failed to extract text for classification.") from e

        elif model_type == "vision":
            if len(self.pages) == 1:
                # Use the single page's content method
                try:
                    single_page = cast("Page", self.pages[0])
                    return single_page._get_classification_content(model_type="vision", **kwargs)
                except Exception as e:
                    logger.error(f"Error getting image from single page for classification: {e}")
                    raise ValueError("Failed to get image from single page.") from e
            elif len(self.pages) == 0:
                raise ValueError("Cannot classify empty PDF using vision model.")
            else:
                raise ValueError(
                    f"Vision classification for a PDF object is only supported for single-page PDFs. "
                    f"This PDF has {len(self.pages)} pages. Use pdf.pages[0].classify() or pdf.classify_pages()."
                )
        else:
            raise ValueError(f"Unsupported model_type for PDF classification: {model_type}")

    # --- End Classification Mixin Implementation ---

    # ------------------------------------------------------------------
    # Unified analysis storage (maps to metadata["analysis"])
    # ------------------------------------------------------------------

    @property
    def analyses(self) -> Dict[str, Any]:
        plumber_pdf = getattr(self, "_pdf", None)
        if plumber_pdf is None:
            raise RuntimeError("Underlying pdfplumber PDF is not initialized.")

        metadata = getattr(plumber_pdf, "metadata", None)
        if metadata is None:
            metadata = {}
            plumber_pdf.metadata = metadata

        analysis_entry = metadata.setdefault("analysis", {})
        if not isinstance(analysis_entry, dict):
            raise TypeError("PDF metadata 'analysis' entry must be a dictionary")

        return cast(Dict[str, Any], analysis_entry)

    @analyses.setter
    def analyses(self, value: Dict[str, Any]):
        plumber_pdf = getattr(self, "_pdf", None)
        if plumber_pdf is None:
            raise RuntimeError("Underlying pdfplumber PDF is not initialized.")

        metadata = getattr(plumber_pdf, "metadata", None)
        if metadata is None:
            metadata = {}
            plumber_pdf.metadata = metadata
        metadata["analysis"] = value

    # Static helper for weakref.finalize to avoid capturing 'self'
    @staticmethod
    def _finalize_cleanup(plumber_pdf, temp_file_obj, is_stream):
        path: Optional[str] = None
        try:
            if plumber_pdf is not None:
                plumber_pdf.close()
        except Exception:
            pass

        if temp_file_obj and not is_stream:
            try:
                path = temp_file_obj.name if hasattr(temp_file_obj, "name") else None
                if path and os.path.exists(path):
                    os.unlink(path)
            except Exception as e:
                logger.warning(f"Failed to clean up temporary file '{path}': {e}")

    def analyze_layout(self, *args, **kwargs) -> "ElementCollection[Region]":
        """
        Analyzes the layout of all pages in the PDF.

        This is a convenience method that calls analyze_layout on the PDF's
        page collection.

        Args:
            *args: Positional arguments passed to pages.analyze_layout().
            **kwargs: Keyword arguments passed to pages.analyze_layout().

        Returns:
            An ElementCollection of all detected Region objects.
        """
        return self.pages.analyze_layout(*args, **kwargs)

    def detect_checkboxes(self, *args, **kwargs) -> "ElementCollection[Region]":
        """
        Detects checkboxes on all pages in the PDF.

        This is a convenience method that calls detect_checkboxes on the PDF's
        page collection.

        Args:
            *args: Positional arguments passed to pages.detect_checkboxes().
            **kwargs: Keyword arguments passed to pages.detect_checkboxes().

        Returns:
            An ElementCollection of all detected checkbox Region objects.
        """
        return self.pages.detect_checkboxes(*args, **kwargs)

    def highlights(self, show: bool = False) -> "HighlightContext":
        """
        Create a highlight context for accumulating highlights.

        This allows for clean syntax to show multiple highlight groups:

        Example:
            with pdf.highlights() as h:
                h.add(pdf.find_all('table'), label='tables', color='blue')
                h.add(pdf.find_all('text:bold'), label='bold text', color='red')
                h.show()

        Or with automatic display:
            with pdf.highlights(show=True) as h:
                h.add(pdf.find_all('table'), label='tables')
                h.add(pdf.find_all('text:bold'), label='bold')
                # Automatically shows when exiting the context

        Args:
            show: If True, automatically show highlights when exiting context

        Returns:
            HighlightContext for accumulating highlights
        """
        from natural_pdf.core.highlighting_service import HighlightContext

        return HighlightContext(self, show_on_exit=show)

Attributes

natural_pdf.PDF.metadata property

Access PDF metadata as a dictionary.

Returns document metadata such as title, author, creation date, and other properties embedded in the PDF file. The exact keys available depend on what metadata was included when the PDF was created.

Returns:

Type	Description
Dict[str, Any]	Dictionary containing PDF metadata. Common keys include 'Title',
Dict[str, Any]	'Author', 'Subject', 'Creator', 'Producer', 'CreationDate', and
Dict[str, Any]	'ModDate'. May be empty if no metadata is available.

Example

pdf = npdf.PDF("document.pdf")
print(pdf.metadata.get('Title', 'No title'))
print(f"Created: {pdf.metadata.get('CreationDate')}")

natural_pdf.PDF.pages property

Access pages as a PageCollection object.

Provides access to individual pages of the PDF document through a collection interface that supports indexing, slicing, and iteration. Pages are lazy-loaded to minimize memory usage.

Returns:

Type	Description
PageCollection	PageCollection object that provides list-like access to PDF pages.

Raises:

Type	Description
AttributeError	If PDF pages are not yet initialized.

Example

pdf = npdf.PDF("document.pdf")

# Access individual pages
first_page = pdf.pages[0]
last_page = pdf.pages[-1]

# Slice pages
first_three = pdf.pages[0:3]

# Iterate over pages
for page in pdf.pages:
    print(f"Page {page.index} has {len(page.chars)} characters")

Functions

natural_pdf.PDF.__enter__()

Context manager entry.

Source code in natural_pdf/core/pdf.py

def __enter__(self):
    """Context manager entry."""
    return self

natural_pdf.PDF.__exit__(exc_type, exc_val, exc_tb)

Context manager exit.

Source code in natural_pdf/core/pdf.py

def __exit__(self, exc_type, exc_val, exc_tb):
    """Context manager exit."""
    self.close()

natural_pdf.PDF.__getitem__(key)

Access pages by index or slice.

Source code in natural_pdf/core/pdf.py

def __getitem__(self, key) -> Union["Page", "PageCollection"]:
    """Access pages by index or slice."""
    if not hasattr(self, "_pages"):
        raise AttributeError("PDF pages not initialized yet.")

    if isinstance(key, slice):
        from natural_pdf.core.page_collection import PageCollection

        # Use the lazy page list's slicing which returns another _LazyPageList
        lazy_slice = self._pages[key]
        # Wrap in PageCollection for compatibility
        return PageCollection(lazy_slice)
    elif isinstance(key, int):
        if 0 <= key < len(self._pages):
            return self._pages[key]
        else:
            raise IndexError(f"Page index {key} out of range (0-{len(self._pages)-1}).")
    else:
        raise TypeError(f"Page indices must be integers or slices, not {type(key)}.")

natural_pdf.PDF.__init__(path_or_url_or_stream, reading_order=True, font_attrs=None, keep_spaces=True, text_tolerance=None, auto_text_tolerance=True, text_layer=True)

Initialize the enhanced PDF object.

Parameters:

Name	Type	Description	Default
path_or_url_or_stream		Path to the PDF file (str/Path), a URL (str), or a file-like object (stream). URLs must start with 'http://' or 'https://'.	required
reading_order	bool	If True, use natural reading order for text extraction. Defaults to True.	True
font_attrs	Optional[List[str]]	List of font attributes for grouping characters into words. Common attributes include ['fontname', 'size']. Defaults to None.	None
keep_spaces	bool	If True, include spaces in word elements during text extraction. Defaults to True.	True
text_tolerance	Optional[dict]	PDFplumber-style tolerance settings for text grouping. Dictionary with keys like 'x_tolerance', 'y_tolerance'. Defaults to None.	None
auto_text_tolerance	bool	If True, automatically scale text tolerance based on font size and document characteristics. Defaults to True.	True
text_layer	bool	If True, preserve existing text layer from the PDF. If False, removes all existing text elements during initialization, useful for OCR-only workflows. Defaults to True.	True

Raises:

Type	Description
TypeError	If path_or_url_or_stream is not a valid type.
IOError	If the PDF file cannot be opened or read.
ValueError	If URL download fails.

Example

# From file path
pdf = npdf.PDF("document.pdf")

# From URL
pdf = npdf.PDF("https://example.com/document.pdf")

# From stream
with open("document.pdf", "rb") as f:
    pdf = npdf.PDF(f)

# With custom settings
pdf = npdf.PDF("document.pdf",
              reading_order=False,
              text_layer=False,  # For OCR-only processing
              font_attrs=['fontname', 'size', 'flags'])

Source code in natural_pdf/core/pdf.py

def __init__(
    self,
    path_or_url_or_stream,
    reading_order: bool = True,
    font_attrs: Optional[List[str]] = None,
    keep_spaces: bool = True,
    text_tolerance: Optional[dict] = None,
    auto_text_tolerance: bool = True,
    text_layer: bool = True,
):
    """Initialize the enhanced PDF object.

    Args:
        path_or_url_or_stream: Path to the PDF file (str/Path), a URL (str),
            or a file-like object (stream). URLs must start with 'http://' or 'https://'.
        reading_order: If True, use natural reading order for text extraction.
            Defaults to True.
        font_attrs: List of font attributes for grouping characters into words.
            Common attributes include ['fontname', 'size']. Defaults to None.
        keep_spaces: If True, include spaces in word elements during text extraction.
            Defaults to True.
        text_tolerance: PDFplumber-style tolerance settings for text grouping.
            Dictionary with keys like 'x_tolerance', 'y_tolerance'. Defaults to None.
        auto_text_tolerance: If True, automatically scale text tolerance based on
            font size and document characteristics. Defaults to True.
        text_layer: If True, preserve existing text layer from the PDF. If False,
            removes all existing text elements during initialization, useful for
            OCR-only workflows. Defaults to True.

    Raises:
        TypeError: If path_or_url_or_stream is not a valid type.
        IOError: If the PDF file cannot be opened or read.
        ValueError: If URL download fails.

    Example:
        ```python
        # From file path
        pdf = npdf.PDF("document.pdf")

        # From URL
        pdf = npdf.PDF("https://example.com/document.pdf")

        # From stream
        with open("document.pdf", "rb") as f:
            pdf = npdf.PDF(f)

        # With custom settings
        pdf = npdf.PDF("document.pdf",
                      reading_order=False,
                      text_layer=False,  # For OCR-only processing
                      font_attrs=['fontname', 'size', 'flags'])
        ```
    """
    self._original_path_or_stream = path_or_url_or_stream
    self._temp_file = None
    self._resolved_path = None
    self._is_stream = False
    self._text_layer = text_layer
    stream_to_open = None

    if hasattr(path_or_url_or_stream, "read"):  # Check if it's file-like
        logger.info("Initializing PDF from in-memory stream.")
        self._is_stream = True
        self._resolved_path = None  # No resolved file path for streams
        self.source_path = "<stream>"  # Identifier for source
        self.path = self.source_path  # Use source identifier as path for streams
        stream_to_open = path_or_url_or_stream
        try:
            if hasattr(path_or_url_or_stream, "read"):
                # If caller provided an in-memory binary stream, capture bytes for potential re-export
                current_pos = path_or_url_or_stream.tell()
                path_or_url_or_stream.seek(0)
                self._original_bytes = path_or_url_or_stream.read()
                path_or_url_or_stream.seek(current_pos)
        except Exception:
            pass
    elif isinstance(path_or_url_or_stream, (str, Path)):
        path_or_url = str(path_or_url_or_stream)
        self.source_path = path_or_url  # Store original path/URL as source
        is_url = path_or_url.startswith("http://") or path_or_url.startswith("https://")

        if is_url:
            logger.info(f"Downloading PDF from URL: {path_or_url}")
            try:
                ssl_context = None
                try:
                    if certifi is not None:
                        ssl_context = ssl.create_default_context(cafile=certifi.where())
                    else:
                        ssl_context = ssl.create_default_context()
                except Exception:
                    ssl_context = ssl.create_default_context()

                with urllib.request.urlopen(path_or_url, context=ssl_context) as response:
                    data = response.read()
                # Load directly into an in-memory buffer — no temp file needed
                buffer = io.BytesIO(data)
                buffer.seek(0)
                self._temp_file = None  # No on-disk temp file
                self._resolved_path = path_or_url  # For repr / get_id purposes
                stream_to_open = buffer  # pdfplumber accepts file-like objects
            except Exception as e:
                logger.error(f"Failed to download PDF from URL: {e}")
                raise ValueError(f"Failed to download PDF from URL: {e}")
        else:
            self._resolved_path = str(Path(path_or_url).resolve())  # Resolve local paths
            stream_to_open = self._resolved_path
        self.path = self._resolved_path  # Use resolved path for file-based PDFs
    else:
        raise TypeError(
            f"Invalid input type: {type(path_or_url_or_stream)}. "
            f"Expected path (str/Path), URL (str), or file-like object."
        )

    logger.info(f"Opening PDF source: {self.source_path}")
    logger.debug(
        f"Parameters: reading_order={reading_order}, font_attrs={font_attrs}, keep_spaces={keep_spaces}"
    )

    try:
        self._pdf = pdfplumber.open(stream_to_open)
    except Exception as e:
        logger.error(f"Failed to open PDF: {e}", exc_info=True)
        self.close()  # Attempt cleanup if opening fails
        raise IOError(f"Failed to open PDF source: {self.source_path}") from e

    # Store configuration used for initialization
    self._reading_order = reading_order
    self._config = {"keep_spaces": keep_spaces}
    self._font_attrs = font_attrs

    # Deprecated managers remain for backwards compatibility but are no longer instantiated.
    self._layout_manager = None

    self.highlighter: HighlightingService = HighlightingService(self)
    self._manager_factories: ManagerFactories = {}
    self._managers: ManagerCache = {}

    # Lazily instantiate pages only when accessed
    self._pages: _LazyPageList = _LazyPageList(
        self, self._pdf, font_attrs=font_attrs, load_text=self._text_layer
    )

    self._element_cache: Dict[str, Any] = {}
    self._exclusions: List[ExclusionSpec] = []
    self._regions: RegionRegistry = []
    self._from_images: bool = False
    self._source_metadata: Optional[Dict[str, Any]] = None

    logger.info(f"PDF '{self.source_path}' initialized with {len(self._pages)} pages.")

    self._initialize_managers()
    self._initialize_highlighter()

    # Remove text layer if requested
    if not self._text_layer:
        logger.info("Removing text layer as requested (text_layer=False)")
        # Text layer is not loaded when text_layer=False, so no need to remove
        pass

    # Analysis results accessed via self.analyses property (see below)

    # --- Automatic cleanup when object is garbage-collected ---
    self._finalizer = weakref.finalize(
        self,
        PDF._finalize_cleanup,
        self._pdf,
        getattr(self, "_temp_file", None),
        getattr(self, "_is_stream", False),
    )

    # --- Text tolerance settings ------------------------------------
    # Users can pass pdfplumber-style keys (x_tolerance, x_tolerance_ratio,
    # y_tolerance, etc.) via *text_tolerance*.  We also keep a flag that
    # enables automatic tolerance scaling when explicit values are not
    # supplied.
    self._config["auto_text_tolerance"] = bool(auto_text_tolerance)
    if text_tolerance:
        # Only copy recognised primitives (numbers / None); ignore junk.
        allowed = {
            "x_tolerance",
            "x_tolerance_ratio",
            "y_tolerance",
            "y_tolerance_ratio",
            "keep_blank_chars",  # passthrough convenience
        }
        for k, v in text_tolerance.items():
            if k in allowed:
                self._config[k] = v

natural_pdf.PDF.__len__()

Return the number of pages in the PDF.

Source code in natural_pdf/core/pdf.py

def __len__(self) -> int:
    """Return the number of pages in the PDF."""
    if not hasattr(self, "_pages"):
        return 0
    return len(self._pages)

natural_pdf.PDF.__repr__()

Return a string representation of the PDF object.

Source code in natural_pdf/core/pdf.py

def __repr__(self) -> str:
    """Return a string representation of the PDF object."""
    if not hasattr(self, "_pages"):
        page_count_str = "uninitialized"
    else:
        page_count_str = str(len(self._pages))

    source_info = getattr(self, "source_path", "unknown source")
    return f"<PDF source='{source_info}' pages={page_count_str}>"

natural_pdf.PDF.add_exclusion(exclusion_func, label=None)

Add an exclusion function to the PDF.

Exclusion functions define regions of each page that should be ignored during text extraction and analysis operations. This is useful for filtering out headers, footers, watermarks, or other administrative content that shouldn't be included in the main document processing.

Parameters:

Name	Type	Description	Default
exclusion_func		A function that takes a Page object and returns a Region to exclude from processing, or None if no exclusion should be applied to that page. The function is called once per page.	required
label	Optional[str]	Optional descriptive label for this exclusion rule, useful for debugging and identification.	None

Returns:

Type	Description
PDF	Self for method chaining.

Raises:

Type	Description
AttributeError	If PDF pages are not yet initialized.

Example

pdf = npdf.PDF("document.pdf")

# Exclude headers (top 50 points of each page)
pdf.add_exclusion(
    lambda page: page.region(0, 0, page.width, 50),
    label="header_exclusion"
)

# Exclude any text containing "CONFIDENTIAL"
pdf.add_exclusion(
    lambda page: page.find('text:contains("CONFIDENTIAL")').above(include_source=True)
    if page.find('text:contains("CONFIDENTIAL")') else None,
    label="confidential_exclusion"
)

# Chain multiple exclusions
pdf.add_exclusion(header_func).add_exclusion(footer_func)

Source code in natural_pdf/core/pdf.py

def add_exclusion(self, exclusion_func, label: Optional[str] = None) -> "PDF":
    """Add an exclusion function to the PDF.

    Exclusion functions define regions of each page that should be ignored during
    text extraction and analysis operations. This is useful for filtering out headers,
    footers, watermarks, or other administrative content that shouldn't be included
    in the main document processing.

    Args:
        exclusion_func: A function that takes a Page object and returns a Region
            to exclude from processing, or None if no exclusion should be applied
            to that page. The function is called once per page.
        label: Optional descriptive label for this exclusion rule, useful for
            debugging and identification.

    Returns:
        Self for method chaining.

    Raises:
        AttributeError: If PDF pages are not yet initialized.

    Example:
        ```python
        pdf = npdf.PDF("document.pdf")

        # Exclude headers (top 50 points of each page)
        pdf.add_exclusion(
            lambda page: page.region(0, 0, page.width, 50),
            label="header_exclusion"
        )

        # Exclude any text containing "CONFIDENTIAL"
        pdf.add_exclusion(
            lambda page: page.find('text:contains("CONFIDENTIAL")').above(include_source=True)
            if page.find('text:contains("CONFIDENTIAL")') else None,
            label="confidential_exclusion"
        )

        # Chain multiple exclusions
        pdf.add_exclusion(header_func).add_exclusion(footer_func)
        ```
    """
    if not hasattr(self, "_pages"):
        raise AttributeError("PDF pages not yet initialized.")

    # ------------------------------------------------------------------
    # Support selector strings and ElementCollection objects directly.
    # Store exclusion and apply only to already-created pages.
    # ------------------------------------------------------------------
    from natural_pdf.elements.element_collection import ElementCollection  # local import

    if isinstance(exclusion_func, str) or isinstance(exclusion_func, ElementCollection):
        # Store for bookkeeping and lazy application
        self._exclusions.append((exclusion_func, label))

        # Don't modify already-cached pages - they will get PDF-level exclusions
        # dynamically through _get_exclusion_regions()
        return self

    # Fallback to original callable / Region behaviour ------------------
    exclusion_data = (exclusion_func, label)
    self._exclusions.append(exclusion_data)

    # Don't modify already-cached pages - they will get PDF-level exclusions
    # dynamically through _get_exclusion_regions()

    return self

natural_pdf.PDF.add_region(region_func, name=None)

Add a region function to the PDF.

Parameters:

Name	Type	Description	Default
region_func	Callable[[Page], Optional[Region]]	A function that takes a Page and returns a Region, or None	required
name	Optional[str]	Optional name for the region	None

Returns:

Type	Description
PDF	Self for method chaining

Source code in natural_pdf/core/pdf.py

def add_region(
    self, region_func: Callable[["Page"], Optional["Region"]], name: Optional[str] = None
) -> "PDF":
    """
    Add a region function to the PDF.

    Args:
        region_func: A function that takes a Page and returns a Region, or None
        name: Optional name for the region

    Returns:
        Self for method chaining
    """
    if not hasattr(self, "_pages"):
        raise AttributeError("PDF pages not yet initialized.")

    region_data = (region_func, name)
    self._regions.append(region_data)

    # Apply only to already-created (cached) pages to avoid forcing page creation
    for i in range(len(self._pages)):
        cached_page = self._pages._cache[i]
        if cached_page is None:
            continue
        try:
            region_instance = region_func(cached_page)
            if region_instance and isinstance(region_instance, Region):
                cached_page.add_region(region_instance, name=name, source="named")
            elif region_instance is not None:
                logger.warning(
                    f"Region function did not return a valid Region for page {cached_page.number}"
                )
        except Exception as e:
            logger.error(f"Error adding region for page {cached_page.number}: {e}")

    return self

natural_pdf.PDF.analyze_layout(*args, **kwargs)

Analyzes the layout of all pages in the PDF.

This is a convenience method that calls analyze_layout on the PDF's page collection.

Parameters:

Name	Type	Description	Default
*args		Positional arguments passed to pages.analyze_layout().	()
**kwargs		Keyword arguments passed to pages.analyze_layout().	{}

Returns:

Type	Description
ElementCollection[Region]	An ElementCollection of all detected Region objects.

Source code in natural_pdf/core/pdf.py

def analyze_layout(self, *args, **kwargs) -> "ElementCollection[Region]":
    """
    Analyzes the layout of all pages in the PDF.

    This is a convenience method that calls analyze_layout on the PDF's
    page collection.

    Args:
        *args: Positional arguments passed to pages.analyze_layout().
        **kwargs: Keyword arguments passed to pages.analyze_layout().

    Returns:
        An ElementCollection of all detected Region objects.
    """
    return self.pages.analyze_layout(*args, **kwargs)

natural_pdf.PDF.apply_ocr(engine=None, languages=None, min_confidence=None, device=None, resolution=None, apply_exclusions=True, detect_only=False, replace=True, options=None, pages=None)

Apply OCR to specified pages of the PDF using batch processing.

Performs optical character recognition on the specified pages, converting image-based text into searchable and extractable text elements. This method supports multiple OCR engines and provides batch processing for efficiency.

Parameters:

Name	Type	Description	Default
engine	Optional[str]	Name of the OCR engine to use. Supported engines include 'easyocr' (default), 'surya', 'paddle', and 'doctr'. If None, uses the global default from natural_pdf.options.ocr.engine.	None
languages	Optional[List[str]]	List of language codes for OCR recognition (e.g., ['en', 'es']). If None, uses the global default from natural_pdf.options.ocr.languages.	None
min_confidence	Optional[float]	Minimum confidence threshold (0.0-1.0) for accepting OCR results. Text with lower confidence will be filtered out. If None, uses the global default.	None
device	Optional[str]	Device to run OCR on ('cpu', 'cuda', 'mps'). Engine-specific availability varies. If None, uses engine defaults.	None
resolution	Optional[int]	DPI resolution for rendering pages to images before OCR. Higher values improve accuracy but increase processing time and memory. Typical values: 150 (fast), 300 (balanced), 600 (high quality).	None
apply_exclusions	bool	If True, mask excluded regions before OCR to prevent processing of headers, footers, or other unwanted content.	True
detect_only	bool	If True, only detect text bounding boxes without performing character recognition. Useful for layout analysis workflows.	False
replace	bool	If True, replace any existing OCR elements on the pages. If False, append new OCR results to existing elements.	True
options	Optional[Any]	Engine-specific options object (e.g., EasyOCROptions, SuryaOptions). Allows fine-tuning of engine behavior beyond common parameters.	None
pages	Optional[Union[Iterable[int], range, slice]]	Page indices to process. Can be: - None: Process all pages - slice: Process a range of pages (e.g., slice(0, 10)) - Iterable[int]: Process specific page indices (e.g., [0, 2, 5])	None

Returns:

Type	Description
PDF	Self for method chaining.

Raises:

Type	Description
ValueError	If invalid page index is provided.
TypeError	If pages parameter has invalid type.
RuntimeError	If OCR engine is not available or fails.

Example

pdf = npdf.PDF("scanned_document.pdf")

# Basic OCR on all pages
pdf.apply_ocr()

# High-quality OCR with specific settings
pdf.apply_ocr(
    engine='easyocr',
    languages=['en', 'es'],
    resolution=300,
    min_confidence=0.8
)

# OCR specific pages only
pdf.apply_ocr(pages=[0, 1, 2])  # First 3 pages
pdf.apply_ocr(pages=slice(5, 10))  # Pages 5-9

# Detection-only workflow for layout analysis
pdf.apply_ocr(detect_only=True, resolution=150)

Note

OCR processing can be time and memory intensive, especially at high resolutions. Consider using exclusions to mask unwanted regions and processing pages in batches for large documents.

Source code in natural_pdf/core/pdf.py

def apply_ocr(
    self,
    engine: Optional[str] = None,
    languages: Optional[List[str]] = None,
    min_confidence: Optional[float] = None,
    device: Optional[str] = None,
    resolution: Optional[int] = None,
    apply_exclusions: bool = True,
    detect_only: bool = False,
    replace: bool = True,
    options: Optional[Any] = None,
    pages: Optional[Union[Iterable[int], range, slice]] = None,
) -> "PDF":
    """Apply OCR to specified pages of the PDF using batch processing.

    Performs optical character recognition on the specified pages, converting
    image-based text into searchable and extractable text elements. This method
    supports multiple OCR engines and provides batch processing for efficiency.

    Args:
        engine: Name of the OCR engine to use. Supported engines include
            'easyocr' (default), 'surya', 'paddle', and 'doctr'. If None,
            uses the global default from natural_pdf.options.ocr.engine.
        languages: List of language codes for OCR recognition (e.g., ['en', 'es']).
            If None, uses the global default from natural_pdf.options.ocr.languages.
        min_confidence: Minimum confidence threshold (0.0-1.0) for accepting
            OCR results. Text with lower confidence will be filtered out.
            If None, uses the global default.
        device: Device to run OCR on ('cpu', 'cuda', 'mps'). Engine-specific
            availability varies. If None, uses engine defaults.
        resolution: DPI resolution for rendering pages to images before OCR.
            Higher values improve accuracy but increase processing time and memory.
            Typical values: 150 (fast), 300 (balanced), 600 (high quality).
        apply_exclusions: If True, mask excluded regions before OCR to prevent
            processing of headers, footers, or other unwanted content.
        detect_only: If True, only detect text bounding boxes without performing
            character recognition. Useful for layout analysis workflows.
        replace: If True, replace any existing OCR elements on the pages.
            If False, append new OCR results to existing elements.
        options: Engine-specific options object (e.g., EasyOCROptions, SuryaOptions).
            Allows fine-tuning of engine behavior beyond common parameters.
        pages: Page indices to process. Can be:
            - None: Process all pages
            - slice: Process a range of pages (e.g., slice(0, 10))
            - Iterable[int]: Process specific page indices (e.g., [0, 2, 5])

    Returns:
        Self for method chaining.

    Raises:
        ValueError: If invalid page index is provided.
        TypeError: If pages parameter has invalid type.
        RuntimeError: If OCR engine is not available or fails.

    Example:
        ```python
        pdf = npdf.PDF("scanned_document.pdf")

        # Basic OCR on all pages
        pdf.apply_ocr()

        # High-quality OCR with specific settings
        pdf.apply_ocr(
            engine='easyocr',
            languages=['en', 'es'],
            resolution=300,
            min_confidence=0.8
        )

        # OCR specific pages only
        pdf.apply_ocr(pages=[0, 1, 2])  # First 3 pages
        pdf.apply_ocr(pages=slice(5, 10))  # Pages 5-9

        # Detection-only workflow for layout analysis
        pdf.apply_ocr(detect_only=True, resolution=150)
        ```

    Note:
        OCR processing can be time and memory intensive, especially at high
        resolutions. Consider using exclusions to mask unwanted regions and
        processing pages in batches for large documents.
    """
    normalized_options = normalize_ocr_options(options)
    engine_name = resolve_ocr_engine_name(
        context=self,
        requested=engine,
        options=normalized_options,
        scope="pdf",
    )
    resolved_languages = resolve_ocr_languages(self, languages, scope="pdf")
    resolved_min_confidence = resolve_ocr_min_confidence(self, min_confidence, scope="pdf")
    resolved_device = resolve_ocr_device(self, device, scope="pdf")

    target_pages = self._get_target_pages(pages)
    if not target_pages:
        logger.warning("No pages selected for OCR processing.")
        return self

    final_resolution = resolution or self._config.get("resolution", 150)
    logger.info(
        "Applying OCR to %d page(s) with engine '%s' at %s DPI.",
        len(target_pages),
        engine_name,
        final_resolution,
    )

    for page in tqdm(target_pages, desc="Applying OCR", leave=False):
        page.apply_ocr(
            engine=engine_name,
            options=normalized_options,
            languages=resolved_languages,
            min_confidence=resolved_min_confidence,
            device=resolved_device,
            resolution=final_resolution,
            detect_only=detect_only,
            apply_exclusions=apply_exclusions,
            replace=replace,
        )

    return self

natural_pdf.PDF.ask(question, *, mode='extractive', pages=None, min_confidence=0.1, model=None, temperature=None, top_p=None, llm_client=None, prompt=None)

Ask a single question about the document content.

Parameters:

Name	Type	Description	Default
question	str	Question string to ask about the document	required
mode	Literal['extractive', 'generative']	"extractive" to extract answer from document, "generative" to generate	'extractive'
pages	Optional[Union[int, Iterable[int], range]]	Specific pages to query (default: all pages)	None
min_confidence	float	Minimum confidence threshold for answers (extractive mode).	0.1
model	Optional[str]	Optional model name for the QA engine or LLM.	None
temperature	Optional[float]	Optional sampling temperature for LLM-backed QA.	None
top_p	Optional[float]	Optional nucleus sampling parameter for LLM-backed QA.	None
llm_client	Optional[Any]	Client instance to use when mode="generative".	None
prompt	Optional[str]	Optional system prompt override for generative QA.	None

Returns:

Type	Description
QAResult	class:QAResult containing answer metadata. Confidence may be None in generative
QAResult	mode; source_elements may be empty if no span is available.

Source code in natural_pdf/core/pdf.py

def ask(
    self,
    question: str,
    *,
    mode: Literal["extractive", "generative"] = "extractive",
    pages: Optional[Union[int, Iterable[int], range]] = None,
    min_confidence: float = 0.1,
    model: Optional[str] = None,
    temperature: Optional[float] = None,
    top_p: Optional[float] = None,
    llm_client: Optional[Any] = None,
    prompt: Optional[str] = None,
) -> QAResult:
    """
    Ask a single question about the document content.

    Args:
        question: Question string to ask about the document
        mode: "extractive" to extract answer from document, "generative" to generate
        pages: Specific pages to query (default: all pages)
        min_confidence: Minimum confidence threshold for answers (extractive mode).
        model: Optional model name for the QA engine or LLM.
        temperature: Optional sampling temperature for LLM-backed QA.
        top_p: Optional nucleus sampling parameter for LLM-backed QA.
        llm_client: Client instance to use when ``mode="generative"``.
        prompt: Optional system prompt override for generative QA.

    Returns:
        :class:`QAResult` containing answer metadata. Confidence may be ``None`` in generative
        mode; ``source_elements`` may be empty if no span is available.
    """
    # Delegate to ask_batch and return the first result
    results = self.ask_batch(
        [question],
        mode=mode,
        pages=pages,
        min_confidence=min_confidence,
        model=model,
        temperature=temperature,
        top_p=top_p,
        llm_client=llm_client,
        prompt=prompt,
    )
    if results:
        return results[0]
    return QAResult(
        {
            "answer": None,
            "confidence": 0.0,
            "found": False,
            "page_num": None,
            "source_elements": [],
        }
    )

natural_pdf.PDF.ask_batch(questions, *, mode='extractive', pages=None, min_confidence=0.1, model=None, temperature=None, top_p=None, llm_client=None, prompt=None)

Ask multiple questions about the document content using batch processing.

This method processes multiple questions efficiently in a single batch, avoiding the multiprocessing resource accumulation that can occur with sequential individual question calls.

Parameters:

Name	Type	Description	Default
questions	List[str]	List of question strings to ask about the document	required
mode	Literal['extractive', 'generative']	"extractive" to extract answer from document, "generative" to generate	'extractive'
pages	Optional[Union[int, Iterable[int], range]]	Specific pages to query (default: all pages)	None
min_confidence	float	Minimum confidence threshold for extractive answers.	0.1
model	Optional[str]	Optional model name for the QA engine or LLM.	None
temperature	Optional[float]	Optional sampling temperature for LLM-backed QA.	None
top_p	Optional[float]	Optional nucleus sampling parameter for LLM-backed QA.	None
llm_client	Optional[Any]	Client instance to use when mode="generative".	None
prompt	Optional[str]	Optional system prompt override for generative QA.	None

Returns:

Type	Description
List[QAResult]	List of :class:QAResult objects containing answer metadata. Confidence may be
List[QAResult]	None in generative mode; source_elements may be empty if no span is available.

Source code in natural_pdf/core/pdf.py

def ask_batch(
    self,
    questions: List[str],
    *,
    mode: Literal["extractive", "generative"] = "extractive",
    pages: Optional[Union[int, Iterable[int], range]] = None,
    min_confidence: float = 0.1,
    model: Optional[str] = None,
    temperature: Optional[float] = None,
    top_p: Optional[float] = None,
    llm_client: Optional[Any] = None,
    prompt: Optional[str] = None,
) -> List[QAResult]:
    """
    Ask multiple questions about the document content using batch processing.

    This method processes multiple questions efficiently in a single batch,
    avoiding the multiprocessing resource accumulation that can occur with
    sequential individual question calls.

    Args:
        questions: List of question strings to ask about the document
        mode: "extractive" to extract answer from document, "generative" to generate
        pages: Specific pages to query (default: all pages)
        min_confidence: Minimum confidence threshold for extractive answers.
        model: Optional model name for the QA engine or LLM.
        temperature: Optional sampling temperature for LLM-backed QA.
        top_p: Optional nucleus sampling parameter for LLM-backed QA.
        llm_client: Client instance to use when ``mode="generative"``.
        prompt: Optional system prompt override for generative QA.

    Returns:
        List of :class:`QAResult` objects containing answer metadata. Confidence may be
        ``None`` in generative mode; ``source_elements`` may be empty if no span is available.
    """
    if not questions:
        return []

    if mode not in ("extractive", "generative"):
        raise ValueError("mode must be either 'extractive' or 'generative'")

    if not isinstance(questions, list) or not all(isinstance(q, str) for q in questions):
        raise TypeError("'questions' must be a list of strings")

    target_pages = self._resolve_qa_pages(pages)

    def _empty_result() -> QAResult:
        return QAResult(
            {
                "answer": None,
                "confidence": 0.0,
                "found": False,
                "page_num": None,
                "source_elements": [],
            }
        )

    if not target_pages:
        logger.warning("No valid pages found for QA processing.")
        return [_empty_result() for _ in questions]

    if mode == "generative":
        return self._ask_batch_generative(
            questions,
            target_pages=target_pages,
            llm_client=llm_client,
            prompt=prompt,
            model=model,
            temperature=temperature,
            top_p=top_p,
        )

    from natural_pdf.qa import get_qa_engine

    qa_engine = get_qa_engine() if model is None else get_qa_engine(model_name=model)

    if temperature is not None:
        set_temp = getattr(qa_engine, "set_temperature", None)
        if callable(set_temp):
            try:
                set_temp(temperature)
            except Exception:
                logger.debug("QA engine rejected temperature override; ignoring.")

    if top_p is not None:
        set_top_p = getattr(qa_engine, "set_top_p", None)
        if callable(set_top_p):
            try:
                set_top_p(top_p)
            except Exception:
                logger.debug("QA engine rejected top_p override; ignoring.")

    logger.info(
        f"Processing {len(questions)} question(s) across {len(target_pages)} page(s) using batch QA..."
    )

    # Collect all page images and metadata for batch processing
    page_images: List[Image.Image] = []
    page_word_boxes: List[Any] = []
    page_metadata: List[Dict[str, Any]] = []

    for page in target_pages:
        # Get page image
        try:
            # Use render() for clean image without highlights
            page_image = page.render(resolution=150)
            if page_image is None:
                logger.warning(f"Failed to render image for page {page.number}, skipping")
                continue

            # Get text elements for word boxes
            elements = page.find_all("text")
            if not elements:
                logger.warning(f"No text elements found on page {page.number}")
                word_boxes = []
            else:
                word_boxes = qa_engine._get_word_boxes_from_elements(
                    elements, offset_x=0, offset_y=0
                )

            page_images.append(page_image)
            page_word_boxes.append(word_boxes)
            page_metadata.append({"page_number": page.number, "page_object": page})

        except Exception as e:
            logger.warning(f"Error processing page {page.number}: {e}")
            continue

    if not page_images:
        logger.warning("No page images could be processed for QA.")
        return [_empty_result() for _ in questions]

    # Process all questions against all pages in batch
    all_results: List[QAResult] = []

    for question_text in questions:
        question_results: List[QAResult] = []

        # Ask this question against each page (but in batch per page)
        for i, (page_image, word_boxes, page_meta) in enumerate(
            zip(page_images, page_word_boxes, page_metadata)
        ):
            try:
                # Use the DocumentQA batch interface
                raw_result = qa_engine.ask(
                    image=page_image,
                    question=question_text,
                    word_boxes=word_boxes,
                    min_confidence=min_confidence,
                )

                page_result: Optional[QAResult]
                if isinstance(raw_result, QAResult):
                    page_result = raw_result
                elif isinstance(raw_result, list) and raw_result:
                    page_result = raw_result[0]
                else:
                    page_result = None

                if page_result and page_result.found:
                    qa_result = QAResult(
                        {
                            "answer": page_result.answer,
                            "confidence": page_result.confidence,
                            "found": page_result.found,
                            "page_num": page_meta["page_number"],
                            "source_elements": getattr(page_result, "source_elements", []),
                            "start": getattr(page_result, "start", -1),
                            "end": getattr(page_result, "end", -1),
                        }
                    )
                    question_results.append(qa_result)

            except Exception as e:
                logger.warning(
                    f"Error processing question '{question_text}' on page {page_meta['page_number']}: {e}"
                )
                continue

        # Sort results by confidence and take the best one for this question
        question_results.sort(key=lambda x: x.get("confidence", 0), reverse=True)

        if question_results:
            all_results.append(question_results[0])
        else:
            # No results found for this question
            all_results.append(_empty_result())

    return all_results

natural_pdf.PDF.classify_pages(labels, model=None, pages=None, analysis_key='classification', using=None, min_confidence=0.0, multi_label=False, **kwargs)

Classifies specified pages of the PDF.

Parameters:

Name	Type	Description	Default
labels	List[str]	List of category names	required
model	Optional[str]	Model identifier ('text', 'vision', or specific HF ID)	None
pages	Optional[Union[Iterable[int], range, slice]]	Page indices, slice, or None for all pages	None
analysis_key	str	Key to store results in page's analyses dict	'classification'
using	Optional[str]	Processing mode ('text' or 'vision')	None
**kwargs		Additional arguments for the ClassificationManager	{}

Returns:

Type	Description
PDF	Self for method chaining

Source code in natural_pdf/core/pdf.py

def classify_pages(
    self,
    labels: List[str],
    model: Optional[str] = None,
    pages: Optional[Union[Iterable[int], range, slice]] = None,
    analysis_key: str = "classification",
    using: Optional[str] = None,
    min_confidence: float = 0.0,
    multi_label: bool = False,
    **kwargs,
) -> "PDF":
    """
    Classifies specified pages of the PDF.

    Args:
        labels: List of category names
        model: Model identifier ('text', 'vision', or specific HF ID)
        pages: Page indices, slice, or None for all pages
        analysis_key: Key to store results in page's analyses dict
        using: Processing mode ('text' or 'vision')
        **kwargs: Additional arguments for the ClassificationManager

    Returns:
        Self for method chaining
    """
    if not labels:
        raise ValueError("Labels list cannot be empty.")

    target_pages = self._get_target_pages(pages)

    if not target_pages:
        logger.warning("No pages selected for classification.")
        return self

    engine_name = kwargs.pop("classification_engine", None)
    engine_obj = get_classification_engine(self, engine_name)
    inferred_using = engine_obj.infer_using(model or engine_obj.default_model("text"), using)
    logger.info(
        f"Classifying {len(target_pages)} pages using model '{model or '(default)'}' (mode: {inferred_using})"
    )

    page_contents = []
    pages_to_classify = []
    logger.debug(f"Gathering content for {len(target_pages)} pages...")

    for page in target_pages:
        try:
            content = page._get_classification_content(model_type=inferred_using, **kwargs)
            page_contents.append(content)
            pages_to_classify.append(page)
        except ValueError as e:
            logger.warning(f"Skipping page {page.number}: Cannot get content - {e}")
        except Exception as e:
            logger.warning(f"Skipping page {page.number}: Error getting content - {e}")

    if not page_contents:
        logger.warning("No content could be gathered for batch classification.")
        return self

    logger.debug(f"Gathered content for {len(pages_to_classify)} pages.")

    try:
        batch_results = run_classification_batch(
            context=self,
            contents=page_contents,
            labels=labels,
            model_id=model or engine_obj.default_model(inferred_using),
            using=inferred_using,
            min_confidence=min_confidence,
            multi_label=multi_label,
            batch_size=8,
            progress_bar=True,
            engine_name=engine_name,
            **kwargs,
        )
    except Exception as e:
        logger.error(f"Batch classification failed: {e}")
        raise ClassificationError(f"Batch classification failed: {e}") from e

    if len(batch_results) != len(pages_to_classify):
        logger.error(
            f"Mismatch between number of results ({len(batch_results)}) and pages ({len(pages_to_classify)})"
        )
        return self

    logger.debug(
        f"Distributing {len(batch_results)} results to pages under key '{analysis_key}'..."
    )
    for page, result_obj in zip(pages_to_classify, batch_results):
        try:
            if not hasattr(page, "analyses") or page.analyses is None:
                page.analyses = {}
            page.analyses[analysis_key] = result_obj
        except Exception as e:
            logger.warning(
                f"Failed to store classification results for page {page.number}: {e}"
            )

    logger.info("Finished classifying PDF pages.")
    return self

natural_pdf.PDF.clear_exclusions()

Clear all exclusion functions from the PDF.

Removes all previously added exclusion functions that were used to filter out unwanted content (like headers, footers, or administrative text) from text extraction and analysis operations.

Returns:

Type	Description
PDF	Self for method chaining.

Raises:

Type	Description
AttributeError	If PDF pages are not yet initialized.

Example

pdf = npdf.PDF("document.pdf")
pdf.add_exclusion(lambda page: page.find('text:contains("CONFIDENTIAL")').above())

# Later, remove all exclusions
pdf.clear_exclusions()

Source code in natural_pdf/core/pdf.py

def clear_exclusions(self) -> "PDF":
    """Clear all exclusion functions from the PDF.

    Removes all previously added exclusion functions that were used to filter
    out unwanted content (like headers, footers, or administrative text) from
    text extraction and analysis operations.

    Returns:
        Self for method chaining.

    Raises:
        AttributeError: If PDF pages are not yet initialized.

    Example:
        ```python
        pdf = npdf.PDF("document.pdf")
        pdf.add_exclusion(lambda page: page.find('text:contains("CONFIDENTIAL")').above())

        # Later, remove all exclusions
        pdf.clear_exclusions()
        ```
    """
    if not hasattr(self, "_pages"):
        raise AttributeError("PDF pages not yet initialized.")

    self._exclusions = []

    # Clear exclusions only from already-created (cached) pages to avoid forcing page creation
    for i in range(len(self._pages)):
        cached_page = self._pages._cache[i]
        if cached_page is None:
            continue
        try:
            cached_page.clear_exclusions()
        except Exception as e:
            logger.warning(f"Failed to clear exclusions from existing page {i}: {e}")
    return self

natural_pdf.PDF.close()

Close the underlying PDF file and clean up any temporary files.

Source code in natural_pdf/core/pdf.py

def close(self):
    """Close the underlying PDF file and clean up any temporary files."""
    if hasattr(self, "_pdf") and self._pdf is not None:
        try:
            self._pdf.close()
            logger.debug(f"Closed pdfplumber PDF object for {self.source_path}")
        except Exception as e:
            logger.warning(f"Error closing pdfplumber object: {e}")
        finally:
            self._pdf = None

    if hasattr(self, "_temp_file") and self._temp_file is not None:
        temp_file_path = None
        try:
            if hasattr(self._temp_file, "name") and self._temp_file.name:
                temp_file_path = self._temp_file.name
                # Only unlink if it exists and _is_stream is False (meaning WE created it)
                if not self._is_stream and os.path.exists(temp_file_path):
                    os.unlink(temp_file_path)
                    logger.debug(f"Removed temporary PDF file: {temp_file_path}")
        except Exception as e:
            logger.warning(f"Failed to clean up temporary file '{temp_file_path}': {e}")

    # Cancels the weakref finalizer so we don't double-clean
    if hasattr(self, "_finalizer") and self._finalizer.alive:
        self._finalizer()

natural_pdf.PDF.deskew(pages=None, resolution=300, angle=None, detection_resolution=72, force_overwrite=False, **deskew_kwargs)

Creates a new, in-memory PDF object containing deskewed versions of the specified pages from the original PDF.

This method renders each selected page, detects and corrects skew using the 'deskew' library, and then combines the resulting images into a new PDF using 'img2pdf'. The new PDF object is returned directly.

Important: The returned PDF is image-based. Any existing text, OCR results, annotations, or other elements from the original pages will not be carried over.

Parameters:

Name	Type	Description	Default
pages	Optional[Union[Iterable[int], range, slice]]	Page indices/slice to include (0-based). If None, processes all pages.	None
resolution	int	DPI resolution for rendering the output deskewed pages.	300
angle	Optional[float]	The specific angle (in degrees) to rotate by. If None, detects automatically.	None
detection_resolution	int	DPI resolution used for skew detection if angles are not already cached on the page objects.	72
force_overwrite	bool	If False (default), raises a ValueError if any target page already contains processed elements (text, OCR, regions) to prevent accidental data loss. Set to True to proceed anyway.	False
**deskew_kwargs		Additional keyword arguments forwarded to the deskew engine. during automatic detection (e.g., max_angle, num_peaks).	{}

Returns:

Type	Description
PDF	A new PDF object representing the deskewed document.

Raises:

Type	Description
ImportError	If 'deskew' or 'img2pdf' libraries are not installed.
ValueError	If force_overwrite is False and target pages contain elements.
FileNotFoundError	If the source PDF cannot be read (if file-based).
IOError	If creating the in-memory PDF fails.
RuntimeError	If rendering or deskewing individual pages fails.

Source code in natural_pdf/core/pdf.py

def deskew(
    self,
    pages: Optional[Union[Iterable[int], range, slice]] = None,
    resolution: int = 300,
    angle: Optional[float] = None,
    detection_resolution: int = 72,
    force_overwrite: bool = False,
    **deskew_kwargs,
) -> "PDF":
    """
    Creates a new, in-memory PDF object containing deskewed versions of the
    specified pages from the original PDF.

    This method renders each selected page, detects and corrects skew using the 'deskew'
    library, and then combines the resulting images into a new PDF using 'img2pdf'.
    The new PDF object is returned directly.

    Important: The returned PDF is image-based. Any existing text, OCR results,
    annotations, or other elements from the original pages will *not* be carried over.

    Args:
        pages: Page indices/slice to include (0-based). If None, processes all pages.
        resolution: DPI resolution for rendering the output deskewed pages.
        angle: The specific angle (in degrees) to rotate by. If None, detects automatically.
        detection_resolution: DPI resolution used for skew detection if angles are not
                              already cached on the page objects.
        force_overwrite: If False (default), raises a ValueError if any target page
                         already contains processed elements (text, OCR, regions) to
                         prevent accidental data loss. Set to True to proceed anyway.
        **deskew_kwargs: Additional keyword arguments forwarded to the deskew engine.
                         during automatic detection (e.g., `max_angle`, `num_peaks`).

    Returns:
        A new PDF object representing the deskewed document.

    Raises:
        ImportError: If 'deskew' or 'img2pdf' libraries are not installed.
        ValueError: If `force_overwrite` is False and target pages contain elements.
        FileNotFoundError: If the source PDF cannot be read (if file-based).
        IOError: If creating the in-memory PDF fails.
        RuntimeError: If rendering or deskewing individual pages fails.
    """
    if not DESKEW_AVAILABLE:
        raise ImportError(
            "Deskew/img2pdf libraries missing. Install with: pip install natural-pdf[deskew]"
        )

    target_pages = self._get_target_pages(pages)  # Use helper to resolve pages

    # --- Safety Check --- #
    if not force_overwrite:
        for page in target_pages:
            # Check if the element manager has been initialized and contains any elements
            if hasattr(page, "has_element_cache") and page.has_element_cache():
                raise ValueError(
                    f"Page {page.number} contains existing elements (text, OCR, etc.). "
                    f"Deskewing creates an image-only PDF, discarding these elements. "
                    f"Set force_overwrite=True to proceed."
                )

    # --- Process Pages --- #
    deskewed_images_bytes = []
    logger.info(f"Deskewing {len(target_pages)} pages (output resolution={resolution} DPI)...")

    for page in tqdm(target_pages, desc="Deskewing Pages", leave=False):
        try:
            # Use page.deskew to get the corrected PIL image
            # Pass down resolutions and kwargs
            deskewed_img = page.deskew(
                resolution=resolution,
                angle=angle,  # Let page.deskew handle detection/caching
                detection_resolution=detection_resolution,
                **deskew_kwargs,
            )

            if not deskewed_img:
                logger.warning(
                    f"Page {page.number}: Failed to generate deskewed image, skipping."
                )
                continue

            # Convert image to bytes for img2pdf (use PNG for lossless quality)
            with io.BytesIO() as buf:
                deskewed_img.save(buf, format="PNG")
                deskewed_images_bytes.append(buf.getvalue())

        except Exception as e:
            logger.error(
                f"Page {page.number}: Failed during deskewing process: {e}", exc_info=True
            )
            # Option: Raise a runtime error, or continue and skip the page?
            # Raising makes the whole operation fail if one page fails.
            raise RuntimeError(f"Failed to process page {page.number} during deskewing.") from e

    # --- Create PDF --- #
    if not deskewed_images_bytes:
        raise RuntimeError("No pages were successfully processed to create the deskewed PDF.")

    logger.info(f"Combining {len(deskewed_images_bytes)} deskewed images into in-memory PDF...")
    try:
        # Use img2pdf to combine image bytes into PDF bytes
        if img2pdf is None:
            raise RuntimeError(
                "img2pdf library is not available despite DESKEW_AVAILABLE flag being set."
            )
        pdf_bytes = img2pdf.convert(deskewed_images_bytes)
        if pdf_bytes is None:
            raise RuntimeError("img2pdf.convert returned no data.")

        # Wrap bytes in a stream
        pdf_stream = io.BytesIO(pdf_bytes)

        # Create a new PDF object from the stream using original config
        logger.info("Creating new PDF object from deskewed stream...")
        new_pdf = PDF(
            pdf_stream,
            reading_order=self._reading_order,
            font_attrs=self._font_attrs,
            keep_spaces=self._config.get("keep_spaces", True),
            text_layer=self._text_layer,
        )
        return new_pdf
    except Exception as e:
        logger.error(f"Failed to create in-memory PDF using img2pdf/PDF init: {e}")
        raise IOError("Failed to create deskewed PDF object from image stream.") from e

natural_pdf.PDF.detect_checkboxes(*args, **kwargs)

Detects checkboxes on all pages in the PDF.

This is a convenience method that calls detect_checkboxes on the PDF's page collection.

Parameters:

Name	Type	Description	Default
*args		Positional arguments passed to pages.detect_checkboxes().	()
**kwargs		Keyword arguments passed to pages.detect_checkboxes().	{}

Returns:

Type	Description
ElementCollection[Region]	An ElementCollection of all detected checkbox Region objects.

Source code in natural_pdf/core/pdf.py

def detect_checkboxes(self, *args, **kwargs) -> "ElementCollection[Region]":
    """
    Detects checkboxes on all pages in the PDF.

    This is a convenience method that calls detect_checkboxes on the PDF's
    page collection.

    Args:
        *args: Positional arguments passed to pages.detect_checkboxes().
        **kwargs: Keyword arguments passed to pages.detect_checkboxes().

    Returns:
        An ElementCollection of all detected checkbox Region objects.
    """
    return self.pages.detect_checkboxes(*args, **kwargs)

natural_pdf.PDF.export_ocr_correction_task(output_zip_path, *, overwrite=False, suggest=None, resolution=300)

Exports OCR results from this PDF into a correction task package. Exports OCR results from this PDF into a correction task package.

Parameters:

Name	Type	Description	Default
output_zip_path	str	The path to save the output zip file.	required
overwrite	bool	When True, replace any existing archive at output_zip_path.	False
suggest		Optional callable that can provide OCR suggestions per region.	None
resolution	int	DPI used when rendering page images for the package.	300

Source code in natural_pdf/core/pdf.py

def export_ocr_correction_task(
    self,
    output_zip_path: str,
    *,
    overwrite: bool = False,
    suggest=None,
    resolution: int = 300,
):
    """
    Exports OCR results from this PDF into a correction task package.
    Exports OCR results from this PDF into a correction task package.

    Args:
        output_zip_path: The path to save the output zip file.
        overwrite: When True, replace any existing archive at ``output_zip_path``.
        suggest: Optional callable that can provide OCR suggestions per region.
        resolution: DPI used when rendering page images for the package.
    """
    try:
        from natural_pdf.utils.packaging import create_correction_task_package

        create_correction_task_package(
            source=self,
            output_zip_path=output_zip_path,
            overwrite=overwrite,
            suggest=suggest,
            resolution=resolution,
        )
    except ImportError:
        logger.error(
            "Failed to import 'create_correction_task_package'. Packaging utility might be missing."
        )
        logger.error(
            "Failed to import 'create_correction_task_package'. Packaging utility might be missing."
        )
    except Exception as e:
        logger.error(f"Failed to export correction task: {e}")
        raise
        logger.error(f"Failed to export correction task: {e}")
        raise

natural_pdf.PDF.extract_tables(selector=None, merge_across_pages=False, method=None, table_settings=None, check_tatr=True)

Extract tables from the document or matching elements.

Parameters:

Name	Type	Description	Default
selector	Optional[str]	Optional selector to filter tables (not yet implemented).	None
merge_across_pages	bool	Whether to merge tables that span across pages (not yet implemented).	False
method	Optional[str]	Extraction strategy to prefer. Mirrors Page.extract_tables.	None
table_settings	Optional[dict]	Per-method configuration forwarded to Page.extract_tables.	None
check_tatr	bool	When True, visit TATR table regions before falling back to page extraction.	True

Returns:

Type	Description
List[Any]	List of extracted tables

Source code in natural_pdf/core/pdf.py

def extract_tables(
    self,
    selector: Optional[str] = None,
    merge_across_pages: bool = False,
    method: Optional[str] = None,
    table_settings: Optional[dict] = None,
    check_tatr: bool = True,
) -> List[Any]:
    """
    Extract tables from the document or matching elements.

    Args:
        selector: Optional selector to filter tables (not yet implemented).
        merge_across_pages: Whether to merge tables that span across pages (not yet implemented).
        method: Extraction strategy to prefer. Mirrors ``Page.extract_tables``.
        table_settings: Per-method configuration forwarded to ``Page.extract_tables``.
        check_tatr: When True, visit TATR table regions before falling back to page extraction.

    Returns:
        List of extracted tables
    """
    if not hasattr(self, "_pages"):
        raise AttributeError("PDF pages not yet initialized.")

    logger.warning("PDF.extract_tables is not fully implemented yet.")
    all_tables = []

    for page in self.pages:
        if hasattr(page, "extract_tables"):
            all_tables.extend(
                page.extract_tables(
                    method=method,
                    table_settings=table_settings,
                    check_tatr=check_tatr,
                )
            )
        else:
            logger.debug(f"Page {page.number} does not have extract_tables method.")

    if selector:
        logger.warning("Filtering extracted tables by selector is not implemented.")

    if merge_across_pages:
        logger.warning("Merging tables across pages is not implemented.")

    return all_tables

natural_pdf.PDF.extract_text(selector=None, preserve_whitespace=True, preserve_line_breaks=True, page_separator='\n', use_exclusions=True, debug_exclusions=False, *, layout=True, x_density=None, y_density=None, x_tolerance=None, y_tolerance=None, line_dir=None, char_dir=None, strip_final=False, strip_empty=False)

Extract text from the entire document or matching elements.

Parameters:

Name	Type	Description	Default
selector	Optional[str]	Optional selector to filter elements	None
preserve_whitespace	bool	Whether to keep blank characters	True
preserve_line_breaks	bool	When False, collapse newlines in each page's text.	True
page_separator	Optional[str]	String inserted between page texts when combining results.	'\n'
use_exclusions	bool	Whether to apply exclusion regions	True
debug_exclusions	bool	Whether to output detailed debugging for exclusions	False
preserve_whitespace	bool	Whether to keep blank characters	True
use_exclusions	bool	Whether to apply exclusion regions	True
debug_exclusions	bool	Whether to output detailed debugging for exclusions	False
layout	bool	Whether to enable layout-aware spacing (default: True).	True
x_density	Optional[float]	Horizontal character density override.	None
y_density	Optional[float]	Vertical line density override.	None
x_tolerance	Optional[float]	Horizontal clustering tolerance.	None
y_tolerance	Optional[float]	Vertical clustering tolerance.	None
line_dir	Optional[str]	Line reading direction override.	None
char_dir	Optional[str]	Character reading direction override.	None
strip_final	bool	When True, strip trailing whitespace from the combined text.	False
strip_empty	bool	When True, drop empty lines from the output.	False

Returns:

Type	Description
str	Extracted text as string

Source code in natural_pdf/core/pdf.py

def extract_text(
    self,
    selector: Optional[str] = None,
    preserve_whitespace: bool = True,
    preserve_line_breaks: bool = True,
    page_separator: Optional[str] = "\n",
    use_exclusions: bool = True,
    debug_exclusions: bool = False,
    *,
    layout: bool = True,
    x_density: Optional[float] = None,
    y_density: Optional[float] = None,
    x_tolerance: Optional[float] = None,
    y_tolerance: Optional[float] = None,
    line_dir: Optional[str] = None,
    char_dir: Optional[str] = None,
    strip_final: bool = False,
    strip_empty: bool = False,
) -> str:
    """
    Extract text from the entire document or matching elements.

    Args:
        selector: Optional selector to filter elements
        preserve_whitespace: Whether to keep blank characters
        preserve_line_breaks: When False, collapse newlines in each page's text.
        page_separator: String inserted between page texts when combining results.
        use_exclusions: Whether to apply exclusion regions
        debug_exclusions: Whether to output detailed debugging for exclusions
        preserve_whitespace: Whether to keep blank characters
        use_exclusions: Whether to apply exclusion regions
        debug_exclusions: Whether to output detailed debugging for exclusions
        layout: Whether to enable layout-aware spacing (default: True).
        x_density: Horizontal character density override.
        y_density: Vertical line density override.
        x_tolerance: Horizontal clustering tolerance.
        y_tolerance: Vertical clustering tolerance.
        line_dir: Line reading direction override.
        char_dir: Character reading direction override.
        strip_final: When True, strip trailing whitespace from the combined text.
        strip_empty: When True, drop empty lines from the output.

    Returns:
        Extracted text as string
    """
    if not hasattr(self, "_pages"):
        raise AttributeError("PDF pages not yet initialized.")

    if selector:
        elements = self.find_all(
            selector,
            apply_exclusions=use_exclusions,
            layout=layout,
            x_density=x_density,
            y_density=y_density,
            x_tolerance=x_tolerance,
            y_tolerance=y_tolerance,
            line_dir=line_dir,
            char_dir=char_dir,
            strip_final=strip_final,
            strip_empty=strip_empty,
        )
        return elements.extract_text(
            preserve_whitespace=preserve_whitespace,
            preserve_line_breaks=preserve_line_breaks,
            layout=layout,
            x_density=x_density,
            y_density=y_density,
            x_tolerance=x_tolerance,
            y_tolerance=y_tolerance,
            line_dir=line_dir,
            char_dir=char_dir,
            strip_final=strip_final,
            strip_empty=strip_empty,
        )

    if debug_exclusions:
        print(f"PDF: Extracting text with exclusions from {len(self.pages)} pages")
        print(f"PDF: Found {len(self._exclusions)} document-level exclusions")

    texts = []
    for page in self.pages:
        texts.append(
            page.extract_text(
                preserve_whitespace=preserve_whitespace,
                preserve_line_breaks=preserve_line_breaks,
                use_exclusions=use_exclusions,
                debug_exclusions=debug_exclusions,
                layout=layout,
                x_density=x_density,
                y_density=y_density,
                x_tolerance=x_tolerance,
                y_tolerance=y_tolerance,
                line_dir=line_dir,
                char_dir=char_dir,
                strip_final=strip_final,
                strip_empty=strip_empty,
            )
        )

    if debug_exclusions:
        print(f"PDF: Combined {len(texts)} pages of text")

    separator = "" if page_separator is None else page_separator
    return separator.join(texts)

natural_pdf.PDF.find(selector=None, *, text=None, apply_exclusions=True, regex=False, case=True, text_tolerance=None, auto_text_tolerance=None, reading_order=True, **extra_kwargs)

Find the first element matching the selector OR text content across all pages.

Provide EITHER selector OR text, but not both.

Parameters:

Name	Type	Description	Default
selector	Optional[str]	CSS-like selector string.	None
text	Optional[Union[str, Sequence[str]]]	Optional text shortcut (equivalent to text:contains(...)).	None
apply_exclusions	bool	Whether to exclude elements in exclusion regions (default: True).	True
regex	bool	Whether to use regex for text search (selector or text) (default: False).	False
case	bool	Whether to do case-sensitive text search (selector or text) (default: True).	True
text_tolerance	Optional[Dict[str, Any]]	Optional dict of tolerance overrides applied temporarily.	None
auto_text_tolerance	Optional[Dict[str, Any]]	Optional overrides controlling automatic tolerance logic.	None
reading_order	bool	Whether to sort matches in reading order when applicable (default: True).	True

Returns:

Type	Description
Optional[Any]	Element object or None if not found.

Source code in natural_pdf/core/pdf.py

def find(
    self,
    selector: Optional[str] = None,
    *,
    text: Optional[Union[str, Sequence[str]]] = None,
    apply_exclusions: bool = True,
    regex: bool = False,
    case: bool = True,
    text_tolerance: Optional[Dict[str, Any]] = None,
    auto_text_tolerance: Optional[Dict[str, Any]] = None,
    reading_order: bool = True,
    **extra_kwargs: Any,
) -> Optional[Any]:
    """
    Find the first element matching the selector OR text content across all pages.

    Provide EITHER `selector` OR `text`, but not both.

    Args:
        selector: CSS-like selector string.
        text: Optional text shortcut (equivalent to ``text:contains(...)``).
        apply_exclusions: Whether to exclude elements in exclusion regions (default: True).
        regex: Whether to use regex for text search (`selector` or `text`) (default: False).
        case: Whether to do case-sensitive text search (`selector` or `text`) (default: True).
        text_tolerance: Optional dict of tolerance overrides applied temporarily.
        auto_text_tolerance: Optional overrides controlling automatic tolerance logic.
        reading_order: Whether to sort matches in reading order when applicable (default: True).

    Returns:
        Element object or None if not found.
    """
    if not hasattr(self, "_pages"):
        raise AttributeError("PDF pages not yet initialized.")

    if selector is not None and text is not None:
        raise ValueError("Provide either 'selector' or 'text', not both.")
    if selector is None and text is None:
        raise ValueError("Provide either 'selector' or 'text'.")

    # Construct selector if 'text' is provided
    effective_selector = ""
    if text is not None:
        effective_selector = build_text_contains_selector(text)
        logger.debug(
            f"Using text shortcut: find(text={text!r}) -> find('{effective_selector}')"
        )
    elif selector is not None:
        effective_selector = selector
    else:
        raise ValueError("Internal error: No selector or text provided.")

    _ = parse_selector(effective_selector)  # Parse once to fail fast on invalid selectors

    selector_kwargs: Dict[str, Any] = dict(extra_kwargs)
    selector_kwargs.update(
        {
            "apply_exclusions": apply_exclusions,
            "regex": regex,
            "case": case,
            "text_tolerance": text_tolerance,
            "auto_text_tolerance": auto_text_tolerance,
            "reading_order": reading_order,
        }
    )

    for page in self.pages:
        element = page.find(selector=effective_selector, **selector_kwargs)
        if element is not None:
            return element

    return None

natural_pdf.PDF.find_all(selector=None, *, text=None, apply_exclusions=True, regex=False, case=True, text_tolerance=None, auto_text_tolerance=None, reading_order=True, **extra_kwargs)

Find all elements matching the selector OR text content across all pages.

Provide EITHER selector OR text, but not both.

Parameters:

Name	Type	Description	Default
selector	Optional[str]	CSS-like selector string.	None
text	Optional[Union[str, Sequence[str]]]	Optional text shortcut (equivalent to text:contains(...)).	None
apply_exclusions	bool	Whether to exclude elements in exclusion regions (default: True).	True
regex	bool	Whether to use regex for text search (selector or text) (default: False).	False
case	bool	Whether to do case-sensitive text search (selector or text) (default: True).	True
text_tolerance	Optional[Dict[str, Any]]	Optional dict of tolerance overrides applied temporarily.	None
auto_text_tolerance	Optional[Dict[str, Any]]	Optional overrides controlling automatic tolerance logic.	None
reading_order	bool	Whether to sort matches in reading order when applicable (default: True).	True

Returns:

Type	Description
ElementCollection	ElementCollection with matching elements.

Source code in natural_pdf/core/pdf.py

def find_all(
    self,
    selector: Optional[str] = None,
    *,
    text: Optional[Union[str, Sequence[str]]] = None,
    apply_exclusions: bool = True,
    regex: bool = False,
    case: bool = True,
    text_tolerance: Optional[Dict[str, Any]] = None,
    auto_text_tolerance: Optional[Dict[str, Any]] = None,
    reading_order: bool = True,
    **extra_kwargs: Any,
) -> "ElementCollection":
    """
    Find all elements matching the selector OR text content across all pages.

    Provide EITHER `selector` OR `text`, but not both.

    Args:
        selector: CSS-like selector string.
        text: Optional text shortcut (equivalent to ``text:contains(...)``).
        apply_exclusions: Whether to exclude elements in exclusion regions (default: True).
        regex: Whether to use regex for text search (`selector` or `text`) (default: False).
        case: Whether to do case-sensitive text search (`selector` or `text`) (default: True).
        text_tolerance: Optional dict of tolerance overrides applied temporarily.
        auto_text_tolerance: Optional overrides controlling automatic tolerance logic.
        reading_order: Whether to sort matches in reading order when applicable (default: True).

    Returns:
        ElementCollection with matching elements.
    """
    if not hasattr(self, "_pages"):
        raise AttributeError("PDF pages not yet initialized.")

    if selector is not None and text is not None:
        raise ValueError("Provide either 'selector' or 'text', not both.")
    if selector is None and text is None:
        raise ValueError("Provide either 'selector' or 'text'.")

    # Construct selector if 'text' is provided
    effective_selector = ""
    if text is not None:
        effective_selector = build_text_contains_selector(text)
        logger.debug(
            f"Using text shortcut: find_all(text={text!r}) -> find_all('{effective_selector}')"
        )
    elif selector is not None:
        effective_selector = selector
    else:
        raise ValueError("Internal error: No selector or text provided.")

    _ = parse_selector(effective_selector)  # Validate selector once

    selector_kwargs: Dict[str, Any] = dict(extra_kwargs)
    selector_kwargs.update(
        {
            "apply_exclusions": apply_exclusions,
            "regex": regex,
            "case": case,
            "text_tolerance": text_tolerance,
            "auto_text_tolerance": auto_text_tolerance,
            "reading_order": reading_order,
        }
    )

    all_elements: List = []
    for page in self.pages:
        page_elements = page.find_all(selector=effective_selector, **selector_kwargs)
        if page_elements:
            all_elements.extend(page_elements.elements)

    from natural_pdf.elements.element_collection import ElementCollection

    return ElementCollection(all_elements)

natural_pdf.PDF.from_images(images, resolution=300, apply_ocr=True, ocr_engine=None, **pdf_options) classmethod

Create a PDF from image(s).

Parameters:

Name	Type	Description	Default
images	Union[Image, List[Image], str, List[str], Path, List[Path]]	Single image, list of images, or path(s)/URL(s) to image files	required
resolution	int	DPI for the PDF (default: 300, good for OCR and viewing)	300
apply_ocr	bool	Apply OCR to make searchable (default: True)	True
ocr_engine	Optional[str]	OCR engine to use (default: auto-detect)	None
**pdf_options		Options passed to PDF constructor	{}

Returns:

Type	Description
PDF	PDF object containing the images as pages

Example

# Simple scan to searchable PDF
pdf = PDF.from_images("scan.jpg")

# From URL
pdf = PDF.from_images("https://example.com/image.png")

# Multiple pages (mix of local and URLs)
pdf = PDF.from_images(["page1.png", "https://example.com/page2.jpg"])

# Without OCR
pdf = PDF.from_images(images, apply_ocr=False)

# With specific engine
pdf = PDF.from_images(images, ocr_engine='surya')

Source code in natural_pdf/core/pdf.py

@classmethod
def from_images(
    cls,
    images: Union["Image.Image", List["Image.Image"], str, List[str], Path, List[Path]],
    resolution: int = 300,
    apply_ocr: bool = True,
    ocr_engine: Optional[str] = None,
    **pdf_options,
) -> "PDF":
    """Create a PDF from image(s).

    Args:
        images: Single image, list of images, or path(s)/URL(s) to image files
        resolution: DPI for the PDF (default: 300, good for OCR and viewing)
        apply_ocr: Apply OCR to make searchable (default: True)
        ocr_engine: OCR engine to use (default: auto-detect)
        **pdf_options: Options passed to PDF constructor

    Returns:
        PDF object containing the images as pages

    Example:
        ```python
        # Simple scan to searchable PDF
        pdf = PDF.from_images("scan.jpg")

        # From URL
        pdf = PDF.from_images("https://example.com/image.png")

        # Multiple pages (mix of local and URLs)
        pdf = PDF.from_images(["page1.png", "https://example.com/page2.jpg"])

        # Without OCR
        pdf = PDF.from_images(images, apply_ocr=False)

        # With specific engine
        pdf = PDF.from_images(images, ocr_engine='surya')
        ```
    """
    import urllib.request

    from PIL import ImageOps

    def _open_image(source: Union[Image.Image, str, Path]) -> Image.Image:
        """Open an image from file path, URL, or return PIL Image as-is."""
        if isinstance(source, Image.Image):
            return source

        source_str = str(source)
        if source_str.startswith(("http://", "https://")):
            # Download from URL
            with urllib.request.urlopen(source_str) as response:
                img_data = response.read()
            return Image.open(io.BytesIO(img_data))
        else:
            # Local file path
            return Image.open(source)

    # Normalize inputs to list of PIL Images
    if isinstance(images, (str, Path)):
        image_list = [_open_image(images)]
    elif isinstance(images, Image.Image):
        image_list = [images]
    elif isinstance(images, list):
        image_list = [_open_image(item) for item in images]
    else:
        raise TypeError(
            "images must be a path, PIL Image, or a list of paths/PIL Image instances."
        )

    # Process images
    processed_images: List[Image.Image] = []
    for img in image_list:
        # Fix EXIF rotation
        img = ImageOps.exif_transpose(img) or img

        # Convert RGBA to RGB (PDF doesn't handle transparency well)
        if img.mode == "RGBA":
            bg = Image.new("RGB", img.size, "white")
            bg.paste(img, mask=img.split()[3])
            img = bg
        elif img.mode not in ["RGB", "L", "1", "CMYK"]:
            img = img.convert("RGB")

        processed_images.append(img)

    # Create PDF at specified resolution
    # Use BytesIO to keep in memory
    pdf_buffer = io.BytesIO()
    processed_images[0].save(
        pdf_buffer,
        "PDF",
        save_all=True,
        append_images=processed_images[1:] if len(processed_images) > 1 else [],
        resolution=resolution,
    )
    pdf_buffer.seek(0)

    # Create PDF object
    pdf = cls(pdf_buffer, **pdf_options)

    # Store metadata about source
    pdf._from_images = True
    pdf._source_metadata = {
        "type": "images",
        "count": len(processed_images),
        "resolution": resolution,
    }

    # Apply OCR if requested
    if apply_ocr:
        pdf.apply_ocr(engine=ocr_engine, resolution=resolution)

    return pdf

natural_pdf.PDF.get_id()

Get unique identifier for this PDF.

Source code in natural_pdf/core/pdf.py

def get_id(self) -> str:
    """Get unique identifier for this PDF."""
    """Get unique identifier for this PDF."""
    return self.path

natural_pdf.PDF.get_manager(key)

Retrieve a manager instance by its key, instantiating it lazily if needed.

Managers are specialized components that handle specific functionality like classification, structured data extraction, or OCR processing. They are instantiated on-demand to minimize memory usage and startup time.

Parameters:

Name	Type	Description	Default
key	str	The manager key to retrieve. Common keys include 'classification' and 'structured_data'.	required

Returns:

Type	Description
Any	The manager instance for the specified key.

Raises:

Type	Description
KeyError	If no manager is registered for the given key.
RuntimeError	If the manager failed to initialize.

Example

pdf = npdf.PDF("document.pdf")
classification_mgr = pdf.get_manager('classification')
structured_data_mgr = pdf.get_manager('structured_data')

Source code in natural_pdf/core/pdf.py

def get_manager(self, key: str) -> Any:
    """Retrieve a manager instance by its key, instantiating it lazily if needed.

    Managers are specialized components that handle specific functionality like
    classification, structured data extraction, or OCR processing. They are
    instantiated on-demand to minimize memory usage and startup time.

    Args:
        key: The manager key to retrieve. Common keys include 'classification'
            and 'structured_data'.

    Returns:
        The manager instance for the specified key.

    Raises:
        KeyError: If no manager is registered for the given key.
        RuntimeError: If the manager failed to initialize.

    Example:
        ```python
        pdf = npdf.PDF("document.pdf")
        classification_mgr = pdf.get_manager('classification')
        structured_data_mgr = pdf.get_manager('structured_data')
        ```
    """
    # Check if already instantiated
    if key in self._managers:
        manager_instance = self._managers[key]
        if manager_instance is None:
            raise RuntimeError(f"Manager '{key}' failed to initialize previously.")
        return manager_instance

    # Not instantiated yet: get factory/class
    if not hasattr(self, "_manager_factories") or key not in self._manager_factories:
        raise KeyError(
            f"No manager registered for key '{key}'. Available: {list(getattr(self, '_manager_factories', {}).keys())}"
        )
    factory_or_class = self._manager_factories[key]
    try:
        resolved = factory_or_class
        # If it's a callable that's not a class, call it to get the class/instance
        if not isinstance(resolved, type) and callable(resolved):
            resolved = resolved()
        # If it's a class, instantiate it
        if isinstance(resolved, type):
            instance = resolved()
        else:
            instance = resolved  # Already an instance
        self._managers[key] = instance
        return instance
    except Exception as e:
        logger.error(f"Failed to initialize manager for key '{key}': {e}")
        self._managers[key] = None
        raise RuntimeError(f"Manager '{key}' failed to initialize: {e}") from e

natural_pdf.PDF.get_sections(start_elements=None, end_elements=None, new_section_on_page_break=False, include_boundaries='both', orientation='vertical')

Extract sections from the entire PDF based on start/end elements.

This method delegates to the PageCollection.get_sections() method, providing a convenient way to extract document sections across all pages.

Parameters:

Name	Type	Description	Default
start_elements		Elements or selector string that mark the start of sections (optional)	None
end_elements		Elements or selector string that mark the end of sections (optional)	None
new_section_on_page_break		Whether to start a new section at page boundaries (default: False)	False
include_boundaries		How to include boundary elements: 'start', 'end', 'both', or 'none' (default: 'both')	'both'
orientation		'vertical' (default) or 'horizontal' - determines section direction	'vertical'

Returns:

Type	Description
ElementCollection	ElementCollection of Region objects representing the extracted sections

Example

Extract sections between headers:

pdf = npdf.PDF("document.pdf")

# Get sections between headers
sections = pdf.get_sections(
    start_elements='text[size>14]:bold',
    end_elements='text[size>14]:bold'
)

# Get sections that break at page boundaries
sections = pdf.get_sections(
    start_elements='text:contains("Chapter")',
    new_section_on_page_break=True
)

Note

You can provide only start_elements, only end_elements, or both. - With only start_elements: sections go from each start to the next start (or end of document) - With only end_elements: sections go from beginning of document to each end - With both: sections go from each start to the corresponding end

Source code in natural_pdf/core/pdf.py

def get_sections(
    self,
    start_elements=None,
    end_elements=None,
    new_section_on_page_break=False,
    include_boundaries="both",
    orientation="vertical",
) -> "ElementCollection":
    """
    Extract sections from the entire PDF based on start/end elements.

    This method delegates to the PageCollection.get_sections() method,
    providing a convenient way to extract document sections across all pages.

    Args:
        start_elements: Elements or selector string that mark the start of sections (optional)
        end_elements: Elements or selector string that mark the end of sections (optional)
        new_section_on_page_break: Whether to start a new section at page boundaries (default: False)
        include_boundaries: How to include boundary elements: 'start', 'end', 'both', or 'none' (default: 'both')
        orientation: 'vertical' (default) or 'horizontal' - determines section direction

    Returns:
        ElementCollection of Region objects representing the extracted sections

    Example:
        Extract sections between headers:
        ```python
        pdf = npdf.PDF("document.pdf")

        # Get sections between headers
        sections = pdf.get_sections(
            start_elements='text[size>14]:bold',
            end_elements='text[size>14]:bold'
        )

        # Get sections that break at page boundaries
        sections = pdf.get_sections(
            start_elements='text:contains("Chapter")',
            new_section_on_page_break=True
        )
        ```

    Note:
        You can provide only start_elements, only end_elements, or both.
        - With only start_elements: sections go from each start to the next start (or end of document)
        - With only end_elements: sections go from beginning of document to each end
        - With both: sections go from each start to the corresponding end
    """
    if not hasattr(self, "_pages"):
        raise AttributeError("PDF pages not yet initialized.")

    return self.pages.get_sections(
        start_elements=start_elements,
        end_elements=end_elements,
        new_section_on_page_break=new_section_on_page_break,
        include_boundaries=include_boundaries,
        orientation=orientation,
    )

natural_pdf.PDF.highlights(show=False)

Create a highlight context for accumulating highlights.

This allows for clean syntax to show multiple highlight groups:

Example

with pdf.highlights() as h: h.add(pdf.find_all('table'), label='tables', color='blue') h.add(pdf.find_all('text:bold'), label='bold text', color='red') h.show()

Or with automatic display

with pdf.highlights(show=True) as h: h.add(pdf.find_all('table'), label='tables') h.add(pdf.find_all('text:bold'), label='bold') # Automatically shows when exiting the context

Parameters:

Name	Type	Description	Default
show	bool	If True, automatically show highlights when exiting context	False

Returns:

Type	Description
HighlightContext	HighlightContext for accumulating highlights

Source code in natural_pdf/core/pdf.py

def highlights(self, show: bool = False) -> "HighlightContext":
    """
    Create a highlight context for accumulating highlights.

    This allows for clean syntax to show multiple highlight groups:

    Example:
        with pdf.highlights() as h:
            h.add(pdf.find_all('table'), label='tables', color='blue')
            h.add(pdf.find_all('text:bold'), label='bold text', color='red')
            h.show()

    Or with automatic display:
        with pdf.highlights(show=True) as h:
            h.add(pdf.find_all('table'), label='tables')
            h.add(pdf.find_all('text:bold'), label='bold')
            # Automatically shows when exiting the context

    Args:
        show: If True, automatically show highlights when exiting context

    Returns:
        HighlightContext for accumulating highlights
    """
    from natural_pdf.core.highlighting_service import HighlightContext

    return HighlightContext(self, show_on_exit=show)

natural_pdf.PDF.save_pdf(output_path, ocr=False, original=False, dpi=300)

Saves the PDF object (all its pages) to a new file.

Choose one saving mode: - ocr=True: Creates a new, image-based PDF using OCR results from all pages. Text generated during the natural-pdf session becomes searchable, but original vector content is lost. Requires 'ocr-export' extras. - original=True: Saves a copy of the original PDF file this object represents. Any OCR results or analyses from the natural-pdf session are NOT included. If the PDF was opened from an in-memory buffer, this mode may not be suitable. Requires 'ocr-export' extras.

Parameters:

Name	Type	Description	Default
output_path	Union[str, Path]	Path to save the new PDF file.	required
ocr	bool	If True, save as a searchable, image-based PDF using OCR data.	False
original	bool	If True, save the original source PDF content.	False
dpi	int	Resolution (dots per inch) used only when ocr=True.	300

Raises:

Type	Description
ValueError	If the PDF has no pages, if neither or both 'ocr' and 'original' are True.
ImportError	If required libraries are not installed for the chosen mode.
RuntimeError	If an unexpected error occurs during saving.

Source code in natural_pdf/core/pdf.py

def save_pdf(
    self,
    output_path: Union[str, Path],
    ocr: bool = False,
    original: bool = False,
    dpi: int = 300,
):
    """
    Saves the PDF object (all its pages) to a new file.

    Choose one saving mode:
    - `ocr=True`: Creates a new, image-based PDF using OCR results from all pages.
      Text generated during the natural-pdf session becomes searchable,
      but original vector content is lost. Requires 'ocr-export' extras.
    - `original=True`: Saves a copy of the original PDF file this object represents.
      Any OCR results or analyses from the natural-pdf session are NOT included.
      If the PDF was opened from an in-memory buffer, this mode may not be suitable.
      Requires 'ocr-export' extras.

    Args:
        output_path: Path to save the new PDF file.
        ocr: If True, save as a searchable, image-based PDF using OCR data.
        original: If True, save the original source PDF content.
        dpi: Resolution (dots per inch) used only when ocr=True.

    Raises:
        ValueError: If the PDF has no pages, if neither or both 'ocr'
                    and 'original' are True.
        ImportError: If required libraries are not installed for the chosen mode.
        RuntimeError: If an unexpected error occurs during saving.
    """
    if not self.pages:
        raise ValueError("Cannot save an empty PDF object.")

    if not (ocr ^ original):  # XOR: exactly one must be true
        raise ValueError("Exactly one of 'ocr' or 'original' must be True.")

    output_path_obj = Path(output_path)
    output_path_str = str(output_path_obj)

    if ocr:
        if create_searchable_pdf is None:
            raise ImportError(
                "Saving with ocr=True requires the OCR export dependencies. "
                'Install with: pip install "natural-pdf[ocr-export]"'
            )
        has_vector_elements = False
        for page in self.pages:
            rects = getattr(page, "rects", None)
            lines = getattr(page, "lines", None)
            curves = getattr(page, "curves", None)
            chars = getattr(page, "chars", None)
            words = getattr(page, "words", None)
            if (
                (rects and len(rects) > 0)  # type: ignore[arg-type]
                or (lines and len(lines) > 0)  # type: ignore[arg-type]
                or (curves and len(curves) > 0)  # type: ignore[arg-type]
                or (
                    chars
                    and any(
                        getattr(el, "source", None) != "ocr"
                        for el in cast(Iterable[Any], chars)
                    )
                )
                or (
                    words
                    and any(
                        getattr(el, "source", None) != "ocr"
                        for el in cast(Iterable[Any], words)
                    )
                )
            ):
                has_vector_elements = True
                break
        if has_vector_elements:
            logger.warning(
                "Warning: Saving with ocr=True creates an image-based PDF. "
                "Original vector elements (rects, lines, non-OCR text/chars) "
                "will not be preserved in the output file."
            )

        logger.info(f"Saving searchable PDF (OCR text layer) to: {output_path_str}")
        try:
            # Delegate to the searchable PDF exporter, passing self (PDF instance)
            create_searchable_pdf(self, output_path_str, dpi=dpi)
        except Exception as e:
            raise RuntimeError(f"Failed to create searchable PDF: {e}") from e

    elif original:
        if create_original_pdf is None:
            raise ImportError(
                "Saving with original=True requires 'pikepdf'. "
                'Install with: pip install "natural-pdf[ocr-export]"'
            )

        # Optional: Add warning about losing OCR data similar to PageCollection
        has_ocr_elements = False
        for page in self.pages:
            if hasattr(page, "find_all"):
                ocr_text_elements = page.find_all("text[source=ocr]")
                if ocr_text_elements:
                    has_ocr_elements = True
                    break
            elif hasattr(page, "words"):  # Fallback
                if any(getattr(el, "source", None) == "ocr" for el in page.words):
                    has_ocr_elements = True
                    break
        if has_ocr_elements:
            logger.warning(
                "Warning: Saving with original=True preserves original page content. "
                "OCR text generated in this session will not be included in the saved file."
            )

        logger.info(f"Saving original PDF content to: {output_path_str}")
        try:
            # Delegate to the original PDF exporter, passing self (PDF instance)
            create_original_pdf(self, output_path_str)
        except Exception as e:
            # Re-raise exception from exporter
            raise e

natural_pdf.PDF.save_searchable(output_path, dpi=300)

DEPRECATED: Use save_pdf(..., ocr=True) instead. Saves the PDF with an OCR text layer, making content searchable.

Requires optional dependencies. Install with: pip install "natural-pdf[ocr-export]"

Parameters:

Name	Type	Description	Default
output_path	Union[str, Path]	Path to save the searchable PDF	required
dpi	int	Resolution for rendering and OCR overlay.	300

Source code in natural_pdf/core/pdf.py

def save_searchable(self, output_path: Union[str, "Path"], dpi: int = 300):
    """
    DEPRECATED: Use save_pdf(..., ocr=True) instead.
    Saves the PDF with an OCR text layer, making content searchable.

    Requires optional dependencies. Install with: pip install \"natural-pdf[ocr-export]\"

    Args:
        output_path: Path to save the searchable PDF
        dpi: Resolution for rendering and OCR overlay.
    """
    logger.warning(
        "PDF.save_searchable() is deprecated. Use PDF.save_pdf(..., ocr=True) instead."
    )
    if create_searchable_pdf is None:
        raise ImportError(
            "Saving searchable PDF requires 'pikepdf'. "
            'Install with: pip install "natural-pdf[ocr-export]"'
        )
    output_path_str = str(output_path)
    # Call the exporter directly, passing self (the PDF instance)
    create_searchable_pdf(self, output_path_str, dpi=dpi)

natural_pdf.PDF.search_within_index(query, search_service, options=None)

Finds relevant documents from this PDF within a search index. Finds relevant documents from this PDF within a search index.

Parameters:

Name	Type	Description	Default
query	Union[str, Path, Image, Region]	The search query (text, image path, PIL Image, Region)	required
search_service	SearchServiceProtocol	A pre-configured SearchService instance	required
options	Optional[SearchOptions]	Optional SearchOptions to configure the query	None
query	Union[str, Path, Image, Region]	The search query (text, image path, PIL Image, Region)	required
search_service	SearchServiceProtocol	A pre-configured SearchService instance	required
options	Optional[SearchOptions]	Optional SearchOptions to configure the query	None

Returns:

Type	Description
List[Dict[str, Any]]	A list of result dictionaries, sorted by relevance
List[Dict[str, Any]]	A list of result dictionaries, sorted by relevance

Raises:

Type	Description
ImportError	If search dependencies are not installed
ValueError	If search_service is None
TypeError	If search_service does not conform to the protocol
FileNotFoundError	If the collection managed by the service does not exist
RuntimeError	For other search failures
ImportError	If search dependencies are not installed
ValueError	If search_service is None
TypeError	If search_service does not conform to the protocol
FileNotFoundError	If the collection managed by the service does not exist
RuntimeError	For other search failures

Source code in natural_pdf/core/pdf.py

def search_within_index(
    self,
    query: Union[str, Path, Image.Image, "Region"],
    search_service: "SearchServiceProtocol",
    options: Optional["SearchOptions"] = None,
) -> List[Dict[str, Any]]:
    """
    Finds relevant documents from this PDF within a search index.
    Finds relevant documents from this PDF within a search index.

    Args:
        query: The search query (text, image path, PIL Image, Region)
        search_service: A pre-configured SearchService instance
        options: Optional SearchOptions to configure the query
        query: The search query (text, image path, PIL Image, Region)
        search_service: A pre-configured SearchService instance
        options: Optional SearchOptions to configure the query

    Returns:
        A list of result dictionaries, sorted by relevance
        A list of result dictionaries, sorted by relevance

    Raises:
        ImportError: If search dependencies are not installed
        ValueError: If search_service is None
        TypeError: If search_service does not conform to the protocol
        FileNotFoundError: If the collection managed by the service does not exist
        RuntimeError: For other search failures
        ImportError: If search dependencies are not installed
        ValueError: If search_service is None
        TypeError: If search_service does not conform to the protocol
        FileNotFoundError: If the collection managed by the service does not exist
        RuntimeError: For other search failures
    """
    if not search_service:
        raise ValueError("A configured SearchServiceProtocol instance must be provided.")

    collection_name = getattr(search_service, "collection_name", "<Unknown Collection>")
    logger.info(
        f"Searching within index '{collection_name}' for content from PDF '{self.path}'"
    )

    service = search_service

    query_input = query
    effective_options = copy.deepcopy(options) if options is not None else TextSearchOptions()

    if isinstance(query, Region):
        logger.debug("Query is a Region object. Extracting text.")
        if not isinstance(effective_options, TextSearchOptions):
            logger.warning(
                "Querying with Region image requires MultiModalSearchOptions. Falling back to text extraction."
            )
        query_input = query.extract_text()
        if not query_input or query_input.isspace():
            logger.error("Region has no extractable text for query.")
            return []

    # Add filter to scope search to THIS PDF
    # Add filter to scope search to THIS PDF
    pdf_scope_filter = {
        "field": "pdf_path",
        "operator": "eq",
        "value": self.path,
    }
    logger.debug(f"Applying filter to scope search to PDF: {pdf_scope_filter}")

    # Combine with existing filters in options (if any)
    if effective_options.filters:
        logger.debug("Combining PDF scope filter with existing filters")
        if (
            isinstance(effective_options.filters, dict)
            and effective_options.filters.get("operator") == "AND"
        ):
            effective_options.filters["conditions"].append(pdf_scope_filter)
        elif isinstance(effective_options.filters, list):
            effective_options.filters = {
                "operator": "AND",
                "conditions": effective_options.filters + [pdf_scope_filter],
            }
        elif isinstance(effective_options.filters, dict):
            effective_options.filters = {
                "operator": "AND",
                "conditions": [effective_options.filters, pdf_scope_filter],
            }
        else:
            logger.warning(
                "Unsupported format for existing filters. Overwriting with PDF scope filter."
            )
            effective_options.filters = pdf_scope_filter
    else:
        effective_options.filters = pdf_scope_filter

    logger.debug(f"Final filters for service search: {effective_options.filters}")

    try:
        results = service.search(
            query=query_input,
            options=effective_options,
        )
        logger.info(f"SearchService returned {len(results)} results from PDF '{self.path}'")
        return results
    except FileNotFoundError as fnf:
        logger.error(f"Search failed: Collection not found. Error: {fnf}")
        raise
        logger.error(f"Search failed: Collection not found. Error: {fnf}")
        raise
    except Exception as e:
        logger.error(f"SearchService search failed: {e}")
        raise RuntimeError("Search within index failed. See logs for details.") from e
        logger.error(f"SearchService search failed: {e}")
        raise RuntimeError("Search within index failed. See logs for details.") from e

natural_pdf.PDF.split(divider, *, include_boundaries='start', orientation='vertical', new_section_on_page_break=False)

Divide the PDF into sections based on the provided divider elements.

Parameters:

Name	Type	Description	Default
divider		Elements or selector string that mark section boundaries	required
include_boundaries	str	How to include boundary elements (default: 'start').	'start'
orientation	str	'vertical' or 'horizontal' (default: 'vertical').	'vertical'
new_section_on_page_break	bool	Whether to split at page boundaries (default: False).	False

Returns:

Type	Description
ElementCollection	ElementCollection of Region objects representing the sections

Example

Split a PDF by chapter titles

chapters = pdf.split("text[size>20]:contains('Chapter')")

Export each chapter to a separate file

for i, chapter in enumerate(chapters): chapter_text = chapter.extract_text() with open(f"chapter_{i+1}.txt", "w") as f: f.write(chapter_text)

Split by horizontal rules/lines

sections = pdf.split("line[orientation=horizontal]")

Split only by page breaks (no divider elements)

pages = pdf.split(None, new_section_on_page_break=True)

Source code in natural_pdf/core/pdf.py

def split(
    self,
    divider,
    *,
    include_boundaries: str = "start",
    orientation: str = "vertical",
    new_section_on_page_break: bool = False,
) -> "ElementCollection":
    """
    Divide the PDF into sections based on the provided divider elements.

    Args:
        divider: Elements or selector string that mark section boundaries
        include_boundaries: How to include boundary elements (default: 'start').
        orientation: 'vertical' or 'horizontal' (default: 'vertical').
        new_section_on_page_break: Whether to split at page boundaries (default: False).

    Returns:
        ElementCollection of Region objects representing the sections

    Example:
        # Split a PDF by chapter titles
        chapters = pdf.split("text[size>20]:contains('Chapter')")

        # Export each chapter to a separate file
        for i, chapter in enumerate(chapters):
            chapter_text = chapter.extract_text()
            with open(f"chapter_{i+1}.txt", "w") as f:
                f.write(chapter_text)

        # Split by horizontal rules/lines
        sections = pdf.split("line[orientation=horizontal]")

        # Split only by page breaks (no divider elements)
        pages = pdf.split(None, new_section_on_page_break=True)
    """
    # Delegate to pages collection
    return self.pages.split(
        divider,
        include_boundaries=include_boundaries,
        orientation=orientation,
        new_section_on_page_break=new_section_on_page_break,
    )

natural_pdf.PDF.update_text(transform, *, selector='text', apply_exclusions=False, pages=None, max_workers=None, progress_callback=None)

Applies corrections to text elements using a callback function.

Parameters:

Name	Type	Description	Default
transform	Callable[[Any], Optional[str]]	Function that takes an element and returns corrected text or None	required
selector	str	Selector to apply corrections to (default: "text")	'text'
apply_exclusions	bool	Whether to honour exclusion regions while selecting text.	False
pages	Optional[Union[Iterable[int], range, slice]]	Optional page indices/slice to limit the scope of correction	None
max_workers	Optional[int]	Maximum number of threads to use for parallel execution	None
progress_callback	Optional[Callable[[], None]]	Optional callback function for progress updates	None

Returns:

Type	Description
PDF	Self for method chaining

Source code in natural_pdf/core/pdf.py

def update_text(
    self,
    transform: Callable[[Any], Optional[str]],
    *,
    selector: str = "text",
    apply_exclusions: bool = False,
    pages: Optional[Union[Iterable[int], range, slice]] = None,
    max_workers: Optional[int] = None,
    progress_callback: Optional[Callable[[], None]] = None,
) -> "PDF":
    """
    Applies corrections to text elements using a callback function.

    Args:
        transform: Function that takes an element and returns corrected text or None
        selector: Selector to apply corrections to (default: "text")
        apply_exclusions: Whether to honour exclusion regions while selecting text.
        pages: Optional page indices/slice to limit the scope of correction
        max_workers: Maximum number of threads to use for parallel execution
        progress_callback: Optional callback function for progress updates

    Returns:
        Self for method chaining
    """
    pages_to_update = self._get_target_pages(pages)

    if not pages_to_update:
        logger.warning("No pages selected for text update.")
        return self

    page_identities = [page.index for page in pages_to_update]
    logger.info(f"Starting text update for pages: {page_identities} with selector='{selector}'")

    for page in pages_to_update:
        try:
            page.update_text(
                transform=transform,
                selector=selector,
                apply_exclusions=apply_exclusions,
                max_workers=max_workers,
                progress_callback=progress_callback,
            )
        except Exception as e:
            logger.error(f"Error during text update on page {page.index}: {e}")
            logger.error(f"Error during text update on page {page.index}: {e}")

    logger.info("Text update process finished.")
    return self

natural_pdf.PDFCollection

Bases: ApplyMixin, ExportMixin, ShapeDetectionMixin, VisualSearchMixin

Source code in natural_pdf/core/pdf_collection.py

class PDFCollection(ApplyMixin, ExportMixin, ShapeDetectionMixin, VisualSearchMixin):
    def __init__(
        self,
        source: Union[str, Iterable[Union[str, "PDF"]]],
        recursive: bool = True,
        **pdf_options: Any,
    ):
        """
        Initializes a collection of PDF documents from various sources.

        Args:
            source: The source of PDF documents. Can be:
                - An iterable (e.g., list) of existing PDF objects.
                - An iterable (e.g., list) of file paths/URLs/globs (strings).
                - A single file path/URL/directory/glob string.
            recursive: If source involves directories or glob patterns,
                       whether to search recursively (default: True).
            **pdf_options: Keyword arguments passed to the PDF constructor.
        """
        self._pdfs: List["PDF"] = []
        self._pdf_options = pdf_options  # Store options for potential slicing later
        self._recursive = recursive  # Store setting for potential slicing

        # Dynamically import PDF class within methods to avoid circular import at module load time
        PDF = self._get_pdf_class()

        if hasattr(source, "__iter__") and not isinstance(source, str):
            source_list = list(source)
            if not source_list:
                return  # Empty list source
            if isinstance(source_list[0], PDF):
                if all(isinstance(item, PDF) for item in source_list):
                    self._pdfs = [cast("PDF", item) for item in source_list]
                    # Don't adopt search context anymore
                    return
                else:
                    raise TypeError("Iterable source has mixed PDF/non-PDF objects.")
            # If it's an iterable but not PDFs, fall through to resolve sources

        # Resolve string, iterable of strings, or single string source to paths/URLs
        resolved_paths_or_urls = self._resolve_sources_to_paths(
            cast(Union[str, Iterable[str]], source)
        )
        self._initialize_pdfs(resolved_paths_or_urls, PDF)  # Pass PDF class

        self._iter_index = 0

        # Initialize internal search service reference (available when search extras installed)
        self._search_service: Optional[Any] = None

    @staticmethod
    def _get_pdf_class():
        """Helper method to dynamically import the PDF class."""
        from natural_pdf.core.pdf import PDF

        return PDF

    # --- Internal Helpers ---

    def _is_url(self, s: str) -> bool:
        return s.startswith(("http://", "https://"))

    def _has_glob_magic(self, s: str) -> bool:
        return py_glob.has_magic(s)

    def _execute_glob(self, pattern: str) -> Set[str]:
        """Glob for paths and return a set of valid PDF paths."""
        found_paths = set()
        # Use iglob for potentially large directories/matches
        paths_iter = py_glob.iglob(pattern, recursive=self._recursive)
        for path_str in paths_iter:
            # Use Path object for easier checking
            p = Path(path_str)
            if p.is_file() and p.suffix.lower() == ".pdf":
                found_paths.add(str(p.resolve()))  # Store resolved absolute path
        return found_paths

    def _resolve_sources_to_paths(self, source: Union[str, Iterable[str]]) -> List[str]:
        """Resolves various source types into a list of unique PDF paths/URLs."""
        final_paths = set()
        sources_to_process = []

        if isinstance(source, str):
            sources_to_process.append(source)
        elif hasattr(source, "__iter__"):
            sources_to_process.extend(list(source))
        else:  # Should not happen based on __init__ checks, but safeguard
            raise TypeError(f"Unexpected source type in _resolve_sources_to_paths: {type(source)}")

        for item in sources_to_process:
            if not isinstance(item, str):
                logger.warning(f"Skipping non-string item in source list: {type(item)}")
                continue

            item_path = Path(item)

            if self._is_url(item):
                final_paths.add(item)  # Add URL directly
            elif self._has_glob_magic(item):
                glob_results = self._execute_glob(item)
                final_paths.update(glob_results)
            elif item_path.is_dir():
                # Use glob to find PDFs in directory, respecting recursive flag
                dir_pattern = (
                    str(item_path / "**" / "*.pdf") if self._recursive else str(item_path / "*.pdf")
                )
                dir_glob_results = self._execute_glob(dir_pattern)
                final_paths.update(dir_glob_results)
            elif item_path.is_file() and item_path.suffix.lower() == ".pdf":
                final_paths.add(str(item_path.resolve()))  # Add resolved file path
            else:
                logger.warning(
                    f"Source item ignored (not a valid URL, directory, file, or glob): {item}"
                )

        return sorted(list(final_paths))

    def _initialize_pdfs(self, paths_or_urls: List[str], PDF_cls: Type["PDF"]):
        """Initializes PDF objects from a list of paths/URLs."""
        logger.info(f"Initializing {len(paths_or_urls)} PDF objects...")
        failed_count = 0
        for path_or_url in tqdm(paths_or_urls, desc="Loading PDFs"):
            try:
                pdf_instance = PDF_cls(path_or_url, **self._pdf_options)
                self._pdfs.append(pdf_instance)
            except Exception as e:
                logger.error(
                    f"Failed to load PDF: {path_or_url}. Error: {e}", exc_info=False
                )  # Keep log concise
                failed_count += 1
        logger.info(f"Successfully initialized {len(self._pdfs)} PDFs. Failed: {failed_count}")

    # --- Public Factory Class Methods (Simplified) ---

    @classmethod
    def from_paths(cls, paths_or_urls: List[str], **pdf_options: Any) -> "PDFCollection":
        """Creates a PDFCollection explicitly from a list of file paths or URLs."""
        # __init__ can handle List[str] directly now
        return cls(paths_or_urls, **pdf_options)

    @classmethod
    def from_glob(cls, pattern: str, recursive: bool = True, **pdf_options: Any) -> "PDFCollection":
        """Creates a PDFCollection explicitly from a single glob pattern."""
        # __init__ can handle single glob string directly
        return cls(pattern, recursive=recursive, **pdf_options)

    @classmethod
    def from_globs(
        cls, patterns: List[str], recursive: bool = True, **pdf_options: Any
    ) -> "PDFCollection":
        """Creates a PDFCollection explicitly from a list of glob patterns."""
        # __init__ can handle List[str] containing globs directly
        return cls(patterns, recursive=recursive, **pdf_options)

    @classmethod
    def from_directory(
        cls, directory_path: str, recursive: bool = True, **pdf_options: Any
    ) -> "PDFCollection":
        """Creates a PDFCollection explicitly from PDF files within a directory."""
        # __init__ can handle single directory string directly
        return cls(directory_path, recursive=recursive, **pdf_options)

    # --- Core Collection Methods ---
    def __len__(self) -> int:
        return len(self._pdfs)

    def __getitem__(self, key) -> Union["PDF", "PDFCollection"]:
        # Use dynamic import here as well
        PDF = self._get_pdf_class()
        if isinstance(key, slice):
            # Create a new collection with the sliced PDFs and original options
            new_collection = PDFCollection.__new__(PDFCollection)  # Create blank instance
            new_collection._pdfs = self._pdfs[key]
            new_collection._pdf_options = self._pdf_options
            new_collection._recursive = self._recursive
            # Search context is not copied/inherited anymore
            return new_collection
        elif isinstance(key, int):
            # Check bounds
            if 0 <= key < len(self._pdfs):
                return self._pdfs[key]
            else:
                raise IndexError(f"PDF index {key} out of range (0-{len(self._pdfs)-1}).")
        else:
            raise TypeError(f"PDF indices must be integers or slices, not {type(key)}.")

    def __iter__(self):
        return iter(self._pdfs)

    def __repr__(self) -> str:
        return f"<PDFCollection(count={len(self._pdfs)})>"

    @property
    def pdfs(self) -> List["PDF"]:
        """Returns the list of PDF objects held by the collection."""
        return self._pdfs

    def show(self, limit: Optional[int] = 30, per_pdf_limit: Optional[int] = 10, **kwargs):
        """
        Display all PDFs in the collection with labels.

        Each PDF is shown with its pages in a grid layout (6 columns by default),
        and all PDFs are stacked vertically with labels.

        Args:
            limit: Maximum total pages to show across all PDFs (default: 30)
            per_pdf_limit: Maximum pages to show per PDF (default: 10)
            **kwargs: Additional arguments passed to each PDF's show() method
                     (e.g., columns, exclusions, resolution, etc.)

        Returns:
            Displayed image in Jupyter or None
        """
        if not self._pdfs:
            print("Empty collection")
            return None

        # Import here to avoid circular imports
        from PIL import ImageDraw, ImageFont

        # Calculate pages per PDF if total limit is set
        if limit and not per_pdf_limit:
            per_pdf_limit = max(1, limit // len(self._pdfs))

        # Collect images from each PDF
        all_images = []
        total_pages_shown = 0

        for pdf in self._pdfs:
            if limit and total_pages_shown >= limit:
                break

            # Calculate limit for this PDF
            pdf_limit = per_pdf_limit
            if limit:
                remaining = limit - total_pages_shown
                pdf_limit = min(per_pdf_limit or remaining, remaining)

            if pdf_limit is None:
                pdf_limit_value = len(pdf.pages)
            else:
                pdf_limit_value = pdf_limit

            # Get PDF identifier
            pdf_name = getattr(pdf, "filename", None) or getattr(pdf, "path", "Unknown")
            if isinstance(pdf_name, Path):
                pdf_name = pdf_name.name
            elif "/" in str(pdf_name):
                pdf_name = str(pdf_name).split("/")[-1]

            # Render this PDF
            try:
                # Get render specs from the PDF
                render_specs = pdf._get_render_specs(
                    mode="show", max_pages=pdf_limit_value, **kwargs
                )

                if not render_specs:
                    continue

                # Get the highlighter and render without displaying
                highlighter = cast("HighlightingService", pdf._get_highlighter())
                pdf_image = highlighter.unified_render(
                    specs=render_specs,
                    layout="grid" if len(render_specs) > 1 else "single",
                    columns=6,
                    **kwargs,
                )

                if pdf_image:
                    # Add label above the PDF image
                    label_height = 40
                    label_bg_color = (240, 240, 240)
                    label_text_color = (0, 0, 0)

                    # Create new image with space for label
                    width, height = pdf_image.size
                    labeled_image = Image.new("RGB", (width, height + label_height), "white")

                    # Draw label background
                    draw = ImageDraw.Draw(labeled_image)
                    draw.rectangle([0, 0, width, label_height], fill=label_bg_color)

                    # Draw label text
                    try:
                        # Try to use a nice font if available
                        font = ImageFont.truetype("Arial", 20)
                    except:
                        # Fallback to default font
                        font = ImageFont.load_default()

                    label_text = f"{pdf_name} ({len(pdf.pages)} pages)"
                    draw.text((10, 10), label_text, fill=label_text_color, font=font)

                    # Paste PDF image below label
                    labeled_image.paste(pdf_image, (0, label_height))

                    all_images.append(labeled_image)
                    if pdf_limit is None:
                        pdf_limit = len(pdf.pages)
                    total_pages_shown += min(pdf_limit_value, len(pdf.pages))

            except Exception as e:
                logger.warning(f"Failed to render PDF {pdf_name}: {e}")
                continue

        if not all_images:
            print("No PDFs could be rendered")
            return None

        # Combine all images vertically
        if len(all_images) == 1:
            combined = all_images[0]
        else:
            # Add spacing between PDFs
            spacing = 20
            total_height = sum(img.height for img in all_images) + spacing * (len(all_images) - 1)
            max_width = max(img.width for img in all_images)

            combined = Image.new("RGB", (max_width, total_height), "white")

            y_offset = 0
            for i, img in enumerate(all_images):
                # Center images if they're narrower than max width
                x_offset = (max_width - img.width) // 2
                combined.paste(img, (x_offset, y_offset))
                y_offset += img.height
                if i < len(all_images) - 1:
                    y_offset += spacing

        # Return the combined image (Jupyter will display it automatically)
        return combined

    @overload
    def find_all(
        self,
        *,
        text: Union[str, Sequence[str]],
        apply_exclusions: bool = True,
        regex: bool = False,
        case: bool = True,
        **kwargs,
    ) -> "ElementCollection": ...

    @overload
    def find_all(
        self,
        selector: str,
        *,
        apply_exclusions: bool = True,
        regex: bool = False,
        case: bool = True,
        **kwargs,
    ) -> "ElementCollection": ...

    def find_all(
        self,
        selector: Optional[str] = None,  # Now optional
        *,
        text: Optional[Union[str, Sequence[str]]] = None,  # New text parameter
        apply_exclusions: bool = True,
        regex: bool = False,
        case: bool = True,
        **kwargs,
    ) -> "ElementCollection":
        """
        Find all elements matching the selector OR text across all PDFs in the collection.

        Provide EITHER `selector` OR `text`, but not both.

        This creates an ElementCollection that can span multiple PDFs. Note that
        some ElementCollection methods have limitations when spanning PDFs.

        Args:
            selector: CSS-like selector string to query elements.
            text: Text content to search for (equivalent to 'text:contains(...)'). Accepts a
                  single string or an iterable of strings (matches any value).
            apply_exclusions: Whether to exclude elements in exclusion regions (default: True).
            regex: Whether to use regex for text search (`selector` or `text`) (default: False).
            case: Whether to do case-sensitive text search (`selector` or `text`) (default: True).
            **kwargs: Additional keyword arguments passed to the find_all method of each PDF.

        Returns:
            ElementCollection containing all matching elements across all PDFs.
        """
        # Validation happens within pdf.find_all

        # Collect elements from all PDFs
        all_elements = []
        for pdf in self._pdfs:
            try:
                # Pass the relevant arguments down to each PDF's find_all
                elements = pdf.find_all(
                    selector=selector,
                    text=text,
                    apply_exclusions=apply_exclusions,
                    regex=regex,
                    case=case,
                    **kwargs,
                )
                all_elements.extend(elements.elements)
            except Exception as e:
                logger.error(f"Error finding elements in {pdf.path}: {e}", exc_info=True)

        return ElementCollection(all_elements)

    def apply_ocr(
        self,
        engine: Optional[str] = None,
        languages: Optional[List[str]] = None,
        min_confidence: Optional[float] = None,
        device: Optional[str] = None,
        resolution: Optional[int] = None,
        apply_exclusions: bool = True,
        detect_only: bool = False,
        replace: bool = True,
        options: Optional[Any] = None,
        pages: Optional[Union[slice, List[int]]] = None,
        max_workers: Optional[int] = None,
    ) -> "PDFCollection":
        """
        Apply OCR to all PDFs in the collection, potentially in parallel.

        Args:
            engine: OCR engine to use (e.g., 'easyocr', 'paddleocr', 'surya')
            languages: List of language codes for OCR
            min_confidence: Minimum confidence threshold for text detection
            device: Device to use for OCR (e.g., 'cpu', 'cuda')
            resolution: DPI resolution for page rendering
            apply_exclusions: Whether to apply exclusion regions
            detect_only: If True, only detect text regions without extracting text
            replace: If True, replace existing OCR elements
            options: Engine-specific options
            pages: Specific pages to process (None for all pages)
            max_workers: Maximum number of threads to process PDFs concurrently.
                         If None or 1, processing is sequential. (default: None)

        Returns:
            Self for method chaining
        """
        PDF = self._get_pdf_class()
        logger.info(
            f"Applying OCR to {len(self._pdfs)} PDFs in collection (max_workers={max_workers})..."
        )

        # Worker function takes PDF object again
        def _process_pdf(pdf: "PDF"):
            """Helper function to apply OCR to a single PDF, handling errors."""
            thread_id = threading.current_thread().name  # Get thread name for logging
            pdf_path = pdf.path  # Get path for logging
            logger.debug(f"[{thread_id}] Starting OCR process for: {pdf_path}")
            start_time = time.monotonic()
            pdf.apply_ocr(  # Call apply_ocr on the original PDF object
                pages=pages,
                engine=engine,
                languages=languages,
                min_confidence=min_confidence,
                device=device,
                resolution=resolution,
                apply_exclusions=apply_exclusions,
                detect_only=detect_only,
                replace=replace,
                options=options,
                # Note: We might want a max_workers here too for page rendering?
                # For now, PDF.apply_ocr doesn't have it.
            )
            end_time = time.monotonic()
            logger.debug(
                f"[{thread_id}] Finished OCR process for: {pdf_path} (Duration: {end_time - start_time:.2f}s)"
            )
            return pdf_path, None

        # Use ThreadPoolExecutor for parallel processing if max_workers > 1
        if max_workers is not None and max_workers > 1:
            futures = []
            with concurrent.futures.ThreadPoolExecutor(
                max_workers=max_workers, thread_name_prefix="OCRWorker"
            ) as executor:
                for pdf in self._pdfs:
                    # Submit the PDF object to the worker function
                    futures.append(executor.submit(_process_pdf, pdf))

            # Use the selected tqdm class with as_completed for progress tracking
            progress_bar = tqdm(
                concurrent.futures.as_completed(futures),
                total=len(self._pdfs),
                desc="Applying OCR (Parallel)",
                unit="pdf",
            )

            for future in progress_bar:
                pdf_path, error = future.result()  # Get result (or exception)
                if error:
                    progress_bar.set_postfix_str(f"Error: {pdf_path}", refresh=True)
                # Progress is updated automatically by tqdm

        else:  # Sequential processing (max_workers is None or 1)
            logger.info("Applying OCR sequentially...")
            # Use the selected tqdm class for sequential too for consistency
            # Iterate over PDF objects directly for sequential
            for pdf in tqdm(self._pdfs, desc="Applying OCR (Sequential)", unit="pdf"):
                _process_pdf(pdf)  # Call helper directly with PDF object

        logger.info("Finished applying OCR across the collection.")
        return self

    def correct_ocr(
        self,
        correction_callback: Callable[[Any], Optional[str]],
        max_workers: Optional[int] = None,
        progress_callback: Optional[Callable[[], None]] = None,
    ) -> "PDFCollection":
        """
        Apply OCR correction to all relevant elements across all pages and PDFs
        in the collection using a single progress bar.

        Args:
            correction_callback: Function to apply to each OCR element.
                                 It receives the element and should return
                                 the corrected text (str) or None.
            max_workers: Max threads to use for parallel execution within each page.
            progress_callback: Optional callback function to call after processing each element.

        Returns:
            Self for method chaining.
        """
        PDF = self._get_pdf_class()  # Ensure PDF class is available
        if not callable(correction_callback):
            raise TypeError("`correction_callback` must be a callable function.")

        logger.info(f"Gathering OCR elements from {len(self._pdfs)} PDFs for correction...")

        # 1. Gather all target elements using the collection's find_all
        #    Crucially, set apply_exclusions=False to include elements in headers/footers etc.
        all_ocr_elements = self.find_all("text[source=ocr]", apply_exclusions=False).elements

        if not all_ocr_elements:
            logger.info("No OCR elements found in the collection to correct.")
            return self

        total_elements = len(all_ocr_elements)
        logger.info(
            f"Found {total_elements} OCR elements across the collection. Starting correction process..."
        )

        # 2. Initialize the progress bar
        progress_bar = tqdm(total=total_elements, desc="Correcting OCR Elements", unit="element")

        def _tick_progress() -> None:
            progress_bar.update()

        for pdf in self._pdfs:
            if not pdf.pages:
                continue
            for page in pdf.pages:
                try:
                    page.update_text(
                        transform=correction_callback,
                        selector="text[source=ocr]",
                        apply_exclusions=False,
                        max_workers=max_workers,
                        progress_callback=_tick_progress,
                    )
                except Exception as e:
                    logger.error(
                        f"Error occurred during correction process for page {getattr(page, 'number', '?')} of PDF {getattr(pdf, 'path', 'unknown')}: {e}",
                        exc_info=True,
                    )
                    continue

        progress_bar.close()

        return self

    def categorize(self, labels: List[str], **kwargs):
        """Categorizes PDFs in the collection based on content or features."""
        # Implementation requires integrating with classification models or logic
        raise NotImplementedError("categorize requires classification implementation.")

    def export_ocr_correction_task(self, output_zip_path: str, **kwargs):
        """
        Exports OCR results from all PDFs in this collection into a single
        correction task package (zip file).

        Args:
            output_zip_path: The path to save the output zip file.
            **kwargs: Additional arguments passed to create_correction_task_package
                      (e.g., image_render_scale, overwrite).
        """
        from natural_pdf.utils.packaging import create_correction_task_package

        # Pass the collection itself (self) as the source
        create_correction_task_package(source=self, output_zip_path=output_zip_path, **kwargs)

    # --- Mixin Required Implementation ---
    def get_indexable_items(self) -> Iterable[Indexable]:
        """Yields Page objects from the collection, conforming to Indexable."""
        if not self._pdfs:
            return  # Return empty iterator if no PDFs

        for pdf in self._pdfs:
            if not pdf.pages:  # Handle case where a PDF might have 0 pages after loading
                logger.warning(f"PDF '{pdf.path}' has no pages. Skipping.")
                continue
            for page in pdf.pages:
                # Optional: Add filtering here if needed (e.g., skip empty pages)
                # Assuming Page object conforms to Indexable
                # We might still want the empty page check here for efficiency
                # if not page.extract_text(use_exclusions=False).strip():
                #     logger.debug(f"Skipping empty page {page.page_number} from PDF '{pdf.path}'.")
                #     continue
                yield page

    # --- Classification Method --- #
    def classify_all(
        self,
        labels: List[str],
        using: Optional[str] = None,
        model: Optional[str] = None,
        analysis_key: str = "classification",
        **kwargs,
    ) -> "PDFCollection":
        """
        Classify each PDF document in the collection using provider-backed batch processing.
        """
        if not labels:
            raise ValueError("Labels list cannot be empty.")

        if not self._pdfs:
            logger.warning("PDFCollection is empty, skipping classification.")
            return self

        mode_desc = f"using='{using}'" if using else f"model='{model}'" if model else "default text"
        logger.info(
            f"Starting batch classification for {len(self._pdfs)} PDFs in collection ({mode_desc})..."
        )

        first_pdf = self._pdfs[0]
        engine_name = kwargs.pop("classification_engine", None)
        engine_obj = get_classification_engine(first_pdf, engine_name)
        inferred_using = engine_obj.infer_using(model or engine_obj.default_model("text"), using)

        pdf_contents: List[Any] = []
        valid_pdfs: List[Any] = []

        logger.info(f"Gathering content from {len(self._pdfs)} PDFs for batch classification...")
        for pdf in self._pdfs:
            try:
                content = pdf._get_classification_content(model_type=inferred_using, **kwargs)
                pdf_contents.append(content)
                valid_pdfs.append(pdf)
            except Exception as exc:
                logger.warning(f"Skipping PDF {pdf.path}: {exc}")

        if not pdf_contents:
            logger.warning("No valid content could be gathered from PDFs for classification.")
            return self

        try:
            batch_results = run_classification_batch(
                context=self,
                contents=pdf_contents,
                labels=labels,
                model_id=model or engine_obj.default_model(inferred_using),
                using=inferred_using,
                min_confidence=kwargs.get("min_confidence", 0.0),
                multi_label=kwargs.get("multi_label", False),
                batch_size=8,
                progress_bar=True,
                engine_name=engine_name,
            )
        except Exception as exc:
            raise ClassificationError(f"Batch classification failed: {exc}") from exc

        if len(batch_results) != len(valid_pdfs):
            raise ClassificationError("Batch result count mismatch with input PDFs")

        processed_count = 0
        for pdf, result_obj in zip(valid_pdfs, batch_results):
            try:
                if not hasattr(pdf, "analyses") or pdf.analyses is None:
                    pdf.analyses = {}
                pdf.analyses[analysis_key] = result_obj
                processed_count += 1
            except Exception as exc:
                logger.warning(f"Failed to store classification result for {pdf.path}: {exc}")

        skipped_count = len(self._pdfs) - processed_count
        final_message = f"Finished batch classification. Processed: {processed_count}"
        if skipped_count > 0:
            final_message += f", Skipped: {skipped_count}"
        logger.info(final_message + ".")

        return self

    # --- End Classification Method --- #

    def _gather_analysis_data(
        self,
        analysis_keys: List[str],
        include_content: bool,
        include_images: bool,
        image_dir: Optional[Path],
        image_format: str,
        image_resolution: int,
    ) -> List[Dict[str, Any]]:
        """
        Gather analysis data from all PDFs in the collection.

        Args:
            analysis_keys: Keys in the analyses dictionary to export
            include_content: Whether to include extracted text
            include_images: Whether to export images
            image_dir: Directory to save images
            image_format: Format to save images
            image_resolution: Resolution for exported images

        Returns:
            List of dictionaries containing analysis data
        """
        if not self._pdfs:
            logger.warning("No PDFs found in collection")
            return []

        all_data = []

        for pdf in tqdm(self._pdfs, desc="Gathering PDF data", leave=False):
            # PDF level data
            pdf_data = {
                "pdf_path": pdf.path,
                "pdf_filename": Path(pdf.path).name,
                "total_pages": len(pdf.pages) if hasattr(pdf, "pages") else 0,
            }

            # Add metadata if available
            if hasattr(pdf, "metadata") and pdf.metadata:
                for k, v in pdf.metadata.items():
                    if v:  # Only add non-empty metadata
                        pdf_data[f"metadata.{k}"] = str(v)

            all_data.append(pdf_data)

        return all_data

Attributes

natural_pdf.PDFCollection.pdfs property

Returns the list of PDF objects held by the collection.

Functions

natural_pdf.PDFCollection.__init__(source, recursive=True, **pdf_options)

Initializes a collection of PDF documents from various sources.

Parameters:

Name	Type	Description	Default
source	Union[str, Iterable[Union[str, PDF]]]	The source of PDF documents. Can be: - An iterable (e.g., list) of existing PDF objects. - An iterable (e.g., list) of file paths/URLs/globs (strings). - A single file path/URL/directory/glob string.	required
recursive	bool	If source involves directories or glob patterns, whether to search recursively (default: True).	True
**pdf_options	Any	Keyword arguments passed to the PDF constructor.	{}

Source code in natural_pdf/core/pdf_collection.py

def __init__(
    self,
    source: Union[str, Iterable[Union[str, "PDF"]]],
    recursive: bool = True,
    **pdf_options: Any,
):
    """
    Initializes a collection of PDF documents from various sources.

    Args:
        source: The source of PDF documents. Can be:
            - An iterable (e.g., list) of existing PDF objects.
            - An iterable (e.g., list) of file paths/URLs/globs (strings).
            - A single file path/URL/directory/glob string.
        recursive: If source involves directories or glob patterns,
                   whether to search recursively (default: True).
        **pdf_options: Keyword arguments passed to the PDF constructor.
    """
    self._pdfs: List["PDF"] = []
    self._pdf_options = pdf_options  # Store options for potential slicing later
    self._recursive = recursive  # Store setting for potential slicing

    # Dynamically import PDF class within methods to avoid circular import at module load time
    PDF = self._get_pdf_class()

    if hasattr(source, "__iter__") and not isinstance(source, str):
        source_list = list(source)
        if not source_list:
            return  # Empty list source
        if isinstance(source_list[0], PDF):
            if all(isinstance(item, PDF) for item in source_list):
                self._pdfs = [cast("PDF", item) for item in source_list]
                # Don't adopt search context anymore
                return
            else:
                raise TypeError("Iterable source has mixed PDF/non-PDF objects.")
        # If it's an iterable but not PDFs, fall through to resolve sources

    # Resolve string, iterable of strings, or single string source to paths/URLs
    resolved_paths_or_urls = self._resolve_sources_to_paths(
        cast(Union[str, Iterable[str]], source)
    )
    self._initialize_pdfs(resolved_paths_or_urls, PDF)  # Pass PDF class

    self._iter_index = 0

    # Initialize internal search service reference (available when search extras installed)
    self._search_service: Optional[Any] = None

natural_pdf.PDFCollection.apply_ocr(engine=None, languages=None, min_confidence=None, device=None, resolution=None, apply_exclusions=True, detect_only=False, replace=True, options=None, pages=None, max_workers=None)

Apply OCR to all PDFs in the collection, potentially in parallel.

Parameters:

Name	Type	Description	Default
engine	Optional[str]	OCR engine to use (e.g., 'easyocr', 'paddleocr', 'surya')	None
languages	Optional[List[str]]	List of language codes for OCR	None
min_confidence	Optional[float]	Minimum confidence threshold for text detection	None
device	Optional[str]	Device to use for OCR (e.g., 'cpu', 'cuda')	None
resolution	Optional[int]	DPI resolution for page rendering	None
apply_exclusions	bool	Whether to apply exclusion regions	True
detect_only	bool	If True, only detect text regions without extracting text	False
replace	bool	If True, replace existing OCR elements	True
options	Optional[Any]	Engine-specific options	None
pages	Optional[Union[slice, List[int]]]	Specific pages to process (None for all pages)	None
max_workers	Optional[int]	Maximum number of threads to process PDFs concurrently. If None or 1, processing is sequential. (default: None)	None

Returns:

Type	Description
PDFCollection	Self for method chaining

Source code in natural_pdf/core/pdf_collection.py

def apply_ocr(
    self,
    engine: Optional[str] = None,
    languages: Optional[List[str]] = None,
    min_confidence: Optional[float] = None,
    device: Optional[str] = None,
    resolution: Optional[int] = None,
    apply_exclusions: bool = True,
    detect_only: bool = False,
    replace: bool = True,
    options: Optional[Any] = None,
    pages: Optional[Union[slice, List[int]]] = None,
    max_workers: Optional[int] = None,
) -> "PDFCollection":
    """
    Apply OCR to all PDFs in the collection, potentially in parallel.

    Args:
        engine: OCR engine to use (e.g., 'easyocr', 'paddleocr', 'surya')
        languages: List of language codes for OCR
        min_confidence: Minimum confidence threshold for text detection
        device: Device to use for OCR (e.g., 'cpu', 'cuda')
        resolution: DPI resolution for page rendering
        apply_exclusions: Whether to apply exclusion regions
        detect_only: If True, only detect text regions without extracting text
        replace: If True, replace existing OCR elements
        options: Engine-specific options
        pages: Specific pages to process (None for all pages)
        max_workers: Maximum number of threads to process PDFs concurrently.
                     If None or 1, processing is sequential. (default: None)

    Returns:
        Self for method chaining
    """
    PDF = self._get_pdf_class()
    logger.info(
        f"Applying OCR to {len(self._pdfs)} PDFs in collection (max_workers={max_workers})..."
    )

    # Worker function takes PDF object again
    def _process_pdf(pdf: "PDF"):
        """Helper function to apply OCR to a single PDF, handling errors."""
        thread_id = threading.current_thread().name  # Get thread name for logging
        pdf_path = pdf.path  # Get path for logging
        logger.debug(f"[{thread_id}] Starting OCR process for: {pdf_path}")
        start_time = time.monotonic()
        pdf.apply_ocr(  # Call apply_ocr on the original PDF object
            pages=pages,
            engine=engine,
            languages=languages,
            min_confidence=min_confidence,
            device=device,
            resolution=resolution,
            apply_exclusions=apply_exclusions,
            detect_only=detect_only,
            replace=replace,
            options=options,
            # Note: We might want a max_workers here too for page rendering?
            # For now, PDF.apply_ocr doesn't have it.
        )
        end_time = time.monotonic()
        logger.debug(
            f"[{thread_id}] Finished OCR process for: {pdf_path} (Duration: {end_time - start_time:.2f}s)"
        )
        return pdf_path, None

    # Use ThreadPoolExecutor for parallel processing if max_workers > 1
    if max_workers is not None and max_workers > 1:
        futures = []
        with concurrent.futures.ThreadPoolExecutor(
            max_workers=max_workers, thread_name_prefix="OCRWorker"
        ) as executor:
            for pdf in self._pdfs:
                # Submit the PDF object to the worker function
                futures.append(executor.submit(_process_pdf, pdf))

        # Use the selected tqdm class with as_completed for progress tracking
        progress_bar = tqdm(
            concurrent.futures.as_completed(futures),
            total=len(self._pdfs),
            desc="Applying OCR (Parallel)",
            unit="pdf",
        )

        for future in progress_bar:
            pdf_path, error = future.result()  # Get result (or exception)
            if error:
                progress_bar.set_postfix_str(f"Error: {pdf_path}", refresh=True)
            # Progress is updated automatically by tqdm

    else:  # Sequential processing (max_workers is None or 1)
        logger.info("Applying OCR sequentially...")
        # Use the selected tqdm class for sequential too for consistency
        # Iterate over PDF objects directly for sequential
        for pdf in tqdm(self._pdfs, desc="Applying OCR (Sequential)", unit="pdf"):
            _process_pdf(pdf)  # Call helper directly with PDF object

    logger.info("Finished applying OCR across the collection.")
    return self

natural_pdf.PDFCollection.categorize(labels, **kwargs)

Categorizes PDFs in the collection based on content or features.

Source code in natural_pdf/core/pdf_collection.py

def categorize(self, labels: List[str], **kwargs):
    """Categorizes PDFs in the collection based on content or features."""
    # Implementation requires integrating with classification models or logic
    raise NotImplementedError("categorize requires classification implementation.")

natural_pdf.PDFCollection.classify_all(labels, using=None, model=None, analysis_key='classification', **kwargs)

Classify each PDF document in the collection using provider-backed batch processing.

Source code in natural_pdf/core/pdf_collection.py

def classify_all(
    self,
    labels: List[str],
    using: Optional[str] = None,
    model: Optional[str] = None,
    analysis_key: str = "classification",
    **kwargs,
) -> "PDFCollection":
    """
    Classify each PDF document in the collection using provider-backed batch processing.
    """
    if not labels:
        raise ValueError("Labels list cannot be empty.")

    if not self._pdfs:
        logger.warning("PDFCollection is empty, skipping classification.")
        return self

    mode_desc = f"using='{using}'" if using else f"model='{model}'" if model else "default text"
    logger.info(
        f"Starting batch classification for {len(self._pdfs)} PDFs in collection ({mode_desc})..."
    )

    first_pdf = self._pdfs[0]
    engine_name = kwargs.pop("classification_engine", None)
    engine_obj = get_classification_engine(first_pdf, engine_name)
    inferred_using = engine_obj.infer_using(model or engine_obj.default_model("text"), using)

    pdf_contents: List[Any] = []
    valid_pdfs: List[Any] = []

    logger.info(f"Gathering content from {len(self._pdfs)} PDFs for batch classification...")
    for pdf in self._pdfs:
        try:
            content = pdf._get_classification_content(model_type=inferred_using, **kwargs)
            pdf_contents.append(content)
            valid_pdfs.append(pdf)
        except Exception as exc:
            logger.warning(f"Skipping PDF {pdf.path}: {exc}")

    if not pdf_contents:
        logger.warning("No valid content could be gathered from PDFs for classification.")
        return self

    try:
        batch_results = run_classification_batch(
            context=self,
            contents=pdf_contents,
            labels=labels,
            model_id=model or engine_obj.default_model(inferred_using),
            using=inferred_using,
            min_confidence=kwargs.get("min_confidence", 0.0),
            multi_label=kwargs.get("multi_label", False),
            batch_size=8,
            progress_bar=True,
            engine_name=engine_name,
        )
    except Exception as exc:
        raise ClassificationError(f"Batch classification failed: {exc}") from exc

    if len(batch_results) != len(valid_pdfs):
        raise ClassificationError("Batch result count mismatch with input PDFs")

    processed_count = 0
    for pdf, result_obj in zip(valid_pdfs, batch_results):
        try:
            if not hasattr(pdf, "analyses") or pdf.analyses is None:
                pdf.analyses = {}
            pdf.analyses[analysis_key] = result_obj
            processed_count += 1
        except Exception as exc:
            logger.warning(f"Failed to store classification result for {pdf.path}: {exc}")

    skipped_count = len(self._pdfs) - processed_count
    final_message = f"Finished batch classification. Processed: {processed_count}"
    if skipped_count > 0:
        final_message += f", Skipped: {skipped_count}"
    logger.info(final_message + ".")

    return self

natural_pdf.PDFCollection.correct_ocr(correction_callback, max_workers=None, progress_callback=None)

Apply OCR correction to all relevant elements across all pages and PDFs in the collection using a single progress bar.

Parameters:

Name	Type	Description	Default
correction_callback	Callable[[Any], Optional[str]]	Function to apply to each OCR element. It receives the element and should return the corrected text (str) or None.	required
max_workers	Optional[int]	Max threads to use for parallel execution within each page.	None
progress_callback	Optional[Callable[[], None]]	Optional callback function to call after processing each element.	None

Returns:

Type	Description
PDFCollection	Self for method chaining.

Source code in natural_pdf/core/pdf_collection.py

def correct_ocr(
    self,
    correction_callback: Callable[[Any], Optional[str]],
    max_workers: Optional[int] = None,
    progress_callback: Optional[Callable[[], None]] = None,
) -> "PDFCollection":
    """
    Apply OCR correction to all relevant elements across all pages and PDFs
    in the collection using a single progress bar.

    Args:
        correction_callback: Function to apply to each OCR element.
                             It receives the element and should return
                             the corrected text (str) or None.
        max_workers: Max threads to use for parallel execution within each page.
        progress_callback: Optional callback function to call after processing each element.

    Returns:
        Self for method chaining.
    """
    PDF = self._get_pdf_class()  # Ensure PDF class is available
    if not callable(correction_callback):
        raise TypeError("`correction_callback` must be a callable function.")

    logger.info(f"Gathering OCR elements from {len(self._pdfs)} PDFs for correction...")

    # 1. Gather all target elements using the collection's find_all
    #    Crucially, set apply_exclusions=False to include elements in headers/footers etc.
    all_ocr_elements = self.find_all("text[source=ocr]", apply_exclusions=False).elements

    if not all_ocr_elements:
        logger.info("No OCR elements found in the collection to correct.")
        return self

    total_elements = len(all_ocr_elements)
    logger.info(
        f"Found {total_elements} OCR elements across the collection. Starting correction process..."
    )

    # 2. Initialize the progress bar
    progress_bar = tqdm(total=total_elements, desc="Correcting OCR Elements", unit="element")

    def _tick_progress() -> None:
        progress_bar.update()

    for pdf in self._pdfs:
        if not pdf.pages:
            continue
        for page in pdf.pages:
            try:
                page.update_text(
                    transform=correction_callback,
                    selector="text[source=ocr]",
                    apply_exclusions=False,
                    max_workers=max_workers,
                    progress_callback=_tick_progress,
                )
            except Exception as e:
                logger.error(
                    f"Error occurred during correction process for page {getattr(page, 'number', '?')} of PDF {getattr(pdf, 'path', 'unknown')}: {e}",
                    exc_info=True,
                )
                continue

    progress_bar.close()

    return self

natural_pdf.PDFCollection.export_ocr_correction_task(output_zip_path, **kwargs)

Exports OCR results from all PDFs in this collection into a single correction task package (zip file).

Parameters:

Name	Type	Description	Default
output_zip_path	str	The path to save the output zip file.	required
**kwargs		Additional arguments passed to create_correction_task_package (e.g., image_render_scale, overwrite).	{}

Source code in natural_pdf/core/pdf_collection.py

def export_ocr_correction_task(self, output_zip_path: str, **kwargs):
    """
    Exports OCR results from all PDFs in this collection into a single
    correction task package (zip file).

    Args:
        output_zip_path: The path to save the output zip file.
        **kwargs: Additional arguments passed to create_correction_task_package
                  (e.g., image_render_scale, overwrite).
    """
    from natural_pdf.utils.packaging import create_correction_task_package

    # Pass the collection itself (self) as the source
    create_correction_task_package(source=self, output_zip_path=output_zip_path, **kwargs)

natural_pdf.PDFCollection.find_all(selector=None, *, text=None, apply_exclusions=True, regex=False, case=True, **kwargs)

find_all(*, text: Union[str, Sequence[str]], apply_exclusions: bool = True, regex: bool = False, case: bool = True, **kwargs) -> ElementCollection

find_all(selector: str, *, apply_exclusions: bool = True, regex: bool = False, case: bool = True, **kwargs) -> ElementCollection

Find all elements matching the selector OR text across all PDFs in the collection.

Provide EITHER selector OR text, but not both.

This creates an ElementCollection that can span multiple PDFs. Note that some ElementCollection methods have limitations when spanning PDFs.

Parameters:

Name	Type	Description	Default
selector	Optional[str]	CSS-like selector string to query elements.	None
text	Optional[Union[str, Sequence[str]]]	Text content to search for (equivalent to 'text:contains(...)'). Accepts a single string or an iterable of strings (matches any value).	None
apply_exclusions	bool	Whether to exclude elements in exclusion regions (default: True).	True
regex	bool	Whether to use regex for text search (selector or text) (default: False).	False
case	bool	Whether to do case-sensitive text search (selector or text) (default: True).	True
**kwargs		Additional keyword arguments passed to the find_all method of each PDF.	{}

Returns:

Type	Description
ElementCollection	ElementCollection containing all matching elements across all PDFs.

Source code in natural_pdf/core/pdf_collection.py

def find_all(
    self,
    selector: Optional[str] = None,  # Now optional
    *,
    text: Optional[Union[str, Sequence[str]]] = None,  # New text parameter
    apply_exclusions: bool = True,
    regex: bool = False,
    case: bool = True,
    **kwargs,
) -> "ElementCollection":
    """
    Find all elements matching the selector OR text across all PDFs in the collection.

    Provide EITHER `selector` OR `text`, but not both.

    This creates an ElementCollection that can span multiple PDFs. Note that
    some ElementCollection methods have limitations when spanning PDFs.

    Args:
        selector: CSS-like selector string to query elements.
        text: Text content to search for (equivalent to 'text:contains(...)'). Accepts a
              single string or an iterable of strings (matches any value).
        apply_exclusions: Whether to exclude elements in exclusion regions (default: True).
        regex: Whether to use regex for text search (`selector` or `text`) (default: False).
        case: Whether to do case-sensitive text search (`selector` or `text`) (default: True).
        **kwargs: Additional keyword arguments passed to the find_all method of each PDF.

    Returns:
        ElementCollection containing all matching elements across all PDFs.
    """
    # Validation happens within pdf.find_all

    # Collect elements from all PDFs
    all_elements = []
    for pdf in self._pdfs:
        try:
            # Pass the relevant arguments down to each PDF's find_all
            elements = pdf.find_all(
                selector=selector,
                text=text,
                apply_exclusions=apply_exclusions,
                regex=regex,
                case=case,
                **kwargs,
            )
            all_elements.extend(elements.elements)
        except Exception as e:
            logger.error(f"Error finding elements in {pdf.path}: {e}", exc_info=True)

    return ElementCollection(all_elements)

natural_pdf.PDFCollection.from_directory(directory_path, recursive=True, **pdf_options) classmethod

Creates a PDFCollection explicitly from PDF files within a directory.

Source code in natural_pdf/core/pdf_collection.py

@classmethod
def from_directory(
    cls, directory_path: str, recursive: bool = True, **pdf_options: Any
) -> "PDFCollection":
    """Creates a PDFCollection explicitly from PDF files within a directory."""
    # __init__ can handle single directory string directly
    return cls(directory_path, recursive=recursive, **pdf_options)

natural_pdf.PDFCollection.from_glob(pattern, recursive=True, **pdf_options) classmethod

Creates a PDFCollection explicitly from a single glob pattern.

Source code in natural_pdf/core/pdf_collection.py

@classmethod
def from_glob(cls, pattern: str, recursive: bool = True, **pdf_options: Any) -> "PDFCollection":
    """Creates a PDFCollection explicitly from a single glob pattern."""
    # __init__ can handle single glob string directly
    return cls(pattern, recursive=recursive, **pdf_options)

natural_pdf.PDFCollection.from_globs(patterns, recursive=True, **pdf_options) classmethod

Creates a PDFCollection explicitly from a list of glob patterns.

Source code in natural_pdf/core/pdf_collection.py

@classmethod
def from_globs(
    cls, patterns: List[str], recursive: bool = True, **pdf_options: Any
) -> "PDFCollection":
    """Creates a PDFCollection explicitly from a list of glob patterns."""
    # __init__ can handle List[str] containing globs directly
    return cls(patterns, recursive=recursive, **pdf_options)

natural_pdf.PDFCollection.from_paths(paths_or_urls, **pdf_options) classmethod

Creates a PDFCollection explicitly from a list of file paths or URLs.

Source code in natural_pdf/core/pdf_collection.py

@classmethod
def from_paths(cls, paths_or_urls: List[str], **pdf_options: Any) -> "PDFCollection":
    """Creates a PDFCollection explicitly from a list of file paths or URLs."""
    # __init__ can handle List[str] directly now
    return cls(paths_or_urls, **pdf_options)

natural_pdf.PDFCollection.get_indexable_items()

Yields Page objects from the collection, conforming to Indexable.

Source code in natural_pdf/core/pdf_collection.py

def get_indexable_items(self) -> Iterable[Indexable]:
    """Yields Page objects from the collection, conforming to Indexable."""
    if not self._pdfs:
        return  # Return empty iterator if no PDFs

    for pdf in self._pdfs:
        if not pdf.pages:  # Handle case where a PDF might have 0 pages after loading
            logger.warning(f"PDF '{pdf.path}' has no pages. Skipping.")
            continue
        for page in pdf.pages:
            # Optional: Add filtering here if needed (e.g., skip empty pages)
            # Assuming Page object conforms to Indexable
            # We might still want the empty page check here for efficiency
            # if not page.extract_text(use_exclusions=False).strip():
            #     logger.debug(f"Skipping empty page {page.page_number} from PDF '{pdf.path}'.")
            #     continue
            yield page

natural_pdf.PDFCollection.show(limit=30, per_pdf_limit=10, **kwargs)

Display all PDFs in the collection with labels.

Each PDF is shown with its pages in a grid layout (6 columns by default), and all PDFs are stacked vertically with labels.

Parameters:

Name	Type	Description	Default
limit	Optional[int]	Maximum total pages to show across all PDFs (default: 30)	30
per_pdf_limit	Optional[int]	Maximum pages to show per PDF (default: 10)	10
**kwargs		Additional arguments passed to each PDF's show() method (e.g., columns, exclusions, resolution, etc.)	{}

Returns:

Type	Description
	Displayed image in Jupyter or None

Source code in natural_pdf/core/pdf_collection.py

def show(self, limit: Optional[int] = 30, per_pdf_limit: Optional[int] = 10, **kwargs):
    """
    Display all PDFs in the collection with labels.

    Each PDF is shown with its pages in a grid layout (6 columns by default),
    and all PDFs are stacked vertically with labels.

    Args:
        limit: Maximum total pages to show across all PDFs (default: 30)
        per_pdf_limit: Maximum pages to show per PDF (default: 10)
        **kwargs: Additional arguments passed to each PDF's show() method
                 (e.g., columns, exclusions, resolution, etc.)

    Returns:
        Displayed image in Jupyter or None
    """
    if not self._pdfs:
        print("Empty collection")
        return None

    # Import here to avoid circular imports
    from PIL import ImageDraw, ImageFont

    # Calculate pages per PDF if total limit is set
    if limit and not per_pdf_limit:
        per_pdf_limit = max(1, limit // len(self._pdfs))

    # Collect images from each PDF
    all_images = []
    total_pages_shown = 0

    for pdf in self._pdfs:
        if limit and total_pages_shown >= limit:
            break

        # Calculate limit for this PDF
        pdf_limit = per_pdf_limit
        if limit:
            remaining = limit - total_pages_shown
            pdf_limit = min(per_pdf_limit or remaining, remaining)

        if pdf_limit is None:
            pdf_limit_value = len(pdf.pages)
        else:
            pdf_limit_value = pdf_limit

        # Get PDF identifier
        pdf_name = getattr(pdf, "filename", None) or getattr(pdf, "path", "Unknown")
        if isinstance(pdf_name, Path):
            pdf_name = pdf_name.name
        elif "/" in str(pdf_name):
            pdf_name = str(pdf_name).split("/")[-1]

        # Render this PDF
        try:
            # Get render specs from the PDF
            render_specs = pdf._get_render_specs(
                mode="show", max_pages=pdf_limit_value, **kwargs
            )

            if not render_specs:
                continue

            # Get the highlighter and render without displaying
            highlighter = cast("HighlightingService", pdf._get_highlighter())
            pdf_image = highlighter.unified_render(
                specs=render_specs,
                layout="grid" if len(render_specs) > 1 else "single",
                columns=6,
                **kwargs,
            )

            if pdf_image:
                # Add label above the PDF image
                label_height = 40
                label_bg_color = (240, 240, 240)
                label_text_color = (0, 0, 0)

                # Create new image with space for label
                width, height = pdf_image.size
                labeled_image = Image.new("RGB", (width, height + label_height), "white")

                # Draw label background
                draw = ImageDraw.Draw(labeled_image)
                draw.rectangle([0, 0, width, label_height], fill=label_bg_color)

                # Draw label text
                try:
                    # Try to use a nice font if available
                    font = ImageFont.truetype("Arial", 20)
                except:
                    # Fallback to default font
                    font = ImageFont.load_default()

                label_text = f"{pdf_name} ({len(pdf.pages)} pages)"
                draw.text((10, 10), label_text, fill=label_text_color, font=font)

                # Paste PDF image below label
                labeled_image.paste(pdf_image, (0, label_height))

                all_images.append(labeled_image)
                if pdf_limit is None:
                    pdf_limit = len(pdf.pages)
                total_pages_shown += min(pdf_limit_value, len(pdf.pages))

        except Exception as e:
            logger.warning(f"Failed to render PDF {pdf_name}: {e}")
            continue

    if not all_images:
        print("No PDFs could be rendered")
        return None

    # Combine all images vertically
    if len(all_images) == 1:
        combined = all_images[0]
    else:
        # Add spacing between PDFs
        spacing = 20
        total_height = sum(img.height for img in all_images) + spacing * (len(all_images) - 1)
        max_width = max(img.width for img in all_images)

        combined = Image.new("RGB", (max_width, total_height), "white")

        y_offset = 0
        for i, img in enumerate(all_images):
            # Center images if they're narrower than max width
            x_offset = (max_width - img.width) // 2
            combined.paste(img, (x_offset, y_offset))
            y_offset += img.height
            if i < len(all_images) - 1:
                y_offset += spacing

    # Return the combined image (Jupyter will display it automatically)
    return combined

natural_pdf.Page

Bases: TextMixin, AnalysisHostMixin, Visualizable

Enhanced Page wrapper built on top of pdfplumber.Page.

This class provides a fluent interface for working with PDF pages, with improved selection, navigation, extraction, and question-answering capabilities. It integrates multiple analysis capabilities through mixins and provides spatial navigation with CSS-like selectors.

The Page class serves as the primary interface for document analysis, offering: - Element selection and spatial navigation - OCR and layout analysis integration - Table detection and extraction - AI-powered classification and data extraction - Visual debugging with highlighting and cropping - Text style analysis and structure detection

Attributes:

Name	Type	Description
index	int	Zero-based index of this page in the PDF.
number	int	One-based page number (index + 1).
width	float	Page width in points.
height	float	Page height in points.
bbox	float	Bounding box tuple (x0, top, x1, bottom) of the page.
chars	List[Any]	Collection of character elements on the page.
words	List[Any]	Collection of word elements on the page.
lines	List[Any]	Collection of line elements on the page.
rects	List[Any]	Collection of rectangle elements on the page.
images	List[Any]	Collection of image elements on the page.
metadata	Dict[str, Any]	Dictionary for storing analysis results and custom data.

Example

Basic usage:

pdf = npdf.PDF("document.pdf")
page = pdf.pages[0]

# Find elements with CSS-like selectors
headers = page.find_all('text[size>12]:bold')
summaries = page.find('text:contains("Summary")')

# Spatial navigation
content_below = summaries.below(until='text[size>12]:bold')

# Table extraction
tables = page.extract_table()

Advanced usage:

# Apply OCR if needed
page.apply_ocr(engine='easyocr', resolution=300)

# Layout analysis
page.analyze_layout(engine='yolo')

# AI-powered extraction
data = page.extract_structured_data(MySchema)

# Visual debugging
page.find('text:contains("Important")').show()

Source code in natural_pdf/core/page.py

class Page(TextMixin, AnalysisHostMixin, Visualizable):
    """Enhanced Page wrapper built on top of pdfplumber.Page.

    This class provides a fluent interface for working with PDF pages,
    with improved selection, navigation, extraction, and question-answering capabilities.
    It integrates multiple analysis capabilities through mixins and provides spatial
    navigation with CSS-like selectors.

    The Page class serves as the primary interface for document analysis, offering:
    - Element selection and spatial navigation
    - OCR and layout analysis integration
    - Table detection and extraction
    - AI-powered classification and data extraction
    - Visual debugging with highlighting and cropping
    - Text style analysis and structure detection

    Attributes:
        index: Zero-based index of this page in the PDF.
        number: One-based page number (index + 1).
        width: Page width in points.
        height: Page height in points.
        bbox: Bounding box tuple (x0, top, x1, bottom) of the page.
        chars: Collection of character elements on the page.
        words: Collection of word elements on the page.
        lines: Collection of line elements on the page.
        rects: Collection of rectangle elements on the page.
        images: Collection of image elements on the page.
        metadata: Dictionary for storing analysis results and custom data.

    Example:
        Basic usage:
        ```python
        pdf = npdf.PDF("document.pdf")
        page = pdf.pages[0]

        # Find elements with CSS-like selectors
        headers = page.find_all('text[size>12]:bold')
        summaries = page.find('text:contains("Summary")')

        # Spatial navigation
        content_below = summaries.below(until='text[size>12]:bold')

        # Table extraction
        tables = page.extract_table()
        ```

        Advanced usage:
        ```python
        # Apply OCR if needed
        page.apply_ocr(engine='easyocr', resolution=300)

        # Layout analysis
        page.analyze_layout(engine='yolo')

        # AI-powered extraction
        data = page.extract_structured_data(MySchema)

        # Visual debugging
        page.find('text:contains("Important")').show()
        ```
    """

    def __init__(
        self,
        page: "PdfPlumberPage",
        parent: "PDF",
        index: int,
        font_attrs=None,
        load_text: bool = True,
    ):
        """Initialize a page wrapper.

        Creates an enhanced Page object that wraps a pdfplumber page with additional
        functionality for spatial navigation, analysis, and AI-powered extraction.

        Args:
            page: The underlying pdfplumber page object that provides raw PDF data.
            parent: Parent PDF object that contains this page and provides access
                to managers and global settings.
            index: Zero-based index of this page in the PDF document.
            font_attrs: List of font attributes to consider when grouping characters
                into words. Common attributes include ['fontname', 'size', 'flags'].
                If None, uses default character-to-word grouping rules.
            load_text: If True, load and process text elements from the PDF's text layer.
                If False, skip text layer processing (useful for OCR-only workflows).

        Note:
            This constructor is typically called automatically when accessing pages
            through the PDF.pages collection. Direct instantiation is rarely needed.

        Example:
            ```python
            # Pages are usually accessed through the PDF object
            pdf = npdf.PDF("document.pdf")
            page = pdf.pages[0]  # Page object created automatically

            # Direct construction (advanced usage)
            import pdfplumber
            with pdfplumber.open("document.pdf") as plumber_pdf:
                plumber_page = plumber_pdf.pages[0]
                page = Page(plumber_page, pdf, 0, load_text=True)
            ```
        """
        self._page = page
        self._parent = parent
        self._index = index
        self._load_text = load_text
        self._text_styles_summary: Dict[str, Any] = {}
        self._text_styles = None  # Lazy-loaded text style analyzer results
        self._exclusions = []  # List to store exclusion functions/regions
        self._skew_angle: Optional[float] = None  # Stores detected skew angle

        self.metadata: Dict[str, Any] = {}

        # Region management
        self._regions = {
            "detected": [],  # Layout detection results
            "named": {},  # Named regions (name -> region)
        }

        if not hasattr(self._parent, "_config"):
            raise AttributeError(
                "Parent PDF is missing _config; cannot initialise Page configuration"
            )

        # Page-scoped configuration begins as a shallow copy of the parent PDF-level
        # configuration so that auto-computed tolerances or other page-specific
        # values do not overwrite siblings.
        self._config = dict(self._parent._config)

        # Initialize ElementManager, passing font_attrs
        self._element_mgr: ElementManager = ElementManager(
            self, font_attrs=font_attrs, load_text=self._load_text
        )
        # self._highlighter = HighlightingService(self) # REMOVED - Use property accessor
        if not hasattr(self, "metadata") or self.metadata is None:
            self.metadata = {}
        self.metadata.setdefault("analysis", {})

        # Initialize the internal variable with a single underscore
        self._layout_analyzer = None

        self._load_elements()
        self._to_image_cache: Dict[tuple, Optional["Image.Image"]] = {}

        # Flag to prevent infinite recursion when computing exclusions
        self._computing_exclusions = False

    def _get_render_specs(
        self,
        mode: Literal["show", "render"] = "show",
        color: Optional[Union[str, Tuple[int, int, int]]] = None,
        highlights: Optional[List[Dict[str, Any]]] = None,
        crop: Union[bool, Literal["content"]] = False,
        crop_bbox: Optional[Tuple[float, float, float, float]] = None,
        **kwargs,
    ) -> List[RenderSpec]:
        """Get render specifications for this page.

        Args:
            mode: Rendering mode - 'show' includes page highlights, 'render' is clean
            color: Default color for highlights in show mode
            highlights: Additional highlight groups to show
            crop: Whether to crop the page
            crop_bbox: Explicit crop bounds
            **kwargs: Additional parameters

        Returns:
            List containing a single RenderSpec for this page
        """
        spec = RenderSpec(page=self)

        elements: Optional[List[Element]] = None
        content_bbox: Optional[Tuple[float, float, float, float]] = None

        def ensure_content_bbox() -> Optional[Tuple[float, float, float, float]]:
            nonlocal elements, content_bbox
            if content_bbox is not None:
                return content_bbox
            if elements is None:
                elements = self.get_elements(apply_exclusions=False)
            if not elements:
                return None
            x_coords: List[float] = []
            y_coords: List[float] = []
            for elem in elements:
                if hasattr(elem, "bbox") and elem.bbox:
                    x0, y0, x1, y1 = elem.bbox
                    x_coords.extend([x0, x1])
                    y_coords.extend([y0, y1])
            if x_coords and y_coords:
                content_bbox = (min(x_coords), min(y_coords), max(x_coords), max(y_coords))
                return content_bbox
            return None

        spec.crop_bbox = resolve_crop_bbox(
            width=self.width,
            height=self.height,
            crop=crop,
            crop_bbox=crop_bbox,
            content_bbox_fn=ensure_content_bbox,
        )

        # Add highlights in show mode
        if mode == "show":
            # Add page's persistent highlights if any
            page_highlights = self._highlighter.get_highlights_for_page(self.index)
            for highlight in page_highlights:
                spec.add_highlight(
                    bbox=highlight.bbox,
                    polygon=highlight.polygon,
                    color=highlight.color,
                    label=highlight.label,
                    element=None,  # Persistent highlights don't have element refs
                )

            # Add additional highlight groups if provided
            if highlights:
                for group in highlights:
                    raw_elements = group.get("elements")
                    if not raw_elements:
                        continue

                    if isinstance(raw_elements, ElementCollection):
                        elements_iter: Iterable[Any] = raw_elements.elements
                    elif isinstance(raw_elements, Iterable):
                        elements_iter = cast(Iterable[Any], raw_elements)
                    else:
                        elements_iter = (raw_elements,)

                    group_color = group.get("color", color)
                    group_label = group.get("label")

                    for elem in elements_iter:
                        spec.add_highlight(element=elem, color=group_color, label=group_label)

            # Handle exclusions visualization
            exclusions_param = kwargs.get("exclusions")
            if exclusions_param:
                # Get exclusion regions
                exclusion_regions = self._get_exclusion_regions(include_callable=True)

                if exclusion_regions:
                    # Determine color for exclusions
                    exclusion_color = (
                        exclusions_param if isinstance(exclusions_param, str) else "red"
                    )

                    # Add exclusion regions as highlights
                    for region in exclusion_regions:
                        spec.add_highlight(
                            element=region,
                            color=exclusion_color,
                            label=f"Exclusion: {region.label or 'unnamed'}",
                        )

        return [spec]

    def to_region(self) -> Region:
        """Return a Region covering the full page."""
        return self.region(0, 0, self.width, self.height)

    qa_target = "page"

    def _qa_context_page_number(self) -> int:
        return self.number

    def _qa_source_elements(self) -> ElementCollection:
        return ElementCollection([])

    def _qa_target_region(self) -> Region:
        return self.to_region()

    def _element_to_region(self, element: Any, label: Optional[str] = None) -> Optional[Region]:
        bbox = extract_bbox(element)
        if bbox is None:
            return None
        return Region(self, bbox, label=label)

    def _exclusion_element_manager(self) -> ElementManager:
        return self._element_mgr

    def _invalidate_exclusion_cache(self) -> None:
        if self._element_mgr:
            self._element_mgr.invalidate_cache()

    @property
    def pdf(self) -> "PDF":
        """Provides public access to the parent PDF object."""
        return self._parent

    @property
    def number(self) -> int:
        """Get page number (1-based)."""
        return self._page.page_number

    @property
    def page_number(self) -> int:
        """Get page number (1-based)."""
        return self._page.page_number

    @property
    def index(self) -> int:
        """Get page index (0-based)."""
        return self._index

    @property
    def width(self) -> float:
        """Get page width."""
        return self._page.width

    @property
    def height(self) -> float:
        """Get page height."""
        return self._page.height

    @property
    def _highlighter(self) -> "HighlightingService":
        """Provides access to the parent PDF's HighlightingService."""
        if not hasattr(self._parent, "highlighter"):
            # This should ideally not happen if PDF.__init__ works correctly
            raise AttributeError("Parent PDF object does not have a 'highlighter' attribute.")
        return self._parent.highlighter

    def get_highlighter(self) -> "HighlightingService":
        """Expose the page-level HighlightingService for Visualizable consumers."""
        return self._highlighter

    def _context_page(self) -> "Page":
        return self

    def clear_exclusions(self) -> "Page":
        """
        Clear all exclusions from the page.
        """
        self._exclusions = []
        return self

    @contextlib.contextmanager
    def without_exclusions(self):
        """
        Context manager that temporarily disables exclusion processing.

        This prevents infinite recursion when exclusion callables themselves
        use find() operations. While in this context, all find operations
        will skip exclusion filtering.

        Example:
            ```python
            # This exclusion would normally cause infinite recursion:
            page.add_exclusion(lambda p: p.find("text:contains('Header')").expand())

            # But internally, it's safe because we use:
            with page.without_exclusions():
                region = exclusion_callable(page)
            ```

        Yields:
            The page object with exclusions temporarily disabled.
        """
        old_value = self._computing_exclusions
        self._computing_exclusions = True
        try:
            yield self
        finally:
            self._computing_exclusions = old_value

    def add_region(
        self, region: "Region", name: Optional[str] = None, *, source: Optional[str] = None
    ) -> "Page":
        """
        Add a region to the page.

        Args:
            region: Region object to add
            name: Optional name for the region
            source: Optional provenance label; if provided it will be recorded on the region.

        Returns:
            Self for method chaining
        """
        # Check if it's actually a Region object
        if not isinstance(region, Region):
            raise TypeError("region must be a Region object")

        # Respect an explicitly provided source, otherwise keep any existing label.
        if source is not None:
            region.source = source
        elif getattr(region, "source", None) is None:
            region.source = "named"

        if name:
            region.name = name
            # Add to named regions dictionary (overwriting if name already exists)
            self._regions["named"][name] = region
        else:
            # Add to detected regions list (unnamed but registered)
            self._regions["detected"].append(region)

        # Add to element manager for selector queries
        self._element_mgr.add_region(region)

        return self

    def add_regions(
        self,
        regions: List["Region"],
        prefix: Optional[str] = None,
        *,
        source: Optional[str] = None,
    ) -> "Page":
        """
        Add multiple regions to the page.

        Args:
            regions: List of Region objects to add
            prefix: Optional prefix for automatic naming (regions will be named prefix_1, prefix_2, etc.)
            source: Optional provenance label applied to each region.

        Returns:
            Self for method chaining
        """
        if prefix:
            # Add with automatic sequential naming
            for i, region in enumerate(regions):
                self.add_region(region, name=f"{prefix}_{i+1}", source=source)
        else:
            # Add without names
            for region in regions:
                self.add_region(region, source=source)

        return self

    # Element manager facade helpers
    def ensure_elements_loaded(self) -> None:
        """Force the underlying element manager to load elements."""
        self._element_mgr.load_elements()

    def invalidate_element_cache(self) -> None:
        """Invalidate the cached elements so they are reloaded on next access."""
        self._element_mgr.invalidate_cache()

    def has_element_cache(self) -> bool:
        """Return True if the element manager currently holds any elements."""
        return self._element_mgr.has_elements()

    def get_all_elements_raw(self) -> List["Element"]:
        """Return all elements without applying exclusions."""
        return list(self._element_mgr.get_all_elements())

    def get_elements_by_type(self, element_type: str) -> List[Any]:
        """Return the elements for a specific backing collection (e.g. 'words')."""
        return list(self._element_mgr.get_elements(element_type))

    def add_element(self, element: Any, element_type: str = "words") -> bool:
        """Add an element to the backing collection."""
        return bool(self._element_mgr.add_element(element, element_type))

    def _infer_element_type(self, element: Any, default: str = "words") -> str:
        """Best-effort inference of element collection name for an object."""
        element_type = getattr(element, "object_type", None)
        if element_type is None and isinstance(element, dict):
            element_type = element.get("object_type")

        if isinstance(element_type, str):
            normalized = element_type.lower()
            if normalized == "word":
                return "words"
            if normalized == "char":
                return "chars"
            if normalized == "rect":
                return "rects"
            if normalized == "line":
                return "lines"
            if normalized == "region":
                return "regions"
            if normalized.endswith("s"):
                return normalized
        return default

    def remove_element(self, element: Any, element_type: Optional[str] = None) -> bool:
        """Remove an element from the backing collection."""
        target_type = element_type or self._infer_element_type(element)
        return bool(self._element_mgr.remove_element(element, target_type))

    def remove_elements_by_source(self, element_type: str, source: str) -> int:
        """Remove all elements of a given type whose source matches."""
        return int(self._element_mgr.remove_elements_by_source(element_type, source))

    def _ocr_element_manager(self) -> ElementManager:
        return self._element_mgr

    def iter_regions(self) -> List["Region"]:
        """Return a list of regions currently registered with the page."""
        return list(self._element_mgr.regions)

    def remove_regions_by_source(self, source: str) -> int:
        """Remove all registered regions that match the requested source."""
        return int(self._element_mgr.remove_elements_by_source("regions", source))

    def remove_regions(
        self,
        *,
        source: Optional[str] = None,
        region_type: Optional[str] = None,
        predicate: Optional[Callable[["Region"], bool]] = None,
    ) -> int:
        """
        Remove regions from the page based on optional filters.

        Args:
            source: Match regions whose ``region.source`` equals this string.
            region_type: Match regions whose ``region.region_type`` equals this string.
            predicate: Additional callable that returns True when a region should be removed.

        Returns:
            The number of regions removed.
        """

        def _matches(region: "Region") -> bool:
            if source is not None and getattr(region, "source", None) != source:
                return False
            if region_type is not None and getattr(region, "region_type", None) != region_type:
                return False
            if predicate is not None and not predicate(region):
                return False
            return True

        removed = 0

        # Remove from element manager, if available
        if hasattr(self, "_element_mgr") and hasattr(self._element_mgr, "regions"):
            regions_list = getattr(self._element_mgr, "regions")
            if isinstance(regions_list, list):
                to_remove = [region for region in regions_list if _matches(region)]
                for region in to_remove:
                    regions_list.remove(region)
                removed += len(to_remove)

        # Remove from detected collection
        detected = self._regions.get("detected", [])
        if detected:
            retained = [region for region in detected if not _matches(region)]
            removed += len(detected) - len(retained)
            self._regions["detected"] = retained

        # Remove from named collection
        named = self._regions.get("named", {})
        if named:
            keys_to_delete = [key for key, region in named.items() if _matches(region)]
            for key in keys_to_delete:
                del named[key]
                removed += 1

        return removed

    def _get_exclusion_regions(self, include_callable=True, debug=False) -> List["Region"]:
        """
        Get all exclusion regions for this page.
        Now handles both region-based and element-based exclusions.
        Assumes self._exclusions contains tuples of (callable/Region/Element, label, method).

        Args:
            include_callable: Whether to evaluate callable exclusion functions
            debug: Enable verbose debug logging for exclusion evaluation

        Returns:
            List of Region objects to exclude, with labels assigned.
        """
        regions = []

        # Combine page-specific exclusions with PDF-level exclusions
        all_exclusions = list(self._exclusions)  # Start with page-specific

        # Add PDF-level exclusions if we have a parent PDF
        if hasattr(self, "_parent") and self._parent and hasattr(self._parent, "_exclusions"):
            # Get existing labels to check for duplicates
            existing_labels = set()
            for exc in all_exclusions:
                if len(exc) >= 2 and exc[1]:  # Has a label
                    existing_labels.add(exc[1])

            for pdf_exclusion in self._parent._exclusions:
                # Check if this exclusion label is already in our list (avoid duplicates)
                label = pdf_exclusion[1] if len(pdf_exclusion) >= 2 else None
                if label and label in existing_labels:
                    continue  # Skip this exclusion as it's already been applied

                # Ensure consistent format (PDF exclusions might be 2-tuples, need to be 3-tuples)
                if len(pdf_exclusion) == 2:
                    # Convert to 3-tuple format with default method
                    pdf_exclusion = (pdf_exclusion[0], pdf_exclusion[1], "region")
                all_exclusions.append(pdf_exclusion)

        if debug:
            print(
                f"\nPage {self.index}: Evaluating {len(all_exclusions)} exclusions ({len(self._exclusions)} page-specific, {len(all_exclusions) - len(self._exclusions)} from PDF)"
            )

        for i, exclusion_data in enumerate(all_exclusions):
            # Handle both old format (2-tuple) and new format (3-tuple) for backward compatibility
            if len(exclusion_data) == 2:
                # Old format: (exclusion_item, label)
                exclusion_item, label = exclusion_data
                method = "region"  # Default to region for old format
            else:
                # New format: (exclusion_item, label, method)
                exclusion_item, label, method = exclusion_data

            exclusion_label = label if label else f"exclusion {i}"

            # Process callable exclusion functions
            if callable(exclusion_item) and include_callable:
                try:
                    if debug:
                        print(f"  - Evaluating callable '{exclusion_label}'...")

                    # Use context manager to prevent infinite recursion
                    with self.without_exclusions():
                        # Call the function - Expects it to return a Region or None
                        region_result = exclusion_item(self)

                    if isinstance(region_result, Region):
                        # Assign the label to the returned region
                        region_result.label = label
                        regions.append(region_result)
                        if debug:
                            print(f"    ✓ Added region from callable '{label}': {region_result}")
                    elif isinstance(region_result, ElementCollection):
                        elements_iter: Iterable[Any] = region_result.elements
                        if debug:
                            print(
                                f"    Converting ElementCollection with {len(region_result.elements)} elements to regions..."
                            )

                        for elem in elements_iter:
                            try:
                                bbox_candidate = getattr(elem, "bbox", None)
                                if (
                                    isinstance(bbox_candidate, (tuple, list))
                                    and len(bbox_candidate) == 4
                                ):
                                    bbox_coords = cast(
                                        Tuple[float, float, float, float],
                                        tuple(
                                            float(v) for v in cast(Sequence[float], bbox_candidate)
                                        ),
                                    )
                                    region = Region(self, bbox_coords, label=label)
                                    regions.append(region)
                                    if debug:
                                        print(f"      ✓ Added region from element: {bbox_coords}")
                                else:
                                    if debug:
                                        print(
                                            f"      ✗ Skipping element without valid bbox: {elem}"
                                        )
                            except Exception as e:
                                if debug:
                                    print(f"      ✗ Failed to convert element to region: {e}")
                                continue

                        if debug and region_result.elements:
                            print(
                                f"    ✓ Converted {len(region_result.elements)} elements from callable '{label}'"
                            )
                    elif isinstance(region_result, Iterable):
                        materialised = list(cast(Iterable[Any], region_result))
                        if not materialised:
                            if debug:
                                print(f"    ✗ Empty iterable returned from callable '{label}'")
                        else:
                            if debug:
                                print(
                                    f"    Converting iterable {type(region_result)} with {len(materialised)} elements to regions..."
                                )
                            for elem in materialised:
                                try:
                                    bbox_candidate = getattr(elem, "bbox", None)
                                    if (
                                        isinstance(bbox_candidate, (tuple, list))
                                        and len(bbox_candidate) == 4
                                    ):
                                        bbox_coords = cast(
                                            Tuple[float, float, float, float],
                                            tuple(
                                                float(v)
                                                for v in cast(Sequence[float], bbox_candidate)
                                            ),
                                        )
                                        region = Region(self, bbox_coords, label=label)
                                        regions.append(region)
                                        if debug:
                                            print(
                                                f"      ✓ Added region from iterable element: {bbox_coords}"
                                            )
                                    else:
                                        if debug:
                                            print(
                                                f"      ✗ Skipping iterable element without valid bbox: {elem}"
                                            )
                                except Exception as e:
                                    if debug:
                                        print(
                                            f"      ✗ Failed to convert iterable element to region: {e}"
                                        )
                                    continue
                    elif region_result:
                        # Check if it's a single Element that can be converted to a Region
                        from natural_pdf.elements.base import Element

                        if isinstance(region_result, Element) or (
                            hasattr(region_result, "bbox") and hasattr(region_result, "expand")
                        ):
                            try:
                                # Convert Element to Region using expand()
                                expanded_region = cast(Any, region_result).expand()
                                if isinstance(expanded_region, Region):
                                    expanded_region.label = label
                                    regions.append(expanded_region)
                                    if debug:
                                        print(
                                            f"    ✓ Converted Element to Region from callable '{label}': {expanded_region}"
                                        )
                                else:
                                    if debug:
                                        print(
                                            f"    ✗ Element.expand() did not return a Region: {type(expanded_region)}"
                                        )
                            except Exception as e:
                                if debug:
                                    print(f"    ✗ Failed to convert Element to Region: {e}")
                        else:
                            logger.warning(
                                f"Callable exclusion '{exclusion_label}' returned non-Region object: {type(region_result)}. Skipping."
                            )
                            if debug:
                                print(
                                    f"    ✗ Callable returned non-Region/None: {type(region_result)}"
                                )
                    else:
                        if debug:
                            print(
                                f"    ✗ Callable '{exclusion_label}' returned None, no region added"
                            )

                except Exception as e:
                    error_msg = f"Error evaluating callable exclusion '{exclusion_label}' for page {self.index}: {e}"
                    print(error_msg)
                    import traceback

                    print(f"    Traceback: {traceback.format_exc().splitlines()[-3:]}")

            # Process direct Region objects (label was assigned in add_exclusion)
            elif isinstance(exclusion_item, Region):
                regions.append(exclusion_item)  # Label is already on the Region object
                if debug:
                    print(f"  - Added direct region '{label}': {exclusion_item}")

            # Process direct Element objects - only convert to Region if method is "region"
            elif hasattr(exclusion_item, "bbox") and hasattr(exclusion_item, "expand"):
                if method == "region":
                    try:
                        # Convert Element to Region using expand()
                        expanded_region = cast(Any, exclusion_item).expand()
                        if isinstance(expanded_region, Region):
                            expanded_region.label = label
                            regions.append(expanded_region)
                            if debug:
                                print(
                                    f"  - Converted direct Element to Region '{label}': {expanded_region}"
                                )
                        else:
                            if debug:
                                print(
                                    f"  - Element.expand() did not return a Region: {type(expanded_region)}"
                                )
                    except Exception as e:
                        if debug:
                            print(f"  - Failed to convert Element to Region: {e}")
                else:
                    # method == "element" - will be handled in _filter_elements_by_exclusions
                    if debug:
                        print(
                            f"  - Skipping element '{label}' (will be handled as element-based exclusion)"
                        )

            # Process string selectors (from PDF-level exclusions)
            elif isinstance(exclusion_item, str):
                selector_str = exclusion_item
                matching_elements = self.find_all(selector_str, apply_exclusions=False)

                if debug:
                    print(
                        f"  - Evaluating selector '{exclusion_label}': found {len(matching_elements)} elements"
                    )

                if method == "region":
                    # Convert each matching element to a region
                    for el in matching_elements:
                        try:
                            bbox_coords = (
                                float(el.x0),
                                float(el.top),
                                float(el.x1),
                                float(el.bottom),
                            )
                            region = Region(self, bbox_coords, label=label)
                            regions.append(region)
                            if debug:
                                print(f"    ✓ Added region from selector match: {bbox_coords}")
                        except Exception as e:
                            if debug:
                                print(f"    ✗ Failed to create region from element: {e}")
                # If method is "element", it will be handled in _filter_elements_by_exclusions

            # Element-based exclusions are not converted to regions here
            # They will be handled separately in _filter_elements_by_exclusions

        if debug:
            print(f"Page {self.index}: Found {len(regions)} valid exclusion regions to apply")

        return regions

    def _filter_elements_by_exclusions(
        self, elements: List["Element"], debug_exclusions: bool = False
    ) -> List["Element"]:
        """
        Filters a list of elements, removing those based on exclusion rules.
        Handles both region-based exclusions (exclude all in area) and
        element-based exclusions (exclude only specific elements).

        Args:
            elements: The list of elements to filter.
            debug_exclusions: Whether to output detailed exclusion debugging info (default: False).

        Returns:
            A new list containing only the elements not excluded.
        """
        # Skip exclusion filtering if we're currently computing exclusions
        # This prevents infinite recursion when exclusion callables use find operations
        if self._computing_exclusions:
            return elements

        # Check both page-level and PDF-level exclusions
        has_page_exclusions = bool(self._exclusions)
        has_pdf_exclusions = (
            hasattr(self, "_parent")
            and self._parent
            and hasattr(self._parent, "_exclusions")
            and bool(self._parent._exclusions)
        )

        if not has_page_exclusions and not has_pdf_exclusions:
            if debug_exclusions:
                print(
                    f"Page {self.index}: No exclusions defined, returning all {len(elements)} elements."
                )
            return elements

        # Get all exclusion regions, including evaluating callable functions
        exclusion_regions = self._get_exclusion_regions(
            include_callable=True, debug=debug_exclusions
        )

        # Collect element-based exclusions
        # Store element bboxes for comparison instead of object ids
        excluded_element_bboxes = set()  # Use set for O(1) lookup

        # Process both page-level and PDF-level exclusions
        all_exclusions = list(self._exclusions) if has_page_exclusions else []
        if has_pdf_exclusions:
            all_exclusions.extend(self._parent._exclusions)

        for exclusion_data in all_exclusions:
            # Handle both old format (2-tuple) and new format (3-tuple)
            if len(exclusion_data) == 2:
                exclusion_item, label = exclusion_data
                method = "region"
            else:
                exclusion_item, label, method = exclusion_data

            # Skip callables (already handled in _get_exclusion_regions)
            if callable(exclusion_item):
                continue

            # Skip regions (already in exclusion_regions)
            if isinstance(exclusion_item, Region):
                continue

            # Handle string selectors for element-based exclusions
            if isinstance(exclusion_item, str) and method == "element":
                selector_str = exclusion_item
                matching_elements = self.find_all(selector_str, apply_exclusions=False)
                elements_iter = getattr(matching_elements, "elements", [])
                for el in cast(Iterable[Any], elements_iter):
                    bbox_vals = extract_bbox(cast(Any, el))
                    if bbox_vals is None:
                        continue
                    excluded_element_bboxes.add(bbox_vals)
                    if debug_exclusions:
                        print(
                            f"  - Added element exclusion from selector '{selector_str}': {bbox_vals}"
                        )

            # Handle element-based exclusions
            elif method == "element":
                bbox = extract_bbox(cast(Any, exclusion_item))
                if bbox is not None:
                    excluded_element_bboxes.add(bbox)
                    if debug_exclusions:
                        print(f"  - Added element exclusion with bbox {bbox}: {exclusion_item}")
                else:
                    logger.warning(
                        f"Page {self.index}: Skipping element exclusion without bounding box: {exclusion_item}"
                    )

        if debug_exclusions:
            print(
                f"Page {self.index}: Applying {len(exclusion_regions)} region exclusions "
                f"and {len(excluded_element_bboxes)} element exclusions to {len(elements)} elements."
            )

        filtered_elements = []
        region_excluded_count = 0
        element_excluded_count = 0

        for element in elements:
            exclude = False

            # Check element-based exclusions first (faster)
            element_bbox = extract_bbox(element)
            if element_bbox in excluded_element_bboxes:
                exclude = True
                element_excluded_count += 1
                if debug_exclusions:
                    print(f"    Element {element} excluded by element-based rule")
            else:
                # Check region-based exclusions
                for region in exclusion_regions:
                    # Use the region's method to check if the element is inside
                    if region._is_element_in_region(element):
                        exclude = True
                        region_excluded_count += 1
                        if debug_exclusions:
                            print(f"    Element {element} excluded by region {region}")
                        break  # No need to check other regions for this element

            if not exclude:
                filtered_elements.append(element)

        if debug_exclusions:
            print(
                f"Page {self.index}: Excluded {region_excluded_count} by regions, "
                f"{element_excluded_count} by elements, keeping {len(filtered_elements)}."
            )

        return filtered_elements

    @contextlib.contextmanager
    def _temporary_text_settings(
        self,
        text_tolerance: Optional[Dict[str, Any]] = None,
        auto_text_tolerance: Optional[Union[bool, Dict[str, Any]]] = None,
    ):
        """
        Temporarily override page-level text tolerance settings and refresh caches.
        """
        overrides: Dict[str, Any] = {}
        if text_tolerance:
            overrides.update(
                {
                    key: value
                    for key, value in dict(text_tolerance).items()
                    if key
                    in {
                        "x_tolerance",
                        "y_tolerance",
                        "x_tolerance_ratio",
                        "y_tolerance_ratio",
                        "keep_blank_chars",
                    }
                }
            )

        auto_override: Optional[bool] = None
        if isinstance(auto_text_tolerance, dict):
            overrides.update(dict(auto_text_tolerance))
            auto_override = True
        else:
            auto_override = auto_text_tolerance

        if auto_override is not None:
            overrides["auto_text_tolerance"] = bool(auto_override)

        if not overrides:
            yield False
            return

        sentinel = object()
        page_config = self._config
        previous_values: Dict[str, Any] = {key: page_config.get(key, sentinel) for key in overrides}
        cache_invalidated = False
        try:
            for key, value in overrides.items():
                if value is None:
                    if key in page_config:
                        del page_config[key]
                        cache_invalidated = True
                elif page_config.get(key) != value:
                    page_config[key] = value
                    cache_invalidated = True

            if cache_invalidated:
                self._element_mgr.invalidate_cache()
            yield cache_invalidated
        finally:
            if not cache_invalidated:
                return

            restore_invalidated = False
            for key, prior in previous_values.items():
                if prior is sentinel:
                    if key in page_config:
                        del page_config[key]
                        restore_invalidated = True
                elif page_config.get(key) != prior:
                    page_config[key] = prior
                    restore_invalidated = True

            if restore_invalidated:
                self._element_mgr.invalidate_cache()

    def find(
        self,
        selector: Optional[str] = None,
        *,
        text: Optional[Union[str, Sequence[str]]] = None,
        overlap: Optional[str] = None,
        apply_exclusions: bool = True,
        regex: bool = False,
        case: bool = True,
        text_tolerance: Optional[Dict[str, Any]] = None,
        auto_text_tolerance: Optional[Union[bool, Dict[str, Any]]] = None,
        reading_order: bool = True,
        near_threshold: Optional[float] = None,
        engine: Optional[str] = None,
    ) -> Optional[Element]:
        """
        Find first element on this page matching selector OR text content.

        Provide EITHER `selector` OR `text`, but not both.

        Args:
            selector: CSS-like selector string.
            text: Optional text shortcut (equivalent to ``text:contains(...)``).
            overlap: Reserved for compatibility with region APIs; ignored for pages.
            apply_exclusions: Whether to exclude elements in exclusion regions (default: True).
            regex: Whether to use regex for text search (`selector` or `text`) (default: False).
            case: Whether to do case-sensitive text search (`selector` or `text`) (default: True).
            text_tolerance: Optional dict of tolerance overrides applied temporarily.
            auto_text_tolerance: Optional overrides controlling automatic tolerance logic.
            reading_order: Whether to sort matches in reading order when applicable (default: True).
            near_threshold: Maximum distance (in points) used by the ``:near`` pseudo-class.
            engine: Optional selector engine name registered via :mod:`natural_pdf.engine_provider`.

        Returns:
            Element object or None if not found.
        """
        effective_selector = normalize_selector_input(
            selector,
            text,
            logger=logger,
            context="Page.find",
        )

        results_collection = execute_selector_query(
            self,
            effective_selector,
            text_tolerance=text_tolerance,
            auto_text_tolerance=auto_text_tolerance,
            regex=regex,
            case=case,
            reading_order=reading_order,
            near_threshold=near_threshold,
            engine=engine,
        )

        elements: List[Element] = list(results_collection.elements) if results_collection else []
        if apply_exclusions and elements:
            elements = self._filter_elements_by_exclusions(elements)

        return cast(Optional[Element], elements[0] if elements else None)

    def find_all(
        self,
        selector: Optional[str] = None,
        *,
        text: Optional[Union[str, Sequence[str]]] = None,
        overlap: Optional[str] = None,
        apply_exclusions: bool = True,
        regex: bool = False,
        case: bool = True,
        text_tolerance: Optional[Dict[str, Any]] = None,
        auto_text_tolerance: Optional[Union[bool, Dict[str, Any]]] = None,
        reading_order: bool = True,
        near_threshold: Optional[float] = None,
        engine: Optional[str] = None,
    ) -> "ElementCollection":
        """
        Find all elements on this page matching selector OR text content.

        Provide EITHER `selector` OR `text`, but not both.

        Args:
            selector: CSS-like selector string.
            text: Text content to search for (equivalent to 'text:contains(...)'). Accepts a
                  single string or an iterable of strings (matches any value).
            overlap: Reserved for compatibility with region APIs; ignored for pages.
            apply_exclusions: Whether to exclude elements in exclusion regions (default: True).
            regex: Whether to use regex for text search (`selector` or `text`) (default: False).
            case: Whether to do case-sensitive text search (`selector` or `text`) (default: True).
            text_tolerance: Optional dict of tolerance overrides applied temporarily.
            auto_text_tolerance: Optional overrides controlling automatic tolerance calculation.
            reading_order: Whether to sort matches in reading order (default: True).
            near_threshold: Maximum distance (in points) used by the ``:near`` pseudo-class.
            engine: Optional selector engine name registered with the selector provider.

        Returns:
            ElementCollection with matching elements.
        """
        effective_selector = normalize_selector_input(
            selector,
            text,
            logger=logger,
            context="Page.find_all",
        )

        results_collection = execute_selector_query(
            self,
            effective_selector,
            text_tolerance=text_tolerance,
            auto_text_tolerance=auto_text_tolerance,
            regex=regex,
            case=case,
            reading_order=reading_order,
            near_threshold=near_threshold,
            engine=engine,
        )

        if apply_exclusions and results_collection:
            filtered_elements = self._filter_elements_by_exclusions(results_collection.elements)
            return ElementCollection(filtered_elements)
        return results_collection

    def _apply_selector(
        self, selector_obj: Dict[str, Any], **kwargs: Any
    ) -> "ElementCollection":  # Removed apply_exclusions arg
        """
        Apply selector to page elements.
        Exclusions are now handled by the calling methods (find, find_all) if requested.

        Args:
            selector_obj: Parsed selector dictionary (single or compound OR selector)
            **kwargs: Additional filter parameters including 'regex' and 'case'

        Returns:
            ElementCollection of matching elements (unfiltered by exclusions)
        """
        from natural_pdf.selectors.parser import _calculate_aggregates, selector_to_filter_func

        selector_kwargs = dict(kwargs)
        selector_kwargs.setdefault("selector_context", self)

        def _apply_relational_only(sel: Dict[str, Any], elements: List[Any]) -> List[Any]:
            relational = sel.get("relational_pseudos")
            if not relational:
                return elements
            return _apply_relational_post_pseudos(
                self,
                {"relational_pseudos": relational},
                elements,
                selector_kwargs,
            )

        def _apply_post_only(sel: Dict[str, Any], elements: List[Any]) -> List[Any]:
            post = sel.get("post_pseudos")
            if not post:
                return elements
            return _apply_relational_post_pseudos(
                self,
                {"post_pseudos": post},
                elements,
                selector_kwargs,
            )

        # Handle compound OR selectors
        if selector_obj.get("type") == "or":
            elements_to_search = self._element_mgr.get_all_elements()

            has_aggregates = False
            for sub_selector in selector_obj.get("selectors", []):
                for attr in sub_selector.get("attributes", []):
                    value = attr.get("value")
                    if isinstance(value, dict) and value.get("type") == "aggregate":
                        has_aggregates = True
                        break
                if has_aggregates:
                    break

            aggregates: Dict[str, Any] = {}
            if has_aggregates:
                for sub_selector in selector_obj.get("selectors", []):
                    sub_type = sub_selector.get("type", "any").lower()
                    if sub_type == "text":
                        sub_elements = self._element_mgr.words
                    elif sub_type == "rect":
                        sub_elements = self._element_mgr.rects
                    elif sub_type == "line":
                        sub_elements = self._element_mgr.lines
                    elif sub_type == "region":
                        sub_elements = self._element_mgr.regions
                    else:
                        sub_elements = elements_to_search

                    sub_aggregates = _calculate_aggregates(sub_elements, sub_selector)
                    aggregates.update(sub_aggregates)

            filter_func = selector_to_filter_func(
                selector_obj, aggregates=aggregates, **selector_kwargs
            )
            matching_elements = [element for element in elements_to_search if filter_func(element)]

            matching_elements = _apply_relational_only(selector_obj, matching_elements)

            if selector_kwargs.get("reading_order", True):
                if all(hasattr(el, "top") and hasattr(el, "x0") for el in matching_elements):
                    matching_elements.sort(key=lambda el: (el.top, el.x0))
                elif matching_elements:
                    logger.warning(
                        "Cannot sort elements in reading order: Missing required attributes (top, x0)."
                    )

            # Handle collection-level pseudo-classes (:first, :last) for OR selectors
            has_first = any(
                any(p.get("name") == "first" for p in sub_selector.get("post_pseudos", []))
                for sub_selector in selector_obj.get("selectors", [])
            )
            has_last = any(
                any(p.get("name") == "last" for p in sub_selector.get("post_pseudos", []))
                for sub_selector in selector_obj.get("selectors", [])
            )

            if has_first:
                matching_elements = matching_elements[:1] if matching_elements else []
            elif has_last:
                matching_elements = matching_elements[-1:] if matching_elements else []

            return ElementCollection(matching_elements)

        element_type = selector_obj.get("type", "any").lower()
        if element_type == "text" or element_type == "word":
            elements_to_search = self._element_mgr.words
        elif element_type == "char":
            elements_to_search = self._element_mgr.chars
        elif element_type in ("rect", "rectangle"):
            elements_to_search = self._element_mgr.rects
        elif element_type == "line":
            elements_to_search = self._element_mgr.lines
        elif element_type == "region":
            elements_to_search = self._element_mgr.regions
        elif element_type == "any":
            elements_to_search = self._element_mgr.get_all_elements()
        else:
            elements_to_search = self._element_mgr.get_all_elements()

        has_aggregates = any(
            isinstance(attr.get("value"), dict) and attr["value"].get("type") == "aggregate"
            for attr in selector_obj.get("attributes", [])
        )

        aggregates: Dict[str, Any] = {}
        if has_aggregates:
            aggregates = _calculate_aggregates(elements_to_search, selector_obj)

        filter_func = selector_to_filter_func(
            selector_obj, aggregates=aggregates, **selector_kwargs
        )
        matching_elements = [element for element in elements_to_search if filter_func(element)]

        matching_elements = _apply_relational_only(selector_obj, matching_elements)

        if selector_kwargs.get("reading_order", True):
            if all(hasattr(el, "top") and hasattr(el, "x0") for el in matching_elements):
                matching_elements.sort(key=lambda el: (el.top, el.x0))
            elif matching_elements:
                logger.warning(
                    "Cannot sort elements in reading order: Missing required attributes (top, x0)."
                )

        # Handle :closest pseudo-class for fuzzy text matching
        for pseudo in selector_obj.get("pseudo_classes", []):
            name = pseudo.get("name")
            if name != "closest" or pseudo.get("args") is None:
                continue

            search_text = str(pseudo["args"]).strip()
            threshold = 0.0
            if not search_text:
                matching_elements = []
                break

            if "@" in search_text and search_text.count("@") == 1:
                text_part, threshold_part = search_text.rsplit("@", 1)
                try:
                    threshold = float(threshold_part)
                    search_text = text_part.strip()
                except (ValueError, TypeError):
                    pass

            ignore_case = not selector_kwargs.get("case", True)
            scored_elements = []
            for el in matching_elements:
                if not getattr(el, "text", None):
                    continue
                el_text = el.text.strip()
                search_term = search_text
                if ignore_case:
                    el_text = el_text.lower()
                    search_term = search_term.lower()

                ratio = _jaro_winkler_similarity(search_term, el_text)
                contains_match = search_term in el_text
                if ratio >= threshold:
                    scored_elements.append((contains_match, ratio, el))

            scored_elements.sort(key=lambda x: (x[0], x[1]), reverse=True)
            matching_elements = [entry[2] for entry in scored_elements]
            break

        matching_elements = _apply_post_only(selector_obj, matching_elements)

        return ElementCollection(matching_elements)

    def create_region(self, x0: float, top: float, x1: float, bottom: float) -> Any:
        """
        Create a region on this page with the specified coordinates.

        Args:
            x0: Left x-coordinate
            top: Top y-coordinate
            x1: Right x-coordinate
            bottom: Bottom y-coordinate

        Returns:
            Region object for the specified coordinates
        """
        from natural_pdf.elements.region import Region

        return Region(self, (x0, top, x1, bottom))

    def region(
        self,
        left: Optional[float] = None,
        top: Optional[float] = None,
        right: Optional[float] = None,
        bottom: Optional[float] = None,
        width: Union[str, float, None] = None,
        height: Optional[float] = None,
    ) -> Any:
        """
        Create a region on this page with more intuitive named parameters,
        allowing definition by coordinates or by coordinate + dimension.

        Args:
            left: Left x-coordinate (default: 0 if width not used).
            top: Top y-coordinate (default: 0 if height not used).
            right: Right x-coordinate (default: page width if width not used).
            bottom: Bottom y-coordinate (default: page height if height not used).
            width: Width definition. Can be:
                    - Numeric: The width of the region in points. Cannot be used with both left and right.
                    - String 'full': Sets region width to full page width (overrides left/right).
                    - String 'element' or None (default): Uses provided/calculated left/right,
                        defaulting to page width if neither are specified.
            height: Numeric height of the region. Cannot be used with both top and bottom.

        Returns:
            Region object for the specified coordinates

        Raises:
            ValueError: If conflicting arguments are provided (e.g., top, bottom, and height)
                        or if width is an invalid string.

        Examples:
            >>> page.region(top=100, height=50)  # Region from y=100 to y=150, default width
            >>> page.region(left=50, width=100)   # Region from x=50 to x=150, default height
            >>> page.region(bottom=500, height=50) # Region from y=450 to y=500
            >>> page.region(right=200, width=50)  # Region from x=150 to x=200
            >>> page.region(top=100, bottom=200, width="full") # Explicit full width
        """
        # Percentage support – convert strings like "30%" to absolute values
        # based on page dimensions.  X-axis params (left, right, width) use
        # page.width; Y-axis params (top, bottom, height) use page.height.

        def _pct_to_abs(val, axis: str):
            if isinstance(val, str) and val.strip().endswith("%"):
                try:
                    pct = float(val.strip()[:-1]) / 100.0
                except ValueError:
                    return val  # leave unchanged if not a number
                return pct * (self.width if axis == "x" else self.height)
            return val

        left = _pct_to_abs(left, "x")
        right = _pct_to_abs(right, "x")
        width = _pct_to_abs(width, "x")
        top = _pct_to_abs(top, "y")
        bottom = _pct_to_abs(bottom, "y")
        height = _pct_to_abs(height, "y")

        is_width_numeric = isinstance(width, (int, float))
        is_width_string = isinstance(width, str)
        width_mode = "element"  # Default mode

        if height is not None and top is not None and bottom is not None:
            raise ValueError("Cannot specify top, bottom, and height simultaneously.")
        if is_width_numeric and left is not None and right is not None:
            raise ValueError("Cannot specify left, right, and a numeric width simultaneously.")
        if is_width_string:
            width_lower = width.lower()
            if width_lower not in ["full", "element"]:
                raise ValueError("String width argument must be 'full' or 'element'.")
            width_mode = width_lower

        final_top = top
        final_bottom = bottom
        final_left = left
        final_right = right

        # Height calculations
        if height is not None:
            if top is not None:
                final_bottom = top + height
            elif bottom is not None:
                final_top = bottom - height
            else:  # Neither top nor bottom provided, default top to 0
                final_top = 0
                final_bottom = height

        # Width calculations (numeric only)
        if is_width_numeric:
            if left is not None:
                final_right = left + width
            elif right is not None:
                final_left = right - width
            else:  # Neither left nor right provided, default left to 0
                final_left = 0
                final_right = width

        # Only default coordinates if they weren't set by dimension calculation
        if final_top is None:
            final_top = 0.0
        if final_bottom is None:
            # Check if bottom should have been set by height calc
            if height is None or top is None:
                final_bottom = float(self.height)

        if final_left is None:
            final_left = 0.0
        if final_right is None:
            # Check if right should have been set by width calc
            if not is_width_numeric or left is None:
                final_right = float(self.width)

        if final_top is None or final_bottom is None or final_left is None or final_right is None:
            raise ValueError("Unable to resolve region coordinates.")

        final_left = float(final_left)
        final_top = float(final_top)
        final_right = float(final_right)
        final_bottom = float(final_bottom)

        if width_mode == "full":
            # Override left/right if mode is full
            final_left = 0
            final_right = self.width

        # Ensure coordinates are within page bounds (clamp)
        final_left = max(0.0, final_left)
        final_top = max(0.0, final_top)
        final_right = min(float(self.width), final_right)
        final_bottom = min(float(self.height), final_bottom)

        # Ensure valid box (x0<=x1, top<=bottom)
        if final_left > final_right:
            logger.warning(f"Calculated left ({final_left}) > right ({final_right}). Swapping.")
            final_left, final_right = final_right, final_left
        if final_top > final_bottom:
            logger.warning(f"Calculated top ({final_top}) > bottom ({final_bottom}). Swapping.")
            final_top, final_bottom = final_bottom, final_top

        from natural_pdf.elements.region import Region

        region = Region(self, (final_left, final_top, final_right, final_bottom))
        return region

    def get_elements(
        self, apply_exclusions=True, debug_exclusions: bool = False
    ) -> List["Element"]:
        """
        Get all elements on this page.

        Args:
            apply_exclusions: Whether to apply exclusion regions (default: True).
            debug_exclusions: Whether to output detailed exclusion debugging info (default: False).

        Returns:
            List of all elements on the page, potentially filtered by exclusions.
        """
        # Get all elements from the element manager
        all_elements = self.get_all_elements_raw()

        # Apply exclusions if requested
        if apply_exclusions:
            return self._filter_elements_by_exclusions(
                all_elements, debug_exclusions=debug_exclusions
            )
        else:
            if debug_exclusions:
                print(
                    f"Page {self.index}: get_elements returning all {len(all_elements)} elements (exclusions not applied)."
                )
            return all_elements

    def filter_elements(
        self, elements: List["Element"], selector: str, **kwargs
    ) -> List["Element"]:
        """
        Filter a list of elements based on a selector.

        Args:
            elements: List of elements to filter
            selector: CSS-like selector string
            **kwargs: Additional filter parameters

        Returns:
            List of elements that match the selector
        """
        from natural_pdf.selectors.parser import parse_selector, selector_to_filter_func

        # Parse the selector
        selector_obj = parse_selector(selector)

        # Create filter function from selector
        filter_func = selector_to_filter_func(selector_obj, **kwargs)

        # Apply the filter to the elements
        matching_elements = [element for element in elements if filter_func(element)]

        # Sort elements in reading order if requested
        if kwargs.get("reading_order", True):
            if all(hasattr(el, "top") and hasattr(el, "x0") for el in matching_elements):
                matching_elements.sort(key=lambda el: (el.top, el.x0))
            else:
                logger.warning(
                    "Cannot sort elements in reading order: Missing required attributes (top, x0)."
                )

        return matching_elements

    def until(
        self,
        selector: str,
        include_endpoint: bool = True,
        *,
        text: Optional[Union[str, Sequence[str]]] = None,
        apply_exclusions: bool = True,
        regex: bool = False,
        case: bool = True,
        text_tolerance: Optional[Dict[str, Any]] = None,
        auto_text_tolerance: Optional[Union[bool, Dict[str, Any]]] = None,
        reading_order: bool = True,
    ) -> Any:
        """
        Select content from the top of the page until matching selector.

        Args:
            selector: CSS-like selector string
            include_endpoint: Whether to include the endpoint element in the region
            **kwargs: Additional selection parameters

        Returns:
            Region object representing the selected content

        Examples:
            >>> page.until('text:contains("Conclusion")')  # Select from top to conclusion
            >>> page.until('line[width>=2]', include_endpoint=False)  # Select up to thick line
        """
        # Find the target element
        target = self.find(
            selector,
            text=text,
            apply_exclusions=apply_exclusions,
            regex=regex,
            case=case,
            text_tolerance=text_tolerance,
            auto_text_tolerance=auto_text_tolerance,
            reading_order=reading_order,
        )
        if not target:
            # If target not found, return a default region (full page)
            from natural_pdf.elements.region import Region

            return Region(self, (0, 0, self.width, self.height))

        # Create a region from the top of the page to the target
        from natural_pdf.elements.region import Region

        # Ensure target has positional attributes before using them
        target_top = getattr(target, "top", 0)
        target_bottom = getattr(target, "bottom", self.height)

        if include_endpoint:
            # Include the target element
            region = Region(self, (0, 0, self.width, target_bottom))
        else:
            # Up to the target element
            region = Region(self, (0, 0, self.width, target_top))

        region.end_element = cast(Element, target)
        return region

    def crop(self, bbox: Optional[Bounds] = None, **kwargs: Any) -> Any:
        """
        Crop the page to the specified bounding box.

        This is a direct wrapper around pdfplumber's crop method.

        Args:
            bbox: Bounding box (x0, top, x1, bottom) or None
            **kwargs: Additional parameters (top, bottom, left, right)

        Returns:
            Cropped page object (pdfplumber.Page)
        """
        # Returns the pdfplumber page object, not a natural-pdf Page
        resolved_bbox: Optional[Tuple[float, float, float, float]]
        if bbox is None:
            resolved_bbox = None
        else:
            bbox_values = extract_bbox(bbox)
            if bbox_values is None:
                raise TypeError("crop expects a 4-tuple bbox or SupportsBBox-compatible object")
            resolved_bbox = bbox_values

        crop_arg: Any = resolved_bbox if resolved_bbox is not None else None
        return self._page.crop(crop_arg, **kwargs)

    def extract_text(
        self,
        preserve_whitespace: bool = True,
        preserve_line_breaks: bool = True,
        use_exclusions: bool = True,
        debug_exclusions: bool = False,
        content_filter=None,
        *,
        layout: bool = True,
        x_density: Optional[float] = None,
        y_density: Optional[float] = None,
        x_tolerance: Optional[float] = None,
        y_tolerance: Optional[float] = None,
        line_dir: Optional[str] = None,
        char_dir: Optional[str] = None,
        strip_final: bool = False,
        strip_empty: bool = False,
        bidi: bool = True,
    ) -> str:
        """
        Extract text from this page, respecting exclusions and using pdfplumber's
        layout engine (chars_to_textmap) if layout arguments are provided or default.

        Args:
            preserve_line_breaks: When False, collapse newlines into spaces for a flattened string.
            use_exclusions: Whether to apply exclusion regions (default: True).
                            Note: Filtering logic is now always applied if exclusions exist.
            debug_exclusions: Whether to output detailed exclusion debugging info (default: False).
            content_filter: Optional content filter to exclude specific text patterns. Can be:
                - A regex pattern string (characters matching the pattern are EXCLUDED)
                - A callable that takes text and returns True to KEEP the character
                - A list of regex patterns (characters matching ANY pattern are EXCLUDED)
            layout: Whether to enable layout-aware spacing (default: True).
            x_density: Horizontal character density override.
            y_density: Vertical line density override.
            x_tolerance: Horizontal clustering tolerance.
            y_tolerance: Vertical clustering tolerance.
            line_dir: Line reading direction override.
            char_dir: Character reading direction override.
            strip_final: When True, strip trailing whitespace from the combined text.
            strip_empty: When True, drop entirely blank lines from the output.
            bidi: Whether to apply bidi reordering when RTL text is detected (default: True).

        Returns:
            Extracted text as string, potentially with layout-based spacing.
        """
        logger.debug(
            "Page %s: extract_text called with layout=%s, x_density=%s, y_density=%s",
            self.number,
            layout,
            x_density,
            y_density,
        )
        debug = debug_exclusions

        # 1. Get Word Elements (triggers load_elements if needed)
        word_elements = self.words
        if not word_elements:
            logger.debug(f"Page {self.number}: No word elements found.")
            return ""

        # 2. Apply element-based exclusions if enabled
        # Check both page-level and PDF-level exclusions
        has_exclusions = bool(self._exclusions) or (
            hasattr(self, "_parent")
            and self._parent
            and hasattr(self._parent, "_exclusions")
            and self._parent._exclusions
        )
        if use_exclusions and has_exclusions:
            # Filter word elements through _filter_elements_by_exclusions
            # This handles both element-based and region-based exclusions
            word_elements = self._filter_elements_by_exclusions(
                word_elements, debug_exclusions=debug
            )
            if debug:
                logger.debug(
                    f"Page {self.number}: {len(word_elements)} words remaining after exclusion filtering."
                )

        # 3. Get region-based exclusions for spatial filtering
        exclusion_regions = []
        apply_exclusions_flag = use_exclusions
        if apply_exclusions_flag and has_exclusions:
            exclusion_regions = self._get_exclusion_regions(include_callable=True, debug=debug)
            if debug:
                logger.debug(
                    f"Page {self.number}: Found {len(exclusion_regions)} region exclusions for spatial filtering."
                )
        elif debug:
            logger.debug(f"Page {self.number}: Not applying exclusions.")

        # 4. Collect All Character Dictionaries from remaining Word Elements
        all_char_dicts = []
        for word in word_elements:
            all_char_dicts.extend(getattr(word, "_char_dicts", []))

        # 5. Spatially Filter Characters (only by regions, elements already filtered above)
        filtered_chars = filter_chars_spatially(
            char_dicts=all_char_dicts,
            exclusion_regions=exclusion_regions,
            target_region=None,  # No target region for full page extraction
            debug=debug,
        )

        # 5. Generate Text Layout using Utility
        # Pass page bbox as layout context
        page_bbox = (0, 0, self.width, self.height)
        # Merge PDF-level default tolerances if caller did not override
        merged_kwargs = {
            "layout": layout,
            "x_density": x_density,
            "y_density": y_density,
            "x_tolerance": x_tolerance,
            "y_tolerance": y_tolerance,
            "line_dir": line_dir,
            "char_dir": char_dir,
        }
        merged_kwargs = {
            key: value
            for key, value in merged_kwargs.items()
            if value is not None or key == "layout"
        }
        tol_keys = ["x_tolerance", "x_tolerance_ratio", "y_tolerance"]
        for k in tol_keys:
            if k not in merged_kwargs:
                if k in self._config:
                    merged_kwargs[k] = self._config[k]
                elif hasattr(self._parent, "_config") and k in self._parent._config:
                    merged_kwargs[k] = self._parent._config[k]

        # Add content_filter to kwargs if provided
        if content_filter is not None:
            merged_kwargs["content_filter"] = content_filter

        result = generate_text_layout(
            char_dicts=filtered_chars,
            layout_context_bbox=page_bbox,
            user_kwargs=merged_kwargs,
        )

        if bidi and result:
            # Quick check for any RTL character
            import unicodedata

            def _contains_rtl(s):
                return any(unicodedata.bidirectional(ch) in ("R", "AL", "AN") for ch in s)

            if _contains_rtl(result):
                try:
                    from bidi.algorithm import get_display  # type: ignore

                    from natural_pdf.utils.bidi_mirror import mirror_brackets

                    normalized_lines = []
                    for line in result.split("\n"):
                        base_dir = (
                            "R"
                            if any(
                                unicodedata.bidirectional(ch) in ("R", "AL", "AN") for ch in line
                            )
                            else "L"
                        )
                        raw_display = get_display(line, base_dir=base_dir)
                        display_line = (
                            raw_display.decode("utf-8", errors="ignore")
                            if isinstance(raw_display, bytes)
                            else str(raw_display)
                        )
                        normalized_lines.append(mirror_brackets(display_line))
                    result = "\n".join(normalized_lines)
                except ModuleNotFoundError:
                    pass  # silently skip if python-bidi not available

        if strip_empty and result:
            result = "\n".join(line for line in result.splitlines() if line.strip())

        if strip_final and result:
            result = "\n".join(line.rstrip() for line in result.splitlines()).strip()

        if result and not preserve_line_breaks:
            normalized = result.replace("\r\n", "\n").replace("\r", "\n")
            if preserve_whitespace:
                result = re.sub(r"\s*\n\s*", " ", normalized)
            else:
                result = " ".join(normalized.split())

        logger.debug(f"Page {self.number}: extract_text finished, result length: {len(result)}.")
        return result

    def extract_table(
        self,
        method: Optional[str] = None,
        table_settings: Optional[dict] = None,
        use_ocr: bool = False,
        ocr_config: Optional[dict] = None,
        text_options: Optional[Dict] = None,
        cell_extraction_func: Optional[Callable[["Region"], Optional[str]]] = None,
        show_progress: bool = False,
        content_filter=None,
        verticals: Optional[List[float]] = None,
        horizontals: Optional[List[float]] = None,
        structure_engine: Optional[str] = None,
    ) -> TableResult:
        """
        Extract the largest table from this page using enhanced region-based extraction.

        Args:
            method: Method to use: 'tatr', 'pdfplumber', 'text', 'stream', 'lattice', or None (auto-detect).
            table_settings: Settings for pdfplumber table extraction.
            use_ocr: Whether to use OCR for text extraction (currently only applicable with 'tatr' method).
            ocr_config: OCR configuration parameters.
            text_options: Dictionary of options for the 'text' method.
            cell_extraction_func: Optional callable function that takes a cell Region object
                                    and returns its string content. For 'text' method only.
            show_progress: If True, display a progress bar during cell text extraction for the 'text' method.
            content_filter: Optional content filter to apply during cell text extraction. Can be:
                - A regex pattern string (characters matching the pattern are EXCLUDED)
                - A callable that takes text and returns True to KEEP the character
                - A list of regex patterns (characters matching ANY pattern are EXCLUDED)
            verticals: Optional list of x-coordinates for explicit vertical table lines.
            horizontals: Optional list of y-coordinates for explicit horizontal table lines.
            structure_engine: Optional structure detection engine forwarded to the underlying
                region so provider-backed engines (e.g., TATR structure consumers) can be leveraged
                before falling back to standard extraction methods.

        Returns:
            TableResult: A sequence-like object containing table rows that also provides .to_df() for pandas conversion.
        """
        # Create a full-page region and delegate to its enhanced extract_table method
        page_region = self.create_region(0, 0, self.width, self.height)
        return page_region.extract_table(
            method=method,
            table_settings=table_settings,
            use_ocr=use_ocr,
            ocr_config=ocr_config,
            text_options=text_options,
            cell_extraction_func=cell_extraction_func,
            show_progress=show_progress,
            content_filter=content_filter,
            verticals=verticals,
            horizontals=horizontals,
            structure_engine=structure_engine,
        )

    def extract_tables(
        self,
        method: Optional[str] = None,
        table_settings: Optional[dict] = None,
        check_tatr: bool = True,
    ) -> List[List[List[Optional[str]]]]:
        """
        Extract all tables from this page with enhanced method support.

        Args:
            method: Method to use: 'pdfplumber', 'stream', 'lattice', or None (auto-detect).
                    'stream' uses text-based strategies, 'lattice' uses line-based strategies.
                    Note: 'tatr' and 'text' methods are not supported for extract_tables.
            table_settings: Settings for pdfplumber table extraction.
            check_tatr: If True (default), first check for TATR-detected table regions
                        and extract from those before falling back to pdfplumber methods.

        Returns:
            List of tables, where each table is a list of rows, and each row is a list of cell values.
        """
        base_settings = table_settings.copy() if table_settings else {}

        def _normalise_table(table: Sequence[Sequence[Optional[str]]]) -> List[List[str]]:
            return [[cell if isinstance(cell, str) else "" for cell in row] for row in table]

        def _as_optional_rows(table: Sequence[Sequence[str]]) -> List[List[Optional[str]]]:
            return [[cell for cell in row] for row in table]

        def _process_pdfplumber_tables(
            raw_tables: Optional[Sequence[Sequence[Sequence[Any]]]],
        ) -> List[List[List[Optional[str]]]]:
            if not raw_tables:
                return []
            processed: List[List[List[Optional[str]]]] = []
            for table in raw_tables:
                normalized_rows: List[List[str]] = []
                for row in table:
                    normalized_row: List[str] = []
                    for cell in row:
                        text_value = "" if cell is None else str(cell)
                        if text_value:
                            text_value = self._apply_rtl_processing_to_text(text_value)
                        normalized_row.append(text_value)
                    normalized_rows.append(normalized_row)
                processed.append(_as_optional_rows(normalized_rows))
            return processed

        def _apply_text_strategy_overrides(settings: Dict[str, Any]) -> None:
            if "text" not in (
                settings.get("vertical_strategy"),
                settings.get("horizontal_strategy"),
            ):
                return

            if "text_x_tolerance" not in settings and "x_tolerance" not in settings:
                x_tol = self.get_config("x_tolerance", None, scope="page")
                if x_tol is not None:
                    settings.setdefault("text_x_tolerance", x_tol)
            if "text_y_tolerance" not in settings and "y_tolerance" not in settings:
                y_tol = self.get_config("y_tolerance", None, scope="page")
                if y_tol is not None:
                    settings.setdefault("text_y_tolerance", y_tol)

            if "snap_tolerance" not in settings and "snap_x_tolerance" not in settings:
                y_tol = self.get_config("y_tolerance", 1, scope="page")
                snap = max(1, round(float(y_tol) * 0.9))
                settings.setdefault("snap_tolerance", snap)
            if "join_tolerance" not in settings and "join_x_tolerance" not in settings:
                join = settings.get("snap_tolerance", 1)
                settings.setdefault("join_tolerance", join)
                settings.setdefault("join_x_tolerance", join)
                settings.setdefault("join_y_tolerance", join)

        def _run_pdfplumber(
            overrides: Optional[Dict[str, Any]] = None, *, force: bool = False
        ) -> List[List[List[Optional[str]]]]:
            settings = base_settings.copy()
            if overrides:
                for key, value in overrides.items():
                    if force or key not in settings:
                        settings[key] = value
            _apply_text_strategy_overrides(settings)
            raw_tables = self._page.extract_tables(settings)
            return _process_pdfplumber_tables(raw_tables)

        def _auto_detect_tables() -> List[List[List[Optional[str]]]]:
            strategies = (
                ("lattice", {"vertical_strategy": "lines", "horizontal_strategy": "lines"}),
                ("stream", {"vertical_strategy": "text", "horizontal_strategy": "text"}),
            )
            for label, overrides in strategies:
                try:
                    tables = _run_pdfplumber(overrides, force=True)
                    if tables:
                        logger.debug(
                            f"Page {self.number}: '{label}' strategy found {len(tables)} tables"
                        )
                        return tables
                except Exception as exc:
                    logger.debug(f"Page {self.number}: '{label}' strategy failed: {exc}")
            return []

        def _extract_tables_via_tatr() -> List[List[List[Optional[str]]]]:
            regions = self.find_all("region[type=table][model=tatr]")
            if not regions:
                logger.debug(
                    f"Page {self.number}: No TATR table regions found, using pdfplumber methods"
                )
                return []

            extracted: List[List[List[Optional[str]]]] = []
            for table_region in regions:
                table_data = table_region.extract_table(method="tatr")
                if table_data:
                    extracted.append(_as_optional_rows(_normalise_table(table_data)))

            if extracted:
                logger.debug(
                    f"Page {self.number}: Successfully extracted {len(extracted)} tables from TATR regions"
                )
            else:
                logger.debug(
                    f"Page {self.number}: TATR regions found but no tables extracted, falling back to pdfplumber"
                )
            return extracted

        if check_tatr:
            try:
                tatr_tables = _extract_tables_via_tatr()
                if tatr_tables:
                    return tatr_tables
            except Exception as exc:
                logger.debug(
                    f"Page {self.number}: Error checking TATR regions: {exc}, falling back to pdfplumber"
                )

        if method is None:
            auto_tables = _auto_detect_tables()
            if auto_tables:
                return auto_tables
            logger.debug(
                f"Page {self.number}: Auto-detection failed to find tables. Returning empty list."
            )
            return []

        settings = table_settings.copy() if table_settings else {}

        if check_tatr:
            try:
                tatr_matches = _extract_tables_via_tatr()
                if tatr_matches:
                    logger.debug(
                        f"Page {self.number}: Returning {len(tatr_matches)} tables from TATR regions."
                    )
                    return tatr_matches
            except Exception as exc:
                logger.debug(
                    f"Page {self.number}: Error checking TATR regions: {exc}; falling back to pdfplumber."
                )

        page_region = self.to_region()
        try:
            return page_region.extract_tables(method=method, table_settings=settings)
        except Exception as exc:
            logger.debug(f"Page {self.number}: Table extraction failed: {exc}")
            return []

    def _load_elements(self):
        """Load all elements from the page via ElementManager."""
        self._element_mgr.load_elements()

    def _create_char_elements(self):
        """DEPRECATED: Use self._element_mgr.chars"""
        logger.warning("_create_char_elements is deprecated. Access via self._element_mgr.chars.")
        return self._element_mgr.chars  # Delegate

    def _process_font_information(self, char_dict):
        """DEPRECATED: Handled by ElementManager"""
        logger.warning("_process_font_information is deprecated. Handled by ElementManager.")
        # ElementManager handles this internally
        pass

    def _group_chars_into_words(self, keep_spaces=True, font_attrs=None):
        """DEPRECATED: Use self._element_mgr.words"""
        logger.warning("_group_chars_into_words is deprecated. Access via self._element_mgr.words.")
        return self._element_mgr.words  # Delegate

    def _process_line_into_words(self, line_chars, keep_spaces, font_attrs):
        """DEPRECATED: Handled by ElementManager"""
        logger.warning("_process_line_into_words is deprecated. Handled by ElementManager.")
        pass

    def _check_font_attributes_match(self, char, prev_char, font_attrs):
        """DEPRECATED: Handled by ElementManager"""
        logger.warning("_check_font_attributes_match is deprecated. Handled by ElementManager.")
        pass

    def _create_word_element(self, chars, font_attrs):
        """DEPRECATED: Handled by ElementManager"""
        logger.warning("_create_word_element is deprecated. Handled by ElementManager.")
        pass

    @property
    def chars(self) -> List[Any]:
        """Get all character elements on this page."""
        return self._element_mgr.chars

    @property
    def words(self) -> List[Any]:
        """Get all word elements on this page."""
        return self._element_mgr.words

    @property
    def rects(self) -> List[Any]:
        """Get all rectangle elements on this page."""
        return self._element_mgr.rects

    @property
    def lines(self) -> List[Any]:
        """Get all line elements on this page."""
        return self._element_mgr.lines

    def add_highlight(
        self,
        bbox: Optional[Tuple[float, float, float, float]] = None,
        color: Optional[Union[Tuple, str]] = None,
        label: Optional[str] = None,
        use_color_cycling: bool = False,
        element: Optional[Any] = None,
        annotate: Optional[List[str]] = None,
        existing: str = "append",
    ) -> "Page":
        """
        Add a highlight to a bounding box or the entire page.
        Delegates to the central HighlightingService.

        Args:
            bbox: Bounding box (x0, top, x1, bottom). If None, highlight entire page.
            color: RGBA color tuple/string for the highlight.
            label: Optional label for the highlight.
            use_color_cycling: If True and no label/color, use next cycle color.
            element: Optional original element being highlighted (for attribute extraction).
            annotate: List of attribute names from 'element' to display.
            existing: How to handle existing highlights ('append' or 'replace').

        Returns:
            Self for method chaining.
        """
        target_bbox = bbox if bbox is not None else (0, 0, self.width, self.height)
        self._highlighter.add(
            page_index=self.index,
            bbox=target_bbox,
            color=color,
            label=label,
            use_color_cycling=use_color_cycling,
            element=element,
            annotate=annotate,
            existing=existing,
        )
        return self

    def add_highlight_polygon(
        self,
        polygon: List[Tuple[float, float]],
        color: Optional[Union[Tuple, str]] = None,
        label: Optional[str] = None,
        use_color_cycling: bool = False,
        element: Optional[Any] = None,
        annotate: Optional[List[str]] = None,
        existing: str = "append",
    ) -> "Page":
        """
        Highlight a polygon shape on the page.
        Delegates to the central HighlightingService.

        Args:
            polygon: List of (x, y) points defining the polygon.
            color: RGBA color tuple/string for the highlight.
            label: Optional label for the highlight.
            use_color_cycling: If True and no label/color, use next cycle color.
            element: Optional original element being highlighted (for attribute extraction).
            annotate: List of attribute names from 'element' to display.
            existing: How to handle existing highlights ('append' or 'replace').

        Returns:
            Self for method chaining.
        """
        self._highlighter.add_polygon(
            page_index=self.index,
            polygon=polygon,
            color=color,
            label=label,
            use_color_cycling=use_color_cycling,
            element=element,
            annotate=annotate,
            existing=existing,
        )
        return self

    def save_image(
        self,
        filename: str,
        width: Optional[int] = None,
        labels: bool = True,
        legend_position: str = "right",
        render_ocr: bool = False,
        include_highlights: bool = True,  # Allow saving without highlights
        resolution: float = 144,
        **kwargs,
    ) -> "Page":
        """
        Save the page image to a file, rendering highlights via HighlightingService.

        Args:
            filename: Path to save the image to.
            width: Optional width for the output image.
            labels: Whether to include a legend.
            legend_position: Position of the legend.
            render_ocr: Whether to render OCR text.
            include_highlights: Whether to render highlights.
            resolution: Resolution in DPI for base image rendering (default: 144 DPI, equivalent to previous scale=2.0).
            **kwargs: Additional args for pdfplumber's internal to_image.

        Returns:
            Self for method chaining.
        """
        # Use export() to save the image
        if include_highlights:
            self.export(
                path=filename,
                resolution=resolution,
                width=width,
                labels=labels,
                legend_position=legend_position,
                render_ocr=render_ocr,
                **kwargs,
            )
        else:
            # For saving without highlights, use render() and save manually
            img = self.render(resolution=resolution, **kwargs)
            if img:
                # Resize if width is specified
                if width is not None and width > 0 and img.width > 0:
                    aspect_ratio = img.height / img.width
                    height = int(width * aspect_ratio)
                    try:
                        img = img.resize((width, height), Image.Resampling.LANCZOS)
                    except Exception as e:
                        logger.warning(f"Could not resize image: {e}")

                # Save the image
                try:
                    if os.path.dirname(filename):
                        os.makedirs(os.path.dirname(filename), exist_ok=True)
                    img.save(filename)
                except Exception as e:
                    logger.error(f"Failed to save image to {filename}: {e}")

        return self

    def clear_highlights(self) -> "Page":
        """
        Clear all highlights *from this specific page* via HighlightingService.

        Returns:
            Self for method chaining
        """
        self._highlighter.clear_page(self.index)
        return self

    def analyze_text_styles(
        self, options: Optional[TextStyleOptions] = None
    ) -> "ElementCollection":
        """
        Analyze text elements by style, adding attributes directly to elements.

        This method uses TextStyleAnalyzer to process text elements (typically words)
        on the page. It adds the following attributes to each processed element:
        - style_label: A descriptive or numeric label for the style group.
        - style_key: A hashable tuple representing the style properties used for grouping.
        - style_properties: A dictionary containing the extracted style properties.

        Args:
            options: Optional TextStyleOptions to configure the analysis.
                        If None, the analyzer's default options are used.

        Returns:
            ElementCollection containing all processed text elements with added style attributes.
        """
        analyzer = TextStyleAnalyzer()
        processed_elements_collection = analyzer.analyze(self, options=options)

        metadata = getattr(self, "metadata", None)
        summary = None
        if isinstance(metadata, dict):
            summary = metadata.get("text_styles_summary")
        self._text_styles_summary = summary or {}

        return processed_elements_collection

    def _create_text_elements_from_ocr(
        self, ocr_results: List[Dict[str, Any]], image_width=None, image_height=None
    ) -> List["TextElement"]:
        """DEPRECATED: Use self._element_mgr.create_text_elements_from_ocr"""
        logger.warning(
            "_create_text_elements_from_ocr is deprecated. Use self._element_mgr version."
        )
        return self._element_mgr.create_text_elements_from_ocr(
            ocr_results, image_width, image_height
        )

    def apply_ocr(
        self,
        engine: Optional[str] = None,
        options: Optional["OCROptions"] = None,
        languages: Optional[List[str]] = None,
        min_confidence: Optional[float] = None,
        device: Optional[str] = None,
        resolution: Optional[int] = None,
        detect_only: bool = False,
        apply_exclusions: bool = True,
        replace: bool = True,
    ) -> "Page":
        """Apply OCR directly to this page."""
        normalized_options = normalize_ocr_options(options)
        engine_name = resolve_ocr_engine_name(
            context=self,
            requested=engine,
            options=normalized_options,
            scope="page",
        )
        resolved_languages = resolve_ocr_languages(self, languages, scope="page")
        resolved_min_conf = resolve_ocr_min_confidence(self, min_confidence, scope="page")
        resolved_device = resolve_ocr_device(self, device, scope="page")

        if replace:
            removed = self.remove_ocr_elements()
            if removed:
                logger.info(
                    f"Page {self.number}: Removed {removed} OCR elements before new OCR run."
                )

        final_resolution = (
            resolution
            if resolution is not None
            else getattr(getattr(self, "_parent", None), "_config", {}).get("resolution", 150)
        )

        try:
            ocr_payload = run_ocr_apply(
                target=self,
                context=self,
                engine_name=engine_name,
                resolution=final_resolution,
                languages=resolved_languages,
                min_confidence=resolved_min_conf,
                device=resolved_device,
                detect_only=detect_only,
                options=normalized_options,
                render_kwargs={},
            )
        except Exception as exc:  # pragma: no cover - defensive
            logger.error(f"Page {self.number}: OCR failed: {exc}", exc_info=True)
            raise

        image_width, image_height = ocr_payload.image_size
        if not image_width or not image_height:
            logger.error(f"Page {self.number}: OCR payload missing image dimensions.")
            return self

        scale_x = self.width / image_width if image_width else 1.0
        scale_y = self.height / image_height if image_height else 1.0
        created_elements = self.create_text_elements_from_ocr(
            ocr_payload.results, scale_x=scale_x, scale_y=scale_y
        )

        logger.info(
            f"Page {self.number}: Added {len(created_elements)} OCR elements using '{engine_name}'."
        )
        return self

    def extract_ocr_elements(
        self,
        engine: Optional[str] = None,
        options: Optional["OCROptions"] = None,
        languages: Optional[List[str]] = None,
        min_confidence: Optional[float] = None,
        device: Optional[str] = None,
        resolution: Optional[int] = None,
    ) -> List["TextElement"]:
        """
        Extract text elements using OCR *without* mutating this page's element store.

        Args:
            engine: Name of the OCR engine.
            options: Engine-specific options object or dict.
            languages: List of engine-specific language codes.
            min_confidence: Minimum confidence threshold.
            device: Device to run OCR on.
            resolution: DPI resolution for rendering page image before OCR.

        Returns:
            List of created TextElement objects derived from OCR results for this page.
        """

        logger.info(f"Page {self.number}: Extracting OCR elements (extract only)...")

        normalized_options = normalize_ocr_options(options)
        engine_name = resolve_ocr_engine_name(
            context=self,
            requested=engine,
            options=normalized_options,
            scope="page",
        )
        resolved_languages = resolve_ocr_languages(self, languages, scope="page")
        resolved_min_conf = resolve_ocr_min_confidence(self, min_confidence, scope="page")
        resolved_device = resolve_ocr_device(self, device, scope="page")

        final_resolution = resolution if resolution is not None else 150
        logger.debug(f"  Using rendering resolution: {final_resolution} DPI")

        ocr_payload = run_ocr_extract(
            target=self,
            context=self,
            engine_name=engine_name,
            resolution=final_resolution,
            languages=resolved_languages,
            min_confidence=resolved_min_conf,
            device=resolved_device,
            detect_only=False,
            options=normalized_options,
            render_kwargs={},
        )

        results = ocr_payload.results
        image_width, image_height = ocr_payload.image_size
        if not image_width or not image_height:
            logger.error("  OCR payload missing image dimensions; aborting extraction.")
            return []

        # Convert results but DO NOT add to ElementManager
        logger.debug("  Converting OCR results to TextElements (extract only)...")
        temp_elements = []
        scale_x = self.width / image_width if image_width else 1
        scale_y = self.height / image_height if image_height else 1
        for result in results:
            if not isinstance(result, Mapping):
                raise TypeError(f"OCR result has unexpected type: {type(result).__name__}")

            bbox_raw = result.get("bbox")
            text_value = result.get("text", "")
            confidence = result.get("confidence", 0.0)
            if not isinstance(bbox_raw, (list, tuple)) or len(bbox_raw) != 4:
                raise ValueError("OCR result missing bbox")
            x0, top, x1, bottom = [float(cast(Union[int, float, str], c)) for c in bbox_raw]
            elem_data = {
                "text": str(text_value),
                "confidence": float(confidence) if confidence is not None else 0.0,
                "x0": x0 * scale_x,
                "top": top * scale_y,
                "x1": x1 * scale_x,
                "bottom": bottom * scale_y,
                "width": (x1 - x0) * scale_x,
                "height": (bottom - top) * scale_y,
                "object_type": "text",  # Using text for temporary elements
                "source": "ocr",
                "fontname": "OCR-extract",  # Different name for clarity
                "size": 10.0,
                "page_number": self.number,
            }
            temp_elements.append(TextElement(elem_data, self))

        logger.info(f"  Created {len(temp_elements)} TextElements from OCR (extract only).")
        return temp_elements

    @property
    def size(self) -> Tuple[float, float]:
        """Get the size of the page in points."""
        return (self._page.width, self._page.height)

    @property
    def layout_analyzer(self) -> "LayoutAnalyzer":
        """Get or create the layout analyzer for this page."""
        if self._layout_analyzer is None:
            self._layout_analyzer = LayoutAnalyzer(self)
        return self._layout_analyzer

    def analyze_layout(
        self,
        engine: Optional[str] = None,
        options: Optional["LayoutOptions"] = None,
        confidence: Optional[float] = None,
        classes: Optional[List[str]] = None,
        exclude_classes: Optional[List[str]] = None,
        device: Optional[str] = None,
        existing: str = "replace",
        model_name: Optional[str] = None,
        client: Optional[Any] = None,  # Add client parameter
    ) -> "ElementCollection[Region]":
        """
        Analyze the page layout using the configured layout engine.
        Adds detected Region objects to the page's element manager.

        Returns:
            ElementCollection containing the detected Region objects.
        """
        analyzer = self.layout_analyzer

        # Clear existing detected regions if 'replace' is specified
        if existing == "replace":
            self.clear_detected_layout_regions()

        # The analyzer's analyze_layout method already adds regions to the page
        # and its element manager. We just need to retrieve them.
        analyzer.analyze_layout(
            engine=engine,
            options=options,
            confidence=confidence,
            classes=classes,
            exclude_classes=exclude_classes,
            device=device,
            existing=existing,
            model_name=model_name,
            client=client,  # Pass client down
        )

        # Retrieve the detected regions from the element manager
        # Filter regions based on source='detected' and potentially the model used if available
        detected_regions = [
            r
            for r in self._element_mgr.regions
            if r.source == "detected" and (not engine or getattr(r, "model", None) == engine)
        ]

        return ElementCollection(detected_regions)

    def clear_detected_layout_regions(self) -> "Page":
        """
        Removes all regions from this page that were added by layout analysis
        (i.e., regions where `source` attribute is 'detected').

        This clears the regions both from the page's internal `_regions['detected']` list
        and from the ElementManager's internal list of regions.

        Returns:
            Self for method chaining.
        """
        removed_count = self.remove_regions(source="detected")
        if removed_count:
            logger.info(f"Page {self.index}: Cleared {removed_count} detected layout regions.")
        else:
            logger.debug(f"Page {self.index}: No detected layout regions to clear.")
        return self

    def get_section_between(
        self,
        start_element=None,
        end_element=None,
        include_boundaries="both",
        orientation="vertical",
    ) -> Optional["Region"]:  # Return Optional
        """
        Get a section between two elements on this page.

        Args:
            start_element: Element marking the start of the section
            end_element: Element marking the end of the section
            include_boundaries: How to include boundary elements: 'start', 'end', 'both', or 'none'
            orientation: 'vertical' (default) or 'horizontal' - determines section direction

        Returns:
            Region representing the section
        """
        # Create a full-page region to operate within
        page_region = self.create_region(0, 0, self.width, self.height)

        # Delegate to the region's method
        try:
            return page_region.get_section_between(
                start_element=start_element,
                end_element=end_element,
                include_boundaries=include_boundaries,
                orientation=orientation,
            )
        except Exception as e:
            logger.error(
                f"Error getting section between elements on page {self.index}: {e}", exc_info=True
            )
            return None

    def split(self, divider, **kwargs: Any) -> "ElementCollection[Region]":
        """Divide the page into sections based on the provided divider elements."""
        base_region = self.create_region(0, 0, self.width, self.height)
        return base_region.split(divider, **kwargs)

    def get_sections(
        self,
        start_elements: Union[str, Sequence[Element], ElementCollection, None] = None,
        end_elements: Union[str, Sequence[Element], ElementCollection, None] = None,
        include_boundaries: str = "start",
        y_threshold: float = 5.0,
        bounding_box: Optional[Bounds] = None,
        orientation: str = "vertical",
        **kwargs: Any,
    ) -> "ElementCollection[Region]":
        """Delegate section extraction to the Region implementation."""

        if bounding_box is not None:
            x0, top, x1, bottom = bounding_box
            base_region = self.create_region(x0, top, x1, bottom)
        else:
            base_region = self.create_region(0, 0, self.width, self.height)

        return base_region.get_sections(
            start_elements=start_elements,
            end_elements=end_elements,
            include_boundaries=include_boundaries,
            orientation=orientation,
            y_threshold=y_threshold,
            **kwargs,
        )

    def __repr__(self) -> str:
        """String representation of the page."""
        return f"<Page number={self.number} index={self.index}>"

    def show_preview(
        self,
        temporary_highlights: List[Dict],
        resolution: float = 144,
        width: Optional[int] = None,
        labels: bool = True,
        legend_position: str = "right",
        render_ocr: bool = False,
    ) -> Optional[Image.Image]:
        """
        Generates and returns a non-stateful preview image containing only
        the provided temporary highlights.

        Args:
            temporary_highlights: List of highlight data dictionaries (as prepared by
                                    ElementCollection._prepare_highlight_data).
            resolution: Resolution in DPI for rendering (default: 144 DPI, equivalent to previous scale=2.0).
            width: Optional width for the output image.
            labels: Whether to include a legend.
            legend_position: Position of the legend.
            render_ocr: Whether to render OCR text.

        Returns:
            PIL Image object of the preview, or None if rendering fails.
        """
        try:
            # Delegate rendering to the highlighter service's preview method
            img = self._highlighter.render_preview(
                page_index=self.index,
                temporary_highlights=temporary_highlights,
                resolution=resolution,
                labels=labels,
                legend_position=legend_position,
                render_ocr=render_ocr,
            )
        except AttributeError:
            logger.error("HighlightingService does not have the required 'render_preview' method.")
            return None
        except Exception as e:
            logger.error(
                f"Error calling highlighter.render_preview for page {self.index}: {e}",
                exc_info=True,
            )
            return None

        # Return the rendered image directly
        return img

    @property
    def text_style_labels(self) -> List[str]:
        """
        Get a sorted list of unique text style labels found on the page.

        Runs text style analysis with default options if it hasn't been run yet.
        To use custom options, call `analyze_text_styles(options=...)` explicitly first.

        Returns:
            A sorted list of unique style label strings.
        """
        summary: dict[str, Any] = getattr(self, "_text_styles_summary", {})
        if not summary:
            metadata = getattr(self, "metadata", None)
            if isinstance(metadata, dict):
                meta_summary = metadata.get("text_styles_summary")
                if isinstance(meta_summary, dict):
                    summary = meta_summary
                    self._text_styles_summary = summary

        if not summary:
            logger.debug(f"Page {self.number}: Running default text style analysis to get labels.")
            self.analyze_text_styles()
            summary = getattr(self, "_text_styles_summary", {})

        if summary:
            labels = {
                style_info["label"] for style_info in summary.values() if "label" in style_info
            }
            return sorted(labels)

        logger.warning(f"Page {self.number}: Text style summary not available after analysis.")
        return []

    def viewer(
        self,
        # elements_to_render: Optional[List['Element']] = None, # No longer needed, from_page handles it
        # include_source_types: List[str] = ['word', 'line', 'rect', 'region'] # No longer needed
    ) -> Optional[Any]:
        """
        Creates and returns an interactive ipywidget for exploring elements on this page.

        Uses InteractiveViewerWidget.from_page() to create the viewer.

        Returns:
            A InteractiveViewerWidget instance ready for display in Jupyter,
            or None if ipywidgets is not installed or widget creation fails.

        Raises:
            # Optional: Could raise ImportError instead of returning None
            # ImportError: If required dependencies (ipywidgets) are missing.
            ValueError: If image rendering or data preparation fails within from_page.
        """
        # Check for availability using the imported flag and class variable
        if not _IPYWIDGETS_AVAILABLE or InteractiveViewerWidget is None:
            logger.error(
                "Interactive viewer requires 'ipywidgets'. "
                'Please install with: pip install "ipywidgets>=7.0.0,<10.0.0"'
            )
            # raise ImportError("ipywidgets not found.") # Option 1: Raise error
            return None  # Option 2: Return None gracefully

        # If we reach here, InteractiveViewerWidget should be the actual class
        try:
            # Pass self (the Page object) to the factory method
            return InteractiveViewerWidget.from_page(self)
        except Exception as e:
            # Catch potential errors during widget creation (e.g., image rendering)
            logger.error(
                f"Error creating viewer widget from page {self.number}: {e}", exc_info=True
            )
            # raise # Option 1: Re-raise error (might include ValueError from from_page)
            return None  # Option 2: Return None on creation error

    def get_id(self) -> str:
        """Returns a unique identifier for the page (required by Indexable protocol)."""
        # Ensure path is safe for use in IDs (replace problematic chars)
        safe_path = re.sub(r"[^a-zA-Z0-9_-]", "_", str(self.pdf.path))
        return f"pdf_{safe_path}_page_{self.page_number}"

    def get_metadata(self) -> Dict[str, Any]:
        """Returns metadata associated with the page (required by Indexable protocol)."""
        # Add content hash here for sync
        metadata = {
            "pdf_path": str(self.pdf.path),
            "page_number": self.page_number,
            "width": self.width,
            "height": self.height,
            "content_hash": self.get_content_hash(),  # Include the hash
        }
        return metadata

    def get_content(self) -> "Page":
        """
        Returns the primary content object (self) for indexing (required by Indexable protocol).
        SearchService implementations decide how to process this (e.g., call extract_text).
        """
        return self  # Return the Page object itself

    def get_content_hash(self) -> str:
        """Returns a SHA256 hash of the extracted text content (required by Indexable for sync)."""
        # Hash the extracted text (without exclusions for consistency)
        # Consider if exclusions should be part of the hash? For now, hash raw text.
        # Using extract_text directly might be slow if called repeatedly. Cache? TODO: Optimization
        text_content = self.extract_text(
            use_exclusions=False, preserve_whitespace=False
        )  # Normalize whitespace?
        return hashlib.sha256(text_content.encode("utf-8")).hexdigest()

    def save_searchable(self, output_path: Union[str, "Path"], dpi: int = 300):
        """
        Saves the PDF page with an OCR text layer, making content searchable.

        Requires optional dependencies. Install with: pip install "natural-pdf[ocr-save]"

        Note: OCR must have been applied to the pages beforehand
                (e.g., pdf.apply_ocr()).

        Args:
            output_path: Path to save the searchable PDF.
            dpi: Resolution for rendering and OCR overlay (default 300).
        """
        # Import moved here, assuming it's always available now
        from natural_pdf.exporters.searchable_pdf import create_searchable_pdf

        # Convert pathlib.Path to string if necessary
        output_path_str = str(output_path)

        create_searchable_pdf(self, output_path_str, dpi=dpi)
        logger.info(f"Searchable PDF saved to: {output_path_str}")

    def update_text(
        self,
        transform: Callable[[Any], Optional[str]],
        *,
        selector: str = "text",
        apply_exclusions: bool = False,
        max_workers: Optional[int] = None,
        progress_callback: Optional[Callable[[], None]] = None,  # Added progress callback
    ) -> "Page":  # Return self for chaining
        """
        Applies corrections to text elements on this page
        using a user-provided callback function, potentially in parallel.

        Finds text elements on this page matching the *selector* argument and
        calls the ``transform`` for each, passing the element itself.
        Updates the element's text if the callback returns a new string.

        Args:
            transform: A function accepting an element and returning
                        `Optional[str]` (new text or None).
            selector: CSS-like selector string to match text elements.
            apply_exclusions: Whether exclusion regions should be honoured (default: False to
                maintain backward compatibility with the base mixin behaviour).
            max_workers: The maximum number of threads to use for parallel execution.
                            If None or 0 or 1, runs sequentially.
            progress_callback: Optional callback function to call after processing each element.

        Returns:
            Self for method chaining.
        """
        logger.info(
            f"Page {self.number}: Starting text update with callback '{transform.__name__}' (max_workers={max_workers}) and selector='{selector}'"
        )

        target_elements_collection = self.find_all(
            selector=selector, apply_exclusions=apply_exclusions
        )
        target_elements = target_elements_collection.elements  # Get the list

        if not target_elements:
            logger.info(f"Page {self.number}: No text elements found to update.")
            return self

        element_pbar = None
        try:
            element_pbar = tqdm(
                total=len(target_elements),
                desc=f"Updating text Page {self.number}",
                unit="element",
                leave=False,
            )

            processed_count = 0
            updated_count = 0
            error_count = 0

            # Define the task to be run by the worker thread or sequentially
            def _process_element_task(element):
                try:
                    # Call the user-provided callback
                    corrected_text = transform(element)

                    # Validate result type
                    if corrected_text is not None and not isinstance(corrected_text, str):
                        logger.warning(
                            f"Page {self.number}: Correction callback for element '{getattr(element, 'text', '')[:20]}...' returned non-string, non-None type: {type(corrected_text)}. Skipping update."
                        )
                        return element, None, None  # Treat as no correction

                    return element, corrected_text, None  # Return element, result, no error
                except Exception as e:
                    logger.error(
                        f"Page {self.number}: Error applying correction callback to element '{getattr(element, 'text', '')[:30]}...' ({element.bbox}): {e}",
                        exc_info=False,  # Keep log concise
                    )
                    return element, None, e  # Return element, no result, error
                finally:
                    if element_pbar:
                        element_pbar.update(1)
                    if progress_callback:
                        try:
                            progress_callback()
                        except Exception as cb_e:
                            # Log error in callback itself, but don't stop processing
                            logger.error(
                                f"Page {self.number}: Error executing progress_callback: {cb_e}",
                                exc_info=False,
                            )

            # Choose execution strategy based on max_workers
            if max_workers is not None and max_workers > 1:
                logger.info(
                    f"Page {self.number}: Running text update in parallel with {max_workers} workers."
                )
                with concurrent.futures.ThreadPoolExecutor(max_workers=max_workers) as executor:
                    # Submit all tasks
                    future_to_element = {
                        executor.submit(_process_element_task, element): element
                        for element in target_elements
                    }

                    # Process results as they complete (progress_callback called by worker)
                    for future in concurrent.futures.as_completed(future_to_element):
                        processed_count += 1
                        try:
                            element, corrected_text, error = future.result()
                            if error:
                                error_count += 1
                                # Error already logged in worker
                            elif corrected_text is not None:
                                # Apply correction if text changed
                                current_text = getattr(element, "text", None)
                                if corrected_text != current_text:
                                    element.text = corrected_text
                                    updated_count += 1
                        except Exception as exc:
                            # Catch errors from future.result() itself
                            element = future_to_element[future]  # Find original element
                            logger.error(
                                f"Page {self.number}: Internal error retrieving correction result for element {element.bbox}: {exc}",
                                exc_info=True,
                            )
                            error_count += 1
                            # Note: progress_callback was already called in the worker's finally block

            else:
                logger.info(f"Page {self.number}: Running text update sequentially.")
                for element in target_elements:
                    # Call the task function directly (it handles progress_callback)
                    processed_count += 1
                    _element, corrected_text, error = _process_element_task(element)
                    if error:
                        error_count += 1
                    elif corrected_text is not None:
                        # Apply correction if text changed
                        current_text = getattr(_element, "text", None)
                        if corrected_text != current_text:
                            _element.text = corrected_text
                            updated_count += 1

            logger.info(
                f"Page {self.number}: Text update finished. Processed: {processed_count}/{len(target_elements)}, Updated: {updated_count}, Errors: {error_count}."
            )

            return self  # Return self for chaining
        finally:
            if element_pbar:
                element_pbar.close()

    def _get_classification_content(self, model_type: str, **kwargs) -> Union[str, Image.Image]:
        if model_type == "text":
            text_content = self.extract_text(
                layout=False, use_exclusions=False
            )  # Simple join, ignore exclusions for classification
            if not text_content or text_content.isspace():
                raise ValueError("Cannot classify page with 'text' model: No text content found.")
            return text_content
        elif model_type == "vision":
            resolution = kwargs.get("resolution", 150)
            # Use render() for clean image without highlights
            img = self.render(resolution=resolution)
            if img is None:
                raise ValueError(
                    "Cannot classify page with 'vision' model: Failed to render image."
                )
            return img
        else:
            raise ValueError(f"Unsupported model_type for classification: {model_type}")

    def _get_metadata_storage(self) -> Dict[str, Any]:
        # Ensure metadata exists
        if not hasattr(self, "metadata") or self.metadata is None:
            self.metadata = {}
        return self.metadata

    @property
    def skew_angle(self) -> Optional[float]:
        """Get the detected skew angle for this page (if calculated)."""
        return self._skew_angle

    def detect_skew_angle(
        self,
        resolution: int = 72,
        grayscale: bool = True,
        force_recalculate: bool = False,
        **deskew_kwargs,
    ) -> Optional[float]:
        """Detect the skew angle of this page using the deskew provider."""
        if self._skew_angle is not None and not force_recalculate:
            logger.debug(f"Page {self.number}: Returning cached skew angle: {self._skew_angle:.2f}")
            return self._skew_angle

        try:
            angle = run_deskew_detect(
                target=self,
                context=self,
                resolution=resolution,
                grayscale=grayscale,
                deskew_kwargs=deskew_kwargs,
            )
        except ImportError:
            raise
        except Exception as exc:
            logger.warning(f"Page {self.number}: Skew detection failed: {exc}", exc_info=True)
            self._skew_angle = None
            return None

        self._skew_angle = angle
        return angle

    def deskew(
        self,
        resolution: int = 300,
        angle: Optional[float] = None,
        detection_resolution: int = 72,
        **deskew_kwargs,
    ) -> Optional[Image.Image]:
        """
        Creates and returns a deskewed PIL image of the page.

        If `angle` is not provided, it will first try to detect the skew angle
        using `detect_skew_angle` (or use the cached angle if available).

        Args:
            resolution: DPI resolution for the output deskewed image.
            angle: The specific angle (in degrees) to rotate by. If None, detects automatically.
            detection_resolution: DPI resolution used for detection if `angle` is None.
            **deskew_kwargs: Additional keyword arguments passed to `deskew.determine_skew`
                                if automatic detection is performed.

        Returns:
            A deskewed PIL.Image.Image object, or None if rendering/rotation fails.

        Raises:
            ImportError: If the 'deskew' library is not installed.
        """
        try:
            result = run_deskew_apply(
                target=self,
                context=self,
                resolution=resolution,
                angle=angle,
                detection_resolution=detection_resolution,
                grayscale=True,
                deskew_kwargs=deskew_kwargs,
            )
        except ImportError:
            raise
        except Exception as exc:
            logger.error(
                f"Page {self.number}: Error during deskewing image generation: {exc}",
                exc_info=True,
            )
            return None

        return result.image

    # Unified analysis storage (maps to metadata["analysis"])

    @property
    def analyses(self) -> Dict[str, Any]:
        if not hasattr(self, "metadata") or self.metadata is None:
            self.metadata = {}
        return self.metadata.setdefault("analysis", {})

    @analyses.setter
    def analyses(self, value: Dict[str, Any]) -> None:
        if not hasattr(self, "metadata") or self.metadata is None:
            self.metadata = {}
        self.metadata["analysis"] = value

    def inspect(self, limit: int = 30) -> "InspectionSummary":
        """
        Inspect all elements on this page with detailed tabular view.
        Equivalent to page.find_all('*').inspect().

        Args:
            limit: Maximum elements per type to show (default: 30)

        Returns:
            InspectionSummary with element tables showing coordinates,
            properties, and other details for each element
        """
        return self.find_all("*").inspect(limit=limit)

    def remove_text_layer(self) -> "Page":
        """
        Remove all text elements from this page.

        This removes all text elements (words and characters) from the page,
        effectively clearing the text layer.

        Returns:
            Self for method chaining
        """
        logger.info(f"Page {self.number}: Removing all text elements...")

        removed_words, removed_chars = self._element_mgr.clear_text_layer()

        logger.info(
            f"Page {self.number}: Removed {removed_words} words and {removed_chars} characters"
        )
        return self

    def _apply_rtl_processing_to_text(self, text: str) -> str:
        """
        Apply RTL (Right-to-Left) text processing to a string.

        This converts visual order text (as stored in PDFs) to logical order
        for proper display of Arabic, Hebrew, and other RTL scripts.

        Args:
            text: Input text string in visual order

        Returns:
            Text string in logical order
        """
        if not text or not text.strip():
            return text

        # Quick check for RTL characters - if none found, return as-is
        import unicodedata

        def _contains_rtl(s):
            return any(unicodedata.bidirectional(ch) in ("R", "AL", "AN") for ch in s)

        if not _contains_rtl(text):
            return text

        try:
            from bidi.algorithm import get_display  # type: ignore

            from natural_pdf.utils.bidi_mirror import mirror_brackets

            # Apply BiDi algorithm to convert from visual to logical order
            # Process line by line to handle mixed content properly
            processed_lines = []
            for line in text.split("\n"):
                if line.strip():
                    # Determine base direction for this line
                    base_dir = "R" if _contains_rtl(line) else "L"
                    logical_line = get_display(line, base_dir=base_dir)
                    if isinstance(logical_line, bytes):
                        try:
                            logical_line = logical_line.decode("utf-8")
                        except UnicodeDecodeError:
                            logical_line = logical_line.decode("utf-8", "ignore")
                    # Apply bracket mirroring for correct logical order
                    processed_lines.append(mirror_brackets(logical_line))
                else:
                    processed_lines.append(line)

            return "\n".join(processed_lines)

        except (ImportError, Exception):
            # If bidi library is not available or fails, return original text
            return text

    @property
    def images(self) -> List[Any]:
        """Get all embedded raster images on this page."""
        return self._element_mgr.images

    def highlights(self, show: bool = False) -> "HighlightContext":
        """
        Create a highlight context for accumulating highlights.

        This allows for clean syntax to show multiple highlight groups:

        Example:
            with page.highlights() as h:
                h.add(page.find_all('table'), label='tables', color='blue')
                h.add(page.find_all('text:bold'), label='bold text', color='red')
                h.show()

        Or with automatic display:
            with page.highlights(show=True) as h:
                h.add(page.find_all('table'), label='tables')
                h.add(page.find_all('text:bold'), label='bold')
                # Automatically shows when exiting the context

        Args:
            show: If True, automatically show highlights when exiting context

        Returns:
            HighlightContext for accumulating highlights
        """
        from natural_pdf.core.highlighting_service import HighlightContext

        return HighlightContext(self, show_on_exit=show)

Attributes

natural_pdf.Page.chars property

Get all character elements on this page.

natural_pdf.Page.height property

Get page height.

natural_pdf.Page.images property

Get all embedded raster images on this page.

natural_pdf.Page.index property

Get page index (0-based).

natural_pdf.Page.layout_analyzer property

Get or create the layout analyzer for this page.

natural_pdf.Page.lines property

Get all line elements on this page.

natural_pdf.Page.number property

Get page number (1-based).

natural_pdf.Page.page_number property

Get page number (1-based).

natural_pdf.Page.pdf property

Provides public access to the parent PDF object.

natural_pdf.Page.rects property

Get all rectangle elements on this page.

natural_pdf.Page.size property

Get the size of the page in points.

natural_pdf.Page.skew_angle property

Get the detected skew angle for this page (if calculated).

natural_pdf.Page.text_style_labels property

Get a sorted list of unique text style labels found on the page.

Runs text style analysis with default options if it hasn't been run yet. To use custom options, call analyze_text_styles(options=...) explicitly first.

Returns:

Type	Description
List[str]	A sorted list of unique style label strings.

natural_pdf.Page.width property

Get page width.

natural_pdf.Page.words property

Get all word elements on this page.

Functions

natural_pdf.Page.__init__(page, parent, index, font_attrs=None, load_text=True)

Initialize a page wrapper.

Creates an enhanced Page object that wraps a pdfplumber page with additional functionality for spatial navigation, analysis, and AI-powered extraction.

Parameters:

Name	Type	Description	Default
page	Page	The underlying pdfplumber page object that provides raw PDF data.	required
parent	PDF	Parent PDF object that contains this page and provides access to managers and global settings.	required
index	int	Zero-based index of this page in the PDF document.	required
font_attrs		List of font attributes to consider when grouping characters into words. Common attributes include ['fontname', 'size', 'flags']. If None, uses default character-to-word grouping rules.	None
load_text	bool	If True, load and process text elements from the PDF's text layer. If False, skip text layer processing (useful for OCR-only workflows).	True

Note

This constructor is typically called automatically when accessing pages through the PDF.pages collection. Direct instantiation is rarely needed.

Example

# Pages are usually accessed through the PDF object
pdf = npdf.PDF("document.pdf")
page = pdf.pages[0]  # Page object created automatically

# Direct construction (advanced usage)
import pdfplumber
with pdfplumber.open("document.pdf") as plumber_pdf:
    plumber_page = plumber_pdf.pages[0]
    page = Page(plumber_page, pdf, 0, load_text=True)

Source code in natural_pdf/core/page.py

def __init__(
    self,
    page: "PdfPlumberPage",
    parent: "PDF",
    index: int,
    font_attrs=None,
    load_text: bool = True,
):
    """Initialize a page wrapper.

    Creates an enhanced Page object that wraps a pdfplumber page with additional
    functionality for spatial navigation, analysis, and AI-powered extraction.

    Args:
        page: The underlying pdfplumber page object that provides raw PDF data.
        parent: Parent PDF object that contains this page and provides access
            to managers and global settings.
        index: Zero-based index of this page in the PDF document.
        font_attrs: List of font attributes to consider when grouping characters
            into words. Common attributes include ['fontname', 'size', 'flags'].
            If None, uses default character-to-word grouping rules.
        load_text: If True, load and process text elements from the PDF's text layer.
            If False, skip text layer processing (useful for OCR-only workflows).

    Note:
        This constructor is typically called automatically when accessing pages
        through the PDF.pages collection. Direct instantiation is rarely needed.

    Example:
        ```python
        # Pages are usually accessed through the PDF object
        pdf = npdf.PDF("document.pdf")
        page = pdf.pages[0]  # Page object created automatically

        # Direct construction (advanced usage)
        import pdfplumber
        with pdfplumber.open("document.pdf") as plumber_pdf:
            plumber_page = plumber_pdf.pages[0]
            page = Page(plumber_page, pdf, 0, load_text=True)
        ```
    """
    self._page = page
    self._parent = parent
    self._index = index
    self._load_text = load_text
    self._text_styles_summary: Dict[str, Any] = {}
    self._text_styles = None  # Lazy-loaded text style analyzer results
    self._exclusions = []  # List to store exclusion functions/regions
    self._skew_angle: Optional[float] = None  # Stores detected skew angle

    self.metadata: Dict[str, Any] = {}

    # Region management
    self._regions = {
        "detected": [],  # Layout detection results
        "named": {},  # Named regions (name -> region)
    }

    if not hasattr(self._parent, "_config"):
        raise AttributeError(
            "Parent PDF is missing _config; cannot initialise Page configuration"
        )

    # Page-scoped configuration begins as a shallow copy of the parent PDF-level
    # configuration so that auto-computed tolerances or other page-specific
    # values do not overwrite siblings.
    self._config = dict(self._parent._config)

    # Initialize ElementManager, passing font_attrs
    self._element_mgr: ElementManager = ElementManager(
        self, font_attrs=font_attrs, load_text=self._load_text
    )
    # self._highlighter = HighlightingService(self) # REMOVED - Use property accessor
    if not hasattr(self, "metadata") or self.metadata is None:
        self.metadata = {}
    self.metadata.setdefault("analysis", {})

    # Initialize the internal variable with a single underscore
    self._layout_analyzer = None

    self._load_elements()
    self._to_image_cache: Dict[tuple, Optional["Image.Image"]] = {}

    # Flag to prevent infinite recursion when computing exclusions
    self._computing_exclusions = False

natural_pdf.Page.__repr__()

String representation of the page.

Source code in natural_pdf/core/page.py

def __repr__(self) -> str:
    """String representation of the page."""
    return f"<Page number={self.number} index={self.index}>"

natural_pdf.Page.add_element(element, element_type='words')

Add an element to the backing collection.

Source code in natural_pdf/core/page.py

def add_element(self, element: Any, element_type: str = "words") -> bool:
    """Add an element to the backing collection."""
    return bool(self._element_mgr.add_element(element, element_type))

natural_pdf.Page.add_highlight(bbox=None, color=None, label=None, use_color_cycling=False, element=None, annotate=None, existing='append')

Add a highlight to a bounding box or the entire page. Delegates to the central HighlightingService.

Parameters:

Name	Type	Description	Default
bbox	Optional[Tuple[float, float, float, float]]	Bounding box (x0, top, x1, bottom). If None, highlight entire page.	None
color	Optional[Union[Tuple, str]]	RGBA color tuple/string for the highlight.	None
label	Optional[str]	Optional label for the highlight.	None
use_color_cycling	bool	If True and no label/color, use next cycle color.	False
element	Optional[Any]	Optional original element being highlighted (for attribute extraction).	None
annotate	Optional[List[str]]	List of attribute names from 'element' to display.	None
existing	str	How to handle existing highlights ('append' or 'replace').	'append'

Returns:

Type	Description
Page	Self for method chaining.

Source code in natural_pdf/core/page.py

def add_highlight(
    self,
    bbox: Optional[Tuple[float, float, float, float]] = None,
    color: Optional[Union[Tuple, str]] = None,
    label: Optional[str] = None,
    use_color_cycling: bool = False,
    element: Optional[Any] = None,
    annotate: Optional[List[str]] = None,
    existing: str = "append",
) -> "Page":
    """
    Add a highlight to a bounding box or the entire page.
    Delegates to the central HighlightingService.

    Args:
        bbox: Bounding box (x0, top, x1, bottom). If None, highlight entire page.
        color: RGBA color tuple/string for the highlight.
        label: Optional label for the highlight.
        use_color_cycling: If True and no label/color, use next cycle color.
        element: Optional original element being highlighted (for attribute extraction).
        annotate: List of attribute names from 'element' to display.
        existing: How to handle existing highlights ('append' or 'replace').

    Returns:
        Self for method chaining.
    """
    target_bbox = bbox if bbox is not None else (0, 0, self.width, self.height)
    self._highlighter.add(
        page_index=self.index,
        bbox=target_bbox,
        color=color,
        label=label,
        use_color_cycling=use_color_cycling,
        element=element,
        annotate=annotate,
        existing=existing,
    )
    return self

natural_pdf.Page.add_highlight_polygon(polygon, color=None, label=None, use_color_cycling=False, element=None, annotate=None, existing='append')

Highlight a polygon shape on the page. Delegates to the central HighlightingService.

Parameters:

Name	Type	Description	Default
polygon	List[Tuple[float, float]]	List of (x, y) points defining the polygon.	required
color	Optional[Union[Tuple, str]]	RGBA color tuple/string for the highlight.	None
label	Optional[str]	Optional label for the highlight.	None
use_color_cycling	bool	If True and no label/color, use next cycle color.	False
element	Optional[Any]	Optional original element being highlighted (for attribute extraction).	None
annotate	Optional[List[str]]	List of attribute names from 'element' to display.	None
existing	str	How to handle existing highlights ('append' or 'replace').	'append'

Returns:

Type	Description
Page	Self for method chaining.

Source code in natural_pdf/core/page.py

def add_highlight_polygon(
    self,
    polygon: List[Tuple[float, float]],
    color: Optional[Union[Tuple, str]] = None,
    label: Optional[str] = None,
    use_color_cycling: bool = False,
    element: Optional[Any] = None,
    annotate: Optional[List[str]] = None,
    existing: str = "append",
) -> "Page":
    """
    Highlight a polygon shape on the page.
    Delegates to the central HighlightingService.

    Args:
        polygon: List of (x, y) points defining the polygon.
        color: RGBA color tuple/string for the highlight.
        label: Optional label for the highlight.
        use_color_cycling: If True and no label/color, use next cycle color.
        element: Optional original element being highlighted (for attribute extraction).
        annotate: List of attribute names from 'element' to display.
        existing: How to handle existing highlights ('append' or 'replace').

    Returns:
        Self for method chaining.
    """
    self._highlighter.add_polygon(
        page_index=self.index,
        polygon=polygon,
        color=color,
        label=label,
        use_color_cycling=use_color_cycling,
        element=element,
        annotate=annotate,
        existing=existing,
    )
    return self

natural_pdf.Page.add_region(region, name=None, *, source=None)

Add a region to the page.

Parameters:

Name	Type	Description	Default
region	Region	Region object to add	required
name	Optional[str]	Optional name for the region	None
source	Optional[str]	Optional provenance label; if provided it will be recorded on the region.	None

Returns:

Type	Description
Page	Self for method chaining

Source code in natural_pdf/core/page.py

def add_region(
    self, region: "Region", name: Optional[str] = None, *, source: Optional[str] = None
) -> "Page":
    """
    Add a region to the page.

    Args:
        region: Region object to add
        name: Optional name for the region
        source: Optional provenance label; if provided it will be recorded on the region.

    Returns:
        Self for method chaining
    """
    # Check if it's actually a Region object
    if not isinstance(region, Region):
        raise TypeError("region must be a Region object")

    # Respect an explicitly provided source, otherwise keep any existing label.
    if source is not None:
        region.source = source
    elif getattr(region, "source", None) is None:
        region.source = "named"

    if name:
        region.name = name
        # Add to named regions dictionary (overwriting if name already exists)
        self._regions["named"][name] = region
    else:
        # Add to detected regions list (unnamed but registered)
        self._regions["detected"].append(region)

    # Add to element manager for selector queries
    self._element_mgr.add_region(region)

    return self

natural_pdf.Page.add_regions(regions, prefix=None, *, source=None)

Add multiple regions to the page.

Parameters:

Name	Type	Description	Default
regions	List[Region]	List of Region objects to add	required
prefix	Optional[str]	Optional prefix for automatic naming (regions will be named prefix_1, prefix_2, etc.)	None
source	Optional[str]	Optional provenance label applied to each region.	None

Returns:

Type	Description
Page	Self for method chaining

Source code in natural_pdf/core/page.py

def add_regions(
    self,
    regions: List["Region"],
    prefix: Optional[str] = None,
    *,
    source: Optional[str] = None,
) -> "Page":
    """
    Add multiple regions to the page.

    Args:
        regions: List of Region objects to add
        prefix: Optional prefix for automatic naming (regions will be named prefix_1, prefix_2, etc.)
        source: Optional provenance label applied to each region.

    Returns:
        Self for method chaining
    """
    if prefix:
        # Add with automatic sequential naming
        for i, region in enumerate(regions):
            self.add_region(region, name=f"{prefix}_{i+1}", source=source)
    else:
        # Add without names
        for region in regions:
            self.add_region(region, source=source)

    return self

natural_pdf.Page.analyze_layout(engine=None, options=None, confidence=None, classes=None, exclude_classes=None, device=None, existing='replace', model_name=None, client=None)

Analyze the page layout using the configured layout engine. Adds detected Region objects to the page's element manager.

Returns:

Type	Description
ElementCollection[Region]	ElementCollection containing the detected Region objects.

Source code in natural_pdf/core/page.py

def analyze_layout(
    self,
    engine: Optional[str] = None,
    options: Optional["LayoutOptions"] = None,
    confidence: Optional[float] = None,
    classes: Optional[List[str]] = None,
    exclude_classes: Optional[List[str]] = None,
    device: Optional[str] = None,
    existing: str = "replace",
    model_name: Optional[str] = None,
    client: Optional[Any] = None,  # Add client parameter
) -> "ElementCollection[Region]":
    """
    Analyze the page layout using the configured layout engine.
    Adds detected Region objects to the page's element manager.

    Returns:
        ElementCollection containing the detected Region objects.
    """
    analyzer = self.layout_analyzer

    # Clear existing detected regions if 'replace' is specified
    if existing == "replace":
        self.clear_detected_layout_regions()

    # The analyzer's analyze_layout method already adds regions to the page
    # and its element manager. We just need to retrieve them.
    analyzer.analyze_layout(
        engine=engine,
        options=options,
        confidence=confidence,
        classes=classes,
        exclude_classes=exclude_classes,
        device=device,
        existing=existing,
        model_name=model_name,
        client=client,  # Pass client down
    )

    # Retrieve the detected regions from the element manager
    # Filter regions based on source='detected' and potentially the model used if available
    detected_regions = [
        r
        for r in self._element_mgr.regions
        if r.source == "detected" and (not engine or getattr(r, "model", None) == engine)
    ]

    return ElementCollection(detected_regions)

natural_pdf.Page.analyze_text_styles(options=None)

Analyze text elements by style, adding attributes directly to elements.

This method uses TextStyleAnalyzer to process text elements (typically words) on the page. It adds the following attributes to each processed element: - style_label: A descriptive or numeric label for the style group. - style_key: A hashable tuple representing the style properties used for grouping. - style_properties: A dictionary containing the extracted style properties.

Parameters:

Name	Type	Description	Default
options	Optional[TextStyleOptions]	Optional TextStyleOptions to configure the analysis. If None, the analyzer's default options are used.	None

Returns:

Type	Description
ElementCollection	ElementCollection containing all processed text elements with added style attributes.

Source code in natural_pdf/core/page.py

def analyze_text_styles(
    self, options: Optional[TextStyleOptions] = None
) -> "ElementCollection":
    """
    Analyze text elements by style, adding attributes directly to elements.

    This method uses TextStyleAnalyzer to process text elements (typically words)
    on the page. It adds the following attributes to each processed element:
    - style_label: A descriptive or numeric label for the style group.
    - style_key: A hashable tuple representing the style properties used for grouping.
    - style_properties: A dictionary containing the extracted style properties.

    Args:
        options: Optional TextStyleOptions to configure the analysis.
                    If None, the analyzer's default options are used.

    Returns:
        ElementCollection containing all processed text elements with added style attributes.
    """
    analyzer = TextStyleAnalyzer()
    processed_elements_collection = analyzer.analyze(self, options=options)

    metadata = getattr(self, "metadata", None)
    summary = None
    if isinstance(metadata, dict):
        summary = metadata.get("text_styles_summary")
    self._text_styles_summary = summary or {}

    return processed_elements_collection

natural_pdf.Page.apply_ocr(engine=None, options=None, languages=None, min_confidence=None, device=None, resolution=None, detect_only=False, apply_exclusions=True, replace=True)

Apply OCR directly to this page.

Source code in natural_pdf/core/page.py

def apply_ocr(
    self,
    engine: Optional[str] = None,
    options: Optional["OCROptions"] = None,
    languages: Optional[List[str]] = None,
    min_confidence: Optional[float] = None,
    device: Optional[str] = None,
    resolution: Optional[int] = None,
    detect_only: bool = False,
    apply_exclusions: bool = True,
    replace: bool = True,
) -> "Page":
    """Apply OCR directly to this page."""
    normalized_options = normalize_ocr_options(options)
    engine_name = resolve_ocr_engine_name(
        context=self,
        requested=engine,
        options=normalized_options,
        scope="page",
    )
    resolved_languages = resolve_ocr_languages(self, languages, scope="page")
    resolved_min_conf = resolve_ocr_min_confidence(self, min_confidence, scope="page")
    resolved_device = resolve_ocr_device(self, device, scope="page")

    if replace:
        removed = self.remove_ocr_elements()
        if removed:
            logger.info(
                f"Page {self.number}: Removed {removed} OCR elements before new OCR run."
            )

    final_resolution = (
        resolution
        if resolution is not None
        else getattr(getattr(self, "_parent", None), "_config", {}).get("resolution", 150)
    )

    try:
        ocr_payload = run_ocr_apply(
            target=self,
            context=self,
            engine_name=engine_name,
            resolution=final_resolution,
            languages=resolved_languages,
            min_confidence=resolved_min_conf,
            device=resolved_device,
            detect_only=detect_only,
            options=normalized_options,
            render_kwargs={},
        )
    except Exception as exc:  # pragma: no cover - defensive
        logger.error(f"Page {self.number}: OCR failed: {exc}", exc_info=True)
        raise

    image_width, image_height = ocr_payload.image_size
    if not image_width or not image_height:
        logger.error(f"Page {self.number}: OCR payload missing image dimensions.")
        return self

    scale_x = self.width / image_width if image_width else 1.0
    scale_y = self.height / image_height if image_height else 1.0
    created_elements = self.create_text_elements_from_ocr(
        ocr_payload.results, scale_x=scale_x, scale_y=scale_y
    )

    logger.info(
        f"Page {self.number}: Added {len(created_elements)} OCR elements using '{engine_name}'."
    )
    return self

natural_pdf.Page.clear_detected_layout_regions()

Removes all regions from this page that were added by layout analysis (i.e., regions where source attribute is 'detected').

This clears the regions both from the page's internal _regions['detected'] list and from the ElementManager's internal list of regions.

Returns:

Type	Description
Page	Self for method chaining.

Source code in natural_pdf/core/page.py

def clear_detected_layout_regions(self) -> "Page":
    """
    Removes all regions from this page that were added by layout analysis
    (i.e., regions where `source` attribute is 'detected').

    This clears the regions both from the page's internal `_regions['detected']` list
    and from the ElementManager's internal list of regions.

    Returns:
        Self for method chaining.
    """
    removed_count = self.remove_regions(source="detected")
    if removed_count:
        logger.info(f"Page {self.index}: Cleared {removed_count} detected layout regions.")
    else:
        logger.debug(f"Page {self.index}: No detected layout regions to clear.")
    return self

natural_pdf.Page.clear_exclusions()

Clear all exclusions from the page.

Source code in natural_pdf/core/page.py

def clear_exclusions(self) -> "Page":
    """
    Clear all exclusions from the page.
    """
    self._exclusions = []
    return self

natural_pdf.Page.clear_highlights()

Clear all highlights from this specific page via HighlightingService.

Returns:

Type	Description
Page	Self for method chaining

Source code in natural_pdf/core/page.py

def clear_highlights(self) -> "Page":
    """
    Clear all highlights *from this specific page* via HighlightingService.

    Returns:
        Self for method chaining
    """
    self._highlighter.clear_page(self.index)
    return self

natural_pdf.Page.create_region(x0, top, x1, bottom)

Create a region on this page with the specified coordinates.

Parameters:

Name	Type	Description	Default
x0	float	Left x-coordinate	required
top	float	Top y-coordinate	required
x1	float	Right x-coordinate	required
bottom	float	Bottom y-coordinate	required

Returns:

Type	Description
Any	Region object for the specified coordinates

Source code in natural_pdf/core/page.py

def create_region(self, x0: float, top: float, x1: float, bottom: float) -> Any:
    """
    Create a region on this page with the specified coordinates.

    Args:
        x0: Left x-coordinate
        top: Top y-coordinate
        x1: Right x-coordinate
        bottom: Bottom y-coordinate

    Returns:
        Region object for the specified coordinates
    """
    from natural_pdf.elements.region import Region

    return Region(self, (x0, top, x1, bottom))

natural_pdf.Page.crop(bbox=None, **kwargs)

Crop the page to the specified bounding box.

This is a direct wrapper around pdfplumber's crop method.

Parameters:

Name	Type	Description	Default
bbox	Optional[Bounds]	Bounding box (x0, top, x1, bottom) or None	None
**kwargs	Any	Additional parameters (top, bottom, left, right)	{}

Returns:

Type	Description
Any	Cropped page object (pdfplumber.Page)

Source code in natural_pdf/core/page.py

def crop(self, bbox: Optional[Bounds] = None, **kwargs: Any) -> Any:
    """
    Crop the page to the specified bounding box.

    This is a direct wrapper around pdfplumber's crop method.

    Args:
        bbox: Bounding box (x0, top, x1, bottom) or None
        **kwargs: Additional parameters (top, bottom, left, right)

    Returns:
        Cropped page object (pdfplumber.Page)
    """
    # Returns the pdfplumber page object, not a natural-pdf Page
    resolved_bbox: Optional[Tuple[float, float, float, float]]
    if bbox is None:
        resolved_bbox = None
    else:
        bbox_values = extract_bbox(bbox)
        if bbox_values is None:
            raise TypeError("crop expects a 4-tuple bbox or SupportsBBox-compatible object")
        resolved_bbox = bbox_values

    crop_arg: Any = resolved_bbox if resolved_bbox is not None else None
    return self._page.crop(crop_arg, **kwargs)

natural_pdf.Page.deskew(resolution=300, angle=None, detection_resolution=72, **deskew_kwargs)

Creates and returns a deskewed PIL image of the page.

If angle is not provided, it will first try to detect the skew angle using detect_skew_angle (or use the cached angle if available).

Parameters:

Name	Type	Description	Default
resolution	int	DPI resolution for the output deskewed image.	300
angle	Optional[float]	The specific angle (in degrees) to rotate by. If None, detects automatically.	None
detection_resolution	int	DPI resolution used for detection if angle is None.	72
**deskew_kwargs		Additional keyword arguments passed to deskew.determine_skew if automatic detection is performed.	{}

Returns:

Type	Description
Optional[Image]	A deskewed PIL.Image.Image object, or None if rendering/rotation fails.

Raises:

Type	Description
ImportError	If the 'deskew' library is not installed.

Source code in natural_pdf/core/page.py

def deskew(
    self,
    resolution: int = 300,
    angle: Optional[float] = None,
    detection_resolution: int = 72,
    **deskew_kwargs,
) -> Optional[Image.Image]:
    """
    Creates and returns a deskewed PIL image of the page.

    If `angle` is not provided, it will first try to detect the skew angle
    using `detect_skew_angle` (or use the cached angle if available).

    Args:
        resolution: DPI resolution for the output deskewed image.
        angle: The specific angle (in degrees) to rotate by. If None, detects automatically.
        detection_resolution: DPI resolution used for detection if `angle` is None.
        **deskew_kwargs: Additional keyword arguments passed to `deskew.determine_skew`
                            if automatic detection is performed.

    Returns:
        A deskewed PIL.Image.Image object, or None if rendering/rotation fails.

    Raises:
        ImportError: If the 'deskew' library is not installed.
    """
    try:
        result = run_deskew_apply(
            target=self,
            context=self,
            resolution=resolution,
            angle=angle,
            detection_resolution=detection_resolution,
            grayscale=True,
            deskew_kwargs=deskew_kwargs,
        )
    except ImportError:
        raise
    except Exception as exc:
        logger.error(
            f"Page {self.number}: Error during deskewing image generation: {exc}",
            exc_info=True,
        )
        return None

    return result.image

natural_pdf.Page.detect_skew_angle(resolution=72, grayscale=True, force_recalculate=False, **deskew_kwargs)

Detect the skew angle of this page using the deskew provider.

Source code in natural_pdf/core/page.py

def detect_skew_angle(
    self,
    resolution: int = 72,
    grayscale: bool = True,
    force_recalculate: bool = False,
    **deskew_kwargs,
) -> Optional[float]:
    """Detect the skew angle of this page using the deskew provider."""
    if self._skew_angle is not None and not force_recalculate:
        logger.debug(f"Page {self.number}: Returning cached skew angle: {self._skew_angle:.2f}")
        return self._skew_angle

    try:
        angle = run_deskew_detect(
            target=self,
            context=self,
            resolution=resolution,
            grayscale=grayscale,
            deskew_kwargs=deskew_kwargs,
        )
    except ImportError:
        raise
    except Exception as exc:
        logger.warning(f"Page {self.number}: Skew detection failed: {exc}", exc_info=True)
        self._skew_angle = None
        return None

    self._skew_angle = angle
    return angle

natural_pdf.Page.ensure_elements_loaded()

Force the underlying element manager to load elements.

Source code in natural_pdf/core/page.py

def ensure_elements_loaded(self) -> None:
    """Force the underlying element manager to load elements."""
    self._element_mgr.load_elements()

natural_pdf.Page.extract_ocr_elements(engine=None, options=None, languages=None, min_confidence=None, device=None, resolution=None)

Extract text elements using OCR without mutating this page's element store.

Parameters:

Name	Type	Description	Default
engine	Optional[str]	Name of the OCR engine.	None
options	Optional[OCROptions]	Engine-specific options object or dict.	None
languages	Optional[List[str]]	List of engine-specific language codes.	None
min_confidence	Optional[float]	Minimum confidence threshold.	None
device	Optional[str]	Device to run OCR on.	None
resolution	Optional[int]	DPI resolution for rendering page image before OCR.	None

Returns:

Type	Description
List[TextElement]	List of created TextElement objects derived from OCR results for this page.

Source code in natural_pdf/core/page.py

def extract_ocr_elements(
    self,
    engine: Optional[str] = None,
    options: Optional["OCROptions"] = None,
    languages: Optional[List[str]] = None,
    min_confidence: Optional[float] = None,
    device: Optional[str] = None,
    resolution: Optional[int] = None,
) -> List["TextElement"]:
    """
    Extract text elements using OCR *without* mutating this page's element store.

    Args:
        engine: Name of the OCR engine.
        options: Engine-specific options object or dict.
        languages: List of engine-specific language codes.
        min_confidence: Minimum confidence threshold.
        device: Device to run OCR on.
        resolution: DPI resolution for rendering page image before OCR.

    Returns:
        List of created TextElement objects derived from OCR results for this page.
    """

    logger.info(f"Page {self.number}: Extracting OCR elements (extract only)...")

    normalized_options = normalize_ocr_options(options)
    engine_name = resolve_ocr_engine_name(
        context=self,
        requested=engine,
        options=normalized_options,
        scope="page",
    )
    resolved_languages = resolve_ocr_languages(self, languages, scope="page")
    resolved_min_conf = resolve_ocr_min_confidence(self, min_confidence, scope="page")
    resolved_device = resolve_ocr_device(self, device, scope="page")

    final_resolution = resolution if resolution is not None else 150
    logger.debug(f"  Using rendering resolution: {final_resolution} DPI")

    ocr_payload = run_ocr_extract(
        target=self,
        context=self,
        engine_name=engine_name,
        resolution=final_resolution,
        languages=resolved_languages,
        min_confidence=resolved_min_conf,
        device=resolved_device,
        detect_only=False,
        options=normalized_options,
        render_kwargs={},
    )

    results = ocr_payload.results
    image_width, image_height = ocr_payload.image_size
    if not image_width or not image_height:
        logger.error("  OCR payload missing image dimensions; aborting extraction.")
        return []

    # Convert results but DO NOT add to ElementManager
    logger.debug("  Converting OCR results to TextElements (extract only)...")
    temp_elements = []
    scale_x = self.width / image_width if image_width else 1
    scale_y = self.height / image_height if image_height else 1
    for result in results:
        if not isinstance(result, Mapping):
            raise TypeError(f"OCR result has unexpected type: {type(result).__name__}")

        bbox_raw = result.get("bbox")
        text_value = result.get("text", "")
        confidence = result.get("confidence", 0.0)
        if not isinstance(bbox_raw, (list, tuple)) or len(bbox_raw) != 4:
            raise ValueError("OCR result missing bbox")
        x0, top, x1, bottom = [float(cast(Union[int, float, str], c)) for c in bbox_raw]
        elem_data = {
            "text": str(text_value),
            "confidence": float(confidence) if confidence is not None else 0.0,
            "x0": x0 * scale_x,
            "top": top * scale_y,
            "x1": x1 * scale_x,
            "bottom": bottom * scale_y,
            "width": (x1 - x0) * scale_x,
            "height": (bottom - top) * scale_y,
            "object_type": "text",  # Using text for temporary elements
            "source": "ocr",
            "fontname": "OCR-extract",  # Different name for clarity
            "size": 10.0,
            "page_number": self.number,
        }
        temp_elements.append(TextElement(elem_data, self))

    logger.info(f"  Created {len(temp_elements)} TextElements from OCR (extract only).")
    return temp_elements

natural_pdf.Page.extract_table(method=None, table_settings=None, use_ocr=False, ocr_config=None, text_options=None, cell_extraction_func=None, show_progress=False, content_filter=None, verticals=None, horizontals=None, structure_engine=None)

Extract the largest table from this page using enhanced region-based extraction.

Parameters:

Name	Type	Description	Default
method	Optional[str]	Method to use: 'tatr', 'pdfplumber', 'text', 'stream', 'lattice', or None (auto-detect).	None
table_settings	Optional[dict]	Settings for pdfplumber table extraction.	None
use_ocr	bool	Whether to use OCR for text extraction (currently only applicable with 'tatr' method).	False
ocr_config	Optional[dict]	OCR configuration parameters.	None
text_options	Optional[Dict]	Dictionary of options for the 'text' method.	None
cell_extraction_func	Optional[Callable[[Region], Optional[str]]]	Optional callable function that takes a cell Region object and returns its string content. For 'text' method only.	None
show_progress	bool	If True, display a progress bar during cell text extraction for the 'text' method.	False
content_filter		Optional content filter to apply during cell text extraction. Can be: - A regex pattern string (characters matching the pattern are EXCLUDED) - A callable that takes text and returns True to KEEP the character - A list of regex patterns (characters matching ANY pattern are EXCLUDED)	None
verticals	Optional[List[float]]	Optional list of x-coordinates for explicit vertical table lines.	None
horizontals	Optional[List[float]]	Optional list of y-coordinates for explicit horizontal table lines.	None
structure_engine	Optional[str]	Optional structure detection engine forwarded to the underlying region so provider-backed engines (e.g., TATR structure consumers) can be leveraged before falling back to standard extraction methods.	None

Returns:

Name	Type	Description
TableResult	TableResult	A sequence-like object containing table rows that also provides .to_df() for pandas conversion.

Source code in natural_pdf/core/page.py

def extract_table(
    self,
    method: Optional[str] = None,
    table_settings: Optional[dict] = None,
    use_ocr: bool = False,
    ocr_config: Optional[dict] = None,
    text_options: Optional[Dict] = None,
    cell_extraction_func: Optional[Callable[["Region"], Optional[str]]] = None,
    show_progress: bool = False,
    content_filter=None,
    verticals: Optional[List[float]] = None,
    horizontals: Optional[List[float]] = None,
    structure_engine: Optional[str] = None,
) -> TableResult:
    """
    Extract the largest table from this page using enhanced region-based extraction.

    Args:
        method: Method to use: 'tatr', 'pdfplumber', 'text', 'stream', 'lattice', or None (auto-detect).
        table_settings: Settings for pdfplumber table extraction.
        use_ocr: Whether to use OCR for text extraction (currently only applicable with 'tatr' method).
        ocr_config: OCR configuration parameters.
        text_options: Dictionary of options for the 'text' method.
        cell_extraction_func: Optional callable function that takes a cell Region object
                                and returns its string content. For 'text' method only.
        show_progress: If True, display a progress bar during cell text extraction for the 'text' method.
        content_filter: Optional content filter to apply during cell text extraction. Can be:
            - A regex pattern string (characters matching the pattern are EXCLUDED)
            - A callable that takes text and returns True to KEEP the character
            - A list of regex patterns (characters matching ANY pattern are EXCLUDED)
        verticals: Optional list of x-coordinates for explicit vertical table lines.
        horizontals: Optional list of y-coordinates for explicit horizontal table lines.
        structure_engine: Optional structure detection engine forwarded to the underlying
            region so provider-backed engines (e.g., TATR structure consumers) can be leveraged
            before falling back to standard extraction methods.

    Returns:
        TableResult: A sequence-like object containing table rows that also provides .to_df() for pandas conversion.
    """
    # Create a full-page region and delegate to its enhanced extract_table method
    page_region = self.create_region(0, 0, self.width, self.height)
    return page_region.extract_table(
        method=method,
        table_settings=table_settings,
        use_ocr=use_ocr,
        ocr_config=ocr_config,
        text_options=text_options,
        cell_extraction_func=cell_extraction_func,
        show_progress=show_progress,
        content_filter=content_filter,
        verticals=verticals,
        horizontals=horizontals,
        structure_engine=structure_engine,
    )

natural_pdf.Page.extract_tables(method=None, table_settings=None, check_tatr=True)

Extract all tables from this page with enhanced method support.

Parameters:

Name	Type	Description	Default
method	Optional[str]	Method to use: 'pdfplumber', 'stream', 'lattice', or None (auto-detect). 'stream' uses text-based strategies, 'lattice' uses line-based strategies. Note: 'tatr' and 'text' methods are not supported for extract_tables.	None
table_settings	Optional[dict]	Settings for pdfplumber table extraction.	None
check_tatr	bool	If True (default), first check for TATR-detected table regions and extract from those before falling back to pdfplumber methods.	True

Returns:

Type	Description
List[List[List[Optional[str]]]]	List of tables, where each table is a list of rows, and each row is a list of cell values.

Source code in natural_pdf/core/page.py

def extract_tables(
    self,
    method: Optional[str] = None,
    table_settings: Optional[dict] = None,
    check_tatr: bool = True,
) -> List[List[List[Optional[str]]]]:
    """
    Extract all tables from this page with enhanced method support.

    Args:
        method: Method to use: 'pdfplumber', 'stream', 'lattice', or None (auto-detect).
                'stream' uses text-based strategies, 'lattice' uses line-based strategies.
                Note: 'tatr' and 'text' methods are not supported for extract_tables.
        table_settings: Settings for pdfplumber table extraction.
        check_tatr: If True (default), first check for TATR-detected table regions
                    and extract from those before falling back to pdfplumber methods.

    Returns:
        List of tables, where each table is a list of rows, and each row is a list of cell values.
    """
    base_settings = table_settings.copy() if table_settings else {}

    def _normalise_table(table: Sequence[Sequence[Optional[str]]]) -> List[List[str]]:
        return [[cell if isinstance(cell, str) else "" for cell in row] for row in table]

    def _as_optional_rows(table: Sequence[Sequence[str]]) -> List[List[Optional[str]]]:
        return [[cell for cell in row] for row in table]

    def _process_pdfplumber_tables(
        raw_tables: Optional[Sequence[Sequence[Sequence[Any]]]],
    ) -> List[List[List[Optional[str]]]]:
        if not raw_tables:
            return []
        processed: List[List[List[Optional[str]]]] = []
        for table in raw_tables:
            normalized_rows: List[List[str]] = []
            for row in table:
                normalized_row: List[str] = []
                for cell in row:
                    text_value = "" if cell is None else str(cell)
                    if text_value:
                        text_value = self._apply_rtl_processing_to_text(text_value)
                    normalized_row.append(text_value)
                normalized_rows.append(normalized_row)
            processed.append(_as_optional_rows(normalized_rows))
        return processed

    def _apply_text_strategy_overrides(settings: Dict[str, Any]) -> None:
        if "text" not in (
            settings.get("vertical_strategy"),
            settings.get("horizontal_strategy"),
        ):
            return

        if "text_x_tolerance" not in settings and "x_tolerance" not in settings:
            x_tol = self.get_config("x_tolerance", None, scope="page")
            if x_tol is not None:
                settings.setdefault("text_x_tolerance", x_tol)
        if "text_y_tolerance" not in settings and "y_tolerance" not in settings:
            y_tol = self.get_config("y_tolerance", None, scope="page")
            if y_tol is not None:
                settings.setdefault("text_y_tolerance", y_tol)

        if "snap_tolerance" not in settings and "snap_x_tolerance" not in settings:
            y_tol = self.get_config("y_tolerance", 1, scope="page")
            snap = max(1, round(float(y_tol) * 0.9))
            settings.setdefault("snap_tolerance", snap)
        if "join_tolerance" not in settings and "join_x_tolerance" not in settings:
            join = settings.get("snap_tolerance", 1)
            settings.setdefault("join_tolerance", join)
            settings.setdefault("join_x_tolerance", join)
            settings.setdefault("join_y_tolerance", join)

    def _run_pdfplumber(
        overrides: Optional[Dict[str, Any]] = None, *, force: bool = False
    ) -> List[List[List[Optional[str]]]]:
        settings = base_settings.copy()
        if overrides:
            for key, value in overrides.items():
                if force or key not in settings:
                    settings[key] = value
        _apply_text_strategy_overrides(settings)
        raw_tables = self._page.extract_tables(settings)
        return _process_pdfplumber_tables(raw_tables)

    def _auto_detect_tables() -> List[List[List[Optional[str]]]]:
        strategies = (
            ("lattice", {"vertical_strategy": "lines", "horizontal_strategy": "lines"}),
            ("stream", {"vertical_strategy": "text", "horizontal_strategy": "text"}),
        )
        for label, overrides in strategies:
            try:
                tables = _run_pdfplumber(overrides, force=True)
                if tables:
                    logger.debug(
                        f"Page {self.number}: '{label}' strategy found {len(tables)} tables"
                    )
                    return tables
            except Exception as exc:
                logger.debug(f"Page {self.number}: '{label}' strategy failed: {exc}")
        return []

    def _extract_tables_via_tatr() -> List[List[List[Optional[str]]]]:
        regions = self.find_all("region[type=table][model=tatr]")
        if not regions:
            logger.debug(
                f"Page {self.number}: No TATR table regions found, using pdfplumber methods"
            )
            return []

        extracted: List[List[List[Optional[str]]]] = []
        for table_region in regions:
            table_data = table_region.extract_table(method="tatr")
            if table_data:
                extracted.append(_as_optional_rows(_normalise_table(table_data)))

        if extracted:
            logger.debug(
                f"Page {self.number}: Successfully extracted {len(extracted)} tables from TATR regions"
            )
        else:
            logger.debug(
                f"Page {self.number}: TATR regions found but no tables extracted, falling back to pdfplumber"
            )
        return extracted

    if check_tatr:
        try:
            tatr_tables = _extract_tables_via_tatr()
            if tatr_tables:
                return tatr_tables
        except Exception as exc:
            logger.debug(
                f"Page {self.number}: Error checking TATR regions: {exc}, falling back to pdfplumber"
            )

    if method is None:
        auto_tables = _auto_detect_tables()
        if auto_tables:
            return auto_tables
        logger.debug(
            f"Page {self.number}: Auto-detection failed to find tables. Returning empty list."
        )
        return []

    settings = table_settings.copy() if table_settings else {}

    if check_tatr:
        try:
            tatr_matches = _extract_tables_via_tatr()
            if tatr_matches:
                logger.debug(
                    f"Page {self.number}: Returning {len(tatr_matches)} tables from TATR regions."
                )
                return tatr_matches
        except Exception as exc:
            logger.debug(
                f"Page {self.number}: Error checking TATR regions: {exc}; falling back to pdfplumber."
            )

    page_region = self.to_region()
    try:
        return page_region.extract_tables(method=method, table_settings=settings)
    except Exception as exc:
        logger.debug(f"Page {self.number}: Table extraction failed: {exc}")
        return []

natural_pdf.Page.extract_text(preserve_whitespace=True, preserve_line_breaks=True, use_exclusions=True, debug_exclusions=False, content_filter=None, *, layout=True, x_density=None, y_density=None, x_tolerance=None, y_tolerance=None, line_dir=None, char_dir=None, strip_final=False, strip_empty=False, bidi=True)

Extract text from this page, respecting exclusions and using pdfplumber's layout engine (chars_to_textmap) if layout arguments are provided or default.

Parameters:

Name	Type	Description	Default
preserve_line_breaks	bool	When False, collapse newlines into spaces for a flattened string.	True
use_exclusions	bool	Whether to apply exclusion regions (default: True). Note: Filtering logic is now always applied if exclusions exist.	True
debug_exclusions	bool	Whether to output detailed exclusion debugging info (default: False).	False
content_filter		Optional content filter to exclude specific text patterns. Can be: - A regex pattern string (characters matching the pattern are EXCLUDED) - A callable that takes text and returns True to KEEP the character - A list of regex patterns (characters matching ANY pattern are EXCLUDED)	None
layout	bool	Whether to enable layout-aware spacing (default: True).	True
x_density	Optional[float]	Horizontal character density override.	None
y_density	Optional[float]	Vertical line density override.	None
x_tolerance	Optional[float]	Horizontal clustering tolerance.	None
y_tolerance	Optional[float]	Vertical clustering tolerance.	None
line_dir	Optional[str]	Line reading direction override.	None
char_dir	Optional[str]	Character reading direction override.	None
strip_final	bool	When True, strip trailing whitespace from the combined text.	False
strip_empty	bool	When True, drop entirely blank lines from the output.	False
bidi	bool	Whether to apply bidi reordering when RTL text is detected (default: True).	True

Returns:

Type	Description
str	Extracted text as string, potentially with layout-based spacing.

Source code in natural_pdf/core/page.py

def extract_text(
    self,
    preserve_whitespace: bool = True,
    preserve_line_breaks: bool = True,
    use_exclusions: bool = True,
    debug_exclusions: bool = False,
    content_filter=None,
    *,
    layout: bool = True,
    x_density: Optional[float] = None,
    y_density: Optional[float] = None,
    x_tolerance: Optional[float] = None,
    y_tolerance: Optional[float] = None,
    line_dir: Optional[str] = None,
    char_dir: Optional[str] = None,
    strip_final: bool = False,
    strip_empty: bool = False,
    bidi: bool = True,
) -> str:
    """
    Extract text from this page, respecting exclusions and using pdfplumber's
    layout engine (chars_to_textmap) if layout arguments are provided or default.

    Args:
        preserve_line_breaks: When False, collapse newlines into spaces for a flattened string.
        use_exclusions: Whether to apply exclusion regions (default: True).
                        Note: Filtering logic is now always applied if exclusions exist.
        debug_exclusions: Whether to output detailed exclusion debugging info (default: False).
        content_filter: Optional content filter to exclude specific text patterns. Can be:
            - A regex pattern string (characters matching the pattern are EXCLUDED)
            - A callable that takes text and returns True to KEEP the character
            - A list of regex patterns (characters matching ANY pattern are EXCLUDED)
        layout: Whether to enable layout-aware spacing (default: True).
        x_density: Horizontal character density override.
        y_density: Vertical line density override.
        x_tolerance: Horizontal clustering tolerance.
        y_tolerance: Vertical clustering tolerance.
        line_dir: Line reading direction override.
        char_dir: Character reading direction override.
        strip_final: When True, strip trailing whitespace from the combined text.
        strip_empty: When True, drop entirely blank lines from the output.
        bidi: Whether to apply bidi reordering when RTL text is detected (default: True).

    Returns:
        Extracted text as string, potentially with layout-based spacing.
    """
    logger.debug(
        "Page %s: extract_text called with layout=%s, x_density=%s, y_density=%s",
        self.number,
        layout,
        x_density,
        y_density,
    )
    debug = debug_exclusions

    # 1. Get Word Elements (triggers load_elements if needed)
    word_elements = self.words
    if not word_elements:
        logger.debug(f"Page {self.number}: No word elements found.")
        return ""

    # 2. Apply element-based exclusions if enabled
    # Check both page-level and PDF-level exclusions
    has_exclusions = bool(self._exclusions) or (
        hasattr(self, "_parent")
        and self._parent
        and hasattr(self._parent, "_exclusions")
        and self._parent._exclusions
    )
    if use_exclusions and has_exclusions:
        # Filter word elements through _filter_elements_by_exclusions
        # This handles both element-based and region-based exclusions
        word_elements = self._filter_elements_by_exclusions(
            word_elements, debug_exclusions=debug
        )
        if debug:
            logger.debug(
                f"Page {self.number}: {len(word_elements)} words remaining after exclusion filtering."
            )

    # 3. Get region-based exclusions for spatial filtering
    exclusion_regions = []
    apply_exclusions_flag = use_exclusions
    if apply_exclusions_flag and has_exclusions:
        exclusion_regions = self._get_exclusion_regions(include_callable=True, debug=debug)
        if debug:
            logger.debug(
                f"Page {self.number}: Found {len(exclusion_regions)} region exclusions for spatial filtering."
            )
    elif debug:
        logger.debug(f"Page {self.number}: Not applying exclusions.")

    # 4. Collect All Character Dictionaries from remaining Word Elements
    all_char_dicts = []
    for word in word_elements:
        all_char_dicts.extend(getattr(word, "_char_dicts", []))

    # 5. Spatially Filter Characters (only by regions, elements already filtered above)
    filtered_chars = filter_chars_spatially(
        char_dicts=all_char_dicts,
        exclusion_regions=exclusion_regions,
        target_region=None,  # No target region for full page extraction
        debug=debug,
    )

    # 5. Generate Text Layout using Utility
    # Pass page bbox as layout context
    page_bbox = (0, 0, self.width, self.height)
    # Merge PDF-level default tolerances if caller did not override
    merged_kwargs = {
        "layout": layout,
        "x_density": x_density,
        "y_density": y_density,
        "x_tolerance": x_tolerance,
        "y_tolerance": y_tolerance,
        "line_dir": line_dir,
        "char_dir": char_dir,
    }
    merged_kwargs = {
        key: value
        for key, value in merged_kwargs.items()
        if value is not None or key == "layout"
    }
    tol_keys = ["x_tolerance", "x_tolerance_ratio", "y_tolerance"]
    for k in tol_keys:
        if k not in merged_kwargs:
            if k in self._config:
                merged_kwargs[k] = self._config[k]
            elif hasattr(self._parent, "_config") and k in self._parent._config:
                merged_kwargs[k] = self._parent._config[k]

    # Add content_filter to kwargs if provided
    if content_filter is not None:
        merged_kwargs["content_filter"] = content_filter

    result = generate_text_layout(
        char_dicts=filtered_chars,
        layout_context_bbox=page_bbox,
        user_kwargs=merged_kwargs,
    )

    if bidi and result:
        # Quick check for any RTL character
        import unicodedata

        def _contains_rtl(s):
            return any(unicodedata.bidirectional(ch) in ("R", "AL", "AN") for ch in s)

        if _contains_rtl(result):
            try:
                from bidi.algorithm import get_display  # type: ignore

                from natural_pdf.utils.bidi_mirror import mirror_brackets

                normalized_lines = []
                for line in result.split("\n"):
                    base_dir = (
                        "R"
                        if any(
                            unicodedata.bidirectional(ch) in ("R", "AL", "AN") for ch in line
                        )
                        else "L"
                    )
                    raw_display = get_display(line, base_dir=base_dir)
                    display_line = (
                        raw_display.decode("utf-8", errors="ignore")
                        if isinstance(raw_display, bytes)
                        else str(raw_display)
                    )
                    normalized_lines.append(mirror_brackets(display_line))
                result = "\n".join(normalized_lines)
            except ModuleNotFoundError:
                pass  # silently skip if python-bidi not available

    if strip_empty and result:
        result = "\n".join(line for line in result.splitlines() if line.strip())

    if strip_final and result:
        result = "\n".join(line.rstrip() for line in result.splitlines()).strip()

    if result and not preserve_line_breaks:
        normalized = result.replace("\r\n", "\n").replace("\r", "\n")
        if preserve_whitespace:
            result = re.sub(r"\s*\n\s*", " ", normalized)
        else:
            result = " ".join(normalized.split())

    logger.debug(f"Page {self.number}: extract_text finished, result length: {len(result)}.")
    return result

natural_pdf.Page.filter_elements(elements, selector, **kwargs)

Filter a list of elements based on a selector.

Parameters:

Name	Type	Description	Default
elements	List[Element]	List of elements to filter	required
selector	str	CSS-like selector string	required
**kwargs		Additional filter parameters	{}

Returns:

Type	Description
List[Element]	List of elements that match the selector

Source code in natural_pdf/core/page.py

def filter_elements(
    self, elements: List["Element"], selector: str, **kwargs
) -> List["Element"]:
    """
    Filter a list of elements based on a selector.

    Args:
        elements: List of elements to filter
        selector: CSS-like selector string
        **kwargs: Additional filter parameters

    Returns:
        List of elements that match the selector
    """
    from natural_pdf.selectors.parser import parse_selector, selector_to_filter_func

    # Parse the selector
    selector_obj = parse_selector(selector)

    # Create filter function from selector
    filter_func = selector_to_filter_func(selector_obj, **kwargs)

    # Apply the filter to the elements
    matching_elements = [element for element in elements if filter_func(element)]

    # Sort elements in reading order if requested
    if kwargs.get("reading_order", True):
        if all(hasattr(el, "top") and hasattr(el, "x0") for el in matching_elements):
            matching_elements.sort(key=lambda el: (el.top, el.x0))
        else:
            logger.warning(
                "Cannot sort elements in reading order: Missing required attributes (top, x0)."
            )

    return matching_elements

natural_pdf.Page.find(selector=None, *, text=None, overlap=None, apply_exclusions=True, regex=False, case=True, text_tolerance=None, auto_text_tolerance=None, reading_order=True, near_threshold=None, engine=None)

Find first element on this page matching selector OR text content.

Provide EITHER selector OR text, but not both.

Parameters:

Name	Type	Description	Default
selector	Optional[str]	CSS-like selector string.	None
text	Optional[Union[str, Sequence[str]]]	Optional text shortcut (equivalent to text:contains(...)).	None
overlap	Optional[str]	Reserved for compatibility with region APIs; ignored for pages.	None
apply_exclusions	bool	Whether to exclude elements in exclusion regions (default: True).	True
regex	bool	Whether to use regex for text search (selector or text) (default: False).	False
case	bool	Whether to do case-sensitive text search (selector or text) (default: True).	True
text_tolerance	Optional[Dict[str, Any]]	Optional dict of tolerance overrides applied temporarily.	None
auto_text_tolerance	Optional[Union[bool, Dict[str, Any]]]	Optional overrides controlling automatic tolerance logic.	None
reading_order	bool	Whether to sort matches in reading order when applicable (default: True).	True
near_threshold	Optional[float]	Maximum distance (in points) used by the :near pseudo-class.	None
engine	Optional[str]	Optional selector engine name registered via :mod:natural_pdf.engine_provider.	None

Returns:

Type	Description
Optional[Element]	Element object or None if not found.

Source code in natural_pdf/core/page.py

def find(
    self,
    selector: Optional[str] = None,
    *,
    text: Optional[Union[str, Sequence[str]]] = None,
    overlap: Optional[str] = None,
    apply_exclusions: bool = True,
    regex: bool = False,
    case: bool = True,
    text_tolerance: Optional[Dict[str, Any]] = None,
    auto_text_tolerance: Optional[Union[bool, Dict[str, Any]]] = None,
    reading_order: bool = True,
    near_threshold: Optional[float] = None,
    engine: Optional[str] = None,
) -> Optional[Element]:
    """
    Find first element on this page matching selector OR text content.

    Provide EITHER `selector` OR `text`, but not both.

    Args:
        selector: CSS-like selector string.
        text: Optional text shortcut (equivalent to ``text:contains(...)``).
        overlap: Reserved for compatibility with region APIs; ignored for pages.
        apply_exclusions: Whether to exclude elements in exclusion regions (default: True).
        regex: Whether to use regex for text search (`selector` or `text`) (default: False).
        case: Whether to do case-sensitive text search (`selector` or `text`) (default: True).
        text_tolerance: Optional dict of tolerance overrides applied temporarily.
        auto_text_tolerance: Optional overrides controlling automatic tolerance logic.
        reading_order: Whether to sort matches in reading order when applicable (default: True).
        near_threshold: Maximum distance (in points) used by the ``:near`` pseudo-class.
        engine: Optional selector engine name registered via :mod:`natural_pdf.engine_provider`.

    Returns:
        Element object or None if not found.
    """
    effective_selector = normalize_selector_input(
        selector,
        text,
        logger=logger,
        context="Page.find",
    )

    results_collection = execute_selector_query(
        self,
        effective_selector,
        text_tolerance=text_tolerance,
        auto_text_tolerance=auto_text_tolerance,
        regex=regex,
        case=case,
        reading_order=reading_order,
        near_threshold=near_threshold,
        engine=engine,
    )

    elements: List[Element] = list(results_collection.elements) if results_collection else []
    if apply_exclusions and elements:
        elements = self._filter_elements_by_exclusions(elements)

    return cast(Optional[Element], elements[0] if elements else None)

natural_pdf.Page.find_all(selector=None, *, text=None, overlap=None, apply_exclusions=True, regex=False, case=True, text_tolerance=None, auto_text_tolerance=None, reading_order=True, near_threshold=None, engine=None)

Find all elements on this page matching selector OR text content.

Provide EITHER selector OR text, but not both.

Parameters:

Name	Type	Description	Default
selector	Optional[str]	CSS-like selector string.	None
text	Optional[Union[str, Sequence[str]]]	Text content to search for (equivalent to 'text:contains(...)'). Accepts a single string or an iterable of strings (matches any value).	None
overlap	Optional[str]	Reserved for compatibility with region APIs; ignored for pages.	None
apply_exclusions	bool	Whether to exclude elements in exclusion regions (default: True).	True
regex	bool	Whether to use regex for text search (selector or text) (default: False).	False
case	bool	Whether to do case-sensitive text search (selector or text) (default: True).	True
text_tolerance	Optional[Dict[str, Any]]	Optional dict of tolerance overrides applied temporarily.	None
auto_text_tolerance	Optional[Union[bool, Dict[str, Any]]]	Optional overrides controlling automatic tolerance calculation.	None
reading_order	bool	Whether to sort matches in reading order (default: True).	True
near_threshold	Optional[float]	Maximum distance (in points) used by the :near pseudo-class.	None
engine	Optional[str]	Optional selector engine name registered with the selector provider.	None

Returns:

Type	Description
ElementCollection	ElementCollection with matching elements.

Source code in natural_pdf/core/page.py

def find_all(
    self,
    selector: Optional[str] = None,
    *,
    text: Optional[Union[str, Sequence[str]]] = None,
    overlap: Optional[str] = None,
    apply_exclusions: bool = True,
    regex: bool = False,
    case: bool = True,
    text_tolerance: Optional[Dict[str, Any]] = None,
    auto_text_tolerance: Optional[Union[bool, Dict[str, Any]]] = None,
    reading_order: bool = True,
    near_threshold: Optional[float] = None,
    engine: Optional[str] = None,
) -> "ElementCollection":
    """
    Find all elements on this page matching selector OR text content.

    Provide EITHER `selector` OR `text`, but not both.

    Args:
        selector: CSS-like selector string.
        text: Text content to search for (equivalent to 'text:contains(...)'). Accepts a
              single string or an iterable of strings (matches any value).
        overlap: Reserved for compatibility with region APIs; ignored for pages.
        apply_exclusions: Whether to exclude elements in exclusion regions (default: True).
        regex: Whether to use regex for text search (`selector` or `text`) (default: False).
        case: Whether to do case-sensitive text search (`selector` or `text`) (default: True).
        text_tolerance: Optional dict of tolerance overrides applied temporarily.
        auto_text_tolerance: Optional overrides controlling automatic tolerance calculation.
        reading_order: Whether to sort matches in reading order (default: True).
        near_threshold: Maximum distance (in points) used by the ``:near`` pseudo-class.
        engine: Optional selector engine name registered with the selector provider.

    Returns:
        ElementCollection with matching elements.
    """
    effective_selector = normalize_selector_input(
        selector,
        text,
        logger=logger,
        context="Page.find_all",
    )

    results_collection = execute_selector_query(
        self,
        effective_selector,
        text_tolerance=text_tolerance,
        auto_text_tolerance=auto_text_tolerance,
        regex=regex,
        case=case,
        reading_order=reading_order,
        near_threshold=near_threshold,
        engine=engine,
    )

    if apply_exclusions and results_collection:
        filtered_elements = self._filter_elements_by_exclusions(results_collection.elements)
        return ElementCollection(filtered_elements)
    return results_collection

natural_pdf.Page.get_all_elements_raw()

Return all elements without applying exclusions.

Source code in natural_pdf/core/page.py

def get_all_elements_raw(self) -> List["Element"]:
    """Return all elements without applying exclusions."""
    return list(self._element_mgr.get_all_elements())

natural_pdf.Page.get_content()

Returns the primary content object (self) for indexing (required by Indexable protocol). SearchService implementations decide how to process this (e.g., call extract_text).

Source code in natural_pdf/core/page.py

def get_content(self) -> "Page":
    """
    Returns the primary content object (self) for indexing (required by Indexable protocol).
    SearchService implementations decide how to process this (e.g., call extract_text).
    """
    return self  # Return the Page object itself

natural_pdf.Page.get_content_hash()

Returns a SHA256 hash of the extracted text content (required by Indexable for sync).

Source code in natural_pdf/core/page.py

def get_content_hash(self) -> str:
    """Returns a SHA256 hash of the extracted text content (required by Indexable for sync)."""
    # Hash the extracted text (without exclusions for consistency)
    # Consider if exclusions should be part of the hash? For now, hash raw text.
    # Using extract_text directly might be slow if called repeatedly. Cache? TODO: Optimization
    text_content = self.extract_text(
        use_exclusions=False, preserve_whitespace=False
    )  # Normalize whitespace?
    return hashlib.sha256(text_content.encode("utf-8")).hexdigest()

natural_pdf.Page.get_elements(apply_exclusions=True, debug_exclusions=False)

Get all elements on this page.

Parameters:

Name	Type	Description	Default
apply_exclusions		Whether to apply exclusion regions (default: True).	True
debug_exclusions	bool	Whether to output detailed exclusion debugging info (default: False).	False

Returns:

Type	Description
List[Element]	List of all elements on the page, potentially filtered by exclusions.

Source code in natural_pdf/core/page.py

def get_elements(
    self, apply_exclusions=True, debug_exclusions: bool = False
) -> List["Element"]:
    """
    Get all elements on this page.

    Args:
        apply_exclusions: Whether to apply exclusion regions (default: True).
        debug_exclusions: Whether to output detailed exclusion debugging info (default: False).

    Returns:
        List of all elements on the page, potentially filtered by exclusions.
    """
    # Get all elements from the element manager
    all_elements = self.get_all_elements_raw()

    # Apply exclusions if requested
    if apply_exclusions:
        return self._filter_elements_by_exclusions(
            all_elements, debug_exclusions=debug_exclusions
        )
    else:
        if debug_exclusions:
            print(
                f"Page {self.index}: get_elements returning all {len(all_elements)} elements (exclusions not applied)."
            )
        return all_elements

natural_pdf.Page.get_elements_by_type(element_type)

Return the elements for a specific backing collection (e.g. 'words').

Source code in natural_pdf/core/page.py

def get_elements_by_type(self, element_type: str) -> List[Any]:
    """Return the elements for a specific backing collection (e.g. 'words')."""
    return list(self._element_mgr.get_elements(element_type))

natural_pdf.Page.get_highlighter()

Expose the page-level HighlightingService for Visualizable consumers.

Source code in natural_pdf/core/page.py

def get_highlighter(self) -> "HighlightingService":
    """Expose the page-level HighlightingService for Visualizable consumers."""
    return self._highlighter

natural_pdf.Page.get_id()

Returns a unique identifier for the page (required by Indexable protocol).

Source code in natural_pdf/core/page.py

def get_id(self) -> str:
    """Returns a unique identifier for the page (required by Indexable protocol)."""
    # Ensure path is safe for use in IDs (replace problematic chars)
    safe_path = re.sub(r"[^a-zA-Z0-9_-]", "_", str(self.pdf.path))
    return f"pdf_{safe_path}_page_{self.page_number}"

natural_pdf.Page.get_metadata()

Returns metadata associated with the page (required by Indexable protocol).

Source code in natural_pdf/core/page.py

def get_metadata(self) -> Dict[str, Any]:
    """Returns metadata associated with the page (required by Indexable protocol)."""
    # Add content hash here for sync
    metadata = {
        "pdf_path": str(self.pdf.path),
        "page_number": self.page_number,
        "width": self.width,
        "height": self.height,
        "content_hash": self.get_content_hash(),  # Include the hash
    }
    return metadata

natural_pdf.Page.get_section_between(start_element=None, end_element=None, include_boundaries='both', orientation='vertical')

Get a section between two elements on this page.

Parameters:

Name	Type	Description	Default
start_element		Element marking the start of the section	None
end_element		Element marking the end of the section	None
include_boundaries		How to include boundary elements: 'start', 'end', 'both', or 'none'	'both'
orientation		'vertical' (default) or 'horizontal' - determines section direction	'vertical'

Returns:

Type	Description
Optional[Region]	Region representing the section

Source code in natural_pdf/core/page.py

def get_section_between(
    self,
    start_element=None,
    end_element=None,
    include_boundaries="both",
    orientation="vertical",
) -> Optional["Region"]:  # Return Optional
    """
    Get a section between two elements on this page.

    Args:
        start_element: Element marking the start of the section
        end_element: Element marking the end of the section
        include_boundaries: How to include boundary elements: 'start', 'end', 'both', or 'none'
        orientation: 'vertical' (default) or 'horizontal' - determines section direction

    Returns:
        Region representing the section
    """
    # Create a full-page region to operate within
    page_region = self.create_region(0, 0, self.width, self.height)

    # Delegate to the region's method
    try:
        return page_region.get_section_between(
            start_element=start_element,
            end_element=end_element,
            include_boundaries=include_boundaries,
            orientation=orientation,
        )
    except Exception as e:
        logger.error(
            f"Error getting section between elements on page {self.index}: {e}", exc_info=True
        )
        return None

natural_pdf.Page.get_sections(start_elements=None, end_elements=None, include_boundaries='start', y_threshold=5.0, bounding_box=None, orientation='vertical', **kwargs)

Delegate section extraction to the Region implementation.

Source code in natural_pdf/core/page.py

def get_sections(
    self,
    start_elements: Union[str, Sequence[Element], ElementCollection, None] = None,
    end_elements: Union[str, Sequence[Element], ElementCollection, None] = None,
    include_boundaries: str = "start",
    y_threshold: float = 5.0,
    bounding_box: Optional[Bounds] = None,
    orientation: str = "vertical",
    **kwargs: Any,
) -> "ElementCollection[Region]":
    """Delegate section extraction to the Region implementation."""

    if bounding_box is not None:
        x0, top, x1, bottom = bounding_box
        base_region = self.create_region(x0, top, x1, bottom)
    else:
        base_region = self.create_region(0, 0, self.width, self.height)

    return base_region.get_sections(
        start_elements=start_elements,
        end_elements=end_elements,
        include_boundaries=include_boundaries,
        orientation=orientation,
        y_threshold=y_threshold,
        **kwargs,
    )

natural_pdf.Page.has_element_cache()

Return True if the element manager currently holds any elements.

Source code in natural_pdf/core/page.py

def has_element_cache(self) -> bool:
    """Return True if the element manager currently holds any elements."""
    return self._element_mgr.has_elements()

natural_pdf.Page.highlights(show=False)

Create a highlight context for accumulating highlights.

This allows for clean syntax to show multiple highlight groups:

Example

with page.highlights() as h: h.add(page.find_all('table'), label='tables', color='blue') h.add(page.find_all('text:bold'), label='bold text', color='red') h.show()

Or with automatic display

with page.highlights(show=True) as h: h.add(page.find_all('table'), label='tables') h.add(page.find_all('text:bold'), label='bold') # Automatically shows when exiting the context

Parameters:

Name	Type	Description	Default
show	bool	If True, automatically show highlights when exiting context	False

Returns:

Type	Description
HighlightContext	HighlightContext for accumulating highlights

Source code in natural_pdf/core/page.py

def highlights(self, show: bool = False) -> "HighlightContext":
    """
    Create a highlight context for accumulating highlights.

    This allows for clean syntax to show multiple highlight groups:

    Example:
        with page.highlights() as h:
            h.add(page.find_all('table'), label='tables', color='blue')
            h.add(page.find_all('text:bold'), label='bold text', color='red')
            h.show()

    Or with automatic display:
        with page.highlights(show=True) as h:
            h.add(page.find_all('table'), label='tables')
            h.add(page.find_all('text:bold'), label='bold')
            # Automatically shows when exiting the context

    Args:
        show: If True, automatically show highlights when exiting context

    Returns:
        HighlightContext for accumulating highlights
    """
    from natural_pdf.core.highlighting_service import HighlightContext

    return HighlightContext(self, show_on_exit=show)

natural_pdf.Page.inspect(limit=30)

Inspect all elements on this page with detailed tabular view. Equivalent to page.find_all('*').inspect().

Parameters:

Name	Type	Description	Default
limit	int	Maximum elements per type to show (default: 30)	30

Returns:

Type	Description
InspectionSummary	InspectionSummary with element tables showing coordinates,
InspectionSummary	properties, and other details for each element

Source code in natural_pdf/core/page.py

def inspect(self, limit: int = 30) -> "InspectionSummary":
    """
    Inspect all elements on this page with detailed tabular view.
    Equivalent to page.find_all('*').inspect().

    Args:
        limit: Maximum elements per type to show (default: 30)

    Returns:
        InspectionSummary with element tables showing coordinates,
        properties, and other details for each element
    """
    return self.find_all("*").inspect(limit=limit)

natural_pdf.Page.invalidate_element_cache()

Invalidate the cached elements so they are reloaded on next access.

Source code in natural_pdf/core/page.py

def invalidate_element_cache(self) -> None:
    """Invalidate the cached elements so they are reloaded on next access."""
    self._element_mgr.invalidate_cache()

natural_pdf.Page.iter_regions()

Return a list of regions currently registered with the page.

Source code in natural_pdf/core/page.py

def iter_regions(self) -> List["Region"]:
    """Return a list of regions currently registered with the page."""
    return list(self._element_mgr.regions)

natural_pdf.Page.region(left=None, top=None, right=None, bottom=None, width=None, height=None)

Create a region on this page with more intuitive named parameters, allowing definition by coordinates or by coordinate + dimension.

Parameters:

Name	Type	Description	Default
left	Optional[float]	Left x-coordinate (default: 0 if width not used).	None
top	Optional[float]	Top y-coordinate (default: 0 if height not used).	None
right	Optional[float]	Right x-coordinate (default: page width if width not used).	None
bottom	Optional[float]	Bottom y-coordinate (default: page height if height not used).	None
width	Union[str, float, None]	Width definition. Can be: - Numeric: The width of the region in points. Cannot be used with both left and right. - String 'full': Sets region width to full page width (overrides left/right). - String 'element' or None (default): Uses provided/calculated left/right, defaulting to page width if neither are specified.	None
height	Optional[float]	Numeric height of the region. Cannot be used with both top and bottom.	None

Returns:

Type	Description
Any	Region object for the specified coordinates

Raises:

Type	Description
ValueError	If conflicting arguments are provided (e.g., top, bottom, and height) or if width is an invalid string.

Examples:

>>> page.region(top=100, height=50)  # Region from y=100 to y=150, default width
>>> page.region(left=50, width=100)   # Region from x=50 to x=150, default height
>>> page.region(bottom=500, height=50) # Region from y=450 to y=500
>>> page.region(right=200, width=50)  # Region from x=150 to x=200
>>> page.region(top=100, bottom=200, width="full") # Explicit full width

Source code in natural_pdf/core/page.py

def region(
    self,
    left: Optional[float] = None,
    top: Optional[float] = None,
    right: Optional[float] = None,
    bottom: Optional[float] = None,
    width: Union[str, float, None] = None,
    height: Optional[float] = None,
) -> Any:
    """
    Create a region on this page with more intuitive named parameters,
    allowing definition by coordinates or by coordinate + dimension.

    Args:
        left: Left x-coordinate (default: 0 if width not used).
        top: Top y-coordinate (default: 0 if height not used).
        right: Right x-coordinate (default: page width if width not used).
        bottom: Bottom y-coordinate (default: page height if height not used).
        width: Width definition. Can be:
                - Numeric: The width of the region in points. Cannot be used with both left and right.
                - String 'full': Sets region width to full page width (overrides left/right).
                - String 'element' or None (default): Uses provided/calculated left/right,
                    defaulting to page width if neither are specified.
        height: Numeric height of the region. Cannot be used with both top and bottom.

    Returns:
        Region object for the specified coordinates

    Raises:
        ValueError: If conflicting arguments are provided (e.g., top, bottom, and height)
                    or if width is an invalid string.

    Examples:
        >>> page.region(top=100, height=50)  # Region from y=100 to y=150, default width
        >>> page.region(left=50, width=100)   # Region from x=50 to x=150, default height
        >>> page.region(bottom=500, height=50) # Region from y=450 to y=500
        >>> page.region(right=200, width=50)  # Region from x=150 to x=200
        >>> page.region(top=100, bottom=200, width="full") # Explicit full width
    """
    # Percentage support – convert strings like "30%" to absolute values
    # based on page dimensions.  X-axis params (left, right, width) use
    # page.width; Y-axis params (top, bottom, height) use page.height.

    def _pct_to_abs(val, axis: str):
        if isinstance(val, str) and val.strip().endswith("%"):
            try:
                pct = float(val.strip()[:-1]) / 100.0
            except ValueError:
                return val  # leave unchanged if not a number
            return pct * (self.width if axis == "x" else self.height)
        return val

    left = _pct_to_abs(left, "x")
    right = _pct_to_abs(right, "x")
    width = _pct_to_abs(width, "x")
    top = _pct_to_abs(top, "y")
    bottom = _pct_to_abs(bottom, "y")
    height = _pct_to_abs(height, "y")

    is_width_numeric = isinstance(width, (int, float))
    is_width_string = isinstance(width, str)
    width_mode = "element"  # Default mode

    if height is not None and top is not None and bottom is not None:
        raise ValueError("Cannot specify top, bottom, and height simultaneously.")
    if is_width_numeric and left is not None and right is not None:
        raise ValueError("Cannot specify left, right, and a numeric width simultaneously.")
    if is_width_string:
        width_lower = width.lower()
        if width_lower not in ["full", "element"]:
            raise ValueError("String width argument must be 'full' or 'element'.")
        width_mode = width_lower

    final_top = top
    final_bottom = bottom
    final_left = left
    final_right = right

    # Height calculations
    if height is not None:
        if top is not None:
            final_bottom = top + height
        elif bottom is not None:
            final_top = bottom - height
        else:  # Neither top nor bottom provided, default top to 0
            final_top = 0
            final_bottom = height

    # Width calculations (numeric only)
    if is_width_numeric:
        if left is not None:
            final_right = left + width
        elif right is not None:
            final_left = right - width
        else:  # Neither left nor right provided, default left to 0
            final_left = 0
            final_right = width

    # Only default coordinates if they weren't set by dimension calculation
    if final_top is None:
        final_top = 0.0
    if final_bottom is None:
        # Check if bottom should have been set by height calc
        if height is None or top is None:
            final_bottom = float(self.height)

    if final_left is None:
        final_left = 0.0
    if final_right is None:
        # Check if right should have been set by width calc
        if not is_width_numeric or left is None:
            final_right = float(self.width)

    if final_top is None or final_bottom is None or final_left is None or final_right is None:
        raise ValueError("Unable to resolve region coordinates.")

    final_left = float(final_left)
    final_top = float(final_top)
    final_right = float(final_right)
    final_bottom = float(final_bottom)

    if width_mode == "full":
        # Override left/right if mode is full
        final_left = 0
        final_right = self.width

    # Ensure coordinates are within page bounds (clamp)
    final_left = max(0.0, final_left)
    final_top = max(0.0, final_top)
    final_right = min(float(self.width), final_right)
    final_bottom = min(float(self.height), final_bottom)

    # Ensure valid box (x0<=x1, top<=bottom)
    if final_left > final_right:
        logger.warning(f"Calculated left ({final_left}) > right ({final_right}). Swapping.")
        final_left, final_right = final_right, final_left
    if final_top > final_bottom:
        logger.warning(f"Calculated top ({final_top}) > bottom ({final_bottom}). Swapping.")
        final_top, final_bottom = final_bottom, final_top

    from natural_pdf.elements.region import Region

    region = Region(self, (final_left, final_top, final_right, final_bottom))
    return region

natural_pdf.Page.remove_element(element, element_type=None)

Remove an element from the backing collection.

Source code in natural_pdf/core/page.py

def remove_element(self, element: Any, element_type: Optional[str] = None) -> bool:
    """Remove an element from the backing collection."""
    target_type = element_type or self._infer_element_type(element)
    return bool(self._element_mgr.remove_element(element, target_type))

natural_pdf.Page.remove_elements_by_source(element_type, source)

Remove all elements of a given type whose source matches.

Source code in natural_pdf/core/page.py

def remove_elements_by_source(self, element_type: str, source: str) -> int:
    """Remove all elements of a given type whose source matches."""
    return int(self._element_mgr.remove_elements_by_source(element_type, source))

natural_pdf.Page.remove_regions(*, source=None, region_type=None, predicate=None)

Remove regions from the page based on optional filters.

Parameters:

Name	Type	Description	Default
source	Optional[str]	Match regions whose region.source equals this string.	None
region_type	Optional[str]	Match regions whose region.region_type equals this string.	None
predicate	Optional[Callable[[Region], bool]]	Additional callable that returns True when a region should be removed.	None

Returns:

Type	Description
int	The number of regions removed.

Source code in natural_pdf/core/page.py

def remove_regions(
    self,
    *,
    source: Optional[str] = None,
    region_type: Optional[str] = None,
    predicate: Optional[Callable[["Region"], bool]] = None,
) -> int:
    """
    Remove regions from the page based on optional filters.

    Args:
        source: Match regions whose ``region.source`` equals this string.
        region_type: Match regions whose ``region.region_type`` equals this string.
        predicate: Additional callable that returns True when a region should be removed.

    Returns:
        The number of regions removed.
    """

    def _matches(region: "Region") -> bool:
        if source is not None and getattr(region, "source", None) != source:
            return False
        if region_type is not None and getattr(region, "region_type", None) != region_type:
            return False
        if predicate is not None and not predicate(region):
            return False
        return True

    removed = 0

    # Remove from element manager, if available
    if hasattr(self, "_element_mgr") and hasattr(self._element_mgr, "regions"):
        regions_list = getattr(self._element_mgr, "regions")
        if isinstance(regions_list, list):
            to_remove = [region for region in regions_list if _matches(region)]
            for region in to_remove:
                regions_list.remove(region)
            removed += len(to_remove)

    # Remove from detected collection
    detected = self._regions.get("detected", [])
    if detected:
        retained = [region for region in detected if not _matches(region)]
        removed += len(detected) - len(retained)
        self._regions["detected"] = retained

    # Remove from named collection
    named = self._regions.get("named", {})
    if named:
        keys_to_delete = [key for key, region in named.items() if _matches(region)]
        for key in keys_to_delete:
            del named[key]
            removed += 1

    return removed

natural_pdf.Page.remove_regions_by_source(source)

Remove all registered regions that match the requested source.

Source code in natural_pdf/core/page.py

def remove_regions_by_source(self, source: str) -> int:
    """Remove all registered regions that match the requested source."""
    return int(self._element_mgr.remove_elements_by_source("regions", source))

natural_pdf.Page.remove_text_layer()

Remove all text elements from this page.

This removes all text elements (words and characters) from the page, effectively clearing the text layer.

Returns:

Type	Description
Page	Self for method chaining

Source code in natural_pdf/core/page.py

def remove_text_layer(self) -> "Page":
    """
    Remove all text elements from this page.

    This removes all text elements (words and characters) from the page,
    effectively clearing the text layer.

    Returns:
        Self for method chaining
    """
    logger.info(f"Page {self.number}: Removing all text elements...")

    removed_words, removed_chars = self._element_mgr.clear_text_layer()

    logger.info(
        f"Page {self.number}: Removed {removed_words} words and {removed_chars} characters"
    )
    return self

natural_pdf.Page.save_image(filename, width=None, labels=True, legend_position='right', render_ocr=False, include_highlights=True, resolution=144, **kwargs)

Save the page image to a file, rendering highlights via HighlightingService.

Parameters:

Name	Type	Description	Default
filename	str	Path to save the image to.	required
width	Optional[int]	Optional width for the output image.	None
labels	bool	Whether to include a legend.	True
legend_position	str	Position of the legend.	'right'
render_ocr	bool	Whether to render OCR text.	False
include_highlights	bool	Whether to render highlights.	True
resolution	float	Resolution in DPI for base image rendering (default: 144 DPI, equivalent to previous scale=2.0).	144
**kwargs		Additional args for pdfplumber's internal to_image.	{}

Returns:

Type	Description
Page	Self for method chaining.

Source code in natural_pdf/core/page.py

def save_image(
    self,
    filename: str,
    width: Optional[int] = None,
    labels: bool = True,
    legend_position: str = "right",
    render_ocr: bool = False,
    include_highlights: bool = True,  # Allow saving without highlights
    resolution: float = 144,
    **kwargs,
) -> "Page":
    """
    Save the page image to a file, rendering highlights via HighlightingService.

    Args:
        filename: Path to save the image to.
        width: Optional width for the output image.
        labels: Whether to include a legend.
        legend_position: Position of the legend.
        render_ocr: Whether to render OCR text.
        include_highlights: Whether to render highlights.
        resolution: Resolution in DPI for base image rendering (default: 144 DPI, equivalent to previous scale=2.0).
        **kwargs: Additional args for pdfplumber's internal to_image.

    Returns:
        Self for method chaining.
    """
    # Use export() to save the image
    if include_highlights:
        self.export(
            path=filename,
            resolution=resolution,
            width=width,
            labels=labels,
            legend_position=legend_position,
            render_ocr=render_ocr,
            **kwargs,
        )
    else:
        # For saving without highlights, use render() and save manually
        img = self.render(resolution=resolution, **kwargs)
        if img:
            # Resize if width is specified
            if width is not None and width > 0 and img.width > 0:
                aspect_ratio = img.height / img.width
                height = int(width * aspect_ratio)
                try:
                    img = img.resize((width, height), Image.Resampling.LANCZOS)
                except Exception as e:
                    logger.warning(f"Could not resize image: {e}")

            # Save the image
            try:
                if os.path.dirname(filename):
                    os.makedirs(os.path.dirname(filename), exist_ok=True)
                img.save(filename)
            except Exception as e:
                logger.error(f"Failed to save image to {filename}: {e}")

    return self

natural_pdf.Page.save_searchable(output_path, dpi=300)

Saves the PDF page with an OCR text layer, making content searchable.

Requires optional dependencies. Install with: pip install "natural-pdf[ocr-save]"

OCR must have been applied to the pages beforehand

(e.g., pdf.apply_ocr()).

Parameters:

Name	Type	Description	Default
output_path	Union[str, Path]	Path to save the searchable PDF.	required
dpi	int	Resolution for rendering and OCR overlay (default 300).	300

Source code in natural_pdf/core/page.py

def save_searchable(self, output_path: Union[str, "Path"], dpi: int = 300):
    """
    Saves the PDF page with an OCR text layer, making content searchable.

    Requires optional dependencies. Install with: pip install "natural-pdf[ocr-save]"

    Note: OCR must have been applied to the pages beforehand
            (e.g., pdf.apply_ocr()).

    Args:
        output_path: Path to save the searchable PDF.
        dpi: Resolution for rendering and OCR overlay (default 300).
    """
    # Import moved here, assuming it's always available now
    from natural_pdf.exporters.searchable_pdf import create_searchable_pdf

    # Convert pathlib.Path to string if necessary
    output_path_str = str(output_path)

    create_searchable_pdf(self, output_path_str, dpi=dpi)
    logger.info(f"Searchable PDF saved to: {output_path_str}")

natural_pdf.Page.show_preview(temporary_highlights, resolution=144, width=None, labels=True, legend_position='right', render_ocr=False)

Generates and returns a non-stateful preview image containing only the provided temporary highlights.

Parameters:

Name	Type	Description	Default
temporary_highlights	List[Dict]	List of highlight data dictionaries (as prepared by ElementCollection._prepare_highlight_data).	required
resolution	float	Resolution in DPI for rendering (default: 144 DPI, equivalent to previous scale=2.0).	144
width	Optional[int]	Optional width for the output image.	None
labels	bool	Whether to include a legend.	True
legend_position	str	Position of the legend.	'right'
render_ocr	bool	Whether to render OCR text.	False

Returns:

Type	Description
Optional[Image]	PIL Image object of the preview, or None if rendering fails.

Source code in natural_pdf/core/page.py

def show_preview(
    self,
    temporary_highlights: List[Dict],
    resolution: float = 144,
    width: Optional[int] = None,
    labels: bool = True,
    legend_position: str = "right",
    render_ocr: bool = False,
) -> Optional[Image.Image]:
    """
    Generates and returns a non-stateful preview image containing only
    the provided temporary highlights.

    Args:
        temporary_highlights: List of highlight data dictionaries (as prepared by
                                ElementCollection._prepare_highlight_data).
        resolution: Resolution in DPI for rendering (default: 144 DPI, equivalent to previous scale=2.0).
        width: Optional width for the output image.
        labels: Whether to include a legend.
        legend_position: Position of the legend.
        render_ocr: Whether to render OCR text.

    Returns:
        PIL Image object of the preview, or None if rendering fails.
    """
    try:
        # Delegate rendering to the highlighter service's preview method
        img = self._highlighter.render_preview(
            page_index=self.index,
            temporary_highlights=temporary_highlights,
            resolution=resolution,
            labels=labels,
            legend_position=legend_position,
            render_ocr=render_ocr,
        )
    except AttributeError:
        logger.error("HighlightingService does not have the required 'render_preview' method.")
        return None
    except Exception as e:
        logger.error(
            f"Error calling highlighter.render_preview for page {self.index}: {e}",
            exc_info=True,
        )
        return None

    # Return the rendered image directly
    return img

natural_pdf.Page.split(divider, **kwargs)

Divide the page into sections based on the provided divider elements.

Source code in natural_pdf/core/page.py

def split(self, divider, **kwargs: Any) -> "ElementCollection[Region]":
    """Divide the page into sections based on the provided divider elements."""
    base_region = self.create_region(0, 0, self.width, self.height)
    return base_region.split(divider, **kwargs)

natural_pdf.Page.to_region()

Return a Region covering the full page.

Source code in natural_pdf/core/page.py

def to_region(self) -> Region:
    """Return a Region covering the full page."""
    return self.region(0, 0, self.width, self.height)

natural_pdf.Page.until(selector, include_endpoint=True, *, text=None, apply_exclusions=True, regex=False, case=True, text_tolerance=None, auto_text_tolerance=None, reading_order=True)

Select content from the top of the page until matching selector.

Parameters:

Name	Type	Description	Default
selector	str	CSS-like selector string	required
include_endpoint	bool	Whether to include the endpoint element in the region	True
**kwargs		Additional selection parameters	required

Returns:

Type	Description
Any	Region object representing the selected content

Examples:

>>> page.until('text:contains("Conclusion")')  # Select from top to conclusion
>>> page.until('line[width>=2]', include_endpoint=False)  # Select up to thick line

Source code in natural_pdf/core/page.py

def until(
    self,
    selector: str,
    include_endpoint: bool = True,
    *,
    text: Optional[Union[str, Sequence[str]]] = None,
    apply_exclusions: bool = True,
    regex: bool = False,
    case: bool = True,
    text_tolerance: Optional[Dict[str, Any]] = None,
    auto_text_tolerance: Optional[Union[bool, Dict[str, Any]]] = None,
    reading_order: bool = True,
) -> Any:
    """
    Select content from the top of the page until matching selector.

    Args:
        selector: CSS-like selector string
        include_endpoint: Whether to include the endpoint element in the region
        **kwargs: Additional selection parameters

    Returns:
        Region object representing the selected content

    Examples:
        >>> page.until('text:contains("Conclusion")')  # Select from top to conclusion
        >>> page.until('line[width>=2]', include_endpoint=False)  # Select up to thick line
    """
    # Find the target element
    target = self.find(
        selector,
        text=text,
        apply_exclusions=apply_exclusions,
        regex=regex,
        case=case,
        text_tolerance=text_tolerance,
        auto_text_tolerance=auto_text_tolerance,
        reading_order=reading_order,
    )
    if not target:
        # If target not found, return a default region (full page)
        from natural_pdf.elements.region import Region

        return Region(self, (0, 0, self.width, self.height))

    # Create a region from the top of the page to the target
    from natural_pdf.elements.region import Region

    # Ensure target has positional attributes before using them
    target_top = getattr(target, "top", 0)
    target_bottom = getattr(target, "bottom", self.height)

    if include_endpoint:
        # Include the target element
        region = Region(self, (0, 0, self.width, target_bottom))
    else:
        # Up to the target element
        region = Region(self, (0, 0, self.width, target_top))

    region.end_element = cast(Element, target)
    return region

natural_pdf.Page.update_text(transform, *, selector='text', apply_exclusions=False, max_workers=None, progress_callback=None)

Applies corrections to text elements on this page using a user-provided callback function, potentially in parallel.

Finds text elements on this page matching the selector argument and calls the transform for each, passing the element itself. Updates the element's text if the callback returns a new string.

Parameters:

Name	Type	Description	Default
transform	Callable[[Any], Optional[str]]	A function accepting an element and returning Optional[str] (new text or None).	required
selector	str	CSS-like selector string to match text elements.	'text'
apply_exclusions	bool	Whether exclusion regions should be honoured (default: False to maintain backward compatibility with the base mixin behaviour).	False
max_workers	Optional[int]	The maximum number of threads to use for parallel execution. If None or 0 or 1, runs sequentially.	None
progress_callback	Optional[Callable[[], None]]	Optional callback function to call after processing each element.	None

Returns:

Type	Description
Page	Self for method chaining.

Source code in natural_pdf/core/page.py

def update_text(
    self,
    transform: Callable[[Any], Optional[str]],
    *,
    selector: str = "text",
    apply_exclusions: bool = False,
    max_workers: Optional[int] = None,
    progress_callback: Optional[Callable[[], None]] = None,  # Added progress callback
) -> "Page":  # Return self for chaining
    """
    Applies corrections to text elements on this page
    using a user-provided callback function, potentially in parallel.

    Finds text elements on this page matching the *selector* argument and
    calls the ``transform`` for each, passing the element itself.
    Updates the element's text if the callback returns a new string.

    Args:
        transform: A function accepting an element and returning
                    `Optional[str]` (new text or None).
        selector: CSS-like selector string to match text elements.
        apply_exclusions: Whether exclusion regions should be honoured (default: False to
            maintain backward compatibility with the base mixin behaviour).
        max_workers: The maximum number of threads to use for parallel execution.
                        If None or 0 or 1, runs sequentially.
        progress_callback: Optional callback function to call after processing each element.

    Returns:
        Self for method chaining.
    """
    logger.info(
        f"Page {self.number}: Starting text update with callback '{transform.__name__}' (max_workers={max_workers}) and selector='{selector}'"
    )

    target_elements_collection = self.find_all(
        selector=selector, apply_exclusions=apply_exclusions
    )
    target_elements = target_elements_collection.elements  # Get the list

    if not target_elements:
        logger.info(f"Page {self.number}: No text elements found to update.")
        return self

    element_pbar = None
    try:
        element_pbar = tqdm(
            total=len(target_elements),
            desc=f"Updating text Page {self.number}",
            unit="element",
            leave=False,
        )

        processed_count = 0
        updated_count = 0
        error_count = 0

        # Define the task to be run by the worker thread or sequentially
        def _process_element_task(element):
            try:
                # Call the user-provided callback
                corrected_text = transform(element)

                # Validate result type
                if corrected_text is not None and not isinstance(corrected_text, str):
                    logger.warning(
                        f"Page {self.number}: Correction callback for element '{getattr(element, 'text', '')[:20]}...' returned non-string, non-None type: {type(corrected_text)}. Skipping update."
                    )
                    return element, None, None  # Treat as no correction

                return element, corrected_text, None  # Return element, result, no error
            except Exception as e:
                logger.error(
                    f"Page {self.number}: Error applying correction callback to element '{getattr(element, 'text', '')[:30]}...' ({element.bbox}): {e}",
                    exc_info=False,  # Keep log concise
                )
                return element, None, e  # Return element, no result, error
            finally:
                if element_pbar:
                    element_pbar.update(1)
                if progress_callback:
                    try:
                        progress_callback()
                    except Exception as cb_e:
                        # Log error in callback itself, but don't stop processing
                        logger.error(
                            f"Page {self.number}: Error executing progress_callback: {cb_e}",
                            exc_info=False,
                        )

        # Choose execution strategy based on max_workers
        if max_workers is not None and max_workers > 1:
            logger.info(
                f"Page {self.number}: Running text update in parallel with {max_workers} workers."
            )
            with concurrent.futures.ThreadPoolExecutor(max_workers=max_workers) as executor:
                # Submit all tasks
                future_to_element = {
                    executor.submit(_process_element_task, element): element
                    for element in target_elements
                }

                # Process results as they complete (progress_callback called by worker)
                for future in concurrent.futures.as_completed(future_to_element):
                    processed_count += 1
                    try:
                        element, corrected_text, error = future.result()
                        if error:
                            error_count += 1
                            # Error already logged in worker
                        elif corrected_text is not None:
                            # Apply correction if text changed
                            current_text = getattr(element, "text", None)
                            if corrected_text != current_text:
                                element.text = corrected_text
                                updated_count += 1
                    except Exception as exc:
                        # Catch errors from future.result() itself
                        element = future_to_element[future]  # Find original element
                        logger.error(
                            f"Page {self.number}: Internal error retrieving correction result for element {element.bbox}: {exc}",
                            exc_info=True,
                        )
                        error_count += 1
                        # Note: progress_callback was already called in the worker's finally block

        else:
            logger.info(f"Page {self.number}: Running text update sequentially.")
            for element in target_elements:
                # Call the task function directly (it handles progress_callback)
                processed_count += 1
                _element, corrected_text, error = _process_element_task(element)
                if error:
                    error_count += 1
                elif corrected_text is not None:
                    # Apply correction if text changed
                    current_text = getattr(_element, "text", None)
                    if corrected_text != current_text:
                        _element.text = corrected_text
                        updated_count += 1

        logger.info(
            f"Page {self.number}: Text update finished. Processed: {processed_count}/{len(target_elements)}, Updated: {updated_count}, Errors: {error_count}."
        )

        return self  # Return self for chaining
    finally:
        if element_pbar:
            element_pbar.close()

natural_pdf.Page.viewer()

Creates and returns an interactive ipywidget for exploring elements on this page.

Uses InteractiveViewerWidget.from_page() to create the viewer.

Returns:

Type	Description
Optional[Any]	A InteractiveViewerWidget instance ready for display in Jupyter,
Optional[Any]	or None if ipywidgets is not installed or widget creation fails.

Raises:

Type	Description
# Optional	Could raise ImportError instead of returning None
# ImportError	If required dependencies (ipywidgets) are missing.
ValueError	If image rendering or data preparation fails within from_page.

Source code in natural_pdf/core/page.py

def viewer(
    self,
    # elements_to_render: Optional[List['Element']] = None, # No longer needed, from_page handles it
    # include_source_types: List[str] = ['word', 'line', 'rect', 'region'] # No longer needed
) -> Optional[Any]:
    """
    Creates and returns an interactive ipywidget for exploring elements on this page.

    Uses InteractiveViewerWidget.from_page() to create the viewer.

    Returns:
        A InteractiveViewerWidget instance ready for display in Jupyter,
        or None if ipywidgets is not installed or widget creation fails.

    Raises:
        # Optional: Could raise ImportError instead of returning None
        # ImportError: If required dependencies (ipywidgets) are missing.
        ValueError: If image rendering or data preparation fails within from_page.
    """
    # Check for availability using the imported flag and class variable
    if not _IPYWIDGETS_AVAILABLE or InteractiveViewerWidget is None:
        logger.error(
            "Interactive viewer requires 'ipywidgets'. "
            'Please install with: pip install "ipywidgets>=7.0.0,<10.0.0"'
        )
        # raise ImportError("ipywidgets not found.") # Option 1: Raise error
        return None  # Option 2: Return None gracefully

    # If we reach here, InteractiveViewerWidget should be the actual class
    try:
        # Pass self (the Page object) to the factory method
        return InteractiveViewerWidget.from_page(self)
    except Exception as e:
        # Catch potential errors during widget creation (e.g., image rendering)
        logger.error(
            f"Error creating viewer widget from page {self.number}: {e}", exc_info=True
        )
        # raise # Option 1: Re-raise error (might include ValueError from from_page)
        return None  # Option 2: Return None on creation error

natural_pdf.Page.without_exclusions()

Context manager that temporarily disables exclusion processing.

This prevents infinite recursion when exclusion callables themselves use find() operations. While in this context, all find operations will skip exclusion filtering.

Example

# This exclusion would normally cause infinite recursion:
page.add_exclusion(lambda p: p.find("text:contains('Header')").expand())

# But internally, it's safe because we use:
with page.without_exclusions():
    region = exclusion_callable(page)

Yields:

Type	Description
	The page object with exclusions temporarily disabled.

Source code in natural_pdf/core/page.py

@contextlib.contextmanager
def without_exclusions(self):
    """
    Context manager that temporarily disables exclusion processing.

    This prevents infinite recursion when exclusion callables themselves
    use find() operations. While in this context, all find operations
    will skip exclusion filtering.

    Example:
        ```python
        # This exclusion would normally cause infinite recursion:
        page.add_exclusion(lambda p: p.find("text:contains('Header')").expand())

        # But internally, it's safe because we use:
        with page.without_exclusions():
            region = exclusion_callable(page)
        ```

    Yields:
        The page object with exclusions temporarily disabled.
    """
    old_value = self._computing_exclusions
    self._computing_exclusions = True
    try:
        yield self
    finally:
        self._computing_exclusions = old_value

natural_pdf.PageCollection

Bases: TextMixin, ApplyMixin, SectionsCollectionMixin, ShapeDetectionMixin, CheckboxDetectionMixin, Visualizable, Sequence['Page']

Represents a collection of Page objects, often from a single PDF document. Provides methods for batch operations on these pages.

Source code in natural_pdf/core/page_collection.py

class PageCollection(
    TextMixin,
    ApplyMixin,
    SectionsCollectionMixin,
    ShapeDetectionMixin,
    CheckboxDetectionMixin,
    Visualizable,
    Sequence["Page"],
):
    """
    Represents a collection of Page objects, often from a single PDF document.
    Provides methods for batch operations on these pages.
    """

    def __init__(self, pages: Sequence["Page"] | Iterable["Page"]):
        """
        Initialize a page collection.

        Args:
            pages: List or sequence of Page objects (can be lazy)
        """
        # Store the sequence as-is to preserve lazy behavior
        # Only convert to list if we need list-specific operations
        if isinstance(pages, Sequence):
            self.pages: Sequence["Page"] = pages
        else:
            # Fallback for non-sequence types – materialise to preserve ordering
            self.pages = list(pages)

    def __len__(self) -> int:
        """Return the number of pages in the collection."""
        return len(self.pages)

    @overload
    def __getitem__(self, idx: int) -> "Page": ...

    @overload
    def __getitem__(self, idx: slice) -> Sequence["Page"]: ...

    def __getitem__(self, idx: Union[int, slice]) -> Union["Page", Sequence["Page"]]:
        """Support indexing and slicing."""
        if isinstance(idx, slice):
            return PageCollection(self.pages[idx])
        return self.pages[idx]

    def __iter__(self) -> Iterator["Page"]:
        """Support iteration."""
        return iter(self.pages)

    def __repr__(self) -> str:
        """Return a string representation showing the page count."""
        return f"<PageCollection(count={len(self)})>"

    @property
    def elements(self) -> Sequence["Page"]:
        """Alias to expose pages for APIs expecting an elements attribute."""
        return self.pages

    def _get_items_for_apply(self) -> Iterator["Page"]:
        """
        Override ApplyMixin's _get_items_for_apply to preserve lazy behavior.

        Returns an iterator that yields pages on-demand rather than materializing
        all pages at once, maintaining the lazy loading behavior.
        """
        return iter(self.pages)

    def _get_page_indices(self) -> List[int]:
        """
        Get page indices without forcing materialization of pages.

        Returns:
            List of page indices for the pages in this collection.
        """
        # Handle different types of page sequences efficiently
        indices = getattr(self.pages, "_indices", None)
        if indices is not None:
            return list(indices)
        else:
            return [p.index for p in self.pages]

    def _resolve_parent_pdf(self) -> "PDF":
        """
        Resolve the parent PDF instance shared by pages in this collection.

        Raises:
            RuntimeError: If the collection is empty or pages do not expose a parent PDF.
        """
        if not self.pages:
            raise RuntimeError("PageCollection is empty; cannot resolve parent PDF.")

        parent = getattr(self.pages[0], "_parent", None)
        if parent is None:
            raise RuntimeError("Pages in this collection do not expose a parent PDF.")

        return cast("PDF", parent)

    def extract_text(
        self,
        separator: str = "\n",
        apply_exclusions: bool = True,
        **kwargs,
    ) -> str:
        """
        Extract text from all pages in the collection.

        Args:
            keep_blank_chars: Whether to keep blank characters (default: True)
            apply_exclusions: Whether to apply exclusion regions (default: True)
            strip: Whether to strip whitespace from the extracted text.
            **kwargs: Additional extraction parameters

        Returns:
            Combined text from all pages
        """
        keep_blank_chars = kwargs.pop("keep_blank_chars", True)
        if keep_blank_chars is None:
            keep_blank_chars = True
        strip = kwargs.pop("strip", None)
        explicit_strip_final = kwargs.pop("strip_final", None)
        explicit_strip_empty = kwargs.pop("strip_empty", None)

        texts: List[str] = []

        for page in self.pages:
            text = page.extract_text(
                preserve_whitespace=keep_blank_chars,
                use_exclusions=apply_exclusions,
                strip_final=(
                    explicit_strip_final
                    if explicit_strip_final is not None
                    else (strip if strip is not None else False)
                ),
                strip_empty=explicit_strip_empty if explicit_strip_empty is not None else False,
                **kwargs,
            )
            texts.append(text)

        return separator.join(texts)

    def apply_ocr(
        self,
        engine: Optional[str] = None,
        # --- Common OCR Parameters (Direct Arguments) ---
        languages: Optional[List[str]] = None,
        min_confidence: Optional[float] = None,  # Min confidence threshold
        device: Optional[str] = None,
        resolution: Optional[int] = None,  # DPI for rendering
        apply_exclusions: bool = True,  # New parameter
        replace: bool = True,  # Whether to replace existing OCR elements
        # --- Engine-Specific Options ---
        options: Optional[Any] = None,  # e.g., EasyOCROptions(...)
    ) -> "PageCollection":
        """
        Applies OCR to all pages within this collection using batch processing.

        This delegates the work to the parent PDF object's `apply_ocr` method.

        Args:
            engine: Name of the OCR engine (e.g., 'easyocr', 'paddleocr').
            languages: List of language codes (e.g., ['en', 'fr'], ['en', 'ch']).
                       **Must be codes understood by the specific selected engine.**
                       No mapping is performed.
            min_confidence: Minimum confidence threshold for detected text (0.0 to 1.0).
            device: Device to run OCR on (e.g., 'cpu', 'cuda', 'mps').
            resolution: DPI resolution to render page images before OCR (e.g., 150, 300).
            apply_exclusions: If True (default), render page images for OCR with
                              excluded areas masked (whited out). If False, OCR
                              the raw page images without masking exclusions.
            replace: If True (default), remove any existing OCR elements before
                    adding new ones. If False, add new OCR elements to existing ones.
            options: An engine-specific options object (e.g., EasyOCROptions) or dict.

        Returns:
            Self for method chaining.

        Raises:
            RuntimeError: If pages lack a parent PDF or parent lacks `apply_ocr`.
            (Propagates exceptions from PDF.apply_ocr)
        """
        if not self.pages:
            logger.warning("Cannot apply OCR to an empty PageCollection.")
            return self

        parent_pdf = self._resolve_parent_pdf()

        if not hasattr(parent_pdf, "apply_ocr") or not callable(parent_pdf.apply_ocr):
            raise RuntimeError("Parent PDF object does not have the required 'apply_ocr' method.")

        # Get the 0-based indices of the pages in this collection
        page_indices = self._get_page_indices()

        logger.info(f"Applying OCR via parent PDF to page indices: {page_indices} in collection.")

        # Delegate the batch call to the parent PDF object, passing direct args and apply_exclusions
        parent_pdf.apply_ocr(
            pages=page_indices,
            engine=engine,
            languages=languages,
            min_confidence=min_confidence,  # Pass the renamed parameter
            device=device,
            resolution=resolution,
            apply_exclusions=apply_exclusions,  # Pass down
            replace=replace,  # Pass the replace parameter
            options=options,
        )
        # The PDF method modifies the Page objects directly by adding elements.

        return self  # Return self for chaining

    def _iter_sections(self) -> Iterable["_SectionHost"]:
        return cast(Iterable["_SectionHost"], iter(self.pages))

    def split(
        self,
        divider: BoundarySource,
        *,
        include_boundaries: str = "start",
        orientation: str = "vertical",
        new_section_on_page_break: bool = False,
    ) -> "ElementCollection[Region]":
        """
        Divide this page collection into sections based on the provided divider elements.

        Args:
            divider: Elements or selector string that mark section boundaries
            include_boundaries: How to include boundary elements (default: 'start').
            orientation: 'vertical' or 'horizontal' (default: 'vertical').
            new_section_on_page_break: Whether to split at page boundaries (default: False).

        Returns:
            ElementCollection of Region objects representing the sections

        Example:
            # Split a PDF by chapter titles
            chapters = pdf.pages.split("text[size>20]:contains('CHAPTER')")

            # Split by page breaks
            page_sections = pdf.pages.split(None, new_section_on_page_break=True)

            # Split multi-page document by section headers
            sections = pdf.pages[10:20].split("text:bold:contains('Section')")
        """
        sections = self.get_sections(
            start_elements=divider,
            include_boundaries=include_boundaries,
            orientation=orientation,
            new_section_on_page_break=new_section_on_page_break,
        )

        # Add initial section if there's content before the first divider
        if sections and divider is not None:
            # Get all elements across all pages
            all_elements = []
            for page in self.pages:
                all_elements.extend(page.get_elements())

            if all_elements:
                # Find first divider
                if isinstance(divider, str):
                    # Search for first matching element
                    first_divider = None
                    for page in self.pages:
                        match = page.find(divider)
                        if match:
                            first_divider = match
                            break
                else:
                    # divider is already elements
                    first_divider = None
                    if isinstance(divider, Iterable):
                        first_candidate = next(iter(divider), None)
                    else:
                        first_candidate = divider
                    if isinstance(first_candidate, SupportsGeometry):
                        first_divider = first_candidate

                if first_divider and all_elements[0] != first_divider:
                    # There's content before the first divider
                    # Get section from start to first divider
                    initial_sections = self.get_sections(
                        start_elements=None,
                        end_elements=[first_divider],
                        include_boundaries="none",
                        orientation=orientation,
                    )
                    if initial_sections:
                        sections = ElementCollection([initial_sections[0]] + list(sections))

        return sections

    def get_sections(
        self,
        start_elements: BoundarySource = None,
        end_elements: BoundarySource = None,
        new_section_on_page_break: bool = False,
        include_boundaries: str = "both",
        orientation: str = "vertical",
    ) -> "ElementCollection":
        """Extract logical sections across this collection of pages.

        This delegates to :class:`natural_pdf.flows.flow.Flow`, which already
        implements the heavy lifting for cross-segment section extraction and
        returns either :class:`Region` or :class:`FlowRegion` objects as
        appropriate.  The arrangement is chosen based on the requested
        orientation so that horizontal sections continue to work for rotated
        content.
        """

        from natural_pdf.elements.element_collection import ElementCollection
        from natural_pdf.flows.flow import Flow

        if len(self) == 0:
            return ElementCollection([])

        arrangement = "vertical" if orientation == "vertical" else "horizontal"

        flow = Flow(self, arrangement=arrangement)
        sections = flow.get_sections(
            start_elements=start_elements,
            end_elements=end_elements,
            new_section_on_page_break=new_section_on_page_break,
            include_boundaries=include_boundaries,
            orientation=orientation,
        )
        section_list = sections.elements if hasattr(sections, "elements") else list(sections)
        cleaned = sanitize_sections(section_list, orientation=orientation)
        if len(cleaned) == len(section_list):
            return sections
        return ElementCollection(cleaned)

    def _gather_analysis_data(
        self,
        analysis_keys: List[str],
        include_content: bool,
        include_images: bool,
        image_dir: Optional[Path],
        image_format: str,
        image_resolution: int,
    ) -> List[Dict[str, Any]]:
        """
        Gather analysis data from all pages in the collection.

        Args:
            analysis_keys: Keys in the analyses dictionary to export
            include_content: Whether to include extracted text
            include_images: Whether to export images
            image_dir: Directory to save images
            image_format: Format to save images
            image_resolution: Resolution for exported images

        Returns:
            List of dictionaries containing analysis data
        """
        if not self.elements:
            logger.warning("No pages found in collection")
            return []

        all_data = []

        for page in self.elements:
            # Basic page information
            page_data = {
                "page_number": page.number,
                "page_index": page.index,
                "width": page.width,
                "height": page.height,
            }

            # Add PDF information if available
            pdf_obj = getattr(page, "pdf", None)
            if pdf_obj is not None:
                pdf_path = getattr(pdf_obj, "path", None)
                if pdf_path:
                    page_data["pdf_path"] = pdf_path
                    page_data["pdf_filename"] = Path(pdf_path).name

            # Include extracted text if requested
            if include_content:
                try:
                    page_data["content"] = page.extract_text(preserve_whitespace=True)
                except Exception as e:
                    logger.error(f"Error extracting text from page {page.number}: {e}")
                    page_data["content"] = ""

            # Save image if requested
            if include_images:
                if image_dir is None:
                    logger.error("image_dir must be provided when include_images=True")
                else:
                    try:
                        # Create image filename
                        pdf_name = "unknown"
                        if pdf_obj is not None:
                            pdf_path = getattr(pdf_obj, "path", None)
                            if pdf_path:
                                pdf_name = Path(pdf_path).stem

                        image_filename = f"{pdf_name}_page_{page.number}.{image_format}"
                        image_path = image_dir / image_filename

                        # Save image
                        page.save_image(
                            str(image_path), resolution=image_resolution, include_highlights=True
                        )

                        # Add relative path to data
                        page_data["image_path"] = str(
                            Path(image_path).relative_to(image_dir.parent)
                        )
                    except Exception as e:
                        logger.error(f"Error saving image for page {page.number}: {e}")
                        page_data["image_path"] = None

            # Add analyses data
            if hasattr(page, "analyses") and page.analyses:
                for key in analysis_keys:
                    if key not in page.analyses:
                        raise KeyError(f"Analysis key '{key}' not found in page {page.number}")

                    # Get the analysis result
                    analysis_result = page.analyses[key]

                    # If the result has a to_dict method, use it
                    if hasattr(analysis_result, "to_dict"):
                        analysis_data = analysis_result.to_dict()
                    else:
                        # Otherwise, use the result directly if it's dict-like
                        try:
                            analysis_data = dict(analysis_result)
                        except (TypeError, ValueError):
                            # Last resort: convert to string
                            analysis_data = {"raw_result": str(analysis_result)}

                    # Add analysis data to page data with the key as prefix
                    for k, v in analysis_data.items():
                        page_data[f"{key}.{k}"] = v

            all_data.append(page_data)

        return all_data

    # --- Deskew Method --- #

    def deskew(
        self,
        resolution: int = 300,
        detection_resolution: int = 72,
        force_overwrite: bool = False,
        **deskew_kwargs,
    ) -> "PDF":  # Changed return type
        """
        Creates a new, in-memory PDF object containing deskewed versions of the pages
        in this collection.

        This method delegates the actual processing to the parent PDF object's
        `deskew` method.

        Important: The returned PDF is image-based. Any existing text, OCR results,
        annotations, or other elements from the original pages will *not* be carried over.

        Args:
            resolution: DPI resolution for rendering the output deskewed pages.
            detection_resolution: DPI resolution used for skew detection if angles are not
                                  already cached on the page objects.
            force_overwrite: If False (default), raises a ValueError if any target page
                             already contains processed elements (text, OCR, regions) to
                             prevent accidental data loss. Set to True to proceed anyway.
            **deskew_kwargs: Additional keyword arguments passed to `deskew.determine_skew`
                             during automatic detection (e.g., `max_angle`, `num_peaks`).

        Returns:
            A new PDF object representing the deskewed document.

        Raises:
            ImportError: If 'deskew' or 'img2pdf' libraries are not installed (raised by PDF.deskew).
            ValueError: If `force_overwrite` is False and target pages contain elements (raised by PDF.deskew),
                        or if the collection is empty.
            RuntimeError: If pages lack a parent PDF reference, or the parent PDF lacks the `deskew` method.
        """
        if not self.pages:
            logger.warning("Cannot deskew an empty PageCollection.")
            raise ValueError("Cannot deskew an empty PageCollection.")

        parent_pdf = self._resolve_parent_pdf()

        if not parent_pdf or not hasattr(parent_pdf, "deskew") or not callable(parent_pdf.deskew):
            raise RuntimeError(
                "Parent PDF reference not found or parent PDF lacks the required 'deskew' method."
            )

        # Get the 0-based indices of the pages in this collection
        page_indices = self._get_page_indices()
        logger.info(
            f"PageCollection: Delegating deskew to parent PDF for page indices: {page_indices}"
        )

        # Delegate the call to the parent PDF object for the relevant pages
        # Pass all relevant arguments through (no output_path anymore)
        return parent_pdf.deskew(
            pages=page_indices,
            resolution=resolution,
            detection_resolution=detection_resolution,
            force_overwrite=force_overwrite,
            **deskew_kwargs,
        )

    # --- End Deskew Method --- #

    def _get_render_specs(
        self,
        mode: Literal["show", "render"] = "show",
        color: Optional[Union[str, Tuple[int, int, int]]] = None,
        highlights: Optional[List[Dict[str, Any]]] = None,
        crop: Union[bool, Literal["content"]] = False,
        crop_bbox: Optional[Tuple[float, float, float, float]] = None,
        **kwargs,
    ) -> List[RenderSpec]:
        """Get render specifications for this page collection.

        For page collections, we return specs for all pages that will be
        rendered into a grid layout.

        Args:
            mode: Rendering mode - 'show' includes highlights, 'render' is clean
            color: Color for highlighting pages in show mode
            highlights: Additional highlight groups to show
            crop: Whether to crop pages
            crop_bbox: Explicit crop bounds
            **kwargs: Additional parameters

        Returns:
            List of RenderSpec objects, one per page
        """
        specs = []

        # Get max pages from kwargs if specified
        max_pages = kwargs.get("max_pages")
        pages_to_render = self.pages[:max_pages] if max_pages else self.pages

        for page in pages_to_render:
            if hasattr(page, "_get_render_specs"):
                # Page has the new unified rendering
                page_specs = page._get_render_specs(
                    mode=mode,
                    color=color,
                    highlights=highlights,
                    crop=crop,
                    crop_bbox=crop_bbox,
                    **kwargs,
                )
                specs.extend(page_specs)
            else:
                # Fallback for pages without unified rendering
                spec = RenderSpec(page=page)
                if crop_bbox:
                    spec.crop_bbox = crop_bbox
                specs.append(spec)

        return specs

    def save_pdf(
        self,
        output_path: Union[str, Path],
        ocr: bool = False,
        original: bool = False,
        dpi: int = 300,
    ):
        """
        Saves the pages in this collection to a new PDF file.

        Choose one saving mode:
        - `ocr=True`: Creates a new, image-based PDF using OCR results. This
          makes the text generated during the natural-pdf session searchable,
          but loses original vector content. Requires 'ocr-export' extras.
        - `original=True`: Extracts the original pages from the source PDF,
          preserving all vector content, fonts, and annotations. OCR results
          from the natural-pdf session are NOT included. Requires 'ocr-export' extras.

        Args:
            output_path: Path to save the new PDF file.
            ocr: If True, save as a searchable, image-based PDF using OCR data.
            original: If True, save the original, vector-based pages.
            dpi: Resolution (dots per inch) used only when ocr=True for
                 rendering page images and aligning the text layer.

        Raises:
            ValueError: If the collection is empty, if neither or both 'ocr'
                        and 'original' are True, or if 'original=True' and
                        pages originate from different PDFs.
            ImportError: If required libraries ('pikepdf', 'Pillow')
                         are not installed for the chosen mode.
            RuntimeError: If an unexpected error occurs during saving.
        """
        if not self.pages:
            raise ValueError("Cannot save an empty PageCollection.")

        if not (ocr ^ original):  # XOR: exactly one must be true
            raise ValueError("Exactly one of 'ocr' or 'original' must be True.")

        output_path_obj = Path(output_path)
        output_path_str = str(output_path_obj)

        if ocr:
            if create_searchable_pdf is None:
                raise ImportError(
                    "Saving with ocr=True requires 'pikepdf' and 'Pillow'. "
                    'Install with: pip install \\"natural-pdf[ocr-export]\\"'  # Escaped quotes
                )

            # Check for non-OCR vector elements (provide a warning)
            has_vector_elements = False
            for page in self.pages:
                # Simplified check for common vector types or non-OCR chars/words
                rects = getattr(page, "rects", [])
                lines = getattr(page, "lines", [])
                curves = getattr(page, "curves", [])
                chars = getattr(page, "chars", [])
                words = getattr(page, "words", [])

                if (
                    rects
                    or lines
                    or curves
                    or any(getattr(el, "source", None) != "ocr" for el in chars)
                    or any(getattr(el, "source", None) != "ocr" for el in words)
                ):
                    has_vector_elements = True
                    break
            if has_vector_elements:
                logger.warning(
                    "Warning: Saving with ocr=True creates an image-based PDF. "
                    "Original vector elements (rects, lines, non-OCR text/chars) "
                    "on selected pages will not be preserved in the output file."
                )

            logger.info(f"Saving searchable PDF (OCR text layer) to: {output_path_str}")
            try:
                # Delegate to the searchable PDF exporter function
                # Pass `self` (the PageCollection instance) as the source
                create_searchable_pdf(self, output_path_str, dpi=dpi)
                # Success log is now inside create_searchable_pdf if needed, or keep here
                # logger.info(f"Successfully saved searchable PDF to: {output_path_str}")
            except Exception as e:
                logger.error(f"Failed to create searchable PDF: {e}", exc_info=True)
                # Re-raise as RuntimeError for consistency, potentially handled in exporter too
                raise RuntimeError(f"Failed to create searchable PDF: {e}") from e

        elif original:
            # ---> MODIFIED: Call the new exporter
            if create_original_pdf is None:
                raise ImportError(
                    "Saving with original=True requires 'pikepdf'. "
                    'Install with: pip install \\"natural-pdf[ocr-export]\\"'  # Escaped quotes
                )

            # Check for OCR elements (provide a warning) - keep this check here
            has_ocr_elements = False
            for page in self.pages:
                # Use find_all which returns a collection; check if it's non-empty
                if hasattr(page, "find_all"):
                    ocr_text_elements = page.find_all("text[source=ocr]")
                    if ocr_text_elements:  # Check truthiness of collection
                        has_ocr_elements = True
                        break
                elif hasattr(page, "words"):  # Fallback check if find_all isn't present?
                    if any(getattr(el, "source", None) == "ocr" for el in page.words):
                        has_ocr_elements = True
                        break

            if has_ocr_elements:
                logger.warning(
                    "Warning: Saving with original=True preserves original page content. "
                    "OCR text generated in this session will not be included in the saved file."
                )

            logger.info(f"Saving original pages PDF to: {output_path_str}")
            try:
                # Delegate to the original PDF exporter function
                # Pass `self` (the PageCollection instance) as the source
                create_original_pdf(self, output_path_str)
                # Success log is now inside create_original_pdf
                # logger.info(f"Successfully saved original pages PDF to: {output_path_str}")
            except Exception as e:
                # Error logging is handled within create_original_pdf
                # Re-raise the exception caught from the exporter
                raise e  # Keep the original exception type (ValueError, RuntimeError, etc.)
            # <--- END MODIFIED

    def to_flow(
        self,
        arrangement: Literal["vertical", "horizontal"] = "vertical",
        alignment: Literal["start", "center", "end", "top", "left", "bottom", "right"] = "start",
        segment_gap: float = 0.0,
    ) -> "Flow":
        """
        Convert this PageCollection to a Flow for cross-page operations.

        This enables treating multiple pages as a continuous logical document
        structure, useful for multi-page tables, articles spanning columns,
        or any content requiring reading order across page boundaries.

        Args:
            arrangement: Primary flow direction ('vertical' or 'horizontal').
                        'vertical' stacks pages top-to-bottom (most common).
                        'horizontal' arranges pages left-to-right.
            alignment: Cross-axis alignment for pages of different sizes:
                      For vertical: 'left'/'start', 'center', 'right'/'end'
                      For horizontal: 'top'/'start', 'center', 'bottom'/'end'
            segment_gap: Virtual gap between pages in PDF points (default: 0.0).

        Returns:
            Flow object that can perform operations across all pages in sequence.

        Example:
            Multi-page table extraction:
            ```python
            pdf = npdf.PDF("multi_page_report.pdf")

            # Create flow for pages 2-4 containing a table
            table_flow = pdf.pages[1:4].to_flow()

            # Extract table as if it were continuous
            table_data = table_flow.extract_table()
            df = table_data.df
            ```

            Cross-page element search:
            ```python
            # Find all headers across multiple pages
            headers = pdf.pages[5:10].to_flow().find_all('text[size>12]:bold')

            # Analyze layout across pages
            regions = pdf.pages.to_flow().analyze_layout(engine='yolo')
            ```
        """
        from natural_pdf.flows.flow import Flow

        return Flow(
            segments=self,  # Flow constructor now handles PageCollection
            arrangement=arrangement,
            alignment=alignment,
            segment_gap=segment_gap,
        )

    def analyze_layout(self, *args, **kwargs) -> "ElementCollection[Region]":
        """
        Analyzes the layout of each page in the collection.

        This method iterates through each page, calls its analyze_layout method,
        and returns a single ElementCollection containing all the detected layout
        regions from all pages.

        Args:
            *args: Positional arguments to pass to each page's analyze_layout method.
            **kwargs: Keyword arguments to pass to each page's analyze_layout method.
                      A 'show_progress' kwarg can be included to show a progress bar.

        Returns:
            An ElementCollection of all detected Region objects.
        """
        all_regions = []

        show_progress = kwargs.pop("show_progress", True)

        iterator = self.pages
        if show_progress:
            try:
                from tqdm.auto import tqdm

                iterator = tqdm(self.pages, desc="Analyzing layout")
            except ImportError:
                pass  # tqdm not installed

        for page in iterator:
            # Each page's analyze_layout method returns an ElementCollection
            regions_collection = page.analyze_layout(*args, **kwargs)
            if regions_collection:
                all_regions.extend(regions_collection.elements)

        return ElementCollection(all_regions)

    def detect_checkboxes(self, *args, **kwargs) -> "ElementCollection[Region]":
        """
        Detects checkboxes on each page in the collection.

        This method iterates through each page, calls its detect_checkboxes method,
        and returns a single ElementCollection containing all detected checkbox
        regions from all pages.

        Args:
            *args: Positional arguments to pass to each page's detect_checkboxes method.
            **kwargs: Keyword arguments to pass to each page's detect_checkboxes method.
                      A 'show_progress' kwarg can be included to show a progress bar.

        Returns:
            An ElementCollection of all detected checkbox Region objects.
        """
        all_checkboxes = []

        show_progress = kwargs.pop("show_progress", True)

        iterator = self.pages
        if show_progress:
            try:
                from tqdm.auto import tqdm

                iterator = tqdm(self.pages, desc="Detecting checkboxes")
            except ImportError:
                pass  # tqdm not installed

        for page in iterator:
            # Each page's detect_checkboxes method returns an ElementCollection
            checkbox_collection = page.detect_checkboxes(*args, **kwargs)
            if checkbox_collection:
                all_checkboxes.extend(checkbox_collection.elements)

        return ElementCollection(all_checkboxes)

    def highlights(self, show: bool = False) -> "HighlightContext":
        """
        Create a highlight context for accumulating highlights.

        This allows for clean syntax to show multiple highlight groups:

        Example:
            with pages.highlights() as h:
                h.add(pages.find_all('table'), label='tables', color='blue')
                h.add(pages.find_all('text:bold'), label='bold text', color='red')
                h.show()

        Or with automatic display:
            with pages.highlights(show=True) as h:
                h.add(pages.find_all('table'), label='tables')
                h.add(pages.find_all('text:bold'), label='bold')
                # Automatically shows when exiting the context

        Args:
            show: If True, automatically show highlights when exiting context

        Returns:
            HighlightContext for accumulating highlights
        """
        from natural_pdf.core.highlighting_service import HighlightContext

        return HighlightContext(self, show_on_exit=show)

    def groupby(self, by: Union[str, Callable], *, show_progress: bool = True) -> "PageGroupBy":
        """
        Group pages by selector text or callable result.

        Args:
            by: CSS selector string or callable function
            show_progress: Whether to show progress bar during computation (default: True)

        Returns:
            PageGroupBy object supporting iteration and dict-like access

        Examples:
            # Group by header text
            for title, pages in pdf.pages.groupby('text[size=16]'):
                print(f"Section: {title}")

            # Group by callable
            for city, pages in pdf.pages.groupby(lambda p: p.find('text:contains("CITY")').extract_text()):
                process_city_pages(pages)

            # Quick exploration with indexing
            grouped = pdf.pages.groupby('text[size=16]')
            grouped.info()                    # Show all groups
            first_section = grouped[0]        # First group
            last_section = grouped[-1]       # Last group

            # Dict-like access by name
            madison_pages = grouped.get('CITY OF MADISON')
            madison_pages = grouped['CITY OF MADISON']  # Alternative

            # Disable progress bar for small collections
            grouped = pdf.pages.groupby('text[size=16]', show_progress=False)
        """
        from natural_pdf.core.page_groupby import PageGroupBy

        return PageGroupBy(self, by, show_progress=show_progress)

Attributes

natural_pdf.PageCollection.elements property

Alias to expose pages for APIs expecting an elements attribute.

Functions

natural_pdf.PageCollection.__getitem__(idx)

__getitem__(idx: int) -> 'Page'

__getitem__(idx: slice) -> Sequence['Page']

Support indexing and slicing.

Source code in natural_pdf/core/page_collection.py

def __getitem__(self, idx: Union[int, slice]) -> Union["Page", Sequence["Page"]]:
    """Support indexing and slicing."""
    if isinstance(idx, slice):
        return PageCollection(self.pages[idx])
    return self.pages[idx]

natural_pdf.PageCollection.__init__(pages)

Initialize a page collection.

Parameters:

Name	Type	Description	Default
pages	Sequence['Page'] | Iterable['Page']	List or sequence of Page objects (can be lazy)	required

Source code in natural_pdf/core/page_collection.py

def __init__(self, pages: Sequence["Page"] | Iterable["Page"]):
    """
    Initialize a page collection.

    Args:
        pages: List or sequence of Page objects (can be lazy)
    """
    # Store the sequence as-is to preserve lazy behavior
    # Only convert to list if we need list-specific operations
    if isinstance(pages, Sequence):
        self.pages: Sequence["Page"] = pages
    else:
        # Fallback for non-sequence types – materialise to preserve ordering
        self.pages = list(pages)

natural_pdf.PageCollection.__iter__()

Support iteration.

Source code in natural_pdf/core/page_collection.py

def __iter__(self) -> Iterator["Page"]:
    """Support iteration."""
    return iter(self.pages)

natural_pdf.PageCollection.__len__()

Return the number of pages in the collection.

Source code in natural_pdf/core/page_collection.py

def __len__(self) -> int:
    """Return the number of pages in the collection."""
    return len(self.pages)

natural_pdf.PageCollection.__repr__()

Return a string representation showing the page count.

Source code in natural_pdf/core/page_collection.py

def __repr__(self) -> str:
    """Return a string representation showing the page count."""
    return f"<PageCollection(count={len(self)})>"

natural_pdf.PageCollection.analyze_layout(*args, **kwargs)

Analyzes the layout of each page in the collection.

This method iterates through each page, calls its analyze_layout method, and returns a single ElementCollection containing all the detected layout regions from all pages.

Parameters:

Name	Type	Description	Default
*args		Positional arguments to pass to each page's analyze_layout method.	()
**kwargs		Keyword arguments to pass to each page's analyze_layout method. A 'show_progress' kwarg can be included to show a progress bar.	{}

Returns:

Type	Description
'ElementCollection[Region]'	An ElementCollection of all detected Region objects.

Source code in natural_pdf/core/page_collection.py

def analyze_layout(self, *args, **kwargs) -> "ElementCollection[Region]":
    """
    Analyzes the layout of each page in the collection.

    This method iterates through each page, calls its analyze_layout method,
    and returns a single ElementCollection containing all the detected layout
    regions from all pages.

    Args:
        *args: Positional arguments to pass to each page's analyze_layout method.
        **kwargs: Keyword arguments to pass to each page's analyze_layout method.
                  A 'show_progress' kwarg can be included to show a progress bar.

    Returns:
        An ElementCollection of all detected Region objects.
    """
    all_regions = []

    show_progress = kwargs.pop("show_progress", True)

    iterator = self.pages
    if show_progress:
        try:
            from tqdm.auto import tqdm

            iterator = tqdm(self.pages, desc="Analyzing layout")
        except ImportError:
            pass  # tqdm not installed

    for page in iterator:
        # Each page's analyze_layout method returns an ElementCollection
        regions_collection = page.analyze_layout(*args, **kwargs)
        if regions_collection:
            all_regions.extend(regions_collection.elements)

    return ElementCollection(all_regions)

natural_pdf.PageCollection.apply_ocr(engine=None, languages=None, min_confidence=None, device=None, resolution=None, apply_exclusions=True, replace=True, options=None)

Applies OCR to all pages within this collection using batch processing.

This delegates the work to the parent PDF object's apply_ocr method.

Parameters:

Name	Type	Description	Default
engine	Optional[str]	Name of the OCR engine (e.g., 'easyocr', 'paddleocr').	None
languages	Optional[List[str]]	List of language codes (e.g., ['en', 'fr'], ['en', 'ch']). Must be codes understood by the specific selected engine. No mapping is performed.	None
min_confidence	Optional[float]	Minimum confidence threshold for detected text (0.0 to 1.0).	None
device	Optional[str]	Device to run OCR on (e.g., 'cpu', 'cuda', 'mps').	None
resolution	Optional[int]	DPI resolution to render page images before OCR (e.g., 150, 300).	None
apply_exclusions	bool	If True (default), render page images for OCR with excluded areas masked (whited out). If False, OCR the raw page images without masking exclusions.	True
replace	bool	If True (default), remove any existing OCR elements before adding new ones. If False, add new OCR elements to existing ones.	True
options	Optional[Any]	An engine-specific options object (e.g., EasyOCROptions) or dict.	None

Returns:

Type	Description
'PageCollection'	Self for method chaining.

Raises:

Type	Description
RuntimeError	If pages lack a parent PDF or parent lacks apply_ocr.

Source code in natural_pdf/core/page_collection.py

def apply_ocr(
    self,
    engine: Optional[str] = None,
    # --- Common OCR Parameters (Direct Arguments) ---
    languages: Optional[List[str]] = None,
    min_confidence: Optional[float] = None,  # Min confidence threshold
    device: Optional[str] = None,
    resolution: Optional[int] = None,  # DPI for rendering
    apply_exclusions: bool = True,  # New parameter
    replace: bool = True,  # Whether to replace existing OCR elements
    # --- Engine-Specific Options ---
    options: Optional[Any] = None,  # e.g., EasyOCROptions(...)
) -> "PageCollection":
    """
    Applies OCR to all pages within this collection using batch processing.

    This delegates the work to the parent PDF object's `apply_ocr` method.

    Args:
        engine: Name of the OCR engine (e.g., 'easyocr', 'paddleocr').
        languages: List of language codes (e.g., ['en', 'fr'], ['en', 'ch']).
                   **Must be codes understood by the specific selected engine.**
                   No mapping is performed.
        min_confidence: Minimum confidence threshold for detected text (0.0 to 1.0).
        device: Device to run OCR on (e.g., 'cpu', 'cuda', 'mps').
        resolution: DPI resolution to render page images before OCR (e.g., 150, 300).
        apply_exclusions: If True (default), render page images for OCR with
                          excluded areas masked (whited out). If False, OCR
                          the raw page images without masking exclusions.
        replace: If True (default), remove any existing OCR elements before
                adding new ones. If False, add new OCR elements to existing ones.
        options: An engine-specific options object (e.g., EasyOCROptions) or dict.

    Returns:
        Self for method chaining.

    Raises:
        RuntimeError: If pages lack a parent PDF or parent lacks `apply_ocr`.
        (Propagates exceptions from PDF.apply_ocr)
    """
    if not self.pages:
        logger.warning("Cannot apply OCR to an empty PageCollection.")
        return self

    parent_pdf = self._resolve_parent_pdf()

    if not hasattr(parent_pdf, "apply_ocr") or not callable(parent_pdf.apply_ocr):
        raise RuntimeError("Parent PDF object does not have the required 'apply_ocr' method.")

    # Get the 0-based indices of the pages in this collection
    page_indices = self._get_page_indices()

    logger.info(f"Applying OCR via parent PDF to page indices: {page_indices} in collection.")

    # Delegate the batch call to the parent PDF object, passing direct args and apply_exclusions
    parent_pdf.apply_ocr(
        pages=page_indices,
        engine=engine,
        languages=languages,
        min_confidence=min_confidence,  # Pass the renamed parameter
        device=device,
        resolution=resolution,
        apply_exclusions=apply_exclusions,  # Pass down
        replace=replace,  # Pass the replace parameter
        options=options,
    )
    # The PDF method modifies the Page objects directly by adding elements.

    return self  # Return self for chaining

natural_pdf.PageCollection.deskew(resolution=300, detection_resolution=72, force_overwrite=False, **deskew_kwargs)

Creates a new, in-memory PDF object containing deskewed versions of the pages in this collection.

This method delegates the actual processing to the parent PDF object's deskew method.

Important: The returned PDF is image-based. Any existing text, OCR results, annotations, or other elements from the original pages will not be carried over.

Parameters:

Name	Type	Description	Default
resolution	int	DPI resolution for rendering the output deskewed pages.	300
detection_resolution	int	DPI resolution used for skew detection if angles are not already cached on the page objects.	72
force_overwrite	bool	If False (default), raises a ValueError if any target page already contains processed elements (text, OCR, regions) to prevent accidental data loss. Set to True to proceed anyway.	False
**deskew_kwargs		Additional keyword arguments passed to deskew.determine_skew during automatic detection (e.g., max_angle, num_peaks).	{}

Returns:

Type	Description
'PDF'	A new PDF object representing the deskewed document.

Raises:

Type	Description
ImportError	If 'deskew' or 'img2pdf' libraries are not installed (raised by PDF.deskew).
ValueError	If force_overwrite is False and target pages contain elements (raised by PDF.deskew), or if the collection is empty.
RuntimeError	If pages lack a parent PDF reference, or the parent PDF lacks the deskew method.

Source code in natural_pdf/core/page_collection.py

def deskew(
    self,
    resolution: int = 300,
    detection_resolution: int = 72,
    force_overwrite: bool = False,
    **deskew_kwargs,
) -> "PDF":  # Changed return type
    """
    Creates a new, in-memory PDF object containing deskewed versions of the pages
    in this collection.

    This method delegates the actual processing to the parent PDF object's
    `deskew` method.

    Important: The returned PDF is image-based. Any existing text, OCR results,
    annotations, or other elements from the original pages will *not* be carried over.

    Args:
        resolution: DPI resolution for rendering the output deskewed pages.
        detection_resolution: DPI resolution used for skew detection if angles are not
                              already cached on the page objects.
        force_overwrite: If False (default), raises a ValueError if any target page
                         already contains processed elements (text, OCR, regions) to
                         prevent accidental data loss. Set to True to proceed anyway.
        **deskew_kwargs: Additional keyword arguments passed to `deskew.determine_skew`
                         during automatic detection (e.g., `max_angle`, `num_peaks`).

    Returns:
        A new PDF object representing the deskewed document.

    Raises:
        ImportError: If 'deskew' or 'img2pdf' libraries are not installed (raised by PDF.deskew).
        ValueError: If `force_overwrite` is False and target pages contain elements (raised by PDF.deskew),
                    or if the collection is empty.
        RuntimeError: If pages lack a parent PDF reference, or the parent PDF lacks the `deskew` method.
    """
    if not self.pages:
        logger.warning("Cannot deskew an empty PageCollection.")
        raise ValueError("Cannot deskew an empty PageCollection.")

    parent_pdf = self._resolve_parent_pdf()

    if not parent_pdf or not hasattr(parent_pdf, "deskew") or not callable(parent_pdf.deskew):
        raise RuntimeError(
            "Parent PDF reference not found or parent PDF lacks the required 'deskew' method."
        )

    # Get the 0-based indices of the pages in this collection
    page_indices = self._get_page_indices()
    logger.info(
        f"PageCollection: Delegating deskew to parent PDF for page indices: {page_indices}"
    )

    # Delegate the call to the parent PDF object for the relevant pages
    # Pass all relevant arguments through (no output_path anymore)
    return parent_pdf.deskew(
        pages=page_indices,
        resolution=resolution,
        detection_resolution=detection_resolution,
        force_overwrite=force_overwrite,
        **deskew_kwargs,
    )

natural_pdf.PageCollection.detect_checkboxes(*args, **kwargs)

Detects checkboxes on each page in the collection.

This method iterates through each page, calls its detect_checkboxes method, and returns a single ElementCollection containing all detected checkbox regions from all pages.

Parameters:

Name	Type	Description	Default
*args		Positional arguments to pass to each page's detect_checkboxes method.	()
**kwargs		Keyword arguments to pass to each page's detect_checkboxes method. A 'show_progress' kwarg can be included to show a progress bar.	{}

Returns:

Type	Description
'ElementCollection[Region]'	An ElementCollection of all detected checkbox Region objects.

Source code in natural_pdf/core/page_collection.py

def detect_checkboxes(self, *args, **kwargs) -> "ElementCollection[Region]":
    """
    Detects checkboxes on each page in the collection.

    This method iterates through each page, calls its detect_checkboxes method,
    and returns a single ElementCollection containing all detected checkbox
    regions from all pages.

    Args:
        *args: Positional arguments to pass to each page's detect_checkboxes method.
        **kwargs: Keyword arguments to pass to each page's detect_checkboxes method.
                  A 'show_progress' kwarg can be included to show a progress bar.

    Returns:
        An ElementCollection of all detected checkbox Region objects.
    """
    all_checkboxes = []

    show_progress = kwargs.pop("show_progress", True)

    iterator = self.pages
    if show_progress:
        try:
            from tqdm.auto import tqdm

            iterator = tqdm(self.pages, desc="Detecting checkboxes")
        except ImportError:
            pass  # tqdm not installed

    for page in iterator:
        # Each page's detect_checkboxes method returns an ElementCollection
        checkbox_collection = page.detect_checkboxes(*args, **kwargs)
        if checkbox_collection:
            all_checkboxes.extend(checkbox_collection.elements)

    return ElementCollection(all_checkboxes)

natural_pdf.PageCollection.extract_text(separator='\n', apply_exclusions=True, **kwargs)

Extract text from all pages in the collection.

Parameters:

Name	Type	Description	Default
keep_blank_chars		Whether to keep blank characters (default: True)	required
apply_exclusions	bool	Whether to apply exclusion regions (default: True)	True
strip		Whether to strip whitespace from the extracted text.	required
**kwargs		Additional extraction parameters	{}

Returns:

Type	Description
str	Combined text from all pages

Source code in natural_pdf/core/page_collection.py

def extract_text(
    self,
    separator: str = "\n",
    apply_exclusions: bool = True,
    **kwargs,
) -> str:
    """
    Extract text from all pages in the collection.

    Args:
        keep_blank_chars: Whether to keep blank characters (default: True)
        apply_exclusions: Whether to apply exclusion regions (default: True)
        strip: Whether to strip whitespace from the extracted text.
        **kwargs: Additional extraction parameters

    Returns:
        Combined text from all pages
    """
    keep_blank_chars = kwargs.pop("keep_blank_chars", True)
    if keep_blank_chars is None:
        keep_blank_chars = True
    strip = kwargs.pop("strip", None)
    explicit_strip_final = kwargs.pop("strip_final", None)
    explicit_strip_empty = kwargs.pop("strip_empty", None)

    texts: List[str] = []

    for page in self.pages:
        text = page.extract_text(
            preserve_whitespace=keep_blank_chars,
            use_exclusions=apply_exclusions,
            strip_final=(
                explicit_strip_final
                if explicit_strip_final is not None
                else (strip if strip is not None else False)
            ),
            strip_empty=explicit_strip_empty if explicit_strip_empty is not None else False,
            **kwargs,
        )
        texts.append(text)

    return separator.join(texts)

natural_pdf.PageCollection.get_sections(start_elements=None, end_elements=None, new_section_on_page_break=False, include_boundaries='both', orientation='vertical')

Extract logical sections across this collection of pages.

This delegates to :class:natural_pdf.flows.flow.Flow, which already implements the heavy lifting for cross-segment section extraction and returns either :class:Region or :class:FlowRegion objects as appropriate. The arrangement is chosen based on the requested orientation so that horizontal sections continue to work for rotated content.

Source code in natural_pdf/core/page_collection.py

def get_sections(
    self,
    start_elements: BoundarySource = None,
    end_elements: BoundarySource = None,
    new_section_on_page_break: bool = False,
    include_boundaries: str = "both",
    orientation: str = "vertical",
) -> "ElementCollection":
    """Extract logical sections across this collection of pages.

    This delegates to :class:`natural_pdf.flows.flow.Flow`, which already
    implements the heavy lifting for cross-segment section extraction and
    returns either :class:`Region` or :class:`FlowRegion` objects as
    appropriate.  The arrangement is chosen based on the requested
    orientation so that horizontal sections continue to work for rotated
    content.
    """

    from natural_pdf.elements.element_collection import ElementCollection
    from natural_pdf.flows.flow import Flow

    if len(self) == 0:
        return ElementCollection([])

    arrangement = "vertical" if orientation == "vertical" else "horizontal"

    flow = Flow(self, arrangement=arrangement)
    sections = flow.get_sections(
        start_elements=start_elements,
        end_elements=end_elements,
        new_section_on_page_break=new_section_on_page_break,
        include_boundaries=include_boundaries,
        orientation=orientation,
    )
    section_list = sections.elements if hasattr(sections, "elements") else list(sections)
    cleaned = sanitize_sections(section_list, orientation=orientation)
    if len(cleaned) == len(section_list):
        return sections
    return ElementCollection(cleaned)

natural_pdf.PageCollection.groupby(by, *, show_progress=True)

Group pages by selector text or callable result.

Parameters:

Name	Type	Description	Default
by	Union[str, Callable]	CSS selector string or callable function	required
show_progress	bool	Whether to show progress bar during computation (default: True)	True

Returns:

Type	Description
'PageGroupBy'	PageGroupBy object supporting iteration and dict-like access

Examples:

Group by header text

for title, pages in pdf.pages.groupby('text[size=16]'): print(f"Section: {title}")

Group by callable

for city, pages in pdf.pages.groupby(lambda p: p.find('text:contains("CITY")').extract_text()): process_city_pages(pages)

Quick exploration with indexing

grouped = pdf.pages.groupby('text[size=16]') grouped.info() # Show all groups first_section = grouped[0] # First group last_section = grouped[-1] # Last group

Dict-like access by name

madison_pages = grouped.get('CITY OF MADISON') madison_pages = grouped['CITY OF MADISON'] # Alternative

Disable progress bar for small collections

grouped = pdf.pages.groupby('text[size=16]', show_progress=False)

Source code in natural_pdf/core/page_collection.py

def groupby(self, by: Union[str, Callable], *, show_progress: bool = True) -> "PageGroupBy":
    """
    Group pages by selector text or callable result.

    Args:
        by: CSS selector string or callable function
        show_progress: Whether to show progress bar during computation (default: True)

    Returns:
        PageGroupBy object supporting iteration and dict-like access

    Examples:
        # Group by header text
        for title, pages in pdf.pages.groupby('text[size=16]'):
            print(f"Section: {title}")

        # Group by callable
        for city, pages in pdf.pages.groupby(lambda p: p.find('text:contains("CITY")').extract_text()):
            process_city_pages(pages)

        # Quick exploration with indexing
        grouped = pdf.pages.groupby('text[size=16]')
        grouped.info()                    # Show all groups
        first_section = grouped[0]        # First group
        last_section = grouped[-1]       # Last group

        # Dict-like access by name
        madison_pages = grouped.get('CITY OF MADISON')
        madison_pages = grouped['CITY OF MADISON']  # Alternative

        # Disable progress bar for small collections
        grouped = pdf.pages.groupby('text[size=16]', show_progress=False)
    """
    from natural_pdf.core.page_groupby import PageGroupBy

    return PageGroupBy(self, by, show_progress=show_progress)

natural_pdf.PageCollection.highlights(show=False)

Create a highlight context for accumulating highlights.

This allows for clean syntax to show multiple highlight groups:

Example

with pages.highlights() as h: h.add(pages.find_all('table'), label='tables', color='blue') h.add(pages.find_all('text:bold'), label='bold text', color='red') h.show()

Or with automatic display

with pages.highlights(show=True) as h: h.add(pages.find_all('table'), label='tables') h.add(pages.find_all('text:bold'), label='bold') # Automatically shows when exiting the context

Parameters:

Name	Type	Description	Default
show	bool	If True, automatically show highlights when exiting context	False

Returns:

Type	Description
'HighlightContext'	HighlightContext for accumulating highlights

Source code in natural_pdf/core/page_collection.py

def highlights(self, show: bool = False) -> "HighlightContext":
    """
    Create a highlight context for accumulating highlights.

    This allows for clean syntax to show multiple highlight groups:

    Example:
        with pages.highlights() as h:
            h.add(pages.find_all('table'), label='tables', color='blue')
            h.add(pages.find_all('text:bold'), label='bold text', color='red')
            h.show()

    Or with automatic display:
        with pages.highlights(show=True) as h:
            h.add(pages.find_all('table'), label='tables')
            h.add(pages.find_all('text:bold'), label='bold')
            # Automatically shows when exiting the context

    Args:
        show: If True, automatically show highlights when exiting context

    Returns:
        HighlightContext for accumulating highlights
    """
    from natural_pdf.core.highlighting_service import HighlightContext

    return HighlightContext(self, show_on_exit=show)

natural_pdf.PageCollection.save_pdf(output_path, ocr=False, original=False, dpi=300)

Saves the pages in this collection to a new PDF file.

Choose one saving mode: - ocr=True: Creates a new, image-based PDF using OCR results. This makes the text generated during the natural-pdf session searchable, but loses original vector content. Requires 'ocr-export' extras. - original=True: Extracts the original pages from the source PDF, preserving all vector content, fonts, and annotations. OCR results from the natural-pdf session are NOT included. Requires 'ocr-export' extras.

Parameters:

Name	Type	Description	Default
output_path	Union[str, Path]	Path to save the new PDF file.	required
ocr	bool	If True, save as a searchable, image-based PDF using OCR data.	False
original	bool	If True, save the original, vector-based pages.	False
dpi	int	Resolution (dots per inch) used only when ocr=True for rendering page images and aligning the text layer.	300

Raises:

Type	Description
ValueError	If the collection is empty, if neither or both 'ocr' and 'original' are True, or if 'original=True' and pages originate from different PDFs.
ImportError	If required libraries ('pikepdf', 'Pillow') are not installed for the chosen mode.
RuntimeError	If an unexpected error occurs during saving.

Source code in natural_pdf/core/page_collection.py

def save_pdf(
    self,
    output_path: Union[str, Path],
    ocr: bool = False,
    original: bool = False,
    dpi: int = 300,
):
    """
    Saves the pages in this collection to a new PDF file.

    Choose one saving mode:
    - `ocr=True`: Creates a new, image-based PDF using OCR results. This
      makes the text generated during the natural-pdf session searchable,
      but loses original vector content. Requires 'ocr-export' extras.
    - `original=True`: Extracts the original pages from the source PDF,
      preserving all vector content, fonts, and annotations. OCR results
      from the natural-pdf session are NOT included. Requires 'ocr-export' extras.

    Args:
        output_path: Path to save the new PDF file.
        ocr: If True, save as a searchable, image-based PDF using OCR data.
        original: If True, save the original, vector-based pages.
        dpi: Resolution (dots per inch) used only when ocr=True for
             rendering page images and aligning the text layer.

    Raises:
        ValueError: If the collection is empty, if neither or both 'ocr'
                    and 'original' are True, or if 'original=True' and
                    pages originate from different PDFs.
        ImportError: If required libraries ('pikepdf', 'Pillow')
                     are not installed for the chosen mode.
        RuntimeError: If an unexpected error occurs during saving.
    """
    if not self.pages:
        raise ValueError("Cannot save an empty PageCollection.")

    if not (ocr ^ original):  # XOR: exactly one must be true
        raise ValueError("Exactly one of 'ocr' or 'original' must be True.")

    output_path_obj = Path(output_path)
    output_path_str = str(output_path_obj)

    if ocr:
        if create_searchable_pdf is None:
            raise ImportError(
                "Saving with ocr=True requires 'pikepdf' and 'Pillow'. "
                'Install with: pip install \\"natural-pdf[ocr-export]\\"'  # Escaped quotes
            )

        # Check for non-OCR vector elements (provide a warning)
        has_vector_elements = False
        for page in self.pages:
            # Simplified check for common vector types or non-OCR chars/words
            rects = getattr(page, "rects", [])
            lines = getattr(page, "lines", [])
            curves = getattr(page, "curves", [])
            chars = getattr(page, "chars", [])
            words = getattr(page, "words", [])

            if (
                rects
                or lines
                or curves
                or any(getattr(el, "source", None) != "ocr" for el in chars)
                or any(getattr(el, "source", None) != "ocr" for el in words)
            ):
                has_vector_elements = True
                break
        if has_vector_elements:
            logger.warning(
                "Warning: Saving with ocr=True creates an image-based PDF. "
                "Original vector elements (rects, lines, non-OCR text/chars) "
                "on selected pages will not be preserved in the output file."
            )

        logger.info(f"Saving searchable PDF (OCR text layer) to: {output_path_str}")
        try:
            # Delegate to the searchable PDF exporter function
            # Pass `self` (the PageCollection instance) as the source
            create_searchable_pdf(self, output_path_str, dpi=dpi)
            # Success log is now inside create_searchable_pdf if needed, or keep here
            # logger.info(f"Successfully saved searchable PDF to: {output_path_str}")
        except Exception as e:
            logger.error(f"Failed to create searchable PDF: {e}", exc_info=True)
            # Re-raise as RuntimeError for consistency, potentially handled in exporter too
            raise RuntimeError(f"Failed to create searchable PDF: {e}") from e

    elif original:
        # ---> MODIFIED: Call the new exporter
        if create_original_pdf is None:
            raise ImportError(
                "Saving with original=True requires 'pikepdf'. "
                'Install with: pip install \\"natural-pdf[ocr-export]\\"'  # Escaped quotes
            )

        # Check for OCR elements (provide a warning) - keep this check here
        has_ocr_elements = False
        for page in self.pages:
            # Use find_all which returns a collection; check if it's non-empty
            if hasattr(page, "find_all"):
                ocr_text_elements = page.find_all("text[source=ocr]")
                if ocr_text_elements:  # Check truthiness of collection
                    has_ocr_elements = True
                    break
            elif hasattr(page, "words"):  # Fallback check if find_all isn't present?
                if any(getattr(el, "source", None) == "ocr" for el in page.words):
                    has_ocr_elements = True
                    break

        if has_ocr_elements:
            logger.warning(
                "Warning: Saving with original=True preserves original page content. "
                "OCR text generated in this session will not be included in the saved file."
            )

        logger.info(f"Saving original pages PDF to: {output_path_str}")
        try:
            # Delegate to the original PDF exporter function
            # Pass `self` (the PageCollection instance) as the source
            create_original_pdf(self, output_path_str)
            # Success log is now inside create_original_pdf
            # logger.info(f"Successfully saved original pages PDF to: {output_path_str}")
        except Exception as e:
            # Error logging is handled within create_original_pdf
            # Re-raise the exception caught from the exporter
            raise e  # Keep the original exception type (ValueError, RuntimeError, etc.)

natural_pdf.PageCollection.split(divider, *, include_boundaries='start', orientation='vertical', new_section_on_page_break=False)

Divide this page collection into sections based on the provided divider elements.

Parameters:

Name	Type	Description	Default
divider	BoundarySource	Elements or selector string that mark section boundaries	required
include_boundaries	str	How to include boundary elements (default: 'start').	'start'
orientation	str	'vertical' or 'horizontal' (default: 'vertical').	'vertical'
new_section_on_page_break	bool	Whether to split at page boundaries (default: False).	False

Returns:

Type	Description
'ElementCollection[Region]'	ElementCollection of Region objects representing the sections

Example

Split a PDF by chapter titles

chapters = pdf.pages.split("text[size>20]:contains('CHAPTER')")

Split by page breaks

page_sections = pdf.pages.split(None, new_section_on_page_break=True)

Split multi-page document by section headers

sections = pdf.pages[10:20].split("text:bold:contains('Section')")

Source code in natural_pdf/core/page_collection.py

def split(
    self,
    divider: BoundarySource,
    *,
    include_boundaries: str = "start",
    orientation: str = "vertical",
    new_section_on_page_break: bool = False,
) -> "ElementCollection[Region]":
    """
    Divide this page collection into sections based on the provided divider elements.

    Args:
        divider: Elements or selector string that mark section boundaries
        include_boundaries: How to include boundary elements (default: 'start').
        orientation: 'vertical' or 'horizontal' (default: 'vertical').
        new_section_on_page_break: Whether to split at page boundaries (default: False).

    Returns:
        ElementCollection of Region objects representing the sections

    Example:
        # Split a PDF by chapter titles
        chapters = pdf.pages.split("text[size>20]:contains('CHAPTER')")

        # Split by page breaks
        page_sections = pdf.pages.split(None, new_section_on_page_break=True)

        # Split multi-page document by section headers
        sections = pdf.pages[10:20].split("text:bold:contains('Section')")
    """
    sections = self.get_sections(
        start_elements=divider,
        include_boundaries=include_boundaries,
        orientation=orientation,
        new_section_on_page_break=new_section_on_page_break,
    )

    # Add initial section if there's content before the first divider
    if sections and divider is not None:
        # Get all elements across all pages
        all_elements = []
        for page in self.pages:
            all_elements.extend(page.get_elements())

        if all_elements:
            # Find first divider
            if isinstance(divider, str):
                # Search for first matching element
                first_divider = None
                for page in self.pages:
                    match = page.find(divider)
                    if match:
                        first_divider = match
                        break
            else:
                # divider is already elements
                first_divider = None
                if isinstance(divider, Iterable):
                    first_candidate = next(iter(divider), None)
                else:
                    first_candidate = divider
                if isinstance(first_candidate, SupportsGeometry):
                    first_divider = first_candidate

            if first_divider and all_elements[0] != first_divider:
                # There's content before the first divider
                # Get section from start to first divider
                initial_sections = self.get_sections(
                    start_elements=None,
                    end_elements=[first_divider],
                    include_boundaries="none",
                    orientation=orientation,
                )
                if initial_sections:
                    sections = ElementCollection([initial_sections[0]] + list(sections))

    return sections

natural_pdf.PageCollection.to_flow(arrangement='vertical', alignment='start', segment_gap=0.0)

Convert this PageCollection to a Flow for cross-page operations.

This enables treating multiple pages as a continuous logical document structure, useful for multi-page tables, articles spanning columns, or any content requiring reading order across page boundaries.

Parameters:

Name	Type	Description	Default
arrangement	Literal['vertical', 'horizontal']	Primary flow direction ('vertical' or 'horizontal'). 'vertical' stacks pages top-to-bottom (most common). 'horizontal' arranges pages left-to-right.	'vertical'
alignment	Literal['start', 'center', 'end', 'top', 'left', 'bottom', 'right']	Cross-axis alignment for pages of different sizes: For vertical: 'left'/'start', 'center', 'right'/'end' For horizontal: 'top'/'start', 'center', 'bottom'/'end'	'start'
segment_gap	float	Virtual gap between pages in PDF points (default: 0.0).	0.0

Returns:

Type	Description
'Flow'	Flow object that can perform operations across all pages in sequence.

Example

Multi-page table extraction:

pdf = npdf.PDF("multi_page_report.pdf")

# Create flow for pages 2-4 containing a table
table_flow = pdf.pages[1:4].to_flow()

# Extract table as if it were continuous
table_data = table_flow.extract_table()
df = table_data.df

Cross-page element search:

# Find all headers across multiple pages
headers = pdf.pages[5:10].to_flow().find_all('text[size>12]:bold')

# Analyze layout across pages
regions = pdf.pages.to_flow().analyze_layout(engine='yolo')

Source code in natural_pdf/core/page_collection.py

def to_flow(
    self,
    arrangement: Literal["vertical", "horizontal"] = "vertical",
    alignment: Literal["start", "center", "end", "top", "left", "bottom", "right"] = "start",
    segment_gap: float = 0.0,
) -> "Flow":
    """
    Convert this PageCollection to a Flow for cross-page operations.

    This enables treating multiple pages as a continuous logical document
    structure, useful for multi-page tables, articles spanning columns,
    or any content requiring reading order across page boundaries.

    Args:
        arrangement: Primary flow direction ('vertical' or 'horizontal').
                    'vertical' stacks pages top-to-bottom (most common).
                    'horizontal' arranges pages left-to-right.
        alignment: Cross-axis alignment for pages of different sizes:
                  For vertical: 'left'/'start', 'center', 'right'/'end'
                  For horizontal: 'top'/'start', 'center', 'bottom'/'end'
        segment_gap: Virtual gap between pages in PDF points (default: 0.0).

    Returns:
        Flow object that can perform operations across all pages in sequence.

    Example:
        Multi-page table extraction:
        ```python
        pdf = npdf.PDF("multi_page_report.pdf")

        # Create flow for pages 2-4 containing a table
        table_flow = pdf.pages[1:4].to_flow()

        # Extract table as if it were continuous
        table_data = table_flow.extract_table()
        df = table_data.df
        ```

        Cross-page element search:
        ```python
        # Find all headers across multiple pages
        headers = pdf.pages[5:10].to_flow().find_all('text[size>12]:bold')

        # Analyze layout across pages
        regions = pdf.pages.to_flow().analyze_layout(engine='yolo')
        ```
    """
    from natural_pdf.flows.flow import Flow

    return Flow(
        segments=self,  # Flow constructor now handles PageCollection
        arrangement=arrangement,
        alignment=alignment,
        segment_gap=segment_gap,
    )

natural_pdf.Region

Bases: TextMixin, TabularRegionMixin, AnalysisHostMixin, Visualizable

Represents a rectangular region on a page.

Regions are fundamental building blocks in natural-pdf that define rectangular areas of a page for analysis, extraction, and navigation. They can be created manually or automatically through spatial navigation methods like .below(), .above(), .left(), and .right() from elements or other regions.

Regions integrate multiple analysis capabilities through mixins and provide: - Element filtering and collection within the region boundary - OCR processing for the region area - Table detection and extraction - AI-powered classification and structured data extraction - Visual rendering and debugging capabilities - Text extraction with spatial awareness

The Region class supports both rectangular and polygonal boundaries, making it suitable for complex document layouts and irregular shapes detected by layout analysis algorithms.

Attributes:

Name	Type	Description
page	'Page'	Reference to the parent Page object.
bbox	Tuple[float, float, float, float]	Bounding box tuple (x0, top, x1, bottom) in PDF coordinates.
x0	float	Left x-coordinate.
top	float	Top y-coordinate (minimum y).
x1	float	Right x-coordinate.
bottom	float	Bottom y-coordinate (maximum y).
width	float	Region width (x1 - x0).
height	float	Region height (bottom - top).
polygon	List[Tuple[float, float]]	List of coordinate points for non-rectangular regions.
label		Optional descriptive label for the region.
metadata	Dict[str, Any]	Dictionary for storing analysis results and custom data.

Example

Creating regions:

pdf = npdf.PDF("document.pdf")
page = pdf.pages[0]

# Manual region creation
header_region = page.region(0, 0, page.width, 100)

# Spatial navigation from elements
summary_text = page.find('text:contains("Summary")')
content_region = summary_text.below(until='text[size>12]:bold')

# Extract content from region
tables = content_region.extract_table()
text = content_region.get_text()

Advanced usage:

# OCR processing
region.apply_ocr(engine='easyocr', resolution=300)

# AI-powered extraction
data = region.extract_structured_data(MySchema)

# Visual debugging
region.show(highlights=['tables', 'text'])

Source code in natural_pdf/elements/region.py

class Region(TextMixin, TabularRegionMixin, AnalysisHostMixin, Visualizable):
    """Represents a rectangular region on a page.

    Regions are fundamental building blocks in natural-pdf that define rectangular
    areas of a page for analysis, extraction, and navigation. They can be created
    manually or automatically through spatial navigation methods like .below(), .above(),
    .left(), and .right() from elements or other regions.

    Regions integrate multiple analysis capabilities through mixins and provide:
    - Element filtering and collection within the region boundary
    - OCR processing for the region area
    - Table detection and extraction
    - AI-powered classification and structured data extraction
    - Visual rendering and debugging capabilities
    - Text extraction with spatial awareness

    The Region class supports both rectangular and polygonal boundaries, making it
    suitable for complex document layouts and irregular shapes detected by layout
    analysis algorithms.

    Attributes:
        page: Reference to the parent Page object.
        bbox: Bounding box tuple (x0, top, x1, bottom) in PDF coordinates.
        x0: Left x-coordinate.
        top: Top y-coordinate (minimum y).
        x1: Right x-coordinate.
        bottom: Bottom y-coordinate (maximum y).
        width: Region width (x1 - x0).
        height: Region height (bottom - top).
        polygon: List of coordinate points for non-rectangular regions.
        label: Optional descriptive label for the region.
        metadata: Dictionary for storing analysis results and custom data.

    Example:
        Creating regions:
        ```python
        pdf = npdf.PDF("document.pdf")
        page = pdf.pages[0]

        # Manual region creation
        header_region = page.region(0, 0, page.width, 100)

        # Spatial navigation from elements
        summary_text = page.find('text:contains("Summary")')
        content_region = summary_text.below(until='text[size>12]:bold')

        # Extract content from region
        tables = content_region.extract_table()
        text = content_region.get_text()
        ```

        Advanced usage:
        ```python
        # OCR processing
        region.apply_ocr(engine='easyocr', resolution=300)

        # AI-powered extraction
        data = region.extract_structured_data(MySchema)

        # Visual debugging
        region.show(highlights=['tables', 'text'])
        ```
    """

    def __init__(
        self,
        page: "Page",
        bbox: Tuple[float, float, float, float],
        polygon: Optional[List[Tuple[float, float]]] = None,
        parent: Optional["Region"] = None,
        label: Optional[str] = None,
    ):
        """Initialize a region.

        Creates a Region object that represents a rectangular or polygonal area on a page.
        Regions are used for spatial navigation, content extraction, and analysis operations.

        Args:
            page: Parent Page object that contains this region and provides access
                to document elements and analysis capabilities.
            bbox: Bounding box coordinates as (x0, top, x1, bottom) tuple in PDF
                coordinate system (points, with origin at bottom-left).
            polygon: Optional list of coordinate points [(x1,y1), (x2,y2), ...] for
                non-rectangular regions. If provided, the region will use polygon-based
                intersection calculations instead of simple rectangle overlap.
            parent: Optional parent region for hierarchical document structure.
                Useful for maintaining tree-like relationships between regions.
            label: Optional descriptive label for the region, useful for debugging
                and identification in complex workflows.

        Example:
            ```python
            pdf = npdf.PDF("document.pdf")
            page = pdf.pages[0]

            # Rectangular region
            header = Region(page, (0, 0, page.width, 100), label="header")

            # Polygonal region (from layout detection)
            table_polygon = [(50, 100), (300, 100), (300, 400), (50, 400)]
            table_region = Region(page, (50, 100, 300, 400),
                                polygon=table_polygon, label="table")
            ```

        Note:
            Regions are typically created through page methods like page.region() or
            spatial navigation methods like element.below(). Direct instantiation is
            used mainly for advanced workflows or layout analysis integration.
        """
        self._page: "Page" = page
        self._bbox: Tuple[float, float, float, float] = bbox
        self._polygon: Optional[List[Tuple[float, float]]] = polygon

        self.metadata: Dict[str, Any] = {}
        # Analysis results live under self.metadata['analysis'] via property

        # Standard attributes for all elements
        self.object_type: str = "region"  # For selector compatibility
        self.start_element: Optional["Element"] = None
        self.end_element: Optional["Element"] = None
        self._boundary_exclusions: Optional[str] = None

        # Layout detection attributes
        self.region_type: Optional[str] = None
        self.normalized_type: Optional[str] = None
        self.confidence: Optional[float] = None
        self.model: Optional[str] = None
        self.is_checked: Optional[bool] = None
        self.checkbox_state: Optional[str] = None
        self.original_class: Optional[str] = None

        # Region management attributes
        self.name: Optional[str] = None
        self.label = label
        self.source: Optional[str] = None  # Will be set by creation methods

        # Hierarchy support for nested document structure
        self.parent_region: Optional["Region"] = parent
        self.child_regions: List["Region"] = []
        self.text_content: Optional[str] = None  # Direct text content (e.g., from Docling)
        self.associated_text_elements: List[TextElement] = (
            []
        )  # Native text elements that overlap with this region
        self.source_element: Optional[Union["Element", "Region"]] = None
        self.includes_source: bool = False
        self.boundary_element: Optional["Element"] = None

        if self.parent_region is not None:
            self.parent_region.child_regions.append(self)

        self._cached_text: Optional[str] = None
        self._cached_elements: Optional[ElementCollection] = None
        self._cached_bbox: Optional[Tuple[float, float, float, float]] = None
        self._exclusions: List[ExclusionSpec] = []

    def _exclusion_element_manager(self):
        return self.page._element_mgr

    def _element_to_region(self, element: Any, label: Optional[str] = None) -> Optional["Region"]:
        bbox = extract_bbox(element)
        if not bbox:
            return None
        # Clamp to this region's bounds
        x0 = max(self.x0, bbox[0])
        top = max(self.top, bbox[1])
        x1 = min(self.x1, bbox[2])
        bottom = min(self.bottom, bbox[3])
        if x0 >= x1 or top >= bottom:
            return None
        return Region(self.page, (x0, top, x1, bottom), label=label)

    def _invalidate_exclusion_cache(self) -> None:
        self._cached_text = None
        self._cached_elements = None

    def _ocr_element_manager(self):
        return self.page._element_mgr

    def _qa_context_page_number(self) -> int:
        return self.page.number

    def _qa_source_elements(self) -> "ElementCollection":
        from natural_pdf.elements.element_collection import ElementCollection

        return ElementCollection([])

    def _qa_target_region(self) -> "Region":
        return self

    def _qa_normalize_result(self, result: Any) -> Union[Dict[str, Any], List[Dict[str, Any]]]:
        return self._normalize_qa_output(result)

    def _qa_blank_result(self, question: QuestionInput) -> Union[QAResult, List[QAResult]]:
        return super()._qa_blank_result(question)

    def _evaluate_local_exclusions(
        self, include_callable: bool = True, debug: bool = False
    ) -> List["Region"]:
        if not getattr(self, "_exclusions", None):
            return []
        return self._evaluate_exclusion_entries(self._exclusions, include_callable, debug)

    def _get_exclusion_regions(
        self, include_callable: bool = True, debug: bool = False
    ) -> List["Region"]:
        local = self._evaluate_local_exclusions(include_callable=include_callable, debug=debug)
        page_regions = self.page._get_exclusion_regions(
            include_callable=include_callable, debug=debug
        )
        return local + page_regions

    def remove_ocr_elements(self) -> int:
        if not hasattr(self.page, "get_elements_by_type"):
            return 0

        removed_count = 0

        def _safe_remove(elem, element_type: Optional[str] = None):
            nonlocal removed_count
            if self.page.remove_element(elem, element_type=element_type):
                removed_count += 1

        for word in list(self.page.get_elements_by_type("words")):
            if getattr(word, "source", None) == "ocr" and self.intersects(word):
                _safe_remove(word, element_type="words")

        for char in list(self.page.get_elements_by_type("chars")):
            char_source = (
                char.get("source") if isinstance(char, dict) else getattr(char, "source", None)
            )
            if char_source != "ocr":
                continue
            if isinstance(char, dict):
                bbox_tuple = (
                    float(char.get("x0", 0.0)),
                    float(char.get("top", 0.0)),
                    float(char.get("x1", 0.0)),
                    float(char.get("bottom", 0.0)),
                )
            else:
                bbox_tuple = getattr(char, "bbox", None)
            if not bbox_tuple:
                continue
            if get_bbox_overlap(self.bbox, bbox_tuple) is not None:
                _safe_remove(char, element_type="chars")

        return removed_count

    def to_region(self) -> "Region":
        """Regions already satisfy the section surface; return self."""
        return self

    def _get_render_specs(
        self,
        mode: Literal["show", "render"] = "show",
        color: Optional[Union[str, Tuple[int, int, int]]] = None,
        highlights: Optional[Union[List[Dict[str, Any]], bool]] = None,
        crop: Union[bool, int, "Region", Literal["wide"]] = True,  # Default to True for regions
        crop_bbox: Optional[Tuple[float, float, float, float]] = None,
        **kwargs,
    ) -> List[RenderSpec]:
        """Get render specifications for this region.

        Args:
            mode: Rendering mode - 'show' includes highlights, 'render' is clean
            color: Color for highlighting this region in show mode
            highlights: Additional highlight groups to show, or False to disable all highlights
            crop: Cropping mode:
                - False: No cropping
                - True: Crop to region bounds (default for regions)
                - int: Padding in pixels around region
                - 'wide': Full page width, cropped vertically to region
                - Region: Crop to the bounds of another region
            crop_bbox: Explicit crop bounds (overrides region bounds)
            **kwargs: Additional parameters

        Returns:
            List containing a single RenderSpec for this region's page
        """

        spec = RenderSpec(page=self.page)

        spec.crop_bbox = resolve_crop_bbox(
            width=self.page.width,
            height=self.page.height,
            crop=crop,
            crop_bbox=crop_bbox,
            content_bbox_fn=lambda: self.bbox,
        )

        # Add highlights in show mode (unless explicitly disabled with highlights=False)
        if mode == "show" and highlights is not False:
            # Only highlight this region if:
            # 1. We're not cropping, OR
            # 2. We're cropping but color was explicitly specified, OR
            # 3. We're cropping to another region (not tight crop)
            if not crop or color is not None or (crop and not isinstance(crop, bool)):
                spec.add_highlight(
                    bbox=self.bbox,
                    polygon=self.polygon if self.has_polygon else None,
                    color=color or "blue",
                    label=self.label or self.name or "Region",
                )

            # Add additional highlight groups if provided (and highlights is a list)
            if highlights and isinstance(highlights, list):
                for group in highlights:
                    elements = group.get("elements", [])
                    group_color = group.get("color", color)
                    group_label = group.get("label")

                    for elem in elements:
                        spec.add_highlight(element=elem, color=group_color, label=group_label)

        return [spec]

    def create_region(
        self,
        left: float,
        top: float,
        right: float,
        bottom: float,
        *,
        relative: bool = True,
        label: Optional[str] = None,
    ) -> "Region":
        """Create a child region anchored to this region.

        Args:
            left: Left coordinate. Interpreted relative to this region when ``relative`` is True.
            top: Top coordinate.
            right: Right coordinate.
            bottom: Bottom coordinate.
            relative: When True (default), coordinates are treated as offsets from this
                region's bounds. Set to False to provide absolute page coordinates.
            label: Optional label to assign to the new region.

        Returns:
            The newly created child region.
        """

        page = getattr(self, "page", None)
        if page is None:
            raise ValueError("Cannot create a sub-region without an associated page")

        if relative:
            abs_left = self.x0 + left
            abs_top = self.top + top
            abs_right = self.x0 + right
            abs_bottom = self.top + bottom
        else:
            abs_left = left
            abs_top = top
            abs_right = right
            abs_bottom = bottom

        child_region = page.region(left=abs_left, top=abs_top, right=abs_right, bottom=abs_bottom)
        child_region.parent_region = self
        self.child_regions.append(child_region)
        if label is not None:
            child_region.label = label
        return child_region

    def _direction(
        self,
        direction: str,
        size: Optional[float] = None,
        cross_size: str = "full",
        include_source: bool = False,
        until: Optional[str] = None,
        include_endpoint: bool = True,
        offset: Optional[float] = None,
        apply_exclusions: bool = True,
        multipage: Optional[bool] = None,
        within: Optional["Region"] = None,
        anchor: str = "start",
        **kwargs,
    ) -> Union["Region", "FlowRegion"]:
        """
        Region-specific wrapper around :py:meth:`DirectionalMixin._direction`.

        It performs any pre-processing required by *Region* (none currently),
        delegates the core geometry work to the mix-in implementation via
        ``super()``, then attaches region-level metadata before returning the
        new :class:`Region` instance.
        """

        effective_offset = offset
        if effective_offset is None:
            effective_offset = _layout_options().directional_offset

        # Delegate to the shared implementation on DirectionalMixin
        result = super()._direction(
            direction=direction,
            size=size,
            cross_size=cross_size,
            include_source=include_source,
            until=until,
            include_endpoint=include_endpoint,
            offset=effective_offset,
            apply_exclusions=apply_exclusions,
            multipage=multipage,
            within=within,
            anchor=anchor,
            **kwargs,
        )

        if isinstance(result, Region):
            # Post-process: make sure callers can trace lineage and flags
            result.source_element = self
            result.includes_source = include_source

        return result

    def above(
        self,
        height: Optional[float] = None,
        width: str = "full",
        include_source: bool = False,
        until: Optional[str] = None,
        include_endpoint: bool = True,
        offset: Optional[float] = None,
        apply_exclusions: bool = True,
        multipage: Optional[bool] = None,
        within: Optional["Region"] = None,
        anchor: str = "start",
        **kwargs,
    ) -> Union["Region", "FlowRegion"]:
        """
        Select region above this region.

        Args:
            height: Height of the region above, in points
            width: Width mode - "full" for full page width or "element" for element width
            include_source: Whether to include this region in the result (default: False)
            until: Optional selector string to specify an upper boundary element
            include_endpoint: Whether to include the boundary element in the region (default: True)
            offset: Pixel offset when excluding source/endpoint (default: None, uses natural_pdf.options.layout.directional_offset)
            multipage: Override global multipage behaviour; defaults to None meaning use global option.
            **kwargs: Additional parameters

        Returns:
            Region object representing the area above
        """
        return self._direction(
            direction="above",
            size=height,
            cross_size=width,
            include_source=include_source,
            until=until,
            include_endpoint=include_endpoint,
            offset=offset,
            apply_exclusions=apply_exclusions,
            multipage=multipage,
            within=within,
            anchor=anchor,
            **kwargs,
        )

    def below(
        self,
        height: Optional[float] = None,
        width: str = "full",
        include_source: bool = False,
        until: Optional[str] = None,
        include_endpoint: bool = True,
        offset: Optional[float] = None,
        apply_exclusions: bool = True,
        multipage: Optional[bool] = None,
        within: Optional["Region"] = None,
        anchor: str = "start",
        **kwargs,
    ) -> Union["Region", "FlowRegion"]:
        """
        Select region below this region.

        Args:
            height: Height of the region below, in points
            width: Width mode - "full" for full page width or "element" for element width
            include_source: Whether to include this region in the result (default: False)
            until: Optional selector string to specify a lower boundary element
            include_endpoint: Whether to include the boundary element in the region (default: True)
            offset: Pixel offset when excluding source/endpoint (default: None, uses natural_pdf.options.layout.directional_offset)
            multipage: Override global multipage behaviour; defaults to None meaning use global option.
            **kwargs: Additional parameters

        Returns:
            Region object representing the area below
        """
        return self._direction(
            direction="below",
            size=height,
            cross_size=width,
            include_source=include_source,
            until=until,
            include_endpoint=include_endpoint,
            offset=offset,
            apply_exclusions=apply_exclusions,
            multipage=multipage,
            within=within,
            anchor=anchor,
            **kwargs,
        )

    def left(
        self,
        width: Optional[float] = None,
        height: str = "element",
        include_source: bool = False,
        until: Optional[str] = None,
        include_endpoint: bool = True,
        offset: Optional[float] = None,
        apply_exclusions: bool = True,
        multipage: Optional[bool] = None,
        within: Optional["Region"] = None,
        anchor: str = "start",
        **kwargs,
    ) -> Union["Region", "FlowRegion"]:
        """
        Select region to the left of this region.

        Args:
            width: Width of the region to the left, in points
            height: Height mode - "full" for full page height or "element" for element height
            include_source: Whether to include this region in the result (default: False)
            until: Optional selector string to specify a left boundary element
            include_endpoint: Whether to include the boundary element in the region (default: True)
            offset: Pixel offset when excluding source/endpoint (default: None, uses natural_pdf.options.layout.directional_offset)
            multipage: Override global multipage behaviour; defaults to None meaning use global option.
            **kwargs: Additional parameters

        Returns:
            Region object representing the area to the left
        """
        return self._direction(
            direction="left",
            size=width,
            cross_size=height,
            include_source=include_source,
            until=until,
            include_endpoint=include_endpoint,
            offset=offset,
            apply_exclusions=apply_exclusions,
            multipage=multipage,
            within=within,
            anchor=anchor,
            **kwargs,
        )

    def right(
        self,
        width: Optional[float] = None,
        height: str = "element",
        include_source: bool = False,
        until: Optional[str] = None,
        include_endpoint: bool = True,
        offset: Optional[float] = None,
        apply_exclusions: bool = True,
        multipage: Optional[bool] = None,
        within: Optional["Region"] = None,
        anchor: str = "start",
        **kwargs,
    ) -> Union["Region", "FlowRegion"]:
        """
        Select region to the right of this region.

        Args:
            width: Width of the region to the right, in points
            height: Height mode - "full" for full page height or "element" for element height
            include_source: Whether to include this region in the result (default: False)
            until: Optional selector string to specify a right boundary element
            include_endpoint: Whether to include the boundary element in the region (default: True)
            offset: Pixel offset when excluding source/endpoint (default: None, uses natural_pdf.options.layout.directional_offset)
            multipage: Override global multipage behaviour; defaults to None meaning use global option.
            **kwargs: Additional parameters

        Returns:
            Region object representing the area to the right
        """
        return self._direction(
            direction="right",
            size=width,
            cross_size=height,
            include_source=include_source,
            until=until,
            include_endpoint=include_endpoint,
            offset=offset,
            apply_exclusions=apply_exclusions,
            multipage=multipage,
            within=within,
            anchor=anchor,
            **kwargs,
        )

    @property
    def type(self) -> str:
        """Element type."""
        # Return the specific type if detected (e.g., from layout analysis)
        # or 'region' as a default.
        return self.region_type or "region"  # Prioritize specific region_type if set

    @property
    def page(self) -> "Page":
        """Get the parent page."""
        return self._page

    @property
    def bbox(self) -> Tuple[float, float, float, float]:
        """Get the bounding box as (x0, top, x1, bottom)."""
        return self._bbox

    @property
    def x0(self) -> float:
        """Get the left coordinate."""
        return self._bbox[0]

    @property
    def top(self) -> float:
        """Get the top coordinate."""
        return self._bbox[1]

    @property
    def x1(self) -> float:
        """Get the right coordinate."""
        return self._bbox[2]

    @property
    def bottom(self) -> float:
        """Get the bottom coordinate."""
        return self._bbox[3]

    @property
    def width(self) -> float:
        """Get the width of the region."""
        return self.x1 - self.x0

    @property
    def height(self) -> float:
        """Get the height of the region."""
        return self.bottom - self.top

    @property
    def has_polygon(self) -> bool:
        """Check if this region has polygon coordinates."""
        return self._polygon is not None and len(self._polygon) >= 3

    @property
    def polygon(self) -> List[Tuple[float, float]]:
        """Get polygon coordinates if available, otherwise return rectangle corners."""
        if self._polygon:
            return self._polygon
        else:
            # Create rectangle corners from bbox as fallback
            return [
                (self.x0, self.top),  # top-left
                (self.x1, self.top),  # top-right
                (self.x1, self.bottom),  # bottom-right
                (self.x0, self.bottom),  # bottom-left
            ]

    def _context_page(self) -> "Page":
        return self.page

    def _context_region_config(self, key: str, sentinel: object) -> Any:
        region_cfg = self.metadata.get("config")
        if isinstance(region_cfg, dict) and key in region_cfg:
            return region_cfg[key]
        return sentinel

    @property
    def origin(self) -> Optional[Union["Element", "Region"]]:
        """The element/region that created this region (if it was created via directional method)."""
        return getattr(self, "source_element", None)

    @property
    def endpoint(self) -> Optional["Element"]:
        """The element where this region stopped (if created with 'until' parameter)."""
        return getattr(self, "boundary_element", None)

    def exclude(self):
        """
        Exclude this region from text extraction and other operations.

        This excludes everything within the region's bounds.
        """
        self.page.add_exclusion(self, method="region")

    def highlight(
        self,
        label: Optional[str] = None,
        color: Optional[Union[Tuple, str]] = None,
        use_color_cycling: bool = False,
        annotate: Optional[List[str]] = None,
        existing: str = "append",
    ) -> None:
        """
        Highlight this region on the page.

        Args:
            label: Optional label for the highlight
            color: Color tuple/string for the highlight, or None to use automatic color
            use_color_cycling: Force color cycling even with no label (default: False)
            annotate: List of attribute names to display on the highlight (e.g., ['confidence', 'type'])
            existing: How to handle existing highlights ('append' or 'replace').

        Returns:
            None
        """
        # Access the highlighter service correctly
        highlighter = self.page._highlighter

        # Prepare common arguments
        highlight_args = {
            "page_index": self.page.index,
            "color": color,
            "label": label,
            "use_color_cycling": use_color_cycling,
            "element": self,  # Pass the region itself so attributes can be accessed
            "annotate": annotate,
            "existing": existing,
        }

        # Call the appropriate service method
        if self.has_polygon:
            highlight_args["polygon"] = self.polygon
            highlighter.add_polygon(**highlight_args)
        else:
            highlight_args["bbox"] = self.bbox
            highlighter.add(**highlight_args)

        return None

    def save(
        self,
        filename: str,
        resolution: Optional[float] = None,
        labels: bool = True,
        legend_position: str = "right",
    ) -> "Region":
        """
        Save the page with this region highlighted to an image file.

        Args:
            filename: Path to save the image to
            resolution: Resolution in DPI for rendering (default: uses global options, fallback to 144 DPI)
            labels: Whether to include a legend for labels
            legend_position: Position of the legend

        Returns:
            Self for method chaining
        """
        image_options = _image_options()
        if resolution is None:
            default_resolution = image_options.resolution
            resolution = float(default_resolution) if default_resolution is not None else 144.0
        else:
            resolution = float(resolution)

        # Highlight this region if not already highlighted
        self.highlight()

        # Save the highlighted image
        self._page.save_image(
            filename, resolution=resolution, labels=labels, legend_position=legend_position
        )
        return self

    def save_image(
        self,
        filename: str,
        resolution: Optional[float] = None,
        crop: bool = False,
        include_highlights: bool = True,
        **kwargs,
    ) -> "Region":
        """
        Save an image of just this region to a file.

        Args:
            filename: Path to save the image to
            resolution: Resolution in DPI for rendering (default: uses global options, fallback to 144 DPI)
            crop: If True, only crop the region without highlighting its boundaries
            include_highlights: Whether to include existing highlights (default: True)
            **kwargs: Additional parameters for rendering

        Returns:
            Self for method chaining
        """
        image_options = _image_options()
        if resolution is None:
            default_resolution = image_options.resolution
            resolution = float(default_resolution) if default_resolution is not None else 144.0
        else:
            resolution = float(resolution)

        # Use export() to save the image
        if include_highlights:
            # With highlights, use export() which includes them
            self.export(
                path=filename,
                resolution=resolution,
                crop=crop,
                **kwargs,
            )
        else:
            # Without highlights, use render() and save manually
            image = self.render(resolution=resolution, crop=crop, **kwargs)
            if image is not None:
                image.save(filename)
            else:
                logger.error(f"Failed to render region image for saving to {filename}")

        return self

    def trim(
        self,
        padding: int = 1,
        threshold: float = 0.95,
        resolution: Optional[float] = None,
        pre_shrink: float = 0.5,
    ) -> "Region":
        """
        Trim visual whitespace from the edges of this region.

        Similar to Python's string .strip() method, but for visual whitespace in the region image.
        Uses pixel analysis to detect rows/columns that are predominantly whitespace.

        Args:
            padding: Number of pixels to keep as padding after trimming (default: 1)
            threshold: Threshold for considering a row/column as whitespace (0.0-1.0, default: 0.95)
                      Higher values mean more strict whitespace detection.
                      E.g., 0.95 means if 95% of pixels in a row/column are white, consider it whitespace.
            resolution: Resolution for image rendering in DPI (default: uses global options, fallback to 144 DPI)
            pre_shrink: Amount to shrink region before trimming, then expand back after (default: 0.5)
                       This helps avoid detecting box borders/slivers as content.

        Returns
        ------

        New Region with visual whitespace trimmed from all edges

        Examples
        --------

        ```python
        # Basic trimming with 1 pixel padding and 0.5px pre-shrink
        trimmed = region.trim()

        # More aggressive trimming with no padding and no pre-shrink
        tight = region.trim(padding=0, threshold=0.9, pre_shrink=0)

        # Conservative trimming with more padding
        loose = region.trim(padding=3, threshold=0.98)
        ```
        """
        image_options = _image_options()
        if resolution is None:
            default_resolution = image_options.resolution
            resolution = float(default_resolution) if default_resolution is not None else 144.0
        else:
            resolution = float(resolution)

        # Pre-shrink the region to avoid box slivers
        work_region = (
            self.expand(left=-pre_shrink, right=-pre_shrink, top=-pre_shrink, bottom=-pre_shrink)
            if pre_shrink > 0
            else self
        )

        # Get the region image
        # Use render() for clean image without highlights, with cropping
        image = work_region.render(resolution=resolution, crop=True)
        if image is None:
            raise RuntimeError(f"Region {self.bbox}: render() returned None during trimming.")

        # Convert to grayscale for easier analysis
        import numpy as np

        # Convert PIL image to numpy array
        img_array = np.array(image.convert("L"))  # Convert to grayscale
        height, width = img_array.shape

        if height == 0 or width == 0:
            raise ValueError(f"Region {self.bbox}: rendered image has zero dimensions.")

        # Normalize pixel values to 0-1 range (255 = white = 1.0, 0 = black = 0.0)
        normalized = img_array.astype(np.float32) / 255.0

        # Find content boundaries by analyzing row and column averages

        # Analyze rows (horizontal strips) to find top and bottom boundaries
        row_averages = np.mean(normalized, axis=1)  # Average each row
        content_rows = row_averages < threshold  # True where there's content (not whitespace)

        # Find first and last rows with content
        content_row_indices = np.where(content_rows)[0]
        if len(content_row_indices) == 0:
            # No content found, return a minimal region at the center
            raise ValueError(f"Region {self.bbox}: no content detected during trimming.")

        top_content_row = max(0, content_row_indices[0] - padding)
        bottom_content_row = min(height - 1, content_row_indices[-1] + padding)

        # Analyze columns (vertical strips) to find left and right boundaries
        col_averages = np.mean(normalized, axis=0)  # Average each column
        content_cols = col_averages < threshold  # True where there's content

        content_col_indices = np.where(content_cols)[0]
        if len(content_col_indices) == 0:
            # No content found in columns either
            raise ValueError(f"Region {self.bbox}: no column content detected during trimming.")

        left_content_col = max(0, content_col_indices[0] - padding)
        right_content_col = min(width - 1, content_col_indices[-1] + padding)

        # Convert trimmed pixel coordinates back to PDF coordinates
        scale_factor = resolution / 72.0  # Scale factor used in render()

        # Calculate new PDF coordinates and ensure they are Python floats
        trimmed_x0 = float(work_region.x0 + (left_content_col / scale_factor))
        trimmed_top = float(work_region.top + (top_content_row / scale_factor))
        trimmed_x1 = float(
            work_region.x0 + ((right_content_col + 1) / scale_factor)
        )  # +1 because we want inclusive right edge
        trimmed_bottom = float(
            work_region.top + ((bottom_content_row + 1) / scale_factor)
        )  # +1 because we want inclusive bottom edge

        # Ensure the trimmed region doesn't exceed the work region boundaries
        final_x0 = max(work_region.x0, trimmed_x0)
        final_top = max(work_region.top, trimmed_top)
        final_x1 = min(work_region.x1, trimmed_x1)
        final_bottom = min(work_region.bottom, trimmed_bottom)

        # Ensure valid coordinates (width > 0, height > 0)
        if final_x1 <= final_x0 or final_bottom <= final_top:
            raise ValueError(f"Region {self.bbox}: trimming produced invalid dimensions.")

        # Create the trimmed region
        trimmed_region: Region = Region(self.page, (final_x0, final_top, final_x1, final_bottom))

        # Expand back by the pre_shrink amount to restore original positioning
        if pre_shrink > 0:
            trimmed_region = cast(
                Region,
                trimmed_region.expand(
                    left=pre_shrink, right=pre_shrink, top=pre_shrink, bottom=pre_shrink
                ),
            )

        # Copy relevant metadata
        trimmed_region.region_type = self.region_type
        trimmed_region.normalized_type = self.normalized_type
        trimmed_region.confidence = self.confidence
        trimmed_region.model = self.model
        trimmed_region.name = self.name
        trimmed_region.label = self.label
        trimmed_region.source = "trimmed"  # Indicate this is a derived region
        trimmed_region.parent_region = self

        logger.debug(
            f"Region {self.bbox}: Trimmed to {trimmed_region.bbox} (padding={padding}, threshold={threshold}, pre_shrink={pre_shrink})"
        )
        return trimmed_region

    def clip(
        self,
        obj: Optional[Any] = None,
        left: Optional[float] = None,
        top: Optional[float] = None,
        right: Optional[float] = None,
        bottom: Optional[float] = None,
    ) -> "Region":
        """
        Clip this region to specific bounds, either from another object with bbox or explicit coordinates.

        The clipped region will be constrained to not exceed the specified boundaries.
        You can provide either an object with bounding box properties, specific coordinates, or both.
        When both are provided, explicit coordinates take precedence.

        Args:
            obj: Optional object with bbox properties (Region, Element, TextElement, etc.)
            left: Optional left boundary (x0) to clip to
            top: Optional top boundary to clip to
            right: Optional right boundary (x1) to clip to
            bottom: Optional bottom boundary to clip to

        Returns:
            New Region with bounds clipped to the specified constraints

        Examples:
            # Clip to another region's bounds
            clipped = region.clip(container_region)

            # Clip to any element's bounds
            clipped = region.clip(text_element)

            # Clip to specific coordinates
            clipped = region.clip(left=100, right=400)

            # Mix object bounds with specific overrides
            clipped = region.clip(obj=container, bottom=page.height/2)
        """
        from natural_pdf.elements.base import extract_bbox

        # Start with current region bounds
        clip_x0 = self.x0
        clip_top = self.top
        clip_x1 = self.x1
        clip_bottom = self.bottom

        # Apply object constraints if provided
        if obj is not None:
            obj_bbox = extract_bbox(obj)
            if obj_bbox is None:
                raise TypeError(
                    f"Region {self.bbox}: cannot extract bbox from clipping object {type(obj)}. "
                    "Object must expose bbox or x0/top/x1/bottom attributes."
                )
            obj_x0, obj_top, obj_x1, obj_bottom = obj_bbox
            clip_x0 = max(clip_x0, obj_x0)
            clip_top = max(clip_top, obj_top)
            clip_x1 = min(clip_x1, obj_x1)
            clip_bottom = min(clip_bottom, obj_bottom)

        # Apply explicit coordinate constraints (these take precedence)
        if left is not None:
            clip_x0 = max(clip_x0, left)
        if top is not None:
            clip_top = max(clip_top, top)
        if right is not None:
            clip_x1 = min(clip_x1, right)
        if bottom is not None:
            clip_bottom = min(clip_bottom, bottom)

        # Ensure valid coordinates
        if clip_x1 <= clip_x0 or clip_bottom <= clip_top:
            raise ValueError(
                f"Region {self.bbox}: clipping resulted in invalid bounds "
                f"({clip_x0}, {clip_top}, {clip_x1}, {clip_bottom})."
            )

        # Create the clipped region
        clipped_region = Region(self.page, (clip_x0, clip_top, clip_x1, clip_bottom))

        # Copy relevant metadata
        clipped_region.region_type = self.region_type
        clipped_region.normalized_type = self.normalized_type
        clipped_region.confidence = self.confidence
        clipped_region.model = self.model
        clipped_region.name = self.name
        clipped_region.label = self.label
        clipped_region.source = "clipped"  # Indicate this is a derived region
        clipped_region.parent_region = self

        logger.debug(
            f"Region {self.bbox}: Clipped to {clipped_region.bbox} "
            f"(constraints: obj={type(obj).__name__ if obj else None}, "
            f"left={left}, top={top}, right={right}, bottom={bottom})"
        )
        return clipped_region

    def region(
        self,
        left: Optional[float] = None,
        top: Optional[float] = None,
        right: Optional[float] = None,
        bottom: Optional[float] = None,
        width: Union[str, float, None] = None,
        height: Optional[float] = None,
        relative: bool = False,
    ) -> "Region":
        """
        Create a sub-region within this region using the same API as Page.region().

        By default, coordinates are absolute (relative to the page), matching Page.region().
        Set relative=True to use coordinates relative to this region's top-left corner.

        Args:
            left: Left x-coordinate (absolute by default, or relative to region if relative=True)
            top: Top y-coordinate (absolute by default, or relative to region if relative=True)
            right: Right x-coordinate (absolute by default, or relative to region if relative=True)
            bottom: Bottom y-coordinate (absolute by default, or relative to region if relative=True)
            width: Width definition (same as Page.region())
            height: Height of the region (same as Page.region())
            relative: If True, coordinates are relative to this region's top-left (0,0).
                     If False (default), coordinates are absolute page coordinates.

        Returns:
            Region object for the specified coordinates, clipped to this region's bounds

        Examples:
            # Absolute coordinates (default) - same as page.region()
            sub = region.region(left=100, top=200, width=50, height=30)

            # Relative to region's top-left
            sub = region.region(left=10, top=10, width=50, height=30, relative=True)

            # Mix relative positioning with this region's bounds
            sub = region.region(left=region.x0 + 10, width=50, height=30)
        """
        # If relative coordinates requested, convert to absolute
        if relative:
            left = (self.x0 + left) if left is not None else None
            top = (self.top + top) if top is not None else None
            right = (self.x0 + right) if right is not None else None
            bottom = (self.top + bottom) if bottom is not None else None

            # For numeric width/height with relative coords, we need to handle the calculation
            # in the context of absolute positioning

        # Use the parent page's region method to create the region with all its logic
        region_kwargs: Dict[str, Any] = {}
        if left is not None:
            region_kwargs["left"] = left
        if top is not None:
            region_kwargs["top"] = top
        if right is not None:
            region_kwargs["right"] = right
        if bottom is not None:
            region_kwargs["bottom"] = bottom
        if width is not None:
            region_kwargs["width"] = width
        if height is not None:
            region_kwargs["height"] = height

        new_region = self.page.region(**region_kwargs)

        # Clip the new region to this region's bounds
        return new_region.clip(self)

    def get_elements(
        self, selector: Optional[str] = None, apply_exclusions=True, **kwargs
    ) -> List["Element"]:
        """
        Get all elements within this region.

        Args:
            selector: Optional selector to filter elements
            apply_exclusions: Whether to apply exclusion regions
            **kwargs: Additional parameters for element filtering

        Returns:
            List of elements in the region
        """
        if selector:
            # Find elements on the page matching the selector
            page_elements = self.page.find_all(
                selector, apply_exclusions=apply_exclusions, **kwargs
            )
            # Filter those elements to only include ones within this region
            elements = [e for e in page_elements if self._is_element_in_region(e)]
        else:
            # Get all elements from the page
            page_elements = self.page.get_elements(apply_exclusions=apply_exclusions)
            # Filter to elements in this region
            elements = [e for e in page_elements if self._is_element_in_region(e)]

        # Apply boundary exclusions if this is a section with boundary settings
        if hasattr(self, "_boundary_exclusions") and self._boundary_exclusions != "both":
            excluded_ids = set()

            if self._boundary_exclusions == "none":
                # Exclude both start and end elements
                if hasattr(self, "start_element") and self.start_element:
                    excluded_ids.add(id(self.start_element))
                if hasattr(self, "end_element") and self.end_element:
                    excluded_ids.add(id(self.end_element))
            elif self._boundary_exclusions == "start":
                # Exclude only end element
                if hasattr(self, "end_element") and self.end_element:
                    excluded_ids.add(id(self.end_element))
            elif self._boundary_exclusions == "end":
                # Exclude only start element
                if hasattr(self, "start_element") and self.start_element:
                    excluded_ids.add(id(self.start_element))

            if excluded_ids:
                elements = [e for e in elements if id(e) not in excluded_ids]

        return elements

    def attr(self, name: str) -> Any:
        """
        Get an attribute value from this region.

        This method provides a consistent interface for attribute access that works
        on both individual regions/elements and collections. When called on a single
        region, it simply returns the attribute value. When called on collections,
        it extracts the attribute from all items.

        Args:
            name: The attribute name to retrieve (e.g., 'text', 'width', 'height')

        Returns:
            The attribute value, or None if the attribute doesn't exist

        Examples:
            # On a single region
            region = page.find('text:contains("Title")').expand(10)
            width = region.attr('width')  # Same as region.width

            # Consistent API across elements and regions
            obj = page.find('*:contains("Title")')  # Could be element or region
            text = obj.attr('text')  # Works for both
        """
        return getattr(self, name, None)

    def extract_text(
        self,
        granularity: str = "chars",
        apply_exclusions: bool = True,
        debug: bool = False,
        *,
        overlap: str = "center",
        newlines: Union[bool, str] = True,
        content_filter=None,
        **kwargs,
    ) -> str:
        """
        Extract text from this region, respecting page exclusions and using pdfplumber's
        layout engine (chars_to_textmap).

        Args:
            granularity: Level of text extraction - 'chars' (default) or 'words'.
                - 'chars': Character-by-character extraction (current behavior)
                - 'words': Word-level extraction with configurable overlap
            apply_exclusions: Whether to apply exclusion regions defined on the parent page.
            debug: Enable verbose debugging output for filtering steps.
            overlap: How to determine if words overlap with the region (only used when granularity='words'):
                - 'center': Word center point must be inside (default)
                - 'full': Word must be fully inside the region
                - 'partial': Any overlap includes the word
            newlines: Whether to strip newline characters from the extracted text.
            content_filter: Optional content filter to exclude specific text patterns. Can be:
                - A regex pattern string (characters matching the pattern are EXCLUDED)
                - A callable that takes text and returns True to KEEP the character
                - A list of regex patterns (characters matching ANY pattern are EXCLUDED)
            **kwargs: Additional layout parameters passed directly to pdfplumber's
                      `chars_to_textmap` function (e.g., layout, x_density, y_density).
                      See Page.extract_text docstring for more.

        Returns:
            Extracted text as string, potentially with layout-based spacing.
        """
        # Validate granularity parameter
        if granularity not in ("chars", "words"):
            raise ValueError(f"granularity must be 'chars' or 'words', got '{granularity}'")

        # Allow 'debug_exclusions' for backward compatibility
        debug = kwargs.get("debug", debug or kwargs.get("debug_exclusions", False))
        logger.debug(
            f"Region {self.bbox}: extract_text called with granularity='{granularity}', overlap='{overlap}', kwargs: {kwargs}"
        )

        # Handle word-level extraction
        if granularity == "words":
            # Use find_all to get words with proper overlap and exclusion handling
            word_elements = self.find_all(
                "text", overlap=overlap, apply_exclusions=apply_exclusions
            )

            # Join the text from all matching words
            text_parts = []
            for word in word_elements:
                word_text = word.extract_text()
                if word_text:  # Skip empty strings
                    text_parts.append(word_text)

            result = " ".join(text_parts)

            # Apply newlines processing if requested
            if newlines is False:
                result = result.replace("\n", " ").replace("\r", " ")
            elif isinstance(newlines, str):
                result = result.replace("\n", newlines).replace("\r", newlines)

            return result

        # Original character-level extraction logic follows...
        # 1. Get Word Elements potentially within this region (initial broad phase)
        # Optimization: Could use spatial query if page elements were indexed
        page_words = self.page.words  # Get all words from the page

        # 2. Gather all character dicts from words potentially in region
        # We filter precisely in filter_chars_spatially
        all_char_dicts = []
        for word in page_words:
            # Quick bbox check to avoid processing words clearly outside
            if get_bbox_overlap(self.bbox, word.bbox) is not None:
                all_char_dicts.extend(getattr(word, "_char_dicts", []))

        if not all_char_dicts:
            logger.debug(f"Region {self.bbox}: No character dicts found overlapping region bbox.")
            return ""

        # 3. Get Relevant Exclusions (overlapping this region)
        apply_exclusions_flag = kwargs.get("apply_exclusions", apply_exclusions)
        exclusion_regions = []
        if apply_exclusions_flag:
            all_page_exclusions = self._get_exclusion_regions(include_callable=True, debug=debug)
            overlapping_exclusions = []
            for excl in all_page_exclusions:
                if get_bbox_overlap(self.bbox, excl.bbox) is not None:
                    overlapping_exclusions.append(excl)
            exclusion_regions = overlapping_exclusions
            if debug:
                logger.debug(
                    f"Region {self.bbox}: Found {len(all_page_exclusions)} total exclusions, "
                    f"{len(exclusion_regions)} overlapping this region."
                )
        elif debug:
            logger.debug(f"Region {self.bbox}: Not applying exclusions (apply_exclusions=False).")

        # Add boundary element exclusions if this is a section with boundary settings
        if hasattr(self, "_boundary_exclusions") and self._boundary_exclusions != "both":
            boundary_exclusions = []

            if self._boundary_exclusions == "none":
                # Exclude both start and end elements
                if hasattr(self, "start_element") and self.start_element:
                    boundary_exclusions.append(self.start_element)
                if hasattr(self, "end_element") and self.end_element:
                    boundary_exclusions.append(self.end_element)
            elif self._boundary_exclusions == "start":
                # Exclude only end element
                if hasattr(self, "end_element") and self.end_element:
                    boundary_exclusions.append(self.end_element)
            elif self._boundary_exclusions == "end":
                # Exclude only start element
                if hasattr(self, "start_element") and self.start_element:
                    boundary_exclusions.append(self.start_element)

            # Add boundary elements as exclusion regions
            for elem in boundary_exclusions:
                if hasattr(elem, "bbox"):
                    exclusion_regions.append(elem)
                    if debug:
                        logger.debug(
                            f"Adding boundary exclusion: {elem.extract_text().strip()} at {elem.bbox}"
                        )

        # 4. Spatially Filter Characters using Utility
        # Pass self as the target_region for precise polygon checks etc.
        filtered_chars = filter_chars_spatially(
            char_dicts=all_char_dicts,
            exclusion_regions=exclusion_regions,
            target_region=self,  # Pass self!
            debug=debug,
        )

        # 5. Generate Text Layout using Utility
        # Add content_filter to kwargs if provided
        final_kwargs = kwargs.copy()
        if content_filter is not None:
            final_kwargs["content_filter"] = content_filter

        result = generate_text_layout(
            char_dicts=filtered_chars,
            layout_context_bbox=self.bbox,  # Use region's bbox for context
            user_kwargs=final_kwargs,  # Pass kwargs including content_filter
        )

        # Flexible newline handling (same logic as TextElement)
        if isinstance(newlines, bool):
            if newlines is False:
                replacement = " "
            else:
                replacement = None
        else:
            replacement = str(newlines)

        if replacement is not None:
            result = result.replace("\n", replacement).replace("\r", replacement)

        logger.debug(f"Region {self.bbox}: extract_text finished, result length: {len(result)}.")
        return result

    def find(
        self,
        selector: Optional[str] = None,
        *,
        text: Optional[Union[str, Sequence[str]]] = None,
        overlap: Optional[str] = "full",
        apply_exclusions: bool = True,
        regex: bool = False,
        case: bool = True,
        text_tolerance: Optional[Dict[str, Any]] = None,
        auto_text_tolerance: Optional[Union[bool, Dict[str, Any]]] = None,
        reading_order: bool = True,
        near_threshold: Optional[float] = None,
        engine: Optional[str] = None,
    ) -> Optional["Element"]:
        """
        Find the first element in this region matching the selector OR text content.

        Provide EITHER `selector` OR `text`, but not both.

        Args:
            selector: CSS-like selector string.
            text: Text content to search for (equivalent to 'text:contains(...)'). Accepts a
                  single string or an iterable of strings (matches any value).
            overlap: How to determine if elements overlap with the region: 'full' (fully inside),
                     'partial' (any overlap), or 'center' (center point inside).
                     (default: "full")
            apply_exclusions: Whether to exclude elements in exclusion regions (default: True).
            regex: Whether to use regex for text search (`selector` or `text`) (default: False).
            case: Whether to do case-sensitive text search (`selector` or `text`) (default: True).
            text_tolerance: Optional mapping of pdfplumber-style tolerance overrides applied
                temporarily while evaluating the selector.
            auto_text_tolerance: Optional overrides for automatic tolerance calculation.
            reading_order: Whether to return the first match according to natural reading order
                (default: True). When False the raw selector ordering is preserved.
            near_threshold: Maximum distance (in points) used by the ``:near`` pseudo-class.
            engine: Optional selector engine name registered with the selector provider.
        Returns:
            First matching element or None.
        """
        # Delegate validation and selector construction to find_all
        elements = self.find_all(
            selector=selector,
            text=text,
            overlap=overlap,
            apply_exclusions=apply_exclusions,
            regex=regex,
            case=case,
            text_tolerance=text_tolerance,
            auto_text_tolerance=auto_text_tolerance,
            reading_order=reading_order,
            near_threshold=near_threshold,
            engine=engine,
        )
        return elements.first if elements else None

    def find_all(
        self,
        selector: Optional[str] = None,
        *,
        text: Optional[Union[str, Sequence[str]]] = None,
        overlap: Optional[str] = "full",
        apply_exclusions: bool = True,
        regex: bool = False,
        case: bool = True,
        text_tolerance: Optional[Dict[str, Any]] = None,
        auto_text_tolerance: Optional[Union[bool, Dict[str, Any]]] = None,
        reading_order: bool = True,
        near_threshold: Optional[float] = None,
        engine: Optional[str] = None,
    ) -> "ElementCollection":
        """
        Find all elements in this region matching the selector OR text content.

        Provide EITHER `selector` OR `text`, but not both.

        Args:
            selector: CSS-like selector string.
            text: Text content to search for (equivalent to 'text:contains(...)'). Accepts a
                  single string or an iterable of strings (matches any value).
            overlap: How to determine if elements overlap with the region: 'full' (fully inside),
                     'partial' (any overlap), or 'center' (center point inside).
                     (default: "full")
            apply_exclusions: Whether to exclude elements in exclusion regions (default: True).
            regex: Whether to use regex for text search (`selector` or `text`) (default: False).
            case: Whether to do case-sensitive text search (`selector` or `text`) (default: True).
            text_tolerance: Optional mapping of pdfplumber-style tolerance overrides applied
                temporarily while evaluating the selector.
            auto_text_tolerance: Optional overrides for automatic tolerance calculation.
            reading_order: Whether to sort matches according to natural reading order (default: True).
            near_threshold: Maximum distance (in points) used by the ``:near`` pseudo-class.
            engine: Optional selector engine name registered with the selector provider.
        Returns:
            ElementCollection with matching elements.
        """
        from natural_pdf.elements.element_collection import ElementCollection

        overlap_mode = (overlap or "full").lower()
        if overlap_mode not in {"full", "partial", "center"}:
            raise ValueError(
                f"Invalid overlap value: {overlap_mode}. Must be 'full', 'partial', or 'center'"
            )

        page_results = self.page.find_all(
            selector=selector,
            text=text,
            apply_exclusions=apply_exclusions,
            regex=regex,
            case=case,
            text_tolerance=text_tolerance,
            auto_text_tolerance=auto_text_tolerance,
            reading_order=reading_order,
            near_threshold=near_threshold,
            engine=engine,
        )

        if not page_results:
            return ElementCollection([])

        filtered = self._filter_elements_by_overlap_mode(
            page_results.elements,
            overlap_mode,
        )

        unique: List["Element"] = []
        seen_ids: set[int] = set()
        for element in filtered:
            marker = id(element)
            if marker in seen_ids:
                continue
            seen_ids.add(marker)
            unique.append(element)

        return ElementCollection(unique)

    def _filter_elements_by_overlap_mode(
        self,
        elements: Sequence["Element"],
        overlap_mode: str,
    ) -> List["Element"]:
        """
        Apply region overlap filtering while preserving the upstream ordering.
        """
        if not elements:
            return []

        if overlap_mode == "full":
            return [el for el in elements if self.contains(el)]
        if overlap_mode == "partial":
            return [el for el in elements if self.intersects(el)]
        # overlap_mode == "center"
        return [el for el in elements if self.is_element_center_inside(el)]

    def apply_ocr(self, replace: bool = True, **ocr_params: Any) -> "Region":
        """
        Apply OCR to this region and return the created text elements.

        This method supports two modes:
        1. **Built-in/registered OCR engines** – pass parameters like ``engine='easyocr'`` or
           ``languages=['en']`` and the request is routed through the shared
           :class:`~natural_pdf.engine_provider.EngineProvider` registry.
        2. **Custom OCR Function** – pass a *callable* under the keyword ``function`` (or
           ``ocr_function``). The callable will receive *this* Region instance and should
           return the extracted text (``str``) or ``None``.  Internally the call is
           delegated to :pymeth:`apply_custom_ocr` so the same logic (replacement, element
           creation, etc.) is re-used.

        Examples
        ---------
        ```python
        def llm_ocr(region):
            image = region.render(resolution=300, crop=True)
            return my_llm_client.ocr(image)
        region.apply_ocr(function=llm_ocr)
        ```

        Args:
            replace: Whether to remove existing OCR elements first (default ``True``).
            **ocr_params: Parameters for the built-in OCR manager *or* the special
                          ``function``/``ocr_function`` keyword to trigger custom mode.

        Returns
        -------
            Self – for chaining.
        """
        custom_func_candidate = ocr_params.pop("function", None) or ocr_params.pop(
            "ocr_function", None
        )
        if callable(custom_func_candidate):
            custom_func = cast(CustomOCRCallable, custom_func_candidate)
            # Delegate to the specialised helper while preserving key kwargs
            return self.apply_custom_ocr(
                ocr_function=custom_func,
                source_label=ocr_params.pop("source_label", "custom-ocr"),
                replace=replace,
                confidence=ocr_params.pop("confidence", None),
                add_to_page=ocr_params.pop("add_to_page", True),
            )

        if replace:
            removed = self.remove_ocr_elements()
            if removed:
                logger.info(
                    f"Region {self.bbox}: Removed {removed} existing OCR elements before re-applying OCR."
                )
            else:
                logger.debug(
                    f"Region {self.bbox}: No overlapping OCR elements found before applying new OCR."
                )

        normalized_options = normalize_ocr_options(ocr_params.get("options"))
        engine_name = resolve_ocr_engine_name(
            context=self,
            requested=ocr_params.get("engine"),
            options=normalized_options,
            scope="region",
        )
        resolved_languages = resolve_ocr_languages(
            self, ocr_params.get("languages"), scope="region"
        )
        resolved_min_conf = resolve_ocr_min_confidence(
            self, ocr_params.get("min_confidence"), scope="region"
        )
        resolved_device = resolve_ocr_device(self, ocr_params.get("device"), scope="region")
        detect_only = bool(ocr_params.get("detect_only", False))

        # Determine rendering resolution from parameters
        final_resolution = ocr_params.get("resolution")
        if final_resolution is None:
            parent_pdf = getattr(self.page, "_parent", None)
            if parent_pdf is not None:
                final_resolution = parent_pdf._config.get("resolution", 150)
            else:
                final_resolution = 150
        logger.debug(
            "Region %s: Applying OCR via '%s' at %s DPI",
            self.bbox,
            engine_name,
            final_resolution,
        )

        ocr_payload = run_ocr_apply(
            target=self,
            context=self,
            engine_name=engine_name,
            resolution=final_resolution,
            languages=resolved_languages,
            min_confidence=resolved_min_conf,
            device=resolved_device,
            detect_only=detect_only,
            options=normalized_options,
            render_kwargs={"crop": True},
        )

        image_width, image_height = ocr_payload.image_size

        valid_results: List[Mapping[str, Any]] = []
        for entry in ocr_payload.results:
            if isinstance(entry, MappingABC):
                valid_results.append(entry)
            else:
                raise TypeError(
                    f"Region {self.bbox}: OCR result has unexpected type: {type(entry).__name__}"
                )

        scale_x = self.width / image_width if image_width else 1.0
        scale_y = self.height / image_height if image_height else 1.0
        created_elements = self.page.create_text_elements_from_ocr(
            valid_results, scale_x=scale_x, scale_y=scale_y
        )
        logger.info(f"Region {self.bbox}: Added {len(created_elements)} elements from OCR.")
        return self

    def apply_custom_ocr(
        self,
        ocr_function: CustomOCRCallable,
        source_label: str = "custom-ocr",
        replace: bool = True,
        confidence: Optional[float] = None,
        add_to_page: bool = True,
    ) -> "Region":
        """
        Apply a custom OCR function to this region and create text elements from the results.

        This is useful when you want to use a custom OCR method (e.g., an LLM API,
        specialized OCR service, or any custom logic) instead of the built-in OCR engines.

        Args:
            ocr_function: A callable that takes a Region and returns the OCR'd text (or None).
                          The function receives this region as its argument and should return
                          the extracted text as a string, or None if no text was found.
            source_label: Label to identify the source of these text elements (default: "custom-ocr").
                          This will be set as the 'source' attribute on created elements.
            replace: If True (default), removes existing OCR elements in this region before
                     adding new ones. If False, adds new OCR elements alongside existing ones.
            confidence: Optional confidence score for the OCR result (0.0-1.0).
                        If None, defaults to 1.0 if text is returned, 0.0 if None is returned.
            add_to_page: If True (default), adds the created text element to the page.
                         If False, creates the element but doesn't add it to the page.

        Returns:
            Self for method chaining.

        Example:
            # Using with an LLM
            def ocr_with_llm(region):
                image = region.render(resolution=300, crop=True)
                # Call your LLM API here
                return llm_client.ocr(image)

            region.apply_custom_ocr(ocr_with_llm)

            # Using with a custom OCR service
            def ocr_with_service(region):
                img_bytes = region.render(crop=True).tobytes()
                response = ocr_service.process(img_bytes)
                return response.text

            region.apply_custom_ocr(ocr_with_service, source_label="my-ocr-service")
        """
        # If replace is True, remove existing OCR elements in this region
        if replace:
            logger.info(
                f"Region {self.bbox}: Removing existing OCR elements before applying custom OCR."
            )

            removed_count = 0

            # Helper to remove a single element safely
            def _safe_remove(elem):
                nonlocal removed_count
                page_obj = getattr(elem, "page", None)
                if page_obj is None or not hasattr(page_obj, "remove_element"):
                    return
                etype = getattr(elem, "object_type", None)
                success = bool(page_obj.remove_element(elem, element_type=etype))
                if success:
                    removed_count += 1

            # Remove ALL OCR elements overlapping this region
            # Remove elements with source=="ocr" (built-in OCR) or matching the source_label (previous custom OCR)
            for word in list(self.page.get_elements_by_type("words")):
                word_source = getattr(word, "source", "")
                # Match built-in OCR behavior: remove elements with source "ocr" exactly
                # Also remove elements with the same source_label to avoid duplicates
                if (word_source == "ocr" or word_source == source_label) and self.intersects(word):
                    _safe_remove(word)

            # Also remove char dicts if needed (matching built-in OCR)
            for char in list(self.page.get_elements_by_type("chars")):
                # char can be dict or TextElement; normalize
                char_src = (
                    char.get("source") if isinstance(char, dict) else getattr(char, "source", None)
                )
                if char_src == "ocr" or char_src == source_label:
                    # Rough bbox for dicts
                    if isinstance(char, dict):
                        cx0, ctop, cx1, cbottom = (
                            char.get("x0", 0),
                            char.get("top", 0),
                            char.get("x1", 0),
                            char.get("bottom", 0),
                        )
                    else:
                        cx0, ctop, cx1, cbottom = char.x0, char.top, char.x1, char.bottom
                    # Quick overlap check
                    if not (
                        cx1 < self.x0 or cx0 > self.x1 or cbottom < self.top or ctop > self.bottom
                    ):
                        _safe_remove(char)

            if removed_count > 0:
                logger.info(f"Region {self.bbox}: Removed {removed_count} existing OCR elements.")

        # Call the custom OCR function
        logger.debug(f"Region {self.bbox}: Calling custom OCR function...")
        ocr_text = ocr_function(self)
        if ocr_text is not None and not isinstance(ocr_text, str):
            raise TypeError(
                f"Custom OCR function for region {self.bbox} returned {type(ocr_text).__name__}, "
                "expected str or None."
            )

        # Create text element if we got text
        if ocr_text is not None:
            # Use the to_text_element method to create the element
            self.to_text_element(
                text_content=ocr_text,
                source_label=source_label,
                confidence=confidence,
                add_to_page=add_to_page,
            )

            logger.info(
                f"Region {self.bbox}: Created text element with {len(ocr_text)} chars"
                f"{' and added to page' if add_to_page else ''}"
            )
        else:
            logger.debug(f"Region {self.bbox}: Custom OCR function returned None (no text found)")

        return self

    def get_section_between(
        self,
        start_element=None,
        end_element=None,
        include_boundaries="both",
        orientation="vertical",
    ):
        """
        Get a section between two elements within this region.

        Args:
            start_element: Element marking the start of the section
            end_element: Element marking the end of the section
            include_boundaries: How to include boundary elements: 'start', 'end', 'both', or 'none'
            orientation: 'vertical' (default) or 'horizontal' - determines section direction

        Returns:
            Region representing the section
        """
        # Get elements only within this region first
        elements = self.get_elements()

        if not elements:
            raise ValueError(f"Region {self.bbox}: no elements available for section extraction.")

        # Sort elements in reading order
        elements.sort(key=lambda e: (e.top, e.x0))

        if start_element and start_element not in elements:
            logger.debug("Start element not found in region, using first element.")
            start_element = elements[0]
        elif start_element is None:
            start_element = elements[0]

        if end_element and end_element not in elements:
            logger.debug("End element not found in region, using last element.")
            end_element = elements[-1]
        elif end_element is None:
            end_element = elements[-1]

        # Validate orientation parameter
        if orientation not in ["vertical", "horizontal"]:
            raise ValueError(f"orientation must be 'vertical' or 'horizontal', got '{orientation}'")

        # Use centralized section utilities
        from natural_pdf.utils.sections import calculate_section_bounds, validate_section_bounds

        # Calculate section boundaries
        bounds = calculate_section_bounds(
            start_element=start_element,
            end_element=end_element,
            include_boundaries=include_boundaries,
            orientation=orientation,
            parent_bounds=self.bbox,
        )

        # Validate boundaries
        if not validate_section_bounds(bounds, orientation):
            # Return an empty region at the start position
            x0, top, _, _ = bounds
            return Region(self.page, (x0, top, x0, top))

        # Create new region
        section = Region(self.page, bounds)

        # Store the original boundary elements and exclusion info
        section.start_element = start_element
        section.end_element = end_element
        section._boundary_exclusions = include_boundaries

        return section

    def get_sections(
        self,
        start_elements: Union[str, Sequence["Element"], "ElementCollection", None] = None,
        end_elements: Union[str, Sequence["Element"], "ElementCollection", None] = None,
        include_boundaries: str = "both",
        orientation: str = "vertical",
        **kwargs: Any,
    ) -> "ElementCollection[Region]":
        """
        Get sections within this region based on start/end elements.

        Args:
            start_elements: Elements or selector string that mark the start of sections
            end_elements: Elements or selector string that mark the end of sections
            include_boundaries: How to include boundary elements: 'start', 'end', 'both', or 'none'
            orientation: 'vertical' (default) or 'horizontal' - determines section direction

        Returns:
            List of Region objects representing the extracted sections
        """
        from natural_pdf.elements.element_collection import ElementCollection
        from natural_pdf.utils.sections import extract_sections_from_region

        def _normalize_section_boundary(
            arg: Union[str, Sequence["Element"], "ElementCollection", None]
        ) -> Union[str, List["Element"], None]:
            if arg is None or isinstance(arg, str):
                return arg

            if isinstance(arg, ElementCollection):
                return list(arg)

            if isinstance(arg, Sequence):
                return list(arg)

            # Fallback: wrap single element-like object
            return [arg]  # type: ignore[list-item]

        # Legacy tolerances (y_threshold/x_threshold) are now handled internally;
        # accept and ignore them for backward compatibility.
        legacy_thresholds = {"y_threshold", "x_threshold"}
        for legacy_key in list(kwargs.keys()):
            if legacy_key in legacy_thresholds:
                kwargs.pop(legacy_key, None)

        # Use centralized section extraction logic
        sections = extract_sections_from_region(
            region=self,
            start_elements=_normalize_section_boundary(start_elements),
            end_elements=_normalize_section_boundary(end_elements),
            include_boundaries=include_boundaries,
            orientation=orientation,
            **kwargs,
        )

        return ElementCollection(sections)

    def split(self, divider, **kwargs) -> "ElementCollection[Region]":
        """
        Divide this region into sections based on the provided divider elements.

        Args:
            divider: Elements or selector string that mark section boundaries
            **kwargs: Additional parameters passed to get_sections()
                - include_boundaries: How to include boundary elements (default: 'start')
                - orientation: 'vertical' or 'horizontal' (default: 'vertical')

        Returns:
            ElementCollection of Region objects representing the sections

        Example:
            # Split a region by bold text
            sections = region.split("text:bold")

            # Split horizontally by vertical lines
            sections = region.split("line[orientation=vertical]", orientation="horizontal")
        """
        # Default to 'start' boundaries for split (include divider at start of each section)
        if "include_boundaries" not in kwargs:
            kwargs["include_boundaries"] = "start"

        sections = self.get_sections(start_elements=divider, **kwargs)

        # Add section before first divider if there's content
        if sections and hasattr(sections[0], "start_element"):
            first_divider = sections[0].start_element
            if first_divider:
                # Get all elements before the first divider
                all_elements = self.get_elements()
                if all_elements and all_elements[0] != first_divider:
                    # Create section from start to just before first divider
                    initial_section = self.get_section_between(
                        start_element=None,
                        end_element=first_divider,
                        include_boundaries="none",
                        orientation=kwargs.get("orientation", "vertical"),
                    )
                    if initial_section and initial_section.get_elements():
                        sections.insert(0, initial_section)

        return sections

    def create_cells(self):
        """
        Create cell regions for a detected table by intersecting its
        row and column regions, and add them to the page.

        Assumes child row and column regions are already present on the page.

        Returns:
            Self for method chaining.
        """
        # Ensure this is called on a table region
        if self.region_type not in (
            "table",
            "tableofcontents",
        ):  # Allow for ToC which might have structure
            raise ValueError(
                f"create_cells should be called on a 'table' or 'tableofcontents' region, not '{self.region_type}'"
            )

        # Find rows and columns associated with this page
        # Remove the model-specific filter
        rows = self.page.find_all("region[type=table-row]")
        columns = self.page.find_all("region[type=table-column]")

        # Filter to only include those that overlap with this table region
        def is_in_table(element):
            # Use a simple overlap check (more robust than just center point)
            # Check if element's bbox overlaps with self.bbox
            return (
                hasattr(element, "bbox")
                and element.x0 < self.x1  # Ensure element has bbox
                and element.x1 > self.x0
                and element.top < self.bottom
                and element.bottom > self.top
            )

        table_rows = [r for r in rows if is_in_table(r)]
        table_columns = [c for c in columns if is_in_table(c)]

        if not table_rows or not table_columns:
            # Use page's logger if available
            logger_instance = getattr(self._page, "logger", logger)
            logger_instance.warning(
                f"Region {self.bbox}: Cannot create cells. No overlapping row or column regions found."
            )
            return self  # Return self even if no cells created

        # Sort rows and columns
        table_rows.sort(key=lambda r: r.top)
        table_columns.sort(key=lambda c: c.x0)

        # Create cells and add them to the page's element manager
        created_count = 0
        for row in table_rows:
            for column in table_columns:
                # Calculate intersection bbox for the cell
                cell_x0 = max(row.x0, column.x0)
                cell_y0 = max(row.top, column.top)
                cell_x1 = min(row.x1, column.x1)
                cell_y1 = min(row.bottom, column.bottom)

                # Only create a cell if the intersection is valid (positive width/height)
                if cell_x1 > cell_x0 and cell_y1 > cell_y0:
                    # Create cell region at the intersection
                    cell = self.page.create_region(cell_x0, cell_y0, cell_x1, cell_y1)
                    # Set metadata
                    cell.source = "derived"
                    cell.region_type = "table-cell"  # Explicitly set type
                    cell.normalized_type = "table-cell"  # And normalized type
                    # Inherit model from the parent table region
                    cell.model = self.model
                    cell.parent_region = self  # Link cell to parent table region

                    # Register the cell region with the page (tracks provenance and manager)
                    self.page.add_region(cell, source="derived")
                    created_count += 1

        # Optional: Add created cells to the table region's children
        # self.child_regions.extend(cells_created_in_this_call) # Needs list management

        logger_instance = getattr(self._page, "logger", logger)
        logger_instance.info(
            f"Region {self.bbox} (Model: {self.model}): Created and added {created_count} cell regions."
        )

        return self  # Return self for chaining

    @staticmethod
    def _normalize_qa_output(result: Any) -> Union[Dict[str, Any], List[Dict[str, Any]]]:
        """Convert QA engine results into plain dictionaries for typing compliance."""
        from natural_pdf.qa.qa_result import QAResult

        if isinstance(result, QAResult):
            return dict(result)

        if isinstance(result, list):
            normalized: List[Dict[str, Any]] = []
            for item in result:
                if isinstance(item, QAResult):
                    normalized.append(dict(item))
                elif isinstance(item, MappingABC):
                    normalized.append(dict(item))
                else:
                    normalized.append({"value": item})
            return normalized

        if isinstance(result, MappingABC):
            return dict(result)

        raise TypeError(f"Unexpected QA result type {type(result).__name__}")

    def add_child(self, child):
        """
        Add a child region to this region.

        Used for hierarchical document structure when using models like Docling
        that understand document hierarchy.

        Args:
            child: Region object to add as a child

        Returns:
            Self for method chaining
        """
        self.child_regions.append(child)
        child.parent_region = self
        return self

    def get_children(self, selector=None):
        """
        Get immediate child regions, optionally filtered by selector.

        Args:
            selector: Optional selector to filter children

        Returns:
            List of child regions matching the selector
        """
        import logging

        logger = logging.getLogger("natural_pdf.elements.region")

        if selector is None:
            return self.child_regions

        # Use existing selector parser to filter
        try:
            selector_obj = parse_selector(selector)
            filter_func = selector_to_filter_func(selector_obj)  # Removed region=self
            matched = [child for child in self.child_regions if filter_func(child)]
            logger.debug(
                f"get_children: found {len(matched)} of {len(self.child_regions)} children matching '{selector}'"
            )
            return matched
        except Exception as e:
            logger.error(f"Error applying selector in get_children: {e}", exc_info=True)
            return []  # Return empty list on error

    def get_descendants(self, selector=None):
        """
        Get all descendant regions (children, grandchildren, etc.), optionally filtered by selector.

        Args:
            selector: Optional selector to filter descendants

        Returns:
            List of descendant regions matching the selector
        """
        import logging

        logger = logging.getLogger("natural_pdf.elements.region")

        all_descendants = []
        queue = list(self.child_regions)  # Start with direct children

        while queue:
            current = queue.pop(0)
            all_descendants.append(current)
            # Add current's children to the queue for processing
            if hasattr(current, "child_regions"):
                queue.extend(current.child_regions)

        logger.debug(f"get_descendants: found {len(all_descendants)} total descendants")

        # Filter by selector if provided
        if selector is not None:
            try:
                selector_obj = parse_selector(selector)
                filter_func = selector_to_filter_func(selector_obj)  # Removed region=self
                matched = [desc for desc in all_descendants if filter_func(desc)]
                logger.debug(f"get_descendants: filtered to {len(matched)} matching '{selector}'")
                return matched
            except Exception as e:
                logger.error(f"Error applying selector in get_descendants: {e}", exc_info=True)
                return []  # Return empty list on error

        return all_descendants

    def __add__(
        self, other: Union["Element", "Region", "ElementCollection"]
    ) -> "ElementCollection":
        """Add regions/elements together to create an ElementCollection.

        This allows intuitive combination of regions using the + operator:
        ```python
        complainant = section.find("text:contains(Complainant)").right(until='text')
        dob = section.find("text:contains(DOB)").right(until='text')
        combined = complainant + dob  # Creates ElementCollection with both regions
        ```

        Args:
            other: Another Region, Element or ElementCollection to combine

        Returns:
            ElementCollection containing all elements
        """
        from natural_pdf.elements.base import Element
        from natural_pdf.elements.element_collection import ElementCollection

        # Create a list starting with self
        elements: List[Union[Element, "Region"]] = [self]

        # Add the other element(s)
        if isinstance(other, (Element, Region)):
            elements.append(other)
        elif isinstance(other, ElementCollection):
            elements.extend(cast(Iterable[Union[Element, "Region"]], list(other)))
        elif hasattr(other, "__iter__") and not isinstance(other, (str, bytes)):
            # Handle other iterables but exclude strings
            elements.extend(cast(Iterable[Union[Element, "Region"]], list(other)))
        else:
            raise TypeError(f"Cannot add Region with {type(other)}")

        return ElementCollection(elements)

    def __radd__(
        self, other: Union["Element", "Region", "ElementCollection"]
    ) -> "ElementCollection":
        """Right-hand addition to support ElementCollection + Region."""
        if other == 0:
            # This handles sum() which starts with 0
            from natural_pdf.elements.element_collection import ElementCollection

            return ElementCollection([self])
        return self.__add__(other)

    def __repr__(self) -> str:
        """String representation of the region."""
        poly_info = " (Polygon)" if self.has_polygon else ""
        name_info = f" name='{self.name}'" if self.name else ""
        type_info = f" type='{self.region_type}'" if self.region_type else ""
        source_info = f" source='{self.source}'" if self.source else ""

        # Add checkbox state if this is a checkbox
        checkbox_info = ""
        if self.region_type == "checkbox" and hasattr(self, "is_checked"):
            state = "checked" if self.is_checked else "unchecked"
            checkbox_info = f" [{state}]"

        return f"<Region{name_info}{type_info}{source_info}{checkbox_info} bbox={self.bbox}{poly_info}>"

    def update_text(
        self,
        transform: Callable[[Any], Optional[str]],
        *,
        selector: str = "text",
        apply_exclusions: bool = False,
    ) -> "Region":
        """Apply *transform* to every text element matched by *selector* inside this region.

        The heavy lifting is delegated to :py:meth:`TextMixin.update_text`; this
        override simply ensures the search is scoped to the region.
        """

        TextMixin.update_text(self, transform, selector=selector, apply_exclusions=apply_exclusions)
        return self

    def _get_classification_content(
        self, model_type: str, **kwargs
    ) -> Union[str, "Image"]:  # Use "Image" for lazy import
        if model_type == "text":
            text_content = self.extract_text(layout=False)  # Simple join for classification
            if not text_content or text_content.isspace():
                raise ValueError("Cannot classify region with 'text' model: No text content found.")
            return text_content
        elif model_type == "vision":
            resolution = kwargs.get("resolution", 150)
            img = self.render(
                resolution=resolution,
                crop=True,  # Just the region content
            )
            if img is None:
                raise ValueError(
                    "Cannot classify region with 'vision' model: Failed to render image."
                )
            return img
        else:
            raise ValueError(f"Unsupported model_type for classification: {model_type}")

    def _get_metadata_storage(self) -> Dict[str, Any]:
        # Ensure metadata exists
        if not hasattr(self, "metadata") or self.metadata is None:
            self.metadata = {}
        return self.metadata

    def analyze_text_table_structure(
        self,
        snap_tolerance: int = 10,
        join_tolerance: int = 3,
        min_words_vertical: int = 3,
        min_words_horizontal: int = 1,
        intersection_tolerance: int = 3,
        expand_bbox: Optional[Dict[str, int]] = None,
        **kwargs,
    ) -> Optional[Dict]:
        """
        Analyzes the text elements within the region (or slightly expanded area)
        to find potential table structure (lines, cells) using text alignment logic
        adapted from pdfplumber.

        Args:
            snap_tolerance: Tolerance for snapping parallel lines.
            join_tolerance: Tolerance for joining collinear lines.
            min_words_vertical: Minimum words needed to define a vertical line.
            min_words_horizontal: Minimum words needed to define a horizontal line.
            intersection_tolerance: Tolerance for detecting line intersections.
            expand_bbox: Optional dictionary to expand the search area slightly beyond
                         the region's exact bounds (e.g., {'left': 5, 'right': 5}).
            **kwargs: Additional keyword arguments passed to
                      find_text_based_tables (e.g., specific x/y tolerances).

        Returns:
            A dictionary containing 'horizontal_edges', 'vertical_edges', 'cells' (list of dicts),
            and 'intersections', or None if pdfplumber is unavailable or an error occurs.
        """

        # Determine the search region (expand if requested)
        search_region = self
        if expand_bbox and isinstance(expand_bbox, dict):

            def _coerce_expand_value(value: Any) -> Union[float, bool, str]:
                if isinstance(value, (bool, str)):
                    return value
                return float(value)

            sanitized_expand: Dict[str, Any] = {}
            if "amount" in expand_bbox and expand_bbox["amount"] is not None:
                sanitized_expand["amount"] = float(expand_bbox["amount"])
            for key in ("left", "right", "top", "bottom"):
                if key in expand_bbox and expand_bbox[key] is not None:
                    sanitized_expand[key] = _coerce_expand_value(expand_bbox[key])
            if "width_factor" in expand_bbox and expand_bbox["width_factor"] is not None:
                sanitized_expand["width_factor"] = float(expand_bbox["width_factor"])
            if "height_factor" in expand_bbox and expand_bbox["height_factor"] is not None:
                sanitized_expand["height_factor"] = float(expand_bbox["height_factor"])
            if "apply_exclusions" in expand_bbox and expand_bbox["apply_exclusions"] is not None:
                sanitized_expand["apply_exclusions"] = bool(expand_bbox["apply_exclusions"])

            search_region = self.expand(**sanitized_expand)
            logger.debug(f"Expanded search region for text table analysis to: {search_region.bbox}")

        # Find text elements within the search region
        text_elements = search_region.find_all(
            "text", apply_exclusions=False
        )  # Use unfiltered text
        if not text_elements:
            logger.info(f"Region {self.bbox}: No text elements found for text table analysis.")
            return {"horizontal_edges": [], "vertical_edges": [], "cells": [], "intersections": {}}

        # Extract bounding boxes
        bboxes = [element.bbox for element in text_elements if hasattr(element, "bbox")]
        if not bboxes:
            logger.info(f"Region {self.bbox}: No bboxes extracted from text elements.")
            return {"horizontal_edges": [], "vertical_edges": [], "cells": [], "intersections": {}}

        # Call the utility function
        try:
            analysis_results = find_text_based_tables(
                bboxes=bboxes,
                snap_tolerance=snap_tolerance,
                join_tolerance=join_tolerance,
                min_words_vertical=min_words_vertical,
                min_words_horizontal=min_words_horizontal,
                intersection_tolerance=intersection_tolerance,
                **kwargs,  # Pass through any extra specific tolerance args
            )
            # Store results in the region's analyses cache
            self.analyses["text_table_structure"] = analysis_results
            return analysis_results
        except ImportError:
            logger.error("pdfplumber library is required for 'text' table analysis but not found.")
            return None
        except Exception as e:
            logger.error(f"Error during text-based table analysis: {e}", exc_info=True)
            return None

    def get_text_table_cells(
        self,
        snap_tolerance: int = 10,
        join_tolerance: int = 3,
        min_words_vertical: int = 3,
        min_words_horizontal: int = 1,
        intersection_tolerance: int = 3,
        expand_bbox: Optional[Dict[str, int]] = None,
        **kwargs,
    ) -> "ElementCollection[Region]":
        """
        Analyzes text alignment to find table cells and returns them as
        temporary Region objects without adding them to the page.

        Args:
            snap_tolerance: Tolerance for snapping parallel lines.
            join_tolerance: Tolerance for joining collinear lines.
            min_words_vertical: Minimum words needed to define a vertical line.
            min_words_horizontal: Minimum words needed to define a horizontal line.
            intersection_tolerance: Tolerance for detecting line intersections.
            expand_bbox: Optional dictionary to expand the search area slightly beyond
                         the region's exact bounds (e.g., {'left': 5, 'right': 5}).
            **kwargs: Additional keyword arguments passed to
                      find_text_based_tables (e.g., specific x/y tolerances).

        Returns:
            An ElementCollection containing temporary Region objects for each detected cell,
            or an empty ElementCollection if no cells are found or an error occurs.
        """
        from natural_pdf.elements.element_collection import ElementCollection

        # 1. Perform the analysis (or use cached results)
        if "text_table_structure" in self.analyses:
            analysis_results = self.analyses["text_table_structure"]
            logger.debug("get_text_table_cells: Using cached analysis results.")
        else:
            analysis_results = self.analyze_text_table_structure(
                snap_tolerance=snap_tolerance,
                join_tolerance=join_tolerance,
                min_words_vertical=min_words_vertical,
                min_words_horizontal=min_words_horizontal,
                intersection_tolerance=intersection_tolerance,
                expand_bbox=expand_bbox,
                **kwargs,
            )

        # 2. Check if analysis was successful and cells were found
        if analysis_results is None or not analysis_results.get("cells"):
            logger.info(f"Region {self.bbox}: No cells found by text table analysis.")
            return ElementCollection([])  # Return empty collection

        # 3. Create temporary Region objects for each cell dictionary
        cell_regions = []
        for cell_data in analysis_results["cells"]:
            cell_region = self.page.region(**cell_data)
            cell_region.region_type = "table-cell"
            cell_region.normalized_type = "table-cell"
            cell_region.model = "pdfplumber-text"
            cell_region.source = "volatile"
            cell_region.parent_region = self
            cell_regions.append(cell_region)

        # 4. Return the list wrapped in an ElementCollection
        logger.debug(f"get_text_table_cells: Created {len(cell_regions)} temporary cell regions.")
        return ElementCollection(cell_regions)

    def to_text_element(
        self,
        text_content: Optional[Union[str, Callable[["Region"], Optional[str]]]] = None,
        source_label: str = "derived_from_region",
        object_type: str = "word",  # Or "char", controls how it's categorized
        default_font_size: float = 10.0,
        default_font_name: str = "RegionContent",
        confidence: Optional[float] = None,  # Allow overriding confidence
        add_to_page: bool = False,  # NEW: Option to add to page
    ) -> "TextElement":
        """
        Creates a new TextElement object based on this region's geometry.

        The text for the new TextElement can be provided directly,
        generated by a callback function, or left as None.

        Args:
            text_content:
                - If a string, this will be the text of the new TextElement.
                - If a callable, it will be called with this region instance
                  and its return value (a string or None) will be the text.
                - If None (default), the TextElement's text will be None.
            source_label: The 'source' attribute for the new TextElement.
            object_type: The 'object_type' for the TextElement's data dict
                         (e.g., "word", "char").
            default_font_size: Placeholder font size if text is generated.
            default_font_name: Placeholder font name if text is generated.
            confidence: Confidence score for the text. If text_content is None,
                        defaults to 0.0. If text is provided/generated, defaults to 1.0
                        unless specified.
            add_to_page: If True, the created TextElement will be added to the
                         region's parent page. (Default: False)

        Returns:
            A new TextElement instance.

        Raises:
            ValueError: If the region does not have a valid 'page' attribute.
        """
        actual_text: Optional[str] = None
        if isinstance(text_content, str):
            actual_text = text_content
        elif callable(text_content):
            try:
                actual_text = text_content(self)
            except Exception as e:
                logger.error(
                    f"Error executing text_content callback for region {self.bbox}: {e}",
                    exc_info=True,
                )
                actual_text = None  # Ensure actual_text is None on error

        final_confidence = confidence
        if final_confidence is None:
            final_confidence = 1.0 if actual_text is not None and actual_text.strip() else 0.0

        if not hasattr(self, "page") or self.page is None:
            raise ValueError("Region must have a valid 'page' attribute to create a TextElement.")

        # Create character dictionaries for the text
        char_dicts = []
        if actual_text:
            # Create a single character dict that spans the entire region
            # This is a simplified approach - OCR engines typically create one per character
            char_dict = {
                "text": actual_text,
                "x0": self.x0,
                "top": self.top,
                "x1": self.x1,
                "bottom": self.bottom,
                "width": self.width,
                "height": self.height,
                "object_type": "char",
                "page_number": self.page.page_number,
                "fontname": default_font_name,
                "size": default_font_size,
                "upright": True,
                "direction": 1,
                "adv": self.width,
                "source": source_label,
                "confidence": final_confidence,
                "stroking_color": (0, 0, 0),
                "non_stroking_color": (0, 0, 0),
            }
            char_dicts.append(char_dict)

        elem_data = {
            "text": actual_text,
            "x0": self.x0,
            "top": self.top,
            "x1": self.x1,
            "bottom": self.bottom,
            "width": self.width,
            "height": self.height,
            "object_type": object_type,
            "page_number": self.page.page_number,
            "stroking_color": getattr(self, "stroking_color", (0, 0, 0)),
            "non_stroking_color": getattr(self, "non_stroking_color", (0, 0, 0)),
            "fontname": default_font_name,
            "size": default_font_size,
            "upright": True,
            "direction": 1,
            "adv": self.width,
            "source": source_label,
            "confidence": final_confidence,
            "_char_dicts": char_dicts,
        }
        text_element = TextElement(elem_data, self.page)

        if add_to_page:
            add_as_type = (
                "words"
                if object_type == "word"
                else "chars" if object_type == "char" else object_type
            )
            if hasattr(self.page, "add_element"):
                self.page.add_element(text_element, element_type=add_as_type)
                logger.debug(
                    "TextElement created from region %s and added to page %s as %s.",
                    self.bbox,
                    getattr(self.page, "page_number", "N/A"),
                    add_as_type,
                )
                if char_dicts and object_type == "word":
                    for char_dict in char_dicts:
                        self.page.add_element(char_dict, element_type="chars")
            else:
                page_num_str = (
                    str(self.page.page_number) if hasattr(self.page, "page_number") else "N/A"
                )
                raise AttributeError(
                    f"Cannot add TextElement to page {page_num_str}: page does not expose add_element"
                )

        return text_element

    # Unified analysis storage (maps to metadata["analysis"])

    @property
    def analyses(self) -> Dict[str, Any]:
        if not hasattr(self, "metadata") or self.metadata is None:
            self.metadata = {}
        return self.metadata.setdefault("analysis", {})

    @analyses.setter
    def analyses(self, value: Dict[str, Any]):
        if not hasattr(self, "metadata") or self.metadata is None:
            self.metadata = {}
        self.metadata["analysis"] = value

    # New helper: build table from pre-computed table_cell regions

    def _apply_rtl_processing_to_text(self, text: str) -> str:
        """
        Apply RTL (Right-to-Left) text processing to a string.

        This converts visual order text (as stored in PDFs) to logical order
        for proper display of Arabic, Hebrew, and other RTL scripts.

        Args:
            text: Input text string in visual order

        Returns:
            Text string in logical order
        """
        if not text or not text.strip():
            return text

        # Quick check for RTL characters - if none found, return as-is
        import unicodedata

        def _contains_rtl(s):
            return any(unicodedata.bidirectional(ch) in ("R", "AL", "AN") for ch in s)

        if not _contains_rtl(text):
            return text

        try:
            from bidi.algorithm import get_display  # type: ignore

            from natural_pdf.utils.bidi_mirror import mirror_brackets

            # Apply BiDi algorithm to convert from visual to logical order
            # Process line by line to handle mixed content properly
            processed_lines = []
            for line in text.split("\n"):
                if line.strip():
                    # Determine base direction for this line
                    base_dir = "R" if _contains_rtl(line) else "L"
                    logical_line = get_display(line, base_dir=base_dir)
                    logical_line_str = (
                        logical_line if isinstance(logical_line, str) else str(logical_line)
                    )
                    # Apply bracket mirroring for correct logical order
                    processed_lines.append(mirror_brackets(logical_line_str))
                else:
                    processed_lines.append(line)

            return "\n".join(processed_lines)

        except (ImportError, Exception):
            # If bidi library is not available or fails, return original text
            return text

    def _apply_content_filter_to_text(self, text: str, content_filter) -> str:
        """
        Apply content filter to a text string.

        Args:
            text: Input text string
            content_filter: Content filter (regex, callable, or list of regexes)

        Returns:
            Filtered text string
        """
        if not text or content_filter is None:
            return text

        import re

        if isinstance(content_filter, str):
            # Single regex pattern - remove matching parts
            try:
                return re.sub(content_filter, "", text)
            except re.error:
                return text  # Invalid regex, return original

        elif isinstance(content_filter, list):
            # List of regex patterns - remove parts matching ANY pattern
            try:
                result = text
                for pattern in content_filter:
                    result = re.sub(pattern, "", result)
                return result
            except re.error:
                return text  # Invalid regex, return original

        elif callable(content_filter):
            # Callable filter - apply to individual characters
            try:
                filtered_chars = []
                for char in text:
                    if content_filter(char):
                        filtered_chars.append(char)
                return "".join(filtered_chars)
            except Exception:
                return text  # Function error, return original

        return text

    # Interactive Viewer Support

    def viewer(
        self,
        *,
        resolution: int = 150,
        include_chars: bool = False,
        include_attributes: Optional[List[str]] = None,
    ) -> Optional[Any]:
        """Create an interactive ipywidget viewer for **this specific region**.

        The method renders the region to an image (cropped to the region bounds) and
        overlays all elements that intersect the region (optionally excluding noisy
        character-level elements).  The resulting widget offers the same zoom / pan
        experience as :py:meth:`Page.viewer` but scoped to the region.

        Parameters
        ----------
        resolution : int, default 150
            Rendering resolution (DPI).  This should match the value used by the
            page-level viewer so element scaling is accurate.
        include_chars : bool, default False
            Whether to include individual *char* elements in the overlay.  These
            are often too dense for a meaningful visualisation so are skipped by
            default.
        include_attributes : list[str], optional
            Additional element attributes to expose in the info panel (on top of
            the default set used by the page viewer).

        Returns
        -------
        InteractiveViewerWidgetType | None
            The widget instance, or ``None`` if *ipywidgets* is not installed or
            an error occurred during creation.
        """

        # Dependency / environment checks
        if not _IPYWIDGETS_AVAILABLE or InteractiveViewerWidget is None:
            logger.error(
                "Interactive viewer requires 'ipywidgets'. "
                'Please install with: pip install "ipywidgets>=7.0.0,<10.0.0"'
            )
            return None

        try:
            # Render region image (cropped) and encode as data URI
            import base64
            from io import BytesIO

            # Use unified render() with crop=True to obtain just the region
            img = self.render(resolution=resolution, crop=True)
            if img is None:
                logger.error(f"Failed to render image for region {self.bbox} viewer.")
                return None

            buf = BytesIO()
            img.save(buf, format="PNG")
            img_str = base64.b64encode(buf.getvalue()).decode()
            image_uri = f"data:image/png;base64,{img_str}"

            # Prepare element overlay data (coordinates relative to region)
            scale = resolution / 72.0  # Same convention as page viewer

            # Gather elements intersecting the region
            region_elements = self.get_elements(apply_exclusions=False)

            # Optionally filter out chars
            if not include_chars:
                region_elements = [
                    el for el in region_elements if str(getattr(el, "type", "")).lower() != "char"
                ]

            default_attrs = [
                "text",
                "fontname",
                "size",
                "bold",
                "italic",
                "color",
                "linewidth",
                "is_horizontal",
                "is_vertical",
                "source",
                "confidence",
                "label",
                "model",
                "upright",
                "direction",
            ]

            if include_attributes:
                default_attrs.extend([a for a in include_attributes if a not in default_attrs])

            elements_json: List[dict] = []
            for idx, el in enumerate(region_elements):
                x0 = (el.x0 - self.x0) * scale
                y0 = (el.top - self.top) * scale
                x1 = (el.x1 - self.x0) * scale
                y1 = (el.bottom - self.top) * scale

                elem_dict = {
                    "id": idx,
                    "type": getattr(el, "type", "unknown"),
                    "x0": round(x0, 2),
                    "y0": round(y0, 2),
                    "x1": round(x1, 2),
                    "y1": round(y1, 2),
                    "width": round(x1 - x0, 2),
                    "height": round(y1 - y0, 2),
                }

                for attr_name in default_attrs:
                    if hasattr(el, attr_name):
                        val = getattr(el, attr_name)
                        if not isinstance(val, (str, int, float, bool, list, dict, type(None))):
                            val = str(val)
                        elem_dict[attr_name] = val
                elements_json.append(elem_dict)

            viewer_data = {"page_image": image_uri, "elements": elements_json}

            # Instantiate the widget directly using the prepared data
            return InteractiveViewerWidget(pdf_data=viewer_data)

        except Exception as e:
            logger.error(f"Error creating viewer for region {self.bbox}: {e}", exc_info=True)
            return None

    def within(self):
        """Context manager that constrains directional operations to this region.

        When used as a context manager, all directional navigation operations
        (above, below, left, right) will be constrained to the bounds of this region.

        Returns:
            RegionContext: A context manager that yields this region

        Examples:
            ```python
            # Create a column region
            left_col = page.region(right=page.width/2)

            # All directional operations are constrained to left_col
            with left_col.within() as col:
                header = col.find("text[size>14]")
                content = header.below(until="text[size>14]")
                # content will only include elements within left_col

            # Operations outside the context are not constrained
            full_page_below = header.below()  # Searches full page
            ```
        """
        return RegionContext(self)

Attributes

natural_pdf.Region.bbox property

Get the bounding box as (x0, top, x1, bottom).

natural_pdf.Region.bottom property

Get the bottom coordinate.

natural_pdf.Region.endpoint property

The element where this region stopped (if created with 'until' parameter).

natural_pdf.Region.has_polygon property

Check if this region has polygon coordinates.

natural_pdf.Region.height property

Get the height of the region.

natural_pdf.Region.origin property

The element/region that created this region (if it was created via directional method).

natural_pdf.Region.page property

Get the parent page.

natural_pdf.Region.polygon property

Get polygon coordinates if available, otherwise return rectangle corners.

natural_pdf.Region.top property

Get the top coordinate.

natural_pdf.Region.type property

Element type.

natural_pdf.Region.width property

Get the width of the region.

natural_pdf.Region.x0 property

Get the left coordinate.

natural_pdf.Region.x1 property

Get the right coordinate.

Functions

natural_pdf.Region.__add__(other)

Add regions/elements together to create an ElementCollection.

This allows intuitive combination of regions using the + operator:

complainant = section.find("text:contains(Complainant)").right(until='text')
dob = section.find("text:contains(DOB)").right(until='text')
combined = complainant + dob  # Creates ElementCollection with both regions

Parameters:

Name	Type	Description	Default
other	Union['Element', 'Region', 'ElementCollection']	Another Region, Element or ElementCollection to combine	required

Returns:

Type	Description
'ElementCollection'	ElementCollection containing all elements

Source code in natural_pdf/elements/region.py

def __add__(
    self, other: Union["Element", "Region", "ElementCollection"]
) -> "ElementCollection":
    """Add regions/elements together to create an ElementCollection.

    This allows intuitive combination of regions using the + operator:
    ```python
    complainant = section.find("text:contains(Complainant)").right(until='text')
    dob = section.find("text:contains(DOB)").right(until='text')
    combined = complainant + dob  # Creates ElementCollection with both regions
    ```

    Args:
        other: Another Region, Element or ElementCollection to combine

    Returns:
        ElementCollection containing all elements
    """
    from natural_pdf.elements.base import Element
    from natural_pdf.elements.element_collection import ElementCollection

    # Create a list starting with self
    elements: List[Union[Element, "Region"]] = [self]

    # Add the other element(s)
    if isinstance(other, (Element, Region)):
        elements.append(other)
    elif isinstance(other, ElementCollection):
        elements.extend(cast(Iterable[Union[Element, "Region"]], list(other)))
    elif hasattr(other, "__iter__") and not isinstance(other, (str, bytes)):
        # Handle other iterables but exclude strings
        elements.extend(cast(Iterable[Union[Element, "Region"]], list(other)))
    else:
        raise TypeError(f"Cannot add Region with {type(other)}")

    return ElementCollection(elements)

natural_pdf.Region.__init__(page, bbox, polygon=None, parent=None, label=None)

Initialize a region.

Creates a Region object that represents a rectangular or polygonal area on a page. Regions are used for spatial navigation, content extraction, and analysis operations.

Parameters:

Name	Type	Description	Default
page	'Page'	Parent Page object that contains this region and provides access to document elements and analysis capabilities.	required
bbox	Tuple[float, float, float, float]	Bounding box coordinates as (x0, top, x1, bottom) tuple in PDF coordinate system (points, with origin at bottom-left).	required
polygon	Optional[List[Tuple[float, float]]]	Optional list of coordinate points [(x1,y1), (x2,y2), ...] for non-rectangular regions. If provided, the region will use polygon-based intersection calculations instead of simple rectangle overlap.	None
parent	Optional['Region']	Optional parent region for hierarchical document structure. Useful for maintaining tree-like relationships between regions.	None
label	Optional[str]	Optional descriptive label for the region, useful for debugging and identification in complex workflows.	None

Example

pdf = npdf.PDF("document.pdf")
page = pdf.pages[0]

# Rectangular region
header = Region(page, (0, 0, page.width, 100), label="header")

# Polygonal region (from layout detection)
table_polygon = [(50, 100), (300, 100), (300, 400), (50, 400)]
table_region = Region(page, (50, 100, 300, 400),
                    polygon=table_polygon, label="table")

Note

Regions are typically created through page methods like page.region() or spatial navigation methods like element.below(). Direct instantiation is used mainly for advanced workflows or layout analysis integration.

Source code in natural_pdf/elements/region.py

def __init__(
    self,
    page: "Page",
    bbox: Tuple[float, float, float, float],
    polygon: Optional[List[Tuple[float, float]]] = None,
    parent: Optional["Region"] = None,
    label: Optional[str] = None,
):
    """Initialize a region.

    Creates a Region object that represents a rectangular or polygonal area on a page.
    Regions are used for spatial navigation, content extraction, and analysis operations.

    Args:
        page: Parent Page object that contains this region and provides access
            to document elements and analysis capabilities.
        bbox: Bounding box coordinates as (x0, top, x1, bottom) tuple in PDF
            coordinate system (points, with origin at bottom-left).
        polygon: Optional list of coordinate points [(x1,y1), (x2,y2), ...] for
            non-rectangular regions. If provided, the region will use polygon-based
            intersection calculations instead of simple rectangle overlap.
        parent: Optional parent region for hierarchical document structure.
            Useful for maintaining tree-like relationships between regions.
        label: Optional descriptive label for the region, useful for debugging
            and identification in complex workflows.

    Example:
        ```python
        pdf = npdf.PDF("document.pdf")
        page = pdf.pages[0]

        # Rectangular region
        header = Region(page, (0, 0, page.width, 100), label="header")

        # Polygonal region (from layout detection)
        table_polygon = [(50, 100), (300, 100), (300, 400), (50, 400)]
        table_region = Region(page, (50, 100, 300, 400),
                            polygon=table_polygon, label="table")
        ```

    Note:
        Regions are typically created through page methods like page.region() or
        spatial navigation methods like element.below(). Direct instantiation is
        used mainly for advanced workflows or layout analysis integration.
    """
    self._page: "Page" = page
    self._bbox: Tuple[float, float, float, float] = bbox
    self._polygon: Optional[List[Tuple[float, float]]] = polygon

    self.metadata: Dict[str, Any] = {}
    # Analysis results live under self.metadata['analysis'] via property

    # Standard attributes for all elements
    self.object_type: str = "region"  # For selector compatibility
    self.start_element: Optional["Element"] = None
    self.end_element: Optional["Element"] = None
    self._boundary_exclusions: Optional[str] = None

    # Layout detection attributes
    self.region_type: Optional[str] = None
    self.normalized_type: Optional[str] = None
    self.confidence: Optional[float] = None
    self.model: Optional[str] = None
    self.is_checked: Optional[bool] = None
    self.checkbox_state: Optional[str] = None
    self.original_class: Optional[str] = None

    # Region management attributes
    self.name: Optional[str] = None
    self.label = label
    self.source: Optional[str] = None  # Will be set by creation methods

    # Hierarchy support for nested document structure
    self.parent_region: Optional["Region"] = parent
    self.child_regions: List["Region"] = []
    self.text_content: Optional[str] = None  # Direct text content (e.g., from Docling)
    self.associated_text_elements: List[TextElement] = (
        []
    )  # Native text elements that overlap with this region
    self.source_element: Optional[Union["Element", "Region"]] = None
    self.includes_source: bool = False
    self.boundary_element: Optional["Element"] = None

    if self.parent_region is not None:
        self.parent_region.child_regions.append(self)

    self._cached_text: Optional[str] = None
    self._cached_elements: Optional[ElementCollection] = None
    self._cached_bbox: Optional[Tuple[float, float, float, float]] = None
    self._exclusions: List[ExclusionSpec] = []

natural_pdf.Region.__radd__(other)

Right-hand addition to support ElementCollection + Region.

Source code in natural_pdf/elements/region.py

def __radd__(
    self, other: Union["Element", "Region", "ElementCollection"]
) -> "ElementCollection":
    """Right-hand addition to support ElementCollection + Region."""
    if other == 0:
        # This handles sum() which starts with 0
        from natural_pdf.elements.element_collection import ElementCollection

        return ElementCollection([self])
    return self.__add__(other)

natural_pdf.Region.__repr__()

String representation of the region.

Source code in natural_pdf/elements/region.py

def __repr__(self) -> str:
    """String representation of the region."""
    poly_info = " (Polygon)" if self.has_polygon else ""
    name_info = f" name='{self.name}'" if self.name else ""
    type_info = f" type='{self.region_type}'" if self.region_type else ""
    source_info = f" source='{self.source}'" if self.source else ""

    # Add checkbox state if this is a checkbox
    checkbox_info = ""
    if self.region_type == "checkbox" and hasattr(self, "is_checked"):
        state = "checked" if self.is_checked else "unchecked"
        checkbox_info = f" [{state}]"

    return f"<Region{name_info}{type_info}{source_info}{checkbox_info} bbox={self.bbox}{poly_info}>"

natural_pdf.Region.above(height=None, width='full', include_source=False, until=None, include_endpoint=True, offset=None, apply_exclusions=True, multipage=None, within=None, anchor='start', **kwargs)

Select region above this region.

Parameters:

Name	Type	Description	Default
height	Optional[float]	Height of the region above, in points	None
width	str	Width mode - "full" for full page width or "element" for element width	'full'
include_source	bool	Whether to include this region in the result (default: False)	False
until	Optional[str]	Optional selector string to specify an upper boundary element	None
include_endpoint	bool	Whether to include the boundary element in the region (default: True)	True
offset	Optional[float]	Pixel offset when excluding source/endpoint (default: None, uses natural_pdf.options.layout.directional_offset)	None
multipage	Optional[bool]	Override global multipage behaviour; defaults to None meaning use global option.	None
**kwargs		Additional parameters	{}

Returns:

Type	Description
Union['Region', 'FlowRegion']	Region object representing the area above

Source code in natural_pdf/elements/region.py

def above(
    self,
    height: Optional[float] = None,
    width: str = "full",
    include_source: bool = False,
    until: Optional[str] = None,
    include_endpoint: bool = True,
    offset: Optional[float] = None,
    apply_exclusions: bool = True,
    multipage: Optional[bool] = None,
    within: Optional["Region"] = None,
    anchor: str = "start",
    **kwargs,
) -> Union["Region", "FlowRegion"]:
    """
    Select region above this region.

    Args:
        height: Height of the region above, in points
        width: Width mode - "full" for full page width or "element" for element width
        include_source: Whether to include this region in the result (default: False)
        until: Optional selector string to specify an upper boundary element
        include_endpoint: Whether to include the boundary element in the region (default: True)
        offset: Pixel offset when excluding source/endpoint (default: None, uses natural_pdf.options.layout.directional_offset)
        multipage: Override global multipage behaviour; defaults to None meaning use global option.
        **kwargs: Additional parameters

    Returns:
        Region object representing the area above
    """
    return self._direction(
        direction="above",
        size=height,
        cross_size=width,
        include_source=include_source,
        until=until,
        include_endpoint=include_endpoint,
        offset=offset,
        apply_exclusions=apply_exclusions,
        multipage=multipage,
        within=within,
        anchor=anchor,
        **kwargs,
    )

natural_pdf.Region.add_child(child)

Add a child region to this region.

Used for hierarchical document structure when using models like Docling that understand document hierarchy.

Parameters:

Name	Type	Description	Default
child		Region object to add as a child	required

Returns:

Type	Description
	Self for method chaining

Source code in natural_pdf/elements/region.py

def add_child(self, child):
    """
    Add a child region to this region.

    Used for hierarchical document structure when using models like Docling
    that understand document hierarchy.

    Args:
        child: Region object to add as a child

    Returns:
        Self for method chaining
    """
    self.child_regions.append(child)
    child.parent_region = self
    return self

natural_pdf.Region.analyze_text_table_structure(snap_tolerance=10, join_tolerance=3, min_words_vertical=3, min_words_horizontal=1, intersection_tolerance=3, expand_bbox=None, **kwargs)

Analyzes the text elements within the region (or slightly expanded area) to find potential table structure (lines, cells) using text alignment logic adapted from pdfplumber.

Parameters:

Name	Type	Description	Default
snap_tolerance	int	Tolerance for snapping parallel lines.	10
join_tolerance	int	Tolerance for joining collinear lines.	3
min_words_vertical	int	Minimum words needed to define a vertical line.	3
min_words_horizontal	int	Minimum words needed to define a horizontal line.	1
intersection_tolerance	int	Tolerance for detecting line intersections.	3
expand_bbox	Optional[Dict[str, int]]	Optional dictionary to expand the search area slightly beyond the region's exact bounds (e.g., {'left': 5, 'right': 5}).	None
**kwargs		Additional keyword arguments passed to find_text_based_tables (e.g., specific x/y tolerances).	{}

Returns:

Type	Description
Optional[Dict]	A dictionary containing 'horizontal_edges', 'vertical_edges', 'cells' (list of dicts),
Optional[Dict]	and 'intersections', or None if pdfplumber is unavailable or an error occurs.

Source code in natural_pdf/elements/region.py

def analyze_text_table_structure(
    self,
    snap_tolerance: int = 10,
    join_tolerance: int = 3,
    min_words_vertical: int = 3,
    min_words_horizontal: int = 1,
    intersection_tolerance: int = 3,
    expand_bbox: Optional[Dict[str, int]] = None,
    **kwargs,
) -> Optional[Dict]:
    """
    Analyzes the text elements within the region (or slightly expanded area)
    to find potential table structure (lines, cells) using text alignment logic
    adapted from pdfplumber.

    Args:
        snap_tolerance: Tolerance for snapping parallel lines.
        join_tolerance: Tolerance for joining collinear lines.
        min_words_vertical: Minimum words needed to define a vertical line.
        min_words_horizontal: Minimum words needed to define a horizontal line.
        intersection_tolerance: Tolerance for detecting line intersections.
        expand_bbox: Optional dictionary to expand the search area slightly beyond
                     the region's exact bounds (e.g., {'left': 5, 'right': 5}).
        **kwargs: Additional keyword arguments passed to
                  find_text_based_tables (e.g., specific x/y tolerances).

    Returns:
        A dictionary containing 'horizontal_edges', 'vertical_edges', 'cells' (list of dicts),
        and 'intersections', or None if pdfplumber is unavailable or an error occurs.
    """

    # Determine the search region (expand if requested)
    search_region = self
    if expand_bbox and isinstance(expand_bbox, dict):

        def _coerce_expand_value(value: Any) -> Union[float, bool, str]:
            if isinstance(value, (bool, str)):
                return value
            return float(value)

        sanitized_expand: Dict[str, Any] = {}
        if "amount" in expand_bbox and expand_bbox["amount"] is not None:
            sanitized_expand["amount"] = float(expand_bbox["amount"])
        for key in ("left", "right", "top", "bottom"):
            if key in expand_bbox and expand_bbox[key] is not None:
                sanitized_expand[key] = _coerce_expand_value(expand_bbox[key])
        if "width_factor" in expand_bbox and expand_bbox["width_factor"] is not None:
            sanitized_expand["width_factor"] = float(expand_bbox["width_factor"])
        if "height_factor" in expand_bbox and expand_bbox["height_factor"] is not None:
            sanitized_expand["height_factor"] = float(expand_bbox["height_factor"])
        if "apply_exclusions" in expand_bbox and expand_bbox["apply_exclusions"] is not None:
            sanitized_expand["apply_exclusions"] = bool(expand_bbox["apply_exclusions"])

        search_region = self.expand(**sanitized_expand)
        logger.debug(f"Expanded search region for text table analysis to: {search_region.bbox}")

    # Find text elements within the search region
    text_elements = search_region.find_all(
        "text", apply_exclusions=False
    )  # Use unfiltered text
    if not text_elements:
        logger.info(f"Region {self.bbox}: No text elements found for text table analysis.")
        return {"horizontal_edges": [], "vertical_edges": [], "cells": [], "intersections": {}}

    # Extract bounding boxes
    bboxes = [element.bbox for element in text_elements if hasattr(element, "bbox")]
    if not bboxes:
        logger.info(f"Region {self.bbox}: No bboxes extracted from text elements.")
        return {"horizontal_edges": [], "vertical_edges": [], "cells": [], "intersections": {}}

    # Call the utility function
    try:
        analysis_results = find_text_based_tables(
            bboxes=bboxes,
            snap_tolerance=snap_tolerance,
            join_tolerance=join_tolerance,
            min_words_vertical=min_words_vertical,
            min_words_horizontal=min_words_horizontal,
            intersection_tolerance=intersection_tolerance,
            **kwargs,  # Pass through any extra specific tolerance args
        )
        # Store results in the region's analyses cache
        self.analyses["text_table_structure"] = analysis_results
        return analysis_results
    except ImportError:
        logger.error("pdfplumber library is required for 'text' table analysis but not found.")
        return None
    except Exception as e:
        logger.error(f"Error during text-based table analysis: {e}", exc_info=True)
        return None

natural_pdf.Region.apply_custom_ocr(ocr_function, source_label='custom-ocr', replace=True, confidence=None, add_to_page=True)

Apply a custom OCR function to this region and create text elements from the results.

This is useful when you want to use a custom OCR method (e.g., an LLM API, specialized OCR service, or any custom logic) instead of the built-in OCR engines.

Parameters:

Name	Type	Description	Default
ocr_function	CustomOCRCallable	A callable that takes a Region and returns the OCR'd text (or None). The function receives this region as its argument and should return the extracted text as a string, or None if no text was found.	required
source_label	str	Label to identify the source of these text elements (default: "custom-ocr"). This will be set as the 'source' attribute on created elements.	'custom-ocr'
replace	bool	If True (default), removes existing OCR elements in this region before adding new ones. If False, adds new OCR elements alongside existing ones.	True
confidence	Optional[float]	Optional confidence score for the OCR result (0.0-1.0). If None, defaults to 1.0 if text is returned, 0.0 if None is returned.	None
add_to_page	bool	If True (default), adds the created text element to the page. If False, creates the element but doesn't add it to the page.	True

Returns:

Type	Description
'Region'	Self for method chaining.

Example

Using with an LLM

def ocr_with_llm(region): image = region.render(resolution=300, crop=True) # Call your LLM API here return llm_client.ocr(image)

region.apply_custom_ocr(ocr_with_llm)

Using with a custom OCR service

def ocr_with_service(region): img_bytes = region.render(crop=True).tobytes() response = ocr_service.process(img_bytes) return response.text

region.apply_custom_ocr(ocr_with_service, source_label="my-ocr-service")

Source code in natural_pdf/elements/region.py

def apply_custom_ocr(
    self,
    ocr_function: CustomOCRCallable,
    source_label: str = "custom-ocr",
    replace: bool = True,
    confidence: Optional[float] = None,
    add_to_page: bool = True,
) -> "Region":
    """
    Apply a custom OCR function to this region and create text elements from the results.

    This is useful when you want to use a custom OCR method (e.g., an LLM API,
    specialized OCR service, or any custom logic) instead of the built-in OCR engines.

    Args:
        ocr_function: A callable that takes a Region and returns the OCR'd text (or None).
                      The function receives this region as its argument and should return
                      the extracted text as a string, or None if no text was found.
        source_label: Label to identify the source of these text elements (default: "custom-ocr").
                      This will be set as the 'source' attribute on created elements.
        replace: If True (default), removes existing OCR elements in this region before
                 adding new ones. If False, adds new OCR elements alongside existing ones.
        confidence: Optional confidence score for the OCR result (0.0-1.0).
                    If None, defaults to 1.0 if text is returned, 0.0 if None is returned.
        add_to_page: If True (default), adds the created text element to the page.
                     If False, creates the element but doesn't add it to the page.

    Returns:
        Self for method chaining.

    Example:
        # Using with an LLM
        def ocr_with_llm(region):
            image = region.render(resolution=300, crop=True)
            # Call your LLM API here
            return llm_client.ocr(image)

        region.apply_custom_ocr(ocr_with_llm)

        # Using with a custom OCR service
        def ocr_with_service(region):
            img_bytes = region.render(crop=True).tobytes()
            response = ocr_service.process(img_bytes)
            return response.text

        region.apply_custom_ocr(ocr_with_service, source_label="my-ocr-service")
    """
    # If replace is True, remove existing OCR elements in this region
    if replace:
        logger.info(
            f"Region {self.bbox}: Removing existing OCR elements before applying custom OCR."
        )

        removed_count = 0

        # Helper to remove a single element safely
        def _safe_remove(elem):
            nonlocal removed_count
            page_obj = getattr(elem, "page", None)
            if page_obj is None or not hasattr(page_obj, "remove_element"):
                return
            etype = getattr(elem, "object_type", None)
            success = bool(page_obj.remove_element(elem, element_type=etype))
            if success:
                removed_count += 1

        # Remove ALL OCR elements overlapping this region
        # Remove elements with source=="ocr" (built-in OCR) or matching the source_label (previous custom OCR)
        for word in list(self.page.get_elements_by_type("words")):
            word_source = getattr(word, "source", "")
            # Match built-in OCR behavior: remove elements with source "ocr" exactly
            # Also remove elements with the same source_label to avoid duplicates
            if (word_source == "ocr" or word_source == source_label) and self.intersects(word):
                _safe_remove(word)

        # Also remove char dicts if needed (matching built-in OCR)
        for char in list(self.page.get_elements_by_type("chars")):
            # char can be dict or TextElement; normalize
            char_src = (
                char.get("source") if isinstance(char, dict) else getattr(char, "source", None)
            )
            if char_src == "ocr" or char_src == source_label:
                # Rough bbox for dicts
                if isinstance(char, dict):
                    cx0, ctop, cx1, cbottom = (
                        char.get("x0", 0),
                        char.get("top", 0),
                        char.get("x1", 0),
                        char.get("bottom", 0),
                    )
                else:
                    cx0, ctop, cx1, cbottom = char.x0, char.top, char.x1, char.bottom
                # Quick overlap check
                if not (
                    cx1 < self.x0 or cx0 > self.x1 or cbottom < self.top or ctop > self.bottom
                ):
                    _safe_remove(char)

        if removed_count > 0:
            logger.info(f"Region {self.bbox}: Removed {removed_count} existing OCR elements.")

    # Call the custom OCR function
    logger.debug(f"Region {self.bbox}: Calling custom OCR function...")
    ocr_text = ocr_function(self)
    if ocr_text is not None and not isinstance(ocr_text, str):
        raise TypeError(
            f"Custom OCR function for region {self.bbox} returned {type(ocr_text).__name__}, "
            "expected str or None."
        )

    # Create text element if we got text
    if ocr_text is not None:
        # Use the to_text_element method to create the element
        self.to_text_element(
            text_content=ocr_text,
            source_label=source_label,
            confidence=confidence,
            add_to_page=add_to_page,
        )

        logger.info(
            f"Region {self.bbox}: Created text element with {len(ocr_text)} chars"
            f"{' and added to page' if add_to_page else ''}"
        )
    else:
        logger.debug(f"Region {self.bbox}: Custom OCR function returned None (no text found)")

    return self

natural_pdf.Region.apply_ocr(replace=True, **ocr_params)

Apply OCR to this region and return the created text elements.

This method supports two modes: 1. Built-in/registered OCR engines – pass parameters like engine='easyocr' or languages=['en'] and the request is routed through the shared :class:~natural_pdf.engine_provider.EngineProvider registry. 2. Custom OCR Function – pass a callable under the keyword function (or ocr_function). The callable will receive this Region instance and should return the extracted text (str) or None. Internally the call is delegated to :pymeth:apply_custom_ocr so the same logic (replacement, element creation, etc.) is re-used.

Examples

def llm_ocr(region):
    image = region.render(resolution=300, crop=True)
    return my_llm_client.ocr(image)
region.apply_ocr(function=llm_ocr)

Parameters:

Name	Type	Description	Default
replace	bool	Whether to remove existing OCR elements first (default True).	True
**ocr_params	Any	Parameters for the built-in OCR manager or the special function/ocr_function keyword to trigger custom mode.	{}

Returns

Self – for chaining.

Source code in natural_pdf/elements/region.py

def apply_ocr(self, replace: bool = True, **ocr_params: Any) -> "Region":
    """
    Apply OCR to this region and return the created text elements.

    This method supports two modes:
    1. **Built-in/registered OCR engines** – pass parameters like ``engine='easyocr'`` or
       ``languages=['en']`` and the request is routed through the shared
       :class:`~natural_pdf.engine_provider.EngineProvider` registry.
    2. **Custom OCR Function** – pass a *callable* under the keyword ``function`` (or
       ``ocr_function``). The callable will receive *this* Region instance and should
       return the extracted text (``str``) or ``None``.  Internally the call is
       delegated to :pymeth:`apply_custom_ocr` so the same logic (replacement, element
       creation, etc.) is re-used.

    Examples
    ---------
    ```python
    def llm_ocr(region):
        image = region.render(resolution=300, crop=True)
        return my_llm_client.ocr(image)
    region.apply_ocr(function=llm_ocr)
    ```

    Args:
        replace: Whether to remove existing OCR elements first (default ``True``).
        **ocr_params: Parameters for the built-in OCR manager *or* the special
                      ``function``/``ocr_function`` keyword to trigger custom mode.

    Returns
    -------
        Self – for chaining.
    """
    custom_func_candidate = ocr_params.pop("function", None) or ocr_params.pop(
        "ocr_function", None
    )
    if callable(custom_func_candidate):
        custom_func = cast(CustomOCRCallable, custom_func_candidate)
        # Delegate to the specialised helper while preserving key kwargs
        return self.apply_custom_ocr(
            ocr_function=custom_func,
            source_label=ocr_params.pop("source_label", "custom-ocr"),
            replace=replace,
            confidence=ocr_params.pop("confidence", None),
            add_to_page=ocr_params.pop("add_to_page", True),
        )

    if replace:
        removed = self.remove_ocr_elements()
        if removed:
            logger.info(
                f"Region {self.bbox}: Removed {removed} existing OCR elements before re-applying OCR."
            )
        else:
            logger.debug(
                f"Region {self.bbox}: No overlapping OCR elements found before applying new OCR."
            )

    normalized_options = normalize_ocr_options(ocr_params.get("options"))
    engine_name = resolve_ocr_engine_name(
        context=self,
        requested=ocr_params.get("engine"),
        options=normalized_options,
        scope="region",
    )
    resolved_languages = resolve_ocr_languages(
        self, ocr_params.get("languages"), scope="region"
    )
    resolved_min_conf = resolve_ocr_min_confidence(
        self, ocr_params.get("min_confidence"), scope="region"
    )
    resolved_device = resolve_ocr_device(self, ocr_params.get("device"), scope="region")
    detect_only = bool(ocr_params.get("detect_only", False))

    # Determine rendering resolution from parameters
    final_resolution = ocr_params.get("resolution")
    if final_resolution is None:
        parent_pdf = getattr(self.page, "_parent", None)
        if parent_pdf is not None:
            final_resolution = parent_pdf._config.get("resolution", 150)
        else:
            final_resolution = 150
    logger.debug(
        "Region %s: Applying OCR via '%s' at %s DPI",
        self.bbox,
        engine_name,
        final_resolution,
    )

    ocr_payload = run_ocr_apply(
        target=self,
        context=self,
        engine_name=engine_name,
        resolution=final_resolution,
        languages=resolved_languages,
        min_confidence=resolved_min_conf,
        device=resolved_device,
        detect_only=detect_only,
        options=normalized_options,
        render_kwargs={"crop": True},
    )

    image_width, image_height = ocr_payload.image_size

    valid_results: List[Mapping[str, Any]] = []
    for entry in ocr_payload.results:
        if isinstance(entry, MappingABC):
            valid_results.append(entry)
        else:
            raise TypeError(
                f"Region {self.bbox}: OCR result has unexpected type: {type(entry).__name__}"
            )

    scale_x = self.width / image_width if image_width else 1.0
    scale_y = self.height / image_height if image_height else 1.0
    created_elements = self.page.create_text_elements_from_ocr(
        valid_results, scale_x=scale_x, scale_y=scale_y
    )
    logger.info(f"Region {self.bbox}: Added {len(created_elements)} elements from OCR.")
    return self

natural_pdf.Region.attr(name)

Get an attribute value from this region.

This method provides a consistent interface for attribute access that works on both individual regions/elements and collections. When called on a single region, it simply returns the attribute value. When called on collections, it extracts the attribute from all items.

Parameters:

Name	Type	Description	Default
name	str	The attribute name to retrieve (e.g., 'text', 'width', 'height')	required

Returns:

Type	Description
Any	The attribute value, or None if the attribute doesn't exist

Examples:

On a single region

region = page.find('text:contains("Title")').expand(10) width = region.attr('width') # Same as region.width

Consistent API across elements and regions

obj = page.find('*:contains("Title")') # Could be element or region text = obj.attr('text') # Works for both

Source code in natural_pdf/elements/region.py

def attr(self, name: str) -> Any:
    """
    Get an attribute value from this region.

    This method provides a consistent interface for attribute access that works
    on both individual regions/elements and collections. When called on a single
    region, it simply returns the attribute value. When called on collections,
    it extracts the attribute from all items.

    Args:
        name: The attribute name to retrieve (e.g., 'text', 'width', 'height')

    Returns:
        The attribute value, or None if the attribute doesn't exist

    Examples:
        # On a single region
        region = page.find('text:contains("Title")').expand(10)
        width = region.attr('width')  # Same as region.width

        # Consistent API across elements and regions
        obj = page.find('*:contains("Title")')  # Could be element or region
        text = obj.attr('text')  # Works for both
    """
    return getattr(self, name, None)

natural_pdf.Region.below(height=None, width='full', include_source=False, until=None, include_endpoint=True, offset=None, apply_exclusions=True, multipage=None, within=None, anchor='start', **kwargs)

Select region below this region.

Parameters:

Name	Type	Description	Default
height	Optional[float]	Height of the region below, in points	None
width	str	Width mode - "full" for full page width or "element" for element width	'full'
include_source	bool	Whether to include this region in the result (default: False)	False
until	Optional[str]	Optional selector string to specify a lower boundary element	None
include_endpoint	bool	Whether to include the boundary element in the region (default: True)	True
offset	Optional[float]	Pixel offset when excluding source/endpoint (default: None, uses natural_pdf.options.layout.directional_offset)	None
multipage	Optional[bool]	Override global multipage behaviour; defaults to None meaning use global option.	None
**kwargs		Additional parameters	{}

Returns:

Type	Description
Union['Region', 'FlowRegion']	Region object representing the area below

Source code in natural_pdf/elements/region.py

def below(
    self,
    height: Optional[float] = None,
    width: str = "full",
    include_source: bool = False,
    until: Optional[str] = None,
    include_endpoint: bool = True,
    offset: Optional[float] = None,
    apply_exclusions: bool = True,
    multipage: Optional[bool] = None,
    within: Optional["Region"] = None,
    anchor: str = "start",
    **kwargs,
) -> Union["Region", "FlowRegion"]:
    """
    Select region below this region.

    Args:
        height: Height of the region below, in points
        width: Width mode - "full" for full page width or "element" for element width
        include_source: Whether to include this region in the result (default: False)
        until: Optional selector string to specify a lower boundary element
        include_endpoint: Whether to include the boundary element in the region (default: True)
        offset: Pixel offset when excluding source/endpoint (default: None, uses natural_pdf.options.layout.directional_offset)
        multipage: Override global multipage behaviour; defaults to None meaning use global option.
        **kwargs: Additional parameters

    Returns:
        Region object representing the area below
    """
    return self._direction(
        direction="below",
        size=height,
        cross_size=width,
        include_source=include_source,
        until=until,
        include_endpoint=include_endpoint,
        offset=offset,
        apply_exclusions=apply_exclusions,
        multipage=multipage,
        within=within,
        anchor=anchor,
        **kwargs,
    )

natural_pdf.Region.clip(obj=None, left=None, top=None, right=None, bottom=None)

Clip this region to specific bounds, either from another object with bbox or explicit coordinates.

The clipped region will be constrained to not exceed the specified boundaries. You can provide either an object with bounding box properties, specific coordinates, or both. When both are provided, explicit coordinates take precedence.

Parameters:

Name	Type	Description	Default
obj	Optional[Any]	Optional object with bbox properties (Region, Element, TextElement, etc.)	None
left	Optional[float]	Optional left boundary (x0) to clip to	None
top	Optional[float]	Optional top boundary to clip to	None
right	Optional[float]	Optional right boundary (x1) to clip to	None
bottom	Optional[float]	Optional bottom boundary to clip to	None

Returns:

Type	Description
'Region'	New Region with bounds clipped to the specified constraints

Examples:

Clip to another region's bounds

clipped = region.clip(container_region)

Clip to any element's bounds

clipped = region.clip(text_element)

Clip to specific coordinates

clipped = region.clip(left=100, right=400)

Mix object bounds with specific overrides

clipped = region.clip(obj=container, bottom=page.height/2)

Source code in natural_pdf/elements/region.py

def clip(
    self,
    obj: Optional[Any] = None,
    left: Optional[float] = None,
    top: Optional[float] = None,
    right: Optional[float] = None,
    bottom: Optional[float] = None,
) -> "Region":
    """
    Clip this region to specific bounds, either from another object with bbox or explicit coordinates.

    The clipped region will be constrained to not exceed the specified boundaries.
    You can provide either an object with bounding box properties, specific coordinates, or both.
    When both are provided, explicit coordinates take precedence.

    Args:
        obj: Optional object with bbox properties (Region, Element, TextElement, etc.)
        left: Optional left boundary (x0) to clip to
        top: Optional top boundary to clip to
        right: Optional right boundary (x1) to clip to
        bottom: Optional bottom boundary to clip to

    Returns:
        New Region with bounds clipped to the specified constraints

    Examples:
        # Clip to another region's bounds
        clipped = region.clip(container_region)

        # Clip to any element's bounds
        clipped = region.clip(text_element)

        # Clip to specific coordinates
        clipped = region.clip(left=100, right=400)

        # Mix object bounds with specific overrides
        clipped = region.clip(obj=container, bottom=page.height/2)
    """
    from natural_pdf.elements.base import extract_bbox

    # Start with current region bounds
    clip_x0 = self.x0
    clip_top = self.top
    clip_x1 = self.x1
    clip_bottom = self.bottom

    # Apply object constraints if provided
    if obj is not None:
        obj_bbox = extract_bbox(obj)
        if obj_bbox is None:
            raise TypeError(
                f"Region {self.bbox}: cannot extract bbox from clipping object {type(obj)}. "
                "Object must expose bbox or x0/top/x1/bottom attributes."
            )
        obj_x0, obj_top, obj_x1, obj_bottom = obj_bbox
        clip_x0 = max(clip_x0, obj_x0)
        clip_top = max(clip_top, obj_top)
        clip_x1 = min(clip_x1, obj_x1)
        clip_bottom = min(clip_bottom, obj_bottom)

    # Apply explicit coordinate constraints (these take precedence)
    if left is not None:
        clip_x0 = max(clip_x0, left)
    if top is not None:
        clip_top = max(clip_top, top)
    if right is not None:
        clip_x1 = min(clip_x1, right)
    if bottom is not None:
        clip_bottom = min(clip_bottom, bottom)

    # Ensure valid coordinates
    if clip_x1 <= clip_x0 or clip_bottom <= clip_top:
        raise ValueError(
            f"Region {self.bbox}: clipping resulted in invalid bounds "
            f"({clip_x0}, {clip_top}, {clip_x1}, {clip_bottom})."
        )

    # Create the clipped region
    clipped_region = Region(self.page, (clip_x0, clip_top, clip_x1, clip_bottom))

    # Copy relevant metadata
    clipped_region.region_type = self.region_type
    clipped_region.normalized_type = self.normalized_type
    clipped_region.confidence = self.confidence
    clipped_region.model = self.model
    clipped_region.name = self.name
    clipped_region.label = self.label
    clipped_region.source = "clipped"  # Indicate this is a derived region
    clipped_region.parent_region = self

    logger.debug(
        f"Region {self.bbox}: Clipped to {clipped_region.bbox} "
        f"(constraints: obj={type(obj).__name__ if obj else None}, "
        f"left={left}, top={top}, right={right}, bottom={bottom})"
    )
    return clipped_region

natural_pdf.Region.create_cells()

Create cell regions for a detected table by intersecting its row and column regions, and add them to the page.

Assumes child row and column regions are already present on the page.

Returns:

Type	Description
	Self for method chaining.

Source code in natural_pdf/elements/region.py

def create_cells(self):
    """
    Create cell regions for a detected table by intersecting its
    row and column regions, and add them to the page.

    Assumes child row and column regions are already present on the page.

    Returns:
        Self for method chaining.
    """
    # Ensure this is called on a table region
    if self.region_type not in (
        "table",
        "tableofcontents",
    ):  # Allow for ToC which might have structure
        raise ValueError(
            f"create_cells should be called on a 'table' or 'tableofcontents' region, not '{self.region_type}'"
        )

    # Find rows and columns associated with this page
    # Remove the model-specific filter
    rows = self.page.find_all("region[type=table-row]")
    columns = self.page.find_all("region[type=table-column]")

    # Filter to only include those that overlap with this table region
    def is_in_table(element):
        # Use a simple overlap check (more robust than just center point)
        # Check if element's bbox overlaps with self.bbox
        return (
            hasattr(element, "bbox")
            and element.x0 < self.x1  # Ensure element has bbox
            and element.x1 > self.x0
            and element.top < self.bottom
            and element.bottom > self.top
        )

    table_rows = [r for r in rows if is_in_table(r)]
    table_columns = [c for c in columns if is_in_table(c)]

    if not table_rows or not table_columns:
        # Use page's logger if available
        logger_instance = getattr(self._page, "logger", logger)
        logger_instance.warning(
            f"Region {self.bbox}: Cannot create cells. No overlapping row or column regions found."
        )
        return self  # Return self even if no cells created

    # Sort rows and columns
    table_rows.sort(key=lambda r: r.top)
    table_columns.sort(key=lambda c: c.x0)

    # Create cells and add them to the page's element manager
    created_count = 0
    for row in table_rows:
        for column in table_columns:
            # Calculate intersection bbox for the cell
            cell_x0 = max(row.x0, column.x0)
            cell_y0 = max(row.top, column.top)
            cell_x1 = min(row.x1, column.x1)
            cell_y1 = min(row.bottom, column.bottom)

            # Only create a cell if the intersection is valid (positive width/height)
            if cell_x1 > cell_x0 and cell_y1 > cell_y0:
                # Create cell region at the intersection
                cell = self.page.create_region(cell_x0, cell_y0, cell_x1, cell_y1)
                # Set metadata
                cell.source = "derived"
                cell.region_type = "table-cell"  # Explicitly set type
                cell.normalized_type = "table-cell"  # And normalized type
                # Inherit model from the parent table region
                cell.model = self.model
                cell.parent_region = self  # Link cell to parent table region

                # Register the cell region with the page (tracks provenance and manager)
                self.page.add_region(cell, source="derived")
                created_count += 1

    # Optional: Add created cells to the table region's children
    # self.child_regions.extend(cells_created_in_this_call) # Needs list management

    logger_instance = getattr(self._page, "logger", logger)
    logger_instance.info(
        f"Region {self.bbox} (Model: {self.model}): Created and added {created_count} cell regions."
    )

    return self  # Return self for chaining

natural_pdf.Region.create_region(left, top, right, bottom, *, relative=True, label=None)

Create a child region anchored to this region.

Parameters:

Name	Type	Description	Default
left	float	Left coordinate. Interpreted relative to this region when relative is True.	required
top	float	Top coordinate.	required
right	float	Right coordinate.	required
bottom	float	Bottom coordinate.	required
relative	bool	When True (default), coordinates are treated as offsets from this region's bounds. Set to False to provide absolute page coordinates.	True
label	Optional[str]	Optional label to assign to the new region.	None

Returns:

Type	Description
'Region'	The newly created child region.

Source code in natural_pdf/elements/region.py

def create_region(
    self,
    left: float,
    top: float,
    right: float,
    bottom: float,
    *,
    relative: bool = True,
    label: Optional[str] = None,
) -> "Region":
    """Create a child region anchored to this region.

    Args:
        left: Left coordinate. Interpreted relative to this region when ``relative`` is True.
        top: Top coordinate.
        right: Right coordinate.
        bottom: Bottom coordinate.
        relative: When True (default), coordinates are treated as offsets from this
            region's bounds. Set to False to provide absolute page coordinates.
        label: Optional label to assign to the new region.

    Returns:
        The newly created child region.
    """

    page = getattr(self, "page", None)
    if page is None:
        raise ValueError("Cannot create a sub-region without an associated page")

    if relative:
        abs_left = self.x0 + left
        abs_top = self.top + top
        abs_right = self.x0 + right
        abs_bottom = self.top + bottom
    else:
        abs_left = left
        abs_top = top
        abs_right = right
        abs_bottom = bottom

    child_region = page.region(left=abs_left, top=abs_top, right=abs_right, bottom=abs_bottom)
    child_region.parent_region = self
    self.child_regions.append(child_region)
    if label is not None:
        child_region.label = label
    return child_region

natural_pdf.Region.exclude()

Exclude this region from text extraction and other operations.

This excludes everything within the region's bounds.

Source code in natural_pdf/elements/region.py

def exclude(self):
    """
    Exclude this region from text extraction and other operations.

    This excludes everything within the region's bounds.
    """
    self.page.add_exclusion(self, method="region")

natural_pdf.Region.extract_text(granularity='chars', apply_exclusions=True, debug=False, *, overlap='center', newlines=True, content_filter=None, **kwargs)

Extract text from this region, respecting page exclusions and using pdfplumber's layout engine (chars_to_textmap).

Parameters:

Name	Type	Description	Default
granularity	str	Level of text extraction - 'chars' (default) or 'words'. - 'chars': Character-by-character extraction (current behavior) - 'words': Word-level extraction with configurable overlap	'chars'
apply_exclusions	bool	Whether to apply exclusion regions defined on the parent page.	True
debug	bool	Enable verbose debugging output for filtering steps.	False
overlap	str	How to determine if words overlap with the region (only used when granularity='words'): - 'center': Word center point must be inside (default) - 'full': Word must be fully inside the region - 'partial': Any overlap includes the word	'center'
newlines	Union[bool, str]	Whether to strip newline characters from the extracted text.	True
content_filter		Optional content filter to exclude specific text patterns. Can be: - A regex pattern string (characters matching the pattern are EXCLUDED) - A callable that takes text and returns True to KEEP the character - A list of regex patterns (characters matching ANY pattern are EXCLUDED)	None
**kwargs		Additional layout parameters passed directly to pdfplumber's chars_to_textmap function (e.g., layout, x_density, y_density). See Page.extract_text docstring for more.	{}

Returns:

Type	Description
str	Extracted text as string, potentially with layout-based spacing.

Source code in natural_pdf/elements/region.py

def extract_text(
    self,
    granularity: str = "chars",
    apply_exclusions: bool = True,
    debug: bool = False,
    *,
    overlap: str = "center",
    newlines: Union[bool, str] = True,
    content_filter=None,
    **kwargs,
) -> str:
    """
    Extract text from this region, respecting page exclusions and using pdfplumber's
    layout engine (chars_to_textmap).

    Args:
        granularity: Level of text extraction - 'chars' (default) or 'words'.
            - 'chars': Character-by-character extraction (current behavior)
            - 'words': Word-level extraction with configurable overlap
        apply_exclusions: Whether to apply exclusion regions defined on the parent page.
        debug: Enable verbose debugging output for filtering steps.
        overlap: How to determine if words overlap with the region (only used when granularity='words'):
            - 'center': Word center point must be inside (default)
            - 'full': Word must be fully inside the region
            - 'partial': Any overlap includes the word
        newlines: Whether to strip newline characters from the extracted text.
        content_filter: Optional content filter to exclude specific text patterns. Can be:
            - A regex pattern string (characters matching the pattern are EXCLUDED)
            - A callable that takes text and returns True to KEEP the character
            - A list of regex patterns (characters matching ANY pattern are EXCLUDED)
        **kwargs: Additional layout parameters passed directly to pdfplumber's
                  `chars_to_textmap` function (e.g., layout, x_density, y_density).
                  See Page.extract_text docstring for more.

    Returns:
        Extracted text as string, potentially with layout-based spacing.
    """
    # Validate granularity parameter
    if granularity not in ("chars", "words"):
        raise ValueError(f"granularity must be 'chars' or 'words', got '{granularity}'")

    # Allow 'debug_exclusions' for backward compatibility
    debug = kwargs.get("debug", debug or kwargs.get("debug_exclusions", False))
    logger.debug(
        f"Region {self.bbox}: extract_text called with granularity='{granularity}', overlap='{overlap}', kwargs: {kwargs}"
    )

    # Handle word-level extraction
    if granularity == "words":
        # Use find_all to get words with proper overlap and exclusion handling
        word_elements = self.find_all(
            "text", overlap=overlap, apply_exclusions=apply_exclusions
        )

        # Join the text from all matching words
        text_parts = []
        for word in word_elements:
            word_text = word.extract_text()
            if word_text:  # Skip empty strings
                text_parts.append(word_text)

        result = " ".join(text_parts)

        # Apply newlines processing if requested
        if newlines is False:
            result = result.replace("\n", " ").replace("\r", " ")
        elif isinstance(newlines, str):
            result = result.replace("\n", newlines).replace("\r", newlines)

        return result

    # Original character-level extraction logic follows...
    # 1. Get Word Elements potentially within this region (initial broad phase)
    # Optimization: Could use spatial query if page elements were indexed
    page_words = self.page.words  # Get all words from the page

    # 2. Gather all character dicts from words potentially in region
    # We filter precisely in filter_chars_spatially
    all_char_dicts = []
    for word in page_words:
        # Quick bbox check to avoid processing words clearly outside
        if get_bbox_overlap(self.bbox, word.bbox) is not None:
            all_char_dicts.extend(getattr(word, "_char_dicts", []))

    if not all_char_dicts:
        logger.debug(f"Region {self.bbox}: No character dicts found overlapping region bbox.")
        return ""

    # 3. Get Relevant Exclusions (overlapping this region)
    apply_exclusions_flag = kwargs.get("apply_exclusions", apply_exclusions)
    exclusion_regions = []
    if apply_exclusions_flag:
        all_page_exclusions = self._get_exclusion_regions(include_callable=True, debug=debug)
        overlapping_exclusions = []
        for excl in all_page_exclusions:
            if get_bbox_overlap(self.bbox, excl.bbox) is not None:
                overlapping_exclusions.append(excl)
        exclusion_regions = overlapping_exclusions
        if debug:
            logger.debug(
                f"Region {self.bbox}: Found {len(all_page_exclusions)} total exclusions, "
                f"{len(exclusion_regions)} overlapping this region."
            )
    elif debug:
        logger.debug(f"Region {self.bbox}: Not applying exclusions (apply_exclusions=False).")

    # Add boundary element exclusions if this is a section with boundary settings
    if hasattr(self, "_boundary_exclusions") and self._boundary_exclusions != "both":
        boundary_exclusions = []

        if self._boundary_exclusions == "none":
            # Exclude both start and end elements
            if hasattr(self, "start_element") and self.start_element:
                boundary_exclusions.append(self.start_element)
            if hasattr(self, "end_element") and self.end_element:
                boundary_exclusions.append(self.end_element)
        elif self._boundary_exclusions == "start":
            # Exclude only end element
            if hasattr(self, "end_element") and self.end_element:
                boundary_exclusions.append(self.end_element)
        elif self._boundary_exclusions == "end":
            # Exclude only start element
            if hasattr(self, "start_element") and self.start_element:
                boundary_exclusions.append(self.start_element)

        # Add boundary elements as exclusion regions
        for elem in boundary_exclusions:
            if hasattr(elem, "bbox"):
                exclusion_regions.append(elem)
                if debug:
                    logger.debug(
                        f"Adding boundary exclusion: {elem.extract_text().strip()} at {elem.bbox}"
                    )

    # 4. Spatially Filter Characters using Utility
    # Pass self as the target_region for precise polygon checks etc.
    filtered_chars = filter_chars_spatially(
        char_dicts=all_char_dicts,
        exclusion_regions=exclusion_regions,
        target_region=self,  # Pass self!
        debug=debug,
    )

    # 5. Generate Text Layout using Utility
    # Add content_filter to kwargs if provided
    final_kwargs = kwargs.copy()
    if content_filter is not None:
        final_kwargs["content_filter"] = content_filter

    result = generate_text_layout(
        char_dicts=filtered_chars,
        layout_context_bbox=self.bbox,  # Use region's bbox for context
        user_kwargs=final_kwargs,  # Pass kwargs including content_filter
    )

    # Flexible newline handling (same logic as TextElement)
    if isinstance(newlines, bool):
        if newlines is False:
            replacement = " "
        else:
            replacement = None
    else:
        replacement = str(newlines)

    if replacement is not None:
        result = result.replace("\n", replacement).replace("\r", replacement)

    logger.debug(f"Region {self.bbox}: extract_text finished, result length: {len(result)}.")
    return result

natural_pdf.Region.find(selector=None, *, text=None, overlap='full', apply_exclusions=True, regex=False, case=True, text_tolerance=None, auto_text_tolerance=None, reading_order=True, near_threshold=None, engine=None)

Find the first element in this region matching the selector OR text content.

Provide EITHER selector OR text, but not both.

Parameters:

Name	Type	Description	Default
selector	Optional[str]	CSS-like selector string.	None
text	Optional[Union[str, Sequence[str]]]	Text content to search for (equivalent to 'text:contains(...)'). Accepts a single string or an iterable of strings (matches any value).	None
overlap	Optional[str]	How to determine if elements overlap with the region: 'full' (fully inside), 'partial' (any overlap), or 'center' (center point inside). (default: "full")	'full'
apply_exclusions	bool	Whether to exclude elements in exclusion regions (default: True).	True
regex	bool	Whether to use regex for text search (selector or text) (default: False).	False
case	bool	Whether to do case-sensitive text search (selector or text) (default: True).	True
text_tolerance	Optional[Dict[str, Any]]	Optional mapping of pdfplumber-style tolerance overrides applied temporarily while evaluating the selector.	None
auto_text_tolerance	Optional[Union[bool, Dict[str, Any]]]	Optional overrides for automatic tolerance calculation.	None
reading_order	bool	Whether to return the first match according to natural reading order (default: True). When False the raw selector ordering is preserved.	True
near_threshold	Optional[float]	Maximum distance (in points) used by the :near pseudo-class.	None
engine	Optional[str]	Optional selector engine name registered with the selector provider.	None

Returns: First matching element or None.

Source code in natural_pdf/elements/region.py

def find(
    self,
    selector: Optional[str] = None,
    *,
    text: Optional[Union[str, Sequence[str]]] = None,
    overlap: Optional[str] = "full",
    apply_exclusions: bool = True,
    regex: bool = False,
    case: bool = True,
    text_tolerance: Optional[Dict[str, Any]] = None,
    auto_text_tolerance: Optional[Union[bool, Dict[str, Any]]] = None,
    reading_order: bool = True,
    near_threshold: Optional[float] = None,
    engine: Optional[str] = None,
) -> Optional["Element"]:
    """
    Find the first element in this region matching the selector OR text content.

    Provide EITHER `selector` OR `text`, but not both.

    Args:
        selector: CSS-like selector string.
        text: Text content to search for (equivalent to 'text:contains(...)'). Accepts a
              single string or an iterable of strings (matches any value).
        overlap: How to determine if elements overlap with the region: 'full' (fully inside),
                 'partial' (any overlap), or 'center' (center point inside).
                 (default: "full")
        apply_exclusions: Whether to exclude elements in exclusion regions (default: True).
        regex: Whether to use regex for text search (`selector` or `text`) (default: False).
        case: Whether to do case-sensitive text search (`selector` or `text`) (default: True).
        text_tolerance: Optional mapping of pdfplumber-style tolerance overrides applied
            temporarily while evaluating the selector.
        auto_text_tolerance: Optional overrides for automatic tolerance calculation.
        reading_order: Whether to return the first match according to natural reading order
            (default: True). When False the raw selector ordering is preserved.
        near_threshold: Maximum distance (in points) used by the ``:near`` pseudo-class.
        engine: Optional selector engine name registered with the selector provider.
    Returns:
        First matching element or None.
    """
    # Delegate validation and selector construction to find_all
    elements = self.find_all(
        selector=selector,
        text=text,
        overlap=overlap,
        apply_exclusions=apply_exclusions,
        regex=regex,
        case=case,
        text_tolerance=text_tolerance,
        auto_text_tolerance=auto_text_tolerance,
        reading_order=reading_order,
        near_threshold=near_threshold,
        engine=engine,
    )
    return elements.first if elements else None

natural_pdf.Region.find_all(selector=None, *, text=None, overlap='full', apply_exclusions=True, regex=False, case=True, text_tolerance=None, auto_text_tolerance=None, reading_order=True, near_threshold=None, engine=None)

Find all elements in this region matching the selector OR text content.

Provide EITHER selector OR text, but not both.

Parameters:

Name	Type	Description	Default
selector	Optional[str]	CSS-like selector string.	None
text	Optional[Union[str, Sequence[str]]]	Text content to search for (equivalent to 'text:contains(...)'). Accepts a single string or an iterable of strings (matches any value).	None
overlap	Optional[str]	How to determine if elements overlap with the region: 'full' (fully inside), 'partial' (any overlap), or 'center' (center point inside). (default: "full")	'full'
apply_exclusions	bool	Whether to exclude elements in exclusion regions (default: True).	True
regex	bool	Whether to use regex for text search (selector or text) (default: False).	False
case	bool	Whether to do case-sensitive text search (selector or text) (default: True).	True
text_tolerance	Optional[Dict[str, Any]]	Optional mapping of pdfplumber-style tolerance overrides applied temporarily while evaluating the selector.	None
auto_text_tolerance	Optional[Union[bool, Dict[str, Any]]]	Optional overrides for automatic tolerance calculation.	None
reading_order	bool	Whether to sort matches according to natural reading order (default: True).	True
near_threshold	Optional[float]	Maximum distance (in points) used by the :near pseudo-class.	None
engine	Optional[str]	Optional selector engine name registered with the selector provider.	None

Returns: ElementCollection with matching elements.

Source code in natural_pdf/elements/region.py

def find_all(
    self,
    selector: Optional[str] = None,
    *,
    text: Optional[Union[str, Sequence[str]]] = None,
    overlap: Optional[str] = "full",
    apply_exclusions: bool = True,
    regex: bool = False,
    case: bool = True,
    text_tolerance: Optional[Dict[str, Any]] = None,
    auto_text_tolerance: Optional[Union[bool, Dict[str, Any]]] = None,
    reading_order: bool = True,
    near_threshold: Optional[float] = None,
    engine: Optional[str] = None,
) -> "ElementCollection":
    """
    Find all elements in this region matching the selector OR text content.

    Provide EITHER `selector` OR `text`, but not both.

    Args:
        selector: CSS-like selector string.
        text: Text content to search for (equivalent to 'text:contains(...)'). Accepts a
              single string or an iterable of strings (matches any value).
        overlap: How to determine if elements overlap with the region: 'full' (fully inside),
                 'partial' (any overlap), or 'center' (center point inside).
                 (default: "full")
        apply_exclusions: Whether to exclude elements in exclusion regions (default: True).
        regex: Whether to use regex for text search (`selector` or `text`) (default: False).
        case: Whether to do case-sensitive text search (`selector` or `text`) (default: True).
        text_tolerance: Optional mapping of pdfplumber-style tolerance overrides applied
            temporarily while evaluating the selector.
        auto_text_tolerance: Optional overrides for automatic tolerance calculation.
        reading_order: Whether to sort matches according to natural reading order (default: True).
        near_threshold: Maximum distance (in points) used by the ``:near`` pseudo-class.
        engine: Optional selector engine name registered with the selector provider.
    Returns:
        ElementCollection with matching elements.
    """
    from natural_pdf.elements.element_collection import ElementCollection

    overlap_mode = (overlap or "full").lower()
    if overlap_mode not in {"full", "partial", "center"}:
        raise ValueError(
            f"Invalid overlap value: {overlap_mode}. Must be 'full', 'partial', or 'center'"
        )

    page_results = self.page.find_all(
        selector=selector,
        text=text,
        apply_exclusions=apply_exclusions,
        regex=regex,
        case=case,
        text_tolerance=text_tolerance,
        auto_text_tolerance=auto_text_tolerance,
        reading_order=reading_order,
        near_threshold=near_threshold,
        engine=engine,
    )

    if not page_results:
        return ElementCollection([])

    filtered = self._filter_elements_by_overlap_mode(
        page_results.elements,
        overlap_mode,
    )

    unique: List["Element"] = []
    seen_ids: set[int] = set()
    for element in filtered:
        marker = id(element)
        if marker in seen_ids:
            continue
        seen_ids.add(marker)
        unique.append(element)

    return ElementCollection(unique)

natural_pdf.Region.get_children(selector=None)

Get immediate child regions, optionally filtered by selector.

Parameters:

Name	Type	Description	Default
selector		Optional selector to filter children	None

Returns:

Type	Description
	List of child regions matching the selector

Source code in natural_pdf/elements/region.py

def get_children(self, selector=None):
    """
    Get immediate child regions, optionally filtered by selector.

    Args:
        selector: Optional selector to filter children

    Returns:
        List of child regions matching the selector
    """
    import logging

    logger = logging.getLogger("natural_pdf.elements.region")

    if selector is None:
        return self.child_regions

    # Use existing selector parser to filter
    try:
        selector_obj = parse_selector(selector)
        filter_func = selector_to_filter_func(selector_obj)  # Removed region=self
        matched = [child for child in self.child_regions if filter_func(child)]
        logger.debug(
            f"get_children: found {len(matched)} of {len(self.child_regions)} children matching '{selector}'"
        )
        return matched
    except Exception as e:
        logger.error(f"Error applying selector in get_children: {e}", exc_info=True)
        return []  # Return empty list on error

natural_pdf.Region.get_descendants(selector=None)

Get all descendant regions (children, grandchildren, etc.), optionally filtered by selector.

Parameters:

Name	Type	Description	Default
selector		Optional selector to filter descendants	None

Returns:

Type	Description
	List of descendant regions matching the selector

Source code in natural_pdf/elements/region.py

def get_descendants(self, selector=None):
    """
    Get all descendant regions (children, grandchildren, etc.), optionally filtered by selector.

    Args:
        selector: Optional selector to filter descendants

    Returns:
        List of descendant regions matching the selector
    """
    import logging

    logger = logging.getLogger("natural_pdf.elements.region")

    all_descendants = []
    queue = list(self.child_regions)  # Start with direct children

    while queue:
        current = queue.pop(0)
        all_descendants.append(current)
        # Add current's children to the queue for processing
        if hasattr(current, "child_regions"):
            queue.extend(current.child_regions)

    logger.debug(f"get_descendants: found {len(all_descendants)} total descendants")

    # Filter by selector if provided
    if selector is not None:
        try:
            selector_obj = parse_selector(selector)
            filter_func = selector_to_filter_func(selector_obj)  # Removed region=self
            matched = [desc for desc in all_descendants if filter_func(desc)]
            logger.debug(f"get_descendants: filtered to {len(matched)} matching '{selector}'")
            return matched
        except Exception as e:
            logger.error(f"Error applying selector in get_descendants: {e}", exc_info=True)
            return []  # Return empty list on error

    return all_descendants

natural_pdf.Region.get_elements(selector=None, apply_exclusions=True, **kwargs)

Get all elements within this region.

Parameters:

Name	Type	Description	Default
selector	Optional[str]	Optional selector to filter elements	None
apply_exclusions		Whether to apply exclusion regions	True
**kwargs		Additional parameters for element filtering	{}

Returns:

Type	Description
List['Element']	List of elements in the region

Source code in natural_pdf/elements/region.py

def get_elements(
    self, selector: Optional[str] = None, apply_exclusions=True, **kwargs
) -> List["Element"]:
    """
    Get all elements within this region.

    Args:
        selector: Optional selector to filter elements
        apply_exclusions: Whether to apply exclusion regions
        **kwargs: Additional parameters for element filtering

    Returns:
        List of elements in the region
    """
    if selector:
        # Find elements on the page matching the selector
        page_elements = self.page.find_all(
            selector, apply_exclusions=apply_exclusions, **kwargs
        )
        # Filter those elements to only include ones within this region
        elements = [e for e in page_elements if self._is_element_in_region(e)]
    else:
        # Get all elements from the page
        page_elements = self.page.get_elements(apply_exclusions=apply_exclusions)
        # Filter to elements in this region
        elements = [e for e in page_elements if self._is_element_in_region(e)]

    # Apply boundary exclusions if this is a section with boundary settings
    if hasattr(self, "_boundary_exclusions") and self._boundary_exclusions != "both":
        excluded_ids = set()

        if self._boundary_exclusions == "none":
            # Exclude both start and end elements
            if hasattr(self, "start_element") and self.start_element:
                excluded_ids.add(id(self.start_element))
            if hasattr(self, "end_element") and self.end_element:
                excluded_ids.add(id(self.end_element))
        elif self._boundary_exclusions == "start":
            # Exclude only end element
            if hasattr(self, "end_element") and self.end_element:
                excluded_ids.add(id(self.end_element))
        elif self._boundary_exclusions == "end":
            # Exclude only start element
            if hasattr(self, "start_element") and self.start_element:
                excluded_ids.add(id(self.start_element))

        if excluded_ids:
            elements = [e for e in elements if id(e) not in excluded_ids]

    return elements

natural_pdf.Region.get_section_between(start_element=None, end_element=None, include_boundaries='both', orientation='vertical')

Get a section between two elements within this region.

Parameters:

Name	Type	Description	Default
start_element		Element marking the start of the section	None
end_element		Element marking the end of the section	None
include_boundaries		How to include boundary elements: 'start', 'end', 'both', or 'none'	'both'
orientation		'vertical' (default) or 'horizontal' - determines section direction	'vertical'

Returns:

Type	Description
	Region representing the section

Source code in natural_pdf/elements/region.py

def get_section_between(
    self,
    start_element=None,
    end_element=None,
    include_boundaries="both",
    orientation="vertical",
):
    """
    Get a section between two elements within this region.

    Args:
        start_element: Element marking the start of the section
        end_element: Element marking the end of the section
        include_boundaries: How to include boundary elements: 'start', 'end', 'both', or 'none'
        orientation: 'vertical' (default) or 'horizontal' - determines section direction

    Returns:
        Region representing the section
    """
    # Get elements only within this region first
    elements = self.get_elements()

    if not elements:
        raise ValueError(f"Region {self.bbox}: no elements available for section extraction.")

    # Sort elements in reading order
    elements.sort(key=lambda e: (e.top, e.x0))

    if start_element and start_element not in elements:
        logger.debug("Start element not found in region, using first element.")
        start_element = elements[0]
    elif start_element is None:
        start_element = elements[0]

    if end_element and end_element not in elements:
        logger.debug("End element not found in region, using last element.")
        end_element = elements[-1]
    elif end_element is None:
        end_element = elements[-1]

    # Validate orientation parameter
    if orientation not in ["vertical", "horizontal"]:
        raise ValueError(f"orientation must be 'vertical' or 'horizontal', got '{orientation}'")

    # Use centralized section utilities
    from natural_pdf.utils.sections import calculate_section_bounds, validate_section_bounds

    # Calculate section boundaries
    bounds = calculate_section_bounds(
        start_element=start_element,
        end_element=end_element,
        include_boundaries=include_boundaries,
        orientation=orientation,
        parent_bounds=self.bbox,
    )

    # Validate boundaries
    if not validate_section_bounds(bounds, orientation):
        # Return an empty region at the start position
        x0, top, _, _ = bounds
        return Region(self.page, (x0, top, x0, top))

    # Create new region
    section = Region(self.page, bounds)

    # Store the original boundary elements and exclusion info
    section.start_element = start_element
    section.end_element = end_element
    section._boundary_exclusions = include_boundaries

    return section

natural_pdf.Region.get_sections(start_elements=None, end_elements=None, include_boundaries='both', orientation='vertical', **kwargs)

Get sections within this region based on start/end elements.

Parameters:

Name	Type	Description	Default
start_elements	Union[str, Sequence['Element'], 'ElementCollection', None]	Elements or selector string that mark the start of sections	None
end_elements	Union[str, Sequence['Element'], 'ElementCollection', None]	Elements or selector string that mark the end of sections	None
include_boundaries	str	How to include boundary elements: 'start', 'end', 'both', or 'none'	'both'
orientation	str	'vertical' (default) or 'horizontal' - determines section direction	'vertical'

Returns:

Type	Description
'ElementCollection[Region]'	List of Region objects representing the extracted sections

Source code in natural_pdf/elements/region.py

def get_sections(
    self,
    start_elements: Union[str, Sequence["Element"], "ElementCollection", None] = None,
    end_elements: Union[str, Sequence["Element"], "ElementCollection", None] = None,
    include_boundaries: str = "both",
    orientation: str = "vertical",
    **kwargs: Any,
) -> "ElementCollection[Region]":
    """
    Get sections within this region based on start/end elements.

    Args:
        start_elements: Elements or selector string that mark the start of sections
        end_elements: Elements or selector string that mark the end of sections
        include_boundaries: How to include boundary elements: 'start', 'end', 'both', or 'none'
        orientation: 'vertical' (default) or 'horizontal' - determines section direction

    Returns:
        List of Region objects representing the extracted sections
    """
    from natural_pdf.elements.element_collection import ElementCollection
    from natural_pdf.utils.sections import extract_sections_from_region

    def _normalize_section_boundary(
        arg: Union[str, Sequence["Element"], "ElementCollection", None]
    ) -> Union[str, List["Element"], None]:
        if arg is None or isinstance(arg, str):
            return arg

        if isinstance(arg, ElementCollection):
            return list(arg)

        if isinstance(arg, Sequence):
            return list(arg)

        # Fallback: wrap single element-like object
        return [arg]  # type: ignore[list-item]

    # Legacy tolerances (y_threshold/x_threshold) are now handled internally;
    # accept and ignore them for backward compatibility.
    legacy_thresholds = {"y_threshold", "x_threshold"}
    for legacy_key in list(kwargs.keys()):
        if legacy_key in legacy_thresholds:
            kwargs.pop(legacy_key, None)

    # Use centralized section extraction logic
    sections = extract_sections_from_region(
        region=self,
        start_elements=_normalize_section_boundary(start_elements),
        end_elements=_normalize_section_boundary(end_elements),
        include_boundaries=include_boundaries,
        orientation=orientation,
        **kwargs,
    )

    return ElementCollection(sections)

natural_pdf.Region.get_text_table_cells(snap_tolerance=10, join_tolerance=3, min_words_vertical=3, min_words_horizontal=1, intersection_tolerance=3, expand_bbox=None, **kwargs)

Analyzes text alignment to find table cells and returns them as temporary Region objects without adding them to the page.

Parameters:

Name	Type	Description	Default
snap_tolerance	int	Tolerance for snapping parallel lines.	10
join_tolerance	int	Tolerance for joining collinear lines.	3
min_words_vertical	int	Minimum words needed to define a vertical line.	3
min_words_horizontal	int	Minimum words needed to define a horizontal line.	1
intersection_tolerance	int	Tolerance for detecting line intersections.	3
expand_bbox	Optional[Dict[str, int]]	Optional dictionary to expand the search area slightly beyond the region's exact bounds (e.g., {'left': 5, 'right': 5}).	None
**kwargs		Additional keyword arguments passed to find_text_based_tables (e.g., specific x/y tolerances).	{}

Returns:

Type	Description
'ElementCollection[Region]'	An ElementCollection containing temporary Region objects for each detected cell,
'ElementCollection[Region]'	or an empty ElementCollection if no cells are found or an error occurs.

Source code in natural_pdf/elements/region.py

def get_text_table_cells(
    self,
    snap_tolerance: int = 10,
    join_tolerance: int = 3,
    min_words_vertical: int = 3,
    min_words_horizontal: int = 1,
    intersection_tolerance: int = 3,
    expand_bbox: Optional[Dict[str, int]] = None,
    **kwargs,
) -> "ElementCollection[Region]":
    """
    Analyzes text alignment to find table cells and returns them as
    temporary Region objects without adding them to the page.

    Args:
        snap_tolerance: Tolerance for snapping parallel lines.
        join_tolerance: Tolerance for joining collinear lines.
        min_words_vertical: Minimum words needed to define a vertical line.
        min_words_horizontal: Minimum words needed to define a horizontal line.
        intersection_tolerance: Tolerance for detecting line intersections.
        expand_bbox: Optional dictionary to expand the search area slightly beyond
                     the region's exact bounds (e.g., {'left': 5, 'right': 5}).
        **kwargs: Additional keyword arguments passed to
                  find_text_based_tables (e.g., specific x/y tolerances).

    Returns:
        An ElementCollection containing temporary Region objects for each detected cell,
        or an empty ElementCollection if no cells are found or an error occurs.
    """
    from natural_pdf.elements.element_collection import ElementCollection

    # 1. Perform the analysis (or use cached results)
    if "text_table_structure" in self.analyses:
        analysis_results = self.analyses["text_table_structure"]
        logger.debug("get_text_table_cells: Using cached analysis results.")
    else:
        analysis_results = self.analyze_text_table_structure(
            snap_tolerance=snap_tolerance,
            join_tolerance=join_tolerance,
            min_words_vertical=min_words_vertical,
            min_words_horizontal=min_words_horizontal,
            intersection_tolerance=intersection_tolerance,
            expand_bbox=expand_bbox,
            **kwargs,
        )

    # 2. Check if analysis was successful and cells were found
    if analysis_results is None or not analysis_results.get("cells"):
        logger.info(f"Region {self.bbox}: No cells found by text table analysis.")
        return ElementCollection([])  # Return empty collection

    # 3. Create temporary Region objects for each cell dictionary
    cell_regions = []
    for cell_data in analysis_results["cells"]:
        cell_region = self.page.region(**cell_data)
        cell_region.region_type = "table-cell"
        cell_region.normalized_type = "table-cell"
        cell_region.model = "pdfplumber-text"
        cell_region.source = "volatile"
        cell_region.parent_region = self
        cell_regions.append(cell_region)

    # 4. Return the list wrapped in an ElementCollection
    logger.debug(f"get_text_table_cells: Created {len(cell_regions)} temporary cell regions.")
    return ElementCollection(cell_regions)

natural_pdf.Region.highlight(label=None, color=None, use_color_cycling=False, annotate=None, existing='append')

Highlight this region on the page.

Parameters:

Name	Type	Description	Default
label	Optional[str]	Optional label for the highlight	None
color	Optional[Union[Tuple, str]]	Color tuple/string for the highlight, or None to use automatic color	None
use_color_cycling	bool	Force color cycling even with no label (default: False)	False
annotate	Optional[List[str]]	List of attribute names to display on the highlight (e.g., ['confidence', 'type'])	None
existing	str	How to handle existing highlights ('append' or 'replace').	'append'

Returns:

Type	Description
None	None

Source code in natural_pdf/elements/region.py

def highlight(
    self,
    label: Optional[str] = None,
    color: Optional[Union[Tuple, str]] = None,
    use_color_cycling: bool = False,
    annotate: Optional[List[str]] = None,
    existing: str = "append",
) -> None:
    """
    Highlight this region on the page.

    Args:
        label: Optional label for the highlight
        color: Color tuple/string for the highlight, or None to use automatic color
        use_color_cycling: Force color cycling even with no label (default: False)
        annotate: List of attribute names to display on the highlight (e.g., ['confidence', 'type'])
        existing: How to handle existing highlights ('append' or 'replace').

    Returns:
        None
    """
    # Access the highlighter service correctly
    highlighter = self.page._highlighter

    # Prepare common arguments
    highlight_args = {
        "page_index": self.page.index,
        "color": color,
        "label": label,
        "use_color_cycling": use_color_cycling,
        "element": self,  # Pass the region itself so attributes can be accessed
        "annotate": annotate,
        "existing": existing,
    }

    # Call the appropriate service method
    if self.has_polygon:
        highlight_args["polygon"] = self.polygon
        highlighter.add_polygon(**highlight_args)
    else:
        highlight_args["bbox"] = self.bbox
        highlighter.add(**highlight_args)

    return None

natural_pdf.Region.left(width=None, height='element', include_source=False, until=None, include_endpoint=True, offset=None, apply_exclusions=True, multipage=None, within=None, anchor='start', **kwargs)

Select region to the left of this region.

Parameters:

Name	Type	Description	Default
width	Optional[float]	Width of the region to the left, in points	None
height	str	Height mode - "full" for full page height or "element" for element height	'element'
include_source	bool	Whether to include this region in the result (default: False)	False
until	Optional[str]	Optional selector string to specify a left boundary element	None
include_endpoint	bool	Whether to include the boundary element in the region (default: True)	True
offset	Optional[float]	Pixel offset when excluding source/endpoint (default: None, uses natural_pdf.options.layout.directional_offset)	None
multipage	Optional[bool]	Override global multipage behaviour; defaults to None meaning use global option.	None
**kwargs		Additional parameters	{}

Returns:

Type	Description
Union['Region', 'FlowRegion']	Region object representing the area to the left

Source code in natural_pdf/elements/region.py

def left(
    self,
    width: Optional[float] = None,
    height: str = "element",
    include_source: bool = False,
    until: Optional[str] = None,
    include_endpoint: bool = True,
    offset: Optional[float] = None,
    apply_exclusions: bool = True,
    multipage: Optional[bool] = None,
    within: Optional["Region"] = None,
    anchor: str = "start",
    **kwargs,
) -> Union["Region", "FlowRegion"]:
    """
    Select region to the left of this region.

    Args:
        width: Width of the region to the left, in points
        height: Height mode - "full" for full page height or "element" for element height
        include_source: Whether to include this region in the result (default: False)
        until: Optional selector string to specify a left boundary element
        include_endpoint: Whether to include the boundary element in the region (default: True)
        offset: Pixel offset when excluding source/endpoint (default: None, uses natural_pdf.options.layout.directional_offset)
        multipage: Override global multipage behaviour; defaults to None meaning use global option.
        **kwargs: Additional parameters

    Returns:
        Region object representing the area to the left
    """
    return self._direction(
        direction="left",
        size=width,
        cross_size=height,
        include_source=include_source,
        until=until,
        include_endpoint=include_endpoint,
        offset=offset,
        apply_exclusions=apply_exclusions,
        multipage=multipage,
        within=within,
        anchor=anchor,
        **kwargs,
    )

natural_pdf.Region.region(left=None, top=None, right=None, bottom=None, width=None, height=None, relative=False)

Create a sub-region within this region using the same API as Page.region().

By default, coordinates are absolute (relative to the page), matching Page.region(). Set relative=True to use coordinates relative to this region's top-left corner.

Parameters:

Name	Type	Description	Default
left	Optional[float]	Left x-coordinate (absolute by default, or relative to region if relative=True)	None
top	Optional[float]	Top y-coordinate (absolute by default, or relative to region if relative=True)	None
right	Optional[float]	Right x-coordinate (absolute by default, or relative to region if relative=True)	None
bottom	Optional[float]	Bottom y-coordinate (absolute by default, or relative to region if relative=True)	None
width	Union[str, float, None]	Width definition (same as Page.region())	None
height	Optional[float]	Height of the region (same as Page.region())	None
relative	bool	If True, coordinates are relative to this region's top-left (0,0). If False (default), coordinates are absolute page coordinates.	False

Returns:

Type	Description
'Region'	Region object for the specified coordinates, clipped to this region's bounds

Examples:

Absolute coordinates (default) - same as page.region()

sub = region.region(left=100, top=200, width=50, height=30)

Relative to region's top-left

sub = region.region(left=10, top=10, width=50, height=30, relative=True)

Mix relative positioning with this region's bounds

sub = region.region(left=region.x0 + 10, width=50, height=30)

Source code in natural_pdf/elements/region.py

def region(
    self,
    left: Optional[float] = None,
    top: Optional[float] = None,
    right: Optional[float] = None,
    bottom: Optional[float] = None,
    width: Union[str, float, None] = None,
    height: Optional[float] = None,
    relative: bool = False,
) -> "Region":
    """
    Create a sub-region within this region using the same API as Page.region().

    By default, coordinates are absolute (relative to the page), matching Page.region().
    Set relative=True to use coordinates relative to this region's top-left corner.

    Args:
        left: Left x-coordinate (absolute by default, or relative to region if relative=True)
        top: Top y-coordinate (absolute by default, or relative to region if relative=True)
        right: Right x-coordinate (absolute by default, or relative to region if relative=True)
        bottom: Bottom y-coordinate (absolute by default, or relative to region if relative=True)
        width: Width definition (same as Page.region())
        height: Height of the region (same as Page.region())
        relative: If True, coordinates are relative to this region's top-left (0,0).
                 If False (default), coordinates are absolute page coordinates.

    Returns:
        Region object for the specified coordinates, clipped to this region's bounds

    Examples:
        # Absolute coordinates (default) - same as page.region()
        sub = region.region(left=100, top=200, width=50, height=30)

        # Relative to region's top-left
        sub = region.region(left=10, top=10, width=50, height=30, relative=True)

        # Mix relative positioning with this region's bounds
        sub = region.region(left=region.x0 + 10, width=50, height=30)
    """
    # If relative coordinates requested, convert to absolute
    if relative:
        left = (self.x0 + left) if left is not None else None
        top = (self.top + top) if top is not None else None
        right = (self.x0 + right) if right is not None else None
        bottom = (self.top + bottom) if bottom is not None else None

        # For numeric width/height with relative coords, we need to handle the calculation
        # in the context of absolute positioning

    # Use the parent page's region method to create the region with all its logic
    region_kwargs: Dict[str, Any] = {}
    if left is not None:
        region_kwargs["left"] = left
    if top is not None:
        region_kwargs["top"] = top
    if right is not None:
        region_kwargs["right"] = right
    if bottom is not None:
        region_kwargs["bottom"] = bottom
    if width is not None:
        region_kwargs["width"] = width
    if height is not None:
        region_kwargs["height"] = height

    new_region = self.page.region(**region_kwargs)

    # Clip the new region to this region's bounds
    return new_region.clip(self)

natural_pdf.Region.right(width=None, height='element', include_source=False, until=None, include_endpoint=True, offset=None, apply_exclusions=True, multipage=None, within=None, anchor='start', **kwargs)

Select region to the right of this region.

Parameters:

Name	Type	Description	Default
width	Optional[float]	Width of the region to the right, in points	None
height	str	Height mode - "full" for full page height or "element" for element height	'element'
include_source	bool	Whether to include this region in the result (default: False)	False
until	Optional[str]	Optional selector string to specify a right boundary element	None
include_endpoint	bool	Whether to include the boundary element in the region (default: True)	True
offset	Optional[float]	Pixel offset when excluding source/endpoint (default: None, uses natural_pdf.options.layout.directional_offset)	None
multipage	Optional[bool]	Override global multipage behaviour; defaults to None meaning use global option.	None
**kwargs		Additional parameters	{}

Returns:

Type	Description
Union['Region', 'FlowRegion']	Region object representing the area to the right

Source code in natural_pdf/elements/region.py

def right(
    self,
    width: Optional[float] = None,
    height: str = "element",
    include_source: bool = False,
    until: Optional[str] = None,
    include_endpoint: bool = True,
    offset: Optional[float] = None,
    apply_exclusions: bool = True,
    multipage: Optional[bool] = None,
    within: Optional["Region"] = None,
    anchor: str = "start",
    **kwargs,
) -> Union["Region", "FlowRegion"]:
    """
    Select region to the right of this region.

    Args:
        width: Width of the region to the right, in points
        height: Height mode - "full" for full page height or "element" for element height
        include_source: Whether to include this region in the result (default: False)
        until: Optional selector string to specify a right boundary element
        include_endpoint: Whether to include the boundary element in the region (default: True)
        offset: Pixel offset when excluding source/endpoint (default: None, uses natural_pdf.options.layout.directional_offset)
        multipage: Override global multipage behaviour; defaults to None meaning use global option.
        **kwargs: Additional parameters

    Returns:
        Region object representing the area to the right
    """
    return self._direction(
        direction="right",
        size=width,
        cross_size=height,
        include_source=include_source,
        until=until,
        include_endpoint=include_endpoint,
        offset=offset,
        apply_exclusions=apply_exclusions,
        multipage=multipage,
        within=within,
        anchor=anchor,
        **kwargs,
    )

natural_pdf.Region.save(filename, resolution=None, labels=True, legend_position='right')

Save the page with this region highlighted to an image file.

Parameters:

Name	Type	Description	Default
filename	str	Path to save the image to	required
resolution	Optional[float]	Resolution in DPI for rendering (default: uses global options, fallback to 144 DPI)	None
labels	bool	Whether to include a legend for labels	True
legend_position	str	Position of the legend	'right'

Returns:

Type	Description
'Region'	Self for method chaining

Source code in natural_pdf/elements/region.py

def save(
    self,
    filename: str,
    resolution: Optional[float] = None,
    labels: bool = True,
    legend_position: str = "right",
) -> "Region":
    """
    Save the page with this region highlighted to an image file.

    Args:
        filename: Path to save the image to
        resolution: Resolution in DPI for rendering (default: uses global options, fallback to 144 DPI)
        labels: Whether to include a legend for labels
        legend_position: Position of the legend

    Returns:
        Self for method chaining
    """
    image_options = _image_options()
    if resolution is None:
        default_resolution = image_options.resolution
        resolution = float(default_resolution) if default_resolution is not None else 144.0
    else:
        resolution = float(resolution)

    # Highlight this region if not already highlighted
    self.highlight()

    # Save the highlighted image
    self._page.save_image(
        filename, resolution=resolution, labels=labels, legend_position=legend_position
    )
    return self

natural_pdf.Region.save_image(filename, resolution=None, crop=False, include_highlights=True, **kwargs)

Save an image of just this region to a file.

Parameters:

Name	Type	Description	Default
filename	str	Path to save the image to	required
resolution	Optional[float]	Resolution in DPI for rendering (default: uses global options, fallback to 144 DPI)	None
crop	bool	If True, only crop the region without highlighting its boundaries	False
include_highlights	bool	Whether to include existing highlights (default: True)	True
**kwargs		Additional parameters for rendering	{}

Returns:

Type	Description
'Region'	Self for method chaining

Source code in natural_pdf/elements/region.py

def save_image(
    self,
    filename: str,
    resolution: Optional[float] = None,
    crop: bool = False,
    include_highlights: bool = True,
    **kwargs,
) -> "Region":
    """
    Save an image of just this region to a file.

    Args:
        filename: Path to save the image to
        resolution: Resolution in DPI for rendering (default: uses global options, fallback to 144 DPI)
        crop: If True, only crop the region without highlighting its boundaries
        include_highlights: Whether to include existing highlights (default: True)
        **kwargs: Additional parameters for rendering

    Returns:
        Self for method chaining
    """
    image_options = _image_options()
    if resolution is None:
        default_resolution = image_options.resolution
        resolution = float(default_resolution) if default_resolution is not None else 144.0
    else:
        resolution = float(resolution)

    # Use export() to save the image
    if include_highlights:
        # With highlights, use export() which includes them
        self.export(
            path=filename,
            resolution=resolution,
            crop=crop,
            **kwargs,
        )
    else:
        # Without highlights, use render() and save manually
        image = self.render(resolution=resolution, crop=crop, **kwargs)
        if image is not None:
            image.save(filename)
        else:
            logger.error(f"Failed to render region image for saving to {filename}")

    return self

natural_pdf.Region.split(divider, **kwargs)

Divide this region into sections based on the provided divider elements.

Parameters:

Name	Type	Description	Default
divider		Elements or selector string that mark section boundaries	required
**kwargs		Additional parameters passed to get_sections() - include_boundaries: How to include boundary elements (default: 'start') - orientation: 'vertical' or 'horizontal' (default: 'vertical')	{}

Returns:

Type	Description
'ElementCollection[Region]'	ElementCollection of Region objects representing the sections

Example

Split a region by bold text

sections = region.split("text:bold")

Split horizontally by vertical lines

sections = region.split("line[orientation=vertical]", orientation="horizontal")

Source code in natural_pdf/elements/region.py

def split(self, divider, **kwargs) -> "ElementCollection[Region]":
    """
    Divide this region into sections based on the provided divider elements.

    Args:
        divider: Elements or selector string that mark section boundaries
        **kwargs: Additional parameters passed to get_sections()
            - include_boundaries: How to include boundary elements (default: 'start')
            - orientation: 'vertical' or 'horizontal' (default: 'vertical')

    Returns:
        ElementCollection of Region objects representing the sections

    Example:
        # Split a region by bold text
        sections = region.split("text:bold")

        # Split horizontally by vertical lines
        sections = region.split("line[orientation=vertical]", orientation="horizontal")
    """
    # Default to 'start' boundaries for split (include divider at start of each section)
    if "include_boundaries" not in kwargs:
        kwargs["include_boundaries"] = "start"

    sections = self.get_sections(start_elements=divider, **kwargs)

    # Add section before first divider if there's content
    if sections and hasattr(sections[0], "start_element"):
        first_divider = sections[0].start_element
        if first_divider:
            # Get all elements before the first divider
            all_elements = self.get_elements()
            if all_elements and all_elements[0] != first_divider:
                # Create section from start to just before first divider
                initial_section = self.get_section_between(
                    start_element=None,
                    end_element=first_divider,
                    include_boundaries="none",
                    orientation=kwargs.get("orientation", "vertical"),
                )
                if initial_section and initial_section.get_elements():
                    sections.insert(0, initial_section)

    return sections

natural_pdf.Region.to_region()

Regions already satisfy the section surface; return self.

Source code in natural_pdf/elements/region.py

def to_region(self) -> "Region":
    """Regions already satisfy the section surface; return self."""
    return self

natural_pdf.Region.to_text_element(text_content=None, source_label='derived_from_region', object_type='word', default_font_size=10.0, default_font_name='RegionContent', confidence=None, add_to_page=False)

Creates a new TextElement object based on this region's geometry.

The text for the new TextElement can be provided directly, generated by a callback function, or left as None.

Parameters:

Name	Type	Description	Default
text_content	Optional[Union[str, Callable[['Region'], Optional[str]]]]	If a string, this will be the text of the new TextElement. If a callable, it will be called with this region instance and its return value (a string or None) will be the text. If None (default), the TextElement's text will be None.	None
source_label	str	The 'source' attribute for the new TextElement.	'derived_from_region'
object_type	str	The 'object_type' for the TextElement's data dict (e.g., "word", "char").	'word'
default_font_size	float	Placeholder font size if text is generated.	10.0
default_font_name	str	Placeholder font name if text is generated.	'RegionContent'
confidence	Optional[float]	Confidence score for the text. If text_content is None, defaults to 0.0. If text is provided/generated, defaults to 1.0 unless specified.	None
add_to_page	bool	If True, the created TextElement will be added to the region's parent page. (Default: False)	False

Returns:

Type	Description
'TextElement'	A new TextElement instance.

Raises:

Type	Description
ValueError	If the region does not have a valid 'page' attribute.

Source code in natural_pdf/elements/region.py

def to_text_element(
    self,
    text_content: Optional[Union[str, Callable[["Region"], Optional[str]]]] = None,
    source_label: str = "derived_from_region",
    object_type: str = "word",  # Or "char", controls how it's categorized
    default_font_size: float = 10.0,
    default_font_name: str = "RegionContent",
    confidence: Optional[float] = None,  # Allow overriding confidence
    add_to_page: bool = False,  # NEW: Option to add to page
) -> "TextElement":
    """
    Creates a new TextElement object based on this region's geometry.

    The text for the new TextElement can be provided directly,
    generated by a callback function, or left as None.

    Args:
        text_content:
            - If a string, this will be the text of the new TextElement.
            - If a callable, it will be called with this region instance
              and its return value (a string or None) will be the text.
            - If None (default), the TextElement's text will be None.
        source_label: The 'source' attribute for the new TextElement.
        object_type: The 'object_type' for the TextElement's data dict
                     (e.g., "word", "char").
        default_font_size: Placeholder font size if text is generated.
        default_font_name: Placeholder font name if text is generated.
        confidence: Confidence score for the text. If text_content is None,
                    defaults to 0.0. If text is provided/generated, defaults to 1.0
                    unless specified.
        add_to_page: If True, the created TextElement will be added to the
                     region's parent page. (Default: False)

    Returns:
        A new TextElement instance.

    Raises:
        ValueError: If the region does not have a valid 'page' attribute.
    """
    actual_text: Optional[str] = None
    if isinstance(text_content, str):
        actual_text = text_content
    elif callable(text_content):
        try:
            actual_text = text_content(self)
        except Exception as e:
            logger.error(
                f"Error executing text_content callback for region {self.bbox}: {e}",
                exc_info=True,
            )
            actual_text = None  # Ensure actual_text is None on error

    final_confidence = confidence
    if final_confidence is None:
        final_confidence = 1.0 if actual_text is not None and actual_text.strip() else 0.0

    if not hasattr(self, "page") or self.page is None:
        raise ValueError("Region must have a valid 'page' attribute to create a TextElement.")

    # Create character dictionaries for the text
    char_dicts = []
    if actual_text:
        # Create a single character dict that spans the entire region
        # This is a simplified approach - OCR engines typically create one per character
        char_dict = {
            "text": actual_text,
            "x0": self.x0,
            "top": self.top,
            "x1": self.x1,
            "bottom": self.bottom,
            "width": self.width,
            "height": self.height,
            "object_type": "char",
            "page_number": self.page.page_number,
            "fontname": default_font_name,
            "size": default_font_size,
            "upright": True,
            "direction": 1,
            "adv": self.width,
            "source": source_label,
            "confidence": final_confidence,
            "stroking_color": (0, 0, 0),
            "non_stroking_color": (0, 0, 0),
        }
        char_dicts.append(char_dict)

    elem_data = {
        "text": actual_text,
        "x0": self.x0,
        "top": self.top,
        "x1": self.x1,
        "bottom": self.bottom,
        "width": self.width,
        "height": self.height,
        "object_type": object_type,
        "page_number": self.page.page_number,
        "stroking_color": getattr(self, "stroking_color", (0, 0, 0)),
        "non_stroking_color": getattr(self, "non_stroking_color", (0, 0, 0)),
        "fontname": default_font_name,
        "size": default_font_size,
        "upright": True,
        "direction": 1,
        "adv": self.width,
        "source": source_label,
        "confidence": final_confidence,
        "_char_dicts": char_dicts,
    }
    text_element = TextElement(elem_data, self.page)

    if add_to_page:
        add_as_type = (
            "words"
            if object_type == "word"
            else "chars" if object_type == "char" else object_type
        )
        if hasattr(self.page, "add_element"):
            self.page.add_element(text_element, element_type=add_as_type)
            logger.debug(
                "TextElement created from region %s and added to page %s as %s.",
                self.bbox,
                getattr(self.page, "page_number", "N/A"),
                add_as_type,
            )
            if char_dicts and object_type == "word":
                for char_dict in char_dicts:
                    self.page.add_element(char_dict, element_type="chars")
        else:
            page_num_str = (
                str(self.page.page_number) if hasattr(self.page, "page_number") else "N/A"
            )
            raise AttributeError(
                f"Cannot add TextElement to page {page_num_str}: page does not expose add_element"
            )

    return text_element

natural_pdf.Region.trim(padding=1, threshold=0.95, resolution=None, pre_shrink=0.5)

Trim visual whitespace from the edges of this region.

Similar to Python's string .strip() method, but for visual whitespace in the region image. Uses pixel analysis to detect rows/columns that are predominantly whitespace.

Parameters:

Name	Type	Description	Default
padding	int	Number of pixels to keep as padding after trimming (default: 1)	1
threshold	float	Threshold for considering a row/column as whitespace (0.0-1.0, default: 0.95) Higher values mean more strict whitespace detection. E.g., 0.95 means if 95% of pixels in a row/column are white, consider it whitespace.	0.95
resolution	Optional[float]	Resolution for image rendering in DPI (default: uses global options, fallback to 144 DPI)	None
pre_shrink	float	Amount to shrink region before trimming, then expand back after (default: 0.5) This helps avoid detecting box borders/slivers as content.	0.5

Returns

New Region with visual whitespace trimmed from all edges

Examples

# Basic trimming with 1 pixel padding and 0.5px pre-shrink
trimmed = region.trim()

# More aggressive trimming with no padding and no pre-shrink
tight = region.trim(padding=0, threshold=0.9, pre_shrink=0)

# Conservative trimming with more padding
loose = region.trim(padding=3, threshold=0.98)

Source code in natural_pdf/elements/region.py

def trim(
    self,
    padding: int = 1,
    threshold: float = 0.95,
    resolution: Optional[float] = None,
    pre_shrink: float = 0.5,
) -> "Region":
    """
    Trim visual whitespace from the edges of this region.

    Similar to Python's string .strip() method, but for visual whitespace in the region image.
    Uses pixel analysis to detect rows/columns that are predominantly whitespace.

    Args:
        padding: Number of pixels to keep as padding after trimming (default: 1)
        threshold: Threshold for considering a row/column as whitespace (0.0-1.0, default: 0.95)
                  Higher values mean more strict whitespace detection.
                  E.g., 0.95 means if 95% of pixels in a row/column are white, consider it whitespace.
        resolution: Resolution for image rendering in DPI (default: uses global options, fallback to 144 DPI)
        pre_shrink: Amount to shrink region before trimming, then expand back after (default: 0.5)
                   This helps avoid detecting box borders/slivers as content.

    Returns
    ------

    New Region with visual whitespace trimmed from all edges

    Examples
    --------

    ```python
    # Basic trimming with 1 pixel padding and 0.5px pre-shrink
    trimmed = region.trim()

    # More aggressive trimming with no padding and no pre-shrink
    tight = region.trim(padding=0, threshold=0.9, pre_shrink=0)

    # Conservative trimming with more padding
    loose = region.trim(padding=3, threshold=0.98)
    ```
    """
    image_options = _image_options()
    if resolution is None:
        default_resolution = image_options.resolution
        resolution = float(default_resolution) if default_resolution is not None else 144.0
    else:
        resolution = float(resolution)

    # Pre-shrink the region to avoid box slivers
    work_region = (
        self.expand(left=-pre_shrink, right=-pre_shrink, top=-pre_shrink, bottom=-pre_shrink)
        if pre_shrink > 0
        else self
    )

    # Get the region image
    # Use render() for clean image without highlights, with cropping
    image = work_region.render(resolution=resolution, crop=True)
    if image is None:
        raise RuntimeError(f"Region {self.bbox}: render() returned None during trimming.")

    # Convert to grayscale for easier analysis
    import numpy as np

    # Convert PIL image to numpy array
    img_array = np.array(image.convert("L"))  # Convert to grayscale
    height, width = img_array.shape

    if height == 0 or width == 0:
        raise ValueError(f"Region {self.bbox}: rendered image has zero dimensions.")

    # Normalize pixel values to 0-1 range (255 = white = 1.0, 0 = black = 0.0)
    normalized = img_array.astype(np.float32) / 255.0

    # Find content boundaries by analyzing row and column averages

    # Analyze rows (horizontal strips) to find top and bottom boundaries
    row_averages = np.mean(normalized, axis=1)  # Average each row
    content_rows = row_averages < threshold  # True where there's content (not whitespace)

    # Find first and last rows with content
    content_row_indices = np.where(content_rows)[0]
    if len(content_row_indices) == 0:
        # No content found, return a minimal region at the center
        raise ValueError(f"Region {self.bbox}: no content detected during trimming.")

    top_content_row = max(0, content_row_indices[0] - padding)
    bottom_content_row = min(height - 1, content_row_indices[-1] + padding)

    # Analyze columns (vertical strips) to find left and right boundaries
    col_averages = np.mean(normalized, axis=0)  # Average each column
    content_cols = col_averages < threshold  # True where there's content

    content_col_indices = np.where(content_cols)[0]
    if len(content_col_indices) == 0:
        # No content found in columns either
        raise ValueError(f"Region {self.bbox}: no column content detected during trimming.")

    left_content_col = max(0, content_col_indices[0] - padding)
    right_content_col = min(width - 1, content_col_indices[-1] + padding)

    # Convert trimmed pixel coordinates back to PDF coordinates
    scale_factor = resolution / 72.0  # Scale factor used in render()

    # Calculate new PDF coordinates and ensure they are Python floats
    trimmed_x0 = float(work_region.x0 + (left_content_col / scale_factor))
    trimmed_top = float(work_region.top + (top_content_row / scale_factor))
    trimmed_x1 = float(
        work_region.x0 + ((right_content_col + 1) / scale_factor)
    )  # +1 because we want inclusive right edge
    trimmed_bottom = float(
        work_region.top + ((bottom_content_row + 1) / scale_factor)
    )  # +1 because we want inclusive bottom edge

    # Ensure the trimmed region doesn't exceed the work region boundaries
    final_x0 = max(work_region.x0, trimmed_x0)
    final_top = max(work_region.top, trimmed_top)
    final_x1 = min(work_region.x1, trimmed_x1)
    final_bottom = min(work_region.bottom, trimmed_bottom)

    # Ensure valid coordinates (width > 0, height > 0)
    if final_x1 <= final_x0 or final_bottom <= final_top:
        raise ValueError(f"Region {self.bbox}: trimming produced invalid dimensions.")

    # Create the trimmed region
    trimmed_region: Region = Region(self.page, (final_x0, final_top, final_x1, final_bottom))

    # Expand back by the pre_shrink amount to restore original positioning
    if pre_shrink > 0:
        trimmed_region = cast(
            Region,
            trimmed_region.expand(
                left=pre_shrink, right=pre_shrink, top=pre_shrink, bottom=pre_shrink
            ),
        )

    # Copy relevant metadata
    trimmed_region.region_type = self.region_type
    trimmed_region.normalized_type = self.normalized_type
    trimmed_region.confidence = self.confidence
    trimmed_region.model = self.model
    trimmed_region.name = self.name
    trimmed_region.label = self.label
    trimmed_region.source = "trimmed"  # Indicate this is a derived region
    trimmed_region.parent_region = self

    logger.debug(
        f"Region {self.bbox}: Trimmed to {trimmed_region.bbox} (padding={padding}, threshold={threshold}, pre_shrink={pre_shrink})"
    )
    return trimmed_region

natural_pdf.Region.update_text(transform, *, selector='text', apply_exclusions=False)

Apply transform to every text element matched by selector inside this region.

The heavy lifting is delegated to :py:meth:TextMixin.update_text; this override simply ensures the search is scoped to the region.

Source code in natural_pdf/elements/region.py

def update_text(
    self,
    transform: Callable[[Any], Optional[str]],
    *,
    selector: str = "text",
    apply_exclusions: bool = False,
) -> "Region":
    """Apply *transform* to every text element matched by *selector* inside this region.

    The heavy lifting is delegated to :py:meth:`TextMixin.update_text`; this
    override simply ensures the search is scoped to the region.
    """

    TextMixin.update_text(self, transform, selector=selector, apply_exclusions=apply_exclusions)
    return self

natural_pdf.Region.viewer(*, resolution=150, include_chars=False, include_attributes=None)

Create an interactive ipywidget viewer for this specific region.

The method renders the region to an image (cropped to the region bounds) and overlays all elements that intersect the region (optionally excluding noisy character-level elements). The resulting widget offers the same zoom / pan experience as :py:meth:Page.viewer but scoped to the region.

Parameters

resolution : int, default 150 Rendering resolution (DPI). This should match the value used by the page-level viewer so element scaling is accurate. include_chars : bool, default False Whether to include individual char elements in the overlay. These are often too dense for a meaningful visualisation so are skipped by default. include_attributes : list[str], optional Additional element attributes to expose in the info panel (on top of the default set used by the page viewer).

Returns

InteractiveViewerWidgetType | None The widget instance, or None if ipywidgets is not installed or an error occurred during creation.

Source code in natural_pdf/elements/region.py

def viewer(
    self,
    *,
    resolution: int = 150,
    include_chars: bool = False,
    include_attributes: Optional[List[str]] = None,
) -> Optional[Any]:
    """Create an interactive ipywidget viewer for **this specific region**.

    The method renders the region to an image (cropped to the region bounds) and
    overlays all elements that intersect the region (optionally excluding noisy
    character-level elements).  The resulting widget offers the same zoom / pan
    experience as :py:meth:`Page.viewer` but scoped to the region.

    Parameters
    ----------
    resolution : int, default 150
        Rendering resolution (DPI).  This should match the value used by the
        page-level viewer so element scaling is accurate.
    include_chars : bool, default False
        Whether to include individual *char* elements in the overlay.  These
        are often too dense for a meaningful visualisation so are skipped by
        default.
    include_attributes : list[str], optional
        Additional element attributes to expose in the info panel (on top of
        the default set used by the page viewer).

    Returns
    -------
    InteractiveViewerWidgetType | None
        The widget instance, or ``None`` if *ipywidgets* is not installed or
        an error occurred during creation.
    """

    # Dependency / environment checks
    if not _IPYWIDGETS_AVAILABLE or InteractiveViewerWidget is None:
        logger.error(
            "Interactive viewer requires 'ipywidgets'. "
            'Please install with: pip install "ipywidgets>=7.0.0,<10.0.0"'
        )
        return None

    try:
        # Render region image (cropped) and encode as data URI
        import base64
        from io import BytesIO

        # Use unified render() with crop=True to obtain just the region
        img = self.render(resolution=resolution, crop=True)
        if img is None:
            logger.error(f"Failed to render image for region {self.bbox} viewer.")
            return None

        buf = BytesIO()
        img.save(buf, format="PNG")
        img_str = base64.b64encode(buf.getvalue()).decode()
        image_uri = f"data:image/png;base64,{img_str}"

        # Prepare element overlay data (coordinates relative to region)
        scale = resolution / 72.0  # Same convention as page viewer

        # Gather elements intersecting the region
        region_elements = self.get_elements(apply_exclusions=False)

        # Optionally filter out chars
        if not include_chars:
            region_elements = [
                el for el in region_elements if str(getattr(el, "type", "")).lower() != "char"
            ]

        default_attrs = [
            "text",
            "fontname",
            "size",
            "bold",
            "italic",
            "color",
            "linewidth",
            "is_horizontal",
            "is_vertical",
            "source",
            "confidence",
            "label",
            "model",
            "upright",
            "direction",
        ]

        if include_attributes:
            default_attrs.extend([a for a in include_attributes if a not in default_attrs])

        elements_json: List[dict] = []
        for idx, el in enumerate(region_elements):
            x0 = (el.x0 - self.x0) * scale
            y0 = (el.top - self.top) * scale
            x1 = (el.x1 - self.x0) * scale
            y1 = (el.bottom - self.top) * scale

            elem_dict = {
                "id": idx,
                "type": getattr(el, "type", "unknown"),
                "x0": round(x0, 2),
                "y0": round(y0, 2),
                "x1": round(x1, 2),
                "y1": round(y1, 2),
                "width": round(x1 - x0, 2),
                "height": round(y1 - y0, 2),
            }

            for attr_name in default_attrs:
                if hasattr(el, attr_name):
                    val = getattr(el, attr_name)
                    if not isinstance(val, (str, int, float, bool, list, dict, type(None))):
                        val = str(val)
                    elem_dict[attr_name] = val
            elements_json.append(elem_dict)

        viewer_data = {"page_image": image_uri, "elements": elements_json}

        # Instantiate the widget directly using the prepared data
        return InteractiveViewerWidget(pdf_data=viewer_data)

    except Exception as e:
        logger.error(f"Error creating viewer for region {self.bbox}: {e}", exc_info=True)
        return None

natural_pdf.Region.within()

Context manager that constrains directional operations to this region.

When used as a context manager, all directional navigation operations (above, below, left, right) will be constrained to the bounds of this region.

Returns:

Name	Type	Description
RegionContext		A context manager that yields this region

Examples:

# Create a column region
left_col = page.region(right=page.width/2)

# All directional operations are constrained to left_col
with left_col.within() as col:
    header = col.find("text[size>14]")
    content = header.below(until="text[size>14]")
    # content will only include elements within left_col

# Operations outside the context are not constrained
full_page_below = header.below()  # Searches full page

Source code in natural_pdf/elements/region.py

def within(self):
    """Context manager that constrains directional operations to this region.

    When used as a context manager, all directional navigation operations
    (above, below, left, right) will be constrained to the bounds of this region.

    Returns:
        RegionContext: A context manager that yields this region

    Examples:
        ```python
        # Create a column region
        left_col = page.region(right=page.width/2)

        # All directional operations are constrained to left_col
        with left_col.within() as col:
            header = col.find("text[size>14]")
            content = header.below(until="text[size>14]")
            # content will only include elements within left_col

        # Operations outside the context are not constrained
        full_page_below = header.below()  # Searches full page
        ```
    """
    return RegionContext(self)

Functions

natural_pdf.configure_logging(level=logging.INFO, handler=None)

Configure logging for the natural_pdf package.

Parameters:

Name	Type	Description	Default
level		Logging level (e.g., logging.INFO, logging.DEBUG)	INFO
handler		Optional custom handler. Defaults to a StreamHandler.	None

Source code in natural_pdf/__init__.py

def configure_logging(level=logging.INFO, handler=None):
    """Configure logging for the natural_pdf package.

    Args:
        level: Logging level (e.g., logging.INFO, logging.DEBUG)
        handler: Optional custom handler. Defaults to a StreamHandler.
    """
    # Avoid adding duplicate handlers
    if any(isinstance(h, logging.StreamHandler) for h in logger.handlers):
        return

    if handler is None:
        handler = logging.StreamHandler()
        formatter = logging.Formatter("%(name)s - %(levelname)s - %(message)s")
        handler.setFormatter(formatter)

    logger.addHandler(handler)
    logger.setLevel(level)

    logger.propagate = False

natural_pdf.set_option(name, value)

Set a global Natural PDF option.

Parameters:

Name	Type	Description	Default
name	str	Option name in dot notation (e.g., 'layout.auto_multipage')	required
value		New value for the option	required

Example

import natural_pdf as npdf npdf.set_option('layout.auto_multipage', True) npdf.set_option('ocr.engine', 'surya')

Source code in natural_pdf/__init__.py

def set_option(name: str, value):
    """
    Set a global Natural PDF option.

    Args:
        name: Option name in dot notation (e.g., 'layout.auto_multipage')
        value: New value for the option

    Example:
        import natural_pdf as npdf
        npdf.set_option('layout.auto_multipage', True)
        npdf.set_option('ocr.engine', 'surya')
    """
    parts = name.split(".")
    obj = options

    # Navigate to the right section
    for part in parts[:-1]:
        if hasattr(obj, part):
            obj = getattr(obj, part)
        else:
            raise KeyError(f"Unknown option section: {part}")

    # Set the final value
    final_key = parts[-1]
    if hasattr(obj, final_key):
        setattr(obj, final_key, value)
    else:
        raise KeyError(f"Unknown option: {name}")