# PharmaShip: A Chinese Pharmaceutical Shipping Document Benchmark with Reading Order Supervision

> Analysis of PharmaShip (arXiv 2025), a 161-document Chinese pharmaceutical shipping dataset with SER, entity linking, and ISDR-style reading order prediction annotations, benchmarked against five layout-aware models.

- Authors: Bevaya AI Team
- Published: 2026-04-02
- Paper: Preprint (arXiv) <https://arxiv.org/abs/2512.23714>
- Dataset: PharmaShip Dataset and Code <https://github.com/KevinYuLei/PharmaShip>
- Source: Bevaya Labs, <https://labs.bevaya.ai/>

---


## TL;DR

PharmaShip is a 161-document Chinese pharmaceutical shipping dataset with ISDR-style reading order graph annotations alongside SER and entity linking tasks. It extends the entity-centric annotation paradigm of EC-FUNSD to the pharmaceutical logistics domain, where dense tabular layouts and long-form documents stress-test proximity-based layout heuristics. Five layout-aware baselines are benchmarked, and reading-order-enhanced variants (RORE) consistently improve SER and EL, with the largest benefit on entity linking.

## What kind of paper is this?

**Dominant:** $\Psi\_{\text{Resource}}$: The headline contribution is the PharmaShip dataset, the first publicly available benchmark for pharmaceutical shipping documents with reading order graph annotations. Dataset construction, annotation methodology, and statistics occupy the majority of the paper.

**Secondary:** $\Psi\_{\text{Evaluation}}$: The controlled benchmark evaluation of five models on SER, EL, and ROP tasks provides diagnostic findings about the complementary roles of pixel features, geometric encoding, and reading order regularization.

## What is the motivation?

Existing VrDU benchmarks (FUNSD, CORD, SROIE) target general administrative forms and consumer receipts. They share two limitations that make them unsuitable for pharmaceutical shipping documents:

1. **Segment-entity coupling.** Block-level annotations implicitly tie semantic labels to visual regions, encouraging models to rely on spatial proximity heuristics that fail when visually adjacent segments belong to different semantic categories (e.g., drug name adjacent to batch number in a dense table).
2. **No reading order supervision.** Without reading order annotations, models default to top-to-bottom, left-to-right assumptions. This fails on multi-column tables and field groups where the reading path is non-trivial.

PharmaShip targets Chinese pharmaceutical circulation documents: delivery notes, shipment forms, and outbound sales slips. These documents have heterogeneous templates, dense tabular layouts, and long-form content frequently exceeding the default 512-token limit of standard PTLMs.

## What is the novelty?

### Dataset: PharmaShip

161 annotated scanned documents sourced from multiple pharmaceutical enterprises in China. Each document is provided in JSON format with:

- **SER annotations:** Token-level semantic entity labels (header, question, answer, other), following the entity-centric paradigm of EC-FUNSD.
- **EL annotations:** Directed relation triples over recognized entities $(e\_i, e\_j) \mapsto r\_{ij} \in \mathcal{R}$.
- **ROP annotations:** ISDR reading order links as a directed acyclic graph over segments, following the formulation of ROOR (Zhang et al., 2024):

$$
f\_{ROP}: (s\_i, s\_j) \mapsto p\_{ij} \in \mathcal{P}
$$

where $\mathcal{P}$ is the set of immediate succession links.

PharmaShip is substantially denser than ROOR on a per-document basis: 70.16 segments per sample versus 53.57 for ROOR, 415.91 words per sample versus 157.27, and ~108 reading order relations per sample versus ~55.

### Key Design Choices

**Entity-centric annotation:** Semantic labels are assigned independently of visual block boundaries, decoupling segment identity from layout proximity. This removes the "false coupling" that inflates scores on FUNSD-style benchmarks.

**Positional embedding extension:** Because pharmaceutical documents routinely exceed 512 tokens, positional embeddings are expanded from 512 to 2048 for all three backbones. Newly added positions are initialized per each repository's default. This prevents truncation of late-page entities and long-range relations.

**Class-imbalance correction for EL:** The public ROOR implementation over-samples the "no-relation" class. The authors rebalance positive/negative pairs and apply class-weighted optimization to stabilize gradients.

## What experiments were performed?

### Baselines

Five models spanning pixel-aware and geometry-aware families:

| Model | Params | Notes |
|:---|:---|:---|
| LiLT[InfoXLM] | base | Pixel-free; layout via box/position interactions |
| LayoutLMv3-base-chinese | 133M | Pixel-aware joint token+image pre-training |
| RORE-LayoutLMv3 | 133M+12 | LayoutLMv3 + reading order relation injection |
| GeoLayoutLM | 399M | Explicit multi-level geometric encodings |
| RORE-GeoLayoutLM | 399M+24 | GeoLayoutLM + reading order relation injection |

All use Chinese pre-trained checkpoints, identical OCR preprocessing and bounding boxes, and the same page renderings. Fine-tuned up to 500 epochs with early stopping (50-epoch patience).

### Results on PharmaShip (Table III)

**SER:**

| Model | Precision | Recall | F1 |
|:---|:---|:---|:---|
| LiLT | 73.15 | 78.77 | 75.85 |
| LayoutLMv3 | 81.06 | 86.08 | 83.49 |
| RORE-LayoutLMv3 | 82.13 | 86.08 | **84.06** (+0.57) |
| GeoLayoutLM | 77.71 | 83.91 | 80.69 |
| RORE-GeoLayoutLM | 78.81 | 85.21 | 81.88 (+1.19) |

**EL:**

| Model | Precision | Recall | F1 |
|:---|:---|:---|:---|
| LiLT | 51.22 | 82.35 | 63.16 |
| LayoutLMv3 | 67.49 | 90.32 | 77.25 |
| RORE-LayoutLMv3 | 68.81 | 91.45 | **78.53** (+1.28) |
| GeoLayoutLM | 77.91 | 73.28 | 75.52 |
| RORE-GeoLayoutLM | 78.42 | 74.79 | 76.56 (+1.04) |

**ROP:**

| Model | Precision | Recall | F1 |
|:---|:---|:---|:---|
| LayoutLMv3 (word-level) | 81.91 | 96.20 | 88.48 |
| LayoutLMv3 (segment-level) | 62.64 | 81.51 | 70.84 |

Cross-dataset results (Table II) show that the same models score 5-9 F1 points lower on PharmaShip than on FUNSD/CORD/SROIE, confirming that entity-centric annotation and layout complexity expose weaknesses masked by simpler benchmarks.

## What are the outcomes/conclusions?

1. **RORE consistently improves SER and EL.** Reading order injection provides +0.57 to +1.28 F1 gains on PharmaShip, with the largest benefit for EL (entity linking relies more on logical ordering than local entity semantics).
2. **Pixel features and geometry are complementary.** LayoutLMv3 (pixel-aware) leads on SER; GeoLayoutLM (geometry-aware) has higher EL precision. Neither alone is sufficient.
3. **Word-level ROP is feasible; segment-level ROP is harder.** LayoutLMv3 achieves 88.48 F1 at word-level but only 70.84 at segment-level, with boundary ambiguity and long-range crossings in dense tables driving the gap.
4. **Positional embedding extension matters.** Expanding from 512 to 2048 positions prevents truncation of late-page content and stabilizes performance on long pharmaceutical forms.
5. **Entity-centric annotation reveals layout heuristic fragility.** Decoupling segments from semantic entities removes the block-level bias that inflates scores on simpler benchmarks; PharmaShip scores are consistently lower, reflecting real task difficulty.

### Limitations

- Dataset is small (161 documents). Generalization of benchmark conclusions to other pharmaceutical document types or other institutions is not validated.
- All documents are Chinese; no multilingual or non-Chinese evaluation.
- Only LayoutLMv3 is tested for ROP; no comparison against dedicated ROP models (e.g., LayoutReader, the ROOR baseline directly).
- No hardware or training time details reported.
- CC-BY-NC-4.0 license restricts commercial use.

## Reproducibility

### Models

- LiLT[InfoXLM], LayoutLMv3-base-chinese, GeoLayoutLM: official Chinese pre-trained checkpoints used as-is.
- RORE variants: ROOR framework applied using official ROOR implementations with corrected class weighting.
- No new model weights released.

### Algorithms

- AdamW, weight decay $10^{-2}$, learning rate $10^{-4}$, batch size 16, 2% linear warm-up, linear decay (for GeoLayoutLM EL; others follow Huang et al. 2022 schedules).
- Up to 500 epochs, early stopping at 50-epoch patience on validation F1.
- Positional embeddings extended from 512 to 2048; dynamic padding within mini-batches.
- Class-imbalance rebalancing for EL positive/negative pairs.

### Data

- 161 annotated scanned documents from Chinese pharmaceutical enterprises.
- Released at [github.com/KevinYuLei/PharmaShip](https://github.com/KevinYuLei/PharmaShip) under CC-BY-NC-4.0.
- Train/validation/test splits follow EC-FUNSD conventions; exact split sizes not stated in the paper.
- Annotation process: not described in detail; no inter-annotator agreement reported.

### Evaluation

- SER and EL: F1 (precision and recall also reported).
- ROP: F1 at word-level and segment-level separately.
- Cross-dataset evaluation on FUNSD, CORD, SROIE, ROOR for context.
- No error bars or significance tests.

### Hardware

Not reported.

## BibTeX

```bibtex
@article{xie2025pharmaship,
  title={PharmaShip: An Entity-Centric, Reading-Order-Supervised Benchmark for Chinese Pharmaceutical Shipping Documents},
  author={Xie, Tingwei and Zhou, Tianyi and Song, Yonghong},
  journal={arXiv preprint arXiv:2512.23714},
  year={2025}
}
```