# CED: Catalog Extraction from Documents with TRACER > TRACER is a transition-based shift-reduce parser that builds table-of-contents trees from plain text segments, introduced alongside the ChCatExt benchmark corpus of 650 manually annotated Chinese business documents. - Authors: Hunter Heidenreich - Published: 2026-04-02 - Paper: Preprint (arXiv) - Paper: Published (ICDAR 2023) - Code: CatalogExtraction (code) - Dataset: ChCatExt (dataset) - Model: TRACER model weights - Source: Bevaya Labs, --- ## TL;DR This paper defines the Catalog Extraction from Documents (CED) task, which parses a flat sequence of OCR text segments into a hierarchical table-of-contents tree. The authors release ChCatExt, a 650-document manually annotated Chinese corpus, and propose TRACER, a transition-based parser built on a lightweight RoBERTa backbone that uses four actions (Sub-Heading, Sub-Text, Concat, Reduce) to incrementally build the catalog tree. TRACER outperforms the pipeline baseline by 5.3 F1 points and the tagging baseline by 4.1 F1 points, and shows meaningful cross-domain transfer when pre-trained on a large Wikipedia corpus. ## What kind of paper is this? **Dominant:** $\Psi\_{\text{Method}}$: The paper's primary contribution is TRACER, a transition-based parsing algorithm with a defined action set for catalog tree construction. The architecture design, ablation on constraints, and extensive transfer experiments all center on validating this method. **Secondary:** $\Psi\_{\text{Resource}}$: The paper also releases ChCatExt, the first annotated corpus for the CED task, and a large-scale Wikipedia pre-training corpus with automatically extracted catalog structures. ## What is the motivation? Long documents such as credit rating reports, financial announcements, and bid specifications contain information that is sparsely distributed across tens of pages. Feeding an entire document into an information extraction (IE) pipeline is computationally impractical; a prior structuring step that locates sections and their hierarchical relationships can dramatically reduce the search space for downstream IE tasks. The obvious solution, handcrafted regular expressions, breaks down across three challenges: 1. Section title formatting varies widely across document sources, making rules non-reusable. 2. Catalog hierarchies can reach five or six levels deep, and naive rule systems cannot distinguish fine-grained heading levels reliably. 3. OCR tools frequently split a single sentence across multiple segments at page breaks or line breaks, so the raw text stream does not align with semantic boundaries. No prior dataset existed for this task, and no prior model addressed all three challenges jointly. ## What is the novelty? The core novelty is TRACER (Transition-based CAtalog extRaction), which reformulates catalog tree construction as a shift-reduce parsing problem operating on document-scale input. **Transition system.** The parser maintains two data structures: a catalog tree stack $S$ and an input queue $Q$ of text segments. At each step, the stack top $s$ and the queue front $q$ are compared, and one of four actions is emitted: - **Sub-Heading**: $q$ becomes a child heading node of $s$. - **Sub-Text**: $q$ becomes a child text node of $s$. - **Concat**: $q$ is the continuation of $s$; their contents are concatenated (handles OCR splits). - **Reduce**: $s$ is replaced by its parent, moving up the tree when $q$'s level is at or above $s$'s level. This action vocabulary sidesteps the need to pre-define a maximum heading depth, making the parser applicable to arbitrarily deep catalog structures. **Model architecture.** Each step encodes $s$ and $q$ independently through a pre-trained language model (RBT3, a compact Chinese RoBERTa variant), concatenates the representations, and passes them through a two-layer FFN with ReLU and dropout: $$ \begin{aligned} g &= s \Vert q \\\\ o &= \text{FFN}(g) \\\\ p(A \mid s, q) &= \text{softmax}(o) \end{aligned} $$ At inference, the predicted action is the argmax over the action set $A$: $$ a\_i = \underset{a \in A}{\text{argmax}} \; p(A \mid s, q) $$ Training uses standard cross-entropy loss: $$ \mathcal{L} = -\sum\_i \mathbb{I}\_{y\_a = a\_i} \log p(a\_i \mid s, q) $$ Two hard constraints prevent illegal tree states: the first action must be Sub-Heading or Sub-Text (not Reduce or Concat onto an empty stack); text nodes may only be followed by Reduce or Concat actions. When the top-scoring action is illegal, the model falls back to the highest-scoring legal action. **Wikipedia pre-training.** To improve cross-domain generalization, the authors train TRACER on 214,989 Chinese Wikipedia articles with automatically derived catalog structures (filtered from 665,355 documents to those with depth 2 to 4), producing a WikiBert checkpoint for transfer experiments. ## What experiments were performed? **Dataset.** ChCatExt contains 650 manually annotated documents across three domain-specific subsets: 100 bid announcements (BidAnn), 300 financial announcements (FinAnn), and 250 credit rating reports (CreRat). Documents are split 80/10/10 for train, development, and test. OCR segmentation is simulated by chunking paragraphs with 50% probability and splitting heading strings at lengths of 7 to 20 characters using a Chinese tokenizer. **Evaluation metric.** Micro-averaged F1 over predicted tree nodes, where each node is represented as a tuple (level, type, content). Precision and recall are: $$ P = \frac{N\_r}{N\_p}, \quad R = \frac{N\_r}{N\_g}, \quad F1 = \frac{2PR}{P + R} $$ where $N\_r$ is correctly matched tuples, $N\_p$ is predicted tuples, and $N\_g$ is gold tuples. **Baselines.** Two baselines were constructed because no prior systems exist for CED: 1. **Classification Pipeline**: A two-stage system that first predicts segment concatenation using a [CLS]-based classifier, then predicts depth level with PLM + TextCNN. 2. **Tagging**: A joint sequence tagger using PLM + LSTM + CRF to simultaneously predict BIO concatenation tags and depth/type labels. **Experiment settings.** Experiments ran on a single NVIDIA Titan Xp GPU. RBT3 was used as the backbone PLM. Training used AdamW with learning rate 2e-5, batch size 20, dropout 0.5, for 10 epochs. Results are averages over 5 random seed trials. **Transfer experiments.** Four tables explore cross-domain and low-resource transfer: training on one subset and evaluating on another; training on only $k \in \\{3, 5, 10\\}$ source documents; sequential fine-tuning on source then $k$ target documents; and joint training on the concatenated source plus $k$ target documents. ## What are the outcomes/conclusions? **Main results.** On ChCatExt, TRACER achieves 82.390% overall F1, compared to 77.085% for the Classification Pipeline and 78.269% for the Tagging baseline. The gains are consistent across heading (90.486% vs. 87.601% and 87.846%) and text nodes (84.328% vs. 82.047% and 81.344%). The authors attribute the advantage to TRACER reasoning about pairwise structural relationships between nodes rather than assigning absolute depth labels, and to the fact that TRACER does not require a predefined maximum depth. Removing the two decoding constraints drops overall F1 by 0.794%, confirming they are useful but not load-bearing for correctness. **Wikipedia pre-training did not help on in-domain evaluation.** TRACER with WikiBert scores 82.063% overall, marginally below the vanilla TRACER (82.390%), likely because the Wikipedia domain differs substantially from Chinese business documents. **Transfer.** Cross-domain zero-shot transfer is weak in all configurations, with BidAnn-trained models scoring only 7.4% and 2.4% on FinAnn and CreRat respectively. WikiBert improves transfer in 6 out of 9 cross-domain pairs (Table 4) and in 23 out of 27 few-shot pairs (Table 5). In low-resource settings, WikiBert pre-training is clearly beneficial. When $k$ target documents are available, concatenating source and target data tends to outperform sequential fine-tuning as $k$ grows. **Depth-level analysis.** F1 degrades at deeper levels, and TRACER fails entirely on level-1 text nodes (none exist in the gold data) and level-5 heading nodes (too rare). The authors note that pairwise encoding without global history is a structural limitation for deeper predictions. **Limitations acknowledged by the authors.** The domain gap between Wikipedia and business documents limits the utility of Wikipedia pre-training for in-domain settings. Comparing only two nodes at a time discards global structural context, which the authors identify as a key source of errors at deeper hierarchy levels. The dataset is Chinese-only business documents, and generalization to other languages or document types is not evaluated. **Limitations not acknowledged.** The simulation of OCR errors by random chunking is a rough approximation; real OCR errors include character substitutions, merged lines, and layout-induced reordering that chunking alone does not capture. The evaluation metric treats node content as an exact match, which may be brittle to minor string normalization differences. No statistical significance testing is reported; results are averages over 5 seeds but without confidence intervals. ## Reproducibility ### Models - **Backbone**: RBT3, a 3-layer Chinese RoBERTa variant from the HFL group, available on Hugging Face (`hfl/rbt3`, Apache-2.0). Parameter count is not stated explicitly in the paper, but RBT3 is a compact model designed for efficiency. - **Head**: Two-layer FFN with ReLU activation and 0.5 dropout; output dimension matches the 4-class action space. - **Released checkpoints** (GitHub releases, Apache-2.0): - `transducer_DomainMix_1227.zip`: vanilla TRACER trained on the full ChCatExt mix. - `transducer_plm4w_-1_DomainMix_1227.zip`: TRACER initialized from WikiBert, fine-tuned on ChCatExt. - `wiki_plm_4w.zip`: the WikiBert PLM checkpoint (RBT3 trained for 40k steps on the Wikipedia corpus), ready for use as initialization in transfer experiments. ### Algorithms - **Optimizer**: AdamW, learning rate 2e-5, no other schedule details reported. - **Batch size**: 20 segments per batch. - **Training duration**: 10 epochs for domain-specific training; 40k steps for Wikipedia pre-training. - **Dropout**: 0.5 in the FFN. - **Decoding**: Greedy (argmax) action selection at each step, with constraint-based fallback to the next legal action when the top prediction is illegal. - **Model selection**: Best checkpoint on the development set; final evaluation on the test set. - **Runs**: 5 trials with different random seeds; average reported. ### Data - **ChCatExt**: 650 manually annotated Chinese business documents (100 BidAnn, 300 FinAnn, 250 CreRat), split 80/10/10. Annotation was performed by pairs of annotators with expert adjudication; each document took up to 20 minutes to label. - **Wikipedia corpus**: 214,989 Chinese Wikipedia articles filtered for catalog depth 2 to 4 from a dump dated 2021-12-20. Catalog structure is automatically derived from Wikipedia heading markup. - **OCR simulation**: Paragraphs chunked with 50% probability into segments of 70 to 100 characters; headings split at 7 to 20 character boundaries using jieba. - **Availability**: Code and all data are released at `https://github.com/Spico197/CatalogExtraction` under Apache-2.0. The GitHub releases page includes `ChCatExt.zip` (annotated data), `Wiki.zip` (pre-training corpus), `DataForAnalysisExp.zip` (data-scale analysis splits), and `OriginalRawData.zip`. Source document URLs are listed in the paper (Hebei bid platform, cninfo.com.cn, chinaratings.com.cn, dfratings.com), but the raw PDFs are not redistributed. ### Evaluation - **Metric**: Micro-averaged F1 over (level, type, content) node tuples. Heading-only and text-only sub-scores are also reported but on disjoint subsets, so they are not averages of the overall score. - **Baselines**: Both baselines use RBT3 as their PLM. The classification pipeline uses TextCNN; the tagging baseline uses LSTM + CRF. Maximum heading depth is set to 8 for both baselines. - **Benchmark**: ChCatExt only; no external benchmark exists for this task. - **Statistical rigor**: 5 random seed runs with averages; no standard deviations or confidence intervals are reported for the main results (standard deviation is mentioned once in passing for a data-scale ablation). ### Hardware - **Training**: Single NVIDIA Titan Xp (12 GB VRAM). No GPU-hour estimates are reported. - **Inference**: Not characterized separately. - **Cost**: Not reported. - **Local feasibility**: RBT3 is compact; the system should be runnable on a single consumer GPU given the small batch size and short sequence lengths (individual segment pairs rather than full documents). ## BibTeX ```bibtex @inproceedings{zhu2023ced, title={CED: Catalog Extraction from Documents}, author={Zhu, Tong and Zhang, Guoliang and Li, Zechang and Yu, Zijian and Ren, Junfei and Wu, Mengsong and Wang, Zhefeng and Huai, Baoxing and Chao, Pingfu and Chen, Wenliang}, booktitle={Document Analysis and Recognition -- ICDAR 2023}, pages={120--136}, year={2023}, publisher={Springer}, doi={10.1007/978-3-031-41679-8_8} } ```