# Seg2Act: Global Context-aware Action Generation for Document Logical Structuring > Seg2Act reframes document logical structure extraction as a one-pass action generation task, using a generative LM conditioned on a global context stack to iteratively assign each text segment an action that places it in the hierarchy. - Authors: Hunter Heidenreich - Published: 2026-04-02 - Paper: Preprint (arXiv) - Paper: Published (EMNLP 2024) - Code: Seg2Act Code - Source: Bevaya Labs, --- ## TL;DR Seg2Act treats document logical structure extraction as a one-pass action generation task: a generative language model reads text segments through a sliding window and predicts a symbolic action per segment, while a global context stack tracks the partially built hierarchy. On ChCatExt and HierDoc benchmarks, the method outperforms both text-only transition-based baselines and multimodal pipeline methods using only textual input. ## What kind of paper is this? **Dominant:** $\Psi\_{\text{Method}}$: The paper's headline contribution is an end-to-end generative framework for document logical structuring. It proposes a new action vocabulary, a global context stack mechanism, and a multi-segment multi-action inference strategy, and validates them with ablation studies and SOTA comparisons. ## What is the motivation? Document logical structuring (recovering the hierarchical tree of headings and paragraphs from a document's text segments) is important for downstream tasks like retrieval, summarization, and question answering over long documents. Existing methods share three failure modes: 1. **Local encoding:** Pipeline methods encode text segments in isolation or in pairs, missing long-range structural cues. 2. **Multi-stage error propagation:** Decomposing the task into heading detection, node classification, and relationship prediction lets errors accumulate across stages. 3. **Weak generalization:** Because each stage is designed around structural conventions of a specific document type, adapting to new document families requires redesigning the pipeline. Transition-based methods such as TRACER partially address these issues but still rely on pairwise local context and shift-reduce operations that force each segment to participate in multiple prediction steps. ## What is the novelty? Seg2Act contributes three tightly coupled ideas: **1. A one-to-one action vocabulary.** Rather than pairwise transition decisions, Seg2Act defines exactly three actions, each mapping one text segment to one position in the logical tree: - **New Level-$k$ Heading** (symbol: $k$ consecutive `+` characters): adds the segment as a level-$k$ heading under the last level-$(k-1)$ heading. - **New Paragraph** (symbol: `*`): adds the segment as a paragraph under the current heading. - **Concatenation** (symbol: `=`): appends the segment to the last node, handling OCR line splits. This one-to-one design reduces the total number of predictions to $N$ (the number of segments) versus the multiple-pass decisions in shift-reduce parsing. **2. A global context stack.** To give the model awareness of the evolving structure, Seg2Act maintains a stack $S$ of the nodes along the ancestor path of the most recently added node. The stack is serialized in the same symbolic vocabulary (`+`, `*`) as the action set and prepended to the model input at each step. Update rules are: - **New Level-$k$ Heading:** pop nodes until the stack top is a level-$(k-1)$ heading, then push the new node. - **New Paragraph:** pop the top paragraph (if any), then push the new paragraph. - **Concatenation:** append text to the top node; stack shape is unchanged. This keeps the globally relevant structural context within a bounded input length. **3. Multi-segment multi-action (MSMA) strategy.** Processing 853 segments one-by-one (the HierDoc average) is slow and limits context. Seg2Act instead processes a window of $w\_I$ segments and predicts $w\_O$ actions in a single forward pass. The total number of steps is $\lceil N / w\_O \rceil$. The two window sizes are independent: a wider $w\_I$ provides more look-ahead context, while $w\_O$ controls throughput. Training uses teacher forcing with cross-entropy loss: $$ \mathcal{L} = -\sum\_{i=1}^{|D|} \log P(y\_{i:i+w\_I-1} \mid s\_i, x\_{i:i+w\_I-1};\, \Theta) $$ where $s\_i$ is the global context stack at step $i$ and $\Theta$ are the model parameters. At training time, $w\_I = w\_O$ by default (one-pass mode). Hard inference constraints enforce structural validity: only tokens from the predefined set $\\{$`+`, `*`, `=`, `\n`$\\}$ are allowed, concatenation is blocked when only the root node is on the stack, and heading levels cannot skip (a predicted level-$k$ heading is clamped to the current maximum depth plus one if needed). ## What experiments were performed? **Datasets.** Two benchmarks: - **ChCatExt** (Zhu et al., 2023): 650 Chinese documents (bid announcements, financial announcements, credit rating reports) with full heading and paragraph annotations. - **HierDoc** (Hu et al., 2022): 650 English scientific documents with Table-of-Contents (heading-only) annotations; average 853.38 segments per document. **Metrics.** Node-level F1 for headings, paragraphs, and all nodes combined; TEDS for ToC structure on HierDoc; and DocAcc (the fraction of documents whose predicted structure exactly matches the ground truth). **Baselines.** - *Text-only:* TRACER (shift-reduce transition parser with a pretrained LM encoder). - *Multimodal:* MTD (BERT + ResNet attention-based ToC extractor) and CMM (RoBERTa + heuristic tree initialization, then node-level refinement). **Backbone models.** GPT2-Medium (~345 M parameters), Baichuan-7B (7B parameters, fine-tuned with LoRA: rank $r=8$, $\alpha=16$), Baichuan-13B, and Qwen1.5 (0.5B, 1.8B, 4B) in the model size ablation. **Transfer learning.** Models are pretrained on the Wiki corpus (provided by Zhu et al.) for 10,000 steps, then evaluated zero-shot, few-shot (3 and 5 examples), and full-shot on each ChCatExt sub-corpus. **Ablations.** - Removing the MSMA strategy ($w\_I = w\_O = 1$). - Removing the symbolic component of the global context stack (leaving only text). - Removing the text component (leaving only symbols). - Removing the full stack (both text and symbols). **Optimizer.** AdamW, learning rate $3 \times 10^{-4}$, 10 epochs, batch size 128, $w\_I = w\_O = 3$ by default. Experiments run on one NVIDIA A100 GPU; results averaged over 5 random seeds. ## What are the outcomes/conclusions? **Supervised setting.** On ChCatExt with the Baichuan-7B backbone, Seg2Act achieves 92.63 total F1 and 63.69 DocAcc, versus TRACER at 89.55 F1 and 53.85 DocAcc, a +9.84 DocAcc gap. On HierDoc, Seg2Act with Baichuan-7B reaches 98.1 heading detection F1 and 96.3 TEDS, outperforming the multimodal MTD (96.1 / 87.2) and CMM (97.0 / 88.1) using text alone. **Transfer learning.** In zero-shot, few-shot, and full-shot transfer across ChCatExt sub-corpora, Seg2Act averages +15.28, +6.04, and +10.65 F1 over SEG2ACT-T (the transition-based variant), respectively. With only 5 labeled examples, performance drops only 3.98 F1 points on average from the full-shot setting, suggesting the action vocabulary generalizes well across document families. **Ablations.** Removing the hierarchical symbols from the global context stack costs 6.46 DocAcc. Removing the text portion costs more (12.92 DocAcc). Removing the full stack degrades DocAcc by 18.77 points, confirming that global context is the primary driver of document-level accuracy. **Model size.** Qwen1.5-1.8B matches Baichuan-7B's DocAcc (both 63.69) at roughly half the inference time (4.27 s vs. 8.13 s per document), and the 1.8B model is $0.65\times$ faster than Qwen1.5-4B while achieving comparable accuracy. Qwen1.5-4B achieves 65.23 DocAcc with a moderate speed penalty. Results suggest the action generation framework can run effectively on smaller models. **Limitations acknowledged by the authors.** Seg2Act relies on a correctly ordered sequence of text segments; disrupted segment order (e.g., from imperfect OCR column detection) is not handled. The generative model may occasionally emit the wrong number of actions for a given window, causing skipped segments. Visual information is not used, so purely layout-driven cues such as font size and indentation are unavailable unless the text encoder can infer them from OCR output. **Limitations not addressed.** Evaluation is restricted to two Chinese and English benchmarks with relatively clean ground-truth segmentations. Performance on noisier OCR pipelines, PDF documents with complex layouts (multi-column, tables embedded in the flow), or non-standard document types remains untested. The paper does not report statistical significance tests, though it does average over five seeds. Computational overhead of the autoregressive generation loop at inference compared to a classifier-based approach is not analyzed for latency-sensitive production settings. ## Reproducibility ### Models - **GPT2-Medium:** ~345 M parameters, standard causal LM decoder, fine-tuned fully on the task. - **Baichuan-7B:** 7B-parameter causal LM, fine-tuned with LoRA (rank $r=8$, alpha $\alpha=16$). Specific layers adapted are not specified in the paper. - **Baichuan-13B:** Larger variant tested in the appendix; LoRA is used by analogy with the 7B setting (specific rank/alpha not re-stated for 13B). - **Qwen1.5 (0.5B, 1.8B, 4B):** Additional model size ablation; training details assumed consistent with Baichuan-7B setting. - Model weights for Seg2Act fine-tuned models are not released. Base model weights (GPT2, Baichuan-7B, Qwen1.5) are publicly available. ### Algorithms - **Optimizer:** AdamW with learning rate $3 \times 10^{-4}$. - **Schedule:** 10 training epochs; no warmup schedule reported. - **Batch size:** 128. - **Input/output windows:** $w\_I = w\_O = 3$ (default). - **Loss:** Teacher-forcing cross-entropy over action tokens. - **Inference:** Greedy decoding with hard constraints via `LogitsProcessor` from the HuggingFace Transformers library. - **Pretraining for transfer:** 10,000 steps on Wiki corpus, then fine-tuned with LoRA. - Gradient clipping and mixed-precision training are not mentioned. ### Data - **ChCatExt:** 650 Chinese documents with logical tree annotations; split details and download location are not described in this paper. The dataset was provided by Zhu et al. (2023) alongside the TRACER work. - **HierDoc:** 650 English scientific documents with ToC annotations; provided by Hu et al. (2022) via ICPR 2022. Public availability should be verified at the original source. - **Wiki corpus:** Used for pretraining in transfer experiments; provided by Zhu et al. (2023). License and availability are not discussed in this paper. - Annotation process for ChCatExt and HierDoc is described in the original dataset papers, not here. ### Evaluation - **Metrics:** Node-level F1 (heading, paragraph, total); TEDS for ToC; DocAcc (exact match at document level). DocAcc is newly introduced here and is straightforward to compute. - **Benchmarks:** ChCatExt (all three sub-corpora: BidAnn, FinAnn, CreRat) and HierDoc. - **Baselines:** TRACER results on ChCatExt were re-run by the authors (TRACER\*) because original results used a different backbone. CMM and MTD results are taken from the original publications. - **Statistical rigor:** Results are averaged over 5 random seeds. No confidence intervals or significance tests are reported. - **Reproducibility gap:** Fine-tuned model weights are not released. The code repository (linked in frontmatter artifacts) has no declared license file, which limits derivative use. ### Hardware - **Training hardware:** Single NVIDIA A100 GPU. Total GPU-hours not reported. - **Inference time:** Per-document inference time with Baichuan-7B at $w\_I = w\_O = 3$ is approximately 8.13 seconds (from Table 6 in the paper). Qwen1.5-0.5B achieves ~4.01 seconds per document. - Memory requirements and VRAM usage are not reported. - The LoRA fine-tuning is intended to reduce GPU memory overhead for the 7B model, suggesting the A100 (40 GB or 80 GB) is used but may be manageable at lower VRAM with quantization. ## BibTeX ```bibtex @inproceedings{li-etal-2024-seg2act, title = "{S}eg2{A}ct: Global Context-aware Action Generation for Document Logical Structuring", author = "Li, Zichao and He, Shaojie and Liao, Meng and Chen, Xuanang and Lu, Yaojie and Lin, Hongyu and Lu, Yanxiong and Han, Xianpei and Sun, Le", booktitle = "Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing", year = "2024", publisher = "Association for Computational Linguistics", doi = "10.18653/v1/2024.emnlp-main.1003", } ```