Skip to content

docs: add SeaTunnel 3.0+ integration guide - #464

Merged
imbajin merged 61 commits into
apache:masterfrom
hugegraph:add-seatunnel-integration-doc
Sep 17, 2026
Merged

imbajin merged 61 commits into
apache:masterfrom
hugegraph:add-seatunnel-integration-doc

Conversation

@zhupitertop

@zhupitertop zhupitertop commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Summary

Before → after: the ToolChain pages mixed Loader, Tools, and SeaTunnel, while the SeaTunnel path lacked a clear version boundary, migration flow, and bilingual examples → the navigation now separates visualization, import, and export/migration, with task-oriented Chinese and English guides.

  • Add SeaTunnel 3.0+ import and export/migration guides. The import guide covers HugeGraph Sink; the export/migration guide covers HugeGraph Source.
  • Explain graph2graph as copying data from source graph A to target graph B, and graph2any as exporting graph data to files or other systems.
  • Compare Loader, Tools, and SeaTunnel with ✅ / ⚠️ / ❌ markers, covering local or distributed execution, Zeta/Spark/Flink, Web UI, connector breadth, SQL-CDC, scheduling, usability, and throughput trade-offs.
  • Link the stable 3.0.0-release branch and keep the obsolete 2.3.13 Sink details in one legacy note.
  • Group references and use matching superscript citations in both SeaTunnel pages; add the HOCON configuration guide link.
  • Use stable site-absolute links for cross-page references so versioned site validation resolves the import guide correctly.
  • Add four bilingual image2.5 diagrams: tool choice, records to graph, ID and endpoint preservation, and workflow summary. The sequence follows overview → detail → choice.
  • Reorganize the ToolChain sidebar while preserving existing article URLs and a dedicated sidebar entry for every subpage.
  • Nest the Computer configuration and benchmark 0.4.4 entries under their parent pages without changing their existing public URLs, and mark the benchmark section for a future rewrite.
  • Remove temporary scripts and language fallback rules, and address all review threads.

Visuals

The diagrams are generated in matching Chinese and English versions and are embedded in the relevant pages.

Focus Chinese English
Tool choice Loader 与 SeaTunnel 的区别 Loader vs SeaTunnel
Records to graph 从记录到图 Records to graph
ID and endpoints 保留顶点 ID Preserve vertex IDs
Workflow summary 按工作流选择工具 Choose by workflow

Validation

  • Hugo Extended build passed for the site (285 English / 283 Chinese pages).
  • 67 repository tests passed; link validation, punctuation checks, and git diff --check passed.
  • Version tooling validation passed for the latest site artifact.
  • All 54 review threads are resolved. GitHub Actions: workflow.

@dosubot dosubot Bot added size:L This PR changes 100-499 lines, ignoring generated files. enhancement New feature or request labels Aug 6, 2026
@zhupitertop
zhupitertop force-pushed the add-seatunnel-integration-doc branch from cf93179 to 4ae78ff Compare August 6, 2026 15:24
@dosubot dosubot Bot added size:M This PR changes 30-99 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Aug 6, 2026

@imbajin imbajin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: yes. Summary: The PR adds an unrelated release workflow that uses obsolete artifact rules, and its SeaTunnel guide contains unsupported claims and non-runnable configuration. Evidence: actionlint /tmp/validate-release-new-464.yml; exact-head patch; and comparison with the parent release workflow.

Comment thread .github/workflows/validate-release-new.yml Outdated
Comment thread .github/workflows/validate-release-new.yml Outdated
Comment thread .github/workflows/validate-release-new.yml Outdated
Comment thread .github/workflows/validate-release-new.yml Outdated
Comment thread .github/workflows/validate-release-new.yml Outdated
Comment thread content/cn/docs/quickstart/toolchain/hugegraph-seatunnel-connector.md Outdated
Comment thread content/cn/docs/quickstart/toolchain/hugegraph-seatunnel-connector.md Outdated
Comment thread content/cn/docs/quickstart/toolchain/hugegraph-seatunnel-connector.md Outdated
Comment thread content/cn/docs/introduction/_index.md Outdated
Comment thread content/cn/docs/introduction/_index.md Outdated

@imbajin imbajin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: yes. Summary: The new release workflow still fails valid artifacts or validates them incompletely, and the SeaTunnel documentation has unsupported compatibility and reference claims. Evidence: exact-head workflow/source review, actionlint, macOS grep reproduction, and current Apache documentation checks at 4ae78ff.

Comment thread .github/workflows/validate-release-new.yml Outdated
Comment thread .github/workflows/validate-release-new.yml Outdated
Comment thread .github/workflows/validate-release-new.yml Outdated
Comment thread .github/workflows/validate-release-new.yml Outdated
Comment thread .github/workflows/validate-release-new.yml Outdated
Comment thread .github/workflows/validate-release-new.yml Outdated
Comment thread .github/workflows/validate-release-new.yml Outdated
Comment thread .github/workflows/validate-release-new.yml Outdated
Comment thread content/cn/docs/quickstart/toolchain/hugegraph-seatunnel-connector.md Outdated
Comment thread content/cn/docs/quickstart/toolchain/hugegraph-seatunnel-connector.md Outdated

@imbajin imbajin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: yes. Summary: The new source-package license validation still has fail-open gaps that can let unreviewed files pass. Evidence: exact-head static review of validate-release-new.yml at 4ae78ff.

Comment thread .github/workflows/validate-release-new.yml Outdated
Comment thread .github/workflows/validate-release-new.yml Outdated
@dosubot dosubot Bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels Aug 7, 2026

@imbajin imbajin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: yes. Summary: The Chinese introduction page contains an unmatched fenced code block, so the rest of the page can render as code and its navigation/content is broken. Evidence: exact-head source has five triple-backtick fences with the extra opener at line 41; the latest deploy check is green but does not catch this Markdown structure.

Comment thread content/cn/docs/introduction/_index.md Outdated
@imbajin imbajin changed the title Add seatunnel integration doc docs: add SeaTunnel Source and Sink quick start Aug 11, 2026
imbajin
imbajin previously approved these changes Aug 11, 2026

@imbajin imbajin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final review on a136574: +1. The final diff is scoped to the Chinese SeaTunnel guide and its diagram, aligns the Source/Sink examples with current Next/dev behavior, documents the cross-graph ID constraints, and passes the Hugo build. Independent review score: 9.3/10, with no Critical or Important findings.

@dosubot dosubot Bot added the lgtm This PR has been approved by a maintainer label Aug 11, 2026

@imbajin imbajin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: yes. Summary: The quick-start mixes the released 2.3.13 distribution with dev-only HugeGraph connector APIs and contains several non-runnable Docker, Kafka, and migration examples. Evidence: the 2.3.13 HugeGraphSinkFactory requires schema_config; official SeaTunnel Docker/Source/Kafka documentation and exact-head source checks confirm the remaining incompatibilities.

Comment thread content/cn/docs/quickstart/toolchain/hugegraph-seatunnel-connector.md Outdated
Comment thread content/cn/docs/quickstart/toolchain/hugegraph-seatunnel-connector.md Outdated
Comment thread content/cn/docs/quickstart/toolchain/hugegraph-seatunnel-connector.md Outdated
Comment thread content/cn/docs/quickstart/toolchain/hugegraph-seatunnel-connector.md Outdated
Comment thread content/cn/docs/quickstart/toolchain/hugegraph-seatunnel-connector.md Outdated
Comment thread content/cn/docs/quickstart/toolchain/hugegraph-seatunnel-connector.md Outdated
Comment thread content/cn/docs/quickstart/toolchain/hugegraph-seatunnel-connector.md Outdated
Comment thread content/cn/docs/quickstart/toolchain/hugegraph-seatunnel-connector.md Outdated
Comment thread content/cn/docs/quickstart/toolchain/hugegraph-seatunnel-connector.md Outdated

@imbajin imbajin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: no. Summary: The graph2graph prerequisite wording conflicts with the exact dev connector contract and should be corrected before users follow the migration steps. Evidence: the linked f1a1a0a SeaTunnel HugeGraph Sink documentation and the exact-head guide at line 259.

Comment thread content/cn/docs/quickstart/toolchain/hugegraph-seatunnel-connector.md Outdated

@bitflicker64 bitflicker64 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: yes. Summary: The guide matches upstream SeaTunnel on option names, defaults, client versions and dev status, but the section 4.2 edge example cannot pass startup validation against the schema declared in section 3.1, the 3.1 PropertyKey table is missing since, the CN introduction page loses its ecosystem overview, and a PR_DESCRIPTION.md scratch file is committed at the repository root. Evidence: upstream connector docs at tag 2.3.13 and at commit f1a1a0a, plus SchemaValidator, HugeGraphSink, HugeGraphSinkFactory and the mappers at 2.3.13. Not executed: no SeaTunnel, HugeGraph, MySQL or Kafka instance was started and Hugo was not run locally, so findings 3 to 5 come from reading the connector source rather than from an observed job failure.

Comment thread PR_DESCRIPTION.md Outdated
Comment thread content/cn/docs/introduction/_index.md Outdated
Comment thread content/cn/docs/quickstart/toolchain/hugegraph-seatunnel-connector.md Outdated
Comment thread content/cn/docs/quickstart/toolchain/hugegraph-seatunnel-connector.md Outdated
spr1lng and others added 12 commits September 13, 2026 17:17
docs: add Apache SeaTunnel connector integration guide (CN only)

Add HugeGraph-SeaTunnel Connector documentation as an ecosystem entry
point for users who want to sync data between HugeGraph and external
systems via Apache SeaTunnel.

Changes:
- Add CN SeaTunnel connector doc in quickstart/toolchain/
- Add SeaTunnel to toolchain table and architecture diagram in
  CN introduction page
- Follow existing Spark-connector documentation style and structure

Co-Authored-By: Claude <noreply@anthropic.com>
@
docs: add SeaTunnel integration architecture diagrams

Add GPT-generated illustrations for the SeaTunnel Connector documentation:

- Architecture diagram: HugeGraph + SeaTunnel data integration overview
- Source/Sink diagram: bidirectional data flow between HugeGraph and external systems
- Full-stack ecosystem diagram: complete HugeGraph toolchain visualization

Placed in connector doc and introduction page for visual enhancement.

Co-Authored-By: Claude <noreply@anthropic.com>
@
docs: fix PR review issues for SeaTunnel connector doc

Address reviewer feedback:

- B1: Fix SeaTunnel installation link (/docs/start-v2/ -> /docs/getting-started/)
- B2: Replace fake config params (url/graph/label) with real connector
  parameters (host/port/graph_name/mappings) from SeaTunnel docs
- C: Add version compatibility matrix, note Source connector is dev-only
- D: Add English docs synchronization (connector doc + introduction page)
- E: Fix ASCII diagram border alignment (all lines now 70 chars wide)

Co-Authored-By: Claude <noreply@anthropic.com>
@
docs: remove EN files, keep CN only for review

Revert English docs to original state; Chinese version will be
reviewed first before adding English back.

Co-Authored-By: Claude <noreply@anthropic.com>
@
fix: correct version matrix and connector doc link

- Java: 8+ -> 11+ (HugeGraph Client 1.5.0+ requires Java 11)
- SeaTunnel: 2.3.12+ -> 2.3.13+ (bundles HugeGraph Client 1.5.0)
- HugeGraph Server: 1.0.0+ -> 1.5.0+ (match client version in connector)
- Link: generic /docs/connector-v2/ -> specific HugeGraph Sink page

Co-Authored-By: Claude <noreply@anthropic.com>
@
- add a single-node import walkthrough with verification
- document HugeGraph Source reads and graph clone flows
- align examples and limits with SeaTunnel dev
- remove unrelated workflow and obsolete assets
- add Loader vs SeaTunnel selection guide for new users
- add Kafka / Flink+Spark engines / graph migration scenarios with collapsible sections
- add mermaid and ASCII diagrams per scenario
- align config options with connector-hugegraph source
- point SeaTunnel doc links to latest pages (Source doc links to dev branch file)
- move version matrix to top and deduplicate
- add Docker-first deployment guide (K8s link, binary as reference)
- replace FakeSource examples with LocalFile CSV
- collapse engine section to doc links
- add graph_space to all examples
- add multi-graph batch migration example
- drop license section, unnumbered references at end
- use schema_config (2.3.13 API) in quick-start and Kafka examples, scope mappings/Source to dev preview
- fix Docker path: note JDK8 image and missing plugins, add JDK11 Dockerfile sample, correct doc link
- fix container networking guidance (host.docker.internal / docker network)
- fix Kafka option topics -> topic
- fix host:port duplication in migration examples
- correct check_vertex description and diagram output props
- mark batch migration as pseudocode with per-table binding note
- clarify Tools, Loader, and SeaTunnel selection\n- align JDBC, Kafka, and dev migration examples\n- add concise diagrams and fold long configs\n- replace stale architecture asset and version links
- detect pre-regrouping release layouts
- retain flat ToolChain pages in archived sidebars
- keep historical navigation metadata stable
- remove terminal periods from SeaTunnel callouts
- keep Chinese and English quote styling aligned
- place Vermeer first in the computing navigation source

- keep materialized sections in the same order

- align sidebar child lookup order with the source tree
- move the Computer quick start into a page bundle\n- place the configuration reference under the Computer page\n- preserve the former config URL with bilingual aliases\n- keep historical version sidebars on flat routes\n- update navigation metadata and regression coverage
- make 0.5.6 the benchmark section index
- place the 0.4.4 report beneath the 0.5.6 page
- retain the legacy 0.4.4 route alias
- preserve archived navigation layouts and regression coverage
- introduce Vermeer before Computer in both languages
- align page overview with sidebar priority
- record Vermeer-first historical navigation trees
- validate archived metadata after the new page hierarchy
@imbajin imbajin changed the title docs: add SeaTunnel 3.0 dev integration guide docs: add SeaTunnel 3.0+ integration guide Sep 16, 2026

@bitflicker64 bitflicker64 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: no. Summary: The SeaTunnel guide at this head matches the connector on the 3.0.0-release branch, including the new max_insert_errors row, and the ToolChain, Computer and benchmark moves keep aliases for the old URLs. The two Chinese links to the migration section use a fragment that the heading does not generate, so they open the page at the top. Evidence: compared apache/seatunnel 35b2716...3.0.0-release (6 commits, no connector source changes) and read HugeGraphOptions and setup.md on that branch; checked front matter aliases, docs_nav.json and the versioning.py archive fallback; rendered the headings with Hugo 0.166 to get the IDs; latest-head CI (link validation, unit tests, versioned builds) is green.

Comment thread content/cn/docs/quickstart/toolchain/_index.md Outdated
Comment thread content/cn/docs/quickstart/toolchain/export-migration/_index.md Outdated
- move Sink and shared setup into the graph import section
- add a Source guide for graph2graph and graph2any jobs
- update bilingual ToolChain links and navigation
- retain the legacy monolithic route as an alias
- align fallback defaults with the 3.0+ HugeGraph Sink reference
- explain the explicit false setting used by examples
- document Loader's measured 1–2 million records/s peak with caveats
- keep Chinese and English comparison tables synchronized
- keep the 2.3.13 warning in one article tail section
- remove duplicate legacy-version references from Source guide
- preserve the shared setup link between import and export pages
- align Sink page titles with the import navigation style
- add support markers to the Loader, Tools, and SeaTunnel table
- describe SeaTunnel coverage of import, export, and migration
- call out SQL-CDC and dozens of connector types
- remove the HugeGraph prefix from the Tools page title
- use matching import and export wording in both languages
- identify SeaTunnel Source in the export page title
- keep sidebar labels consistent with the task categories
- replace the standalone legacy-version heading with a quote
- keep the 2.3.13 compatibility warning at the page end
- synchronize Chinese and English Sink pages
- document native Spark, Flink, and Zeta engine support
- explain direct Source to Sink flow without staging files
- clarify adaptation needed for existing in-memory DAGs
- link the official engine and Connector V2 references
- render the table legend as a callout
- add superscript citation markers to sourced claims
- order official references and align Chinese and English pages
- explain graph2graph as source graph A to target graph B
- show the Source to Transform to Sink data path
- distinguish graph2any exports to external systems
- incorporate current master documentation updates
- retain the SeaTunnel and ToolChain changes on the latest base
- resolve the pull request mergeability conflict
- group references by connector, setup, engine, and source
- preserve superscript citation mapping in both guides
- align Chinese and English reference lists
- remove duplicate unordered-list markers from references\n- keep grouped headings for quick scanning\n- retain superscript citation numbers in Chinese and English docs
- replace reference bullets with superscript entries\n- add matching citations to Source and LocalFile guidance\n- keep Chinese and English export pages in sync
- avoid versioned site resolving relative links under export pages\n- keep Chinese and English Source guides aligned\n- point citations to the existing import page route
- use absolute import guide links in rendered pages\n- keep versioned site validation on the canonical route\n- align Chinese and English Source references
- keep existing article files at their public routes\n- group ToolChain, Computer, and benchmark entries in the sidebar\n- update navigation metadata, links, and versioning tests
- accept already normalized legacy links\n- refresh historical navigation fingerprints\n- cover idempotent content repair with a regression test

@imbajin imbajin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Congrats~ We could enhance the doc in future work 🔢

@imbajin
imbajin merged commit 3287b60 into apache:master Sep 17, 2026
9 checks passed
@imbajin
imbajin deleted the add-seatunnel-integration-doc branch September 17, 2026 18:36
github-actions Bot pushed a commit that referenced this pull request Sep 17, 2026
Before → after: the ToolChain pages mixed Loader, Tools, and SeaTunnel, while the SeaTunnel path lacked a clear version boundary, migration flow, and bilingual examples → the navigation now separates visualization, import, and export/migration, with task-oriented Chinese and English guides.

---------

Co-authored-by: imbajin <jin@apache.org> 3287b60
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request lgtm This PR has been approved by a maintainer size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants