跳至正文
OpenAgentSkill
浏览已开放 · 目录记录日期 2026-11-21 · 自动更新和账户服务暂时关闭。
搜索与研究

neo4j-kafka-skill

Configure and operate the Neo4j Connector for Kafka (sink + source) and the

neo4j-contrib · 已有 AI 审核记录

安装与使用

运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。

npx skills add neo4j-contrib/neo4j-skills --skill neo4j-kafka-skill

技能说明


name: neo4j-kafka-skill description: Configure and operate the Neo4j Connector for Kafka (sink + source) and the native Neo4j CDC API. Covers Cypher/Pattern/CUD sink strategies, CDC-based and query-based source, exactly-once semantics, DLQ error handling, Confluent Cloud managed connector, schema registry (Avro/JSON), and native db.cdc.query cursor-loop patterns (Neo4j 5.13+ Enterprise/Aura BC/VDC). Use when streaming Kafka events into Neo4j, streaming Neo4j changes to Kafka, or querying Neo4j change events without Kafka. Does NOT handle Cypher query authoring — use neo4j-cypher-skill. Does NOT handle bulk CSV/file import — use neo4j-import-skill. Does NOT handle GDS algorithms — use neo4j-gds-skill. allowed-tools: Bash WebFetch version: 1.0.4

Neo4j Kafka Skill

When to Use

  • Writing Kafka events into Neo4j (sink connector — Cypher, Pattern, CDC, CUD strategies)
  • Streaming Neo4j changes to Kafka topics (source connector — CDC or query-based)
  • Querying Neo4j change events natively without Kafka (db.cdc.query)
  • Configuring Confluent Cloud managed Neo4j sink connector
  • Setting up schema registry (Avro/JSON Schema) for typed Kafka messages
  • Enabling exactly-once semantics or dead-letter queue on sink

When NOT to Use

  • Cypher query authoring → neo4j-cypher-skill
  • Bulk CSV/JSON file import → neo4j-import-skill
  • GDS algorithms → neo4j-gds-skill
  • Live app write patterns → neo4j-cypher-skill

Decision Table — Which connector strategy?

Use caseStrategy
Custom transformation of Kafka payload → graphSink: Cypher
Mirror another Neo4j CDC sourceSink: CDC (schema or source-id sub-strategy)
Map Kafka JSON fields to graph nodes/rels with no codeSink: Pattern
Consume pre-formatted CUD JSON messagesSink: CUD
Stream all Neo4j changes to Kafka (real-time)Source: CDC (Neo4j 5.13+ EE/Aura BC/VDC)
Stream specific query results on a scheduleSource: Query
Consume CDC events in-process, no KafkaNative CDC API (db.cdc.query)

Prerequisites

  • Neo4j Connector for Kafka ≥ 5.5.2 (download from neo4j.com/labs/kafka or Confluent Hub) — 5.5.2 processes incoming messages per partition instead of per topic; 5.5.1 forces static labels in generated queries
  • Kafka Connect ≥ 3.x or Confluent Platform ≥ 7.x
  • For CDC source/sink: Neo4j 5.13+ Enterprise Edition, AuraDB Business Critical, or AuraDB VDC
  • For query source: any Neo4j edition
  • Java 11+

Core Connection Config (all connectors)

{
  "neo4j.uri": "neo4j+s://your-instance.databases.neo4j.io:7687",
  "neo4j.authentication.type": "BASIC",
  "neo4j.authentication.basic.username": "neo4j",
  "neo4j.authentication.basic.password": "${file:/opt/secrets.properties:neo4j.password}",
  "neo4j.database": "neo4j"
}

Authentication types: BASIC | BEARER | KERBEROS | CUSTOM | NONE

Never hardcode passwords — use Kafka Connect secrets provider (${file:...} or ${env:...}).


Sink Connector

Strategy 1 — Cypher

Connector auto-prepends UNWIND $events AS __value — write query using __value:

{
  "connector.class": "org.neo4j.connectors.kafka.sink.Neo4jConnector",
  "topics": "person-creates,person-updates",
  "neo4j.uri": "neo4j+s://...",
  "neo4j.authentication.type": "BASIC",
  "neo4j.authentication.basic.username": "neo4j",
  "neo4j.authentication.basic.password": "secret",
  "neo4j.cypher.topic.person-creates":
    "MERGE (p:Person {id: __value.id}) SET p += __value.properties",
  "neo4j.cypher.topic.person-updates":
    "MATCH (p:Person {id: __value.id}) SET p += __value.properties",
  "neo4j.cypher.bind-value-as": "__value",
  "neo4j.cypher.bind-key-as": "__key",
  "neo4j.cypher.bind-header-as": "__header"
}

MERGE pattern — idempotent upsert:

MERGE (p:Person {id: __value.id})
ON CREATE SET p.createdAt = datetime(), p += __value.properties
ON MATCH  SET p.updatedAt = datetime(), p += __value.properties

Strategy 2 — Pattern

No Cypher needed — map message fields to graph via pattern syntax:

{
  "neo4j.pattern.topic.users": "(:User{!userId, name, email})",
  "neo4j.pattern.topic.friendships":
    "(:User{!userId: from.userId})-[:KNOWS{since}]->(:User{!userId: to.userId})"
}

Pattern rules:

  • !prop = key property (used for MERGE)
  • prop: field.path = map from nested message field
  • * = map all message fields
  • -prop = exclude property (cannot mix with inclusions)

Strategy 3 — CDC (mirror another Neo4j)

{
  "neo4j.cdc.schema.topics": "neo4j-cdc-events"
}

Or with source-id tracking (stores elementId as property):

{
  "neo4j.cdc.source-id.topics": "neo4j-cdc-events",
  "neo4j.cdc.source-id.label-name": "SourceEvent",
  "neo4j.cdc.source-id.property-name": "sourceId"
}

Exactly-Once Semantics (EOS)

Requires: connector ≥ 5.3.0 (Cypher/Pattern/CDC strategies), connector ≥ 5.3.1 (CUD strategy), Kafka broker EOS support, and a NODE KEY constraint.

Step 1 — Create constraint:

CREATE CONSTRAINT kafka_offset_key IF NOT EXISTS
FOR (n:__KafkaOffset)
REQUIRE (n.strategy, n.topic, n.partition) IS NODE KEY;

Step 2 — Add to connector config:

{
  "neo4j.eos-offset-label": "__KafkaOffset"
}

Without EOS: connector provides at-least-once — write idempotent Cypher (MERGE, not CREATE).

Error Handling / DLQ

{
  "errors.tolerance": "all",
  "errors.log.enable": "true",
  "errors.log.include.messages": "true",
  "errors.deadletterqueue.topic.name": "neo4j-dlq",
  "errors.deadletterqueue.context.headers.enable": "true",
  "errors.deadletterqueue.topic.replication.factor": "3"
}

errors.tolerance=none (default) — stops on first error. Use all + DLQ for production.


Source Connector

CDC-Based Source (recommended, Neo4j 5.13+)

{
  "connector.class": "org.neo4j.connectors.kafka.source.Neo4jConnec
阅读完整来源 ↗