跳至正文
OpenAgentSkill
浏览已开放 · 目录记录日期 2026-11-21 · 自动更新和账户服务暂时关闭。
其他技能

neo4j-spark-skill

Use when reading from or writing to Neo4j with Apache Spark or Databricks using the

neo4j-contrib · 已有静态检查记录

安装与使用

运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。

npx skills add neo4j-contrib/neo4j-skills --skill neo4j-spark-skill

技能说明


name: neo4j-spark-skill description: Use when reading from or writing to Neo4j with Apache Spark or Databricks using the Neo4j Connector for Apache Spark 6.0 (org.neo4j.connectors:spark) or 5.x (org.neo4j:neo4j-connector-apache-spark). Covers SparkSession setup, DataFrame reads via labels/Cypher/relationship scan, DataFrame writes with SaveMode, node.keys for MERGE, relationship write mapping, partition and batch tuning, PySpark and Scala examples, Databricks cluster config, Databricks secrets for credentials, Delta Lake to Neo4j pipelines. Does NOT handle Cypher authoring — use neo4j-cypher-skill. Does NOT handle the Python bolt driver — use neo4j-driver-python-skill. Does NOT handle GDS algorithms — use neo4j-gds-skill. version: 1.0.3 allowed-tools: Bash WebFetch

Neo4j Connector for Apache Spark

When to Use

  • Reading Neo4j nodes/relationships into Spark DataFrames
  • Writing Spark DataFrames to Neo4j as nodes or relationships
  • Databricks notebooks connecting to Neo4j
  • Delta Lake → Neo4j ingestion pipelines
  • Partitioned parallel reads from large Neo4j graphs

When NOT to Use

  • Python bolt driver / execute_query → neo4j-driver-python-skill
  • Cypher query writing → neo4j-cypher-skill
  • GDS graph algorithms → neo4j-gds-skill
  • Spring Boot + Neo4j → neo4j-spring-data-skill

Version Matrix

ConnectorSparkScalaJavaDatabricks RuntimeNeo4jMaven coordinate
6.0.x4.0, 4.12.1317+17.3 LTS5.x, 2025.x, 2026.xorg.neo4j.connectors:spark:6.0.0-s_2.13
5.5.x / 5.4.x3.4, 3.52.12, 2.138+14.3–16.4 LTS4.4, 5.x, 2025.x, 2026.xorg.neo4j:neo4j-connector-apache-spark_2.13:5.5.0_for_spark_3

Group ID changed in 6.0 — org.neo4j:neo4j-connector-apache-spark_<scala> is now a relocation POM pointing at org.neo4j.connectors:spark. On Spark 3.x stay on 5.5.x.

6.0 breaking changes

ChangeMigration
Spark baseline 3.5 → 4.0/4.1; Scala 2.12 and Java 8–11 droppedUpgrade to 5.5.0 first, then Spark 4.x + Scala 2.13 + Java 17
Maven coordinate org.neo4j.connectors:spark:<version>-s_2.13Replace old _for_spark_3 coordinate
schema.optimization.type removedschema.optimization.node.keys, schema.optimization.relationship.keys, schema.optimization
$stream.offset in partitioned reads removedUse partitions + query.count
;-separated multi-statement script removedscript.1, script.2, … script.N — executed in numbered order
relationship.save.strategy default native → keysSet .option("relationship.save.strategy", "native") explicitly to keep old behaviour
query option rewritten for Data Source V2 predicate push-downNo action; verify plans on upgrade

Setup

Standalone Spark (PySpark)

from pyspark.sql import SparkSession

spark = (SparkSession.builder
    .appName("neo4j-app")
    .config("spark.jars.packages",
            "org.neo4j.connectors:spark:6.0.0-s_2.13")   # Spark 3.x: org.neo4j:neo4j-connector-apache-spark_2.13:5.5.0_for_spark_3
    .config("neo4j.url", "neo4j+s://xxxx.databases.neo4j.io")
    .config("neo4j.authentication.type", "basic")
    .config("neo4j.authentication.basic.username", "neo4j")
    .config("neo4j.authentication.basic.password", "password")
    .getOrCreate())

Standalone Spark (Scala)

val spark = SparkSession.builder
  .appName("neo4j-app")
  .config("spark.jars.packages",
    "org.neo4j.connectors:spark:6.0.0-s_2.13")
  .config("neo4j.url", "neo4j+s://xxxx.databases.neo4j.io")
  .config("neo4j.authentication.type", "basic")
  .config("neo4j.authentication.basic.username", "neo4j")
  .config("neo4j.authentication.basic.password", "password")
  .getOrCreate()

Databricks — Cluster Installation

  1. Cluster → Libraries → Install New → Maven
  2. Coordinate org.neo4j.connectors:spark:6.0.0-s_2.13 on DBR 17.3 LTS; org.neo4j:neo4j-connector-apache-spark_2.13:5.5.0_for_spark_3 on DBR 14.3–16.4 LTS
  3. Cluster → Advanced Options → Spark tab — add config:
    neo4j.url neo4j+s://xxxx.databases.neo4j.io
    neo4j.authentication.type basic
    neo4j.authentication.basic.username {{secrets/neo4j/username}}
    neo4j.authentication.basic.password {{secrets/neo4j/password}}
    
  4. Use Single user access mode (Unity Catalog shared mode not supported)

Databricks — Secrets (preferred over plaintext)

# Store credentials once:
# databricks secrets create-scope --scope neo4j
# databricks secrets put --scope neo4j --key url
# databricks secrets put --scope neo4j --key username
# databricks secrets put --scope neo4j --key password

neo4j_url  = dbutils.secrets.get(scope="neo4j", key="url")
neo4j_user = dbutils.secrets.get(scope="neo4j", key="username")
neo4j_pass = dbutils.secrets.get(scope="neo4j", key="password")

spark.conf.set("neo4j.url", neo4j_url)
spark.conf.set("neo4j.authentication.type", "basic")
spark.conf.set("neo4j.authentication.basic.username", neo4j_user)
spark.conf.set("neo4j.authentication.basic.password", neo4j_pass)

Key Configuration Options

OptionDescriptionDefault
neo4j.urlBolt/Neo4j URI— (required)
neo4j.authentication.typenone, basic, kerberos, bearerbasic
neo4j.authentication.basic.usernameUsernamedriver default
neo4j.authentication.basic.passwordPassworddriver default
neo4j.authentication.bearer.tokenBearer token—
neo4j.databaseTarget databasedriver default
neo4j.access.moderead or writeread
neo4j.encryption.enabledTLS (ignored with +s/+ssc URI)false
neo4j.db.transaction.timeoutTransaction timeout (ms)driver default
neo4j.db.transaction.metadata.<key>Custom transaction metadata surfaced in que
阅读完整来源 ↗