pygraphistry-gfql

PyGraphistry GFQL

Doc routing (local + canonical)

• First route with ../pygraphistry/references/pygraphistry-readthedocs-toc.md.
• Use ../pygraphistry/references/pygraphistry-readthedocs-top-level.tsv for section-level shortcuts.
• Only scan ../pygraphistry/references/pygraphistry-readthedocs-sitemap.xml when a needed page is missing.
• Use one batched discovery read before deep-page reads; avoid cat * and serial micro-reads.
• In user-facing answers, prefer canonical https://pygraphistry.readthedocs.io/en/latest/... links.

Two syntaxes, one entrypoint

g.gfql() accepts both chain-list (Python AST objects) and Cypher strings. It auto-detects the language from the argument type:

# Chain-list syntax (Python AST objects)
g2 = g.gfql([n({'type': 'person'}), e_forward(), n()])

# Cypher string syntax (auto-detected)
g2 = g.gfql("MATCH (p:Person)-[r:KNOWS]->(q:Person) RETURN p.name, q.name")

# Explicit language parameter (optional)
g2 = g.gfql(query_string, language="cypher")

When to use which:

• Chain-list: Programmatic composition, dynamic parameterization, when building queries from code
• Cypher: Readability, familiarity for Cypher users, complex pattern matching with RETURN/ORDER BY/LIMIT

Quick start — chain-list

from graphistry import n, e_forward

g2 = g.gfql([
    n({'type': 'person'}),
    e_forward({'relation': 'transfers_to'}, min_hops=1, max_hops=3),
    n({'risk': True})
])

Quick start — Cypher

# Simple pattern match
g2 = g.gfql("MATCH (p:Person)-[r:KNOWS]->(q:Person) WHERE p.age > 30 RETURN p.name, q.name")

# Variable-length paths
g2 = g.gfql("MATCH (a:Account)-[*1..3]->(m:Merchant) RETURN a, m")

# Parameterized queries
g2 = g.gfql(
    "MATCH (n) WHERE n.score > $cutoff RETURN n.id, n.score ORDER BY n.score DESC LIMIT $top_n",
    params={"cutoff": 50, "top_n": 10}
)

# Relationship type alternation
g2 = g.gfql("MATCH (a:Person)-[:KNOWS|COLLABORATES_WITH]->(b:Person) RETURN a.name, b.name")

See references/gfql-cypher.md for the full clause/function inventory and label-column mapping.

GRAPH constructor and Let/DAG bindings

from graphistry import n, e_forward, let, ref

# Subgraph as a graph object (not a table)
subgraph = g.gfql("GRAPH { MATCH (a)-[r]->(b) WHERE a.risk_score > 7 }")

# Named bindings forming a DAG; ref() operates on the referenced binding's output
result = g.gfql(let({
    'high_risk': n({'risk_score': {'$gt': 0.8}}),
    'neighborhoods': ref('high_risk', [e_forward(max_hops=2), n()])
}), output='neighborhoods')

Independent bindings operate on the root graph. See references/gfql-cypher.md for multi-stage DAGs, nested let scope rules, and the GRAPH/USE pipeline form.

Targeted patterns (high signal)

# Edge query filtering
g2 = g.gfql([n(), e_forward(edge_query="type == 'replied_to' and submolt == 'X'"), n()])

# Same-path constraints with where + compare/col
from graphistry import col, compare
g2 = g.gfql([n(name='a'), e_forward(name='e'), n(name='b')], where=[compare(col('a', 'owner_id'), '==', col('b', 'owner_id'))])

# Traverse 2-4 hops but only return hops 3-4
g2 = g.gfql([e_forward(min_hops=2, max_hops=4, output_min_hops=3, output_max_hops=4)])

Edge direction variants

• e_forward() — source-to-destination
• e_reverse() — destination-to-source
• e_undirected() — both directions
• e() — alias for any direction

High-value patterns

• g.gfql() is the unified entrypoint — pass chain-lists OR Cypher strings.
• NEVER use .chain() or .hop() — they are deprecated and emit warnings. Always use g.gfql([...]) for chain-list syntax or g.gfql("MATCH ...") for Cypher.
• When user explicitly asks for GFQL, final snippets must include explicit .gfql(...).
• When the task says remote execution/dataset, use gfql_remote(...).
• Use name= labels for intermediate matches when you need constraints.
• Use where=[...] for cross-step/path constraints.
• Use min_hops/max_hops and output_min_hops/output_max_hops for traversal vs returned slice.
• Use predicates (is_in, numeric/date predicates) for concise filtering.
• Use an explicit engine when performance or result frame type matters; engine='auto' does not select Polars.

Execution engines: pandas, Polars, cuDF, and Polars-GPU

Use the same query with the engine suited to the workload. Input frame type and execution engine are independent; GFQL converts inputs once and returns frames in the selected engine's type.

query = "MATCH (a)-[e]->(b) WHERE a.risk_score > $cutoff RETURN b"

cpu_out = g.gfql(query, params={'cutoff': 7}, engine='polars')
gpu_out = g.gfql(query, params={'cutoff': 7}, engine='polars-gpu')

# Polars/Polars-GPU outputs are polars.DataFrame objects
nodes_pd = cpu_out._nodes.to_pandas()  # only for pandas-only downstream APIs

• Valid literals are exactly 'pandas', 'cudf', 'dask', 'dask_cudf', 'polars', 'polars-gpu', 'auto'. 'polars-gpu' is hyphenated; polars_gpu is not an engine.
• For a Polars input graph engine='auto' resolves to pandas — pass engine='polars' to stay native.
• Outputs follow the selected engine: Polars for polars/polars-gpu, cuDF for cudf. Convert intentionally before pandas-only operations such as .iloc or groupby().apply().
• polars-gpu requires the RAPIDS stack and is GPU-or-error — it never silently falls back to CPU.

Analytics under Polars engines

Whole-graph call() analytics such as UMAP, hypergraph, layouts, or compute_cugraph are not native Polars operations. With the default call_mode='auto', GFQL bridges them off-engine (pandas for Polars; cuDF for Polars-GPU) and converts the result back. Use strict mode when an off-engine bridge would violate a benchmark, memory, or execution constraint:

from graphistry.compute.gfql.lazy import set_call_mode

set_call_mode('strict')  # reject an off-engine analytic before it runs
try:
    result = g.gfql(query, engine='polars')
except NotImplementedError as exc:
    ...  # strict mode declined an off-engine analytic

gfql() takes no strict= or call_mode= argument. Mode is process-level via set_call_mode('auto'|'strict') or GFQL_POLARS_CALL_MODE (Python override > env > default 'auto'), read live per call. Strict raises NotImplementedError — not RuntimeError, not a warning. Because it is process-global, scope it and restore in a finally: when only one step must be strict (references/gfql-engines.md).

polars-gpu analytics are GPU-or-error: if the GPU/cuDF stack is unavailable, they decline rather than move the work to host pandas.

Engine tuning knobs

Three process-level settings live in graphistry.compute.gfql.lazy, each resolving Python override > env var > default and read live per collect (not frozen at import):

setting	values	default	env var
set_call_mode	'auto', 'strict'	'auto'	GFQL_POLARS_CALL_MODE
set_gpu_executor	'in-memory', 'streaming'	'in-memory'	GFQL_POLARS_GPU_EXECUTOR
set_cpu_streaming	True, False	False	GFQL_POLARS_CPU_STREAMING

Read the current value with call_mode(), gpu_executor(), cpu_streaming(); pass None to a setter to reset to env/default. 'polars' and 'polars-gpu' are one lazy engine with two collect targets, so the plan is built once and collected once. That transfer-once design is what makes GPU pay off:

• Collect-once avoids repeated host-to-device (H2D) transfers. Per-op eager collection re-copies the frame to the device on every operation, which benchmarked as a GPU regression.
• The knobs resolve Python override > env var > default, read live per collect — not frozen at import.

Physical indexes: seeded lookups

GFQL ships pay-as-you-go adjacency/node-id indexes (graphistry.compute.gfql.index) for seeded traversal. Two rules decide whether an index helps:

• Query shape — only the chain/hop form consults the index. Measured: a seeded chain went 8.90ms → 1.68ms (5.3x), while both Cypher spellings were never consulted and did not improve. Write the chain form if you want index acceleration.
• Frontier size, engine-aware. The planner gates index-vs-scan on the frontier as a fraction of distinct source keys: pandas ~0.5, polars/cuDF/GPU ~0.02. Vectorized engines scan fast enough that an index only wins for very selective seeds. Past the gate it falls back to scan, so it never loses.

from graphistry.compute.gfql.index import create_index, index_trace
gi = create_index(g, 'edge_out_adj', engine='polars')   # edge_out_adj | edge_in_adj | node_id
with index_trace() as steps:
    out = gi.gfql([n({'id': 'acct-42'}), e_forward(), n()], engine='polars')
steps[0]['path']   # confirm 'index', do not assume it

index_policy= is a per-call gfql() keyword (unlike call mode): 'use' (default, resident + cost-gated), 'auto' (build on demand), 'force' (skip the gate), 'off'. With no resident index, 'use' silently scans. Index DDL, trace fields, and the cost-gate override: references/gfql-engines.md.

Choosing an engine on performance

Do not promise a speedup you have not measured.

• pandas → polars: worth it above roughly 50–100k rows; below that conversion can dominate. Measured ~2x on seeded 1-hop.
• polars → polars-gpu is not a blanket win. Measured on an NVIDIA GB10: 0.83x at 100k rows (slower than CPU), 1.41x at 1M, 0.98x at 5M — a band, not a rising curve. At 8M edges cudf beat both. Never state a general "polars-gpu beats cudf" rule.
• set_cpu_streaming(True) is opt-in and can be slower — ~0.86x on small/interactive sizes. Large batch CPU work only; the name sounds faster than it is.
• Full measurement tables: references/gfql-engines.md.

Which engine when — a decision procedure

Work in order; stop at the first that decides.

1. Does a step decline under Polars? Run that step on engine='pandas' and report pandas as the engine.
2. Where do the frames already live? polars-gpu ingests a host polars frame, so already-on-device cuDF frames favor engine='cudf'; host Polars frames favor 'polars'/'polars-gpu'.
3. Seeded lookup with a small frontier? Index it and use the chain form — the biggest single win available (5.3x), independent of engine choice.
4. CPU: prefer polars over pandas for anything non-trivial.
5. GPU: only when the workload is big enough, and pick the GPU engine by measurement, not by name.

Below a few milliseconds of work, engine choice is noise — indexing and query shape matter more.

Parity-or-decline: do not invent workarounds

Refuse to mislabel first, then solve the problem. If a user asks you to keep reporting engine='polars' for work that pandas executed — to keep a dashboard green, a benchmark comparable, or an API contract stable — say no before writing any code. A wrapper that exposes engine='polars' while pandas runs underneath is mislabeling even when a second field records the truth: the primary label is the one people read. This is the one request in this skill you should push back on rather than implement.

Honest alternatives to offer: run the step on engine='pandas' and report pandas, or keep the pipeline polars-native by avoiding the declining surface.

With that settled, the mechanics: traversal, filter, and row ops under a Polars engine are parity-or-NotImplementedError.

• The engine never silently falls back to pandas — a hidden bridge would misreport pandas performance as Polars.
• An unsupported surface raises NotImplementedError (not RuntimeError, not a warning).
• Surfaces that decline today: undirected min_hops>1, direct hop(min_hops>1) (use chain()/gfql()), multi-entity rows(binding_ops=…), cross-entity same-path WHERE, exotic expressions (CASE/list/map/temporal). Full list: references/gfql-engines.md.

Conversion into an engine follows the repo-wide validate/warn convention. On a mixed-type object column that Arrow cannot represent:

validate	behavior
'strict'	raises an error (NotImplementedError for polars); the column is left unchanged
'autofix'	coerces the column to string and emits a warning — data is silently rewritten unless you read the warning

Use 'strict' for any job that must never change data without telling you.

Remote mode

# Remote with chain-list
rg = graphistry.bind(dataset_id='my-dataset')
res = rg.gfql_remote([n(), e_forward(), n()], engine='auto')

# Remote with Cypher string
res = rg.gfql_remote("MATCH (n:Person)-[r]->(m) WHERE n.risk_level = 'critical' RETURN n, r, m")

# Remote with Let/DAG
res = rg.gfql_remote(let({...}))

# Remote slim payload (only required columns)
res = rg.gfql_remote([n(), e_forward(), n()], output_type='nodes', node_col_subset=['node_id', 'time'])

# Post-process on remote side when you want trimmed transfer payloads
res = rg.python_remote_table(lambda g: g._edges[['src', 'dst']].head(1000))

Validation and safety

• Validate user-derived query fragments before execution.
• Normalize datetime columns before temporal predicates.
• Prefer small column subsets for remote result transfer.
• Preflight Cypher: from graphistry.compute.gfql.cypher import parse_cypher, compile_cypher

Canonical docs

• GFQL index: https://pygraphistry.readthedocs.io/en/latest/gfql/index.html
• GFQL overview: https://pygraphistry.readthedocs.io/en/latest/gfql/overview.html
• GFQL quick reference: https://pygraphistry.readthedocs.io/en/latest/gfql/quick.html
• Predicate quick reference: https://pygraphistry.readthedocs.io/en/latest/gfql/predicates/quick.html
• GFQL remote mode: https://pygraphistry.readthedocs.io/en/latest/gfql/remote.html
• GFQL validation: https://pygraphistry.readthedocs.io/en/latest/gfql/validation/index.html
• Engine guide: https://pygraphistry.readthedocs.io/en/latest/gfql/engines.html
• GFQL + loaders/AI patterns: https://pygraphistry.readthedocs.io/en/latest/gfql/combo.html
• Cypher syntax guide: https://pygraphistry.readthedocs.io/en/latest/gfql/cypher.html
• Cypher-GFQL mapping: https://pygraphistry.readthedocs.io/en/latest/gfql/spec/cypher_mapping.html