SemQL Specification

Version 1.0 · Noetive Broker

SemQL is a query language for defining geometric predicates over high-dimensional semantic space. It combines a human-readable SQL-like text syntax with a JSON wire format for programmatic use. Both representations are equivalent and losslessly interconvertible.

1. Grammar

1.1 Text Syntax (EBNF)

query           = match_clause
                  [ namespace_clause ]
                  [ window_clause ]
                  [ limit_clause ] ;

match_clause    = "MATCH" expression ;

expression      = term { "OR" term } ;
term            = factor { "AND" factor } ;
factor          = [ "NOT" ] atom ;
atom            = clause | "(" expression ")" ;

clause          = distance_clause
                | direction_clause
                | contrast_clause ;

(* --- Clause definitions --- *)

distance_clause = "DISTANCE" "(" anchor ")" [ distance_opts ] ;
distance_opts   = "WITHIN" number
                | "TOP" integer ;

direction_clause = "DIRECTION" "(" anchor_list ")" [ direction_opts ] ;
direction_opts   = "CONE" number ;

contrast_clause  = "CONTRAST" "("
                     "ATTRACT" anchor_list ","
                     "REPEL" anchor_list
                   ")" [ "WITHIN" number ] ;

(* --- Namespace and window --- *)

namespace_clause   = "NAMESPACE" namespace_selector ;
namespace_selector = namespace_item { "," namespace_item } ;
namespace_item     = namespace_ref | "ALL" | "GLOBAL" ;
namespace_ref      = [ "NOT" ] string_literal ;

window_clause     = "WINDOW" duration ;
limit_clause      = "LIMIT" integer ;

(* --- Primitives --- *)

anchor            = string_literal                      (* natural language text *)
                  | vector_literal ;                    (* raw float vector *)

anchor_list       = "[" anchor { "," anchor } "]"
                  | anchor ;

vector_literal    = "[" number { "," number } "]" ;
string_literal    = '"' { character } '"' ;
duration          = integer time_unit ;
time_unit         = "s" | "m" | "h" | "d" | "w" ;
number            = float | integer ;

1.2 Text Syntax Examples

Simple nearest-neighbor search:

MATCH DISTANCE("payment reconciliation failure")
NAMESPACE "org:acme-corp"
LIMIT 20

Composed multi-clause subscription:

MATCH DIRECTION(["customer frustration", "billing complaint"]) CONE 0.4
  AND CONTRAST(
        ATTRACT ["enterprise", "high-value account"],
        REPEL   ["self-serve", "free tier"]
      )
NAMESPACE "org:acme-corp", GLOBAL
WINDOW 48h

Boolean composition with grouping:

MATCH (
        DIRECTION("payment gateway") CONE 0.3
    AND DISTANCE("timeout error") WITHIN 0.2
  )
  OR (
        DIRECTION("database connection") CONE 0.3
    AND CONTRAST(
          ATTRACT ["connection pool", "resource exhaustion"],
          REPEL   ["query optimization", "index tuning"]
        )
  )
NAMESPACE "org:acme-corp"

Negation:

MATCH DIRECTION("infrastructure security") CONE 0.3
  AND NOT DISTANCE("routine monitoring") WITHIN 0.2
NAMESPACE "org:acme-corp", NOT "org:acme-staging"

2. JSON Wire Format

The JSON format is the canonical representation used by the Noetive API. Every text query compiles to this format. Clients may submit queries in either format.

2.1 Top-Level Query

{
  "match": <expression>,
  "namespace": <namespace_selector>,   // optional
  "window": "<ISO 8601 duration>",     // optional
  "limit": <integer>                   // optional
}

2.2 Expressions

Boolean composition uses and, or, and not keys. A bare clause object is also a valid expression.

AND:

{ "and": [ <expression>, <expression>, ... ] }

OR:

{ "or": [ <expression>, <expression>, ... ] }

NOT:

{ "not": <expression> }

Bare clause (implicit single-expression):

{ "distance": { ... } }

2.3 Anchors

An anchor is either a string (natural language text the broker embeds) or an array of numbers (raw embedding vector).

"payment reconciliation failure"

[0.182, -0.041, 0.389, 0.057]

2.4 Clauses

DISTANCE

Nearest-neighbor sphere in embedding space.

Field	Type	Required	Default	Description
`anchor`	string \| float[]	yes	—	Center point
`within`	number	no	—	Minimum cosine similarity (0.0–1.0); 0 means no floor
`top_k`	integer	no	—	Return top-k nearest
`metric`	string	no	`"cosine"`	`"cosine"`, `"euclidean"`, or `"dot"`

Exactly one of within or top_k should be provided. If neither is set, the clause acts as a scoring signal without a hard threshold.

{ "distance": { "anchor": "payment failure", "within": 0.3 } }

{ "distance": { "anchor": [0.18, -0.04, 0.39], "top_k": 10 } }

DIRECTION

Cone in embedding space — thematic alignment regardless of magnitude.

Field	Type	Required	Default	Description
`toward`	string \| string[]	yes	—	Direction concept(s)
`cone`	number	no	`0.3`	Half-angle in radians

When toward is an array, the direction vector is the normalized mean of all embedded concepts.

{ "direction": { "toward": ["customer frustration", "billing complaint"], "cone": 0.4 } }

{ "direction": { "toward": "infrastructure security", "cone": 0.25 } }

CONTRAST

Attract/repel vector arithmetic.

Field	Type	Required	Default	Description
`attract`	string[]	yes	—	Concepts to attract toward
`repel`	string[]	no	—	Concepts to repel from
`within`	number	no	—	Minimum cosine similarity from composite vector (0.0–1.0); 0 means no floor

When both attract and repel are present, the composite vector is normalize(mean(embed(attract)) - mean(embed(repel))). When repel is absent, the composite vector is normalize(mean(embed(attract))).

{
  "contrast": {
    "attract": ["enterprise", "high-value account"],
    "repel": ["self-serve", "free tier"],
    "within": 0.4
  }
}

2.5 Namespace Selector

{
  "include": ["org:acme-corp", "org:acme-eu"],
  "exclude": ["org:acme-staging"],
  "global": true,
  "all": false
}

Field	Type	Default	Description
`include`	string[]	`[]`	Namespace names to include (exact match — no wildcards)
`exclude`	string[]	`[]`	Namespace names to exclude (exact match — no wildcards)
`global`	boolean	`false`	Include the well-known global namespace
`all`	boolean	`false`	Include every namespace the caller has access to

The text-form keywords GLOBAL and ALL map to these booleans and may appear as items inside the comma-separated list alongside literal names (e.g. NAMESPACE "monsters", GLOBAL or NAMESPACE "dinoco", ALL). global and all are independent — either, both, or neither may be set.

3. Full Examples

3.1 Finance: Risk Signal Detection

Text:

MATCH DIRECTION(["sovereign debt concern", "emerging market stress"]) CONE 0.4
  AND CONTRAST(
        ATTRACT ["credit spreads", "bond yields"],
        REPEL   ["routine monetary policy", "scheduled rate decision"]
      )
NAMESPACE "org:hedgefund", GLOBAL

JSON:

{
  "match": {
    "and": [
      {
        "direction": {
          "toward": ["sovereign debt concern", "emerging market stress"],
          "cone": 0.4
        }
      },
      {
        "contrast": {
          "attract": ["credit spreads", "bond yields"],
          "repel": ["routine monetary policy", "scheduled rate decision"]
        }
      }
    ]
  },
  "namespace": { "include": ["org:hedgefund"], "global": true }
}

3.2 Ad Serving: Contextual Matching

Text:

MATCH DISTANCE("luxury automotive lifestyle") WITHIN 0.3
  AND DIRECTION(["purchase intent", "aspiration"]) CONE 0.3
  AND CONTRAST(
        ATTRACT ["premium brand", "high-income lifestyle"],
        REPEL   ["budget", "discount", "coupon"]
      )
NAMESPACE "adnet:publisher-inventory"

JSON:

{
  "match": {
    "and": [
      { "distance": { "anchor": "luxury automotive lifestyle", "within": 0.3 } },
      { "direction": { "toward": ["purchase intent", "aspiration"], "cone": 0.3 } },
      {
        "contrast": {
          "attract": ["premium brand", "high-income lifestyle"],
          "repel": ["budget", "discount", "coupon"]
        }
      }
    ]
  },
  "namespace": { "include": ["adnet:publisher-inventory"] }
}

3.3 Retail: Demand Signal

Text:

MATCH DIRECTION(["unmet need", "product frustration"]) CONE 0.4
  AND CONTRAST(
        ATTRACT ["home kitchen", "small appliance"],
        REPEL   ["professional equipment", "commercial grade"]
      )
NAMESPACE "org:retailco-eu", "org:retailco-us", NOT "org:retailco-internal"
WINDOW 30d

JSON:

{
  "match": {
    "and": [
      {
        "direction": {
          "toward": ["unmet need", "product frustration"],
          "cone": 0.4
        }
      },
      {
        "contrast": {
          "attract": ["home kitchen", "small appliance"],
          "repel": ["professional equipment", "commercial grade"]
        }
      }
    ]
  },
  "namespace": { "include": ["org:retailco-eu", "org:retailco-us"], "exclude": ["org:retailco-internal"] },
  "window": "P30D"
}

4. Duration Format

Text syntax shorthand: 30s, 15m, 24h, 7d, 4w.

JSON format: ISO 8601 durations — "PT30S", "PT15M", "PT24H", "P7D", "P28D".

The text parser accepts both formats. The JSON serializer always emits ISO 8601.

5. Reserved Words

MATCH AND OR NOT DISTANCE DIRECTION CONTRAST ATTRACT REPEL
WITHIN TOP CONE WINDOW NAMESPACE ALL GLOBAL LIMIT

All reserved words are case-insensitive in the text syntax. The JSON format uses lowercase keys exclusively.

6. Future Work

The following features are planned but not part of v1.0:

Trajectory clause — semantic drift detection over time
Region clause — learned or explicit boundaries in embedding space
Anomaly clause — novelty detection relative to history
Resonance clause — periodic pattern detection
Named references — $variable, MSG("id"), CENTROID("id") anchor types
Composite anchors — weighted combinations of multiple anchors
Clause weights — soft scoring contribution per clause