NaviServer Built-in Commands – 5.1.0
ns_json - JSON parser and generator for HTTP/WebAuthn needs.
The command ns_json provides a JSON (RFC 8259) parser and generator integrated directly into NaviServer.
The parser is designed for server-side use cases where predictable behavior, explicit type handling, and tight integration with NaviServer data structures are required. It parses exactly one JSON value from the input and reports errors via Tcl exceptions.
In addition to returning conventional Tcl structures, the command supports output formats that preserve JSON type information and lexical representations explicitly (the triple format|. This enables reliable round-tripping between JSON input and output and avoids unintended data transformations.
The ns_json triples ensemble provides structured access to the internal triples representation produced by ns_json parse -output triples. These commands allow querying and updating JSON data in its typed intermediate form without re-parsing or serializing JSON text.
The implementation performs strict validation of JSON syntax as defined by RFC 8259. Optional validation can be enabled to enforce additional constraints on numeric values when integrating with Tcl numeric semantics. The command does not perform application-level transformations or coercions beyond these well-defined checks.
Features such as comments, JSON5 extensions, or schema validation are not supported.
The following design goals guided the implementation:
Strict adherence to JSON syntax as defined in RFC 8259.
Deterministic and explicit handling of JSON types.
Preservation of numeric and string lexemes to support lossless round-trips.
Efficient parsing directly from in-memory buffers.
Support for incremental parsing via byte consumption reporting.
Integration with Tcl values and NaviServer ns_set structures.
No dependency on external JSON libraries or Tcl packages.
The command supports multiple output formats to cover common integration needs:
dict returns a Tcl representation using dict/list/scalars.
triples returns a flat list of triples NAME TYPE VALUE.
set returns a flattened ns_set suitable for form handling.
These last two formats make JSON type information explicit and are intended for programmatic processing and re-emission.
The ns_json command is not only intended as a utility for application developers, but is also designed as an integral part of NaviServer’s internal request processing infrastructure.
NaviServer has a long-standing model for handling structured request data, most notably for application/x-www-form-urlencoded and multipart/form-data requests, where incoming data is parsed early and exposed through well-defined core data structures such as ns_set. Higher-level facilities (e.g., page contracts, filters, and form handling) are built on top of these parsed representations.
The ns_json parser extends this model to JSON-based requests. It is implemented in C, exposes a C-level parsing interface, and integrates directly with NaviServer core abstractions. This allows JSON request bodies to be parsed automatically and made available to both internal code and application-level Tcl code in a uniform and predictable way.
In particular, the set output format is designed to mirror classical form handling as closely as possible. JSON objects and arrays are flattened into an ns_set, structural paths are encoded into keys, and JSON type information is preserved using sidecar fields. This enables existing infrastructure such as page contracts and request filters to operate on JSON input without requiring JSON-specific logic.
Beyond application-level use, ns_json is intended to be used internally by NaviServer itself. By exposing a small, well-defined C API and by relying on common core data structures, the same parser can be reused for tasks such as automatic parsing of JSON POST requests, incremental or streaming request bodies, and other protocol-level features.
Most existing Tcl JSON libraries focus on convenience for scripting and map JSON values directly to Tcl dicts and lists. While suitable for many application use cases, this approach loses explicit JSON type information and often normalizes numeric values in ways that prevent reliable round-tripping.
The triples output format addresses this by representing JSON values as explicit NAME TYPE VALUE triples. JSON types and lexical representations are preserved, enabling lossless round-tripping back to JSON text and precise downstream processing. This representation is well suited for internal processing and for building higher-level functionality on top of the core parser.
Rather than attempting to be a general-purpose JSON framework, ns_json focuses on predictable behavior, explicit typing, strict JSON syntax validation, and deep integration with NaviServer internals. Applications that require schema validation, extended JSON dialects, or document-oriented querying should use higher-level libraries built on top of this foundation.
Parse a single JSON value from value and return the parsed Tcl value. Multiple output formats are supported for mapping JSON values to Tcl data structures.
By default, the entire input must consist of exactly one JSON value (optionally surrounded by whitespace). Trailing non-whitespace data in the input is treated as an error unless the option -scan is specified.
When the option -scan is used, the result is a two-element list. The first element is the parsed value (or handle, depending on -output), and the second element is the number of bytes consumed from the input. This is useful for concatenated JSON streams or when JSON is embedded in larger payloads.
The following options are supported by ns_json parse:
Convert a Tcl value to its JSON representation and return the resulting JSON text.
The option -type controls how the Tcl value is interpreted and encoded as JSON. The default is auto, which selects the JSON type based on the Tcl value.
The following options are supported by ns_json value:
Quick examples:
# Pretty-print a JSON object via triples (lossless round-trip of values)
set JSON {{"user":{"id":7,"name":"Alice","flags":{"admin":false,"active":true}}}}
# Parse to typed triples and re-emit pretty-printed JSON
set triples [ns_json parse -output triples $JSON]
ns_json value -pretty $triples
## output:
## {
## "user": {
## "id": 7,
## "name": "Alice",
## "flags": {
## "admin": false,
## "active": true
## }
## }
## }
ns_json value -type object {a number 1 b boolean 1 c string x n null {}}
## output: {"a":1,"b":true,"c":"x","n":null}
ns_json value -type array {0 number 1 1 number 2 2 number 3 3 null {}}
## output: [1,2,3,null]
# Canonical root-wrapped triples document
set t [ns_json parse -output triples {{"user":{"id":7}}}]
## output: {} object {user object {id number 7}}
ns_json value $t
## output: {"user":{"id":7}}
Returns a Tcl value representing JSON null.
The returned value has a dedicated internal Tcl object type and can be used to construct or update JSON values unambiguously, without relying on string contents.
The string representation of the null value defaults to the empty string, but this textual form is only a presentation detail. Null detection should use ns_json isnull or explicit JSON type information.
# Create a JSON null value explicitly
set n [ns_json null]
ns_json isnull $n
## output: 1
# Use it when updating triples
set t1 [ns_json parse -output triples {{"user":{"name":"Alice"}}}]
set t2 [ns_json triples setvalue -path {user name} $t1 $n]
ns_json value $t2
## output: {"user":{"name":null}}
Returns a boolean indicating whether value is a Tcl value of the dedicated JSON-null object type.
This command checks the Tcl object type, not the string representation. Therefore, plain Tcl strings such as "", null, or any caller-supplied -nullvalue mapping are not considered JSON null unless they are represented by the dedicated JSON-null object type returned by ns_json null.
When working with parsed JSON in set or triples mode, JSON types can also be distinguished via .type sidecars or via ns_json triples gettype.
set n1 [ns_json parse {null}]
set n2 [ns_json parse -nullvalue XXX {null}]
list \
[ns_json isnull $n1] \
[ns_json isnull $n2] \
[ns_json isnull null] \
[ns_json isnull XXX]
## output: 1 1 0 0
Interpret a flattened key produced by set output and return a dict:
key <unescaped-base-key> field <field-or-empty>
The base key is the path key with escaping reversed and sidecar suffix removed. Currently the sidecar field .type is recognized. Other sidecar fields may be added in the future.
Return the escaped form of an arbitrary string for use as a single path segment in set output keys. The escaping rules are:
~ is encoded as ~0
/ is encoded as ~1
. is encoded as ~2
Return the unescaped form of an escaped path segment (inverse of keyencode).
Return the value located at the specified path within a triples structure.
Exactly one of -path or -pointer must be specified.
-path value specifies a Tcl list describing the hierarchical location.
-pointer value specifies a JSON Pointer (RFC 6901) string like /a/b. For convenience, it also accepts fragment form #/a/b (the leading # is ignored). A trailing / denotes an empty reference token and is not ignored.
-indices causes the command to return a Tcl index path suitable for use with lindex or lset on the original triples list instead of the value itself.
-output json|triples controls the output format. The default is json.
json (default) returns the selected value formatted as JSON text. Note that for strings, the commands provides the quoting needed for JSON.
triples returns the stored Tcl representation. For scalar values this is the scalar Tcl value. For container values this is the container-content triples list.
-pretty applies only when -output json is used and formats the JSON output with indentation.
When selecting the whole document (e.g. via -pointer "" or -pointer #), the behavior depends on the output mode:
With -output json, the entire JSON document is returned.
With -output triples, the root VALUE is returned (i.e., the container-content triples list for objects or arrays, or the scalar value for scalar documents).
Selecting the whole document requires a canonical root-wrapped triples representation. If the triples input is not root-wrapped, an error is raised.
set t [ns_json parse -output triples {{"o":{"x":1},"s":"Bob"}}]
# Default: JSON projection
ns_json triples getvalue -pointer /o $t
## output: {"x":1}
ns_json triples getvalue -pointer /s $t
## output: "Bob"
# return the full document
ns_json triples getvalue -pointer # $t
##output: {"o":{"x":1},"s":"Bob"}
# Raw triples/Tcl representation
ns_json triples getvalue -output triples -pointer /o $t
## output: x number 1
Return the JSON type token at the specified path within a triples structure.
The result is one of: string, number, boolean, null, object, or array.
The location within the triples structure is specified via either -path or -pointer. Exactly one of these options must be provided.
-path value specifies a Tcl list describing the hierarchical location.
-pointer value specifies a JSON Pointer (RFC 6901) string. A trailing / denotes an empty reference token and is not ignored.
With -indices, the command returns an index path pointing to the type element inside the triples list. This index path can be used with lindex or lset.
set t [ns_json parse -output triples {{"user":{"flags":{"active":true}}}}]
ns_json triples gettype -pointer /user/flags/active $t
## output: boolean
Update the value at the specified path within a triples structure and return a modified triples list.
The location to be updated within the triples structure is specified via either -path or -pointer. Exactly one of these options must be provided.
-path value specifies a Tcl list describing the hierarchical location.
-pointer value specifies a JSON Pointer (RFC 6901) string. A trailing / denotes an empty reference token and is not ignored.
The optional -type flag controls how the new value is interpreted:
auto (default): infer the type from the value, while preserving and validating existing leaf types where applicable (for example, number slots remain numeric and reject invalid number lexemes).
Explicit types (string, number, boolean, null, object, array) enforce validation and normalization of the supplied value.
With number, valid JSON number lexemes are preserved. Other Tcl numeric forms are accepted when they can be interpreted as numbers and are normalized to a JSON-compatible number representation.
For container types (object and array), the supplied value must be a triples list. The empty list represents an empty container.
Setting a null value can be done using ns_json null, or via a null value retrieved from triples, or via -type null.
set t1 [ns_json parse -output triples {{"user":{"name":"Alice"}}}]
ns_json triples gettype -path {user name} $t1
## output: string
set t2 [ns_json triples setvalue -path {user name} $t1 [ns_json null]]
ns_json triples gettype -path {user name} $t2
## output: null
Derive a JSON Schema-like description from one or more JSON instances represented as canonical root-wrapped triples documents.
The generated schema is canonicalized to ensure a stable and deterministic structure. Schema objects are emitted with a fixed member order ($schema, type, properties, items, required, anyOf). Property names are sorted lexicographically and arrays such as required are normalized to a stable order. This canonicalization makes the output predictable and suitable for comparisons in tests, diffs, or generated documentation.
The generated schemata use a small subset of the JSON Schema vocabulary and are valid JSON Schema documents. They can therefore be used directly with standard JSON Schema validators, although they represent inferred structure rather than human certified validation constraints.
Schema generation can be useful when inspecting unknown JSON structures, documenting API responses, or deriving validation rules from example data. The command analyzes the provided JSON instances and produces a schema describing the observed structure and value types.
When multiple input documents are provided, the individual schema fragments are merged into a single schema describing all observed instances.
The result is returned as JSON text. The generated schema follows a JSON Schema-like structure and includes a top-level $schema field naming the supported draft.
-pretty returns the generated schema in pretty-printed form.
-required includes required arrays in object schemas. With a single input document, all observed properties are considered required. With multiple input documents, required is computed using intersection semantics, so only properties present in every input document remain required.
Without -required, the generated schema omits required fields.
Each input value must be a canonical root-wrapped triples document, such as one returned by ns_json parse -output triples.
set t [ns_json parse -output triples {{"id":1,"email":"a@example.com"}}]
ns_json triples schema -pretty $t
## output:
## {
## "$schema": "https://json-schema.org/draft/2020-12/schema",
## "type": "object",
## "properties": {
## "email": { "type": "string" },
## "id": { "type": "number" }
## }
## }
When multiple instances are provided, the generated schema reflects the combined structure of all observed values.
set t1 [ns_json parse -output triples {{"id":1,"email":"a@example.com"}}]
set t2 [ns_json parse -output triples {{"id":2}}]
set t3 [ns_json parse -output triples {{"id":3,"email":"c@example.com"}}]
ns_json triples schema -pretty -required $t1 $t2 $t3
## output:
## {
## "$schema": "https://json-schema.org/draft/2020-12/schema",
## "type": "object",
## "properties": {
## "email": { "type": "string" },
## "id": { "type": "number" }
## },
## "required": ["id"]
## }
When a property is observed with different scalar types across multiple instances, the generated schema uses a compact type union.
set t1 [ns_json parse -output triples {{"email":"a@example.com"}}]
set t2 [ns_json parse -output triples {{"email":null}}]
ns_json triples schema -pretty $t1 $t2
## output:
## {
## "$schema": "https://json-schema.org/draft/2020-12/schema",
## "type": "object",
## "properties": {
## "email": { "type": ["null", "string"] }
## }
## }
Check whether the provided triples structure conforms to the specified JSON schema.
The schema must be provided as a Tcl dictionary representing a JSON Schema document. Such schemata can be generated automatically using ns_json triples schema, or constructed manually.
Currently, the command supports the subset of JSON Schema that is generated by ns_json triples schema. This includes the keywords type, properties, items, required, and anyOf. Other schema keywords are rejected by default, but can be ignored by specifying the -ignoreunsupported flag.
The command verifies that the triples structure matches the supported subset of JSON Schema. When a mismatch is detected, an error is raised describing the problem and the location within the document using a JSON Pointer style path.
Supported schema keywords include type, properties, items, required, and anyOf. These are sufficient to represent the schemata generated by ns_json triples schema.
The optional -ignoreunsupported flag allows additional schema keywords to appear in the schema document. Unsupported keywords are ignored rather than rejected. By default, the command reports an error when encountering unsupported schema keywords.
The result of a successful match is the boolean value 1.
The following example demonstrates a typical workflow. First, a JSON document is parsed into the triples representation. A schema is then derived from this instance using ns_json triples schema. Finally, another JSON document is parsed and checked against the derived schema using ns_json triples match.
set json1 {
{
"user": {
"id": 1,
"email": "alice@example.com"
}
}
}
set t1 [ns_json parse -output triples $json1]
# derive a schema from the example document
set schema [ns_json triples schema -required $t1]
set json2 {
{
"user": {
"email": "bob@example.com"
"id": 2,
}
}
}
set t2 [ns_json parse -output triples $json2]
# check whether the new document conforms to the schema
ns_json triples match -schema $schema $t2
## output: 1
If a mismatch occurs, the command raises an exception describing the violation and the location within the document.
set json3 {
{
"user": {
"id": "not-a-number",
"email": "bob@example.com"
}
}
}
set t3 [ns_json parse -output triples $json3]
ns_json triples match -schema $schema $t3
## error: ns_json: schema mismatch at /user/id: expected number, got string
The triples representation uses a single uniform structure. A JSON document is represented in canonical form as a triples document, which is a container-content triples list wrapped in an explicit root triple:
A triples document has the form:
"" TYPE VALUE
Here, TYPE is the JSON type of the document and VALUE is the corresponding triple notation of the content.
For JSON objects and arrays, VALUE is a container-content triples list of the form, which is a flat list of one or more triples:
NAME TYPE VALUE NAME TYPE VALUE ...
This same container-content structure is used recursively for nested objects and arrays.
The explicit root wrapper makes the JSON document type unambiguous, even for scalar documents, and ensures that JSON Pointer navigation (including selection of the whole document) behaves consistently. It also avoids heuristic container detection and preserves full round-trip fidelity.
In triples lists:
NAME is:
empty for triple documents (root wrapper)
an object member name (string) for JSON objects, or
the decimal string form of the zero-based array index for JSON arrays.
TYPE is one of: object, array, string, number, boolean, or null.
VALUE is:
for object and array: a nested container-content triples list,
for string: a Tcl string,
for number: a Tcl string representing the JSON number lexeme,
for boolean: a Tcl boolean value,
for null: the current null value (default null).
Example: triples document (canonical root wrapper)
set t [ns_json parse -output triples {{"o":{"x":1},"s":"Bob","n":null}}]
## output: {} object {o object {x number 1} s string Bob n null {} }
Example: container-content triples list (object VALUE only)
## VALUE part of the root object above:
o object {x number 1} s string Bob n null {}
The triples representation preserves JSON type information explicitly and allows efficient navigation and modification without re-parsing JSON text. The canonical root wrapper makes the document type unambiguous and enables consistent JSON Pointer navigation, including selection of the entire document.
Note: it is also possible to call the JSON parser to parse scalars based on the JSON rules. When producing triple output, these are returned with the root wrapper, which allows for round trips without explicit typing.
set t [ns_json parse -output triples 1]
## output: {} number 1
ns_json value $t
## output: 1
# Note: recognition works only for proper root wrappers.
# Here we have a NAME value which is not emtpy.
ns_json value {0 number 1}
## output: "0 number 1"
Parse JSON as dict
ns_json parse {{"a":1,"b":true,"c":"x"}}
## a 1 b true c x
# Nested objects become nested dicts.
set d [ns_json parse {{
"user": {
"id": 7,
"name": "Alice",
"flags": {
"admin": false,
"active": true
}
},
"meta": {
"count": 2
}
}}]
# -> user {id 7 name Alice flags {admin false active true}} meta {count 2}
# Access nested values with dict get:
dict get $d user name ;# -> Alice
dict get $d user flags active ;# -> true
dict get $d meta count ;# -> 2
Parse JSON with explicit types (triples)
ns_json parse -output triples {{"a":1,"b":true}}
## {} object {a number 1 b boolean true}
Parse JSON to a flattened ns_set
set s [ns_json parse -output set {{
"x": { "y":"z" },
"arr": [10,20]
}}]
ns_set get $s x/y
## output: z
ns_set get $s x/y.type
## output: string
ns_set get $s arr/0
## output: 10
ns_set get $s arr/0.type
## output: int
ns_set format $s
## output:
ns_json:
x/y.type: string
x/y: z
arr/0.type: number
arr/0: 10
arr/1.type: number
arr/1: 20
Scan mode
ns_json parse -scan {{"a":1}{"b":2}}
## output: {{a 1} 7}
# Parse the first JSON object and return the number of bytes consumed,
# leaving a useful trailer (e.g., a checksum) after the JSON.
set data {{"a":1} CRC32=0x3A5F2C19}
set r [ns_json parse -scan $data]
## output: {{a 1} 7}
# The consumed byte count lets you slice off the trailer:
lassign $r d consumed
set trailer [string range {{"a":1} CRC32=0x3A5F2C19} $consumed end]
## output: CRC32=0x3A5F2C19
Encode JSON strings safely
ns_json value -type string {a"b\c}
## output: "a\"b\\c"
Generate JSON from triples
ns_json value -type object {a number 1 b boolean 1 c string x n null {}}
## output: {"a":1,"b":true,"c":"x","n":null}
Key helpers for set output
ns_json keyencode {a.b}
## output: a~2b
ns_json keyinfo {a~2b.type}
## output: key a.b field type
Example: Navigating and Updating JSON Structures via Triples (4 steps)
Assume a JSON document with the following structure, where a response from a sever contains an array of invoices, and every invoice contains a BillEmail (irrelevant parts omitted):
{
"QueryResponse": {
"Invoice": [{
...,
"BillEmail": {
"Address": "joe.user@nowhere.com"
},
...
}],
...
}
}
1) Parse the JSON document into a triples representation for convenient navigation and modification:
set t [ns_json parse -output triples $JSON]
The variable t now contains the triples representation of the JSON document.
2) Retrieve the BillEmail object of the first invoice (the default output format is JSON):
ns_json triples getvalue -pointer /QueryResponse/Invoice/0/BillEmail $t
## output: {"Address":"joe.user@nowhere.com"}
3) Update the email address inside the triples structure:
set t2 [ns_json triples setvalue -pointer /QueryResponse/Invoice/0/BillEmail/Address $t "customer@somewhere.com"]
4) Inspect the updated BillEmail object using formatted JSON output:
ns_json triples getvalue -pretty -pointer /QueryResponse/Invoice/0/BillEmail $t2
## output:
{
"Address": "customer@somewhere.com"
}