NaviServer Built-in Commands – 5.1.0
ns_json - JSON parser and generator for HTTP/WebAuthn needs.
The command ns_json provides a JSON (RFC 8259) parser and generator integrated directly into NaviServer.
The parser is designed for server-side use cases where predictable behavior, explicit type handling, and tight integration with NaviServer data structures are required. It parses exactly one JSON value from the input and reports errors via Tcl exceptions.
In addition to returning conventional Tcl structures, the command supports output formats that preserve JSON type information and lexical representations explicitly (the tripple format|. This enables reliable round-tripping between JSON input and output and avoids unintended data transformations.
The ns_json triples ensemble provides structured access to the internal triples representation produced by ns_json parse -output triples. These commands allow querying and updating JSON data in its typed intermediate form without re-parsing or serializing JSON text.
The implementation performs strict validation of JSON syntax as defined by RFC 8259. Optional validation can be enabled to enforce additional constraints on numeric values when integrating with Tcl numeric semantics. The command does not perform application-level transformations or coercions beyond these well-defined checks.
Features such as comments, JSON5 extensions, or schema validation are intentionally not supported.
The following design goals guided the implementation:
Strict adherence to JSON syntax as defined in RFC 8259.
Deterministic and explicit handling of JSON types.
Preservation of numeric and string lexemes to support lossless round-trips.
Efficient parsing directly from in-memory buffers.
Support for incremental parsing via byte consumption reporting.
Integration with Tcl values and NaviServer ns_set structures.
No dependency on external JSON libraries or Tcl packages.
The command supports multiple output formats to cover common integration needs:
dict returns a Tcl representation using dict/list/scalars.
triples returns a flat list of triples NAME TYPE VALUE.
set returns a flattened ns_set suitable for form handling.
These last two formats make JSON type information explicit and are intended for programmatic processing and re-emission.
The ns_json command is not only intended as a utility for application developers, but is also designed as an integral part of NaviServer’s internal request processing infrastructure.
NaviServer has a long-standing model for handling structured request data, most notably for application/x-www-form-urlencoded and multipart/form-data requests, where incoming data is parsed early and exposed through well-defined core data structures such as ns_set. Higher-level facilities (e.g., page contracts, filters, and form handling) are built on top of these parsed representations.
The ns_json parser extends this model to JSON-based requests. It is implemented in C, exposes a C-level parsing interface, and integrates directly with NaviServer core abstractions. This allows JSON request bodies to be parsed automatically and made available to both internal code and application-level Tcl code in a uniform and predictable way.
In particular, the set output format is designed to mirror classical form handling as closely as possible. JSON objects and arrays are flattened into an ns_set, structural paths are encoded into keys, and JSON type information is preserved using sidecar fields. This enables existing infrastructure such as page contracts and request filters to operate on JSON input without requiring JSON-specific logic.
Beyond application-level use, ns_json is intended to be used internally by NaviServer itself. By exposing a small, well-defined C API and by relying on common core data structures, the same parser can be reused for tasks such as automatic parsing of JSON POST requests, incremental or streaming request bodies, and other protocol-level features.
Most existing Tcl JSON libraries focus on convenience for scripting and map JSON values directly to Tcl dicts and lists. While suitable for many application use cases, this approach loses explicit JSON type information and often normalizes numeric values in ways that prevent reliable round-tripping.
The triples output format addresses this by representing JSON values as explicit NAME TYPE VALUE triples. JSON types and lexical representations are preserved, enabling lossless round-tripping back to JSON text and precise downstream processing. This representation is well suited for internal processing and for building higher-level functionality on top of the core parser.
Rather than attempting to be a general-purpose JSON framework, ns_json focuses on predictable behavior, explicit typing, strict JSON syntax validation, and deep integration with NaviServer internals. Applications that require schema validation, extended JSON dialects, or document-oriented querying should use higher-level libraries built on top of this foundation.
Parse a single JSON value from value and return the parsed Tcl value. Multiple output formats are supported for mapping JSON values to Tcl data structures.
By default, the entire input must consist of exactly one JSON value (optionally surrounded by whitespace). Trailing non-whitespace data in the input is treated as an error unless the option -scan is specified.
When the option -scan is used, the result is a two-element list. The first element is the parsed value (or handle, depending on -output), and the second element is the number of bytes consumed from the input. This is useful for concatenated JSON streams or when JSON is embedded in larger payloads.
The following options are supported by ns_json parse:
Convert a Tcl value to its JSON representation and return the resulting JSON text.
The option -type controls how the Tcl value is interpreted and encoded as JSON. The default is auto, which selects the JSON type based on the Tcl value.
The following options are supported by ns_json value:
Quick examples:
ns_json value -type object {a number 1 b boolean 1 c string x n null __NS_JSON_NULL__}
## output: {"a":1,"b":true,"c":"x","n":null}
ns_json value -type array {0 number 1 1 number 2 2 number 3 3 null __NS_JSON_NULL__}
## output: [1,2,3,null]
# AUTO container detection (explicit container marker present)
set JSON {{"user":{"id":7}}}
set t [ns_json parse -output triples $JSON]
## output: user object {id number 7}
ns_json value $t
## output: {"user":{"id":7}}
# Note: Scalar-looking triples are not auto-detected as containers
ns_json value {0 number 1}
## output: "0 number 1"
Return a boolean indicating whether value is the distinguished JSON null sentinel used in dict output.
Interpret a flattened key produced by set output and return a dict:
key <unescaped-base-key> field <field-or-empty>
The base key is the path key with escaping reversed and sidecar suffix removed. Currently the sidecar field .type is recognized. Other sidecar fields may be added in the future.
Return the escaped form of an arbitrary string for use as a single path segment in set output keys. The escaping rules are:
~ is encoded as ~0
/ is encoded as ~1
. is encoded as ~2
Return the unescaped form of an escaped path segment (inverse of keyencode).
Return the value located at the specified path within a triples structure.
Exactly one of -path or -pointer must be specified.
-path value specifies a Tcl list describing the hierarchical location.
-pointer value specifies a JSON Pointer (RFC 6901) string. A trailing / denotes an empty reference token and is not ignored.
-indices causes the command to return a Tcl index path suitable for use with lindex or lset on the original triples list instead of the value itself.
-output json|triples controls the output format. The default is json.
json (default) returns the selected value formatted as JSON text. Note that for strings, the commands provides the quoting needed for JSON.
triples returns the stored Tcl representation. For scalar values this is the scalar Tcl value. For container values this is the container-content triples list.
-pretty applies only when -output json is used and formats the JSON output with indentation.
set t [ns_json parse -output triples {{"o":{"x":1},"s":"Bob"}}]
# Default: JSON projection
ns_json triples getvalue -pointer /o $t
## output: {"x":1}
ns_json triples getvalue -pointer /s $t
## output: "Bob"
# Raw triples/Tcl representation
ns_json triples getvalue -output triples -pointer /o $t
## output: x number 1
Return the JSON type token at the specified path within a triples structure.
The result is one of: string, number, boolean, null, object, or array.
The location within the triples structure is specified via either -path or -pointer. Exactly one of these options must be provided.
-path value specifies a Tcl list describing the hierarchical location.
-pointer value specifies a JSON Pointer (RFC 6901) string. A trailing / denotes an empty reference token and is not ignored.
With -indices, the command returns an index path pointing to the type element inside the triples list. This index path can be used with lindex or lset.
set t [ns_json parse -output triples {{"user":{"flags":{"active":true}}}}]
ns_json triples gettype -pointer /user/flags/active $t
## output: boolean
Update the value at the specified path within a triples structure and return a modified triples list.
The location to be updated within the triples structure is specified via either -path or -pointer. Exactly one of these options must be provided.
-path value specifies a Tcl list describing the hierarchical location.
-pointer value specifies a JSON Pointer (RFC 6901) string. A trailing / denotes an empty reference token and is not ignored.
The optional -type flag controls how the new value is interpreted:
auto (default): infer the type from the value, while preserving and validating existing leaf types where applicable (for example, number slots remain numeric and reject invalid number lexemes).
Explicit types (string, number, boolean, null, object, array) enforce validation and normalization of the supplied value.
For container types (object and array), the supplied value must be a triples list. The empty list represents an empty container.
Setting a null value can be done using the sentinel __NS_JSON_NULL__, which stores a null leaf.
set t [ns_json parse -output triples {{"user":{"name":"Alice"}}}]
ns_json triples gettype -path {user name} $t
## output: string
set t2 [ns_json triples setvalue -path {user name} $t __NS_JSON_NULL__]
ns_json triples gettype -path {user name} $t2
## output: null
The triples representation is a typed Tcl list format used internally by ns_json. It is produced by:
ns_json parse -output triples
ns_json triples getvalue -output triples
A triples list has the structure:
key-or-index type value key-or-index type value ...
Each logical element consists of three consecutive fields:
A key (for objects) or numeric index (for arrays)
A type token
A value
Valid type tokens are: string, number, boolean, null, object, and array.
The meaning of the value field depends on the type:
string: the value is a Tcl string containing the decoded JSON string (without surrounding quotes).
number: the value is a Tcl string containing the original JSON number lexeme. The lexeme must conform to JSON number syntax.
boolean: the value is the canonical Tcl representation of the JSON boolean (true or false).
null: the value is the null sentinel __NS_JSON_NULL__.
object: the value is a triples list describing the object members. The empty list represents an empty object.
array: the value is a triples list describing the array elements. The empty list represents an empty array.
Example:
set t [ns_json parse -output triples {{"o":{"x":1},"s":"Bob","n":null}}]
## output: o object {x number 1} s string Bob n null __NS_JSON_NULL__
The triples representation preserves JSON type information explicitly and allows efficient navigation and modification without re-parsing JSON text.
Parse JSON as dict
ns_json parse {{"a":1,"b":true,"c":"x"}}
## a 1 b true c x
# Nested objects become nested dicts.
set d [ns_json parse {{
"user": {
"id": 7,
"name": "Alice",
"flags": {
"admin": false,
"active": true
}
},
"meta": {
"count": 2
}
}}]
# -> user {id 7 name Alice flags {admin false active true}} meta {count 2}
# Access nested values with dict get:
dict get $d user name ;# -> Alice
dict get $d user flags active ;# -> true
dict get $d meta count ;# -> 2
Parse JSON with explicit types (triples)
ns_json parse -output triples {{"a":1,"b":true}}
## a number 1 b boolean true
Parse JSON to a flattened ns_set
set s [ns_json parse -output set {{
"x": { "y":"z" },
"arr": [10,20]
}}]
ns_set get $s x/y
## output: z
ns_set get $s x/y.type
## output: string
ns_set get $s arr/0
## output: 10
ns_set get $s arr/0.type
## output: int
ns_set format $s
## output:
ns_json:
x/y.type: string
x/y: z
arr/0.type: number
arr/0: 10
arr/1.type: number
arr/1: 20
Scan mode
ns_json parse -scan {{"a":1}{"b":2}}
## output: {{a 1} 7}
# Parse the first JSON object and return the number of bytes consumed,
# leaving a useful trailer (e.g., a checksum) after the JSON.
set data {{"a":1} CRC32=0x3A5F2C19}
set r [ns_json parse -scan $data]
## output: {{a 1} 7}
# The consumed byte count lets you slice off the trailer:
lassign $r d consumed
set trailer [string range {{"a":1} CRC32=0x3A5F2C19} $consumed end]
## output: CRC32=0x3A5F2C19
Encode JSON strings safely
ns_json value -type string {a"b\c}
## output: "a\"b\\c"
Generate JSON from triples
ns_json value -type object {a number 1 b boolean 1 c string x n null __NS_JSON_NULL__}
## output: {"a":1,"b":true,"c":"x","n":null}
Key helpers for set output
ns_json keyencode {a.b}
## output: a~2b
ns_json keyinfo {a~2b.type}
## output: key a.b field type
Assume a JSON document with the following structure, where a response from a sever contains an array of invoices, and every invoice contains a BillEmail (irrelevant parts omitted):
{
"QueryResponse": {
"Invoice": [{
...,
"BillEmail": {
"Address": "joe.user@nowhere.com"
},
...
}],
...
}
}
Parse the JSON document into triples:
set t [ns_json parse -output triples $JSON]
Retrieve the BillEmail object of the first invoice (default output is JSON):
ns_json triples getvalue -pointer /QueryResponse/Invoice/0/BillEmail $t
## output: {"Address":"joe.user@nowhere.com"}
Update the email address:
set t2 [ns_json triples setvalue -pointer /QueryResponse/Invoice/0/BillEmail/Address $t "customer@somewhere.com"]
Inspect the updated BillEmail object (using formatted JSON output):
ns_json triples getvalue -pretty -pointer /QueryResponse/Invoice/0/BillEmail $t2
## output:
{
"Address": "customer@somewhere.com"
}