NaviServer - programmable web server
4.99  5.0

[ Main Table Of Contents | Table Of Contents | Keyword Index ]

nsv(n) 4.99.30 naviserver "NaviServer Built-in Commands"

Name

nsv - NaviServer nsv Commands

Table Of Contents

Synopsis

Description

The nsv commands provide a high performance data sharing mechanism. This facility is much flexible alternative to the obsolete ns_share command. The model uses an array syntax and includes more features. In addition, lock contention is managed in a much more scalable way--something that is not possible with the obsolete ns_share facility.

COMMANDS

nsv_array get array ?pattern?
nsv_array set array value-list
nsv_array reset array value-list
nsv_array exists array
nsv_array size array
nsv_array names array ?pattern?

Commands for the most part mirror the corresponding Tcl command for ordinary variables.

 % nsv_array set shared_array { key1 value1 key2 value2 }
 
 % nsv_array get shared_array
 key1 value1 key2 value2
 
 % nsv_array reset shared_array { key3 value3 }
 
 % nsv_array exists shared_array
 1
 
 % nsv_array size shared_array
 1
 
 % nsv_array names shared_array
 key3
 
 % nsv_array set shared_array [array get tmp_shared_array]
 % array set tmp_shared_array [nsv_array get shared_array]
 % nsv_array reset shared_array [array get tmp_shared_array]
nsv_bucket ?bucket-nr?

Return a list of all the array names with lock counts from the specified bucket. If no bucket-nr is specified, return a list of all arrays from all buckets. This command is mainly for performance tuning. When e.g. the number of locks for a certain bucket is high one can use this command to determine the arrays with their usages from this bucket.

 set buckets ""
 set count -1
 foreach b [nsv_bucket] {
    append buckets "[incr count]: " [lsort -integer -index 1 -decreasing $b] \n
 }
 ns_log notice $buckets
nsv_dict append array key dictkey ?value ...?
nsv_dict exists array key dictkey ?dictkey ...?
nsv_dict get ?-varname varname? array key ?dictkey ...?
nsv_dict getdef ?-varname varname? array key ?dictkey ...? default
nsv_dict incr array key ?increment?
nsv_dict keys array key ?pattern?
nsv_dict lappend array key dictkey ?value ...?
nsv_dict set array key dictkey ?dictkey ...? value
nsv_dict size array key
nsv_dict unset array key dictkey ?dictkey ...?

Implementation of the Tcl dict command for shared variables. The commands work similar as nsv_set and nsv_get, except that the value of these commands are structure values in form of dicts.

The main difference to plain Tcl is the option ?-varname varname?. When this option is provided, the function returns 0 or 1 depending on success and returns in the success case the value in the provided variable. When the varname option is provided, the behavior is similar to the optional last argument in nsv_get or ns_cache_get The option ?-varname varname? has the advantage to test and get the value with a single locked command. This avoids race conditions and reduces the number of locks for the application.

 % nsv_dict set personnel 4711 name gustaf
 name gustaf
 
 % nsv_dict set personnel 4711 sex m
 name gustaf sex m
 
 % nsv_dict get personnel 4711 name
 gustaf
 
 % nsv_dict get personnel 4711
 name gustaf sex m
nsv_exists array key

Test whether a key exists in the nsv array.

 % nsv_exists shared_array key1
 1
 % nsv_exists shared_array key2
 0
nsv_get array key ?varName?

Get the value for the key from the nsv array. If the optional varName is provided, the function returns on success 1 and on failure 0. On success, it binds the variable varName. If the variable name is not provided, it returns on success the value and raises on failure an error (similar to ns_cache_get). With the optional variable name, this function allows an atomic check for existence followed by a get operation.

 % nsv_get shared_array key1
 value1
nsv_incr arrayName key ?increment?

If increment is supplied then its value (which must be an integer) is added to the value of the element key; otherwise 1 is added to the value of the element key. Unlike the Tcl equivalent if key does not exists it is created. Returns the new value of the element specified by key. Internally interlocked so it is thread safe, no mutex required.

 % nsv_incr shared_array foo
 1
 % nsv_incr shared_array foo -1
 0
nsv_append array key value ?value ...?

Append all of the value arguments to the current value of variable key in the array. If key doesn't exist, it is given a value equal to the concatenation of all the value arguments

 % nsv_append shared_array key1 foo
 value1foo
nsv_lappend array key value ?value ...?

Append all of the value arguments as list elements to variable key in the array. If key doesn't exist, it is created as a list with elements given by the value arguments

 % nsv_lappend shared_array key1 value2
 value1 value2
nsv_names ?pattern?

Return a list of all the nsvs in use, optionally only those matching pattern. If no matching nsvs are in use returns the empty string.

 % nsv_names
 shared_array
nsv_set ?-default? ?-reset? array key ?value?

Set the value for a key in an nsv array. Returns the value the key is set to. The various options are especially useful to implement atomic operations.

-default

When this flag is specified nothing is changed in case the key key of array has already an value. Otherwise it sets the value. This operation is similar to SETNX in REDIS (set if no exists).

-reset

When this flag is specified and a value is given, the command resets the value for key and returns the old value. This operation is similar to GETSET in REDIS (get the old value and set it new).

When this flag is specified but no value is provided, the command returns the value for key and unsets resets it.

 % nsv_set shared_array key1 value1
 value1
nsv_unset ?-nocomplain? ?--? array ?key?

Unset an array or a single key from an array. If successful returns an empty string. When -nocomplain is specified the command does not complain when the specified array or key does not exist.

 % nsv_unset shared_array key1
 % nsv_unset shared_array

Migrating From ns_share

Migrating from ns_share is straightforward. If your init.tcl included commands such as:

 ns_share myshare
 set myshare(lock) [ns_mutex create]

use instead:

 nsv_set myshare lock [ns_mutex create]

In your procedures, instead of:

 proc myproc {} {
     ns_share myshare
     ns_mutex lock $myshare(lock)
     ...
 }

use:

 proc myproc {} {
     ns_mutex lock [nsv_get myshare lock]
     ...
 }

and within an ADP page, instead of:

 <%
  ns_share myshare
  ns_puts $myshare(key1)
 %>
 
 <%=$myshare(key2)%>

use:

 <%
 ns_puts [nsv_get myshare key1]
 %>
 
 <%=[nsv_get myshare key2]%>

Notice that, unlike ns_share, no command is required to define the shared array. The first attempt at setting the variable through any means will automatically create the array. Also notice that only arrays are supported. However, to migrate from ns_share you can simply package up all existing ns_share scalars into a single array with a short name, perhaps just ".". For example, if you had:

 ns_share mylock myfile
 set myfile /tmp/some.file
 set mylock [ns_mutex create]

you can use:

 nsv_set . myfile /tmp/some.file
 nsv_set . mylock [ns_mutex create]

Multithreading Features

One advantages of nsv is built in interlocking for thread safety. For example, consider a case of a "increment-by-one" unique id system. Here's the ns_share solution:

 ns_share ids
 set ids(lock) [ns_mutex create]
 set ids(next) 0
 
 proc nextid {} {
     ns_share ids
     ns_mutex lock $ids(lock)
     set next [incr ids(next)]
     ns_mutex unlock $ids(lock)
     return $next
 }

and here's an nsv solution:

 nsv_set ids next 0
 
 proc nextid {} {
     return [nsv_incr ids next]
 }

Note that the nsv solution does not need a mutex as the nsv_incr command is internally interlocked.

Compatibility with Tcl Arrays

Another useful feature of nsv is the nsv_array command which works much like the Tcl array command. This can be used to import and export values from ordinary Tcl arrays. For example, to copy from Tcl use:

 nsv_array set meta [array get tmpmeta]

and to copy to Tcl use:

 array set metacopy [nsv_array get meta]

As with all other nsv command, nsv_array is atomic and no explicit locking is required. This feature can be used to construct a new nsv array by first filling up an ordinary temporary Tcl array via some time consuming process and then swapping it into place as above. While the new temporary array is being constructed, other threads can access the old array without delay or inconsistent data. You can even reset a complete nsv array in one step with "reset". For example, instead of:

 ns_share lock meta
 set lock [ns_mutex create]
 ns_mutex lock $lock
 unset meta
 array set meta [array get tmpmeta]
 ns_mutex unlock $lock

you can simply use:

 nsv_array reset meta [array get tmpmeta]

The reset option will flush and then reset all values atomically, eliminating the need for the explicit lock.

Configuration

All accesses to shared variables are protected by a mutex. Attention should be taken for array operations which have to iterate over many shared variables to return the result, since this will lead to reduced scalability due too long lasting locks.

Number of buckets

The nsv system uses a common multithreading technique to reduce the potential for lock contention which is to split the locks to achieve finer grained locking. This technique groups arrays randomly into buckets and only the arrays within a particular bucket share a lock. The number of buckets to be used can be configured by setting the nsvbuckets Tcl parameters, e.g.:

 ns_section  ns/server/${server}/tcl {
   # Number of buckets in Tcl hash table for nsv vars
   ns_param nsvbuckets 16
 }

The default value for nsvbuckets is 8 which should be reasonable for most applications. Note that you can monitor the lock contention by viewing the results of "ns_info locks" command after the server has been running for some time. The nsv locks all have names of the form "nsv:##". If you find many lock attempts which did not succeeded immediately, try increasing nsvbuckets.

Mutex Locks vs. RWLocks

An RWLock allows concurrent read access and a single writer, while a mutex locks allows only a single reader or a single writer at the same time. The RWLock has more overhead to support concurrency, but in case there are more read than write operations, RWLocks are better (but this might be different by the support for RWLocks). In general, for Web server applications (like e.g. OpenACS), most nsv variables have a write ratio of way below 1%. However, applications might be different, so NaviServer allows the specify in the configuration file, whether mutex locks or rwlocks might be used.

 ns_section  ns/server/${server}/tcl {
   ns_param nsvrwlocks false   ;# default true
 }

See Also

ns_cache, ns_set, ns_urlspace, nsd

Keywords

configuration, data structure, mutex, nsv, server built-in, shared, variables