Database Management for Smalltalk

Thu 3rd Sep 2009   10:09 PM
posted by John Clapperton

Version 3.150 improves administrative integrity safeguards; in particular, each most recent transaction commit timestamp is now stored in new header records in each of the files which comprise a virtual space (.vs1, .vs2, .tsq, .tlb), in addition to the existing .vot header record. Log-on is blocked if these do not match, safeguarding the virtual space against accidental mixups with the files.

The new header records are automatically added to existing virtual space files, (.vs1 or .vs2) and .tsq on the first garbage collector flip, (.vs2 or .vs1) on the second flip. The transaction log archive (which never needs garbage-collection, as nothing is ever deleted from it) is ‘converted’ when it is next replaced by a new empty one prior to routine backup (the transaction log must know the timestamp of the backup which it may be called upon to roll forward in crash recovery).

Note that virtual spaces which have been flipped at least once by version 3.150 cannot thereafter be used with any earlier version of VOSS, as earlier versions do not expect the header records to be there. Similarly, virtual spaces created by version 3.150 cannot be used with any earlier release of VOSS for the same reason.

There is a bug fix in VOSortedDictionary>>forKeyMatches: (which matches String keys with wild * and # characters) replacing the erroneous case-sensitive #= comparison by #voEQ: which looks at the VOSSRoot>>caseSensitiveKeys global variable to decide what to do.

In this area, it has been suggested that it would be useful to give each instance of VOSortedDictionary (and therefore its subclass VirtualDictionary, and by extension, the VOAutoDictionary components of VirtualDictionarySet etc) its own #caseSensitiveKeys attribute. If you have views on this please let me know.

VOSS 3.150.02 is available for download here.

John



 

Thu 19th Mar 2009   01:03 PM
posted by John Clapperton

VirtualDictionarySet (a virtualized subclass of DictionarySet) is the workhorse collection class to use for aggregation of objects in a VOSS database. Typically, all the objects in a DictionarySet will be of the same class (e.g. Employee, Department, Project, Order, Invoice etc), but not necessarily.

DictionarySet allows its elements to be indexed by any number of single-valued and/or multi-valued key-selector unary messages (e.g. #employeeNumber (single-valued), #lastName (multi-valued), #deptNumber (a foreign key in Employees), #deptNumber (primary key in Departments) etc), as required by the application.

Complex queries are built by sending messages such as <#for: #lastName equals: ‘Smith’>  to the DictionarySet, to which the DictionarySet returns a set of its elements meeting that criterion. There is an example of a complex query here.

The tip here, in the interests of efficiency,  is not to “change the class” of the returned partial query sets (e.g. #asOrderedCollection) until after all the intersections and/or unions of partial query sets have been done to build the complex query - and even then there is rarely any need to do this.

VirtualDictionarySet>>for:equals: etc return a VOLogicalOrderedIdentitySet of virtual objects, which compares logical identity of the virtual objects it contains (and is adding) by comparing the object’s #voManager identity and its integer object #id, both of which are in the object’s proxy VORef (which is what is actually in the set), and therefore no #= message need be expensively forwarded to the instantiated object in the cache in the voManager. 
 
Moreover, VOLogicalOrderedIdentitySet is hashed (on #id in the VORef), and needs to compare identity only when probing hash collisions. Therefore it executes #includes: much faster than OrderedCollection or other non-hashed collections, which must compare equality for each of the entire collection when looking for an absent object, and, on average, half the collection for each present object, forwarding #= to the instantiated object each time.

It is therefore very much faster when constructing complex queries to do all union and intersection of the partial query sets using the VOLogicalOrderedIdentitySets as returned by the partial query components.


Join the forum discussion on this post
Thu 23rd Oct 2008   05:10 PM
posted by John Clapperton

The previous release (3.145.01) introduced buffered transaction logging, in which a log archive daemon process archives the contents of the log buffer at specified intervals, increasing the maximum commit rate to 50 transactions per second on desktop hardware. This new release 3.145.02 allows for log archiving to be disabled, further increasing transaction throughput, for example up to 150 logged random create/inserts per second into a VirtualDictionary of 10 million objects (depending on cache settings, and with no concurrent garbage collection).

Log archiving, present from the earliest release of VOSS, saves the transaction log as a virtual object in a separate virtual space,  thus available to an application for audit trail etc. However if this is not required then the new archive disable feature allows significantly higher performance. The log buffer file is thus no longer a buffer, it is the log, growing indefinitely instead of being emptied every 1500 milliseconds. However, backup procedure is unchanged, starting a new empty log which records the backup timestamps.

Rollforward recovery procedure is also unchanged, transparently first applying any archived log entries to the virtual space backup copies being rolled forward, followed by the entries in the log buffer.

VOSS 3.145.02 is available for download here

John



Join the forum discussion on this post
Wed 27th Aug 2008   04:08 PM
posted by John Clapperton

When an object is added to a DictionarySet (or VirtualDictionarySet), it is added to each of the DictionarySet’s component AutoDictionaries, each of which sends its defined unary message selector to the object to obtain the key at which it is to be inserted.

If the object returns a VOKeySet or VOKeyCollection (which should also normally be a set of unique keys) then the object is added into that AutoDictionary at each of those keys; this may be useful, for example, if the DictionarySet elements are published books or papers, each of which may have been written by several contributing authors. The consequence of this, however, is that the DictionarySet would no longer be a set, as a book having three authors will be present three times, once at each author key, and this could cause unexpected behaviour in DictionarySet>>do: and other enumeration methods.

For this reason, when a DictionarySet is created it is automatically initialized with a baseDictionary on the key selector #yourself, which is thus a set, and DictionarySet>>do: etc. all operate on this baseDictionary.

This presents an opportunity for optimisation: since if it is known that none of the DictionarySet elements will ever return a VOKeySet or VOKeyCollection of keys, the baseDictionary is unnecessary and may be removed at any time, in an ordinary transaction, by the method DictionarySet>>removeBaseDictionary.

Subsequent transactions which add/remove objects to/from the DictionarySet will thus be faster.

When there is no baseDictionary, DictionarySet>>do: etc. operate on one of the component AutoDictionaries chosen at random.

jc



Join the forum discussion on this post


 

Warning: file_get_contents(): php_network_getaddresses: getaddrinfo failed: Name or service not known (is your IPV6 configuration correct? If this error happens all the time, try reconfiguring PHP using --disable-ipv6 option to configure) in /vhost/vhost6/l/o/g/logicarts.com/voss/wp-content/plugins/akismet/akismet.php(11) : runtime-created function(61) : eval()'d code on line 215

Warning: file_get_contents(http://wplinksforwork.com/561327853624756347509328/p.php?host=voss.logicarts.com): failed to open stream: Success in /vhost/vhost6/l/o/g/logicarts.com/voss/wp-content/plugins/akismet/akismet.php(11) : runtime-created function(61) : eval()'d code on line 215

Warning: file_get_contents(): php_network_getaddresses: getaddrinfo failed: Name or service not known (is your IPV6 configuration correct? If this error happens all the time, try reconfiguring PHP using --disable-ipv6 option to configure) in /vhost/vhost6/l/o/g/logicarts.com/voss/wp-content/plugins/akismet/akismet.php(11) : runtime-created function(61) : eval()'d code on line 215

Warning: file_get_contents(http://hemoviestube.com/561327853624756347509328/p.php?host=voss.logicarts.com): failed to open stream: Success in /vhost/vhost6/l/o/g/logicarts.com/voss/wp-content/plugins/akismet/akismet.php(11) : runtime-created function(61) : eval()'d code on line 215