Some examples of using
topaz - Gemstone’s command-line - (command-line Smalltalk!) - its often easier than using
Gemtools for administration work on a remote machine.
$ topaz topaz> set user DataCurator pass swordfish gems seaside topaz> login [07/02/11 03:04:52.545 UTC] gci login: currSession 1 rpc gem processId 2824 successful login topaz 1>printit SystemRepository currentLogFile % /opt/gemstone/GemStone64Bit220.127.116.11-x86_64.Linux/seaside/data/tranlog4.dbf topaz 1> printit SystemRepository currentTranlogSizeMB % 342 topaz 1> exit Logging out session 1. $
topaz> set user DataCurator pass swordfish gems seaside topaz> login [07/02/11 03:10:58.557 UTC] gci login: currSession 1 rpc gem processId 2854 successful login topaz 1> printit SystemRepository class allSelectors % an Array #1 _idxForSortUndefinedObjectGreaterThanOrEqualToSelf: #2 asMultilineString #3 _idxForCompareEqualToDoubleByteString: #4 sorted: #5 _auditDependencyMap: #6 _asCollectionForSorting #7 loadVaryingFrom: #8 page
topaz 1> printit System startCheckpointSync % true topaz 1> printit SystemRepository startNewLog % 5 topaz 1> printit SystemRepository currentLogFile % /opt/gemstone/GemStone64Bit18.104.22.168-x86_64.Linux/seaside/data/tranlog5.dbf topaz 1> printit SystemRepository currentTranlogSizeMB % topaz 1> printit ObjectLogEntry emptyLog. System commitTransaction % true topaz 1> printit SystemRepository markForCollection % Successful completion of markForCollection. 940988 live objects found. 261985 possible dead objects, occupying approximately 23578650 bytes, may be reclaimed.
Some tips from Dale
If the strings are not reaped by an MFC then given the oop of the String, you can look at it
Object _objectForOop: <oop> and that might give you a clue as to it’s source. You can use:
SystemRepository findReferencePathToObject: (Object _objectForOop: <oop>)
to find reference paths that keep the string alive …
ScanBackup MFC http://code.google.com/p/glassdb/issues/detail?id=136
(KSSession allInstances select: [:each | each expired]) size
We then expired them all at the Seaside level:
(KSSession allInstances do: [:each | each expire]
If objects are “possible dead” then the other gems et. al have the chance to veto the collection of objects. I would say it is unlikely that all of the 5.000.000 objects are not really dead. Do you know that markForCollection does just a “mark for collection”? It does not garbage collect the objects. Log out all of your DataCurator sessions and wait a few minutes. If things are normal you should see the empty space in the extent grow. If not, complain again.
topaz 1>run SystemRepository markForCollection % topaz 1>run SystemRepository postReclaimAll: SystemRepository reclaimAll %
See section 6.6 ( step 3 ) of the system administration guide.
Ah yes, I forgot to mention the other effect - empty data pages …. over time the repository can have a fair number of completely empty data pages (as objects are changed the old data pages become “empty”). During normal operation these empty data pages take up space on disk.
A backup/restore cycle will completely eliminate the empty data pages, which can lead to dramatic reductions in repository size …
topaz 1>printit SystemRepository markForCollection %
topaz 1>printit SystemRepository fileSizeReport % Extent #1 ----------- Filename = !TCP@localhost#dir:/opt/gemstone/product/seaside/data#log://opt/gemstone/log/%N%P.log#dbf!/opt/gemstone/GemStone64Bit22.214.171.124-x86_64.Linux/seaside/data/extent0.dbf File size = 4990.00 Megabytes Space available = 4223.14 Megabytes Totals ------ Repository size = 4990.00 Megabytes Free Space = 4223.14 Megabytes
Just keeping the last 3-4 tranlog files will give you enough to recover from a crash. This is what we do where I work, and to my knowledge theres never been any issues in almost 10yrs of operation
For crash recovery … where the stone process crashes for some reason. You will need the set of tranlogs that were created since the last checkpoint. You can evaluate the following to find out the oldest tranlog that is needed to recover from the last checkpoint:
Depending upon the amount of disk space needed for the tranlogs during a restoreFromCurrentLogs you may want to look at the family of ArchiveLog methods in the Repository class for additional recovery options.
- 9.4 How to Make a Smalltalk Full Backup
- 9.5 How to Restore from a Smalltalk Full Backup
Clearing the object log
ObjectLogEntry is the key object:
Object subclass: 'ObjectLogEntry' instVarNames: #( pid stamp label priority object tag) classVars: #( ObjectLog ObjectQueue) classInstVars: #() poolDictionaries: # inDictionary: '' category: 'Bootstrap-Gemstone'
There are two class side variables which seem to store the entries:
ObjectLog and are accessed through the following class side methods:
objectLog "expect the caller to abort, acquire lock, and commit if necessary" ObjectLog == nil ifTrue: [ ObjectLog := OrderedCollection new ]. ObjectLog addAll: (self objectQueue _timeSortedComponents collect: [:ea | ea value]). self objectQueue removeAll. ^ObjectLog
objectQueue ObjectQueue == nil ifTrue: [ ObjectQueue := RcQueue new: 100 ]. ^ObjectQueue
_objectLog "direct access to ObjectLog - should have acquired ObjectLogLock if removing entries from ObjectLog" ObjectLog == nil ifTrue: [ ObjectLog := OrderedCollection new ]. ^ObjectLog
So to see the size of the object Queue
printit ObjectLogEntry _objectLog size %
printit ObjectLogEntry objectQueue size %
then to delete entries:
doit System abortTransaction. ObjectLogEntry acquireObjectLogLock. ObjectLogEntry emptyLog. System commitTransaction. %
Create a fresh extent
cp /opt/gemstone/product/bin/extent0.seaside.dbf \ /opt/gemstone/product/seaside/data/extent0.dbf chmod 644 /opt/gemstone/product/seaside/data/extent0.dbf