Environment for creative processing of text and numerical data |  |
Environment for creative processing of text and numerical data | |
Survo is constantly developed further, and new versions are released. Perfect backward compatibility is strictly maintained. As new functions do not exclude existing ones, the user does not need to re-learn how to use Survo.
On this page we tell about the new functions of SURVO MM since the version 1.0 was released.
(PRINT?)
SURVO66 operation for emulating SURVO 66 (SURVO66?)
http://www.survo.fi/demos/#ex106
GDIST operation for computing distance distributions between two random
points in a metric network (GDIST?)
(ECHO?)
LST commands for processing lists of 'words' in the edit field (LST?)
(FILL?:4)
(WCOPY?)
(SURVOPOINT?)
LOAD <Survo>\U\D\SPOINT
FILE specification, the TAB command saves the table also as
a new Survo data file (TABFILE?)
(MOUSE?:6).
SHADOW SET command makes the same for shadow lines as the standard
SET command does for normal edit lines (SHADOW? SET?).
SPECS (extending the set of primary specifications
on the command line) is available for a limited set of operations
(SPECS?).
/HIGHLIGHT
TABTEST has been extended by the common Pearson goodness of fit test
(GFIT?).
COLOR(n) specifications (SHADING?)
POINT specification of the form POINT=...[rotate(variable)]...
(POINT?)
WORDS is a new command
either for
(WORDS2?)
F1 J works as F2 J but makes possible to complete a phrase consisting
of two or more words by a model found elsewhere in the edit field. (PREJ?)
(FENCE?)
In scatter plots, points can be connected by lines and arrows (vectors)
by entering (in connection with PLOT scheme) a specification of the form
ARROWS=A,[plus other parameters describing properties of vectors].
A is an incidence matrix saved as a Survo matrix file and its non-zero
elements indicate points to be connected. The values (when needed)
tell individually the type of line (vector) to be drawn. Thus the line
width, type, and color as well as the type of arrow heads can be
specified separately for each vector.
This option is available in PostScript graphics only.
(ARROWS?)
Examples:
F2 M J (Sucro /J)
KEYWORDS
by default) where a list of keywords is displayed.
The edit file of keywords can be edited and extended by new keywords.
(SEARCH?:8)
F1 V
F1 V displays the extended form
of the name on the message line.
(F1?F)
/COLORMAT is a new sucro for creating 'color mats' with gradually
changing shades of colors.
Examples:
http://www.survo.fi/papers/ColorChanges.pdf
(COLORMAT?)
/GS-PDF sucro has been modified so that the PDF file created
using Ghostscript is opened by the default reader program
of the computer (Adobe Reader, Adobe Acrobat, Sumatra etc.) and
with specification OPEN=0 it's not opened at all. The latter option
is useful when PDF files are created automatically as a part of
a larger work scheme or a sucro.
DD desktop program now opens any file pointed to by the cursor
when 'O' (Open) is pressed. The program to be used depends on
the file type and the configuration of the computer, similarly
as it is with the /OPEN sucro.
STAT operation are saved as a matrix file STAT.M
(STAT?:sp:sp:M).
GEOM is a new program for describing geometric ruler and compass
constructions. By GEOM it is also possible to calculate the statistical
accuracy of a given construction. The principle and usage of GEOM is
documented in
CODES commands are extended so that they can be used for bytewise
editing of (large) files. For example, parts of a file can be removed
so that it is then possible to convert a file to a Survo data file.
F1 function key (being previously a Help key only) works now
as a new prefix key (like F2) but for different tasks.
F1 codes.
More information by F1?
TRANSPOSE is a new command for transposing a table or a data set
given as a text file. TRANSPOSE thus exchanges the "rows" and
"columns" (TRANSPOSE?).
/MONITOR is a family of sucros for monitoring selected files (typically
matrix and data files) in another Survo window.
More information is
obtained by activating
/MONITOR-HELP
and by watching the Flash demo
http://www.survo.fi/flash/e_monitor.html
REVERSE is a new command for changing words, lines or bytes on the
lines in opposite order (REVERSE?).
F2 J (see PREJ?) have been
extended so that one can copy word by word texts on multiple lines
in the edit field by the space bar until the first empty line.
Earlier this was possible within one line only. It is also possible
to undo the last selection by the BACKSPACE key.
/CONTFRAC for computation of a simple continued fraction
presentation of a positive real number
/LOAD2 <pathname_of_an_edit_field>
WINDOW command.
This is one of the best ways for working simultaneously with several Survo sessions.
WINDOW 18,0,0
WINDOW?).
The same thing is done permanently by the command
SYS window=18,0,0
It inserts (or replaces) line window=18,0,0 in system file SURVO.APU
and this setting becomes valid when Survo is started for the next time.
These activities have no effect if properties of the main window have
been altered by clicking the Survo icon in the title bar.
To get them working, see instructions (LNKFILE?).
SHOW command have been extended.
If the file to be shown is an edit file, the first line can also be given by a symbolic label (appearing in the control column).
The first line can also be specified by a text appearing on that line
and this applies also to text files (SHOW2?)
SHOW <edit or text file name> / SHOWLOAD=L1,L2
loads lines from L1 to L2 from the file directly to the current edit
field. The output starts from the edit line below the command line.
The line labels can be numerical, symbolic (in edit files), or
indicated by texts given in double quotes. (SHOWLOAD?)
For example, it is easy to insert essential lines (with shadows)
from a Survo edit file to the current edit field by the command
SHOW <edit_file> / SHOWLOAD=2,END INSERT=1
where INSERT=1 allows SHOW to add empty lines automatically.
Thus SHOW should never overwrite text in the edit field.
OTHER?).
LINEINS is a new command for insertion of new lines between given
edit lines (LINEINS?).
INSERTL command inserts empty lines in the edit field (INSERT?).
DELETEL command deletes several consecutive lines (DELETE?).
SHOW command (SHOW?).
(CLIP?) the clipboard can be
maintained by keys ctrl+ins (COPY), shift+ins (PASTE).
QUANTA is a new module for estimating quanta from a univariate
data set.
(G)PLOT <data> / TYPE=PYRAMID
|EXAMPLE| by BAR?:TE)
MAT #EIGLAN(A,k,S,L)
L and their eigenvectors S
of an n*n symmetric matrix A by the Lanczos method.
This is a faster method than creating the entire spectral decomposition
of A (by MAT SPECTRAL DECOMPOSITION OF A TO S,L) when n is 200 or more
and k is small.
It is substantially faster when n is several thousands and thus really
suitable e.g. in classical multidimensional scaling where k is typically
very small, often 2.
Therefore a new version /CSCAL2 using MAT #EIGLAN has been made of
/CSCAL.
/LOAD <Survo>\U\R\INDEX
POINT_TYPE=<type_variable>.
<type_variable>
are 0,1,...,11.
/P_TRIPLE
LOAD SR
POINT_COLOR=<color_variable>.
Only integer values for the color variable are permitted and for each
value 'n' a specification
FILL(n)=c,m,y,k
has to be given in the current PLOT scheme (POINTC?).
/INFL_R makes influence curves for the correlation coefficient
of two variables of any data set as contour plots.
The user may indicate a new point in the graph by the mouse and see
its effect to the correlation coefficient (/INFL_R?).
/INFL_R DECA,Height,Weight,0.05
GPLOT
scheme by the command
COMPARE <sample1>,<sample2>,L / TEST=Q-Q
(Q-Q?)
LINREG produces estimates of parameters in an orthogonal regression
model of one regressand and one regressor as a 2x1 matrix file OREG.M
as a by-product (LINREG?).
FILE MASK operation for selecting various attributes to variables in
data files has been extended by several options needed in sucros and
for handling large sets of variables (FMASK?).
FILE MEDIT operation is
available in the Survo help system (MEDIT?).
It includes plenty of |EXAMPLE| sucros. Many details are best revealed
by watching these examples and FILE MEDIT setups generated by the
sucros.
REGDIAG gives more results as matrix files helping additional
computations (REGDIAG?:M).
(MAT SAVE) also rowwise comments may be given.
These comments are saved in the matrix file and will appear in their
proper places when using MAT LOAD (MATSAVE?:A, third page).
RG.M (a collection of various scalar parameters and results of REGDIAG)
is a good example of such a self-commenting matrix.
CLF techniques recently developed by Robert I. Jennrich (ROTATE?).
ROTATE works now also in tutorial mode.
This enables making teaching programs about this topic as sucros.
ROTATE? (F, second page) gives a link to an |EXAMPLE| sucro which
demonstrates graphical rotation by a small example.
(ROTATE?) has been generalized
according to principles suggested by Lauri Tarkkonen.
The sucro command
/EXTFACTA <data_file>,<#_of_factors>
creates a setup for carrying out graphical rotation in this extended
form after the user has selected the variables into the analysis by
masks A (visible), B (invisible), and external variables by masks X.
EXTFACTA makes the correlation matrix of all variables (A,B,X), computes
the factor matrix FACT.M from A,B-variables (by ML method by default),
and the loadings of external variables (matrix EXTFACT.M) directly from
correlations between A,B- and X-variables and from FACT.M (without a
need to compute factor scores).
The setup when applied to the DECA data will be, for example:
MASK=--ABAAABBBABXX /EXTFACTA DECA,3 Setup for graphical factor rotation generated by /EXTFACTA: Primary factor matrix saved in FACT.M _ /MATSHOW FACT.M EXTERNAL=EXTFACT.M Loadings of external variables _ /MATSHOW EXTFACT.M FSHOW=FSHOW.M Variables shown in displays _ /MATSHOW FSHOW.M ROTATE FACT.M,3,CUR+1 / METHOD=GRAPHICAL
SURVIVAL is a new operation for survival analysis programmed by
Kim Huuhko (SURVIVAL?)
A new alternative is the mrand() function using the Mersenne Twister algorithm by Takuji Nishimura and Makoto Matsumoto (1998, modified 2002). It has a huge period 2^19937-1 (a Mersenne prime), i.e. about 10^6000.
Currently mrand() is available in operations VAR, TRANSFORM BY #UNIFORM,
MAT #TRANSFORM, MNSIMUL. (RAND?)
{play sound <WAV_file>}
is a new code in sucros for playing sound files
asynchronously while a sucro is running.
START /GS-PDF sucro.
',' key.
During an interrupt the user can scroll the edit field, type
temporary text, edit it and activate Survo commands, but not sucros.
The tutorial continues thereafter from the break point by pressing
the keys F2 and ','.
Sounds indicating interrupts etc. are set by a 'tut_sounds' line
in the SURVO.APU file (TUTSOUND?).
(ERROR?).
T2TEST <data1>,<data2>,<output_line>
(T2TEST?).
COVTEST <output_line> / SAMPLES=<data1>,<data2>,...
(COVTEST?).
(ERROR?)
If a misspelled command is activated, Survo tries to offer suitable correct alternatives.
A corresponding checking procedure will be extended to specification
words as well. Currently only a small portion of Survo operations allow
specification checking. For example, all forms of PLOT and its relatives
enable this extension.
Checking is based on Levenshtein (edit) measure which tells the number of edit operations (insertion, deletion, substitution) needed to transform one string to another.
This measure is used also in new heuristic forms of the FIND command:
FIND<word>~ / Find word that best resembles <word> in the edit field.FIND<word>~~ / Case-insensitive search by the same principle.(HFIND?)
STRDIST
command (STRDIST?).
ARMAX models by ESTIMATE (ARMA?)
/DEVPLOT for making mean-deviation bar plots e.g. in attitude
surveys (DEVPLOT?)
Z for working temporarily with a "slave survo". See e.g.
/Z ?
NTERM operation for guessing the general term in integer sequences
(by Reijo Sund).
REGDIAG
(DURBIN?).
ESTIMATE (AR?)
(MOUSE6?).
(SESSION?)
/GS-PDF for previews and printing of reports (GHOSTSCR?)
FILE COPY it is possible to make sure (by parameter NEW) that
the target file is deleted before copying by using
FILE COPY <original_data> TO NEW <target_file>.
(FCOPY?)
POWERS <data> / POW_VARS=<list_of_variables> DEGREE=<d>
POW_VARS as (new) variables.
POWERS is a tool for generating variables for polynomial regression
analysis and its generalizations, for example. (POWERS?)
(PLOT)
CONF_BAND specifications are available for plotting confidence intervals
of various types in simple linear regression (CONFBAND?).
TREND specification also a trend line where X is explained by Y
can be plotted (TREND?).
FILE EXPAND command increases the maximum number of variables and
the length of each record of observations in a Survo data file
(FEXPAND?).
MAT G=MPINV(A) computes the Moore-Penrose inverse G of any m*n matrix A
(MPINV?).
LINEDEL command deletes lines in the edit field under various
conditions. It also saves deleted lines in a new .EDT file if needed.
LINEDEL in many ways augments tools (such as SCRATCH, ERASE, CLEAR)
for cleaning of edit fields.
(LINEDEL?)
TIME COUNT FILE opens (and closes) a text file for saving execution
times of successive operations (TIMES?).
/BCA
PCOPY operation for splitting huge files and for combining small
ones. PCOPY enables copying a large file to another computer when
the file doesn't fit to a diskette or to a CD (PCOPY?).
FILE MEDIT operation extended (MEDIT?).
(SAMPLES?).
/COLORMAP-PRINT
COLORMAP.PS is created in the current data path
of Survo. This file can be printed, for example, by the command
/PS-PDF COLORMAP.PS / (if Adobe Acrobat is available)
(OPEN? LINK?)
make a sample of size m from rows ofMAT C=#SAMPLE(A,m,URS,seed_number)/ Unrestricted Random SamplingMAT C=#SAMPLE(A,m,SRS,seed_number)/ Simple Random Sampling
A and save the rows
of the sample as a new matrix file C.
MAT C=#SORT(A,k)/ Sorting rows ofAaccording to the k'th column
(MATEXT?)
VAR and MAT #TRANSFORM operations for
Gamma, beta, Weibull, and exponential distributions (FUNCSTAT?).
Example: x=22.222222 gamma.F(5,2,x)=0.98598831089672 is the same as
chi2.F(10,x)=0.98598831089672
MAT C=NULL(A)/Cis an orthonormal basis of the null space ofA.MAT C=BASIS(A)/Cis an orthonormal basis of the column space ofA. (MAT?:BC, see alsoMAXDET?)MAT C=RANK(A)/C(1,1)=the rank ofA(Cis a 1x1 matrix)MAT C=TRACE(A)/Cistr(A).MAT C=DET(A)/Cis the determinant ofA.MAT C=LDET(A)/Cis the logarithm ofDET(A).
MAT C=#INTSCAL(A,n)
A to integers by first evaluating each element
as the n'th convergent of a continued fraction expansion of A (default
n=10) and then finding the least common multiple of denominators.
A is a numerical constant or a 1x1 matrix,
MAT C=#INTSCAL(A,n)
A for up to n terms (default n=10)
and saves both partial quotients and convergents etc. in a matrix
file C. (INTSCAL?)
OPEN opens files created by other programs using a suitable program.
INDEX operation associates /OPEN command with the following file types:
.PDF, .DOC, .XLS, .PPT, .JPG, .JPEG, .GIF, .HTM, .HTML, .PNG, .RTF, .TIF, .TIFF
IND and CASES conditions have been extended.
In conditions of the IND type 'values' MISSING and !MISSING are
accepted. Example: IND=Length,!MISSING (IND?)
CASES type, spaces and wild characters can
be used by entering proper extra specifications (CASES?).
These extensions are valid in all standard SURVO MM operations
dealing with IND, CASES, and SELECT specifications.
FILE AGGR: New MISSING function (AGGR?:3)
MAT #CONVOLUTION(C,A,B) computes the convolution C of vectors A and B.
MAT #CONVOLUTION -operation (MAT?:X,A)
(CENTRAL?):
/DISTRSUM (without parameters)
DISTRSUM is assisted by another sucro
/SUMDISTR <vector_of_probabilities>
SUMDISTR computes the probabilities of sum distributions stepwise
by the MAT #CONVOLUTION operation.
|EXAMPLE| sucro is available for studying the central limit
theorem by simulation. This sucro is also an example of using the
TRANSFORM <data> by #DISTR(P)
operation.
(TRANSF2?:|EXAMPLE|).
The Help system contains also examples about Survo usage. So far the user has had to load them into the current edit field in order to see how they are working.
Now it is possible to activate examples directly from the Help system.
When a 'button' |EXAMPLE| appears in the Help window a demo related
to the current subject will be started by double-clicking this button.
In fact, another copy of SURVO MM will then be doing this task with a
sucro automatically. When the example is terminated, the user must exit
this second Survo and then he/she will be back in the original
Survo session.
This option highly improves learning Survo usage.
As a long-term goal we are going to extend the help system to cover
"all" activities with illustrative |EXAMPLE|'s, but this requires
a lot of work. However, already now there are plenty of demo sucros
related to the examples of the Survo book (1992); most of them
will be connected to the Help system in the nearest future.
Currently |EXAMPLE|'s can be found in about 30 places and they can be
found systematically, for example, by activating
/HSEARCH |EXAMPLE|
Various |EXAMPLE|'s:
COUNT? 4 examples of various forms of theCOUNTcommand FORM? Example on page 36 of the Survo book PLOT?:1,X,G NewGROUPINGspecification in bar charts MOUSE2? Use of the mouse when positioning texts in screen graphics CORRESP? (Old) example related to correspondence analysis
DAT-function for displaying and editorial computing with values
in Survo data sets (data files, data matrices in the edit field, etc.).
It is especially useful in sucro applications dealing with individual
data values but it is also a nice tool for 'information retrieval'
(DAT?, also |EXAMPLE| included).
STAT operation the default output length of string variables
(on nominal scale) in frequency distributions is 8 and this has been
possible to raise by a MAXSTRING specification. MAXSTRING has, however,
been defective by working only for up to lengths of about 20.
Now this flaw has been corrected and much longer lengths are permitted.
HBAR, %HBAR, etc.) the names of the
cases can have up to 100 characters.
CORRTEST operation has been extended to make comparisons of correlation
coefficients of readily-computed results (CORRTEST?).
GROUPING specification (PLOT?:1,X,G).
CONFMEAN=<confidence_level>
(e.g. 0.95)
the STAT module computes a confidence interval for the mean of
each variable by assuming normality of the sample.
PLOT and GPLOT) where 'space' is indicated
by '_' and 'comma' by ;
the characters _ and ; can be obtained by combinations \_ and \;
(PLOT?:T).
LOGMEAN module for computing the logarithmic mean
for several arguments (LOGMEAN?),
see also a
paper on the subject.
LOADW: New SPLIT=SP option for splitting long lines without breaking
words. Also text in the Windows clipboard can be copied to the edit
field by LOADW CLIPBOARD (or simply by keys F2 M P).
MULTIPLE?
ARIT operations for multiple precision arithmetics
are available as a preliminary version.
Examples are found by activating the command
LOAD <Survo>\U\D\LONGARIT
GPLOT) various texts, headers, scale notations, etc.
can be both rotated and moved (TEXTROT?).
These features have been available already earlier in PostScript plots
of Survo.
NORMAL , BRIGHT ) by soft buttons SYSTEM COLOR .
F is available for function keys, certain special
keys, and key combinations.
MARKOV?.
New functions are demonstrated in the tutorial
/MARKOV-DEMO2
START field as a hypertext.
LOAD <Survo>\U\ESTART
FIND, REPLACE, and alt-F5 key have been case-sensitive.
There are now two means to make those searches in a case-insensitive
way. See FIND?:C (Case-insensitive searches).
MINSTAT operation for computing basic statistics.
MINSTAT saves the table of results also as a matrix file STAT.M
thus simplifying usage of basic statistics in sucros, for example
(MINSTAT?).
XJITTER and YJITTER specifications)
in an ordinary scatter plot is possible. This feature has been
available for a long time in scatter plot matrices (TYPE=DRAFTS).
Jittering makes the graph more informative when scales of variables
are roughly discrete. In such cases (without jittering) many points
would overlap each other (JITTER?).
ESTIMATE operation allows using the density and cumulative distribution
functions (phi and PHI) of the standard normal distribution (EFUNC?).
SCATPLOT for making scatterplot matrices using graphics windows:
scales, regression lines, smooths, jitterings, contour ellipses etc.
available.
TABDATA and TABMAT.
FILE LOAD operation.
This enables creating automatic, verbal reports for individual cases
(FLOAD?:2).
GPLOT:
When combining enhanced metafiles (by INFILE, OUTFILE technique) sizes
of files grow exponentially in Windows NT/2000 environments. This is
an obvious bug by Microsoft.
The problem has been solved by means of Survo by filtering out
unnecessary statements from those metafiles. This change has no
effect in GPLOT usage.
c (copy) and p (paste)
(see CLIPBOARD?)
FILE COPY operation a set of MATCH fields can be given in the form
MATCH=<match variable 1>,<match variable 2>,...
LMSELECT for computing linear regression model selection
statistics for all combinations of regressors.
INDEX /PRINTERS gives information of local printers.
DCONTOUR, MATSHOW, TUTSHOW, CODESHOW, and BOXCOX developed further.
/LOAD <Survo>\U\MATRIX\INDEX,1,7
LARGE?).
PLAY command has been extended so that WAV files can be created
from Survo data files. These functions will be still developed.
Examples in Finnish are found by activating the command:
/LOAD <Survo>\U\SND\PLAYDATA
ROBREG module (programmed by Reino Siren)
performs robust estimation of the parameters of a linear regression
model by the method of Least Median of Squares (Rousseeuw 1984,
Rousseeuw & Leroy 1987). Instead of minimizing the sum of squared
residuals (OLS), the median of squared residuals is minimized, giving
estimates which are highly robust to outliers in the data.
PLAY SOUND(S) command for playback of sound files.
SYSTEM WINDOW .
PLOT or GPLOT by using
a LINE=POLYGON specification.
GPLOT a ROP specification tells the Raster OPtion for color
combinations of overlapping grphical objects.
/ROPDEMO gives an example of features mentioned above.
INDEX operation.
PS-PDF for displaying a PS file by converting it to PDF format by
using Adobe Acrobat Distiller
STAT operations sums, sums of squares, etc. can be computed
by specification
SUMS=<1,2,3,or 4>
.
FILE SELECT new specifications MARK and NMATCH.
POLYGONS specification is now available also in PostScript plotting.
COMB operation.
FILE SORT the (combined) sort key can be saved in the sorted file
by a KEY_SAVED specification. This new feature is helpful in
FILE AGGR since this key is often used as the aggregating variable.
FILE LOAD missing values can be replaced by a given string
by using
MISSING=<string>
specification.
GPLOT, polygons can be drawn and filled by a POLYGONS specification.