uvsort - UNIX sort utility with mainframe functionality [librefs]

uvsort - sort utility for Unix/Linux systems - CONTENTS

A1.	uvsort is similar to the 'uvcp' utility with the addition of sort keys
A2.	uvsort FACILITY also available from within the 'uvcopy' utility
A3.	Jan 2004 enhancements - up to 20 input files & 60 output files
A4.	Nov 2004 enhancements - maintain input sequence & field type 'g'

B1.	uvsort FUNCTION Summary
B2.	uvsort INSTRUCTION Summary (same as uvcp)

C1.	Sort Key field data types, sequence codes
C2.	Sort Key formats & options
C3.	Sort collation sequence for field type 'g'
C4.	sort key option 'u' (unique) to drop records with duplicate keys
C5.	sort key option 's' (select) to select records with duplicate keys
C6.	sort key option 'f1' to write separate files for each key1 dup set
C7.	Year 2000 field type & options to modify window
C8.	Illustrated demo of sliding windows

D1.	FILE DEFINITION parameters
	(fili1/filo1, typ, rcs, isk)

E1.	File TYP's & file TYP OPTIONS
	(typ=RSF/RSV/ISF/IDX/LST/RDW - see details ahead)
E3.	Indexed Sequential Variable record length files (typ=IDX)
	compatible with Micro Focus COBOL IDXFORMAT3
E4.	File type options (a thru z)
E7.	File type options for typ=IDX files (f,v,x,y,z)

F1.	Run options a-z
F3.	Run option examples (assigning more memory, stop & bypass counts)
F4.	Run option 'f2' to input all files from a directory
	- optionally matching any specified prefix &/or suffix
F5.	SORTDIR environmental variable for the sort work subdir.
	Interrupt processing can be enabled via run option e1.

G1.	Select & Delete records
G2.	Instruction option 'o#' to select/delete records to different output files
G3.	option u1 to code blanks as underscores & u2 to code commas as at-signs
	- for sel/del, also mvc & rep (documented in uvcp.htm)

H1.	SUM function - accumulates fields into 1 record for duplicate key sets.

I1.	Instruction option 'o' to execute instructions
	at sort output time vs sort input time (the default).
I2.	Instruction option 'o#' to execute instructions on different output files

J1.	FILE DECLARATION - 4 METHODS

K1.	MULTI-file declaration - 3 METHODS

Goto: Begin this document , End this document , UVSI Home-Page

uvsort - sort utility for Unix/Linux - CONTENTS (continued)

L0.	uvsort examples - using demo files provided
	demo file /home/uvadm/dat1/sales1
	- 64 byte records, mainframe type, fixed length records, no LineFeeds

L1.	ex#1 - sort sales1 demo file on 1 field (product#)
L2.	ex#2 - sort sales1 demo file on 2 fields (salesman# & product#)
	- also change output file to typ=LSTt to insert LineFeeds
L3.	Sort sales file by product within salesman
	select HAMmers for salesmen 21,44,& 65
L4.	Sort & Load an Indexed file with 2 keys (customer# & city-name)
L5.	Read, Sort,& re-Write an Indexed file back into the same file
	- a good way to reorganize indexed files & drop deleted records
L6.	Create customer telephone list - sorted by telephone#
	- reformat 256 byte record to 50 bytes (cust#, name,& telephone#)

M1.	Using uvsort to rebuild indexed files from the data partition of
	ISAM files with corrupted Indices. (both Microfocus & MBP COBOL)

N1.	Record Sequential Variable (typ=RSV) files
	- compatible with Micro Focus COBOL IDXFORMAT3 files

O1.	Test/demo uxsort creating Indexed Sequential Variable files (typ=IDXf3/f8)
O2.	uvhd hexdump of a sample IDXFORMAT3 file
O3.	uvhd hexdump of a sample IDXFORMAT8 file
O4.	IDXf3/IDXf8 file header & record header layout
O5.	uxsort scripts used at UVSI to write various IDX files & read back
O6.	uxsortIDXf3w test/demo script - writes 6 IDXf3 files with various options
O7.	uxsortIDXf8w test/demo script - writes 6 IDXf8 files with various options
O8.	directory listings after writing IDXf3/IDXf8 files

P1.	typ=RDW (default typ=RDWz4) Record Descriptor Word variable length files
	- 4 byte record prefix with 2 byte binary record size & 2 unused nulls
	- using 'uvhd' to display & convert EBCDIC RDW files to ASCII text
	- using 'uvcp' to convert EBCDIC RDW files to ASCII fixed length files
	typ=RDWz2 - option 'z2' for 2 byte record prefix (vs 4 above)
	typ=RDWz8 - option 'z8' same as RDWz4, but bypasses any BDW prefixes

Q1.	SyncSort to uvsort converter (new as of May 2010)
	'syncsort2uvsort1' converts unix/linux SyncSort parameters to uvsort
	sample SyncSort parameters & uvsort equivalent
	- 3 output files with selections by salesman & amounts > $200.00

R1.	syncsortsplit1 - separating SyncSorts from scripts
	SyncSort parameters usually embedded in scripts, uvsort converter expects
	uvsort parameters to be in a separate file, called from the script.

S1.	CoSort to uvsort converter (new as of May 2010)
	'cosort2uvsort1' converts unix/linux SyncSort parameters to uvsort
	sample CoSort parameters & uvsort equivalent
	- 3 output files with selections by salesman & amounts > $200.00

T1.	uvsort vs MFSORT (Micro Focus COBOL standalone sort utility)
	- samples of mfsort & equivalent uvsort
	- sort time tests on big files (1 gig, 4 gig,& 8 gig)

Goto: Begin this document , End this document , UVSI Home-Page

A1. uvsort Overview

Uvsort is a very efficient and powerful utility that provides mainframe sorting capability to UNIX systems. The UNIX system sort is unsatisfactory for mainframe style data processing. For example specifying fixed sort fields is extremely awkward and packed fields cannot be handled.

uvsort example


 uvsort "fili1=infile,rcs=64,typ=RSF,filo1=outfile,typ=ISF,isk1=0(6),key1=0(6)"
 ==============================================================================

This example reads a file of fixed size records. 'rcs=64' specifies ReCord Size is 64 bytes, and 'typ=RSF' indicates Record Sequential Fixed. 'key1=0(6)' specifies sorting on the 1st 6 bytes (displacement 0 & length 6). The output file (filo1) is 'typ=ISF' Indexed Sequential Fixed and 'isk1=0(6)' means the Indexed Sequential Key is also the 1st 6 bytes (same as sort key).

Uvsort is a very efficient and powerful utility that provides mainframe sorting capability to UNIX systems. Uvsort is twice as fast as the UNIX sort and six times as fast as 'vcsort' (MBP COBOL standalone utility).

The UNIX system sort is totally unsatisfactory for mainframe style data processing. The unix system sort cannot handle fixed record sizes without LineFeeds, cannot sort on packed decimal fields,& cannot read or write Indexed files (as in the example above).

Uvsort provides for 300 record selects (or deletes) that may be based on multiple and/or conditions comparing record fields to constants. Uvsort also provides for 400 instructions for reformatting records (move,add,subtract,multiply,compare,skip,etc)

UVsort is very similar to the uvcp utility with the addition of sort keys. Keys may be specified as separate parameters or as a string, for example:

        key1=0(8),key2=20(15),key3=30(5pd)
                      -or-
        keys=(0,8,c,a,20,15,c,a,30,5,p,d)

Up to 9 keys are allowed in format 1 & up to 15 keys in format 2.

This uvsort section will repeat some documentation for file declarations which are similar to uvcp but not identical (re multifile input).

This uvsort section will not repeat any documentation for the processing instructions (add,sub,mpy,mvc,etc) since they are identical to the uvcp utility & have already been documented in the uvcp section.

Goto: Begin this document , End this document , UVSI Home-Page

A2. uvsort: similar sort facility is also provided in uvcopy

alternate version 'uxsort' for IDXFORMAT3/8 files

'uxsort' is an alternate version of uvsort that includes support for Micro Focus COBOL IDXFORMAT3/8 files (Indexed Sequential Variable length records). You must execute uxsort to read or write 'typ=IDX' files. Note that Fixed length Indexed files (compatible with Micro Focus COBOL IDXFORMAT1) are supported by both uvsort & uxsort.

'uxsort' must be compiled with the Micro Focus 'cob' command. Please see install.doc which describes the use of the 'cobccuv' script which makes it easy to compile Vancouver Utility programs using the 'cob' command.

similar sort facility also provided in uvcopy

Uvcopy also includes the uvsort facility, implemented in a more powerful manner, but not as convenient for simple sorts, because uvsort can specify all parameters on the command line.

In uvcopy, the sort is implemented as several instructions for the various sort phases, which provides a lot of flexibility.

For example, you can reformat the records as you are giving them to the sort, then execute the sort,& then load an indexed file as you are retrieving the records from the sort (possibly appending sequence#s to make unique keys).

See the sort instructions documented at the end of uvcopy3.doc (sxo,sxp,sxs,sxg,sxc).

For some particularly good examples of using the sort within uvcopy please see the following jobs in other sections of the documentation.

   index  - in section TEXTjobs.doc
          - create an index from a document containing keywords & page ejects

   permute - also in the TEXTjobs section.
           - create a permuted index from a table of contents

Goto: Begin this document , End this document , UVSI Home-Page

A3. Jan2004 Enhancements to uvsort

Jan2004 Enhancements to uvsort

uvsort now allows you to specify up to 20 input files & up to 60 output files. On input you may specify record selects/deletes/reformats that are applied to all records. On output you may specify record selects/deletes/reformats that are different for different output files.

A maximum of 20 input & 60 output files apply when you specifically name the desired files (fili1=...,fili2=...,fili20=...,filo1=...,filo2=...,filo60=...).

An alternative to specifying multiple input filenames is to use the new run option 'f2', to input all files from a specified directory, OR all filenames matching a specified prefix &/or suffix.

For output files, you may specify option 'f1' on key1 (key1f1) to output multiple separate files for duplicate record key sets based on key1. This is a unique & powerful feature that can be used to write out separate files for each customer#, salesman#, state/province, etc.

The output filenames are automatically created in the specified output directory, with an optional prefix, followed by the current key1 data value.

These enhancements make uvsort even more powerful than SYNCSORT.

links to enhancements

C5.	sort key option 'f1' to write separate files for each key1 duplicate set
F4.	Run option 'f2' to input all files from a directory
	- optionally matching any specified prefix &/or suffix
G2.	Instruction option 'o#' to select/delete records to different output files
I2.	Instruction option 'o#' to execute instructions on different output files
H1.	SUM function - accumulates fields into 1 record for duplicate key sets.

JCL converter also enhanced

The mainframe JCL to Korn shell script converter has been updated to use these new uvsort enhancements when converting SYNCSORTs to uvsort. Please see http://www.uvsoftware.ca/MVSJCL2.htm#6B1 for an example of converting a SYNCSORT with multiple selects/deletes for multiple output files.

Goto: Begin this document , End this document , UVSI Home-Page

A4. Jan2004 Enhancements to uvsort

Nov 2004 Enhancements to uvsort

uvsort now maintains the original file record sequence. This is effective when multiple records exist for the specified sort keys. This feature is implemented by automatically adding an extra low order sort field with an input record sequence number. For best performance, uvsort uses the 'qsort' algorithm which does not guarantee original record sequence.

sort field type 'g'

Sort field type 'g' changes the character collation sequence. The exact sequence is shown on page C3 of http://www.uvsoftware.ca/uvsort.htm#C3. This was requested by customers converting DB2 UDB applications from the mainframe to unix/linux. Other sites might use this if they want UPPER & lower case letters to sort together (vs the separate ranges of ASCII).

Goto: Begin this document , End this document , UVSI Home-Page

B1. uvsort FUNCTION SUMMARY

 fili1=infilename           - declare input filename
 fili20=...                 - up to 20 input files may be declared
 fili1=dir/*                - input files from directory option (rop=f2)

 typ=LST,RSF,RST,RSV,ISF,IDX,RUF - declare file type
 typ=LSTt                          & possible options appended to file type

 rcs=9999                   - declare record size

 filo1=outfilename          - declare output filename
 filo60=...                 - up to 60 output files may be declared
 filo1=dir/*                - multi-file output by option 'f1' on key1 breaks

 iskx=start(length)         - declare indexed file keys

 keys=(0,8,c,a,30,5,p,d)    - declare sort keys (format 1)
         -or-
 key1=0(8),key2=30(5pd)     - declare sort keys (alternate format)
                            - options to delete or select duplicate keys

 rop=abc...xyz              - declare misc run options

 sel=start(lth):constant     - select records with spcfd value in spcfd cols
 del=start(lth):constant     - delete records with spcfd value in spcfd cols

prm=paramfile

declare name of parameter file containing uvsort parameters
use for complex sorts with many sels/dels &/or output files
not required for simple sorts since all parameters
may be specified on the command line
see example on page 'J1'
could use the sort facility of uvcopy vs complex uvsorts

Goto: Begin this document , End this document , UVSI Home-Page

B2. uvsort INSTRUCTION SUMMARY

      logical:  mvc, mvn, clr
   arithmetic:  add, sub, mpy, div, pac, unp
 compare/skip:  cmc, cmn, skp
 scan/replace:  rep, scn, sqz
   translates:  tra, tre, trl, tru
        other:  anc, orc, chx, hxc, tim
          I/O:  put

Note	the instructions are the same as for the 'uvcp' utility please refer to the uvcp documentation

Goto: Begin this document , End this document , UVSI Home-Page

C1. uvsort: sort KEY specification

sort key format #1 - up to 28 keys allowed

keys=(displacement,length,data-type,sequence, ... etc )

 keys=(dsp1,lth1,typ1,seq1,dsp2,lth2,typ2,seq2,----etc----)

 keys=(0,8,c,a)            - sortkey = cols 1-8, character, ascending

 keys=(0,8)                - same (character,ascending is the default)

 keys=(0,8,c,a,228,3,p,d)  - sortkey #1 = cols 1-8 character, ascending
                           - sortkey #2 = cols 229-231 packed, descending

sort key format #2 - up to 9 keys allowed

key#=displacement(length)

key1=dsp1(lth1),key2=dsp2(lth2),key3=dsp3(lth3),etc

 key1=0(8ca)            - sortkey #1 is cols 1-8
                        - character ascending

 key1=0(8)              - sortkey #1 is cols 1-8
                        - character ascending is the default

 key2=228(3pd)          - sortkey #2 is cols 229-231
                        - packed decimal, descending

Sort key format#2 allows fewer sort fields, but more options, which will be described on the following pages.

Goto: Begin this document , End this document , UVSI Home-Page

C2. uvsort: sort key specification

sort field data types

a	alphabetic (case insensetive, lower & upper treated equally)

b	binary (bit for bit) - same as character (see below)

c	character ASCII (the default)

e	EBCDIC collating sequence (for ASCII data sort fields)

f	first blank high, same as type 'c' (character ASCII), except that if the 1st byte of the field is a blank, it will be sorted as though it were a tilde x'7E'

g	alternate collating sequence for DB2 UDB with some German characters sorts UPPER & lower case alphas together (vs separate ranges) see exact sequence on the next page

h	sort all blanks high regardless of field location (re type f) otherwise the same as type 'c' (character ASCII)

i	integer (2 or 4 bytes) Intel reversed byte architecture

p	packed decimal considers sign in right nibble of right hand byte x'0D' is negative, all else is positive

u	unsigned integer (length must be 2 or 4)

special field type for the Year2000 problem (2 digit years)
default window is '50' meaning years 50-99 = 1950-1999
& years 00-49 = 2000-2049
specify run option 'y__' to modify window & 'z1' for sliding windows
rop=y70z1 in 1998 means 68-99=1968-1999,& 00-67=2000-2067
see 'rop' run options in sub-section 6 ahead --->

z	zoned decimal considers sign in left nibble of right hand byte negative if x'40' bit present, else positive

sort field sequence codes

a	ascending sequence (the default)

d	descending sequence

Goto: Begin this document , End this document , UVSI Home-Page

C3. uvsort: Sort Key Specifications

Sort Sequence for field type 'g'


 uvsort "fili1=infile,rcs=80,typ=RSF,filo1=outfile,key2=20(30g)"
 ============================================================*==

Field type 'g' can be used to sort to the sequence shown in the table below:

 00-1F - control characters, reassigned to x'FF' to sort high
 20-2F - B _ - , ; : ! ? / . ` ^ ~ ' " (  ('B' is Blank)
 30-3F - ) [ ] { } @ $ * \ & # % + < = >
 40-4F - | 0 1 2 3 4 5 6 7 8 9 a A a A b  (2nd 'a A' with diaeresis)
 50-5F - B c C d D e E f F g G h H i I j
 60-6F - J k K l L m M n N o O o O p P q  (2nd o O with diaeresis)
 70-7F - Q r R s S s t T u U u U v V w W  (2nd 's' sharp, 2nd 'u U' diaeresis)
 80-85 - x X y Y z Z
 86-FE - not used
       - some German characters reasigned to lower sort sequences
 FF    - null & control characters reassigned to sort high

ASCII ISO 8859-1 sort sequence

For comparison the normal (default) character sort sequence is shown below:

 00-1F - control characters (null,soh,stx,etc,...)
 20-2F - B !  " # $ % & ' () * + , - .  /   ('B' is Blank)
 30-3F - 0 1 2 3 4 5 6 7 8 9 : ; < = > ?
 40-4F - @ A B C D E F G H I J K L M N O
 50-5F - P Q R S T U V W X Y Z [ \ ] ^ _
 60-6F - ` a b c d e f g h i j k l m n o
 70-7F - p q r s t u v w x y z { | } ~ D    ('D' is DEL char)
 80-9F - extended control characters
 AF-FF - extended characters for European languages
       - not used in US ASCII ISO 646 (subset of ISO 8859)
       - a few characters German language characters are shown
       - field type 'g' reassigns these to lower values (see above)
 0xC4 - 'a' with diaeresis
 0xD6 - 'O' with diaeresis
 0xDC - 'U' with diaeresis
 0xDF - 'S' sharp
 0xE4 - 'A' with diaeresis
 0xF6 - 'o' with diaeresis
 0xFC - 'u' with diaeresis

Goto: Begin this document , End this document , UVSI Home-Page

C4. Sort Key Options - Unique Keys (key#u1/2) - Drop Duplicate Keys

option 'u' (key#u1/2/4) - drop duplicate records

Option 'u' in the 5th byte of 'key#u' specifies 'unique keys' (data reduction) which outputs only 1 record (or none) for the keys at the specified level '#'.

key#u1

will drop duplicates EXCEPT for the FIRST record of a duplicate set
at the specified key level#. ('u1' is the default if u1/2/4 omitted)

key#u2

will drop duplicates EXCEPT for the LAST record of a duplicate set
at the specified key level#.

key#u4

will drop ALL records of DUPLICATE KEY SETS, retaining only those
records that had no duplicate keys at the specified key level.

Note	In April 2005, options u1 & u2 were changed to u2 & u4 and u1 is now a new feature to retail First record of a dup set

ex#1 - option 'u1' keep only 1st of dup sets

given

the sales detail file listed on page 8 of uvtrain.doc
with salesman# cols 11-12, cust# 1-6, $amount 54-62.

problem

create a list of customers for each salesman, listing only 1 record
of multiple sales to any 1 customer for that salesman.

key1=10(2)

sortkey #1 (salesman# major)

key2u1=0(6)

sortkey #2 (customer# minor)
option 'u1' for unique keys on customer within slsmn

key3=53(9)

optionally specify key3, so that the one record in
each duplicate key set will be the highest $amount.

ex#2 - option 'u4' drop all dup sets

given

1 directory of COBOL source programs (gl101.bat,gl102.bat,etc)
2nd directory of compiled programs (gl101.int,gl102.int,etc)

required

create a list of any missing source programs or compiled programs
(ie - a list of filenames without matches in the other directory)


 #1. ls cobsdir >tmp/cobslist
 #2. ls cobxdir >tmp/cobxlist


 #3. uvsort "fili1=tmp/cobslist,fili2=tmp/cobxlist,rcs=80,typ=LSTt\
             filo1=tmp/misslist,key1u4=0(20)"
                               ============

run option 's' - insert dup count at specified column#

rop=s##

run option 's' may specify a displacement where the dup count
is to be inserted in the last record of a duplicate set.
run option s (rop=s99) may be used with user option 'u'
(this page) as well as with user option 's' (next page)
please see the example shown on the next page --->

Goto: Begin this document , End this document , UVSI Home-Page

C5. Sort Key Options (key#s1/2) - SELECT records for DUPLICATE Keys

option 's' (key#s1/2) - select duplicate records

Option 's' in the 5th byte of 'key#s' specifies selection of records for duplicate key sets at the specified key level#. IE - drop all records that had no duplicate keys at the specified key level#.

key#s1

select only the FIRST record of a duplicate set
(at the specified key# level)

key#s2

select only the LAST record of a duplicate set
(at the specified key# level)

key#s4

select all records of a duplicate set
(at the specified key# level)

key#s4b

select all records of a duplicate set, with option 'b'
to insert a blank line between duplicate sets

run option 's' - insert dup count at specified column#

rop=s##

run option 's' may specify a displacement where the dup count
is to be inserted in the last record of a duplicate set.
displacement is zero relative, length is always 6 bytes

ex#3 - option 's2' - select last rec of dup set

given

the sales detail file listed on page 8 of uvtrain.doc

required

determine how many sales invoices for each customer


 uvsort "fili1=dat1/sales1,rcs=64,typ=RSF,filo1=sales1dups,\
 ==========================================================
 rcs=80,typ=LSTt,key1s2=0(6),key2=13(6),rop=s70"
 ===============================================
                     ^^                 ^^^^^^^

key1s2=0(6)

select the last record of each duplicate set
0(6) defines the customer#

key2=13(6)

also sort on the date, to ensure that the last customer
record in a dup set is the latest date

rop=s70

optional, to insert the duplicate record count into
columns 71-76 of the last record in each duplicate set.

Goto: Begin this document , End this document , UVSI Home-Page

C6. uvsort: key1f1 option to write multiple files

key1f1 option to write multiple files

This is a unique & powerful feature that can be used to write out separate files for each customer#, salesman#, state/province, etc. Whenever a control break occurs in key1, a new file is opened whose filename is taken from the declared filo1=... with the new key1 data appended.

For example, we can sort the dat1/sales3 demo file by salesman# (bytes 10-11) & write separate files to the tmp/ subdir for each salesman# encountered.


 uvsort "fili1=dat1/sales3,typ=RSF,rcs=64,filo1=tmp/sm*,key1f1=10(2)"
 ====================================================================

1st 6 of 20 records in dat1/sales3 demo file

 130140    21 940802 IN111001  HAM001  00020+ 0000001 00000020+
 130140    21 940802 IN111001  SCR012  00021+ 0001001 00021021+
 130140    21 940802 IN111001  CHR001  00022+ 0002001 00044022+
 139923    35 950802 IN111002  TAB013  00023+ 0003001 00069023+
 139923    35 950807 IN111002  TAB013  00024+ 0004001 00096024+
 150825    44 960804 IN1122    HAM001  00025+ 0005001 00125025+

Option f1 on key1f1 will write a separate output file for each salesman#, in the directory specified by filo1 & with any specified prefix:

tmp/sm21	output file for salesman 21 (6 records for demo)
tmp/sm35	output file for salesman 35 (4 records for demo)
tmp/sm44	output file for salesman 44 (6 records for demo)

Rules for option f1 (key1f1)

Option f1 can only be coded on key1 (key1f1=...)
Only 1 output file may be declared, for example: filo1=tmp/sm*
The output filename must include a directory (at least 1 '/') & optionally may include a filename prefix (on which key1 data will be appended).
You may append an '*' on the directory/filenameprefix*, but this is just a visual reminder. Only the option 'f1' on key1f1 is required.
There is no limit on the number of output files that could be created but you would use this option where you knew the outputs are manageable.
You might use this option when you need to send out separate files to each salesman, or each customer, etc.

Note	Also see option 'rop=f2' which is the reverse of 'key1f1'. inputs all files from a directory (see page 'F4').

Note	Also see demo parameter files uvsortsplit1/2 on pages 'J1' & 'J2'

Goto: Begin this document , End this document , UVSI Home-Page

C7. Year 2000 field type (y) - Illustrated Demonstration


    key1=30(6y)     - field type 'y' specifies Year2000 windowed sort
    =========^=
                    - 1st 2 digits assumed to be the year (no century)
                    - the default window is '50', meaning:
                      years 50-99 are considered 1950-1999,
                      years 00-49 are considered 2000-2049.

Year2000 sort demo

Note: You can run the following Y2K sort demo using the supplied test file.


 uvsort "fili1=tf/testY2K,rcs=80,typ=LSTt,filo1=tmp1,key1=30(6y)
 ===============================================================

INPUT file (uvadm/tf/testY2K)

          1         2         3         4         5
 12345678901234567890123456789012345678901234567890
                             purchase        expiry  <-- sort field 31-36
 cust#   product description   yymmdd policy yymmdd

 00000 test Year2000 date sort 000101 0-year 000101
 11111 test Year2000 date sort 100101 1-year 110101
 22222 test Year2000 date sort 200101 2-year 220101
 33333 test Year2000 date sort 301231 3-year 331231
 44444 test Year2000 date sort 400101 4-year 440101
 55555 test Year2000 date sort 500101 5-year 550101
 66666 test Year2000 date sort 601231 6-year 661231
 77777 test Year2000 date sort 700101 7-year 770101
 88888 test Year2000 date sort 800101 8-year 880101
 99999 test Year2000 date sort 901231 9-year 991231

Y2Ksort OUTPUT results

 55555 test Year2000 date sort 500101 5-year 550101  <-- Y2K windowed sort
 66666 test Year2000 date sort 601231 6-year 661231      (default window 50)
 77777 test Year2000 date sort 700101 7-year 770101
 88888 test Year2000 date sort 800101 8-year 880101
 99999 test Year2000 date sort 901231 9-year 991231
 00000 test Year2000 date sort 000101 0-year 000101
 11111 test Year2000 date sort 100101 1-year 110101
 22222 test Year2000 date sort 200101 2-year 220101
 33333 test Year2000 date sort 301231 3-year 331231
 44444 test Year2000 date sort 400101 4-year 440101

---> Rerun with run options to modify window value & with sliding option


 uvsort "fili1=tf/testY2K,rcs=80,typ=LSTt,filo1=tmp1,key1=30(6y),rop=y70z1"
 ==========================================================================
                                                              ^  ^^^^^^^^^
 ---> Try 'key1=30(6yd)' for descending sort on type y fields.

Goto: Begin this document , End this document , UVSI Home-Page

C8. Year 2000 sort demo - Options to change window & specify sliding

Options to modify window & specify sliding

The default window is '50', you can change with run options 'y__' & 'z1'. NOTE - run options are specified on the 'rop' command.

rop=y35z1

set Year2000 window for field type 'y' on sort keys
(& also for any sel1y/del1y selects/deletes)
option 'y__' assumes 1st 2 bytes of field is year (no century)
'y35' means that years 35-99 = 1935-1999,
00-34 = 2000-2034

'y35z1' option 'z1' specifies sliding windows
in 1998, the window value would be '33'
meaning years 33-99 = 1933-1999,
00-32 = 2000-2032

Y2K demo sort with run options y35z1

<--- Please see the input file listed on the previous page

---> Rerun with run options to modify window value & with sliding option


 uvsort "fili1=tf/testY2K,rcs=80,typ=LSTt,filo1=tmp1,key1=44(6y),rop=y35z1"
 ==========================================================================
                                                              ^  ^^^^^^^^^
 cat tmp1       - display sort output file
 ========
                  ** output results **

 33333 test Year2000 date sort 301231 3-year 331231
 44444 test Year2000 date sort 400101 4-year 440101
 55555 test Year2000 date sort 500101 5-year 550101
 66666 test Year2000 date sort 601231 6-year 661231
 77777 test Year2000 date sort 700101 7-year 770101
 88888 test Year2000 date sort 800101 8-year 880101
 99999 test Year2000 date sort 901231 9-year 991231
 00000 test Year2000 date sort 000101 0-year 000101
 11111 test Year2000 date sort 100101 1-year 110101
 22222 test Year2000 date sort 200101 2-year 220101

Note

we have changed the sort field to the 2nd date in cols 45-50
to better demonstrate the sliding window option.
with option 'z1' the window option y35 is adjusted to 33
& the record with date '331231' in cols 45-50 sorts low.
assuming it is now 1998 (35 + 98 - 100 = 33)
so 33-99 are 1933-1999 & 00-32 are 2000-2032

Now	Rerun the sort without option 'z1' & note that the record with date '331231' in cols 45-50 will sort high (vs low) since the window is then 35 (1935-1999 & 2000-2034)

Goto: Begin this document , End this document , UVSI Home-Page

D1. uvsort: FILE DEFINITION PARAMETERS

      fili1=, typ=, rcs=, [isk#=,]  filo1=, [typ=,] [rcs=,] [isk#=]

fili1

specifies input filename (fili1=pathname)
up to 20 input files
also see run option rop=f2 to input multi-files from directory

filo1

specifies output filename (filo1=pathname)
up to 60 output files
also see option 'f1' on key1f1 to output multi-files based on
control breaks on key1.

Note	following keywords apply to the preceding file (fili1 or filo1) keywords specified for fili1 also default for filo1 unless respecified

typ	file type codes - 3 upper case alpha + misc options a-z RSF/RSV/ISF/IDX/LST - see details ahead file options - misc options may be appended to the 3 alpha codes - see details on next page

rcs	record size for the preceding file (fili1/filo1) may omit declaring rcs for output file (will default same as input) input ISAM files know their rcs, but you have to specify something to get past the error-check

isk?	declare isk1-isk9 for the preceding file (fili1/filo1) required only for filo1 to build/load a new Indexed file

        isk1=0(8)         - declare 1st key (bytes 1-8)
        isk2=59(15)       - declare 2nd key (bytes 61-75)

        isk1p1=0(8)       - declare key#1 part#1
        isk1p2=20(6)      - declare key#1 part#2
        isk1p3=30(4)      - declare key#1 part#3

fili1/filo1 example


 uvsort "fili1=xx,typ=RSF,rcs=80,filo1=yy,typ=ISF,isk1=0(8),key1=0(8)"
 =====================================================================

Note	the 'isk1=0(8)' could be omitted in this case, since it would be defaulted from 'key1=0(8)' when output typ=ISF. you would have to code any additional ISAM keys (isk2,isk3,etc).

Goto: Begin this document , End this document , UVSI Home-Page

D2. Indexed file definition - additional options

alternate format for ISAM keys

 uvsort "fili1=x,typ=RSF,rcs=80,filo1=y,typ=ISF,isk1=0(8),isk2=59(5),key1=0(8)"
 uvsort "fili1=x,typ=RSF,rcs=80,filo1=y,typ=ISF,isks=(0,8,59,5),key1=0(8)"

ISAM option for DUPS/NODUPS

 uvsort "fili1=x,typ=RSF,rcs=80,filo1=y,typ=ISF,isks=(0,8,40,20),key1=0(8)"
 uvsort "fili1=x,typ=RSF,rcs=80,filo1=y,typ=ISF,isks=(0,8n,40,20d),key1=0(8)"

the above illustrates the defaults - nodups on the 1st key,
and dups allowed on secondary keys.
You only need to code a 'd' on the primary key for dups,
or an 'n' on a secondary key for nodups.

 uvsort "fili1=x,typ=RSF,rcs=80,filo1=y,typ=ISF,isks=(0,8,40,20),key1=0(8)"

the above specifies nodups allowed on key #2
(key#1 defaults to nodups & key#2 to dups allowed)

 uvsort "fili1=x,typ=RSF,rcs=80,\
       filo1=y,typ=ISF,isk1=0(8),isk2=40(20n),key1=0(8)"

could also specify the 'd' or the 'n' on the original isk format

do not need to specify ISAM keys for input files

 uvsort "fili1=x,rcs=80,typ=ISF,isks=(0,8,40,20),filo1=y,typ=RST,key1=0(8)"
 uvsort "fili1=x,rcs=80,typ=ISF,filo1=y,typ=RST,key1=0(8)"

the above are the same, you never need to specify ISAM keys
for input files (only for output file creation).

option 'k' for key of reference (read sequence)

 uvsort "fili1=x,rcs=80,typ=ISFk2,filo1=y,typ=RST,key1=0(8)"

the above option 'k2' of file 'typ=ISFk2' specifies the key
of reference as key #2, to cause record retrieval in sequence
by the 2nd key.

Goto: Begin this document , End this document , UVSI Home-Page

E1. TYP CODES for fili1/filo1 - Sequential files

File type codes are 3 upper case alpha + misc options (a-z), for example:


       uvsort "fili1=test1,typ=RSF,rcs=80,filo1=test2,typ=LST,key1=0(8)"
       ====================*******====================*******===========

RSF	Record Sequential Fixed (record size)

RST	Record Sequential Terminated (fixed rcsz) Terminated by LF in last byte of stated record size (whether input or output)

RSR

Record Sequential Relative (Micro Focus COBOL compatible)
actual record size is 1 greater than declared
extra byte x'0A' if valid, x'00' if deleted
on output - the x'0A' is always written
on input - 1 extra byte is read but not tested by default
- option 'r1' required to drop record if last byte x'00'

RUF	UNIBOL file type if input: will bypass the 8 byte header if output: will create the 8 byte header as follows: bytes 00-05 - x'07ff00430000' bytes 06-07 - record size in binary

LST

Line Sequential Terminated
by LF *after* last byte of rcs
LST input files are usually variable rcsz
LST output files would have the LF after the stated rcs
the 't' option would truncate any trailing traling blanks
to create variable output rcsz's if the data varies.
use LST only when required (for variable length text records)
LST is 4 times slower than RSF or RST because it reads or writes
1 character at a time (vs 1 record at a time).
record size determined by position of the LineFeed, not by rcs=xxx
you should specify record size generously to avoid splitting records

ISF	Indexed Seqential Fixed record length D-ISAM/C-ISAM/IDXFORMAT1 see more explanation on the next page --->

RSV	Record Sequential Variable files compatible with Micro Focus COBOL IDXFORMAT3 files, see more explanation on the next page --->

IDX	Indexed Sequential Variable files compatible with Micro Focus COBOL IDXFORMAT3/8 files, see more explanation 2 pages ahead --->

RDWz4

'Record Descriptor Word' variable length files (RDW defaults to RDWz4)
4 byte record prefix with 2 byte binary record size & 2 unused nulls
record size in prefix is the total record size including the prefix
2 byte record size is 'BIG-END' binary (like mainframe or SUN,AIX,DEC)
'little-end' machines (Intel,AMD) need to switch ends to perform math

RDWz2

option 'z2' for 2 byte prefixes (vs 4 default above)

RDWz8

option 'z8' same as RDWz4, but bypasses any BDW prefixes

Goto: Begin this document , End this document , UVSI Home-Page

E2. TYP CODES for fili1/filo1 - Indexed files

typ=ISF Indexed Sequential Fixed files

typ=ISF

Indexed Sequential Fixed records (D-ISAM for Micro Focus COBOL)
(MBP-ISAM & SCO-ISAM now obsolete)


 uvsort "fili1=dat1/sales2,typ=LST,rcs=64,filo1=xx,typ=ISF,isk1=0(6),key1=0(6)"
 ==============================***=====================***=====================

The example above will convert a text input file to an Indexed Sequential Fixed record size file with key in 1st 6 bytes. This is an easy way to create indexed files - just use the editor to create the keys & data fields, with no need to worry about record sizes or key sequences, then use uvsort to create the Indexed file of fixed data records with .idx partition.

export ISDATEXT=.dat

This applies only to D-ISAM typ=ISF Indexed files. You may export ISDATEXT=".dat" if you wish to use '.dat' extensions on D-ISAM files read or written by uvcp (+ uvsort + uvcopy). The default is no extension on data partition, which is compatible with Micro Focus COBOL. Add to /etc/profile if your site always needs '.dat'. Add the export to /etc/profile if your site always needs '.dat'.

typ=RSV MF COBOL file type IDXFORMAT3

typ=RSV

Record Sequential Variable files compatible with Micro Focus
COBOL IDXFORMAT3 files (data partition only not indexed).
As of Nov2002 uvcp & uvsort support OUTPUT as well as INPUT RSV files
For input record size declared does not matter, make sure it is
big enough, for example:


 uvsort "fili1=indir/paymast,typ=RSV,rcs=4000,\
 ==============================================
         filo1=outdir/paymast,typ=ISF,rcs=200,isk1=0(6),key1=0(6)"
         =========================================================

The above would convert a Record Sequential Variable (Micro Focus IDXFORMAT3) data file to an Indexed Sequential Fixed record length file. The 'ISF' file is compatible with MF COBOL fixed length Indexed files & C-ISAM files.

Please see pages 'N1' thru 'N3' for more details & examples --->

Goto: Begin this document , End this document , UVSI Home-Page

E3. file TYP codes for fili1/filo1 (continued)

typ=IDX Indexed Sequential Variable files

'typ=IDX' declares Indexed Sequential Variable length record files compatible with Micro Focus COBOL IDXFORMAT1,3,8 files. See the 'File Handling' documentation for Micro Focus Object or Server Express COBOL.

The Micro Focus COBOL 'EXTFH' file handler is compiled into an alternate version of uvsort called 'uxsort' using the 'cobccuv' script as explained in install.doc.

'uxsort' supports the same file types as uvsort with typ=IDX in addition. Two versions of the program are required since only Micro Focus COBOL users (server express 2.2+) could compile 'uxsort' whereas any user can compile 'uvsort' which uses D-ISAM to support Indexed Sequential FIXED record length files.

Note that 'typ=RSV' (Record Sequential Variable with no indexed keys) files are supported by both uvsort & uxsort. 'uxsort' must be used when either input or output or both files are typ=IDX.

example - sort text file & load Indexed Variable


 uxsort "fili1=dat1/idxfL,typ=LST,rcs=256\
 =========================================
        ,filo1=tmp/idxf3v,typ=IDXf3v60y4,rcs=252,isk1=0(6d),key1=0(6)"
        ==============================================================

This would read text file dat1/idxfL & build an Indexed Sequential Variable file tmp/idxf3v whose key is the 1st 6 bytes of each record. The output records could vary from 60 to 252 bytes depending on the last nonblank character of input record. 'rcs=252' sets the maximum size & option 'v60' sets the minimum. The records will actually vary from 64 to 256 allowing for the record header (see pages 'N1' to 'N3' which explain the record format).

You can run this example using a test file provided (uvadm/dat1/idxfL).

Goto: Begin this document , End this document , UVSI Home-Page

E4. uvsort: file TYP OPTIONS

File options may be appended to the 3 byte file typ code (lower case letters a-z with possible numeric suffixes).

file 'typ' option examples


   uvsort "fili1=text1,typ=LSTb1c1,rcs=80,filo1=text2,typ=LSTtd3,key1=0(20)"
   =========================================================================
                              ^^^^                           ^^^
 a   - append to the file's existing contents
       (vs default of erasing existing contents)

b1	drop blank records

c1	convert unprintable characters (< x'20' or > x'7E') to blanks
c2	convert unprintable characters (< x'20' or > x'7E') to periods OK for output file typ=LST since LF inserted after record converted

d1	McIntosh file, record terminator is CR x'0D' for typ=LST & typ=RST files (a single CR vs LF for unix vs CRLF for DOS option d1) CR is appended for output records & removed from input records

d2	Unix files (the default), inserts LF x'0A' on output, removes on input for typ=LST & typ=RST files

d3	MSDOS file, inserts CR (and LF) when writing typ=LST/RST files removes when reading into the input area

d4	insert null as terminator on output typ=LST files
d8	insert tilde as terminator on output typ=LST files

e2	insert EBCDIC Line-Feed x'25' in last byte of record (typ=RSTe2)
e1	insert Carriage-Return x'0D' in 2nd last byte (use with e2+e1=e3)
e3	insert CR+LF in last 2 bytes of record (applies only to 'typ=RSTe3'

f#	format for typ=IDX files
f1	fixed length Indexed records, compatible with C-ISAM, same as typ=RSF
f3	variable length Indexed files (file size < 2 gig)
f8	variable length Indexed files (allowing file size > 2 gig) IDXf3 & IDXf8 could be fixed records in variable format

g_	for typ=IDX/RSV MF COBOL ISAM variable length Indexed/Sequential files
g4	get active data records only (default)
g2	get deleted data records only
g6	get both active & deleted data records option 'g' in this uvcp is similar to record display option 'v' in the uvhd display utility (v1=filehdr,v2=deleted,v4=data,v8=all)

Goto: Begin this document , End this document , UVSI Home-Page

E5. file TYP OPTIONS (continued)

wait on locked file, retries every 10 seconds until lock released
(when other process closes the file)
retry time is gradually increased to every 5 minutes after 30 mins
displays waiting message showing total wait time
you would use option 'h1' with EXCLUSIVE lock ('l0' or not coded)
typ=ISFh1l1 would not make sense, since it will never get a lock
See file lock test results in separate section uvlocks.htm

j1	translate filename to lower case (before opening)
j2	translate filename to UPPER case (before opening)
j4	translate any extension to the opposite case

Option 'j' useful when you are processing all files in directory (with uvcopyx) And you want the output files in a different case than input files.

k#	key of reference for an ISAM input file typ=ISFk2 would cause reading in sequence by key #2

l_	file locking options for DISAM indexed files & for UNIX Sequential files as of January 1999 see more discussion & test/demo's in uvlocks.doc (follows uvcopy5)

l0	EXCLUSIVE (default) RDlock+WRlock other jobs cannot access, unless they specify READONLY for ISAM or NO-locking for non-indexed sequential files. always use for output files (building new ISAM files).

l1	READONLY (inhibits all locking) works regardless of other jobs lock modes intended for read only, attempted updates by the READONLY job will be rejected (error code 9 BAD FILE# ?)

l2	AUTOLOCK (applies only to ISAM files) only 1 record locked at 1 time in this file records are locked by the 'red' instruction & unlocked by whatever instruction follows (might be 'upd' to update the record)

l4	MANULOCK (applies only to ISAM files) assumes the programmer will handle locking manually using the 'lck' uvcopy instruction ('islock' DISAM function) Use this option if you want to open 2 paths for the same file (fili1 & filr1 for the same filename with typ=ISFl4)

Note	you probably wont use ISAM 'RECORD locking' options l2 & l4 it is easier to use the 'FILE locking' options (l0,l1 for ISAM files; l0,l1,l8 for Sequential files)

l8	NO locking, used only for Non-Indexed Sequential files For non-indexed files, only lock options l0, l1, l8 apply.

Goto: Begin this document , End this document , UVSI Home-Page

E6. file TYP OPTIONS (continued)

m#	file progress messages every m# records
m100000	default is every 100,000 records

n1	inhibit errmsg if data exceeds record length specified for typ=LST ie, if LineFeed x'0A' not found before specified rcs=... causes over-length records to be split into 2 records. use this when you don't want errmsgs & it is OK to split to 2 recs

n4	typ=RDW: specifies 4 byte record-size prefix Numeric digits vs default binary 2 bytes + 2 bytes null also see typ=RDW & options z2/z4/z8

r1	for typ=RSRr1 Relative files (MicroFocus COBOL compatible) drop record if extra byte = x'00'

s4	detailed ISAM file key statistics (in addition to record counts) typ=ISFs4 will display key start byte & length for each key

t	truncate Line Variable records will scan back from the end of each record & insert the LF after the last non-blank data character

file typ option 'v' for typ=ISF

v#	max recsize for typ=ISF C-ISAM variable length Indexed files min recsize specified by rcs=___

note	above variable length C-ISAM indexed files NOT compatible with MF COBOL UVSI recommends following IDXf3 for variable length Indexed files Micro Focus COBOL compatible

w2	Warning prompt if output file exists may overwrite, change name, or cancel

Goto: Begin this document , End this document , UVSI Home-Page

E7. file TYP OPTIONS (continued)

file typ options for typ=IDX & typ=RSV

Note	you must use 'uxsort' (vs uvsort) to process IDXFORMAT3 & IDXFORMAT8 options v,x,y apply mostly to uxsort but option v might be used with uvsort for file typ=RSV

IDXf1	fixed length Indexed records, compatible with C-ISAM, same as typ=RSF
IDXf3	variable length Indexed files (file size < 2 gig)
IDXf8	variable length Indexed files (allowing file size > 2 gig) IDXF3 & IDXf8 could be fixed records in variable format

v9999

minimum record size option for typ=IDX & typ=RSV files
For example: 'typ=IDXv400,rcs=5000' v400 declares minimum size 400
while rcs=5000 declares the maximum size 5000.
if option v is not specified, all records will be fixed size rcs=____
and the file type will be fixed (vs variable)
if option v is specified with no value, min size is set = max size
and the file type will be variable
also see type 'y4' below to determine record size by scanning back
from max size to the last nonblank/nonnull beyond min size

x	file organization for typ=IDX files
x1	Sequential, no Index created
x2	Indexed, assumed if 'isk' (Indexed Seqntl Key) is declared

y	Recording Mode
y1	Fixed Length records, all records forced to size spcfd by rcs=____ default for if option 'v' NOT specified (w or w/o min recsize)
y2	Variable Length records default if option 'v' IS specified (w or w/o min recsize value) min recsize specified by option 'v', max recsize by rcs=____
y4	determine the record size for each record by scanning back from max size to last non-blank or non-null beyond the min size
y8	fixed length records in variable length format in case you want to copy a variable length file & output all records as the max size specified on rcs=___ (but in variable length format

RDW variable length files with 4 byte record prefixes (default)
(typ=RDW defaults to typ=RDWz4)
4 byte record prefix with 2 byte binary record size & 2 unused nulls
record size in prefix is the total record size including the prefix
2 byte record size is 'BIG-END' binary (like mainframe or SUN,AIX,DEC)
'little-end' machines (Intel,AMD) need to switch ends to perform math

z2	option 'z2' for 2 byte prefixes (vs 4 byte default above)

z8	option 'z8' same as RDWz4, but bypasses any BDW prefixes

n4	note option n4 = 4 byte recsize Numeric digits (vs Binary)

Goto: Begin this document , End this document , UVSI Home-Page

E8. uvsort file TYP OPTIONS (continued)

output record size defaults for variable length files

The output record size for IDXf3/IDXf8 variable length records defaults to the corresponding input record size. The record size is passed thru the sort by appending it on the end of the record & using it on the output phase.

This works well if both input & output are variable length (typ=IDXf3v/IDXf8v). If input was fixed (typ=RSF/ISF/IDXf1) & output was variable (typ=IDXf3/IDXf8v), then output records would be in the variable length format, but all same size.

See pages 'N1' thru 'O3' for examples of RSV/IDX & options f,v,x,y --->

Goto: Begin this document , End this document , UVSI Home-Page

F1. uvsort: 'rop' - RUN OPTIONS

'rop' is used to declare various options for the entire run. Do not confuse with file options which apply only to a file (fili1/filo1) & which are appended to the file typ code (example: typ=LSTtd). Do not confuse with instruction options which are appended to instruction opcodes (see the uvcp documentation).

rop=m256 <-- example, option 'm' request 256 meg MAXMEMORY for the sort

b99999

input file record bypass count

i99999	input file record stop count
o99999	put file record stop count

Note	options 'i' & 'o' are duplicated here out of alpha sequence since they are related

 d=sortworkdir
        - declare the sort work directory (default is 'tmp' in current dir)
        - must enclose in parens, for example:  rop=(d=sortwork)
        - also see the 'SORTDIR' environmental variable on page 'F5'.

Note	run option 'd' was removed in November 2004 use the 'SORTDIR' environmental variable

e1	enable interupt (usually '^C' or the DELETE key) will report current I/O record counts & allow close or continue disabled by default since 'uvsort's within scripts might be zombied enable when you run uvcp from command line by appending ',rop=e1'

f2	input all files (unlimited number) from the fili1 directory or all files matching a specified prefix &/or suffix (if any). see the demo on page 'F4'.

h2	halt at end of uvsort, until operator presses enter

i99999

input file record stop count

Goto: Begin this document , End this document , UVSI Home-Page

F2. uvsort: rop options - continued

m64	sort max memory in megabytes (default m64 = 64 meg) max is m2000 (2 Gig), but would not be efficiently used unless the file is greater than 6.4 gigabytes

WorkFiles * MaxMemory = max filesize for efficient sort
(see workfiles specified by option 'w' below)
100 workfiles * 64 meg max memory = 6.4 gig (max filesize efficient)
not much to gain by increasing maxmemory, unless file > 6.4 gig
workfiles * maxmemory >= filesize means sort will be mostly
input time + output time (no intermediate sort passes)

o99999

input file record stop count

Note	both input & output stop counts are provided since the records selected could be very different (due to sorting).

r99999

declare maximum record size to be 99999 (or whatever)
vs the default max 32768
this also changes the max size allowed for the 'w' work area

s999	column# (zero relative displacement) for record count of duplicate record sets (see 'key' specifications).

t1	(default) puts any SUM totals into FIRST record of the dup key set
t2	puts any SUM totals into the LAST record of the dup key set see the 'SUM' feature discussed on page 'H1'

v0	inhibit the sort progress messages (% of merge pass completion) Actually you will seldom see these messages because it takes a very large sort before they are displayed.

w100	workfiles maximum, default is 100 & this is max recommended.

y70	set Year2000 window for field type 'y' on sort keys & also for any sel1y/del1y selects/deletes option 'y__' assumes 1st 2 bytes of field is the year (no century) 'y70' means that years 70-99 = 1970-1999, 00-69 = 2000-2069

z	option 'z1' specifies sliding windows, for example in 1998

y70z1

the window value would be '68' meaning 68-99 = 1968-1999,
00-67 = 2000-2067

See uvsort examples using various Run OPtions on the next page --->

Goto: Begin this document , End this document , UVSI Home-Page

F3. uvsort: rop options - continued

uvsort options for performance

Most sort times are determined by input & output times. Intermediate sort merges are required only for files larger than 6.4 gigabytes, assuming the default sort memory of 64 megs & 100 sort work files. As the input file is read, each 64 meg portion is sorted & written to up to 100 work files in the tmp subdir. These are then merged during the output phase.

See options 'm' & 'w' (above) for sort memory & number of work files (default 64 megs & 100 work files max). There is no advantage increasing the memory unless the file is > 6.4 gig.

rop options - complete examples


 #1. uvsort "fili1=bigfile,typ=RSF,rcs=351,filo1=bigsorted,key1=71(20),rop=m256"
     ==================================================================********=

rop=m256 - assign 256 megs for internal sort memory (vs 64 megs default)


 #2. uvsort "fili1=warmas1a,typ=RSF,rcs=64,filo1=wm2,key1=44(6y),rop=y70z1"
     ============================================================*********=

     rop=y70z1 - set Year2000 sliding window '70' for sort field type 'y'
                 & also for any sel1y/del1y selects/deletes


 #3. uvsort "fili1=xx,typ=RSF,rcs=256,filo1=yy,key1=0(8),rop=b8000,i4000"
     ====================================================***************

     rop=b8000 - bypass the 1st 8000 records in the input file
         i4000 - stop input after 4000 records (8001-12000)

Goto: Begin this document , End this document , UVSI Home-Page

F4. uvsort: rop options - continued

rop=f2 input all files from directory

Run option f2 allow you to input all files from a directory, or all files matching a specified prefix &/or suffix.

The fili1 pathname must include at least 1 '/' & exactly 1 '*' (a directory & optionally a prefix &/or suffix)

This is the reverse of option 'f1' on key1f1 (see page 'C6'), which writes multiple separate files to a directory based on key1 control breaks.

For example, we could recombine the multiple salesman files that were created on the key1f1 demo on page 'C6'.

ex#1	'fili1=tmp/*' will read all files from the tmp subdir.

ex#2	'fili1=tmp/*.txt' reads all files matching suffix '.txt'

ex#3	'fili1=tmp/sm*.txt' reads files matching prefix 'sm' & suffix '.txt'

complete example


 uvsort "fili1=tmp/sm*,typ=RSF,rcs=64,filo1=tmp/allsm,key1=10(2),rop=f2"
 =======================================================================

Goto: Begin this document , End this document , UVSI Home-Page

F5. environmental variable for sortwork directory


 SORTDIR=/u2/sortwork; export SORTDIR          (example)
 ====================================

'SORTDIR' is an optional environmental variable that may be used to assign the sortwork directory. You could code this in your .profile.

The default sortwork directory is 'tmp' within the current directory (wherever uvsort is executed - not the system '/tmp' subdir). The sortwork directory will be created if it does not already exist.

Note that sort work (merge) files are not created if the data file size is less than the memory assigned to uvsort via run option 'm' (see page 'F2'). The default is 'rop=m64' which assigns 64 megabytes to internal sort work You could increase up to 2 gigabytes, but you wont see much improvement unless your file is larger than about 6 gigabytes, which is the point when uvsort would need to do multi-pass merges.

uvsort creates sort work files using the process ID as part of the filename. The process ID is converted to hexadecimal, for example:

tmp/SORT0001E6.001 tmp/SORT0001E6.002 ---- etc ---

If uvsort fails for whatever reason (hardware, software, or no space left in filesystem where tmp subdir located), then the SORT work files will be left in the tmp subdir & you might want to remove them before rerunning the sort. uvsort automatically removes SORT work files only if more than 2 days old (in case multiple uvsorts are running at the same time).


 find tmp/SORT* -mtime +2 -exec rm -f {} \;
 ==========================================

interupt processing

You may press the 'interupt' key at any time. Interrupt causes a display of current input/output record counts & prompts you for a response. Enter null to continue or 'q' to quit. If uvsort is on the input phase, quit will end the input phase & goto the sort & output phases. If uvsort was already on the output phase, it will simply close files & end program.

Interupts must be enabled via option 'e1', for example:


 uvsort "fili1=dat1/sales2,rcs=80,typ=LSTt,filo1=tmp/sales2,key1=30(6),rop=e"
 ======================================================================*****

Goto: Begin this document , End this document , UVSI Home-Page

G1. uvsort: sel/del - SELECT/DELETE RECORDS

      sel1=start(length):dataconstant

      sel1=10(2):21          - select records with '21' in cols 11-12 (slsmn)
      sel2=30(3):HAM           AND 'HAM' in cols 31-33 (product code)
      sel3=38(6)>000025        AND >'000025' in cols 49-54 (quantity)
                               ('AND' condition if 4th byte increases)


      sel1=10(2):21,sel2=30(3):HAM,sel3=38(6)>000025  <-- code contigously
      ==============================================      with ',' separators

      sel1=10(2):21          - select records with '21' in cols 11-12 (slsmn)
      sel1=10(2):44            OR with '44' in cols 11-12
      sel1=10(2):85            OR with '85' in cols 11-12
                               (OR condition if 4th byte reverts to '1')

      del1=90(2)>55          - delete if cols 91-92 > '55'
      del2=99(1)!0xff          AND if column 100 not = x'ff'

'sel' & 'del' are mutually exclusive within Inputs & within Outputs. You can have all dels for Input files & all sels for Output files, but do not mix sels/dels within Inputs or Outputs. You can have a maximum of 500 sels/dels.

The AND/OR condition is indicated by the 4th byte, sel1, sel2, etc to sel99 max. The 'AND' condition is indicated as long as 4th byte increases (sel1,sel2,etc). The 'OR' condition is indicated whenever 4th byte reverts to '1'.

sel/del complete example

Given: the uvadm/dat1/sales2 file illustrated below:

          1         2         3         4         5         6
 1234567890123456789012345678901234567890123456789012345678901234
 cust#   slmn YYMMDD invoice  product    qty   price    amount
 ================================================================
 130140    21 940802 IN111001  HAM001  000020 0000001 000000020
 139923    35 950802 IN111002  TAB013  000023 0003001 000069023
 150825    44 960804 IN1122    HAM001  000025 0005001 000125025
 223240    65 980816 CR955     HAM001  000028 0008001 000224028
 400002    85 010812 CR245     HAX129  00001v 0006001 00009601v

Required: sort by product within salesman & select following records:


 #1. Salesman 21,44,& 65 (columns 10-11)
 #2. Product hammers ('HAM' in cols 31-33)
 #3. sales QTY 10 or more (columns 39-44)

 uvsort "fili1=dat1/sales2,rcs=64,typ=RSF,filo1=tmp/s2,key1=10(2),key2=30(6)\
 ,sel1=10(2):21,sel2=30(3):HAM,sel3=38(6)>000009\
 ,sel1=10(2):44,sel2=30(3):HAM,sel3=38(6)>000009\
 ,sel1=10(2):65,sel2=30(3):HAM,sel3=38(6)>000009"

Goto: Begin this document , End this document , UVSI Home-Page

G2. uvsort: sel/del (continued) - option 'o'

option 'o' & 'o#' on sel/del

Option 'o' on sel/del functions cause the record selection to be performed at output time (vs input time). Without any number attached these selects/ deletes will be applied to all output files.

Option 'o#' may specify the output file# to which the selects/deletes are to be applied. For example, to select different salesmen to separate files:


 uvsort "fili1=dat1/sales3,rcs=64,typ=RSF,key1=10(2)\
 ===================================================
 ,filo1=tmp/sm21,filo2=tmp/sm35,filo3=tmp/sm44\
 =============================================
 ,sel1o1=10(2):21,sel1o2=10(2):35,sel1o3=10(2):44"
 =================================================

This is somewhat similar to option 'f1' on key1f1 already demonstrated on page 'C6', but this can be much more selective. For example, we can add additional conditions to select by product & quantity as well as salesman#:


 ,sel1o1=10(2):21,sel2o1=30(3):HAM,sel3o1=38(6)>000009\
 =====================================================
 ,sel1o2=10(2):35,sel2o2=30(3):HAM,sel3o2=38(6)>000009\
 =====================================================
 ,sel1o3=10(2):44,sel2=30(3):HAM,sel3=38(6)>000009\
 =================================================

Rules for option 'o' on sel/del

Output files (filo1=...,filo2=...,etc) must have been declared to match the option 'o#'s specified (o1,o2,...,o60).
The option 'o#'s must appear in sequence, but in fact any out of sequence o#s will be included with the current highest o# yet encountered.
This means if you inadvertently omit the o# option as shown above for the sel2 & sel3 on the 3rd line, the conditions are still properly applied to filo3 as declared on the sel1o3.

Goto: Begin this document , End this document , UVSI Home-Page

G3. uvsort: sel/del - SELECT/DELETE RECORDS

option 'u1' - allow coding ' 's as '_'s

Option 'u1' allows you to code blanks as underscores, which is convenient because blanks are not allowed in the uvcp command string. The alternative would be to code blanks in hexadecimal (' ' = 0x20). Here are some examples:

option 'u2' - allow coding ','s as '@'s

Option 'u2' allows you to code commas as at-signs, which is convenient because commas are used to separate commands in the command string and therefore can not be coded as constants.

examples - coding ' ' as '_' & ',' ar '@'


 sel1u1=0(1):_,sel1u2=0(1):@,sel...
 ==================================
 - select record if 1st byte blank ' ' OR 1st byte comma ','


 sel1u1=0(1):_,sel2u2=1(1):@,sel...
 ==================================
 - select record if 1st byte blank ' ' AND 2nd byte comma ','


 sel1u3=0(2):_@,sel...
 ==================================
 - select record if 1st byte blank ' ' AND 2nd byte comma ','
 - coding a 2 byte constant (vs separate 1 byte constants)
 - note option u3 = (u1 + u2)

option 'y' for Year2000 windowed date field selections

     sel1y=20(6)>980630   - select records with dates in cols 21-26
                            from 19980701 up to 20491231

Option 'y' assumes 1st 2 bytes of specified field is the 2 digit year regardless of field length (yymmdd, yymm, yy).

The default window is '50', may be changed by run option 'y' to specify the window value,& option 'z1' for sliding windows.

Note	run options are specified on the 'rop' command (ex: rop=y50z1)

     rop=y50  -------- means 50-99 = 1950-1999,& 00-49 = 2000-2049

     rop=y50z1 in 1998 means 48-99 = 1948-1999,& 00-47 = 2000-2047

     sel1y=20(2)>98,rop=y70z1  - select records with years in cols 21-22
                                 from 1998 to 2067 (rop=y70z1) if now 1998

Goto: Begin this document , End this document , UVSI Home-Page

H1. uvsort: SUM Function

 sum1=38(6z),sum2=53(9z)        <-- sum example

The 'sum' function accumulates fields from multiple records into 1 record for each duplicate key set. Consider the following complete uvsort example:


 uvsort="fili1=dat1/sales2,rcs=64,typ=RSF,filo1=tmp/sales2,key1=0(6)\
 ===================================================================
 ,sum1=38(6z),sum2=53(9z)"
 =========================

input file

          1         2         3         4         5         6
 1234567890123456789012345678901234567890123456789012345678901234
 cust#   slmn YYMMDD invoice  product    qty   price    amount
 ================================================================
 130140    21 940802 IN111001  HAM001  000020 0000001 000000020
 130140    21 940802 IN111001  SCR012  000021 0001001 000021021
 130140    21 940802 IN111001  CHR001  000022 0002001 000044022
 139923    35 950802 IN111002  TAB013  000023 0003001 000069023
 139923    35 950807 IN111002  TAB013  000024 0004001 000096024
 150825    44 960804 IN1122    HAM001  000025 0005001 000125025
 150825    44 960804 IN1122    HAX129  000026 0006001 000156026

                     ** output file **
                                       (sum1)           (sum2)
 cust#   slmn YYMMDD invoice  product    qty   price    amount
 ================================================================
 130140    21 940802 IN111001  HAM001  000063 0000001 000065063
 139923    35 950802 IN111002  TAB013  000047 0003001 000165047
 150825    44 960804 IN1122    HAM001  000051 0005001 000281051

Note that qty & amount have been accumulated into 1 record for each customer#. By default the SUM totals go into the FIRST record of the dup key record set. If you wish to use the LAST record of the dup key set, specify run option 't2'. For example:


 uvsort="fili1=dat1/sales2,rcs=64,typ=RSF,filo1=tmp/sales2,key1=0(6)\
 ===================================================================
 ,sum1=38(6z),sum2=53(9z),rop=t2"
 =========================******

Goto: Begin this document , End this document , UVSI Home-Page

I1. uvsort: Instruction Option 'o' for output time

This uvsort.doc documents only functions (fili1,filo1,keys,sel/del,etc), but not instructions (mvc,mvn,add,rep,etc) which were documented in uvcp.doc.

      logical:  mvc, mvn, clr
   arithmetic:  add, sub, mpy, div, pac, unp
 compare/skip:  cmc, cmn, skp
 scan/replace:  rep, scn, sqz
   translates:  tra, tre, trl, tru
        other:  anc, orc, chx, hxc, tim

Some instructions support 'options' (appended to the instruction code), and these were also documented in uvcp.doc. Instruction option 'o' is unique to uvsort & may be coded on most instructions. Option 'o' identifies instructions to be executed at sort 'output time' vs sort 'input time' (the default).

option 'o' example


 mvno=70(9)53,divo=70(9)38(6)
 ============================

The complete uvsort coding might be:


 uvsort="fili1=dat1/sales2,rcs=64,typ=LSTt,filo1=tmp/sales2,rcs=80,key1=0(6)\
 ===========================================================================
 ,sum1=38(6z),sum2=53(9z),mvno=64(9)53,divo=64(9)38(6)"
 =========================****************************

The 'sum' functions accumulate the qty & amount into 1 record for each customer. The 'mvno' moves the accumulated amount to the end of the record and the 'divo' calculates the average sale (total sales/total qty). The 'o' option is essential to force these instructions to be executed at sort output time (vs the default sort input time).

example input file

          1         2         3         4         5         6         7         8
 12345678901234567890123456789012345678901234567890123456789012345678901234567890
 cust#   slmn YYMMDD invoice  product    qty   price    amount
 ================================================================================
 130140    21 940802 IN111001  HAM001  000020 0000001 000000020
 130140    21 940802 IN111001  SCR012  000021 0001001 000021021
 130140    21 940802 IN111001  CHR001  000022 0002001 000044022
 139923    35 950802 IN111002  TAB013  000023 0003001 000069023
 139923    35 950807 IN111002  TAB013  000024 0004001 000096024

                     ** example output file **
                                       (sum1)           (sum2)
 cust#   slmn YYMMDD invoice  product    qty   price    amount   avg amount
 ===============================================================================
 130140    21 940802 IN111001  CHR001  000063 0002001 000065063  000001032
 139923    35 950807 IN111002  TAB013  000047 0004001 000165047  000003511

Goto: Begin this document , End this document , UVSI Home-Page

I2. uvsort: Instruction Option 'o#' for specified files

option 'o#' on instructions for filo1,filo2,etc

Instructions coded with option 'o', but without a number (as on prior page) are applied to all output files.

Instructions coded with o1,o2,o3,...o60 are applied to the corresponding output file (filo1,filo2,filo3,...filo60).

Option 'o' on instructions could be combined with option 'o' on selects/deletes. For example, we will append the salesman's name on the output records as we select the different salesmen to separate files.


 uvsort "fili1=dat1/sales3,rcs=64,typ=RSF,key1=10(2)\
 ===================================================
 ,filo1=tmp/sm21,filo2=tmp/sm35,filo3=tmp/sm44\
 =============================================
 ,sel1o1=10(2):21,sel1o2=10(2):35,sel1o3=10(2):44\
 ================================================
 ,mvco1=64(4):John,mvco2=64(5):Henry,mvco3=64(6):George"
 =======================================================

Goto: Begin this document , End this document , UVSI Home-Page

J1. uvsort: I/O file declaration - 3 methods

declaring filenames on the COMMAND LINE


    uvsort "fili1=dat1/sales2,rcs=64,typ=RSF,filo1=tmp/sales2,key1=10(8)"
    ========***************==================***************============

declaring input filenames via the QUERY option


    uvsort "fili1=?,rcs=64,typ=RSF,filo1=tmp/allsales,key1=10(8)"
    ========*******=============================================

the '?' will cause the filename to be solicited from the operator
would be used in a script when the filename varies from run to run

declaring Filenames in a PARAMETER FILE


    uvsort pf/demo/uvsortsplit1  <-- execute instructions in parameter file
    ===========================

 # uvsortsplit1 - split sales to separate files for specified salesman
 #              - output separate file for salesmen 21,35,44,65,85
 #              - select sales only if extended amount > $ 500.00
 #              - see doc at http://www.uvsoftware.ca/uvsort.htm#J1
 #
 # Compare to 'uvsortsplit2' using 'f1' option on 'key1f1=...'
 # - allows unlimited number of output files with 1 line of code
 # But this 'uvsortsplit1' more flexible if you do not want all salesmen
 # - or if sort key not salesman#
 #
 # export PFD=/home/uvadm/pf/demo  <-- Vancouver Utility Parameter File Demos
 # uvsort $PFD/uvsortsplit2        <-- execute uvsort on this parameter file
 #
 fili1=dat1/sales2,rcs=64,typ=RST
 key1=10(8),key2=53(9z)
 filo1=tmp1/sales2_21,rcs=64,typ=RST,sel1o1=10(2):21,sel2o1=53(9)>000050000
 filo2=tmp1/sales2_35,sel1o2=10(2):35,sel2o2=53(9)>000050000
 filo3=tmp1/sales2_44,sel1o3=10(2):44,sel2o3=53(9)>000050000
 filo4=tmp1/sales2_65,sel1o4=10(2):65,sel2o4=53(9)>000050000
 filo5=tmp1/sales2_85,sel1o5=10(2):85,sel2o5=53(9)>000050000

Goto: Begin this document , End this document , UVSI Home-Page

J2. uvsort: I/O file declaration - 3 methods

notes re Multiple Files in uvsort Parameter File

A parameter file would be used when the command line would be too long to be comfortable with, especially if the command line is to be entered from the UNIX prompt, and to a lesser extent when the command is within a script file. Use a parameter file for complex sorts with many sels/dels &/or output files.

Alternatively, you could use the sort facility within the 'uvcopy' utility (vs lengthy complex uvsorts). The uvcopy instructions are more powerful & easier to code but not as compact as uvsort fucntions such as sel/del.

Alternative for Multiple Output Files

 # uvsortsplit2 - demo uvsort split sales to separate files for each salesman
 #              - method #2 using 'f1' option on 'key1f1=...'
 #              - allows unlimited number of output files with 1 line of code
 #              - see doc at http://www.uvsoftware.ca/uvsort.htm#C6 & J2
 #
 # Compare to 'uvsortsplit1' which specifies salesman# & multiple output files
 # (but more flexible if you do not want all slsmn or if sort key not slsmn#)
 #
 # export PFD=/home/uvadm/pf/demo  <-- Vancouver Utility Parameter File Demos
 # uvsort $PFD/uvsortsplit2        <-- execute uvsort on this parameter file
 #
 fili1=dat1/sales2,typ=RST,rcs=64,filo1=tmp1/sales_*,key1f1=10(2)
 #===============================================================
 #
 # Alternative to executing uvsort on this parameter file
 # - can enter uvsort command at shell prompt as follows:
 #
 # uvsort "fili1=dat1/sales2,typ=RST,rcs=64,filo1=tmp1/sales_*,key1f1=10(2)"
 #==========================================================================

Goto: Begin this document , End this document , UVSI Home-Page

K1. uvsort: MULTIPLE INPUT FILES - 3 methods

declaring multiple input files on the COMMAND LINE


    uvsort "fili1=sales1,fili2=sales2,rcs=128,typ=RSF,filo1=sales,key1=10(8)"
    ========*************************========================================

up to 20 input files may be declared (fili1,fili2,etc to fili8)

run option rop=f2 to input all files from directory - or all files matching a specified prefix &/or suffix


    uvsort "fili1=dat1/sales*,rcs=64,typ=RSF,filo1=allsales,key1=10(8),rop=f2"
    ==============***********==========================================******=

declaring multiple input files via the '??' QUERY OPTION


    uvsort "fili1=??,rcs=64,typ=RSF,filo1=allsales,key1=10(8)"
    ========********==========================================

the '??' causes uvsort to query the operator for the filenames
enter null to end the query process
the '??' method can be used either on an operator entered command
line or from within a script to query the operator

Note	The '??' was removed in Jan2004 when major enhancements made to allow multiple inputs from a directory. May be revived sometime in future if anybody requests it.

Goto: Begin this document , End this document , UVSI Home-Page

L0. uvsort examples

Demo file '/home/uvadm/dat1/sales1' for examples 1-3

          1         2         3         4         5         6
 1234567890123456789012345678901234567890123456789012345678901234
 cust#   slmn YYMMDD invoice  product    qty   price    amount
 ================================================================
 130140    21 940802 IN111001  HAM001  000020 0000001 000000020
 130140    21 940802 IN111001  SCR012  000021 0001001 000021021
 130140    21 940802 IN111001  CHR001  000022 0002001 000044022
 139923    35 950802 IN111002  TAB013  000023 0003001 000069023
 139923    35 950807 IN111002  TAB013  000024 0004001 000096024
 150825    44 960804 IN1122    HAM001  000025 0005001 000125025
 150825    44 960804 IN1122    HAX129  000026 0006001 000156026
 201120    44 970807 CR5234    WHIP75  000027 0007001 000189027
 223240    65 980816 CR955     HAM001  000028 0008001 000224028
 223240    65 980816 IN441     BBQ001  000029 0009001 000261029
 308685    21 990812 IN6605    SAW051  00001p 0000001 00000001p
 308685    21 990812 IN6605    WHIP75  00001q 0001001 00001101q
 308685    21 990812 CR8835    TAB013  00001r 0002001 00002401r
 315512    44 000805 IN2251    HAM001  00001s 0003001 00003901s
 315512    44 000805 IN2251    SAW051  00001t 0004001 00005601t
 315512    44 000805 IN2255    WHIP75  00001u 0005001 00007501u
 400002    85 010812 CR245     HAX129  00001v 0006001 00009601v
 406082    35 020815 IN33001   BBQ001  00001w 0007001 00011901w
 406082    35 020815 IN33001   TAB013  00001x 0008001 00014401x
 406082    65 020816 IN441     HAM001  00001y 0009001 00017101y

Notes re demo files

'dat1/sales1' is a typical mainframe file - fixed record length & no LineFeeds. This means you cannot view/print using the unix vi, cat, more, lp, etc.


 vi dat1/sales1   <-- try to view with 'vi'
 ==============

'vi' will report the entire file as 1 line & the records we know to be 64 bytes will be all run together on the 80 byte screen lines.


 uvhd dat1/sales1 r256  <-- use 'uvhd' to display files w/o LineFeeds
 =====================

                  10        20        30        40        50        60
 r#    1 0123456789012345678901234567890123456789012345678901234567890123
       0 130140    21 940802 IN111001  HAM001  00002{ 0000001 00000002{
      64 130140    21 940802 IN111001  SCR012  00002A 0001001 00002102A
     128 139923    35 950802 IN111002  CHR001  00002B 0002001 00004402B
     192 139923    35 950802 IN111002  TAB013  00002C 0003001 00006902C

The record size is actually 64, but was specified as 256, which explains why you see the 1st 4 records in alignment (256/4 = 64).

Goto: Begin this document , End this document , UVSI Home-Page

L1. uvsort examples

uvsort ex#1 - simple sort

Given

the dat1/sales1 file illustrated above.

Required

sort by product#

uvsort ex#1 - solution


 uvsort "fili1=dat1/sales1,rcs=64,typ=RSF,filo1=tmp/s1,key1=30(6)"
 =================================================================

output file

 223240    65 980816 IN441     BBQ001  000029 0009001 000261029
 406082    35 020815 IN33001   BBQ001  00001w 0007001 00011901w
 130140    21 940802 IN111001  CHR001  000022 0002001 000044022
 130140    21 940802 IN111001  HAM001  000020 0000001 000000020
              - - - - - some records omitted - - - - -
 406082    35 020815 IN33001   TAB013  00001x 0008001 00014401x
 201120    44 970807 CR5234    WHIP75  000027 0007001 000189027
 308685    21 990812 IN6605    WHIP75  00001q 0001001 00001101q
 315512    44 000805 IN2255    WHIP75  00001u 0005001 00007501u

Notes

The output file is shown with records on separate lines, but as explained on the previous page, they would in fact be all run together since there are no LineFeeds in this typical mainframe file & the logical record size of 64 is not a multiple or sub-multiple of the 80 byte screen size.

There is another version of file 'dat1/sales2' which does have LineFeeds (in the 64th byte, unused for data in this case). If you wish you can use this file for easier test/demos.

Goto: Begin this document , End this document , UVSI Home-Page

L2. uvsort examples

uvsort ex#2 - simple sort

Given

the dat1/sales1 file illustrated on page 'L1'.

Required

sort by product# within salesman.
also create a text output file
(with LineFeeds, so we can inspect with vi,cat,more,etc)

uvsort ex#2 - solution


 uvsort "fili1=dat1/sales1,rcs=64,typ=RSF,filo1=tmp/s1,typ=LSTt\
 ================================================================
 ,key1=10(2),key2=30(6)"
 =======================

The command is shown on 2 lines for this documentation & but you can enter it all on 1 line or you can use the '\' to complete the command on a 2nd line.

output file

 130140    21 940802 IN111001  HAM001  00002{ 0000001 00000002{
 308685    21 990812 IN6605    SAW001  00001} 0000001 00000001}
 130140    21 940802 IN111001  SCR012  00002A 0001001 00002102A
              - - - - - some records omitted - - - - -
 223240    65 980816 CR955     HAM001  00002H 0008001 00022402H
 406082    65 020816 IN441     HAM001  00001R 0009001 00017101R
 400002    85 010812 CR245     HAX129  00001O 0006001 00009601O

Notes

Note that the records are in sequence 1st by key1=10(2) (salesman), 2nd by key2=30(6) (product#).
The input typ=RSF means 'Record Sequential Fixed' (no LineFeeds).
The output file type is now changed to 'typ=LSTt' which means 'Line Sequential Terminated (with LineFeeds)'.
The 't' option on 'LSTt' truncates trailing blanks. The LineFeed is inserted after the right hand non-blank.

Goto: Begin this document , End this document , UVSI Home-Page

L3. uvsort examples

uvsort ex#3 - selecting records

Given

the dat1/sales1 file illustrated on page 'L1'.

Required

sort by product within salesman & select following records:


 #1. Salesman 21,44,& 65 (columns 10-11)


 #2. Product hammers ('HAM' in cols 31-33)

uvsort solution


 uvsort "fili1=dat1/sales2,rcs=64,typ=RSF,filo1=tmp/sales2,key1=10(2),key2=30(6)\
 ================================================================================
 ,sel1=10(2):21,sel2=30(3):HAM,sel1=10(2):44,sel2=30(3):HAM\
 ===========================================================
 ,sel1=10(2):65,sel2=30(3):HAM"
 ==============================

output file


 cust#   slmn YYMMDD invoice  product    qty   price    amount
 ================================================================
 130140    21 940802 IN111001  HAM001  00002{ 0000001 00000002{
 150825    44 960804 IN1122    HAM001  00002E 0005001 00012502E
 315512    44 000805 IN2251    HAM001  00001L 0003001 00003901L
 223240    65 980816 CR955     HAM001  00002H 0008001 00022402H
 406082    65 020816 IN441     HAM001  00001R 0009001 00017101R

Notes

We have used dat1/sales2 as the input file for this example, so you will find it easier to check the outputs with 'vi'.
'dat1/sales2' already has LineFeeds (even though typ=RSF does not usually have LineFeeds), so we did not need to specify the output as 'typ=LSTt' to insert LineFeeds. See more explanations given in examples #1 & #2 above.

Goto: Begin this document , End this document , UVSI Home-Page

L4. uvsort examples

uvsort ex#4 - loading indexed (ISAM) files

given

input test file uvadm/dat1/custmas1,
record layout & 1st record (of 32) is listed below.
since the record contains many packed fields, we used the 'uvhd'
(hexdump utility) to list the 1st record.

           001-006 - customer#
           009-009 - Delete flag 'D'
           011-035 - customer name
           036-060 - address
           061-076 - city
           078-079 - province
           081-090 - postal code
           091-102 - phone#
           103-120 - contact name
           121-180 - this year sales 12 x 5 byte packed fields
           181-240 - last year sales 12 x 5 byte packed fields
           241-256 - unused (no LineFeed at end record)

Note	above layout is ONE relative you must specify ZERO relative displacements for uvsort (subtract 1)


    uvhd dat1/custmas1 r256  - execute uvhd to display 1st record
    =======================

                       1         2         3         4         5         6
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 130140    EVERGREEN MOTORS LTD.    1815 BOWEN ROAD          NANA
             3333332222454545444244545524542222233332445442544422222222224444
             130140000056527255E0DF4F230C44E0000181502F75E02F140000000000E1E1
          64 IMO          BC V9S1H1    604-754-5531                  ..4V|...
             4442222222222442535343222233323332333322222222222222222201357000
             9DF00000000002306931810000604D754D55310000000000000000000246C000
         128 .........W0....`........)X}..f3.....\.................4V}...f...
             0000000005300016000000002570063100095000000000000000013570016000
             0C0000C0270D0540C0000C0098D0263C0444C0000C0000C0000C0246D0056C00
         192 ...............................f.....<........f.C
             0000008900000000880000000018000680001300000000694222222222222222
             00C0026C0000C0023C0000C0083C0056D0012C0000C0016D3000000000000000

REQUIRED

sort & load an ISAM file with 2 keys
customer# (cols 1-6) & city name (cols 61-76)

uvsort solution


 uvsort "fili1=dat1/custmas1,rcs=256,typ=RSF,filo1=tmp/custmas1,typ=ISF,\
 =======================================================================
 isk1=0(6),isk2=60(16),key1=0(6),key2=60(16)"
 ===========================================

The sort keys are coded as key1,key2,etc. 'isk' is the code for 'Indexed Sequential Key', which could be different than the sort keys, but usually the same.

Goto: Begin this document , End this document , UVSI Home-Page

L5. uvsort examples

uvsort ex#5 - Re-Organizing Indexed files

Example #2 (on the previous page) sorted a 'sequential' file & loaded an 'indexed' file. This example will input an indexed file, sort,& write back to the same file.

Writing back to the same file could be a problem if your machine just happened to crash at while writing the output, so you should ensure that a backup has preceded a job such as this.

Sorting & reloading an indexed file should be done periodically to reorganize the indexed file & drop any deleted records. Deleted records could be both physical & logical.

'Reorganizing' means to put the data partition records back into sequence to improve sequential access times. Randomly added records will be at the end of the data partition & reading, sorting,& reloading will put them back into physical sequence depending on their key fields.

Physical deletes are usually performed by COBOL program 'delete' commands. The micro Focus COBOL file handler changes the delete flag (an extra byte at end of each record) from x'0A' to x'00'. The uvsort program wont see these records because the D-ISAM file handler (compatible with COBOL file handler) drops them automatically.

Logical deletes are probably also performed by a COBOL program, but rather than use the 'delete' command, the program sets a data byte to some agreed upon value that other programs will check. In our uvsort example below we will assume that a 'D' in byte 8 (column 9) identifies a logically deleted record. uvsort will drop it due to the 'del=8(1):D'.

Indexed file sort/reload demo


 #1. cp dat1/custmast* tmp      - copy custmast & custmast.idx to tmp subdir
     =====================


 #2. uvsort "fili1=tmp/custmast,rcs=256,typ=ISF,filo1=tmp/custmast\
     =============================================================
     ,key1=0(6),isk1=0(6d),del=8(1):D"
     =================================

Console Log

 uvsort:050330:115707: uvsort fili1=tmp/custmast,filo1=tmp/custmast
 uvsort:050330:115707: EOF fili01 30 rds, 8224 size; tmp/custmast
 uvsort:050330:115707: DISAM: recs=30,rcsz=256,keys=7; tmp/custmast
 uvsort:050330:115707: sort phase complete, used 64 megabytes, sort output begin
 uvsort:050330:115707: EOF filo01 27 wrts, 0 size; tmp/custmast
 uvsort:050330:115707: DISAM: recs=27,rcsz=256,keys=1; tmp/custmast

Goto: Begin this document , End this document , UVSI Home-Page

L6. uvsort examples

uvsort ex#6 - reformatting records

given

input test file uvadm/dat1/custmas1
same file as on previous example
input record layout is same as previous example
(see 1st record listed on prior page)
output record layout as follows:

           001-006 - customer#
           009-009 - delete flag (D or X)
           011-035 - customer name
           036-047 - phone# (from 091-102 of input)

REQUIRED

Create a customer Telephone Number list
sort by telephone#
reformat the record to include only cust#, cust name,& telephone#

uvsort solution


 uvsort "fili1=dat1/custmas1,rcs=256,typ=RSF,filo1=tmp/phones,typ=LSTt,rcs=80\
 ============================================================================
 clr=35(45):0x20,mvc=35(12)90,key1=35(12)"
 =========================================

sample output file

 313720    MONITREX ENGINEERING LTD
 306959  X 356582 ALBERTA LTD.          -   -
 208060    E&L TRUCKING              403-231-7312
 201120    ALLTYPE RENTAL LTD.       403-246-5274
 308685    FOOTHILLS ELECTRIC        403-296-4856
 211140  D FORD NEW HOLLAND          403-312-2760
 318833    TOP NOTCH CONSTRUCTION    403-385-2965
 224700    OTTO MOBILES WESTERN LTD. 403-467-3197
 223240  X NICHOLSON CHEVROLET LTD.  403-467-4257
 234300    TERRITORIAL REWIND        403-512-3948
 204700    CASE POWER EQUIPMENT      403-538-1743
 231550    SPARROW ELECTRIC CO. LTD. 403-742-6139
 301120  D ALBERTA GAS CHEMICALS LTD 403-952-2475
 401210    COAST RANGE CONSTRUCTION  604 681 2626

We might have dropped the deleted records (X or D in column 9), by adding the following to the uvsort command string:


 ,del1=8(1):D,del1=8(1):X
 ========================

The 'mvc=35(12)90' is performed during input, so the sort key is specified as 'key1=35(12)'. The 'key=...' is specified following the 'mvc=...', but in fact the command sequence does not matter. You can perform 'mvc's at output time using the 'o' option (mvco=... see page 'I2').

Goto: Begin this document , End this document , UVSI Home-Page

M1. rebuilding Indexed files from the data partition of corrupted file s

Microfocus COBOL Indexed files (IDXFORMAT1)

 Data    partition filename: 'filename'     <-- NO .dat extension
                             'filename.dat'   - OR with .dat if ISDATEXT=".dat"
 Indexed partition filename: 'filename.idx' <-- Indexed part always ".idx"

C-ISAM data partition filename: 'filename.dat' <-- C-ISAM convention

No control area at begining of data records (vs MBP ISAM files)

Data records are 1 byte larger than record size loaded because a LineFeed is appended to each data record. The LineFeed is converted to a x'00' if the record is deleted

recovering Microfocus data records

The following will demonstrate the process by loading an Indexed file, pretending that the Index is corrupted,& then rebuilding the Indexed file from the data partition only.


 #0. Create an Indexed file for recovery test/demo


     uvcp "fili1=dat1/custmas2,rcs=128,typ=RSF,filo1=tmp/cm2,typ=ISF,isk1=0(6)"
     ==========================================================================


 #1. Copy the data partition to a save file


     cp tmp/cm2.dat tmp/cm2.sav
     ==========================


 #2. erase the original filename.dat & filename.idx


     rm tmp/cm2.dat tmp/cm2.idx
     ==========================


 #3. sort/reload the Indexed file from the data partition


     uvsort "fili1=tmp/cm2.sav,rcs=129,typ=RSF,filo1=tmp/cm2,rcs=128,typ=ISF\
     =======================================================================
            ,isk1=0(6),key1=0(6),del=128:0x00"
            =================================

Note	Also see alternative methods using uvcopy in section uvcopy4.doc recovISMF - recover Microfocus COBOL files with corrupted indices recovISMB - recover MBP COBOL ISAM files with corrupted indices

Goto: Begin this document , End this document , UVSI Home-Page

M2. rebuilding Indexed files from the data partition of corrupted files

10b - MBP COBOL Indexed files

      Data    partition filename is: 'filename.DAT'
      Indexed partition filename is: 'filename.KEY'

588 byte binary control area at begining of data records (Microfocus ISAM files do not have).

Data records are 2 bytes larger than record size loaded because a 2 byte binary record size is prepended to each data record. The record size is converted to a x'0000' if the record is deleted.

Note	See tip# 11b in UNIXtips.doc for an illustration of the control area at the begining of an MBP COBOL ISAM file.

recovering MBP data records from an Indexed file

The following will demonstrate the process by loading an Indexed file, pretending that the Index is corrupted,& then rebuilding the Indexed file from the data partition only.


 #0. Create an Indexed file for recovery test/demo


     uvcp "fili1=dat1/custmas2,rcs=128,typ=RSF,filo1=tmp/cm2,typ=ISF,isk1=0(6)"
     ==========================================================================


 #1. Copy the data partition to a save file


     cp tmp/cm2.DAT tmp/cm2.sav1
     ===========================


 #2. erase the original filename.DAT & filename.KEY


     rm tmp/cm2.DAT tmp/cm2.KEY
     ==========================


 #3. drop the control area from the begining of the data partition
    (using the UNIX 'dd' utility)


     dd if=tmp/cm2.sav1 of=tmp/cm2.sav2 bs=588 skip=1
     ================================================


 #4. sort/reload the Indexed file from the data partition


     uvsort "fili1=tmp/cm2.sav2,rcs=130,typ=RSF,filo1=tmp/cm2,rcs=128,typ=ISF\
     ========================================================================
             ,isk1=0(6),key1=0(6),del=0(2):0x0000,mvc=0(128)2"
              ================================================

Goto: Begin this document , End this document , UVSI Home-Page

N1. Record Sequential Variable length files (typ=RSV)

As of Nov 2002 'typ=RSV' supports OUTPUT files as well as INPUT files. 'typ=RSV' provides compatibility with Micro Focus COBOL IDXFORMAT3 Variable length files, but only for Sequential. Indexed variable length support is provided by an alternate version of uvsort (uxsort linked with EXTFH).

For complete documentation on these files, please see the 'FILE STRUCTURES' chapter in Micro Focus Object COBOL or Server Express COBOL, or the web site http://supportline.microfocus.com/documentation/books/ocds4140/fhfile.htm.

'typ=RSV' files have 128 byte file header. Each record has 2 or 4 byte record header/prefix which has x'40' in the first nibble to identify a data record, followed by the record size in the next 12 bits or 28 bits, depending on max recsize <= 4095 or > 4095 bytes.

uvcp/uvsort RSV Output Implementation

The maximum rcsz is specified by 'rcs=____' & the minimum rcsz is specified by option 'v' of 'typ=RSVv____'. Record size determined by scanning backward to last non-blank & round up to a multiple of 4 (includes 2/4 byte rec hdr).
If minimum rcsz (option v) is not specified, all output records will be written with maximum record size & blank padded as necessary.
'uvsort' allows any combination of INPUT/OUTPUT file types (LST,RSF,ISF,RSV) & allows you to convert files from any type to any other type. For example you can create test files by using 'vi' to create text records & then use uvcp/uvsort to convert the text file (typ=LST) to a typ=RSV output file.

uvcp/uvsort typ=RSV examples


 #1. uvcp "fili1=dat1/testRSV1,rcs=256,typ=LST,filo1=tmp/a,rcs=254,typ=RSV"
     ======================================================================

reads text file (typ=LST) of records less than 256 bytes
writes Rec Seq Variable (typ=RSV) records
all records will be 256 bytes (2 byte recsize hdr + 254 bytes data)


 #2. uvcp "fili1=dat1/testRSV1,rcs=256,typ=LST,filo1=tmp/b,rcs=254,typ=RSVv62"
     =========================================================================

    Same as ex#1 but writes typ=RSV records varying from 128 to 256
    - 064 bytes (2 hdr + 062 data) if bytes 062+ all blank
      ??? bytes (2 hdr + ??? data) depending on last nonblank (multiple of 4)
    - 256 bytes (2 hdr + 254 data) is maximum


 #3. uvcp "fili1=dat1/testRSV1,rcs=256,typ=LST,filo1=tmp/c,rcs=5000,typ=RSVv400"
     ===========================================================================

    This example writes 4 byte record headers (vs 2 bytes in above examples)
    because the maximum record size is > 4095 bytes (rcs=5000).
    Record sizes vary from 404 to 5004 depending on last nonblank
    -  404 bytes (4 hdr +  400 data) if bytes 400+ all blank
      ???? bytes (4 hdr + ???? data) depending on last nonblank (multiple of 4)
    - 5004 bytes (4 hdr + 5000 data) is maximum

Goto: Begin this document , End this document , UVSI Home-Page

N2. Record Sequential Variable length files (typ=RSV)

using uvcp to create a typ=RSV file


 #2. uvcp "fili1=dat1/testRSV3,rcs=256,typ=LST,filo1=tmp/test3,rcs=254,typ=RSVv80"
    =============================================================================

input text file (3 records)


 #1 testRSV3 - text to create Rec Seqntl Variable(RSV) file to test uvcp & uvsort
 #2 uvcp "fili1=dat1/testRSV3,rcs=256,typ=LST,filo1=tmp/test3,rcs=254,typ=RSVv80"
 #3 output records vary from min(84) to max(256) depending on last nonblank-----|
 -----this record will be 164 (data 160 + 2 byte rechdr + 2 round to mult 4)---->

output RSV file (displayed with uvhd)


 uvhd tmp/test3 v      <-- display file using uvhd with option 'v'
 ================

                       1         2         3         4         5         6
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 0~......030114102019.................>.......................P..
             370000003333333333330000000000000000030000000000000000000F000500
             0E00000003011410201900000000000000000E0100000000100000000E000000
          64 ................................................................
             0000000000000000000000000000000000000000000000000000000000000000
             0000000000000000000000000000000000000000000000000000000000000000
                       1         2         3         4         5         6
 r#        2 0123456789012345678901234567890123456789012345678901234567890123
         128 @P#1 testRSV3 - text to create Rec Seqntl Variable (RSV) file to
             4523276775553222767727626766762566256767625676666622555226666276
             00310453423630D0458404F032514502530351E4C0612912C5082369069C504F
          64  test uvcp/uvsort
             27677277672777677222
             0453405630F563F24000
                       1         2         3         4         5         6
 r#        3 0123456789012345678901234567890123456789012345678901234567890123
         212 @P#2 uvcp "fili1=dat1/testRSV3,rcs=256,typ=LST,filo1=tmp/test3,r
             4523277672266663366732767755532767333327773455266663376727677327
             0032056300269C91D4141F45342363C233D256C490DC34C69CF1D4D0F45343C2
          64 cs=254,typ=RSVv80"
             67333327773555733222
             33D254C490D236680200
                       1         2         3         4         5         6
 r#        4 0123456789012345678901234567890123456789012345678901234567890123
         296 @.#3 output records vary from min(84) to max(256) depending on l
             4A23267777727666767276772676626662332276266723332266766666626626
             00330F540540253F24306129062FD0D9E884904F0D188256904505E49E70FE0C
          64 ast nonblank-----|-----this record will be 164 (data 160 + 2 byt
             6772666666662222272222276672766676276662662333226676233322232677
             1340EFE2C1EBDDDDDCDDDDD48930253F24079CC025016408414101600B020294
         128 e rechdr + 2 round to mult 4)---->
             627666672223276766276267672322222322
             502538420B0202F5E404F0D5C4049DDDDE00

Goto: Begin this document , End this document , UVSI Home-Page

N3. Record Sequential Variable length files (typ=RSV)

File Header record for typ=RSV (& typ=IDX) files

                       1         2         3         4         5         6
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 0~......021125110120.................>.......................~..
             370000003333333333330000000000000000030000000000000000000F000700
             0E00000002112511012000000000000000000E0100000000100000000E000E00
          64 ................................................................
             0000000000000000000000000000000000000000000000000000000000000000
             0000000000000000000000000000000000000000000000000000000000000000

00-00 - x'30' identifies the file header record (vs x'40' for data records)

 01-01 - data size of this record x'7E' = (7*16+14*1) = 112+14 = 126
       - total size is 128 including the 2 byte record header

 00-03 - If max rcsz > 4095, the record header would be 4 bytes x'3000007C'
       - 7*16 + 12*1) = 112+12 = 124 + 4 byte header = 128 total record size

08-19 - file creation date/time stamp (yymmddHHMMSS)

36-36 - reserved value x'3E' required for Micro Focus COBOL IDXFORMAT3 files

39-39 - file ORGanization x'01' = Sequential

48-48 - Recording Mode x'01' = Variable format

 54-57 - Maximum record length
       - In this example x'000000FE' = (15*16+14*1) = 240+14 = 254

 58-61 - Minimum record length
       - In this example x'0000007E' = (07*16+14*1) = 112+14 = 126

typ=IDX files same as typ=RSV with index added

The file structure described above for typ=RSV files also applies to typ=IDX files which include an indexed partition & some extra system records in the data partition. You must use the alternate program 'uxsort' (vs 'uvsort) if either input or output or both files are typ=IDX.

Goto: Begin this document , End this document , UVSI Home-Page

O1. 'uxsort' test/demo Indexed Seqntl Variable length files

test/demo uxsort for Indexed Seqntl Variable length files

'uxsort' supports Indexed Sequential Variable length record files & must be compiled with the Micro Focus COBOL 'cob' command as shown on page D1 of install.htm. You can demo uxsort using test files provided as shown below:

dat1/idxtest - test file to demo uxsort

 0010.0060 idxtest text file to create Var Lth Indexed files.
 0020.0064 - input to COBOL,uxcp,uxcopy to create IDXf1,3,8 files
 0030.0065 - 01-04=ISAM key, 05=separator, 06-09=size, 10-400 text
 0040.0067 - output record varies from 64 to 256, depending on 06-09
 0050.0080 - here is an 80 byte record------------------------------------------>
 0060.0160 - here is a 160 byte record...........................................
 0070.0240 - here is a 240 byte record (last record in file).....................
 ===============================================================================>

uxsort - read text (typ=LST), write Indexed file (typ=IDXf3)


 #1. uxsort "fili1=dat1/idxtest,rcs=256,typ=LST\
     ===========================================
             ,filo1=tmp/idxf3v60,rcs=254,typ=IDXf3v60,isk1=0(4),key1=0(4)"
             ============================================================


 #2. uvhd tmp/idxf3v60.dat v    <-- examine output data partition with 'uvhd'
     =======================      - need option 'v' for IDXf3 files

Note	See uvhd output (in vertical hexadecimal) on the next page --->

uxsort - read Indexed (typ=IDXf3), write text (typ=LST)


 #3. uxsort "fili1=tmp/idxf3v60,rcs=256,typ=IDXf3,filo1=tmp/idxf3v60vL,typ=LSTt,key1=0(4)"
     ============================================================================


 #4. vi tmp/idxf3v60L           <-- examine output file with 'vi'
     ================

This converts the typ=IDXf3 file back to a text file (typ=LSTt) which should be the same as the original file used in step #1. There is no need to specify the indexed keys or the 'v' option for input files.

Goto: Begin this document , End this document , UVSI Home-Page

O2. 'uxsort' test/demo Indexed Seqntl Variable length files

uvhd hexdump of IDXf3 varlth indexed file


 #1a. uvhd tmp/IDXf3v60.dat v   <-- examine output data partition with 'uvhd'
      =======================     - uvhd option 'v' for IDXf3 or IDXf8 files

                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 0~......0705141326133507051413261335.>.......................<..
             370000003333333333333333333333333333030000000000000000000F000300
             0E00000007051413261335070514132613350E0200000000100000000E000C00
          64 ............................................X"8*................
             0000000000000000000000000000000000000000000052320000000000000000
             00000000000020100000000000000000000000000000828A0000000000000000
                      10        20        30        40        50        60
 r#        2 0123456789012345678901234567890123456789012345678901234567890123
         384 @<0010.0060 idxtest text file to create Var Lth Indexed files...
             4333332333326677677276772666627626766762567247624666766266667200
             0C0010E00600948453404584069C504F032514506120C4809E45854069C53E00
                      10        20        30        40        50        60
 r#        3 0123456789012345678901234567890123456789012345678901234567890123
         448 @@0020.0064 - input to COBOL,uxcp,uxcopy to create IDXf1,3,8 fil
             4433332333322266777276244444277672776677276267667624456323232666
             000020E00640D09E05404F03F2FCC5830C583F0904F0325145094861C3C8069C
          64 es..
             6700
             5300
                      10        20        30        40        50        60
 r#        4 0123456789012345678901234567890123456789012345678901234567890123
         516 @A0030.0065 - 01-04=ISAM key, 05=separator, 06-09=size, 10-400 t
             4433332333322233233345442667223337676767672233233376762233233327
             010030E00650D001D04D931D0B59C005D3501214F2C006D09D39A5C010D40004
          64 ext.
             6770
             5840
                      10        20        30        40        50        60
 r#        5 0123456789012345678901234567890123456789012345678901234567890123
         584 @C0040.0067 - output record varies from 64 to 256, depending on
             4433332333322267777727666762767667267662332762333226676666662662
             030040E00670D0F540540253F240612953062FD06404F0256C04505E49E70FE0
          64 06-09...
             33233000
             06D09000
                      10        20        30        40        50        60
 r#        6 0123456789012345678901234567890123456789012345678901234567890123
         656 @P0050.0080 - here is an 80 byte record-------------------------
             4533332333322266762672662332677627666762222222222222222222222222
             000050E00800D0852509301E080029450253F24DDDDDDDDDDDDDDDDDDDDDDDDD
          64 ----------------->..
             22222222222222222300
             DDDDDDDDDDDDDDDDDE00
                ----- last 3 records omitted to keep this on 1 page -----

Goto: Begin this document , End this document , UVSI Home-Page

O3. 'uxsort' test/demo Indexed Seqntl Variable length files

create IDXf8 varlth indexed file & examine with uvhd


 #1. uxsort "fili1=dat1/idxtest,rcs=256,typ=LST\
     ==========================================
             ,filo1=tmp/IDXf8v60,rcs=254,typ=IDXf8v60,isk1=0(4),key1=0(4)"
             =============================================================


 #2. uvhd tmp/IDXf8v60.dat v  <-- examine output data partition with 'uvhd'
     =======================

                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 3.......0705141522382607051415223827.>.......................<..
             3F0000003333333333333333333333333333030000000000000000000F000300
             3E00000007051415223826070514152238270E0200180000106000000E000C00
          64 ............................................X"8*X"8*............
             0000000000000000000000000000000000000000000052325232000000000011
             00000007000022100000000000000000000000000008828A828A000000000000
         128 ................................................................
             00000011000000000000000000000000000000000000000F0000000000000000
             000000004244010400000040000000C0000000000000003E0000000000000000
                     ----- bytes 192-960 removed (all x'00's) -----
         960 ................................................................
                      10        20        30        40        50        60
 r#        2 0123456789012345678901234567890123456789012345678901234567890123
        3328 @<0010.0060 idxtest text file to create Var Lth Indexed files...
             4333332333326677677276772666627626766762567247624666766266667200
             0C0010E00600948453404584069C504F032514506120C4809E45854069C53E00
                      10        20        30        40        50        60
 r#        3 0123456789012345678901234567890123456789012345678901234567890123
        3392 @@0020.0064 - input to COBOL,uxcp,uxcopy to create IDXf1,3,8 fil
             4433332333322266777276244444277672776677276267667624456323232666
             000020E00640D09E05404F03F2FCC5830C583F0904F0325145094861C3C8069C
          64 es..
             6700
             5300
                      10        20        30        40        50        60
 r#        4 0123456789012345678901234567890123456789012345678901234567890123
        3464 @A0030.0065 - 01-04=ISAM key, 05=separator, 06-09=size, 10-400 t
             4433332333322233233345442667223337676767672233233376762233233327
             010030E00650D001D04D931D0B59C005D3501214F2C006D09D39A5C010D40004
          64 ext.
             6770
             5840
                      10        20        30        40        50        60
 r#        5 0123456789012345678901234567890123456789012345678901234567890123
        3536 @C0040.0067 - output record varies from 64 to 256, depending on
             4433332333322267777727666762767667267662332762333226676666662662
             030040E00670D0F540540253F240612953062FD06404F0256C04505E49E70FE0
          64 06-09...
             33233000
             06D09000
                    ----- records 6,7,8 not shown to fit 1 page -----

Goto: Begin this document , End this document , UVSI Home-Page

O4. 'uxsort' test/demo Indexed Seqntl Variable length files

IDXORMAT3/IDXFORMAT8 file header layout

The Vancouver Utilities (uxcopy,uxsort,uxqrpg,uxcp,uvhd) are compatible with Micro Focus COBOL variable length sequential & Indexed files (IDXFORMAT3 & 8).

We will document the most important fields in the file header for IDXf3/IDXf8. If desired, you can find all fields defined on the Micro Focus website.

Note that you can investigate the file headers using 'uvhd' with option 'v' for IDXFORMAT3 or IDXFORMAT8. See sample on the previous page.

 000-003 - file header record length
         - 1st 4 bits x'30' identifies a system record
         - next 12 bits is header length if max record length is < 4095
         - next 28 bits is header length if max record length is => 4095

008-014 - creation date/time yymmddhhmmsscc (indexed files only)

 039-039 - organization (1=seqntl, 2=indexed, 3=relative)
 043-043 - file type (3=IDXFORMAT3, 8=IDXFORMAT8)
 048-048 - recording mode (0=fixed, 1=variable)

 054-057 - maximum record length (big end format)
         - example recsize 80 = x'00000050'
 058-061 - minimum record length

 068-071 - record count (4 bytes Big-End Binary)
         - Indexed files only (not in variable length non-indexed file hdrs)

128-135 - offset to End-of-File (8 bytes binary big end format)

 140-141  - number of keys in Indexed file

record header layout

 000-000 - 1st 4 bits identifies the record type
 x'1_'   - system record - duplicate occurrence record
 x'2_'   - deleted record
 x'3_'   - system record
 x'4_'   - user data record
 x'5_'   - reduced user data record
 x'6_'   - pointer record
 x'7_'   - user data record referenced by a pointer record
 x'8_'   - reduced user data record referenced by a pointer record

 000-001 - next 12 bits is the record length if max recsize < 4095
 000-003 - next 28 bits is the record length if max recsize =? 4095
 002-... - record data if max recsize < 4095
 004-... - record data if max recsize => 4095

Note	all records are padded with x'00's to a multiple of 4 bytes

Goto: Begin this document , End this document , UVSI Home-Page

O5. 'uxsort' test/demo Indexed Seqntl Variable length files

Test typ=IDX file options

See page 'E6' which documents file options 'v' 'x' 'y' 'z' to modify the default behavior of the Micro Focus COBOL EXTFH file handler. Note that these tests are the same as #1 on the previous page with the addition of options x,y,or z on the output file typ=IDXv60__?__

scripts to test typ=IDX file options

The next few pages will list some scripts used at UV Software to test/demo using uxsort to write IDXFORMAT3/IDXFORMAT8 files & read them back to verify.

These scripts are supplied in /home/uvadm/sf/demo/..., which is in the PATH of the supplied profile. You can run these at your site. See instructions & explanations embedded within the scripts.

O6.	uxsortIDXf3w - write various IDXf3 file formats
	- sorts a text file & writes out 6 files with various options

O7.	uxsortIDXf8w - write various IDXf8 file formats
	- reads a text file & writes out 6 files with various options

can use 'uxcp' scripts to read back

uxcpIDXf3r

read back IDXf3 files & convert back to text files
to verify result same as original text file input

uxcpIDXf8r

read back various IDXf8 files written by uxcpIDXf8w
to verify result same as original text file input

Note	we will list the scripts only for uxsortIDXf3 & uxsortIDXf8 you can all scripts in /home/uvadm/sf/demo/... you can see some uxcp scripts listed at uvcp.htm#J4

O8.	tmp1 subdir listings after uxsortIDXf3w & after uxsortIDXf8w

Note	see VSECOBOL.htm for samples of COBOL with IDXf3/IDXf8 files also includes scripts to prove uxsort compatibility with COBOL

Goto: Begin this document , End this document , UVSI Home-Page

O6. 'uxsort' test/demo Indexed Seqntl Variable length files

 # uxsortIDXf3w - write various IDXFORMAT3 files to test uxsort
 #              - by Owen Townsend, Jan28/06, updated May14/07
 #              - IDXFORMAT8 files created by Micro Focus COBOL EXTFH
 #              - file handler used by uxcp, uxsort, uxcopy, uxqrpg
 #
 #*uxsortIDXf3w - write various IDXf8 files for read back by uxcpIDXf3r
 #                   ** can use 'uxcp' scripts to read back **
 # uxcpIDXf3r   - read back various IDXf8 files written by uxsortIDXf3w
 #              - converts output files back to text records
 # uxcpIDXf3c   - copy various IDXf8 files written by uxsortIDXf3w
 #              - writes output files with same types & record-sizes
 #
 # OPTIONS: f1=IDXFORMAT1, f3=IDXFORMAT3, f8=IDXFORMAT8 (see www.microfocus.com)
 # x1=noindex, x2=indexed; y1=fixed, y2=variable, y4=scanback to last non-blank
 # v=min recsize, (maxsize declared by rcs=___)
 #
 # 1. typ=LST-->IDXf8 indexed, varfmt but all 252+4 since no option v minsize
 uxsort "fili1=dat1/idxtest,typ=LST,rcs=252,filo1=tmp1/idxf8,typ=IDXf8,isk1=0(4)"
 #===============================================================================
 #
 # 2. typ=LST-->IDXf8v60 indexed, variable recsize pass thru from input text recs
 uxsort "fili1=dat1/idxtest,typ=LST,rcs=252,filo1=tmp1/idxf8v60,typ=IDXf8v60,isk1=0(4)"
 #===============================================================================
 #
 # 3. typ=LST-->IDXf8v60y4 indexed variable option y4 recsize at last nonblank
 uxsort "fili1=dat1/idxtest,typ=LST,rcs=252,filo1=tmp1/idxf8v60y4,typ=IDXf8v60y4,isk1=0(4)"
 #===============================================================================
 #
 # 4. typ=LST-->IDXf8x1 noindex(x1), fixed 252, no IDX filehdr, same as typ=RSF
 uxsort "fili1=dat1/idxtest,typ=LST,rcs=252,filo1=tmp1/idxf8x1,typ=IDXf8x1,isk1=0(4)"
 #===============================================================================
 #note - must read back using typ=RSF, since no index & fixed lth (no v)
 #
 # 5. typ=LST-->IDXf8v60x1 noindex(x1), varfmt, but all 252+4
 uxsort "fili1=dat1/idxtest,typ=LST,rcs=252,filo1=tmp1/idxf8v60x1,typ=IDXf8v60x1,isk1=0(4)"
 #===============================================================================
 #
 # 6. typ=LST-->IDXf8v60x1y4 noindex(x1), variable 60-252, depending on LNB (y4)
 uxsort "fili1=dat1/idxtest,typ=LST,rcs=252,filo1=tmp1/idxf8v60x1y4,typ=IDXf8v60x1y4,isk1=0(4)"
 #===============================================================================
 #
 echo "You can examine output files with 'uvhd', example for output#1 above"
 echo " "
 echo "uvhd tmp1/idxf8.dat v   <-- use 'v' option for IDXf8 files"
 echo "====================="
 #
 #Note - script may bomb on uxcp #6 if you only have 1 Micro Focus COBOL license
 #     - But you can add following to your profile to wait 1 minute (vs bomb)
 #
 # export ASLMFRETRY="5,15"
 # ========================

Goto: Begin this document , End this document , UVSI Home-Page

O7. 'uxsort' test/demo Indexed Seqntl Variable length files

 # uxsortIDXf8w - write various IDXFORMAT files to test uxsort
 #            - by Owen Townsend, Jan28/06, updated May14/07
 #            - IDXFORMAT8 files created by Micro Focus COBOL EXTFH
 #            - file handler used by uxcp, uxsort, uxcopy, uxqrpg
 #
 #*uxsortIDXf8w - write various IDXf8 files for read back by uxcpIDXf8r
 #                   ** can use 'uxcp' scripts to read back **
 # uxcpIDXf8r   - read back various IDXf8 files written by uxsortIDXf8w
 #              - converts output files back to text records
 # uxcpIDXf8c   - copy various IDXf8 files written by uxsortIDXf8w
 #              - writes output files with same types & record-sizes
 #
 # OPTIONS: f1=IDXFORMAT1, f3=IDXFORMAT3, f8=IDXFORMAT8 (see www.microfocus.com)
 # x1=noindex, x2=indexed; y1=fixed, y2=variable, y4=scanback to last non-blank
 # v=min recsize, (maxsize declared by rcs=___)
 #
 # 1. typ=LST-->IDXf8 indexed, varfmt but all 252+4 since no option v minsize
 uxsort "fili1=dat1/idxtest,typ=LST,rcs=252,filo1=tmp1/idxf8,typ=IDXf8,isk1=0(4)"
 #===========================================================================
 #
 # 2. typ=LST-->IDXf8v60 indexed, variable recsize pass thru from input text recs
 uxsort "fili1=dat1/idxtest,typ=LST,rcs=252,filo1=tmp1/idxf8v60,typ=IDXf8v60,isk1=0(4)"
 #===============================================================================
 #
 # 3. typ=LST-->IDXf8v60y4 indexed variable option y4 recsize at last nonblank
 uxsort "fili1=dat1/idxtest,typ=LST,rcs=252,filo1=tmp1/idxf8v60y4,typ=IDXf8v60y4,isk1=0(4)"
 #===============================================================================
 #
 # 4. typ=LST-->IDXf8x1 noindex(x1), fixed 252, no IDX filehdr, same as typ=RSF
 uxsort "fili1=dat1/idxtest,typ=LST,rcs=252,filo1=tmp1/idxf8x1,typ=IDXf8x1,isk1=0(4)"
 #===============================================================================
 #note - must read back using typ=RSF, since no index & fixed lth (no v)
 #
 # 5. typ=LST-->IDXf8v60x1 noindex(x1), varfmt, but all 252+4
 uxsort "fili1=dat1/idxtest,typ=LST,rcs=252,filo1=tmp1/idxf8v60x1,typ=IDXf8v60x1,isk1=0(4)"
 #===============================================================================
 #
 # 6. typ=LST-->IDXf8v60x1y4 noindex(x1), variable 60-252, depending on LNB (y4)
 uxsort "fili1=dat1/idxtest,typ=LST,rcs=252,filo1=tmp1/idxf8v60x1y4,typ=IDXf8v60x1y4,isk1=0(4)"
 #===============================================================================
 #
 echo "You can examine output files with 'uvhd', example for output#1 above"
 echo " "
 echo "uvhd tmp1/idxf8.dat v   <-- use 'v' option for IDXf8 files"
 echo "====================="
 #
 #Note - script may bomb on uxcp #6 if you only have 1 Micro Focus COBOL license
 #     - But you can add following to your profile to wait 1 minute (vs bomb)
 #
 # export ASLMFRETRY="5,15"
 # ========================

Goto: Begin this document , End this document , UVSI Home-Page

O8. 'uxsort' test/demo Indexed Seqntl Variable length files

input text file for uxsortIDXf3/uxsortIDXf8

 0010.0060 idxtest text file to create Var Lth Indexed files.
 0020.0064 - input to COBOL,uxcp,uxcopy to create IDXf1,3,8 files
 0030.0065 - 01-04=ISAM key, 05=separator, 06-09=size, 10-400 text
 0040.0067 - output record varies from 64 to 256, depending on 06-09
 0050.0080 - here is an 80 byte record------------------------------------------>
 0060.0160 - here is a 160 byte record...........................................
 0070.0240 - here is a 240 byte record (last record in file).....................
 ===============================================================================>

tmp1 subdir listings after uxsortIDXf3w


 #1. -rw-rw-r--    1 uvadm    users     2176 May 14 13:50 idxf3.dat
     -rw-rw-r--    1 uvadm    users     3072 May 14 13:50 idxf3.idx


 #2. -rw-rw-r--    1 uvadm    users     1148 May 14 13:50 idxf3v60.dat
     -rw-rw-r--    1 uvadm    users     3072 May 14 13:50 idxf3v60.idx


 #3. -rw-rw-r--    1 uvadm    users     1148 May 14 13:50 idxf3v60y4.dat
     -rw-rw-r--    1 uvadm    users     3072 May 14 13:50 idxf3v60y4.idx


 #4. -rw-rw-r--    1 uvadm    users     1920 May 14 13:50 idxf3x1


 #5. -rw-rw-r--    1 uvadm    users      892 May 14 13:50 idxf3v60x1


 #6. -rw-rw-r--    1 uvadm    users      892 May 14 13:50 idxf3v60x1y4

tmp1 subdir listings after uxsortIDXf8w


 #1. -rw-rw-r--    1 uvadm    users     4864 May 14 15:26 idxf8.dat


 #2. -rw-rw-r--    1 uvadm    users     4112 May 14 15:26 idxf8v60.dat


 #3. -rw-rw-r--    1 uvadm    users     4112 May 14 15:26 idxf8v60y4.dat


 #4. -rw-rw-r--    1 uvadm    users     1764 May 14 15:26 idxf8x1


 #5. -rw-rw-r--    1 uvadm    users      892 May 14 15:26 idxf8v60x1


 #6. -rw-rw-r--    1 uvadm    users      892 May 14 15:26 idxf8v60x1y4

Note	IDXf3 Indexed files have 2 partitions .dat & .idx IDXf8 files have only the .dat partition, but it includes indexes unless option 'x1' is specified for a non-indexed file

Please see 'uvhd' of IDXF3v60 on page 'O2' & IDXf8v60 on page 'O3'.

Please see uvcopy6.htm for IDXf3/IDXf8 test/demos (similar to the above), but for 'uxcopy' vs 'uxcp' or 'uxsort'.

Goto: Begin this document , End this document , UVSI Home-Page

P1. uvsort: RDW (Record Descriptor Word) variable length files

RDW (Record Descriptor Word) variable length files are often used to FTP variable length EBCDIC files from the mainframe to unix/linux/windows systems. uvsort provides for the following types of RDW files:

RDWz4

4 byte record prefix with 2 byte binary record size & 2 unused nulls
record size in prefix is the total record size including the prefix
2 byte record size is 'BIG-END' binary (like mainframe or SUN,AIX,DEC)
'little-end' machines (Intel,AMD) need to switch ends to perform math
RDWz4 is the default (typ=RDW defaults to typ=RDWz4)

RDWz2	option 'z2' specifies record prefix only 2 bytes (2 nulls omitted) record size in prefix is the data record size excluding the prefix
RDWz8	option 'z8' same as RDWz4, but bypasses any BDW prefixes

RDWn4

option 'n4' specifies 4 byte record prefix is Numeric digits (vs binary)
record size in prefix is the total record size including the prefix

test/demo files supplied in /home/uvadm/dat1

dat1/testLST	text file originally used to create testRDW
dat1/testRDW	RDW file created from testLST
dat1/testRDWe	RDW file, same as above in EBCDIC


 uvhd dat1/testRDW z4   <-- investigate RDW file (using uvhd)
 ====================     - only 1st of 8 records shown below

                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 ....DELL10 - Dell Inc.
             010044443322246662466222
             080045CC100D045CC09E3E00

Note	please see uvcp.htm#K1 thru K6 for extensive illustrations of RDW files and conversions of EBCDIC RDW files to ASCII text & fixed length files

Here in uvsort, we will give an example of sorting the ASCII testRDW file to a text file, and then sorting the EBCDIC testRDWe file & translating to ASCII at the same time.

sort RDW ASCII file to a text file


 uvsort "fili1=dat1/testRDW,rcs=64,typ=RDWz4,filo1=tmp/testz4,typ=LSTt,key1=0(6)"
 ================================================================================

text records sorted from RDW file

     DELL00 - Dell Inc.
     HP0000 - Hewlett Packard
     IBM000 - International Business Machines
     MFC000 - Micro Focus COBOL
     MS0000 - Microsoft Corp.
     REDHAT - Red Hat Linux
     SUN000 - Sun Microsystems Ltd
     UVSI00 - UV Software Inc.

Goto: Begin this document , End this document , UVSI Home-Page

P2. RDW (Record Descriptor Word) variable length files

ex#2 sort RDW EBCDIC file translating to ASCII

Instead of variable length text, we will create fixed length output records that are required for most COBOL input files. For this demo, we will convert to fixed 64 byte records & insert a LineFeed in the last byte so we can have it both ways (fixed length for COBOL, but LFs allow investigation with vi).


 uvsort "fili1=dat1/testRDWe,rcs=64,typ=RDWz4,filo1=tmp/testz4,typ=RST,tra=0(64)"
 ================================================================================

tra=0(64)

translates the records to ASCII.

typ=RST

output 'Record Sequential Terminated' (inserts LineFeed in last byte)

confirm conversion OK with uvhd


 uvhd tmp/testz4 r64h2     <-- display output file with uvhd
 =====================       - option r64 for record size
                             - option 'h2' for hex display to see LineFeed
                             - only 1st record shown below

                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 DELL10 - Dell Inc.                                             .
             4444332224666246622222222222222222222222222222222222222222222220
             45CC100D045CC09E3E000000000000000000000000000000000000000000000A

Confirmations shown by uvhd

Binary record prefixes have been dropped off, because output typ=RST extracts the data record only & omits the typ=RDW record prefix.
LineFeeds in last byte of each record (x'0A's).
Output is ASCII 1st char 'D' is x'44' here vs x'C4' on page 'P1' Translation performed by instruction 'tra=0(64)'.
See pages uvcp.htm for other instructions you can use to reformat data (mvc, clr, rep, tre, tru, trl, sel, del, etc...).

see DATAcnv1 for more RDW examples

Note	also see DATAcnv1.htm for more examples of data conversions stripping BDW/RDW record prefixes converting variable length files to fixed length translating from EBCDIC to ASCII, allowing for packed & signed fields

Goto: Begin this document , End this document , UVSI Home-Page

Q1. uvsort - unix sort utility with mainframe functionality

SyncSort to uvsort converter

For several years, 'uvsort' has had the ability to do just about everything that 'SyncSort' can do, but we had to manually code the equivalent uvsort parameters.

For several years, we have had a JCL converter that would convert any mainframe SORT or SYNCSORT parameters to the equivalent uvsort parameters, but the unix/linux SyncSort parameters are completely different.

Now (as of May 2010), we have automatic SyncSort & CoSort to uvsort converters. 'syncsort2uvsort1' will convert unix/linux SyncSort parameters to the equivalent uvsort parameters.

Here is our sample syncsort 'syncsort11' & the converted equivalent uvsort 'uvsort11'. Following the samples we will present the operating instructions to convert 1 at a time or all syncsorts in a directory. See similar samples & doc for CoSort conversions following the SyncSort presentation.

sample SyncSort

 syncsort \
 /fields custno 1 char 6, slsmn 11 char 2, idate 14 char 6, inv 21 char 8, \
         product 31 char 6, qty 39 zd 6, price 46 zd 7, amount 54 zd 9 \
 /condition oldsale idate gt '"500101"' and idate lt '"991232"' \
 /condition slsmn21 slsmn eq '"21"' and amount gt 20000 \
 /condition slsmn44 slsmn eq '"44"' and amount gt 20000 \
 /condition slsmn65 slsmn eq '"65"' and amount gt 20000 \
 /keys slsmn, amount \
 /infile $SORTIN1 stlf 64 /omit \ oldsale \
 /outfile $OUTDIR/slsmn21 stlf 64 /include slsmn21 \
 /outfile $OUTDIR/slsmn44 stlf 64 /include slsmn44 \
 /outfile $OUTDIR/slsmn65 stlf 64 /include slsmn65 \
 /end

equivalent uvsort

 # syncsort11 - uvsort replacing SyncSort
 #         - converted at UV_Software on 2010/05/01_15:57:22
 #         - call this parameter file as follows:
 # export SORTIN=...      #<-- define input file
 # export SORTOUT=...     #<-- define output file
 # export PF=...          #<-- define ParamterFile dirctory in profiles
 # uvsort $PF/syncsort11  #<-- execute uvsort on this parameter file
 # =====================
 fili01=${SORTIN1},typ=LST,rcs=64
 filo01=${OUTDIR}/slsmn21,typ=LSTt,rcs=64
 filo02=${OUTDIR}/slsmn44,typ=LSTt,rcs=64
 filo03=${OUTDIR}/slsmn65,typ=LSTt,rcs=64
 keys=(10,2,c,a,53,9,c,a)
 del1i00=13(6c)>500101,del2i00=13(6c)<991232
 sel1o01=10(2c):21,sel2o01=53(9z)>20000
 sel1o02=10(2c):44,sel2o02=53(9z)>20000
 sel1o03=10(2c):65,sel2o03=53(9z)>20000

Goto: Begin this document , End this document , UVSI Home-Page

Q2. SyncSort to uvsort converter

converting SyncSort to uvsort

If you have the May 2010 version of Vancouver Utilities, you could convert the sample pf/demo/syncsort11 to pf/demo/uvsort11, and then execute it on the supplied test file dat1/sales2. The relevant subdirs & files are:

 /home/uvadm/
 :-----bin/                   <-- Vancouver Utility executable programs
 :     :-----uvsort
 :-----pf/                    <-- VU supplied Parameter Files
 :-----pf/IBM/                <-- conversion jobs
 :     :-----syncsort2uvsort1 <-- syncsort to uvsort converter
 :-----pf/demo/               <-- demo Parameter Files
 :     :-----syncsort11       <-- sample syncsort parameters
 :     :-----uvsort11         <-- converted equivalent uvsort
 :-----dat1/                  <-- Data files subdir
 :     :-----sales2           <-- data file input for converted syncsort
 :-----tmp/                   <-- Temp subdir for output files
 :     :-----sales11
 :     :-----sales44          <-- 3 output files from uvsort11
 :     :-----sales65


 #1. Login uvadm --> /home/uvadm


 #2. uvcopy syncsort2uvsort1,fili1=pf/demo/syncsort11,filo1=pf/demo/uvsort11
     =======================================================================
     - convert sample SyncSort to uvsort equivalent parameters


 #3a. export SORTIN1=dat1/sales2  <-- export input file for uvsort11
      ==========================
 #3b. export OUTDIR=tmp           <-- export output subdir for uvsort11
      =================


 #4. uvsort pf/demo/uvsort11      <-- execute uvsort on parameter file uvsort11
     =======================

You could shorten the 'uvsort' command line if you export a $SYMBOL for the Parameter Files subdir, for example:


 #3c. export PF=/home/uvadm/pf/demo/
      ==============================


 #4a. uvsort $PF/uvsort11      <-- execute uvsort on parameter file uvsort11
      ===================

You would code 'PF' in your profile to define your own Parameter Files subdir. This would allow you to run your uvsorts from any location without having to enter long pathnames.

If you have hundreds of SyncSorts, you could convert all SyncSorts to equivalent uvsorts as follows:


 uvcopyx syncsort2uvsort1 syncsortdir uvsortdir uop=q0i7
 =======================================================

Goto: Begin this document , End this document , UVSI Home-Page

Q3. SyncSort to uvsort converter

Input data file for SyncSort/uvsort demo

          1         2         3         4         5         6
 1234567890123456789012345678901234567890123456789012345678901234
 cust#   slmn YYMMDD invoice  product    qty   price    amount
 ================================================================
 130140    21 990802 IN111001  HAM001  000020 0000101 000002020
 130140    21 000802 IN111001  SCR012  000021 0001001 000021021
 130140    21 010802 IN111001  CHR001  000022 0010001 000220022
 139923    35 980802 IN111002  TAB013  000023 0000201 000004623
 139923    35 990807 IN111002  TAB013  000024 0002001 000048024
 150825    44 010804 IN1122    HAM001  000025 0020001 000500025
 150825    44 020804 IN1122    HAX129  000026 0000301 000007826
 201120    44 030807 CR5234    WHIP75  000027 0003001 000081027
 223240    65 010816 CR955     HAM001  000028 0030001 000840028
 223240    65 020816 IN441     BBQ001  000029 0000101 000002929
 308685    21 020812 IN6605    SAW051  00001p 0000101 00000101p
 308685    21 030812 IN6605    WHIP75  00001q 0001001 00001101q
 308685    21 040812 CR8835    TAB013  00001r 0010001 00012001r
 315512    44 040805 IN2251    HAM001  00001s 0000201 00000261s
 315512    44 050805 IN2251    SAW051  00001t 0002001 00002801t
 315512    44 060805 IN2255    WHIP75  00001u 0020001 00030001u
 400002    85 990812 CR245     HAX129  00001v 0000101 00000161v
 406082    35 010815 IN33001   BBQ001  00001w 0001001 00001701w
 406082    35 020815 IN33001   TAB013  00001x 0010001 00018001x
 406082    65 030816 IN441     HAM001  00001y 0000101 00000191y

Output data files for SyncSort/uvsort demo

 130140    21 000802 IN111001  SCR012  000021 0001001 000021021
 130140    21 010802 IN111001  CHR001  000022 0010001 000220022

 201120    44 030807 CR5234    WHIP75  000027 0003001 000081027
 150825    44 010804 IN1122    HAM001  000025 0020001 000500025

 223240    65 010816 CR955     HAM001  000028 0030001 000840028

notes re I/O data files

Note the 3 separate output files for salesmen 21,44,& 65 (defined by SORTOF01, SORTOF02, SORTOF03).
Note the selections by slsmn# & amount > $200.00. Satisfy yourself that the selections are correct
Note that the amount field for the last 10 records of the input file are negative & therefore fail the test for 'sel2o01=53(9z)>20000'.

Goto: Begin this document , End this document , UVSI Home-Page

R1. uvsort - unix sort utility with mainframe functionality

separating SyncSorts from scripts

SyncSort parameters are usually embedded in scripts, but the uvsort converter expects the uvsort parameters to be in a separate file, which is called from the script.

'syncsortsplit1' will split the scripts directory to separate directories of syncsort parameters & modified scripts with 1 line calls to the equivalent uvsort parameters.

Here is a sample script with 2 syncsorts and the outputs of syncsortsplit1 and syncsort2uvsort1.

sample script with 2 syncsorts

 # script11 - sample script with SyncSort to demo syncsortsplit1
 #          - see doc at www.uvsoftware.ca/uvsort.htm#Q1
 # sort#1 - by product & amount descending
 export SORTIN1=$RUNDATA/dat1/sales2
 export SORTOUT=$RUNDATA/tmp/sales2product
 syncsort \
 /fields custno 1 char 6, slsmn 11 char 2, product 31 char 6, amount 54 zd 9 \
 /keys product, amount descending \
 /infile $SORTIN1 stlf 64 \
 /outfile $SORTOUT stlf 64 \
 /end
 #
 # sort#2 - by slsmn, product,& amount descending
 export SORTIN1=$RUNDATA/dat1/sales2
 export SORTOUT=$RUNDATA/tmp/sales2slsmn
 syncsort \
 /fields custno 1 char 6, slsmn 11 char 2, product 31 char 6, amount 54 zd 9 \
 /keys slsmn, product, amount descending \
 /infile $SORTIN1 stlf 64 \
 /outfile $SORTOUT stlf 64 \
 /end

script output (syncsorts replaced by calls)

 # script11 - sample script with SyncSort to demo syncsortsplit1
 #          - see doc at www.uvsoftware.ca/uvsort.htm#Q1
 # sort#1 - by product & amount descending
 export SORTIN1=$RUNDATA/dat1/sales2
 export SORTOUT=$RUNDATA/tmp/sales2product
 uvsort $PF/script11_sort01
 #=========================
 #
 # sort#2 - by slsmn, product,& amount descending
 export SORTIN1=$RUNDATA/dat1/sales2
 export SORTOUT=$RUNDATA/tmp/sales2slsmn
 uvsort $PF/script11_sort02
 #=========================

Goto: Begin this document , End this document , UVSI Home-Page

R2. separating SyncSorts from scripts

syncsort parameter file outputs

 # script11_sort01
 syncsort \
 /fields custno 1 char 6, slsmn 11 char 2, product 31 char 6, amount 54 zd 9 \
 /keys product, amount descending \
 /infile $SORTIN1     stlf 64 \
 /outfile $SORTOUT    stlf 64 \
 /end

 # script11_sort01
 syncsort \
 /fields custno 1 char 6, slsmn 11 char 2, product 31 char 6, amount 54 zd 9 \
 /keys product, amount descending \
 /infile $SORTIN1     stlf 64 \
 /outfile $SORTOUT    stlf 64 \
 /end

equivalent uvsort parameter files

 # # script11_sort01
 # script11_sort01 - uvsort replacing SyncSort
 #         - converted at UV_Software on 2010/05/10_15:53:34
 #         - call this parameter file as follows:
 # export SORTIN=...      #<-- define input file
 # export SORTOUT=...     #<-- define output file
 # export PF=...          #<-- define ParamterFile dirctory in profiles
 # uvsort $PF/script11_sort01   #<-- execute uvsort on this parameter file
 # ====================
 fili01=${SORTIN1},typ=LST,rcs=64
 filo01=${SORTOUT},typ=LSTt,rcs=64
 keys=(30,6,c,a,53,9,c,d)

 # # script11_sort02
 # script11_sort02 - uvsort replacing SyncSort
 #         - converted at UV_Software on 2010/05/10_15:53:34
 #         - call this parameter file as follows:
 # export SORTIN=...      #<-- define input file
 # export SORTOUT=...     #<-- define output file
 # export PF=...          #<-- define ParamterFile dirctory in profiles
 # uvsort $PF/script11_sort02   #<-- execute uvsort on this parameter file
 # ====================
 fili01=${SORTIN1},typ=LST,rcs=64
 filo01=${SORTOUT},typ=LSTt,rcs=64
 keys=(10,2,c,a,30,6,c,a,53,9,c,d)

Goto: Begin this document , End this document , UVSI Home-Page

R3. separating SyncSorts from scripts

syncsortsplit1 - subdirs for demo

If you have the May 2010 version of Vancouver Utilities, you could demo syncsortsplit1 and syncsort2uvsort1 as shown below. Relevant subdirs/files are:

 /home/uvadm/
 :-----bin/                   <-- Vancouver Utility executable programs
 :     :-----uvsort
 :-----pf/                    <-- VU supplied Parameter Files
 :-----pf/IBM/                <-- conversion jobs
 :     :-----syncsortsplit1   <-- separate syncsorts from scripts
 :     :-----syncsort2uvsort1 <-- syncsort to uvsort converter
 :-----sf/demo/               <-- demo scripts
 :     :-----script11         <-- sample script with 2 syncsorts
 :-----dat1/                  <-- Data files subdir
 :     :-----sales2           <-- data file input for converted syncsort
 :-----tmp1/
 :     :-----script11         <-- copy script here for demo input
 :-----tmp2/
 :     :-----script11     <-- script output syncsorts replaced by calls to uvsort
 :-----tmp3/
 :     :-----script11_sort01  <-- syncsorts extracted from script
 :     :-----script11_sort02
 :-----tmp4/
 :     :-----script11_sort01  <-- converted to uvsorts
 :     :-----script11_sort02

Goto: Begin this document , End this document , UVSI Home-Page

R4. separating SyncSorts from scripts

syncsortsplit1 - execute demo


 #1. Login uvadm --> /home/uvadm


 #2. mkdir tmp1 tmp2 tmp3 tmp4   <-- make subdirs for demo
     =========================


 #3. cp sf/demo/script11 tmp1    <-- isolate demo input in separate directory
     ========================        (could be multiple scripts, we only have 1)


 #4. uvcopy syncsortsplit1,fild1=tmp1,fild2=tmp2,fild3=tmp3
     ======================================================
     - split scripts to scripts with calls to uvsort & syncsort parameters


 #4a. vi tmp2/script11          <-- observe output script
      ================            - syncsorts replaced by calls to uvsort


 #4b. vi tmp3/script11_sort01   <-- observe syncsort prmtrs in sep subdir
      =======================
 #4c. vi tmp3/script11_sort02   <-- could be multiple syncsorts in 1 script
      =======================       (we have 2 in our demo)


 #5. uvcopyx syncsort2uvsort1 tmp3 tmp4 uop=q0i7
     ===========================================
     - convert syncsort parameters to uvsort equivalent


 #5a. vi tmp4/script11_sort01   <-- observe uvsort prmtrs in sep subdir
      =======================
 #5b. vi tmp4/script11_sort02
      =======================


 #6a. export PF=/home/uvadm/tmp4   <-- export location: uvsort Parameter Files
      ==========================


 #6b. export RUNDATA=/home/uvadm   <-- export location: superdir for execution
      ==========================


 #7. ksh tmp2/script11          <-- execute script calling 2 uvsorts
     =================


 #8a. vi tmp/sales2product      <-- observe output from 1st sort
      ====================


 #8b. vi tmp/sales2slsmn        <-- observe output from 2nd sort
      ==================

Goto: Begin this document , End this document , UVSI Home-Page

S1. uvsort - unix sort utility with mainframe functionality

CoSort to uvsort converter

In May 2010 (in addition to the SyncSort converter), we also developed a 'CoSort' to 'uvsort' converter. 'cosort2uvsort1' will convert unix/linux CoSort parameters to the equivalent uvsort parameters.

Here is our sample CoSort 'cosort21' & the converted equivalent uvsort 'uvsort21'. On the next page we will present the operating instructions to convert 1 at a time or all CoSorts in a directory.

sample CoSort

 /sort
 /infiles=(SORTIN)
 /length=64
 /field=(custno,  position=1, size=6)
 /field=(slsmn,   position=11, size=2)
 /field=(idate,   position=14, size=6)
 /field=(inv,     position=21, size=8)
 /field=(product, position=31, size=6)
 /field=(qty,     position=39, size=6, numeric)
 /field=(price,   position=46, size=7, numeric)
 /field=(amount,  position=54, size=9, numeric)
 /condition=(oldsale, test=(idate gt "500101" and idate lt "991232"))
 /condition=(slsmn21, test=(slsmn eq "21" and amount gt 20000))
 /condition=(slsmn44, test=(slsmn eq "44" and amount gt 20000))
 /condition=(slsmn65, test=(slsmn eq "65" and amount gt 20000))
 /key=(slsmn,  ascending)
 /key=(amount, descending)
 /omit=(condition=oldsale)
 /outfile=(SORTOF01) /include=(condition=slsmn21)
 /outfile=(SORTOF02) /include=(condition=slsmn44)
 /outfile=(SORTOF03) /include=(condition=slsmn65)

equivalent uvsort

 # cosort21 - uvsort replacing COSORT
 #         - converted at UV_Software on 2010/05/01_16:41:56
 #         - call this parameter file as follows:
 # export SORTIN=...      #<-- define input file
 # export SORTOUT=...     #<-- define output file
 # export PF=...          #<-- define ParamterFile dirctory (in profiles)
 # uvsort $PF/cosort21   #<-- execute uvsort on this parameter file
 # ====================
 fili01=${SORTIN},typ=RSF,rcs=00064
 filo01=${SORTOF01},typ=RSF,rcs=64
 filo02=${SORTOF02},typ=RSF,rcs=64
 filo03=${SORTOF03},typ=RSF,rcs=64
 keys=(10,2,c,a,53,9,c,a)
 del1i00=13(6c)>500101,del2i00=13(6c)<991232
 sel1o01=10(2c):21,sel2o01=53(9z)>20000
 sel1o02=10(2c):44,sel2o02=53(9z)>20000
 sel1o03=10(2c):65,sel2o03=53(9z)>20000

Goto: Begin this document , End this document , UVSI Home-Page

S2. CoSort to uvsort converter

converting CoSort to uvsort

If you have the May 2010 version of Vancouver Utilities, you could convert the sample pf/demo/cosort21 to pf/demo/uvsort21, and then execute it on the supplied test file dat1/sales2. The relevant subdirs & files are:

 /home/uvadm/
 :-----bin/                   <-- Vancouver Utility executable programs
 :     :-----uvsort
 :-----pf/                    <-- VU supplied Parameter Files
 :-----pf/IBM/                <-- conversion jobs
 :     :-----cosort2uvsort1   <-- CoSort to uvsort converter
 :-----pf/demo/               <-- demo Parameter Files
 :     :-----cosort21         <-- sample CoSort parameters
 :     :-----uvsort21         <-- converted equivalent uvsort
 :-----dat1/                  <-- Data files subdir
 :     :-----sales2           <-- data file input for converted CoSort
 :-----tmp/                   <-- Temp subdir for output files
 :     :-----sales11
 :     :-----sales44          <-- 3 output files from uvsort21
 :     :-----sales65


 #1. Login uvadm --> /home/uvadm


 #2. uvcopy cosort2uvsort1,fili1=pf/demo/cosort21,filo1=pf/demo/uvsort21
     ===================================================================
     - convert sample CoSort to uvsort equivalent parameters


 #3a. export SORTIN1=dat1/sales2  <-- export input file for uvsort21
      ==========================
 #3b. export OUTDIR=tmp           <-- export output subdir for uvsort21
      =================


 #4. uvsort pf/demo/uvsort21      <-- execute uvsort on parameter file uvsort21
     =======================

You could shorten the 'uvsort' command line if you export a $SYMBOL for the Parameter Files subdir, for example:


 #3c. export PF=/home/uvadm/pf/demo/
      ==============================


 #4a. uvsort $PF/uvsort21      <-- execute uvsort on parameter file uvsort21
      ===================

You would code 'PF' in your profile to define your own Parameter Files subdir. This would allow you to run your uvsorts from any location without having to enter long pathnames.

If you have hundreds of CoSorts, you could convert all CoSorts to equivalent uvsorts as follows:


 uvcopyx cosort2uvsort1 cosortdir uvsortdir uop=q0i7
 ===================================================

Goto: Begin this document , End this document , UVSI Home-Page

T1. uvsort vs MFSORT (Micro Focus standalone sort utility)

mfsort to uvsort converter (not yet)

I have not yet (as of July 2010) written an 'mfsort to uvsort' converter, but it will be easy to do when the need arises, since mfsort is similar to syncsort & cosort (see above).

In this section I will present a sample mfsort & equivalent uvsort, and document how I performed some timings between the two. I have not yet been able to get timings for syncsort & cosort since I do not have those products.

sample mfsort

 # mfsort2 - sample MFSORT (Micro Focus standalone sort utility)
 #         - by Owen Townsend, UV Software, July 02/2010
 #         - this script stored at /home/uvadm/sf/demo/mfsort2 (in uvadm PATH)
 #
 # See www.uvsoftware.ca/uvsort.htm#T1 doc re demo MFSORT vs uvsort timings
 # - uvsort.htm#T2 shows how to setup big file for sort timings
 #
 echo "mfsort2, input dat1/cm1big, Begun at: $(date)"
 mfsort "sort fields(1,6,ch,a) \
         use dat1/cm1big org sq record f,256 \
         give dat2/cm1bigs"
 echo "mfsort2, output dat2/cm1bigs, Ended at: $(date)"
 ls -l dat1/cm1big
 ls -l dat2/cm1bigs

equivalent uvsort

 # uvsort2 - sample uvsort (UV Software Vancouver Utilities)
 #         - for comparison to MFSORT (Micro Focus standalone sort utility)
 #         - by Owen Townsend, UV Software, July 02/2010
 #         - this script stored at /home/uvadm/sf/demo/uvsort2 (in uvadm PATH)
 #
 # See www.uvsoftware.ca/uvsort.htm#T1 doc re demo uvsort vs MFSORT timings
 # - uvsort.htm#T2 shows how to setup big file for sort timings
 #
 echo "uvsort2, input dat1/cm1big, Begun at: $(date)"
 uvsort "fili1=dat1/cm1big,rcs=256,typ=RSF,filo1=dat2/cm1bigs,key1=0(6)"
 #======================================================================
 echo "uvsort2, output dat2/cm1bigs, Ended at: $(date)"
 ls -l dat1/cm1big
 ls -l dat2/cm1bigs

Goto: Begin this document , End this document , UVSI Home-Page

T2. sort time comparison - uvsort vs mfsort

script to make BIG file

 #!/bin/ksh
 # makebig1 - make a big file by concatenating spcfd no of times
 echo "makebig1 - make a big file by concatenating spcfd no of times"
 f1="$1"; f2="$2"; integer n="$3"
 if [[ -f "$f1" && -n "$f2" && -n "$n" ]]; then :
    else echo "usage: makebig1 file1 file2 ##  <-- arg1/2=filenames, arg3 number"
         echo "       ======================="
         exit 9; fi
 # if output file does not exist, create empty file now
 if [[ ! -f $f2 ]]; then >$f2; fi
 integer i=0
 while ((i < n))
   do cat $f1 >> $f2
      ((i+=1))
   done
 echo "completed $n concats of $f1 to $f2"
 exit 0

directories for sort time tests

For these tests, I created a separate user 'testbig' with homedir in file system /home4/... where I had lots of space to create & sort BIG files. Here are the relevant directories & files:

 /home/testbig
 :-----dat1           <-- subdir for input files
 :     :-----cm1      <-- 8K demo file copied from /home/uvadm/dat1/custmas1
 :     :                - 32 records of 256 bytes = 8192
 :     :-----cm1big   <-- 1 Gig file created by concatenating cm1 125,000 times
 :     :                - 8192 * 125,000 = 1,024,000,000
 :-----dat2           <-- subdir for output files
 :     :-----cm1bigs  <-- output file from uvsort or mfsort
 :     :                - see scripts mfsort2 & uvsort2 listed on prior page


 uvhd dat1/custmas1 r256 <-- use 'uvhd' to display 1st record of test file
 =======================   - customer N&A with 24 * 5 byte packed sales history

                       1         2         3         4         5         6
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 130140    EVERGREEN MOTORS LTD.    1815 BOWEN ROAD          NANA
             3333332222454545444244545524542222233332445442544422222222224444
             130140000056527255E0DF4F230C44E0000181502F75E02F140000000000E1E1
          64 IMO          BC V9S1H1    604-754-5531                  ..4V|...
             4442222222222442535343222233323332333322222222222222222201357000
             9DF00000000002306931810000604D754D55310000000000000000000246C000
         128 .........W0....`........)X}..f3.....\.................4V}...f...
             0000000005300016000000002570063100095000000000000000013570016000
             0C0000C0270D0540C0000C0098D0263C0444C0000C0000C0000C0246D0056C00
         192 ...............................f.....<........f.C
             0000008900000000880000000018000680001300000000694222222222222222
             00C0026C0000C0023C0000C0083C0056D0012C0000C0016D3000000000000000

Goto: Begin this document , End this document , UVSI Home-Page

T3. sort time comparison - uvsort vs mfsort

make BIG file & perform timing tests

First setup a user with homedir in a file system with lots of space. We setup userid 'testbig' at /home4/testbig & subdirs as illustrated above. You could setup the subdirs in your homedir if enough space in /home/... Our biggest test is an 8 gig file so you need 24 gig (input, output, work).


 #2. Login as testbig --> /home4/testbig (or login as yourself)
     ================


 #3. cp /home/uvadm/dat1/custmas1 dat1/cm1 <-- copy 8K demo file
     =====================================   - 32 records * 256 = 8192


 #4. makebig1 dat1/cm1 dat1/cm1big 125000  <-- run script to make big file
     ====================================    - 8192 * 125000 = 1,024,000,000


 #5a. uvsort2    <-- run uvsort script (see listing on page 'T1' above)
      =======      - see timings below


 #5b. uvsort "fili1=dat1/cm1big,rcs=256,typ=RSF,filo1=dat2/cm1bigs,key1=0(6)"
      =======================================================================
      - alternative, run uvsort directly, all prameters on cmd line (vs script)

      100703:184706:uvsort: uvsort fili1=dat1/cm1big,filo1=dat2/cm1bigs
      100703:184711:uvsort: inrecs  = 1000000 fili01 dat1/cm1big
      100703:184718:uvsort: inrecs  = 2000000 fili01 dat1/cm1big
      100703:184723:uvsort: inrecs  = 3000000 fili01 dat1/cm1big
      100703:184730:uvsort: inrecs  = 4000000 fili01 dat1/cm1big
      100703:184730:uvsort: EOF fili1 4000000 rds, 1024000000 size; dat1/cm1big
      100703:184730:uvsort: sort complete, used 64 megabytes, sort output begin
      100703:184733:uvsort: outrecs = 1000000 filo01 dat2/cm1bigs
      100703:184736:uvsort: outrecs = 2000000 filo01 dat2/cm1bigs
      100703:184739:uvsort: outrecs = 3000000 filo01 dat2/cm1bigs
      100703:184742:uvsort: outrecs = 4000000 filo01 dat2/cm1bigs
      100703:184747:uvsort: EOF filo1 4000000 wrts, 1024000000 size; dat2/cm1bigs
      uvsort start 2010/07/03_18:47:06 end 18:47:47 elapsed 00:41


 #6a. mfsort2    <-- run MFSORT script (see listing on page 'T1' above)
      =======


 #6b. mfsort "sort fields(1,6,ch,a) \
              use dat1/cm1big org sq record f,256 \
              give dat2/cm1bigs"
      =============================================
      - alternative, run mfsort directly, all prameters on cmd line (vs script)

      mfsort2, input dat1/cm1big, Begun at: Sat Jul  3 18:48:04 PDT 2010
      mfsort2, output dat2/cm1bigs, Ended at: Sat Jul  3 18:48:53 PDT 2010

Note	MFSORT time = (12:43:55 - 12:43:09) = 49 seconds uvsort time = 41 seconds (almost 20% faster)

Goto: Begin this document , End this document , UVSI Home-Page

T4. sort time comparison - uvsort vs mfsort

sort times for 4 GIG file


 #1. makebig1 dat1/cm1 dat1/cm1big 500000  <-- run script to make big file
     ====================================    - 8192 * 500000 = 4,096,000,000


 #2. uvsort2    <-- run uvsort script (see listing on page 'T1' above)
     =======      - console display log similar to 1 gig test above
                  - time = 6 minutes, 36 seconds


 #3. mfsort2    <-- run MFSORT script (see listing on page 'T1' above)
     =======      - console display log similar to 1 gig test above
                  - time = 7 minutes, 04 seconds

Note	on a 4 gig sort, uvsort is about 15% faster than MFSORT

sort times for 8 GIG file


 #1. makebig1 dat1/cm1 dat1/cm1big 1000000  <-- run script to make big file
     =====================================    - 8192 * 1000000 = 8,192,000,000


 #2. uvsort2    <-- run uvsort script (see listing on page 'T1' above)
     =======      - console display log similar to 1 gig test above
                  - time = 15 minutes, 01 second


 #3. mfsort2    <-- run MFSORT script (see listing on page 'T1' above)
     =======

      mfsort2, input dat1/cm1big, Begun at: Sat Jul  3 19:12:26 PDT 2010
      SORT013U: I/O error on dataset 'dat1/cm1big'
      SORT014U: Status = 9/194 (FILE TOO BIG)
      SORT020U: SORT(EXTSM) failed - sort engine status = 9/194

option for files > 4 Gig

For MFSORT to sort files > 4 Gig, you need to setup a control file 'extfh.cfg' with option 'FILEMAXSIZE=8', and you need to export its location using environmental variable 'EXTFH', for example:

 # extfh.cfg - File Handler Configuration file for Micro Focus COBOL
 #           - 'FILEMAXSIZE=8' required for 'mfsort' to sort files > 4 Gig
 #           - see more at www.uvsoftware.ca/mvscobol.htm#5D2 OR at:
 #             http://supportline.microfocus.com/Documentation/books/sx40sp/fh...
 #
 # export EXTFH=home/uvadm/ctl/extfh.cfg  #<-- define location of this file
 #
 #---------------------------------------------------------------------------
 [XFH-DEFAULT]
 FILEMAXSIZE=8
 IDXNAMETYPE=2
 #---------------------------------------------------------------------------

Goto: Begin this document , End this document , UVSI Home-Page

T5. sort time comparison - uvsort vs mfsort

rerun 8 gig mfsort test

For MFSORT to sort files > 4 Gig, we need to setup a control file 'extfh.cfg' with option 'FILEMAXSIZE=8' (see listing above), and we need to export its location using environmental variable 'EXTFH'.

If you have Vancouver Utilities installed you could use the supplied extfh.cfg & export its location (as suggested in the listing above). If you do not have Vancouver Utilities, you might proceed as follows:


 #3a. mkdir ctl         <-- make directory for control file
      =========


 #3b. vi ctl/extfh.cfg  <-- create extfh.cfg with 'FILEMAXSIZE=8' (see above)
      ================


 #3c. export EXTFH=ctl/extfh.cfg  <-- export control file for MFSORT
      ==========================


 #3d. export TMPDIR=/tmp  <-- optional variable for sort work files
      ==================    - defaults to current directory if unspecified


 #3e. mfsort2    <-- RE-RUN MFSORT script (see listing on page 'T1' above)
      =======

      mfsort2, input dat1/cm1big,   Begun at: Wed Jul 7 16:50:24 PDT 2010
      mfsort2, output dat2/cm1bigs, Ended at: Wed Jul 7 17:07:52 PDT 2010
      Elapsed time = 17 minutes, 28 seconds

conclusions

uvsort is 15 to 20% faster than mfsort, which is probably not enough to justify buying the Vancouver Utilities if that were the only reason.

'uvsort' is just 1 part of the Vancouver Utilities, which are designed to replace mainframe utilities on Unix/Linux/windows systems (mainframe utilities such as SORT,IDCAMS,IEBGENER,FILEAID,EASYTRIEVE,etc).

Be sure to try out the 1 FREE utility 'uvhd', which you may download from www.uvsoftware.ca/libuvhd.htm. uvhd is all my customers favorite utility.

'uvhd' is a an interactive file investigation utility with 19 help screens. It displays files with packed/binary fields in vertical hexadecimal & prompts for commands to browse, search, select, update, print, etc.

Goto: Begin this document , End this document , UVSI Home-Page

Visitor Counters for ThisYear and LastYear

ThisYear=000192 (J=0,F=0,M=0,A=0,M=0,J=0,J=170,A=22,S=0,O=0,N=0,D=0)
LastYear=001778 (J=125,F=142,M=151,A=125,M=124,J=130,J=126,A=146,S=153,O=209,N=180,D=167)