'uvcp' is the command line version of the 'uvcopy' utility. You can reformat files with a 1 line command. 'uvcp' does not require a separate parameter file as does the more powerful 'uvcopy'.

A1. Overview of uvcp & uxcp
- uxcp is an alternate version of uvcp to support
  Indexed Sequential Variable record length files
  compatible with Micro Focus COBOL IDXFORMAT3
B1. Introductory examples
C1. FUNCTION summary & Interrupt processing
D1. File definition parameters (fili1/filo1, typ, rcs, isk)
D2.  ISAM option for Duplicate keys Allowed on first & secondary keys
D3.  No need to specify ISAM keys for input files (read from .idx partition)
- ISAM keys copied to output file if both input & output typ=ISF
- option 'k' for key of reference (read sequence)
E1. I/O File Declaration - 3 methods
 (command line, environmental variables,& parameter file)
F1. File types - typ=LST,RSF,ISF,RDW,STL,IDX
 LST=Line Seqntl Termninated, RSF=Record Seqntl Fixed, ISF=Indexed Seqntl Fixed
 RDW=Record Descriptor Word, STL=STandard Language AIX COBOL
F3.  Indexed Sequential FIXED-length record files (typ=ISF)
F4.  Indexed Sequential VARIABLE-length record files (typ=IDX)
 compatible with Micro Focus COBOL IDXFORMAT1/3/8
 requires special version of uvsort, named 'uxsort'
F5.  options for file typ=LST... (text files)
F6.  file typ options applicable to multiple file types
F7.  file typ options unique to typ=RDW files
F8.  file typ options unique to typ=RSV & typ=RSR
F9.  file typ options for FIXED-length Indexed files
F10.   file typ options for VARIABLE-length Indexed files
F11.   output record size defaults for variable length files
G1. Run options (rop=a1b2c3...z9999)
H1. Select & Delete records (sel/del)
- option u1 to code blanks as underscores
- option u2 to code commas as at-signs
I1. Record Sequential Variable (typ=RSV) files
- compatible with Micro Focus COBOL IDXFORMAT3 files
- also applies to the data partition of typ=IDX files

Goto:   Begin this document End this document UVSI Home-Page

uvcp Contents (continued)

J1. Test/demo Indexed Seqntl Variable length files
- convert supplied text file to typ=IDX & back to typ=LST
J2.  uvhd (hexdump display) of IDXf3 file
J3.  uvhd (hexdump display) of IDXf8 file
J4.  IDXf3/IDXf8 file header & record header layout
J5.  scripts to test/demo file typ=IDX options (f,v,x,y).
 uxcpIDXf3w - write various IDXf3 formats, run prior to uxcpIDXf3r
 uxcpIDXf3r - read back IDXf3 files & convert back to text files
 uxcpIDXf3c - copy IDXf3 files to same output file types & record-sizes
 uxcpIDXf8w - write various IDXf8 files for read back by uxcpIDXf8r
 uxcpIDXf8r - read back various IDXf8 files written by uxcpIDXf8w
 uxcpIDXf8c - copy IDXf8 files to same output file types & record-sizes
J9. tmp1 subdir listings of files created by above scripts
K1. typ=RDW (default typ=RDWz4) Record Descriptor Word variable length files
- 4 byte record prefix with 2 byte binary record size & 2 unused nulls
- using 'uvhd' to display & convert EBCDIC RDW files to ASCII text
- using 'uvcp' to convert EBCDIC RDW files to ASCII fixed length files
K6.  typ=RDWz2 - uvcp conversion for 2 byte record prefix (vs 4 above)
L1. INSTRUCTION summary
L2.  instruction formats & operand areas
M0. Instruction details
- mvc,mvn,edt,clr,tra,tre,trl,tru,trp,trb,add,sub,mpy,div,pac,unp,seq
  anc,orc,chx,hxc,cmc,skp,cmn,rep,scn,sqz,tim,put
N1. Instruction options (repeat, op length increment inhibit, etc)
O1. special working storages 'a' 'b' 'w' 'x' 'y'
P1. uvcp restrictions
Q1. example #1 - reformatting records
R1. example #2 - processing multiple same size contiguous fields
S1. Using 'mkscript' to generate 'uvcp's to copy & reorganize directories
 of data files (re-sequences & drops deleted records in ISAM files)
T1. Converting TEXT files to FIXED record length files
- uvcp with input typ=LST & output typ=RSF
T4. Converting FIXED record length files to TEXT files
- uvcp with input typ=RSF & output typ=LST
U0. uvcpfix1 & uvcpfix2 - scripts calling the 'uvcp' utility that make it
 easier to enter the commands (vs calling uvcp directly).
 The scripts copy the data file to tmp/... You can then inspect & if OK,
 copy the tmp/ file to the desired location or overwrite the input file.

Goto:   Begin this document End this document UVSI Home-Page

uvcp Contents (continued)

U1. uvcpfix1 Ex#1 - copy Text files changing file-type, record-size,& reformat
U2.  uvcpfix2 Ex#2 - copy Fixed-Length records changing type, size,& reformat
U3.  uvcpfix1 script listing - copy/modify Text record files (LineFeeds)
U4.  uvcpfix2 script listing - copy/modify Fixed-Length records (No LineFeeds)
V1. demo file typ=EOR create output files with EBCDIC End-Of-Record marker x'FF01'
EORt1 - option 't1' translates to EBCDIC
EORt4 - option 't4' shortens record at last non-blank (EBCDIC x'40')
EORt5 - t5=(t1+t4) translates to EBCDIC & shortens record at last non-blank
W1. demo file typ=STL STandard Language file system (used by AIX CBOL)
STLs - STL Sequential files (input & output)
STLi - STL Indexed files (input only)
STLr - STL Relative record files (input only)
W2.  Converting STL to other file types (RSF,LST,RDW,etc)
- may convert STLi & STLr to STLs (only output for typ STL)
W3.  STL file formats - sample uvhd display in vertical hexadecimal
- including file header records, System records,& Deleted records
- illustrating the record prefixes for 3 STL file types
W5.  STL Indexed file - sample uvhd display in vertical hexadecimal
W7.  STL Relative file - sample uvhd display in vertical hexadecimal
W9. uvcp Run OPtions to prefix output with seq# & recsize
- use to recover data from corrupted files to recreate Indexed
  or Relative files, rebuilding indexes.
W10. sample uvcp file copies with test data & recovery options.
- illustrating output records with sequence#s & record sizes

Goto:   Begin this document End this document UVSI Home-Page

A1. uvcp Introduction & Overview

'uvcp' is a mini version of the uvcopy utility that can accept all it's parameters on the command line rather than requiring a separate parameter file as does uvcopy. This makes it convenient for use within scripts to perform jobs that do not require the power of uvcopy.

The program can perform record selection, reformatting, translation, and field conversion such as packed to zoned or to binary. uvcp is more than adequate as a replacement for the mainframe DATA utility which has a limited record selection & record reformatting capabilities.

uvcp is driven by 'keyword=argument's which may be entirely specified on the command line for simple file copies. For more complex operations the command line would specify the name of a parameter file which would contain any lengthy list of instructions, but it is better to use the uvcopy utility in this situation.

Note that 'uvsort' is compatible with 'uvcp'. uvsort uses the same functions and instructions, but also allows you to specify 'sort keys'.

alternate version 'uxcp' for IDXFORMAT3 files

'uxcp' is an alternate version of uvcp that includes support for Micro Focus COBOL IDXFORMAT3 files (Indexed Sequential Variable length records). You must execute uxcp to read or write 'typ=IDX' files. Note that Fixed length Indexed files (compatible with Micro Focus COBOL IDXFORMAT1) are supported by both uvcp & uxcp.

'uxcp' must be compiled with the Micro Focus 'cob' command. Please see install.doc which describes the use of the 'cobccuv' script which makes it easy to compile Vancouver Utility programs using the 'cob' command.

Goto:   Begin this document End this document UVSI Home-Page

B1. uvcp INTRODUCTORY EXAMPLES

  1. Given: file 'dat1/sales1' of 64 byte records (written by a COBOL program) has no linefeeds & therefore cannot be edited, displayed,or printed with the usual UNIX or MSDOS tools.

    Required: copy the file to 'tmp/sales1' appending linefeeds so it can be
              investigated with the editor.

    Solution: uvcp "fili1=dat1/sales1,rcs=64,typ=RSF,filo1=tmp/sales1,typ=LST"
              ===============================================================
Note
  • input 'typ=RSF' specifies Record Seqntl Fixed record sizes
    & no LineFeeds are present (typical COBOL file)
  • output 'typ=LST' specifies Line Seqntl Terminated records
    which can be easily edited, displayed, or printed.
  1. Given: the same file as in example 1 above.

    Required: translate the file to UPPER case EBCDIC for export to an
              IBM or Unisys S80 mainframe.

    uvcp "fili1=dat1/sales1,rcs=64,typ=RSF,filo1=tmp/sales1,tru=0(64),tre=0(64)"
    ============================================================================
Note
  • 'tru=0(64)' translates each 64 byte record to UPPER case
  • 'tre=0(64)' translates each 64 byte record to EBCDIC
  1. required - backup an indexed master file to a unix sequential file - dat1/custmas1 is an Indexed Fixed rcsz file of 256 byte records - copy to bak1/custmas1 (Unix,Sequential,Fixed same rcsz)


 solution: uvcp "fili1=dat1/custmas1,typ=ISF,rcs=256,filo1=bak1/custmas1,typ=RSF"
           ======================================================================
  1. required - reload the indexed file with key#1 1-6 & key#2 61-75


 uvcp "fili1=bak1/custmas1,typ=RSF,rcs=256,filo1=dat1/caustmas1,typ=ISF,isk1=0(6),isk2=59(15)"
 ================================================================================
Note
  • ISAM key parameter is 'isk' not 'key' because 'key' is reserved
    for the sort key specification in the uvsort utility
  • quotes are required (by the unix shell) around parameters
    when parenthesis are present as for 'isk' & 'mvc' above

Goto:   Begin this document End this document UVSI Home-Page

B2. uvcp INTRODUCTORY EXAMPLES (continued)

  1. given - a file of name & address updates has been prepared with a text editor (Unix Line Variable records with linefeeds)

    required - convert the variable length records to fixed length 80 byte
               records as required for existing programs.
             - the records should have a linefeed in col 80 so the text
               editor can still be used to view (& carefully update the file)
               (will create a Unix Sequential file Terminated by LFs)

               uvcp "fili1=tmp/updts,typ=LST,filo1=dat1/updts,typ=RST,rcs=80"
               ==============================================================
  1. given - a file of name&adrs records similar to the above but keyed with customer# in 1-6 & name&adrs in 7-?? (80 cols max)

    required - move the name&adrs from cols 7-79 max to cols 11-89 max and
               insert a constant date '9212' in cols 7-10 (out rcsz 90 bytes)

    uvcp "fili1=tmp/ups,typ=LST,filo1=dat1/ups,rcs=90,mvc=10(80)6,mvc=6(4):9212"
    ============================================================================
  1. given - a sequential file of 200 byte A/P history records with 12 x 5 byte packed fields in cols 21-80 (current yr monthly acums)

required - zero out the 12 months accumulators to begin a new year


    uvcp "fili1=dat1/aphist,typ=RSF,rcs=200,filo1=tmp/aphist,mvnx12=20(p5):0"
    =========================================================================
Note
  • the 'x12' of 'mnvx12' will 'repeat' the move numeric 12 times
    (converting the constant numeric zero to packed 5 bytes)
  • we could also have had another instrn to 1st move this year fields
    to last year which might be 'mvc=80(60)20"
  • after checking output tmp/aphist, copy back to (overwrite) dat1/aphist
  1. Given - a large file (5 gigabytes) of 100 byte records - filename "saleshist"

    Required - split the file into multiple files, max 2 gigabytes
             - 20,000,000 records * 100 = 2,000,000,000 bytes
             - so output files can be transferred to an old system
               with a 2 gig file limit

    uvcp "fili1=dat1/saleshist,typ=RSF,rcs=100,filo1=tmp/saleshist,rop=x20000000"
    =============================================================================

output: saleshist_001 (2 gig), saleshist_002 (2 gig), saleshist_003 (1 gig)

Ilustrates Run OPtion 'x' (rop=x____) to split output into multiple files of a max number of records (except for last output file).

Goto:   Begin this document End this document UVSI Home-Page

B3. uvcp INTRODUCTORY EXAMPLES (continued)

  1. required - an easy way of creating empty Indexed files - sometimes required for batch or online programs to accumulate records from an initial state of zero.

Note
  • this script is also documented in scripts.doc
    & is present in the 'uvadm/sf' directory.
 # makeISF0 - create empty D-ISAM file
 #
 #usage:   makeISF0 filename rcsz keys
 #         ===========================
 #
 #example: makeISF0 isamtest 255 0,8,50,20
 #         ===============================
 #
 if [[ $# -ne 3 ]]; then
    echo "usage: makeISF0 filename rcsz key1strt,key1lth,key2strt,key2lth,etc"
    exit 9; fi
 #
 uvcp "fili1=/dev/null,typ=RSF,rcs=$2,filo1=$1,typ=ISF,isks=($3)"
 #
 exit 0
Note
  • See uvtrain.htm for several more uvcp examples & exercises

Goto:   Begin this document End this document UVSI Home-Page

C1. uvcp FUNCTION SUMMARY

 fili1=infilename           - declare input filename
 typ=LST,RSF,RST,RSV,ISF,IDX,STL - declare file type
 typ=LSTt                          & possible options appended to file type
 rcs=9999                   - declare record size
 filo1=outfilename          - declare output filename
 iskx=start(length)         - declare indexed file keys
 rop=abc...xyz              - declare misc run options

sel=start(lth):constant - select records with spcfd value in spcfd cols

del=start(lth):constant - delete records with spcfd value in spcfd cols

 prm=paramfile              - declare name of a parameter file which would
                              contain any lengthy list of instructions
                            - not normally required since all parameters
                              may be specified on the command line
                              (use the sort facility of uvcopy if required)

interrupt processing

You may press the 'interrupt' key while uvcp is copying files. Interrupt causes uvcp to display current input/output record counts & prompts you for a response. Enter 'q' to close files & end the program, or null to continue. Interrupt processing must be enabled via run option 'e1' (rop=e1) for example:


 uvcp "fili1=infile,rcs=80,typ=LST,filo1=outfile,rcs=80,typ=LSTt,rop=e1"
 =======================================================================

Goto:   Begin this document End this document UVSI Home-Page

D1. uvcp: FILE DEFINITION PARAMETERS

      fili1=, typ=, rcs=, [isk#=,]  filo1=, [typ=,] [rcs=,] [isk#=]
  fili1 - specifies input filename (fili1=pathname)
        - may specify as null (fili1=,) to cause environment search for
          variables fili1/filo1 & retrieve equated values for filenames
          (initftab subfunction also used with COBOL programs)
  filo1 - specifies output filename (filo1=pathname)
        - same rules as for fili1
Note
  • following keywords apply to the preceding file (fili1 or filo1)
  • keywords specified for fili1 also default for filo1 unless respecified
typ
  • file type codes - 3 upper case alpha + misc options a-z
    RSF/RSV/RST/ISF/IDX/LST/STL - see details ahead
    file options - misc options may be appended to the 3 alpha codes
    - see details on next page
rcs
  • record size for the preceding file (fili1/filo1)
  • may omit declaring rcs for output file (will default same as input)
  • input ISAM files know their rcs, but you have to specify something
    to get past the error-check
isk?
  • declare isk1-isk9 for the preceding file (fili1/filo1)
  • required only for filo1 typ=ISF/IDX to build/load a new Indexed file
  • not required when the input file is typ=ISF/IDX,& you wish to accept
    the default behavior of automatically copying the input file keys.
        isk1=0(8)         - declare 1st key (bytes 1-8)
        isk2=59(15)       - declare 2nd key (bytes 61-75)
        isk1p1=0(8)       - declare key#1 part#1
        isk1p2=20(6)      - declare key#1 part#2
        isk1p3=30(4)      - declare key#1 part#3

or all on 1 line

isk1p1=0(8),isk1p2=20(6),isk1p3=30(4),isk2p1=100(30),isk2p2=150(15)

fili1/filo1 example


    uvcp "fili1=xx,typ=RSF,rcs=80,filo1=yy,typ=ISF,isk1=0(8),isk2=59(15)"
    =====================================================================

alternate format for ISAM keys


    uvcp "fili1=xx,typ=RSF,rcs=80,filo1=yy,typ=ISF,isks=(0,8,59,15)"
    ================================================================

Goto:   Begin this document End this document UVSI Home-Page

D2. uvcp: Indexed file definition - additional options

ISAM option for DUPS/NODUPS


 uvcp "fili1=x,typ=RSF,rcs=80,filo1=y,typ=ISF,isk1=0(8),isk2=40(6),isk3=60(9)"
 ================================================================================
 - omitting any Dups/Nodups options on isk1/isk2/isk3

 uvcp "fili1=x,typ=RSF,rcs=80,filo1=y,typ=ISF,isk1=0(8n),isk2=40(6d),isk3=60(9d)"
 ================================================================================
 - defaults same as this (Nodups isk1, Dups isk2,isk3,etc)

 uvcp "fili1=x,typ=RSF,rcs=80,filo1=y,typ=ISF,isk1=0(8d),isk2=40(6n)"
 ================================================================================

ISAM option DUPS/NODUPS - alternate format


 uvcp "fili1=x,typ=RSF,rcs=80,filo1=y,typ=ISF,isks=(0,8,40,20,60,10)"
 ====================================================================
 - omitting any Dups/Nodups options on isks=(...) format

 uvcp "fili1=x,typ=RSF,rcs=80,filo1=y,typ=ISF,isks=(0,8n,40,20d,60,10d)"
 =======================================================================
 - defaults same as this (Nodups on 1st key, Dups Allowed on 2nd,3rd,etc)

 uvcp "fili1=x,typ=RSF,rcs=80,filo1=y,typ=ISF,isks=(0,8d,40,20n)"
 ====================================================================

Goto:   Begin this document End this document UVSI Home-Page

D3. uvcp: Indexed file definition - additional options

No need to specify ISAM keys for input files

You do not need to specify ISAM keys for input files (only required when creating output indexed files). You may specify 'isk' for input files if desired, but you will get an error if the declared 'isks' do not match the existing indexed file keys. For example, the following 2 commands are the same:

uvcp "fili1=x,rcs=80,typ=ISF,isks=(0,8,40,20,60,10),filo1=y,typ=RST"

uvcp "fili1=x,rcs=80,typ=ISF,filo1=y,typ=RST"

Input Indexed keys copied to output file

If both input & output files are indexed (typ=ISF or typ=IDX) and no output keys are declared, then the input file keys are automatically used for the output file. For example if file 'x' has keys as shown above, the following would create indexed file 'y' with the same keys:

uvcp "fili1=x,rcs=80,typ=ISF,filo1=y,typ=ISF"

option 'k' for key of reference (read sequence)

You can specify option 'k' to specify the Key of Reference. For example option 'k3' of file 'typ=ISFk3' specifies the key of reference as key #3, to cause record retrieval in sequence by the 3rd key.

uvcp "fili1=x,rcs=80,typ=ISFk3,filo1=y,typ=RST"

Goto:   Begin this document End this document UVSI Home-Page

E1. uvcp: I/O file declaration

  1. declaring filenames on the COMMAND LINE


      uvcp "fili1=dat1/sales1,rcs=64,typ=RSF,filo1=tmp/sales1"
      ======*****************================****************
  1. declaring filenames in a PARAMETER FILE


    command line:    uvcp prmfilexx
                     ==============
    parameter file:  fili1=dat1/custsales,rcs=128,typ=RSF
                     filo1=prodsales
                     mpy=b60(8):120          increase price by 20%
                     div=b60(8):100

Goto:   Begin this document End this document UVSI Home-Page

F1. uvcp data file utility

file TYP codes

File type codes are 3 upper case alpha + misc options (a-z), for example:


       uvcp "fili1=test1,typ=RSF,rcs=80,filo1=test2,typ=LST"
       ==================*******====================*******
RSF
  • Record Sequential Fixed (record size)
RST
  • Record Sequential Terminated (fixed rcsz)
    Terminated by LF *in* last byte of stated record size
    (whether input or output)
RSR
  • Record Sequential Relative (Micro Focus COBOL compatible)
  • actual record size is 1 greater than declared
  • extra byte x'0A' if valid, x'00' if deleted
    on output - the x'0A' is always written
    on input - 1 extra byte is read but not tested by default
    - option 'r1' required to drop record if last byte x'00'
RUF
  • UNIBOL file type
  • if input: will bypass the 8 byte header
  • if output: will create the 8 byte header as follows:
    bytes 00-05 x'07ff00430000', bytes 06-07 record size in binary
LST
  • Line Sequential Terminated, with LineFeed *after* last byte of recsize
  • LST input files are usually variable rcsz
  • LST output files would have the LF after the stated rcs
  • the 't' option would truncate any trailing traling blanks
    to create variable output rcsz's if the data varies.
  • use LST only when required (for variable length text records)
  • LST is 4 times slower than RSF or RST because it reads or writes
    1 character at a time (vs 1 record at a time).
ISF
  • Indexed Seqential Fixed record length D-ISAM/C-ISAM/IDXFORMAT1
  • see more explanation on the next page --->
RSV
  • Record Sequential Variable files compatible with Micro Focus COBOL
    IDXFORMAT3 files, see more explanation on the next page --->
IDX
  • Indexed Sequential Variable files compatible with Micro Focus COBOL
    IDXFORMAT3/8 files, see more explanation 2 pages ahead --->

Goto:   Begin this document End this document UVSI Home-Page

F2. file TYP codes for fili1/filo1 (continued)

RDWz4
  • 'Record Descriptor Word' variable length files (RDW defaults to RDWz4)
  • 4 byte record prefix with 2 byte binary record size & 2 unused nulls
  • record size in prefix is the total record size including the prefix
  • record size is 'BIG-END' binary (like mainframe or SUN,AIX,DEC)
  • 'little-end' machines (Intel,AMD) need to switch ends to perform math
RDWz2
  • RDW files wiith ONLY 2 byte headers (vs z4 4 byte standard)
  • recsize in hdr DATA SIZE ONLY (vs z4 recsize in hdr = Total Header+Data)
RDWz18
  • (z2+z16) similar to z4 STANDARD RDW
  • recsize in hdr is TOTAL slot size (includes 2 byte hdr)
  • 2 byte hdr, recsize 1st 2 bytes binary BIG-end
Note
  • would be better if RDW types z2 & z18 were switched, to be consistent with
    RDWz4 which has total size in header (standard) vs data only (non-standard).
  • But RDWz2 format was requested prior to RDW18 (z2+16 bits = z18 code)
  • And RDWz18 was implemented much later in March 2022
EOR
  • to create output files with EBCDIC End-Of-Record marker x'FF01'
EORt1
  • option 't1' translates to EBCDIC
EORt4
  • option 't4' shortens record at last non-blank (EBCDIC x'40')
EORt5
  • t5=(t1+t4) translates to EBCDIC & shortens record at last non-blank
EOR
  • see test/demos on pages 'V1' - 'V3'
STL
  • STandard Language file system, used by AIX COBOL
STLs
  • STL Sequential files
STLi
  • STL Indexed files
STLr
  • STL Relative record files
STL
  • see test/demo STL files on pages 'W1' - 'W5'
  • uvcp instructions to convert to 'typ=LSTt'
  • uvhd hexdumps of file headers & record prefixes

Goto:   Begin this document End this document UVSI Home-Page

F3. uvcp: file TYP codes for fili1/filo1 (continued)

typ=ISF Indexed Sequential Fixed files

typ=ISF
  • Indexed Seqential Fixed (record length ISAM files)

       uvcp "fili1=test1,typ=LST,rcs=80,filo1=test2,typ=ISF,isk1=0(6)"
       ======================***========================***===========

The example above will convert a text input file to an Indexed Sequential Fixed record size file with key in 1st 6 bytes. This is an easy way to create indexed files - just use the editor to create the keys & data fields, with no need to worry about record sizes, then use uvcp to create the ISAM data partition of fixed records & indexed partition.

export ISDATEXT=.dat

You may export ISDATEXT=dat if you wish to use '.dat' extensions on D-ISAM files read or written by uvcp (+ uvsort + uvcopy). The default is no extension on data partition, which is compatible with Micro Focus COBOL. Add to /etc/profile if your site always needs '.dat'. See options for Micro Focus to use '.dat' extensions (EXTFH & IDXNAMETYPE=2). See extfh.cfg listed at MVSCOBOL.htm#5D2 or VSECOBOL.htm#5D2

typ=RSV for COBOL non-indexed IDXFORMAT3


 uvcp "fili1=dat1/pmast,typ=RSV,rcs=4000,filo1=tmp/pm,typ=ISF,rcs=200,isk1=0(6)"
 ===============================================================================

The above would convert a Record Sequential Variable (Micro Focus IDXFORMAT3) data file to an Indexed Sequential Fixed record length file. The 'ISF' file is compatible with MF COBOL fixed length Indexed files & C-ISAM files.


 uvhd dat1/pmast v    <-- see uvhd display on page 'I3'
 =================

Please see pages 'I1' thru 'I3' for more details & examples --->

Goto:   Begin this document End this document UVSI Home-Page

F4. uvcp: file TYP codes for fili1/filo1 (continued)

typ=IDX Indexed Sequential Variable files

'typ=IDX' declares Indexed Sequential Variable length record files compatible with Micro Focus COBOL IDXFORMAT3 files. See the 'File Handling' documentation for Micro Focus Object or Server Express COBOL.

The Micro Focus COBOL 'EXTFH' file handler is compiled into an alternate version of uvcp called 'uxcp' using the 'cobccuv' script as explained in install.doc.

'uxcp' supports the same file types as uvcp with typ=IDX in addition. Two versions of the program are required since only Micro Focus COBOL users (server express 2.2+) could compile 'uxcp' whereas any user can compile 'uvcp' which uses D-ISAM to support Indexed Sequential FIXED record length files.

Note that 'typ=RSV' (Record Sequential Variable with no indexed keys) files are supported by both uvcp & uxcp. 'uxcp' must be used when either input or output or both files are typ=IDX.

example - convert text file to Indexed Variable


 uxcp "fili1=x,typ=LST,rcs=256,filo1=tmp/y,typ=IDXf3v62,rcs=254,isk1=0(6d)"
 ==========================================================================

This would read text file 'x' & build an Indexed Sequential Variable file 'y' whose key is the 1st 6 bytes of each record. The output records could vary from 62 to 254 bytes depending on the last nonblank character of input record. 'rcs=254' sets the maximum size & option 'v62' on typ=... sets the minimum size. The records will actually vary from 64 to 256 allowing for the 2 byte record header (see page 'J2' which displays a sample IDXf3 file).

You can run this example using a test file provided (uvadm/dat1/testRSV1) in place of file 'x' above.

Please see pages 'J1' thru 'J8' for more details & examples --->

Goto:   Begin this document End this document UVSI Home-Page

F5. uvcp: file TYP OPTIONS

File options may be appended to the 3 byte file typ code - lower case letters a-z with possible numeric suffixes - for example:


       uvcp "fili1=text1,typ=LSTb1c2,rcs=80,filo1=text2,typ=LSTt3"
       =========================****===========================**=
Note
  • we will 1st discuss options that apply to typ=LST (text files)
  • then discuss options applicable to multiple types in alphbetic sequence
  • then discuss options applicable to less used file types (RDW,RSV,Indexed)

options for file typ=LST... (text files)

c1
  • convert any unprintable chars to blanks
c2
  • convert any unprintable chars to periods
c4
  • convert nulls to blanks
c8
  • convert tabs to blanks
c16
  • remove trailing blanks,nulls,LFs <-- DEFAULT
  • c16 set by any LST unless LSTt4 both in&out
c32
  • also blank CR on scanback if c16 (set by typ=LST)
c64
  • stop read on formfeed x'0C' (use s0 to not stop on LF)
c128
  • determine output length by 1st null vs last nonblank
  • you are unlikely to need any of c16,c32,c64,c128.
d_
  • option d (delete chars) not used by uvsort typ=LST (March 2019+)
e_
  • option e (erase chars) not used by uvsort typ=LST (March 2019+)
t1
  • stop read on CR <-- 't' options for Input file typ=LSTt#
t2
  • stop read on LF <-- DEFAULT t2 stop read on LF
t4
  • stop read on LF & save length to write same length on output typ=LSTt4
t8
  • stop read on tilde '~'
Note
  • above Input stop read options s1-s8 changed to t1-t8 to match Output options
    so you can specify only on input & let default to output as shown on #3 below
t1
  • terminate write with CR <-- 't' options for Output file typ=LSTt#
t2
  • terminate write with LF <-- DEFAULT t2 stop write on LF
t4
  • write same length as input & terminate LF if t4 or CR/LF if t5
t8
  • terminate write with tilde '~'

Notes re options for typ=LST...

  1. You normally need no options for typ=LST, since the defaults are what you would expect for text files - stop read on LineFeed x'0A' & append LF x'0A' on output writes (after the last non-blank).

  2. File typ options (like most Vancouver Utility options) are "binary" and may be combined. For example, typ=LSTt3 would terminate output records with both a CR & an LF x'0D' + x'0A' (MSDOS file).

  3. Output file typ options may be omitted & defaulted to input options, for example:


   uvsort "fili1=text1,typ=LST,rcs=80,filo1=text2,key1=0(20)"
   ==========================================================

Goto:   Begin this document End this document UVSI Home-Page

F6. file TYP OPTIONS (continued)

file typ options applicable to multiple file types

a
  • append to the file's existing contents (vs default erasing existing contents)
b1
  • reduce multi consecutive blank lines to 1 blank line
b2
  • drop blank records
h1
  • wait on locked file, retries every 10 seconds until lock released
    (when other process closes the file)
  • retry time is gradually increased to every 5 minutes after 30 mins
  • displays waiting message showing total wait time
  • you would use option 'h1' with EXCLUSIVE lock ('l0' or not coded)
  • typ=ISFh1l1 would not make sense, since it will never get a lock
  • See file lock test results in separate section uvlocks.htm
j1
  • translate filename to lower case (before opening)
j2
  • translate filename to UPPER case (before opening)
j4
  • translate any extension to the opposite case

Option 'j' useful when you are processing all files in directory (with uvcopyx) And you want the output files in a different case than input files.

l_
  • file locking options for DISAM indexed files
    & for UNIX Sequential files as of January 1999
  • see more discussion & test/demo's in uvlocks.doc (follows uvcopy5)
l0
  • EXCLUSIVE (default) RDlock+WRlock
  • other jobs cannot access, unless they specify READONLY for ISAM
    or NO-locking for non-indexed sequential files.
  • always use for output files (building new ISAM files).
l1
  • READONLY
  • multiple jobs could read same file if all jobs specify RDlock
  • open error if another job has opened with WRlock (or Exclusive)
    OR use option 'h1' to wait until other jobs with WRlock/Exclock close
l8
  • NO locking, used only for Sequential files (not Indexed)
  • For non-indexed files, only lock options l0, l1, l8 apply.
m#
  • file progress messages every m# records
m1000000
  • default is every million records
Note
  • EOF elapsed time stats are reported if > 5 seconds
  • no option to change this (as of Oct 2018)
n1
  • inhibit errmsg if data exceeds record length specified for typ=LST
  • ie, if LineFeed x'0A' not found before specified rcs=...
  • causes over-length records to be split into 2 records.
  • better to increase record-size to avoid splitting records
w2
  • Warning prompt if output file exists
  • may overwrite, change name, or cancel

Goto:   Begin this document End this document UVSI Home-Page

F7. file TYP OPTIONS (continued)

file typ options unique to typ=RDW files

n4
  • typ=RDW: specifies 4 byte record-size prefix Numeric digits
  • vs default binary 2 bytes + 2 bytes null
  • also see typ=RDW & options z2/z4/z8
t
  • controls record padding & truncation for typ=RDW files
t0
  • typ=RDWt0 - default blank fills
t1
  • typ=RDWt1 - null fills to end of record
t2
  • typ=RDWt2 - scans back to last non-blank or non-null
    - to shorten output record
t3
  • typ=RDWt3 - same as RDWt2
    - since either nulls or blanks are truncated
  • RDWt3 might apply to mainframe SORT conversion of OUTREC FIELDS=(1,80,100)
  • (1,80,100) missing length from 100,___ means move remainder of var lth record
  • would generate uvsort moves "mvco=0(80)0,mvco=80(0)100"
  • length (0) causes uvsort to move remainder of var lth record
z4
  • STANDARD RDW - 4 byte hdr, hdr-value = TOTAL (hdr + data)
    (typ=RDW defaults to typ=RDWz4)
  • recsize in hdr is total slot size (includes hdr 4 bytes)
  • 4 byte hdr, recsize 1st 2 bytes binary BIG-end, 3&4 unused
  • 'BIG-END' binary used on mainframes or SUN,AIX,DEC
  • 'little-end' machines (Intel,AMD) need to switch ends to perform math
z2
  • NON-STANDATD RDW (before z16 added)
  • recsize in hdr DATA SIZE ONLY (not including hdr 2 bytes)
  • 2 byte hdr, recsize 2 bytes binary BIG-end
z18
  • (z2+z16) similar to z4 STANDARD RDW
  • recsize in hdr is TOTAL slot size (includes 2 byte hdr)
  • 2 byte hdr, recsize 1st 2 bytes binary BIG-end
Note
  • would be better if RDW types z2 & z18 were switched, to be consistent with
    RDWz4 which has total size in header (standard) vs data only (non-standard).
  • But RDWz2 format was requested prior to RDW18 (z2+16 bits = z18 code)
  • And RDWz18 was implemented much later in March 2022

RDW processing by uvcp,uvsort,uvcopy

Note that the binary values in RDW headers are "BIG-END" binary, native to manframes, but Vancouver Utilities (uvcp,uvsort,uvcopy) usually run on INTEL or AMD machines which are "little-end".

uvcp/uvsort/uvcopy detect the machine type & automatically switch the binary ends as required after reading & before processing & calculating data size to read after the RDW header value.

And the programs automatically switch the little-end calculated values back to BIG-END for writing in the RDW output file headers.

Goto:   Begin this document End this document UVSI Home-Page

F8. file TYP OPTIONS (continued)

file typ options unique to typ=RSV & typ=RSR

g_
  • for typ=IDX/RSV MF COBOL ISAM variable length Indexed/Sequential files
g4
  • get active data records only (default)
g2
  • get deleted data records only
g6
  • get both active & deleted data records
  • option 'g' in this uvcp is similar to record display option 'v'
    in the uvhd display utility (v1=filehdr,v2=deleted,v4=data,v8=all)
q8
  • round up typ=RSV slot size to a nultiple of 8 (vs default 4)
  • use with option v (typ=RSVvq8), may be required if recsize >4095
  • see option v in alpha sequence below
r1
  • for typ=RSRr1 Relative files (MicroFocus COBOL compatible)
  • drop record if extra byte = x'00'
v9999
  • minimum record size option for typ=IDX & typ=RSV files
    For example: 'typ=IDXv400,rcs=5000' v400 declares minimum size 400
    while rcs=5000 declares the maximum size 5000.
  • if option v not specified or specified with no value on typ=IDX or typ=RSV,
    all data records will be fixed size rcs=____, but with variable length
    record headers.
v#y4
  • also see type 'y4' below to determine record size by scanning back
    from max size to the last nonblank/nonnull beyond min size
v#q8
  • q8 roundup output record size (pad x'20's) to multiple of 8 (vs 4)
  • may be required if record size > 4095

Goto:   Begin this document End this document UVSI Home-Page

F9. file TYP OPTIONS (continued)

file typ options for FIXED-length Indexed files

f#
  • format for typ=IDX files
f1
  • fixed length Indexed records, compatible with C-ISAM, same as typ=RSF
f3
  • variable length Indexed files (file size < 2 gig)
f8
  • variable length Indexed files (allowing file size > 2 gig)
  • IDXf3 & IDXf8 could be fixed records in variable format
g_
  • for typ=IDX/RSV MF COBOL ISAM variable length Indexed/Sequential files
g4
  • get active data records only (default)
g2
  • get deleted data records only
g6
  • get both active & deleted data records
  • option 'g' in this uvcp is similar to record display option 'v'
    in the uvhd display utility (v1=filehdr,v2=deleted,v4=data,v8=all)
k#
  • key of reference for an ISAM input file
    typ=ISFk2 would cause reading in sequence by key #2
p#
  • parts in multi-part keyed files
l2
  • AUTOLOCK (applies only to ISAM files in uvcopy utility, not uvsort)
  • only 1 record locked at 1 time in this file
  • records are locked by the 'red' instruction
    & unlocked by whatever instruction follows
    (might be 'upd' to update the record)
l4
  • MANULOCK (applies only to ISAM files in uvcopy utility, not uvssort)
  • assumes the programmer will handle locking manually
    using the 'lck' uvcopy instruction ('islock' DISAM function)
  • Use this option if you want to open 2 paths for the same file
    (fili1 & filr1 for the same filename with typ=ISFl4)
Note
  • you probably wont use ISAM 'RECORD locking' options l2 & l4
  • it is easier to use the 'FILE locking' options
    (l0,l1 for ISAM files; l0,l1,l8 for Sequential files)
o4
  • detailed ISAM file key statistics (in addition to record counts)
    typ=ISFs4 will display key start byte & length for each key

Goto:   Begin this document End this document UVSI Home-Page

F10. file TYP OPTIONS (continued)

file typ options for VARIABLE-length Indexed files

Note
  • you must use 'uxsort' (vs uvsort) to process IDXFORMAT3 & IDXFORMAT8
  • options v,x,y apply mostly to uxsort
  • but option v might be used with uvsort for file typ=RSV
IDXf1
  • fixed length Indexed records, compatible with C-ISAM, same as typ=RSF
IDXf3
  • variable length Indexed files (file size < 2 gig)
IDXf8
  • variable length Indexed files (allowing file size > 2 gig)
  • IDXF3 & IDXf8 could be fixed records in variable format
v9999
  • minimum record size option for typ=IDX & typ=RSV files
    For example: 'typ=IDXv400,rcs=5000' v400 declares minimum size 400
    while rcs=5000 declares the maximum size 5000.
  • if option v not specified or specified with no value on typ=IDX or typ=RSV,
    all data records will be fixed size rcs=____, but with variable length
    record headers.
v#y4
  • also see type 'y4' below to determine record size by scanning back
    from max size to the last nonblank/nonnull beyond min size
v#q8
  • q8 roundup output record size (pad x'20's) to multiple of 8 (vs 4)
  • may be required if record size > 4095
x_
  • Not used
y
  • Recording Mode
y1
  • Fixed Length records, all records forced to size spcfd by rcs=____
  • default for if option 'v' NOT specified (w or w/o min recsize)
y2
  • Variable Length records
  • default if option 'v' IS specified (w or w/o min recsize value)
  • min recsize specified by option 'v', max recsize by rcs=____
y4
  • determine the record size for each record by scanning back from
    max size to last non-blank or non-null beyond the min size
y8
  • fixed length records in variable length format
  • in case you want to copy a variable length file & output all records
    as the max size specified on rcs=___ (but in variable length format

Goto:   Begin this document End this document UVSI Home-Page

F11. file TYP OPTIONS (continued)

output record size defaults for variable length files

The output record size for IDXf3/IDXf8 variable length records defaults to the corresponding input record size. The record size is passed thru the sort by appending it on the end of the record & using it on the output phase.

This works well if both input & output are variable length (typ=IDXf3v/IDXf8v). If input was fixed (typ=RSF/ISF/IDXf1) & output was variable (typ=IDXf3/IDXf8v), then output records would be in the variable length format, but all same size.

See pages 'J1' thru 'J9' for scripts to test/demo IDXf3/IDXf8 options.

Goto:   Begin this document End this document UVSI Home-Page

G1. uvcp: 'rop' - RUN OPTIONS

'rop' is used to declare various options for the run, for example:

 rop=x1000   <-- set max records per output file, splits output to multi-files
               - appends _001,_002,etc on declared output filename

Do not confuse with file options which apply only to a file (fili1/filo1) & which are appended to the file typ code (example: typ=LSTtd). Do not confuse with instruction options which apply only to the current instruction & are appended to the instruction opcode.


 uvsort "fili1=...,typ=...,rcs=...,filo1=xxx,key1=...,rop=x1000"
 ===============================================================
 - complete example, option 'x1000' multiple files 1000 records each
   and option 'v' 0 to inhibit sort progress messages

 export UVCPROP=r65536   <-- set max recsize 65,536 for all uvcp's
 =====================
 - alternate way to specify run options via Environmental Variable 'UVCPROP'
 - could define in your profile to specify options desired for all uvsort's
b99999
  • input file record bypass count
i99999
  • input file record stop count
o99999
  • output file record stop count
Note
  • both input & output stop counts are provided since you might
    also be selecting or deleting records
j#
  • input every j#th record (create smaller test files)
j100
  • input every 100th record (drop 99 of each 100 records)
e1
  • enable interupt (usually '^C' or the DELETE key)
  • will report current I/O record counts & allow close or continue
  • disabled by default since 'uvcp's within scripts might be zombied
  • enable when you run uvcp from command line by appending ',rop=e1'
n1
  • inhibit wait for reply on all prompts soliciting operator input
  • intended for unattended night shifts
  • could append on each uvcp "fili1=....,rop=n1"
  • could code 'export UVCPROP=n1' at begining of a night shift script
  • or put in profile of night shift to inhibit all prompts all scripts
p_
  • Prefix output records with record-number & record-size
p1
  • prefix with record-number only (9 digits & '_' separator)
p2
  • also include record-size (code with p1 + p2 = p3)
p3
  • both record# & record-size
  • 123456789_12345_<---- record data ---->

Goto:   Begin this document End this document UVSI Home-Page

G2. uvcp: 'rop' - Run Options - EXAMPLES

r99999
  • declare maximum record size to be 99999 (or whatever)
    vs the default max 32768
  • this also changes the max size allowed for the 'w' work area
t15
  • default time (seconds) to report EOF stats if exceeded
  • do not confuse with file typ option 'm' progress reporting
    typ=RSFm1000000 default reports every million records
x99999
  • set Max records per output file, causes multi-file output
  • appends _001,_002,etc on declared output filename
  • this option added in October 2016
y70
  • set Year2000 window for option 'y__' on select/delete commands
  • option 'y__' assumes 1st 2 bytes of field is the year (no century)
  • 'y70' means years 70-99 = 1970-1999, 00-69 = 2000-2069
z
  • option 'z1' specifies sliding windows, for example in 1998
y70z1
  • window value would be '68', 68-99 = 1968-1999, 00-67 = 2000-2067

UVCPROP - env-var to specify uvsort Run OPtions

Environmental Variable 'UVCPROP' can be defined in your profile to specify options for all uvcp executions (vs appending ,rop=... on the uvcp command). There are only a few options that you might want to specify this way, for example:


 export UVSORTROP=r65536n1    <-- specify run options for all uvcp's
 #========================
 - r65536 max recsize 65,536 vs default 32,768
 - n1 inhibit wait for reply on all prompts soliciting operator input

Goto:   Begin this document End this document UVSI Home-Page

G3. uvcp: 'rop' - Run Options - EXAMPLES

rop options - complete examples


 #1. uvcp "fili1=xx,typ=RSF,rcs=64,filo1=yy,sel1y=44(6y),rop=b2000,o500
     ====================================================**************

b2000 - bypasses the 1st 2000 records of the input file

    o500  - sets a stop count to end output after 500 records
            (not necessarily #2000-2500 due to select)
Note
  • you don't need commas, could specify as: "rop=b2000o500"

 #2. uvcp "fili1=dat1/saleshist,typ=RSF,rcs=100,filo1=tmp/saleshist,rop=x20000000"
     =============================================================================
      This illustrates Run OPtion 'x' (rop=x____) to split output into multiple
      files of a Max number of records (except for last output file).
      For example, given a large file (5 gigabytes) of 100 byte records,
      You might need to split into multiple files of 2 gig each, for transfer
      to an older system with the 2 gig file limit.
       Use 'rop=x20000000' 20,000,000 records * 100 = 2,000,000,000 bytes
       output: saleshist_001 (2 gig), saleshist_002 (2 gig), saleshist_003 (1 gig)

 #3. uvcp "fili1=dat1/filex,typ=STLr,rcs=4096,filo1=tmp/filex,typ=LSTt,rop=p3"
     =========================================================================
      You can use 'rop=p1' to Prefix output records with a 9 digits sequence#
      separated from record-data with an underscore (123456789_<---data--->)
      Use 'rop=p3' to include record-size (123456789_12345_<---data--->)
      You might need this to reload a corrupted STL Relative file.
      You would write a COBOL program to recreate the file with same size
      records and in the same relative record# locations.

 #4. uvcp "fili1=dat1/warmas1a,typ=RSF,rcs=64,filo1=wm2,sel1y=44(2)>98,rop=y70z1"
     ==================================================================*********
     rop=y70z1 - set Year2000 sliding window '70' for sel1y/del1y
                 (select/delete on windowed date fields)
     in 1998 the window value would be '68' meaning 68-99 = 1968-1999,
                                                    00-67 = 2000-2067

Goto:   Begin this document End this document UVSI Home-Page

G4. uvcp: OPTION CODING RULES

example1: rop=r8000i1000t300

Options must be 1 contiguous string with no blanks (except in quotes)

Valid options are the 26 lower case letters. The letter options may be specified in any sequence. Each letter used should only be specified once in the string, but the last specification would override previous specs.

The option letter may be followed by a numeric value which is often bit coded for suboptions (a7 includes a1+a2+a4).

example2: rop=(a,b,c768,d1,e='embedded blanks')

String values must be assigned via the '=' separator & the string must be enclosed in quotes if any embedded blanks. Hex values are indicated by enclosing in circumflexes

Comma separators are optional if no value or only numeric values are assigned, but are mandatory following string values.

When the '=' or ',' is required/desired within the option string, you must enclose the option string within (...) to not confuse the parsing of the overall keyword & argument prmtr string which must then be enclosed in quotes to prevent the shell choking on the parenthesis - also true if any isk1=start(length) keywords are present.

3 KINDS OF OPTIONS

  1. 'run' options - specified via the 'rop' keyword parameter apply to the entire run (see above).

  2. 'file' options - appended to the mandatory 3 character file typ code apply only to that file (fili1/filo1).

  3. 'instruction' options - appended to the 3rd byte of the opcode for example:

       sqzmb=20(80):0x20   - options 'mb' squeeze multiple blanks to 1
                             vs the default of all blanks

Goto:   Begin this document End this document UVSI Home-Page

H1. uvcp: sel/del - SELECT/DELETE RECORDS

       sel=start(length):dataconstant
       sel=0(2):a1                 - select if bytes 1-2 = 'a1'
       sel1=78(2):BC               - select records with 'BC' in cols 79-80
       sel2=8(1)!D                   that do not have a 'D' in column 9
       del1=90(2)>55               - delete if 91-92 > '55'
       del2=99(1)!0xff               and if 100 not = x'ff'

'sel' & 'del' are mutually exclusive.

The 'and/or' condition is indicated by 4th byte of sel/del. sel1/sel2/etc indicates 'and' conditions. The 'or' condition is indicated when 4th byte reverts to 1 or blank.

'sel's or 'del's are stored in a common table at setup time for execution at run time (100 entries allowed).

option 'y' for Year2000 windowed date field selections

     sel1y=20(6)>980630   - select records with dates in cols 21-26
                            from 19980701 up to 20491231

Option 'y' assumes 1st 2 bytes of specified field is the 2 digit year regardless of field length (yymmdd, yymm, yy).

The default window is '50', may be changed by run option 'y__' to specify the window value,& option 'z1' for sliding windows.

Note
  • run options are specified on the 'rop' command (ex: rop=y50z1)
      rop=y50  -------- means 50-99 = 1950-1999,& 00-49 = 2000-2049
      rop=y50z1 in 1998 means 48-99 = 1948-1999,& 00-47 = 2000-2047
      sel1y=20(2)>98,rop=y70z1  - select records with years in cols 21-22
                                  from 1998 to 2067 (rop=y70z1) if now 1998

Goto:   Begin this document End this document UVSI Home-Page

H2. uvcp: sel/del - SELECT/DELETE RECORDS

option 'u1' - allow coding ' 's as '_'s

Option 'u1' allows you to code blanks as underscores, which is convenient because blanks are not allowed in the uvcp command string. The alternative would be to code blanks in hexadecimal (' ' = 0x20). Here are some examples:

option 'u2' - allow coding ' 's as '_'s

Option 'u2' allows you to code commas as at-signs, which is convenient because commas are used to separate commands in the command string and therefore can not be coded as constants.

examples - coding ' ' as '_' & ',' ar '@'


 sel1u1=0(1):_,sel1u2=0(1):@,...
 ===============================
 - select record if 1st byte blank ' ' OR 1st byte comma ','

 sel1u1=0(1):_,sel2u2=1(1):@,...
 ===============================
 - select record if 1st byte blank ' ' AND 2nd byte comma ','

 sel1u3=0(2):_@,...
 ===============================
 - select record if 1st byte blank ' ' AND 2nd byte comma ','
 - coding a 2 byte constant (vs separate 1 byte constants)
 - note option u3 = (u1 + u2)

Goto:   Begin this document End this document UVSI Home-Page

I1. uvcp: Record Sequential Variable length files (typ=RSV)

As of Nov 2002 'typ=RSV' supports OUTPUT files as well as INPUT files. 'typ=RSV' provides compatibility with Micro Focus COBOL IDXFORMAT3 Variable length files, but only for Sequential. Indexed variable length support will be provided in Jan 2003 by an alternate version of uvcp (uxcp linked with EXTFH).

For complete documentation on these files, please see the 'FILE STRUCTURES' chapter in Micro Focus Object COBOL or Server Express COBOL, or the web site https://supportline.microfocus.com/documentation/books/ocds4140/fhfile.htm.

'typ=RSV' files have 128 byte file header or 1024 bytes if recsize > 4095. Each record has 2 or 4 byte record header/prefix which has x'40' in the first nibble to identify a data record, followed by the record size in the next 12 bits or 28 bits, depending on max recsize <= 4095 or > 4095 bytes.

uvcp/uvsort RSV Output Implementation

  1. The maximum rcsz is specified by 'rcs=____' & the minimum rcsz is specified by option 'v' of 'typ=RSVv____'. Record size determined by scanning backward to last non-blank & round up to a multiple of 4 (includes 2/4 byte rec hdr).

  2. If minimum rcsz (option v) is NOT specified, all output records will be written with maximum record size & blank padded as necessary.

  3. 'uvcp' allows any combination of I/O file types (LST,RSF,ISF,IDX,RSV) & allows you to convert files from any type to any other type. For example you can create test files by using 'vi' to create text records & then use uvcp/uvsort to convert the text file (typ=LST) to a typ=RSV output file.

    uvsort typ=RSV examples

  4. uvcp "fili1=dat1/testRSV1,rcs=256,typ=LST,filo1=tmp/a,rcs=254,typ=RSV"

  1. uvcp "fili1=dat1/testRSV1,rcs=256,typ=LST,filo1=tmp/b,rcs=254,typ=RSVv62"

    Same as ex#1 but writes typ=RSV records varying from 64 to 256
    - 064 bytes (2 hdr + 062 data) if bytes 62+ all blank
      ??? bytes (2 hdr + ??? data) depending on last nonblank (multiple of 4)
    - 256 bytes (2 hdr + 254 data) is maximum
  1. uvcp "fili1=dat1/testRSV1,rcs=256,typ=LST,filo1=tmp/c,rcs=5000,typ=RSVv400"

    This example writes 4 byte record headers (vs 2 bytes in above examples)
    because the maximum record size is > 4095 bytes (rcs=5000).
    Record sizes vary from 404 to 5004 depending on last nonblank
    - 0404 bytes (4 hdr + 0400 data) if bytes 400+ all blank
    - ???? bytes (4 hdr + ???? data) depending on last nonblank in 400-5000
    - 5004 bytes (4 hdr + 5000 data) is maximum

Goto:   Begin this document End this document UVSI Home-Page

I2. Record Sequential Variable length files (typ=RSV)

using uvcp to create a typ=RSV file


 2. uvcp "fili1=dat1/testRSV3,rcs=256,typ=LST,filo1=tmp/RSV3,rcs=254,typ=RSVv80"
    =============================================================================

input text file (3 records)


 #1 testRSV3 - text to create Rec Seqntl Variable(RSV) file to test uvcp&uvsort
 #2 uvcp "fili1=dat1/testRSV3,rcs=256,typ=LST,filo1=tmp/testx,rcs=254,typ=RSVv80"
 #3 output records vary from min(82) to max(256) depending on last nonblank-----|
 -----this record will be 164 (data 160 + 2 byte rechdr + 2 round to mult 4)---->

output RSV file (displayed with uvhd)


 uvhd tmp/RSV3 v      <-- display file using uvhd with option 'v'
 ===============
                       1         2         3         4         5         6
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 0~......030114102019.................>.......................P..
             370000003333333333330000000000000000030000000000000000000F000500
             0E00000003011410201900000000000000000E0100000000100000000E000000
          64 ................................................................
             0000000000000000000000000000000000000000000000000000000000000000
             0000000000000000000000000000000000000000000000000000000000000000
                       1         2         3         4         5         6
 r#        2 0123456789012345678901234567890123456789012345678901234567890123
         128 @P#1 testRSV3 - text to create Rec Seqntl Variable (RSV) file to
             4523276775553222767727626766762566256767625676666622555226666276
             00310453423630D0458404F032514502530351E4C0612912C5082369069C504F
          64  test uvcp/uvsort
             27677277672777677222
             0453405630F563F24000
                       1         2         3         4         5         6
 r#        3 0123456789012345678901234567890123456789012345678901234567890123
         212 @P#2 uvcp "fili1=dat1/testRSV3,rcs=256,typ=LST,filo1=tmp/test3,r
             4523277672266663366732767755532767333327773455266663376727677327
             0032056300269C91D4141F45342363C233D256C490DC34C69CF1D4D0F45343C2
          64 cs=254,typ=RSVv80"
             67333327773555733222
             33D254C490D236680200
                       1         2         3         4         5         6
 r#        4 0123456789012345678901234567890123456789012345678901234567890123
         296 @.#3 output records vary from min(84) to max(256) depending on l
             4A23267777727666767276772676626662332276266723332266766666626626
             00330F540540253F24306129062FD0D9E884904F0D188256904505E49E70FE0C
          64 ast nonblank-----|-----this record will be 164 (data 160 + 2 byt
             6772666666662222272222276672766676276662662333226676233322232677
             1340EFE2C1EBDDDDDCDDDDD48930253F24079CC025016408414101600B020294
         128 e rechdr + 2 round to mult 4)---->
             627666672223276766276267672322222322
             502538420B0202F5E404F0D5C4049DDDDE00

Goto:   Begin this document End this document UVSI Home-Page

I3. Record Sequential Variable length files (typ=RSV)

File Header record for typ=RSV (or typ=IDX) files

                       1         2         3         4         5         6
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 0~......021125110120.................>.......................~..
             370000003333333333330000000000000000030000000000000000000F000700
             0E00000002112511012000000000000000000E0100000000100000000E000E00
          64 ................................................................
             0000000000000000000000000000000000000000000000000000000000000000
             0000000000000000000000000000000000000000000000000000000000000000

00-00 - x'30' identifies the file header record (vs x'40' for data records)

 01-01 - data size of file header record x'7E' = (7*16+14*1) = 112+14 = 126
       - total size is 128 including the 2 byte record header
 00-03 - If max rcsz > 4095, the file record header would be 4 bytes x'300003FC'
       - 3*256 +15*16 +12*1 = 768+240+12 + 4 byte header = 1024 total record size

08-19 - file creation date/time stamp (yymmddHHMMSS)

36-36 - reserved value x'3E' required for Micro Focus COBOL IDXFORMAT3 files

39-39 - file ORGanization x'01' = Sequential

48-48 - Recording Mode x'01' = Variable format

 54-57 - Maximum record length
       - In this example x'000000FE' = (15*16+14*1) = 240+14 = 254
 58-61 - Minimum record length
       - In this example x'0000007E' = (07*16+14*1) = 112+14 = 126

typ=IDX files same as typ=RSV with index added

The file structure described above for typ=RSV files also applies to typ=IDX files which include an indexed partition & some extra system records in the data partition. You must use the alternate program 'uxcp' (vs 'uvcp) if either input or output or both files are typ=IDX.

Goto:   Begin this document End this document UVSI Home-Page

J1. 'uxcp' test/demo Indexed Seqntl Variable length files

test/demo uxcp for Indexed Seqntl Variable length files

'uxcp' supports Indexed Sequential Variable length record files & must be compiled with the Micro Focus COBOL 'cob' command as shown on page D1 of install.htm. You can demo uxcp using test files provided as shown below:

dat1/idxtest - test file to demo uxcp

 0010.0060 idxtest text file to create Var Lth Indexed files.
 0020.0064 - input to COBOL,uxcp,uxcopy to create IDXf1,3,8 files
 0030.0065 - 01-04=ISAM key, 05=separator, 06-09=size, 10-400 text
 0040.0067 - output record varies from 64 to 256, depending on 06-09
 0050.0080 - here is an 80 byte record------------------------------------------>
 0060.0160 - here is a 160 byte record...........................................
 0070.0240 - here is a 240 byte record (last record in file).....................
 ===============================================================================>

uxcp - READ text file (typ=LST), WRITE Indexed file (typ=IDX)


 #1. uxcp "fili1=dat1/idxtest,rcs=256,typ=LST\
     ==========================================
             ,filo1=tmp/IDXf3v60,rcs=254,typ=IDXf3v60,isk1=0(4)"
             ===================================================

 #2. uvhd tmp/IDXf3v60.dat v  <-- examine output data partition with 'uvhd'
     =======================    - use 'v' option for IDXf3 files
Note
  • See uvhd output (in vertical hexadecimal) on the next page --->

uxcp - READ Indexed file (typ=IDX), WRITE text file (typ=LSTt)


 #3. uxcp "fili1=tmp/IDXf3v60,rcs=256,typ=IDXf3,filo1=tmp/idxf3.txt,typ=LSTt"
     ========================================================================

 #4. vi tmp/idxf3.txt          <-- examine output file with 'vi'
     ================

This converts the typ=IDX file back to a text file (typ=LSTt) which should be the same as the original file used in step #1. Note that you do not need to specify option 'v' on input files (only when creating output files).

Goto:   Begin this document End this document UVSI Home-Page

J2. uvcp: 'uxcp' test/demo Indexed Seqntl Variable length files

output uvhd hexdump of IDXf3 varlth indexed file


 #1a. uvhd tmp/IDXf3v60.dat v   <-- examine output data partition with 'uvhd'
      =======================
                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 0~......0705141326133507051413261335.>.......................<..
             370000003333333333333333333333333333030000000000000000000F000300
             0E00000007051413261335070514132613350E0200000000100000000E000C00
          64 ............................................X"8*................
             0000000000000000000000000000000000000000000052320000000000000000
             00000000000020100000000000000000000000000000828A0000000000000000
                      10        20        30        40        50        60
 r#        2 0123456789012345678901234567890123456789012345678901234567890123
         384 @<0010.0060 idxtest text file to create Var Lth Indexed files...
             4333332333326677677276772666627626766762567247624666766266667200
             0C0010E00600948453404584069C504F032514506120C4809E45854069C53E00
                      10        20        30        40        50        60
 r#        3 0123456789012345678901234567890123456789012345678901234567890123
         448 @@0020.0064 - input to COBOL,uxcp,uxcopy to create IDXf1,3,8 fil
             4433332333322266777276244444277672776677276267667624456323232666
             000020E00640D09E05404F03F2FCC5830C583F0904F0325145094861C3C8069C
          64 es..
             6700
             5300
                      10        20        30        40        50        60
 r#        4 0123456789012345678901234567890123456789012345678901234567890123
         516 @A0030.0065 - 01-04=ISAM key, 05=separator, 06-09=size, 10-400 t
             4433332333322233233345442667223337676767672233233376762233233327
             010030E00650D001D04D931D0B59C005D3501214F2C006D09D39A5C010D40004
          64 ext.
             6770
             5840
                      10        20        30        40        50        60
 r#        5 0123456789012345678901234567890123456789012345678901234567890123
         584 @C0040.0067 - output record varies from 64 to 256, depending on
             4433332333322267777727666762767667267662332762333226676666662662
             030040E00670D0F540540253F240612953062FD06404F0256C04505E49E70FE0
          64 06-09...
             33233000
             06D09000
                      10        20        30        40        50        60
 r#        6 0123456789012345678901234567890123456789012345678901234567890123
         656 @P0050.0080 - here is an 80 byte record-------------------------
             4533332333322266762672662332677627666762222222222222222222222222
             000050E00800D0852509301E080029450253F24DDDDDDDDDDDDDDDDDDDDDDDDD
          64 ----------------->..
             22222222222222222300
             DDDDDDDDDDDDDDDDDE00
                ----- last 3 records omitted to keep this on 1 page -----

Goto:   Begin this document End this document UVSI Home-Page

J3. 'uxcp' test/demo Indexed Seqntl Variable length files

create IDXf8 varlth indexed file & examine with uvhd


 #1. uxcp "fili1=dat1/idxtest,rcs=256,typ=LST\
     ==========================================
             ,filo1=tmp/IDXf8v60,rcs=254,typ=IDXf8v60,isk1=0(4)"
             ===================================================

 #2. uvhd tmp/IDXf8v60.dat v  <-- examine output data partition with 'uvhd'
     =======================
                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 3.......0705141522382607051415223827.>.......................<..
             3F0000003333333333333333333333333333030000000000000000000F000300
             3E00000007051415223826070514152238270E0200180000106000000E000C00
          64 ............................................X"8*X"8*............
             0000000000000000000000000000000000000000000052325232000000000011
             00000007000022100000000000000000000000000008828A828A000000000000
         128 ................................................................
             00000011000000000000000000000000000000000000000F0000000000000000
             000000004244010400000040000000C0000000000000003E0000000000000000
                     ----- bytes 192-960 removed (all x'00's) -----
         960 ................................................................
                      10        20        30        40        50        60
 r#        2 0123456789012345678901234567890123456789012345678901234567890123
        3328 @<0010.0060 idxtest text file to create Var Lth Indexed files...
             4333332333326677677276772666627626766762567247624666766266667200
             0C0010E00600948453404584069C504F032514506120C4809E45854069C53E00
                      10        20        30        40        50        60
 r#        3 0123456789012345678901234567890123456789012345678901234567890123
        3392 @@0020.0064 - input to COBOL,uxcp,uxcopy to create IDXf1,3,8 fil
             4433332333322266777276244444277672776677276267667624456323232666
             000020E00640D09E05404F03F2FCC5830C583F0904F0325145094861C3C8069C
          64 es..
             6700
             5300
                      10        20        30        40        50        60
 r#        4 0123456789012345678901234567890123456789012345678901234567890123
        3464 @A0030.0065 - 01-04=ISAM key, 05=separator, 06-09=size, 10-400 t
             4433332333322233233345442667223337676767672233233376762233233327
             010030E00650D001D04D931D0B59C005D3501214F2C006D09D39A5C010D40004
          64 ext.
             6770
             5840
                      10        20        30        40        50        60
 r#        5 0123456789012345678901234567890123456789012345678901234567890123
        3536 @C0040.0067 - output record varies from 64 to 256, depending on
             4433332333322267777727666762767667267662332762333226676666662662
             030040E00670D0F540540253F240612953062FD06404F0256C04505E49E70FE0
          64 06-09...
             33233000
             06D09000
                    ----- records 6,7,8 not shown to fit 1 page -----

Goto:   Begin this document End this document UVSI Home-Page

J4. 'uxcp' test/demo Indexed Seqntl Variable length files

IDXORMAT3/IDXFORMAT8 file header layout

The Vancouver Utilities (uxcopy,uxsort,uxqrpg,uxcp,uvhd) are compatible with Micro Focus COBOL variable length sequential & Indexed files (IDXFORMAT3 & 8).

We will document the most important fields in the file header for IDXf3/IDXf8. If desired, you can find all fields defined on the Micro Focus website.

Note that you can investigate the file headers using 'uvhd' with option 'v' for IDXFORMAT3 or IDXFORMAT8. See sample on the previous page.

 000-003 - file header record length
         - 1st 4 bits x'30' identifies a system record
         - next 12 bits is header length if max record length is < 4095
         - next 28 bits is header length if max record length is => 4095

008-014 - creation date/time yymmddhhmmsscc (indexed files only)

 039-039 - organization (1=seqntl, 2=indexed, 3=relative)
 043-043 - file type (3=IDXFORMAT3, 8=IDXFORMAT8)
 048-048 - recording mode (0=fixed, 1=variable)
 068-071 - record count (4 bytes Big-End Binary)
         - Indexed files only (not in variable length non-indexed file hdrs)
 054-057 - maximum record length (big end format)
         - example recsize 80 = x'00000050'
 058-061 - minimum record length

128-135 - offset to End-of-File (8 bytes binary big end format)

 140-141  - number of keys in Indexed file

record header layout

 000-000 - 1st 4 bits identifies the record type
 x'1_'   - system record - duplicate occurrence record
 x'2_'   - deleted record
 x'3_'   - system record
 x'4_'   - user data record
 x'5_'   - reduced user data record
 x'6_'   - pointer record
 x'7_'   - user data record referenced by a pointer record
 x'8_'   - reduced user data record referenced by a pointer record

000-001 - next 12 bits is the record length if max recsize < 4095 000-003 - next 28 bits is the record length if max recsize =? 4095 002-... - record data if max recsize < 4095 004-... - record data if max recsize => 4095

Note
  • all records are padded with x'00's to a multiple of 4 bytes

Goto:   Begin this document End this document UVSI Home-Page

J5. 'uxcp' test/demo Indexed Seqntl Variable length files

scripts to test typ=IDX file options

The next few pages will list several scripts used at UV Software to test/demo using uxcp to write IDXFORMAT3/IDXFORMAT8 files & read them back to verify.

These scripts are supplied in /home/uvadm/sf/demo/..., which is in the PATH of the supplied profile. You can run these at your site. See instructions & explanations embedded within the scripts.

uxcpIDXf3w
  • write various IDXf3 formats, run prior to uxcpIDXf3r
  • reads a text file & writes out 6 files with various options
uxcpIDXf3r
  • read back IDXf3 files & convert back to text files
    to verify result same as original text file input
uxcpIDXf3c
  • copy IDXf3 files to same output file types & record-sizes
  • to prove we can copy unchanged (passing recsizes to output)
  • but you can add selects/deletes & modify instructions
Note
  • we will list the scripts only for the IDXf8 files
  • IDXf3 scripts are similar & you can see in /home/uvadm/sf/demo/...
J6. uxcpIDXf8w - write various IDXf8 files for read back by uxcpIDXf8r
  - reads a text file & writes out 6 files with various options
J7. uxcpIDXf8r - read back various IDXf8 files written by uxcpIDXf8w
  to verify result same as original text file input
J8. uxcpIDXf8c - copy IDXf3 files to same output file types & record-sizes
  - to prove we can copy unchanged (passing recsizes to output)
  - but you can add selects/deletes & modify instructions
J9. tmp1 subdir listings after uxcpIDXf3w & after uxcpIDXf8w
Note
  • see VSECOBOL.htm for samples of COBOL with IDXf3/IDXf8 files
  • also includes scripts to prove uxcp compatibility with COBOL

Goto:   Begin this document End this document UVSI Home-Page

J6. 'uxcp' test/demo Indexed Seqntl Variable length files

script to WRITE typ=IDXFORMAT8 files

 # uxcpIDXf8w - write various IDXFORMAT files to test uxcp
 #            - by Owen Townsend, Jan28/06, updated Feb22/07
 #            - IDXFORMAT8 files created by Micro Focus COBOL EXTFH
 #            - file handler used by uxcp, uxsort, uxcopy, uxqrpg
 #
 #*uxcpIDXf8w - write various IDXf8 files for read back by uxcpIDXf8r <--this job
 # uxcpIDXf8r - read back various IDXf8 files written by uxcpIDXf8w
 # uxcpIDXf8c - Copy IDXf3 files to same output file types & record sizes
 # uxcpIDXf3w - write various IDXf3 formats in tmp1, run prior to uxcpIDXf3r
 # uxcpIDXf3r - read back IDXf3 files & convert back to text files in tmp2
 # uxcpIDXf3c - Copy IDXf3 files to same output file types & record sizes
 #
 # OPTIONS: f1=IDXFORMAT1, f3=IDXFORMAT3, f8=IDXFORMAT8 (see www.microfocus.com)
 # x1=noindex, x2=indexed; y1=fixed, y2=variable, y4=scanback to last non-blank
 # v=min recsize, (maxsize declared by rcs=___)
 #
 # 1. typ=LST-->IDXf8 indexed, data maxsz 252+4 (since no minsz v optn)
 uxcp "fili1=dat1/idxtest,typ=LST,rcs=252,filo1=tmp1/idxf8,typ=IDXf8,isk1=0(4)"
 #===========================================================================
 #
 # 2. typ=LST-->IDXf8v60 indexed, variable recsize passed thru from input
 uxcp "fili1=dat1/idxtest,typ=LST,rcs=252,filo1=tmp1/idxf8v60,typ=IDXf8v60,isk1=0(4)"
 #===============================================================================
 #
 # 3. typ=LST-->IDXf8v60y4 indexed variable min v60+4, depending on last nonblank
 uxcp "fili1=dat1/idxtest,typ=LST,rcs=252,filo1=tmp1/idxf8v60y4,typ=IDXf8v60y4,isk1=0(4)"
 #===============================================================================
 #
 # 4. typ=LST-->IDXf8x1 noindex(x1), fixed 252, no IDX filehdr, same as typ=RSF
 uxcp "fili1=dat1/idxtest,typ=LST,rcs=252,filo1=tmp1/idxf8x1,typ=IDXf8x1,isk1=0(4)"
 #===============================================================================
 #note - must read back using typ=RSF, since no index & fixed lth (no v)
 #
 # 5. typ=LST-->IDXf8v60x1 noindex(x1), varfmt, but all 252+4
 uxcp "fili1=dat1/idxtest,typ=LST,rcs=252,filo1=tmp1/idxf8v60x1,typ=IDXf8v60x1,isk1=0(4)"
 #===============================================================================
 #
 # 6. typ=LST-->IDXf8v60x1y4 noindex(x1), variable 60-252, depending on LNB
 uxcp "fili1=dat1/idxtest,typ=LST,rcs=252,filo1=tmp1/idxf8v60x1y4,typ=IDXf8v60x1y4,isk1=0(4)"
 #===============================================================================
 #
 echo "You can examine output files with 'uvhd', example for output#1 above"
 echo " "
 echo "uvhd tmp1/idxf8.dat v   <-- use 'v' option for IDXf8 files"
 echo "====================="
 #
 #Note - script may bomb on uxcp #6 if you only have 1 Micro Focus COBOL license
 #     - But you can add following to your profile to wait 1 minute (vs bomb)
 #
 # export ASLMFRETRY="5,15"
 # ========================

Goto:   Begin this document End this document UVSI Home-Page

J7. 'uxcp' test/demo Indexed Seqntl Variable length files

script to READ typ=IDXFORMAT8 files

 # uxcpIDXf8r - read back various IDXFORMAT3 files to test uxcp
 #              (files written by prior job uxcpIDXf8w)
 #            - this job converts back to text files (typ=LSTt)
 #            - by Owen Townsend, Jan28/06, updated Feb22/07
 #            - IDXFORMAT8 files created by Micro Focus COBOL EXTFH
 #            - file handler used by uxcp, uxsort, uxcopy, uxqrpg
 #
 # uxcpIDXf8w - write various IDXf8 files for read back by uxcpIDXf8r
 #*uxcpIDXf8r - read back various IDXf8 files written by uxcpIDXf8w  <--this job
 # uxcpIDXf8c - Copy IDXf3 files to same output file types & record sizes
 # uxcpIDXf3w - write various IDXf3 formats, run prior to uxcpIDXf3r
 # uxcpIDXf3r - read back IDXf3 files & convert back to text files
 # uxcpIDXf3c - Copy IDXf3 files to same output file types & record sizes
 #
 # options: f1=IDXFORMAT1, f3=IDXFORMAT3, f8=IDXFORMAT8 (see www.microfocus.com)
 # x1=seqntl, x2=indexed; y1=fixed, y2=variable, y4=set rcsz after last nonblank
 # v=min recsize, (maxsize declared by rcs=___)
 #
 # 1. typ=IDXf8-->LSTt - input IDXf8 indexed fixed (no option v)
 uxcp "fili1=tmp1/idxf8,typ=IDXf8,rcs=252,filo1=tmp2/idxf8L,typ=LSTt"
 #===================================================================
 #
 # 2. typ=IDXf8v60-->LSTt - input variable from 60+4 to 252+4
 uxcp "fili1=tmp1/idxf8v60,typ=IDXf8,rcs=252,filo1=tmp2/idxf8v60L,typ=LSTt"
 #=========================================================================
 #
 # 3. typ=IDXf8v60y4-->LSTt - input variable determined by y4 scanback to LNB
 uxcp "fili1=tmp1/idxf8v60y4,typ=IDXf8,rcs=252,filo1=tmp2/idxf8v60y4L,typ=LSTt"
 #=============================================================================
 #
 # 4. typ=IDXf8x1-->LSTt - input IDXf8 noindex, all 252+4
 uxcp "fili1=tmp1/idxf8x1,typ=RSF,rcs=252,filo1=tmp2/idxf8x1L,typ=LSTt"
 #=======================================================================
 #note - must read back using typ=RSF, since no index & fixed lth (no v)
 #
 # 5. typ=IDXf8v60x1-->LSTt - input IDXf8 noindex, varfmt, all 252+4
 uxcp "fili1=tmp1/idxf8v60x1,typ=IDXf8,rcs=252,filo1=tmp2/idxf8v60x1L,typ=LSTt"
 #=============================================================================
 #
 # 6. typ=IDXf8v60x1y4-->LSTt - input IDXf8 noindex, varfmt, 60+4 to 252+4, y4
 uxcp "fili1=tmp1/idxf8v60x1y4,typ=IDXf8,rcs=252,filo1=tmp2/idxf8v60x1y4L,typ=LSTt"
 #=============================================================================
 #
 echo "All outfiles in tmp2/... should be same as original input to uxcpIDXf3w"
 echo " "
 echo "--> cat tmp2/idxf8L  <-- display output file, example for #1 above"
 echo "    ==============="
 #
 #Note - on inputs you only need to specify the file type (IDXf1,IDXf3,IDXf8)
 #     - options v,x,y need not be specified
 #     - EXTFH file handler reads existing files OK

Goto:   Begin this document End this document UVSI Home-Page

J8. 'uxcp' test/demo Indexed Seqntl Variable length files

script to COPY typ=IDXFORMAT8 files

 # uxcpIDXf8c - copy various IDXFORMAT3 files to test uxcp
 #              (files written by prior job uxcpIDXf8w)
 #            - copy the IDXf3 files to same output file types & recsizes
 #            - IDXFORMAT8 files created by Micro Focus COBOL EXTFH
 #            - file handler used by uxcp, uxsort, uxcopy, uxqrpg
 #            - by Owen Townsend, Jan28/06, updated May14/07
 #
 # uxcpIDXf8w - write various IDXf8 files for read back by uxcpIDXf8r
 # uxcpIDXf8r - read back various IDXf8 files written by uxcpIDXf8w
 #*uxcpIDXf8c - Copy IDXf3 files to same output file types & recsizes <--this job
 # uxcpIDXf3w - write various IDXf3 formats, run prior to uxcpIDXf3r
 # uxcpIDXf3r - read back IDXf3 files & convert back to text files
 # uxcpIDXf3c - Copy IDXf3 files to same output file types & record sizes
 #
 # options: f1=IDXFORMAT1, f3=IDXFORMAT3, f8=IDXFORMAT8 (see www.microfocus.com)
 # x1=seqntl, x2=indexed; y1=fixed, y2=variable, y4=set rcsz after last nonblank
 # v=min recsize, (maxsize declared by rcs=___)
 #
 # 1. copy typ=IDXf8 indexed but fixed 252+4 since no optn v
 uxcp "fili1=tmp1/idxf8,typ=IDXf8,rcs=252,filo1=tmp3/idxf8,typ=IDXf8"
 #===================================================================
 #
 # 2. copy typ=IDXf8v60 variable, passes input recsizes to output file
 uxcp "fili1=tmp1/idxf8v60,typ=IDXf8,rcs=252,filo1=tmp3/idxf8v60,typ=IDXf8v60"
 #============================================================================
 #
 # 3. copy typ=IDXf8v60y4, option y4 scans back to last nonblank for out recsize
 uxcp "fili1=tmp1/idxf8v60y4,typ=IDXf8,rcs=252,filo1=tmp3/idxf8v60y4,typ=IDXf8v60"
 #===============================================================================
 #
 # 4. copy typ=IDXf8x1 noindex, option x1 (fixed lth), all out recs 252+4 bytes
 uxcp "fili1=tmp1/idxf8x1,typ=RSF,rcs=252,filo1=tmp3/idxf8x1,typ=IDXf8x1"
 #=======================================================================
 #note - must read back using typ=RSF, since no index & fixed lth (no v)
 #
 # 5. copy typ=IDXf8v60x1 noindex, varfmt, all 252+4
 uxcp "fili1=tmp1/idxf8v60x1,typ=IDXf8,rcs=252,filo1=tmp3/idxf8v60x1,typ=IDXf8v60x1"
 #=============================================================================
 #
 # 6. copy typ=IDXf8v60x1y4 noindex, varfmt, 60+4 to 252+4, y4
 uxcp "fili1=tmp1/idxf8v60x1y4,typ=IDXf8,rcs=252,filo1=tmp3/idxf8v60x1y4,typ=IDXf8v60x1y4"
 #=============================================================================
 #
 echo "All outfiles in tmp3/... should be same as original input to uxcpIDXf3w"
 echo " "
 echo "--> cat tmp3/idxf8L  <-- display output file, example for #1 above"
 echo "    ==============="
 #
 #Note - on inputs you only need to specify the file type (IDXf1,IDXf3,IDXf8)
 #     - options v,x,y need not be specified
 #     - EXTFH file handler reads existing files OK

Goto:   Begin this document End this document UVSI Home-Page

J9. 'uxcp' test/demo Indexed Seqntl Variable length files

input text file for uxcpIDXf3/uxcpIDXf8

 0010.0060 idxtest text file to create Var Lth Indexed files.
 0020.0064 - input to COBOL,uxcp,uxcopy to create IDXf1,3,8 files
 0030.0065 - 01-04=ISAM key, 05=separator, 06-09=size, 10-400 text
 0040.0067 - output record varies from 64 to 256, depending on 06-09
 0050.0080 - here is an 80 byte record------------------------------------------>
 0060.0160 - here is a 160 byte record...........................................
 0070.0240 - here is a 240 byte record (last record in file).....................
 ===============================================================================>

tmp1 subdir listings after uxcpIDXf3w


 #1. -rw-rw-r--    1 uvadm    users     2176 May 14 13:50 idxf3.dat
     -rw-rw-r--    1 uvadm    users     3072 May 14 13:50 idxf3.idx

 #2. -rw-rw-r--    1 uvadm    users     1148 May 14 13:50 idxf3v60.dat
     -rw-rw-r--    1 uvadm    users     3072 May 14 13:50 idxf3v60.idx

 #3. -rw-rw-r--    1 uvadm    users     1148 May 14 13:50 idxf3v60y4.dat
     -rw-rw-r--    1 uvadm    users     3072 May 14 13:50 idxf3v60y4.idx

 #4. -rw-rw-r--    1 uvadm    users     1920 May 14 13:50 idxf3x1

 #5. -rw-rw-r--    1 uvadm    users      892 May 14 13:50 idxf3v60x1

 #6. -rw-rw-r--    1 uvadm    users      892 May 14 13:50 idxf3v60x1y4

tmp1 subdir listings after uxcpIDXf8w


 #1. -rw-rw-r--    1 uvadm    users     4864 May 14 15:26 idxf8.dat

 #2. -rw-rw-r--    1 uvadm    users     4112 May 14 15:26 idxf8v60.dat

 #3. -rw-rw-r--    1 uvadm    users     4112 May 14 15:26 idxf8v60y4.dat

 #4. -rw-rw-r--    1 uvadm    users     1764 May 14 15:26 idxf8x1

 #5. -rw-rw-r--    1 uvadm    users      892 May 14 15:26 idxf8v60x1

 #6. -rw-rw-r--    1 uvadm    users      892 May 14 15:26 idxf8v60x1y4
Note
  • IDXf3 Indexed files have 2 partitions .dat & .idx
  • IDXf8 files have only the .dat partition, but it includes indexes
  • unless option 'x1' is specified for a non-indexed file

Please see 'uvhd' of IDXF3v60 on page 'J2' & IDXf8v60 on page 'J3'.

Please see uvcopy6.htm for IDXf3/IDXf8 test/demos (similar to the above), but for 'uxcopy' vs 'uxcp'.

Goto:   Begin this document End this document UVSI Home-Page

K1. uvcp: RDW (Record Descriptor Word) variable length files

RDW (Record Descriptor Word) variable length files are often used to FTP variable length EBCDIC files from the mainframe to unix/linux/windows systems.

uvcp provides for the following types of RDW files:

RDWz4
  • 4 byte record prefix with 2 byte binary record size & 2 unused nulls
  • record size in prefix is the total record size including the prefix
  • 2 byte record size is 'BIG-END' binary (like mainframe or SUN,AIX,DEC)
  • 'little-end' machines (Intel,AMD) need to switch ends to perform math
  • RDWz4 is the default (typ=RDW defaults to typ=RDWz4)
RDWz2
  • option 'z2' specifies record prefix only 2 bytes (2 nulls omitted)
  • record size in prefix is the data record size excluding the prefix
RDWz18
  • (z2+z16) similar to z4 STANDARD RDW
  • recsize in hdr is TOTAL slot size (includes 2 byte hdr)
  • 2 byte hdr, recsize 1st 2 bytes binary BIG-end
RDWn4
  • option 'n4' specifies 4 byte record prefix is Numeric digits (vs binary)
  • record size in prefix is the total record size including the prefix

test/demo files supplied in /home/uvadm/dat1

dat1/testLST
  • text file originally used to create the testRDW files
dat1/testRDW
  • RDW file created from testLST
dat1/testRDWe
  • RDW file, same as above in EBCDIC

We will illustrate how to convert RDW EBCDIC files using both 'uvcp' & 'uvhd'.

You could use 'uvhd' when there you only have a few files to do,& when the interactive utility is preferred.

You could use 'uvcp' when you want to write a script to convert several files and for repeatability.

Note that neither of uvhd or uvcp can be used for files with packed or binary fields. For those please see DATAcnv1.htm to generate data conversion jobs that preserve packed & binary fields as defined by COBOL copybook record layouts. For high volume conversions of files with packed/binary fields see MVSDATA.htm or VSEDATA.htm.

We will 1st investigate the testRDWe file using 'uvhd'. We can not use unix tools (vi, lp, etc) since the file is EBCDIC & contains binary record sizes.

Goto:   Begin this document End this document UVSI Home-Page

K2. processing RDW variable length files with uvcp

Investigating RDW files with uvhd


 uvhd dat1/testRDWe as3 <-- examine the RDW test file
 ======================   - option 'a' translates character line to ASCII
                          - option 's3' to space between scale & 3 line groups
                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 ....DELL00 - Dell Inc.  ....HP0000 - Hewlett Packard....IBM000 -
             0100CCDDFF464C8994C984440100CDFFFF464C8A98AA4D8898980200CCDFFF46
             080045330000045330953B000C008700000008563533071321940C0092400000
          64  International Business Machines....MFC000 - Micro Focus COBOL
             4C9A8998A899894CAA898AA4D888898A0200DCCFFF464D88994C98AA4CDCDD44
             0953595139651302429552204138955200004630000004939606634203626300
         128 ....MS0000 - Microsoft Corp.....REDHAT - Red Hat Linux  ....SUN0
             0100DEFFFF464D8899A98A4C99940100DCCCCE464D884C8A4D89AA440200EEDF
             0C0042000000049396266303697B0C0095481300095408130395470004002450
         192 00 - Sun Microsystems Ltd   ....UVSI00 - UV Software Inc.
             FF464EA94D8899AAAA89A4DA84440200EEECFF464EE4E98AA8984C984444
             00000245049396282354203340000000452900000450266361950953B000

The entire file is only 252 bytes & contains 8 short variable length records:

uvhd option 'z4' to recognize RDW files

Option 'z4' tells uvhd to look for the 'RDW' record prefixes & show 1 record at a time (vs multiple records as fits in a 256 byte block). The default is to show any file in 256 byte blocks (4 groups of 3 64 byte lines for characters, zones,& digits).

See the next page where we will specify option 'z4' to show RDW files 1 record at a time. You can press enter to browse forward until EOF reached. Then you could enter '1' to return to the begining of the file.

Goto:   Begin this document End this document UVSI Home-Page

K3. processing RDW variable length files with uvcp

Investigating RDW files using option 'z4'


 uvhd dat1/testRDWe z4a   <-- display RDW file using option 'z4'
 ======================     - option 'a' to display character line in ASCII
                            - null entries browse forward til EOF reached
                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 ....DELL10 - Dell Inc.
             0100CCDDFF464C8994C98444
             080045331000045330953B00
                      10        20        30        40        50        60
 r#        2 0123456789012345678901234567890123456789012345678901234567890123
          24 ....HP0010 - Hewlett Packard
             0100CDFFFF464C8A98AA4D889898
             0C00870010000856353307132194
                      10        20        30        40        50        60
 r#        3 0123456789012345678901234567890123456789012345678901234567890123
          52 ....IBM010 - International Business Machines
             0200CCDFFF464C9A8998A899894CAA898AA4D888898A
             0C009240100009535951396513024295522041389552
                      10        20        30        40        50        60
 r#        4 0123456789012345678901234567890123456789012345678901234567890123
          96 ....MFC010 - Micro Focus COBOL
             0200DCCFFF464D88994C98AA4CDCDD44
             00004630100004939606634203626300
                      10        20        30        40        50        60
 r#        5 0123456789012345678901234567890123456789012345678901234567890123
         128 ....MS0010 - Microsoft Corp.
             0100DEFFFF464D8899A98A4C9994
             0C0042001000049396266303697B
                      10        20        30        40        50        60
 r#        6 0123456789012345678901234567890123456789012345678901234567890123
         156 ....REDH10 - Red Hat Linux
             0100DCCCFF464D884C8A4D89AA44
             0C00954810000954081303954700
                      10        20        30        40        50        60
 r#        7 0123456789012345678901234567890123456789012345678901234567890123
         184 ....SUN010 - Sun Microsystems Ltd
             0200EEDFFF464EA94D8899AAAA89A4DA8444
             040024501000024504939628235420334000
                      10        20        30        40        50        60
 r#        8 0123456789012345678901234567890123456789012345678901234567890123
         220 ....UVSI10 - UV Software Inc.
             0200EEECFF464EE4E98AA8984C984444
             0000452910000450266361950953B000
Note
  • option 'a' translated the character line to ASCII (so you can read it)
  • But the zones & digits lines show the data is EBCDIC
  • For example the 'D' of Dell is x'C4' (vs x'44' in ASCII)

Goto:   Begin this document End this document UVSI Home-Page

K4. RDW (Record Descriptor Word) variable length files

convert EBCDIC RDWz4 files to ASCII text

On this page we will show you how to convert EBCDIC RDWz4 files to ASCII text using 'uvhd'. On the next page we will do the same with 'uvcp'.

We will specify uvhd options 'z4a3p4y7' which mean:

z4
  • recognize RDWz4 files
a3
  • a3=a1+a2
a1
  • translate character line of display to ASCII
a2
  • translate any 'Write' command output to ASCII
p4
  • drop 4 byte prefix on write output
y7
  • y7=y1+y2+y4
y1
  • insert CR at end of record
y2
  • insert LF at end of record
y4
  • scan back to last nonblank to insert the CR/LF

 uvhd dat1/testRDWe z4a3p4y7   <-- display 1st record & wait for command
 ===========================
                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 ....DELL10 - Dell Inc.
             0100CCDDFF464C8994C98444
             080045331000045330953B00
 ---> w99999  <-- write all records (8) to tmp/testRDWe_yymmdd_hhmmssW
                - output filename will be date/time stamped
                - On Dec 18, 20007 at 12:15 tmp/testRDWe_071218_121500W
 ---> q       <--- quit uvhd

 vi tmp/testRDWe_071218_121500W
 ==============================
 vi tmp/*00W       <-- shortcut to display desired file
 ===========

text records extracted from RDWz4 file

DELL00 - Dell Inc. HP0000 - Hewlett Packard IBM000 - International Business Machines MFC000 - Micro Focus COBOL MS0000 - Microsoft Corp. REDHAT - Red Hat Linux SUN000 - Sun Microsystems Ltd UVSI00 - UV Software Inc.

Goto:   Begin this document End this document UVSI Home-Page

K5. processing RDWz4 variable length files with uvcp

using uvcp to convert EBCDIC RDWz4 files

We will now use 'uvcp' to perform the conversion from EBCDIC RDWz4 to ASCII.

Instead of variable length text, we will create fixed length output records that are required for most COBOL input files. For this demo, we will convert to fixed 64 byte records & insert a LineFeed in the last byte so we can have it both ways (fixed length for COBOL, but LFs allow investigation with vi).


 uvcp "fili1=dat1/testRDWe,rcs=64,typ=RDWz4,filo1=tmp/testz4,typ=RST,tra=0(64)"
 ==============================================================================
tra=0(64)
  • translates the records to ASCII.
typ=RST
  • output 'Record Sequential Terminated' (inserts LineFeed in last byte)

confirm conversion OK with uvhd


 uvhd tmp/testz4 r64h2     <-- display output file with uvhd
 =====================       - option r64 for record size
                             - option 'h2' for hex display to see LineFeed
                             - only 1st record shown below
                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 DELL10 - Dell Inc.                                             .
             4444332224666246622222222222222222222222222222222222222222222220
             45CC100D045CC09E3E000000000000000000000000000000000000000000000A

Confirmations shown by uvhd

  1. Binary record prefixes have been dropped off, because output typ=RST extracts the data record only & omits the typ=RDWz4 record prefix.

  2. LineFeeds in last byte of each record (x'0A's).

  3. Output is ASCII 1st char 'D' is x'44' here vs x'C4' on page 'K2'&'K3' Translation performed by instruction 'tra=0(64)'.

  4. See pages 'M0'-M19 for other instructions you can use to reformat data (mvc, clr, rep, tre, tru, trl, trp, trb, sel, del, etc...).

    see DATAcnv1 for more RDW examples

Note
  • also see DATAcnv1.htm for more examples of data conversions
  • stripping BDW/RDW record prefixes
  • converting variable length files to fixed length
  • translating from EBCDIC to ASCII, allowing for packed & signed fields

Goto:   Begin this document End this document UVSI Home-Page

K6. processing RDWz2 variable length files with uvcp

using uvcp to convert RDWz2 files

This page is similar to the previous page, but for typ=RDWz2 (2 byte prefixes) vs RDWz4 (4 byte prefixes). We will use uvhd to show the 1st 2 input records, then the 'uvcp' command with 'typ=RDWz2',& then the 1st 2 output records (with binary record length prefix stripped & blank filled to 64 bytes).

EBCDIC input file typ=RDWz2 (2 byte hdrs)


 uvhd dat1/testRDWez2 z2a    <-- display RDW file, option z2 for 2 byte hdrs
 ========================      - option 'a' to translate char line to Ascii
                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 ..DELL10 - Dell Inc.
             01CCDDFF464C8994C98444
             0445331000045330953B00
                      10        20        30        40        50        60
 r#        2 0123456789012345678901234567890123456789012345678901234567890123
          22 ..HP0010 - Hewlett Packard
             01CDFFFF464C8A98AA4D889898
             08870010000856353307132194
Note
  • 1st recsize = x'0014' = 1*16+4=20 (but scale shows 22 bytes)
  • the prefix size (2 bytes) is EXCLUDED from record size (add 2 bytes)

strip typ=RDWz2 hdrs, translate, write fixed lth records


 uvcp "fili1=dat1/testRDWez2,rcs=64,typ=RDWz2,filo1=tmp/testz2,typ=RST,tra=0(64)"
 ================================================================================

confirm conversion OK with uvhd


 uvhd tmp/testz2 r64h2     <-- display output file with uvhd
 =====================       - option r64 for record size
                             - option 'h2' for hex display to see LineFeeds
                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 DELL10 - Dell Inc.                                             .
             4444332224666246622222222222222222222222222222222222222222222220
             45CC100D045CC09E3E000000000000000000000000000000000000000000000A
                      10        20        30        40        50        60
 r#        2 0123456789012345678901234567890123456789012345678901234567890123
          64 HP0010 - Hewlett Packard                                       .
             4533332224676677256666762222222222222222222222222222222222222220
             8000100D0857C5440013B124000000000000000000000000000000000000000A

Goto:   Begin this document End this document UVSI Home-Page

L1. uvcp INSTRUCTION SUMMARY

      logical:  mvc, mvn, edt, clr
   arithmetic:  add, sub, mpy, div, pac, unp, seq
 compare/skip:  cmc, cmn, skp
 scan/replace:  rep, scn, sqz
   translates:  tra, tre, trl, tru, trp, trb
        other:  anc, orc, chx, hxc, tim
          I/O:  put

uvcp INSTRUCTION FORMAT SUMMARY

 mvc=tobyte(lth)frombyte         - move character data field
 mvn=to(lth)from(lth)            - move numeric data field (right adjusted)
 edt=to(lth)from(lth):zz;zzz.99  - edit
 clr=start(lth):constant         - clear an area to a specified value
 tra=start(length)               - translate to ASCII  (from EBCDIC)
 tre=start(length)               - translate to EBCDIC (from ASCII)
 trl=start(length)               - translate to lower case (any upper case)
 tru=start(length)               - translate to UPPER case (any lower case)
 trp=start(length)               - translate unprintable characters to '.' periods
 trb=start(length)               - translate unprintable characters to ' ' blanks
 add=op1dsp(lth)op2dsp(lth)      - add op2 data to op1 data, result in op1
 sub=op1dsp(lth)op2dsp(lth)      - subtract op2 data from op1, result in op1
 mpy=op1dsp(lth)op2dsp(lth)      - multiply op2 by op1, result in op1
 div=op1dsp(lth)op2dsp(lth)      - divide op1 by op2, result in op1
 pac=op1dsp(lth)op2dsp(lth)      - pack op2 data into op1 area
 unp=op1dsp(lth)op2dsp(lth)      - unpack op2 data into op1 area
 seq=op1dsp(lth)                 - sequence#
 anc=start(lth):mask             - and character (erase bits)
 orc=start(lth):mask             - or character  (add bits)
 chx=op1start(lth)op2start       - convert op2 hex representation to data in op1
 hxc=op1start(lth)op2start       - convert op2 data to hex representation in op1
 rep=strt(lth):patrn:alt         - replace pattern with alternate
 scn=start(lth):patrn            - scan for a pattern, set cc equal if found
 sqz=start(lth):constant         - squeeze out the op2 character from op1
 cmc=op1strt(lth)op2strt         - compare op1 to op2 & set condition code
 skp?=n                          - test condition code & skip n instrns
 cmn=op1strt(lth)op2strt(lth)    - compare numeric & set condition code
                                 - op1 & op2 may be any combination of
                                   packed, zoned, or binary
 put=b0(rcsz)                    - write current record (exception output)

Goto:   Begin this document End this document UVSI Home-Page

L2. uvcp: INSTRUCTION FORMATS & OPERAND AREAS

    unp=100(9)20(5)           - sample instruction
                                unpack bytes 21-25 of the input record
                                into bytes 101-109 of the output record

opcode=op1start(op1lth)op2start(op2lth) - instruction format

           receiving-operand <-- sending-operand
    unp=b100(9)a20(5)         - op1 defaults to outrec & op2 to inrec
                              - may identify explicitly by alpha prefixes
        b100                  - 'b' identifies the output record area
        w___                  - 'w' identifies the 'work' area (misc use)
               a20            - 'a' identifies the input record area
               w__            - 'w' op2 could specify the work area
               h__            - 'b' op2 could specify the outrec area
               :________      - ':' identifies a constant
               :0x______      - ':0x' identifies a hexadecimal constant
        _100                  - start bytes are displacements within the area
                                (the number of bytes preceding the 1st byte)
                                (or the offset within the area)
            (9)               - the length of the data field
                              - startbyte + length cannot exceed area size
                                - outrec & inrec depend on rcs= parameter
                                - work area max is 32768 unless increased
                                  by rop=r99999 parameter
               :_________       - constant max length is 64 bytes
    mvc=200(100)0             - some instructions such as move character
                                do not require op2 length since it is always
                                the same as the output length
    mvn=100(9za)20(5pe)       - instructions such as move numeric allow
                                field type codes within the length specs
                              - this example would convert a packed ebcdic
                                field to a zoned ascii field
    mvc=b0(rcsz)a0        <---- this instruction is automatically performed
                                by uvcp before your 1st instruction is
                                executed.
    clr=b0(rcsz):0x20         - sometimes you may want to clear the outrec
                                area if you are radically changing the layout
                                & want to ensure blanks between your fields.

Goto:   Begin this document End this document UVSI Home-Page

M0. uvcp: INSTRUCTION REFERENCE

      logical:  mvc, mvn, edt, clr
   arithmetic:  add, sub, mpy, div, pac, unp, seq
 compare/skip:  cmc, cmn, skp
 scan/replace:  rep, scn, sqz
   translates:  tra, tre, trl, tru, trp, trb
        other:  anc, orc, chx, hxc, tim
          I/O:  put

The following pages will describe each instruction in detail in the sequence listed above.

Note that 'uvsort' provides the same instruction set as this 'uvcp'. The instructions are documented in detail ONLY here in uvcp.doc.

Goto:   Begin this document End this document UVSI Home-Page

M1. uvcp: 'mvc' - move characters

'mvc' is the 1st instruction to be documented (vs prior items that are termed 'functions'). Instructions are validated at setup time and stored in memory for execution on each record cycle (at run time). uvcp allows for 200 instructions.

The move instruction documentation will include detailed description of instruction operands,lengths,etc which will also apply to all instructions following.

     mvc=tobyte(length)frombyte     - format & examples
     mvc=op1dspl(op1lth)op2dsp
     mvc=100(30)0                  move from 1-30 to 101-130
     mvc=b100(30)a0                same with area codes explicit
                                   (op1 defaults to 'b' output record)
                                   (op2 defaults to 'a' input record)
     mvc=w0(200)b100             - move bytes 101-200 of the output record
                                   area to bytes 1-200 of the work area
     mvc=130(07):updated           ':' indicates constant vs frombyte
     mvc=137(02):0x0d0a            ':0x' indicates hex constant

The following rules apply to most instructions (not just 'mvc')

  1. operand 1 is the receiving operand & operand 2 is the sending operand.

  2. operand addresses are zero relative displacements within the area.

  3. instructions such as mvc require no length on operand 2, since it is always the same as operand 1.

  4. op1 defaults to the output record area if no nonnumeric prefix is coded.

  5. op1 prefixes may be: 'b' - the output record area (default), max = output record size 'w' - the work area, max = 32768 (unless increased by rop=r99999 option)

  6. op2 defaults to the input record area if no non-numeric prefix is coded.

  7. op2 prefixes may be: 'a' - the input record area (default), max = input record size 'b' - the output record area, max = output record size 'w' - the work area, max = 32768 (unless increased by rop=r99999 option) 'x' - system information area (date,time,etc) ':' - indicates constant data following ':0x' - indicates a hexadecimal constant following

Goto:   Begin this document End this document UVSI Home-Page

M1a. 'mvc' - move characters

option 'u1' - allow coding ' 's as '_'s

Option 'u1' allows you to code blanks as underscores, which is convenient because blanks are not allowed in the uvcp command string. The alternative would be to code blanks in hexadecimal (' ' = 0x20). Here are some examples:

option 'u2' - allow coding ' 's as '_'s

Option 'u2' allows you to code commas as at-signs, which is convenient because commas are used to separate commands in the command string and therefore can not be coded as constants.

examples - coding ' ' as '_' & ',' ar '@'


 mvcu1=0(1):_,mvcu2=1(1):@,...
 =============================
 - move a ' ' blank to the 1st byte & a ',' comma to the 2nd byte

 mvcu3=0(2):_@,...
 =================
 - move a ' ' blank to the 1st byte & a ',' comma to the 2nd byte
 - coding a 2 byte constant (vs separate 1 byte constants)
 - note option u3 = (u1 + u2)

Goto:   Begin this document End this document UVSI Home-Page

M2. uvcp: 'mvn' - move numeric data (right adjusted & left zero filled)

'mvn' will pack, unpack, convert to binary or decimal and convert signs as required depending on data type codes (in the length specification).

      mvn=todsp(tolth/typ)fromdsp(fromlth/typ)
      mvn=todsp(tolth/typ):constant
      mvn=100(9)20(7)              - move 21-27 to 101-109
                                     right adjusted, left zero filled
                                     (defaults to zoned ASCII numeric)
      mvn=30(-9za)30(9za-)         - moves 31-39 input to 31-39 output
                                     shifting sign trailing to leading
                                     '-' present if neg, absent if pos
                                     '+' means + for pos, - for neg
      mvn=200(4bs)50(9za)          - converts 51-59 from zoned ASCII
                                     to 201-204 binary long word
                                     's' means switch big-end/little-end
      mvn=160(5pa):235-            - store constant neg 235 in 161-165
                                     packed ASCII (sign in zone)
      mvn=80(9za)20(5pe)           - unpacks 21-25 into 81-89
                                     21-25 is packed EBCDIC
                                     81-89 is zoned ASCII (zoned sign)
      mvn=80(9za)80(9zx)           - convert 81-89 zoned type 'x' (see below)
                                     to zoned ASCII numeric
                                   - type 'x' corrects zoned numeric signs
                                     translated from EBCDIC to ASCII
                                     before testing possible zoned signs
                                     (see explanation next page)

Goto:   Begin this document End this document UVSI Home-Page

M2a. summary of numeric data type & sign codes coded in op1/op2 lengths

 1st alpha letter defines the data type
   z - zoned numeric (default)
   p - packed decimal
   b - binary - length should be either 2 or 4 bytes
              - usually on a halfword or fullword boundary
 2nd alpha letter defines the codeset (or indicates binary switch)
   a - ASCII (default)
   e - EBCDIC (probably from an mainframe)
   s - causes a big-end/little-end switch
       when converting data between INTEL chip machines & others
   x - ASCII zoned numeric translated from EBCDIC that may have had
       signs in the units zone (see explanation below)
 separate sign byte codes
   '-' sign indicates a separate sign byte (vs zone of units digit)
       ('-' is coded if negative, else positive is assumed)
   '+' sign indicates a separate sign byte, but code '+' for positive
       as well as '-' for negative
   '-' or '+' in the 1st byte of the length indicates a leading sign byte
       else the sign byte will be trailing

data type code 'x'

Data type code 'x' is 'ASCII zoned numeric translated from EBCDIC'.

data type code 'x' is provided to correct the units position of numeric zoned (unpacked) fields that have been translated from EBCDIC to ASCII without allowing for possible zoned signs.

EBCDIC zone + sign is x'C0', + 0123456789 becomes {ABCDEFGHI EBCDIC zone - sign is x'D0', - 0123456789 becomes }JKLMNOPQR

Code 'x' converts positive units digits to 0123456789 (numeric zones) but negative units digit depends on whether the uvcp/uvcopy/uvsort utility has been generated with DISAM for Microfocus COBOL or not since conventions are different for Microfocus COBOL & MBP-COBOL(obsolete)

Microfocus zone
  • sign is x'70', - 0123456789 becomes pqrstuvwxy
MBP COBOL zoned
  • sign is x'40', - 0123456789 becomes @ABCDEFGHI

Goto:   Begin this document End this document UVSI Home-Page

M3. 'edt' - edit op2 into op1 using edit mask in op3

Feb 2019 - edit instruction added

edt=to(lth)from(lth):zz;zzz.99 - edit

      edt=71(13):z;zzz;zzz.99-
      edt=71(13):z;zzz;zzz.99+
      edt=71(13):-z;zzz;zzz.99
      edt=71(13):+z;zzz;zzz.99

In Feb 2019, we added the edit instruction, here is our test script:

 # testsortedt1 - test uvsort edit instrn Feb27/2019
 #  - script to test/demo 'edt' instruction, stored in $UV/sf/adm/testsortedt1
 uvsort "fili1=dat1/sales2,rcs=100,typ=LSTt,filo1=tmp1/sales2,edt=71(13)53(9):z;zzz;zzz.99-"
 #==========================================================================================
 exit 0

You can test as follows:


 #1. Login as uvadm or yourself if your profile setup for Vancouver Utilities
 #2. mkdir tmp1       <-- make output directory (if not existing)
 #3. cat dat1/sales2  <-- display input file
 #4. testsortedt1     <-- execute script
     ============
 #5. cat tmp1/sales2  <-- display output file

Goto:   Begin this document End this document UVSI Home-Page

M4. 'clr' - clear op1 area to the 1st byte of op2 (usually a 1 byte constant)

      clr=tobyte(lth):constant
      clr=tobyte(lth)frombyte
      clr=b0(200):0x20         - blank out 1st 200 bytes of the output
                                 record area
      clr=b0(200)              - may omit op2 to default to blank (0x20)
      clr=w0(500):0xff         - fill 1st 500 bytes of work area
                                 with hex FF
      clr=b100(100)a0          - fill 101-200 of outrec with the
                                 value from byte 1 of input record

Goto:   Begin this document End this document UVSI Home-Page

M5. uvcp: 'tra' - translate to ASCII (from EBCDIC)

      tra=op1dsp(lth)
      tra=0(40)                - translate 1-40 to ASCII from EBCDIC
                               - defaults to the output record area
                                 since no alpha prefix on op1
      tra=b100(28)             - translate 100-128
                                 (possibly bypassing packed fields 40-99 ?)

Here is the complete uvcp command to translate a 152 byte EBCDIC record to ASCII (OK if no packed fields & no zoned signed fields).


 uvcp "fili1=infile,typ=LST,rcs=152,filo1=outfile,tra=0(152)"
 =================================================^^^^^^^^^^=

option 's' to correct zoned signs

Option 's' of 'tras' will translate zoned signs from EBCDIC to ASCII for use with Micro Focus COBOL. You could use this to fix the signs in files that have been FTP'd in text mode from the mainframe to unix/linux/windows.

FTP from the mainframe in text mode will automatically translate EBCDIC to ASCII. Most people know that you can't do this if there are any packed or binary fields, but some don't realize that any unpacked signs need to be corrected from the mainframe conventions to Micro Focus COBOL conventions.

      In EBCDIC +'0123456789' is '{ABCDEFGHI' (x'C0' thru x'C9')
      In EBCDIC -'0123456789' is '}JKLMNOPQR' (x'D0' thru x'D9')
      In ASCII  +'0123456789' is '0123456789' (x'30' thru x'39') (no diff)
      In ASCII  -'0123456789' is 'pqrstuvwxy' (x'70' thru x'79')

For example if the file had a zoned signed field in bytes 41-49 (0 rel), then we could use the following uvcp to correct the units zone in byte 49.


 uvcp "fili1=infile,typ=LST,rcs=100,filo1=outfile,tras=49(1)"
 ====================================================^=======

We could also have used 'mvn=41(9za)41(9ze)', but the advantage of 'tras' is that we can correct multiple contiguous signed fields with 1 instruction. For example if we had 12 * 9 byte signed fields in bytes 41-112, we could use:


 uvcp "fili1=infile,typ=LST,rcs=154,filo1=outfile,tras=41(112)"
 ====================================================^=========

The recsize is specified as 154 because FTP text would have added 2 bytes for CR/LF to the original 152 (40 + 112). If we wanted to drop the CR/LF to get back to the original record length of the mainframe file, (so we don't have to change the COBOL programs) we should use the following:


 uvcp "fili1=infile,typ=LST,rcs=154,filo1=outfile,tras=41(112),rcs=152,typ=RSF"
 ====================================================^=========================

Goto:   Begin this document End this document UVSI Home-Page

M5e. 'tre' - translate to EBCDIC (from ASCII)

      tre=op1dsp(lth)
      tre=b0(500)               - translate the 500 byte output record area
                                  to EBCDIC (from ASCII)
      tre=w50(30)               - translate bytes 51-80 of the work area

option 's' to correct zoned signs

Option 's' of 'tres' will translate zoned signs from ASCII to EBCDIC. You could use this to fix the signs in files that have been FTP'd in text mode from unix/linux/windows back to the mainframe.

'trp' - translate any unprintable characters to '.' periods

      trp=op1dsp(lth)
      trp=b0(500)               - translate the 500 byte output record area
                                  any unprintable characters to '.' periods

'trb' - translate any unprintable characters to ' ' blanks

      trb=op1dsp(lth)
      trb=b0(500)               - translate the 500 byte output record area
                                  any unprintable characters to ' ' blanks

Goto:   Begin this document End this document UVSI Home-Page

M6. 'trl' - translate to lower case (any existing upper case)

      trl=op1dsp(lth)
      trl=b0(100)                 - translate bytes 1-100 of the output
                                    record area to lower case
      trl=w200(100)               - translate bytes 201-300 of the work
                                    area to lower case

'tru' - translate to UPPER case (any existing lower case)

      tru=op1dsp(lth)
      tru=b100(100)               - translate 101-200 of the output record
                                    area to UPPER case
      tru=w200(100)               - translate 201-300 of the work area
                                    area to UPPER case

Goto:   Begin this document End this document UVSI Home-Page

M7. 'add' - add numeric, packed, or binary

Add will pack, unpack, convert to binary or decimal, and convert signs as required depending on data type codes (in the length specification).

      add=todsp(tolth/typ)fromdsp(fromlth/typ)
      add=todsp(tolth/typ):constant
      add=100(9)20(7)              - add 21-27 to 101-109
                                     & store result in 101-109 (op1)
                                     (defaults to zoned ASCII numeric)
                                     (op1 dflts recout, op2 to recinput)
      add=b80(9za)a20(5pe)         - adds 21-25(outrec) to 81-89(inrec)
                                     21-25 is packed EBCDIC
                                     81-89 is zoned ASCII (zoned sign)
      add=w30(-9za):100            - adds constant 100 to bytes 31-39
                                     in the work area
                                     leading '-' sign if negative
                                     '-' present if neg, absent if pos
                                     '+' means + for pos, - for neg
      add=200(4bs)b50(9za)         - adds 51-59 to 201-204
                                     51-59 is zoned ASCII in outrec
                                     201-204 is a binary long word
                                     in outrec area (by default)
                                     's' means switch big-end/little-end
                                     on both retrieval & store result
      add=160(5pa):235-            - adds  constant neg 235 to 161-165
                                     161-165 packed ASCII (sign in zone)
    ------------------ instruction options --------------------

Please note the 'repeat' options which are described further below & that apply to add (& mvn,edt,sub,mpy,div,pac,unp,seq,anc,orc,rep,sqz)

x99
  • repeat option
  • the instruction will be repeated the specified number of times
    incrementing the operands by the op1 & op2 lengths specified
  • to perform operations on multiple same size contiguous fields
    with 1 instruction (add multiple corresponding, crossfoot, etc)
j/k/l
  • these options may be used to inhibit or modify the default
    incrementing of operand lengths before the repetition
    (to achieve cross-footing or field skipping etc)

Goto:   Begin this document End this document UVSI Home-Page

M8. 'sub' - subtract numeric, packed,or binary

      sub=todsp(tolth/typ)fromdsp(fromlth/typ)
      sub=todsp(tolth/typ):constant
      sub=100(9)20(7)              - subtract 21-27 from 101-109
                                     (defaults to zoned ASCII numeric)
                                     op1 defaults to outrec area (h)
                                     op2 defaults to  inrec area (r)
      sub=w150(-9za):125           - subs constant 125 from work area bytes
                                     151-159 zoned ASCII
                                     with leading '-' sign if negative
Note
  • add,sub,mpy,& div set the internal condition code & may be followed
    by the 'skp' instruction to test & skip instructions
  • the condition code is set equal for zero, > for pos, < for neg
      sub=b60(5pa)b65(5pa)         - subtract bytes 66-70 from 61-65
      skp<=2                       - skip next 2 instrns if negative

Goto:   Begin this document End this document UVSI Home-Page

M9. 'mpy' - multiply

Multiply will pack, unpack, convert to binary or decimal, and convert signs as required depending on data type codes (in the length specification).

      mpy=todsp(tolth/typ)fromdsp(fromlth/typ)
      mpy=todsp(tolth/typ):constant
      mpy=100(9)20(7)              - multiply 101-109 by 21-27
                                     & store result in 101-109 (op1)
                                     (defaults to zoned ASCII numeric)
                                     op1 dflts to recout, op2 to recinput
      mpy=b80(9za)a20(5pe)         - multiply 81-89 by 21-25
                                     21-25 is packed EBCDIC
                                     81-89 is zoned ASCII (zoned sign)
      mpy=w30(-9za):115            - multiply work area bytes 31-39
                                     by constant 115
                                     '-' present if neg, absent if pos
                                     '+' means + for pos, - for neg
      mpy=200(4bs)50(9za)          - multiply 51-59 by 201-204
                                     51-59 is zoned ASCII
                                     201-204 is a binary long word
                                     's' means switch big-end/little-end
                                     on both retrieval & store result
      mpy=160(5pa):235-            - mpys 161-165 by constant -235
                                     161-165 packed ASCII (sign in zone)

'div' - divide

      div=todsp(tolth/typ)fromdsp(fromlth/typ)
      div=todsp(tolth/typ):constant
      div=100(9)20(7)              - divide 101-109 by 21-27
                                     & store result in 101-109 (op1)
                                     (defaults to zoned ASCII numeric)
                                     op1 dflts to recout, op2 to recinput
      div=w30(-9za):115            - divide work area bytes 31-39
                                     by constant 115
Note
  • division by zero results in zero

Goto:   Begin this document End this document UVSI Home-Page

M10. 'pac' - pack the op2 data into the op1 area

      pac=op1dsp(op1lth)op2dsp(op2lth)
      pac=100(5)50(9)                - pack 51-59 into 101-105
operation
  • switches nibbles in right hand byte
  • packs digits from right to left until left hand of op1 reached
  • any extra length in op2 data is ignored
  • if left side op2 reached 1st, extra bytes op1 are 0x00 filled
Note
  • in most cases you should use the 'mvn' instruction (see above)
    since mvn ensures the proper sign which may be different in the
    packed & unpacked formats
  • mvn also allows conversion to separate sign bytes leading or trailing

'unp' - unpack the op2 data into the op1 area.

      unp=op1dsp(op1lth)op2dsp(op2lth)
      unp=100(9)50(5)                - unpack 51-55 into 101-109
operation
  • switches nibbles in right hand byte
  • unpacks digits from right to left until left side op1 reached
  • any extra length in op2 data is ignored
  • if left side op2 reached 1st, extra bytes op1 are 0x30 filled
Note
  • in most cases you should use the 'mvn' instruction (see above)
    since mvn ensures the proper sign which may be different in the
    packed & unpacked formats
  • mvn allows conversion to separate sign bytes leading or trailing

Goto:   Begin this document End this document UVSI Home-Page

M11. 'seq' - sequence number

Most instructions are common to both uvcp & uvsort, but documented only here in uvcp.doc. This 'seq' documentation is duplicated in uvsort, since sequence# is most applicable to uvsort & would probably be executed at sort output time.

Note
  • option 'o' applies only to uvsort (not this uvcp)
      seq=displacement(length)    <-- instruction format
      seq=75(5)     - sequence# in 75-79 (cols 76-80)
                    - 5 bytes zoned numeric (unpacked)
      seq=75(5p)    - 5 bytes packed
      seq=76(5b)    - 4 bytes binary
      seqo=75(5)    - sequence# at Output time (vs input time)
                    - more applicable to sorts
      seqo1=75(5)   - seq# Output file #1
      seqo2=75(5)   - seq# Output file #2
      seqo3=75(5)   - seq# Output file #3
      seqo1c1=75(5)   - seq# Outfile#1 using counter#1 (default)
      seqo2c2=75(5)   - seq# Outfile#2 using counter#2 (default)
      seqo1c1=75(5)   - seq# Outfile#1 using counter#1 (default)
      seqo1c2=80(5)   - 2nd seq# in Outfile#1 using counter#2
      seqo1c2=85(5)   - can have multiple seq#s in 1 file
      seqo1i10=75(5)  - Increment by 10 (vs 1 default)
      seqo1b1000i10=75(5) - Begin at 1000 & Increment by 10
option 'o#'
  • specifies Output file# 1-40
  • common to most instructions (not just seq#)
option 'c#'
  • Counter#
  • defaults to c0 for input files
  • defaults same as 'o#' for output files
  • specify explicitly for multiple ctrs in 1 file
option 'i#'
  • Increment, defaults to 1
option 'b#'
  • Begin value (before 1st increment)
  • dfaults to zero

Goto:   Begin this document End this document UVSI Home-Page

M12. 'anc' - AND Character

Perform a logical 'and' on the op1 data with the op2 mask op2 is usually a constant, but could address a data area. Use 'anc' to clear bits in the data that are 0's in the mask.

      anc=op1dsp(lth):0xconstant (op2 usually constant)
      anc=op1dsp(lth)op2dsp
      anc=20(10):0x0f                - clear the zones of bytes 21-30
      anc=20(10)10                   - op2 could be an adrs, but unusual
      ancm=20(3):0x0f0c0f            - erase 0 bits in the mask of bytes
                                       21-23, option 'm' required to allow
                                       op2 to be more than 1 byte

'orc' - or character

Perform a logical 'or' on the op1 data with the op2 mask op2 is usually a constant, but could address a data area. Use 'or' to add bits in the data that are 1's in the mask.

      orc=op1dsp(lth):0xconstant (op2 usually constant)
      orc=20(10):0x30               - set zones of byte 21-30 to 0x30's
      orc=20(10)10                  - op2 could be an adrs, but unusual
      orcm=20(3):0x303040           - set zones of 21-23 to 0x303040
                                      option 'm' required to allow
                                      op2 to be more than 1 byte

Goto:   Begin this document End this document UVSI Home-Page

M13. 'chx' - convert character representation of hex to true hex data

      chx=0(16)0          - converts the hex representation characters
                            in the 1st 32 bytes of input record
                            to true data in the 1st 16 bytes of outrec
 sample op2  input = 3132332041424320616263207E5E3E2E
        op1 output = 123 ABC abc ~^>.
Notes
  • output will be 1/2 as long as input (op2 lth need not be coded)
  • This instruction might be used when you needed to create a file
    containing binary data that is not easily entered by a text editor
  • For example if you needed some kind of binary control file of 16
    byte records, you could enter the hex representation into a
    temporary file 32 characters per line & then run uvcp to convert
    to the desired binary file.
  • The complete uvcp command line might be as follows:
         uvcp "fili1=temp1,typ=LST,filo1=binaryctl,typ=RSF,rcs=32,chx=0(16)0"

'hxc' - convert data to hex representation

      hxc=0(32)0          - converts the data in the 1st 16 bytes of the
                            input record to hex representation in the
                            1st 32 bytes of the output record

sample op2 input = 123 ABC abc ~^>. sample op1 output = 3132332041424320616263207E5E3E2E

Note
  • output will be twice as long as input (op2 lth need not be coded)

uvcp "fili1=binary,typ=RSF,rcs=16,filo1=hexrep,typ=RST,rcs=33,hxc=0(32)0"

Goto:   Begin this document End this document UVSI Home-Page

M14. 'rep' - replace patterns with alternates

'rep' will scan the op1 area replacing all occurrences of the op2 pattern with the op3 replacement.

The op1 area will be truncated or blank right filled depending on whether The replacement is longer than or shorter than the original pattern.

      rep=op1dsp(op1lth):searchpattern:replacementpattern
      rep=op1dsp(op1lth)op2dsp(op2lth)op3dsp(op3lth)
      rep=op1dsp(op1lth):constant:constant
      rep=0(128):AS400:UNIX          - search outrec area replacing all
                                       occurrences of 'AS400' with 'UNIX'
      rep=b100(200)a10(5)w20(8)      - scan outrec 101-200 replacing any
                                       patterns matching 11-15 of inrec
                                       with the data at 21-28 of work area
      rep=0(128):0x0a:0x0d0a         - scan the outrec area replacing any
                                       line feeds with CR & LF
                                       (demos hex constants id by ':0x')
      skp?=nn                        - may use the skip instrn to test
                                       whether any replacements were made
      skp:=1                         - skip 1 instrn if any replcmnts made
      skp!=2                         - skip 2 instrns if no replacements made

Goto:   Begin this document End this document UVSI Home-Page

M14a. 'rep' - replace patterns with alternates

option 'u1' - allow coding ' 's as '_'s

Option 'u1' allows you to code blanks as underscores, which is convenient because blanks are not allowed in the uvcp command string. The alternative would be to code blanks in hexadecimal (' ' = 0x20). Here are some examples:

option 'u2' - allow coding ' 's as '_'s

Option 'u2' allows you to code commas as at-signs, which is convenient because commas are used to separate commands in the command string and therefore can not be coded as constants.

examples - coding ' ' as '_' & ',' ar '@'


 repu1=0(1):_:X,repu2=1(1):@:Y,...
 =================================
 - replace any blank ' ' in 1st byte with 'X'
 - and replace any comma ',' in 2nd byte with 'Y'

 repu3=0(2):_@:XY,...
 ====================
 - replace ' ,' (blank+comma) in 1st 2 bytes with 'XY'
 - note option u3 = (u1 + u2)
 - not same as above since both conditions must be met

 repu3=0(1):_:@,repu3=1(1):@:_,...
 =================================
 - replace any blank ' ' in 1st byte with comma ','
 - and replace any comma ',' in 2nd byte with blank ' '
 - note option u3 = (u1 + u2)

Goto:   Begin this document End this document UVSI Home-Page

M15. 'scn' - scan for a pattern & set cc = if found

'scn' searches the op1 area for any occurrence of the data pattern contained in op2 & sets the internal condition code equal if found which may then be tested by a following 'skp' instruction.

      scn=op1dsp(op1lth):constant
      scn=op1dsp(op1lth):op2dsp(op2lth)
      scn=a0(128):comp-3            - scan input record area for pattern
                                      'comp-3' & set cc equal if found
      skp!=2                        - skip next 2 instrns if not found

Goto:   Begin this document End this document UVSI Home-Page

M16. 'sqz' - squeeze out blanks from the op1 area

      sqz=start(length):constant  - format
      sqz=20(80):0x20             - squeeze all blanks between words
                                    in bytes 21-100 of outrec area
                                    blank filling on the right
                                    (0x20 is an ASCII blank in hex)
      sqz=20(80)                  - may omit op2 to default to blank
      sqzm=20(80):0x20            - squeeze multiple blanks to 1 blank
                                    between words in op1 (m option)
                                  - see other options described below
      sqz=w0(10)w10               - op2 could be an adrs but unusual
                                  - this would squeeze out whatever char
                                    was in byte# 11 of w/s from the 1st
                                    10 bytes of w/s

sqz options

'sqz' with no options will squeeze all blanks from between the words in the op1 area & blank fill on the right as required

option f
  • ensures that any blanks before the first word in the op1 area
    will be squeezed out so the data will be left adjusted
        m  - squeezes multiple blanks to 1 blank between the words
        c1 - option 'c1' same as 'm' (compatible with uvcopy sqz option)
        b  - will convert any characters less than a blank (0x00-0x20)
             into a blank so they will then be squeezed
        h  - will convert any characters greater than '~' (0x7e-0xff)
             into a blank so they will then be squeezed

Goto:   Begin this document End this document UVSI Home-Page

M17. 'cmc' - compare op1 to op2 & set internal condition code = > <

M17a. 'skp?' - test internal condition code to '?' code & skip if match

      cmc=op1dsp(op1lth)op2dsp          - compare op1 to op2 & set cc
      cmc=op1dsp(op1lth):constant       - compare op1 to a constant
      skp?=nn                           - skip if ? matches internal cc
      cmc=0(2):a5               - compare cols 1-2 to constant 'a5'
      skp:=1                    - if equal skip the next instrn
                                  (':' colon used for equal condition)
      cmc=b10(5)w10             - compare bytes 11-15 of outrec area
                                  to bytes 11-15 of the work area
      skp>=2                    - if > skip the next 2 instrns
      ---                         else fall thru to this instrn
      skp.=1                      skip (.)unconditionally 1 instrn
      ---
Notes
  • you may use the following codes in the 4th byte of 'skp' to
    test the internal condition code set by 'cmc'
  • ':' equal (cannot use '=' since that is used to sep keywords)
  • '!' not equal (exclamation mark)
  • '>' greater than
  • '<' less than
  • '.' unconditional skip (period)
Note
  • the 'skp' instruction may also be used following several other
    instructions that also set the condition code
  • add,sub,mpy,div,mvn,edt set cc equal if zero, > if pos, < if neg
  • rep sets cc equal if any replacements were made
  • scn (scan) sets cc equal if the search pattern was found in op1
Note
  • use 'skp!=99' when you want to modify some records & copy others
  • for example, given following file, flag '*' in col 8 of HAMmers

BBQ010 BAR-B-Q HAM010 CLAW HAMMER HAM035 JACK HAMMER SAW011 HAND SAW SAW012 RIP SAW

uvcp "fili1=dat1/products,rcs=128,typ=LSTt,filo1=tmp/products\ ,cmc=0(3):HAM,skp!=99,mvc=7(1):*" =============================================================

Note
  • you could have several more cmc's & skp!'s for multiple conditions
  • the 'skp!=99' could have been 'skp!=1' or any number greater than
    the instrucitons you need to identify the desired condition
  • but in an odd quirk, it must be less than the record size
    which does not matter for typ=LSTt
  • this bug will be fixed soon

Goto:   Begin this document End this document UVSI Home-Page

M18. 'cmn' - compare numeric & set internal condition code = > <

M18a. 'skp?' - test internal condition code to '?' code & skip if match

      cmn=op1dsp(op1lth)op2dsp(op2lth)  - compare op1 to op2 & set cc
      cmn=op1dsp(op1lth):constant       - compare op1 to a constant
      skp?=nn                           - skip if ? matches internal cc
      cmn=0(8):125              - compare cols 1-8 to constant '125'
                                - numeric compare rt adjust, left 0 fill
      cmn=0(8z)20(5p)           - compare cols 1-8 (zoned num default)
                                  to cols 21-25 packed
      cmn=20(5p)40(4b)          - compare packed 21-25 to binary long 41-44
Note
  • see the prior 'cmc' (compare character) documentation for all the
    possible condition codes etc

Goto:   Begin this document End this document UVSI Home-Page

M19. 'put' - write the record currently existing in the output record area

     put=b0(256)                  - write the current outrec area data
Note
  • the output record area may be defined in op1 as shown above
    but this version of UNIX UVCOPY (uvcp) will always write
    the output record area defined by the filo1 'rcs' parameter.
Note
  • if no 'put's are coded, uvcp will write out each input record
    automatically (unless a 'sel' or 'del' condition applies)
  • if any 'put' is coded, then you must use a put to write out
    all records desired
example
  • drop records that have all zero amounts in the 12 x 5 byte
    packed fields in record bytes 101-160
      mvn=w0(6p):0                   - clear some w/s for accumulation
      addx12j0=w0(6p)a100(5)         - accumulate the 12 fields & set cc
      skp:=1                         - test cc & skip next instrn if zero
      put=b0(200)                    - write the record if nonzero
      ---

Goto:   Begin this document End this document UVSI Home-Page

N1. uvcp: INSTRUCTION OPTIONS THAT APPLY TO MULTIPLE INSTRUCTIONS

The documentation above has described options that apply to specific instructions, but there are some options that apply to entire classes of instructions.

The options described below apply to the following instructions: mvn,edt,add,sub,mpy,div,pac,unp,seq,anc,orc,rep,sqz.

x99
  • repeat option
  • the instruction will be repeated the specified number of times
    incrementing the operands by the op1 & op2 lengths specified
  • to perform operations on multiple same size contiguous fields
    with 1 instruction (add multiple corresponding, crossfoot, etc)
j0/j9
  • optional with the 'repeat' option
  • changes the op1 increment from the default (op1 length) to
    whatever is specified on the 'j' option
  • would specify j0 to crossfoot multi op2 fields into 1 op1 field
  • would specify j 2 * field length to process every other field (etc)
k0/k9
  • optional with the 'repeat' option
  • changes the op2 increment from the default (op2 length) to
    whatever is specified on the 'k' option
  • would specify k0 to move or add 1 op2 value into multiple op1 fields
  • would specify k 2 * field length to process every other field (etc)
l0/l9
  • optional with the 'repeat' option
  • changes the op3 increment from the default (op3 length) to
    whatever is specified on the 'l' option

examples

 addx12=160(5p)100(5p)     - adds the 12 5 byte packed fields in 101-160
                             to the 12 5 byte packed fields in 161-120
 addx12j0=w0(4b)160(5p)    - adds the 36 5 byte packed fields in 101-280
                             to the 4 byte binary field in work area w1-w4
                           - the 'j0' option prevents op1 from being
                             incremented, to get the cross-foot effect
 addx12j10=b40(5p)100(5p)  - adds to the 12 x 5 byte packed fields 41-100
                             from every 2 nd 5 byte field in 101-220
                           - option 'j10' causes op2 to be incremented
                             by 10 vs the op2 length of 5

Goto:   Begin this document End this document UVSI Home-Page

O1. uvcp: WORKING STORAGES

There are 3 working storages which the user may address by prefixing the displacements of the item desired with the letters 'w','x',or 'y'

w
  • general purpose work area, initialized to blanks
  • default size 32768 bytes, may increase via rop=r99999
x
  • system information of character type (layout below)
y
  • system information of binary long type (layout below)

LAYOUT OF WORKING STORAGE 'x'

          --------- input file information -----------
x0(80)
  • input file name
x80(80)
  • input filename #2 (uvsort)
x560(80)
  • input filename #8 (uvsort)
x640(16)
  • input file type
  • 3 alpha + options
x656(16)
  • last modification date
x672(28)
  • file typ options
x700(104)
  • input file optns longs
x804(4)
  • input record counter
x808(4)
  • file size (ftell bytes)
x812(4)
  • record size
x816(4)
  • current rcsz (Line Var)
          --------- output file information -----------
x1000(80)
  • output file name
x1080(16)
  • output file type
  • 3 alpha + options
x1096(16)
  • last modification date
x1112(28)
  • file typ options
x1140(104)
  • input file optns longs
x1244(4)
  • output record counter
x1248(4)
  • file size (ftell bytes)
x1252(4)
  • record size

Goto:   Begin this document End this document UVSI Home-Page

O2. LAYOUT OF WORKING STORAGE 'y'

          ---- system dates & times (begin) ----
y00(16)
  • program begin date & time
    ccyymmddhhmmss__
y20(19)
  • program begin date&time edited
    ccyy/mm/dd_hh:mm:ss
          ---- system dates/times (tim instrn/prgm end) ----
y40(16)
  • current date & time
    ccyymmddhhmmss__
y60(19)
  • date & time edited
    ccyy/mm/dd_hh:mm:ss
y80(8)
  • program version (compile date yyyymmdd)
y90(19)
  • program license# (yyyymmdd_99V_930630)
y100(30)
  • site name (company name)
y140(30)
  • registered person name
y170(30)
  • parameter filename
y200(100)
  • msgw reply area
y300(10)
  • program name executable (uvcp)
y310(10)
  • program name source (uvcp.c)
y320(10)
  • ISAM compile type
  • D-ISAM, SCOISAM(obsolete), MBPISAM(obsolete)
y330(8)
  • machine type
  • LNX,AIX,HP,SUN,DOS
y338(1)
  • forward/backward slash unix/dos
y340(8)
  • today yyyymmdd
y350(20)
  • date/time stamp
  • :ddmmyy:HHMMSS:
y370(8)
  • license expiry
y380(20)
  • $JOBID exported from parent script
y400(30)
  • job stamp
  • ddmmyy:HHMMSS:$JOBID2 for console msg prefix
y430(70)
  • parameter file pathname (from command line)
  • PFPATH1+root,PFPATH2+root
y504(8b)
  • program begin time (unix time secs since 1970)
y512(8b)
  • microseconds
y536(4b)
  • milliseconds
y520(8b)
  • program end time (unix time secs since 1970)
y528(8b)
  • microseconds
y540(4b)
  • milliseconds

Goto:   Begin this document End this document UVSI Home-Page

P1. uvcp RESTRICTIONS

'uvcp' is a mini version of the 'uvcopy' utility & is not intended to include all the facilities of that powerful utility.

uvcp was primarily intended to provide at least the functionality of the mainframe DATA utility to assist converting JCL to UNIX scripts.

The uvcp instruction format is restrictive compared to uvcopy because the intention was to allow all instructions to be specified on the command line & not be restricted to a parameter file.

The full featured version (uvcopy) allows a more user friendly format but restricts instructions (vs functions) to the parameter file since their syntax is incompatible with UNIX shell rules.

 ------------------ example of uvcp instruction format ----------------
       unp=100(9)20(5)
       mvc=200(60):cannot_have_blanks_equalsigns_commas_in_uvcp_constants
       cmc=109(1):0x20           (use hex constants for blanks,equals,etc)
 ---------------- example of uvcopy instruction formats ----------------
       unp   100(9),20(5)
       mvc   200(60),'no problem to have blanks, commas, = signs in uvcopy'
       cmc   109(1),x'3f'                   (uvcopy hex constants)
 ---------------------- restriction summary ---------------------------
 1 - no embedded blanks, commas, equal signs in uvcp parameters
     - can use hexadecimal constants for these in most cases
     - could use underscores to separate words in character constants

2 - uvcp allows for 100 record selects or deletes (sel/del)

3 - uvcp allows for 100 instructions (mvc,mvn,edt,add,sub,etc) but this could be easily changed & the program recompiled

4 - must enclose command line in double quotes if any parenthesis present as on "isk=0(8)" (unix shell restriction)

5 - uvcp has a limited instruction set compared to the OS3 UVCOPY or the UNIX full featured uvcopy

6 - uvcp cannot skip to a label or use index registers as per UVCOPY or uvcopy

Goto:   Begin this document End this document UVSI Home-Page

Q1. uvcp: example #1 - reformatting records

given
  • a customer master file on mag tape from an EBCDIC mainframe
  • 256 byte records containing the following relevant fields:
            01-08=cust#,09=status,10-34=company,35-59=adrs1,60-84=adrs2,
            85-109=adrs3,110-116=zip,117-122=telephone(packed),
            176-200=contact-name
required
  • convert the above file to an ASCII Indexed 2 key file
    of 110 byte name & address records as follows:
            001-008 = cust#  - to be key1
            009-009 = status - any 'd' status records to be deleted
            010-034 = contact name (from 176-200 of input record)
            035-059 = adrs1
            060-084 = adrs2 & adrs3 squeezed together
            085-093 = zip code - to be key2
            094-103 = telephone# unpacked
            104-104 = line feed terminator so backup versions
                      can be easily viewed with a line editor
solution
  • for anything more complex than simple file copies it is best to
    prepare a parameter file with the editor & then call the utility
    specifying only the param filename on the command line
 # cnvtcm - parameter file for the uvcp utility
 #        - to convert the mainframe EBCDIC customer master file to a
 #          UNIX ASCII 2 key Indexed contact name & address file
 fili1=/dev/rmt0,typ=RSF,rcs=256
 filo1=contacts,typ=ISF,rcs=104,isk1=0(8),isk2=84(9)
 del=8(1):0xc4              # delete records with EBCDIC 'D' in col 9
 mvc=w0(256)a0              # move entire record to the work area
 tra=w0(256)                # translate entire work area to ASCII
 mvc=0(9)w0                 # move cust# & status from w/a to outrec
 mvc=b9(25)w175             # move contact name from w/a to outrec
 mvc=b34(25)w35             # move adrs1 from w/a to outrec
 sqzm=w59(50):0x20          # squeeze adrs2 & adrs3 together in w/a
 mvc=b59(25)w59             # move combined adrs2+adrs3 to adrs2 out
 mvc=b84(7)w109             # move zip code from w/a to outrec
 unp=b93(10)a116(6)         # unpack the telephone#
 orc=b102(1):0x30           # ensure zone of units digit is ASCII numeric
 mvc=b103:0x0a              # insert line feed in last byte of record
 put=b0(104)                # write out the record

After preparing the parameter file with the editor, you may execute it with the command line shown below:


 uvcp cnvtcm
 ===========

Goto:   Begin this document End this document UVSI Home-Page

Q2. notes re: cnvtcm record conversion on prior page

The entire record was 1st moved to the work area & translated to ASCII in the work area in order to give us complete flexibility in then reformatting the unpacked data from the work area & the packed data from the record area

This example includes some 'overkill' in order to demonstrate more of the available instructions

'put' is unnecessary since it is automatic unless we use 'skp's.

'clr' is unnecessary since the output record area is automatically cleared before each record is processed.

         mvc=b103(1):0x0a

The above instrn to put a line feed in the last byte is not required since typ=ISF,rcs=104 actually writes 105 byte records with LF in last byte. (Indexed files use extra byte as a status byte, changed to x'00' if deleted)

          unp=b93(10)a116(6)
          orc=b102(1):0x30

The above 2 instrns could be replaced with the 1 'mvn' shown below since mvn ensures zones conform to the data type specified (zoned ASCII)

          mvn=b93(10za)a116(6pe)

Goto:   Begin this document End this document UVSI Home-Page

R1. example #2 - reformatting records with multi same size numeric fields

given:
  • a G/L file from an mainframe EBCDIC system, layout as follows:
    001-020 = gl#
    021-060 = description
    061-120 = this year 12 months x 5 byte packed decimal
    121-180 = budget 12 months x 5 byte packed decimal
    181-240 = last year 12 months x 5 byte packed decimal
required:
  • (a) convert to UNIX ASCII leaving fields packed as is
  • (b) convert the packed fields to unpacked ASCII numeric
    with separate leading '-' sign byte if negative
  • (c) convert the packed fields to binary long integers

solutions:

 (a) uvcp "fili1=/def/rmt0,typ=RSF,rcs=240,filo1=glmaster,typ=ISF,isk1=0(20),
           tra=0(60)"
 (b) uvcp "fili1=/def/rmt0,typ=RSF,rcs=240,filo1=glmaster,typ=ISF,isk1=0(20),
           rcs=240,tra=0(60),mvnx36=60(-10za)60(5pe)"
 (c) uvcp "fili1=/def/rmt0,typ=RSF,rcs=240,filo1=glmaster,typ=ISF,isk1=0(20),
           rcs=240,tra=0(60),mvnx36=60(4b)60(5pe)"

Notes:

 (a) - translates only the gl# & the description (not packed fields)
     - you do not need to code 'mvc=0(240)0' before the 'tra' instrn
       since uvcp automatically moves the input record to the
       output record area before your 1st instrn is executed.
 (b) - 'mvn' with repeat option 'x36' converts all 36 packed fields from
       5 byte packed EBCDIC to 10 byte zoned ASCII with separate leading
       sign byte '-' if negative.
 (c) - 'mvn' with repeat option 'x36' converts all 36 packed fields from
       5 byte packed EBCDIC to 4 byte binary long integers.
     - note that the 's' option to switch from big end to little end
       is only required if you were converting binary mainframe fields.

Goto:   Begin this document End this document UVSI Home-Page

S1. using 'mkscript' to generate 'uvcp's to reorganize data file directories

NOTE

Make script from directory of data files & a command template with patterns to be replaced by current file values. The template may contain the following patterns, which will be replaced by the actual values for the current file:

FNAME
  • filename
TYP
  • file type (RSF or ISF)
RSIZE
  • record size
FSIZE
  • file size
KEYS
  • Indexed keys ex: (0,12)
RCOUNT
  • record count
DATETIME
  • date/time of last modify

The default template is for 'uvcp' (data file copy utility), which will copy all files from the input directory to a 2nd directory, reorganizing indexed files (sequencing active records & dropping deleted records). The default template & a corresponding generated script line might be:


 cmnd: uvcopy mkscript,fild1=dat1,filo1=sf/reorg1,filr1=ctlI/ctlfile1
       ==============================================================
       enter script template or null for default -->       <-- null for default

 dflt: uvcp "fili1=$DIR1/FNAME,typ=TYPl1,rcs=RSIZE,filo1=$DIR2/FNAME #keys=KEYS"
       =========================================================================

sample generated script

 # sf/reorg1 - created by mkscript 1998/12/17_17:14:03
 #           - uvcp template as follows:
 # uvcp "fili1=$DIR1/FNAME,typ=TYPl1,rcs=RSIZE,filo1=$DIR2/FNAME" #keys=KEYS
 uvcp "fili1=$DIR1/custmas1,typ=RSFl1,rcs=256,filo1=$DIR2/custmas1" #keys=
 uvcp "fili1=$DIR1/custmas2,typ=ISFl1,rcs=256,filo1=$DIR2/custmas2" #keys=(0,6,60,16)
 uvcp "fili1=$DIR1/warmas1,typ=RSFl1,rcs=64,filo1=$DIR2/warmas1" #keys=
 uvcp "fili1=$DIR1/warmas2,typ=ISFl1,rcs=64,filo1=$DIR2/warmas2" #keys=(0,12)

 2.  vi sf/reorg1     - inspect & modify created script
     ============

Goto:   Begin this document End this document UVSI Home-Page

T1. uvcp: converting TEXT files to FIXED record lengths & vice-versa

using uvcp to convert 'text' to 'fixed'

You may need to convert text files to fixed if the data is input for a COBOL program expecting fixed length records (with no LineFeed terminators). Text records are terminated by LineFeeds LF=x'0A' or CarriageReturn+LineFeed CR+LF=x'0D0A'. Text records may vary in length since trailing blanks are usually omitted & the LineFeed inserted after the last nonblank. Text files on unix systems need only the LineFeed, but files from windows or mainframes may have both CR & LF. For our example we will show both.

sample text file dat1/products

BBQ010 BAR-B-Q CHR015 LAWN CHAIR HAM010 CLAW HAMMER HAM020 BALL PEEN HAMMER


 uvhd dat1/products h2t   <-- uvhd with 'h2' option to see the LineFeeds
 ======================       (option 't' tells uvhd to end records on x'0A')
                  10        20        30        40        50        60
 r#    1 0123456789012345678901234567890123456789012345678901234567890123
       0 BBQ010   BAR-B-Q..  <-- LineFeed shows as '.' on character line
         445333222445242500  <-- note: zones & digits CR+LF=x'0D'+x'0A'
         221010000212D2D1DA
              - - - showing only 1st & 4th records to save space - - -
                  10        20        30        40        50        60
 r#    4 0123456789012345678901234567890123456789012345678901234567890123
      61 HAM020   BALL PEEN HAMMER..
         444333222444425444244444500
         81D02000021CC0055E081DD52DA

 uvcp "fili1=dat1/products,typ=LST,rcs=256,filo1=tmp/products,typ=RSF,rcs=80"
 ============================================================================
 - convert text file to fixed length 80 byte records with no CR/LF terminators

 uvhd tmp/products h2r80   <-- uvhd with option h2 for hex display
 =======================     - option r80 for fixed record-size
                  10        20        30        40        50        60
 r#    1 0123456789012345678901234567890123456789012345678901234567890123
       0 BBQ010   BAR-B-Q
         4453332224452425222222222222222222222222222222222222222222222222
         221010000212D2D1000000000000000000000000000000000000000000000000
      64
         2222222222222222
         0000000000000000
              - - - showing only 1st & 4th records to save space - - -
 r#    4 0123456789012345678901234567890123456789012345678901234567890123
     240 HAM020   BALL PEEN HAMMER
         4443332224444254442444445222222222222222222222222222222222222222
         81D02000021CC0055E081DD52000000000000000000000000000000000000000
      64
         2222222222222222
         0000000000000000

Goto:   Begin this document End this document UVSI Home-Page

T2. uvcp: converting TEXT files to FIXED record lengths & vice-versa

Notes re uvcp on page above


 uvcp "fili1=dat1/products,typ=LST,rcs=256,filo1=tmp/products,typ=RSF,rcs=80"
 ============================================================================
 - convert text file to fixed length 80 byte records with no CR/LF terminators
  1. Text record 'typ=LST' (Line Sequential Terminated)

  2. Fixed record 'typ=RSF' (Record Sequential Fixed)

  3. For text files (typ=LST), allow extra length (rcs=256 in this example) to ensure the terminating LineFeed will be found within the area assigned (to avoid record splitting).

  4. Or could use 'typ=RST' (Record Sequential Terminated) puts a LineFeed into the last byte of a fixed record size. A good idea if the last byte is not part of a data field, allows you to use editor on fixed files.

    common problem when files FTP'd from mainframe

If you FTP files from the mainframe in text mode, FTP translates to ASCII and appends CR+LF on the end of each record, as shown below using our 'customers' demo file. These records were fixed length 102 bytes on the mainframe with no LineFeeds. Here are the 1st 2 records shown by uvhd using option 'h2' for vertical hexadecimal display & option 't' to tell uvhd to look for LineFeeds x'0A' to terminate records.


 uvhd dat1/customers h2t   <-- uvhd display in hex using LF terminators
 =======================
                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 130140    EVERGREEN MOTORS LTD.    1815 BOWEN ROAD          NANA
             3333332222454545444244545524542222233332445442544422222222224444
             130140000056527255E0DF4F230C44E0000181502F75E02F140000000000E1E1
          64 IMO          BC V9S1H1    604-754-5531..
             4442222222222442535343222233323332333300
             9DF00000000002306931810000604D754D5531DA
                      10        20        30        40        50        60
 r#        2 0123456789012345678901234567890123456789012345678901234567890123
         104 132588    GEECOE GENERATOR SERVICE UNIT 170 - 2851 SIMPSON  RICH
             3333332222444444244445454525455444254452333222333325445544225444
             13258800007553F5075E5214F20352693505E9401700D02851039D03FE002938
          64 MOND         BC V6X2R2    604-278-4488..
             4444222222222442535353222233323332333300
             DFE40000000002306682220000604D278D4488DA

The problem is that the COBOL program on unix expects fixed 102 byte records so we need to drop off the extra 2 bytes (CR+LF) appended by FTP.

Please see the solution (via uvcp) on the next page --->

Goto:   Begin this document End this document UVSI Home-Page

T3. uvcp: converting TEXT files to FIXED record lengths & vice-versa

uvcp to drop CR/LF (convert text to fixed)


 uvcp "fili1=dat1/customers,typ=LST,rcs=256,filo1=tmp/customers,typ=RSF,rcs=102"
 ===============================================================================
 - drop off CR/LF (created by FTP) & force output recsize = 102

 uvhd tmp/customers h2r102   <-- uvhd with options h2=hex, r102=record-size
 ==========================
                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 130140    EVERGREEN MOTORS LTD.    1815 BOWEN ROAD          NANA
             3333332222454545444244545524542222233332445442544422222222224444
             130140000056527255E0DF4F230C44E0000181502F75E02F140000000000E1E1
          64 IMO          BC V9S1H1    604-754-5531
             44422222222224425353432222333233323333
             9DF00000000002306931810000604D754D5531
                      10        20        30        40        50        60
 r#        2 0123456789012345678901234567890123456789012345678901234567890123
         102 132588    GEECOE GENERATOR SERVICE UNIT 170 - 2851 SIMPSON  RICH
             3333332222444444244445454525455444254452333222333325445544225444
             13258800007553F5075E5214F20352693505E9401700D02851039D03FE002938
          64 MOND         BC V6X2R2    604-278-4488
             44442222222224425353532222333233323333
             DFE40000000002306682220000604D278D4488

Above we used uvcp to restore the record size to agree with the COBOL program.

Alternative - modify the COBOL program

We might consider changing the COBOL program to accept a text type file vs fixed length. Change ORGANIZATION in the SELECT clause as shown below:

ORGANIZATION RECORD SEQUENTIAL

ORGANIZATION LINE SEQUENTIAL

  1. You can only do this if there are NO packed or binary fields in the records.

  2. You would have to change the ORGANIZATION in all programs that read the file (see the cross reference reports in Part_9)

  3. An advantage is that you could then view the file using the 'vi' editor

  4. A disadvantage is that processing would be somewhat slower, but this would be insignificant unless you had huge files.

Goto:   Begin this document End this document UVSI Home-Page

T4. uvcp: converting TEXT files to FIXED record lengths & vice-versa

using 'uvcp' to convert 'fixed' to 'text'

Fixed length records without LineFeeds cannot be viewed or printed with the usual unix/linux utilities (vi,more,lp,etc). To these utilties, the file appears as 1 long line. Try the following:


 #0. cd /home/uvadm     <-- easy access to dat/... test/demo files
     ==============

 #1. vi dat1/sales1     <-- try vi on a file w/o LineFeeds
     ==============
     - the screen will be filled with data, but records are not distinct.
     - and 'vi' will display --> noEOL, 1 Line, 1280 Chars

 #2. uvhd dat1/sales1   <-- try uvhd
     ================
                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 130140    21 940802 IN111001  HAM001  00002{ 0000001 00000002{
             3333332222332333333244333333224443332233333723333333233333333722
             130140000021094080209E1110010081D0010000002B00000001000000002B00
          64 130140    21 940802 IN111001  SCR012  00002A 0001001 00002102A
             3333332222332333333244333333225453332233333423333333233333333422
             130140000021094080209E111001003320120000002100001001000002102100
         128 139923    35 950802 IN111002  CHR001  00002B 0002001 00004402B
             3333332222332333333244333333224453332233333423333333233333333422
             139923000035095080209E111002003820010000002200002001000004402200

 #3. uvcp "fili1=dat1/sales1,typ=RSF,rcs=64,filo1=tmp/sales1,typ=LSTt"
     =================================================================
     - use uvcp to convert file type from RSF (Record Seqntl Fixed)
     - to LSTt (Line Seqntl Terminated)
     - option 't' inserts the LF after the last nonblank

 #4. vi tmp/sales1        <-- use vi to view the converted file (typ=LST)
     =============

130140 21 940802 IN111001 HAM001 00002{ 0000001 00000002{ 130140 21 940802 IN111001 SCR012 00002A 0001001 00002102A 139923 35 950802 IN111002 CHR001 00002B 0002001 00004402B

Note
  • we are showing only the 1st 3 of 20 records
  • vi does not show the LineFeeds (use uvhd if you wish to see them)

Goto:   Begin this document End this document UVSI Home-Page

U0. uvcp: copy/modify data files (text & fixed length)

uvcpfix1 & uvcpfix2

uvcpfix1 & uvcpfix2 are scripts to make it easy to change file-types, record-sizes, reformat records, translate, etc. These scripts use 'uvcp', but the coding required is much simpler than coding the uvcp comamnd directly, see examples below:

U1. Ex#1 - uvcpfix1 convert text to Fixed


 Required - we need to create a new file of product codes & descriptions
 ========   for a COBOL program that expects fixed-length 32 byte records.

 Method - use the editor to create a text file of products & descriptions
 ====== - then convert the file to Fixed-Length using uvcpfix1

 Extra - also store a Line-Feed in the last byte for unix convenience
 ===== - not required for the COBOL program
       - but allows us to use unix utilities on a fixed length file
       - possible only when last byte unused

 #1. Login --> /home/userxx
 #2. mkdir dat1 tmp     <-- make subdirs (if not already existing)

 #3. vi dat1/products   <-- use editor to create text file of
     ================       product codes & descriptions, as follows:
      BBQ010   BAR-B-Q
      CHR015   LAWN CHAIR
      HAM010   CLAW HAMMER
        --- etc ---

 #4. uvcpfix1 dat1/products 32 RST
     =============================
     - convert dat1/products from text to fixed-lenth in tmp/products
     - arg3 'RST' inserts LF x'0A' in the last byte
     - see all file-type codes on pages 'F1' - F7.

 #5. uvhd tmp/products r32   <-- check output file with uvhd
     =====================     - note LF x'0A' in last byte
                  10        20        30        40        50        60
 r#    1 0123456789012345678901234567890123456789012345678901234567890123
       0 BBQ010   BAR-B-Q               .
         44533322244524252222222222222220
         221010000212D2D1000000000000000A
 rec#=1 rsize=32 fptr=0 fsize=288 rcount=9

 #6. cp tmp/products dat1
     ====================
     - save output file by copying back to dat1/products
     - overwrites input (could rename if desired)

Goto:   Begin this document End this document UVSI Home-Page

U2. uvcpfix1 & uvcpfix2 - scripts to copy/modify data files

Ex#2 - uvcpfix2 change Fixed to Text & reformat records


 Given - customer master & sales history file (see uvhd of 1st record below)
 ===== - 256 byte records with 24 x 5 byte packed fields (thisyr & lastyr sales)
       - packed fields & no LineFeeds mean 'vi' can not be used

 Required - create a 'Customer Telephone List' from customer master & sales file
 ======== - customer#, customer-name, telephone#,& contact-name

 #1. Login --> /home/userxx
 #2. mkdir dat1 tmp     <-- make subdirs (if not already existing)

 #3. cp $UV/dat1/custmas1 dat1   <-- copy demo file to your homedir
     =========================

 #4. uvhd /home/uvadm/dat1/custmas1  <-- inspect cust master file
     ==============================    - to see where desired fields are
                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 130140    EVERGREEN MOTORS LTD.    1815 BOWEN ROAD          NANA
             3333332222454545444244545524542222233332445442544422222222224444
             130140000056527255E0DF4F230C44E0000181502F75E02F140000000000E1E1
          64 IMO          BC V9S1H1    250-754-5531 LARRY WRENCH     ..4V|...
             4442222222222442535343222233323332333324455525544442222201357000
             9DF00000000002306931810000250D754D55310C12290725E38000000246C000
         128 .........W0....`........)X}..f3.....\.................4V}.......
             0000000005300016000000002570063100095000000000000000013570000000
             0C0000C0270D0540C0000C0098D0263C0444C0000C0000C0000C0246D0000C00
         192 .E|...V}.......................f.....<........f.C 19950531
             0470005700000000880000000018000680001300000000694233333333222222
             35C0046D0000C0023C0000C0083C0056D0012C0000C0016D3019950531000000
 rec#=1 rsize=256 fptr=0 fsize=8192 rcount=32

 #5. uvcpfix2 dat1/custmas1 256 65 LSTt 'mvc=35(30)90'
     =================================================
     - convert fixed 256 record to 65 byte text (1st 35 + 90-119)
     - we need to move (mvc) tel# & contact-name from 90-119 to 35-64
     - file type 'LSTt' inserts the Line-Feed after the last non-blank
     - instructions (mvc) must be enclosed in single quotes
     - writes output to tmp/custmas1 (for inspection & relocation if OK)

Goto:   Begin this document End this document UVSI Home-Page


 #6. vi tmp/custmas1     <-- inspect output (1st 3 records shown below)
     ===============
      130140    EVERGREEN MOTORS LTD.    250-754-5531 LARRY WRENCH
      132588    GEECOE GENERATORS        604-278-4488 HARRY LIGHT
      139923    JOHNSTONE BOILER & TANKS 604-320-1845 GEORGE BROWN

 #6a. uvhd tmp/custmas1 t <-- inspect with uvhd to see the LineFeed x'0A'
      ===================   - inserted after last non-blank
 rec#=1 rsize=61 fptr=0 fsize=1665 rcount=32
                  10        20        30        40        50        60
 r#    1 0123456789012345678901234567890123456789012345678901234567890123
       0 130140    EVERGREEN MOTORS LTD.    250-754-5531 LARRY WRENCH.
         3333332222454545444244545524542222233323332333324455525544440
         130140000056527255E0DF4F230C44E0000250D754D55310C12290725E38A
 rec#=2 rsize=58 fptr=61 fsize=1665 rcount=32
                  10        20        30        40        50        60
 r#    2 0123456789012345678901234567890123456789012345678901234567890123
      61 132588    GEECOE GENERATOR SERVICES604-278-4488 JOE LIGHT.
         3333332222444444244445454525455444533323332333324442444450
         13258800007553F5075E5214F2035269353604D278D44880AF50C9784A

 #7. cp tmp/custmas1 dat1/custelephones
     ==================================
     - save output file by copy/rename from tmp/custmas1 to dat1/custelephones

file 'TYP' codes used by Vancouver Utiltiies

 LST  - Line Sequential Terminated by LineFeed x'0A', after specified recsize
 LSTt - same, but LineFeed inserted after last non-blank (shortens records)
 RSF  - Record Sequential Fixed record length
 RST  - Record Sequential fixlth Terminated by LineFeed (OK if last byte unused)
 RSR  - Record Sequential fixlth Relative, LF extra, actual recsize=specified+1
 ISF  - Indexed Sequential Fixed record length (compatible with Micro Focus COBOL)
 IDX  - Indexed Sequential Variable record length (compatible with Micro Focus COBOL)
 RSV  - Record Sequential Variable length (compatible with Micro Focus COBOL)
 RDW  - Variable record length with recsize in 1st 2 bytes binary (FTP option)
 RDWz4  - 4 byte record prefix with 2 byte binary record size & 2 unused nulls
        - record size in prefix is the total record size including the prefix
 RDWz2  - RDW files wiith ONLY 2 byte headers (vs z4 4 byte standard)
        - recsize in hdr DATA SIZE ONLY (vs z4 recsize in hdr = Total Header+Data)
 RDWz18 - recsize in hdr is TOTAL slot size, includes hdr (like z4)
        - 2 byte hdr, recsize 1st 2 bytes binary BIG-end

Goto:   Begin this document End this document UVSI Home-Page

U3. uvcpfix1 & uvcpfix2 - scripts to copy/modify data files

script listing - uvcpfix1 - modify Text files

 #!/bin/ksh
 # uvcpfix1 - copy/modify a text file (with LineFeeds) changing (file type, layout)
 #          - copies to tmp subdir & prompts for vi,cat,more, or uvlp12(etc)
 #          - may then move from tmp to other desired location
 #          - by Owen Townsend, UV Software, Feb 2014 (uvcpL2F renamed)
 #          - may enter uvcp commands as arg 4 to move,clear,translate fields
 #          - see all uvcp instructions at www.uvsoftware.ca/uvcp.htm#M0 - M19
 #          - see file type codes at www.uvsoftware.ca/uvcp.htm#F1 - F7
 # uvcpfix2 - alternate to copy/modify Fixed-Length record files
 fpn="$1"; typ="$3"; cmd="$4";  # capture filename, type, & optional instrns
 integer rcs="$2";              # capture recsize (ensure numeric)
 if [[ -f "$fpn" && -n "$typ" && $rcs -gt 0 ]]; then :
 else echo "usage: uvcpfix1 filename outrecsize outtype ['mvc=,clr=,tra=,etc']"
      echo "       ==============================================================="
      echo " - arg1 is a text file with LineFeeds (max recsize 4096)"
      echo " - arg2 is record size for fixed length output file types"
      echo " - arg3 must be output-file-type (RSF/RST/RSR/LST/LSTt)"
      echo " - arg4 optional instructions, see www.uvsoftware.ca/uvcp.htm"
      echo " "
      echo "example1: uvcpfix1 dat1/produpdts 64 RSF"
      echo "          =============================="
      echo " - convert dat1/produpdts text file to fixlth 64 bytes no LFs"
      echo "   for input to COBOL update that demands fixlth no LFs"
      echo " - typ RSF removes any existing LFs & blank fills to spcfd recsize"
      echo " - could specify RST insert LF in last byte (if unused)"
      echo " - could specify RSR insert LF after spcfd recsize (recsize+1)"
      echo " - could specify LST, would be copying to same filetype ?"
      echo " - might use LSTtc2 (option c2) to convert unprintables to '.' periods"
      echo " - might use LSTtd3 (option d3) insert CR & LF (CR old Windows DOS)"
      echo " - option w2 appended on all types to Warn overwrite same filenames"
      echo " "
      echo "example2: uvcpfix1 dat1/produpdts 64 RST 'mvc=50(8):20140201'"
      echo "          ==================================================="
      echo " - same as above & also store a date in cols 51-58"
      echo " - output typ 'RST' stores LineFeed in last byte (allows vi)"
      exit 1; fi
 #
 fbn=$(basename $fpn);          # get file basename (drop directories)
 #
 uvcp "fili1=$fpn,typ=LSTt,rcs=4096,filo1=tmp/$fbn,typ=${typ}w2,rcs=$rcs,$cmd"
 #============================================================================
 echo "$fpn converted to tmp/$fbn"
 echo "enter command: (uvhd is recommended when no LineFeeds)"
 echo "- or null & then move tmp/$fbn to another desired location"
 read reply
 if [[ "$reply" = "uvhd" ]]; then uvhd tmp/$fbn r$rcs
 elif [[ -n "$reply" ]]; then $reply tmp/$fbn
 fi
 exit 0

Goto:   Begin this document End this document UVSI Home-Page

U4. uvcpfix1 & uvcpfix2 - scripts to copy/modify data files

script listing - uvcpfix2 - modify Fixed-Record-Length files

 # uvcpfix2 - convert datafile from Fixed length (RSF) to LST/LSTt/RST/etc
 #          - copies to tmp subdir & prompts for vi,cat,more, or uvlp12(etc)
 #          - may then move from tmp to other desired location
 #          - by OT, UV Software, 2005, updated Feb2014, renamed from uvcpF2L
 #          - may enter uvcp commands as arg 4 to move,clear,translate fields
 #          - see all uvcp instructions at www.uvsoftware.ca/uvcp.htm#M0 - M19
 #          - see file type codes at www.uvsoftware.ca/uvcp.htm#F1 - F7
 # uvcpfix1 - alternate to copy/modify text files (records terminated by LineFeeds)
 fpn="$1"; typ="$4"; cmd="$5";  # capture filename, type, & optional instrns
 integer rcsi="$2";              # capture recsize (ensure numeric)
 integer rcso="$3";              # capture recsize (ensure numeric)
 if [[ -f "$fpn" && -n "$typ" && $rcsi -gt 0 && $rcso -gt 0 ]]; then :
 else echo "usage: uvcpfix2 filename insize outsize type [mvc=,clr=,tra=,etc]"
      echo "       =========================================================="
      echo " - arg1 is a fixed record length file (no LineFeeds requried)"
      echo " - arg2 is record-size for fixed length INput file"
      echo " - arg3 is rec-size for fixed lth OUTput file or max-size for text output"
      echo " - arg4 must be output-file-type (RSF/RST/RSR/LST/LSTt)"
      echo " - arg5 optional instructions, see www.uvsoftware.ca/uvcp.htm"
      echo " "
      echo "example1: uvcpfix2 dat1/sales1 64 65 LST"
      echo "          =============================="
      echo " - LST adds LF after each 64 bytes, actual recsize = 65"
      echo " - could specify RST insert LF in col 64 (if col 64 unused)"
      echo " - could specify LSTt (option t) to insert LF after last non-blank"
      echo " - could specify LSTtc2 (option c2) convert unprintables to '.' periods"
      echo " - could specify LSTtc2d3 (option d3) insert CR & LF (CR old Windows DOS)"
      echo " - option w2 appended on all types to Warn overwrite same filenames"
      echo " "
      echo "example2: uvcpfix2 dat1/custmas1 256 65 LSTt 'mvc=35(30)90'"
      echo "          ================================================="
      echo " - converts fix-lth 256 byte records to text file 65 max bytes"
      echo " - moves 90-119 (tel# & contact) to 35-65 making a telephone list"
      echo "   (cust# 0-6, cust-name 10-34, tel# 35-36, contact 37=64)"
      exit 1; fi
 # verify filesize evenly divisible by recsize
 fs=$(stat -c%s $fpn)
 ((rem = fs % rcsi))
 if ((rem)); then
    echo "file size $fs not evenly divisible by recsize $rs, remndr=$rem"
    exit 2; fi
 fbn=$(basename $fpn);          # get file basename (drop directories)
 uvcp "fili1=$fpn,typ=RSF,rcs=$rcsi,filo1=tmp/$fbn,typ=${typ}w2,rcs=$rcso,$cmd"
 #=============================================================================
 echo "$fpn converted to tmp/$fbn"
 echo "enter command: vi,cat,more, or uvlp12,14,18,etc (1st page only)"
 echo "- or null & then move file from tmp/$fbn to another desired location"
 read reply
 if [[ "$reply" == uvlp* ]]; then $reply tmp/$fbn e1
 elif [[ -n "$reply" ]]; then $reply tmp/$fbn
 fi
 exit 0

Goto:   Begin this document End this document UVSI Home-Page

V1. uvcp file typ=EOR test/demo

File typ=EOR was added in Jan 2015 to facilitate FTP of variable length records back to the mainframe (FTP options binary & 'STRU R').

'typ=EOR' will append the EBCDIC End-Of-Record marker x'FF01' on the end of the input file records, with options to translate to EBCDIC,& shorten records to the last non-blank (EBCDIC > x'40' or ASCII > x'20').

typ=EORt0
  • does not translate to EBCDIC (may already be EBCDIC)
  • output recsize Fixed+2 if specified after filo1=...
  • else recsize taken from input records (may be variable)
typ=EORt1
  • translates to EBCDIC
  • output recsize Fixed+2 if specified after filo1=...
  • else recsize taken from input records (may be variable)
typ=EORt2
  • does not translate, shortens to lastbyte > x'20'
  • might be used to FTP ASCII records to mainframe ?
typ=EORt4
  • does not translate, shortens to lastbyte > x'40'
typ=EORt5
  • translates to EBCDIC & shortens to lastbyte > x'40'

demo dat1/testEORt1, output rcs= omitted

dat1/testEORt1 - test uvcp typ=EORt1 translate to EBCDIC & append x'FF01' at EOR, depending rcs= on fili1/filo1 rcs= fili1 only, use insize(may vary), keep trailing blanks


 uvcp "fili1=dat1/testEORt1,rcs=64,typ=LST,filo1=tmp1/testEORt1,typ=EORt1"
 =========================================================================
 uvhd /home/uvadm/tmp1/testEORt1 at=FF01
 =======================================
 rec#=1 rsize=58 fptr=0 fsize=183 rcount=3
                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 dat1/testEORt1 - test uvcp typ=EORt1 translate to EBCDIC..
             88AF6A8AACDDAF464A8AA4AA894AA97CDDAF4A989A98A84A94CCCCCCF0
             413113523569310003523045370387E5693103915231350360523493F1
 rec#=2 rsize=61 fptr=58 fsize=183 rcount=3
                      10        20        30        40        50        60
 r#        2 0123456789012345678901234567890123456789012345678901234567890123
          58 & append x'FF01' at EOR, depending rcs= on fili1/filo1     ..
             548998984A7CCFF748A4CDD64889898898498A749948898F68899F44444F0
             0017755407D6601D0130569B04575549570932E06506939116936100000F1
 rec#=3 rsize=64 fptr=119 fsize=183 rcount=3
                      10        20        30        40        50        60
 r#        3 0123456789012345678901234567890123456789012345678901234567890123
         119 rcs= fili1 only, use insize(may vary), keep trailing blanks   ..
             98A748898F4999A64AA8489A8A8498A4A89A56498894A9889898489899A444F0
             932E06939106538B04250952995D41805198DB025570391939570231522000F1
Note
  • uvhd options 'at=FF01'
  • option 'a' translates char line to ASCII (zones/digits lines EBCDIC)
  • option 't=FF01' specifies End-Of-Record ID for uvhd (new Jan 2015)

Goto:   Begin this document End this document UVSI Home-Page

V2. uvcp file typ=EOR test/demo

demo dat1/testEORt1, output rcs= specified

dat1/testEORt1 - test uvcp typ=EORt1 translate to EBCDIC & append x'FF01' at EOR, depending rcs= on fili1/filo1 rcs= also on filo1, causes fixed output recisze+2


 uvcp "fili1=dat1/testEORt1a,rcs=64,typ=LST,filo1=tmp1/testEORt1a,rcs=62,typ=EORt1"
 ================================================================******===========
 uvhd /home/uvadm/tmp1/testEORt1a at=FF01
 ========================================
 rec#=1 rsize=64 fptr=0 fsize=192 rcount=3
                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 dat1/testEORt1 - test uvcp typ=EORt1 translate to EBCDIC      ..
             88AF6A8AACDDAF464A8AA4AA894AA97CDDAF4A989A98A84A94CCCCCC444444F0
             413113523569310003523045370387E5693103915231350360523493000000F1
 rec#=2 rsize=64 fptr=64 fsize=192 rcount=3
                      10        20        30        40        50        60
 r#        2 0123456789012345678901234567890123456789012345678901234567890123
          64 & append x'FF01' at EOR, depending rcs= on fili1/filo1        ..
             548998984A7CCFF748A4CDD64889898898498A749948898F68899F44444444F0
             0017755407D6601D0130569B04575549570932E06506939116936100000000F1
 rec#=3 rsize=64 fptr=128 fsize=192 rcount=3
                      10        20        30        40        50        60
 r#        3 0123456789012345678901234567890123456789012345678901234567890123
         128 rcs= also on filo1, causes fixed output recisze+2             ..
             98A7489A949948899F6488AA8A488A8849AA9AA49888AA84F4444444444444F0
             932E01326065069361B0314252069754064374309539295E20000000000000F1
Note
  • 'rcs=62' specified here on filo1=... output (vs omitted on V1 above)
  • causes output records to be fixed 62 bytes + 2 for x'FF01' = 64
  • which (on purpose) matches the uvhd segment length 64

Goto:   Begin this document End this document UVSI Home-Page

V3. uvcp file typ=EOR test/demo

demo dat1/testEORt5, shorten to last Nonblank

dat1/testEORt5 - test uvcp typ=EORt5 translate to EBCDIC, append x'FF01',& remove trailing blanks x'40' option t5 These records blank filled to 63 bytes +LF, will be shortened


 uvcp "fili1=dat1/testEORt5,rcs=64,typ=LST,filo1=tmp1/testEORt5,typ=EORt5"
 =========================================================================
 uvhd /home/uvadm/tmp1/testEORt5 at=FF01
 =======================================
 rec#=1 rsize=59 fptr=0 fsize=179 rcount=3
                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 dat1/testEORt5 - test uvcp typ=EORt5 translate to EBCDIC,..
             88AF6A8AACDDAF464A8AA4AA894AA97CDDAF4A989A98A84A94CCCCCC6F0
             413113523569350003523045370387E5693503915231350360523493BF1
 rec#=2 rsize=57 fptr=59 fsize=179 rcount=3
                      10        20        30        40        50        60
 r#        2 0123456789012345678901234567890123456789012345678901234567890123
          59 append x'FF01',& remove trailing blanks x'40' option t5..
             8998984A7CCFF76549899A84A9889898489899A4A7FF7499A8994AFF0
             17755407D6601DB00954655039193957023152207D40D0673965035F1
 rec#=3 rsize=63 fptr=116 fsize=179 rcount=3
                      10        20        30        40        50        60
 r#        3 0123456789012345678901234567890123456789012345678901234567890123
         116 These records blank filled to 63 bytes +LF, will be shortened..
             E88A84988998A48989948899884A94FF48AA8A44DC64A8994884A899A8988F0
             385250953694202315206933540360630283520E36B069330250286935554F1
Note
  • This demos EORt5 option t5=(t1+t4) translates to EBCDIC
  • AND shortens records at last non-blank (EBCDIC x'40')

Goto:   Begin this document End this document UVSI Home-Page

W1. uvcp - Data file Utility

file type STL

File 'typ=STL' is the STandard Language file system used by AIX COBOL, added to uvcp in February 2016. These are complex file formats with file headers & record headers. There are 3 subtypes as follows:

typ=STLs
  • STL Sequential files (input & output)
typ=STLi
  • STL Indexed files (input only)
typ=STLr
  • STL Relative record files (input only)

File typ=STL Indexed & Relative is provided for input files only. You can copy them to output file types (LSTt,RSF,RST,RSV,RDW,ISF,IDXf1,& STLs Sequential).

converting STL Sequential,Indexed,& Relative to typ=LSTt

Here are examples converting the 3 demo files provided in $UV/dat1/... We will convert them to 'typ=LSTt' which can be easily checked with vi.


 uvcp "fili1=dat1/testSTLs,typ=STLs,rcs=4096,filo1=tmp/teststls,typ=LSTt"
 =======================================================================
 - convert 'typ=STLs' Sequential to 'typ=LSTt' Line Sequential

 uvcp "fili1=dat1/testSTLi,typ=STLi,rcs=4096,filo1=tmp/teststli,typ=LSTt"
 =======================================================================
 - convert 'typ=STLi' Indexed to 'typ=LSTt' Line Sequential

 uvcp "fili1=dat1/testSTLr,typ=STLr,rcs=4096,filo1=tmp/teststlr,typ=LSTt"
 =======================================================================
 - convert 'typ=STLr' Relative to 'typ=LSTt' Line Sequential

We specify 'rcs=4096' or a size you know is larger than the largest record expected. Record size specified by rcs=... will init I/O areas to blanks, The actual record size is specified in the file header &/or as a prefix on each record & could vary from record to record.

Output 'typ=LSTt' inserts a LineFeed x'0A' after the last nonblank, truncating records depending on data present. So the output records could vary from 4096 bytes down to 1 byte.

Goto:   Begin this document End this document UVSI Home-Page

W2. file typ=STL

converting STL Indexed to typ=RSF


 uvcp "fili1=dat1/testSTLi,typ=STLi,rcs=4096,filo1=tmp/teststli,typ=RSF"
 =======================================================================
 - convert 'typ=STLi' Indexed to 'typ=RSF' Record Sequential Fixed
 In this case the output records would all be 4096 bytes (blank filled on right).
 - regardless of whether Input records are variable or fixed.

 uvcp "fili1=dat1/testSTLi,typ=STLi,rcs=4096,filo1=tmp/teststli,typ=RSF,rcs=200"
 ===============================================================================
 - can specify record size on output side (after filo1=...) as desired

 uvcp "fili1=dat1/testSTLi,typ=STLi,rcs=200,filo1=tmp/teststli,typ=RST"
 ======================================================================
 - or can specify on input side if you know records are <= 200
 - this example specifies output 'typ=RST' which inserts a LineFeed in last byte
   which allows you to use unix tools (vi,etc) on the output file
   (if you know the last byte is unused for data)

converting STL Indexed (or Relative) to STL Sequential


 uvcp "fili1=dat1/testSTLi,typ=STLi,rcs=4096,filo1=tmp/teststls,typ=STLs"
 =======================================================================
 - convert 'typ=STLi' Indexed to 'typ=STLs' Seqquential
 - output record size is determined by input record size & could be variable

 uvcp "fili1=dat1/testSTLi,typ=STLi,rcs=4096,filo1=tmp/teststls,typ=STLs,rcs=200"
 ================================================================================
 - But if you specify a record size on the output side (after filo1=...)
   then the output file size will be fixed

use 'uvhd' to investigate STL files

Note that 'uvcp' automatically bypasses file headers, system records,& deleted records (in Indexed & Relative files). You could see deleted records in an Indexed file as follows:


 uvhd dat1/testSTLi x4y4   <-- display STL indexed file (x4=Indexed files)
 =======================     - add option 'y4' to see Deleted records

See https://uvsoftware.ca/uvhd.htm#5J5 for options y1,y2,y4 to show file headers, System records,& Deleted records. Use y7 to show all types.

Goto:   Begin this document End this document UVSI Home-Page

W3. STL file formats

STLs Sequential file format

Here is a uvhd (hexdump) of the STL sequential demo file provided.


 uvhd /home/cothern/testlibs/d1/testSTLs x2y1 <-- option x2=sequential STL
 ============================================   - option y1=show file header
                                                  vs default Data records only
 records=11  rsize=112 fsize=1102
 rec#=1 rsize=112 fptr=0 fsize=1102 records=11
                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 ...p ...STLS....................................................
             0007210055450000000000000000000000000000000000000000000000000000
             0000054034C31234000100000000000000000000000000000000000000000000
          64 .......[.......[.......[.......................N
             000000050000000500000005000000000001000000000004
             0000000B0000000B0000000B0000000A000000000000004E
 rec#=2 rsize=99 fptr=112 fsize=1102 records=11
                      10        20        30        40        50        60
 r#        2 0123456789012345678901234567890123456789012345678901234567890123
         112 ...[000000001
             0005333333333222222222222222222222222222222222222222222222222222
             000B000000001000000000000000000000000000000000000000000000000000
          64                       000000001...[
             22222222222222222222223333333330005
             0000000000000000000000000000001000B
 rec#=3 rsize=99 fptr=211 fsize=1102 records=11
                      10        20        30        40        50        60
 r#        3 0123456789012345678901234567890123456789012345678901234567890123
         211 ...[000000002
             0005333333333222222222222222222222222222222222222222222222222222
             000B000000002000000000000000000000000000000000000000000000000000
          64                       000000002...[
             22222222222222222222223333333330005
             0000000000000000000000000000002000B
 rec#=4 rsize=99 fptr=310 fsize=1102 records=11
                      10        20        30        40        50        60
 r#        4 0123456789012345678901234567890123456789012345678901234567890123
         310 ...[000000003
             0005333333333222222222222222222222222222222222222222222222222222
             000B000000003000000000000000000000000000000000000000000000000000
          64                       000000003...[
             22222222222222222222223333333330005
             0000000000000000000000000000003000B

For the uvhd hexdump above, option 'x2' specifies an STL Sequential file. Option 'y1' will show the file header as well as the Data records.

Goto:   Begin this document End this document UVSI Home-Page

W4. STL file formats

Notes re STL Sequential file format

The 1st 4 bytes of the file hdr is x'00000070' hex, which is 112 decimal. This defines the displacement to the begining of the 1st data record, which you can see is 112 decimal, (1st line 64 + 48 from 2nd line = 112).

The data records are preceded & followed by the record length in binary x'00000050' (horizontal hex vs vertical hex in uvhd) - is decimal 80 (5*16) The x'50' (recsize 80) conincides with the ASCII code for letter 'P'.


 uvcp "fili1=dat1/testSTLs,typ=STLs,rcs=4096,filo1=tmp/teststls,typ=LSTt"
 =======================================================================
 - convert 'typ=STLs' Sequential to 'typ=LSTt' Line Sequential

Goto:   Begin this document End this document UVSI Home-Page

W5. file typ=STL

STLi Indexed file format


 uvhd /home/cothern/testlibs/d1/testSTLi x4y7  <-- option x4=Indexed STL
 ============================================  <-- option y7=show all types
                                                   vs y4=Data records only
 records=12 deleted=3 rsize=2048 fsize=7094
 rec#=1 rsize=2048 fptr=0 fsize=7094 records=12 deleted=3
                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 .... ...STLI...................................................p
             0000210055440000000000000000001000000000000000000000000000001007
             0080054034C900000000000A0000008000000000000000000000000000010B10
             <------- showing only 1st 64 bytes of 2048 file header -------->
 rec#=2 rsize=4096 fptr=2048 fsize=7094 records=12 deleted=3
                      10        20        30        40        50        60
 r#        2 0123456789012345678901234567890123456789012345678901234567890123
        2048  .............................................._...............|
             2000000000000000000000000000000000000010000000150000001100000017
             00070000000000800000000000000000000000800000008F0000009D0000009C
             <------ showing only 1st 64 bytes of 4096 system record ------->
 rec#=3 rsize=95 fptr=6144 fsize=7094 records=12 deleted=3
                      10        20        30        40        50        60
 r#        3 0123456789012345678901234567890123456789012345678901234567890123
        6144 @.._000000001ISAM
             4005333333333454422222222222222222222222222222222222222222222222
             000F000000001931D00000000000000000000000000000000000000000000000
          64                       000000001
             2222222222222222222222333333333
             0000000000000000000000000000001
 rec#=4 rsize=95 fptr=6239 fsize=7094 records=12 deleted=3
                      10        20        30        40        50        60
 r#        4 0123456789012345678901234567890123456789012345678901234567890123
        6239 @.._000000002ISAM
             4005333333333454422222222222222222222222222222222222222222222222
             000F000000002931D00000000000000000000000000000000000000000000000
          64                       000000002
             2222222222222222222222333333333
             0000000000000000000000000000002
 rec#=5 rsize=95 fptr=6334 fsize=7094 records=12 deleted=3
                      10        20        30        40        50        60
 r#        5 0123456789012345678901234567890123456789012345678901234567890123
        6334 ..._..................._........................................
             8005000000000000000000050000000000000000000000000000000000000000
             000F0000000000000000000F0000000000000000000000000000000000000000
          64 ...............................
             0000000000000000000000000000000
             0000000000000000000000000000000

For the uvhd hexdump above, option 'x4' specifies an STL Indexed file. With no other options, we would see only the Data records, but we added option 'y7' to show All records (y1=file Hdr, y2=System recs, y4=Deleted recs).

Goto:   Begin this document End this document UVSI Home-Page

W6. STLi Indexed file format

Notes re STLi Indexed file format

The Indexed & Relative files file header records are 2048 bytes. See file hdr size in 1st 4 bytes x'00000800' = 8 * 256 = 2048 bytes.

The 2nd record at 2048 begins with x'2000000A'. x'20' identifies a System record which are always 4096 bytes. The last 3 bytes of the 4 byte record header are usually the record length, but not for these system records. I am not sure if the x'__000007' has any significance ?

The 1st data record begins at 2048 + 4096 = 6144 bytes & begins with x'4000005F' The x'40' identifies a valid data record, Deleted records would be x'80'. The x'__00005F' is the record length including the 4 byte record hdr (5*16)+(15*1) = 95.


 uvcp "fili1=dat1/testSTLi,typ=STLi,rcs=4096,filo1=tmp/teststli,typ=LSTt"
 =======================================================================
 - convert 'typ=STLi' Indexed to 'typ=LSTt' Line Sequential

Goto:   Begin this document End this document UVSI Home-Page

W7. STLr Relative file format

uvhd STL Relative test file


 uvhd /home/cothern/testlibs/d1/testSTLr x8y7  <-- option x8=relative STL
 ============================================  <-- option y7=show all types
                                                   vs y4=Data records only
 records=12 deleted=3 rsize=2048 fsize=7134
 rec#=1 rsize=2048 fptr=0 fsize=7134 records=12 deleted=3
                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 .... ...STLI...................................................p
             0000210055440000000000000000001000000000000000000000000000001007
             0080054034C900000000000A0000008000000000000000000000000000010B10
             <------- showing only 1st 64 bytes of 2048 file header -------->
 rec#=2 rsize=4096 fptr=2048 fsize=7134 records=12 deleted=3
                      10        20        30        40        50        60
 r#        2 0123456789012345678901234567890123456789012345678901234567890123
        2048  ..............................................c.......)........
             2000000000000000000000000000000000000010000000160000001200000018
             000700000000008000000000000000000000008000000083000000990000009C
             <------ showing only 1st 64 bytes of 4096 system record ------->
 rec#=3 rsize=99 fptr=6144 fsize=7134 records=12 deleted=3
                      10        20        30        40        50        60
 r#        3 0123456789012345678901234567890123456789012345678901234567890123
        6144 @..c....000000001REL
             4006000033333333354422222222222222222222222222222222222222222222
             0003000100000000125C00000000000000000000000000000000000000000000
          64                           000000001
             22222222222222222222222222333333333
             00000000000000000000000000000000001
 rec#=4 rsize=99 fptr=6243 fsize=7134 records=12 deleted=3
                      10        20        30        40        50        60
 r#        4 0123456789012345678901234567890123456789012345678901234567890123
        6243 @..c....000000002REL
             4006000033333333354422222222222222222222222222222222222222222222
             0003000200000000225C00000000000000000000000000000000000000000000
          64                           000000002
             22222222222222222222222222333333333
             00000000000000000000000000000000002
 rec#=5 rsize=99 fptr=6342 fsize=7134 records=12 deleted=3
                      10        20        30        40        50        60
 r#        5 0123456789012345678901234567890123456789012345678901234567890123
        6342 ...c...................c........................................
             8006000000000000000000060000000000000000000000000000000000000000
             0003000000000000000000030000000000000000000000000000000000000000
          64 ...................................
             00000000000000000000000000000000000
             00000000000000000000000000000000000

For the uvhd hexdump above, option 'x8' specifies an STL Relative record file. With no other options, we would see only the Data records, but we added option 'y7' to show All records (y1=file Hdr, y2=System recs, y4=Deleted recs).

Goto:   Begin this document End this document UVSI Home-Page

W8. STLr Relative file format

Notes re STL Relative files

Relative files are the same as Indexed files except that the records have an 8 byte prefix - the 4 byte record size and a 4 byte relative record#. Note that the file Relative header has 'STLI' in bytes 8-11 (same as Indexed). You might have expected 'STLR', since Sequential file headers have 'STLS'

Relative file header records are 2048 bytes. See file hdr size in 1st 4 bytes x'00000800' = 8 * 256 = 2048 bytes.

The record at 2048 begins with x'2000000A'. x'20' identifies this as a system record which is always 4096 bytes. The last 3 bytes of the record header is usually the record length, but not for system records.

The 1st data record begins at 2048 + 4096 = 6144 bytes & begins with x'40000063' The x'40' identifies a valid data record, Deleted records would be x'80'. See deleted record #5 above. x'__000063' is the record length including an 8 byte record hdr (6*16)+(3*1)=99.

Relative records have a binary record number in the last 4 bytes of the 8 byte hdr. x'00000001' - for 1st record, x'00000002' for 2nd, etc


 uvcp "fili1=dat1/testSTLr,typ=STLr,rcs=4096,filo1=tmp/teststlr,typ=LSTt"
 =======================================================================
 - convert 'typ=STLr' Relative to 'typ=LSTt' Line Sequential

Goto:   Begin this document End this document UVSI Home-Page

W9. uvcp Run OPtions to prefix output with seq# & recsize

Run option p1
  • insert 9 digit sequence# at begining of output records
Run option p2
  • insert 9 digit Relative record# (vs a sequence#)
  • applies to typ=STLr (vs typ=STLi or STLs)
Run option p4
  • insert 5 digit record-size after the sequence# & the data

These options are provided to allow you to recover STL files with corrupted indexes (for Indexed & Relative files) - especially Relative files where you might want to restore the original relative record#s & record sizes.

You would need to write an AIX COBOL program to read the uvcp output records aand re-create the STL file, rebuilding the Indexes.

example 'uvcp's with Run OPtions p1/p2/p4


 uvcp "fili1=dat1/testSTLs,typ=STLs,rcs=4096,filo1=tmp/teststls,typ=LSTt,rop=p1"
 ===============================================================================
 - rop=p1 inserts 9 digit record seq# at begining of output records

 uvcp "fili1=dat1/testSTLi,typ=STLi,rcs=4096,filo1=tmp/teststli,typ=LSTt,rop=p5"
 ===============================================================================
 - rop=p5 (p1+p4), p4 inserts the record-size between seq# & the data

 uvcp "fili1=dat1/testSTLr,typ=STLr,rcs=4096,filo1=tmp/teststlr,typ=LSTt,rop=p7"
 ===============================================================================
 - rop=p7 (p1+p2+p4), p2 uses the Relative record# ffrom the prefix vs sequence#

Here are some early examples of the output records (more later in this section).

 000000001ISAM.....data 91 bytes.....000000001           <-- rop=p0 data only

000000001_000000001ISAM.....data 91 bytes.....000000001 <-- rop=p1 inserts seq#

000000001_00091_000000001REL.....data 91 bytes.....000000001 <-- rop=p7

'rop=p7' (p1+p2+p4=p7) for Relative record files, p2 inserts the RelRec# instead of a sequence# (see later examples for REL files with deleted records), p4 inserts the record-size, see '00091_' above.

Goto:   Begin this document End this document UVSI Home-Page

W10. uvcp Run OPtions to prefix output with seq# & recsize

sample uvcp file copies with test data

See the sample uvcp commands to convert typ=STL files to typ=LSTt. We converted our test files to 'typ=LSTt' which can be easily checked with vi, but files with packed/binary in the data should be converted to typ=RSF, since typ=LSTt outputs could not be read back reliably (LineFeeds within the data would cause record splitting).

We allowed max record size of 4096, which does not matter for STL inputs specify the actual record size in the 4 byte prefix. 'rcs=4096' will also apply to the output file, but 'typ=LSTt' will be shorten them back to the last non-blank.

copy STL Relative file with rop=p7

See the sample input file, 1st 5 records listed on page 'W7' (with uvhd).


 uvcp "fili1=dat1/testSTLr,typ=STLr,rcs=4096,filo1=tmp/teststlr,typ=LSTt,rop=p7"
 ===============================================================================
 - rop=p7 (p1+p2+p4), p2 uses the Relative record# ffrom the prefix vs sequence#
 - output file listed below
      000000001_00091_000000001REL.....data 91 bytes.....000000001
      000000002_00091_000000002REL.....data 91 bytes.....000000001
      000000004_00091_000000004REL.....data 91 bytes.....000000001
      000000005_00091_000000005REL.....data 91 bytes.....000000001
      000000007_00091_000000007REL.....data 91 bytes.....000000001
      000000009_00091_000000009REL.....data 91 bytes.....000000001
      000000010_00091_000000010REL.....data 91 bytes.....000000001

Notes re output from REL file using rop=p7

  1. Option p2 inserts the relative record# (vs a sequence#).Note that there is no '000000003,000000006,000000008 because those records were deleted in our test file (listed on page 'W7'.

  2. option p4 inserts the data record-size. You might need this to recover the data from a corrupted REL file. You would write an AIX COBOL program to read the uvcp output & recreate the REL file.

  3. If the data contains packed/binary fields, you should use output typ=RSF (not typ=LSTt, which could cause record splitting by x'0A' in binary fields). Then choose an approriate output record size, for this file, the minimum size would be 107, (rr#=10 + rcs=6 + data=91 = 107). But I suggest you use 128 (a multiple of 64 for uvhd inspection convenience). The uvcp command would then be:


 uvcp "fili1=dat1/testSTLr,typ=STLr,rcs=4096,filo1=tmp/teststlr,typ=RSF,rcs=128,rop=p7"
 ======================================================================================
  1. Your COBOL program would then read 128 byte fixed size records & extract the desired data fields (rr#=10, rcs=6, data=91).

    Goto:   Begin this document End this document UVSI Home-Page

    Visitor Counters for ThisYear and LastYear