CMPjobs - Compare Fixed & Variable length data files - CONTENTS

CMPjobs - Contents


Part_0 - Overview of data compare utilities

Part_1 - Compare 'Fixed Length' Data files
- utility 'uvcmp1' will print pairs of records that are different
- differences highlighted with '*'s under 64 byte record segments
- uvcmp1 intended for files with packed/binary fields & no LineFeeds
  (use unix 'diff' for text files with LineFeeds)
- options provided for up to 4 'exclude fields' (date/time stamps, etc)
- scripts uvcmpFA1 (ASCII) & uvcmpFE1 (EBCDIC) make it easier
  to run uvcmp1 (shorter command lines)
- uvcmp1all, script to compare all data files in 2 sub-directories

Part_2 - Compare 'Variable Length RDW' Data files
- using uvcopy job 'uvcmp2' for RDW files (vs 'uvcmp1' for Fixed length)
- RDW files have 'Record Descriptor Words' in 1st 4 bytes each record
- binary record size in 1st 2 bytes (next 2 bytes null/unused)
- script 'uvcmpRDW1' makes it easier to run uvcmp2 (shorter commands)

Part_3 - Compare Micro Focus COBOL Variable Length Data files
- using uvcopy job 'uvcmp3' for IDX files (vs 'uvcmp1' or 'uvcmp2')
- may be sequential or Indexed (IDXFORMAT3 or IDXFORMAT8)
- file header record 128/1024 & binary record size in 1st 2 bytes
- script 'uvcmpIDX81' makes the command line shorter & easier to key

Part_4 - Compare Fixed length files that may have INSERTED records
- may use uvcmpI1 directly if files are already INDEXED
- else use scripts uvcmpFAI1/uvcmpFEI1 to load sequential files to
  Indexed files before calling uvcmpI1
- need to rerun with files reversed to report inserts in file2
- use uvcmpFAI2/uvcmpFEI2 to show inserts in both files on 1 report
- step1 unmatI2 writes unmatched records in file1 to tmp2/...
  by reading file2 sequentially & reading file1 randomly by key
- uvcmpI2 then reads 3 files data1, data2,& unmatched in data1

Part_5 - Compare 'entire directories' of data files
- 'uvcmp5' uses a control file to supply the record sizes
- script 'uvcmp5all' makes it easier to run
- first run 'unixcmpsplit' to separate large non-matching files
- uses unix 'cmp' to eliminate matching files before uvcmp5 is run
  (since 'cmp' is much faster than uvcmp5)

Part_6 - Compare Data files 'Field by Field'
- COBOL copybooks used to generate data compare jobs
- may compare files with different record layouts (as for Y2K)
  but can be used anytime record layouts are changed for any reason
- unlimited field excludes (for date/time stamps or any reason)
- script 'uvcopyxx' allows you to run all field by field compare jobs

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_0 CMPjobs.doc - Data File Comparison Jobs - OVERVIEW

1A0. Data File Comparison Jobs - OVERVIEW

These 'CMPjobs' (compare jobs) were developed for Year 2000 reasons, but have proven useful for many reasons long after Year 2000 work has ended.

The CMPjobs can be extremely valuable anytime you make major changes to soft- ware systems, involving parallel runs, to prove the validity of the changes.

This was proven the first time these jobs were used. An analyst had examined the printouts from the parallel runs & pronounced the test OK, but running CMPjobs on the data files highlighted several problems which would have been disastrous, had the system gone into production at that point.

These jobs to compare data files and create reports showing pairs of mismatched records, with differences highlighted by '*'s.

Records are printed in 64 byte segments. If packed or binary fields are present the segments are printed in vertical hexadecimal using 5 lines per segment (characters, zones, digits, '*' mismatches,& space between segments)

These jobs are useful for fixed & variable record length files without linefeeds that may have packed or binary fields. These files are commonly used by COBOL applications that may have been converted from a mainframe.

Note that the UNIX 'diff' utility is marvelous for text files, but cannot be used for files without linefeeds or that have packed or binary data.

The UNIX 'cmp' utility can be used for these files, but is very inadequate in showing you where the differences are. By default it quits on the first mismatch & the long option is not nice (1 byte mismatches per line in octal).

However UNIX 'cmp' is very QUICK to determine whether files are different. You can then run the uvcmp jobs only on the files which are in fact different. This plan will save time if you have very large files.

The uvcmp... jobs have a 'stop print' option, which prevents generating voluminous reports when you probably only need the first page to determine the problem, before fixing and retesting.

They also have a terminate option which allows the run to continue to count the mismatches for totals at end of job. If you have very large files, you can save time by setting the terminate count low (1000 for example) for the initial testing. For the final testing you would set it high, to ensure comparisons are performed on all records in the file.

Note that the above descriptions apply to all of the uvcmp... jobs The next page will describe the jobs in detail, highlighting the differences, and recommending which job you should use in various situations.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

0A2. CMPjobs.doc - Data File Comparison Jobs - OVERVIEW

uvcopy jobs for comparing data files

There are 2 ways to run the data comparison jobs. You can run the 'uvcopy jobs' directly or you can run 'scripts' that call the uvcopy jobs, which is easier because it makes the command lines shorter.

First we will describe the most commonly used uvcopy jobs followed by several scripts that call them. Two pages ahead we will describe the less used jobs to compare entire directories & perform field by field comparisons for 2 files with different record layouts.

uvcmp1 - compare Fixed length Sequential files

uvcmpI1 - Fixed Length allowing Inserts (but 2 reports)

uvcmpI2 - Fixed Length allowing Inserts (on 1 report)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

0A3. CMPjobs.doc - Data File Comparison Jobs - OVERVIEW

uvcmp2 - uvcopy job to compare RDW files

uvcmp2
  • compare Variable Length RDW files directly (new as of Jan2009)
  • prior to Jan2009, we converted to fixed & compared via uvcmp1
  • RDW (Record Descriptor Word) files have recsize in binary prefixes
  • RDWz4 (4 byte prefix default) & RDWz2 option 'z2' for 2 byte prefix
uvcmp3
  • compare IDXFORMAT3/IDXFORMAT8 files directly (new as of Jan2009)
  • prior to Jan2009, we converted to fixed & compared via uvcmp1
  • IDX files are Micro Focus COBOL Variable length files
  • must execute with 'uxcopy' which has MF COBOL EXTFH (file handler)
    (vs uvcopy which does not)

scripts to run uvcmp1,2,3 jobs - Fixed or Variable

uvcmpFA1
  • compare Fixed length ASCII data files (Part_1)
uvcmpFE1
  • compare Fixed length EBCDIC data files (Part_1)
uvcmpRDW1
  • compare RDW (Record Descriptor Word) variable lth files (Part_2)
  • record size in binary prefix (4 bytes default, option z2 2 bytes)
  • runs uvcopy job 'uvcmp2' to compare the 2 variable length RDW files
  • option 'y1' to translate to RBCDIC files to ASCII reports
  • option 'z2' for RDWz2 files with 2 byte prefixes (vs 4 default)
uvcmpIDX81
  • compare Micro Focus COBOL IDXFORMAT3/8 Variable lth files (Part_3)
  • runs uvcopy job 'uvcmp3' to compare 2 variable length IDX files
  • sequential or Indexed

scripts to run uvcmpI1 - allow Inserts but on 2 reports

uvcmpFAI1
  • compare Fixed ASCII files that may have INSERTED records (Part_4)
  • uvcopy loadISF1 used to load sequential files to INDEXED
  • then uvcmpI1 called to compare the 2 Indexed files
  • need to rerun with files reversed to detect inserts in 1st file
uvcmpFEI1
  • same as uvcmpFAI1, but for EBCDIC files (vs ASCII)

scripts to run uvcmpI2 - allows Inserts/Deletes on 1 report

uvcmpFAI2
  • compare Fixed ASCII files that may have INSERTED records (Part_4)
  • creates 1 report with inserts/deletes in both files
  • step1 unmatI2 writes unmatched records in file1 to tmp2/...
    by reading file2 sequentially & reading file1 randomly by key
  • uvcmpI2 then reads 3 files data1, data2,& unmatched in data1
uvcmpFEI2
  • same as uvcmpFAI2, but for EBCDIC files (vs ASCII)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

0A4. CMPjobs.doc - Data File Comparison Jobs - OVERVIEW

uvcmp5 - compare Directories of Fixed Seqntl files

unixcmpall

unixcmpsplit

uvcmp5all

uvcmp6 - generate jobs to perform field by field comparisons

recommendation

For casual use (1 pair of files at a time), assuming same size records, & less than 5 exclude fields (date/time stamps that need to be ignored), I recommend using uvcmp1, uvcmpI1,& their various scripts, because there is no preparation time required.

For serious work (comparing dozens of files at a time), I recommend running 'uvcmp5all' on all files in the 2 directories & printing all mismatch reports. The stop count allows you to limit the reports to 1 or 2 pages each.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_1 CMPjobs - Comparing Sequential files

Part 1 - comparing Fixed Length Sequential files


1A1. Overview of 'uvcmp1' for Fixed length record comparisons
- alternatives uvcmp2,3,5,6 (documented in later parts)
- scripts to run uvcmp1 (shorter commands)

1B1. Setup Directories to demo uvcmp1
- may setup 'cmpdata' directory in your homedir
- cd cmpdata & run script 'cmpdatadirs' to setup 10 subdirs required

1C1. test file#1 supplied for uvcmp1 demo (dat1/warmas1)
test file#2 with differences (dat1/warmas1a)

1D1. uvcmp1 demo#1 - Compare 2 ASCII datafiles - Operating Instructions

1D2. uvcmp1 demo#1 - sample report

1E1. uvcmp1 demo#2 - Exclude options

1E2. uvcmp1 demo#2 - Exclude Op. Instrns. & sample report

1F1. scripts to make uvcmp1 easier to run
- will use script 'uvcmpFE1' for demo#3
- will use EBCDIC files for demo#3 (warmas1E & warmas1Ea)
- uvcopy command vs script equivalent

1F2. script uvcmpFE1 demo#3 - Op. Instrns & sample report

1G1. demo#4 - creating test file for packed field comparison

1G2. demo#4 - Packed field Op. Instrns. & sample report

1H1. uvcmp1all - script to compare ALL files in 2 directories, for example:
- comparing re-converted EBCDIC fiels to original mainframe files
  to verify conversions

1X1. Summary of comparison jobs & scripts documented in this Part.
- listings of some scripts (uvcmpFA1, uvcmpFE1, uvcmp1all)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1A1. CMPjobs: Data File Comparisons

uvcopy jobs to compare Sequential files

'uvcmp_' jobs will print pairs of records that are different & highlight the differences with '*'s between the 64 byte record segments. This is usefull for comparing files with packed/binary fields, or that do not have LineFeeds.

UNIX 'diff' is good for text files, & UNIX 'cmp' (for binary files) is OK for indicating the files are the same or different, but very poor at illustrating where the differences are.

uvcmp... job summary

 *uvcmp1 - compare Fixed length Sequential files (used here in Part 1)
         - option y1 for EBCDIC (vs y0 ASCII default)
         - option 'r' (for Record-size) must be specifeid
         - allows up to 4 EXCLUDE fields defined by options a/b c/d e/f g/h
           (example: a44b6 - exclude field#1, a44=displacement, b6=length)
         - uvcmp1 does NOT allow for inserted records
uvcmpI1
  • compare Fixed length Indexed files allowing for INSERTED records
  • called by scripts uvcmpFAI1 & uvcmpFEI1 which load sequential
    files to Indexed files prior to calling uvcmpI1
  • need to rerun with files reversed to detect inserts in 1st file
uvcmpI2
  • same as uvcmpI1, but allows for inserts in both files, on 1 report
  • prior job 'unmatI2' writes unmatched records in file1 to tmp2/...
    by reading file2 sequentially & reading file1 randomly by key
  • uvcmpI2 then reads 3 files data1, data2,& unmatched in data1
uvcmp2
  • compare Variable Length RDW files directly (new as of Jan2009)
  • prior to Jan2009, we converted to fixed & compared via uvcmp1
  • RDW (Record Descriptor Word) files have recsize in binary prefixes
  • RDWz4 (4 byte prefix default) & RDWz2 option 'z2' for 2 byte prefix
uvcmp3
  • compare IDXFORMAT3/IDXFORMAT8 files directly (new as of Jan2009)
  • prior to Jan2009, we converted to fixed & compared via uvcmp1
  • IDX files are Micro Focus COBOL Variable length files
  • must execute with 'uxcopy' which has MF COBOL EXTFH (file handler)
uvcmp5
  • compare all files in 2 DIRECTORIES of Fixed Seqntl files
  • must prepare a control file with record sizes & file types
  • use with the 'uvcmp5all' script, which processes all pairs of files
    in 2 directories, & writes the reports to a 3rd directory.
uvcmp6
  • generate jobs to perform field by field comparisons
  • 'data compare jobs' generated from copybooks
  • Allows different record layouts (for Y2K or any layout change)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1A2. Data File Comparisons for Fixed Length Sequential files

scripts to run comparison jobs in Part 1

uvcmpFA1
  • compare Fixed length ASCII Sequential files
uvcmpFE1
  • compare Fixed length EBCDIC Sequential files

scripts to run comparison jobs in other Parts

uvcmpRDW1
  • compare ASCII variable length files (Part_2)
  • runs uvcmp2 to compare the 2 variable length RDW files
  • specify option 'y1' to translate to RBCDIC files to ASCII reports
uvcmpIDX81
  • compare Micro Focus COBOL Variable length files (Part_3)
  • runs uvcmp3 to compare the 2 variable length IDXFORMAT3/8 files
  • sequential or Indexed
uvcmpFAI1
  • compare Fixed ASCII files that may have INSERTED records (Part_4)
  • uvcopy loadISF1 used to load sequential files to INDEXED
  • then uvcmpI1 called to compare the 2 Indexed files
uvcmpFEI1
  • same as uvcmpFAI1, but for EBCDIC files (vs ASCII)
uvcmpFAI2
  • compare Fixed ASCII files that may have INSERTED records (Part_4)
  • creates 1 report with inserts/deletes in both files
  • step1 unmatI2 writes unmatched records in file1 to tmp2/...
    by reading file2 sequentially & reading file1 randomly by key
  • uvcmpI2 then reads 3 files data1, data2,& unmatched in data1
uvcmpFEI2
  • same as uvcmpFAI2, but for EBCDIC files (vs ASCII)
uvcmp1all
  • script to compare ALL files in 2 directories, for example:
  • comparing re-converted EBCDIC fiels to original mainframe files
    to verify conversions

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1B1. Data File Comparisons for Fixed Length Sequential files

Directory 'cmpdata' & subdirs for comparisons

I recommend you setup a 'cmpdata' within your homedir & then make the subdirs required for comparison job uvcmp1 & its scripts uvcmpFA1 & uvcmpFE1.


 #1. Login ---> /home/userxx     <-- Login or change to your home directory

 #2. mkdir cmpdata               <-- make superdir for subdirs shown above
     =============

 #3. cd cmpdata                  <-- change into the comparison superdir
     ==========

 #4. mkdir dat1 dat2 rptcmp tmp  <-- make subdirs required
     ==========================

 #5a. cp /home/uvadm/dat1/warmas1 dat1
      =================================
      - copy supplied test files to subdirs in your homedir

 #5b. cp /home/uvadm/dat1/warmas1a dat1
      ==================================
      - can store 2nd demo file in same subdir as 1st since filenames different
        (store in dat2/... if filenames same)

 #6a. vi dat1/warmas1    <-- view the demo files
      ================
 #6b. vi dat1/warmas1a
      ================

 #7a. vi dat1/warmas1   <-- print the demo files
      ================       see the files listed on the next page -->
 #7b. vi dat1/warmas1a
      ================

subdirs used to compare files

 /home/userxx                 <-- your home directory
 :-----...
 :-----cmpdata                <-- working directory for datafile compares
 :     :-----dat1             <-- subdir to store files to be compared
 :     :     :-----warmas1        <-- sample files for comparison
 :     :     :-----warmas1a
 :     :-----rptcmp           <-- subdir to receive comparison reports
 :     :     :-----warmas1        <-- comparison report demo#1

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1C1. Data File Comparisons for Fixed Length Sequential files

We will demonstrate uvcmp1 on the following 2 pages using the test files shown below:

test file#1 /home/uvadm/dat1/warmas1

 cust# prod#  description   purch-date  expire-date   price
           1         2         3         4         5         6         7
 01234567890123456789012345678901234567890123456789012345678901234567890
 12345 12345 sony television   950101 1-year 960101 000002911
 22222 00000 dish washer       960101 2-year 980101 000004922
 33333 11111 refrigerator      961231 3-year 991231 000006733
 33333 22222 air conditioner   970101 3-year 000101 000006744
 44444 00000 freezer           970101 4-year 010101 000009955
 55555 11111 clothes dryer     991231 5-year 041231 000055566
 66666 00000 micro wave        001231 6-year 061231 000066677
 88888 00000 HP computer       030101 3-year 070101 000129999

dat1/warmas1a - Alternate test file#2 with differences

 12345 12345 sony television   950101 1-year 960101 000002911
 22222 00000 dish CRASHER      960101 2-year 980101 000004922
 33333 11111 refrigerator      961215 4-year 001215 000005500
 33333 22222 air conditioner   970101 3-year 000101 000006744
 44444 00000 freezer           970101 4-year 010101 000009955
 55555 11111 clothes SHRINKER  990615 6-year 051201 000066699
 66666 00000 micro wave        001215 6-year 999999 000066677
 88888 00000 HP computer       030101 3-year 070101 000129999

Note that we have created some intentional mismatches in the 2nd file.

The operating instructions are shown on the following page --->

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1D1. Data File Comparisons for Fixed Length Sequential files

uvcmp1 demo#1 - Operating Instructions


 uvcopy uvcmp1,fili1=dat1/warmas1,fili2=dat1/warmas1a,uop=r64
 ============================================================
 uop=q1a0b0c0d0e0f0g0h0p30r256s8t500000u3x2y0  - option defaults
       a#              - dsplcmnt to exclude field #1
       b#              - length   to exclude field #1
         c#            - dsplcmnt to exclude field #2
         d#            - length   to exclude field #2
           e#          - dsplcmnt to exclude field #3
           f#          - length   to exclude field #3
             g#        - dsplcmnt to exclude field #4
             h#        - length   to exclude field #4
               n0      - print column scale only at begin report
               n1      - print column scale for each record mismatch pair
                 p30   - line# to test overflow at begin record only
                  r256 - record size
                s8     - StopPrint count, after s# mismatched records
              t500000  - StopRead count, after s# mismats, else read to EOF
            u1         - stop print at last segment with mismatch
            u2         - inhibit printing all matching segments after 1st
          x#           - hex or char print (x1 char, x2 hex, x0 auto)
        y0             - do NOT translate, file already ASCII, default(y0)
        y1             - translate char lines to ASCII (for EBCDIC files)
 090218:093751:uvcmp1: uvcopy ver=20090212 pf=/home/uvadm/pf/util/uvcmp1
 uvcopy DISAM ext=dat LNX L64  license=20090212V site=UV_Software
 User OPtion (uop) defaults  = q1p30r256s8t500000u3x2y0r64
 -->null to accept or enter/override -->
 ==================== EOF or StopPrint/StopRead count reached  ==============
 File1Total=8, File2Total=8, StopPrint=8, StopRead=500000
 File1Reads=8, TotalMisMatches=4, MisMatchesPrinted=4
 090218:093752:uvcmp1: EOF fili01 rds=8 size=512: dat1/warmas1
 090218:093752:uvcmp1: EOF fili02 rds=8 size=512: dat1/warmas1a
 090218:093752:uvcmp1: EOF fild03 size=4096: rptcmp
 090218:093752:uvcmp1: EOF filo03 wrts=53 size=3058: rptcmp/warmas1 hits=4
 EOJ, Output File written to: rptcmp/warmas1
 default command = null, OR enter: more,print,uvlpr12,edit,null -->

enter command: vi,cat,more,lp,uvlp12,null --> vi <-- enter vi to view rpt

See differences report listed on the next page --->

Note the response to the options prompt was 'r64', specifying the record sizes as 64 bytes. We specified the options twice here to emphasize their importance, once on the command line & again at the prompt (which would be redundant).

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1D2. Data File Comparisons for Fixed Length Sequential files

uvcmp1 demo#1 - sample report

 uvcmp1 - compare 2 files, print mismatched records, '*' flag diffs
 2009/02/18_09:37:51 uop=q1p30r256s8t500000u3x2y0r64
 recsize  reccount   file-size  typ Report=rptcmp/warmas1
 1:   64         8           0  RSF  File1=dat1/warmas1
 2:   64         8           0  RSF  File2=dat1/warmas1a
                       1         2         3         4         5         6
 f#rec#byte# 0123456789012345678901234567890123456789012345678901234567890123
 ============================================================================
 1    2    0 22222 00000 dish washer       960101 2-year 980101 000004922   .
             3333323333326676276766722222223333332327667233333323333333332220
             22222000000049380713852000000096010102D951209801010000004922000A
                              *******
 2    2      22222 00000 dish CRASHER      960101 2-year 980101 000004922   .
             3333323333326676245454452222223333332327667233333323333333332220
             22222000000049380321385200000096010102D951209801010000004922000A
                              *******
 1    3    0 33333 11111 refrigerator      961231 3-year 991231 000006733   .
             3333323333327667666767672222223333332327667233333323333333332220
             3333301111102562975214F200000096123103D951209912310000006733000A
                                               ** *      **  **      ****
 2    3      33333 11111 refrigerator      961215 4-year 001215 000005500   .
             3333323333327667666767672222223333332327667233333323333333332220
             3333301111102562975214F200000096121504D951200012150000005500000A
                                               ** *      **  **      ****
 1    6    0 55555 11111 clothes dryer     991231 5-year 041231 000055566   .
             3333323333326667667267767222223333332327667233333323333333332220
             5555501111103CF48530429520000099123105D951200412310000055566000A
                                 ********    **** *       *  *      *****
 2    6      55555 11111 clothes SHRINKER  990615 6-year 051201 000066699   .
             3333323333326667667254544445223333332327667233333323333333332220
             5555501111103CF485303829EB520099061506D951200512010000066699000A
                                 ********    **** *       *  *      *****
 1    7    0 66666 00000 micro wave        001231 6-year 061231 000066677   .
             3333323333326667627676222222223333332327667233333323333333332220
             666660000000D932F071650000000000123106D951200612310000066677000A
                                               **        ******
 2    7      66666 00000 micro wave        001215 6-year 999999 000066677   .
             3333323333326667627676222222223333332327667233333323333333332220
             666660000000D932F071650000000000121506D951209999990000066677000A
                                               **        ******
 ==================== EOF or StopPrint/StopRead count reached  ==============
 File1Total=8, File2Total=8, StopPrint=8, StopRead=500000
 File1Reads=8, TotalMisMatches=4, MisMatchesPrinted=4

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1D3. Data File Comparisons for Fixed Length Sequential files

Notes re uvcmp1 report on prior page

The page headings give you the information you need to know about the file (record-size, no of records, filesize, filenames, options in effect, date).

The options specified at the prompt (or via uop= on the command line), are appended to & override the default options. Please see the options described on the previous page.

The EOF count statistics are vital, indicating whether the report shows you all the mismatches, or whether the stop count was reached. The stop count saves wasting a lot of paper; you probably want to fix the programs & rerun the program & the comparison jobs. You might set the stop count & termination count options higher as your testing gets cleaner to ensure you compare to end of file with no mismatches.

enter arguments on command line or at prompts

You could enter just 'uvcopy uvcmp1' & enter filenames & options at the prompts, but it is better to enter at least the filenames on the command line, so you can use the shell history if you need to repeat the command.


 uvcopy uvcmp1  <-- will prompt for filenames & options if not on command
 =============

For uvcmp1 (intended for 1 pair of files at a time) you could wait for the prompt to specify options, but for uvcmp5/uvcmp6 it is usually advantageous to specify options on the command line & include the 'q0' option to disable the prompts for multiple files in the directory.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1E1. Data File Comparisons for Fixed Length Sequential files

uvcmp1 demo#2 - Exclude options

There are options to exclude up to 4 fields from the mismatch report (for date/time stamps or any fields desired). When you run uvcmp1, it will display the option descriptions & prompt you for changes to the defaults. Please see the options displayed for demo#1 on page '1D1' above.

For demo#2, we will exclude just the 'day' of the 'purchase-date' & the entire 'expiry-date' (see the demo file listed on page '1C1', so the options will be:


 uop=r64a34b2c44d6  <-- options for demo#2 (exclude fields)
 =================
     r64            - record size option mandatory
        a34         - dsplcmnt to 1st exclude field
           b2       - length of 1st exclude field
             c44    - dsplcmnt to 2nd exclude field
                d6  - length of 2nd exclude field

Here is the complete command line. You could enter just 'uvcopy uvcmp1' & wait for the prompts to enter filenames & options, BUT it is better to enter them on the command line, so you can use your command line history to easily repeat & modify prior commands.

uvcmp1 demo#2 Exclude - Operating Instructions


 uvcopy uvcmp1,fili1=dat1/warmas1,fili2=dat1/warmas1a,uop=r64a34b2c44d6
 ======================================================================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1E2. Data File Comparisons for Fixed Length Sequential files

uvcmp1 demo#2 Exclude - sample report

 uvcmp1 - compare 2 files, print mismatched records, '*' flag diffs
 2009/02/18_09:40:57 uop=q1p30r256s8t500000u3x2y0r64a34b2c44d6
 recsize  reccount   file-size  typ Report=rptcmp/warmas1
 1:   64         8           0  RSF  File1=dat1/warmas1
 2:   64         8           0  RSF  File2=dat1/warmas1a
                       1         2         3         4         5         6
 f#rec#byte# 0123456789012345678901234567890123456789012345678901234567890123
 ============================================================================
 1    2    0 22222 00000 dish washer       960101 2-year 980101 000004922   .
             3333323333326676276766722222223333332327667233333323333333332220
             22222000000049380713852000000096010102D951209801010000004922000A
                              *******
 2    2      22222 00000 dish CRASHER      960101 2-year 980101 000004922   .
             3333323333326676245454452222223333332327667233333323333333332220
             22222000000049380321385200000096010102D951209801010000004922000A
                              *******
 1    3    0 33333 11111 refrigerator      961231 3-year 991231 000006733   .
             3333323333327667666767672222223333332327667233333323333333332220
             3333301111102562975214F200000096123103D951209912310000006733000A
                                                  *                  ****
 2    3      33333 11111 refrigerator      961215 4-year 001215 000005500   .
             3333323333327667666767672222223333332327667233333323333333332220
             3333301111102562975214F200000096121504D951200012150000005500000A
                                                  *                  ****
 1    6    0 55555 11111 clothes dryer     991231 5-year 041231 000055566   .
             3333323333326667667267767222223333332327667233333323333333332220
             5555501111103CF48530429520000099123105D951200412310000055566000A
                                 ********    **   *                 *****
 2    6      55555 11111 clothes SHRINKER  990615 6-year 051201 000066699   .
             3333323333326667667254544445223333332327667233333323333333332220
             5555501111103CF485303829EB520099061506D951200512010000066699000A
                                 ********    **   *                 *****
 ==================== EOF or StopPrint/StopRead count reached  ==============
 File1Total=8, File2Total=8, StopPrint=8, StopRead=500000
 File1Reads=8, TotalMisMatches=3, MisMatchesPrinted=3

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1E3. Data File Comparisons for Fixed Length Sequential files

uvcmp1 demo#2 Exclude - sample report Notes

Compare demo#2 report above on page '1E2' (with exclude field options) to the prior demo#1 report on page '1D2' (without excludes) & note the following:

  1. There are no '**' under the purchase date day (bytes 34-35), because this field was excluded by options a34b2 (dsplcmnt 34, length 2).

  2. There are no '******' under the expiry date (bytes 44-49), because this field was excluded by options c44d6 (dsplcmnt 44, length 6).

  3. The 'micro wave' record is omitted because the only diffs were in the exclude fields.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1F1. Data File Comparisons for Fixed Length Sequential files

scripts to make uvcmp1 easier to run

uvcmpFA1
  • compare Fixed length SEQUENTIAL ASCII Sequential files (Part 1)
uvcmpFE1
  • compare Fixed length SEQUENTIAL EBCDIC Sequential files (Part 1)
uvcmpRDW1
  • compare ASCII variable length files (Part_2)
uvcmpIDX81
  • compare Micro Focus COBOL Variable length files (Part_3)
uvcmpFAI1
  • compare Fixed ASCII files that may have INSERTED records (Part_4)
uvcmpFEI1
  • compare Fixed EBCDIC files that may have INSERTED records (Part_4)

script uvcmpFE1 demo

We will demo script 'uvcmpFE1' which makes it easier to run the 'uvcmp1' uvcopy utility. As well as illustrating the script commands, we will also demonstrate comparing EBCDIC files (vs the prior ASCII files).

dat1/warmas1
  • ASCII file#1 used on prior uvcmp1 demos
dat1/warmas1a
  • ASCII file#2 (with some differences)
dat1/warmas1E
  • EBCDIC file#1 equivalent of ASCII warmas1
dat1/warmas1Ea
  • EBCDIC file#2 (with some differences)
Note
  • for EBCDIC files, you should specify option 'y1'
  • so you can read the character lines
  • but still see the EBCDIC zone & digit lines

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1F2. Data File Comparisons for Fixed Length Sequential files

uvcopy command vs script equivalent


 uvcopy uvcmp1,fili1=dat1/warmas1E,fili2=dat1/warmas1Ea,uop=r64y1
 ================================================================
 - uvcopy command to execute uvcmp1
 - using option 'y1' to translate character lines to ASCII

script to compare 2 Fixlth EBCDIC files


 uvcmpFE1 dat1/warmas1E dat1/warmas1Ea r64y1   <-- script to execute uvcmp1
 ===========================================
Note
  • omitting filenames & options causes a 'usage' display as follows:

 uvcmpFE1   <-- calling script without filenames & options
 ========     - causes the 'usage' display shown below:

 usage:   uvcmpFE1 file1 file2 recsize
          ============================

 example1: uvcmpFE1 dat1/warmas1E dat1/warmas1Ea r64
           =========================================

 ex2: uvcmpFE1 dat1/warmas1E dat1/warmas1Ea r64a34b2c44d6y1
      =====================================================

demo#3 - using script 'uvcmpFE1'


 uvcmpFE1 dat1/warmas1E dat1/warmas1Ea r64 <-- script to call uvcmp1
 =========================================   - uvcmpFE1 for EBCDIC files

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1F3. Data File Comparisons for Fixed Length Sequential files

script 'uvcmpFE1' - sample report

 uvcmp1 - compare 2 files, print mismatched records, '*' flag diffs
 2009/02/18_09:43:20 uop=q1p30r256s8t500000u3x2y0q1y1r64
 recsize  reccount   file-size  typ Report=rptcmp/warmas1E
 1:   64         8           0  RSF  File1=dat1/warmas1E
 2:   64         8           0  RSF  File2=dat1/warmas1Ea
                       1         2         3         4         5         6
 f#rec#byte# 0123456789012345678901234567890123456789012345678901234567890123
 ============================================================================
 1    2    0 22222 00000 dish washer       960101 2-year 980101 000004922   .
             FFFFF4FFFFF488A84A8A8894444444FFFFFF4F6A8894FFFFFF4FFFFFFFFF4440
             222220000000492806128590000000960101020851909801010000004922000A
                              *******
 2    2      22222 00000 dish CRASHER      960101 2-year 980101 000004922   .
             FFFFF4FFFFF488A84CDCECCD444444FFFFFF4F6A8894FFFFFF4FFFFFFFFF4440
             222220000000492803912859000000960101020851909801010000004922000A
                              *******
 1    3    0 33333 11111 refrigerator      961231 3-year 991231 000006733   .
             FFFFF4FFFFF4988988898A99444444FFFFFF4F6A8894FFFFFF4FFFFFFFFF4440
             333330111110956997591369000000961231030851909912310000006733000A
                                               ** *      **  **      ****
 2    3      33333 11111 refrigerator      961215 4-year 001215 000005500   .
             FFFFF4FFFFF4988988898A99444444FFFFFF4F6A8894FFFFFF4FFFFFFFFF4440
             333330111110956997591369000000961215040851900012150000005500000A
                                               ** *      **  **      ****
 1    6    0 55555 11111 clothes dryer     991231 5-year 041231 000055566   .
             FFFFF4FFFFF4899A88A489A8944444FFFFFF4F6A8894FFFFFF4FFFFFFFFF4440
             555550111110336385204985900000991231050851900412310000055566000A
                                 ********    **** *       *  *      *****
 2    6      55555 11111 clothes SHRINKER  990615 6-year 051201 000066699   .
             FFFFF4FFFFF4899A88A4ECDCDDCD44FFFFFF4F6A8894FFFFFF4FFFFFFFFF4440
             555550111110336385202899525900990615060851900512010000066699000A
                                 ********    **** *       *  *      *****
 1    7    0 66666 00000 micro wave        001231 6-year 061231 000066677   .
             FFFFF4FFFFF4988994A8A844444444FFFFFF4F6A8894FFFFFF4FFFFFFFFF4440
             666660000000493960615500000000001231060851900612310000066677000A
                                               **        ******
 2    7      66666 00000 micro wave        001215 6-year 999999 000066677   .
             FFFFF4FFFFF4988994A8A844444444FFFFFF4F6A8894FFFFFF4FFFFFFFFF4440
             666660000000493960615500000000001215060851909999990000066677000A
                                               **        ******
 ==================== EOF or StopPrint/StopRead count reached  ==============
 File1Total=8, File2Total=8, StopPrint=8, StopRead=500000
 File1Reads=8, TotalMisMatches=4, MisMatchesPrinted=4

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1F4. Data File Comparisons for Fixed Length Sequential files

Notes re warmas1E report on prior page

  1. The character line of this demo#3 report (using EBCDIC files) looks the same as in the demo#1 report on page '1D2', because option 'y1' has translated the character line to ASCII (so we can read it).

  2. But you can see that the zone & digit lines are in EBCDIC. For example the 1st byte of the 1st line (dish washer) is character '3', zone 'F', & digit '3'. IE a '3' in EBCDIC is a x'F3' vs '30' in ASCII. The 'd' (of dish washer) is x'84' vs x'64' in ASCII.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1G1. Data File Comparisons for Fixed Length Sequential files

demo #4 - packed fields

For demo#4, we will use demo file dat1/custmas1 & ask you to copy & create a 2nd mismatched file by using the 'uvhd' utility (which you will find invaluable for investigating files with packed/binary date).

  1. cp dat1/custmas1 tmp - copy demo file to tmp for modify

 2. uvhd tmp/custmas1 r256u   - enter uvhd to update 256 byte records
    =======================   - option 'u' to allow update
                              - uvhd displays 1st record & prompts for command
                              - we will then enter '6' to goto record #6
 r#       1           1         2         3         4         5         6
 b#       0 0123456789012345678901234567890123456789012345678901234567890123
          0 130140    EVERGREEN MOTORS LTD.    1815 BOWEN ROAD          NANA
            3333332222454545444244545524542222233332445442544422222222224444
            130140000056527255E0DF4F230C44E0000181502F75E02F140000000000E1E1
         64 IMO          BC V9S1H1    604-754-5531                  ........
            4442222222222442535343222233323332333322222222222222222200000000
            9DF00000000002306931810000604D754D55310000000000000000000000C000
        128 .........W0....`........)X}..f3.....\.......................f...
            0000000005300016000000002570063100095000000000000000000000016000
            0C0000C0270D0540C0000C0098D0263C0444C0000C0000C0000C0000C0056C00
        192 ...............................f.....<........f.C
            0000008900000000880000000018000680001300000000694222222222222222
            00C0026C0000C0023C0000C0083C0056D0012C0000C0016D3000000000000000

--> 6 <-- enter '6' to goto record #6

 r#       6           1         2         3         4         5         6
 b#    1280 0123456789012345678901234567890123456789012345678901234567890123
          0 147615  X O'CONNER R.V. CENTRE     44430 YALE ROAD WEST     CHIL
            3333332252424444452525224445542222233333254442544425455222224444
            1476150080F73FEE5202E6E035E4250000044430091C502F140753400000389C
         64 LIWACK       BC V2P6J1    604-858-4611BONNIE o"conner   ........
            4454442222222442535343222233323332333344444426266666722200000000
            C9713B00000002306206A10000604D858D46112FEE950F23FEE520000000C000
        128 ...4Vx........BT|...W.............. .........GO.................
            0013579000000045700059000000000000920000000004400000000000000000
            0C2468C0000C0024C0017C0000C0000C0380C0000C0057F0000C0000C0000C00
        192 ...........0.....\.........\........`...........
            0000000000830000150000000995000000086000000000002222222222222222
            00C0000C0220C0058C0000C0214C0000C0210C0000C0000C0000000000000000

Enter the following 'update' commands to make changes for uvcmp1 demo#4 (we have already shown the changes in the display above).

  1. u 99(2),'61' - change tel# 4161 to 4611
  2. u 109(8),'o"conner' - add Bonnie's last name
  3. u 130(5),x'123456789C' - modify a packed decimal field
  4. u 174(1),x'4F' - change the sign in a packed decimal field

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1G2. Data File Comparisons for Fixed Length Sequential files

demo #4 packed fields - Operating Instructions


 uvcopy uvcmp1,fili1=dat1/custmas1,fili2=tmp/custmas1,uop=r256
 ===========================================================
 - use the uvcopy job directly (longer command)

 uvcmpFA1 dat1/custmas1 tmp/custmas1 r256  <-- OR use the script (easier)
 ========================================
 uvcmpFA1 uvcmp1 - compare 2 files, print mismatched records, '*' flag diffs
 2008/05/24_16:50:28 uop=q1p30r256s8t500000v2y0q1r256
 recsize  reccount   file-size  typ Report=rptcmp/custmas1
 1:  256        32           0  RSF  File1=dat1/custmas1
 2:  256        32           0  RSF  File2=tmp/custmas1
                       1         2         3         4         5         6
 f#rec#byte# 0123456789012345678901234567890123456789012345678901234567890123
 ============================================================================
 1    6    0 147615  X O'CONNER R.V. CENTRE     44430 YALE ROAD WEST     CHIL
             3333332252424444452525224445542222233333254442544425455222224444
             1476150080F73FEE5202E6E035E4250000044430091C502F140753400000389C
          64 LIWACK       BC V2P6J1    604-858-4161BONNIE            ........
             4454442222222442535343222233323332333344444422222222222200000000
             C9713B00000002306206A10000604D858D41612FEE950000000000000000C000
                                                **        ********
         128 ..............BT|...W.............. .........GL.................
             0000000000000045700059000000000000920000000004400000000000000000
             0C0000C0000C0024C0017C0000C0000C0380C0000C0057C0000C0000C0000C00
               *****                                       *
 2    6      147615  X O'CONNER R.V. CENTRE     44430 YALE ROAD WEST     CHIL
             3333332252424444452525224445542222233333254442544425455222224444
             1476150080F73FEE5202E6E035E4250000044430091C502F140753400000389C
          64 LIWACK       BC V2P6J1    604-858-4611BONNIE o"conner   ........
             4454442222222442535343222233323332333344444426266666722200000000
             C9713B00000002306206A10000604D858D46112FEE950F23FEE520000000C000
                                                **        ********
         128 ...4Vx........BT|...W.............. .........GO.................
             0013579000000045700059000000000000920000000004400000000000000000
             0C2468C0000C0024C0017C0000C0000C0380C0000C0057F0000C0000C0000C00
               *****                                       *
 ==================== EOF or StopPrint/StopRead count reached  ==============
 File1Total=32, File2Total=32, StopPrint=8, StopRead=500000
 File1Reads=32, TotalMisMatches=1, MisMatchesPrinted=1

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1H1. converting EBCDIC data-files back to ASCII

uvcmp1all - compare all files in 2 directories

Here is a good illustration of the usefulness of the 'uvcmp1all' script:

When we convert mainframe EBCDIC files to ASCII, we can verify the conversions by converting back to EBCDIC & comparing the re-converted EBCDIC files to the original mainframe files.

You might also need the ASCII to EBCDIC conversions to return some files to the mainframe if you can not switch all applications over to unix/linux at the same time.

Generate jobs: EBCDIC-->ASCII & ASCII-->EBCDIC

 cpys -------> maps --------> pfx1 --------> pfx2 ---------> pfx3 -------> pfx4
      cobmap1       uvdata51       uvdata52       copy/edit       uvdata55

Execute jobs: EBCDIC-->ASCII, ASCII-->EBCDIC,& compare

 Main -----> d1ebc ----------> d2asc ----------> d3ebc --------> rptcmp
 frame FTP   uvcopyxx 'pfx3/*'  uvcopyxx 'pfx4/*'   **uvcmp1all** mismatch rpts

generate ASCII-->EBCDIC jobs - for ALL data-files


 #1. mkdir pfx4       <-- make subdir for ASCII-->EBCDIC jobs
     ==========           (if not already existing)

 #2. uvcopyx uvdata55 pfx3 pfx4 uop=q0i7   <-- generate ALL jobs
     ===================================

Execute ALL jobs to convert ALL data-files


 #3. uvcopyxx 'pfx4/*'                     <-- execute ALL jobs
     =================

Compare ALL re-converted files to original EBCDIC files


 #4. mkdir rptcmp           <-- make subdir for mismatch reprots
     ============

 #5. uvcmp1all d1ebc d3ebc  <-- compare all original files to reconverted files
     =====================

 #6a. vi rptcmp/*          <-- inspect all mismatch reports
      ===========

 #6b. uvlpd12 rptcmp       <-- print all mismatch reports
      ==============

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1X0. Data File Comparisons for Fixed Length Sequential files

uvcopy jobs used in Part 1

uvcmp1
  • compare 2 files, record by record & flag differences with '*'s
  • compares record pairs in FIXED record length files w/o LineFeeds

scripts used in Part 1


1X1. uvcmpFA1 - compare 2 Fixed length ASCII Sequential files

1X2. uvcmpFE1 - compare 2 Fixed length EBCDIC Sequential files

1X3. uvcmp1all - script to compare ALL files in 2 directories, for example:
- comparing re-converted EBCDIC fiels to original mainframe files
to verify conversions

The scripts will be listed below since they are only 1 or 2 pages each.

The uvcopy job (uvcmp1) will not be listed because it is much longer, but you can view it or print it as follows:


 #1. cd /home/uvadm
     ==============

 #2. vi pf/util/uvcmp1
     =================

 #3. uvlp12 pf/util/uvcmp1
     =====================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1X1. Data File Comparisons for Fixed Length Sequential files

script 'uvcmpFA1' - compare Ascii Fixed length files

 #!/bin/ksh
 # uvcmpFA1 - script to compare 2 Fixed length ASCII data files
 #          - may have packed & binary fields
 #          - by Owen Townsend, October 2007, May 2008
 #Feb18/09 - rename uvcmp2->5, uvcmp3->6, RDW->uvcmp2, IDX->uvcmp3
 #
 #            ** scripts not allowing for inserts/deletes **
 #
 #*uvcmpFA1 (this script) - compare 2 fixed length ASCII data files
 # uvcmpFE1 - compare 2 fixed length EBCDIC data files
 # uvcmpRDW1 - compare RDW variable length files - ASCII or EBCDIC (option y1)
 # uvcmpIDX81 - compare Micro Focus COBOL IDXFORMAT3/8 Variable length files
 #
 #              ** scripts allowing for INSERTS/DELETES **
 #
 # uvcmpFAI2 - FixLth ASCII w inserted recs by loading Index files typ=ISF
 # uvcmpFEI2 - FixLth EBCDIC w inserted recs by loading Index files typ=ISF
 #
 # - the 2 datafiles may have same names if in different subdirs
 # - run these scripts from a working directory with subdirs holding datafiles
 # - will make subdir rptcmp for report (if not already existing)
 # - output report will be created in subdir: rptcmp/filename1
 #
 export JOBID2=uvcmpFA1  # so uvcopy uvcmp1 can show script name on report
 if [[ -f "$1" && -f "$2" && -n "$3" ]]; then :
    else echo "usage:   uvcmpFA1 file1 file2 options/recsize"
         echo "         ===================================="
         echo "example1: uvcmpFA1 dat1/warmas1 dat1/warmas1a r64"
         echo "          ======================================="
         echo "compare 2 Fixed length ASCII data files"
         echo " - arg1 & arg2 are filenames that must be present"
         echo " - arg3 option alphas+numerics, example r64a34b2c44d6"
         echo " - Record-size optn 'r' must be specified"
         echo " "
         echo "ex2: uvcmpFE1 dat1/warmas1 dat1/warmas1a r64a34b2c44d6"
         echo "     ================================================="
         echo " - ex#2 demos exclude field options (displacements & lengths)"
         echo " - a34b2 excludes 34-35, c44d6 excludes 44-49"
         echo " - see all options described in /home/uvadm/pf/util/uvcmp1"
         echo " "
         echo "ERROR - arg1 & arg2 must be files, arg3(options) must be spcfd"
         exit 99; fi
 #
 df1="$1"; df2="$2"; optns="$3";
 if [[ ! -d rptcmp ]]; then mkdir rptcmp; fi
 #
 uvcopy uvcmp1,fili1=$df1,fili2=$df2,fild3=rptcmp,uop=q1$optns
 #============================================================
 # - uvcmp1 creates comparison report in rptcmp/filename1
 #   and prompts for disposition, may enter vi, more, uvlp12, etc
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1X2. Data File Comparisons for Fixed Length Sequential files

script 'uvcmpFE1' - compare EBCDIC Fixed length files

 #!/bin/ksh
 # uvcmpFE1 - script to compare 2 Fixed length EBCDIC data files
 #          - files may have packed & binary fields
 #          - by Owen Townsend, October 2007, May 2008
 #Feb18/09 - rename uvcmp2->5, uvcmp3->6, RDW->uvcmp2, IDX->uvcmp3
 #
 #            ** scripts not allowing for inserts/deletes **
 #
 # uvcmpFA1 - compare 2 fixed length ASCII data files
 #*uvcmpFE1 (this script) - compare 2 fixed length EBCDIC data files
 # uvcmpRDW1 - compare RDW variable length files - ASCII or EBCDIC (option y1)
 # uvcmpIDX81 - compare Micro Focus COBOL IDXFORMAT3/8 Variable length files
 #
 #              ** scripts allowing for INSERTS/DELETES **
 #
 # uvcmpFAI2 - FixLth ASCII w inserted recs by loading Index files typ=ISF
 # uvcmpFEI2 - FixLth EBCDIC w inserted recs by loading Index files typ=ISF
 #
 # - the 2 datafiles may have same names if in different subdirs
 # - run these scripts from a working directory with subdirs holding datafiles
 # - will make subdir rptcmp for report (if not already existing)
 # - output report will be created in subdir: rptcmp/filename1
 #
 export JOBID2=uvcmpFE1  # so uvcopy uvcmp1 can show script name on report
 echo "uvcmpFE1 - compare Fixed lth Ebcdic files (Nov28/07)"
 if [[ -f "$1" && -f "$2" && -n "$3" ]]; then :
    else echo "usage:   uvcmpFE1 file1 file2 recsize"
         echo "         ============================"
         echo "example1: uvcmpFE1 dat1/warmas1 dat1/warmas1a r64"
         echo "          ======================================="
         echo "compare 2 Fixed length EBCDIC data files"
         echo " - arg1 & arg2 are filenames that must be present"
         echo " - arg3 option alphas+numerics, example r64a34b2c44d6y1"
         echo " - Record-size optn 'r' must be specified"
         echo " "
         echo "ex2: uvcmpFE1 dat1/warmas1 dat1/warmas1a r64a34b2c44d6y1"
         echo "     ==================================================="
         echo " - ex#2 demos exclude field options (displacements & lengths)"
         echo " - a34b2 excludes 34-35, c44d6 excludes 44-49, y1 EBCDIC:ASCII"
         echo " - see all options described in /home/uvadm/pf/util/uvcmp1"
         echo " "
         echo "ERROR - arg1 & arg2 must be files, arg3(options) must be spcfd"
         exit 99; fi
 #
 df1="$1"; df2="$2"; optns="$3";
 if [[ ! -d rptcmp ]]; then mkdir rptcmp; fi
 #
 uvcopy uvcmp1,fili1=$df1,fili2=$df2,fild3=rptcmp,uop=q1y1$optns
 #==============================================================
 # - uvcmp1 creates comparison report in rptcmp/filename1
 #   and prompts for disposition, may enter vi, more, uvlp12, etc
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1X3. Data File Comparisons for Fixed Length Sequential files

uvcmp1all - compare ALL files in 2 directories

 #!/bin/ksh
 # uvcmp1all - script to compare all files in 2 directories
 #           - by Owen Towsnend, Feb 2010 at Everis
 # calls 'uvcmp1' for fixed length files w/o LineFeeds
 # - shows record pairs in vertical hexadecimal
 #   with mismatches flagged with '*'s
 # uses Indexed control file to get record size (ctl/ctlfile4I)
 # - same control file as used for EBCDIC to ASCII conversion
 #
 echo "uvcmp1all - compare all files in 2 directories, using uvcmp1"
 echo "- to show record pairs in vertical hexadecimal with '*' mismatches"
 echo "- for files with packed/binary & no LineFeeds (use diff for text files)"
 d1="$1"; d2="$2";
 if [[ -d "$1" && -d "$2" ]]; then :
    else echo "usage: uvcmp1all dir1 dir2 [options]"
         echo "       ============================="
         echo "ex1:   uvcmp1all d2asc.old d2asc"
         echo "       ========================="
         echo "ex2:   uvcmp1all d1ebc d3ebc y1"
         echo "       ========================"
         echo "- option y1 for EBCDIC not req'd if dirname contains *ebc*"
         exit 99; fi
 echo "uvcmp1 $d1/... $d2/..."
 echo "- will write mismatch reports to subdir rptcmp/..."
 reply=x;
 until [[ "$reply" = "y" || "$reply" = "n" ]]
    do echo "- clear all old mismatch reports from rptcmp/... y/n ?"
       read reply; done
 if [[ "$reply" == "y" ]]; then rm -f rptcmp/*; echo "rptcmp/* removed"; fi
 x=0;y=0;
 for d1f in $d1/*
   { let x=x+1
     f=$(basename $d1f)
     if [[ $f == *.idx ]]; then continue; fi
     if [[ ! -f $d2/$f ]]; then
        echo "bypassing nomatch in $d2 for $d1/$f"
        continue; fi
     #
     # use uvcopy job to get recsize & key options from control file
     # - will write to: /tmp/${LOGNAME}.uvcmp
     # remove any '.dat' (not coded in control file)
     fx=${f%.dat}
     uvcopy uvcmpget2,filr1=${CNVDATA}/ctl/ctlfile4I\
 ,filo1=/tmp/${LOGNAME}.uvcmp,arg1=$fx
     if (( $? != 0))
        then echo "datafilename not in control file"; exit 2;fi
     #
     # open the file written by uvcopy & retrieve options into $variables
     exec 3< /tmp/${LOGNAME}.uvcmp   # open file written by uvcopy
     read -u3 copyname ctlops other  # read file contents into variables
     exec 3<&-                       # close file
     #
     # if dirname contains 'ebc' add option y1 to trnslt chars to ASCII
     if [[ $d1 == *ebc* ]]; then ctlops=${ctlops}y1; fi
     #
     echo "file# $x $d1/$f $d2/$f"
     uvcopy uvcmp1,fili1=$d1/$f,fili2=$d2/$f,rop=r0,uop=q0i7$ctlops
     #=============================================================
     if (( $? != 0)); then ((y+=1)); fi
   }
 echo "$y mismatches of $x total files compared in $d1 & $d2"
 echo "vi rptcmp/*      <-- view all mismatch reports"
 echo "==========="
 echo "uvlpd12 rptcmp   <-- print all mismatch reports"
 echo "=============="
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_2 CMPjobs - Comparing data files

Part 2 - comparing Variable Length RDW data files


2A1. Comparing Variable Length RDW files - Overview
'uvcmp2' to compare Variable Length RDW files
vs 'uvcmp1' for FIXED Length files (see Part_1)
'uvcmpRDW1' script easier way to run uvcmp2 (shorter commands)
Variable Length file types (RDW, BDW/RDW, BDW/RDW unblocked, Micro Focus)

2B1. Setup Directories to demo uvcmp2
- may setup 'cmpdata' directory in your homedir
- make subdirs & copy test/demo files from /home/uvadm/dat1/...

2C1. testRDWe & testRDWea - test files supplied for VarLth comparison demo
- display with 'uvhd', since binary RDW conflicts with 'vi'

2D1. Op. Instrns to compare RDW files using uvcopy job 'uvcmp2' directly
2D2. console log
2D3. sample report

2E1. Op. Instrns to compare RDW files using script 'uvcmpRDW1'
2E2. console log
2E3. sample report

2F1. creating table summary stats of record sizes in variable length files
uvcopy job 'RDWstat1' Operating Instructions & sample report

2X1. Summary of comparison jobs & scripts documented in this Part.
- uvcopy jobs uvcmp2, RDWstat1
- scripts uvcmpRDW1

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2A1. Comparing Variable Length RDW Data Files

comparing variable length RDW files - Overview

We will document 2 ways to compare variable length RDW files

  1. using uvcopy job 'uvcmp2' directly

  2. using script 'uvcmpRDW1' which calls uvcmp2 (easiest method).

    types of Variable Length files

  3. RDWz4 - records have 4 byte a 'Record Descriptor Word' (prefix) - record size in 1st 2 bytes, next 2 bytes x'0000' unused - the binary value is the total slot (-4 for data-size) - record size is in binary 'Big-End' (mainframe vs Intel Little-End) - we will illustrate RDWz4 file compares here in Part 2.

  4. RDWz2 - records have 2 byte a 'Record Descriptor Word' (prefix) (the 2 null bytes are omitted) - the binary value is the data-size (+2 for total slot size)

  5. BDW/RDW - Blocks have 4 byte 'BDW's containing records with 4 byte 'RDW's - this is the traditional IBM variable length tape format

  6. BDW+RDW - unblocked records where every record has both BDW & RDW the same.

  7. Micro Focus variable length Sequential or Indexed (IDXFORMAT3/IDXFORMAT8) - file header 128 or 1024 bytes - each record has 2 or 4 byte prefix (not same as RDW) - see format described in uvhd.htm#X1 thru X5. - could convert to fixed length using uvhd with option 'v' (or uxcp, uxsort, uxcopy) - we will illustrate Micro Focus varlth compares later in Part_3.

    sample variable length RDW data files

Before documenting the comparison procedures, we will illustrate the RDW variable length file format on the next few pages.


2B1. testRDWe - sample VARIABLE length data file in RDW format.

2B2. testRDWea - Alternate test file for comparison
- with some intentional mismatches

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2B1. Comparing Variable Length RDW Data Files

subdirs used to compare varlth files

 /home/userxx                <-- your home directory
 :-----...
 :-----cmpdata               <-- working directory for datafile compares
 :     :
 :     :-----dat1            <-- subdir to store test files
 :     :     :-----testRDWe      <-- sample RDW file for comparison
 :     :     :-----testRDWea     <-- 'a'lternate file with some mismatches
 :     :
 :     :-----rptcmp          <-- subdir to receive comparison reports
 :     :     :-----testRDWe      <-- comparison report
 :     :                           - name same as file#1
 :     :
 :     :-----tmp             <-- subdir required by uvhd for any Printing
 :     :     :-----testRDWe_090217_124800P  <-- Print file created by uvhd

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2B2. Comparing Variable Length RDW Data Files

setup subdirs & copy test/demo files


 #1. login to your homedir & mkdir cmpdata (if not already existing)

 #2. cd cmpdata                 <-- change into comparison directory
     ==========

 #3. mkdir dat1 rptcmp tmp      <-- make subdirs required
     =====================

 #4a. cp /home/uvadm/dat1/testRDWe dat1
      =================================
      - copy 1st EBCDIC varlth file to be compared to dat1/...

 #4b. cp /home/uvadm/dat1/testRDWea dat1
      ==================================
      - copy 2nd EBCDIC varlth file to be compared to dat1/...

 #5. uvhd dat1/testRDWe z4a
     ======================
     Investigate 1st variable lth file with options:
     z4 - use binary recsize in 1st 2 bytes of record
     a  - translate uvhd character line from EBCDIC to ASCII

 #5a. --> p8     <-- Print 8 records (all)
                   - see printout below
Note
  • see the demo file listed on the next page (via uvhd)
  • can NOT list with text utilities (lp,vi,etc) due to binary RDW

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2C1. Comparing Variable Length RDW Data Files

testRDWe - sample VARIABLE length RDW data file


 uvhd dat1/testRDWe z4a1  <-- option z4 = variable lth binary 1st 2 bytes
 =======================    - option a1 = translate char line to Ascii
                            - zones & digits lines show EBCDIC codes
                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 ....DELL10 - Dell Inc.
             0100CCDDFF464C8994C98444
             080045331000045330953B00
 r#        2
          24 ....HP0010 - Hewlett Packard
             0100CDFFFF464C8A98AA4D889898
             0C00870010000856353307132194
 r#        3
          52 ....IBM010 - International Business Machines
             0200CCDFFF464C9A8998A899894CAA898AA4D888898A
             0C009240100009535951396513024295522041389552
 r#        4
          96 ....MFC010 - Micro Focus COBOL
             0200DCCFFF464D88994C98AA4CDCDD44
             00004630100004939606634203626300
 r#        5
         128 ....MS0010 - Microsoft Corp.
             0100DEFFFF464D8899A98A4C9994
             0C0042001000049396266303697B
 r#        6
         156 ....REDH10 - Red Hat Linux
             0100DCCCFF464D884C8A4D89AA44
             0C00954810000954081303954700
 r#        7
         184 ....SUN010 - Sun Microsystems Ltd
             0200EEDFFF464EA94D8899AAAA89A4DA8444
             040024501000024504939628235420334000
 r#        8
         220 ....UVSI10 - UV Software Inc.
             0200EEECFF464EE4E98AA8984C984444
             0000452910000450266361950953B000
  1. RDW format stores RecLth in 1st 2 bytes binary (next 2 bytes null/unused)

  2. RecLth prefix for 1st rec is x'0018' = ((1*16)+(8*1)) = 24 bytes

  3. RecLth is Big-End binary, right adjusted & zero left filled (like mainframe).

  4. Many Unix/Linux systems are Intel architecture, left adjusted & 0 right fill. To perform calculations on Intel, binary 24 would be x'1800' (not x'0018'). uvcopy provides option 's' (switch), specify as (2bs) vs (2b).

  5. See mode doc for RDW & BDW files on pages X1-X5 of uvhd.htm#X1.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2C2. Comparing Variable Length RDW Data Files

testRDWea - Alternate file for compare (with mismatches)


 uvhd dat1/testRDWea z4a1  <-- option z4 = variable lth binary 1st 2 bytes
 ========================    - option a1 = translate char line to Ascii
                             - zones & digits lines show EBCDIC codes
                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 ....DELL10 - Dell Inc.
             0100CCDDFF464C8994C98444
             080045331000045330953B00
 r#        2
          24 ....HP0010 - Hewlett Packard
             0100CDFFFF464C8A98AA4D889898
             0C00870010000856353307132194
 r#        3
          52 ....IBM010 - International Business Mothers
             0200CCDFFF464C9A8998A899894CAA898AA4D9A889A4
             0C009240100009535951396513024295522046385920
 r#        4
          96 ....MFC010 - Micro Focus COBOL
             0200DCCFFF464D88994C98AA4CDCDD44
             00004630100004939606634203626300
 r#        5
         128 ....MS0010 - Microsoft Corp.
             0100DEFFFF464D8899A98A4C9994
             0C0042001000049396266303697B
 r#        6
         156 ....REDH10 - Red Hat Linux
             0100DCCCFF464D884C8A4D89AA44
             0C00954810000954081303954700
 r#        7
         184 ....SUN010 - Sun Microsystems Corp
             0200EEDFFF464EA94D8899AAAA89A4C99944
             040024501000024504939628235420369700
 r#        8
         220 ....UVSI10 - UV Software Inc.
             0200EEECFF464EE4E98AA8984C984444
             0000452910000450266361950953B000
  1. Can you see the differences between this file 'testRDWea' & 'testRDWe' listed on the previous page.

  2. If not, it is a good reason to use these comparison jobs, especially when your data files could be thousands of times bigger.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2D1. Comparing Variable Length RDW Data Files

compare RDW files method #1 - use uvcmp2 directly

Assuming the 2 EBCDIC RDW files have already been copied to subdir dat1/... in your homedir (as shown on page '2B1').


 #1. uvcopy uvcmp2,fili1=dat2/testRDWe,fili2=dat2/testRDWea,uop=y1z4
     ===============================================================
     - run comparison utility
       option y1 - translate char line to ASCII (zone & digit lines EBCDIC)
       option z4 - RDW file with 4 byte recsize prefixes (vs 2 bytes)
     - see the console log displayed on the next page --->

 #2. uvlp12 rptcmp/testRDWe
     ======================
     - print output report

alternative 'script uvcmpRDW1' vs 'uvcopy uvcmp2'


 #1a. uvcmpRDW1 dat1/testRDWe dat1/testRDWea z4y1  <-- execute alternative
      ===========================================      (shorter command)
      - option 'z4' (4 byte prefixes) is default & may be omitted
      - option 'y1' translates character line to ASCII (for EBCDIC files)
Note
  • see script 'uvcmpRDW1' Op. Instrns. on page '2E1'
  • see script 'uvcmpRDW1' listing on page '2X1'

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2D2. Comparing Variable Length RDW Data Files

console log from uvcmp2 2 FixLth files written by uvhd


 uvcopy uvcmp2,fili1=dat2/testRDWe,fili2=dat2/testRDWea,uop=y1z4
 ===============================================================
 uvcmp2 - compare files & print diffs in hex with "*" markers (Nov28/07)
 uop=q1a0b0c0d0e0f0g0h0p30r8192s8t500000u3x2y0  - option defaults
       a#              - dsplcmnt to exclude field #1
       b#              - length   to exclude field #1
         c#            - dsplcmnt to exclude field #2
         d#            - length   to exclude field #2
           e#          - dsplcmnt to exclude field #3
           f#          - length   to exclude field #3
             g#        - dsplcmnt to exclude field #4
             h#        - length   to exclude field #4
               n0      - print column scale only at begin report
               n1      - print column scale for each record mismatch pair
                p30    - line# to test overflow at begin record only
                  r8192- record size (largest RDW recsize, default 8192)
                s8     - StopPrint count, after s# mismatched records
              t500000  - StopRead count, after s# mismats, else read to EOF
            u1         - stop print at last segment with mismatch
            u2         - inhibit printing all matching segments after 1st
          x#           - hex or char print (x1 char, x2 hex, x0 auto)
        y0             - do NOT translate, file already ASCII, default(y0)
        y1             - translate char lines to ASCII (for EBCDIC files)
      z2               - RDWz2 2 byte RDW recsize prefixes (vs RDWz4 dflt)
 090217:131431:uvcmp2: uvcopy ver=20090212 pf=/home/uvadm/pf/util/uvcmp2
 uvcopy DISAM ext=dat LNX L64  license=20090212V site=UV_Software
 User OPtion (uop) defaults  = q1p30r8192s8t500000u3x2y0y1
 -->null to accept or enter/override -->
 ==================== EOF or StopPrint/StopRead count reached  ==============
 File1Total=0, File2Total=0, StopPrint=8, StopRead=500000
 File1Reads=8, TotalMisMatches=2, MisMatchesPrinted=2
 090217:131431:uvcmp2: EOF fili01 rds=8 size=252: dat1/testRDWe
 090217:131431:uvcmp2: EOF fili02 rds=8 size=252: dat1/testRDWea
 090217:131431:uvcmp2: EOF fild03 size=4096: rptcmp
 090217:131431:uvcmp2: EOF filo03 wrts=32 size=1496: rptcmp/testRDWe hits=2
 EOJ, Output File written to: rptcmp/testRDWe
 default command = null, OR enter: more,print,uvlpr12,edit,null -->

See differences report listed on the next page --->

Note
  • we will not demo 'exclude field' options here in Part 2
  • see exclude field options on page '1E1' & sample report on '1E2'
  • you can rerun using the same or different exclude field options

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2D3. Comparing Variable Length RDW Data Files

testRDWe - sample comparison report

 ${JOBID2} uvcmp2 - compare RDW files, print mismatched records, '*' flag diffs
 2009/02/17_13:14:31 uop=q1p30r8192s8t500000u3x2y0y1
 recsize  reccount   file-size  typ Report=rptcmp/testRDWe
 1: 8192         0*          0  RDW  File1=dat1/testRDWe
 2: 8192         0*          0  RDW  File2=dat1/testRDWea
                       1         2         3         4         5         6
 f#rec#byte# 0123456789012345678901234567890123456789012345678901234567890123
 ============================================================================
 1    3    0 IBM010 - International Business Machines
     40      CCDFFF464C9A8998A899894CAA898AA4D888898A
             9240100009535951396513024295522041389552
                                              ** ****
 2    3      IBM010 - International Business Mothers
             CCDFFF464C9A8998A899894CAA898AA4D9A889A4
             9240100009535951396513024295522046385920
                                              ** ****
 1    7    0 SUN010 - Sun Microsystems Ltd
     32      EEDFFF464EA94D8899AAAA89A4DA8444
             24501000024504939628235420334000
                                       ****
 2    7      SUN010 - Sun Microsystems Corp
             EEDFFF464EA94D8899AAAA89A4C99944
             24501000024504939628235420369700
                                       ****
 ==================== EOF or StopPrint/StopRead count reached  ==============
 File1Total=0, File2Total=0, StopPrint=8, StopRead=500000
 File1Reads=8, TotalMisMatches=2, MisMatchesPrinted=2

Notes

  1. As you can see '*'s are used to indicate misscompares.

  2. Note that f#rec#byte# shows the file (1 or 2), record#,& byte# of segment where mismatches occur.

  3. The number under 'f#rec#' (for file#1) is the record-size of the pair. If file#2 different you would see another error message.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2E1. Comparing Variable Length RDW Data Files

compare varlth files - script uvcmpRDW1

Script 'uvcmpRDW1' makes it easier to compare variable length RDW files because:

  1. The command line is shorter.

  2. The script assumes you are in a working directory where you have permissions & groupID match to allow you to make subdirs & write files.

  3. The script will make subdir rptcmp if not already present in current working directory.

    compare varlth files - script 'uvcmpRDW1' Op. Instrns.


 #1. uvcmpRDW1 dat1/testRDWe dat1/testRDWea z4y1    <-- execute script
     ===========================================
      - option 'z4' (4 byte prefixes) is default & may be omitted
      - option 'y1' translates character line to ASCII (for EBCDIC files)

 #2a. vi rptcmp/testRDWe        <-- view report
      ==================

 #2b. uvlp12 rptcmp/testRDWe    <-- print report
      ======================
Note
  • see the 'uvcmpRDW1' script listed on page '2X1'.

uvcmpRDWAI2 compare RDW files allowing inserts/deletes

Also see 'uvcmpRDWAI2/uvcmpRDWEI2' on pages '4H1' - 4H4, to compare variable length RDW files allowing for isnerts & deletes.


 uvcmpRDWAI2 dat1/vendormas3 dat1/vendormas32 r64k0l6
 ====================================================
 - compare 2 RDW files, report created in rptcmp/vendormas3

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2F1. Comparing Variable Length RDW Data Files

creating table summary of record sizes in RDW files

'RDWstat1' is a uvcopy job that you can run to create a summary table of record-sizes found in all records of a variable file typ=RDW or typ=RDWz2 (variable length records with 4 byte or 2 byte prefixes).

demo with supplied test file


 #1. cd $UV      <-- change to /home/uvadm

 #2. uvcopy RDWstat1,fili1=dat1/testRDW,filo2=tmp/testRDW_recsizes
     =============================================================

 #3. vi tmp/testRDW_recsizes   <-- inspect report
     =======================

sample report

 RDWstat1  2007/12/19_18:10:29  record-sizes in dat1/testRDW
 tbl#001 pg#001     -argument-
 line#  count    %  record-sizes
     1       1  12  00020
     2       3  37  00024
     3       2  25  00028
     4       1  12  00032
     5       1  12  00040
             8*100   *TOTAL*

create reports for all files in directory


 #1. cd $CMPDATA     <-- change to file compare superdir

 #2. mkdir tmp1      <-- make tmp1 subdir if not existing
 #2a. rm -f tmp1/*   <-- OR remove all files if tmp1 already exists

 #3. uvcopyx RDWstat1 d0ebc tmp1 uop=q0i7,rop=r0
     ===========================================
     - create table summary recsize counts for all files in d0ebc subdir
     - output reports in tmp1 with same names as datafiles in d0ebc

 #4. uvlpd12 tmp1     <-- print all reports in tmp subdir
     ============
Note
  • see the RDWstat1 job listed on page '2X2'

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2X0. Comparing Variable Length RDW Data Files

scripts used in Part 2


2X1. uvcmpRDW1 - compare 2 variable length RDW files
- option y1/y0 EBCDIC or ASCII data files
- option z2/z4 for 2 byte or 4 byte RDW recsize prefixes
- listed on the next page.

uvcopy jobs used in Part 2

uvcmp2
  • compare 2 files, record by record & flag differences with '*'s
  • compares record pairs in FIXED record length files w/o LineFeeds

2X2. RDWstat1 - create table summary of record sizes in variable length files
- listed 2 pages below

uvcopy job 'uvcmp2' is not listed because it is too long, but you can view or print it as follows:


 #1. cd /home/uvadm
     ==============

 #2. vi pf/util/uvcmp2
     =================

 #3. uvlp12 pf/util/uvcmp2
     =====================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2X1. Comparing Variable Length RDW Data Files

uvcmpRDW1 - script to run uvcmp2

 #!/bin/ksh
 # uvcmpRDW1 - compare 2 Variable length EBCDIC files with 4 byte record headers
 #           - RDW big-end binary recsize in 1st 2 bytes, next 2 bytes x'0000'
 #           - by Owen Townsend, February 2009
 #           - 1 of several file comparison scripts for different file types
 #Feb18/09 - rename uvcmp2->5, uvcmp3->6, RDW->uvcmp2, IDX->uvcmp3
 #
 #*uvcmpRDW1 (this script) - compare 2 RDW variable length files
 #          - either ASCII or EBCDIC (if option 'y1')
 # uvcmpFA1 - compare 2 fixed length ASCII data files
 # uvcmpFE1 - compare 2 fixed length EBCDIC data files
 # uvcmpFAI2 - FixLth ASCII w inserted recs by loading Index files typ=ISF
 # uvcmpFEI2 - FixLth EBCDIC w inserted recs by loading Index files typ=ISF
 #
 # This script (uvcmpRDW1) does not allow for inserted or deleted records
 # - since RDW files are used mostly for file transfers from mainframe
 # - not for COBOL production files (since COBOL does not process RDW files)
 # - we could provide scripts to allow inserts/deletes in RDW files if required
 #
 # This script is an easier way to use the 'uvcmp2' comparison job for RDW files
 # - shorter commands
 #
 export JOBID2=uvcmpRDW1  # so uvcopy uvcmp2 can show script name on report
 if [[ -f "$1" && -f "$2" && -n "$3" ]]; then :
    else echo "usage:   uvcmpRDW1 file1 file2 [options]"
         echo "         =============================="
         echo "example: uvcmpRDW1 dat1/testRDWe dat1/testRDWea y1z4"
         echo "         ==========================================="
         echo "compare RDW files (variable length recsize in binary prefixes)"
         echo " - arg1 & arg2 are filenames that must be present"
         echo " - arg3 options (all options displayed by uvcmp2)"
         echo "        y1 - translate EBCDIC to ASCII (y0=no translate)"
         echo "        z4 - 4 byte recsize prefixes default (vs z2 2 bytes)"
         echo " "
         echo "ERROR - arg1 & arg2 must be files, arg3(options) must be spcfd"
         exit 99; fi
 df1="$1"; df2="$2"; optns="$3";
 if [[ -z "$optns" ]]; then optns=y0; fi
 if [[ ! -d rptcmp ]]; then mkdir rptcmp; fi
 #
 uvcopy uvcmp2,fili1=$df1,fili2=$df2,fild3=rptcmp,uop=q1$optns
 #============================================================
 # - uvcmp2 creates comparison report in rptcmp/filename1
 #   and prompts for disposition, may enter vi, more, uvlp12, etc
 exit 0
 #

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2X2. Comparing Variable Length RDW Data Files

RDWstat1 - table summarize record-sizes in RDW files

 # RDWstat1 - create summary table of record-sizes used in variable file
 #          - for variable lth records with 4 byte hdrs (standard RDW)
 #          - by Owen Townsend, Dec 12, 2007
 #Dec19/07 - replace getvr2 subrtn with file typ=RDW & RDWz2 (optional)
 #
 #           ** create report for 1 file (for testing) **
 #
 # uvcopy RDWstat1,fili1=d0ebc/datafilename,filo2=tmp1/reportname
 # ==============================================================
 #
 #           ** create reports for all files in directory **
 #
 # 1. cd $CNVDATA     <-- change to conversion superdir
 #                     - subdir d0ebc contains EBCDIC var lth files
 #
 # 2. mkdir tmp1      <-- make tmp1 subdir if not existing
 # 2a. rm -f tmp1/*   <-- OR remove all files if tmp1 already exists
 #
 # 3. uvcopyx RDWstat1 d0ebc tmp1 uop=q0i7,rop=r0
 #    ===========================================
 #    - create table summary recsize counts for all files in d0ebc subdir
 #    - output reports in tmp1 with same names as datafiles in d0ebc
 #
 # 4. uvlpd12 tmp1     <-- print all reports in tmp subdir
 #    ============
 #
 #                  ** sample report **
 #
 # RDWstat1  2006/12/17_18:15:17  record-sizes in d0ebc/E2121656
 # tbl#001 pg#001     -argument-
 # line#  count    %  record-sizes
 #     1  10,552  16  00046
 #     2   4,451   7  00065
 #     3  23,347  37  00066
 #     4     367   0  00068
 #     5  21,010  33  00083
 #       - - - etc - - -
 #    18       3   0  00218
 #    19     441   0  00233
 #    20     813   1  00239
 #        62,115*100   *TOTAL*
 #

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

 # This job designed for variable lth records typ=RDW & typ=RDWz2
 # typ=RDW (standard RDW) - 4 byte record prefixes
 #         - binary recsize in 1st 2 bytes (3rd & 4th x'0000' unused)
 #         - binary recsize of data (includes the 4 byte prefix)
 #         - blk/rec lth binary BIG-END format, need 's'witch option for INTEL
 # typ=RDWz2 (non-std RDW) - 2 byte record prefixes (2 nulls omitted)
 #         - binary recsize does not include the 2 byte prefix
 #
 opr='$jobname - create summary table of record-sizes used in variable file'
 opr='uop=q1z0 - option defaults (message only, see uop=... in job)'
 opr='      z0 - RDW 4 byte prefix, recsize 1st 2 BIG-END binary'
 opr='      z2 - RDW 2 byte prefix, recsize 1st 2 BIG-END binary'
 uop=q1z0      # option defaults
 rop=r1        # option to prompt for report disposition
 was=a16384
 fili1=?d0ebc/input,rcs=8192,typ=RDW        #Dec19/07 typ=RDW replaces getvr
 filo1=?tmp1/$jobname.rpt,typ=LSTt,rcs=128
 @run
 #Dec19/07 - replace subrtn getvr with typ=RDW & RDWz2
 #         - for nonstd 2 byte prefix vs std 4 bytes
 # if uop=z2, append file option z2 for typ=RDWz2
        tsb    o26(1),x'02'            uop z2 for nonstd RDW ?
        skp!   1
        cata8  $fili1+180(15),'z2'
        opn    all
 #
 # begin loop to get records & build summary table of recsizes
 # - to be dumped at end of file
 man20  get    fili1,a0                 get next RDW record data
        skp>   eof                      (cc set > at EOF)
        mvn    c0(5),$rv                cnvrt binary recsize to numeric
        tblt1f1 c0(5),'record-sizes'    build table in memory
        skp    man20                    repeat get loop til EOF
 #
 # EOF - close files & end job
 eof    mvfv1  f0(80),'record-sizes in $fili1'
        tbpt1s1 filo1,f0(50)            dump table from memory to file
        cls    all
        eoj
 #

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_3 CMPjobs - Comparing data files

Part 3 - comparing Variable Length IDXFORMAT3/8 files


3A1. Comparing Variable Length IDXFORMAT3/IDXFORMAT8 files - Overview
'uvcmp3' to compare Variable Length IDXFORMAT files
vs 'uvcmp1' for FIXED Length files (see Part_1)
vs 'uvcmp2' for Variable Length RDW files (see Part_2)
'uvcmpIDX81' script easier way to run uvcmp3 (shorter commands)
Variable Length file types (IDXFORMAT vs RDW)

3B1. Setup Directories to demo uvcmp3
- may setup 'cmpdata' directory in your homedir
- make subdirs & copy test/demo files from /home/uvadm/dat1/...

3C1. testIDX81 & testIDX81a - test files supplied for VarLth comparison demo
- display with 'uvhd', since you cannot use 'vi' for IDXFORMAT files

3D1. Op. Instrns to compare IDX files using uxcopy job 'uvcmp3' directly
3D2. console log
3D3. sample report

3E1. Op. Instrns to compare IDX files using script 'uvcmpIDX81'
3E2. console log
3E3. sample report

3F1. creating table summary stats of record sizes in variable length files
uxcopy job 'IDXstat1' Operating Instructions & sample report

3X1. Summary of comparison jobs & scripts documented in this Part.
- uxcopy jobs uvcmp3, IDXstat1
- scripts uvcmpIDX81

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3A1. Comparing Variable Length IDXFORMAT3/8 Data Files

comparing variable length IDXFORMAT3/8 files - Overview

We will document 2 ways to compare variable length IDX files

  1. using uxcopy job 'uvcmp3' directly

  2. using script 'uvcmpIDX81' which calls uvcmp3 (easiest method).

    types of Variable Length files

  3. RDWz4 - records have 4 byte a 'Record Descriptor Word' (prefix) - record size in 1st 2 bytes, next 2 bytes x'0000' unused - the binary value is the total slot (-4 for data-size) - record size is in binary 'Big-End' (mainframe vs Intel Little-End) - see RDWz4 file compares in Part 2.

  4. RDWz2 - records have 2 byte a 'Record Descriptor Word' (prefix) (the 2 null bytes are omitted) - the binary value is the data-size (+2 for total slot size)

  5. Micro Focus variable length Sequential or Indexed (IDXFORMAT3/IDXFORMAT8) - file header 128 or 1024 bytes - each record has 2 or 4 byte prefix (but not same as RDW) - see format described in uvhd.htm#V1 thru V5. - could convert to fixed length using uvhd with option 'v' (or uxcp, uxsort, uxcopy) - we will illustrate Micro Focus varlth compares here in Part_3.

    sample variable length IDX data files

Before documenting the comparison procedures, we will illustrate the IDX variable length file format on the next few pages.


3B1. testIDX81 - sample VARIABLE length data file in IDX format.

3B2. testIDX81a - Alternate test file for comparison
- with some intentional mismatches

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3B1. Comparing Variable Length IDX Data Files

subdirs used to compare varlth files

 /home/userxx                 <-- your home directory
 :-----...
 :-----cmpdata                <-- working directory for datafile compares
 :     :
 :     :-----dat1             <-- subdir to store test files
 :     :     :-----testIDX81      <-- sample IDX file for comparison
 :     :     :-----testIDX81a     <-- 'a'lternate file with some mismatches
 :     :
 :     :-----rptcmp           <-- subdir to receive comparison reports
 :     :     :-----testIDX81      <-- comparison report
 :     :                           - name same as file#1
 :     :
 :     :-----tmp              <-- subdir required by uvhd for any Printing
 :     :     :-----testIDX81_090217_124800P  <-- Print file created by uvhd

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3B2. Comparing Variable Length IDX Data Files

setup subdirs & copy test/demo files


 #1. login to your homedir & mkdir cmpdata (if not already existing)

 #2. cd cmpdata                 <-- change into comparison directory
     ==========

 #3. mkdir dat1 rptcmp tmp      <-- make subdirs required
     =====================

 #4a. cp /home/uvadm/dat1/testIDX81 dat1
      ==================================
      - copy 1st EBCDIC varlth file to be compared to dat1/...

 #4b. cp /home/uvadm/dat1/testIDX81a dat1
      ===================================
      - copy 2nd EBCDIC varlth file to be compared to dat1/...

 #5. uvhd dat1/testIDX81 v
     =====================
     Investigate 1st variable lth file with options 'v'
     - for IDXFORMAT3/IDXFORMAT8 files

 #5a. --> p99     <-- Print All records (we know < 99)
                    - see printout below
Note
  • see the demo file listed on the next page (via uvhd)
  • can NOT list with text utilities (lp,vi,etc) due to binary in IDX files

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3C1. Comparing Variable Length IDX Data Files

testIDX81 - sample VARIABLE length IDX data file


 uvhd dat1/testIDX81 v  <-- option v = variable lth IDXFORMAT
 =====================    - option a1 = translate char line to Ascii
                          - zones & digits lines show EBCDIC codes
                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 0~......0711180743423507111807434235.>...................@......
             3700000033333333333333333333333333330300000000000000000004000100
             0E00000007111807434235071118074342350E01000000001000000000000000
          64 ............................................X"8*................
             0000000000000000000000000000000000000000000052320000000000000000
             00000000000000100000000000000000000000000000828A0000000000000000
                      10        20        30        40        50        60
 r#        2 0123456789012345678901234567890123456789012345678901234567890123
         128 @.DELL00 - Dell Inc.
             41444433222466624662
             0245CC000D045CC09E3E
 r#        3
         148 @.HP0000 - Hewlett Packard..
             4145333322246766772566667600
             088000000D0857C5440013B12400
 r#        4
         176 @(IBM000 - International Business Machines..
             42444333222467676676666624776667724666666700
             0892D0000D09E452E149FE1C02539E5330D1389E5300
 r#        5
         220 @.MFC000 - Micro Focus COBOL
             4144433322246676246677244444
             0AD630000D0D932F06F35303F2FC
 r#        6
         248 @.MS0000 - Microsoft Corp...
             4145333322246676766724677200
             08D300000D0D932F3F6403F20E00
 r#        7
         276 @.REDHAT - Red Hat Linux
             415444452225662467246677
             062548140D025408140C9E58
 r#        8
         300 @.SUN000 - Sun Microsystems Ltd.
             41554333222576246676777766724760
             0D35E0000D035E0D932F39345D30C440
 r#        9
         332 @.UVSI00 - UV Software Inc..
             4155543322255256677676246620
             095639000D05603F64712509E3E0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3C2. Comparing Variable Length IDX Data Files

Notes re: Micro Focus Varlth file shown above

  1. Micro Focus Variable Length files have a 128 byte header record (see above). See uvhd.htm#V5 for the layout & location of important fields uvhd shows this record by default, but does not show other system records (which may be among the data records).

  2. Micro Focus varlth records are identified by x'4_' in 1st byte of the record. The record length is stored in next 12 bits.

  3. RecLth prefix for 1st rec is x'4012' = ((1*16)+(2*1)) = 18 bytes.

  4. RecLth is Big-End binary, right adjusted & zero left filled (like mainframe).

  5. Many Unix/Linux systems are Intel architecture, left adjusted & 0 right fill. To perform calculations on Intel, binary 18 would be x'1200' (not x'0012'). uvcopy provides option 's' (switch), specify as (2bs) vs (2b).

  6. Can you see the differences between this file 'testIDX81' & 'testIDX81a' listed on the next page.

  7. If not, it is a good reason to use these comparison jobs, especially when your data files could be thousands of times bigger.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3C3. Comparing Variable Length IDX Data Files

testIDX81a - Alternate file for compare (with mismatches)


 uvhd dat1/testIDX81a v  <-- option v = variable lth IDXFORMAT3/8 files
 ======================
                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 0~......0711180747307707111807473077.>...................@......
             3700000033333333333333333333333333330300000000000000000004000100
             0E00000007111807473077071118074730770E01000000001000000000000000
          64 ............................................X"8*................
             0000000000000000000000000000000000000000000052320000000000000000
             00000000000000100000000000000000000000000000828A0000000000000000
                      10        20        30        40        50        60
 r#        2 0123456789012345678901234567890123456789012345678901234567890123
         128 @.DELL00 - Dell Inc.
             41444433222466624662
             0245CC000D045CC09E3E
 r#        3
         148 @.HP0000 - Hewlett Packard..
             4145333322246766772566667600
             088000000D0857C5440013B12400
 r#        4
         176 @'IBM000 - International Business Mothers...
             42444333222467676676666624776667724676677000
             0792D0000D09E452E149FE1C02539E5330DF48523000
 r#        5
         220 @.MFC000 - Micro Focus COBOL
             4144433322246676246677244444
             0AD630000D0D932F06F35303F2FC
 r#        6
         248 @.MS0000 - Microsoft Corp...
             4145333322246676766724677200
             08D300000D0D932F3F6403F20E00
 r#        7
         276 @.REDHAT - Red Hat Linux
             415444452225662467246677
             062548140D025408140C9E58
 r#        8
         300 @.SUN000 - Sun Macrosystems Ltd.
             41554333222576246676777766724760
             0D35E0000D035E0D132F39345D30C440
 r#        9
         332 @.UVSI00 - UV Software Inc..
             4155543322255256677676246620
             095639000D05603F64712509E3E0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3D1. Comparing Variable Length IDX Data Files

compare IDX files method #1 - use uvcmp3 directly

Assuming the 2 EBCDIC IDX files have already been copied to subdir dat1/... in your homedir (as shown on page '3B1').


 #1. uxcopy uvcmp3,fili1=dat2/testIDX81,fili2=dat2/testIDX81a,uop=q1
     ===============================================================
     - run comparison utility
       option q1 = display prompts for option changes
     - see the console log displayed on the next page --->

 #2. uvlp12 rptcmp/testIDX81
     ======================
     - print output report

alternative 'script uvcmpIDX81' vs 'uvcopy uvcmp2'


 #1a. uvcmpIDX81 dat1/testIDX81 dat1/testIDX81a q1  <-- execute alternative
      ============================================      (shorter command)
Note
  • see script 'uvcmpIDX81' Op. Instrns. on page '3E1'
  • see script 'uvcmpIDX81' listing on page '3X1'

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3D2. Comparing Variable Length IDX Data Files

console log from uvcmp3 2 FixLth files written by uvhd


 uxcopy uvcmp3,fili1=dat2/testIDX81,fili2=dat2/testIDX81a,uop=q1
 ===============================================================
 uvcmp3 - compare files & print diffs in hex with "*" markers (Nov28/07)
 uop=q1a0b0c0d0e0f0g0h0p30r8192s8t500000u3x2y0  - option defaults
       a#              - dsplcmnt to exclude field #1
       b#              - length   to exclude field #1
         c#            - dsplcmnt to exclude field #2
         d#            - length   to exclude field #2
           e#          - dsplcmnt to exclude field #3
           f#          - length   to exclude field #3
             g#        - dsplcmnt to exclude field #4
             h#        - length   to exclude field #4
               n0      - print column scale only at begin report
               n1      - print column scale for each record mismatch pair
                p30    - line# to test overflow at begin record only
                  r8192- record size (largest RDW recsize, default 8192)
                s8     - StopPrint count, after s# mismatched records
              t500000  - StopRead count, after s# mismats, else read to EOF
            u1         - stop print at last segment with mismatch
            u2         - inhibit printing all matching segments after 1st
          x#           - hex or char print (x1 char, x2 hex, x0 auto)
        y0             - do NOT translate, file already ASCII, default(y0)
        y1             - translate char lines to ASCII (for EBCDIC files)
 090217:151629:uvcmp3: uxcopy ver=20090212 pf=/home/uvadm/pf/util/uvcmp3
 uxcopy DISAM ext=dat LNX LL64 license=20090212U site=UV_Software_Inc.
 User OPtion (uop) defaults  = q1p30r8192s8t500000u3x2y0
 -->null to accept or enter/override -->
 record pair size mismatch fili1=40, fili2=39, enter=continue
 ==================== EOF or StopPrint/StopRead count reached  ==============
 File1Total=0, File2Total=0, StopPrint=8, StopRead=500000
 File1Reads=8, TotalMisMatches=2, MisMatchesPrinted=2
 090217:151630:uvcmp3: EOF fili01 rds=8 size=360: dat1/testIDX81
 090217:151630:uvcmp3: EOF fili02 rds=8 size=360: dat1/testIDX81a
 090217:151630:uvcmp3: EOF fild03 size=4096: rptcmp
 090217:151630:uvcmp3: EOF filo03 wrts=32 size=1454: rptcmp/testIDX81hits=2
 EOJ, Output File written to: rptcmp/testIDX81
 default command = null, OR enter: more,print,uvlpr12,edit,null -->

See differences report listed on the next page --->

Note
  • we will not demo 'exclude field' options here in Part 3
  • see exclude field options on page '1E1' & sample report on '1E2'
  • you can rerun using the same or different exclude field options

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3D3. Comparing Variable Length IDX Data Files

testIDX81 - sample comparison report

 uvcmpIDX81 uvcmp3 - compare RDW files, print mismatched records, '*' flag diffs
 2009/02/17_15:36:51 uop=q1p30r8192s8t500000u3x2y0q1r8192
 recsize  reccount   file-size  typ Report=rptcmp/testIDX81
 1: 8192         0           0  IDX  File1=dat1/testIDX81
 2: 8192         0           0  IDX  File2=dat1/testIDX81a
                       1         2         3         4         5         6
 f#rec#byte# 0123456789012345678901234567890123456789012345678901234567890123
 ============================================================================
 1    3    0 IBM000 - International Business Machines
     39      4443332224676766766666247766677246666667
             92D0000D09E452E149FE1C02539E5330D1389E53
                                              ** ****
 2    3      IBM000 - International Business Mothers
             4443332224676766766666247766677246766772
             92D0000D09E452E149FE1C02539E5330DF485230
                                              ** ****
 1    7    0 SUN000 - Sun Microsystems Ltd
     29      55433322257624667677776672476
             35E0000D035E0D932F39345D30C44
                           *
 2    7      SUN000 - Sun Macrosystems Ltd
             55433322257624667677776672476
             35E0000D035E0D132F39345D30C44
                           *
 ==================== EOF or StopPrint/StopRead count reached  ==============
 File1Total=0, File2Total=0, StopPrint=8, StopRead=500000
 File1Reads=8, TotalMisMatches=2, MisMatchesPrinted=2

Notes

  1. As you can see '*'s are used to indicate misscompares.

  2. Note that f#rec#byte# shows the file (1 or 2), record#,& byte# of segment where mismatches occur.

  3. The number under 'f#rec#' (for file#1) is the record-size of the pair. If file#2 different you would see another error message.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3E1. Comparing Variable Length IDX Data Files

compare varlth files - script uvcmpIDX81

Script uvcmpIDX81 makes it easier to compare variable length IDX files because:

  1. The command line is shorter.

  2. The script assumes you are in a working directory where you have permissions & groupID match to allow you to make subdirs & write files.

  3. The script will make subdir rptcmp if not already present in current working directory.

    compare varlth files - script 'uvcmpIDX81' Op. Instrns.


 #1. uvcmpIDX81 dat1/testIDX81 dat1/testIDX81a r64    <-- execute script
     =========================================

 #2a. vi rptcmp/testIDX81        <-- view report
      ==================

 #2b. uvlp12 rptcmp/testIDX81    <-- print report
      ======================
Note
  • see the 'uvcmpIDX81' script listed on page '3X1'.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3F1. Comparing Variable Length IDX Data Files

creating table summary of record sizes in IDX files

'IDXstat1' is a uxcopy job that you can run to create a summary table of record-sizes found in all records of a variable file typ=IDX or typ=IDXz2 (variable length records with 4 byte or 2 byte prefixes).

demo with supplied test file


 #1. cd $UV      <-- change to /home/uvadm

 #2. uxcopy IDXstat1,fili1=dat1/testIDX81,filo2=tmp/testIDX81_recsizes
     =================================================================

 #3. vi tmp/testIDX81_recsizes   <-- inspect report
     =========================

sample report

 IDXstat1  2009/02/17_15:29:16  record-sizes in dat1/testIDX81 dat1/testIDX81
 tbl#001 pg#001     -argument-            -acum#1-    %
 line#  count    %  record-sizes
    1        1  12  00018                        1   12
    2        1  12  00022                        1   12
    3        2  25  00024                        2   25
    4        1  12  00025                        1   12
    5        1  12  00026                        1   12
    6        1  12  00029                        1   12
    7        1  12  00040                        1   12
             8*100   *TOTAL*                     8 *100

create reports for all files in directory


 #1. cd $CMPDATA     <-- change to file compare superdir

 #2. mkdir tmp1      <-- make tmp1 subdir if not existing
 #2a. rm -f tmp1/*   <-- OR remove all files if tmp1 already exists

 #3. uxcopyx IDXstat1 d0ebc tmp1 uop=q0i7,rop=r0
     ===========================================
     - create table summary recsize counts for all files in d0ebc subdir
     - output reports in tmp1 with same names as datafiles in d0ebc

 #4. uvlpd12 tmp1     <-- print all reports in tmp subdir
     ============
Note
  • see the IDXstat1 job listed on page '3X2'

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3X0. Comparing Variable Length IDX Data Files

scripts used in Part 3


3X1. uvcmpIDX81 - compare 2 variable length IDX files
- listed on the next page.

uxcopy jobs used in Part 3

uvcmp3
  • compare 2 files, record by record & flag differences with '*'s
  • compares record pairs in FIXED record length files w/o LineFeeds

3X2. IDXstat1 - create table summary of record sizes in variable length files
- listed 2 pages below

uxcopy job 'uvcmp3' is not listed because it is too long, but you can view or print it as follows:


 #1. cd /home/uvadm
     ==============

 #2. vi pf/util/uvcmp3
     =================

 #3. uvlp12 pf/util/uvcmp3
     =====================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3X1. Comparing Variable Length IDX Data Files

uvcmpIDX81 - script to run uvcmp3

 #!/bin/ksh
 # uvcmpIDX81 - compare 2 Variable length IDXFORMAT8 files
 #            - using 'uvcmp3' for IDXFORMAT files
 #              (vs 'uvcmp1' for Fiexd lth or 'uvcmp2' for RDW files)
 #            - by Owen Townsend, Feb 2009
 #Feb18/09 - rename uvcmp2->5, uvcmp3->6, RDW->uvcmp2, IDX->uvcmp3
 #
 # See documentation at www.uvsoftware.ca/cmpjobs.htm#Part_3
 #  Part 1 documents compares for fixed length files (both ASCII & EBCDIC)
 #  Part 2 documents compares for RDW variable length files (ASCII & EBCDIC)
 # *Part 3 (this script) documents compares for IDXFORMAT8 variable length files
 #  - no need to provide EBCDIC versions for IDXFORMAT8 (not mainframe files)
 # - the 2 datafiles may have same names if in different subdirs
 # - run this script from a working directory with subdirs holding datafiles
 # - output report will be created in subdir: rptcmp/filename1
 #
 export JOBID2=uvcmpIDX81  # so uxcopy uvcmp3 can show script name on report
 if [[ -f "$1" && -f "$2" ]]; then :
    else echo "usage:   uvcmpIDX81 file1 file2 [options]"
         echo "         ================================"
         echo "example: uvcmpIDX81 dat1/testIDX81 dat1/testIDX82"
         echo "         ========================================"
         echo "compare Variable length IDXFORMAT8 files"
         echo " - arg1 & arg2 are filenames that must be present"
         echo " - arg3 options (may also enter at uvcmp3 prompt)"
         echo " - see options described in /home/uvadm/pf/util/uvcmp3"
         echo " "
         echo "ERROR - arg1 & arg2 must be files, arg3(options) may be spcfd"
         exit 99; fi
 #
 df1="$1"; df2="$2"; optns="$3";
 if [[ -z "$optns" ]]; then optns=r8192; fi
 if [[ ! -d rptcmp ]]; then mkdir rptcmp; fi
 #
 uxcopy uvcmp3,fili1=$df1,fili2=$df2,fild3=rptcmp,uop=q1$optns
 #============================================================
 # - uvcmp3 creates comparison report in rptcmp/filename1
 #   and prompts for disposition, may enter vi, more, uvlp12, etc
 exit 0
 #

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3X2. Comparing Variable Length IDX Data Files

IDXstat1 - table summarize record-sizes in IDX files

 # IDXstat1 - create summary table of record-sizes used in variable file
 #          - for variable lth records IDXFORMAT3 or IDXFORMAT8
 #          - Micro Focus COBOL compatible
 #          - must execute with 'uxcopy' (not 'uvcopy')
 #            (uxcopy has IDXFORMAT file handler, uvcopy does not)
 #          - by Owen Townsend, Feb 2009
 #
 #           ** create report for 1 file (for testing) **
 #
 # uxcopy IDXstat1,fili1=d0ebc/datafilename,filo2=tmp/reportname
 # =============================================================
 #
 #           ** create reports for all files in directory **
 #
 # 1. cd $CNVDATA     <-- change to conversion superdir
 #                     - subdir d0ebc contains EBCDIC var lth files
 #
 # 2. mkdir tmp       <-- make tmp subdir if not existing
 # 2a. rm -f tmp/*    <-- OR remove all files if tmp already exists
 #
 # 3. uxcopyx IDXstat1 d0ebc tmp uop=q0i7,rop=r0
 #    ==========================================
 #    - create table summary recsize counts for all files in d0ebc subdir
 #    - output reports in tmp with same names as datafiles in d0ebc
 #
 # 4. uvlpd12 tmp     <-- print all reports in tmp subdir
 #    ===========
 #
 #                  ** sample report **
 #
 # IDXstat1  2006/12/17_18:15:17  record-sizes in d0ebc/E2121656
 # tbl#001 pg#001     -argument-
 # line#  count    %  record-sizes
 #     1  10,552  16  00046
 #     2   4,451   7  00065
 #     3  23,347  37  00066
 #     4     367   0  00068
 #     5  21,010  33  00083
 #       - - - etc - - -
 #    18       3   0  00218
 #    19     441   0  00233
 #    20     813   1  00239
 #        62,115*100   *TOTAL*
 #

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

 # This job designed for variable lth records typ=IDXf3 or typ=IDXf8
 #
 opr='$jobname - create summary table of record-sizes used in variable file'
 #uop=q1        # option defaults
 rop=r1        # option to prompt for report disposition
 was=a16384
 fili1=?d0ebc/input,rcs=8192,typ=IDXf8
 filo1=?tmp/$jobname.rpt,typ=LSTt,rcs=128
 @run
        opn    all
 #
 # begin loop to get records & build summary table of recsizes
 # - to be dumped at end of file
 man20  get    fili1,a0                 get next IDXf3/f8 record data
        skp>   eof                      (cc set > at EOF)
        mvn    c0(5),$rv                cnvrt binary recsize to numeric
        tblt1f1 c0(5),'record-sizes'    build table in memory
        skp    man20                    repeat get loop til EOF
 #
 # EOF - close files & end job
 eof    mvfv1  f0(80),'record-sizes in $fili1'
        tbpt1s1 filo1,f0(50)            dump table from memory to file
        cls    all
        eoj
 #

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_4 CMPjobs - Comparing data files

Part 4 - comparing files that may have Inserted records


4A1. 'uvcmpI1' to compare Indexed files with Inserted records
- alternatives uvcmp1, uvcmp2, uvcmp3, uvcmp5,& uvcmp6
 'uvcmpI2' allows for inserts in both files, on 1 report
- scripts to run uvcmpI1 (shorter commands)

4B1. Setup Directories to demo uvcmpI1
- may setup 'cmpdata' directory in your homedir
  with subdirs required for uvcmpI1 & scripts uvcmpFAI1 & uvcmpFEI1
  (dat1, rptcmp, tmp, tmp1, tmp2)

4C1. test file#1 supplied for uvcmpI1 demo (dat1/warmas1I)
test file#2 with differences (dat1/warmas1Ia)

4D1. uvcopy jobs & scripts used for test/demos
uvcopy jobs - loadISF1, uvcmpI1, unmatI2, uvcmpI2
scripts uvcmpFAI1/uvcmpFEI1 show inserts, but need to rerun with files
reversed to detect inserts in 1st file
scripts uvcmpFAI2/uvcmpFEI2 show inserts in both files on 1 report

4E1. uvcmpFAI1/uvcmpI1 demo#1 - Compare ASCII files with Inserted records
- 1st run detects inserted record in file#1, but not file#2
4E2. - console log & sample report

4F1. uvcmpFAI1/uvcmpI1 demo#2 - Compare ASCII files with Inserted records
- 2nd run detects inserted record in file#2, but not file#1
- need to rerun with files reversed to detect inserts in 1st file
4E2. - console log & sample report

4G1. uvcmpFAI2/uvcmpI2 demo#3 - Compare files with Inserts/Deletes (on 1 report)
- same as uvcmpI1, but allows for inserts in both files, on 1 report
- prior job 'unmatI2' writes unmatched records in file1 to tmp2/...
  by reading file2 sequentially & reading file1 randomly by key
- uvcmpI2 then reads 3 files data1, data2,& unmatched in data1
- options i1/i2 to ignore inserts/deletes OR mismatched pairs
 uvcmpFEI2 - for EBCDIC files (vs ASCII files)

4G2. - console log & sample report

4H1. uvcmpRDWAI2/uvcmpRDWEI2 - for RDW ASCII/EBCDIC files with inserts/deletes
- similar to above uvcmpFAI2/uvcmpFEI2
4H2.  sample report from uvcmpRDWAI2 comparing vendormas3 & vendormas32
4H3.  options for max recsize & key displacment/length
4H4.  unique key required to allow for inserts & deletes

4X1. Summary of comparison jobs & scripts documented in this Part.
 uvcopy jobs - loadISF1 & uvcmpI1 - called by scripts
 scripts uvcmpFAI1 & uvcmpFEI1
- scripts listed since they are short

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4A1. comparing Fixed length files that may have Inserted records

Part 4 shows you how to compare Fixed length files that may have Inserted or Deleted records. See Part_0 for a summary of all file compare jobs.

uvcmpI1 - compare files that may have Inserted records

uvcmpI2 - compare files that may have Inserted records

Note
  • uvcmpI1 & uvcmpI2 are 'uvcopy jobs' and they are much easier to use
    via the several scripts described below.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4A2. comparing Fixed length files that may have Inserted records

scripts to run uvcmpI1 (allow Inserts in 1 file)

uvcmpFAI1
  • compare Fixed 'ASCII' files that may have INSERTED/DELETED records
  • uvcopy loadISF1 used to load sequential files to INDEXED
  • then uvcmpI1 called to compare the 2 Indexed files
    (see description on prior page)
  • need to rerun with files reversed to detect inserts in 1st file
uvcmpFEI1
  • same as uvcmpFAI1, but for EBCDIC files (vs ASCII)

scripts to run uvcmpI2 (allow inserts in both files)

uvcmpFAI2
  • compare Fixed ASCII files that may have INSERTED records
  • creates 1 report allowing for inserts/deletes in both files
  • step1 unmatI2 writes unmatched records in file1 to tmp2/...
    by reading file2 sequentially & reading file1 randomly by key
  • uvcmpI2 then reads 3 files data1, data2,& unmatched in data1
uvcmpFEI2
  • same as uvcmpFAI2, but for EBCDIC files (vs ASCII)
uvcmpRDWAI2
  • compare variable length RDW ASCII files allowing inserts/deletes
uvcmpRDWEI2
  • same as uvcmpRDWAI2, but for EBCDIC files (vs ASCII)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4B1. comparing Fixed length files that may have Inserted records

preparation for uvcmpI1 test/demo


 #1. Login ---> /home/userxx     <-- Login or change to your home directory

 #2. mkdir cmpdata               <-- make superdir for subdirs shown above
     =============

 #3. cd cmpdata                  <-- change into the comparison superdir
     ==========

 #4. mkdir dat1 dat2 rptcmp tmp  <-- make subdirs required
     ==========================

 #5a. cp /home/uvadm/dat1/warmas1I dat1
      =================================
      - copy supplied test files to subdirs in your homedir

 #5b. cp /home/uvadm/dat1/warmas1Ia dat1
      ==================================
      - can store 2nd demo file in same subdir as 1st since filenames different
        (store in dat2/... if filenames same)

 #6a. vi dat1/warmas1I    <-- view the demo files
      ================      - see listed on page '4C1'
 #6b. vi dat1/warmas1Ia
      ================

for varlth RDW files


 #7. cp /home/uvadm/dat1/vendormas3* dat1
     ====================================
     - copies 4 RDW files for demos on pages '4H1' - 4H4
     - ASCII files vendormas3/vendormas32
     - EBCDIC files vendormas3e/vendormas3e2

 #8a. uvhd /home/uvadm/dat1/vendormas3* dat1 z4
      =========================================
      - inspect RDW files (requires uvhd due to binary prefixes)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4B2. comparing Fixed length files that may have Inserted records

subdirs used to compare files

 /home/userxx                 <-- your home directory
 :-----...
 :-----cmpdata                <-- working directory for datafile compares
 :     :-----dat1             <-- subdir to store files to be compared
 :     :     :-----warmas1I       <-- sample files for comparison
 :     :     :-----warmas1Ia
 :     :-----tmp1                 <-- tmp1 subdir for Indexed version of file1
 :     :     :-----warmas1I.dat
 :     :     :-----warmas1I.idx
 :     :-----tmp2                 <-- tmp2 subdir for Indexed version of file2
 :     :     :-----warmas1Ia.dat
 :     :     :-----warmas1Ia.idx
 :     :-----rptcmp               <-- subdir to receive comparison reports
 :     :     :-----warmas1I           <-- comparison report demo#1
 :     :     :-----warmas1Ia          <-- comparison report demo#2
                                        - with files reversed
                                        - to detect inserted records in 2nd file

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4C1. comparing Fixed length files that may have Inserted records

We will demonstrate uvcmpI1 on the following 2 pages using the test files shown below:

test file#1 /home/uvadm/dat1/warmas1I

 cust# prod#  description   purch-date  expire-date   price
           1         2         3         4         5         6         7
 01234567890123456789012345678901234567890123456789012345678901234567890
 12345 12345 sony television   950101 1-year 960101 000002911
 22222 00000 dish washer       960101 2-year 980101 000004922
 33333 11111 refrigerator      961231 3-year 991231 000006733
 33333 22222 air conditioner   970101 3-year 000101 000006744
 44444 00000 freezer           970101 4-year 010101 000009955
 55555 11111 clothes dryer     991231 5-year 041231 000055566
 66666 00000 micro wave        001231 6-year 061231 000066677
 77777 11111 panasonic HDTV 50 020601 5-year 070601 000219999  <-- Inserted
 88888 00000 HP computer       030101 3-year 070101 000129999

dat1/warmas1Ia - test file#2 with diffs & inserts

 12345 12345 sony television   950101 1-year 960101 000002911
 22222 00000 dish CRASHER      960101 2-year 980101 000004922
 33333 11111 refrigerator      961215 4-year 001215 000005500
 33333 22222 air conditioner   970101 3-year 000101 000006744
 44444 00000 freezer           970101 4-year 010101 000009955
 55555 11111 clothes SHRINKER  990615 6-year 051201 000066699
 55555 22222 miele ultra dryer 010301 5-year 050301 000099999  <-- Inserted
 66666 00000 micro wave        001215 6-year 999999 000066677
 88888 00000 HP computer       030101 3-year 070101 000129999
  1. These testfiles are similar to those used in Part_1, but each file has one INSERTED record

  2. The 1st demo can only detect inserted records in file#1, since we read sequentially by key & use that key to read randomly in file#2.

  3. The 2nd demo will reverse the order of the filenames on the command line which will detect inserted records in the 2nd file.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4D1. comparing Fixed length files that may have Inserted records

scripts to make uvcmpI1 easier to run

uvcmpFAI1
  • Fixed ASCII files that may have INSERTED/DELETED records
  • we will demo only script 'uvcmpFAI1'
uvcmpFEI1
  • same as uvcmpFAI1, but for EBCDIC files (vs ASCII)
uvcmpFAI2
  • compare Fixed ASCII files that may have INSERTED records
  • creates 1 report with inserts/deletes in both files

other scripts used in previous Parts

uvcmpFA1 - Fixed length SEQUENTIAL ASCII data files (see Part_1) uvcmpFE1 - Fixed length SEQUENTIAL EBCDIC data files (see Part_1) uvcmpRDW1 - compare variable length RDW files (Part_2) uvcmpIDX81 - MICRO FOCUS COBOL Variable length files (see Part_3)

test/demo files used in Part 4

dat1/warmas1I
  • ASCII file#1 for demos in Part 4 (with diffs & inserts)
dat1/warmas1Ia
  • ASCII file#2 for demos in Part 4 (with diffs & inserts)

test/demo files used in previous Parts

dat1/warmas1
  • ASCII file#1 used on prior uvcmp1 demos
dat1/warmas1a
  • ASCII file#2 (with some differences)
dat1/warmas1E
  • EBCDIC file#1 used for demos in previous Parts
dat1/warmas1Ea
  • EBCDIC file#2 (with some differences)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4D2. comparing Fixed length files that may have Inserted records

uvcopy jobs

loadISF1
  • load sequential file into Indexed file prior to compare job
  • run for both files to be compared
  • output to subdirs tmp1/... & tmp2/... (allows for same filenames)
uvcmpI1
  • data file comparison job for Indexed files
  • allowing for Inserted records (in file#1)
  • can run with files reversed to detect inserted records in 2nd file
uvcmpI2
  • same, but allows for inserts/deletes in both files, on 1 report
  • prior job 'unmatI2' writes unmatched records in file1 to tmp2/...
    by reading file2 sequentially & reading file1 randomly by key
  • uvcmpI2 then reads 3 files data1, data2,& unmatched in data1
  • option i1 to ignore inserts/deletes, show only mismatched pairs
  • option i2 to ignore mismatched pairs, show only inserts/deletes

uvcopy commands - hard way


 #1a. uvcopy loadISF1,fili1=dat1/warmas1I,filo1=tmp1/warmas1I,uop=q0i7r64k0l11
      ========================================================================

 #1b. uvcopy loadISF1,fili1=dat1/warmas1Ia,filo1=tmp2/warmas1Ia,uop=q0i7r64k0l11
      ==========================================================================

 #2. uvcopy uvcmpI1,fili1=tmp1/warmas1I,fili2=tmp2/warmas1Ia,uop=r64k0l11
     ====================================================================

script method - easier, shorter command


 uvcmpFAI1 dat1/warmas1I dat1/warmas1Ia r64k0l11
 ===============================================
Note
  • we will illustrate console log & resulting report on following pages:

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4E1. comparing Fixed length files that may have Inserted records

uvcmpFAI1 demo#1 - inserted record in file#2


 uvcmpFAI1 dat1/warmas1I dat1/warmas1Ia r64k0l11
 ===============================================
 uvcmpFAI1 - compare Fixed lth Ebcdic Indexed files (Dec11/07)
 090218:112644:uvcmpFAI1: EOF fili01 rds=9 size=576: dat1/warmas1I
 090218:112644:uvcmpFAI1: EOF filo01 wrts=9 size=0: tmp1/warmas1I
 090218:112644:uvcmpFAI1: EOF fili01 rds=9 size=576: dat1/warmas1Ia
 090218:112644:uvcmpFAI1: EOF filo01 wrts=9 size=0: tmp2/warmas1Ia
Note
  • above console displays from 2 runs of loadISF1 to load Indexed files
  • console displays minimized by uop=q0i7 (no need to change at run time)
 uvcmpI1 - compare files & print diffs in hex with "*" markers (Nov28/07)
 uop=q1a0b0c0d0e0f0g0h0k0l0p30r256s8t500000x2y0u1  - option defaults
       a#               - dsplcmnt to exclude field #1
       b#               - length   of exclude field #1
         c#             - dsplcmnt to exclude field #2
         d#             - length   of exclude field #2
           e#           - dsplcmnt to exclude field #3
           f#           - length   of exclude field #3
             g#         - dsplcmnt to exclude field #4
             h#         - length   of exclude field #4
               k#       - dsplcmnt to Indexed key field#1
                 l#     - length   of Indexed key field#1
                  n0    - print column scale only at begin report
                  n1    - print column scale for each record mismatch pair
                   p30  - line# to test overflow at begin record only
                  r256  - record size
                s8      - StopPrintCount, after s# mismatched records
              t500000   - StopReadCount, after s# mismats, else read to EOF
            u1          - stop print at last segment with mismatch
            u2          - inhibit printing all matching segments after 1st
          x#            - hex or char print (x1 char, x2 hex, x0 auto)
        y0              - do NOT translate, file already ASCII, default(y0)
        y1              - translate char lines to ASCII (for EBCDIC files)
 090218:112644:uvcmpFAI1: uvcopy ver=20090212 pf=/home/uvadm/pf/util/uvcmpI1
 uvcopy DISAM ext=dat LNX L64  license=20090212V site=UV_Software
 User OPtion (uop) defaults  = q1a0b0c0d0e0f0g0h0k0l0p30r256s8t500000x2y0u1q1r64k0l11
 -->null to accept or enter/override -->
 ==================== EOF or StopPrint/StopRead count reached  ==============
 File1Total=9, File2Total=9, StopPrint=8, StopRead=500000
 File1Reads=9, TotalErrs=5, MisCompares=4, File2NoRec=1
 090218:112644:uvcmpFAI1: EOF fili01 rds=9 size=585: tmp1/warmas1I
 090218:112644:uvcmpFAI1: EOF fili02 rds=9 size=585: tmp2/warmas1Ia
 090218:112644:uvcmpFAI1: EOF fild03 size=4096: rptcmp
 090218:112644:uvcmpFAI1: EOF filo03 wrts=60 size=3466: rptcmp/warmas1I hits=5
 EOJ, Output File written to: rptcmp/warmas1I
 default command = null, OR enter: more,print,uvlpr12,edit,null -->

See differences report listed on the next page --->

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4E2. comparing Fixed length files that may have Inserted records

uvcmpI1 demo#1 - sample report

 uvcmpFAI1 uvcmpI1 - compare 2 files, print mismatched records, '*' flag diffs
 2009/02/18_11:19:25 uop=q1a0b0c0d0e0f0g0h0k0l0p30r256s8t500000x2y0u1q1r64k0l11
 recsize  reccount   file-size  typ Report=rptcmp/warmas1I
 1:   64         9         585  ISF  File1=tmp1/warmas1I
 2:   64         9         585  ISF  File2=tmp2/warmas1Ia
                       1         2         3         4         5         6
 f#rec#byte# 0123456789012345678901234567890123456789012345678901234567890123
 ============================================================================
 1    2    0 22222 00000 dish washer       960101 2-year 980101 000004922   .
             3333323333326676276766722222223333332327667233333323333333332220
             22222000000049380713852000000096010102D951209801010000004922000A
                              *******
 2    2      22222 00000 dish CRASHER      960101 2-year 980101 000004922   .
             3333323333326676245454452222223333332327667233333323333333332220
             22222000000049380321385200000096010102D951209801010000004922000A
                              *******
 1    3    0 33333 11111 refrigerator      961231 3-year 991231 000006733   .
             3333323333327667666767672222223333332327667233333323333333332220
             3333301111102562975214F200000096123103D951209912310000006733000A
                                               ** *      **  **      ****
 2    3      33333 11111 refrigerator      961215 4-year 001215 000005500   .
             3333323333327667666767672222223333332327667233333323333333332220
             3333301111102562975214F200000096121504D951200012150000005500000A
                                               ** *      **  **      ****
 1    6    0 55555 11111 clothes dryer     991231 5-year 041231 000055566   .
             3333323333326667667267767222223333332327667233333323333333332220
             5555501111103CF48530429520000099123105D951200412310000055566000A
                                 ********    **** *       *  *      *****
 2    6      55555 11111 clothes SHRINKER  990615 6-year 051201 000066699   .
             3333323333326667667254544445223333332327667233333323333333332220
             5555501111103CF485303829EB520099061506D951200512010000066699000A
                                 ********    **** *       *  *      *****
 1    7    0 66666 00000 micro wave        001231 6-year 061231 000066677   .
             3333323333326667627676222222223333332327667233333323333333332220
             666660000000D932F071650000000000123106D951200612310000066677000A
                                               **        ******
 2    8      66666 00000 micro wave        001215 6-year 999999 000066677   .
             3333323333326667627676222222223333332327667233333323333333332220
             666660000000D932F071650000000000121506D951209999990000066677000A
                                               **        ******
 1    8    0 77777 11111 panasonic HDTV 50 020601 5-year 070601 000219999   .
             3333323333327666766662445523323333332327667233333323333333332220
             77777011111001E13FE9308446050002060105D951200706010000219999000A
             ****************************************************************
 2                           *** NoRecord in file#2 ***
 ==================== EOF or StopPrint/StopRead count reached  ==============
 File1Total=9, File2Total=9, StopPrint=8, StopRead=500000
 File1Reads=9, TotalErrs=5, MisCompares=4, File2NoRec=1

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4E3. comparing Fixed length files that may have Inserted records

Notes re uvcmpI1 report on prior page

  1. Note the '*** Record NoMatch ***' for record#8 (panasonic HDTV). Note the 2 files listed on page '4C1'.

  2. Next we will run demo#2 with the filenames reversed on the command line to detect inserted records in file#2.

  3. compare report above ('4E2') to the report on page '4F2' with files reversed Note the '*** Record NoMatch ***' for record#7 (miele ultra dryer), compared to the report on page '4E2' where inserted record was #8 HDTV. See the 2 files listed on page '4C1'.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4F1. comparing Fixed length files that may have Inserted records

uvcmpFAI1 demo#2 - inserted record in file#2


 uvcmpFAI1 dat1/warmas1Ia dat1/warmas1I r64k0l11
 ===============================================
 uvcmpFAI1 - compare Fixed lth Ebcdic Indexed files (Dec11/07)
 090218:112541:uvcmpFAI1: EOF fili01 rds=9 size=576: dat1/warmas1Ia
 090218:112541:uvcmpFAI1: EOF filo01 wrts=9 size=0: tmp1/warmas1Ia
 090218:112541:uvcmpFAI1: EOF fili01 rds=9 size=576: dat1/warmas1I
 090218:112541:uvcmpFAI1: EOF filo01 wrts=9 size=0: tmp2/warmas1I
 uvcmpI1 - compare files & print diffs in hex with "*" markers (Nov28/07)
 uop=q1a0b0c0d0e0f0g0h0k0l0p30r256s8t500000x2y0u1  - option defaults
       a#               - dsplcmnt to exclude field #1
       b#               - length   of exclude field #1
         c#             - dsplcmnt to exclude field #2
         d#             - length   of exclude field #2
           e#           - dsplcmnt to exclude field #3
           f#           - length   of exclude field #3
             g#         - dsplcmnt to exclude field #4
             h#         - length   of exclude field #4
               k#       - dsplcmnt to Indexed key field#1
                 l#     - length   of Indexed key field#1
                  n0    - print column scale only at begin report
                  n1    - print column scale for each record mismatch pair
                   p30  - line# to test overflow at begin record only
                  r256  - record size
                s8      - StopPrintCount, after s# mismatched records
              t500000   - StopReadCount, after s# mismats, else read to EOF
            u1          - stop print at last segment with mismatch
            u2          - inhibit printing all matching segments after 1st
          x#            - hex or char print (x1 char, x2 hex, x0 auto)
        y0              - do NOT translate, file already ASCII, default(y0)
        y1              - translate char lines to ASCII (for EBCDIC files)
 090218:112541:uvcmpFAI1: uvcopy ver=20090212 pf=/home/uvadm/pf/util/uvcmpI1
 uvcopy DISAM ext=dat LNX L64  license=20090212V site=UV_Software
 User OPtion (uop) defaults  = q1a0b0c0d0e0f0g0h0k0l0p30r256s8t500000x2y0u1q1r64k0l11
 -->null to accept or enter/override -->
 ==================== EOF or StopPrint/StopRead count reached  ==============
 File1Total=9, File2Total=9, StopPrint=8, StopRead=500000
 File1Reads=9, TotalErrs=5, MisCompares=4, File2NoRec=1
 090218:112541:uvcmpFAI1: EOF fili01 rds=9 size=585: tmp1/warmas1Ia
 090218:112541:uvcmpFAI1: EOF fili02 rds=9 size=585: tmp2/warmas1I
 090218:112541:uvcmpFAI1: EOF fild03 size=4096: rptcmp
 090218:112541:uvcmpFAI1: EOF filo03 wrts=60 size=3467: rptcmp/warmas1Ia hits=5
 EOJ, Output File written to: rptcmp/warmas1Ia
 default command = null, OR enter: more,print,uvlpr12,edit,null -->

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4F2. comparing Fixed length files that may have Inserted records

uvcmpFAI1 report#2 - inserted record in file#2

 uvcmpFAI1 uvcmpI1 - compare 2 files, print mismatched records, '*' flag diffs
 2009/02/18_11:25:41 uop=q1a0b0c0d0e0f0g0h0k0l0p30r256s8t500000x2y0u1q1r64k0l11
 recsize  reccount   file-size  typ Report=rptcmp/warmas1Ia
 1:   64         9         585  ISF  File1=tmp1/warmas1Ia
 2:   64         9         585  ISF  File2=tmp2/warmas1I
                       1         2         3         4         5         6
 f#rec#byte# 0123456789012345678901234567890123456789012345678901234567890123
 ============================================================================
 1    2    0 22222 00000 dish CRASHER      960101 2-year 980101 000004922   .
             3333323333326676245454452222223333332327667233333323333333332220
             22222000000049380321385200000096010102D951209801010000004922000A
                              *******
 2    2      22222 00000 dish washer       960101 2-year 980101 000004922   .
             3333323333326676276766722222223333332327667233333323333333332220
             22222000000049380713852000000096010102D951209801010000004922000A
                              *******
 1    3    0 33333 11111 refrigerator      961215 4-year 001215 000005500   .
             3333323333327667666767672222223333332327667233333323333333332220
             3333301111102562975214F200000096121504D951200012150000005500000A
                                               ** *      **  **      ****
 2    3      33333 11111 refrigerator      961231 3-year 991231 000006733   .
             3333323333327667666767672222223333332327667233333323333333332220
             3333301111102562975214F200000096123103D951209912310000006733000A
                                               ** *      **  **      ****
 1    6    0 55555 11111 clothes SHRINKER  990615 6-year 051201 000066699   .
             3333323333326667667254544445223333332327667233333323333333332220
             5555501111103CF485303829EB520099061506D951200512010000066699000A
                                 ********    **** *       *  *      *****
 2    6      55555 11111 clothes dryer     991231 5-year 041231 000055566   .
             3333323333326667667267767222223333332327667233333323333333332220
             5555501111103CF48530429520000099123105D951200412310000055566000A
                                 ********    **** *       *  *      *****
 1    7    0 55555 22222 miele ultra dryer 010301 5-year 050301 000099999   .
             3333323333326666627677626776723333332327667233333323333333332220
             555550222220D95C505C421042952001030105D951200503010000099999000A
             ****************************************************************
 2                           *** NoRecord in file#2 ***
 1    8    0 66666 00000 micro wave        001215 6-year 999999 000066677   .
             3333323333326667627676222222223333332327667233333323333333332220
             666660000000D932F071650000000000121506D951209999990000066677000A
                                               **        ******
 2    7      66666 00000 micro wave        001231 6-year 061231 000066677   .
             3333323333326667627676222222223333332327667233333323333333332220
             666660000000D932F071650000000000123106D951200612310000066677000A
                                               **        ******
 ==================== EOF or StopPrint/StopRead count reached  ==============
 File1Total=9, File2Total=9, StopPrint=8, StopRead=500000
 File1Reads=9, TotalErrs=5, MisCompares=4, File2NoRec=1

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4G1. comparing Fixed length files that may have Inserted records

uvcmpFAI2 demo#3 - inserts in both files on 1 report


 uvcmpFAI2 dat1/warmas1Ia dat1/warmas1I r64k0l11
 ===============================================
 uvcmpFAI2 - compare Fixed lth ASCII Indexed files (May20/08)
 110311:105326:uvcmpFAI2: EOF fili01 rds=9 size=576: dat1/warmas1Ia
 110311:105326:uvcmpFAI2: EOF filo01 wrts=9 size=0: tmp1/warmas1Ia
 110311:105326:uvcmpFAI2: EOF fili01 rds=9 size=576: dat1/warmas1I
 110311:105326:uvcmpFAI2: EOF filo01 wrts=9 size=0: tmp2/warmas1I
 unmatI2 - select unmatched records in file#2 for following uvcmpI2
 uop=q1r256k0l0 - option defaults
       r256     - record size
           k0   - dsplcmnt to Indexed key
             l0 - length of Indexed key (must specify)
 110311:105326:uvcmpFAI2: uvcopy ver=20110307 pf=/home/uvadm/pf/util/unmatI2
 uvcopy DISAM ext=.dat LNX L64  license=110307_00V_930601 site=UV_Software
 110311:105326:uvcmpFAI2: EOF fili01 rds=9 size=585: tmp1/warmas1Ia
 110311:105326:uvcmpFAI2: EOF fili02 rds=9 size=585: tmp2/warmas1I
 110311:105326:uvcmpFAI2: EOF filo03 wrts=1 size=0: tmp2/warmas1I.unm

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

 uvcmpI2 - compare files allowing for inserts/deletes
 uop=q1a0b0c0d0e0f0g0h0k0l0p30r256s8t500000x2y1u3  - option defaults
       a#               - dsplcmnt to exclude field #1
       b#               - length   of exclude field #1
         c#             - dsplcmnt to exclude field #2
         d#             - length   of exclude field #2
           e#           - dsplcmnt to exclude field #3
           f#           - length   of exclude field #3
             g#         - dsplcmnt to exclude field #4
             h#         - length   of exclude field #4
               i1       - ignore inserts/deletes, show mismatched pairs
               i2       - ignore mismatched pairs, show inserts/deletes
                 k#     - dsplcmnt to Indexed key field#1
                   l#   - length   of Indexed key field#1
                    n0  - print column scale only at begin report
                    n1  - print column scale for each record mismatch pair
                  p30   - line# to test overflow at begin record only
                 r256   - record size
               s8       - StopPrintCount, after s# mismatched records
             t500000    - StopReadCount, after s# mismats, else read to EOF
           u1           - stop print at last segment with mismatch
           u2           - inhibit printing all matching segments after 1st
         x#             - hex or char print (x1 char, x2 hex, x0 auto)
       y0               - do NOT translate, file already ASCII, default(y0)
       y1               - translate char lines to ASCII (for EBCDIC files)
 110311:105326:uvcmpFAI2: uvcopy ver=20110307 pf=/home/uvadm/pf/util/uvcmpI2
 uvcopy DISAM ext=.dat LNX L64  license=110307_00V_930601 site=UV_Software
 User OPtion (uop) defaults  = q1a0b0c0d0e0f0g0h0k0l0p30r256s8t500000x2y0u3q1r64k0l11
 -->null to accept or enter/override -->
 ==================== EOF or StopPrint/StopRead count reached  ==============
 File1Total=9, File2Total=9, StopPrint=8, StopRead=500000
 File1Reads=9, TotalErrs=6, MisCompares=4, File1NoRec=1, File2NoRec=1
 110311:105328:uvcmpFAI2: EOF fili01 rds=9 size=585: tmp1/warmas1Ia
 110311:105328:uvcmpFAI2: EOF fili02 rds=10 size=585: tmp2/warmas1I
 110311:105328:uvcmpFAI2: EOF fili03 rds=1 size=65: tmp2/warmas1I.unm
 110311:105328:uvcmpFAI2: EOF fild04 size=4096: rptcmp
 110311:105328:uvcmpFAI2: EOF filo04 wrts=27 size=1561: rptcmp/warmas1Ia hits=6
 EOJ, Output File written to: rptcmp/warmas1Ia
 default command = null, OR enter: more,print,uvlpr12,edit,null -->

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4G2. comparing Fixed length files that may have Inserted records

uvcmpFAI2 report#2 - inserts both files, 1 report

 uvcmpFAI2 uvcmpI2 - compare 2 files, print mismatched records, '*' flag diffs
 2009/02/18_11:31:55 uop=q1a0b0c0d0e0f0g0h0k0l0p30r256s8t500000x2y0u3q1r64k0l11
 recsize  reccount   file-size  typ Report=rptcmp/warmas1Ia
 1:   64         9         585  ISF  File1=tmp1/warmas1Ia
 2:   64         9         585  ISF  File2=tmp2/warmas1I
                       1         2         3         4         5         6
 f#rec#byte# 0123456789012345678901234567890123456789012345678901234567890123
 ============================================================================
 1    2    0 22222 00000 dish CRASHER      960101 2-year 980101 000004922   .
             3333323333326676245454452222223333332327667233333323333333332220
             22222000000049380321385200000096010102D951209801010000004922000A
                              *******
 2    2      22222 00000 dish washer       960101 2-year 980101 000004922   .
             3333323333326676276766722222223333332327667233333323333333332220
             22222000000049380713852000000096010102D951209801010000004922000A
                              *******
 1    3    0 33333 11111 refrigerator      961215 4-year 001215 000005500   .
             3333323333327667666767672222223333332327667233333323333333332220
             3333301111102562975214F200000096121504D951200012150000005500000A
                                               ** *      **  **      ****
 2    3      33333 11111 refrigerator      961231 3-year 991231 000006733   .
             3333323333327667666767672222223333332327667233333323333333332220
             3333301111102562975214F200000096123103D951209912310000006733000A
                                               ** *      **  **      ****
 1    6    0 55555 11111 clothes SHRINKER  990615 6-year 051201 000066699   .
             3333323333326667667254544445223333332327667233333323333333332220
             5555501111103CF485303829EB520099061506D951200512010000066699000A
                                 ********    **** *       *  *      *****
 2    6      55555 11111 clothes dryer     991231 5-year 041231 000055566   .
             3333323333326667667267767222223333332327667233333323333333332220
             5555501111103CF48530429520000099123105D951200412310000055566000A
                                 ********    **** *       *  *      *****

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

 1    7    0 55555 22222 miele ultra dryer 010301 5-year 050301 000099999   .
             3333323333326666627677626776723333332327667233333323333333332220
             555550222220D95C505C421042952001030105D951200503010000099999000A
             ****************************************************************
 2                           *** NoRecord in file#2 ***
 1    8    0 66666 00000 micro wave        001215 6-year 999999 000066677   .
             3333323333326667627676222222223333332327667233333323333333332220
             666660000000D932F071650000000000121506D951209999990000066677000A
                                               **        ******
 2    7      66666 00000 micro wave        001231 6-year 061231 000066677   .
             3333323333326667627676222222223333332327667233333323333333332220
             666660000000D932F071650000000000123106D951200612310000066677000A
                                               **        ******
 1                           *** NoRecord in file#1 ***
 2    8+   0 77777 11111 panasonic HDTV 50 020601 5-year 070601 000219999   .
             3333323333327666766662445523323333332327667233333323333333332220
             77777011111001E13FE9308446050002060105D951200706010000219999000A
             ****************************************************************
 ==================== EOF or StopPrint/StopRead count reached  ==============
 File1Total=9, File2Total=9, StopPrint=8, StopRead=500000
 File1Reads=9, TotalErrs=6, MisCompares=4, File1NoRec=1, File2NoRec=1

Options i1/i2 ignore inserts/deletes OR mismatched pairs

option i1
  • ignore inserts/deletes, show only mismatched pairs
option i2
  • ignore mismatched pairs, show only inserts/deletes

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4H1. comparing RDW files that may have Inserted records

compare RDW files allowing inserts/deletes

In March 2011, 2 new scripts were added to make it easier to compare variable length RDW files allowing for inserts & deletes.

uvcmpRDWAI2
  • compare RDW ASCII files allowing inserts/deletes
uvcmpRDWEI2
  • compare RDW EBCDIC files allowing inserts/deletes

You can demo these scripts using the following supplied test files. These can be copied from $UV/dat1 to your homedir/subdir as previously shown on page '4B1'.


 cp $UV/dat1/vendormas3* dat1/  <-- copy supplied test files to your homedir
 =============================
vendormas3
  • RDW ASCII compare file #1
vendormas32
  • RDW ASCII compare file #2
vendormas3e
  • RDW EBCDIC compare file #1
vendormas3e2
  • RDW EBCDIC compare file #2

vendormas2 TEXT file used to create vendormas3 RDW file

      DELL10 M  Dell Inc.
      HP0010 M  Hewlett Packard
      IBM010 M  International Business Machines
      MFC010 M  Micro Focus COBOL
      MS0010 M  Microsoft Corp.
      REDH10 M  Red Hat Linux
      SUN010 M  Sun Microsystems Ltd
      UVSI10 M  UV Software Inc.

vendormas22 TEXT file used to create vendormas32 RDW file

      DELL10 M  Dell Inc.
      HP0010 M  Hewlett Packrat
      IBM010 M  International Business Machines
      MFC010 M  Micro Focus COBOL
      MS0010 M  Microhard Corp.
      REDH10 M  Red Hat Linux Incorporated
      UVSI10 M  UV Software Inc.

uvcp to convert TEXT files to RDW files


 uvcp "fili1=dat1/vendormas2,typ=LST,rcs=64,filo1=dat1/vendormas3,typ=RDW"
 =========================================================================

 uvcp "fili1=dat1/vendormas2,typ=LST,rcs=64,filo1=dat1/vendormas3e,typ=RDW,tre=0(64)"
 ================================================================================

UV Software has already created the test files as shown above. We list the 'uvcp' commands here in case you are interested in how to create RDW files from TEXT files.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4H2. comparing RDW files that may have Inserted records

execute uvcmpRDWAI2 to compare 2 RDW files


 uvcmpRDWAI2 dat1/vendormas3 dat1/vendormas32 r64k0l6
 ====================================================
 - compare 2 RDW files, report created in rptcmp/vendormas3 (listed below)
 - see options 'r64k0l6' explained on following page -->
 uvcmpRDWAI2 uvcmpI2 - compare 2 files, print mismatched records, '*' flag diffs
 2011/03/12_06:40:48 uop=q1a0b0c0d0e0f0g0h0k0l0p30r256s8t500000x2y0u3q1r64k0l6
 recsize  reccount   file-size  typ Report=rptcmp/vendormas3
 1:   64         8         520  ISF  File1=tmp1/vendormas3
 2:   64         7         455  ISF  File2=tmp2/vendormas32
                       1         2         3         4         5         6
 f#rec#byte# 0123456789012345678901234567890123456789012345678901234567890123
 ============================================================================
 1    2    0 HP0010 M  Hewlett Packard
             4533332422467667725666676222222222222222222222222222222222222222
             8000100D00857C5440013B124000000000000000000000000000000000000000
                                   ***
 2    2      HP0010 M  Hewlett Packrat
             4533332422467667725666767222222222222222222222222222222222222222
             8000100D00857C5440013B214000000000000000000000000000000000000000
                                   ***
 1    5    0 MS0010 M  Microsoft Corp.
             4533332422466767667246772222222222222222222222222222222222222222
             D300100D00D932F3F6403F20E000000000000000000000000000000000000000
                            ****
 2    5      MS0010 M  Microhard Corp.
             4533332422466766676246772222222222222222222222222222222222222222
             D300100D00D932F812403F20E000000000000000000000000000000000000000
                            ****
 1    6    0 REDH10 M  Red Hat Linux
             5444332422566246724667722222222222222222222222222222222222222222
             2548100D0025408140C9E5800000000000000000000000000000000000000000
                                     ************
 2    6      REDH10 M  Red Hat Linux Incorporated
             5444332422566246724667724666776767662222222222222222222222222222
             2548100D0025408140C9E5809E3F20F214540000000000000000000000000000
                                     ************
 1    7    0 SUN010 M  Sun Microsystems Ltd
             5543332422576246676777766724762222222222222222222222222222222222
             35E0100D0035E0D932F39345D30C440000000000000000000000000000000000
             ****************************************************************
 2                           *** NoRecord in file#2 ***
 ==================== EOF or StopPrint/StopRead count reached  ==============
 File1Total=8, File2Total=7, StopPrint=8, StopRead=500000
 File1Reads=8, TotalErrs=4, MisCompares=3, File1NoRec=0, File2NoRec=1

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4H3. comparing RDW files that may have Inserted records

Options for uvcmpRDWAI2


 uvcmpRDWAI2 dat1/vendormas3 dat1/vendormas32 r64k0l6
 ====================================================
 - compare 2 RDW files, note options explained below:
r64
  • record length, choose a record length larger than largest record in file
  • pick a multiple of 64, since 'uvcmp' dumps records in 64 byte segments
k0
  • Key start byte displacement
  • vendor master records (see previous page) have vendor# in 1st 6 bytes
l6
  • Length of key (vendor# is 6 bytes long)

uvcmpRDWEI2 - compare EBCDIC RDW files


 uvcmpRDWEI2 dat1/vendormas3e dat1/vendormas3e2 r64k0l6
 ======================================================
 - compare 2 EBCDIC RDW files

 vi rptcmp/vendormas3e   <-- inspect report (created in subdir rptcmp/...)
 =====================

Since the report is similar to the ASCII version (on previous page), we will show only the 1st pair of mismatched records. Note the vertical hex now shows the EBCDIC zones & digits, but character line is translated to ASCII, so we can read it on the unix/linux systems.

 1    2    0 HP0010 M  Hewlett Packard
             CDFFFF4D44C8A98AA4D889898422222222222222222222222222222222222222
             8700100400856353307132194000000000000000000000000000000000000000
                                   ****
 2    2      HP0010 M  Hewlett Packrat
             CDFFFF4D44C8A98AA4D88998A222222222222222222222222222222222222222
             8700100400856353307132913000000000000000000000000000000000000000
                                   ****

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4H4. comparing RDW files that may have Inserted records

unique key required

Records must have some unique field to be used as a key to load the Indexed files required to allow for inserts & deletes. If the files did not have any unique key, we could modify the script to create a sequential# key as files are converted to Indexed files.


 vi $UV/sf/util/uvcmpRDWAI2   <-- inspect script to compare 2 RDW files
 ==========================

This script is similar to 'uvcmpFAI2' listed on page '4X2'. Note line 70 loads the 1st file into an Indexed files as follows (line 75 loads the 2nd file).


 uvcopy loadISF1,fili1=$df1,typ=RDWz8,filo1=tmp1/$f1,uop=q0i7$optns
 #=================================================================
 # - load 1st seqntl file to Indexed file for uvcmpI2 to match by key

We could copy/modify this job to insert a 10 byte key (sequential#) as the file is being load to the Indexed file. We might name the new job 'loadISF2'.


 #1. cd $UV   <-- change to /home/uvadm

 #2. cp pf/util/loadISF1 pf/util/loadISF2 - copy/rename for alternate version
     ====================================

 #3. vi pf/util/loadISF2 - edit alternate job to insert sequence# key
     ===================

We would also copy/rename uvcmpRDWAI2 & modify to call the alternate job loadISF2 on lines 70 & 75.


 #4. cp sf/util/uvcmpRDWAI2 sf/util/uvcmpRDWAI3 - copy/rename for alt version
     ==========================================

 #5. vi pf/util/uvcmpRDWAI2 - edit alt job to call loadISF3 (vs loadISF2)
     ======================

The alternate jobs/scripts have not yet been created as of March 2011, but let us know if you need them.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4X0. comparing Fixed length files that may have Inserted records

uvcopy jobs used in Part 4

uvcmpI1
  • data file comparison job for Indexed files
  • allowing for Inserted records (in file#1)
  • can run with files reversed to detect inserted records in 2nd file
uvcmpI2
  • same, but allows for inserts/deletes in both files, on 1 report
  • prior job 'unmatI2' writes unmatched records in file1 to tmp2/...
    by reading file2 sequentially & reading file1 randomly by key
  • uvcmpI2 then reads 3 files data1, data2,& unmatched in data1
  • option i1 to ignore inserts/deletes, show only mismatched pairs
  • option i2 to ignore mismatched pairs, show only inserts/deletes

scripts used in Part 4


4X1. uvcmpFAI1 - compare Fixed ASCII files that may have INSERTs & DELETEs
____ uvcmpFEI1
  • same as uvcmpFAI1, but for EBCDIC files (vs ASCII)

4X2. uvcmpFAI2 - compare Fixed ASCII files that may have INSERTED records
- creates 1 report with inserts/deletes in both files
____ uvcmpFEI2
  • same as uvcmpFAI2, but for EBCDIC files (vs ASCII)
____ uvcmpRDWAI2
  • compare RDW ASCII files allowing inserts/deletes
  • not listed (but very similar to uvcmpFAI1)
____ uvcmpRDWEI2
  • compare RDW EBCDIC files allowing inserts/deletes

Two scripts will be listed below since they are short. The uvcopy jobs will not be listed because they are much longer, but you can view or print as follows:


 #1. cd /home/uvadm
     ==============

 #2. vi pf/util/uvcmpI1
     ==================

 #3. uvlp12 pf/util/uvcmpI1
     ======================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4X1. comparing Fixed length files that may have Inserted records

uvcmpFAI1 - compare files that may have INSERTs

 #!/bin/ksh
 # uvcmpFAI1 - script to compare 2 Fixed length ASCII Indexed files
 #           - by Owen Townsend, December 2007, May 2008
 #Feb18/09 - rename uvcmp2->5, uvcmp3->6, RDW->uvcmp2, IDX->uvcmp3
 #
 #            ** scripts allowing INSERTs/DELETEs BUT on 2 reports **
 #
 #*uvcmpFAI1 - FixLth ASCII w inserted recs by loading Index typ=ISF
 # uvcmpFEI1 - FixLth EBCDIC w inserted recs by loading Index typ=ISF
 # uvcmpFAX1 - FixLth ASCII w inserted recs by loading Index typ=IDXf1
 # uvcmpFEX1 - FixLth EBCDIC w inserted recs by loading Index typ=IDXf1
 #    - sort/load Indexed files, read 1st seqntl by key, read 2nd randomly by key
 #    - need to repeat with files reversed to ensure all mismatches detected
 #    - above scritps now obsoleted by following scripts
 #
 #         ** scripts allowing for INSERTs/DELETEs on 1 report **
 #
 # uvcmpFAI2 - FixLth ASCII w inserted recs by loading Index typ=ISF
 # uvcmpFEI2 - FixLth EBCDIC w inserted recs by loading Index typ=ISF
 #    - pre-compare step unmatI2 writes unmatched records in file1 to tmp2/...
 #      by reading file2 sequentially & reading file1 randomly by key
 #    - uvcmpI2 then reads 3 files data1, data2,& data2 recs w/o matches in data1
 #
 #            ** scripts NOT allowing Inserts/deletes **
 #
 # uvcmpFA1 - compare Fixed length ASCII data files
 # uvcmpFE1 - compare Fixed length EBCDIC data files
 # uvcmpRDW1 - compare RDW variable length files - ASCII or EBCDIC (option y1)
 # uvcmpIDX81 - compare Micro Focus COBOL IDXFORMAT3/8 Variable length files
 #    - 1 pass only, no need to load Indexed files, or rerun files reversed
 #      (since no inserts/deletes are allowed for these scripts)
 #    - the 2 datafiles may have same names if in different subdirs
 #    - run this script from a working directory with subdirs holding datafiles
 #    - output report will be created in subdir: rptcmp/filename1
 #      (will make subdir ./rptcmp if not already existing)
 #

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

 export JOBID2=uvcmpFAI1  # so uvcopy uvcmp1 can show script name on report
 echo "uvcmpFAI1 - compare Fixed lth Ebcdic Indexed files (Dec11/07)"
 if [[ -f "$1" && -f "$2" && -n "$3" ]]; then :
    else echo "usage:   uvcmpFAI1 file1 file2 Recsize/Keydsp/Keylth/Excludes"
         echo "         ===================================================="
         echo "example1: uvcmpFAI1 dat1/warmas1I dat1/warmas1Ia r64k0l6"
         echo "          =============================================="
         echo "compare 2 Fixed length ASCII Indexed data files"
         echo " - arg1 & arg2 are filenames that must be present"
         echo " - arg3 options alphas+numerics, example r64k0l6"
         echo " - r64=Recsize, k0=Keydsplcmnt, l6=keyLength"
         echo " "
         echo "ex2: uvcmpFAI1 dat1/warmas1I dat1/warmas1Ia r64k0l6a34b2c44d6"
         echo "     ========================================================"
         echo " - ex#2 demos exclude field excludes (displacements & lengths)"
         echo " - a34b2 excludes 34-35, c44d6 excludes 44-49"
         echo " - see all options described in /home/uvadm/pf/util/uvcmp1"
         echo " "
         echo "ERROR - arg1 & arg2 must be files, arg3(options) must be spcfd"
         exit 99; fi
 #
 df1="$1"; df2="$2"; optns="$3";
 f1=$(basename $df1)
 f2=$(basename $df2)
 #
 if [[ ! -d tmp1 ]]; then mkdir tmp1; fi
 if [[ ! -d tmp2 ]]; then mkdir tmp2; fi
 if [[ ! -d rptcmp ]]; then mkdir rptcmp; fi
 #
 uvcopy loadISF1,fili1=$df1,filo1=tmp1/$f1,uop=q0i7$optns
 #=======================================================
 if (($?)); then exit 91; fi   # exit if load fails
 # - load 1st seqntl file to Indexed file for uvcmpI1 to match by key
 #
 uvcopy loadISF1,fili1=$df2,filo1=tmp2/$f2,uop=q0i7$optns
 #=======================================================
 if (($?)); then exit 92; fi   # exit if load fails
 # - load 2nd seqntl file to Indexed file for uvcmpI1 to match by key
 #
 uvcopy uvcmpI1,fili1=tmp1/$f1,fili2=tmp2/$f2,fild3=rptcmp,uop=q1$optns
 #=====================================================================
 # - uvcmpI1 creates comparison report in rptcmp/filename1
 #   and prompts for disposition, may enter vi, more, uvlp12, etc
 exit 0
 #

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4X2. comparing Fixed length files that may have Inserted records

'uvcmpFAI2' - allow inserts in both files on 1 report

 #!/bin/ksh
 # uvcmpFAI2 - script to compare 2 Fixed length ASCII Indexed files
 #           - by Owen Townsend, May 2008
 #Mar11/11 - option i1 ignore inserts/deletes, show only mismatched pairs
 #           option i2 ignore mismatched pairs, show only inserts/deletes
 #
 #         ** scripts allowing for INSERTs/DELETEs on 1 report **
 #
 #*uvcmpFAI2 - FixLth ASCII w inserted recs by loading Index typ=ISF
 # uvcmpFEI2 - FixLth EBCDIC w inserted recs by loading Index typ=ISF
 #    - pre-compare step unmatI2 writes unmatched records in file1 to tmp2/...
 #      by reading file2 sequentially & reading file1 randomly by key
 #    - uvcmpI2 then reads 3 files data1, data2,& data2 recs w/o matches in data1
 #
 #            ** scripts allowing INSERTs/DELETEs BUT on 2 reports **
 #
 # uvcmpFAI1 - FixLth ASCII w inserted recs by loading Index typ=ISF
 # uvcmpFEI1 - FixLth EBCDIC w inserted recs by loading Index typ=ISF
 # uvcmpFAX1 - FixLth ASCII w inserted recs by loading Index typ=IDXf1
 # uvcmpFEX1 - FixLth EBCDIC w inserted recs by loading Index typ=IDXf1
 #    - sort/load Indexed files, read 1st seqntl by key, read 2nd randomly by key
 #    - need to repeat with files reversed to ensure all mismatches detected
 #    - above scritps now obsoleted by uvcmpFAI2 & uvcmpFEI2
 #
 #            ** scripts NOT allowing Inserts/deletes **
 #
 # uvcmpFA1 - compare Fixed length ASCII data files
 # uvcmpFE1 - compare Fixed length EBCDIC data files
 # uvcmpRDW1 - compare RDW variable length files - ASCII or EBCDIC (option y1)
 # uvcmpIDX81 - compare Micro Focus COBOL IDXFORMAT3/8 Variable length files
 #    - 1 pass only, no need to load Indexed files, or rerun files reversed
 #    - use these scripts for sequential files without inserts & deletes
 #    - simpler, faster, 1 step process (no need for Indexed files)
 #
 # - the 2 datafiles may have same names if in different subdirs
 # - run this script from a working directory with subdirs holding datafiles
 # - output report will be created in subdir: rptcmp/filename1
 #   (will make subdir ./rptcmp if not already existing)
 #

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

 export JOBID2=uvcmpFAI2  # so uvcopy uvcmpI2 can show script name on report
 echo "uvcmpFAI2 - compare Fixed lth ASCII Indexed files (May20/08)"
 if [[ -f "$1" && -f "$2" && -n "$3" ]]; then :
    else echo "usage:   uvcmpFAI2 file1 file2 Recsize/Keydsp/Keylth/Excludes"
         echo "         ===================================================="
         echo "example1: uvcmpFAI2 dat1/warmas1I dat1/warmas1Ia r64k0l6"
         echo "          =============================================="
         echo "compare 2 Fixed length ASCII Indexed data files"
         echo " - arg1 & arg2 are filenames that must be present"
         echo " - arg3 options alphas+numerics, example r64k0l6"
         echo " - r64=Recsize, k0=Keydsplcmnt, l6=keyLength"
         echo " "
         echo "ex2: uvcmpFAI2 dat1/warmas1I dat1/warmas1Ia r64k0l6a34b2c44d6"
         echo "     =========================================================="
         echo " - ex#2 demos exclude field excludes (displacements & lengths)"
         echo " - a34b2 excludes 34-35, c44d6 excludes 44-49"
         echo " - see all options described in /home/uvadm/pf/util/uvcmpI2"
         echo " "
         echo "ERROR - arg1 & arg2 must be files, arg3(options) must be spcfd"
         exit 99; fi
 #
 df1="$1"; df2="$2"; optns="$3";
 f1=$(basename $df1)
 f2=$(basename $df2)
 #
 if [[ ! -d tmp1 ]]; then mkdir tmp1; fi
 if [[ ! -d tmp2 ]]; then mkdir tmp2; fi
 if [[ ! -d rptcmp ]]; then mkdir rptcmp; fi
 #
 uvcopy loadISF1,fili1=$df1,filo1=tmp1/$f1,uop=q0i7$optns
 #=======================================================
 if (($?)); then exit 91; fi   # exit if load fails
 # - load 1st seqntl file to Indexed file for uvcmpI2 to match by key
 #
 uvcopy loadISF1,fili1=$df2,filo1=tmp2/$f2,uop=q0i7$optns
 #=======================================================
 if (($?)); then exit 92; fi   # exit if load fails
 # - load 2nd seqntl file to Indexed file for uvcmpI2 to match by key
 #
 uvcopy unmatI2,fili1=tmp1/$f1,fili2=tmp2/$f2,filo3=tmp2/$f2.unm,uop=q0$optns
 #===========================================================================
 # pass #1 - detect unmatched records in file2 & write to tmp2/filename2.unm
 #         - by reading file2 seqntly & file1 randomly by key
 #
 uvcopy uvcmpI2,fili1=tmp1/$f1,fili2=tmp2/$f2,fili3=tmp2/$f2.unm\
 ,fild4=rptcmp,uop=q1$optns
 #===============================================================
 # pass #2 - create the report, reads file1 seqntly, random read file2 by key,
 #         - detects matched keys with mismatched data & unmatched recs in file2
 #         - unmatched recs in file1 are merged in from unmatI2 output above
 #
 # report created: rptcmp/filename1
 # - prompts for disposition, may enter vi, more, uvlp12, etc
 exit 0
 #

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_5 uvcmp5 - compare all files in 2 directories

compare all files in 2 directories - Contents


5A1. Comparing 'entire directories' of data files - Overview
- 'uvcmp5' uses a control file to supply the record sizes
  (vs manual entry for uvcmp1 used in Parts 1,2,3,4)
- script 'uvcmp5all' makes it easier to run
- first run 'unixcmpsplit' to isolate non-matching files
- uses unix 'cmp' to eliminate matching files before uvcmp5 is run
  (since 'cmp' is much faster than uvcmp5)

5B1. unixcmpall - script to run unix 'cmp' for all pairs of files in 2 subdirs
- creates a report used by unixcmpsplit to remove matches

5C1. unixcmpsplit - move non-matching files out to 2 compare directories
- saves running uvcmp5 on matching files (unix 'cmp' much faster).

5D1. uvcmp5 - compare all files in 2 directories & create comparison reports
- uses a control file to supply record sizes
  (recsize was entered manually for uvcmp1)

5E1. preparation to demo uvcmp5 - compare all files in 2 subdirs

5F1. Execute test/demo uvcmp5 - compare all files in 2 subdirs

5G1. vtocr1 - VTOC reports may assist your directory comparisons
- could run before & after batch updates

5X0. Summary of scripts & uvcopy jobs used in this Part
- listings of scripts (& uvcopy jobs only if short)
- unixcmpsplit, unixcmpall, uvcmp5all,

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5A1. uvcmp5 - compare all files in 2 directories

compare all files in 2 directories - Overview

'uvcmp5' can be used to compare all Fixed Length Sequential files in 2 directories & generate difference reports in subdir rptcmp.

uvcmp5 is driven by the 'uvcmp5all' script & a pre-edited control file to supply the record sizes, so we can perform the comparisons with no interventions. (vs manual entry for uvcmp1 used in Parts 1,2,3,4)

The control file may be prepared with the vi editor & loaded into an ISAM file with the supplied utility job (uvcopy loadctlI).

You might 1st run script 'unixcmpsplit' if you have a lot of large files. unixcmpsplit uses unix 'cmp' to eliminate matching files before uvcmp5 is run (since 'cmp' is much faster than uvcmp5).

The UNIX 'cmp' utility is inadequate for reporting the differences between the files, BUT is very QUICK to determine whether the files are different.

We will also demonstrate the 'unixcmpall' script (also based on UNIX cmp), which simply creates a brief report indicating matching/mismatching files.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5A2. uvcmp5 - compare all files in 2 directories

Directory 'cmpdata' & subdirs for comparisons

I recommend you setup a 'cmpdata' within your homedir & then use the the 'cmpdatadirs' script to setup subdirs required for uvcmp1, uvcmp5,& uvcmp6 (see procedures following the directory illustrations).

 /home/userxx
 :-----cmpdata          <-- superdir for DATA comparisons
 :     :-----cpy1         - copybooks
 :     :-----cpy2         - copybooks for modified layouts (as in Y2K)
 :     :-----dat0         - EBCDIC files transferred from mainframe
                          - may need to convert variable to fixed in dat1
                          - might need to copy/rename/reformat dat0/... to dat1
 :     :-----dat1         - subdir to hold 1st file(s) to be compared
                          - can put demo files here since filenames are different
 :     :-----dat2         - subdir to hold 2nd file(s) to be compared
                            (need separate subdir if 2nd filenames same as 1st)
 :     :-----map1         - copybook layouts
 :     :-----map1I        - copybook layouts loaded to Indexed files
 :     :-----map2         - modified layouts (as in Y2K) OR same as map1
 :     :-----pfc1         - uvcmp6 comparison jobs generated from layouts
 :     :-----pfc2         - comparison jobs copied here for any mods & execution
 :     :-----rptcmp       - reports generated here
 :     :-----tmp          - for misc outputs (keep working dir free of files)

The cpy1,2,map1,1I,map2,pfc1,2 subdirs are needed to generate 'uvcmp6' jobs that allow different file layouts for the files to be compared (as in Y2K).

preparation for uvcmp1 test/demo


 #1. Login ---> /home/userxx  <-- Login or change to your home directory

 #2. mkdir cmpdata            <-- make superdir for subdirs shown above
     =============

 #3. cd cmpdata               <-- change into the comparison superdir
     ==========

 #4. cmpdatadirs              <-- run script to make subdirs shown above
     ===========

 #5a. cp /home/uvadm/dat1/warmas1 dat1
      ================================
      - copy supplied test files to subdirs in your homedir

 #5b. cp /home/uvadm/dat1/warmas1a dat1
      =================================
      - can store 2nd demo file in same subdir as 1st since filenames different
        (store in dat2/... if filenames different)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5B1. compare all files in 2 directories

unixcmpall - test/demo

 1a. cd $HOME        - change to your home directory
 1b. mkdir cmpdata   - make a sub-directory for the test/demo
 1c. cd cmpdata      - change into it
 2a. mkdir dat1      - make subsubdir for original data files
 2b. mkdir dat2      - make subsubdir for modified data files
 2c. mkdir rptcmp    - make subsubdir for mismatch reports

3a. cp /home/uvadm/dat1/warmas* dat1 - copy some test files to dat1 3b. cp /home/uvadm/dat1/warmas* dat2 - & dat2, then modify some in dat2

 4a. vi dat2/*     - create some differences to illustrate mismatch reports
                   - for the demo report below, we modified as follows:
                     warmas3 rec#5: change 'dish washer' to 'disk CRASHER'
                     warmas6 rec#6: change 'clothes dryer' to 'clothes SHRINKER'
     Note - warmas9 has packed fields & we will change packed date field in
            record #2, byte 60-63 from x'0990101C' to x'0990909C'
          - You can't do this with 'vi', so use 'uvhd' as follows:
 4b. uvhd dat2/warmas9 r64u   - call uvhd to update warmas9 (rcsz 64)
                              - goto rec#2 & enter following command:
     u 60(4),x'0990909C'      - change bytes 60-63 to x'0990909C'

 5a. unixcmpall dat1 dat2       <-- run unixcmpall script (using UNIX cmp)
     ====================         - report will be displayed on screen (stdout)

 5b. unixcmpall dat1 dat2 >rptcmp/cmpall.rpt  <-- to capture report in a file
     =======================================

unixcmpall - demo report

file# 1 cmp dat1/warmas1 dat2/warmas1 file# 2 cmp dat1/warmas1x dat2/warmas1x file# 3 cmp dat1/warmas3 dat2/warmas3 dat1/warmas3 dat2/warmas3 differ: char 274, line 5 file# 4 cmp dat1/warmas6 dat2/warmas6 dat1/warmas6 dat2/warmas6 differ: char 341, line 6 file# 5 cmp dat1/warmas6x dat2/warmas6x file# 6 cmp dat1/warmas9 dat2/warmas9 dat1/warmas9 dat2/warmas9 differ: char 127, line 1 >> 3 mismatches of 6 total files compared in dat1 & dat2

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5C1. uvcmp5 - compare all files in 2 directories

unixcmpsplit - eliminate matching files prior to uvcmp5all

We will now demo the 'unixcmpsplit' script (based on UNIX cmp) to move the non-matching files from 2 directories out to 2 other directories, prior to running the 'uvcmp5all' script (based on uvcopy uvcmp5, see next section).

The demo preparation is the similar to unixcmpall (shown on the previous page). We only need to make 2 more subdirs to receive the non-matching files & run unixcmpsplit vs unixcmpall. Here are the additions & replacement OP. Instrns:

 2e. mkdir dat1X               - make subdirs to receive non-matching files
 2f. mkdir dat2X                 (selected out by unixcmpsplit)

 5a. unixcmpsplit dat1 dat2 dat1X dat2X  <-- run unixcmpsplit script
     ==================================    - report displayed on screen

Note - To rerun the demo, we must move the non-matching files back

5b. mv dat1X/* dat1 5c. mv dat2X/* dat2


 5d. unixcmpsplit dat1 dat2 dat1X dat2X >rptcmp/unixcmpsplit.rpt
     ===========================================================
                                     - capture report in a file

 5e. uvlp12 rptcmp/unixcmpsplit.rpt  - print the unixcmpsplit report
     ==============================

unixcmpsplit - demo report

file# 1 cmp dat1/warmas1 dat2/warmas1 file# 2 cmp dat1/warmas1x dat2/warmas1x file# 3 cmp dat1/warmas3 dat2/warmas3 dat1/warmas3 dat2/warmas3 differ: char 274, line 5

file# 4 cmp dat1/warmas6 dat2/warmas6 dat1/warmas6 dat2/warmas6 differ: char 341, line 6

file# 5 cmp dat1/warmas6x dat2/warmas6x file# 6 cmp dat1/warmas9 dat2/warmas9 dat1/warmas9 dat2/warmas9 differ: char 127, line 1

>> 3 matching of 6 total files compared in dat1 & dat2 >> 3 non-matching files have been moved to directories dat1X & dat2X

See the 'unixcmpsplit' script listed at the end of this Part --->

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5D1. uvcmp5 - compare all files in 2 directories

uvcmp5 - compare 2 directories, creating reports in a 3rd

'uvcmp5' is similar to uvcmp1, but includes 2 significant enhancements.

  1. You do not have to enter the the record size for sequential files. This problem is solved by using a control file (ctlcmp1) containing record sizes & file-types for all files.

  2. The 'uvcmp5all' script allows you to compare all files in 2 directories (with 1 command), outputting mismatch reports to a 3rd directory, which can then all be printed with 1 command (uvlpd12 dirname).

Uvcmp2 does require the 1-time preparation of the control file, but this is well worth the time if you want to compare dozens of files multiple times.

The control file may be prepared with the editor & loaded into an ISAM file with the supplied utility job (uvcopy loadctlI). You would hard-code it's pathname within the uvcmp5 job.

For casual 1-shot use, use the uvcmp1 job which requires no preparation (& you can enter all required parameters on the command line).

contents of this section

Note that the reports generated are the same as for uvcmp1. Several demo reports were illustrated for uvcmp1 in Part_1 and they will not be repeated here. Please see the following pages for:

  1. Sample control file (~uvadm/ctl/ctlcmp1) listed on the next page --->

  2. OPERATING INSTRUCTIONS & CONSOLE OUTPUT for uvcmp5all demo run, comparing all files in 2 directories & creating mismatch reports in a 3rd, See Op. Instrns. & console output 3 pages ahead --->

  3. VTOC reports for the 2 directories to be compared. - file statistics showing record counts, record sizes, file sizes, indexed keys, etc - run this first, check for same files present & same record counts in matched pairs of files. See samples 4 pages ahead --->

  4. 'uvcmp5all' script listed on page '5X1' at end of Part 5.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5D2. uvcmp5/uvcmp5all - compare 2 directories, creating reports in a 3r d

sample control file

 # ctlcmp1 - control file to generate complete jobs from skeletons
 #         - used by uvdata52 to encode data-file-names & ISAM keys
 #
 # uvcmp5 - this ctlcmp1 must be converted to an ISAM file so uvcmp5 can
 #          determine the record sizes of sequential files, use loadctlI:
 #
 #          uvcopy loadctlI,fili1=ctl/ctlcmp1,filo1=ctl/ctlcmp1I
 #          ====================================================
 #
 # uvcmp6 - the procedure of generating field by field comparison jobs for
 #          each file uses the uvdata52 utility which converts the uvcmp5
 #          skeletons to complete uvcopy jobs with data filenames & keys.
 #        - uvdata52 uses this file directly as a sequential text file
 #          to relate the data files to the copybooks & code ISAM key info.
 #
 custmas1       cpy=custmas1 rcs=00256
 custmast       cpy=custmas1 rcs=00256  keys=(0,6,nn,10,25,nn,60,16,nn,etc)
 warmas1        cpy=warmas1  rcs=00064
 warmas3        cpy=warmas3  rcs=00064
 warmas6        cpy=warmas6  rcs=00064
 warmas8        cpy=warmas8  rcs=00064 keys=(0,6,nn)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5E1. uvcmp5 - compare all files in 2 directories

uvcmp5 test/demo - preparation

You can run the following demo, using the test files provided in uvadm/dat1 We will setup a test sub-directory (within your home directory), change into it and setup several sub-sub-directories. This will keep our home directory uncluttered & make it easier to clean up after the test/demo.

Sub-directories are made for the different types of files, so that we can keep the same filenames for different versions of the files. These data files will also be used for the demos in the next section.

Multiple sub-directories work well if we have many files to be compared We can compare 2 entire directories & output the corresponding mismatch reports to a 3rd directory.

 1a. cd $HOME        - change to your home directory
 1b. mkdir cmpdata   - make a sub-directory for the test/demo
 1c. cd cmpdata      - change into it
 2a. mkdir dat1      - make subsubdir for original data files
 2b. mkdir dat2      - make subsubdir for modified data files
 2c. mkdir rptcmp    - make subsubdir for mismatch reports
 2d. mkdir ctl       - make directory for control files
 2e. mkdir tmp       - subdir for misc outputs (keep working dir clean)

copy test files to dat1 & modify in dat2

3a. cp /home/uvadm/dat1/custmas1 dat1 3b. cp /home/uvadm/dat1/warmas1 dat1 3c. cp /home/uvadm/dat1/warmas6 dat1


 3d. uvcp "fili1=/home/uvadm/dat1/warmas8,rcs=64,typ=RSF,filo1=dat1/warmas8\
     =======================================================================
           ,typ=ISF,isk1=0(6)"      - convert warmas8 to ISAM (see uvcp.doc)
           ===================
  1. cp dat1/* dat2 - duplicate all dat1 files into dat2 subdir

  2. vi dat2/* - create some differences to illustrate mismatch reports - don't modify record sizes since these are fixed length

    load control file


 6a. cp /home/uvadm/ctl/ctlcmp1 ctl/ctlcmp1
     ======================================

 6b. uvcopy loadctlI,fili1=ctl/ctlcmp1,filo1=ctl/ctlcmp1I
     ====================================================
     - copy ctlcmp1 & create indexed version for uvcmp5

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5F1. uvcmp5 - compare all files in 2 directories

uvcmp5all test/demo - execution


 uvcmp5all dat1 dat2 rptcmp filr4=ctl/ctlcmp1I,uop=q0i7s5
 ========================================================

uvcmp5all demo console output

compare files in dir dat1 to dir dat2, write mismatch reports to rptcmp

EOF fili1 32 rds, 0 wrts, 8192 size, fname=dat1/custmas1 EOF fili2 32 rds, 0 wrts, 8192 size, fname=dat2/custmas1 EOF filo3 0 rds, 45 wrts, 2689 size, fname=rptcmp/custmas1 1 hits EOF filr4 1 rds, 0 wrts, 0 upds, 0 dels, 906 size, fname=ctl/ctlcmp1I ctl/ctlcmp1I DISAM STATS: recs=6,rcsz=150,keys=1

EOF fili1 7 rds, 0 wrts, 448 size, fname=dat1/warmas1 EOF fili2 7 rds, 0 wrts, 448 size, fname=dat2/warmas1 EOF filo3 0 rds, 24 wrts, 1161 size, fname=rptcmp/warmas1 2 hits EOF filr4 1 rds, 0 wrts, 0 upds, 0 dels, 906 size, fname=ctl/ctlcmp1I ctl/ctlcmp1I DISAM STATS: recs=6,rcsz=150,keys=1

EOF fili1 7 rds, 0 wrts, 448 size, fname=dat1/warmas6 EOF fili2 7 rds, 0 wrts, 448 size, fname=dat2/warmas6 EOF filo3 0 rds, 45 wrts, 2151 size, fname=rptcmp/warmas6 5 hits EOF filr4 1 rds, 0 wrts, 0 upds, 0 dels, 906 size, fname=ctl/ctlcmp1I ctl/ctlcmp1I DISAM STATS: recs=6,rcsz=150,keys=1

EOF fili1 6 rds, 0 wrts, 0 upds, 0 dels, 390 size, fname=dat1/warmas8 dat1/warmas8 DISAM STATS: recs=6,rcsz=64,keys=1 EOF fili2 6 rds, 0 wrts, 0 upds, 0 dels, 426 size, fname=dat2/warmas8 dat2/warmas8 DISAM STATS: recs=6,rcsz=70,keys=1 EOF filo3 0 rds, 45 wrts, 2183 size, fname=rptcmp/warmas8 5 hits EOF filr4 1 rds, 0 wrts, 0 upds, 0 dels, 906 size, fname=ctl/ctlcmp1I ctl/ctlcmp1I DISAM STATS: recs=6,rcsz=150,keys=1

5 files compared in dat1 & dat2, mismatch reports in rptcmp

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5G1. uvcmp5 - compare all files in 2 directories

VTOC reports - aid file compares

You will find the 'VTOC' reports a valuable asset in comparing the resulting directories of files from your parallel tests. Run these first & review any differences in files present & record counts within the files.

Note that 'ls -l' does not give you any record counts, because UNIX does not carry a record size within directory entries (as did mainframes). The 'vtocr1' utility uses the same control file that we created on page '5D2' for the 'uvcmp5' job to know the record sizes for file compares.

print VTOC reports for the 2 data directories


 uvcopy vtocr1,fild1=dat1,filr1=ctl/ctlcmp1I
 ===========================================
 vtocr1 prompts for report disposition
 report=tmp/vtocr1.rpt - more/vi/uvlp12 --------> uvlp12 <-- print at 12 cpi

 uvcopy vtocr1,fild1=dat2,filr1=ctl/ctlcmp1I  --> uvlp12 <- print dat2 report
 ===========================================

sample VTOC report for dat1

 vtocr1    Directories: dat1
 1998/11/15_08:08:47                           options: q1a1s2k2s1
 file#   file-size   rcsz  records  kloc klen  last-modify   filename
    1        8,192   256        32            19981114 16:19  custmas1
    2          448    64         7            19981114 09:23  warmas1
    3          448    64         7            19981102 18:13  warmas6
    4          390    64+        6      0  6  19981104 09:35  warmas8
         Total Records          59

sample VTOC report for dat2

 vtocr1    Directories: dat2
 1998/11/15_08:11:09                           options: q1a1s2k2s1
 file#   file-size   rcsz  records  kloc klen  last-modify   filename
    1        8,192   256        32            19981114 16:23  custmas1
    2          448    64         7            19981114 09:32  warmas1
    3          561    70         8*           19981105 09:39  warmas6
    4          426    70+        6      0  6  19981104 09:35  warmas8
         Total Records          60

The '+' beside the rcsz of warmas8 (64+ in dat1 & 70+ in dat2) indicates that the physical record size is 1 greater (ISAM record delete flag x'0A' at EOR).

The '*' beside record count (8*) of warmas6 in dat2 indicates the file size is not evenly divisible by the record size. See explanation next page --->

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5G2. uvcmp5/uvcmp5all - compare 2 directories, creating reports in a 3r d

sample VTOC report for dat2

 vtocr1    Directories: dat2
 1998/11/15_08:11:09                           options: q1a1s2k2s1
 file#   file-size   rcsz  records  kloc klen  last-modify   filename
    1        8,192   256        32            19981114 16:23  custmas1
    2          448    64         7            19981114 09:32  warmas1
    3          561    70         8*           19981105 09:39  warmas6
    4          426    70+        6      0  6  19981104 09:35  warmas8
         Total Records          60

why the '*' beside record count

The '*' beside record count (8*) of warmas6 in dat2 indicates the file size is not evenly divisible by the record size.

In this case it is because our control file has the correct record size for warmas6 in dat1, but not for warmas6 in dat2.

The records in warmas6 & warmas8 of dat2 have been expanded to 70 bytes to test 'uvcmp6' (see next section) which can do field by field comparisons of different record sizes (due to date field expansion).

The solution is to create a different control file for the dat2 directory of expanded record sizes. We could simply copy our ctl/ctlcmp1 to say ctl/ctlcmp2,& correct the record sizes with the editor.

Note that uvcmp5all requires the Indexed version of the control file, and we don't want to attempt to edit the Indexed file directly. Is is much safer to copy the non-indexed version of the control file, modify that,& use it to create the indexed version, for example:


 1. cp ctl/ctlcmp1 ctl/ctlcmp2    - copy sequential version of ctlcmp1
    ==========================

 2. vi ctl/ctlcmp2                - update ctlcmp2 for rcsz's in dat2
    ==============

 3. uvcopy loadctlI,fili1=ctl/ctlcmp2,filo1=ctl/ctlcmp2I
    ====================================================
                                   - create Indexed version of ctlcmp2
                                     for use by vtocr1 report utility

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5X0. Summary of scripts & uvcopy jobs used in Part 5

scripts used in Part 5 & listed here


5X1. unixcmpall - script using UNIX 'cmp' to compare all files in 2 subdirs
& create a report showing which files are different.

5X2. unixcmpsplit - script using UNIX 'cmp' to compare all files in 2 subdirs
& move non-matching files out to 2 other subdirs

5X3. uvcmp5all - script to execute uvcmp5 for every pair of files in the
2 input directories

summary of uvcopy jobs used in Part 5

uvcmp5
  • similar to uvcmp1, but uses a control file to supply record sizes
    & file types for all files in 2 directories to be compared.
  • uvcmp5 is best used with the 'uvcmp5all' script, which processes
    all pairs of files in 2 directories, & writes the reports to a
    3rd directory.

The uvcopy jobs in this section are not listed here to save space. They are of course readily available in the /home/uvadm/pf/... directories and you can inspect them with the editor or list them as follows. Using uvcmp1 as an example:


 cd /home/uvadm         - change to uvadm home directory
 ==============
 vi pf/util/uvcmp5      - inspect uvcmp5 (or any uvcopy job desired)
 =================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5X1. scripts used in Part 6 & listed here

unixcmpall - compare all files in 2 subdirs

 #!/bin/ksh
 # unixcmpall - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # unixcmpall - use UNIX 'cmp' to compare corresponding files in 2 directories
 #            - very fast since it stops at the 1st miscompare (no -l option)
 #            - for large files run this 1st to determine which files are diff
 #            - then run uvcmp5 on the diff files to get the diff reports
 #
 # unixcmpall-l - alternate script with '-l' (long) option
 #              - do NOT use this, creates very long reports (1 byte per line)
 #              - use 'uvcmp5all' which shows 64 byte diffs per line
 #
 if [[ -d "$1" && -d "$2" ]]; then :
    else echo "usage: unixcmpall dir1 dir2"
         echo "       ===================="
         echo " - arg1 & arg2 must be directories"
         exit 9; fi
 #
 x=0;y=0;
 for i in $1/*
   { let x=x+1
     f=${i##*/}
     echo "file# $x cmp $1/$f $2/$f"
     cmp    $1/$f $2/$f >tmp/$f
     #=============================
     if (($? != 0))
        then echo "                     mis-compares--->tmp/$f"
        let y=y+1
     fi
   }
 echo ">> $y mismatches of $x total files compared in $1 & $2"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5X2. scripts used in Part 6 & listed here

unixcmpsplit - move non-matching files to other subdirs

 #!/bin/ksh
 # unixcmpsplit - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # unixcmpsplit - use UNIX 'cmp' to compare all files in 2 directories
 #                & move Non-Matching files out to 2 other directories
 #
 # - UNIX 'cmp' will quickly determine if files are same or different,
 #   but very inadequate for showing where & what the differences are.
 #
 # - We recommend running unixcmpsplit to select out Non-Matching files prior to
 #   running 'unixcmpall' which is very good at reporting the differences.
 # - see complete documentation in CMPjobs.doc
 #
 if [[ -d "$1" && -d "$2" && -d "$3" && -d "$4" ]]; then :
 else echo "usage: unixcmpsplit dir1 dir2 dir1X dir2X"
      echo "       =================================="
      echo " - arg1,2,3,4 must be subdirs"
      exit 9; fi
 #
 x=0;y=0;z=0;
 for i in $1/*
   { let x=x+1
     f=${i##*/}
     echo "file# $x cmp $1/$f $2/$f"
     cmp $1/$f $2/$f          # compare current pair of files in dir1 & dir2
     if (($? == 0))
        let y=y+1
     else
        mv -i $1/$f $3        # move non-matching files to dir3 & dir4
        mv -i $2/$f $4
        let z=z+1
     fi
   }
 echo ">> $y matching files in $x total compared in $1 & $2"
 echo ">> $z Mis-Matching files moved to subdirs $3 & $4"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5X3. scripts used in Part 5 & listed here

uvcmp5all - execute uvcmp5 for every pair of files in 2 subdirs

 #!/bin/ksh
 # uvcmp5all - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # uvcmp5all - compare all files in 2 directories with output to a 3rd
 #           - script calls 'uvcmp5' for each pair of files in 2 dirs
 #           - writes the mismatch reports to a 3rd directory
 # - could run uvcmp5 manually for any pair of files, but this script saves
 #   a lot of manual entry when you have multiple files to compare.
 #
 #usage: uvcmp5all indir1 indir2 reportdir [ctlfile,options]
 #       ===================================================
 #example:
 # 1. cd /home/uvadm/tstcmp
 # 2. mkdir dat1,dat2,rptcmp,ctl,etc, copy test data from uvadm/dat1, etc
 #    - see demo setup in COMPjobs.doc
 #
 # 3. uvcmp5all dat1 dat2 rptcmp uop=q0i7s3,filr4=ctlI/ctlcmp1
 #    ========================================================
 #
 # - uvcmp5 requires a 1 time preparation of a control file of information
 #   about the files at your site (filename,recsize,filetype,& ISAM keys)
 # - Hard-code full pathname of the control file in the uvcmp5 job since it
 #   does not change often & saves you entering it on the command line
 # - arg4 then used only for options & you can set your desired defaults
 #   in the script or in the uvcmp5 uvcopy job
 # - you will probably default option q0 to inhibit prompts on each file
 #   & option i7 to inhibit extraneous console messages (leaving EOF stats)
 # - you might want to modify some options from run to run, such as the
 #   print stop count (default s3) & the terminate count (default t5000)
 # - be sure to use terminate count t99999999 for your final comparison
 #   tests (when most problems resolved) to ensure no mismatches ignored.
 #
 # The script command above would generate 1 uvcopy command for each file
 # in the input#1 directory, for example:
 #
 # uvcopy uvcmp5,fili1=dat1/file1,fili2=dat2/file1,filo3=rptcmp/file1\
 # ===================================================================
 #                                ,filr4=ctlI/ctlcmp1,uop=q0i7s5
 #                                ==============================
 #
 echo "compare files in dir $1 to dir $2, write mismatch reports to $3"
 if [[ (-d "$1") && (-d "$2") && (-d "$3") ]]; then :
 else echo "ERR usage: uvcmp5all indir1 indir2 outdir uop=q0i7"; exit 1; fi
 x=0
 for i in $1/*
 do
   f=${i##*/}
   if [[ $f != *.idx ]]; then
      uvcopy uvcmp5,fili1=$1/$f,fili2=$2/$f,filo3=$3/$f,uop=q0i7,$4
      let x=x+1
   fi
 done
 echo "$x files compared in $1 & $2, mismatch reports in $3 "
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_6 uvcmp6 - field by field comparisons

Field by Field comparisons - Contents


6A1. uvcmp6 - field by field comparison - Overview
- allows date/time stamps to be removed
- allows comparing different size records (after Y2K expansion)
- separate job per file, automatically generated from copybooks

6B1. Illustrate 'field by field' data file comparisons
- comparing files with same record sizes, but with date/time stamps.
- sample/demo: data-file, copybook, generated compare job, mismatch-
  report, demo operating instructions, options.
- Was used for Year2000 testing after DATE WINDOWING changes
  but could be used anytime you need to exclude some fields

6C1. Preparation to generate field by field compare jobs
- from 1 or 2 copybooks (2nd could have different layout)

6D1. use 'uvcmp6' to Generate field by field compare jobs 1 at a time
6D2. Execute generated field by field compare job

6E1. script 'genuvcmp6' to generate field by field compare job

6F1. File comparison demo for Year2000 19xx testing for DATE EXPANSION
- Compare files that have different record sizes due to Y2K expansions.
  or whenever record layouts are changed for any reason.
- sample/demo: data-file, copybook, generated compare job, mismatch-
  report, demo operating-instructions.

6G1. Operating Instructions to generate comparison jobs for all copybooks
in a directory & all corresponding data files.
6G1. Preparation for generating for all copybooks & data files.
6G3. Generating comparison jobs for all copybooks & all data files.
6G4. Alternative generation for duplicate fieldnames due to qualification
6G5. Sample comparison job fully completed with filenames & indexed keys.
6G6. Executing the fully generated jobs (1 at a time or all or by prefix).
6G7. Inspecting, printing,& using the mismatch reports.

6I1. Operating Summary & Recommendations

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Field by Field comparisons - Contents (continued)


6J0. Modifying generated jobs.
- summary of functions performed by called subroutines.

6J1. summary uvcmp6 - generates uvcopy comparison jobs from copybooks
- generated uvcopy jobs
- uvcmp6.subs - supplied subrtns called by generated jobs

6J2. Alternate subrtn 'cmpp' vs 'cmp', to compare data files from the
- mainframe to data files created on UNIX.

6J3. Splitting combined fields to eliminate subfields

6J4. Generating 1 instruction per field (vs combining like fields)

6K1. Inserting record-type tests for files with redefined records

6L1. Corrections for occurs clauses with mixed data types

6M1. Exceptions, Problems,& Solutions
      1. options h,j,k to allow comparison where 1 file has an expanded date
         within the 1st key (used to chain to the matching record in 2nd file)
      2. Complement dates cause problems if present within the 1st key, which
         is used to chain to the 2nd file.
         Alternate versions of uvcmp1, uvcmp5,& uvcmp6.subs are provided
         (renamed as uvcmp1X, uvcmp5X,& uvcmp6X.subs) for this problem.

6N1. Alternative method to compare data files field by field, showing
corresponding COBOL copy-book field-names (uvhdcob & uvhddif2).
Sample report illustrated here, complete doc in uvhdcob.doc of vol 1.

6O1. Applications
1. Year2000 19xx testing (windows, expansion, or combinations)
2. Year2000 20xx testing (windows, expansion, or combinations)
3. Mainframe to UNIX conversions

6X1. Listings of the comparison scripts documented in this Part.
- uvcmp1, uvcmp5, uvcmp6 & uvcmp6.subs genuvcmp6

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6A1. comparing data files Field by Field based on COBOL copybooks

Comparing data files field by field - Overview

Part_5 shows you how to automatically generate uvcopy jobs to compare data file records (field by field) based on the COBOL copybooks, tables of date-field-names,& a control file of data file names & related copybooks.

'uvcmp6' is the most important part of this process, but there are several other jobs involved which are documented in other sections of the Vancouver Utilities documentation.

  cobmap1 - documented in COBaids.doc
          - creates 'cobmaps' from COBOL copybooks, encoding the field
            length, type, start,& end bytes on the right hand side.
          - copies the entire copybook directory to an alternate directory
            while adding the 'map' information.
  reform1 - documented in DATAcnvt1.doc
          - loads the old cobmap (copybook record layout) into an indexed
            file, so uvcmp6 will have access to the old field locations
            as well as the new copybook locations (primary input file).
 *uvcmp6  - the HEART of the process, but the complete procedure depends on
            several other pre-programmed jobs in the Vancouver Utilities.
          - uvcmp6 generates a skeleton uvcopy for each pair of copybook/maps.
          - the skeleton job will have 'bal' instructions to compare the
            records field by field. The job contains tables of fieldnames
            that should not be compared (date/time stamps, etc).
  uvdata52 - documented in DATAcnvt1.doc
          - uvdata52 will complete the skeletons created by uvcmp6, supplying
            the actual data-file names & any indexed keys, from a control
            file which relates the data-file names to the copybook names.
          - A sample control file (uvadm/ctl/ctlcmp1) is supplied & you can
            use this as a guide to create your version.
  uvcopyxx - script supplied to execute all the generated uvcopy jobs
            to increment the dates in multiple files. Or you can, of course,
            execute them individually to convert one file at a time.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6B1. comparing data files Field by Field based on COBOL copybooks

Field by Field comparison - Illustration

Assume we have modified our programs for Year2000 using 'windows' solutions, and we want to test our modified programs using 19xx data, to verify that we have not broken the programs while modifying for Year 2000. We might proceed as follows:

  1. Run test data files through the original programs & save outputs We will refer to these as: '19xx data outputs from original programs'

  2. Run test data files through the modified programs & save outputs We will refer to these as: '19xx data outputs from windowed programs'

  3. Generate & execute the jobs to compare the 2 sets of data files, to prove that our modifications have not broken anything.
Note
  • We would later repeat this procedure using the 20xx data (dates aged
    into 2000), to prove the Year 2000 modifications.

demo data file /home/uvadm/dat1/warmas6

cust# prod# description purch-date expire-date date-time-stamp

 12345 12345 sony television   970101 1-year 980101 981201091501
 22222 11111 refrigerator      970101 2-year 990101 981201091502
 33333 22222 dish washer       961231 3-year 991231 981201103003
 33333 11111 refrigerator      970101 3-year 000101 981201103004
 44444 33333 freezer           970101 4-year 010101 981201124505
 55555 55555 clothes dryer     971231 5-year 021231 981201124506
 66666 66666 blank/zero dates         6-year 000000 981201125507

cobmap1 (copybook record layout) for data file above

 cobmap1  start-end bytes for cobol record fields    199906061246  pg# 0001
 cpy1/warmas6                                      RCSZ=0064  bgn-end lth typ
 *warmas6 - warranty master record (warmas1 with date-time vs
     05 wm-cust             pic  x(6).                        000-0005 006
     05 wm-prod             pic  x(6).                        006-0011 006
     05 wm-descrip          pic  x(18).                       012-0029 018
     05 wm-purch-date       pic  x(6).                        030-0035 006
     05 wm-policy           pic  x(8).                        036-0043 008
     05 wm-expiry.
        10 wm-exp-year      pic  9(2).                        044-0045 002 n  02
        10 wm-exp-month     pic  9(2).                        046-0047 002 n  02
        10 wm-exp-day       pic  9(2).                        048-0049 002 n  02
     05 filler001           pic  x(1).                        050-0050 001
     05 wm-date-time        pic  x(12).                       051-0062 012
     05 filler002           pic  x(1).                        063-0063 001
 *RCSZ=0064                                                       0064

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6B2. comparing data files Field by Field based on COBOL copybooks

sample uvcopy job to compare 2 data files

The previous page showed a demo data file & corresponding copybook, that we are using to illustrate the 'uvcmp6' utility, which automatically generates a uvcopy job (interpretive code) to compare data files, field by field. This allows disabling fields that should not be compared (date/time stamps).

This also allows comparing records of different layouts, which will be illustrated in the next section (after date-field expansions).

 opr='JOBNAME warmas6 - generated by cobmap1,uvcmp6,uvdata52'
 rop=j64000k16000,uop=q1p30s3t1000v0
 was=a8000b8000c8000d8000e8000f8000g8000h8000i8000j8000
 fili1=FILEIN1,rcs=0064,typ=RSF
 fili2=FILEIN2,rcs=0070,typ=RSF
 filo3=rptcmp/FILEOUT,rcs=128,typ=LSTt
 @run
        bal    cmpa                      open files, write report headers
 # begin loop to get each record pair, compare, report(if mismatch), until EOF
 loop   bal    cmpb                      get record pair & init possible output
        skp>   eof
        bal    cmpc,'a0000b0030c0000d0030','wm-cust : wm-descrip'
        bal    cmpc,'a0030b0006c0032d0006','wm-purch-date c0030d0008'
        bal    cmpc,'a0036b0008c0038d0008','wm-policy'
        bal    cmpc,'a0044b0002c0048d0002','wm-exp-year c0046d0004'
        bal    cmpc,'a0046b0004c0050d0004','wm-exp-month : wm-exp-day'
        bal    cmpc,'a0050b0001c0054d0001','filler001'
 #2     bal    cmpc,'a0051b0012c0057d0012','wm-date-time c0055d0014'
        bal    cmpc,'a0063b0001c0069d0001','filler002'
        bal    cmpd                      output records if any mismatches
        skp    loop                      repeat loop until EOF
 # EOF - write reports stats, close files,& end job
 eof    bal    cmpe                      write rpt stats, close files, end job
 @pf2=uvcmp6.subs  #load(at run-time) common subrtns (cmpa,cmpb,cmpc,cmpd,cmpe)

Please see the sample report (data mismatches) on the next page --->

Note that this utility compares multiple contiguous fields of the same type with 1 'bal' instruction. The first & last fieldnames are shown in operand 3 on the right side (eg - 'wm-cust : wm-descript'). The copybook on the previous page shows you this includes wm-prod. The total length is coded as option 'b' (b0030) 30 bytes combined.

Uvcmp3 contains a table of fieldnames to force breaking contiguous compares. This table includes names such as 'date', 'year', etc & you can add to it.

A 2nd table is used to inhibit compares on fields such as date/time stamps. This table might include names such as 'time', 'stamp', etc.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6B3. comparing data files Field by Field based on COBOL copybooks

sample report - data mismatches (CHAR only)

Here is a sample report created by the uvcopy job shown on the previous page (which was automatically generated from the copybook).

Note that, we created some intentional mismatches by using the vi editor to change some data in one of the files (cmpdata/dat2/warmas6).

 uvcmp6 - compare 2 files & "*" differences (uops=q0p30s3t1000v0)
 record-size     count   file-size  typ  filename
 file1:   64         7         448  RSF  dat1/warmas6
 file2:   64         7         448  RSF  dat2/warmas6
 1998/11/08_15:32:57   1         2         3         4         5         6
 f#rec#byte# 0123456789012345678901234567890123456789012345678901234567890123
 ============================================================================
 1    3    0 33333 22222 dish washer       961231 3-year 991231 981201103003.
                              *******             *       *
 2    3      33333 22222 dish CRASHER      961231 2-year 981231 981202133333.
                              *******             *       *
 1    6    0 55555 55555 clothes dryer     971231 5-year 021231 981201124506.
                                 ********         *       *
 2    6      55555 55555 clothes SHRINKER  971231 6-year 031231 981201162666.
                                 ********         *       *
 ======================= EOF or stop count reached ==========================
 Total=000007, read=000007, mismatch=000002, stop=000003, term=000100

Please see the operating instructions for this demo on the next page --->

Note the '*'s marking the differences between the record pairs, and also note that there are no '*'s under bytes 51-62 (on the right hand side), even though these fields are clearly different.

This field is a date/time stamp which is excluded from the comparison by table #2 in the 'uvcmp6' generation job. It's fieldname is 'wm-date-time' & is excluded because it has a match on 'time' in table #2. Actually the 'bal' comparison instruction is generated, but #commented out, so you can reinstate, in case of table #2 unintended matches.

Note that the page headings give you the information you need to know about the file (rcsz, no of records, filesize, filenames, options in effect, date.

The EOF count statistics are vital, indicating whether the report shows you all the mismatches, or whether the stop count was reached. The stop count saves wasting a lot of paper; you probably want to fix the programs & rerun the program & the comparison jobs. You might set the stop count higher as your testing gets cleaner to ensure you compare to end of file.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6B4. comparing data files Field by Field based on COBOL copybooks

sample report - data mismatches (CHAR & HEX)

The report below was created from the same pair of data files as was the report on the previous page, but we specified the *'v2'* option to force vertical hexadecimal record displays.

The default option is 'v0' which automatically determines whether to print char only or both char+hex, depending on the presence of packed/binary data.

You are prompted to change the default options when you run the compare job but if desired you can specify the option on the command line as follows: (Please see the detailed operating instructions on page '6D1' ahead)


 uvcopy pfc2/warmas6,fili1=dat1/warmas6,fili2=dat2/warmas6
 =========================================================
                    ,filo3=rptcmp/warmas6x,uop=v2
                    =============================

sample report - data mismatches (CHAR & HEX)

 uvcmp6 - compare 2 files & "*" differences (uops=q0p30s3t1000v0v2)
 record-size     count   file-size  typ  filename
 file1:   64         7         448  RSF  dat1/warmas6
 file2:   64         7         448  RSF  dat2/warmas6
 1998/11/08_15:30:09   1         2         3         4         5         6
 f#rec#byte# 0123456789012345678901234567890123456789012345678901234567890123
 ============================================================================
 1    3    0 33333 22222 dish washer       961231 3-year 991231 981201103003.
             3333323333326676276766722222223333332327667233333323333333333330
             33333022222049380713852000000096123103D951209912310981201103003A
                              *******             *       *
 2    3      33333 22222 dish CRASHER      961231 2-year 981231 981202133333.
             3333323333326676267676672222223333332327667233333323333333333330
             33333022222049380321385200000096123102D951209812310981202133333A
                              *******             *       *
 1    6    0 55555 55555 clothes dryer     971231 5-year 021231 981201124506.
             3333323333326667667267767222223333332327667233333323333333333330
             5555505555503CF48530429520000097123105D951200212310981201124506A
                                 ********         *       *
 2    6      55555 55555 clothes SHRINKER  971231 6-year 031231 981201162666.
             3333323333326667667276766667223333332327667233333323333333333330
             5555505555503CF485303829EB520097123106D951200312310981201162666A
                                 ********         *       *
 ======================= EOF or stop count reached ==========================
 Total=000007, read=000007, mismatch=000002, stop=000003, term=000100
NOTE
  • option 'v2' forces HEX print (vs 'v0' for CHAR print on prior page)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6B5. comparing data files Field by Field based on COBOL copybooks

Investigating the differences

Now you have data 'difference' reports, how do you find the problems ? You may want to investigate the records just before the differences occurred.

'uvhd' is the perfect tool for displaying the desired records, either by record# (shown on the difference reports, or by searching for patterns.

sample 'uvhd' display


 uvhd dat2/warmas6 r64   <-- initiate uvhd on dat/warmas6 with record size 64
 =====================
 dat2/warmas6 bytes=448 recs=7 mdate=1998110309 today=199811050647
 r#       1           1         2         3         4         5         6
 b#       0 0123456789012345678901234567890123456789012345678901234567890123
          0 12345 12345 sony television   970101 1-year 980101 981201091111.
            3333323333327667276667676662223333332327667233333323333333333330
            1234501234503FE9045C56939FE00097010101D951209801010981201091111A
 r#       2           1         2         3         4         5         6
 b#      64 0123456789012345678901234567890123456789012345678901234567890123
          0 22222 11111 refrigerator      970101 2-year 990101 981201091222.
            3333323333327667666767672222223333332327667233333323333333333330
            2222201111102562975214F200000097010102D951209901010981201091222A
 r#       3           1         2         3         4         5         6
 b#     128 0123456789012345678901234567890123456789012345678901234567890123
          0 33333 22222 dish CRASHER      961231 2-year 981231 981202133333.
            3333323333326676267676672222223333332327667233333323333333333330
            33333022222049380321385200000096123102D951209812310981202133333A

fname=dat2/warmas6 fsz=448 rsz=64 recs=7 current#=3 fptr=128 null=next,r#=rec,b#=byte,+/-recs,s=search,u=update,p=print,w=write ,q=quit,?=help -->

When you start uvhd it displays the first record & prompts for your command.

   s '961231'          <-- search for 961231 anywhere in records
   s 30(6),'961231'    <-- search for 961231 in bytes 30-35 of records

See detailed documentation for uvhd in uvhd.doc .

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6C1. comparing data files Field by Field based on COBOL copybooks

test/demo - preparation (1 at a time)

These demo Op. Instrns. generate compare jobs '1 FILE AT A TIME', BUT for production, I recommend the 'ALL FILES in DIRECTORY' procedures. See '6G1' + ahead --->

We will setup a test sub-directory (within your home directory), change into it and setup several sub-sub-directories. This will keep our home directory uncluttered & make it easier to clean up after the test/demo.

Sub-directories are made for the different types of files, so that we can keep the same name as the original copybook.

Multiple sub-directories works well if we have many files to be compared We can generate these jobs for an entire directory of COBOL copybooks, and their corresponding data files (see page '6G1' +).

This procedure provides for comparing data files before & after expansion (or any kind of conversion) by allowing you to specify 2 different copybooks (1 for the original file & 1 for the expanded or converted file).

In this case (19xx windows testing) the data files are the same, and we will specify the same copybook for the old file as for the new file. The next demo (expansion testing, page '6E1' +) will need 2 copybooks.

Directories Required for uvcmp6

 /home/userxx
 :-----cmpdata          <-- superdir for DATA comparisons
 :     :-----cpy1         - copybooks
 :     :-----cpy2         - copybooks for modified layouts (as in Y2K)
 :     :-----dat0         - EBCDIC files transferred from mainframe
                          - may need to convert variable to fixed in dat1
 :     :-----dat1         - data files for comparison to dat2/...
 :     :-----dat2         - data files 2nd set (same names as in dat1)
 :     :-----map1         - copybook layouts
 :     :-----map1I        - copybook layouts loaded to Indexed files
 :     :-----map2         - modified layouts (as in Y2K) OR same as map1
 :     :-----pfc1         - uvcmp6 comparison jobs generated from layouts
 :     :-----pfc2         - comparison jobs copied here for any mods & execution
 :     :-----rptcmp       - reports generated here
 :     :-----tmp          - for misc outputs (keep working dir free of files)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6C2. comparing data files Field by Field based on COBOL copybooks

Setup Directories for uvcmp6

You may perform uvcmp6 data file comparisons within a subdirectory of your home directory (as illustrated above). We will refer to your userid as 'userxx' & will use 'cmpdata' as the superdir containing the several subdirs.

You must have setup your profile as documented in ADMjobs.htm#Part_1.

The profiles now include 'export CMPDATA=$HOME/cmpdata' & an alias 'cdk' to take you there.


 #1. Login ---> /home/userxx  <-- Login or change to your home directory

 #2. mkdir cmpdata            <-- make superdir for subdirs shown above
     =============

 #3. cd cmpdata               <-- change into data comparison superdir
     ==========

 #4. cmpdatadirs              <-- run script to make subdirs shown above
     ===========

 #5a. cp /home/uvadm/cobol/cpys/warmas6 cpy1  - copy demo copybook to subdir
      ======================================
 #5b. cp /home/uvadm/cobol/cpys/warmas6 cpy2  - same copybook for new layouts
      ======================================    (diff for Y2K or any reformats)

 #6a. cp /home/uvadm/dat1/warmas6 dat1  - copy test-data to original subdir
      ================================
 #6a. cp /home/uvadm/dat1/warmas6 dat2  - copy test-data to modified subdir
      ================================

 #7.  vi dat2/warmas6  - create some differences to illustrate uvcmp6
      ===============  - don't modify record sizes since these are fixed length

The following demos generate 1 compare job at a time. For production we will generate jobs for all copybooks & all data files. The Instructions are shorter since you specify directories not filenames. See page '6G1' ahead --->

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6D1. comparing data files Field by Field based on COBOL copybooks

Demo Op. Instructions (1 file at a time)

 <----- see preparations on the previous page.

 1. uvcopy cobmap1,fili1=cpy1/warmas6,filo1=map1/warmas6
    ====================================================
                                - run cobmap1 for the original copybook

 2. uvcopy cobmap1,fili1=cpy2/warmas6,filo1=map2/warmas6
    ====================================================
                                - run cobmap1 for the new copybook

 3. uvcopy reform1,fili1=map1/warmas6,filo1=map1I/warmas6
    =====================================================
                                - load old map into an Indexed file

 4. uvcopy uvcmp6,fili1=map2/warmas6,filr1=map1I/warmas6,filo1=pfc1/warmas6\
    ========================================================================
                 ,arg2=e9999p30s3t9999v0   (arg2 defaults as shown OK)
                 =======================
    - generate uvcopy job to compare files
  1. uvcopy uvdata52 - optional, omit for this demo - could insert correct filenames & Indexed file keys - we will supply filenames on command line for this demo. - see page '6G1' + which will illustrate 'uvdata52'. - would definitely use it for production with multiple files,

 6. cp pfc1/warmas6 pfc2  <-- copy job to alt subdir before modify/execute
    ====================    - to protect edits from inadvertent regen

 7. vi pfc2/warmas6         - modify generated job if required
    ===============         - add record-type tests for redefined fields
                            - might change data-file-names

 8. uvcopy pfc2/warmas6,fili1=dat1/warmas6,fili2=dat2/warmas6,filo3=rptcmp/warmas6
    =============================================================================
                            - execute the job to compare the data file

 8a. uvcopy pfc2/warmas6    <-- same as above (files default as shown)
     ===================      - OK if you changed datafilenames to match copybook

 9a. vi rptcmp/warmas6        - inspect report for differences (* markers)
     =================

 9b. uvlp12 rptcmp/warmas6    - print report
     =====================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6D2. comparing data files Field by Field based on COBOL copybooks

Options - for generation job & for generated jobs

Uvcmp3 has 'options' that affect behavior at generation time & 'arguments' that affect behavior of the generated jobs at execution time.

The arguments specified to the uvcmp6 generator are passed thru to the generated jobs as the 'default options' at execution time.

uvcmp6 prompt for generation option changes

 uop=q1g0c14   - option defaults
     q1        - prompt operator for option changes (q0 inhibits)
       g0      - combine contiguous same type fields into 1 compare
                 (breaks on fieldname matches to table date/time)
       g1      - generate separate bal/cmpc instrns for each field
         c__   - control action when field lth expanded
         c14   - default compare leftmost bytes if new field > 14 bytes
                 default compare rightmost bytes if new field <= 14 bytes
         c08   - change compare left/right decision to 8 bytes
         c00   - would disable any change to generated dsplcmnts(lengths)
 User OPtion (uop) defaults  = q1g0c14
 null to accept or re-specify (1 or more) -->

uvcmp6 prompt for arguments (default options for generated jobs)

enter options for generated jobs (defaults=q1p30s3t1000v0) null to accept defaults, or respecify all options

q1
  • prompt operator to allow option change at execution time
q0
  • disable prompts at execution time (use defaults)
  • 30 lines per page on report before 'eject'
  • stop count (after 3 mismatched records)
  • terminate count to count mismatches after stop print reached
  • could specify high number to read to end-of-file
  • auto determination vertical hex or character output
  • force character only
  • force vertical hex

specifying options & arguments on uvcmp6 command


 uvcopy uvcmp6,fili1=map2/warmas6,filr1=map1I/warmas6,filo1=pfc1/warmas6\
 ========================================================================
        ,arg2=q1e9999p30s3t9999v0,uop=q1g0c14  <-- defaults no need to enter
        =====================================
Note
  • 'arg2' on uvcmp6 is passed thru to the generated jobs to become
    the options at execution time. You can change these defaults
    within the 'uvcmp6' job code, so you wont have to enter on cmd line.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6E1. comparing data files Field by Field based on COBOL copybooks

script 'genuvcmp6' to generate uvcmp6 job

Page '6D1' presented the 'step by step' method of generating a uvcmp6 job to compare data files 'field by field'.

Alternatively, you can use the 'genuvcmp6' script to generate the uvcmp6 job with 1 command assuming the following:

  1. You are logged in using the profiles documented in ADMjobs.htm#Part_1.

  2. You have changed into the 'cmpdata' directory with the subdirs shown on page '6C1'.

3a. The copybook describing the 1st data file has been stored in cpy1/...

 3b. The copybook describing the 2nd data file has been stored in cpy2/...
     - will be same as the 1st if no layout differences exist

4a. The 1st data file is stored in dat1/...

4b. The 2nd data file is stored in dat2/...

  1. All filenames are the same & no .extensions are present. - this works because they are in different subdirs - if desired the data files could have different names because you can modify the generated uvcmp6 job after generation & before execution.

    Generation


 6. genuvcmp6 copybookname   <-- call script to generate uvcmp6 job
    ======================     - generated job stored in pfc1/...

 6a. genuvcmp6 warmas6       <-- demo with test file used on page '6D1'
     =================

Execution


 7. cp pfc1/warmas6 pfc2    <-- copy to alt subdir before edit/execute
    ====================      - protects edits in case inadvertent regen

 8. vi pfc2/warmas6         <-- modify generated job if required
    ===============           - might change data-file-names ?

 9. uvcopy pfc2/warmas6     <-- execute the file comparison job
    ===================

 10a. vi rptcmp/warmas6     <-- inspect the comparison report
      =================

 10b. vi rptcmp/warmas6     <-- print the comparison report
      =================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6F1. comparing data files Field by Field based on COBOL copybooks

Y2K 19xx DATE EXPANSION testing

Assume we have modified our programs for Year2000 using DATE EXPANSION, and we want to test our modified programs using 19xx data, to verify that we have not broken the programs while modifying for Year 2000. We might proceed as follows:

  1. Run original test data files through original programs & save outputs Refer to these as: '19xx original data outputs from original programs'

  2. Expand date fields in test set, creating a new set of test data files. Refer to these as: '19xx expanded data for input to expanded programs' - see reform2 job in DATAcnvt3.doc in vol 4 &/or REFORMjobs in vol 3.

  3. Run expanded test data files through the modified programs & save outputs Refer to these as: '19xx expanded data outputs from expanded programs'

  4. Generate & execute the jobs to compare the 2 sets of output data files, to prove that our modifications have not broken anything.
Note
  • We would later repeat this procedure using the 20xx data (dates aged
    into 2000), to prove the Year 2000 modifications.

demo data file - ORIGINAL before expansion

cust# prod# description purch-date expire-date date-time-stamp

 12345 12345 sony television   970101 1-year 980101 981201091501
 22222 11111 refrigerator      970101 2-year 990101 981201091502
 33333 22222 dish washer       961231 3-year 991231 981201103003
 33333 11111 refrigerator      970101 3-year 000101 981201103004
 44444 33333 freezer           970101 4-year 010101 981201124505
 55555 55555 clothes dryer     971231 5-year 021231 981201124506
 66666 66666 blank/zero dates         6-year 000000 981201125507

demo data file - after date EXPANSION

cust# prod# description purch-date expire-date date-time-stamp

 12345 12345 sony television   19970101 1-year 19980101 19981201091501
 22222 11111 refrigerator      19970101 2-year 19990101 19981201091502
 33333 22222 dish CRASHER      19961231 3-year 19991231 19981201103003
 33333 11111 refrigerator      19970101 3-year 20000101 19981201103004
 44444 33333 freezer           19970101 4-year 20010101 19981201124505
 55555 55555 clothes SHRINKER  19971231 5-year 20021231 19981201124506
 66666 66666 blank/zero dates           6-year 20000000 19981201125507
 77777 77777 extra rec file2   19980101 7-year 20050101 19980101164559XX

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6F2. comparing data files Field by Field based on COBOL copybooks

cobmap for ORIGINAL data file before expansion

 cobmap1  start-end bytes for cobol record fields    199906061246  pg# 0001
 cpy1/warmas6                                      RCSZ=0064  bgn-end lth typ
 *warmas6 - warranty master record (warmas1 with date-time vs
     05 wm-cust             pic  x(6).                        000-0005 006
     05 wm-prod             pic  x(6).                        006-0011 006
     05 wm-descrip          pic  x(18).                       012-0029 018
     05 wm-purch-date       pic  x(6).                        030-0035 006
     05 wm-policy           pic  x(8).                        036-0043 008
     05 wm-expiry.
        10 wm-exp-year      pic  9(2).                        044-0045 002 n  02
        10 wm-exp-month     pic  9(2).                        046-0047 002 n  02
        10 wm-exp-day       pic  9(2).                        048-0049 002 n  02
     05 filler001           pic  x(1).                        050-0050 001
     05 wm-date-time        pic  x(12).                       051-0062 012
     05 filler002           pic  x(1).                        063-0063 001
 *RCSZ=0064                                                       0064

cobmap1 for test file - after date EXPANSION

 cobmap1  start-end bytes for cobol record fields    199906061246  pg# 0001
 cpy2/warmas6                                      RCSZ=0070  bgn-end lth typ
 *warmas6x - warmas6 expanded date,year,date-time stamp
     05 wm-cust             pic  x(6).                        000-0005 006
     05 wm-prod             pic  x(6).                        006-0011 006
     05 wm-descrip          pic  x(18).                       012-0029 018
     05 wm-purch-date       pic  x(8).   <-- 8 bytes vs 6     030-0037 008
     05 wm-policy           pic  x(8).                        038-0045 008
     05 wm-expiry.
        10 wm-exp-year      pic  9(4).   <-- 4 bytes vs 2     046-0049 004 n  04
        10 wm-exp-month     pic  9(2).                        050-0051 002 n  02
        10 wm-exp-day       pic  9(2).                        052-0053 002 n  02
     05 filler001           pic  x(1).                        054-0054 001
     05 wm-date-time        pic  x(14).  <-- 14 bytes vs 12   055-0068 014
     05 filler002           pic  x(1).                        069-0069 001
 *RCSZ=0070                                                       0070

NOTES

The records have been expanded from 64 bytes to 70 bytes, since there were 3 date fields expanded by 2 bytes each.

This forces the record display onto 2 lines, since records are displayed in 64 byte segments.

See the sample report 2 pages ahead --->

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6F3. comparing data files Field by Field based on COBOL copybooks

sample uvcopy job to compare 2 data files

The previous pages showed demo data files & corresponding copybooks, that we are using to illustrate the 'uvcmp6' utility, which automatically generates a uvcopy job (interpretive code) to compare data files, field by field. This allows disabling fields that should not be compared (date/time stamps).

This also allows comparing records of different layouts, which is required due to date field expansion. This is illustrated on the next page --->

sample uvcopy job to compare 2 data files

 opr='JOBNAME warmas6 - generated by cobmap1,uvcmp6,uvdata52'
 rop=j64000k16000,uop=q1p30s3t1000v0
 was=a8000b8000c8000d8000e8000f8000g8000h8000i8000j8000
 fili1=FILEIN1,rcs=0064,typ=RSF
 fili2=FILEIN2,rcs=0070,typ=RSF
 filo3=rptcmp/FILEOUT,rcs=128,typ=LSTt
 @run
        bal    cmpa                      open files, write report headers
 # begin loop to get each record pair, compare, report(if mismatch), until EOF
 loop   bal    cmpb                      get record pair & init possible output
        skp>   eof
        bal    cmpc,'a0000b0030c0000d0030','wm-cust : wm-descrip'
        bal    cmpc,'a0030b0006c0032d0006','wm-purch-date c0030d0008'
        bal    cmpc,'a0036b0008c0038d0008','wm-policy'
        bal    cmpc,'a0044b0002c0048d0002','wm-exp-year c0046d0004'
        bal    cmpc,'a0046b0004c0050d0004','wm-exp-month : wm-exp-day'
        bal    cmpc,'a0050b0001c0054d0001','filler001'
 #2     bal    cmpc,'a0051b0012c0057d0012','wm-date-time c0055d0014'
        bal    cmpc,'a0063b0001c0069d0001','filler002'
        bal    cmpd                      output records if any mismatches
        skp    loop                      repeat loop until EOF
 # EOF - write reports stats, close files,& end job
 eof    bal    cmpe                      write rpt stats, close files, end job
 @pf2=uvcmp6.subs  #load(at run-time) common subrtns (cmpa,cmpb,cmpc,cmpd,cmpe)

Please see the sample report (data mismatches) on the next page --->

Note that this utility compares multiple contiguous fields of the same type with 1 'bal' instruction. The first & last fieldnames are shown in operand 3 on the right side (eg - 'wm-cust : wm-descript'). The copybook on the previous page shows you this includes wm-prod. The total length is coded as option 'b' (b0030) 30 bytes combined.

Uvcmp3 contains a table of fieldnames to force breaking contiguous compares. This table includes names such as 'date', 'year', etc & you can add to it.

A 2nd table is used to inhibit compares on fields such as date/time stamps. This table might include names such as 'time', 'stamp', etc.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6F4. comparing data files Field by Field based on COBOL copybooks

Here is a sample report created by the uvcopy job shown on the previous page (which was automatically generated from the copybook). Note that, we created some intentional mismatches by using the vi editor to change some data in one of the files (cmpdata/dat2/warmas6).

sample report - data mismatches

 uvcmp6 - compare 2 files & "*" differences (uops=q0p30s3t1000v0)
 record-size     count   file-size  typ  filename
 file1:   64         7         448  RSF  dat1/warmas6
 file2:   70         8*        561  RSF  dat2/warmas6
 1998/11/08_15:32:57   1         2         3         4         5         6
 f#rec#byte# 0123456789012345678901234567890123456789012345678901234567890123
 ============================================================================
 1    3    0 33333 22222 dish washer       961231 3-year 991231 981201103003.
                              *******             *       *
 2    3      33333 22222 dish CRASHER      19961231 2-year 19981231 199812333
                              *******               *         *
          64 33333.
 1    6    0 55555 55555 clothes dryer     971231 5-year 021231 981201124506.
                                 ********         *       *
 2    6      55555 55555 clothes SHRINKER  19971231 6-year 20031231 199812666
                                 ********           *         *
          64 66666.
 1    8    0 ~EOF~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
             ***************************************************            *
 2    8      77777 77777 extra rec file2   19981105 7-year 20000000 BADRECORD
             ******************************  **************  *******
          64 LENGTH
                  *
 ======================= EOF or stop count reached ==========================
 Total=000007, read=000009, mismatch=000004, stop=000003, term=000100

Please see the operating instructions for this demo on the next page --->

Since the records have been expanded to 70 bytes, it now requires 2 lines to display records from file #2 (records are displayed in 64 byte segments). Note that file2 has an extra record after EOF already reached in file2. If the extra record occurs any sooner we would have mismatches from there on.

The extra record also has wrong length (not 70 bytes like others in file2). This causes the '*' flag beside the file2 count - see '8*' in page headings. The count is flagged when filesize/recsize is not evenly divisible. For ISAM files the extra byte (delete control byte) is allowed for by adding 1 to the recsize before division.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6F5. comparing data files Field by Field based on COBOL copybooks

demo - expanded file comparison - preparation

The following files are supplied to demo uvcmp6 on different record sizes:

    uvadm/dat1/warmas6     - demo data-file supplied in 'dat1' directory
    uvadm/dat1/warmas6x    - same data file after dates expanded

uvadm/cobol/cpys/warmas6 - demo copy-book for the warmas6 data file above uvadm/cobol/cpys/warmas6x - demo copy-book for the expanded warmas6x above


 3a. cp /home/uvadm/cobol/cpys/warmas6 cpy1
     ======================================
     - copy demo copybook for original file to cpy1 subdir

 3b. cp /home/uvadm/cobol/cpys/warmas6x cpy2/warmas6
     ===============================================
     - change name of expanded copy-book in cpy2 to match cpy1

 4a. cp /home/uvadm/dat1/warmas6 dat1
     ================================
     - copy original test-data to dat1 subdir

 4b. cp /home/uvadm/dat1/warmas6x dat2/warmas6
     =========================================
     - change name of expanded data-file in dat2 to match dat1

See test/demo job executions on the next page --->

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6F6. comparing data files Field by Field based on COBOL copybooks

NOTE
  • this is a demo & generates 1 compare job at a time. For serious work
    please see page '6G1' + which presents the Operating Instructions
    to generate comparison jobs for all copybooks & all data files.
 <----- see preparations on the previous page.

Demo Operating Instructions


 1. uvcopy cobmap1,fili1=cpy1/warmas6,filo1=map1/warmas6
    ====================================================
                                - run cobmap1 for the original copybook

 2. uvcopy cobmap1,fili1=cpy2/warmas6,filo1=map2/warmas6
    ====================================================
                                - run cobmap1 for the new copybook

 3. uvcopy reform1,fili1=map1/warmas6,filo1=map1I/warmas6
    =====================================================
                                - load old map into an Indexed file

 4. uvcopy uvcmp6,fili1=map2/warmas6,filr1=map1I/warmas6,filo1=pfc1/warmas6\
    ========================================================================
                 ,arg2=e9999p30s3t9999v0    <-- defaults, no need to enter
                 =======================
    - generate uvcopy job to compare files
 5a. uvcopy uvdata52  - optional for demo (see pages '6G1'+)
                      - definitely use for production (serious work)
                      - copies jobs from pfc1 to pfc2 inserting datafilenames
                        using a control file to relate to copybooknames
     - - - - OR - - - -    <-- use 5b. if you do not use 5a.

 5b. cp pfc1/warmas6 pfc2  <-- copy job to alt subdir before modify/execute
     ====================    - to protect edits from inadvertent regen

 6. vi pfc2/warmas6         - modify generated job if required
    ===============         - add record-type tests for redefined fields
                            - might change data-file-names

 7. uvcopy pfc2/warmas6,fili1=dat1/warmas6,fili2=dat2/warmas6,filo3=rptcmp/warmas6
    ============================================================================
                              - execute the job to compare the data file

 7a. uvcopy pfc2/warmas6    <-- same as above (files default as shown)
     ===================      - OK if data-file-names match copy-book-names

 8a. vi rptcmp/warmas6        - inspect report for differences (* markers)
     =================

 8b. uvlp12 rptcmp/warmas6    - print report
     =====================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6F7. comparing data files Field by Field based on COBOL copybooks

Notes Re Executing Generated uvcmp6 Compare jobs

The previous page suggested executing the generated compare job as follows:


 uvcopy pfc2/warmas6,fili1=dat1/warmas6,fili2=dat2/warmas6,filo3=rptcmp/warmas6
 ==============================================================================

Be patient, we will eventually show you a better way begining on page '6G1'.

This long command is necessary at this point because the jobs in 'pfc2' have been generated using only the information in the copybooks, which does not include the actual data-file-names or Indexed key info.

If desired, you could edit the correct data filenames into the pfc2/jobs (replacing the generic FILEIN & FILEOUT), then you could run as follows:


 uvcopy pfc2/warmas6
 ===================

The next section '6G1' + provides a better solution, using a control file to relate the actual data-file-names & indexed key info to the copy-book info. The following illustrates 2 lines from the control file used in section '6G1'+.

 # ctlcmp1 - control file to generate complete jobs from skeletons
 #         - used by uvdata52 to encode data-file-names & ISAM keys
 warmas6        cpy=warmas6  rcs=00064
 warmas8        cpy=warmas8  rcs=00064 keys=(0,6,nn)

An additional job 'uvdata52' is provided to read the control file & for each data-file, randomly retrieve the corresponding skeleton uvcopy (previously generated from the copybook & now in the pfc1 subdir).

The actual filenames & any ISAM key info will be inserted into the skeleton jobs from pfc1 to create complete uvcopy jobs in the pfc2 subdir.

It is also important to realize that 1 copy-book could represent the data file record layouts for several data-files. The control file will have 1 line for each data-file, but the total number could be more than the total number of copy-books. In fact it is usually less because there are many other copy-books that do not represent any data-file.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6G1. comparing data files Field by Field based on COBOL copybooks

generate comparison jobs for All files in Directory

Previous operating instructions were demos for comparing 1 file at a time, but for industrial strength serious work, we want to generate the compare jobs for all copy-books & all data-files.

If you are ready real work, you will use your own directories of copybooks & data-files, but if you are not yet confident in using these techniques, I suggest you use first try these procedures using the supplied test files.

demo Op. Instrns. - Preparation

 1a. cd $HOME        - change to your home directory
 1b. mkdir cmpdata   - make a sub-directory for the test/demo
 1c. cd cmpdata      - change into it

make sub-directories

 2a. mkdir cpy1      - for original copybooks
 2b. mkdir cpy2      - for expanded copybooks
 2c. mkdir map1      - for original copybook 'cobmaps' (record layouts)
 2d. mkdir map2      - for new copybook cobmaps
 2e. mkdir map1I     - for original cobmaps loaded into indexed files
 2f. mkdir pfc1      - for generated uvcopy jobs (to compare files)
 2g. mkdir pfc2      - for uvcopy jobs (with filenames & indexed keys)
 2h. mkdir rptcmp    - for mismatch reports
 2i. mkdir ctl       - for control file (relate copybook to datafiles)
 2j. mkdir dat1      - for test data file inputs
 2k. mkdir dat2      - for test data file outputs
 2i. mkdir tmp       - subdir for misc outputs (keep working dir clean)

copy demo copybooks to cmpdata, duplicate,& modify


 3a. cp /home/uvadm/cobol/cpys/warmas*  cpy1
     =======================================
     - copy all warmas* copybooks to cmpdata/cpy1
       (warmas1,warmas1x,warmas3,warmas6,warmas6x,warmas8,warmas8x)
 3b.  move the 'x' (expanded versions) to cpy2, renaming (removing the 'x')
                                                 to match the names in cpy1.
 3b1. mv cpy1/warmas1x     cpy2/warmas1
 3b2. mv cpy1/warmas6x     cpy2/warmas6
 3b3. mv cpy1/warmas8x     cpy2/warmas8

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6G2. comparing data files Field by Field based on COBOL copybooks

copy demo data-files to cmpdata, duplicate,& modify


 4a. cp /home/uvadm/dat1/warmas* dat1  - copy warmas* datafiles to cmpdata/dat1
     ================================
 4b.  move the 'x' (expanded versions) to dat2, renaming (removing the 'x')
                                                 to match the names in dat1.
 4b1. mv dat1/warmas1x     dat2/warmas1
 4b2. mv dat1/warmas6x     dat2/warmas6
 4b3. mv dat1/warmas8x     dat2/warmas8
 4b4. mv dat1/warmas8x.idx dat2/warmas8.idx
Note
  • The 'x' versions have expanded dates with '19' inserted.
  • These were created automatically (from the non x versions)
    by 'reform2' (see REFORMjobs.doc).

 uvcp "fili1=dat2/warmas6,rcs=70,typ=RSF,filo1=dat2/warmas8,typ=ISF,isk1=0(6)"
 =============================================================================

copy control file & modify

 5a. cp /home/uvadm/ctl/ctlcmp1 ctl     - copy control file to cmpdata subdir

 5b. vi ctl/ctlcmp1                     - modify (remove extraneous files)
     ==============
 # ctlcmp1 - control file to generate complete jobs from skeletons
 #         - used by uvdata52 to encode data-file-names & ISAM keys
 #
 warmas1        cpy=warmas1  rcs=00064
 warmas3        cpy=warmas3  rcs=00064
 warmas6        cpy=warmas6  rcs=00064
 warmas8        cpy=warmas8  rcs=00064 keys=(0,6,nn)

The control file is input to the 'uvdata52' step (see next page), which completes the skeleton compare jobs created by 'uvcmp6', by inserting the actual data filenames & any indexed file keys.

Preparing the control file is the only non-automatic part of this project. An accurate control file is vital to your file comparison success.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6G3. comparing data files Field by Field based on COBOL copybooks

<--- this assumes you have completed the preparations on the previous page.

Operating Instructions - Generation


 1.  uvcopyx cobmap1 cpy1 map1 uop=q0p0i7
     ====================================
     - convert original copybooks into cobmap1 layouts

 2.  uvcopyx cobmap1 cpy2 map2 uop=q0p0i7
     ====================================
     - convert modified copybooks into cobmap1 layouts

 3.  uvcopyx reform1 map1 map1I uop=q0i7
     ===================================
     - convert original cobmap1 into ISAM files (for reform2)

 4.  uvcopyxr uvcmp6 map2 pfc1 map1I uop=q0i7g0c14,arg2=q1e9999p30s3t9999v0
     ======================================================================
     - generate uvcopy jobs to compare data-files

 5.  uvcopy uvdata52,fili1=ctl/ctlcmp1,fild2=pfc1,fild3=pfc2\
     ========================================================
 5a.       ,arg1=/u2/prod/dat1,arg2=/u2/prod/dat2   <- paths to data-files
           ======================================
 5b.       ,arg1=${UVDATA1},arg2=${UVDATA2}         <- alternative (see note)
           ================================

The fully generated jobs in subdir 'pfc2' are renamed for the datafiles, rather than the copybooks as were the skeleton jobs in 'pfc1'. This makes sense because there could be multiple data-files for 1 copybook.

See a "Sample Fully Generated Job" on the next page ---> See the "Execution Operating Instructions" 2 pages ahead --->

 uvcopy   pfc2/warmas6       - execute job any 1 job to compare 2 files
 uvcopyxx 'pfc2/*'            - execute all compare jobs in pfc2 directory

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6G4. comparing data files Field by Field based on COBOL copybooks

Alternative for duplicate fieldnames due to qualification

These procedures use 'reform1a' (vs 'reform1') & include an extra step 'reform1b'. An extra directory is required 'map2a'.


 1.  uvcopyx cobmap1 cpy1 map1 uop=q0p0i7
     ====================================
     - convert original copybooks into cobmap1 layouts

 2.  uvcopyx cobmap1 cpy2 map2 uop=q0p0i7
     ====================================
     - convert modified copybooks into cobmap1 layouts

 3.  uvcopyx reform1a map1 map1I uop=q0i7         <-- reform1a vs reform1
     ====================================
     - convert original cobmap1 into ISAM files (for reform2)

 3a. uvcopyxr reform1b map2 map2a map1I uop=q0i7  <-- reform1b creates map2a
     ===========================================

 4.  uvcopyxr uvcmp6 map2a pfc1 map1I uop=q0i7g0c14,arg2=q1e9999p30s3t9999v0
     =======================================================================
     - generate uvcopy jobs to compare data-files
Note
  • input subdir map2a (vs map2 above)

 5.  uvcopy uvdata52,fili1=ctl/ctlcmp1,fild2=pfc1,fild3=pfc2
     ======================================================
reform1a
  • converts map1 files to ISAM files keyed on fieldnames
  • appends sequence#s -02, -03, etc on duplicate fieldnames
    found within any 1 copybook.
reform1b
  • copies map2 to map2a appending seq#s -02, -03, etc
    to match the corresponding fieldnames in map1I
note
  • if you need to rerun reform1b, you must 1st rerun reform1a
    to ensure fieldnames will match

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6G5. comparing data files Field by Field based on COBOL copybooks

Sample Fully Generated Job

The demo jobs generated on pages '6D1' + were incomplete, and filenames were supplied on the command line at execution time. This is OK for a demo, but not good if you have 100 file compare jobs to run.

The comparison jobs created in this section '6G1' + (see prior page) are complete, including actual data filenames & indexed keys if any. This is achieved by editing the 'control file' (see page '6G2'), and running an extra step 'uvdata52' which is driven by the control file to relate the copybook names to the actual datafile names.

This makes the jobs easier to run, just enter: 'uvcopy + jobname'.


 uvcopy pfc2/warmas8   <--- command to execute fully completed comparison job
 ===================

sample comparison job - completed by uvdata52

 opr='warmas8 warmas8 - generated by cobmap1,uvcmp6,uvdata52'
 rop=j64000k16000,uop=q1p30s3t1000v0
 was=a8000b8000c8000d8000e8000f8000g8000h8000i8000j8000
 fili1=${UVDATA1}/warmas8,rcs=0064,typ=ISFl1,isks=(0,6n)
 fili2=${UVDATA2}/warmas8,rcs=0070,typ=ISFl1,isks=(0,6n)
 filo3=rptcmp/warmas8,rcs=128,typ=LSTt
 @run
        bal    cmpa                      open files, write report headers
 # begin loop to get each record pair, compare, report(if mismatch), until EOF
 loop   bal    cmpb                      get record pair & init possible output
        skp>   eof
        bal    cmpc,'a0000b0030c0000d0030','wm-cust : wm-descrip'
        bal    cmpc,'a0030b0006c0032d0006','wm-purch-date c0030d0008'
        bal    cmpc,'a0036b0008c0038d0008','wm-policy'
        bal    cmpc,'a0000b0000c0048d0000','wm-exp-cent c0046d0002'
        bal    cmpc,'a0044b0002c0048d0002','wm-exp-year'
        bal    cmpc,'a0046b0004c0050d0004','wm-exp-month : wm-exp-day'
        bal    cmpc,'a0050b0001c0054d0001','filler001'
 #2     bal    cmpc,'a0051b0012c0057d0012','wm-date-time c0055d0014'
        bal    cmpc,'a0063b0001c0069d0001','filler002'
        bal    cmpd                      output records if any mismatches
        skp    loop                      repeat loop until EOF
 # EOF - write reports stats, close files,& end job
 eof    bal    cmpe                      write rpt stats, close files, end job
 @pf2=uvcmp6.subs  #load(at run-time) common subrtns (cmpa,cmpb,cmpc,cmpd,cmpe)

Compare this fully completed job to the demo job on page '6E3' and note:

  1. Actual data file names are now coded, instead of FILEIN1/FILEIN2/FILEOUT
  2. Indexed file typ=ISF & keys=(...) are inserted if indicated by the control file.
  3. Jobnames (in subdir pfc2) now match the datafilenames (from control file) rather than the copybooknames. This is necessary because there could be multiple data files for any 1 copybook.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6G6. comparing data files Field by Field based on COBOL copybooks

Executing the Fully Generated Jobs

To be completely flexible we should be able to execute our comparison jobs from anywhere. This can be achieved by environmental variables.

When you ran 'uvdata52' you were prompted for pathnames to the 2 sets of data files to be compared,& we suggested that you specify ${UVDATA1} & ${UVDATA2}. You would then add exports to your .profile, for example:


 export UVDATA1=$HOME/cmpdata/dat1; export UVDATA2=$HOME/cmpdata/dat2
 ====================================================================

Then during testing, you could modify the pathnames simply by exporting different values at your shell prompt (in effect until you logoff).

The other problem to be resolved is 'How does the uvcopy interpreter know where to find our generated comparison jobs ?'

Uvcopy provides environmental variable PFPATH to be searched. The provided profiles (see ADMjobs.htm#Part_1) includes the following:

export UV=/home/uvadm


 export PFPATH=$UV/pf/adm:$UV/pf/demo:$UV/pf/IBM:$UV/pf/util
 ===========================================================

If you have generated the comparison jobs in $HOME/cmpdata/pfc2, you could include that subdir in the PFPATH as follows:


 export PFPATH=$PFPATH:$HOME/cmpdata/pfc2
 ========================================

executing any 1 data file compare job

Simply enter 'uvcopy' (the interpreter) followed by the name of the parameter file (interpretive code / uvcopy job), which matches the name of the data file, for example:


 uvcopy custmast      - execute job to compare the pair of custmast files
 ===============        (cmpdata/dat1/custmast & cmpdata/dat2/custmast)

executing compare jobs for multiple data files

'uvcopyxx' is a script which will repeatedly execute the uvcopy interpreter for all parameter files found in the specified directory. uvcopyxx allows for pattern matching for, but you must specify the full pathname:


 uvcopyxx 'pfc2/*'                 - execute all compare jobs in pfc2 directory
 ===============                   (assuming you are in $HOME/cmpdata)

 uvcopyxx '$HOME/cmpdata/pfc2/gl*'  - execute all jobs for the GL system
 =================================    (regardless of current directory)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6G7. comparing data files Field by Field based on COBOL copybooks

Inspecting, Printing & Using the Mismatch Reports

The mismatch reports will be created in subdir 'rptcmp' within whatever directory you are in when you run the comparison jobs. I recommend you clean it out before you begin each new series of comparisons.


 vi rptcmp/gl.master       -inspect mismatch report for the gl.master file
 ===================

You can inspect the reports with vi, but I recommend printing them out, so you have your terminal free to inspect the data files (with uvhd) using the hard-copy mismatch reports as your guide.


 uvlpd12 rptcmp           - print all mismatch reports
 ==============

using uvhd to inspect data files

You will find uvhd an invaluable tool for inspecting your data files especially if you have packed/binary files without linefeeds. See details in uvhd.htm.


 uvhd dat1/warmas6 r64        - investigate problem file original
 =====================

 uvhd dat2/warmas6 r70        - investigate problem file expanded
 =====================

With uvhd you can inspect any record in the file, for example you might want to see the data records just prior to a mismatch report difference. You can search for any desired pattern (& restrict the search to specified columns).

You can also print out any desired records in hex similar to mismatch reports. For example, when you records you want to print, use the immediate print:

 i3     <-- print the next 3 records
 ==

See the sample uvhd printout on page '6B5'. The printout will go to the laser printer configured in the 'uvlp12' script. See the install.doc guide in volume 1 or see /home/uvadm/sf/util/uvlp12.

print VTOC reports, check record counts


 uvcopy vtocr1,fild1=dat1,filr1=ctl/ctlcmp1I  --> uvlp12 <- print at 12 cpi
 ===========================================

 uvcopy vtocr1,fild1=dat2,filr1=ctl/ctlcmp2I  --> uvlp12 <- print at 12 cpi
 ===========================================

See sample VTOC reports on page '5G1'.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6I1. comparing data files Field by Field based on COBOL copybooks

first test/demo

I recommend you initially work your way thru the 1 file at a time demos begining on page '6B1', to get familiar with these procedures.

First run the demos with the demo copybooks & data files supplied in ~uvadm, then try them with some of your own copybooks & data files. For these you, can use the 'gencmp3a' script (see page '6X3') to automatically perform the multiple steps required to generate the comparison job.

If you only have a few data files to compare you can stick with gencmp3a.

generating comparison jobs for ALL copybooks & ALL data-files

If you have more than a few data files to compare, I recommend using the operating instructions begining on page '6G1' to generate complete jobs for all copybooks & all data files.

The instructions for ALL are in fact shorter than for 1 at a time & it is very fast to generate the uvcopy comparison jobs for hundreds of files.

The only manual editing required is to prepare the control file which relates the copybook names to the actual data file names, record sizes,& indexed file keys (if applicable).

Generating the comparison jobs for ALL files is very convenient , because you can then run comparisons whenever required during your testing.

For your first mass generation, I recommend you follow the step by step instructions begining on page '6G1'. The 'gencmp3b' script might be used if you have to subsequently regenerate all jobs due to changes in copybooks. If you are only changing the control file, you could simply run the 'uvdata52' job (which is the last step of generation).

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6J0. comparing data files Field by Field based on COBOL copybooks


6J1. summary of functions performed by called subroutines.

6J2. alternate subrtn 'cmpp' vs 'cmp. This allows you to compare data files
from the mainframe to data files created on UNIX.
 6J3. To split combined fields to eliminate subfields, not to be compared
    - The generation job contains a table of fieldnames that force separate
      'bal' compare instructions. This table contains date,year,yy, etc.
    - A 2nd table contains patterns that cause the 'bal' instruction to be
      #commented out (date/time stamps, that will not compare).
    - You can modify these tables (within ~uvadm/pf/util/uvcmp6), but it will
      never be perfect & you may want to split combined fields, or modify code
      that compares different length fields due to date expansion.

6J4. Option 'g1' could have been used to generate separate instructions for
each copybook/data field, but you might not want to use it because the
generated jobs would be much longer, especially for big copybooks with
hundreds of fields.
 6K1. Inserting record-type tests for files with redefined records.
    - Some files have multiple record layouts depending on record type codes.
      The code to compare the multiple sets of record fields will have been
      generated, but you need to insert code to test the record type & skip
      to the proper set of generated bal instructions.
 6L1. Corrections for occurs clauses with mixed data types.
    - Under these circumstances uvcmp6 will generate code only for the 1st
      set of occurs fields. You may have to insert a code loop to test
      all fields in the occurs set.

Please see these modifications illustrated on the following pages --->

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6J1. comparing data files Field by Field based on COBOL copybooks

'uvcmp6' & 'uvcmp6.subs' vs generated worker jobs

'uvcmp6' generates a worker uvcopy job for each pair of copybook/maps. The generated jobs have 'bal' instructions to compare the records field by field.

The 'bal's (branch & link) to subroutines to perform the various functions required in the comparison job. These subrtns are loaded at execution time from ~uvadm/pf/util/uvcmp6.subs (stored in Vancouver Utility home directory).

This is advantageous since it keeps the comparison worker jobs to the absolute minimum no of lines & allows any changes to the common subrtns to be made once to the master copy.

There are 5 subrtns called by the comparison jobs:

  1. cmpa - begin job, open files, create report headers, etc

  2. cmpb - get each pair of records, clear mismatch *mark areas

  3. cmpc - compare each pair of fields within records, if mismatch: store *markers in mismatch relative position
 3a. cmpp  - alternate subrtn 'cmpp' vs 'cmp', to compare data files from
             the mainframe to data files created on UNIX.
  1. cmpd - at end of each record cycle, test for any mismatches if mismatch: print out record pair & mismatch *markers

  2. cmpe - at end of file, print count stats & close files

These subrtns are stored in /home/uvadm/pf/util/uvcmp6.subs. You probably wont have to change them, since you can usually achieve the desired effect by modifying the worker jobs that call these subrtns.

If you do require changes, please call UV Software for assistance.

The uvcmp6 master generation job is not listed here, but is available in /home/uvadm/pf/util/uvcmp6. This of course is more complex and call UV Software if you think you need to change something.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6J1. comparing data files Field by Field based on COBOL copybooks

alternate subrtn 'cmpp' vs 'cmpc'

This allows you to compare data files from the mainframe to data files created on UNIX. Mainframes generated x'_C' signs in arithmetic operations, but Micro Focus COBOL generates C signs for comp-3 signed, and F signs for comp-3 unsigned.

Subrtn cmpp (alternate to cmpc) considers packed field signs C or F to match equal. Bytes with either x'_C' or x'_F' are assumed to be part of packed fields & match equal if the units digit matches. Uvcmp3 always generates 'bal cmpc' for all fields, and you can simply change these to 'bal cmpp' if you wish packed signs C/F to be considered a match.

For example, demo file custmas1 has 12 * 5 = 60 bytes packed fields for this year & last year sales, and generates the following bal's:

        bal    cmpc,'a0000b0006c0000d0006','cm-num'
        bal    cmpc,'a0006b0114c0006d0114','cm-delete : cm-contact'
        bal    cmpc,'a0120b0060c0120d0060','cm-thisyr-sales occur 12'
        bal    cmpc,'a0180b0060c0180d0060','cm-lastyr-sales occur 12'
        bal    cmpc,'a0250b0006c0250d0006','filler003'

You can simply change the bal's for the packed fields as follows:

        bal    cmpp,'a0120b0060c0120d0060','cm-thisyr-sales occur 12'
        bal    cmpp,'a0180b0060c0180d0060','cm-lastyr-sales occur 12'

Actually, you can change all the 'cmpc's to 'cmpp's. It will run a little slower, but you will save a lot of time not having to determine which 'bal's have packed fields.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6J3. comparing data files Field by Field based on COBOL copybooks

sample modification - splitting/disabling fields

Here is just the 'bal' compare instructions extracted from the complete job, that was listed on page '6E3'.

        bal    cmpc,'a0000b0030c0000d0030','wm-cust : wm-descrip'
        bal    cmpc,'a0030b0006c0032d0006','wm-purch-date c0030d0008'
        bal    cmpc,'a0036b0008c0038d0008','wm-policy'
        bal    cmpc,'a0044b0002c0048d0002','wm-exp-year c0046d0004'
        bal    cmpc,'a0046b0004c0050d0004','wm-exp-month : wm-exp-day'
        bal    cmpc,'a0050b0001c0054d0001','filler001'
 #2     bal    cmpc,'a0051b0012c0057d0012','wm-date-time c0055d0014'
        bal    cmpc,'a0063b0001c0069d0001','filler002'

Referring to the copybook (listed on page '6E2') you can see that the first instruction compares 3 contiguous fields for a total of 30 bytes.

     05 wm-cust             pic  x(6).                        006    000-0005
     05 wm-prod             pic  x(6).                        006    006-0011
     05 wm-descrip          pic  x(18).                       018    012-0029
        bal    cmpc,'a0000b0030c0000d0030','wm-cust : wm-descrip'
        bal    cmpc,'a____b____c____d____','1stfield : lastfield'
                     ---file1--|--file2--
                    '-dspl-lth-|-dspl-lth'

The above should clarify the coding which uses option letters a,b,c,d. (note that displacements are zero relative)

option 'a' = displacement to first byte of current field in file #1 option 'b' = length of current field in file #1 option 'c' = displacement to first byte of current field in file #2 option 'b' = length of current field in file #2

EXAMPLE
  • suppose you need to eliminate the comparison on wm-prod
    (as if it were a date/time stamp)
Solution
  • You could recode the comparisons on these 3 fields as follows:
        bal    cmpc,'a0000b0006c0000d0006','wm-cust'
 ###    bal    cmpc,'a0006b0006c0006d0006','wm-prod'
        bal    cmpc,'a0012b0018c0012d0018','wm-descrip'

Wm-prod has been commented out (using ### for emphasis). Of course you could simply omit it, but #commenting out allows easy future changes.

OR you could have generated 1 instruction per field (via option g1). See illustration on the next page --->

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6J4. comparing data files Field by Field based on COBOL copybooks

OPTION to generate SEPARATE INSTRUCTION for each field

   g0      - combine contiguous same type fields into 1 compare (default)
             (breaks on fieldname matches to table date/time)
   g1      - generate separate bal/cmpc instrns for each field

Option 'g0' is the default & you can see the resulting separate instructions on page '6B2'. If you specified *'g1'* the results would be as follows:

 opr='JOBNAME warmas6 - generated by cobmap1,uvcmp6,uvdata52'
 rop=j64000k16000,uop=q1p30s3t1000v0
 was=a8000b8000c8000d8000e8000f8000g8000h8000i8000j8000
 fili1=FILEIN1,rcs=0064,typ=RSF
 fili2=FILEIN2,rcs=0070,typ=RSF
 filo3=rptcmp/FILEOUT,rcs=128,typ=LSTt
 @run
        bal    cmpa                      open files, write report headers
 # begin loop to get each record pair, compare, report(if mismatch), until EOF
 loop   bal    cmpb                      get record pair & init possible output
        skp>   eof
        bal    cmpc,'a0000b0006c0000d0006','wm-cust'
        bal    cmpc,'a0006b0006c0006d0006','wm-prod'
        bal    cmpc,'a0012b0018c0012d0018','wm-descrip'
        bal    cmpc,'a0030b0006c0032d0006','wm-purch-date c0030d0008'
        bal    cmpc,'a0036b0008c0038d0008','wm-policy'
        bal    cmpc,'a0044b0002c0048d0002','wm-exp-year c0046d0004'
        bal    cmpc,'a0046b0002c0050d0002','wm-exp-month'
        bal    cmpc,'a0048b0002c0052d0002','wm-exp-day'
        bal    cmpc,'a0050b0001c0054d0001','filler001'
 #2     bal    cmpc,'a0051b0012c0057d0012','wm-date-time c0055d0014'
        bal    cmpc,'a0063b0001c0069d0001','filler002'
        bal    cmpd                      output records if any mismatches
        skp    loop                      repeat loop until EOF
 # EOF - write reports stats, close files,& end job
 eof    bal    cmpe                      write rpt stats, close files, end job
 @pf2=uvcmp6.subs  #load(at run-time) common subrtns (cmpa,cmpb,cmpc,cmpd,cmpe)
        bal    cmpc,'a0000b0030c0000d0030','wm-cust : wm-descrip'

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6K1. comparing data files Field by Field based on COBOL copybooks

Comparing files with Multiple Record Types

In many cases, you will not have to modify the compare jobs, for files with multiple record types. The compare jobs are less likely to need modifications than are the other date processing jobs (REFORMjobs, AGEjobs, & VALjobs). This is because we can usually compare the entire record and we do not have to process the dates individually as for the other jobs. Even if there are multiple record types, we can usually compare the entire record & not have to care that some have dates & some do not.

There are 2 situations where we would need to test record types:

  1. When we are comparing unexpanded files to expanded files - uvcmp6 generates instructions to compare all 6 digits of expanded dates to the right most 6 digits of the 8 digit expanded dates. - expanded files with multiple record types obviously require added code

  2. When the files are the same record size, but some record types have time-stamps (to be bypassed) & other record types do not, or they are in different locations.

We will illustrate the 2nd situation on the next few pages using the warmas3 file, which has 2 record types (dtl & note). Let's pretend that the expiry date is a time stamp & cannot be compared whereas the entire note record should be compared.

 12345 12345 sony television   970101 1-year 980101 000002500
 12345 12345*warmas3 alternate warmas file, demo multi rec types
 22222 11111 refrigerator      970101 2-year 990101 000004500
 22222 22222 dish washer       971231 2-year 991231 000004500
 22222 22222*rectype col 12, blank for detail, '*' for note recs
 33333 11111 refrigerator      970101 3-year 000101 000006500
 33333 11111*special 3 yr warranty to John Smith GE refrigerator

copybook for warmas3 (multiple record types)

 cobmap1  start-end bytes for cobol record fields    199909151419  pg# 0001
 cpy1/warmas3                   wm-rec             RCSZ=0064  bgn-end lth typ
  01  wm-rec.
      05 wm-cust             pic  x(6).                       000-0005 006
      05 wm-prod             pic  x(5).                       006-0010 005
      05 wm-type             pic  x(1).  <-- Record Type      011-0011 001
      05 wm-dtl.
         10 wm-descrip          pic  x(18).                   012-0029 018
         10 wm-purch-date       pic  x(6).                    030-0035 006
         10 wm-policy           pic  x(8).                    036-0043 008
         10 wm-expiry_date      pic  x(6). <-- time-stamp ?   044-0049 006
         10 filler001           pic  x(1).                    050-0050 001
         10 wm-paid-amt         pic  9(9).                    051-0059 009 n  09
         10 filler002           pic  x(4).                    060-0063 004
      05 wm-note redefines wm-dtl.
         10 wm-comment          pic  x(52).                   012-0063 052
 *RCSZ=0064                                                       0064

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6K2. comparing data files Field by Field based on COBOL copybooks

uvcopy job - BEFORE record type changes

 opr='warmas3 warmas3 - generated by cobmap,uvcmp6,uvdata52'
 arg1=warmas3,uop=q1p30s3t10000v0
 was=a8000b8000c8000d8000e8000f8000g8000h8000i8000j8000
 fili1=${UVDATA1}/warmas3,rcs=0064,typ=RSF
 fili2=${UVDATA2}/warmas3,rcs=0064,typ=RSF
 filo3=${CMPRPTS}/warmas3,rcs=128,typ=LSTt
 @run
        bal    cmpa                      open files, write report headers
 # begin loop to get each record pair, compare, report(if mismatch), until EOF
 loop   bal    cmpb                      get record pair & init possible output
        skp>   eof
 #      ---                       <-- R/T tests required ?
        bal    cmpc,'a0000b0030c0000d0030','wm-cust : wm-descrip'
        bal    cmpc,'a0030b0006c0030d0006','wm-purch-date'
        bal    cmpc,'a0036b0008c0036d0008','wm-policy'
        bal    cmpc,'a0044b0006c0044d0006','wm-expiry_date'
        bal    cmpc,'a0050b0014c0050d0014','filler001 : filler002'
        skp    put1
 #      ---                         <-- redefined record, R/T code reqd?
        bal    cmpc,'a0012b0052c0012d0052','wm-comment'
        skp    put1
 #----------------------------------------------------------------------
 cmpall bal    cmpc,'a0b0064c0d0064','allrecord'  (use if no dates)
 put1   bal    cmpd                      report if any mismatches
        skp    loop                      repeat loop until EOF
 #EOF - write reports stats, close files,& end job
 eof    bal    cmpe                      write rpt stats, close files, end job
 @pf2=uvcmp6.subs  #load(at run-time) common subrtns (cmpa,cmpb,cmpc,cmpd,cmpe)

Note that different instructions are generated for the different record definitions and it is simply a matter of inserting some code to test the record type & conditionally execute the proper set of instructions.

See the modified job listed on the next page --->

Note that only bytes 12-63 (52 bytes) of the comment record will be compared, since only bytes 12-63 were redefined in the copybook.

Also note that uvcmp6 has generated the 'unixcmpall' instruction for just such situations. On the next page, we could change 'skp= note' to 'skp= cmpall' & eliminate or just ignore the 'bal' generated for the comment record.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6K3. comparing data files Field by Field based on COBOL copybooks

uvcopy job - AFTER inserting code to test record-type

 opr='warmas3 warmas3 - generated by cobmap,uvcmp6,uvdata52'
 arg1=warmas3,uop=q1p30s3t10000v0
 was=a8000b8000c8000d8000e8000f8000g8000h8000i8000j8000
 fili1=${UVDATA1}/warmas3,rcs=0064,typ=RSF
 fili2=${UVDATA2}/warmas3,rcs=0064,typ=RSF
 filo3=${CMPRPTS}/warmas3,rcs=128,typ=LSTt
 @run
        bal    cmpa                      open files, write report headers
 # begin loop to get each record pair, compare, report(if mismatch), until EOF
 loop   bal    cmpb                      get record pair & init possible output
        skp>   eof
 #-------------------------------------------------------------------------
        cmc    a11(1),' '           detail record ?                <-INSERT
        skp=   dtl                                                 <-INSERT
        cmc    a11(1),'*'           note record ?                  <-INSERT
        skp=   note                                                <-INSERT
        msgw   'Invalid record type - enter to copy unchanged ?'   <-INSERT
        skp    put1                                                <-INSERT
 #-------------------------------------------------------------------------
 dtl    bal    cmpc,'a0000b0030c0000d0030','wm-cust : wm-descrip'
        bal    cmpc,'a0030b0006c0030d0006','wm-purch-date'
        bal    cmpc,'a0036b0008c0036d0008','wm-policy'
 ###    bal    cmpc,'a0044b0006c0044d0006','wm-expiry_date'<-disable TIME-STAMP
        bal    cmpc,'a0050b0014c0050d0014','filler001-002'
        skp    put1
 #
 note   bal    cmpc,'a0012b0052c0012d0052','wm-comment'
        skp    put1
 #----------------------------------------------------------------------
 cmpall bal    cmpc,'a0b0064c0d0064','allrecord'  (use if no dates)
 put1   bal    cmpd                      report if any mismatches
        skp    loop                      repeat loop until EOF
 #EOF - write reports stats, close files,& end job
 eof    bal    cmpe                      write rpt stats, close files, end job
 @pf2=uvcmp6.subs  #load(at run-time) common subrtns (cmpa,cmpb,cmpc,cmpd,cmpe)

We inserted 6 instructions & added 2 tags to already generated instructions.

We ### commented out the compare for wm-expiry-date, which we are pretending is a time-stamp. In fact this would be automatically #commented out if the fieldname included 'time'.

In more complex files, there might be 3 or more record types. However it is still a matter of inserting the record type tests - all the code to compare the different record type fields would already be generated.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6L1. comparing data files Field by Field based on COBOL copybooks

Modifications Required for Dates in OCCURS

Dates within occurs is not usually a problem for these 'CMPjobs', as they were for other date processing jobs (REFORMjobs, AGEjobs, VALjobs). This is because we can simply compare the entire occurs area, without having to process dates individually as we do for REFORMjobs, AGEjobs,& VALjobs.

Dates in occurs would be a problem in the following 2 situations:

1 - Comparing unexpanded files with 6 digit dates to expanded files with 8 digit dates. 2 - When the occurs fields also contains other fields which must not be compared (usually time-stamps).

The solution for both of these is add a register loop to avoid having to code a 'bal' instruction for each occurrence. We will illustrate the solution using the warmat1 test file, which has an occurs 4 containing a date & policy type. Let's assume that the policy type is a time stamp.

Here is the copybook/cobmap for our test file. We will compare 2 files using the same file layout (with unexpanded 6 digit dates).

map1/warmat1 - copybook to demo dates in occurs

 cobmap1  start-end bytes for cobol record fields    199905130550  pg# 0001
 cpy1/warmat1                                      RCSZ=0100  bgn-end lth typ
 *warmas1 - warranty master record
     05 wm-cust             pic  x(6).                        000-0005 006
     05 wm-prod             pic  x(6).                        006-0011 006
     05 wm-descrip          pic  x(18).                       012-0029 018
     05 wm-policy occurs 4.
        07 wm-policy-date   pic  x(6).                        030-0035 006
        07 wm-policy-type   pic  x(8).                        036-0043 008
     05 wm-expiry.
        07 wm-exp-year      pic  9(2).                        086-0087 002 n  02
        07 wm-exp-month     pic  9(2).                        088-0089 002 n  02
        07 wm-exp-day       pic  9(2).                        090-0091 002 n  02
     05 wm-paid-amt         pic  9(7).                        092-0098 007 n  07
     05 filler001           pic  x(1).                        099-0099 001
 *RCSZ=0100                                                       0100

Note that only the 1st set of occurs fields are shown in the cobmap. The 2nd,3rd,4th sets would be in bytes 44-57, 58-71,& 72-85.

When you add your code loop, I suggest you insert #comment lines as shown below to check your calculations.

# occurs 4 * (6 date + 8 type) = 56 + 30 start = 86 end

Remember that for this demo, we are pretending that the policy type is a time-stamp field which must not be compared.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6L2. comparing data files Field by Field based on COBOL copybooks

Modifications Required for Dates in OCCURS

Here is the comparison report BEFORE & AFTER the modifications required.

The BEFORE report detects mismatches only within the 1st occurs group, because the cobmap1 utility maps only the 1st occurrence, to avoid long repetitive code in the cobmap reports.

Since the compare jobs are generated from the copybooks/cobamps, they will contain the code only to compare the 1st field of each occurs group. The next 2 pages illustrate the BEFORE & AFTER compare job for this file.

BEFORE Changes for Dates in Occurs

 uvcmp6 file comparison, Copybook=warmat1, uops=q1p30s3t10000v0q0i3
 recsize  reccount  dels   file-size  typ  filename
 1:  100         8     0         800  RSF  dat1/warmat1
 2:  100         8     0         800  RSF  dat2/warmat1
 1999/09/15_09:40:56   1         2         3         4         5         6
 f#rec#byte# 0123456789012345678901234567890123456789012345678901234567890123
 ============================================================================
 1    2    0 22222 11111 refrigerator      970101 21year 970201 22year 970301
                                               **   *
          64  23year 970401 24year 9901010004925.
 2    2      22222 11111 refrigerator      970199 21Xear 970288 22XXar 970377
                                               **   *
          64  23XXXr 970466 24XXXX 9901010004925.
 ======================= EOF or terminate count rea==========================
 Total=000008, read=000008, mismatch=000001, stop=000003, term=010000

AFTER Changes for Dates in Occurs

 uvcmp6 file comparison, Copybook=warmat1, uops=q1p30s3t10000v0
 recsize  reccount  dels   file-size  typ  filename
 1:  100         8     0         800  RSF  dat1/warmat1
 2:  100         8     0         800  RSF  dat2/warmat1
 1999/09/15_11:29:25   1         2         3         4         5         6
 f#rec#byte# 0123456789012345678901234567890123456789012345678901234567890123
 ============================================================================
 1    2    0 22222 11111 refrigerator      970101 21year 970201 22year 970301
                                               **            **            **
          64  23year 970401 24year 9901010004925.
                         **
 2    2      22222 11111 refrigerator      970199 21Xear 970288 22XXar 970377
                                               **            **            **
          64  23XXXr 970466 24XXXX 9901010004925.
                         **
 ======================= EOF or terminate count rea==========================
 Total=000008, read=000008, mismatch=000001, stop=000003, term=010000

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6L3. comparing data files Field by Field based on COBOL copybooks

pfc2/warmat1 - as generated BEFORE modifications

 opr='warmat1 warmat1 - generated by cobmap,uvcmp6,uvdata52'
 arg1=warmat1,uop=q1p30s3t10000v0
 was=a8000b8000c8000d8000e8000f8000g8000h8000i8000j8000
 fili1=${UVDATA1}/warmat1,rcs=0100,typ=RSF
 fili2=${UVDATA2}/warmat1,rcs=0100,typ=RSF
 filo3=${CMPRPTS}/warmat1,rcs=128,typ=LSTt
 @run
        bal    cmpa                      open files, write report headers
 # begin loop to get each record pair, compare, report(if mismatch), until EOF
 loop   bal    cmpb                      get record pair & init possible output
        skp>   eof
 #      ---                       <-- R/T tests required ?
        bal    cmpc,'a0000b0030c0000d0030','wm-cust : wm-descrip'
        bal    cmpc,'a0030b0006c0030d0006','wm-policy-date occur 04'
        bal    cmpc,'a0036b0008c0036d0008','wm-policy-type occur 04'
        bal    cmpc,'a0086b0002c0086d0002','wm-exp-year'
        bal    cmpc,'a0088b0012c0088d0012','wm-exp-month : filler001'
        skp    put1
 #----------------------------------------------------------------------
 cmpall bal    cmpc,'a0b0100c0d0100','allrecord'  (use if no dates)
 put1   bal    cmpd                      report if any mismatches
        skp    loop                      repeat loop until EOF
 #EOF - write reports stats, close files,& end job
 eof    bal    cmpe                      write rpt stats, close files, end job
 @pf2=uvcmp6.subs  #load(at run-time) common subrtns (cmpa,cmpb,cmpc,cmpd,cmpe)
WARNING
  • Do not modify the jobs directly in the pfc2 subdirectory, because
    they would get overwritten if you subsequently had to regenerate
    all jobs (due to mass changes to copybooks, option changes,
    enhancements to uvcmp6 generation jobs, etc).

Setup 'pfc3' for all jobs that require manual modifications.

pfc1
  • directory of uvcopy skeleton jobs generated from cobmap1s,
    before uvdata52 applies correct data filenames & ISAM keys
    using the control file.
pfc2
  • directory of completed uvcopy jobs.
  • DO NOT modify any jobs in this directory.
pfc3
  • COPY any jobs requiring manual changes to this directory
    & perform manual edits here in pfc3
  • Copy all pfc3 to pfc2 after any mass regens of pfc2.
    (may need to cut & paste if important changes in regens)
  • Could copy all jobs from pfc3 to pfc2 before executing
    (to execute all with 1 command or for standardization).

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6L4. comparing data files Field by Field based on COBOL copybooks

pfc3/warmat1 - AFTER modifications

 opr='warmat1 warmat1 - generated by cobmap,uvcmp6,uvdata52'
 arg1=warmat1,uop=q1p30s3t10000v0
 was=a8000b8000c8000d8000e8000f8000g8000h8000i8000j8000
 fili1=${UVDATA1}/warmat1,rcs=0100,typ=RSF
 fili2=${UVDATA2}/warmat1,rcs=0100,typ=RSF
 filo3=${CMPRPTS}/warmat1,rcs=128,typ=LSTt
 @run
        bal    cmpa                      open files, write report headers
 # begin loop to get each record pair, compare, report(if mismatch), until EOF
 loop   bal    cmpb                      get record pair & init possible output
        skp>   eof
 #      ---                       <-- R/T tests required ?
        bal    cmpc,'a0000b0030c0000d0030','wm-cust : wm-descrip'
 #----------------------------------------------------------------------
 ##     bal    cmpc,'a0030b0006c0030d0006'  <-- original date, see new below
 ##     bal    cmpc,'a0036b0008c0036d0008'  <-- original time, removed
 ##occurs with dates & time-stamps to be bypassed, original above, new below
 # - use rgstrs j & k with code loop to compare date only & bypass time
 # - occurs 4 * (6 byte date + 8 byte time) = 56 + 30 start = 86 end
        mvn    $rj,0              init rgstr j (file 1)
        mvn    $rk,0              init rgstr k (file 2)
 lup1   bal    cmpc,'aj30b6ck30d6','date occur 04'    <-- NOTE rgstrs J & k
        add    $rj,14             up file1 rgstr to next occurs
        add    $rk,14             up file2 rgstr to next occurs
        cmn    $rj,52             end 4 loops 4 * (6+8) = 52 ?
        skp<   lup1
 #----------------------------------------------------------------------
        bal    cmpc,'a0086b0002c0086d0002','wm-exp-year'
        bal    cmpc,'a0088b0012c0088d0012','wm-exp-month : filler001'
        skp    put1
 #----------------------------------------------------------------------
 cmpall bal    cmpc,'a0b0100c0d0100','allrecord'  (use if no dates)
 put1   bal    cmpd                      report if any mismatches
        skp    loop                      repeat loop until EOF
 #EOF - write reports stats, close files,& end job
 eof    bal    cmpe                      write rpt stats, close files, end job
 @pf2=uvcmp6.subs  #load(at run-time) common subrtns (cmpa,cmpb,cmpc,cmpd,cmpe)

Note that you must use registers 'j' & 'k' as shown on the following line:

lup1 bal cmpc,'aj30b6ck30d6','date occur 04' <-- NOTE rgstrs J & k

I recommend you insert #comment lines as shown above to check your math.

# occurs input 4 * (6 date + 8 type) = 56 + 30 start = 86 end

Please relate these calcs to the copybook shown on page '6L1'. These #comments are a big help to you or other people who may later need to verify that the job was done correctly.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6M1. uvcmp6 - Exceptions, Problems,& Solutions

option h,j,k for jobs generated by uvcmp6

For sequential (non-keyed) files, the uvcmp... jobs read both files sequentially & assume the matching record pairs can be compared.

For Indexed files, the uvcmp jobs read the 1st file sequentially, and chain to the 2nd file ('chain' means random read by key of 1st file).

Chaining (random read by key) allows for inserted records, which would be a problem if we were reading sequentially & assuming matching pairs.

However, if we have expanded dates in the first key of the 2nd file, we will have a problem using the unexpanded key of the first file to chain to the second file.

Options h,j,k are provided to solve this problem, by allowing for expanded dates prior to the key, and for inserting '19's for the century of any expanded dates within the key, before using the key to chain to the 2nd file.

option 'h'
  • bytes to shift key (in case of expansions prior to the key)
option 'j'
  • displacement to insert '19' for 1st expanded date in key
option 'k'
  • displacement to insert '19' for 2nd expanded date in key

To demo all options in use, assume the old record had 3 * 6 byte dates in the 1st 18 bytes 0(18) & that the key is date2+date3 6(12). Also assume that all dates have been expanded in the 2nd file to 24 bytes, so the key is now 8(16).

               date1,date2,date3.
                     <----key--->
 1st file key: 990122990222990322
               --date1,--date2,--date3.
                       <------key----->
 2nd file key: 199901221999022219990322

For this example, you would enter 'h0j8k16' at the options prompt when you run the 'pfa2' job generated by uvcmp6. This will shift the key up 2 bytes to compensate for the expansion of the 1st date (not part of the key), and insert '19' at bytes 8 & 16 before chaining to the 2nd file.


 uvcopy pfc2/filename,uop=h2j8k16
 ================================

You can enter the options on the command line as shown above, or even better (in case of reruns) edit the generated job to append these options on line 2.

 rop=j64000k16000,uop=q1p30s3t1000v0h2j8k16
                                    ^^^^^^^

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6M2. Exceptions, Problems,& Solutions

complement dates

Some sites use complement dates, to make transactions appear in date sequence.

complement date = 1000000 - yymmdd yymmdd = 1000000 - complement date

For Year2000 (with a window of 68) this was changed to:

complement date = 20680000 - yymmdd yymmdd = 20680000 - complement date

Complement dates are excluded from comparison by using the uvcmp6 generated jobs & inserting their fieldnames into the exclude table within uvcmp6. In the example above, the customer always coded the word 'pseudo' in the complement dates, so this word was added to the exclude table.

However, there is still a problem if the complement date is part of the first key of an Indexed file, which is normally used to chain to the matching record in the second file.

For these cases, the old original uvcmp1,5,6 jobs have been preserved with their names changed to uvcmp1X, uvcmp5X, & uvcmp6X.subs.

For this situation, we would be using the jobs generated by uvcmp6, which normally call uvcmp6.subs at the end of the generated code. All you have to do is change:

 @pf2=uvcmp6.subs
       -to-
 @pf2=uvcmp6X.subs

Of course this is a trade-off, these jobs will not now handle inserted records in the file, which is why the original jobs were changed to read the 1st file sequentially & use its key to chain to the 2nd file.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6N1. uvhdcob/diff script to compare 2 datafiles

The 'uvhdcob' utility & 'uvhddif2' script can be used to compare 2 datafiles & show the differences with the corresponding COBOL copybook names.

Please see complete details in uvhdcob.doc of UVdoc.

                   ** sample output report **
 1c1
 < dat1/warmas1 fsz=576 rsz=64 totrecs=9 current=1
 ---
 > dat2/warmas1 fsz=576 rsz=64 totrecs=9 current=1
 34c34
 < wm-descrip                      12  29    dish washer
 ---
 > wm-descrip                      12  29    dish CRASHER
 76c76
 < wm-descrip                      12  29    clothes dryer
 ---
 > wm-descrip                      12  29    clothes SHRINKER

Note that this method of comparing data-files is not as exact as the uvcmp... jobs documented in CMPjobs.doc because:

  1. uvhdcob only displays the 1st 38 bytes of longer data-fields
  2. uvhdcob only displays the 1st field of occurs

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6O1. uvcmp6 - Comparing data fields field by field - APPLICATIONS

Year2000 19xx testing for Windows

  1. Assuming program library has been duplicated and updated for Year2000.
  2. Assuming a test set of data is available (19xx data input).
  3. Run old programs on 19xx data & store results (19xx data outputs from old/Y1K programs)

  4. Run new programs on same data & store results (19xx data outputs from new/Y2K programs)

  5. Compare: data results from Y1K programs with 19xx data inputs to: data results from Y2K programs with 19xx data inputs - to prove that Year2000 program changes have not broken anything.

    Year2000 20xx testing for Windows

  6. Assuming same programs as above.
  7. Create a new set of test data with dates aged into the 2000 boundary - see the dateage1 job in AGEjobs.doc
  8. Run old programs on 20xx data & store results (20xx data outputs from old/Y1K programs)

  9. Run new programs on same data & store results (20xx data outputs from new/Y2K programs)

  10. Compare: data results from Y1K programs with 20xx data inputs to: data results from Y2K programs with 20xx data inputs - to prove validity of Year 2000 program changes. - Y1K program outputs will be incorrect, but the differences should help you verify your Year2000 program changes.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6O2. uvcmp6 - Comparing data fields field by field - APPLICATIONS

Year2000 19xx testing for Expansion

  1. Assuming program library has been duplicated and updated for Year2000.
 2a. Assuming a test set of data is available (19xx data input).
 2b. Create a new set of test data with EXPANDED dates (19xx data input).
     - see the reform2 job in DATAcnvt1.htm.
  1. Run old programs on 19xx original data & store results (19xx outputs from old/Y1K programs, using 19xx ORIGINAL data)

  2. Run new programs on 19xx expanded data & store results (19xx outputs from new/Y2K programs, using 19xx EXPANDED data)

  3. Compare: data results from Y1K programs with 19xx original data to: data results from Y2K programs with 19xx expanded data - to prove that Year2000 program changes have not broken anything.

    Year2000 20xx testing for Expansion

  4. Assuming program library has been duplicated and updated for Year2000.
 2a. Create test data by aging dates into Year2000 (20xx unexpanded data input).
     - see the dateage1 job in AGEjobs.doc
 2b. Create 2nd set of test data with EXPANDED dates (20xx expanded data input).
     - see the reform2 job in DATAcnvt1.htm.
  1. Run old programs on 20xx aged data & store results (20xx outputs from old/Y1K programs, using 20xx unexpanded data)

  2. Run new programs on 20xx expanded data & store results (20xx outputs from new/Y2K programs, using 20xx EXPANDED data)

  3. Compare: data results from Y1K programs with 20xx unexpanded data to: data results from Y2K programs with 20xx expanded data - to prove validity of Year 2000 program changes. - Y1K program outputs will be incorrect, but the differences should help you verify your Year2000 program changes.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6O3. uvcmp6 - Comparing data fields field by field - APPLICATIONS

Mainframe to UNIX conversions

  1. Assuming programs have been transferred & converted to run on UNIX

  2. Assuming a test set of data available on the mainframe

  3. Run mainframe programs & save output data files
 4a. Transfer mainframe input test data to UNIX & convert
     - EBCDIC to ASCII, except for packed/binary, correct numeric signs
     - see DATAcnv1.htm

4b. Transfer mainframe output data to UNIX & convert (as for inputs)

  1. On UNIX, run converted test data thru converted programs & save outputs

  2. Compare output data files from UNIX programs to the transferred/converted data outputs from the Mainframe programs. - to prove the validity of both data & program conversions.

  3. Optionally, you could reconvert the already converted data (now ASCII) back to EBCDIC & compare to the transferred data files (before conversion) - to prove the validity of the conversion process (separate from the program conversion influences).

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6X0. Summary of scripts & uvcopy jobs used in Part 6

scripts used in Part 6 & listed here


6X1. genuvcmp6 - script to generate uvcmp6 job to compare 2 files field by field
- saves having to key 4 uvcopy jobs with long command lines
(cobmap1, cobmap1, reform1, uvcmp6)

summary of uvcopy jobs used in Part 6

uvcmp6
  • field by field comparison (vs record to record)
  • allows date/time stamps to be removed
  • allows comparing different size records (after Y2K expansion)
  • separate job per file, auto generated from the copybooks
  • most sophisticated of the 3 jobs, details in CMPjobs.doc
uvcmp6.subs
  • subroutines called by the worker jobs generated by uvcmp6
  • are listed here for your convenience in case you need to change
  • call UV Software if you need any help

The uvcopy jobs in this section are not listed here to save space. They are of course readily available in the /home/uvadm/pf/... directories and you can inspect them with the editor or list them as follows. Using uvcmp1 as an example:


 cd /home/uvadm         - change to uvadm home directory
 ==============
 vi pf/util/uvcmp6      - inspect uvcmp6 (or any uvcopy job desired)
 =================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6X3. scripts used in Part 6 & listed here

genuvcmp6 - generate jobs to compare all files in subdirs

 #!/bin/ksh
 # genuvcmp6 - Generate uvcopy job to compare 2 data files field by field
 #           - based on 2 COBOL copybooks
 #             (same or different if datafile#2 has different layout than #1)
 #           - see /home/uvadm/doc/CMPjobs.doc or www.uvsoftware.ca/cmpjobs.htm
 #
 # genuvcmp6 must run from a superdir containing subdirs
 # cpy1,cpy2,dat1,dat2,map1,map2,map1I,pfc1,pfc2,rptcmp,tmp)
 # cpy_=copybooks, map_=layouts from copybooks, pfc_=uvcopy jobs from cobmaps
 # dat1=datafile#1, dat2=datafile#2
 #
 # Multiple subdirs allow us to use the same filename for copybooks, layouts,
 # datafiles,& reports. This simplifies the generation & execution.
 #
 fn="$1"   #capture copybook filename (must be no .cpy extension)
 echo "genuvcmp6 using: cpy1,cpy2,dat1,dat2,map1,map2,map1I,pfc1,pfc2,rptcmp"
 if [[ (-f "cpy1/$fn") && (-f "cpy2/$fn") && ("$fn" != *.*) ]]; then :
    else echo "usage: genuvcmp6 CopyBookName"
         echo "       ======================"
         echo " copybook (arg1) must be stored in cpy1 & cpy2 (with no .extnsn)"
         exit 91; fi
 if [[ -d cpy1 && -d cpy2 && -d map1 && -d map2 && -d map1I && -d pfc1 ]]; then :
    else echo "subdirs missing (cpy1,cpy2,map1,map2,map1I,pfc1)"
         exit 92; fi
 #
 uvcopy cobmap1,fili1=cpy1/$fn,filo1=map1/$fn,uop=q0i7p0
 #======================================================
 uvcopy cobmap1,fili1=cpy2/$fn,filo1=map2/$fn,uop=q0i7p0
 #======================================================
 uvcopy reform1,fili1=map1/$fn,filo1=map1I/$fn,uop=q0i7
 #=====================================================
 uvcopy uvcmp6,fili1=map2/$fn,filr1=map1I/$fn,filo1=pfc1/$fn
 #==========================================================
 exit 0
 #

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Permuted Index of Keywords from ** Headings **

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

5G1 Aid: VTOC reports - aid file compares
0A3 Allow: scripts to run uvcmpI1 - allow Inserts but on 2 reports
4A2 Allow: scripts to run uvcmpI1 (allow Inserts in 1 file)
4A2 Allow: scripts to run uvcmpI2 (allow inserts in both files)
4X2 Allow: 'uvcmpFAI2' - allow inserts in both files on 1 report
0A2 Allowing: uvcmpI1 - Fixed Length allowing Inserts (but 2 reports)
0A2 Allowing: uvcmpI2 - Fixed Length allowing Inserts (on 1 report)
2E1 Allowing: uvcmpRDWAI2 compare RDW files allowing inserts/deletes
4H1 Allowing: compare RDW files allowing inserts/deletes
0A3 Allows: scripts to run uvcmpI2 - allows Inserts/Deletes on 1 report
1D3 Arguments: enter arguments on command line or at prompts
6D2 Arguments: specifying options & arguments on uvcmp6 command
6D2 Arguments: uvcmp6 prompt for arguments (default options for generated jobs)
1X1 Ascii: script 'uvcmpFA1' - compare Ascii Fixed length files
1H1 Ascii-->ebcdic: Execute jobs EBCDIC-->ASCII, ASCII-->EBCDIC,& compare
1H1 Ascii-->ebcdic: Generate jobs EBCDIC-->ASCII & ASCII-->EBCDIC
1H1 Ascii-->ebcdic: generate ASCII-->EBCDIC jobs - for ALL data-files
1D3 At: enter arguments on command line or at prompts

5G2 Beside: why the '*' beside record count
4A2 Both: scripts to run uvcmpI2 (allow inserts in both files)
4G1 Both: uvcmpFAI2 demo#3 - inserts in both files on 1 report
4G2 Both: uvcmpFAI2 report#2 - inserts both files, 1 report
4X2 Both: 'uvcmpFAI2' - allow inserts in both files on 1 report
0A3 Bu: scripts to run uvcmpI1 - allow Inserts but on 2 reports
0A2 But: uvcmpI1 - Fixed Length allowing Inserts (but 2 reports)

6B3 Char: sample report - data mismatches (CHAR only)
6B4 Char: sample report - data mismatches (CHAR & HEX)
6B4 Char: sample report - data mismatches (CHAR & HEX)
6G7 Check: print VTOC reports, check record counts
6J1 Cmpc: alternate subrtn 'cmpp' vs 'cmpc'
1B1 Cmpdata: Directory 'cmpdata' & subdirs for comparisons
5A2 Cmpdata: Directory 'cmpdata' & subdirs for comparisons
6G1 Cmpdata: copy demo copybooks to cmpdata, duplicate,& modify
6G2 Cmpdata: copy demo data-files to cmpdata, duplicate,& modify
6J1 Cmpp: alternate subrtn 'cmpp' vs 'cmpc'
6F2 Cobmap: for ORIGINAL data file before expansion
6B1 Cobmap1: (copybook record layout) for data file above
6F2 Cobmap1: for test file - after date EXPANSION
1D3 Command: enter arguments on command line or at prompts
1F2 Command: uvcopy command vs script equivalent
4D2 Command: script method - easier, shorter command
6D2 Command: specifying options & arguments on uvcmp6 command
4D2 Commands: uvcopy commands - hard way
0A2 Compare: uvcmp1 - compare Fixed length Sequential files
0A3 Compare: uvcmp2 - uvcopy job to compare RDW files
0A4 Compare: uvcmp5 - compare Directories of Fixed Seqntl files
1A1 Compare: uvcopy jobs to compare Sequential files
1B1 Compare: subdirs used to compare files
1F2 Compare: script to compare 2 Fixlth EBCDIC files
1H1 Compare: Compare ALL re-converted files to original EBCDIC files
1H1 Compare: Execute jobs EBCDIC-->ASCII, ASCII-->EBCDIC,& compare
1H1 Compare: uvcmp1all - compare all files in 2 directories
1X1 Compare: script 'uvcmpFA1' - compare Ascii Fixed length files
1X2 Compare: script 'uvcmpFE1' - compare EBCDIC Fixed length files
1X3 Compare: uvcmp1all - compare ALL files in 2 directories
2B1 Compare: subdirs used to compare varlth files
2C2 Compare: testRDWea - Alternate file for compare (with mismatches)
2D1 Compare: RDW files method #1 - use uvcmp2 directly
2E1 Compare: varlth files - script uvcmpRDW1
2E1 Compare: varlth files - script 'uvcmpRDW1' Op. Instrns.
2E1 Compare: uvcmpRDWAI2 compare RDW files allowing inserts/deletes
3B1 Compare: subdirs used to compare varlth files
3C3 Compare: testIDX81a - Alternate file for compare (with mismatches)
3D1 Compare: IDX files method #1 - use uvcmp3 directly
3E1 Compare: varlth files - script uvcmpIDX81
3E1 Compare: varlth files - script 'uvcmpIDX81' Op. Instrns.
4A1 Compare: uvcmpI1 - compare files that may have Inserted records
4A1 Compare: uvcmpI2 - compare files that may have Inserted records
4B2 Compare: subdirs used to compare files
4H1 Compare: RDW files allowing inserts/deletes
4H2 Compare: execute uvcmpRDWAI2 to compare 2 RDW files
4H3 Compare: uvcmpRDWEI2 - compare EBCDIC RDW files
4X1 Compare: uvcmpFAI1 - compare files that may have INSERTs
5A1 Compare: all files in 2 directories - Overview
5D1 Compare: uvcmp5 - compare 2 directories, creating reports in a 3rd
5X1 Compare: unixcmpall - compare all files in 2 subdirs
6B2 Compare: sample uvcopy job to compare 2 data files
6F3 Compare: sample uvcopy job to compare 2 data files
6F3 Compare: sample uvcopy job to compare 2 data files
6F7 Compare: Notes Re Executing Generated uvcmp6 Compare jobs
6G6 Compare: executing any 1 data file compare job
6G6 Compare: executing compare jobs for multiple data files
6X3 Compare: genuvcmp6 - generate jobs to compare all files in subdirs
5G1 Compares: VTOC reports - aid file compares
0A2 Comparing: uvcopy jobs for comparing data files
2A1 Comparing: variable length RDW files - Overview
3A1 Comparing: variable length IDXFORMAT3/8 files - Overview
6A1 Comparing: Comparing data files field by field - Overview
6K1 Comparing: Comparing files with Multiple Record Types
1A2 Comparison: scripts to run comparison jobs in Part 1
1A2 Comparison: scripts to run comparison jobs in other Parts
2D3 Comparison: testRDWe - sample comparison report
3D3 Comparison: testIDX81 - sample comparison report
6B1 Comparison: Field by Field comparison - Illustration
6F5 Comparison: demo - expanded file comparison - preparation
6G1 Comparison: generate comparison jobs for All files in Directory
6G5 Comparison: sample comparison job - completed by uvdata52
6I1 Comparison: generating comparison jobs for ALL copybooks & ALL data-files
0A4 Comparisons: uvcmp6 - generate jobs to perform field by field comparisons
1B1 Comparisons: Directory 'cmpdata' & subdirs for comparisons
5A2 Comparisons: Directory 'cmpdata' & subdirs for comparisons
6M2 Complement: dates
6G5 Completed: sample comparison job - completed by uvdata52
2D2 Console: log from uvcmp2 2 FixLth files written by uvhd
3D2 Console: log from uvcmp3 2 FixLth files written by uvhd
5F1 Console: uvcmp5all demo console output
5D2 Control: sample control file
5E1 Control: load control file
6G2 Control: copy control file & modify
2B2 Copy: setup subdirs & copy test/demo files
3B2 Copy: setup subdirs & copy test/demo files
5E1 Copy: test files to dat1 & modify in dat2
6G1 Copy: demo copybooks to cmpdata, duplicate,& modify
6G2 Copy: control file & modify
6G2 Copy: demo data-files to cmpdata, duplicate,& modify
6B1 Copybook: cobmap1 (copybook record layout) for data file above
6K1 Copybook: for warmas3 (multiple record types)
6L1 Copybook: map1/warmat1 - copybook to demo dates in occurs
6G1 Copybooks: copy demo copybooks to cmpdata, duplicate,& modify
6I1 Copybooks: generating comparison jobs for ALL copybooks & ALL data-files
5G2 Count: why the '*' beside record count
6G7 Counts: print VTOC reports, check record counts

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

1C1 Dat1: dat1/warmas1a - Alternate test file#2 with differences
1C1 Dat1: test file#1 /home/uvadm/dat1/warmas1
4C1 Dat1: dat1/warmas1Ia - test file#2 with diffs & inserts
4C1 Dat1: test file#1 /home/uvadm/dat1/warmas1I
5E1 Dat1: copy test files to dat1 & modify in dat2
5G1 Dat1: sample VTOC report for dat1
6B1 Dat1: demo data file /home/uvadm/dat1/warmas6
5E1 Dat2: copy test files to dat1 & modify in dat2
5G1 Dat2: sample VTOC report for dat2
5G2 Dat2: sample VTOC report for dat2
0A2 Data: uvcopy jobs for comparing data files
2A1 Data: sample variable length RDW data files
2C1 Data: testRDWe - sample VARIABLE length RDW data file
3A1 Data: sample variable length IDX data files
3C1 Data: testIDX81 - sample VARIABLE length IDX data file
5G1 Data: print VTOC reports for the 2 data directories
6A1 Data: Comparing data files field by field - Overview
6B1 Data: cobmap1 (copybook record layout) for data file above
6B1 Data: demo data file /home/uvadm/dat1/warmas6
6B2 Data: sample uvcopy job to compare 2 data files
6B3 Data: sample report - data mismatches (CHAR only)
6B4 Data: sample report - data mismatches (CHAR & HEX)
6B4 Data: sample report - data mismatches (CHAR & HEX)
6F1 Data: demo data file - ORIGINAL before expansion
6F1 Data: demo data file - after date EXPANSION
6F2 Data: cobmap for ORIGINAL data file before expansion
6F3 Data: sample uvcopy job to compare 2 data files
6F3 Data: sample uvcopy job to compare 2 data files
6F4 Data: sample report - data mismatches
6G6 Data: executing any 1 data file compare job
6G6 Data: executing compare jobs for multiple data files
6G7 Data: using uvhd to inspect data files
1H1 Data-files: Execute ALL jobs to convert ALL data-files
1H1 Data-files: generate ASCII-->EBCDIC jobs - for ALL data-files
6G2 Data-files: copy demo data-files to cmpdata, duplicate,& modify
6I1 Data-files: generating comparison jobs for ALL copybooks & ALL data-files
6F1 Date: Y2K 19xx DATE EXPANSION testing
6F1 Date: demo data file - after date EXPANSION
6F2 Date: cobmap1 for test file - after date EXPANSION
6L1 Date: map1/warmat1 - copybook to demo dates in occurs
6L1 Dates: Modifications Required for Dates in OCCURS
6L2 Dates: AFTER Changes for Dates in Occurs
6L2 Dates: BEFORE Changes for Dates in Occurs
6L2 Dates: Modifications Required for Dates in OCCURS
6M2 Dates: complement dates
6D2 Default: uvcmp6 prompt for arguments (default options for generated jobs)
0A3 Delete: scripts to run uvcmpI2 - allows Inserts/Deletes on 1 report
4G2 Delete: Options i1/i2 ignore inserts/deletes OR mismatched pairs
2E1 Deletes: uvcmpRDWAI2 compare RDW files allowing inserts/deletes
4H1 Deletes: compare RDW files allowing inserts/deletes
1D1 Demo: uvcmp1 demo#1 - Operating Instructions
1D2 Demo: uvcmp1 demo#1 - sample report
1E1 Demo: uvcmp1 demo#2 - Exclude options
1E1 Demo: uvcmp1 demo#2 Exclude - Operating Instructions
1E2 Demo: uvcmp1 demo#2 Exclude - sample report
1E3 Demo: uvcmp1 demo#2 Exclude - sample report Notes
1F1 Demo: script uvcmpFE1 demo
1F2 Demo: demo#3 - using script 'uvcmpFE1'
1G1 Demo: #4 - packed fields
1G2 Demo: #4 packed fields - Operating Instructions
2B2 Demo: setup subdirs & copy test/demo files
2F1 Demo: with supplied test file
3B2 Demo: setup subdirs & copy test/demo files
3F1 Demo: with supplied test file
4B1 Demo: preparation for uvcmpI1 test/demo
4D1 Demo: test/demo files used in Part 4
4D1 Demo: test/demo files used in previous Parts
4E1 Demo: uvcmpFAI1 demo#1 - inserted record in file#2
4E2 Demo: uvcmpI1 demo#1 - sample report
4F1 Demo: uvcmpFAI1 demo#2 - inserted record in file#2
4G1 Demo: uvcmpFAI2 demo#3 - inserts in both files on 1 report
5A2 Demo: preparation for uvcmp1 test/demo
5B1 Demo: unixcmpall - demo report
5B1 Demo: unixcmpall - test/demo
5C1 Demo: unixcmpsplit - demo report
5E1 Demo: uvcmp5 test/demo - preparation
5F1 Demo: uvcmp5all demo console output
5F1 Demo: uvcmp5all test/demo - execution
6B1 Demo: data file /home/uvadm/dat1/warmas6
6C1 Demo: test/demo - preparation (1 at a time)
6D1 Demo: Demo Op. Instructions (1 file at a time)
6F1 Demo: data file - ORIGINAL before expansion
6F1 Demo: data file - after date EXPANSION
6F5 Demo: - expanded file comparison - preparation
6F6 Demo: Demo Operating Instructions
6G1 Demo: copy demo copybooks to cmpdata, duplicate,& modify
6G1 Demo: Op. Instrns. - Preparation
6G2 Demo: copy demo data-files to cmpdata, duplicate,& modify
6I1 Demo: first test/demo
6L1 Demo: map1/warmat1 - copybook to demo dates in occurs
1C1 Differences: dat1/warmas1a - Alternate test file#2 with differences
6B5 Differences: Investigating the differences
4C1 Diffs: dat1/warmas1Ia - test file#2 with diffs & inserts
2D1 Directly: compare RDW files method #1 - use uvcmp2 directly
3D1 Directly: compare IDX files method #1 - use uvcmp3 directly
0A4 Directories: uvcmp5 - compare Directories of Fixed Seqntl files
1H1 Directories: uvcmp1all - compare all files in 2 directories
1X3 Directories: uvcmp1all - compare ALL files in 2 directories
5A1 Directories: compare all files in 2 directories - Overview
5D1 Directories: uvcmp5 - compare 2 directories, creating reports in a 3rd
5G1 Directories: print VTOC reports for the 2 data directories
6C1 Directories: Directories Required for uvcmp6
6C2 Directories: Setup Directories for uvcmp6
1B1 Directory: Directory 'cmpdata' & subdirs for comparisons
2F1 Directory: create reports for all files in directory
3F1 Directory: create reports for all files in directory
5A2 Directory: Directory 'cmpdata' & subdirs for comparisons
6G1 Directory: generate comparison jobs for All files in Directory
6J3 Disabling: sample modification - splitting/disabling fields
6B5 Display: sample 'uvhd' display
6G4 Due: Alternative for duplicate fieldnames due to qualification
6G1 Duplicate: copy demo copybooks to cmpdata, duplicate,& modify
6G2 Duplicate: copy demo data-files to cmpdata, duplicate,& modify
6G4 Duplicate: Alternative for duplicate fieldnames due to qualification

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

1F1 Easie: scripts to make uvcmp1 easier to run
4D1 Easie: scripts to make uvcmpI1 easier to run
4D2 Easier: script method - easier, shorter command
1F2 Ebcdic: script to compare 2 Fixlth EBCDIC files
1H1 Ebcdic: Compare ALL re-converted files to original EBCDIC files
1X2 Ebcdic: script 'uvcmpFE1' - compare EBCDIC Fixed length files
4H3 Ebcdic: uvcmpRDWEI2 - compare EBCDIC RDW files
1H1 Ebcdic-->ascii: Execute jobs EBCDIC-->ASCII, ASCII-->EBCDIC,& compare
1H1 Ebcdic-->ascii: Generate jobs EBCDIC-->ASCII & ASCII-->EBCDIC
5C1 Eliminate: unixcmpsplit - eliminate matching files prior to uvcmp5all
1D3 Enter: arguments on command line or at prompts
5X3 Every: uvcmp5all - execute uvcmp5 for every pair of files in 2 subdirs
1E1 Exclude: uvcmp1 demo#2 - Exclude options
1E1 Exclude: uvcmp1 demo#2 Exclude - Operating Instructions
1E2 Exclude: uvcmp1 demo#2 Exclude - sample report
1E3 Exclude: uvcmp1 demo#2 Exclude - sample report Notes
1H1 Execute: Execute ALL jobs to convert ALL data-files
1H1 Execute: Execute jobs EBCDIC-->ASCII, ASCII-->EBCDIC,& compare
4H2 Execute: uvcmpRDWAI2 to compare 2 RDW files
5X3 Execute: uvcmp5all - execute uvcmp5 for every pair of files in 2 subdirs
6F7 Executing: Notes Re Executing Generated uvcmp6 Compare jobs
6G6 Executing: Executing the Fully Generated Jobs
6G6 Executing: any 1 data file compare job
6G6 Executing: compare jobs for multiple data files
5F1 Execution: uvcmp5all test/demo - execution
6E1 Execution: Execution
6F5 Expanded: demo - expanded file comparison - preparation
6F1 Expansion: Y2K 19xx DATE EXPANSION testing
6F1 Expansion: demo data file - ORIGINAL before expansion
6F1 Expansion: demo data file - after date EXPANSION
6F2 Expansion: cobmap for ORIGINAL data file before expansion
6F2 Expansion: cobmap1 for test file - after date EXPANSION
6O2 Expansion: Year2000 19xx testing for Expansion
6O2 Expansion: Year2000 20xx testing for Expansion

0A4 Fiel: uvcmp6 - generate jobs to perform field by field comparisons
0A4 Field: uvcmp6 - generate jobs to perform field by field comparisons
6A1 Field: Comparing data files field by field - Overview
6A1 Field: Comparing data files field by field - Overview
6B1 Field: Field by Field comparison - Illustration
6B1 Field: Field by Field comparison - Illustration
6J4 Field: OPTION to generate SEPARATE INSTRUCTION for each field
6G4 Fieldnames: Alternative for duplicate fieldnames due to qualification
1G1 Fields: demo #4 - packed fields
1G2 Fields: demo #4 packed fields - Operating Instructions
6J3 Fields: sample modification - splitting/disabling fields
6I1 First: test/demo
0A3 Fixe: scripts to run uvcmp1,2,3 jobs - Fixed or Variable
0A2 Fixed: uvcmp1 - compare Fixed length Sequential files
0A2 Fixed: uvcmpI1 - Fixed Length allowing Inserts (but 2 reports)
0A2 Fixed: uvcmpI2 - Fixed Length allowing Inserts (on 1 report)
0A4 Fixed: uvcmp5 - compare Directories of Fixed Seqntl files
1X1 Fixed: script 'uvcmpFA1' - compare Ascii Fixed length files
1X2 Fixed: script 'uvcmpFE1' - compare EBCDIC Fixed length files
1F2 Fixlth: script to compare 2 Fixlth EBCDIC files
2D2 Fixlth: console log from uvcmp2 2 FixLth files written by uvhd
3D2 Fixlth: console log from uvcmp3 2 FixLth files written by uvhd
3C2 Focus: Notes re Micro Focus Varlth file shown above
6G5 Fully: Sample Fully Generated Job
6G6 Fully: Executing the Fully Generated Jobs

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

0A4 Generate: uvcmp6 - generate jobs to perform field by field comparisons
1H1 Generate: Generate jobs EBCDIC-->ASCII & ASCII-->EBCDIC
1H1 Generate: ASCII-->EBCDIC jobs - for ALL data-files
6E1 Generate: script 'genuvcmp6' to generate uvcmp6 job
6G1 Generate: comparison jobs for All files in Directory
6J4 Generate: OPTION to generate SEPARATE INSTRUCTION for each field
6X3 Generate: genuvcmp6 - generate jobs to compare all files in subdirs
6D2 Generated: Options - for generation job & for generated jobs
6D2 Generated: uvcmp6 prompt for arguments (default options for generated jobs)
6F7 Generated: Notes Re Executing Generated uvcmp6 Compare jobs
6G5 Generated: Sample Fully Generated Job
6G6 Generated: Executing the Fully Generated Jobs
6J1 Generated: 'uvcmp6' & 'uvcmp6.subs' vs generated worker jobs
6L3 Generated: pfc2/warmat1 - as generated BEFORE modifications
6M1 Generated: option h,j,k for jobs generated by uvcmp6
6I1 Generating: comparison jobs for ALL copybooks & ALL data-files
6D2 Generation: Options - for generation job & for generated jobs
6D2 Generation: uvcmp6 prompt for generation option changes
6E1 Generation: Generation
6G3 Generation: Operating Instructions - Generation
6E1 Genuvcmp6: script 'genuvcmp6' to generate uvcmp6 job
6X3 Genuvcmp6: - generate jobs to compare all files in subdirs

4D2 Hard: uvcopy commands - hard way
4A1 Have: uvcmpI1 - compare files that may have Inserted records
4A1 Have: uvcmpI2 - compare files that may have Inserted records
4X1 Have: uvcmpFAI1 - compare files that may have INSERTs
5X0 Here: scripts used in Part 5 & listed here
6X0 Here: scripts used in Part 6 & listed here
6B4 Hex: sample report - data mismatches (CHAR & HEX)
6B4 Hex: sample report - data mismatches (CHAR & HEX)
1C1 Home: test file#1 /home/uvadm/dat1/warmas1
4C1 Home: test file#1 /home/uvadm/dat1/warmas1I
6B1 Home: demo data file /home/uvadm/dat1/warmas6

4G2 I2: Options i1/i2 ignore inserts/deletes OR mismatched pairs
3A1 Idx: sample variable length IDX data files
3C1 Idx: testIDX81 - sample VARIABLE length IDX data file
3D1 Idx: compare IDX files method #1 - use uvcmp3 directly
3F1 Idx: creating table summary of record sizes in IDX files
3X2 Idx: IDXstat1 - table summarize record-sizes in IDX files
3A1 Idxformat3: comparing variable length IDXFORMAT3/8 files - Overview
3X2 Idxstat1: IDXstat1 - table summarize record-sizes in IDX files
4G2 Ignore: Options i1/i2 ignore inserts/deletes OR mismatched pairs
6B1 Illustration: Field by Field comparison - Illustration
4A2 Insert: scripts to run uvcmpI1 (allow Inserts in 1 file)
4A2 Insert: scripts to run uvcmpI2 (allow inserts in both files)
4A1 Inserted: uvcmpI1 - compare files that may have Inserted records
4A1 Inserted: uvcmpI2 - compare files that may have Inserted records
4E1 Inserted: uvcmpFAI1 demo#1 - inserted record in file#2
4F1 Inserted: uvcmpFAI1 demo#2 - inserted record in file#2
4F2 Inserted: uvcmpFAI1 report#2 - inserted record in file#2
6K3 Inserting: uvcopy job - AFTER inserting code to test record-type
0A2 Inserts: uvcmpI1 - Fixed Length allowing Inserts (but 2 reports)
0A2 Inserts: uvcmpI2 - Fixed Length allowing Inserts (on 1 report)
0A3 Inserts: scripts to run uvcmpI1 - allow Inserts but on 2 reports
0A3 Inserts: scripts to run uvcmpI2 - allows Inserts/Deletes on 1 report
2E1 Inserts: uvcmpRDWAI2 compare RDW files allowing inserts/deletes
4C1 Inserts: dat1/warmas1Ia - test file#2 with diffs & inserts
4G1 Inserts: uvcmpFAI2 demo#3 - inserts in both files on 1 report
4G2 Inserts: Options i1/i2 ignore inserts/deletes OR mismatched pairs
4G2 Inserts: uvcmpFAI2 report#2 - inserts both files, 1 report
4H1 Inserts: compare RDW files allowing inserts/deletes
4X1 Inserts: uvcmpFAI1 - compare files that may have INSERTs
4X2 Inserts: 'uvcmpFAI2' - allow inserts in both files on 1 report
6G7 Inspect: using uvhd to inspect data files
6G7 Inspecting: Inspecting, Printing & Using the Mismatch Reports
2E1 Instrns.: compare varlth files - script 'uvcmpRDW1' Op. Instrns.
3E1 Instrns.: compare varlth files - script 'uvcmpIDX81' Op. Instrns.
6G1 Instrns.: demo Op. Instrns. - Preparation
6J4 Instruction: OPTION to generate SEPARATE INSTRUCTION for each field
1D1 Instructions: uvcmp1 demo#1 - Operating Instructions
1E1 Instructions: uvcmp1 demo#2 Exclude - Operating Instructions
1G2 Instructions: demo #4 packed fields - Operating Instructions
6D1 Instructions: Demo Op. Instructions (1 file at a time)
6F6 Instructions: Demo Operating Instructions
6G3 Instructions: Operating Instructions - Generation
6B5 Investigating: Investigating the differences

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

4H4 Key: unique key required

6B1 Layout: cobmap1 (copybook record layout) for data file above
1D3 Lin: enter arguments on command line or at prompts
5X0 Listed: scripts used in Part 5 & listed here
6X0 Listed: scripts used in Part 6 & listed here
5E1 Load: control file
2D2 Log: console log from uvcmp2 2 FixLth files written by uvhd
3D2 Log: console log from uvcmp3 2 FixLth files written by uvhd

6O3 Mainframe: Mainframe to UNIX conversions
6L1 Map1: map1/warmat1 - copybook to demo dates in occurs
5C1 Matching: unixcmpsplit - eliminate matching files prior to uvcmp5all
4A1 May: uvcmpI1 - compare files that may have Inserted records
4A1 May: uvcmpI2 - compare files that may have Inserted records
4X1 May: uvcmpFAI1 - compare files that may have INSERTs
2D1 Method: compare RDW files method #1 - use uvcmp2 directly
3D1 Method: compare IDX files method #1 - use uvcmp3 directly
4D2 Method: script method - easier, shorter command
3C2 Micro: Notes re Micro Focus Varlth file shown above
6G7 Mismatch: Inspecting, Printing & Using the Mismatch Reports
4G2 Mismatched: Options i1/i2 ignore inserts/deletes OR mismatched pairs
2C2 Mismatches: testRDWea - Alternate file for compare (with mismatches)
3C3 Mismatches: testIDX81a - Alternate file for compare (with mismatches)
6B3 Mismatches: sample report - data mismatches (CHAR only)
6B4 Mismatches: sample report - data mismatches (CHAR & HEX)
6B4 Mismatches: sample report - data mismatches (CHAR & HEX)
6F4 Mismatches: sample report - data mismatches
5E1 Modif: copy test files to dat1 & modify in dat2
6J3 Modification: sample modification - splitting/disabling fields
6L1 Modifications: Modifications Required for Dates in OCCURS
6L2 Modifications: Modifications Required for Dates in OCCURS
6L3 Modifications: pfc2/warmat1 - as generated BEFORE modifications
6L4 Modifications: pfc3/warmat1 - AFTER modifications
6G1 Modify: copy demo copybooks to cmpdata, duplicate,& modify
6G2 Modify: copy control file & modify
6G2 Modify: copy demo data-files to cmpdata, duplicate,& modify
5X2 Move: unixcmpsplit - move non-matching files to other subdirs

5X2 Non-matching: unixcmpsplit - move non-matching files to other subdirs

6L1 Occurs: Modifications Required for Dates in OCCURS
6L1 Occurs: map1/warmat1 - copybook to demo dates in occurs
6L2 Occurs: AFTER Changes for Dates in Occurs
6L2 Occurs: BEFORE Changes for Dates in Occurs
6L2 Occurs: Modifications Required for Dates in OCCURS
2E1 Op.: compare varlth files - script 'uvcmpRDW1' Op. Instrns.
3E1 Op.: compare varlth files - script 'uvcmpIDX81' Op. Instrns.
6D1 Op.: Demo Op. Instructions (1 file at a time)
6G1 Op.: demo Op. Instrns. - Preparation
1D1 Operating: uvcmp1 demo#1 - Operating Instructions
1E1 Operating: uvcmp1 demo#2 Exclude - Operating Instructions
1G2 Operating: demo #4 packed fields - Operating Instructions
6F6 Operating: Demo Operating Instructions
6G3 Operating: Operating Instructions - Generation
6D2 Option: uvcmp6 prompt for generation option changes
6J4 Option: OPTION to generate SEPARATE INSTRUCTION for each field
6M1 Option: h,j,k for jobs generated by uvcmp6
1E1 Options: uvcmp1 demo#2 - Exclude options
4G2 Options: Options i1/i2 ignore inserts/deletes OR mismatched pairs
4H3 Options: Options for uvcmpRDWAI2
6D2 Options: Options - for generation job & for generated jobs
6D2 Options: specifying options & arguments on uvcmp6 command
6D2 Options: uvcmp6 prompt for arguments (default options for generated jobs)
2A1 Overview: comparing variable length RDW files - Overview
3A1 Overview: comparing variable length IDXFORMAT3/8 files - Overview
5A1 Overview: compare all files in 2 directories - Overview
6A1 Overview: Comparing data files field by field - Overview

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

1G1 Packed: demo #4 - packed fields
1G2 Packed: demo #4 packed fields - Operating Instructions
1D3 Page: Notes re uvcmp1 report on prior page
1F4 Page: Notes re warmas1E report on prior page
4E3 Page: Notes re uvcmpI1 report on prior page
5X3 Pair: uvcmp5all - execute uvcmp5 for every pair of files in 2 subdirs
4G2 Pairs: Options i1/i2 ignore inserts/deletes OR mismatched pairs
1A2 Part: scripts to run comparison jobs in Part 1
1X0 Part: scripts used in Part 1
1X0 Part: uvcopy jobs used in Part 1
2X0 Part: scripts used in Part 2
2X0 Part: uvcopy jobs used in Part 2
3X0 Part: scripts used in Part 3
3X0 Part: uxcopy jobs used in Part 3
4D1 Part: test/demo files used in Part 4
4X0 Part: scripts used in Part 4
4X0 Part: uvcopy jobs used in Part 4
5X0 Part: scripts used in Part 5 & listed here
5X0 Part: summary of uvcopy jobs used in Part 5
6X0 Part: scripts used in Part 6 & listed here
6X0 Part: summary of uvcopy jobs used in Part 6
1A2 Parts: scripts to run comparison jobs in other Parts
4D1 Parts: other scripts used in previous Parts
4D1 Parts: test/demo files used in previous Parts
0A4 Perform: uvcmp6 - generate jobs to perform field by field comparisons
6L3 Pfc2: pfc2/warmat1 - as generated BEFORE modifications
6L4 Pfc3: pfc3/warmat1 - AFTER modifications
4B1 Preparation: for uvcmpI1 test/demo
5A2 Preparation: for uvcmp1 test/demo
5E1 Preparation: uvcmp5 test/demo - preparation
6C1 Preparation: test/demo - preparation (1 at a time)
6F5 Preparation: demo - expanded file comparison - preparation
6G1 Preparation: demo Op. Instrns. - Preparation
4D1 Previous: other scripts used in previous Parts
4D1 Previous: test/demo files used in previous Parts
5G1 Print: VTOC reports for the 2 data directories
6G7 Print: VTOC reports, check record counts
6G7 Printing: Inspecting, Printing & Using the Mismatch Reports
1D3 Prior: Notes re uvcmp1 report on prior page
1F4 Prior: Notes re warmas1E report on prior page
4E3 Prior: Notes re uvcmpI1 report on prior page
5C1 Prior: unixcmpsplit - eliminate matching files prior to uvcmp5all
6D2 Prompt: uvcmp6 prompt for arguments (default options for generated jobs)
6D2 Prompt: uvcmp6 prompt for generation option changes
1D3 Prompts: enter arguments on command line or at prompts

6G4 Qualification: Alternative for duplicate fieldnames due to qualification

0A3 Rdw: uvcmp2 - uvcopy job to compare RDW files
2A1 Rdw: comparing variable length RDW files - Overview
2A1 Rdw: sample variable length RDW data files
2C1 Rdw: testRDWe - sample VARIABLE length RDW data file
2D1 Rdw: compare RDW files method #1 - use uvcmp2 directly
2E1 Rdw: uvcmpRDWAI2 compare RDW files allowing inserts/deletes
2F1 Rdw: creating table summary of record sizes in RDW files
2X2 Rdw: RDWstat1 - table summarize record-sizes in RDW files
4B1 Rdw: for varlth RDW files
4H1 Rdw: compare RDW files allowing inserts/deletes
4H1 Rdw: uvcp to convert TEXT files to RDW files
4H1 Rdw: vendormas2 TEXT file used to create vendormas3 RDW file
4H1 Rdw: vendormas22 TEXT file used to create vendormas32 RDW file
4H2 Rdw: execute uvcmpRDWAI2 to compare 2 RDW files
4H3 Rdw: uvcmpRDWEI2 - compare EBCDIC RDW files
2X2 Rdwstat1: RDWstat1 - table summarize record-sizes in RDW files
1H1 Re-converted: Compare ALL re-converted files to original EBCDIC files
0A4 Recommendation:
2F1 Record: creating table summary of record sizes in RDW files
3F1 Record: creating table summary of record sizes in IDX files
4E1 Record: uvcmpFAI1 demo#1 - inserted record in file#2
4F1 Record: uvcmpFAI1 demo#2 - inserted record in file#2
4F2 Record: uvcmpFAI1 report#2 - inserted record in file#2
5G2 Record: why the '*' beside record count
6B1 Record: cobmap1 (copybook record layout) for data file above
6G7 Record: print VTOC reports, check record counts
6K1 Record: Comparing files with Multiple Record Types
6K1 Record: copybook for warmas3 (multiple record types)
6K2 Record: uvcopy job - BEFORE record type changes
2X2 Record-sizes: RDWstat1 - table summarize record-sizes in RDW files
3X2 Record-sizes: IDXstat1 - table summarize record-sizes in IDX files
6K3 Record-type: uvcopy job - AFTER inserting code to test record-type
4A1 Records: uvcmpI1 - compare files that may have Inserted records
4A1 Records: uvcmpI2 - compare files that may have Inserted records
1D3 Repor: Notes re uvcmp1 report on prior page
1F4 Repor: Notes re warmas1E report on prior page
4E3 Repor: Notes re uvcmpI1 report on prior page
0A3 Run: scripts to run uvcmp1,2,3 jobs - Fixed or Variable
0A3 Run: scripts to run uvcmpI1 - allow Inserts but on 2 reports
0A3 Run: scripts to run uvcmpI2 - allows Inserts/Deletes on 1 report
1A2 Run: scripts to run comparison jobs in Part 1
1A2 Run: scripts to run comparison jobs in other Parts
1F1 Run: scripts to make uvcmp1 easier to run
2X1 Run: uvcmpRDW1 - script to run uvcmp2
3X1 Run: uvcmpIDX81 - script to run uvcmp3
4A2 Run: scripts to run uvcmpI1 (allow Inserts in 1 file)
4A2 Run: scripts to run uvcmpI2 (allow inserts in both files)
4D1 Run: scripts to make uvcmpI1 easier to run

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

1D2 Sample: uvcmp1 demo#1 - sample report
1E2 Sample: uvcmp1 demo#2 Exclude - sample report
1E3 Sample: uvcmp1 demo#2 Exclude - sample report Notes
1F3 Sample: script 'uvcmpFE1' - sample report
2A1 Sample: variable length RDW data files
2C1 Sample: testRDWe - sample VARIABLE length RDW data file
2D3 Sample: testRDWe - sample comparison report
2F1 Sample: report
3A1 Sample: variable length IDX data files
3C1 Sample: testIDX81 - sample VARIABLE length IDX data file
3D3 Sample: testIDX81 - sample comparison report
3F1 Sample: report
4E2 Sample: uvcmpI1 demo#1 - sample report
5D2 Sample: control file
5G1 Sample: VTOC report for dat1
5G1 Sample: VTOC report for dat2
5G2 Sample: VTOC report for dat2
6B2 Sample: uvcopy job to compare 2 data files
6B3 Sample: report - data mismatches (CHAR only)
6B4 Sample: report - data mismatches (CHAR & HEX)
6B4 Sample: report - data mismatches (CHAR & HEX)
6B5 Sample: 'uvhd' display
6F3 Sample: uvcopy job to compare 2 data files
6F3 Sample: uvcopy job to compare 2 data files
6F4 Sample: report - data mismatches
6G5 Sample: Sample Fully Generated Job
6G5 Sample: comparison job - completed by uvdata52
6J3 Sample: modification - splitting/disabling fields
5D1 Section: contents of this section
6J4 Separate: OPTION to generate SEPARATE INSTRUCTION for each field
0A4 Seqntl: uvcmp5 - compare Directories of Fixed Seqntl files
0A2 Sequential: uvcmp1 - compare Fixed length Sequential files
1A1 Sequential: uvcopy jobs to compare Sequential files
2B2 Setup: subdirs & copy test/demo files
3B2 Setup: subdirs & copy test/demo files
6C2 Setup: Setup Directories for uvcmp6
4D2 Shorter: script method - easier, shorter command
3C2 Shown: Notes re Micro Focus Varlth file shown above
2F1 Size: creating table summary of record sizes in RDW files
3F1 Size: creating table summary of record sizes in IDX files
6D2 Specifying: options & arguments on uvcmp6 command
6J3 Splitting: sample modification - splitting/disabling fields
6G1 Sub-directories: make sub-directories
1B1 Subdirs: Directory 'cmpdata' & subdirs for comparisons
1B1 Subdirs: used to compare files
2B1 Subdirs: used to compare varlth files
2B2 Subdirs: setup subdirs & copy test/demo files
3B1 Subdirs: used to compare varlth files
3B2 Subdirs: setup subdirs & copy test/demo files
4B2 Subdirs: used to compare files
5A2 Subdirs: Directory 'cmpdata' & subdirs for comparisons
5X1 Subdirs: unixcmpall - compare all files in 2 subdirs
5X2 Subdirs: unixcmpsplit - move non-matching files to other subdirs
5X3 Subdirs: uvcmp5all - execute uvcmp5 for every pair of files in 2 subdirs
6X3 Subdirs: genuvcmp6 - generate jobs to compare all files in subdirs
6J1 Subrtn: alternate subrtn 'cmpp' vs 'cmpc'
2X2 Summarize: RDWstat1 - table summarize record-sizes in RDW files
3X2 Summarize: IDXstat1 - table summarize record-sizes in IDX files
1A1 Summary: uvcmp... job summary
2F1 Summary: creating table summary of record sizes in RDW files
3F1 Summary: creating table summary of record sizes in IDX files
5X0 Summary: of uvcopy jobs used in Part 5
6X0 Summary: of uvcopy jobs used in Part 6
2F1 Supplied: demo with supplied test file
3F1 Supplied: demo with supplied test file

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

2F1 Table: creating table summary of record sizes in RDW files
2X2 Table: RDWstat1 - table summarize record-sizes in RDW files
3F1 Table: creating table summary of record sizes in IDX files
3X2 Table: IDXstat1 - table summarize record-sizes in IDX files
1C1 Test: dat1/warmas1a - Alternate test file#2 with differences
1C1 Test: file#1 /home/uvadm/dat1/warmas1
2B2 Test: setup subdirs & copy test/demo files
2F1 Test: demo with supplied test file
3B2 Test: setup subdirs & copy test/demo files
3F1 Test: demo with supplied test file
4B1 Test: preparation for uvcmpI1 test/demo
4C1 Test: dat1/warmas1Ia - test file#2 with diffs & inserts
4C1 Test: file#1 /home/uvadm/dat1/warmas1I
4D1 Test: test/demo files used in Part 4
4D1 Test: test/demo files used in previous Parts
5A2 Test: preparation for uvcmp1 test/demo
5B1 Test: unixcmpall - test/demo
5E1 Test: copy test files to dat1 & modify in dat2
5E1 Test: uvcmp5 test/demo - preparation
5F1 Test: uvcmp5all test/demo - execution
6C1 Test: test/demo - preparation (1 at a time)
6F2 Test: cobmap1 for test file - after date EXPANSION
6I1 Test: first test/demo
6K3 Test: uvcopy job - AFTER inserting code to test record-type
3C1 Testidx81: testIDX81 - sample VARIABLE length IDX data file
3D3 Testidx81: testIDX81 - sample comparison report
3C3 Testidx81a: testIDX81a - Alternate file for compare (with mismatches)
6F1 Testing: Y2K 19xx DATE EXPANSION testing
6O1 Testing: Year2000 19xx testing for Windows
6O1 Testing: Year2000 20xx testing for Windows
6O2 Testing: Year2000 19xx testing for Expansion
6O2 Testing: Year2000 20xx testing for Expansion
2C1 Testrdwe: testRDWe - sample VARIABLE length RDW data file
2D3 Testrdwe: testRDWe - sample comparison report
2C2 Testrdwea: testRDWea - Alternate file for compare (with mismatches)
4H1 Text: uvcp to convert TEXT files to RDW files
4H1 Text: vendormas2 TEXT file used to create vendormas3 RDW file
4H1 Text: vendormas22 TEXT file used to create vendormas32 RDW file
4A1 That: uvcmpI1 - compare files that may have Inserted records
4A1 That: uvcmpI2 - compare files that may have Inserted records
4X1 That: uvcmpFAI1 - compare files that may have INSERTs
6K2 Type: uvcopy job - BEFORE record type changes
2A1 Types: of Variable Length files
3A1 Types: of Variable Length files
6K1 Types: Comparing files with Multiple Record Types
6K1 Types: copybook for warmas3 (multiple record types)

4H4 Unique: key required
6O3 Unix: Mainframe to UNIX conversions
0A4 Unixcmpall:
5B1 Unixcmpall: - demo report
5B1 Unixcmpall: - test/demo
5X1 Unixcmpall: - compare all files in 2 subdirs
0A4 Unixcmpsplit:
5C1 Unixcmpsplit: - demo report
5C1 Unixcmpsplit: - eliminate matching files prior to uvcmp5all
5X2 Unixcmpsplit: - move non-matching files to other subdirs
1C1 Uvadm: test file#1 /home/uvadm/dat1/warmas1
4C1 Uvadm: test file#1 /home/uvadm/dat1/warmas1I
6B1 Uvadm: demo data file /home/uvadm/dat1/warmas6
1A1 Uvcmp...: job summary
0A2 Uvcmp1: - compare Fixed length Sequential files
0A3 Uvcmp1: scripts to run uvcmp1,2,3 jobs - Fixed or Variable
1D1 Uvcmp1: demo#1 - Operating Instructions
1D2 Uvcmp1: demo#1 - sample report
1D3 Uvcmp1: Notes re uvcmp1 report on prior page
1E1 Uvcmp1: demo#2 - Exclude options
1E1 Uvcmp1: demo#2 Exclude - Operating Instructions
1E2 Uvcmp1: demo#2 Exclude - sample report
1E3 Uvcmp1: demo#2 Exclude - sample report Notes
1F1 Uvcmp1: scripts to make uvcmp1 easier to run
5A2 Uvcmp1: preparation for uvcmp1 test/demo
1H1 Uvcmp1all: - compare all files in 2 directories
1X3 Uvcmp1all: - compare ALL files in 2 directories
0A3 Uvcmp2: - uvcopy job to compare RDW files
2D1 Uvcmp2: alternative 'script uvcmpRDW1' vs 'uvcopy uvcmp2'
2D1 Uvcmp2: compare RDW files method #1 - use uvcmp2 directly
2D2 Uvcmp2: console log from uvcmp2 2 FixLth files written by uvhd
2X1 Uvcmp2: uvcmpRDW1 - script to run uvcmp2
3D1 Uvcmp2: alternative 'script uvcmpIDX81' vs 'uvcopy uvcmp2'
3D1 Uvcmp3: compare IDX files method #1 - use uvcmp3 directly
3D2 Uvcmp3: console log from uvcmp3 2 FixLth files written by uvhd
3X1 Uvcmp3: uvcmpIDX81 - script to run uvcmp3
0A4 Uvcmp5: - compare Directories of Fixed Seqntl files
5D1 Uvcmp5: - compare 2 directories, creating reports in a 3rd
5E1 Uvcmp5: test/demo - preparation
5X3 Uvcmp5: uvcmp5all - execute uvcmp5 for every pair of files in 2 subdirs
0A4 Uvcmp5all:
5C1 Uvcmp5all: unixcmpsplit - eliminate matching files prior to uvcmp5all
5F1 Uvcmp5all: demo console output
5F1 Uvcmp5all: test/demo - execution
5X3 Uvcmp5all: - execute uvcmp5 for every pair of files in 2 subdirs
0A4 Uvcmp6: - generate jobs to perform field by field comparisons
6C1 Uvcmp6: Directories Required for uvcmp6
6C2 Uvcmp6: Setup Directories for uvcmp6
6D2 Uvcmp6: specifying options & arguments on uvcmp6 command
6D2 Uvcmp6: prompt for arguments (default options for generated jobs)
6D2 Uvcmp6: prompt for generation option changes
6E1 Uvcmp6: script 'genuvcmp6' to generate uvcmp6 job
6F7 Uvcmp6: Notes Re Executing Generated uvcmp6 Compare jobs
6J1 Uvcmp6: 'uvcmp6' & 'uvcmp6.subs' vs generated worker jobs
6M1 Uvcmp6: option h,j,k for jobs generated by uvcmp6
6J1 Uvcmp6.subs: 'uvcmp6' & 'uvcmp6.subs' vs generated worker jobs
1X1 Uvcmpfa1: script 'uvcmpFA1' - compare Ascii Fixed length files
4E1 Uvcmpfai1: uvcmpFAI1 demo#1 - inserted record in file#2
4F1 Uvcmpfai1: uvcmpFAI1 demo#2 - inserted record in file#2
4F2 Uvcmpfai1: uvcmpFAI1 report#2 - inserted record in file#2
4X1 Uvcmpfai1: uvcmpFAI1 - compare files that may have INSERTs
4G1 Uvcmpfai2: uvcmpFAI2 demo#3 - inserts in both files on 1 report
4G2 Uvcmpfai2: uvcmpFAI2 report#2 - inserts both files, 1 report
4X2 Uvcmpfai2: 'uvcmpFAI2' - allow inserts in both files on 1 report
1F1 Uvcmpfe1: script uvcmpFE1 demo
1F2 Uvcmpfe1: demo#3 - using script 'uvcmpFE1'
1F3 Uvcmpfe1: script 'uvcmpFE1' - sample report
1X2 Uvcmpfe1: script 'uvcmpFE1' - compare EBCDIC Fixed length files
0A2 Uvcmpi1: uvcmpI1 - Fixed Length allowing Inserts (but 2 reports)
0A3 Uvcmpi1: scripts to run uvcmpI1 - allow Inserts but on 2 reports
4A1 Uvcmpi1: uvcmpI1 - compare files that may have Inserted records
4A2 Uvcmpi1: scripts to run uvcmpI1 (allow Inserts in 1 file)
4B1 Uvcmpi1: preparation for uvcmpI1 test/demo
4D1 Uvcmpi1: scripts to make uvcmpI1 easier to run
4E2 Uvcmpi1: uvcmpI1 demo#1 - sample report
4E3 Uvcmpi1: Notes re uvcmpI1 report on prior page
0A2 Uvcmpi2: uvcmpI2 - Fixed Length allowing Inserts (on 1 report)
0A3 Uvcmpi2: scripts to run uvcmpI2 - allows Inserts/Deletes on 1 report
4A1 Uvcmpi2: uvcmpI2 - compare files that may have Inserted records
4A2 Uvcmpi2: scripts to run uvcmpI2 (allow inserts in both files)
3D1 Uvcmpidx81: alternative 'script uvcmpIDX81' vs 'uvcopy uvcmp2'
3E1 Uvcmpidx81: compare varlth files - script uvcmpIDX81
3E1 Uvcmpidx81: compare varlth files - script 'uvcmpIDX81' Op. Instrns.
3X1 Uvcmpidx81: uvcmpIDX81 - script to run uvcmp3
2D1 Uvcmprdw1: alternative 'script uvcmpRDW1' vs 'uvcopy uvcmp2'
2E1 Uvcmprdw1: compare varlth files - script uvcmpRDW1
2E1 Uvcmprdw1: compare varlth files - script 'uvcmpRDW1' Op. Instrns.
2X1 Uvcmprdw1: uvcmpRDW1 - script to run uvcmp2
2E1 Uvcmprdwai2: uvcmpRDWAI2 compare RDW files allowing inserts/deletes
4H2 Uvcmprdwai2: execute uvcmpRDWAI2 to compare 2 RDW files
4H3 Uvcmprdwai2: Options for uvcmpRDWAI2
4H3 Uvcmprdwei2: uvcmpRDWEI2 - compare EBCDIC RDW files
0A2 Uvcopy: jobs for comparing data files
0A3 Uvcopy: uvcmp2 - uvcopy job to compare RDW files
1A1 Uvcopy: jobs to compare Sequential files
1F2 Uvcopy: command vs script equivalent
1X0 Uvcopy: jobs used in Part 1
2D1 Uvcopy: alternative 'script uvcmpRDW1' vs 'uvcopy uvcmp2'
2X0 Uvcopy: jobs used in Part 2
3D1 Uvcopy: alternative 'script uvcmpIDX81' vs 'uvcopy uvcmp2'
4D2 Uvcopy: commands - hard way
4D2 Uvcopy: jobs
4X0 Uvcopy: jobs used in Part 4
5X0 Uvcopy: summary of uvcopy jobs used in Part 5
6B2 Uvcopy: sample uvcopy job to compare 2 data files
6F3 Uvcopy: sample uvcopy job to compare 2 data files
6F3 Uvcopy: sample uvcopy job to compare 2 data files
6K2 Uvcopy: job - BEFORE record type changes
6K3 Uvcopy: job - AFTER inserting code to test record-type
6X0 Uvcopy: summary of uvcopy jobs used in Part 6
4H1 Uvcp: to convert TEXT files to RDW files
6G5 Uvdata52: sample comparison job - completed by uvdata52
2D2 Uvhd: console log from uvcmp2 2 FixLth files written by uvhd
3D2 Uvhd: console log from uvcmp3 2 FixLth files written by uvhd
6B5 Uvhd: sample 'uvhd' display
6G7 Uvhd: using uvhd to inspect data files
3X0 Uxcopy: jobs used in Part 3

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

0A3 Variable: scripts to run uvcmp1,2,3 jobs - Fixed or Variable
2A1 Variable: comparing variable length RDW files - Overview
2A1 Variable: sample variable length RDW data files
2A1 Variable: types of Variable Length files
2C1 Variable: testRDWe - sample VARIABLE length RDW data file
3A1 Variable: comparing variable length IDXFORMAT3/8 files - Overview
3A1 Variable: sample variable length IDX data files
3A1 Variable: types of Variable Length files
3C1 Variable: testIDX81 - sample VARIABLE length IDX data file
2B1 Varlth: subdirs used to compare varlth files
2E1 Varlth: compare varlth files - script uvcmpRDW1
2E1 Varlth: compare varlth files - script 'uvcmpRDW1' Op. Instrns.
3B1 Varlth: subdirs used to compare varlth files
3C2 Varlth: Notes re Micro Focus Varlth file shown above
3E1 Varlth: compare varlth files - script uvcmpIDX81
3E1 Varlth: compare varlth files - script 'uvcmpIDX81' Op. Instrns.
4B1 Varlth: for varlth RDW files
4H1 Vendormas2: TEXT file used to create vendormas3 RDW file
4H1 Vendormas22: TEXT file used to create vendormas32 RDW file
4H1 Vendormas3: vendormas2 TEXT file used to create vendormas3 RDW file
4H1 Vendormas32: vendormas22 TEXT file used to create vendormas32 RDW file
5G1 Vtoc: VTOC reports - aid file compares
5G1 Vtoc: print VTOC reports for the 2 data directories
5G1 Vtoc: sample VTOC report for dat1
5G1 Vtoc: sample VTOC report for dat2
5G2 Vtoc: sample VTOC report for dat2
6G7 Vtoc: print VTOC reports, check record counts

1C1 Warmas1: test file#1 /home/uvadm/dat1/warmas1
1C1 Warmas1a: dat1/warmas1a - Alternate test file#2 with differences
1F4 Warmas1e: Notes re warmas1E report on prior page
4C1 Warmas1i: test file#1 /home/uvadm/dat1/warmas1I
4C1 Warmas1ia: dat1/warmas1Ia - test file#2 with diffs & inserts
6K1 Warmas3: copybook for warmas3 (multiple record types)
6B1 Warmas6: demo data file /home/uvadm/dat1/warmas6
6L1 Warmat1: map1/warmat1 - copybook to demo dates in occurs
6L3 Warmat1: pfc2/warmat1 - as generated BEFORE modifications
6L4 Warmat1: pfc3/warmat1 - AFTER modifications
4D2 Way: uvcopy commands - hard way
6O1 Windows: Year2000 19xx testing for Windows
6O1 Windows: Year2000 20xx testing for Windows
6J1 Worker: 'uvcmp6' & 'uvcmp6.subs' vs generated worker jobs
2D2 Written: console log from uvcmp2 2 FixLth files written by uvhd
3D2 Written: console log from uvcmp3 2 FixLth files written by uvhd

6F1 Y2k: Y2K 19xx DATE EXPANSION testing
6O1 Year2000: Year2000 19xx testing for Windows
6O1 Year2000: Year2000 20xx testing for Windows
6O2 Year2000: Year2000 19xx testing for Expansion
6O2 Year2000: Year2000 20xx testing for Expansion

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Visitor Counters for ThisYear and LastYear

ThisYear=000917   (J=152,F=174,M=217,A=189,M=185,J=0,J=0,A=0,S=0,O=0,N=0,D=0)
LastYear=001719   (J=149,F=180,M=137,A=133,M=150,J=109,J=157,A=132,S=125,O=134,N=127,D=186)