| Part_0 | - Introduction & Overview |
| - test files supplied for you to copy to your homedir | |
| - step by step operating instructions to run the test/demos | |
| - then try out on your own files |
| Part_1 | - preparations for Vancouver Utilities self-training |
| - setup your profile & copy testfiles to subdirs in your homedir | |
| - test files provided by UV Software to demo Vancouver Utilities | |
| - customer master files: custmas0 (EBCDIC) & custmas1 (ASCII) | |
| - sales detail files: sales0 (EBCDIC) & sales1 (ASCII) | |
| - vendor master file: vendormas3 sample of RDW variable-length file | |
| - these files used in all parts of this document |
| Part_2 | - 'uvhd' file investigation utility (free download) |
| - display any file in vertical hexadecimal | |
| - ideal for mainframe EBCDIC files transferred to unix/linux | |
| (with packed/binary fields & no LineFeeds required by most unix tools) | |
| - commands to browse,search,select,update,translate,etc |
| Part_3 | - 'uvhdcob' companion utility to 'uvhd' |
| - displays COBOL copybook fieldnames beside data field contents | |
| - similar commands to browse,search,select,update | |
| - + 'verify' using copybook defs of character/numeric/numeric fields | |
| - may correct bad data: zero fill numeric/packed fields with blanks, | |
| and convert unprintable bytes in character fields to '.' periods |
| Part_4 | - uvcp, uvsort,& uvlist - data file reformat, sort,& list utilities |
| - a few examples to give you an idea of their capabilities | |
| - full documentation available in separate manuals | |
| - uvcp.htm, uvsort.htm,& uvlist.htm |
| Part_5 | - FTP mainframe data files to Unix/Linux |
| - must use binary mode if any packed/binary fields | |
| - may use ASCII mode if all unpacked character fields | |
| - ASCII mode advantage automatic translate EBCDIC to ASCII & inserts | |
| LineFeeds which allows use of unix tools (vi,lp,cat,mopre,etc) | |
| - disadvantage increases record sizes, need to modify programs to match | |
| - also need to correct zone signs in unpacked numeric fields | |
| from mainframe conventions to Micro Focus COBOL conventions | |
| - using uvcp to change record lengths & file types & zoned signs | |
| - detecting wrong-sized records in FTP'd files | |
| - selecting desired record-sizes from FTP ASCII files | |
| - EBCDIC to ASCII Data Conversion allowing packed/binary | |
| - these more complex data conversions not covered in detail here | |
| - references given to other documents with the operating instructions |
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| Part_6 | - General Purpose Table Summaries |
| - 'table2' pre-programmed job to accumulate record counts & any 2 | |
| other fields by any desired table argument | |
| - example for sales3 demo file, acum counts,qty,amt by product code | |
| - 'cmtbl2' sample custom written uvcopy job to demo the 'tbl' instruction | |
| - builds 2 tables summarizing this-year & last-year sales | |
| by city & province |
| Part_7 | - Accumulating numeric data fields |
| - easiest using the interactive uvhd utility (free download) | |
| - or use pre-programmed batch job to accumulate 1 field & display total | |
| - use copybooks to generate jobs to accumulate ALL numeric fields | |
| - custom written uvcopy jobs to accumulate multiple fields, | |
| test record types, edit as desired. |
| Part_8 | - Compare 'Variable Length RDW' Data files |
| - Part 8 here in DataUtilities.htm taken from Part 2 in CMPjobs.doc, | |
| which is the complete documentation for data file compare jobs. | |
| - CMPjobs.htm covers all file types (Fixed-Length, Micro Focus, etc) | |
| and alternate jobs to allow for Inserted or Deleted records. | |
| - This Part covering variable-length RDW files was included here in | |
| DataAnalysis.doc because it was short, but gives you a good idea | |
| of the data file compare jobs. |
| Part_9 | - 'gentbl1A' & 'gentbl1E' utilities to table summarize all fields |
| in a data file (gentbl1A for Asciii & gentbl1E for Ebcdic). | |
| - Driven by copybooks to generate uvcopy jobs that read thru files & | |
| create tables of values found in each field defined in the copybook. | |
| - Tabling all values would be too voluminous without limiting options. | |
| - Default option 'm30' limits entries retained to 30, then new entries | |
| added if their value is greater than or less than any existing entry | |
| & a mid-point entry removed to maintain the number of entries to 30. | |
| - Default option 'v10' limits ~invalid entries retained to 10, after 10 | |
| invalid entries are just counted for the ~Invalid Total line. |
Parts 1-6 are introductions to uvhd,uvhdcob,uvcp,uvsort,uvlist,& uvcopy. The examples & tutorials here are the best place to start learning these utilities, but they are all extensively documented elsewhere in separate manuals.
Part_9 is an original work, this is the only manual that documents gentbl1A/1E. It is particularly valuable if you need to investigate new files from outside sources.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
This document is intended as a good place to start learning how the Vancouver Utilities can help you investigate, validate, manipulate, sort,& perform summary table analysis on mainframe type files that have been migrated to Unix/Linux systems.
The Vancouver Utilities are designed to provide the mainframe type data file maintenance, manipulation,& sorting capabilities that are not supplied by standard unix/linux system utilities. For example the unix/linux system sort can not sort on packed fields, can not sort files with fixed record lengths (without line-feeds), and can not sort Indexed files.
In addition to the SORT, Vancouver Utilities provide replacements for mainframe utilities such as IDCAMS, IEBGENER, DITTO, FILE-AID,& EASYTRIEVE. Vancouver Utilities provide the powerful data manipulation & data field addressing by column# that is otherwise lacking on unix/linux systems.
Test data files & copybooks are supplied for you to copy to subdirs in your homedir. Step by step operating instructions help you to run the test/demos. Expected reports are listed for comparison to your results. After the test/demos, please try out these utilities on some of your own files.
VU documentation is created & maintained as text files with unix/linux editors, and automatically converted to HTML by a Vancouver Utility job. The text files are found in /home/uvadm/doc/... after the package is installed. They are given suffixes of .doc to indicate documentation (not MS WORD files).
datautilities.htm - this document
uvhd.htm - file investigation vertical hexdump utility
- uvhd is a free download from freestuff.htm
uvhdcob.htm - displays data field contents with copybook fieldnames
uvcp.htm - command line data file reformat utility
uvsort.htm - Vancouver sort utility replaces unix sort
- inadequate for mainframe type files migrated to unix/linux
uvlist.htm - test file list utility for documentation, scripts, programs
- ensures page headings with filenames, userid, date/times
uvcopy1.htm - powerful data manipulation utility
uvcopy2.htm - record/work areas, options, etc
uvcopy3.htm - all instructions documented with examples
cmpjobs.htm - Data File Compare jobs (for all file types)
JCLcnv1demo.htm - Best document if you are interested in converting mainframe
JCL to Korn shell scripts
- part_4 is also best place to start for DATA file conversions
translating EBCDIC to ASCII preserving packed fields
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| 1A1. | Vancouver Utilities Installation |
| 1A2. | Preparation for Vancouver Utilities Self-Training |
| - setup your profile to use the Vancouver Utilities | |
| - copy demo data files & copybooks to subdirs in your homedir |
| 1A3. | Training subdirs/files copied to your homedir |
| 1B1. | dat1/custmas1 - demo file for several following tutorials |
| - customer master Name,Address,& Sales-History file | |
| - sample of fixed-length data file with packed fields | |
| - 256 byte records with 24 x 5 byte packed fields |
| 1C1. | dat1/sales1,2,3 - sales detail records |
| - cust#,slsmn#,date,inv#,product-code,units,price,amount | |
| - sample fixed length file with unpacked numeric fields | |
| - 64 byte records with unpacked Zoned numeric qty/amount | |
| - sales1,2,3 3 versions with different sign conventions |
| 1D1. | dat1/vendormas3 - sample RDW file (Record Descriptor Word) |
| - variable length records | |
| - record size in binary in 4 byte record prefix |
| 1E1. | dat1/vendormas3B - sample BDW/RDW file (Block & Record Descriptor Words) |
| 1F1. | dat1/vendorpaymas - sample multi-record-type file |
| - vendor Master & Payment records | |
| - text file & RDW equivalent |
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
We recommend setting up userid 'uvadm' & installing the software in its home directory which would usually be /home/uvadm. We allow for alternate locations' by defining environmental variable '$UV' in the profile as follows:
export UV=/home/uvadm #<-- default location of Vancouver Utiltiies ===================== - environmental variable in profile
To use the Vancouver utilities, you must setup your profile as described at https://www.uvsoftware.ca/install.htm#A5. If the package is already installed and you are a new user, you can conCATenate the supplied profile onto your existing profile as follows:
cat /home/uvadm/env/stub_profile >>.profile <-- for Unix/Korn shell users =========================================== cat /home/uvadm/env/stub_profile >>.bash_profile <-- for Linux/Bash shell users ================================================
Please see all installation directories summarized at install.htm#A2. The following are relevant to the training exercises in this document.
$UV/bin/... - contains Vancouver Utility programs (compiled from $UV/src).
(uvhd, uvlist, uvcp, uvsort, uvcopy)
- this directory must be in your PATH (defined in your .profile)
$UV/dat1/... - test data files for UV training exercises
(custmas1, custmas2, custmas3, sales1, sales2, sales3, etc)
- demo file for vi mark exercise (uvtrain_contents)
$UV/mf/cpys - COBOL copybooks describing record layouts for data files
(custmas1.cpy, salesdtl.cpy, etc)
$UV/mf/maps - 'cobmap's created from COBOL copybooks
(custmas1.map, salesdtl.map, etc)
- shows field start/end/length/type on right side
$UV/pf/... - parameter files (pre-programmed jobs) for the uvcopy utility
example: cobmap - create record layouts from COBOL copy-books
- sub-directoried pf/adm, pf/demo, pf/IBM, pf/util
directories defined by env-var PFPATH (see below)
$UV/sf/... - script files supplied with the Vancouver Utilities
example: uvlp12 - script to execute uvlist & pipe to a printer
- sub-directoried sf/adm, sf/demo, sf/IBM, sf/util
- add to your PATH as shown below
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
We recommend you setup subdirs in your home directory & copy the relevant files to these subdirs as follows:
#1. login --> your homedir
#2a. mkdir dat1 <-- make subdir for demo datafiles
==========
#2b. mkdir cpys <-- make subdir for copybooks
==========
#2c. mkdir maps <-- make subdir for copybook layouts (maps)
==========
#2d. mkdir sf <-- make subdir for scripts (Script Files)
==========
#2e. mkdir pf <-- make subdir for uvcopy jobs (Parameter Files)
==========
#2f. mkdir tmp <-- make tmp subdir in your home directory
========= to receive output of various exercises
#3a. cp $UV/dat1/custmas* dat1 <-- customer master files
========================= - sample fixed length with packed fields
#3b. cp $UV/dat1/sales* dat1 <-- sales files
======================= - sample fixed length with numeric fields
#3c. cp $UV/dat1/vendor* dat1 <-- Vendor Master & Payment files
======================== - sample RDW variable length files
#4a. cp $UV/mf/cpys/* cpys <-- COBOL copybooks
=====================
#4b. cp $UV/mf/maps/* maps <-- cobol copybook MAPS (record layouts)
=====================
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
/home/userxx :-----dat1 <-- test/demo data files : :----- custmas0 - EBCDIC file with 24 5 byte packed fields : :----- custmas1 - ASCII file with 24 5 byte packed fields : :----- custmas2 - ASCII file name&address without packed fields : :----- sales0 - EBCDIC sales details with unpacked fields : :----- sales1 - ASCII with EBCDIC sign conventions : :----- sales2 - ASCII with Micro Focus COBOL sign conventions : :----- sales3 - ASCII with separate +/- signs : :----- vendormas3 - Vendor master RDW variable length file : :----- vendorpaymas3 - Vendor payment RDW variable length file
:-----cpys <-- COBOL copybooks : :-----custmas1.cpy - customer master Name&Adrs + thisyr/lastyr sales : :-----sales.cpy - detail sales with zoned signs : :-----sales3.cpy - detail sales with separate +/- signs : :-----vendormas.cpy - vendor master variable length RDW sample file
:-----maps <-- record layouts (maps) from copybooks : :----- custmas1 : :----- sales : :----- sales3 : :----- vendormas
:-----tmp <-- for your output files
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Mainframe files can not be displayed with most unix/linux utilities because they are usually fixed-length EBCDIC & do not have the LineFeeds required by most unix/linux utilities. 'uvhd' is ideal for this task.
uvhd dat1/custmas1 r256as2 - display testfile, option 'r256'=record-size, ========================== 'a'=translate to ASCII, 's2'=double space
now=20110128:1615 uvhd /home/uvadm/dat1/custmas0 a
version=20110114 records=32 rsize=256 fsize=8192
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 130140 EVERGREEN MOTORS LTD. 1815 BOWEN ROAD NANA
FFFFFF4444CECDCDCCD4DDEDDE4DEC44444FFFF4CDECD4DDCC4444444444DCDC
130140000055597955504636920334B000018150266550961400000000005151
64 IMO BC V9S1H1 250-754-5531 LARRY WRENCH ...V@...
CDD4444444444CC4EFEFCF4444FFF6FFF6FFFF4DCDDE4EDCDCC4444401357000
946000000000023059218100002500754055310319980695538000000246C000
128 .........W.....-.........X'..f.....m*..................V'...f...
0000000005300016000000002570063100095000000000000000013570016000
0C0000C0270D0540C0000C0098D0263C0444C0000C0000C0000C0246D0056C00
192 .E@...V'........c..............f..............f.C 19950531
047000570000000088000000001800068000130000000069C4FFFFFFFF444444
35C0046D0000C0023C0000C0083C0056D0012C0000C0016D3019950531000000
rec#=1 rsize=256 fptr=0 fsize=8192 rcount=32
null=next,r#=rec,s=search,u=update,x=undo,p=print,i=iprint,w=write,e=count
,R#=Recsize,h1=char,h2=hex,g=genseq#,c=chkseq#,t=translate(ta,te,tu,tl,tc,tp)
,q=quit,k0=helpoff,k1=helpon,?=HelpScreens -->
As you see above there are 32 records in the custmas0 file and it would take a lot of space it we showed them all with uvhd.
We provide other versions of the customer master file with the Name & Address paortion extracted to a separate file. See 'custmas2' listed on the next page.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
001-006 - customer#
009-009 - delete flag (D)
011-035 - customer name
036-060 - address
061-076 - city
078-079 - province
081-090 - postal code
091-102 - phone#
103-120 - contact name
121-180 - this year sales 12 x 5 byte packed fields
181-240 - last year sales 12 x 5 byte packed fields
241-244 - credit codes
245-250 - date of last change
251-256 - unused (no LineFeed at end record)
We need 'uvhd' to investigate this file. We cannot use the unix/linux utilities (such as vi, cat, more, lp) becuase of the packed fields & no LineFeeds. But, here is 'dat1/custmas3' which has 1st 80 bytes extracted to a separate file.
130140 EVERGREEN MOTORS LTD. 1815 BOWEN ROAD NANAIMO BC 132588 GEECOE GENERATOR SERVICESUNIT 170 - 2851 SIMPSON RICHMOND BC 139923 JOHNSTONE BOILER & TANKS 1250 EAST PENDER STREET VANCOUVER BC 142175 LILLY ELECTRIC (1973) LTD16809 - 24TH AVENUE HOPE BC 145264 D MAGRATH SUPPLIES LTD. 1939 KIRSCHNER ROAD KELOWNA BC 147615 X O'CONNER R.V. CENTRE 44430 YALE ROAD WEST CHILLIWACK BC 149304 POINT GREY GOLF & COUNTRY3350 S.W. MARINE DRIVE VANCOUVER BC 150825 RIGGERS INDUSTRIAL 960 - 6TH AVENUE HOPE BC 152355 SHAW, JOHN 477 CARIBOO CRES. COQUITLAM BC 154689 FRANKLIN RESOUCES 805 - 1199 HOWE STREET VANCOUVER BC 157352 D WHYTE, W. 1150 LODGE ROAD NORTH VANCOUVER BC 173320 X ZENITH TRANSPORT 2381 ROGERS AVE COQUITLAM BC 201120 ALLTYPE RENTAL LTD. BOX 1819 GRANDE PRAIRIE AL 204700 CASE POWER EQUIPMENT 12611-100 ST GRANDE PRAIRIE AL 208060 E&L TRUCKING LAND & CATTLE LTD THREE HILLS AB 211140 D FORD NEW HOLLAND BOX 1, 440 ELLIS ROAD FORT MCMURRAY AB 223240 X NICHOLSON CHEVROLET LTD. 7215 ARGYLL ROAD EDMONTON AL 224700 OTTO MOBILES WESTERN LTD.8441 CORONET ROAD EDMONTON AL 231550 SPARROW ELECTRIC CO. LTD.BOX 88 THREE HILLS AL 239300 TERRITORIAL REWIND BOX 1648 WHITEHORSE YK 237286 WEBER, TOM BOX 5503 FORT MCMURRAY AL 301120 D ALBERTA GAS CHEMICALS LTD3RD. FLOOR, 11456 JASPER EDMONTON AL 306959 X 356582 ALBERTA LTD. DEVITT NURSERY CALGARY AL 308685 FOOTHILLS ELECTRIC 3932 - 3A ST. N.W. CALGARY AL 313720 MONITREX ENGINEERING LTD 7-2280-39 AVE, N.E. WHITEHORSE YK 315512 PARTS PLUS BOX 510 MAIN ST THREE HILLS AB 318833 TOP NOTCH CONSTRUCTION BOX 308, STN J CALGARY AL 400002 ACKLANDS LTD 945 -2ND AVE PRINCE GEORGE BC 401210 COAST RANGE CONSTRUCTION 1103-207 W. HASTINGS ST VANCOUVER BC 402875 HULL, DON & SONS LTD. BOX 1297 PRINCE GEORGE BC 403887 D MILNER, LARRY BOX 28 RAU ROAD R.R.8 DAWSON CITY YK 406082 PRECAM RENTALS LTD: 10116-94TH AVE CHILLIWACK BC
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
These are Fixed length 64 byte records.
| sales0 |
|
| sales1 |
|
| sales2 |
|
| sales3 |
|
01-06 - CUST#
11-12 - SLSMN#
14-19 - DATE
21-28 - INVOICE#
31-36 - PRODUCT#
39-44 - QTY
46-52 - PRICE
54-62 - AMOUNT
64-64 - linefeed terminated
1 2 3 4 5 6
1234567890123456789012345678901234567890123456789012345678901234
CUST# SLM DATE INV# PROD# QTY PRICE AMOUNT
130140 21 940802 IN111001 HAM001 00002{ 0000001 00000002{
130140 21 940802 IN111001 SCR012 00002A 0001001 00002102A
139923 35 950802 IN111002 CHR001 00002B 0002001 00004402B
139923 35 950802 IN111002 TAB013 00002C 0003001 00006902C
139923 35 950807 CR5421 TAB013 00002D 0004001 00009602D
150825 44 960804 IN1122 HAM001 00002E 0005001 00012502E
150825 44 960804 IN1122 PLN012 00002F 0006001 00015602F
201120 44 970807 CR5234 WHIP75 00002G 0007001 00018902G
223240 65 980816 CR955 HAM001 00002H 0008001 00022402H
223240 65 980816 IN441 BIT001 00002I 0009001 00026102I
308685 21 990812 IN6605 SAW001 00001} 0000001 00000001}
308685 21 990812 IN6605 WID115 00001J 0001001 00001101J
308685 21 990812 CR8835 TAB013 00001K 0002001 00002401K
315512 44 000805 IN2251 HAM001 00001L 0003001 00003901L
315512 44 000805 IN2251 SAW051 00001M 0004001 00005601M
315512 44 000805 IN2255 WID115 00001N 0005001 00007501N
400002 85 010812 CR245 HAX129 00001O 0006001 00009601O
406082 35 020815 IN33001 BBQ001 00001P 0007001 00011901P
406089 35 020815 IN33001 TAB013 00001Q 0008001 00014401Q
406082 65 020816 IN441 HAM001 00001R 0009001 00017101R
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
1 2 3 4 5 6
1234567890123456789012345678901234567890123456789012345678901234
CUST# SLM DATE INV# PROD# QTY PRICE AMOUNT
130140 21 940802 IN111001 HAM001 000020 0000001 000000020 130140 21 940802 IN111001 SCR012 000021 0001001 000021021 139923 35 950802 IN111002 HAM001 000022 0002001 000044022 139923 35 950802 IN111002 TAB013 000023 0003001 000069023 139923 35 950807 CR5421 TAB013 000024 0004001 000096024 150825 44 960804 IN1122 HAM001 000025 0005001 000125025 150825 44 960804 IN1122 PLN012 000026 0006001 000156026 201120 44 970807 CR5234 WHIP75 000027 0007001 000189027 223240 65 980816 CR955 HAM001 000028 0008001 000224028 223240 65 980816 IN441 BIT001 000029 0009001 000261029 308685 75 990812 IN6605 SAW001 00001p 0000001 00000001p 308685 75 990812 IN6605 WID115 00001q 0001001 00001101q 308685 75 990812 CR8835 TAB013 00001r 0002001 00002401r 315512 33 000805 IN2251 HAM001 00001s 0003001 00003901s 315512 33 000805 IN2251 SAW051 00001t 0004001 00005601t 315512 33 000805 IN2255 WID115 00001u 0005001 00007501u 400002 85 010812 CR245 HAX129 00001v 0006001 00009601v 406082 55 020815 IN33001 BBQ001 00001w 0007001 00011901w 406082 55 020815 IN33001 TAB013 00001x 0008001 00014401x 406082 65 020816 IN441 HAM001 00001y 0009001 00017101y
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
1 2 3 4 5 6
1234567890123456789012345678901234567890123456789012345678901234
CUST# SLM DATE INV# PROD# QTY PRICE AMOUNT
130140 21 940802 IN111001 HAM001 00020+ 0000001 00000020+ 130140 21 940803 IN111001 SCR012 00021+ 0001001 00021021+ 130140 21 940804 IN111001 CHR001 00022+ 0002001 00044022+ 139923 35 950802 IN111002 TAB013 00023+ 0003001 00069023+ 139923 35 950807 IN111002 TAB013 00024+ 0004001 00096024+ 150825 44 960804 IN1122 HAM001 00025+ 0005001 00125025+ 150825 44 960805 IN1122 HAX129 00026+ 0006001 00156026+ 201120 44 970807 CR5234 WHIP75 00027+ 0007001 00189027+ 223240 65 980816 CR955 HAM001 00028+ 0008001 00224028+ 223240 65 980817 IN441 BBQ001 00029+ 0009001 00261029+ 308685 21 990812 IN6605 SAW051 00010- 0000001 00000010- 308685 21 990813 IN6605 WHIP75 00011- 0001001 00011011- 308685 21 990814 CR8835 TAB013 00012- 0002001 00024012- 315512 44 000805 IN2251 HAM001 00013- 0003001 00039013- 315512 44 000806 IN2251 SAW051 00014- 0004001 00056014- 315512 44 000807 IN2255 WHIP75 00015- 0005001 00075015- 400002 85 010812 CR245 HAX129 00016- 0006001 00096016- 406082 35 020815 IN33001 BBQ001 00017- 0007001 00119017- 406082 35 020816 IN33001 TAB013 00018- 0008001 00144018- 406082 65 020816 IN441 HAM001 00019- 0009001 00171019-
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
uvhd dat1/vendormas3 z4 <-- option 'z4' identifies RDW files ======================= (record size in 4 byte binary prefix)
now=20110521:0920 uvhd /home/uvadm/dat1/vendormas3 z4 version=20110519 lastmod=2011031421 print=p99 records=8 rsize=23 fsize=249
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 ....DELL10 M Dell Inc.
01004444332422466624662
070045CC100D0045CC09E3E
10 20 30 40 50 60
r# 2 0123456789012345678901234567890123456789012345678901234567890123
23 ....HP0010 M Hewlett Packard
010045333324224676677256666762
0E008000100D00857C5440013B1240
10 20 30 40 50 60
r# 3 0123456789012345678901234567890123456789012345678901234567890123
53 .-..IBM010 M International Business Machines
020044433324224676766766666247766677246666667
0D0092D0100D009E452E149FE1C02539E5330D1389E53
10 20 30 40 50 60
r# 4 0123456789012345678901234567890123456789012345678901234567890123
98 ....MFC010 M Micro Focus COBOL
0100444333242246676246677244444
0F00D630100D00D932F06F35303F2FC
10 20 30 40 50 60
r# 5 0123456789012345678901234567890123456789012345678901234567890123
129 ....MS0010 M Microsoft Corp.
01004533332422466767667246772
0D00D300100D00D932F3F6403F20E
10 20 30 40 50 60
r# 6 0123456789012345678901234567890123456789012345678901234567890123
158 ....REDH10 M Red Hat Linux
010054443324225662467246677
0B002548100D0025408140C9E58
10 20 30 40 50 60
r# 7 0123456789012345678901234567890123456789012345678901234567890123
185 ."..SUN010 M Sun Microsystems Ltd
0200554333242257624667677776672476
020035E0100D0035E0D932F39345D30C44
10 20 30 40 50 60
r# 8 0123456789012345678901234567890123456789012345678901234567890123
219 ....UVSI10 M UV Software Inc.
010055543324225525667767624662
0E005639100D005603F64712509E3E
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
uvhd dat1/vendormas3 z4 <-- option 'z4' identifies RDW files =======================
uvhd dat1/vendormas3 z4 <-- start uvhd on desired file
======================= - displays 1st record & prompts for command
--> p8 <-- print(write) 8 records to tmp/...
uvlp12 tmp/vendormas3_110521_181300P <-- list Print command output
====================================
export UVLPDEST=-dlp0
=====================
'-d' is the destination option & 'lp0' is the printer (no space allowed)
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
'vendormas3B' is a BDW/RDW file, similar to 'vendormas3' RDW file shown above. In addition to the 3 byte binary RDW prefix, the records are Blocked & each block is preceded with a BDW prefix.
Here is the 1st block of the file, shown with uvhd using option 'r256' - not option z4 (RDW) or option z8 (BDW). 'r256' displays 256 byte fixed-length records, which in this case happens to be the BDW Block size.
The 1st BDW in 1st 2 bytes is x'0100' which is 256, the RDW of 1st record is in bytes 4&5 x'0018' which is 24 (1*16+8=24). Do you see that there are 8 vendor master records in this block ?
uvhd /home/uvadm/dat1/vendormas3B r256 ======================================
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 ........DELL10 - Dell Inc. ....HP0010 - Hewlett Packard.,..IBM0
0000010044443322246662466222010045333322246766772566667602004443
1000080045CC100D045CC09E3E000C008000100D0857C5440013B1240C0092D0
64 10 - International Business Machines. ..MFC010 - Micro Focus COB
3322246767667666662477666772466666670200444333222466762466772444
100D09E452E149FE1C02539E5330D1389E530000D630100D0D932F06F35303F2
128 OL ....MS0010 - Microsoft Corp.....REDH10 - Red Hat Linux .$..
4422010045333322246676766724677201005444332225662467246677220200
FC000C00D300100D0D932F3F6403F20E0C002548100D025408140C9E58000400
192 SUN010 - Sun Microsystems Ltd . ..UVSI10 - UV Software Inc.
5543332225762466767777667247622202005554332225525667767624662222
35E0100D035E0D932F39345D30C4400000005639100D05603F64712509E3E000
Use option 'z8' to display individual records of a BDW/RDW file. Note that uvhd does not show the 'BDW's, so the RDW of the 1st record below starts at displacement '4'. Here are 1st 2 records (same as RDW on page '1D1').
uvhd /home/uvadm/dat1/vendormas3B z8 ====================================
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
4 ....DELL10 - Dell Inc.
010044443322246662466222
080045CC100D045CC09E3E00
10 20 30 40 50 60
r# 2 0123456789012345678901234567890123456789012345678901234567890123
28 ....HP0010 - Hewlett Packard
0100453333222467667725666676
0C008000100D0857C5440013B124
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
'vendorpaymas' is a demo/sample file with 2 record types (Master & Payment). The file is supplied in 4 different file types:
| vendorpaymas0 |
|
| vendorpaymas1 |
|
| vendorpaymas2 |
|
| vendorpaymas3 |
|
cat dat1/vendorpaymas <-- display text file version (only 16 records) =====================
DELL10 M Dell Inc. DELL10 P 20100131 INV00010 000195000 Dell server 2900 HP0010 M Hewlett Packard HP0010 P 20100215 INV00020 000042500 HP LCD Monitor HP0010 P 20100215 INV00020 000350000 HP XW9400 Workstation IBM010 M International Business Machines IBM010 P 20100321 INV00030 000750000 AIX server IBM010 P 20100322 INV00030 000750000 DB2 database software MFC010 M Micro Focus COBOL MFC010 P 20100420 INV00040 000320000 Micro Focus Developer MS0010 M Microsoft Corp. MS0010 P 20100501 INV00050 000011600 Microsoft Windows 7 REDH10 M Red Hat Linux REDH10 P 20100620 INV00060 000034900 Red Hat Enterprise 5.4 SUN010 M Sun Microsystems Ltd UVSI10 M UV Software Inc.
uvhd /home/uvadm/dat1/vendorpaymas3 z4 <-- display RDW file (1st 2 records) ====================================== - uvhd shows binary rec-size prefixes
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 ....DELL10 M Dell Inc.
01004444332422466624662
070045CC100D0045CC09E3E
rec#=1 rsize=23 fptr=0 fsize=745 rcount=16
10 20 30 40 50 60
r# 2 0123456789012345678901234567890123456789012345678901234567890123
23 .<..DELL10 P 20100131 INV00010 000195000 Dell server 2900
030044443325223333333322445333332233333333324666276776723333
0C0045CC10000020100131009E60001000000195000045CC035265202900
rec#=2 rsize=60 fptr=23 fsize=745 rcount=16
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| 2A1. | uvhd Program Description |
| - data file investigation utility | |
| - ideal for mainframe EBCDIC files with packed/binary | |
| - browse,search,select,update,print,translate,etc |
| 2A2. | uvhd Display format illustrated |
| - displays any file in vertical hexadecimal | |
| - 64 byte segments on 3 lines (characters, zones,& digits) |
| 2B1. | Investigating EBCDIC files |
| - demo file custmas0 EBCDIC vs custmas1 ASCII |
| 2B2. | Display EBCDIC file with option 'a' |
| - to translate character line to ASCII so you can read on unix/linux |
| 2C1. | Packed Decimal fields |
| - sample display in vertical hexadecimal |
| 2D1. | Example of using uvhd to translate EBCDIC to ASCII |
| - character fields only, packed fields must be preserved unchanged | |
| - could use uvhd (free download) for simple record layouts | |
| - the full package has comprehensive EBCDIC to ASCII tools, driven by | |
| the COBOL copybooks to translate high volumes of complex record layouts |
This manual is a good place to start learning uvhd. When you need more details please see complete documentation at https://uvsoftware.ca/uvhd.htm.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
'uvhd' is a file investigation utility. It displays any file in vertical hexadecimal and prompts for commands to browse, search, select, update, scan/replace, print, translate, etc. uvhd is an interactive utility with a command line interface and 23 help screens. uvhd is the favorite utility of most UV Software customers.
uvhd is a free download from: https://www.uvsoftware.ca/freestuff.htm
'uvhd' is offered as a Free sample of the Vancouver Utilties which are often used to convert mainframe JCL, COBOL,& DATA to Unix/Linux/Windows Mainframe JCL is converted to Korn shell scripts, and Mainframe COBOL is converted to Micro Focus COBOL to run under Unix/Linux.
uvhd is especially valuable to investigate mainframe type files with packed fields and no linefeeds (required by most unix/linux utilities). uvhd can investigate all file types: sequential, fixed, variable, indexed, text, binary, C-ISAM, D-ISAM, Micro Focus COBOL IDXFORMAT1,3,8)
uvhd displays any file in vertical hexadecimal, 64 byte segments, in 3 line groups (characters on 1st line, zones on 2nd line,& digits on 3rd line).
When you find a data problem, use the print function to capture the evidence and send it to your associates via email or hardcopy printout. Hardcopy or softcopy printouts of binary/packed files with byte position scales are not possible with most other unix utilities.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Records are displayed 256 bytes at a time by default (if rcsz not spcfd) in 4 groups of 3 vertical hex lines as follows:
...........record in character format (. for non-dspl chars)....
...........hex representation of record zones...................
...........hex representation of record digits..................
uvhd dat1/custmas1 r256s2 - command to display the testfile ========================= options: r256=record-size, s2=double space
now=20110128:0657 uvhd /home/uvadm/dat1/custmas1 r256s2
version=20110114 records=32 rsize=256 fsize=8192
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 130140 EVERGREEN MOTORS LTD. 1815 BOWEN ROAD NANA
3333332222454545444244545524542222233332445442544422222222224444
130140000056527255E0DF4F230C44E0000181502F75E02F140000000000E1E1
64 IMO BC V9S1H1 250-754-5531 LARRY WRENCH ..4V|...
4442222222222442535343222233323332333324455525544442222201357000
9DF00000000002306931810000250D754D55310C12290725E38000000246C000
128 .........W0....`........)X}..f3.....\.................4V}...f...
0000000005300016000000002570063100095000000000000000013570016000
0C0000C0270D0540C0000C0098D0263C0444C0000C0000C0000C0246D0056C00
192 .E|...V}.......................f.....<........f.C 19950531
0470005700000000880000000018000680001300000000694233333333222222
35C0046D0000C0023C0000C0083C0056D0012C0000C0016D3019950531000000
rec#=1 rsize=256 fptr=0 fsize=8192 rcount=32
null=next,r#=rec,s=search,u=update,x=undo,p=print,i=iprint,w=write,e=count
,R#=Recsize,h1=char,h2=hex,g=genseq#,c=chkseq#,t=translate(ta,te,tu,tl,tc,tp)
,q=quit,k0=helpoff,k1=helpon,?=HelpScreens -->
In this example we entered 'q' to quit after the 1st 256 bytes of the file was displayed, but we could have entered a record# or byte# to explore other areas of the file. Note that the current record# & byte# are displayed at the left of the two top lines (the 0-64 byte scale).
The uvhd display is especially useful for displaying data files that do not have line feeds & therefor are not easily displayed by an editor & for files with packed decimal fields.
Any non displayable characters is represented by periods on the character line (1st line of 3 vertical hex lines). The above sample has a lot due to the multiple packed fields.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Both ASCII & EBCDIC versions of a few demo files are provided. Our convention is to use suffix '0' to indicate EBCDIC & '1' for ASCII.
| custmas0 |
|
| custmas1 |
|
| sales0 |
|
| sales1 |
|
uvhd provides option 'a' to view EBCDIC files. First we will show you the display if you run uvhd on the custmas0 EBCDIC file with NO options. The default record size is 256 which happens to be the record size of custmas0.
uvhd dat1/custmas0 <-- investigate EBCDIC file with NO options ==================
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 ......@@@@.........@......@...K@@@@....@.....@....@@@@@@@@@@....
FFFFFF4444CECDCDCCD4DDEDDE4DEC44444FFFF4CDECD4DDCC4444444444DCDC
130140000055597955504636920334B000018150266550961400000000005151
64 ...@@@@@@@@@@..@......@@@@...`...`....@@@@@@@@@@@@@@@@@@........
CDD4444444444CC4EFEFCF4444FFF6FFF6FFFF44444444444444444400000000
946000000000023059218100006040754055310000000000000000000000C000
128 .........W0....`........)X|..f3.....\.......................f...
0000000005300016000000002570063100095000000000000000000000016000
0C0000C0270C0540C0000C0098C0263C0444C0000C0000C0000C0000C0056C00
192 .E|...V}.......................f.....<........f..@@@@@@@@@@@@@@@
047000570000000088000000001800068000130000000069C444444444444444
35C0046D0000C0023C0000C0083C0056D0012C0000C0016D3000000000000000
rec#=1 rsize=256 fptr=0 fsize=8192 rcount=32 -->
Of course we can NOT read the data on unix/linux ASCII systems. 'uvhd' provides option 'a' to perform EBCDIC to ASCII translation on the character line of the 64 byte 3 line groups (characters, zones, digits). You can then 'read' the EBCDIC data. Note that the zones & digit lines will show the EBCDIC values. Here is just the 1st 64 byte segment as shown with option 'a'.
Chars--> 0 130140 EVERGREEN MOTORS LTD. 1815 BOWEN ROAD NANA Zones--> FFFFFF4444CECDCDCCD4DDEDDE4DEC44444FFFF4CDECD4DDCC4444444444DCDC Digits-> 130140000055597955504636920334B000018150266550961400000000005151
See the complete 1st record of custmas0 shown below with option 'a'.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
uvhd dat1/custmas0 r256as2 <-- redisplay EBCDIC demo file, with option 'a'
========================== - option 'r256' is the default & could be omitted
- option 's2' inserts space between 3 line groups
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 130140 EVERGREEN MOTORS LTD. 1815 BOWEN ROAD NANA
FFFFFF4444CECDCDCCD4DDEDDE4DEC44444FFFF4CDECD4DDCC4444444444DCDC
130140000055597955504636920334B000018150266550961400000000005151
64 IMO BC V9S1H1 604-754-5531 ........
CDD4444444444CC4EFEFCF4444FFF6FFF6FFFF44444444444444444400000000
946000000000023059218100006040754055310000000000000000000000C000
128 .........W.....-.........X@..f.....m*.......................f...
0000000005300016000000002570063100095000000000000000000000016000
0C0000C0270C0540C0000C0098C0263C0444C0000C0000C0000C0000C0056C00
192 .E@...V'........c..............f..............f.C
047000570000000088000000001800068000130000000069C444444444444444
35C0046D0000C0023C0000C0083C0056D0012C0000C0016D3000000000000000
rec#=1 rsize=256 fptr=0 fsize=8192 rcount=32 -->
Please compare the EBCDIC file display above with the ASCII file shown 2 pages back at '2B2'.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Note that there are 24 * 5 byte packed fields in 120-240. You can identify the end of each packed field by the sign, x'_C' or x'_D'in the last byte. For example here is the 1st 8 bytes of last segment:
192 .E@...V' <-- character line
04700057 <-- zones line
35C0046D <-- digits line
^ ^ <-- '^' marks sign/end of each packed field
Option 'a' is relevant to sites that are converting from an IBM mainframe to Unix, Linux, or Windows/SFU. The data files are transferred to the new machine where they are converted from EBCDIC to ASCII (see DATAcnv1.htm).
The conversion jobs are generated automatically from the COBOL copybooks. All character type & unpacked numeric fields are translated from EBCDIC to ASCII, while preserving any packed or binary fields unchanged (since they would be destroyed by the translation).
'uvhd' with option 'a' is a vital asset to verify the data conversions, you can spot check the conversion by examining both the input EBCDIC data & the output ASCII file.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Vancouver Utilities has very comprehensive tools to translate EBCDIC files to ASCII based on the COBOL copybook record layouts - so only character fields are translated & packed fields are preserved unchanged (since translating packed fields would destroy them). Please see DATAcnv1.htm#Part_3.
But, here is an example of using uvhd to translate simple record layouts. You might use this if you have only the free 'uvhd' & not the full Vancouver Utilities package required for the comprehensive translation based on the COBOL copybooks.
This exercise assumes you have setup the directories as shown on page '1A2'. If you are using the free uvhd download, we also assume you have downloaded the demo files from https://www.uvsoftware.ca/VUdemofiles.zip and copied the relevant file (custmas0) to subdir dat1/...
#1. mkdir tmp1 <-- make subdir for translated file
========== - to preserve input demo file
#2. cp dat1/custmas0 tmp1 <-- copy EBCDIC demo file to tmp1/custmas0
===================== - to preserve input demo file
#3. uvhd tmp1/custmas0 r256ua <-- initiate uvhd
========================= - option 'u' required to modify records
- option 'a' to show EBCDIC chars in ASCII
#3a. t99e 0(120) <-- translate 1st 120 bytes of all records to Ebcdic
===========
#3b. 1 <-- return to 1st record in file
===
#3c. t99e 240(16) <-- translate last 16 bytes of all records to Ebcdic
============
#3d. q <-- quit uvhd, so we can rerun without option 'a'
===
#4. uvhd tmp1/custmas0 r256 <-- rerun uvhd
======================= - omitting option 'a' (or 'u')
The records in tmp1/custmas0 will now appear as shown on page '2A2' for the ASCII file dat1/custmas1. Note that the packed fields have been preserved.
The 't' command will translate to Ascii,Ebcdic,Upper,Lower,Characters,or Periods (depending on options a,e,u,l,c,p). Some other translate examples:
| ta |
|
| t99e |
|
| tl 0(120) |
|
| tc1 |
|
| tc15 |
|
See full doc for uvhd at https://uvsoftware.ca/uvhd.htm
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| 3A1. | uvhdcob overview |
| - utility to display data field contents with COBOL copybook field names | |
| - sample display using customer master demo file & copybook |
| 3B1. | Preparation to use uvhdcob. |
| - must convert copybook to 'cobmap' record layout | |
| - using utility provided 'uvcopy cobmap1' |
| 3C1. | Operating Instructions for 'uvcopy cobmap1' |
| - sample 'cobmap' for the customer master demo file |
| 3D1. | Verify data command (v) |
| - creating BAD data for demo |
| 3D2. | test/demo verify command |
| - sample display with BAD data fields indicated | |
| Bad-Numeric, Bad-Packed, Bad-Character |
| 3D3. | Notes re verify test/demo |
| 3E1. | Options for Verify command |
| options a-b,c-d,e-f to inhibit Verify on up to 3 areas | |
| - identified by start & end byte displacements | |
| - allows you to search/Verify for other fields in record | |
| options x1,x2,x4 to inhibit stop-on-err for numeric/packed/character fields | |
| - continues to EOF & displays error counts & record#s of 1st & last error | |
| options y1,y2,y4 to specify signs: zoned ASCII, zoned EBCDIC, separate +/- | |
| options z1,z2,z4 to convert all blank numeric/packed fields to zeros | |
| - or convert unprintables in character fields to '.' periods | |
| - option 'p' to print formatted err records to tmp/filename_yymmdd_hhmmssVP | |
| - option 'w' to write original err records to tmp/filename_yymmdd_hhmmssVW | |
| - may follow with lvp/lvw to close files or omit to combine outputs |
| 3F1. | Create verify report for entire file --> v99p99x7 |
| - option p99 creates separate tmp/report with error records only | |
| - option x7 inhibits normal stop/display | |
| - quit uvhdcob & use vi/more/lp/uvlp12 on tmp/report | |
| See sample report for dat1/custmas1x with errs created on 3F1 |
This manual is a good place to start learning uvhdcob. When you need more details please see complete documentation at https://uvsoftware.ca/uvhdcob.htm.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
uvhdcob datafile=dat1/custmas1 20090222:0844 copybook=maps/custmas1 options= record#=1 totalrecs=32 recsize=256 filesize=8192 fileptr=0 rec# 1 fieldname occurs bgn end typ<------ data (hex if typ=p/b) -----> 001 cm-num 0 5 n 130140 002 cm-delete 6 9 003 cm-name 10 34 EVERGREEN MOTORS LTD. 004 cm-adrs 35 59 1815 BOWEN ROAD 005 cm-city 60 75 NANAIMO 006 filler001 76 76 007 cm-prov 77 78 BC 008 filler002 79 79 009 cm-postal 80 89 V9S1H1 010 cm-phone 90 101 250-754-5531 011 cm-contact 102 119 LARRY WRENCH 012 cm-thisyr-sales 012 120 124pns 001234567C 013 cm-lastyr-sales 012 180 184pns 001234567D 014 filler003 240 255 C 19950531 rec#=1 rcount=32 rsize=256 fsize=8192 dat1/custmas1 null=next,r#=rec,s=search,u=update,x=undo,p=print,i=iprint,w=write,t=tally ,v=verify,c=chkseq#,e=exit to uvhd,q=quit,?=help -->
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
You can test/demo uvhdcob by copying supplied test files to subdirs in your homedir.
#1. Login --> your homedir /home/userxx/
=====
#2a. mkdir dat1 <-- setup subdir for data files
==========
#2b. mkdir cpys <-- setup subdir for copybooks
==========
#2c. mkdir maps <-- subdir for cobmaps(record layouts created from copybooks)
==========
#2d. mkdir tmp <-- subdir for uvhdcob outputs (prints, selects, verifies)
=========
#3a. cp $UV/dat1/custmas* dat1 <-- customer master files
========================= - sample fixed length with packed fields
#3b. cp $UV/dat1/sales* dat1 <-- sales files
======================= - sample fixed length with numeric fields
#3c. cp $UV/dat1/vendor* dat1 <-- Vendor Master & Payment files
======================== - sample RDW variable length files
#4a. cp $UV/mf/cpys/* cpys <-- COBOL copybooks
=====================
#4b. cp $UV/mf/maps/* maps <-- cobol copybook MAPS (record layouts)
=====================
#5. uvcopyx cobmap1 cpys maps uop=q0i7 <-- convert copybooks to cobmaps
================================== see illustration next page -->
#6a. uvhdcob dat1/custmas1 maps/custmas <-- test uvhdcob for customer master
================================== see illustration 2 pages ahead -->
#6b. uvhdcob dat1/custmas0 maps/custmas a <-- retest with EBCDIC file
==================================== - option 'a' translate to ASCII
#6c. uvhdcob dat1/sales1 maps/sales <-- test uvhdcob on sales detail
==============================
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
uvhdcob requires prior generation of 'cobmaps' from the COBOL copybooks. You could regenerate the cobmaps as follows:
#1. Login --> your homedir
=====
#2. uvcopyx cobmap1 cpys maps uop=q0i7
==================================
- generate cobmaps for all files in directory
- easier than for 1 at a time (as shown below)
#2a. uvcopy cobmap1,fili1=cpys/custmas1,filo1=maps/custmas1
======================================================
- generate cobmap for 1 copybook at a time
cobmap1 start-end bytes for cobol record fields 199905101503 pg# 0001
cpys/custmas1 RCSZ=0256 bgn-end lth typ
* custmas1 - cobol copy book for custmas1 file (see uvtrain.
10 cm-num pic 9(6). 000-0005 006 n 06
10 cm-delete pic x(4). 006-0009 004
10 cm-name pic x(25). 010-0034 025
10 cm-adrs pic x(25). 035-0059 025
10 cm-city pic x(16). 060-0075 016
10 filler001 pic x. 076-0076 001
10 cm-prov pic x(2). 077-0078 002
10 filler002 pic x. 079-0079 001
10 cm-postal pic x(10). 080-0089 010
10 cm-phone pic x(12). 090-0101 012
10 cm-contact pic x(18). 102-0119 018
10 cm-thisyr-sales pic s9(7)v99 comp-3 occurs 12. 120-0124 005pns 09
10 cm-lastyr-sales pic s9(7)v99 comp-3 occurs 12. 180-0184 005pns 09
10 cm-thisyr-xft pic s9(7)v99 comp-3. 240-0244 005pns 09
10 cm-lastyr-xft pic s9(7)v99 comp-3. 245-0249 005pns 09
10 filler003 pic x(6). 250-0255 006
*RCSZ=0256 0256
Note that this sample cobmap was used for the uvhdcob sample illustrated on the previous page (please relate the fieldnames & locations).
Also note that the 'cobmaps' are very useful to COBOL shops for other reasons (when you need to know field locations for sorting, verifying data, etc).
You may find it convenient to print cobmaps for all copybooks, and mount in a 3 ring binder for easy reference. See documentation for the 'cobmap' utility in COBaids.doc .
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#1. Login --> your homedir
#2. ls -lr <-- list subdirs & files
====== (showing only customer master & sales files)
/home/userxx :-----dat1 <-- subdir for test/demo DATA : :----- custmas0 - EBCDIC file with 24 5 byte packed fields : :----- custmas1 - ASCII file with 24 5 byte packed fields : :----- custmas2 - ASCII file name&address without packed fields : :----- sales0 - EBCDIC sales details with unpacked fields : :----- sales1 - ASCII with EBCDIC sign conventions : :----- sales2 - ASCII with Micro Focus COBOL sign conventions : :----- sales3 - ASCII with separate +/- signs : :----- vendormas3 - Vendor master RDW variable length file : :----- vendorpaymas3 - Vendor payment RDW variable length file :-----cpys <-- COBOL copybooks : :-----custmas1.cpy - customer master Name&Adrs + thisyr/lastyr sales : :-----sales.cpy - detail sales with zoned signs : :-----sales3.cpy - detail sales with separate +/- signs : :-----vendormas.cpy - vendor master variable length RDW sample file :-----maps <-- record layouts (maps) from copybooks : :----- custmas1 : :----- sales : :----- sales3 : :----- vendormas :-----tmp <-- for your output files
#2. uvhdcob dat1/custmas1 maps/custmas <-- run uvhdcob specifying
================================== data-file & copybook
now=20110129:0723 uvhdcob dat1/custmas1 version=20110114 copybook=maps/custmas rec#=1 rcount=32 rsize=256 fsize=8192 fptr=0 rec# 1 occurs bgn end typ<------ data (hex if typ=p/b) -----> 001 cm-num 0 5 n 130140 002 cm-delete 6 9 003 cm-name 10 34 EVERGREEN MOTORS LTD. 004 cm-adrs 35 59 1815 BOWEN ROAD 005 cm-city 60 75 NANAIMO 006 filler001 76 76 007 cm-prov 77 78 BC 008 filler002 79 79 009 cm-postal 80 89 V9S1H1 010 cm-phone 90 101 250-754-5531 011 cm-contact 102 119 LARRY WRENCH 012 cm-thisyr-sales 012 120 124pns 001234567C 013 cm-lastyr-sales 012 180 184pns 001234567D 014 filler003 240 255 C 19950531 rec#=1 rcount=32 rsize=256 fsize=8192 fptr=0 k1=HelpPrompts,?=HelpScreens -->
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
The 'v' command will verify character, numeric,& packed data fields. Verify searches the file for unprintable characters in pic x fields & invalid digits or signs in unpacked or packed numeric fields.
We have provided a doctored version of dat1/custmas1 (dat1/custmas1x) to test/demo uvhdcob. dat1/custmas1x has been updated (using uvhdcob) to create errors in record#s 10 & 20.
Here are the 'update' commands we used to create the BAD data for the 'verify' command test/demo. We will omit the displays & list only the commands.
#1. Login --> your homedir (with subdirs shown on previous page)
=====
#2. cp dat1/custmas1 dat1/custmas1x <-- copy to alt file to make bad data
===============================
#3. uvhdcob dat1/custmas1x maps/custmas1 u <-- option 'u' for update
======================================
#4. --> 10 <-- goto record# 10
#5a. --> u 5(1),'*' <-- create non-numeric data in numeric field (cust#) #5b. --> u 78(1),x'EE' <-- create unprintable data in character field (prov) #5c. --> u 120(1),x'EE' <-- create invalid digits in packed field (thisyr sales)
#6. --> 20 <-- goto record# 20
#7a. --> u 5(2),'XX' <-- create non-num data in numeric field (cust#) #7b. --> u 83(3),x'EEEEEE' <-- create unprintables in char field (province) #7c. --> u 180(5),x'1E2E3E4E5E' <-- create BAD data in packed field (lastyr sls)
#8. uvhdcob dat1/custmas1x maps/custmas1 <-- run on demo file
====================================
- uvhdcob displays 1st record & prompts for command
- we are omitting 1st record display, see verify displays next page
--> v99 <-- search/verify until bad record found or EOF reached
- will display next bad record found & prompt for command
- if desired, could print(formatted) or write(raw) to tmp/...
--> vv <-- may then enter 'vv' to continue search/verify
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
The 'v' verify command searches from the current record until it finds a record with at least 1 bad data field & displays that record. For our demo file (dat1/custmas1x) record# 10 is the 1st record with bad data fields.
--> v99 <-- Verify
rec# 10 occurs bgn end typ<------ data (hex if typ=p/b) -----> 001 cm-num 0 5 n 15468* 31353436382A <-BadN 002 cm-delete 6 9 003 cm-name 10 34 FRANKLIN RESOUCES 004 cm-adrs 35 59 805 - 1199 HOWE STREET 005 cm-city 60 75 VANCOUVER 006 filler001 76 76 007 cm-prov 77 78 B. 42EE <-BadC 008 filler002 79 79 009 cm-postal 80 89 V6C2T6 010 cm-phone 90 101 604-666-5489 011 cm-contact 102 119 RON BENNETT 012 cm-thisyr-sales 012 120 124pns EE0123456C <-BadP 013 cm-lastyr-sales 012 180 184pns 000000000C 014 filler003 240 255 C 19980331
v99 --> 10 recs verified, 4 ERRs found in rec# 10, 1st at byte# 0
2 packed & 1 numeric bad signs/digits, 1 unprintables in char fields
--> vv <-- continue Verify
rec# 20 fieldname occurs bgn end typ<------ data (hex if typ=p/b) -----> 001 cm-num 0 5 n 2343XX 323334335858 <-BadN 002 cm-delete 6 9 003 cm-name 10 34 TERRITORIAL REWIND 004 cm-adrs 35 59 BOX 1648 005 cm-city 60 75 WHITEHORSE 006 filler001 76 76 007 cm-prov 77 78 YK 008 filler002 79 79 009 cm-postal 80 89 Y1A... 593141EEEEEE <-BadC 010 cm-phone 90 101 403-512-3948 011 cm-contact 102 119 012 cm-thisyr-sales 012 120 124pns 000000000C 013 cm-lastyr-sales 012 180 184pns 1E2E3E4E5E <-BadP 014 filler003 240 255 20021130
vv --> 10 recs verified, 10 ERRs found in rec# 20, 1st at byte# 0
5 packed & 2 numeric bad signs/digits, 3 unprintables in char fields
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
--> v99p99x7 <-- create a verify report for the entire file
======= for separate investigation & printing.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
abcdef - options to inhibit verify in 3 areas ID by offset from/to
v99a120b239 - inhibit verify in all fields from dsplcmnt 120 to 239
- to bypass known bad fields (to search for unknown bad fields)
v99a100b120c200d220e300f320 - inhibit verify in 3 record areas max
...x - option 'x' no-stop-on-error, count for display at EOF (or max# spcfd) v99x1 - nostop on errors in numeric fields v99x2 - nostop on errors in packed fields v99x4 - nostop on errors in character (pic x) fields v99x3 - nostop on errors in numeric + packed fields v99x7 - nostop on errors in numeric + packed + character fields
...y - option 'y' specify valid signs for numeric fields
v99y1 - (default) Micro Focus COBOL -0123456789=pqrstuvwxy x'70' zones
v99y2 - EBCDIC signs: +0123456789={ABCDEFGHI, -0123456789=}JKLMNOPQR
- defaults to y2 when input file EBCDIC (option 'a' on command line)
v99y4 - separate +/- signs (leading or trailing)
...z - option 'z' correct blank numeric/packed,& pic x with unprintable chars v99z1 - convert any all blank numeric fields to numeric zeros v99z2 - convert any all blank packed fields to packed zeros with x'_C' sign v99z4 - convert any any unprintable characters in pic x fields to '.' periods v99z7 - all of above
| v99x7p99 |
|
lw <-- may follow write command with 'lw' command to close file for separate
output files or omit for combined outputs from multi write commands
lp <-- similar command close print files for separate or omit for combined
lvw <-- similar command close verify writes for separate or omit for combined
lvp <-- similar command close verify prints for separate or omit for combined
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Use command --> v99p99x7 <-- to create a verify report for the entire file in the tmp/... subdir for separate investigation & printing. Option p99 writes only the error records into tmp/filename_yymmdd_hhmmssVP. Option x7 inhibits the default stop/display on each error record. You can then quit uvhd & use vi/more/lp/uvlp12/... on the tmp/report. Here is a demo using the dat1/custmas file with the 2 bad records created on page '3D1' & displayed on page '3D2'.
#1. uvhdcob dat1/custmas1x maps/custmas1 <-- run uvhdcob with demo files
====================================
--> ........ <-- displays 1st record (display omitted to save space here)
#1a. --> v99p99x7 <-- enter command 'v99' with options 'p99' & 'x7
============
#1b. --> q <-- quit uvhdcob
#2. more tmp/custmas1x_110101_084059VP <-- display verify error records
==================================
rec# 10 occurs bgn end typ<------ data (hex if typ=p/b) -----> 001 cm-num 0 5 n 15468* 31353436382A <-BadN 002 cm-delete 6 9 003 cm-name 10 34 FRANKLIN RESOUCES 004 cm-adrs 35 59 805 - 1199 HOWE STREET 005 cm-city 60 75 VANCOUVER 006 filler001 76 76 007 cm-prov 77 78 B. 42EE <-BadC 008 filler002 79 79 009 cm-postal 80 89 V6C2T6 010 cm-phone 90 101 604-666-5489 011 cm-contact 102 119 RON BENNETT 012 cm-thisyr-sales 012 120 124pns EE0123456C <-BadP 013 cm-lastyr-sales 012 180 184pns 000000000C 014 filler003 240 255 C 19980331
rec# 20 occurs bgn end typ<------ data (hex if typ=p/b) -----> 001 cm-num 0 5 n 2343XX 323334335858 <-BadN 002 cm-delete 6 9 003 cm-name 10 34 TERRITORIAL REWIND 004 cm-adrs 35 59 BOX 1648 005 cm-city 60 75 WHITEHORSE 006 filler001 76 76 007 cm-prov 77 78 YK 008 filler002 79 79 009 cm-postal 80 89 Y1A... 593141EEEEEE20202 <-BadC 010 cm-phone 90 101 403-512-3948 011 cm-contact 102 119 012 cm-thisyr-sales 012 120 124pns 000000000C 013 cm-lastyr-sales 012 180 184pns 1E2E3E4E5E <-BadP 014 filler003 240 255 20021130
v99p99x7 --> 32 recs verified, 2 with errs, 1st at rec# 10, last at rec# 20 ERRs: 12 packed, 3 numeric signs/digits, 4 unprintable characters --> v99p99x7 closing: tmp/cm1_110102_093123VP
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| 4A1. | uvcp demo#1 - reformat records, change file type & record-size |
| 4A2. | uvcp #2 - extract telephone list from Customer Name&Address file |
| 4A3. | uvcp #3 - record selection (keep or drop) |
| 4A4. | uvcp #4 - convert EBCDIC RDW variable length records to ASCII Text file |
| 4A5. | uvcp #5 - convert ASCII text file to EBCDIC RDW variable length records |
| 4A6. | uvcp #6 - create 'BDW/RDW' variable length records |
| 4B1. | uvsort demo#1 - sort sales3 demo file by product# |
| 4B2. | uvsort demo#2 - sort with Unique-Key option |
| - keep only the 1st record of duplicate key set | |
| 4B3. | uvsort demo#3 - 'sum' feature, keep only 1st record of product# group |
| - with accumulated total $amount |
| 4C1. | uvlist - Text file listing utility for scripts, programs, parameter files |
| - page headings with filename, when printed, userID, line#, Page# | |
| - over 50 convenient scripts for various cpi,lpi,Simplex,Duplex,etc |
| 4C2. | sample script 'uvlp12' calling uvlist |
| - add environmental variable 'UVLPDEST' to your profile | |
| - to direct listings to a network printer near you |
Part 4 presents a few brief examples of uvcp, uvsort,& uvlist. These should give you an idea of how useful these utilties may be to you - even better if you can take the time to execute these demos & compare your results to the results listed here.
The complete documentation for these utilties can be found at:
| uvcp | |
| uvsort | |
| uvlist |
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
'uvcp' is handy for reformatting records & extracting desired text info from large records with packed fields. For example, suppose we wish to extract the Name & Address from the customer master & sales history records. You can not do this with the 'vi' editor because there are no LineFeeds and even if there were the packed fields would distort the screen.
vi dat1/custmas1 <-- attempt to display with 'vi' ================ - won't work, get "line too long"
uvhd dat1/custmas1 r256 <-- display 1st record with uvhd OK =======================
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 130140 EVERGREEN MOTORS LTD. 1815 BOWEN ROAD NANA
3333332222454545444244545524542222233332445442544422222222224444
130140000056527255E0DF4F230C44E0000181502F75E02F140000000000E1E1
64 IMO BC V9S1H1 250-754-5531 LARRY WRENCH ..4V|...
4442222222222442535343222233323332333324455525544442222201357000
9DF00000000002306931810000250D754D55310C12290725E38000000246C000
128 .........W0....`........)X}..f3.....\.................4V}...f...
0000000005300016000000002570063100095000000000000000013570016000
0C0000C0270D0540C0000C0098D0263C0444C0000C0000C0000C0246D0056C00
192 .E|...V}.......................f.....<........f.C 19950531
0470005700000000880000000018000680001300000000694233333333222222
35C0046D0000C0023C0000C0083C0056D0012C0000C0016D3019950531000000
Here are the 1st 3 N&A extracted from the 1st 80 bytes of the customer master and sales history records. See the complete file listed on page '1B2'.
130140 EVERGREEN MOTORS LTD. 1815 BOWEN ROAD NANAIMO BC 132588 GEECOE GENERATOR SERVICESUNIT 170 - 2851 SIMPSON RICHMOND BC 139923 JOHNSTONE BOILER & TANKS 1250 EAST PENDER STREET VANCOUVER BC
uvcp "fili1=dat1/custmas1,rcs=256,typ=RSF,filo1=tmp/cm1,rcs=80,typ=RST" =======================================================================
vi tmp/cm1 <-- examine extracted file with 'vi' (no problem) ==========
| note |
|
uvhd tmp/cm1 t <-- option 't' for Text file ==============
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Suppose we wished to create a telephone list from the customer master file. Here are 1st 3 records in the desired format.
130140 EVERGREEN MOTORS LTD. 250-754-5531 LARRY WRENCH 132588 GEECOE GENERATOR SERVICES 604-278-4488 HARRY LIGHT 139923 JOHNSTONE BOILER & TANKS 604-320-1845 GEORGE BROWN
uvcp "fili1=dat1/custmas1,rcs=256,typ=RSF,filo1=tmp/telelist,rcs=80,typ=RST\ ,clr=35(45),mvc=38(12)90,mvc=52(20)102" ============================================================================
Suppose we wanted our telephone list only for the province of 'BC'. We could modify the above command to the following:
uvcp "fili1=dat1/custmas1,rcs=256,typ=RSF,filo1=tmp/telelistBC,rcs=80,typ=RST\ ,clr=35(45),mvc=38(12)90,mvc=52(20)102,sel=77(2):BC" ==============================================================================
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
You can use uvcp to translate EBCDIC to ASCII & convert RDW/BDW variable-length files to any desired format (fixed-length, variable-length, text, Indexed). For example, we will convert EBCDIC RDW file vendormas3E to ASCII text records. 'vendormas3E' is the EBCDIC version of ASCII file 'vendormas3' listed on '1D1'. Here is the 1st record shown with uvhd (note the EBCDIC codes).
uvhd dat1/vendormas3E z4a <-- option 'z4' identifies RDW files
========================= - option 'a' to translate character line to ASCII
- zones & digits lines show the EBCDIC codes
records=8 rsize=23 fsize=249
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 ....DELL10 M Dell Inc.
0100CCDDFF4D44C8994C984
0700453310040045330953B
uvcp "fili1=dat1/vendormas3E,rcs=64,typ=RDWz4,filo1=tmp/vm3A,typ=LSTt,tra=0(64)" ================================================================================
more tmp/vm3A <-- display output file (all 8 records) =============
DELL10 M Dell Inc. HP0010 M Hewlett Packard IBM010 M International Business Machines MFC010 M Micro Focus COBOL MS0010 M Microsoft Corp. REDH10 M Red Hat Linux SUN010 M Sun Microsystems Ltd UVSI10 M UV Software Inc.
'tra=0(64)' translates to ASCII
'typ=LSTt' creates a Text file (inserts a Line-Feed after last non-blank). We can use uvhd to see the Line-Feeds as follows:
uvhd tmp/vm3A t <-- option 't' designates a Text file ===============
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 DELL10 M Dell Inc..
44443324224666246620
45CC100D0045CC09E3EA
| Note |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
We could also do the opposite of the example above - create RDW EBCDIC records from text records.
uvcp "fili1=tmp/vm3A,rcs=64,typ=LST,filo1=tmp/vm3E,typ=RDWz4,tre=0(64)"
=======================================================================
- convert ASCII text file to EBCDIC RDW file
uvhd tmp/vm3E z4a <-- display output file (only 1st record shown below)
=================
records=8 rsize=23 fsize=248
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 ....DELL10 M Dell Inc.
0100CCDDFF4D44C8994C984
0700453310040045330953B
| note |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
This is an academic exercise, but should help you understand BDW/RDW files. When uvhd reads BDW/RDW files, it bypasses any BDWs & shows only the RDWs. uvhd cannot normally write BDW files, but for this exercise we will create a file with 1 Block by prefixing our sample 8 record RDW file with the block size. We will use uvcp to create the 4 byte BDW as a separate file & then concatenate the 2 files.
#1. uvcp "fili1=tmp/vm3A,rcs=64,typ=LST,filo1=tmp/vm3E,typ=RDWz4,tre=0(64)"
=======================================================================
- create the RDW file (same as on prior page)
#2. ls -l tmp <-- get size of RDW file (8 records)
========= - 248 on my tests (+4 for BDW)
#3. echo xx >tmp/xx <-- create dummy file for input on uvcp below
===============
#4. uvcp "fili1=tmp/xx,rcs=4,typ=RSF,filo1=tmp/vm3B,mvc=0(4):0x00FC0000"
====================================================================
- create 4 byte BDW (to be prefixed to tmp/vm3E from above)
- x'00FC' = (15*16 + 12*1) = 252
#5. cat tmp/xx tmp/vm3E >>tmp/vm3EB
===============================
- concat 4 byte BDW + 248 byte RDW file = 252 byte BDW/RDW file
#6. uvhd /home/uvadm/tmp/vm3EB z8a <-- display output file
============================== using option 'z8' for BDW/RDW files
rec#=1 rsize=23 fptr=4 fsize=252 rcount=8
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
4 ....DELL10 M Dell Inc.
0100CCDDFF4D44C8994C984
0700453310040045330953B
| Note |
|
#6. uvhd tmp/vm3EB r64a <-- display 1st 64, note BDW x'00FC' = 252
===================
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 ........DELL10 M Dell Inc.....HP0010 M Hewlett Packard....IBM0
0F000100CCDDFF4D44C8994C9840100CDFFFF4D44C8A98AA4D8898980200CCDF
0C000700453310040045330953B0D0087001004008563533071321940D009240
| note |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
The UNIX system sort is inadequate for sorting mainframe type files because:
See the input file listed on page '1C3'.
uvsort "fili1=dat1/sales3,rcs=64,typ=RST,filo1=tmp/s3,key1=30(6)" =================================================================
cat tmp/s3 <-- display the output file ========== - note sorted by product#
223240 65 980817 IN441 BBQ001 00029+ 0009001 00261029+ 406082 35 020815 IN33001 BBQ001 00017- 0007001 00119017- 130140 21 940804 IN111001 CHR001 00022+ 0002001 00044022+ 130140 21 940802 IN111001 HAM001 00020+ 0000001 00000020+ 150825 44 960804 IN1122 HAM001 00025+ 0005001 00125025+ 223240 65 980816 CR955 HAM001 00028+ 0008001 00224028+ 315512 44 000805 IN2251 HAM001 00013- 0003001 00039013- 406082 65 020816 IN441 HAM001 00019- 0009001 00171019- 150825 44 960805 IN1122 HAX129 00026+ 0006001 00156026+ 400002 85 010812 CR245 HAX129 00016- 0006001 00096016- 308685 21 990812 IN6605 SAW051 00010- 0000001 00000010- 315512 44 000806 IN2251 SAW051 00014- 0004001 00056014- 130140 21 940803 IN111001 SCR012 00021+ 0001001 00021021+ 139923 35 950802 IN111002 TAB013 00023+ 0003001 00069023+ 139923 35 950807 IN111002 TAB013 00024+ 0004001 00096024+ 308685 21 990814 CR8835 TAB013 00012- 0002001 00024012- 406082 35 020816 IN33001 TAB013 00018- 0008001 00144018- 201120 44 970807 CR5234 WHIP75 00027+ 0007001 00189027+ 308685 21 990813 IN6605 WHIP75 00011- 0001001 00011011- 315512 44 000807 IN2255 WHIP75 00015- 0005001 00075015-
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| Demo#2 |
|
uvsort "fili1=dat1/sales3,rcs=64,typ=RST,filo1=tmp/s3,key1u1=30(6)" ===================================================================
cat tmp/s3 <-- display output ==========
223240 65 980817 IN441 BBQ001 00029+ 0009001 00261029+ 130140 21 940804 IN111001 CHR001 00022+ 0002001 00044022+ 130140 21 940802 IN111001 HAM001 00020+ 0000001 00000020+ 150825 44 960805 IN1122 HAX129 00026+ 0006001 00156026+ 308685 21 990812 IN6605 SAW051 00010- 0000001 00000010- 130140 21 940803 IN111001 SCR012 00021+ 0001001 00021021+ 139923 35 950802 IN111002 TAB013 00023+ 0003001 00069023+ 201120 44 970807 CR5234 WHIP75 00027+ 0007001 00189027+
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| Demo#3 |
|
uvsort "fili1=dat1/sales3,rcs=64,typ=RST,filo1=tmp/s3,key1=30(6),sum=53(9+)" ===========================================================================
cat tmp/s3 <-- display output, note:
========== - output only 1 record per key (product#)
- with $amount total summed in cols 54-62
223240 65 980817 IN441 BBQ001 00029+ 0009001 00142012+ 130140 21 940804 IN111001 CHR001 00022+ 0002001 00044022+ 130140 21 940802 IN111001 HAM001 00020+ 0000001 00139041+ 150825 44 960805 IN1122 HAX129 00026+ 0006001 00060010+ 308685 21 990812 IN6605 SAW051 00010- 0000001 00056024- 130140 21 940803 IN111001 SCR012 00021+ 0001001 00021021+ 139923 35 950802 IN111002 TAB013 00023+ 0003001 00002983- 201120 44 970807 CR5234 WHIP75 00027+ 0007001 00103001+
uvsort "fili1=dat1/sales3,rcs=64,typ=RST,filo1=tmp/s3,key1=30(6)\ ,sum1=38(6+),sum2=53(9+)" =================================================================
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
'uvlist' will list ascii text files (on stdout by default) but with the main intention of piping to a printer. Several scripts are provided (uvlp12,etc) which include the pipe to the printer, so you can list your text files with a minimum of keystrokes.
uvlist was originally developed to list the documentation files of the Vancouver Utility package. These files use the '*eject' pattern (anywhere in 1st 15 cols) to signal a new page. uvlist has proven to be of substantial assistance to customers as a general purpose text file list utility for programs, scripts, documentation, parameter files, etc.
uvlist ensures that all pages of listings have page headings that identify the file-name, date-printed, userid, line# of 1st line on page,& page#. uvlist saves you the frustration that can occur when you pick up a listing without page headings to identify where it came from & when it was created.
Many options are provided for laser printers, for example you can list 132 column reports on 8 1/2 width paper using the 'uvlp18' script which includes the option to print at 18 characters per inch.
| uvlp12 |
|
| uvlp12v6 |
|
| uvlp14 |
|
| uvlp12D |
|
| uvlp14L |
|
| uvlp18 |
|
| uvlp13LD |
|
| uvlpd12 |
|
uvlp12 doc/datautilities.doc <-- list this documentation ============================ - defaults to Simplex
uvlp12D doc/datautilities.doc <-- list this documentation Duplex ============================= - Duplex (double sided)
uvlp12D doc/datautilities.doc b51e56 <-- list with options Begin/End page#s ==================================== - begin at page 51, End at page 56
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#!/bin/ksh
# uvlp12 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
# uvlp12 - print a file at 12 cpi (90 chars on 8 1/2 x 11)
# - pg hdngs with: filename, mod-date, today-date, page#s
# - for HP laserjet 4 printers & compatibles
# - by Owen Townsend - UV Software Inc - Dec 1994
#
#usage: uvlp12 filename [group1] [group2] <-- may override default options
# =================================
#
# - 1 of several: uvlp12,uvlp14,uvlp16,uvlp12L,uvlp14L,uvlp12D,uvlp14LD,etc
# - these scripts invokes uvlist & pipe to the spooler
# - scriptnames reflect commonly used uvlist options
# group1: p60 = 60 lines per page
# group2: c12 = 12 cpi = 90 chars across on 8 1/2 " paper
# m280 = default margin offset by 280/720 inch (for 3 hole punches)
# - see uvlist.doc for the many other HP PCL5 compatible print options
#
# .profile should specify environmental variables for 'lp', for example:
# export UVLPDEST="-dlp0" #<-- destination 'lp0' ("-dLPT1" for SFU)
# export UVLPOPTN="-onobanner" #<-- 'nobanner' option for lp
#
if [ -f "$1" ]; then :
else echo "ERROR - $1 is not a file"; exit 1; fi
#
uvlist $1 p60$2 t1c12$3 | lp $UVLPOPTN $UVLPDEST
#===============================================
#note - option 't1' for tray 1 (main paper source on my Lexmark t642)
# - option 't4' for alternate tray (t2 & t3 are manual & envelope)
# - OR use option 'a1' for letter size (& remove tray code)
# and relate letter size to tray# via printer control panel
uvln=$(basename $0)
linesbf=$(wc -l $1); linesb=${linesbf% *}; lines=${linesb##* };
echo "$uvln printing $1 on $UVLPDEST, lines=$lines"
exit 0
The common_profile defines a default printer that may not be convenient for you. You can override the default by adding a line to your .profile (or .bash_profile) as follows:
export UVLPDEST="-dlaserjet2" <-- define printer near you =============================
| Note |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| 5A1. | problems FTP'ing mainframe data files to unix |
| - FTP ASCII mode - corrections required |
| 5A2. | which solution to CR/LF ? |
| - solutions for zoned signs |
| 5B1. | FTP data from Mainframe to Unix/Linux |
| 5B2. | FTP ASCII with demo files |
| 5C1. | investigate files FTP'd from mainframe |
| - sales0 - EBCDIC recsize=64 |
| 5C2. | sales1a - ASCII with CR/LF appended |
| 5D1. | correcting signs after FTP ASCII mode |
| - ftp/sales1a - BEFORE sign correction | |
| - ftp/sales1a - AFTER sign correction |
| 5E1. | correcting Record-Length after FTP ASCII mode |
| - ftp/sales1a - BEFORE record-length correction | |
| - data/sales1a - AFTER record-length correction |
| 5E2. | Best of Both Worlds |
| - data/sales1a - AFTER recsize correct & LF insert |
| 5F1. | uvcp file 'typ' codes |
| 5G1. | using uvhd TRANSLATE sign command (vs uvcp) to correct zoned signs |
| - uvhd is a FREE download from https://www.uvsoftware.ca/freestuff.htm |
| 5G2. | using uvhd WRITE command (vs uvcp) to change record-length drop CR/LF |
| "Best of Both Worlds" - retaining original rec-size, so no program changes | |
| - inserting LF in last byte (if unused) of original rec-size, | |
| allows us to use the various unix tools (vi,lp,cat,more,etc). |
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| 5H1. | Detecting corrupted files after FTP |
| - ftp/sales1x - corrupted file |
| 5H2. | Use 'uvhd' to detect corrupted file |
| 5H3. | Using uvhd with Text file option 't' |
| 5I1. | finding wrong-size records in large files |
| varstat3 - table summarize record sizes |
| 5J1. | correcting wrong-size records |
| - dropping/separating wrong-size records | |
| - console log from splitfile2 |
| 5K1. | splitfile2 - listing of uvcopy job to select desired rec-size |
| 5K2. | varstat3 - listing of uvcopy job to table summarize record-sizes |
| 5L1. | EBCDIC to ASCII Data Conversion allowing packed/binary |
| - these more complex data conversions not covered in detail here | |
| - references given to other documents with the operating instructions |
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Part 5 will illustrate problems that can occur when FTP'ing mainframe files to unix/linux. And we will offer solutions to these problems using Vancouver Utilities (uvhd & uvcp).
You can FTP files in either ASCII/text mode or in BINARY mode. If you are have files with packed decimal or binary fields, you must FTP them in BINARY mode - because the ASCII/text mode EBCDIC to ASCII translation destroys packed/binary fields. Ascii mode also appends a Carriage-Return & Line-Feed making each record 2 bytes longer than on the mainframe.
Migrating mainframe applications to unix/linux will be easier if you can avoid changing record sizes - because you will not have to change the prgrams. If you FTP'd the files in ASCII mode you would have to change the COBOL programs to match - increasing the record length by 2 bytes (for the added CR/LF) & changing file type from the default 'ORGANIZATION SEQUENTIAL' to 'organization LINE sequential' (for Micro Focus COBOL).
But it is not that simple if you have files with packed/binary fields. These files must be FTP'd in BINARY mode & then selectively translate only the character fields, preserving the packed/binary fields which must be the same for both mainframe & Micro Focus COBOL. Vancouver Utilities has tools which use the COBOL copybooks to automatically generate jobs to translate the pic x fields & preserve (copy as is) the comp-3/comp fields.
Please see 'Part_6' if you need to convert files with packed &/or binary fields.
This Part 5 will focus on FTP ASCII mode & compensating for the file changes caused by FTP ASCII. There are 2 changes that will require correction if you are inputting the files to the same application (COBOL?) as on the mainframe that expects the same record layout.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Leaving the CR/LF on the records & modifying the programs to expect LINE Sequential has advantages. You can then use the various unix tools (vi,lp,etc) to view & print the file. Without the LineFeeds you could use 'uvhd' to inspect the files.
The following pages will illustrate the FTP problems & the various solutions.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#1. Login --> your homedir
=====
#2. cdd --> $RUNDATA (RUNDATA=/home/cnvdata1 or cnvdata2, etc)
===
#3. cd data <-- change into the data subdir
=======
#4. ftp xxx.xxx.xxx.xxx <-- FTP to mainframe IP#
===================
#4a. userid--> ..... <-- enter userid
#4b. passwd--> ..... <-- enter password
#4c. ascii <-- translate EBCDIC to ASCII & insert CR/LF
---OR--- (ascii is usually the default)
#4c. binary <-- ensure binary transfer
#4d. site RDW <-- required if any variable length files
#4e. cd ... <-- change to data files directory ??
#4f. get 'XXX.XXX.XXX' <-- get desired data files
#4g. get '...etc....' - 'single quotes' may be required
#5. cdd <-- change back up to working directory (above data/...)
===
On some systems the single quotes are retained & the filename may be UPPER case or lower case depending how we entered the filenames. ON unix we certainly need to remove the single quotes & we may need to change the case. We will show you 2 ways to do this - the 'hard way' & the 'easy' way'.
#5. mv \'XXX.XXX.XXX\' xxx.xxx.xxx <-- the hard way
================================ - see #6. the easy way
- imagine how awkward this would be if hundreds of files
#6a. renameQQ data <-- easy way to remove Quotes from All files in subdir
=============
#6b. renameL data <-- easy way to convert to Lower case (All files in subdir)
============
#6b. renameU data <-- easy way to convert to UPPER case (All files in subdir)
============
| Note |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
We can illustrate FTP ASCII problems by FTP'ing the supplied EBCDIC demo file dat1/sales1 within our unix/linux machine - using IP# 127.0.0.1 (localhost).
/home/uvadm
:-----dat1 <-- supplied test data files
: :-----sales0 <-- EBCDIC fixed length 64 byte records (no LineFeeds)
: : - with zoned signs in unpacked numeric fields
: :-----sales1a <-- ASCII fixed length 66 byte records (CR/LF appended)
: : - as sales0 would be if FTP'd from mainframe in ASCII mode
: :-----sales1x <-- corrupted file (not all same-size records)
- to demo varstat3 detect & splitfile2 to separate
/home/mvstest1 <-- mvstest1 for user self-training /home/userxx <-- OR setup testdata in userhomedir :-----testdata : :-----ftp - receive files FTP'd from $UV/dat1/... : :-----data - copy with corrections to $HOME/testdata/data : : :-----sales0,sales1a,sales1x <-- files copied from $UV/dat1/...
#1. Login --> your homedir #2. cdd --> $RUNDATA (RUNDATA=/home/cnvdata1 or cnvdata2, etc)
#3. mkdir ftp <-- make subdir to receive FTP'd files
- see existing subdirs on page '3B2'
#4. cd ftp <-- change into subdir to receive FTP'd files
#5. ftp localhost <-- FTP within our unix/linux machine
---OR---
ftp 127.0.0.1 - IP# for localhost
=============
#5a. userid--> uvadm <-- to get demo files supplied with VU pkg
#5b. passwd--> ..... <-- enter password
#5c. cd dat1 <-- change to demo data files directory
#5d. ascii <-- translate EBCDIC to ASCII & insert CR/LF
binary - ascii does not work within a unix system
#5e. get sales0 <-- get EBCDIC demo file
#5f. get sales1a <-- get ASCII version with CR/LF
#5g. bye
#6. cdd <-- change back up to working directory (above data/...)
| Note |
|
| sales0 |
|
| sales1a |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
After FTP'ing use 'uvhd' to investigate the files & confirm as expected. First determine the record size (default 256 if not sepcified on uvhd command). If omitted uvhd suggests the next lower & next higher recsize multiple that divides evenly into the filesize. Or look for repeating patterns in the data. Or since we FTP'd in ascii mode look for the 1st Linefeed x'0A', quit uvhd,& re-enter the command with correct record-size.
uvhd ftp/sales0 r64a <-- option r64 for record-size
==================== - option 'a' to display char line to ASCII
so we can read (zones/digits show EBCDIC)
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 130140 21 940802 IN111001 HAM001 00002{ 0000001 00000002{
FFFFFF4444FF4FFFFFF4CDFFFFFF44CCDFFF44FFFFFC4FFFFFFF4FFFFFFFFC44
1301400000210940802095111001008140010000002000000001000000002000
rec#=1 rsize=64 fptr=0 fsize=1280 rcount=20
cust# slm# date invoice# product qty unit-price ext$amount
10 20 30 40 50 60
r# 2 0123456789012345678901234567890123456789012345678901234567890123
64 130140 21 940802 IN111001 SCR012 00002A 0001001 00002102A
FFFFFF4444FF4FFFFFF4CDFFFFFF44ECDFFF44FFFFFC4FFFFFFF4FFFFFFFFC44
1301400000210940802095111001002390120000002100001001000002102100
rec#=2 rsize=64 fptr=64 fsize=1280 rcount=20
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
uvhd ftp/sales1a r66 <-- option r66 for record-size ==================== - omit option 'a' since FTP translated
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 130140 21 940802 IN111001 HAM001 00002{ 0000001 00000002{
3333332222332333333244333333224443332233333723333333233333333722
130140000021094080209E1110010081D0010000002B00000001000000002B00
64 ..
00
DA
rec#=1 rsize=66 fptr=0 fsize=1320 rcount=20
10 20 30 40 50 60
r# 2 0123456789012345678901234567890123456789012345678901234567890123
66 130140 21 940802 IN111001 SCR012 00002A 0001001 00002102A
3333332222332333333244333333225453332233333423333333233333333422
130140000021094080209E111001003320120000002100001001000002102100
64 ..
00
DA
rec#=2 rsize=66 fptr=66 fsize=1320 rcount=20
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
uvcp "fili1=ftp/sales1a,rcs=66,typ=RST,filo1=data/sales1a,tras=43(1),tras=61(1)" ================================================================================
130140 21 940802 IN111001 HAM001 00002{ 0000001 00000002{
130140 21 940802 IN111001 SCR012 00002A 0001001 00002102A
139923 35 950802 IN111002 CHR001 00002B 0002001 00004402B
139923 35 950802 IN111002 TAB013 00002C 0003001 00006902C
139923 35 950807 CR5421 TAB013 00002D 0004001 00009602D
150825 44 960804 IN1122 HAM001 00002E 0005001 00012502E
150825 44 960804 IN1122 PLN012 00002F 0006001 00015602F
201120 44 970807 CR5234 WHIP75 00002G 0007001 00018902G
223240 65 980816 CR955 HAM001 00002H 0008001 00022402H
223240 65 980816 IN441 BIT001 00002I 0009001 00026102I
308685 21 990812 IN6605 SAW001 00001} 0000001 00000001}
308685 21 990812 IN6605 WID115 00001J 0001001 00001101J
308685 21 990812 CR8835 TAB013 00001K 0002001 00002401K
315512 44 000805 IN2251 HAM001 00001L 0003001 00003901L
315512 44 000805 IN2251 SAW051 00001M 0004001 00005601M
315512 44 000805 IN2255 WID115 00001N 0005001 00007501N
400002 85 010812 CR245 HAX129 00001O 0006001 00009601O
406082 35 020815 IN33001 BBQ001 00001P 0007001 00011901P
406089 35 020815 IN33001 TAB013 00001Q 0008001 00014401Q
406082 65 020816 IN441 HAM001 00001R 0009001 00017101R
130140 21 940802 IN111001 HAM001 000020 0000001 000000020 130140 21 940802 IN111001 SCR012 000021 0001001 000021021 139923 35 950802 IN111002 CHR001 000022 0002001 000044022 139923 35 950802 IN111002 TAB013 000023 0003001 000069023 139923 35 950807 CR5421 TAB013 000024 0004001 000096024 150825 44 960804 IN1122 HAM001 000025 0005001 000125025 150825 44 960804 IN1122 PLN012 000026 0006001 000156026 201120 44 970807 CR5234 WHIP75 000027 0007001 000189027 223240 65 980816 CR955 HAM001 000028 0008001 000224028 223240 65 980816 IN441 BIT001 000029 0009001 000261029 308685 21 990812 IN6605 SAW001 00001p 0000001 00000001p 308685 21 990812 IN6605 WID115 00001q 0001001 00001101q 308685 21 990812 CR8835 TAB013 00001r 0002001 00002401r 315512 44 000805 IN2251 HAM001 00001s 0003001 00003901s 315512 44 000805 IN2251 SAW051 00001t 0004001 00005601t 315512 44 000805 IN2255 WID115 00001u 0005001 00007501u 400002 85 010812 CR245 HAX129 00001v 0006001 00009601v 406082 35 020815 IN33001 BBQ001 00001w 0007001 00011901w 406089 35 020815 IN33001 TAB013 00001x 0008001 00014401x 406082 65 020816 IN441 HAM001 00001y 0009001 00017101y
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
If you decide to retain the same record-size (so you do not have to change the programs), you can drop the appended CR/LF using 'uvcp' as shown below using our demo file sales1a which expanded from 64 to 66 by ASCII FTP.
uvcp "fili1=ftp/sales1a,rcs=66,typ=RSF,filo1=data/sales1a,rcs=64,typ=RSF" =========================================================================
uvhd ftp/sales1a r66 <-- option r66 rec-size from ASCII FTP & BEFORE drop CR/LF ==================== - will show only 1st record of 20 in our demo file
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 130140 21 940802 IN111001 HAM001 00002{ 0000001 00000002{
3333332222332333333244333333224443332233333723333333233333333722
130140000021094080209E1110010081D0010000002B00000001000000002B00
64 ..
00
DA
rec#=1 rsize=66 fptr=0 fsize=1320 rcount=20
uvhd data/sales1a r64 <-- option r64 for rec-size AFTER dropping CR/LF =====================
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 130140 21 940802 IN111001 HAM001 00002{ 0000001 00000002{
3333332222332333333244333333224443332233333723333333233333333722
130140000021094080209E1110010081D0010000002B00000001000000002B00
rec#=1 rsize=64 fptr=0 fsize=1280 rcount=20
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Dropping the CR/LF has a disadvantage - you can not use the various unix tools (vi,lp,cat,more,etc) to view & print the records unless the LF is present (the CR is not required for unix/linux systems).
We can keep the LineFeed & the same Record-Length in this case - because there just happens to be 2 unused blanks at the end of our sales1a demo file. The following uvcp specifies output 'typ=RST' (Record Sequential Terminated) which forces a LineFeed into the last byte of 'rcs=64'.
uvcp "fili1=ftp/sales1a,rcs=66,typ=RSF,filo1=data/sales1a,rcs=64,typ=RST" =========================================================================
uvhd data/sales1a r64 <-- option r64 for rec-size after dropping CR/LF =====================
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 130140 21 940802 IN111001 HAM001 00002{ 0000001 00000002{
3333332222332333333244333333224443332233333723333333233333333720
130140000021094080209E1110010081D0010000002B00000001000000002B0A
rec#=1 rsize=64 fptr=0 fsize=1280 rcount=20
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
See all file type codes at uvcp.htm#F1. Here are the more relevant codes:
| RSF |
|
| RST |
|
| LST |
|
| LSTt |
|
| ISF |
|
| RDWz4 |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Page '5D1' used 'uvcp' to correct zoned signs from mainframe EBCDIC conventions to unix Micro Focus COBOL conventions (positives {ABCDEFGHI to 0123456789 & negatives }JKLMNOPQR to pqrstuvy).
uvcp "fili1=ftp/sales1a,rcs=66,typ=RST,filo1=data/sales1a,tras=43(1),tras=61(1)" ================================================================================ - 'uvcp' solution from page '5D1', see 'uvhd' Alternative solution below
If you have not purchased the Vancouver Utilities package, you can do the sign corrections using the 'FREE uvhd' from https://www.uvsoftware.ca/freestuff.htm. Need to enter a separate command for each signed field vs uvcp 1 pass.
#1. Login mvstest1 or yourself --> your homedir
#2. cdd --> $HOME/testdata (with subdirs ftp,data at least)
#3. cp ftp/sales1a data/
====================
- copy FTP'd file to the data/ subdir, since uvhd updates in place
- vs uvcp which copies the file (as seen above)
#4. uvhd data/sales1a r66u <-- option 'r66' for Record-size 66
====================== - option 'u' required for Update
#4a. --> t99s 43(1) <-- translate mainframe signs to unix Micro Focus COBOL
========== - 't99' Translates ALL records (99 assumes 999999999)
- option 's' for Sign correction translate table
- '43(1)' identifies the start byte & (length)
- the units position of the quantity field
#4b. --> 1 <-- return to record #1 (in preparation for 2nd field)
===
#4c. --> t99s 61(1) <-- translate the zoned sign in the units of $amount
==========
#5. head data/sales1a <-- inspect results
================= - see all 20 records Before & After on page '5D1'
- but here are 1st 2 & last record Before & After
1 2 3 4 5 6
012345678901234567890123456789012345678901234567890123456789012345
CUST# SLM DATE INV# PROD# QTY PRICE AMOUNT
130140 21 940802 IN111001 HAM001 00002{ 0000001 00000002{
130140 21 940802 IN111001 SCR012 00002A 0001001 00002102A
406082 65 020816 IN441 HAM001 00001R 0009001 00017101R
- - - 1st 2 & last records BEFORE above, AFTER below - - -
130140 21 940802 IN111001 HAM001 000020 0000001 000000020
130140 21 940802 IN111001 SCR012 000021 0001001 000021021
406082 65 020816 IN441 HAM001 00001y 0009001 00017101y
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Page '5E1' used 'uvcp' to change the record-length - dropping the CR/LF so we would not have to change the programs when migrating from mainframe to unix. You can also change record lengths using the 'FREE uvhd'.
We will use the 'w' (Write) command of uvhd with option 'r' for the desired Record-size. Note that the Write command writes the file out to the tmp/... subdir with a date_time stamp (tmp/sales1a_yymmdd_hhmmssW). You can inspect & if OK - copy back changing the name back to the original name.
#1. Login mvstest1 or yourself --> your homedir
#2. cdd --> $HOME/testdata (with subdirs ftp,data at least)
#3. uvhd data/sales1a r66 <-- option 'r66' for Record-size 66
=====================
#3a. --> w99r64 <-- Write all records to tmp/sales1a_yymmdd_hhmmssW
====== - 'r64' specifies the new Record-size (dropping CR/LF)
#4. uvhd tmp/sales1a_yymmdd_hhmmssW r64 <-- inpsect output file
===================================
#5. cp tmp/sales1a_yymmdd_hhmmssW data/sales1a
==========================================
- copy back renaming back to the original name
Here is the "Best of Both Worlds" solution - retaining original record-size, so we don't have to chaneg programs AND inserting a LineFeed in the last byte (if unused) of original rec-size, so we can use unix tools (vi,lp,cat,more,etc).
#3. uvhd data/sales1a r66 <-- option 'r66' for Record-size 66
=====================
#3b. --> w99r63t2 <-- Write all records to tmp/sales1a_yymmdd_hhmmssW
======== - option 'r63' specifies new Record-size (dropping CR/LF)
- option 't2' appends a LineFeed on rec-size (63+1=64)
#4. uvhd tmp/sales1a_yymmdd_hhmmssW r64 <-- inpsect output file
===================================
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 130140 21 940802 IN111001 HAM001 00002{ 0000001 00000002{
3333332222332333333244333333224443332233333723333333233333333720
130140000021094080209E1110010081D0010000002B00000001000000002B0A
rec#=1 rsize=64 fptr=0 fsize=1280 rcount=20
| Note |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
As mentioned earlier, the 1st thing you should do on receiving files via FTP is to investigate with uvhd to confirm as expected - file type & record size. If fixed-length, all records must be the same size & uvhd can check this.
To illustrate, we have provided 'sales1x' which is similar to the previously listed sales1a, but has 5 short records. Here it is:
130140 21 940802 IN111001 HAM001 00002{ 0000001 00000002{
130140 21 940802 IN111001 SCR012 00002A 0001001 00002102A
139923 35 950802 IN111002 CHR001 00002B 0002001 00004402B
139923 35 950802 IN111002 TAB013 00002C 0003001 00006902C
139923 35 950807 CR5421 TAB013 00002D 0004001 00009602D
150825 44 960804 IN1122 HAM001 00002E 0005001 00012502E
150825 44 960804 IN1122 PLN012 00002F 0006001 00015602F
201120 44 970807 CR5234 WHIP75 00002G 0007001 00018902G
223240 65 980816 CR955 HAM001 00002H 0008001
223240 65 980816 IN441 BIT001 00002I 0009001 00026102I
308685 21 990812 IN6605 SAW001 00001} 0000001 00000001}
308685 21 990812 IN6605 WID115 00001J 0001001 00001101J
308685 21 990812 CR8835 TAB013 00001K
315512 44 000805 IN2251 HAM001 00001L 0003001 00003901L
315512 44 000805 IN2251 SAW051 00001M 0004001
315512 44 000805 IN2255 WID115 00001N 0005001 00007501N
400002 85 010812 CR245 HAX129 00001O
406082 35 020815 IN33001 BBQ001 00001P 0007001 00011901P
406089 35 020815 IN33001 TAB013 00001Q
406082 65 020816 IN441 HAM001 00001R 0009001 00017101R
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Here are the results of using uvhd on corrupted file ftp/sales1x, which has 5 records shorter than the expected 66 (see listing on previous page).
uvhd ftp/sales1x r66 ====================
records=18 rsize=66 fsize=1236 fsize/rsize(rmndr)=48
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 130140 21 940802 IN111001 HAM001 00002{ 0000001 00000002{
3333332222332333333244333333224443332233333723333333233333333722
130140000021094080209E1110010081D0010000002B00000001000000002B00
64 ..
00
DA
filesize NOT multiple of recsize, 48 bytes remain
rec#=1 rsize=66 fptr=0 fsize=1236 rcount=18
If Text file (with LineFeeds), quit/restart with option 't'
If Variable (IDXFORMAT3/8,RDW), quit/restart with optn v,or z
If Fixed length records, change recsize now via command 'R' (R66 current)
next evenly divisible record sizes lower/higher are --> R12/R103
OR you may continue (ignoring these warnings) -->
10 20 30 40 50 60
r# 2 0123456789012345678901234567890123456789012345678901234567890123
66 130140 21 940802 IN111001 SCR012 00002A 0001001 00002102A
3333332222332333333244333333225453332233333423333333233333333422
130140000021094080209E111001003320120000002100001001000002102100
64 ..
00
DA
filesize NOT multiple of recsize, 48 bytes remain
rec#=2 rsize=66 fptr=66 fsize=1236 rcount=18
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
If you are getting "filesize not multiple of recsize" & you can see LineFeeds x'0A' in the data, then quit & change your fixed size option 'r' to option 't' for Text files (record length detrmined by LineFeed at end of each record).
uvhd ftp/sales1x t <-- prior page options 'r66' changed to 't' ================== - option 't' determines recor-size by LineFeeds x'0A'
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 130140 21 940802 IN111001 HAM001 00002{ 0000001 00000002{
3333332222332333333244333333224443332233333723333333233333333722
130140000021094080209E1110010081D0010000002B00000001000000002B00
64 ..
00
DA
rec#=1 rsize=66 fptr=0 fsize=1236 rcount=20
10 20 30 40 50 60
r# 2 0123456789012345678901234567890123456789012345678901234567890123
66 130140 21 940802 IN111001 SCR012 00002A 0001001 00002102A
3333332222332333333244333333225453332233333423333333233333333422
130140000021094080209E111001003320120000002100001001000002102100
64 ..
00
DA
rec#=2 rsize=66 fptr=66 fsize=1236 rcount=20
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
You might run statistics job 'varstat3' on corrupted files to see how many different record-sizes you have & where the 1st record is of each different size.
#1. Login mvstest1 or yourself --> your homedir
#2. cdd --> $HOME/testdata (with subdirs ftp,data at least)
#3. mkdir stats <-- make subdir to receive stats report
#4. uvcopy varstat3,fili1=ftp/sales1x,filo1=stats/sales1x
=====================================================
- create record-sizes report in stats/sales1x
#4a. uvcopy varstat3,fili1=ftp/sales1x <-- Alternative shorter command
================================= - output filename defaults as above
#5. vi stats/sales1x <-- inspect report
================
varstat3 2011/11/09_17:01:11 record-sizes in dat1/sales1x
tbl#0001 tblt1f1 c0(5)
line# count % 1strec# record-sizes
1 3 15 13 00045
2 2 10 9 00053
3 15 75 1 00065
20*100 *TOTAL*
vi $UV/pf/IBM/varstat3
======================
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
You might want to correct the wrong size records, if the application programs must have all same sized records, vs changing program files to 'ORGANIZATION LINE SEQUENTIAL", which would read the file correctly.
We can use 'uvcp' with 'rcs=66', input 'typ=LST' & output 'typ=RST' to force all output recors to 66 bytes with the LF in the last byte.
uvcp "fili1=ftp/sales1x,rcs=66,typ=LST,filo1=data/sales1x,typ=RST" ==================================================================
If you wanted to drop the wrong-size records, you could run the 'splitfile2' uvcopy job, which reads the file & outputs 2 files - 1st with specified record-size, 2nd with all other-sized records (in case you want to inspect).
uvcopy splitfile2,fili1=ftp/sales1x,uop=r65 ===========================================
uop=q1r220 - option defaults
r220 - select data recsize 220 (+1 LF total 221)
r2048 - specify any desired record-size, max 2048
111109:175930:splitfile2: uvcopy ver=20111103 pf=/home/uvadm/pf/demo/splitfile2
uvcopy LNX L64 license=111103_99V_930630 site=UV_Software
User OPtion (uop) defaults = q1r220r65
-->null to accept or enter/override -->
111109:175931:splitfile2: EOF fili01 rds=20 size=1236: ftp/sales1x
111109:175931:splitfile2: EOF filo01 wrts=15 size=990: data/sales1x
111109:175931:splitfile2: EOF filo02 wrts=5 size=241: tmp/sales1xx
export PFPATH=$PFPATH,$UV/pf/adm,$UV/pf/demo,$UV/pf/util,$UV/pf/IBM ===================================================================
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
# splitfile2 - uvcopy job stored at $UV/pf/demo/splitfile2 (in $PFPATH)
# splitfile2 - split file of text records that should all be 1 record-size
# - write 2 output files, 1st file all specified recsize
# 2nd file for all other recsizes
# - a more flexible general purpose job than 'splitfile1'
# which hard-coded 1 particular record size
#
# ** sample Op. Instrns. for splitfile2 **
#
# uvcopy splitfile2,fili1=d1/filex,filo1=d2/filexok,filo2=d2/filexbad,uop=r220
# ============================================================================
# - note 'uop=r220' option to select data recsize 220 (+1 for LF total 221)
# - may specify I/O filenames on comamnd line (as above)
#
# uvcopy splitfile2
# =================
# - may omit fili1=...,filo1=...,filo2=... filenames & uop=...
# - uvcopy will show defaults coded & prompt to enter newnames/new rec-size
#
opr='uop=q1r220 - option defaults'
opr=' r220 - select data recsize 220 (+1 LF total 221)'
opr=' r2048 - specify any desired record-size, max 2048'
uop=q1r220 # option defaults (actual vs above opr msgs)
# # note option 'r' coded as $uopbr on cmn instruction below
fili1=?dat1/filex,typ=LST,rcs=2048 # recsize by LF max 2048
filo1=?dat2/filexok,typ=RST,rcs=2048 # fixed recsize option r, LF last byte
filo2=?dat2/filexbad,typ=LSTt,rcs=2048 #option t of LSTt drops trailing blanks
@run
opn all
# begin loop to get/put records until EOF
# note - get stores recsize in register 'v' ($rv)
loop get fili1,a0 fili1 typ=LST rcs=2048 allows recsize 2048 max
skp> eof
cmn $rv,$uopbr recsize = option 'r' ?
skp! put2
add $rv,1 +1 for LineFeed
put1 put filo1,a0($rv2048) filo1 typ=RST inserts LF last byte option size
skp loop
put2 put filo2,a0 filo2 typ=LST last nonblank determines recsize
skp loop
#
# EOF - close files & end job
eof cls all
eoj
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
# varstat3 - create summary table of record-sizes used in text file
# - by Owen Townsend, Oct 19/2011
#
# uvcopy varstat3,fili1=data/filename,filo2=stats/filename
# ==========================================================
#
# ** sample report **
#
# RDWstat1 2006/12/17_18:15:17 record-sizes in d0ebc/E2121656
# tbl#001 pg#001 -argument-
# line# count % record-sizes
# 1 10,552 16 00046
# 2 4,451 7 00065
# 3 23,347 37 00066
# 4 367 0 00068
# 5 21,010 33 00083
# - - - etc - - -
# 18 3 0 00218
# 19 441 0 00233
# 20 813 1 00239
# 62,115*100 *TOTAL*
#
opr='$jobname - create summary table of record-sizes used in text files'
rop=r1 # option to prompt for report disposition
fili1=?data/input,rcs=2048,typ=LST
filo1=?stats/$fili1,typ=LSTt,rcs=128
@run
opn all
#
# begin loop to get records & build summary table of recsizes
# - to be dumped at end of file
man20 get fili1,a0 get next RDW record data
skp> eof (cc set > at EOF)
mvn c0(5),$rv cnvrt binary recsize to numeric
tblt1f1 c0(5),'record-sizes' build table in memory
skp man20 repeat get loop til EOF
#
# EOF - close files & end job
eof mvfv1 f0(80),'record-sizes in $fili1'
tbpt1s1 filo1,f0(50) dump table from memory to file
cls all
eoj
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
See DATAcnv1.htm#Part_3 for the easiest way to convert data files with packed/binary fields. Script 'gencnvD2' generates conversion jobs, automatically from the COBOL copybooks. The following uvcopy jobs & SQL scripts are generated:
Script gencnvD2 generates all 4 types of conversion jobs & scripts. You might initially need only the 1st (EBCDIC to ASCII retaining same layout) if you wish to migrate quickly to unix/linux with the fewest changes required. Later on you might use the jobs/scripts 2,3,4 to convert flat files to database tables.
The data file conversions documented in DATAcnv1.htm#Part_3 are OK for most mainframe files, but do not allow for more complex files used at some mainframe sites. UV Software does provide solutions for those complex files, which are documented in other manuals.
Below is a comparison of datafile conversion capabilities covered in various UV Software documents:
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
DATAcnv1.htm & JCLcnv2real.htm#Part_4 cover the more complex data file conversions (such as multi Record Types & variable length records).
You can retain original data file names (without manual renaming) by creating a control file to relate the copybook-name to the various data-file names with that layout. The conversion jobs are also named from the datafile names (vs copybook names, because there could be multiple datafiles per copybook).
All data files may be converted with 1 command, which is important for the several reconversions requried during testing & the final 'go live' day.
See SQLconvert.htm for more complete documentation on converting data files to pipe|delimited all text files suitable for loading database tables. Packed/binary fields will be unpacked & edited with decimal points & separate signs suitable for loading database tables. We also generate scripts to create & load database tables. Utility 'sqlcreate1' generates scripts for Oracle. Utility 'sqlcreate2' generates scripts for Microsoft SQL Server.
Or you might want to unpack all packed/binary fields & retain your COBOL batch programs. See DATAcnv1.htm#4I1 thru 4I4 which documents this procedure. You make up a 2nd copybook for each file removing all 'comp-3's & 'comp's. Then run script 'genrfmA' to generate the data conversion jobs automatically. This conversion could be easy if you have used copybooks for all your data files, but obviously more difficult if you have embedded record layouts in your COBOL programs.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| 6A1. | table2 pre-programmed uvcopy job for table summaries |
| - table2 demo#1, summarize sales qty & amount by product |
| 6A2. | table2 demo - command, prompts, & responses |
| 6B1. | table2 demo#2 - customer counts by province |
| 6C1. | custom written uvcopy table summaries |
| cmtbl2 - sample uvcopy job to demo table summaries | |
| - table summarize this-year & last-year sales by city & province |
| 6C2. | cmtbl2 - listing of uvcopy instructions |
| 6D1. | Concatenating fields for table arguments |
| cmtbl2a - alternate uvcopy job to demo tabling by province+cityname |
| 6D2. | cmtbl2a - listing of uvcopy instructions |
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
'table2' is a pre-programmed uvcopy job to count records & accumulate 1 or 2 numeric fields by any desired argument field. We will demonstrate using the 'sales3' file listed on page '1C3'.
table2 2011/05/22_18:19:39 summary of qty & amount by product#
tbl#0001 tblo1t1 n0($rn32) -acum#1- % -acum#2- %
line# count % 1strec# product# qty amount
1 2 10 10 BBQ001 12 12 1,420.12 31
2 1 5 3 CHR001 22 22 440.22 9
3 5 25 1 HAM001 41 41 1,390.41 30
4 2 10 7 HAX129 10 10 600.10 13
5 2 10 11 SAW051 24- 24- 560.24- 12-
6 1 5 2 SCR012 21 21 210.21 4
7 4 20 4 TAB013 17 17 29.83- 0
8 3 15 8 WHIP75 1 1 1,030.01 22
20*100 *TOTAL* 100 *100 4,501.00 *100
Here is the command to call the table2 'uvcopy job' directly, BUT DO NOT use this, RATHER use the table2 'script' which is shorter & easier.
uvcopy table2,fili1=dat1/sales3,filo1=tmp/sales3_tbl,uop=a30b6c38d6e53f9g3r64 ============================================================================= table2 dat1/sales3 RSF a30b6c38d6e53f9g3r64r64 <-- use the table2 'script' ============================================== (shorter than 'uvcopy job')
uop=a0b0c0d0e0f0g0n0p0r0 - option defaults uop=a0 - argument displacement (0 relative) uop= b0 - argument length uop= c0 - acum#1 dsplcmnt (omit c,d,e,f for counts only) uop= d0 - acum#1 length uop= e0 - acum#2 displacement (0 relative) uop= f0 - acum#2 length uop= g1-12 - table formats g1-g12 see uvcopy.doc tbf f1-f12 uop= g3 - edit 1st acum qty, acums 2-6 dollars¢s uop= p0 - acum-fields p0=unpacked, p1=packed, p2=binary uop= r8192 - input record size (max 8192) uop=a30b6c38d6e53f9g3r64 <-- ENTER this for demo with dat1/sales3 page '1C3'
| a30b6 |
|
| c38d6 |
|
| e53f9 |
|
| g3 |
|
| r64 |
|
1 2 3 4 5 6
1234567890123456789012345678901234567890123456789012345678901234
CUST# SLM DATE INV# PROD# QTY PRICE AMOUNT
130140 21 940802 IN111001 HAM001 00020+ 0000001 00000020+
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
The table2 demo on the previous page omitted some of the prompts & responses. Here is a rerun, empahsizing the various prompts & responses. Note that the resonses are ==== underlined.
table2 dat1/sales3 RSF a30b6c38d6e53f9g3r64r64 <-- table2 script ==============================================
- file types: RSF(Fixed,default), LST(Text), RDWz4,RDWz2,RDWz8, ISF,IDXf8 - output report (filo1) will be: tmp/sales3_tbl_product uop=a0b0c0d0e0f0g0n0p0r0 - option defaults uop=a0 - argument displacement (0 relative) uop= b0 - argument length uop= c0 - acum#1 dsplcmnt (omit c,d,e,f for counts only) uop= d0 - acum#1 length uop= e0 - acum#2 displacement (0 relative) uop= f0 - acum#2 length uop= g1 - edit all acums as qty (commas, no decimal) uop= g2 - edit all acums as $ (commas & decimal point) uop= g__ - table formats g1-g12 see uvcopy.doc tbf f1-f12 uop= n0 - argument n0=unpacked, n1=packed, n2=binary uop= p0 - acum-fields p0=unpacked, p1=packed, p2=binary uop= r8192 - input record size (max 8192) uop= note - if file type LST specify r > largest possible uop=a30b6c38d6e53f9g3r64 <-- ENTER this for demo with dat1/sales3 file product#;quantity;amount <-- ENTER this for demo Field Headings Summary of Qty & Amt by Product# <-- ENTER this for demo Report Heading User OPtion (uop) defaults = q1a0b0c0d0e0f0g0n0p0r0a30b6c38d6e53f9g3r64 -->null accept or override --> <-- null response (entered on command line)
enter table header fieldnames separated by semicolons
argument;acum1;acum2 <-- enter fieldnames
--> product;qty;amount <-- entry for sales3 demo
================================================
enter table heading (optional, prints above fieldnames)
--> Qty & Amount by Product# <-- entry for sales3 demo
========================
110523:101907:table2: EOF fili01 rds=20 size=1280: dat1/sales3
110523:101907:table2: EOF filo01 wrts=8 size=1040: tmp/sales3_tbl_product
EOJ, Output File written to: tmp/sales3_tbl_product
default command = more, OR enter: more,print,uvlpr12,edit,null -->
--> more <-- entry for sales3 demo (display report on screen)
===========================================================
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Here is another table2 demo using the customer master file, simply counting records (customers) by province. See customer master file listed on page '1B2' & note province is in bytes 77-78 (zero relative) and the record size is 256.
table2 dat1/custmas1 RSF a77b2r256 ==================================
uop=a0b0c0d0e0f0g0n0p0r0 - option defaults uop=a0 - argument displacement (0 relative) uop= b0 - argument length uop= c0 - acum#1 dsplcmnt (omit c,d,e,f for counts only) uop= d0 - acum#1 length uop= e0 - acum#2 displacement (0 relative) uop= f0 - acum#2 length uop= g1 - edit all acums as qty (commas, no decimal) uop= g2 - edit all acums as $ (commas & decimal point) uop= g__ - table formats g1-g12 see uvcopy.doc tbf f1-f12 uop= n0 - argument n0=unpacked, n1=packed, n2=binary uop= p0 - acum-fields p0=unpacked, p1=packed, p2=binary uop= r8192 - input record size (max 8192) uop= note - if file type LST specify r > largest possible User OPtion (uop) defaults = q1a0b0c0d0e0f0g0n0p0r0a77b2r256 -->null accept or override --> <-- null response (entered on command line)
enter table header fieldnames separated by semicolons
argument;acum1;acum2 <-- enter fieldnames
--> province <-- entry for custmas1 demo
================================================
enter table heading (optional, prints above fieldnames)
--> customer counts by province <-- entry for custmas1 demo
===========================
EOJ, Output File written to: tmp/custmas1_tbl_province
default command = more, OR enter: more,print,uvlpr12,edit,null -->
--> more <-- entry for custmas1 demo (display report on screen)
=============================================================
table2 2011/05/23_11:00:58 customer counts by province
tbl#0001 tblo1t1 n0($rn32) -acum#1- % -acum#2- %
line# count % 1strec# province
1 3 9 15 AB
2 10 31 13 AL
3 16 49 1 BC
4 3 9 20 YK
32*100 *TOTAL* * *
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
The previous pages used 'table2' a pre-programmed uvcopy job to create 1 table summary report with 1 or 2 accumulators by any 1 argument field. Writing your own uvcopy job would allow you to use up to 6 accumulators in each table and have up to 255 tables in 1 job (1 pass thru the file).
'cmtbl2' is an example of a customer written uvcopy job that reads the customer master file (listed on page '1B2') and builds 2 tables to summarize this-year & last-year sales by city & province. Note that this-year & last-year sales must first be cross-footed (instruction 'xft') from the 12 monthly sales fields before being tabled (instruction 'tbl').
Here is the 'uvcopy' command to call the 'cmtbl2' instructions. uvcopy finds its jobs via PFPATH which includes /home/uvadm/pf/demo/... where cmtbl2 is stored. See the 'cmtbl2' job listed after the sample report below.
uvcopy cmtbl2 <-- execute 'uvcopy' to interpret instructions in 'cmtbl2' =============
cmtbl2 2011/05/23_12:44:07 sales by city name (this yr & last yr)
tbl#0001 tblt1f2 a60(16) -acum#1- % -acum#2- %
line# count % 1strec# city name thisyr sales lastyr sales
1 3 9 23 CALGARY 894.56 0 847.27 1
2 2 6 6 CHILLIWACK 4,530.46 4 9,218.36 20
3 2 6 9 COQUITLAM 1,844.48 1
4 1 3 31 DAWSON CITY 66.01 0 4,432.47 9
5 3 9 17 EDMONTON 20,108.35 20 27,919.97 61
6 2 6 16 FORT MCMURRAY 1,000.00- 1- 849.88 1
7 2 6 13 GRANDE PRAIRIE 1,546.42 1 2,192.64 4
8 2 6 4 HOPE 10,837.28 10 173.25
9 1 3 5 KELOWNA .00 0
10 1 3 1 NANAIMO 21,335.56 21 12,152.55- 26-
11 1 3 11 NORTH VANCOUVER .00 0
12 2 6 28 PRINCE GEORGE 1,781.75 1 4,395.21 9
13 1 3 2 RICHMOND 434.98 0 640.04 1
14 3 9 15 THREE HILLS 6,048.13 6 3,276.65 7
15 4 12 3 VANCOUVER 24,112.72 24 1,220.16 2
16 2 6 20 WHITEHORSE 7,157.23 7 2,427.49 5
32*100 *TOTAL* 99,697.93 *100 45,440.84 *100
cmtbl2 2011/05/23_12:44:07 sales by province (this yr & last yr)
tbl#0002 tblt2f2 a77(2) -acum#1- % -acum#2- %
line# count % 1strec# province thisyr sales lastyr sales
1 3 9 15 AB 323.13 0 1,534.06 3
2 10 31 13 AL 27,274.33 27 33,552.35 73
3 16 50 1 BC 64,877.23 65 3,494.47 7
4 3 9 20 YK 7,223.24 7 6,859.96 15
32*100 *TOTAL* 99,697.93 *100 45,440.84 *100
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
# cmtbl2 - table analysis of customer master sales history file
#
# uvcopy cmtbl2 <-- execute uvcopy to interpret this cmtbl2 parameter file
# ============= - files default as coded below on fili1=... & filo1=...
#
opr='$jobname - summarize sales (thisyr&lastyr) by city name & province'
rop=r1 # prompt oprtr for report disposition at EOJ
fili1=?dat1/custmas1,rcs=256,typ=RSF
filo1=?tmp/$fili1.tbl,rcs=80,typ=LSTt
@run
opn all
# begin loop to read all customer master sales history records
# - crossfooting & accumulating (tabling) thisyr & lastyr totals
loop get fili1,a0(256)
skp> eof
xft $ca1,a120(5p),12
xft $ca2,a180(5p),12
tblt1f2 a60(16),'city name;thisyr sales;lastyr sales',$ca1,$ca2
tblt2f2 a77(2),'province;thisyr sales;lastyr sales',$ca1,$ca2
skp loop
# end of file - print/edit the tables to a file for: lp, cat,or vi
eof tbpt1s2 filo1,'sales by city name (this yr & last yr)'
tbpt2s1 filo1,'sales by province (this yr & last yr)'
cls all
eoj
See all uvcopy instructions documented at uvcopy3.htm. uvcopy instructions are all 3 characters, anything after that are options. Here are a few notes on the more vital instructions in the cmtbl2 job.
| get |
|
| xft |
|
| xftt1 |
|
| xftt1f2 |
|
| tbp |
|
| tbpt1s2 |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
You might want to concatenate record fields for table arguments. For example you might want to concatenate province + city-name for 1 table argument, so the table would print in sequence by city within province.
uvcopy cmtbl2a <-- alternate version for table arg concat province+cityname ==============
cmtbl2a 2011/05/28_15:24:39 sales by province+cityname (this yr & last yr)
tbl#0001 tblt1f5 c0(20)
line# count % 1strec# province+cityname thisyr sales %
1 1 3 16 AB FORT MCMURRAY .00 0
2 2 6 15 AB THREE HILLS 323.13 0
3 3 9 23 AL CALGARY 894.56 0
4 3 9 17 AL EDMONTON 20,108.35 20
5 1 3 21 AL FORT MCMURRAY 1,000.00- 1-
6 2 6 13 AL GRANDE PRAIRIE 1,546.42 1
7 1 3 19 AL THREE HILLS 5,725.00 5
8 2 6 6 BC CHILLIWACK 4,530.46 4
9 2 6 9 BC COQUITLAM 1,844.48 1
10 2 6 4 BC HOPE 10,837.28 10
11 1 3 5 BC KELOWNA .00 0
12 1 3 1 BC NANAIMO 21,335.56 21
13 1 3 11 BC NORTH VANCOUVER .00 0
14 2 6 28 BC PRINCE GEORGE 1,781.75 1
15 1 3 2 BC RICHMOND 434.98 0
16 4 12 3 BC VANCOUVER 24,112.72 24
17 1 3 31 YK DAWSON CITY 66.01 0
18 2 6 20 YK WHITEHORSE 7,157.23 7
32*100 *TOTAL* 99,697.93 *100
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
# cmtbl2a - table analysis of customer master sales history file
# - Alt version to demo concatenating fields for table argument
# - use table format 'f5' allow 32 byte arg vs 16 for f2 in cmtbl2
# uvcopy cmtbl2a <-- execute uvcopy to interpret this cmtbl2a parameter file
# ==============
opr='$jobname - summarize sales (thisyr&lastyr) by province + city-name'
rop=r1 # prompt oprtr for report disposition at EOJ
fili1=?dat1/custmas1,rcs=256,typ=RSF
filo1=?tmp/$fili1.tbl,rcs=80,typ=LSTt
@run
opn all
# begin loop to read all customer master sales history records
# - crossfooting & accumulating (tabling) thisyr & lastyr totals
loop get fili1,a0(256)
skp> eof
xft $ca1,a120(5p),12
mvc c0(2),a77 store province to concat city-name
mvc c3(16),a60 concat city-name for argument province+cityname
tblt1f5 c0(20),'province+cityname;thisyr sales;lastyr sales',$ca1
skp loop
# end of file - print/edit the tables to a file for: lp, cat,or vi
eof tbpt1s2 filo1,'sales by province+cityname (this yr & last yr)'
cls all
eoj
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| 7A1. | using 'uvhd' or 'uvhdcob' to accumulate any 1 field (interactively) |
| - may qualify records to be accumulated (via record type, etc) | |
| - see examples using sales detail demo file |
| 7B1. | using 'pre-programmed uvcopy job' to accumulate any 1 field |
| - allows for various file types (fixed,variable,RDW/BDW,IDXFORMAT,text) | |
| - options for field type (numeric, packed, binary) |
| 7C1. | writing 'custom uvcopy job' to accumulate 1 or more fields |
| - complete flexibility to whatever desired | |
| - test record types & accumulate different fields |
| 7D1. | using COBOL copybooks to automatically generate a uvcopy job |
| to accumulate all numeric fields defined in the copybook |
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Our 1st example will accumulate the $amount field in the sales3 detail file. See all 20 records listed on page '1C3'. hen we run uvhd, it will display the first record & wait for our command.
uvhd dat1/sales3 r64 <-- run uvhd on the sales3 demo file ==================== - option r64 for fixed-length 64 byte records
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 130140 21 940802 IN111001 HAM001 00020+ 0000001 00000020+ .
rec#=1 rsize=64 fptr=0 fsize=1280 rcount=20
,k1=HelpPrompts,?=HelpScreens --> a 53(9) <-- enter acum command
=======
a 53(9) --> acum total=450100 from 20 recs of 20 read <-- result =====================================================
EOF, enter -1 to see last record, or 1 return to Begin File rec#=21 rsize=0 fptr=1280 fsize=1280 rcount=20 ,k1=HelpPrompts,?=HelpScreens --> q <-- quit request - program ended
uvhd dat1/sales3 r64 ====================
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 130140 21 940802 IN111001 HAM001 00020+ 0000001 00000020+ .
rec#=1 rsize=64 fptr=0 fsize=1280 rcount=20
,k1=HelpPrompts,?=HelpScreens --> a 53(9),,10(2),'21' <-- acum qualified
=================== for slsmn #21
10 20 30 40 50 60
r# 13 0123456789012345678901234567890123456789012345678901234567890123
768 308685 21 990814 CR8835 TAB013 00012- 0002001 00024012- .
a 53(9),,10(2),'21' --> acum total=30030 from 6 recs of 20 read <-- result ===============================================================
searched to EOF & reset to last record accumulated rec#=13 rsize=64 fptr=768 fsize=1280 rcount=20 ,k1=HelpPrompts,?=HelpScreens --> q <-- quit request - program ended
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
'acum1' is a pre-programmed uvcopy job to accumulate any 1 field in any file. There are 2 ways to use it - directly with uvcopy or via script 'acum1'. Here are examples of both ways, but the script is easier to use (fewer keystrokes).
#1. uvcopy acum1,fili1=dat1/sales,typ=RSF,uop=a53b9e2r64 <-- direct call
====================================================
#2. acum1 dat1/sales3 RSF a53b9e2r64 <-- script 'acum1' calls uvcopy 'acum1'
================================
Either call prompts for options in case you want to add more options or override any specified on the command line.
file types: RSF(Fixed,default), LST(Text), RDWz4,RDWz2,RDWz8, ISF,IDXf8
uop=a0b0c1d0e0f0r0 - default options
a0 - field dsplcmnt (zero relative)
b0 - field length (must specify)
c1 - cross-foot number of fields
c12 - example for 12 months sales
d0 - field type numeric unpacked
d1 - field type binary
d2 - field type packed
e0 - edit 0 decimals
e2 - edit 2 decimals
f0 - ASCII file
f1 - EBCDIC file
r0 - recsize if typ=RSF
User OPtion (uop) defaults = q1a0b0c1d0e0f0r0a53b9e2r64
-->null to accept or enter/override --> <-- null to accept cmd line as is
110524:162621:acum1: EOF fili01 rds=20 size=1280: dat1/sales3 110524:162621:acum1: EOF filo01 wrts=2 size=120: tmp/dat1_sales3_acum1
EOJ, Output File written to: tmp/dat1_sales3_acum1 report command: more,print,uvlpr12,edit,null --> more <-- display reprot
acum1: dsplcmnt 53, length 9, occurs 1, recsize 64: dat1/sales3 field total--> 4,501.00 records= 20 =======================================================
Run the following command on the custmas1 file & see if you get the result shown below. Option 'c12' crossfoots 12 5 byte packed fields starting at 120.
acum1 dat1/custmas1 RSF a120b5c12d2e2r256 =========================================
acum1: dsplcmnt 120, length 5, occurs 12, recsize 256: dat1/custmas1 field total--> 99,697.93 records= 32 =======================================================
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
'acumvpm1' is an example of a custom written 'uvcopy job' to accumulate record fields for final totals. See the job (Parameter/Instruction file) listed on the next page. Note the many #comments documenting the job.
The input file is lsited on page '1F1' but also listed here for convenience. Note 2 record types 'M' or 'P' in byte 7 & Payment amount in bytes 30-38. The uvcopy job must test the Record Type & accumulate the Payment amount.
The file listed below is dat1/vendorpaymas2 which is the 'TEXT' file equivalent of the actual input file dat1/vendorpaymas3 which is an 'RDW' file with binary record-length prefixes (must be listed with uvhd as on page '1F1').
10 20 30 40 50 60
0123456789012345678901234567890123456789012345678901234567890123
================================================================
DELL10 M Dell Inc.
DELL10 P 20100131 INV00010 000195000 Dell server 2900
HP0010 M Hewlett Packard
HP0010 P 20100215 INV00020 000042500 HP LCD Monitor
HP0010 P 20100215 INV00020 000350000 HP XW9400 Workstation
IBM010 M International Business Machines
IBM010 P 20100321 INV00030 000750000 AIX server
IBM010 P 20100322 INV00030 000750000 DB2 database software
MFC010 M Micro Focus COBOL
MFC010 P 20100420 INV00040 000320000 Micro Focus Developer
MS0010 M Microsoft Corp.
MS0010 P 20100501 INV00050 000011600 Microsoft Windows 7
REDH10 M Red Hat Linux
REDH10 P 20100620 INV00060 000034900 Red Hat Enterprise 5.4
SUN010 M Sun Microsystems Ltd
UVSI10 M UV Software Inc.
uvcopy acumvpm1,fili1=dat1/vendorpaymas3,filo1=tmp/vpmtotal ===========================================================
uvcopy acumvpm1 <-- shorter command (files default as above) ===============
110525:100809:acumvpm1: uvcopy ver=20110511 pf=/home/uvadm/pf/demo/acumvpm1 uvcopy DISAM ext=.dat LNX L64 license=110511_99V_930630 site=UV_Software dat1/vendorpaymas3 = default fili01 - null accept or reenter ? tmp/vpmtotal = default filo01 - null accept or reenter ? 110525:100810:acumvpm1: EOF fili01 rds=16 size=745: dat1/vendorpaymas3 110525:100810:acumvpm1: EOF filo01 wrts=1 size=56: tmp/vpmtotal EOJ, Output File written to: tmp/vpmtotal report dispostion, enter: more,print,uvlpr12,edit,null --> more <--
Payment Total = 2454000, input-file: dat1/vendorpaymas3 =======================================================
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| Note |
|
# acumvpm1 - accumulate payments in VendorPayMas3
# - uvcopy job to demo accumulating numeric fields
# - by Owen Townsend, UV Software, April 24/2011
# - documented at www.uvsoftware.ca/datautilities.htm#7C1
#
# Execute uvcopy to interpret this instruction file as follows:
# - default input is dat1/vendorpaymas3 (an RDW demo file)
# - default output is tmp/vpmtotal (option rop=r1 prompts for more,lp,etc)
#
# uvcopy acumvpm1,fili1=dat1/vendorpaymas3,filo1=tmp/vpmtotal
# ===========================================================
#
# uvcopy acumvpm1 <-- shorter command (files default as above)
# ===============
#
# Default input is dat1/vendorpaymas3 (RDW demo file with 2 Record Types)
# - here are 1st 2 records, see all at www.uvsoftware.ca/datautilities.htm#7C1
# - R/T M/P is in byte 7 & payment amount is in bytes 30-39 (0 relative)
#
# DELL10 M Dell Inc.
# DELL10 P 20100131 INV00010 000195000 Dell server 2900
# 10 20 30 40 50 60
# 0123456789012345678901234567890123456789012345678901234567890123
#
rop=r1 # Run OPtions to prompt for report disposition at EOJ (more,lp,etc)
fili1=?dat1/vendorpaymas3,typ=RDWz4,rcs=64
filo1=?tmp/vpmtotal,typ=LSTt,rcs=80
@run
opn all open files
# begin loop to get records, test R/T, accumulate payments until EOF
man20 get fili1,a0(64) get next record
skp> man90 (cc set > at EOF)
cmc a7(1),'P' Payment record (vs Master record) ?
skp! man20 no - return to get next record
add $ca1,a30(9) acum Payment $amount
skp man20 return to get next record
#
# EOF - write total to outfile & end job (option rop=r1 prompts for more,lp,etc)
man90 mvfv1 b0(80),'Payment Total = $ca1, input-file: $fili1'
put filo1,b0(80) write to output file
cls all close files
eoj End Of Job
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
'acumvpm2' is an extension of acumvpm1 above. In addition to accumulating the payment amounts, this job lists the payment records with the payment amount edited on the right hand side (see report at bottom of this page). See the acumvpm2 Instruction file listed on the following page.
uvcopy acumvpm2,fili1=dat1/vendorpaymas3,filo1=tmp/vpmlist ===========================================================
uvcopy acumvpm2 <-- shorter command (files default as above) ===============
110525:100818:acumvpm2: uvcopy ver=20110511 pf=/home/uvadm/pf/demo/acumvpm2 uvcopy DISAM ext=.dat LNX L64 license=110511_99V_930630 site=UV_Software dat1/vendorpaymas3 = default fili01 - null accept or reenter ? tmp/vpmlist = default filo01 - null accept or reenter ? 110525:100819:acumvpm2: EOF fili01 rds=16 size=745: dat1/vendorpaymas3 110525:100819:acumvpm2: EOF filo01 wrts=9 size=684: tmp/vpmlist EOJ, Output File written to: tmp/vpmlist default command = null, OR enter: more,print,uvlpr12,edit,null --> more <--
DELL10 P 20100131 INV00010 000195000 Dell server 2900 1,950.00
HP0010 P 20100215 INV00020 000042500 HP LCD Monitor 425.00
HP0010 P 20100215 INV00020 000350000 HP XW9400 Workstation 3,500.00
IBM010 P 20100321 INV00030 000750000 AIX server 7,500.00
IBM010 P 20100322 INV00030 000750000 DB2 database software 7,500.00
MFC010 P 20100420 INV00040 000320000 Micro Focus Developer 3,200.00
MS0010 P 20100501 INV00050 000011600 Microsoft Windows 7 116.00
REDH10 P 20100620 INV00060 000034900 Red Hat Enterprise 5.4 349.00
Payment Total 24,540.00
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
# acumvpm2 - edit, list, & accumulate payments in VendorPayMas3
# - sample/demo of custom written uvcopy jobs
# - by Owen Townsend, UV Software, April 24/2011
# - documented at www.uvsoftware.ca/datautilities.htm#7C2
#
# Execute uvcopy to interpret this instruction file as follows:
# - default input is dat1/vendorpaymas3 (an RDW demo file)
# - default output is tmp/vpmlist (option rop=r1 prompts for report more,lp,etc)
#
# uvcopy acumvpm2,fili1=dat1/vendorpaymas3,filo1=tmp/vpmlist
# ==========================================================
#
# uvcopy acumvpm2 <-- shorter command (files default as above)
# ===============
#
# Default input is dat1/vendorpaymas3 (RDW demo file with 2 Record Types)
# - here are 1st 2 records, see all at www.uvsoftware.ca/datautilities.htm#7C1
# - R/T M/P is in byte 7 & payment amount is in bytes 30-39 (0 relative)
#
# DELL10 M Dell Inc.
# DELL10 P 20100131 INV00010 000195000 Dell server 2900
# 10 20 30 40 50 60
# 0123456789012345678901234567890123456789012345678901234567890123
#
rop=r1 # Run OPtions to prompt for report disposition at EOJ
fili1=?dat1/vendorpaymas3,typ=RDWz4,rcs=64
filo1=?tmp/vpmlist,typ=LSTt,rcs=80
@run
opn all open files
# begin loop to get records, test R/T, accumulate payments until EOF
man20 get fili1,a0(256) get next record (RDW variable lth)
skp> man90 (cc set > at EOF)
cmc a7(1),'P' Payment record (vs Master record) ?
skp! man20 no - return to get next record
mvc b0(80),a0 move pmt record to output (print) area
add $ca1,a30(9) acum Payment $amount
edt b64(12),a30(9),'zzzz,zzz.99-' edit $amt to outarea
man30 put filo1,b0(80) write (print) to outfile
skp man20 return to get next record
#
# EOF - write total to outfile & end job (option rop=r1 prompt for more,lp,etc)
man90 clr b0(80),' '
mvf b50(20),'Payment Total'
edt b64(12),$ca1,'zzzz,zzz.99-' edit $total to outarea
put filo1,b0(80) write to output file
cls all close files
eoj End Of Job
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
'genacum2' is a Vancouver Utility to generate uvcopy jobs to accumulate all numeric fields (packed/zoned/binary) in 1 or all data files. The jobs are generated automatically from the 'cobmap' record layout (which is generated from the COBOL copybook).
You might use these reports to verify data file conversions, OR to get a 2nd opinion on suspicious looking totals on your COBOL reports.
We will demonstrate using the 'custmas1' copybook & datafile, which has occurs fields for 12 months sales this year & last. Here are the illustrated procedures to generate & execute the 'genacum2' job for custmas1.
#1. Login --> your homedir
#2. ls -l <-- list subdirs created previously
- should be: dat1, cpys, maps, tmp
#3a. mkdir pfa1 pfa2 <-- make subdirs for genacum2 jobs
===============
#3b. mkdir rpts <-- make subdir for output reports
==========
/home/userxx
:-----cpys/ <-- copybooks
: :----custmas1 - copybook 'custmas1' for datafile 'custmas1'
:-----dat1/ <-- data files (previously converted to ASCII)
: :-----custmas1 - datafile 'custmas1'
:-----maps/ <-- record layouts (cobmaps) created from copybooks
: :-----custmas1 - cobmap 'custmas1'
:-----pfa1/ <-- uvcopy jobs generated from cobmaps by 'genacum2'
: :-----custmas1 - uvcopy job 'custmas1' to add all fields in datafile
:-----pfa2/ <-- uvcopy jobs copied here before execution
: : - modified if if multiple record types
: : - &/or modified to insert actual datafile names
:-----rpts/ <-- reports written here
: :-----custmas1 - reports named same as copybook & datafile
(run uvdata52 to change to datafilenames)
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#4. uvcopy cobmap1,fili1=cpys/custmas1,filo1=maps/custmas1
======================================================
- convert copybook to 'cobmap' (record layout)
cobmap1 start-end bytes for cobol record fields 200902200736 pg# 0001
cpys/custmas1.cpy RCSZ=00256 bgn end lth typ
* custmas1 - cobol copy book for custmas1 file (see uvtrain.
10 cm-num pic 9(6). 0000 0005 006 n 6
10 cm-delete pic x(4). 0006 0009 004
10 cm-name pic x(25). 0010 0034 025
10 cm-adrs pic x(25). 0035 0059 025
10 cm-city pic x(16). 0060 0075 016
10 filler001 pic x. 0076 0076 001
10 cm-prov pic x(2). 0077 0078 002
10 filler002 pic x. 0079 0079 001
10 cm-postal pic x(10). 0080 0089 010
10 cm-phone pic x(12). 0090 0101 012
10 cm-contact pic x(18). 0102 0119 018
10 cm-thisyr-sales pic s9(7)v99 comp-3 occurs 12. 0120 0124 005pns 9
10 cm-lastyr-sales pic s9(7)v99 comp-3 occurs 12. 0180 0184 005pns 9
10 filler003 pic x(16). 0240 0255 016
*RCSZ=00256
On the next page, we will run genacum2 to generate a job to add all fields in test file dat1/custmas1 (based on the copybook converted to a cobmap as shown on the previous page).
But first (to understand the datafile), you might run 'uvhdcob' which displays the data field contents beside the COBOL copybook fieldnames.
#5. uvhdcob dat1/cutsmas1 maps/custmas1 <-- execute uvhdcob for custmas1
===================================
rec# 1 fieldname occurs bgn end typ<------ data (hex if typ=p/b) -----> 001 cm-num 0 5 n 130140 002 cm-delete 6 9 003 cm-name 10 34 EVERGREEN MOTORS LTD. 004 cm-adrs 35 59 1815 BOWEN ROAD 005 cm-city 60 75 NANAIMO 006 filler001 76 76 007 cm-prov 77 78 BC 008 filler002 79 79 009 cm-postal 80 89 V9S1H1 010 cm-phone 90 101 250-754-5531 011 cm-contact 102 119 LARRY WRENCH 012 cm-thisyr-sales 012 120 124pns 001234567C 013 cm-lastyr-sales 012 180 184pns 001234567D 014 filler003 240 255 C 19950531
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#6. uvcopy genacum2,fili1=maps/custmas1,filo1=pfa1/custmas1,uop=f0
==============================================================
- convert cobmap to uvcopy job to accumulate All numeric fields
uop=f2m1 - option defaults (for generated jobs)
f0 - Input file default typ=RSF fixed lth (in generated job)
f2 - Input typ=RDWz2 (2 byte BIG-end binary recsize hdrs)
f4 - Input file typ=RDWz4 (4 byte hdrs, bytes 3&4 nulls)
m1 - read copybook record layouts from maps/... in $CNVDATA
m2 - read copybook record layouts from maps/... in $RUNLIBS
User OPtion (uop) defaults = q1f2m1
-->null to accept or enter/override --> f0m0 <-- f0 typ=RSF, m0 ./maps
| Note |
|
# uvcopy job to accumulate all numeric fields in a datafile
opr='jobname=JOBNAME - pfa2 name=datafilename'
opr='copybook=custmas1 - maps/pfa1 name=copybookname'
uop=q0i3,rop=r1 # minimize startup msgs, prompt for report disposition
was=a33000c200000t100000 # 33K maxrec, 200K cpybkmap, 100K acum tbl
fili1=?dat1/custmas1,rcs=00256,typ=RSF
filo1=?rpts/custmas1,rcs=128,typ=LSTt
fili2=?maps/custmas1,rcs=128,typ=LST
@run
opn all
rtb fili2,c0(100),c0(100) read cobmap into area c table
clr t0(100000),x'00' clear area t for accumulator table
loop get fili1,a0 get next record into area 'a'
skp> eof
add $cb3,1 count all record types
# --- --- <-- insert any Record Type tests here
mvn $rt,0 init register t for current record
mvn $rs,0 init register s for current record
add ts9800(8b),1 count 1st (or only) R/T in copybook
bal acum,'0,6,z,1,cm-num'
bal acum,'120,5,p,12,cm-thisyr-sales'
bal acum,'180,5,p,12,cm-lastyr-sales'
skp loop
eof bal dump dump tables of fieldnames & acummulators
cls all
eoj
@pf2=acum2.sub
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#7. uvcopy pfa1/custmas1 <-- execute job to add all numeric fields custmas1
====================
#8. uvlp12 rpts/custmas1 <-- print report
====================
genacum2 RCSZ=00256 DataFile=dat1/custmas1 2009/02/26_09:22:03 copybook=cpys/custmas1.cpy line# Field-Name occurs start lth type field-total
1 cm-num 0005 006 n 7,259,353
2 cm-thisyr-sales 12 0124 005 pns 99,697.93
3 cm-lastyr-sales 12 0184 005 pns 45,440.85
Record count 32 ** Hash-Total 21,773,231 **
The above procedures were for 1 file at a time. For major projects we would want to generate jobs for all copybooks & all datafiles. See the procedures to generate jobs for all copybooks & all datafiles documented at https://www.uvsoftware.ca/datacnv1.htm#8G5.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| 8A1. | Comparing Variable Length RDW files - Overview |
| 'uvcmp2' to compare Variable Length RDW files | |
| vs 'uvcmp1' for FIXED Length files (see CMPjobs.htm#Part_1) | |
| 'uvcmpRDW1' script easier way to run uvcmp2 (shorter commands) | |
| Variable Length file types (RDW, BDW/RDW, BDW/RDW unblocked, Micro Focus) |
| 8B1. | Setup Directories to demo uvcmp2 |
| - may setup 'cmpdata' directory in your homedir | |
| - make subdirs & copy test/demo files from /home/uvadm/dat1/... |
| 8C1. | testRDWe & testRDWea - test files supplied for VarLth comparison demo |
| - display with 'uvhd', since binary RDW conflicts with 'vi' |
| 8D1. | Op. Instrns to compare RDW files using uvcopy job 'uvcmp2' directly |
| 8D2. | console log |
| 8D3. | sample report |
| 8E1. | Op. Instrns to compare RDW files using script 'uvcmpRDW1' |
| 8E2. | console log |
| 8E3. | sample report |
This Part 8 in this document DataUtilities.htm is taken from Part 2 in CMPjobs.htm, which is the complete documentation for file compare jobs. CMPjobs.htm covers all file types (Fixed-Length, Micro Focus Indexed, etc) and alternate jobs to allow for Inserted or Deleted records.
This Part covering variable-length RDW files was included here in DataAnalysis.htm because it was short, but gives you a good idea of the data file compare jobs.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
We will document 2 ways to compare variable length RDW files
Before documenting the comparison procedures, we will illustrate the RDW variable length file format on the next few pages.
| 8B1. | testRDWe - sample VARIABLE length data file in RDW format. |
| 8B2. | testRDWea - Alternate test file for comparison |
| - with some intentional mismatches |
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
/home/userxx <-- your home directory :-----... :-----cmpdata <-- working directory for datafile compares : : : :-----dat1 <-- subdir to store test files : : :-----testRDWe <-- sample RDW file for comparison : : :-----testRDWea <-- 'a'lternate file with some mismatches : : : :-----rptcmp <-- subdir to receive comparison reports : : :-----testRDWe <-- comparison report : : - name same as file#1 : : : :-----tmp <-- subdir required by uvhd for any Printing : : :-----testRDWe_090217_124800P <-- Print file created by uvhd
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#1. login to your homedir & mkdir cmpdata (if not already existing)
#2. cd cmpdata <-- change into comparison directory
==========
#3. mkdir dat1 rptcmp tmp <-- make subdirs required
=====================
#4a. cp /home/uvadm/dat1/testRDWe dat1
=================================
- copy 1st EBCDIC varlth file to be compared to dat1/...
#4b. cp /home/uvadm/dat1/testRDWea dat1
==================================
- copy 2nd EBCDIC varlth file to be compared to dat1/...
#5. uvhd dat1/testRDWe z4a
======================
Investigate 1st variable lth file with options:
z4 - use binary recsize in 1st 2 bytes of record
a - translate uvhd character line from EBCDIC to ASCII
#5a. --> p8 <-- Print 8 records (all)
- see printout below
| Note |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
uvhd dat1/testRDWe z4a1 <-- option z4 = variable lth binary 1st 2 bytes
======================= - option a1 = translate char line to Ascii
- zones & digits lines show EBCDIC codes
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 ....DELL10 - Dell Inc.
0100CCDDFF464C8994C98444
080045331000045330953B00
r# 2
24 ....HP0010 - Hewlett Packard
0100CDFFFF464C8A98AA4D889898
0C00870010000856353307132194
r# 3
52 ....IBM010 - International Business Machines
0200CCDFFF464C9A8998A899894CAA898AA4D888898A
0C009240100009535951396513024295522041389552
r# 4
96 ....MFC010 - Micro Focus COBOL
0200DCCFFF464D88994C98AA4CDCDD44
00004630100004939606634203626300
r# 5
128 ....MS0010 - Microsoft Corp.
0100DEFFFF464D8899A98A4C9994
0C0042001000049396266303697B
r# 6
156 ....REDH10 - Red Hat Linux
0100DCCCFF464D884C8A4D89AA44
0C00954810000954081303954700
r# 7
184 ....SUN010 - Sun Microsystems Ltd
0200EEDFFF464EA94D8899AAAA89A4DA8444
040024501000024504939628235420334000
r# 8
220 ....UVSI10 - UV Software Inc.
0200EEECFF464EE4E98AA8984C984444
0000452910000450266361950953B000
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
uvhd dat1/testRDWea z4a1 <-- option z4 = variable lth binary 1st 2 bytes
======================== - option a1 = translate char line to Ascii
- zones & digits lines show EBCDIC codes
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 ....DELL10 - Dell Inc.
0100CCDDFF464C8994C98444
080045331000045330953B00
r# 2
24 ....HP0010 - Hewlett Packard
0100CDFFFF464C8A98AA4D889898
0C00870010000856353307132194
r# 3
52 ....IBM010 - International Business Mothers
0200CCDFFF464C9A8998A899894CAA898AA4D9A889A4
0C009240100009535951396513024295522046385920
r# 4
96 ....MFC010 - Micro Focus COBOL
0200DCCFFF464D88994C98AA4CDCDD44
00004630100004939606634203626300
r# 5
128 ....MS0010 - Microsoft Corp.
0100DEFFFF464D8899A98A4C9994
0C0042001000049396266303697B
r# 6
156 ....REDH10 - Red Hat Linux
0100DCCCFF464D884C8A4D89AA44
0C00954810000954081303954700
r# 7
184 ....SUN010 - Sun Microsystems Corp
0200EEDFFF464EA94D8899AAAA89A4C99944
040024501000024504939628235420369700
r# 8
220 ....UVSI10 - UV Software Inc.
0200EEECFF464EE4E98AA8984C984444
0000452910000450266361950953B000
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Assuming the 2 EBCDIC RDW files have already been copied to subdir dat1/... in your homedir (as shown on page '8B1').
#1. uvcopy uvcmp2,fili1=dat2/testRDWe,fili2=dat2/testRDWea,uop=y1z4
===============================================================
- run comparison utility
option y1 - translate char line to ASCII (zone & digit lines EBCDIC)
option z4 - RDW file with 4 byte recsize prefixes (vs 2 bytes)
- see the console log displayed on the next page --->
#2. uvlp12 rptcmp/testRDWe
======================
- print output report
#1a. uvcmpRDW1 dat1/testRDWe dat1/testRDWea z4y1 <-- execute alternative
=========================================== (shorter command)
- option 'z4' (4 byte prefixes) is default & may be omitted
- option 'y1' translates character line to ASCII (for EBCDIC files)
| Note |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
uvcopy uvcmp2,fili1=dat2/testRDWe,fili2=dat2/testRDWea,uop=y1z4 ===============================================================
uvcmp2 - compare files & print diffs in hex with "*" markers (Nov28/07)
uop=q1a0b0c0d0e0f0g0h0p30r8192s8t500000u3x2y0 - option defaults
a# - dsplcmnt to exclude field #1
b# - length to exclude field #1
c# - dsplcmnt to exclude field #2
d# - length to exclude field #2
e# - dsplcmnt to exclude field #3
f# - length to exclude field #3
g# - dsplcmnt to exclude field #4
h# - length to exclude field #4
n0 - print column scale only at begin report
n1 - print column scale for each record mismatch pair
p30 - line# to test overflow at begin record only
r8192- record size (largest RDW recsize, default 8192)
s8 - StopPrint count, after s# mismatched records
t500000 - StopRead count, after s# mismats, else read to EOF
u1 - stop print at last segment with mismatch
u2 - inhibit printing all matching segments after 1st
x# - hex or char print (x1 char, x2 hex, x0 auto)
y0 - do NOT translate, file already ASCII, default(y0)
y1 - translate char lines to ASCII (for EBCDIC files)
z2 - RDWz2 2 byte RDW recsize prefixes (vs RDWz4 dflt)
090217:131431:uvcmp2: uvcopy ver=20090212 pf=/home/uvadm/pf/util/uvcmp2
uvcopy DISAM ext=dat LNX L64 license=20090212V site=UV_Software
User OPtion (uop) defaults = q1p30r8192s8t500000u3x2y0y1
-->null to accept or enter/override -->
==================== EOF or StopPrint/StopRead count reached ==============
File1Total=0, File2Total=0, StopPrint=8, StopRead=500000
File1Reads=8, TotalMisMatches=2, MisMatchesPrinted=2
090217:131431:uvcmp2: EOF fili01 rds=8 size=252: dat1/testRDWe 090217:131431:uvcmp2: EOF fili02 rds=8 size=252: dat1/testRDWea 090217:131431:uvcmp2: EOF fild03 size=4096: rptcmp 090217:131431:uvcmp2: EOF filo03 wrts=32 size=1496: rptcmp/testRDWe hits=2 EOJ, Output File written to: rptcmp/testRDWe default command = null, OR enter: more,print,uvlpr12,edit,null -->
See differences report listed on the next page --->
| Note |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
${JOBID2} uvcmp2 - compare RDW files, print mismatched records, '*' flag diffs
2009/02/17_13:14:31 uop=q1p30r8192s8t500000u3x2y0y1
recsize reccount file-size typ Report=rptcmp/testRDWe
1: 8192 0* 0 RDW File1=dat1/testRDWe
2: 8192 0* 0 RDW File2=dat1/testRDWea
1 2 3 4 5 6
f#rec#byte# 0123456789012345678901234567890123456789012345678901234567890123
============================================================================
1 3 0 IBM010 - International Business Machines
40 CCDFFF464C9A8998A899894CAA898AA4D888898A
9240100009535951396513024295522041389552
** ****
2 3 IBM010 - International Business Mothers
CCDFFF464C9A8998A899894CAA898AA4D9A889A4
9240100009535951396513024295522046385920
** ****
1 7 0 SUN010 - Sun Microsystems Ltd
32 EEDFFF464EA94D8899AAAA89A4DA8444
24501000024504939628235420334000
****
2 7 SUN010 - Sun Microsystems Corp
EEDFFF464EA94D8899AAAA89A4C99944
24501000024504939628235420369700
****
==================== EOF or StopPrint/StopRead count reached ============== File1Total=0, File2Total=0, StopPrint=8, StopRead=500000 File1Reads=8, TotalMisMatches=2, MisMatchesPrinted=2
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Script 'uvcmpRDW1' makes it easier to compare variable length RDW files because:
#1. uvcmpRDW1 dat1/testRDWe dat1/testRDWea z4y1 <-- execute script
===========================================
- option 'z4' (4 byte prefixes) is default & may be omitted
- option 'y1' translates character line to ASCII (for EBCDIC files)
#2a. vi rptcmp/testRDWe <-- view report
==================
#2b. uvlp12 rptcmp/testRDWe <-- print report
======================
| Note |
|
Also see 'uvcmpRDWAI2/uvcmpRDWEI2' to compare variable length RDW files allowing for inserts & deletes at CMPjobs.htm#4H1.
uvcmpRDWAI2 dat1/vendormas3 dat1/vendormas32 r64k0l6 ==================================================== - compare 2 RDW files, report created in rptcmp/vendormas3
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| 9A1. | Data table analysis Overview |
| 9B1. | Sample reports |
| 9C1. | Subdirs for Generating & Executing table summary jobs |
| 9C2. | Setting up subdirs in your homedir |
| 9D1. | Generate Table summary jobs - for 1 file at a time |
| 9E1. | Generate table summary job - Illustrated |
| copybook maps/custmas0x will generate pft1/custmas0x table summary job | |
| - copybook maps/custmas0x input listed |
| 9E2. | Execute 'gentbl1E' to Generate table summary job 'custmas0x' |
| - console display, prompts for option changes | |
| - option 'm15' maximum table entries to be maintained | |
| - option 'v4' maximum Invalid table entries to be maintained | |
| - option 'd3' (default) converts invalid numerics to zeros | |
| - option 'd4' clears invalid numeric fields (shows hex on right side) |
| 9E3. | Listing of gnerated table summary job pft1/custmas0x |
| - 14 fields in copybook generate code for 14 summary tables | |
| - thisyear/lastyear sales occurs 12 handled by index register & code loop |
| 9E4. | copy/rename/modify table summary job for easier testing/check/rerun |
| - copy pft1/custmas0x to pft1/cm03v4 & modify | |
| - reduce no of fields from 13 to 3 | |
| - remove crossfoot of 12 months sales to just Jan sales |
| 9E5. | Execute modified table summary job pft1/cm03v4 |
| - output report listed, 3 tables (cust#, city, Jan sales) |
| 9F1. | further modify table summary job to test accumulation for total Jan sales |
| - copy pft1/cm03v4 to pft1/cm03v4a & modify | |
| - reduce to just 1 table for January sales | |
| - add accumulator for Jan sales total | |
| - could accumulate up to 6 fields on right side of report |
| 9F2. | Execute modified table summary job |
| - output report listed (showing Jan sales total) |
| 9G1. | use 'uvhdcob' to investigate problems reported by table summary jobs |
| 9H1. | operating instructions - All files in directory |
| 9I1. | inserting data-file-names in table analysis jobs |
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
'gentbl1A' & 'gentbl1E' are Vancouver Utilities to table summarize all fields in a data file (gentbl1A for Asciii & gentbl1E for Ebcdic).
gentbl1A/gentbl1E are driven by the COBOL copybooks to generate uvcopy jobs that will read thru data files & create a summary table of values found in each & all fields defined in the COBOL copybook.
Obviously tabling all values would be too voluminous without limiting options. Default option 'm30' limits the no of entries retained to 30. After 30, new entries are added if their value is greater than or less than any existing entry, & a mid-point entry is removed to maintain the number of entries to 30.
Default option 'v10' limits the no of ~invalid entries retained to 10. After 10, invalid entries are just counted for the ~Invalid Total line.
You can generate the data analysis/validation jobs for 1 file at a time, or you can generate the jobs for all COBOL copybooks in the directory.
The generated jobs are named the same as the copybooks, and are written to a separate subdir 'pft1'. The generated jobs assume the data files have the same name as the copybook and are stored in subdir 'dat1'.
This means that no changes are required to the generated jobs, if you can copy/rename your data files into subdir 'dat1', but it us very simple to edit the generated jobs with the actual directory/filename where your data files currently exist.
If you have dozens or hundreds of copybooks/datafiles, there is an optional utility 'uvdata52', that uses a control file to relate copybooknames to datafilenames as it copies the jobs from 'pft1' to pft2'. Of course you have to 1st edit/create the control-file.
There are 2 times you can change the 'm' & 'v' options. #1 when you generate the jobs for 1 or all copybooks. #2 you may edit the generated jobs to change the defaults. After executing & investigating the 1st run reports, you may well want to rerun with expanded options for problem fields.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
cm03v4 2011/01/30_12:25:42 Table Analysis all fields in dat1/custmas0x
tbl#0005 tblt005f7d3s0m015v04 a60(16ce)
line# count % 1strec# 10 CM-CITY pic x(16).
1 2 6 23 CALGARY
2 6 18 13 EDMONTON
3 1 3 21 FORT MCMURRAY
4 1 3 1 NANAIMO
5 1 3 11 NORTH VANCOUVER
6 1 3 9 PORT COQUITLAM
7 1 3 30 PRINCE GEORGE
8 1 3 26 THREE HILLS
9 3 9 3 VANCOUVER
10 1 3 16 WARBURG
11 2 6 20 WHITEHORSE
12 1 3 6 ~CHIL.IWACK C3C8C9D3EEC9E6C1C3D24040404
13 1 3 8 ~HOPE. C8D6D7C5EE40404040404040404
14 1 3 5 ~K.LOWNA D2EED3D6E6D5C14040404040404
15 2 6 10 ~VA.COUVER E5C1EEC3D6E4E5C5D9404040404
16 6 18 ~Invalid Total
32*100 *TOTAL*
cm03v4 2011/01/31_16:15:59 Table Analysis all fields in dat1/custmas0x
tbl#0012 tblt012f7d3s2m020v04 a120(5pe)
line# count % 1strec# 10 CM-THISYR-SALES pic s9(7)v99 comp-3 occurs 12.
1 16 50 4 0+
2 1 3 17 1443+
3 1 3 3 1528+
4 1 3 31 2487+
5 1 3 2 4680+
6 1 3 32 5246+
7 1 3 14 21248-
8 1 3 28 27483+
9 1 3 27 38355+
10 2 6 1 1234567+
11 1 3 8 ~ 123+ 000000123E
12 1 3 15 ~ 1200567+ 0012FF567C
13 1 3 10 ~ 1234007+ 001234AA7C
14 2 6 5 ~ 1234560+ 00123456EE
15 6 18 ~Invalid Total
32*100 *TOTAL*
| Note |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
/home/userxx :-----tablesum : :-----cpys <-- subdir for COBOL COPYBOOKs : : :-----custmas0x.cpy <-- copybook for customer master : : :-----sales0.cpy <-- copybook for sales detail : : : : :-----dat1 <-- subdir for test/demo DATA : : :-----custmas0x <-- customer master data file : : :-----sales0 <-- sales detail data file : : : : :-----maps <-- subdir for RECORD LAYOUTs (cobmaps) : : :-----custmas0x <-- cobmap for customer master : : :-----sales0 <-- cobmap for sales detail : : : : :-----pft1 <-- subdir for table analysis JOBS : : :-----custmas0x <-- table analysis job for customer master : : :-----sales0 <-- table analysis job for sales detail : : : : :-----tbls <-- subdir for data table analysis REPORTs : : :-----custmas0x <-- table analysis report customer master : : :-----sales0 <-- table analysis report for sales detail : : : : :-----tmp <-- subdir for uvhd/uvhdcob print/verify files
| Note |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
We suggest you setup these subdirs within a table summary analyze/verify super directory within your homedir.
#1. Login ---> your home dir
=====
#2. mkdir tablesum <-- make superdir for subdirs
==============
#3. mkdir cpys dat1 maps pft1 tbls tmp <-- make subdirs for generation/execution
==================================
#4a. cp /home/uvadm/dat1/custmas0x dat1 <-- copy data-files
==================================
#4b. cp /home/uvadm/dat1/sales0 dat1
===============================
#5a. cp /home/uvadm/mf/cpys/custmas0x dat1 <-- copy COBOL copy-books
=====================================
#5b. cp /home/uvadm/mf/cpys/sales0 dat1
==================================
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| Note |
|
#1. uvcopy cobmap1,fili1=cpys/custmas0x,filo1=maps/custmas0x
========================================================
- convert copybook to record layout (cobmap)
#2. uvcopy gentbl1E,fili1=maps/custmas0x,filo1=pft1/custmas0x,uop=m15v6
===================================================================
- generate verify job from record layout
- default options changed from 'uop=m30v10' as desired ('uop=m15v6' here)
- m15 = max total lines, v6 = max invalid lines
#3. vi pft1/custmas0x <-- inspect output, modify for multi R/T's
================= (no manual changes required for custmas0x)
#4. uvcopy pft1/custmas0x,[fili1=dat1/custmas0x,filo1=tbls/custmas0x]
=================================================================
- execute table analysis/verify job for customer master file
- I/O subdirs/files will default as shown above in [...]
#4a. uvcopy pft1/custmas0x <-- execute (easier) [files default as above]
=====================
#5. vi tbls/custmas0x <-- inspect table analysis/verify report
=================
#5a. uvlp12 tbls/custmas0x <-- print table analysis/verify report
=====================
| Note |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#1. uvcopy cobmap1,fili1=cpys/custmas0x,filo1=maps/custmas0x
========================================================
- convert copybook to record layout (cobmap)
cobmap1 start-end bytes for cobol record fields 201012221438 pg# 0001
cpys/custmas0x.cpy RCSZ=00256 bgn end lth typ
* custmas0x - cobol copy book for custmas0x file (see uvtrain).
10 cm-num pic 9(6). 0000 0005 006 n 6
10 cm-delete pic x(4). 0006 0009 004
10 cm-name pic x(25). 0010 0034 025
10 cm-adrs pic x(25). 0035 0059 025
10 cm-city pic x(16). 0060 0075 016
10 filler001 pic x. 0076 0076 001
10 cm-prov pic x(2). 0077 0078 002
10 filler002 pic x. 0079 0079 001
10 cm-postal pic x(10). 0080 0089 010
10 cm-phone pic x(12). 0090 0101 012
10 cm-contact pic x(18). 0102 0119 018
10 cm-thisyr-sales pic s9(7)v99 comp-3 occurs 12. 0120 0124 005pns 9
10 cm-lastyr-sales pic s9(7)v99 comp-3 occurs 12. 0180 0184 005pns 9
10 filler003 pic x(16). 0240 0255 016
*RCSZ=00256
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#2. uvcopy gentbl1E,fili1=maps/custmas0x,filo1=pft1/custmas0x,uop=d4m15v4
=====================================================================
- run gentbl1E to generate table summary from copybook 'custmas0x'
- displays option defaults & prompts for option changes
- see 'change option defaults *NOTE*' below
gentbl1E - generate uvcopy jobs to table analyze all fields in copybook
uop=f0d3m030v10 - option defaults
f0 - Input file typ=RSF fixed lth (in generated job)
f2 - Input typ=RDWz2 (2 byte BIG-end binary recsize hdrs)
f4 - Input file typ=RDWz4 (4 byte hdrs, bytes 3&4 nulls)
f8 - Input file typ=RDWz8 (BDW + RDW)
d3 - convert invalid digits in numeric fields to zeros
d4 - clear invalid numeric fields, show hex on rightside
m030 - max tbl entries, lower/higher added mid-point removed
v10 - max Invalid entries, excess added for ~Invalid Total
110313:150859:gentbl1E: uvcopy ver=20110313 pf=/home/uvadm/pf/IBM/gentbl1E
uvcopy DISAM ext=.dat LNX L64 license=110313_00V_930601 site=UV_Software
User OPtion (uop) defaults = q1f0d3m030v10
-->null to accept or enter/override --> d4m15v4 <------- change options *NOTE*
110127:163805:gentbl1E: EOF fili01 rds=19 size=1309: maps/custmas0x
110127:163805:gentbl1E: EOF filo01 wrts=47 size=1719: pft1/custmas0x
tblt001f7d3s1m030v10 a0(6ze),'10 CM-NUM pic 9(6).' <-- default options
=========**==*******==============================
tblt001f7d4s1m015v04 a0(6ze),'10 CM-NUM pic 9(6).' <-- sample overrides
=========**==*******==============================
| d3 |
|
| d4 |
|
| m030 |
|
| m015 |
|
| v10 |
|
| v04 |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
# uvcopy job to generate jobs to table analyze all fields in copybook
opr='jobname=JOBNAME - pft2 name=datafilename'
opr='copybook=custmas0x - maps/pft1 name=copybookname'
arg1=JOBNAME,arg2=custmas0x
rop=u400v500000
was=a33000b33000
fili1=?dat1/custmas0x,rcs=00256,typ=RSF
filo1=?tbls/custmas0x,rcs=128,typ=LSTt
@run
opn all
loop get fili1,a0
skp> eof
mvc b0(00256),a0
tra b0(00256) translate to Ascii
tblt001f7d3s1m015v04 a0(6ze),'10 CM-NUM pic 9(6).'
tblt002f7d3s0m015v04 a6(4ce),'10 CM-DELETE pic x(4).'
tblt003f7d3s0m015v04 a10(25ce),'10 CM-NAME pic x(25).'
tblt004f7d3s0m015v04 a35(25ce),'10 CM-ADRS pic x(25).'
tblt005f7d3s0m015v04 a60(16ce),'10 CM-CITY pic x(16).'
tblt006f7d3s0m015v04 a76(1ce),'10 FILLER001 pic x.'
tblt007f7d3s0m015v04 a77(2ce),'10 CM-PROV pic x(2).'
tblt008f7d3s0m015v04 a79(1ce),'10 FILLER002 pic x.'
tblt009f7d3s0m015v04 a80(10ce),'10 CM-POSTAL pic x(10).'
tblt010f7d3s0m015v04 a90(12ce),'10 CM-PHONE pic x(12).'
tblt011f7d3s0m015v04 a102(18ce),'10 CM-CONTACT pic x(18).'
#
mvn $ra,0
mvn $rj,0
oci010 tblt012f7d3s2m015v04 aa120(5pe),'10 CM-THISYR-SALES pic s9(7)v99 comp-3 occurs 12.'
add $ra,00005
add $rj,00001
cmn $rj,00012
skp< oci010
#
mvn $ra,0
mvn $rj,0
oci020 tblt013f7d3s2m015v04 aa180(5pe),'10 CM-LASTYR-SALES pic s9(7)v99 comp-3 occurs 12.'
add $ra,00005
add $rj,00001
cmn $rj,00012
skp< oci020
#
tblt014f7d3s0m015v04 a240(16ce),'10 FILLER003 pic x(16).'
skp loop
#
eof tbp filo1,'Table Analysis all fields in $fili1'
cls all
eoj
| Note |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Before illustrating the execution of the table summary job, we will reduce the number of fields/tables from 14 to 3 (manageable for this documentation).
#3a. cp pft1/custmas0x pft1/cm03v4 <-- copy/rename to preserve original
=============================
#3b. vi pft1/cm03v4 <-- edit generated job, reduce to 3 fields
============== - edited result below (original above)
# uvcopy job to generate jobs to table analyze all fields in copybook
# cm03v4 - custmas0x stripped down to just 3 fields & drop crossfoot 12 mths
# - also see cm03v4a, just 1 field (sales) but also accumulated
# - output name changed to $jobname to preserve each jobs output
rop=u400v500000
was=a33000b33000
fili1=?dat1/custmas0x,rcs=00256,typ=RSF
filo1=?tbls/$jobname,rcs=128,typ=LSTt
@run
opn all
loop get fili1,a0
skp> eof
tblt001f7d3s1m015v04 a0(6ze),'10 CM-NUM pic 9(6).'
tblt005f7d3s0m015v04 a60(16ce),'10 CM-CITY pic x(16).'
oci010 tblt012f7d3s2m015v04 a120(5pe),'10 CM-THISYR-SALES pic s9(7)v99 comp-3 occurs 12.'
skp loop
#
eof tbp filo1,'Table Analysis all fields in $fili1'
cls all
eoj
filo1=?tbls/custmas0x <-- as generated in original job 'custmas0x'
=====================
filo1=?tbls/$jobname <-- changed to '$jobname' to not overwrite custmas0x
==================== - output report will same as jobname ('cm03v4' here)
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#4. uvcopy pft1/cm03v4,[fili1=dat1/custmas0x,filo1=tbls/custmas0x]
==============================================================
- execute table analysis/verify job for customer master file
- I/O subdirs/files will default as shown above in [...]
#4a. uvcopy pft1/custmas0x <-- execute (easier) [files default as above]
=====================
cm03v4 2011/01/31_16:15:59 Table Analysis all fields in dat1/custmas0x
tbl#0001 tblt001f7d3s1m015v04 a0(6ze)
line# count % 1strec# 10 CM-NUM pic 9(6).
1 1 3 1 120140
2 1 3 2 132588
3 1 3 3 139923
4 1 3 4 142175
5 1 3 6 147615
6 1 3 7 149304
7 1 3 13 201120
8 1 3 14 204700
9 1 3 16 211140
10 1 3 26 315512
11 1 3 31 403887
12 1 3 5 ~ 105264 F1EEF5F2F6F4
13 1 3 10 ~ 105688 F1EEF5F6F8F8
14 1 3 15 ~ 205060 F2F0F5EEF6F0
15 1 3 20 ~ 234090 F2F3F4EEF9F0
16 6 18 ~Invalid Total
32*100 *TOTAL*
cm03v4 2011/01/31_16:15:59 Table Analysis all fields in dat1/custmas0x
tbl#0005 tblt005f7d3s0m015v04 a60(16ce)
line# count % 1strec# 10 CM-CITY pic x(16).
1 2 6 23 CALGARY
2 6 18 13 EDMONTON
3 1 3 21 FORT MCMURRAY
4 1 3 1 NANAIMO
5 1 3 11 NORTH VANCOUVER
6 1 3 9 PORT COQUITLAM
7 1 3 30 PRINCE GEORGE
8 1 3 26 THREE HILLS
9 3 9 3 VANCOUVER
10 1 3 16 WARBURG
11 2 6 20 WHITEHORSE
12 1 3 6 ~CHIL.IWACK C3C8C9D3EEC9E6C1C3D24040404
13 1 3 8 ~HOPE. C8D6D7C5EE40404040404040404
14 1 3 5 ~K.LOWNA D2EED3D6E6D5C14040404040404
15 2 6 10 ~VA.COUVER E5C1EEC3D6E4E5C5D9404040404
16 6 18 ~Invalid Total
32*100 *TOTAL*
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
cm03v4 2011/01/31_16:15:59 Table Analysis all fields in dat1/custmas0x
tbl#0012 tblt012f7d3s2m020v04 a120(5pe)
line# count % 1strec# 10 CM-THISYR-SALES pic s9(7)v99 comp-3 occurs 12.
1 16 50 4 0+
2 1 3 17 1443+
3 1 3 3 1528+
4 1 3 31 2487+
5 1 3 2 4680+
6 1 3 32 5246+
7 1 3 14 21248-
8 1 3 28 27483+
9 1 3 27 38355+
10 2 6 1 1234567+
11 1 3 8 ~ 123+ 000000123E
12 1 3 15 ~ 1200567+ 0012FF567C
13 1 3 10 ~ 1234007+ 001234AA7C
14 2 6 5 ~ 1234560+ 00123456EE
15 6 18 ~Invalid Total
32*100 *TOTAL*
tbl#0012 tblt012f7d3s2m015v04 aa120(5pe)
=========================================
See all tbl options at https://uvsoftware.ca/datautilities.htm#tbl. Here is a description of the options used in this case:
| t012 |
|
| f7 |
|
| d3 |
|
| d4 |
|
| s2 |
|
| m015 |
|
| v04 |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
If we had specified option 'd4' (vs default 'd3') the table on the prior page would then appear as shown below - with invalid numeric fields cleared, but still showing the hexadecimal value on the right.
cm03v4 2011/01/31_16:15:59 Table Analysis all fields in dat1/custmas0x
tbl#0012 tblt012f7d3s2m020v04 a120(5pe)
line# count % 1strec# 10 CM-THISYR-SALES pic s9(7)v99 comp-3 occurs 12.
1 16 50 4 0+
2 1 3 17 1443+
3 1 3 3 1528+
4 1 3 31 2487+
5 1 3 2 4680+
6 1 3 32 5246+
7 1 3 14 21248-
8 1 3 28 27483+
9 1 3 27 38355+
10 2 6 1 1234567+
11 1 3 8 ~ 000000123E
12 1 3 15 ~ 0012FF567C
13 1 3 10 ~ 001234AA7C
14 2 6 5 ~ 00123456EE
15 6 18 ~Invalid Total
32*100 *TOTAL*
#1a. vi pft1/custmas0x <-- edit to modify options
=================
tbl#0012 tblt012f7d3s2m015v04 aa120(5pe) <-- d3 default
===================**====================
tbl#0012 tblt012f7d4s2m015v04 aa120(5pe) <-- d4 modified
===================**====================
#1b. uvcopy pft1/custmas0x <-- rerun the table summary job
=====================
#2a. uvcopy gentbl1E,fili1=maps/custmas0x,filo1=pft1/custmas0x,uop=d4m15v4
=====================================================================
- regenerate table summary job with desired options
#3a. uvcopyx gentbl1E maps pft1 uop=q0i7d4m15v4
==========================================
- generate table sumary jobs for ALL copybooks (cobmaps)
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
If desired you can add up to 6 accumulator fields on the right side of the table argument. To illustrate we will reduce out table job to just the 1 table for January sales & define the Jan sales field as an accumulator on the right.
#3a. cp pft1/cm03v4 pft1/cm03v4a <-- copy/rename to preserve original
===========================
#3b. vi pft1/cm03v4a <-- edit generated job, reduce to 1 field (Jan sales)
=============== - edited result below (before editing on page '9E4').
# uvcopy job to generate jobs to table analyze all fields in copybook
# cm03v4a - custmas0x stripped down to just 1 fields & drop crossfoot 12 mths
# - table analyze just Jan sales, but also accumulate
# - output name changed to $jobname to preserve each jobs output
rop=u400v500000
was=a33000b33000
fili1=?dat1/custmas0x,rcs=00256,typ=RSF
filo1=?tbls/$jobname,rcs=128,typ=LSTt
@run
opn all
loop get fili1,a0
skp> eof
oci010 tblt012f7d3s2m015v04 a120(5pe),'Jan-Sales pic s9(7)v99 comp-3;Jan-sales',a120(5pe)
skp loop
#
eof tbp filo1,'Table Analysis all fields in $fili1'
cls all
eoj
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#4. uvcopy pft1/cm03v4a,[fili1=dat1/custmas0x,filo1=tbls/custmas0x]
===============================================================
- execute table analysis/verify job for customer master file
- I/O subdirs/files will default as shown above in [...]
#4a. uvcopy pft1/custmas0x <-- execute (easier) [files default as above]
=====================
cm03v4a 2011/01/31_16:33:30 Table Analysis all fields in dat1/custmas0x
tbl#0012 tblt012f7d3s2m015v04 a120(5pe) -acum#1- %
line# count % 1strec# Jan-Sales pic s9(7)v99 comp-3 Jan-sales
1 16 50 4 0+ 0 0
2 1 3 17 1443+ 1,443 0
3 1 3 3 1528+ 1,528 0
4 1 3 31 2487+ 2,487 0
5 1 3 2 4680+ 4,680 0
6 1 3 32 5246+ 5,246 0
7 1 3 14 21248- 21,248- 0
8 1 3 28 27483+ 27,483 0
9 1 3 27 38355+ 38,355 0
10 2 6 1 1234567+ 2,469,134 33
11 1 3 8 ~ 123+ 000000123E 123 0
12 1 3 15 ~ 1200567+ 0012FF567C 1,200,567 16
13 1 3 10 ~ 1234007+ 001234AA7C 1,234,007 16
14 2 6 5 ~ 1234560+ 00123456EE 2,469,120 33
15 6 18 ~Invalid Total
32*100 *TOTAL* 7,467,492 *100
| 4. | See https://www.uvsoftware.ca/uvcopy3.htm#tbl for complete details. |
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
'uvhdcob' is the best tool to investigate problems reported by the table summary jobs (generated by 'gentbl1E' from COBOL copybooks).
After examining the 'custmas0x' table summaries on the last few pages '9E5' thru '9F2', you might want to see the record#s reported to have invalid fields. Note that the table summaries report the 1st record# with each invalid data field reported & record #5 is the 1st on those reports.
uvhdcob dat1/custmas0x maps/custmas0x a <-- run uvhdcob on datafile & copybook
=====================================
--> displays 1st record & prompts for command
- 1st record display omitted here
--> 5 <-- enter 5 to see 1st record reported to have errors --> v99 <-- OR use verify command to search for records with invalid fields
rec# 5 occurs bgn end typ<------ data (hex if typ=p/b) -----> 001 cm-num 0 5 n 1.5264 F1EEF5F2F6F4 <-BadN 002 cm-delete 6 9 D 003 cm-name 10 34 MAGRATH SUPPLIES LTD. 004 cm-adrs 35 59 1939 KIRSCHNER ROAD 005 cm-city 60 75 K.LOWNA D2EED3D6E6D5C1404 <-BadC 006 filler001 76 76 007 cm-prov 77 78 BC 008 filler002 79 79 009 cm-postal 80 89 V1Y4N7 010 cm-phone 90 101 250-843-2987 011 cm-contact 102 119 012 cm-thisyr-sales 012 120 124pns 00123456EE <-BadP 013 cm-lastyr-sales 012 180 184pns 000000000C 014 filler003 240 255 C 19970131 rec#=5 rcount=32 rsize=256 fsize=8192 fptr=1024 k1=HelpPrompts,?=HelpScreens -->
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#1. uvcopyx cobmap1 cpys maps uop=q0i7p0
====================================
- convert all COBOL copybooks to layouts
#2. uvcopyx gentbl1E maps pft1 uop=q0i7f0m15v6
==========================================
- convert all cobmaps to table analysis/verify jobs
- options 'uop=q0i7f0m15v6'
q0i7 - inhibits prompts (desired when generating multiple jobs)
f0 - Fixed Length files (as specified in copybook/map
m15 = max total lines, v6 = max invalid lines
#3. vi pft1/* <-- inspect all generated jobs, modify for multi R/T's
========= (no changes required for custmas0x or sales0)
#4. uvcopyxx 'pft1/*' <-- execute all jobs (I/O files coded in jobs)
================= - input files in dat1/... output reports in tbls/...
#5. vi tbls/* <-- inspect all table analysis/verify reports
=========
#5a. uvlpd12 tbls <-- print all table analysis/verify reports
============ - 'uvlpd12' prints all files in subdir
- vs 'uvlp12' (used earlier) to print 1 file
| Note |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#0a. mkdir ctl
#0b. mkdir pft2
#0. vi ctl/ctlfile54 <-- create control file for uvdata52
================ - to relate copybook-names to data-file-names
- see sample below
ar.customer.master cpy=custmas typ=RSF rcs=00256
ar.sales.detail cpy=sales typ=RSF rcs=00064
ap.vendor.master cpy=vendormas typ=RDWz4 rcs=08192
| Note |
|
#1. uvcopyx cobmap1 cpys maps uop=q0i7p0
====================================
- convert all COBOL copybooks to layouts
#2. uvcopyx gentbl1E maps pft1 uop=q0i7f0m15v6
==========================================
- convert all cobmaps to table analysis/verify jobs
#3. uvcopy uvdata52,fili1=ctl/ctlfile54,fild2=pft1,fild3=pft2
=========================================================
- insert datafilenames (vs copybooknames) while copying pft1 to pft2/...
- also renames output jobs same as datafilenames (not copybooknames)
#4. vi pft2/* <-- inspect all generated jobs, modify for multi R/T's
========= (no changes required for custmas0x or sales0)
#5. uvcopyxx 'pft2/*' <-- execute all jobs (I/O datafilenames coded in jobs)
================= - input files in dat1/... output reports in tbls/...
#6. vi tbls/* <-- inspect all table analysis/verify reports
=========
#6a. uvlpd12 tbls <-- print all table analysis/verify reports
============ - 'uvlpd12' prints all files in subdir
- vs 'uvlp12' (used earlier) to print 1 file
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page