TABLEjobs are general purpose uvcopy jobs to read various file types, building tables of counts & values to be accumulated by a specified argument field, and dumped to a report file at EOF (for viewing &/or printing).
TABLEjobs should satisfy most of your need to quickly read through files, accumulating 1 or 2 field values by any 1 argument. There is no need to write or understand any uvcopy code.
| A1. | Introduction - several versions of the TABLEjob utilities are provided |
| (table1, table2, table3) | |
| - to allowing for different file types (delimited, fixed, text/keywords) |
| A2. | Setup to run test/demos in your own home directory (vs /home/uvadm) |
| B1. | table1 - for delimited text files |
| - specify argument & acum fields by field# | |
| - example: table summary of regular pay & overtime by employee name |
| C1. | table2 - for fixed fields in fixed records or text records with fixed |
| fields preceding a variable trailing portion | |
| - specify table arguments & acum fields by displacement & length | |
| - example1: table summary of sales qty & amount by product# | |
| - example2: table summary of customers counted by state or province |
| D1. | table3 - for text files with keywords to identify following words to |
| be tabled & summarized | |
| - useful for structured files such as programs, JCL, etc | |
| - In COBOL programs we can summarize copybooks (preceded with ' copy ') | |
| - In JCL we can summarize programs called (preceded with EXEC) |
| E1. | table3d - for entire directories of text files (such as programs or JCL) |
| - similar to table3, but analyzes all files in the directory |
| F1. | table3x - reformats table summaries from table3d into cross-references |
| - The COBOL copybook cross-reference is used as an example. |
Goto: Begin this document , End this document , UVSI Home-Page
If your requirements exceed the limitations of the general purpose jobs, you can custom write your own uvcopy job. Several examples are given and it is easy to copy, rename,& modify these to satisfy your unique requirements.
| G1. | sltbl1 - sample/demo uvcopy job to illustrate custom written table summaries |
| - creates same report as the table1 EX#1 above, but with several | |
| extensions allowed by special purpose jobs. |
| H1. | sltbl2 - similar to sltbl1 but summarizes the counts & amounts by |
| Salesman# & Product# (vs product# only). | |
| - illustrates combining multiple fields for the argument |
| I1. | sltbl3 - illustrates multiple tables in 1 run |
| - accumulates quantity & amount in 3 different tables | |
| by 3 different arguments (customer#, salesman#,& product#). |
| J1. | statlogin1 - table summary of user logins by month & userid |
| - based on /var/log/messages, sysadmins can use this job | |
| - this job applies to all Unix/Linux systems | |
| since the input is a system file present on all systems | |
| - this job table summarizes just 1 item (login userids) | |
| - this example shows you how to table summarize other items |
These jobs demonstrates the power & conciseness of uvcopy. These jobs require as few as 12 lines of interpretive code. It would take thousands of lines of COBOL code to create similar table analysis summaries.
Goto: Begin this document , End this document , UVSI Home-Page
Several versions of TABLEjobs are provided to handle the various file types (Delimited, Fixed Field, Text files with keywords). Here are just a few records from the beginning of each file type accommodated.
emp#|employee name|job |regpay |overtime =========================================== 11911|Thomas Watson|job101|1,550.00|108.50| 21945|Presper Eckert|job200|1000.00|100.00| 31975|Steve Jobs|job303|124.95|8.75|
1 2 3 4 5 6
0123456789012345678901234567890123456789012345678901234567890123
cust# slsm# date invoice# product# qty price amount
130140 21 940802 IN111001 HAM001 00020+ 0000001 00000020+
130140 21 940802 IN111001 SCR012 00021+ 0001001 00021021+
139923 35 950802 IN111002 CHR001 00022+ 0002001 00044022+
For our table3 demos, we will use the COBOL programs in /home/uvadm/mvstest/testlibs/cbl0/... but table3 could be used with various program languages or any structured text file with keywords to identify a following target word to be table summarized & cross-referenced.
For COBOL copybook cross-references, our keywords is 'COPY' and we will then use the following word (copybook name) as our table argument.
000037 01 CUSREC1 COPY CUSMREC. 000037 01 CUSREC2 COPY CUSPREC. 000038 01 CUSLINE COPY CUSLREC.
Note that 'table3' (which processes only 1 file/program) is of limited use. 'table3d' is much more useful because it processes all files in a directory, which is what we need to create meaningful cross-references.
Goto: Begin this document , End this document , UVSI Home-Page
If you are the only user of the Vancouver Utilities, you could run these test/demos in the /home/uvadm directory while logged in as uvadm. See install.htm re setting up uvadm (Vancouver Utility site Administrator)
For multi-user sites it is obviously better for each user to run these test/demos in his or her own home directory. The user/programmer must have setup his/her .profile as described on page A3 of install.htm#A3 This is easy since you can simply copy a supplied file (/home/uvadm/env/profile1) onto the end of your existing .profile.
For multi-user sites, each user/programmer should copy the required test files from /home/uvadm to his/her own home directory. We will setup subdirs with the same names as in /home/uvadm so the operating procedures will be correct whether you run from /home /uvadm or your own home directory.
#1. cd $HOME - change to your home directory
#2. mkdir dat1 tf cbl0 - setup subdirs to receive test files
#3a. cp /home/uvadm/dat1/custmas2 dat1 #3b. cp /home/uvadm/dat1/sales3 dat1 <-- used for most test/demos #3c. cp /home/uvadm/tf/timecards2 tf #3d. cp /home/uvadm/cob1/* cbl0 <-- about 40 very small demo programs
#6. mkdir tmp - setup a 'tmp' subdir if not already present
========= to receive the outputs of the various test/demos
\rm tmp/* - OR remove all files from existing tmp subdir
When you run these tests, please stay at your home directory level (or uvadm). Don't change in & out of the sub-directories to check I/O files. Most test/demos copy files from 'dat1' or 'tf' subdirs to the 'tmp' subdir and you need to be back at the home directory level for the next test. Note that the use of a 'tmp' subdir keeps your home directory clean & short. (subdirs only, no files in your home/working directory). You can remove all temporary files easily anytime (\rm tmp/*).
Goto: Begin this document , End this document , UVSI Home-Page
emp#|employee name|job |regpay |overtime =========================================== 11911|Thomas Watson|job101|1,550.00|108.50| <-- 1st 3 of 12 records 21945|Presper Eckert|job200|1000.00|100.00| 31975|Steve Jobs|job303|124.95|8.75|
uvcopy table1,fili1=infile,filo1=outreport,uop=a?b?c?f?,arg1=fields,arg2=header
===============================================================================
- command format (may specify all filenames, options,& fieldnames)
uvcopy table1,fili1=tf/timecards2,filo1=tmp/table1.tbl,uop=a2b4c5
=================================================================
- sample with I/O filenames & options on command line (the Hard Way)
uvcopy table1 <-- Try this now (the Easy Way)
============= - filenames default as shown above
- you will be prompted for options & fieldnames
uop--> a2b4c5f2 <-- ENTER Options for demo with tf/timecards2 file
========
tf/timecards2 = default fili01 accept/reenter--> <--ENTER null to accept dflt
====
empname;regpay;overtime <-- ENTER this for demo FieldName Headings
=======================
Payroll Summary by Name <-- ENTER this for demo Report Title
=======================
view/print? --> type <-- ENTER to display report (shown below)
====
table1 2003/04/24_11:00:53 Payroll Summary by Employee Name tbl#001 pg#001 -argument- -acum#1- % -acum#2- % -acum##3- % -acum#4- %
line# count % employee name regular overtime
1 3 25 Eckert, Presper 2,200.00 13 200.00 31
2 2 16 Gates, Bill 7,500.00- 46- 675.00- 104-
3 1 8 Jobs, Steve 124.95 8.75 1
4 4 33 Watson, Thomas 11,650.00 71 308.50 47
5 2 16 Wosniak, Steve 9,755.00 60 802.00 124
12*100 *TOTAL* 16,229.95 *100 644.25 *100
Please see the rerun on the next page to better illustrate the options --->
Goto: Begin this document , End this document , UVSI Home-Page
This is a rerun of the table1 demo (shown on the previous page) to better illustrate the OPTION Explanations.
uvcopy table1,fili1=tf/timecards2,filo1=tmp/table1.tbl,uop=a2b4c5
=================================================================
- sample with I/O filenames & options on command line (the Hard Way)
uvcopy table1 <-- Try this now (the Easy Way) ============= - filenames default as shown above
uop=a0b0c0d0e0f1 - option defaults
uop=a0 - argument field# (must specify non-zero)
uop= b0 - 1st field to be accumulated (if non-zero)
uop= c0 - 2nd field to be accumulated (if non-zero)
uop= d0 - 3rd field to be accumulated (if non-zero)
uop= e0 - 4th field to be accumulated (if non-zero)
uop= f# - table format (f1-f12), see uvcopy.doc "tbf"
uop= f1 - 16 byte arg, acum edits all qty (commas, no decimal)
uop= f2 - 16 byte arg, acum edits all $ (commas & decimal pt)
uop= f3 - 16 byte arg, acum #1 qty, acum #2-#6 $
uop= f4-6 - 32 byte arg, edit f4,f5,f6 same as f1,f2,f3
uop= f7-9 - 48 byte arg, edit f7,f8,f9 same as f1,f2,f3
uop= note - may omit options b,c,d,e for record counts only
uop=a2b4c5f2 <-- ENTER this for demo with tf/timecards2 file
========
empname;regpay;overtime <-- ENTER this for demo field headings
=======================
User OPtion (uop) defaults = q1a0b0c0d0e0f1
null to accept or re-specify (1 or more) ----> a2b4c5f2 <-- Enter Options
tf/timecards2 = default file, accept/reenter--> <-- null to accept
enter table fieldnames separated by semicolons
argument;acum1;acum2;acum3;acum4(max) --------> empname;regpay;overtime <--
enter table heading (optional) ---------------> Payroll Summary by EmpName <--
enter to command to view/print ---------------> type <-- to display report
table1 2003/05/08_10:09:19 Payroll Summary by Employee Name tbl#001 pg#001 -argument- -acum#1- % -acum#2- % -acum#3- % -acum#4- % line# count % empname regular overtime
1 3 25 Eckert, Presper 2,200.00 13 200.00 31
2 2 16 Gates, Bill 7,500.00- 46- 675.00- 104-
3 1 8 Jobs, Steve 124.95 0 8.75 1
4 4 33 Watson, Thomas 11,650.00 71 308.50 47
5 2 16 Wosniak, Steve 9,755.00 60 802.00 124
12*100 *TOTAL* 16,229.95 *100 644.25 *100
Goto: Begin this document , End this document , UVSI Home-Page
'table2' solicits the filename & options which define displacement & length of the table argument,& 1 or 2 fields to be accumulated. table2 then reads through the file & displays or prints the accumulated totals sorted & summarized to one line per unique key field value.
1 2 3 4 5 6
0123456789012345678901234567890123456789012345678901234567890123
cust# slsm# date invoice# product# qty price amount
130140 21 940802 IN111001 HAM001 00020+ 0000001 00000020+
130140 21 940802 IN111001 SCR012 00021+ 0001001 00021021+
139923 35 950802 IN111002 CHR001 00022+ 0002001 00044022+
uvcopy table2,fili1=dat1/sales3,uop=a30b6c38d6e53f9g3r64,filo1=tmp/sales3.tbl =============================================================================
uvcopy table2 <-- Try this Now
============= - You only need to enter/remember this much
- you will be prompted for filename & options
- filenames default as shown above (for this demo)
uop--> a30b6c38d6e53f9g3r64 <-- ENTER options for demo with dat1/sales3 file
====================
dat1/sales3 = default fili01 accept/reenter--> <--ENTER null to accept dflt
table2 2003/04/26_20:15:34 Sales Summary by Product# tbl#001 pg#001 -argument- -acum#1- % -acum#2- % line# count % product quantity
1 2 10 BBQ001 12 12 1,420.12 31
2 1 5 CHR001 22 22 440.22 9
3 5 25 HAM001 41 41 1,390.41 30
4 2 10 HAX129 10 10 600.10 13
5 2 10 SAW051 24- 24- 560.24- 12-
6 1 5 SCR012 21 21 210.21 4
7 4 20 TAB013 17 17 29.83-
8 3 15 WHIP75 1 1 1,030.01 22
20*100 *TOTAL* 100 *100 4,501.00 *100
Please see the rerun on the next page to better illustrate the options --->
Goto: Begin this document , End this document , UVSI Home-Page
uvcopy table2,fili1=dat1/sales3,uop=a10b2c38d6e53f9g3r64,filo1=tmp/sales3.tbl ============================================================================= - May specify all filenames & options on the command line (as above)
uvcopy table2,fili1=dat1/sales3 - may specify input filename only
=============================== - will be prompted for options
- output file same as input but in tmp subdir
uvcopy table2 - minimum entry, will be prompted for file & options
=============
uop=a0b0c0d0e0f0g1p0r0 - option defaults
uop=a0 - argument displacement (0 rel)
uop= b0 - argument length
uop= note - may omit options c,d,e,f,p for record counts only
uop= c0 - acum#1 displacement (0 rel)
uop= d0 - acum#1 length
uop= e0 - acum#2 displacement (0 rel)
uop= f0 - acum#2 length
uop= g1 - edit all acums as qty (commas, no decimal)
uop= g2 - edit all acums as $ (commas & decimal point)
uop= g3 - edit acum#1 qty, acums#2-#6 $amounts
uop= g__ - table formats g1-g12 see uvcopy.doc tbf f1-f12
uop= p0 - acum-fields unpacked, "p1" packed, "p2" binary
uop= r8192 - input record size (max 8192)
uop= note - if file type LST specify r > largest possible
uop=a30b6c38d6e53f9g3r64 <-- ENTER this for demo with dat1/sales3 file
====================
product#;quantity;amount <-- ENTER this for demo Field Headings
========================
Summary of Qty & Amt by Product# <-- ENTER this for demo Report Heading
================================
<-- ENTER null at filename prompt to accept demo input
Notes re: input File TYPE prompts
RSF (default) <-- Record Seqntl Fixed (recsize option r must be correct)
LST <-- Line Seqntl Terminated (specify r > largest possible)
Option defaults = q1a0b0c0d0e0f0g1p0r0
null accept or re-specify (1 or more) --> a30b6c38d6e53f9g3r64 <--ENTER for demo
====================
dat1/sales3 = default fili01 accept/reenter--> <--ENTER null to accept dflt
RSF = typ default accept or enter new typ --> <--ENTER null to accept dflt
enter table hdr fieldnames separated by semicolons
--> argument;acum1;acum2 --> product#;quantity;$amount <-- ENTER fieldnames
=========================
enter Report Heading (optional)
--> Sales Summary by Product# <-- ENTER Report Heading
=========================
enter to command to view/print report --> more <-- to display report
====
<-- See the sample report listed on the previous page
Goto: Begin this document , End this document , UVSI Home-Page
table2 is primarily intended for Fixed Field files in Fixed Length records, but has an option for TEXT files ie - variable length terminated by LineFeed. When you run table2 you are prompted to change the default type 'RSF' (Record Sequential Fixed) to 'LST' (Line Sequential Terminated).
The 'LST' option can be used with text files that are variable length (ragged right margin), but have FIXED fields on the left side of the record. For example consider the dat1/sales8 file, which ends with a variable length customer name.
1 2 3 4 5 6 7
01234567890123456789012345678901234567890123456789012345678901234567890123456789
cust# slsm# date invoice# product# qty customer name
130140 21 940802 IN111001 HAM001 00020+ EVERGREEN MOTORS LTD.
150825 44 960804 IN1122 HAX129 00026+ RIGGERS INDUSTRIAL
315512 44 000805 IN2251 HAM001 00013- PARTS PLUS
On mainframes records these would have been blank filled up to a fixed record size (with no LineFeed), but in this case they are terminated with a LineFeed wherever the customer name ends. Use 'uvhd' if you want to see the LineFeeds.
uvhd dat1/sales8 th2 <-- uvhd with 'text option & 'hex' display option ==================== (see uvhd.htm for complete details)
10 20 30 40 50 60
r# 16 0123456789012345678901234567890123456789012345678901234567890123
922 315512 44 000805 IN2251 SAW051 00014- PARTS PLUS.
33333322223323333332443333222254533322333332254555254550
315512000044000080509E225100003170510000014D00124300C53A
The uvhd display (vertical hexadecimal) above shows only record #16 & you can see the LineFeed x'0A' in byte 55 at the end of the record.
Now rerun the table2 demo, specifying dat1/sales8 as the input file, and when prompted enter file type 'LST'. Please see the preceding page for all the option displays & prompts, but here is the essential command & reply:
uvcopy table2,fili1=dat1/sales8,uop=a30b6c38d6r80 <-note 'r80' largest recsize =================================================
RSF = typ default fili01 accept or enter new typ -----> LST <-- for TEXT files
When you use the 'LST' text file option, you must specify the record size option 'r' larger than the largest record possible in the file. We used 'r80' here, but it may be a good idea to always use something like 'r256' to make sure you never cause record splitting which happens if larger records present.
We wont show the results here, but you can run it & see for yourself. Now that you understand the rules, please try 'table2' on some of your own files.
Goto: Begin this document , End this document , UVSI Home-Page
'table2' is very handy when you want to count records by province/state codes, zip codes, etc. For example consider the following file:
filename = /home/uvadm/dat1/custmas2 record size = 128 provice code= 77(2)
cust# customer-name address city prov ================================================================================ 130140 EVERGREEN MOTORS LTD. 1815 BOWEN ROAD NANAIMO BC 132588 GEECOE GENERATOR SERVICESUNIT 170 - 2851 SIMPSON RICHMOND BC 237286 WEBER, TOM BOX 5503 FORT MCMURRAY AL
uvcopy table2,fili1=dat1/custmas2,uop=a77b2r128 =============================================== - a77b2r128 defines table argument dsplcmnt(a77), length(b2), & recsize(r128)
uop=a0b0c0d0e0f0g1p0r0 - option defaults uop=a0 - argument displacement (0 rel) uop= b0 - argument length uop= c0 - acum#1 displacement (0 rel) uop= d0 - acum#1 length uop= e0 - acum#2 displacement (0 rel) uop= f0 - acum#2 length uop= g1 - edit all acums as qty (commas, no decimal) uop= g2 - edit all acums as $ (commas & decimal point) uop= g3 - edit acum#1 qty, acums#2 $ uop= g__ - table formats g1-g12 see uvcopy.doc tbf f1-f12 uop= p0 - acum-fields unpacked, "p1" packed, "p2" binary uop= r8192 - input record size (max 8192) User OPtion (uop) defaults = q1a0b0c0d0e0f0g1p0r0 null to accept or re-specify (1 or more) --> a77b2r128 <-- omit if on cmd line dat1/sales3 = default fili01 accept ---> dat1/custmas2 <-- omit if entered above enter table heading --> Record Counts by Province <-- enter table heading
enter to command to view/print (vi, uvlp12) tmp/custmas2.tbl --> more <-- to display report
table2 2003/04/26_15:10:47 Record counts by State/Province tbl#001 pg#001 -argument- -acum#1- % -acum#2- % line# count % state/prov count
1 3 9 AB 3 9 3 9
2 11 34 AL 11 34 11 34
3 17 53 BC 17 53 17 53
4 1 3 NW 1 3 1 3
32*100 *TOTAL* 32 *100 32 *100
Goto: Begin this document , End this document , UVSI Home-Page
'table3' solicits a keyword preceding the target-word to be summarized. This job will be useful to you if you work with computer languages such as COBOL or JCL or any file which uses keywords to identify following items. For example, we can extract COBOL copybook names from COBOL programs by using the preceding keyword 'copy'. For our demo, our input file will be cbl0/car200.cbl. Here are the copy statements extracted with the 'grep' utility.
01 salerec. copy "saledtl.cpy".
01 custrec. copy "custmas.cpy".
01 detail-line. copy "sdline.cpy".
01 total-line. copy "stline.cpy".
Note that grep picked up any line with 'COPY'. We don't want the COBOL comment lines (with '*' in column 7). table3 has option 'c1' to bypass these lines.
First we will demo 'table3' which analyzes only 1 file at a time. Then we will demo 'table3d' which analyzes an entire directory of programs, and which will be far more useful for COBOL copybook cross-references.
uvcopy table3,fili1=tf/car200.cbl,filo1=tmp/car200.tbl,uop=c1l1w1 =================================================================
uvcopy table3 <-- Try this Now
============= - you will be prompted for filename & options
- filenames default as shown above (for this demo)
uop--> c1l1w1 <-- ENTER options for demo with cbl0/car200.cbl file
Job: table3 File: tf/car200.cbl Keyword(s): copy Qual1: Qual2: Blanked: '" Userops: q1c0f0l0k0m0p1w1c1w1 table3 2007/09/09_11:16:12 Counts by Targetword following specified Keyword tbl#001 pg#001 -argument- line# count % target-word
1 1 16 custmas.cpy
2 1 16 saledtl.cpy
3 1 16 sdline.cpy
4 1 16 stline.cpy
5 1 16 unixproc1.cpy
6 1 16 unixwork1.cpy
6*100 *TOTAL*
See the OPTION explanations on the next page --->
Goto: Begin this document , End this document , UVSI Home-Page
uvcopy table3,fili1=tf/car200.cbl,filo1=tmp/car200.tbl,uop=c1l1w1
=================================================================
- may specify all filenames & options on the command line (as above)
uvcopy table3,fili1=cbl0/car200.cbl - may specify input filename only
=================================== - will be prompted for options
- output in tmp subdir with .tbl extension
uvcopy table3 - minimum entry, will be prompted for file & options =============
uop=q1c0f0l0k0m0p1w1 - option defaults
c1 - bypass comments in COBOL programs (* column 7)
c2 - bypass comments in shell scripts (# column 1)
f0 - do not prepend/append table entry with filename
f1 - prepend table argument with filename
f2 - append table argument with filename
k1 - no blank before keyword (k0 ensures word sep)
k2 - no blank after keyword (allow keyword prefix)
l0 - translate lower case before matching patterns
m2 - look for multiple keywords on 1 line
p1 - qualifier#1 present & qualifier#2 absent
p2 - qualifier#1 present & qualifier#2 present
p4 - qualifier#1 absent & qualifier#2 absent
w1 - target word is 1st word following keyword
w2 - target word is 2nd word following keyword
-------- examples - options, keywords,& spcl chars to be blanked ------
COBOL--> c1f2 <-- options copybook cross-ref
-------> copy <-- keyword copybook xref
JCL/ksh> c2w2 <-- options Korn shell script top-node summary
-------> exportfile:exportgen0:exportgen1 <-- multiple keywords
-------> exportfile DDNAME node1/node2.node3.node4
-------> &\,$=/-(.`);*|" <--- on arg4/prompt, enter spcl chars to blank
-------> abcdefhlpqrsAPQ <--- OR enter these alphas to represent above
User OPtion (uop) defaults = q1c0f0l0k0m0p1w1
null to accept or re-specify (1 or more) --> c1l1w1 <-- or null if on cmd line
enter search pattern (max 64 bytes) --------> copy <-- enter KEYWORD
enter qualifier#1 (null to disable) --> <-- null for this demo
enter qualifier#2 (null to disable) --> <-- null for this demo
display/edit/print/none ? (more/vi/lp/null)--> more <-- display on screen
Goto: Begin this document , End this document , UVSI Home-Page
Job: table3 File: tf/car200.cbl Keyword(s): copy Qual1: Qual2: Blanked: '" Userops: q1c0f0l0k0m0p1w1c1w1 table3 2007/09/09_11:16:12 Counts by Targetword following specified Keyword tbl#001 pg#001 -argument- line# count % target-word
1 1 16 custmas.cpy
2 1 16 saledtl.cpy
3 1 16 sdline.cpy
4 1 16 stline.cpy
5 1 16 unixproc1.cpy
6 1 16 unixwork1.cpy
6*100 *TOTAL*
Goto: Begin this document , End this document , UVSI Home-Page
'table3d' solicits a keyword preceding the target-word to be summarized. This job will be useful to you if you work with computer languages such as COBOL or JCL or any file which uses keywords to identify following items. For example, we can extract COBOL copybook names from COBOL programs by using the preceding keyword 'copy'. For our demo, our input will be: /home/uvadm/mvstest/testlibs/cbl0/... a directory of about 40 COBOL programs, but not all have copybooks.
uvcopy table3d,fild1=cbl0,filo1=tmp/cbl0.tbl,arg1=copy,uop=c1l1w1 =================================================================
uvcopy table3d <-- Try this Now
============== - you will be prompted for filename & options
- filenames default as shown above (for this demo)
uop--> c1l1w1 <-- ENTER options for demo with cbl0/car200.cbl file
Job: table3d Dir: cbl0 Keyword(s): copy Qual1: Qual2: Blanked: '" Userops: q1c0f0l0k0m0p1w1c1l1w1 table3d 2007/09/09_11:23:37 Counts by Targetword following specified Keyword tbl#001 pg#001 -argument- line# count % target-word
1 1 9 custmas
2 5 45 custmas.cpy
3 1 9 paymas.cpy
4 2 18 saledtl.cpy
5 1 9 sdline.cpy
6 1 9 stline.cpy
11*100 *TOTAL*
See the OPTION explanations on the next page --->
Goto: Begin this document , End this document , UVSI Home-Page
uvcopy table3d,fild1=cbl0,filo1=tmp/cbl0.tbl,arg1=copy,uop=c1l1w1
=================================================================
- may specify Directory, Filename & Options on the command line
- Note that 'fild1' identifies a Directory (vs fili1/filo1 for I/O files)
uvcopy table3d,fild1=cbl0 - may specify input directory on command line
========================= - will be prompted for options
- output in tmp subdir with .tbl extension
uvcopy table3d - minimum entry, will be prompted for directory ============== & options
uop=q1c0f0l0k0m0p1w1 - option defaults
c1 - bypass comments in COBOL programs (* column 7)
c2 - bypass comments in shell scripts (# column 1)
f0 - do not prepend/append table entry with filename
f1 - prepend table argument with filename
f2 - append table argument with filename
k1 - no blank before keyword (k0 ensures word sep)
k2 - no blank after keyword (allow keyword prefix)
l0 - translate lower case before matching patterns
m2 - look for multiple keywords on 1 line
p1 - qualifier#1 present & qualifier#2 absent
p2 - qualifier#1 present & qualifier#2 present
p4 - qualifier#1 absent & qualifier#2 absent
w1 - target word is 1st word following keyword
w2 - target word is 2nd word following keyword
-------- examples - options, keywords,& spcl chars to be blanked ------
COBOL--> c1f2 <-- options copybook cross-ref
-------> copy <-- keyword copybook xref
JCL/ksh> c2w2 <-- options Korn shell script top-node summary
-------> exportfile:exportgen0:exportgen1 <-- multiple keywords
-------> exportfile DDNAME node1/node2.node3.node4
-------> &\,$=/-(.`);*|" <--- on arg4/prompt, enter spcl chars to blank
-------> abcdefhlpqrsAPQ <--- OR enter these alphas to represent above
User OPtion (uop) defaults = q1c0f0l0k0m0p1w1c1l1w1
null to accept or re-specify (1 or more) ---> c1w1f2 <-- enter options
enter search word(s), (":" sep if multi) ---> copy <-- enter KEYWORD
enter qualifier#1 (null=disable) ----->
enter qualifier#2 (null=disable) ----->
EOJ, OutFile written to: tmp/cbl0_copy
enter command: vi,cat,more,uvlp12(or null) --> more <-- display on screen
Goto: Begin this document , End this document , UVSI Home-Page
We will repeat the table3d demo from the previous page with 1 change. We will add option 'f2' to append the filename onto the table argument In this case, to create a COBOL copybook name cross-reference.
uvcopy table3d,fild1=cbl0,filo1=tmp/cbl0.tbl,arg1=copy,uop=c1l1w1f2 ===================================================================
Job: table3d Dir: cbl0 Keyword(s): copy
Qual1: Qual2: Blanked: '" Userops: q1c0f0l0k0m0p1w1c1l1w1f2
table3d 2007/09/09_11:53:47 Counts by Targetword following specified Keyword
tbl#001 pg#001 -argument-
line# count % target-word
1 1 9 custmas CAR100.cbl
2 1 9 custmas.cpy CAR120.cbl
3 1 9 custmas.cpy CAR130.cbl
4 1 9 custmas.cpy CAR140.cbl
5 1 9 custmas.cpy CAR150.cbl
6 1 9 custmas.cpy CAR200.cbl
7 1 9 paymas.cpy CPY100.cbl
8 1 9 saledtl.cpy CAR150.cbl
9 1 9 saledtl.cpy CAR200.cbl
10 1 9 sdline.cpy CAR200.cbl
11 1 9 stline.cpy CAR200.cbl
11*100 *TOTAL*
Alternatively we could use option 'f1' to insert the filename prior to the table argument (instead of following). In our COBOL example this would show us a list of the copybooks used within each program.
Job: table3d Dir: cbl0 Keyword(s): copy
Qual1: Qual2: Blanked: '" Userops: q1c0f0l0k0m0p1w1c1l1w1f1
table3d 2007/09/09_11:56:58 Counts by Targetword following specified Keyword
tbl#001 pg#001 -argument-
line# count % target-word
1 1 9 CAR100.cbl custmas
2 1 9 CAR120.cbl custmas.cpy
3 1 9 CAR130.cbl custmas.cpy
4 1 9 CAR140.cbl custmas.cpy
5 1 9 CAR150.cbl custmas.cpy
6 1 9 CAR150.cbl saledtl.cpy
7 1 9 CAR200.cbl custmas.cpy
8 1 9 CAR200.cbl saledtl.cpy
9 1 9 CAR200.cbl sdline.cpy
10 1 9 CAR200.cbl stline.cpy
11 1 9 CPY100.cbl paymas.cpy
11*100 *TOTAL*
Goto: Begin this document , End this document , UVSI Home-Page
table3x will reformat the table summary reports from table3d (see prior page) into cross-reference reports with the table argument on the left and multiple file references spread across the page.
We will demonstrate this using the output of table3d as input to table3x. On the previous page demo, table3d with option 'f2' created a file of COBOL copybook names appended with the file/program name. table3x will now be used to show the copybook name only once on the left & spread the multiple filename references across the page (which shortens the report).
1. uvcopy table3d,fild1=cbl0,filo1=tmp/cbl0.tbl,arg1=copy,uop=c1w1f2
=================================================================
- preceding job to create input for this job (table3x)
2. uvcopy table3x,fili1=tmp/cbl0.tbl,filo1=tmp/cbl0.rpt,uop=a16b14c4s1
===================================================================
- run this job to spread the filenames across the page for each tbl arg
1 1 5 cmrecord cobol4.cob 2 1 5 cmrecord cobol6.cob 3 1 5 cobcopy6 cobol6.cob 4 1 5 cobcopy8 cobol8.cob 5 1 5 copysel9 cobol9.cob 6 1 5 cuslrec cobol9.cob 7 2 10 cusmrec cobol7.cob 8 2 10 cusmrec cobol8.cob 9 2 10 cusmrec cobol9.cob 10 1 5 cusprec cobol7.cob 11 1 5 cusprec cobol8.cob 12 1 5 cusqrec cobol7.cob 13 1 5 warmasr cobolw.cob 14 3 15 warmasr cobolx.cob
Job: table3x create Cross-Reference from table analysis Page# 1 InputFile: tmp/table3d.tbl Options=q1a16b14c4s2t0 2003/04/26_11:55:23 =======================================================================
cmrecord_2 cobol4.cob cobol6.cob cobcopy6 cobol6.cob cobcopy8 cobol8.cob copysel9 cobol9.cob cuslrec cobol9.cob cusmrec_6 cobol7.cob_2 cobol8.cob_2 cobol9.cob_2 cusprec_2 cobol7.cob cobol8.cob cusqrec cobol7.cob warmasr_4 cobolw.cob cobolx.cob_3
Please see Options & report details on the next page --->
Goto: Begin this document , End this document , UVSI Home-Page
We will repeat the run command here to illustrate the Options available. We will redisplay the report output, but not the input which may be seen on the previous page.
2. uvcopy table3x,fili1=tmp/cbl0.tbl,filo1=tmp/cbl0.rpt,uop=a16b14c4s1
===================================================================
uop=q1a16b14c4s2t0 - option defaults
a16 - column to align references
b14 - columns for fixed word horizontal spacing
b30 - max columns, indicates variable (1 blank between)
c4 - count max words per line (c4*b14=52+a16 = 68)
- could increase for print scripts uvlp12,uvlp14,etc
s2 - space2 between keyword1 groups (s1 for space1)
t1 - translate to lower case
User OPtion (uop) defaults = q1a16b14c4s2t0
null to accept or re-specify (1 or more) --> <-- null accept default options
Job: table3x create Cross-Reference from table analysis Page# 1 InputFile: tmp/table3d.tbl Options=q1a16b14c4s2t0 2003/04/26_11:55:23 ======================================================================= cmrecord_2 cobol4.cob cobol6.cob cobcopy6 cobol6.cob cobcopy8 cobol8.cob copysel9 cobol9.cob cuslrec cobol9.cob cusmrec_6 cobol7.cob_2 cobol8.cob_2 cobol9.cob_2 cusprec_2 cobol7.cob cobol8.cob cusqrec cobol7.cob warmasr_4 cobolw.cob cobolx.cob_3
The '_#'s are the total occurrences of the copybook. For example using the last line above, the '_4' of 'warmasr_4' indicates 4 total references to copybook warmasr. cobolw.cob had only 1 reference (no _# appended if only 1). cobolx.cob had 3 references indicated by the '_3' suffix of cobolx.cob_3.
Try this job if you are a programmer. It does not have to be COBOL. These are general purpose jobs & most languages have some keywords that can be used to target some following item to be cross-referenced.
Note that there are many more cross-references documented in XREFjobs.htm (many based on scripts converted from mainframe JCL).
Goto: Begin this document , End this document , UVSI Home-Page
If your requirements exceed the limitations of the above general purpose jobs, you can custom write your own uvcopy job. As an example please see the 'sltbl1' UV demo job & you can run it as follows:
uvcopy sltbl1,fili1=dat1/sales3 <-- run the demo job ===============================
1 2 3 4 5 6
0123456789012345678901234567890123456789012345678901234567890123
cust# slsm# date invoice# product# qty price amount
130140 21 940802 IN111001 HAM001 00020+ 0000001 00000020+
130140 21 940802 IN111001 SCR012 00021+ 0001001 00021021+
130140 21 940802 IN111001 CHR001 00022+ 0002001 00044022+
sltbl1 2003/04/26_16:07:59 sales analysis by product# of quantity & amount tbl#001 pg#001 -argument- -acum#1- % -acum#2- % line# count % product# sales qty sales amt
1 2 10 BBQ001 12 12 1,420.12 31
2 1 5 CHR001 22 22 440.22 9
3 5 25 HAM001 41 41 1,390.41 30
4 2 10 HAX129 10 10 600.10 13
5 2 10 SAW051 24- 24- 560.24- 12-
6 1 5 SCR012 21 21 210.21 4
7 4 20 TAB013 17 17 29.83-
8 3 15 WHIP75 1 1 1,030.01 22
20*100 *TOTAL* 100 *100 4,501.00 *100
<-- compare this custom report to the report on page 'C1' (created by the general purpose table summary job 'table2')
See custom written uvcopy job 'sltbl1' listed on next page --->
Before you look, guess how many instructions were required to create the report shown above.
Goto: Begin this document , End this document , UVSI Home-Page
fili1=?dat1/sales3,typ=RSF,rcs=64
filo1=tmp/$jobname.rpt,typ=LSTt,rcs=80
@run
opn all
# begin loop to get & table until EOF
man20 get fili1,a0 get next record
skp> eof (cc set > at EOF)
tblt1f3 a30(6),'product#; sales qty; sales amt',a38(6),a53(9)
skp man20 return to get next record
#
# EOF - dump table to report file & display report
eof tbpt1s2 filo1,'sales analysis by product# of quantity & amount'
cls all
sysv1 'cat $filo1'
eoj
Only 12 uvcopy instructions were required to create the report shown on the previous page. All uvcopy instructions are documented in uvcopy3.htm. In this case, the 2 most relevant (& powerful) instructions are 'tbl' & 'tbp'.
tbl - build table in memory
===
tblt1f3 - option 't1' for table#1, t2 for table #2, etc
- option 'f3' for format 3 (acum1=quantity, acum2=dollars/cents)
tbp - print table (write memory table to an output file for subsequent
=== viewing or printing (with vi,cat,lp,etc)
tbpt1s2 - option 't1' to write table#1 (match table# on prior tbl)
- option 's2' to space 2 after (FormFeed is the default)
Goto: Begin this document , End this document , UVSI Home-Page
This example is a follow-on to sltbl1 which creates sales analysis by product#. But what if the sales manager wants to see the product sales for each salesman ?
This example will move the salesman# & the product# together, and use the combination as the table argument. Of course we could also combine them the other way around if you wanted to see the salesmen within the products. We will demo the salesman+product#, then you can try the reverse OK ?
uvcopy sltbl2,fili1=dat1/sales3 <-- run the demo job ===============================
sltbl2 2003/04/26_16:08:16 sales analysis by slsmn#/product# of quantity & amount tbl#001 pg#001 -argument- -acum#1- % -acum#2- % line# count % slsmn#/product# sales qty sales amt
1 1 5 21 CHR001 22 22 440.22 9
2 1 5 21 HAM001 20 20 20
3 1 5 21 SAW051 10- 10- 10-
4 1 5 21 SCR012 21 21 210.21 4
5 1 5 21 TAB013 12- 12- 240.12- 5-
6 1 5 21 WHIP75 11- 11- 110.11- 2-
7 1 5 35 BBQ001 17- 17- 1,190.17- 26-
8 3 15 35 TAB013 29 29 210.29 4
9 2 10 44 HAM001 12 12 860.12 19
10 1 5 44 HAX129 26 26 1,560.26 34
11 1 5 44 SAW051 14- 14- 560.14- 12-
12 2 10 44 WHIP75 12 12 1,140.12 25
13 1 5 65 BBQ001 29 29 2,610.29 57
14 2 10 65 HAM001 9 9 530.09 11
15 1 5 85 HAX129 16- 16- 960.16- 21-
20*100 *TOTAL* 100 *100 4,501.00 *100
Goto: Begin this document , End this document , UVSI Home-Page
<-- see report output on the previous page
# sltbl2 - sales by Salesman# & product#
fili1=?dat1/sales3,typ=RSF,rcs=64
filo1=tmp/$jobname.rpt,typ=LSTt,rcs=80
@run
opn all
# begin loop to get & table until EOF
man20 get fili1,a0 get next record
skp> eof (cc set > at EOF)
mvc b0(2),a10 move slsmn# to work area 'c'
mvc b3(6),a30 move product# beside slsmn#
tblt1f3 b0(9),'slsmn#/product#; sales qty; sales amt',a38(6),a53(9)
skp man20 return to get next record
# EOF - dump table to report file & display report
eof tbpt1s1 filo1,'sales analysis by slsmn#/product# of quantity & amount'
cls all
sysv1 'cat $filo1'
eoj
This job 'sltbl2' is the same as 'sltbl1' except for 3 lines:
mvc b0(2),a10 move slsmn# to work area 'c'
mvc b3(6),a30 move product# beside slsmn#
tblt1f3 b0(9),'slsmn#/product#; sales qty; sales amt',a38(6),a53(9)
Two 'mvc' instructions are inserted to move slsmn# & product# together in work area 'c'. Op1 of the 'tbl' instruction is modified to address the concatenated slsm# & product# in b0(9) - vs product# only at a30(6). Op2 field headings changed to 'slsmn#/product#' (vs product# only).
Now here is an exercise for you to try - copy/rename/modify sltbl2 to try the reverse (product+salesman).
#1. cp pf/demo/sltbl2 pf/demo/sltbl2a - copy & rename sltbl2 as sltbl2a
=================================
#2. vi pf/demo/sltbl2a - modify sltbl2a to table on product + salesman
==================
#3. uvcopy sltbl2a - execute your modified job
==============
Goto: Begin this document , End this document , UVSI Home-Page
You can have up to 255 tables in 1 uvcopy job. The number of table entries is limited only by the available memory. 'sltbl3' will demonstrate 3 tables in one job (customer, salesman,& product), using the dat1/sales3 demo file (already illustrated on previous pages).
uvcopy sltbl3,fili1=dat1/sales3 <-- run the demo job ===============================
sltbl3 2003/04/27_14:08:17 sales analysis by CUSTOMER# of quantity & amount
tbl#001 pg#001 -argument- -acum#1- % -acum#2- %
line# count % customer# sales qty sales amt
1 3 15 130140 63 63 650.63 14
2 2 10 139923 47 47 1,650.47 36
3 2 10 150825 51 51 2,810.51 62
4 1 5 201120 27 27 1,890.27 41
5 2 10 223240 57 57 4,850.57 107
6 3 15 308685 33- 33- 350.33- 7-
7 3 15 315512 42- 42- 1,700.42- 37-
8 1 5 400002 16- 16- 960.16- 21-
9 3 15 406082 54- 54- 4,340.54- 96-
20*100 *TOTAL* 100 *100 4,501.00 *100
sltbl3 2003/04/27_14:08:17 sales analysis by SALESMAN# of quantity & amount
tbl#002 pg#001 -argument- -acum#1- % -acum#2- %
line# count % salesman# sales qty sales amt
1 6 30 21 30 30 300.30 6
2 4 20 35 12 12 979.88- 21-
3 6 30 44 36 36 3,000.36 66
4 3 15 65 38 38 3,140.38 69
5 1 5 85 16- 16- 960.16- 21-
20*100 *TOTAL* 100 *100 4,501.00 *100
sltbl3 2003/04/27_14:08:17 sales analysis by PRODUCT# of quantity & amount
tbl#003 pg#001 -argument- -acum#1- % -acum#2- %
line# count % product# sales qty sales amt
1 2 10 BBQ001 12 12 1,420.12 31
2 1 5 CHR001 22 22 440.22 9
3 5 25 HAM001 41 41 1,390.41 30
4 2 10 HAX129 10 10 600.10 13
5 2 10 SAW051 24- 24- 560.24- 12-
6 1 5 SCR012 21 21 210.21 4
7 4 20 TAB013 17 17 29.83-
8 3 15 WHIP75 1 1 1,030.01 22
20*100 *TOTAL* 100 *100 4,501.00 *100
Please see the 'sltbl3' uvcopy code listed on the next page --->
Goto: Begin this document , End this document , UVSI Home-Page
fili1=?dat1/sales3,typ=RSF,rcs=64
filo1=?tmp/sltbl3.rpt,typ=LSTtd,rcs=80
@run
opn all
# begin loop to get & table until EOF
man20 get fili1,a0 get next record
skp> eof (cc set > at EOF)
tblt1f3 a00(6),'customer#; sales qty; sales amt',a38(6),a53(9)
tblt2f3 a10(2),'salesman#; sales qty; sales amt',a38(6),a53(9)
tblt3f3 a30(6),'product#; sales qty; sales amt',a38(6),a53(9)
skp man20 return to get next record
#
# EOF - dump tables to report file & display report
eof tbpt1s2 filo1,'sales analysis by CUSTOMER# of quantity & amount'
tbpt2s2 filo1,'sales analysis by SALESMAN# of quantity & amount'
tbpt3s2 filo1,'sales analysis by PRODUCT# of quantity & amount'
cls all
msgwy 'display report y/n ? (stored in tmp/sltbl3.rpt)'
skp! 1
sysv1 'cat $filo1' 'cat' auto changed to 'type' for WIN/DOS
eoj
<--- Please see the sample output report listed on the previous page.
'sltbl3' is a custom written uvcopy job to illustrate multiple tables in 1 run. The pre-programmed TABLEjobs (table1,2,3 previously documented) should satisfy most of your table analysis requirements, but if not you can do anything desired with a custom written uvcopy job.
Custom write when you need to:
#1. Have more than 1 table per run
#2. Accumulate more than 2 fields (tbl allows up to 6 acums + counts)
#3. Combine multiple fields for the argument
#4. Combine the tabling with other functions, such as reporting each record
(as well as dumping tables at EOF).
Goto: Begin this document , End this document , UVSI Home-Page
Here is a job that you can run on any Unix/Linux system, since the input is a system file present on all systems /var/log/messages.
This job table summarizes just 1 item (login userids) collected by the system. Using this job as an example, you should be able to write other jobs to table summarize other items that interest you form this file or other system files.
You can copy, rename, & modify for other purposes as follows:
cp /home/uvadm/demo/statlogin1 /home/appsadm/pf/statxxx ======================================================= - copy this job to appsadm/pf, rename as you wish,& modify for your purposes - please see ADMjobs.htm to setup uvadm, appsadm,& their profiles - you should create any new jobs in appsadm (not uvadm), since you would lose any changes in uvadm when a new version of Vancouver Utilities is installed.
# statlogin1 - table summary of user logins by month & userid # - based on /var/log/messages # - by Owen Townsend, UV Software, Nov 26/2007 # # uvcopy statlogin1,fili1=/var/log/messages,filo1=stats/login.rpt1 # ================================================================ # uvcopy statlogin1 <-- same & easier (files default as shown above) # ================= # #Note - above works if you have updated root's profile to run uvcopy # - if not, use following procedure: #1. mkdir stats tmp <-- make subdirs in your working directory #2. su root <-- Switch User to root & enter password #3. cp /var/log/messages tmp <-- copy messages file to tmp subdir #4. chmod 777 tmp/messages <-- change permissions on messages file #5. uvcopy statlogin1,fili1=tmp/messages,filo1=stats/login.rpt1 # =========================================================== #6. vi stats/login.rpt1 <-- view repport #7. uvlp12 stats/login.rpt1 <-- print repport # # ** sample input /var/log/messages ** # # Oct 23 06:11:51 uvsoft3 login[16341]: session opened for user laval4 by LOGIN(uid=0) # Oct 23 06:11:51 uvsoft3 -- laval4[16341]: LOGIN ON tty5 BY laval4 # Oct 23 06:13:08 uvsoft3 ftpd[16342]: FTP LOGIN FROM 192.168.0.2, uvsoft2.uvsoft.ca (laval4) # Oct 23 07:15:00 uvsoft3 login[16341]: session closed for user laval4 # Oct 23 07:15:23 uvsoft3 login[16516]: authentication failure; logname=LOGIN uid=0 euid=0 tty=tty5 ruser= rhost= user=mvstest # Oct 23 07:15:33 uvsoft3 login[16516]: session opened for user mvstest by LOGIN(uid=0) # Oct 23 07:15:33 uvsoft3 -- mvstest[16516]: LOGIN ON tty5 BY mvstest # Oct 23 21:00:42 uvsoft3 shutdown: shutting down for system halt
Goto: Begin this document , End this document , UVSI Home-Page
# ** sample output report **
#
# statlogin1 2007/11/26_21:24:24 logins by month & userid
# tbl#001 pg#001 -argument-
# line# count % mth login
# 1 13 5 Nov efunds2
# 2 13 5 Nov laval4
# 3 23 10 Nov mvstest
# 4 27 12 Nov root
# 5 26 11 Nov uvadm
# 6 26 11 Nov uvbak
# 7 8 3 Oct efunds2
# 8 7 3 Oct laval4
# 9 10 4 Oct mvstest
# 10 11 4 Oct root
# 222*100 *TOTAL*
#
# The code scans for 'session opened' & if found
# - then scans for ' user ', extracts following word (userid)
# - moves month (1st 3 bytes) & userid together for table argument
# - see vital instructions 'tbl' (build table) & 'tbp' (print table)
#
rop=r1 # option to prompt for report disposition at EOJ
fili1=/var/log/messages,typ=LST,rcs=256
filo1=stats/login.rpt1,typ=LSTt,rcs=128
@run
opn all
#
# begin loop to get & process messages lines until EOF
man20 get fili1,a0 get next line of messages
skp> man90 (cc set > at EOF)
sqzc1 a0(256),' ' ensure only 1 blank between words
#
# scan for 'session opened' & 'user' login
# table user logins by month & by month+day
scn a0(100),' session opened '
skp! man20
scn a0(100),' user '
skp! man20
clr b0(500),' ' clear workarea
mvu b0(25),ax6,' ' store user login until ending blank
mvc b100(3),a0 store mth (1st 3 bytes)
mvc b104(25),b0 follow with userid
tblt1f4 b100(32),'mth login'
skp man20 return to get next line
#
# EOF - dump tables, close files, prompt for report view (rop=r1), end job
man90 tbpt1 filo1,'logins by month & userid'
cls all
eoj
#
# statlogin2 - also see alternate job to create 4 table summaries,
# #1=month+userid, #2=userid+month
# #3=month+day+userid, #4=userid+month+day
# statlogin1 - creates just the 1st of these 4 table summaries
Goto: Begin this document , End this document , UVSI Home-Page