JCL Conversion - Related Documents

JCLcnv1demo.htm - DEMO conversions, sample JCL, scripts, executions JCLcnv2real.htm - comprehensive instructions for REAL conversions JCLcnv3aids.htm - conversion AIDS (cross-refs,tips,mass changes,etc) *THIS DOC* JCLcnv4gdg.htm - GDG file handler from UV Software DATAcnv1.htm - Data file conversion - Original Documentation DATAcnv2.htm - Data File Conversion - Comprehensive & Complete

JCLcnv3aids - Contents by Part


Part_1 Cross-References available
xrefall - script to run all cross-ref reports
xkshfile1 - list all datafiles in each JCL/ksh/script
xkshfile2 - shows all JCL/ksh/scripts using each data-file
xkshparm1 - list all SYSIN/PARM modules in each JCL/ksh/script
xkshparm2 - shows all JCL/ksh/scripts referencing each SYSIN/PARM
xkshparmsd1 - alternate crossref if parms in multiple subdirs
xkshparmsd2 - alternate crossref if parms in multiple subdirs
xjclproc1 - list all PROCs executed in each JCL/ksh/script
xjclproc2 - shows all JCL/ksh/scripts executing each PROC
xkshprog1 - list all programs executed in each JCL/ksh/script
xkshprog2 - shows all JCL/ksh/scripts executing each program
xkshprog2a - alternate crossref combining all COBOLs
xcobcopy1 - list all 'copy'books used in each program
xcobcopy2 - shows all programs using each 'copy'book
xcobcall1 - list all 'call'ed programs in each program
xcobcall2 - shows all programs 'call'ing each program
xcobfile2 - crossref all PROGRAMS using each external-file
xcobsql1 - list all SQL Includes in any 1 PROGRAM
xcobsql2 - crossref all PROGRAMS using any 1 SQL Include
cobfiles5A - COBOL file report
Statistics (table-summaries) available
statmvsjcl1 - statistics for mainframe JCL
- count programs,PROCs,top-nodes,files used in all JCLs
statksh1 - statistics for Korn shell scripts
- count programs,exportfile/exportgen,filenames in Ksh scripts
statcbl1 - statistics for COBOL programs
- count copybooks, called programs,& file ORG type
mvsfiles5A - script to determine essential Input files for ALL JCL/scripts
- extracts all data files from jcl2 (JCL procs expanded)
- report xmvs/mvsfiles5 '*' flags essential input files
mvsfiles51 - determine essential input files for 1 JCL vs All JCLs
- invaluable during test/debug

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

JCLcnv3aids - Contents by Part


Part_2 Test/Debug Tips & Techniques
 Test Environment, RUNLIBS & RUNDATA superdirs defined in your profile
- allows different programmers to have their own set of Libraries & Data
 See JCL converted to scripts & executed in JCLcnv1demo.htm
 Use these tips for testing the Demo JCL/scripts or for your own JCL/scripts
 Converting Your DATA files - brief review of DATAcnv1.htm.
 run 'testdatainit' before JCL/scripts
- to clear output files & make it easier to see outputs of current test
- use 'joblog' to capture the log from your JCL/script
 Test/Debug for difficult JCL/scripts
- save all data1/* in datasave/ & load data1/ with files for difficult job
 Iterations of test/investigate/modify as required
- Animator/Debugger for Micro Focus COBOL
 Check results of test/debug, investigate output files (use uvhd if packed)
Print-outs using 'uvlp' scripts assist test/debug
 Modifying no of gnerations in GDG control file & reloading Indexed file
 GDG files & step Restart
 Activating 'console-logging' vs 'job-logging'
 File Comparison for files with packed/binary &/or no LineFeeds
- unix 'diff' does not work for these types of files
- UV Software provides the 'uvcmp' utilities
- uvcopy jobs uvcmp1,2,3 & several scripts uvcmpFA1,uvcmpFE1,etc
  to make the uvcopy jobs easier to run.
- uvcmp prints mismatched record pairs in vertical hexadecimal
  flagging differences with '*'s, see sample report comparing
  2 generations of gl.account.master_000001 & _000002

Part_3 Updating control-files to improve JCL conversions
Control-Files supply data-file-info that may be missing in JCL
(record-sizes,file-types,GDG generations,etc)
- required for script conversions of SORTs,IDCAMs,IEBGNENERs,etc
 capturing COBOL info for JCL conversion (record-sizes, file-types)
 control files created by 1st conversion (in subdir ctl/...)
- datajcl51, datajcl52 extracted from the JCL
 JCL converter script 'jcl2ksh51A' does everything
- calls 'jcldata51A' to create datajcl51 & datajcl52
  & load into Indexed file datactl53I.dat/.idx for JCL converter
- calls 'jclxx51' for JCL conversion to Korn shell scripts
- updates datactl53I with datafile info from COBOL conversions
 using LISTCAT mainframe reports to improve JCL conversion
- transfer, translate, store multiple reports in subdir cat0/...
 LISTCAT report sample (mock-up for VU demo files)
 converting Excel spreadsheets of datafile info to control files
Re-Converting JCL with LISTCAT, datamisc,& dataxls
- using 'alldiff2' script to see differences made by re-conversion
- confirm chages as expected & no unintended changes
 Modifying JCL converter 'options'
- example to change fixed length data files to Line Sequential
- if you have very few files with packed/binary fields
 Determine Essential INPUT files for any 1 JCL/script
- script 'mvsfiles51' selects essential input files
- ignoring output files & intermediate created/used within JCL

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

JCLcnv3aids - Contents (continued)


Part_4 Checking conversion results, Searching for potential problems.
Tools to search directories for patterns with qualifiers.
'grep' is the 1st choice to search for 1 pattern.
'selectf1' will search for up to 3 patterns with occurrence counts
=, >, or < specified values. Default is AND, may specify OR.
See example to search for SORT's with 'INCLUDE COND' or 'OMIT COND'
'scan1d' searches a directory for all lines matching 1 pattern
'scan2d' searches a directory for all lines matching any pattern
in a table of pre-edited patterns
'scan3d' selects lines from a starting pattern to an ending pattern,
qualified by a 3rd pattern present anywhere between start & end.

Part_5 Mass changes to all files in directories of JCL &/or COBOL.
- summary of tools available
 Mass change example using 'sed' & script 'runsed1'
- uncomment all '#rm' file deletes after test/debug
- example using 'runsed1' to prompt for sed 's' substitute commands
  & apply while copying all files in directory to a 2nd directory
 Mass change example using sed, runsed1,& runsed2
- change all 'ksh' to 'ksh93', for clients that do not want
  to upgrade their current ksh (ksh88) to ksh93 (best for JCL/scripts)
 Mass change using uvcopy job 'rep1d'
rep1d - copies all files from 1 dir to a 2nd dir
- prompts for 1 pattern to be replaced by a 2nd pattern
- optionally qualified by 2 other patterns (present or absent)
 Mass change using uvcopy job 'rep2d'
rep2d - copies all files from 1 dir to a 2nd dir
- searching/replacing depending on a pre-edited table
  of search/replace/qualify patterns
rep3d - replace blocks of code in all files in a directory
- blocks identified by a start pattern, an end pattern,&
  an optional qualifier pattern between start & end patterns
uvfix1 - easy way to fix various problems in datafiles & scripts.
- allows you to apply the powerful uvcopy instructions
- translates, convert data field type (packed/binary/zoned)
- example given to correct EBCDIC zoned signs to unix standards
uvfixA - same as uvfix1, but processes all files in a directory
  while copying to the tmp1/ directory

Part_6 Optional Conversion Enhancements
 Directories required (Libraries & Data)
 JCL conversion OPTIONS control file (jclunixop51)
- modify options to improve conversions depending on site data types
  datafiles containing many packed/binary fields or mostly character
 Add data file info onto JCL/script file definitions (as #comments)
- appends Record-Sizes & Packed/Binary
- examples of before/after adding datafileinfo as #comments
  ctl/datactl53I - control file info for JCL conversion

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

JCLcnv3aids - Contents (continued)


Part_7 Test/Verify Critical Features in Converted JCL/scripts
 Test 'joblog2' run JCL/scripts serially
 Prove joblog2 serial scheduler works using jar100.ksh
- insert prompt/reply, run job from 2 screens, note delay
- answer prompt on 1st job to complete & allow 2nd job to run
 Test automatic GDG generation# re-sequence at 900000
- gdgrest1 resets gdg#s for 'ONE' file automatically, whenever gdg#
  900000 exceeded. 'gdgreset1' is called by function 'jobend51' which was
  inserted by the JCL converter at the end of all JCL/scripts.
 Test batch job to re-sequence gdg#s > specified threshold
- gdgreset2 will reset GDG#s for 'ALL' files whose GDG#s exceed a
  specified threshold (suggested default 500000). You would run this
  batch job when no JCL/scripts are being executed.

potential Additions

Might add 'IP# replace' to part 6 here in JCLcnv3aids.doc from JCL2cnvreal.doc#1E5 - 1E7 & 2A1

Might add jclunixop51 listing somewhere in this doc ?

select2d, xkshprog2a, xrefdrop1 - noted on page '1A1'

chkrcs1, chkrcs2, chkrcs2A - also noted in Part_6 contents


 uvcopy chkrcs1,fild1=jcl3,filo2=rpts/chkrcs1.rpt
 ================================================
 - check record sizes on uvsort/uvcp in JCL/scripts
 - see MVSJCL.doc or www.uvsoftware.ca/mvsjcl.htm

 uvcopy chkrcs2,fild1=jcl3,filo1=rpts/chkrcs2_fj,filo2=rpts/chkrcs2_jf
 =====================================================================
 - check record sizes on uvsort/uvcp in JCL/scripts
 - report datafilenames for rcs=99999 from saved SORTIN exports

 chkrcs2A jcl3     <-- execute uvcopy job chkrcs2 + sorts
 =============

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_1 JCL test/demo conversions

Cross-References - Contents


1A1. xrefall - script to run all cross-ref reports

1B1. xkshfile1 - list all datafiles in each JCL/ksh/script
1B2. xkshfile2 - shows all JCL/ksh/scripts using each data-file
1C1. xkshparm1 - list all SYSIN/PARM modules in each JCL/ksh/script
1C2. xkshparm2 - shows all JCL/ksh/scripts referencing each SYSIN/PARM
- alternate for sites with duplicate parmnames in multiple subdirs
xkshparmsd1 - alternate parms in each JCL/script
xkshparmsd1 - alternate JCL/scripts referencing each parm
1D1. xjclproc1 - list all PROCs executed in each JCL/ksh/script
1D2. xjclproc2 - shows all JCL/ksh/scripts executing each PROC
1E1. xkshprog1 - list all programs executed in each JCL/ksh/script
1E2. xkshprog2 - shows all JCL/ksh/scripts executing each program
1E3. xkshprog2a - same as xkshprog2, but consolidates all COBOL programs
to highlight the various utilities & 3rd party software

1F1. xcobcopy1 - list all 'copy'books used in each program
1F2. xcobcopy2 - shows all programs using each 'copy'book
xcobcall1 - list all 'called'programs used in each program
xcobcall2 - shows all programs calling each 'called'program
1G1. xcobfile2 - crossref all PROGRAMS using each external-file

1H1. cobfil51 - COBOL Files Report
(not exactly a cross-reference, but a very useful report)
- information extracted from all COBOL programs in directory
- filenames, Input/Output, Org, Access, record size, copybook
- all programmers should have a copy before starting test/debug

1I1. Statistics (table-summaries) available
statmvsjcl1 - statistics for mainframe JCL
- count programs,PROCs,top-nodes,files used in all JCLs
statksh1 - statistics for Korn shell scripts
- count programs,exportfile/exportgen,filenames in Ksh scripts
statcbl1 - statistics for COBOL programs
- count copybooks, called programs,& file ORG type

1J1. jcl3_programs - table summary all programs executed by converted scripts
- highlights mainframe utilities that may need replacement
1J2. jcl3_exportfiles - summary of exportfiles(non-GDG) & exportgens(GDG files)
- count exportfile/exportgen gen#s & subdirs
1J3. jcl3_parms - table summary of all SYSIN/PARM libarary modules
1J4. jcl3_filenames - table summary of all filenames in JCL/ksh/scripts
- may be a long report (vs above short reports)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Cross-References - Contents (continued)


1K1. cbls_copybooks - table summary counts of all copybooks used by all programs
1K2. cbls_ORGtypes - table summary of file ORGANIZATION types used
(SEQUENTIAL,LINE SEQUENTIAL,INDEXED,DYNAMIC,RANDOM,etc)

1L1. mvsfiles5A - script to trace data files in MVS JCL
- extracts all data files from subdir jcl2 (JCL procs expanded)
- identifies the essential INPUT files to execute JCL/scripts
- INVALUABLE during test/debug
mvsfiles51 - for 1 JCL at a time (vs all JCLs in jcl2/*)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1A1. JCL/COBOL test/demo Cross-References

xrefall - create all JCL/COBOL cross-refs


 #1. Login userxx --> /home/userxx  (or whatever Login desired)

 #2. cdl --> $HOME/testlibs (alias cdl='cd $HOME/testlibs)

 #3. xrefall cbls jcl3 ksh      <-- create all cross-ref reports
     =====================          COBOL/copybooks & JCL/scripts

 #4. cobfiles5A cbls cpys maps  <-- create COBOL 'FILES' report
     =========================

cross-ref report selections

Mar31/2017
  • inserted here, but should follow the xref sample reports

 uvcopy select2d,fild1=xref,fild2=xrefmissing,uop=a0b1,arg1='*'
 ==============================================================
 - create 'missing modules' reports for parms, programs, copybooks
Note
  • xkshprog2a report drops the COBOL programs to highlight utilities

 uvcopy xrefdrop1,fili1=xref/xkshprog2a,filo1=xref/xkshprog2a_sum
 ================================================================
 - drops all but 1st & last lines of utilities xref to create short report
   showing the total number of each utility with a few JCL names where used

statistics (table summary counts)


 #4. uvcopy statksh1   <-- create stat reports from Korn shell scripts
     ===============

 #5. uvcopy statcbl1   <-- create stat reports from COBOL programs
     ===============
Note
  • The Cross-References are shown on the next 7 pages '1B1' - '1H1'
  • Then the statistics reports on pages '1I1' - '1K1'

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1A2. JCL/COBOL test/demo Cross-References

xrefallparm - extra crossrefs for parms

`xrefall`above creates reports `xkshparm1` & `xkshparm2`. which is sufficient for most sites. `xrefallparm`creates additional crossrefs as follows:

xkshparmjps1/2
  • crossrefs by Jobname + Program + Step
xkshparmjp1/2
  • crossrefs by Jobname + Program
xkshparmp1/2
  • crossrefs by Program (all jobs & steps combined)
xkshparmp1
  • lists all parms used by each program
  • all instances of program in all job steps
xkshparmp2
  • shows programs using each parm

'xkshparmp1' was the main objective here, to get a list of all parms used by proograms such as FASTLOAD which required special conversions (convert DECIMAL to varchar).


 #1. mkdir step1 step2 step3 step4
     =============================

 #2. xrefallparm jcl3
     ================

xrefalltdsql - crossrefs for TeraData SQLs

xrefalltdsql is for sites that maintain a separate parms subdir for systems such as TeraData (BTEQ,FASTLOAD,MLOAD). The SQL parameters (load/unload tables) are maintained in separate parms subdir 'tdsqls'. xrefalltdsql creates following reports:

xkshtdsqlp1
  • lists tdsqls used by all instances of TeraData
    (bteqm fastload, mload)
xkshtdsql2
  • shows programs using each parm

 #3. xrefalltdsql step4
     ==================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1B1. JCL/COBOL test/demo Cross-References

xkshfile1 - list datafiles in each script

 xkshfile1 ** list all DATAFILES used in each ksh SCRIPT **
 Keyword=exportfile:exportgen  Exclude=autoload:JTMP:SYOT Include=~~ Skip=~~ page#  1
 Directory=/home/userxx/testlibs/jcl3
 Options=q1a16b16c4e0d0j0l0s2w1a16b50c1j4w2q0i7
 =======================================================2011/11/05_16:35:15
 eztlist.ksh      $RUNLIBS/parms/eztlist1
 ___________      data1/ar.customer.eztlist
 ____________3    data1/ar.customer.master
 ftpput1.ksh      FTPPUT01
 jar100.ksh       data1/ar.customer.master
 ___________2     data1/ar.customer.nameadrs.list100
 jar200.ksh       data1/ar.customer.master.indexed
 __________       data1/ar.sales.items
 ___________3     data1/ar.sales.list
 jgl100.ksh       ACCTMAS
 ___________2     ACTLIST
 jgl200.ksh       $RUNLIBS/parms/pgl200s1
 __________       GLMSNEW
 __________       GLMSOLD
 __________       GLTRANS
 __________       SORTOUT
 ___________6     data1/gl.account.tran1
 jgl300.ksh       ACNTMAS
 ___________2     SYSUT1
 jgl320.ksh       ACNTMAS
 ___________2     SYSUT1
 qjtlist.ksh      $RUNLIBS/parms/qjtlist1
 ___________      data1/ar.custmas.tellist
 ____________3    data1/ar.customer.master

**TotalFiles: 9, TotalWords: 24, TotalRefs: 24

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1B2. JCL/COBOL test/demo Cross-References

xkshfile2 - scripts using each data-file

 xkshfile2 ** crossref to show all ksh SCRIPTs using each DATAFILE **
 Keyword=exportfile:exportgen  Exclude=autoload:JTMP:SYOT Include=~~ Skip=~~ page#  1
 Directory=/home/userxx/testlibs/jcl3 Options=q1a16b16c4e0d0j0l0s2w1a40b18c3j4w2q0i7
 =======================================================2011/11/05_16:35:15
 $RUNLIBS/parms/eztlist1                  eztlist.ksh
 $RUNLIBS/parms/pgl200s1                  jgl200.ksh
 $RUNLIBS/parms/qjtlist1                  qjtlist.ksh
 ACCTMAS                                  jgl100.ksh
 ACNTMAS_2                                jgl300.ksh        jgl320.ksh
 ACTLIST                                  jgl100.ksh
 FTPPUT01                                 ftpput1.ksh
 GLMSNEW                                  jgl200.ksh
 GLMSOLD                                  jgl200.ksh
 GLTRANS                                  jgl200.ksh
 SORTOUT                                  jgl200.ksh
 SYSUT1_2                                 jgl300.ksh        jgl320.ksh
 data1/ar.custmas.tellist                  qjtlist.ksh
 data1/ar.customer.eztlist                 eztlist.ksh
 data1/ar.customer.master_3                eztlist.ksh       jar100.ksh        qjtlist.ksh
 data1/ar.customer.master.indexed          jar200.ksh
 data1/ar.customer.nameadrs.list100        jar100.ksh
 data1/ar.sales.items                      jar200.ksh
 data1/ar.sales.list                       jar200.ksh
 data1/gl.account.tran1                    jgl200.ksh

**TotalWords: 20, TotalFiles=24, TotalRefs: 24

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1C1. JCL/COBOL test/demo Cross-References

1C1. xkshparm1 - all PARMs in each script

 xkshparm1 ** crossref to show all ksh SCRIPTS referencing each PARM **
 Keyword=parms  Exclude=~~ Include=exportfile Skip=~~ page#  1
 Directory=/home/userxx/testlibs/jcl3
 Options=q1a16b16c4e0d0j0l0s2w1a16b16c4d1j4l1w1q0i7
 =======================================================2011/11/05_16:35:15
 eztlist.ksh      eztlist1
 jgl200.ksh       pgl200s1
 qjtlist.ksh      qjtlist1

**TotalFiles: 3, TotalWords: 3, TotalRefs: 3

1C2. xkshparm2 - all scripts using each PARM

  xkshparm2 ** crossref to show all ksh SCRIPTS referencing each PARM **
  Keyword=parms  Exclude=~~ Include=exportfile Skip=~~ page#  1
  Directory=/home/userxx/testlibs/jcl3 Options=q1a16b16c4e0d0j0l0s2w1a16b16c4d1j4l1w1q0i7
  =======================================================2011/11/05_16:35:15
  eztlist1         eztlist.ksh
  pgl200s1         jgl200.ksh
  qjtlist1         qjtlist.ksh

**TotalWords: 3, TotalFiles=3, TotalRefs: 3

1C3. xkshparmsd1/xkshparsd2

xkshparmsd1 & xkshparmsd2 are for sites that had duplicate parmnames in different PDS libraries on the mainframe, so on unix we cannot combin all parms into 1 subdir parms/... We must store in multple subdirs

 parms       <-- subdir for all parms when no duplicate names
 parmsds     <-- superdir for multiple subdirs when parmnames not unique
 :-----pgapo_abe_ekstada_vlk        <-- multi subdirs
 :     :--------------------artgrp
 :     :--------------------artgw   <-- parm modules
 :-----pgapo_abe_katalog_vlk
 :     :--------------------adwh400
 :     :--------------------artgw   <-- may be duplicate names

See details for parms in multiple subdirs at JCLcnv2real.htm#1E1 - 1E8.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1D1. JCL/COBOL test/demo Cross-References

xjclproc1 - all PROCs executed in each script

 #xjclproc1 ** crossref all MVS JOBnames for each EXEC PROCname **
 #Keyword=EXEC  Exclude=PGM Include=~~ Skip=~~ page#  1
 #Directory=/home/mvstest1/testlibs1/jcl1
 #Options=q1a16b16c4e0d0g0h1j0l1p0s2w1a16b16c4j15w1q0i7
 #=======================================================2016/02/15_12:38:24
 jgl100.jcl       pgl100
 jgl200.jcl       pgl200

#**TotalFiles: 2, TotalWords: 2, TotalRefs: 2

1D2. xjclproc2 - all scripts executing each PROC

 #xjclproc2 ** crossref all MVS JOBnames for each EXEC PROCname **
 #Keyword=EXEC  Exclude=PGM Include=~~ Skip=~~ page#  1
 #Directory=/home/mvstest1/testlibs1/jcl1 Options=q1a16b16c4e0d0g0h1j0l1p0s2w1a16b16c4j15w1q0i7
 #======================================================2016/02/15_12:38:24
  pgl100           jgl100.jcl
  pgl200           jgl200.jcl

#**TotalWords: 2, TotalFiles=2, TotalRefs: 2 #**Total Missing Procs = 0 **

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1E1. JCL/COBOL test/demo Cross-References

1E1. xkshprog1 - programs executed in each script

 xkshprog1 ** list all PROGRAMs executed in each ksh SCRIPT **
 Keyword=progid:program  Exclude=~~ Include=~~ Skip=  page#  1
 Directory=/home/userxx/testlibs/jcl3
 Options=q1a16b16c4e0d0j0l0s2w1a18b16c4d8j4l1q0i7
 =======================================================2011/11/05_16:35:15
 eztlist.ksh        eztpa00
 ftpput1.ksh        ftp
 jar100.ksh         car100
 jar200.ksh_2       car200          sort
 jgl100.ksh         cgl100
 jgl200.ksh_2       cgl200          sort
 jgl300.ksh_2       idcams          iebgener
 jgl320.ksh_2       idcams          iebgener
 mailsmtp.ksh       iebgener
 qjtlist.ksh        quikjob

**TotalFiles: 10, TotalWords: 14, TotalRefs: 14

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1E2. JCL/COBOL test/demo Cross-References

xkshprog2 - all scripts executing each program

  xkshprog2 ** crossref to show all ksh SCRIPTS executing each PROGRAM **
  Keyword=progid:program  Exclude=~~ Include=~~ Skip=  page#  1
  Directory=/home/userxx/testlibs/jcl3 Options=q1a16b16c4e0d0j0l0s2w1a16b16c4d8j4l1q0i7
  =======================================================2011/11/05_16:35:15
  car100           jar100.ksh
  car200           jar200.ksh
  cgl100           jgl100.ksh
  cgl200           jgl200.ksh
 *eztpa00          eztlist.ksh
 *ftp              ftpput1.ksh
 *idcams_2         jgl300.ksh      jgl320.ksh
 *iebgener_3       jgl300.ksh      jgl320.ksh      mailsmtp.ksh
 *quikjob          qjtlist.ksh
 *sort_2           jar200.ksh      jgl200.ksh

**TotalWords: 10, TotalFiles=14, TotalRefs: 14

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1E3. JCL/COBOL test/demo Cross-References

xkshprog2a - program crossref with all COBOLs consolidated

 #xkshprog2a ** crossref to show all ksh SCRIPTS executing each PROGRAM **
 #Keyword=progid:program  Exclude=~~ Include=~~ Skip=  page#  1
 #Directory=/home/mvstest1/testlibs1/jcl3 Options=q1a16b16c4e0d0g0h1j0l1p0s2w1a16b16c4d8j4l1q0i7
 #======================================================2016/02/15_12:38:24
 *cobol            jar100.ksh      jar200.ksh      jgl100.ksh      jgl200.ksh
  ______6          jgl230.ksh_2
  eztpa00          eztlist.ksh
  ftp              ftpput1.ksh
  icetool          icetool3.ksh
  idcams           jgl320.ksh
  iebgener_2       jgl320.ksh      mailsmtp.ksh
  quikjob          qjtlist.ksh
  sort_3           jar200.ksh      jgl200.ksh      jgl230.ksh

#**TotalWords: 8, TotalFiles=15, TotalRefs: 16

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1F1. JCL/COBOL test/demo Cross-References

1F1. xcobcopy1 - all copybooks used in each program

 xcobcopy1 ** list all COPYBOOKS in each PROGRAM **
 Keyword=copy:include  Exclude=unix Include=.cpy Skip=  page#  1
 Directory=/home/userxx/testlibs/cbls
 Options=q1a16b16c4e0d0j0l0s2w1a16b16c4e3l1q0i7
 =======================================================2011/11/05_16:35:14
 car100.cbl       custmas.cpy
 car200.cbl_4     custmas.cpy     saledtl.cpy     sdline.cpy      stline.cpy

**TotalFiles: 2, TotalWords: 5, TotalRefs: 5

1F2. xcobcopy2 - all programs using each copybook

  xcobcopy2 ** crossref all PROGRAMS using each COPYBOOK **
  Keyword=copy:include  Exclude=unix Include=.cpy Skip=  page#  1
  Directory=/home/userxx/testlibs/cbls Options=q1a16b16c4e0d0j0l0s2w1a16b16c4e3l1q0i7
  =======================================================2011/11/05_16:35:14
  custmas.cpy_2    car100.cbl      car200.cbl
  saledtl.cpy      car200.cbl
  sdline.cpy       car200.cbl
  stline.cpy       car200.cbl

**TotalWords: 4, TotalFiles=5, TotalRefs: 5

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1G1. JCL/COBOL test/demo Cross-References

1G1. xcobfile2 - all PROGRAMS using each external-file

 xcobfile2 ** crossref all PROGRAMS using each external-file **
 Keyword=external  Exclude=~~ Include=~~ Skip=  page#  1
 Directory=/home/userxx/testlibs/cbls Options=q1a16b16c4e0d0j0l0s2w1a16b16c4e1l1q0i7
 =======================================================2011/11/05_16:35:14
 acctmas          cgl100.cbl
 actlist          cgl100.cbl
 custmas_2        car100.cbl      car200.cbl
 glmsnew          cgl200.cbl
 glmsold          cgl200.cbl
 gltrans          cgl200.cbl
 nalist           car100.cbl
 saledtl          car200.cbl
 salelst          car200.cbl

**TotalWords: 9, TotalFiles=10, TotalRefs: 10

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1H1. JCL/COBOL test/demo Cross-References

COBOL Files Report


 cobfiles5A cbls cpys maps  <-- create COBOL 'FILES' report
 =========================
 cobfil51  ** COBOL Files Report ** Dir=cbls  2011/10/23_16:39:38
 progname.cbl DDname       OAM   open recsz  copybook.cpy  FDname / Key    lines
 ================================================================================
 car100.cbl   custmas      SS_   I___   256  custmas.cpy   custmas
 car100.cbl   nalist       L__   O___   120                nalist
                                                                              52
 car200.cbl   saledtl      SS_   I___    64  saledtl.cpy   saledtl
 car200.cbl   custmas      IR_   I___   256  custmas.cpy   custmas
 car200.cbl          key->                                    cm-cust
 car200.cbl   salelst      L__   O___   120  sdline.cpy    salelst
                                                                              65
 cgl100.cbl   acctmas      SS_   I___   128                acctmas
 cgl100.cbl   actlist      L__   O___   120                actlist
                                                                              57
 cgl200.cbl   glmsold      SS_   I___   128                glmsold
 cgl200.cbl   glmsnew      SS_   O___   128                glmsnew
 cgl200.cbl   gltrans      SS_   I___    80                gltrans
                                                                              65
                    ** Note re COBOL Files Report **

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1I1. JCL/COBOL test/demo Cross-Refs & Statistics

JCL/script Statistics (table-summaries)

statmvsjcl1
  • statistics for original mainframe JCL
  • count programs,PROCs,top-nodes,files used in all JCLs
  • not shown here, statksh1 similar
statksh1
  • statistics for converted Korn shell scripts
  • count programs,exportfile/exportgen,filenames in Ksh scripts
statcbl1
  • statistics for COBOL programs
  • count copybooks, called programs,& file ORG type

OP. Instrns. for statksh1 & statcbl1


 #1. Login userxx --> /home/userxx  (or whatever Login desired)

 #2. cdl --> $HOME/testlibs (alias cdl='cd $HOME/testlibs)

 #3. mkdir stats       <-- make subdir for reports
     ===========

 #4. uvcopy statmvsjcl1  <-- create stat reports from MVS JCL
     ==================

 #5. uvcopy statksh1   <-- create stat reports from Korn shell scripts
     ===============

 #6. uvcopy statcbl1   <-- create stat reports from COBOL programs
     ===============

 #7. vi stats/*        <-- view stat reports
     ==========

 #8. uvlpd12 stats     <-- print all reports
     =============         OR print only desired reports
                         - most stats/reports short
                         - BUT, filename reports could be very long

 #ra. uvlp12 stats/jcl3_programs  <-- print table summary of executed programs
      ==========================    - 1 of shorter useful reports
Note
  • 'uvlpd12' is script that prints all files in the subdir
    (1st page is contents, list of filenames printed on following pages)
  • 'uvlp12' is a script that prints only the 1 named file

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1J1. JCL/script/COBOL Statistics (table-summaries)

1J1. jcl3_programs - programs executed by scripts

 statksh1  2011/10/28_13:16:05  Summary table of PROGIDs in jcl3
 tbl#0001  tblt1f4 c0(32)
 line#     count  %    1strec#  PROGID
     1         1   7        19  car100
     2         1   7        47  car200
     3         1   7        25  cgl100
     4         1   7        53  cgl200
     5         1   7        19  eztpa00
     6         1   7        18  ftp
     7         2  14        44  idcams
     8         3  21        22  iebgener
     9         1   7        19  quikjob
    10         2  14        19  sort
              14*100             *TOTAL*

1J2. jcl3_exportfiles(nonGDG) & exportgens(GDGs)

 statksh1  2011/10/28_13:16:05  Summary exportfile/exportgen subdirs in jcl3
 tbl#0002  tblt2f4 e100(32)
 line#     count  %    1strec#  exportfile/gen [0/+1] subdir/
     1        12  30        22  exportfile $JTMP/
     2         3   7        31  exportfile $RUNLIBS/
     3         3   7        30  exportfile $SYOT/
     4        10  25        21  exportfile data1/
     5         5  12        24  exportgen0 0 data1/
     6         6  15        47  exportgen1 +1 $JGDG/
              39*100             *TOTAL*

1J3. jcl3_parms SYSIN/PARM libarary modules

 statksh1  2011/10/28_13:16:05  Summary table of parms/modules in jcl3
 tbl#0004  tblt4f4 d900(30)
 line#     count  %    1strec#  parms/module
     1         1  33        23  eztlist1
     2         1  33        31  pgl200s1
     3         1  33        23  qjtlist1
               3*100             *TOTAL*

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1J4. JCL/script/COBOL Statistics (table-summaries)

1J4. jcl3_filenames - all filenames in ksh/scripts

 statksh1  2011/10/28_13:16:05  Summary table of data1/filenames in jcl3
 tbl#0005  tblt5f7 d300(48)
 line#     count  %    1strec#  subdir/filename
     1         1   2        28  $JGDG/data1/gl.account.acntlist_
     2         3   7        47  $JGDG/data1/gl.account.master_
     3         2   5        29  $JGDG/data1/gl.account.trans_
     4         1   2        29  $JTMP/${JSTEP}_${PROGID}_FTPTMP2
     5         1   2        20  $JTMP/${JSTEP}_${PROGID}_INPUT1
     6         3   7        23  $JTMP/${JSTEP}_${PROGID}_SYSIN
     7         1   2        21  $JTMP/${JSTEP}_${PROGID}_SYSUT1
     8         4  10        25  $JTMP/__tempglm
     9         2   5        22  $JTMP/__tempsls
    10         1   2        23  $RUNLIBS/parms/eztlist1
    11         1   2        31  $RUNLIBS/parms/pgl200s1
    12         1   2        23  $RUNLIBS/parms/qjtlist1
    13         2   5        30  $SYOT/${JSTEP}_SYSOUT
    14         1   2        20  $SYOT/${JSTEP}_SYSUT2_B_SMTP
    15         1   2        22  data1/ar.custmas.tellist
    16         1   2        22  data1/ar.customer.eztlist
    17         3   7        21  data1/ar.customer.master
    18         1   2        50  data1/ar.customer.master.indexed
    19         1   2        22  data1/ar.customer.nameadrs.list100
    20         1   2        21  data1/ar.sales.items
    21         1   2        51  data1/ar.sales.list
    22         4  10        24  data1/gl.account.master_
    23         1   2        28  data1/gl.account.tran1
    24         1   2        28  data1/gl.account.trans_
              39*100             *TOTAL*

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1K1. JCL/script/COBOL Statistics (table-summaries)

1K1. cbls_copybooks - copybooks used by all programs

 statcbl1  2011/10/24_09:36:59  Summary of copybooks in cbls
 tbl#0001  tblt1f4 d0(32)
 line#     count  %    1strec#  copybook
     1         2  15        27  custmas.cpy
     2         1   7        25  saledtl.cpy
     3         1   7        29  sdline.cpy
     4         1   7        30  stline.cpy
     5         4  30        65  unixproc1.cpy
     6         4  30        46  unixwork1.cpy
              13*100             *TOTAL*

1K2. cbls_ORGtypes - file ORGANIZATION types

 statcbl1  2011/10/24_09:36:59  Summary of ORGANIZATION types in cbls
 tbl#0003  tblt3f4 d0(32)
 line#     count  %    1strec#  organization
     1         1  10        15  indexed
     2         3  30        20  line sequential
     3         6  60        10  record sequential
              10*100             *TOTAL*

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1L1. JCL/COBOL test/demo Cross-References

mvsfiles51,5A,5B - Determine Essential INPUT files

Scripts 'mvsfiles51/mvsfiles5A/mvsfiles5B' create "data file trace" reports from jcl2 (MVS JCLs with procs expanded). There are 7 reports created mvsfiles1-mvsfiles7 but the most useful report might be 'xmvs/mvsfiles5' which '*' flags the INPUT files actually needed to run any 1 job or all jobs in subdir jcl2/*.

This report eliminates duplicates & intermediate files (outputs for later inputs) & identifies the essential files required to be be transfered from the mainframe. There are 3 scripts as follows:

mvsfiles51
  • creates the 'essential files' report for 1 job at a time
    (vs all jobs in jcl2/...). Run this script before you attempt to
    debug each JCL/script to determine if you have all INPUT files required
    for a particular job.
  • the reports are created within xmvs/jobname subdirs
  • reports for cgl100.jcl will be xmvs/cgl100/mvsfiles1,2,3,4,5,6,7
  • reports for cgl200.jcl will be xmvs/cgl2l2/mvsfiles1,2,3,4,5,6,7
mvsfiles5B
  • runs 'mvsfiles51' for all JCLs found in jcl2/...
  • will create 1 jobname/ subdir for each JCL in jcl2/...
mvsfiles5A
  • creates 1 set of the 7 reports for All JCLs found in jcl2/...
  • creates these reports in subdir xmvsA/... (vs xmvs/...)

For this demo, we have run script mvsfiles5A based on 2 jobs (cgl100.jcl & cgl230.jcl) listed on pages '2C1' & '2E1'.

Note
  • you may find that 'mvsfiles51' reports in xmvs/jobname are the most useful
  • run mvsfiles51 before you test/debug each JCL/script to know the essential
    files required for that 1 JCL/script.
  • whereas the combined reports in xmvsA/... could be very large (all JCLs)

pre-requisites for mvsfiles51/mvsfiles5A/mvsfiles5B

You have already run 'jcl2ksh51A' (on page '3F3'), but you must now run 'cobfiles5A' before you run mvsfiles51 or mvsfiles5A. cobfiles5A creates Indexed files from COBOL copybooks & programs to supply record-sizes & packed field indicators.


 #1. jcl2ksh51A all   <-- converts all jcl0/* --> jcl1 --> jcl2 --> jcl3
     ==============     - creates ctl/datactl53I.dat/.idx used by mvsfiles53

 #2. cobfiles5A cbls cpys maps <-- creates xref/cobfiles report and also:
     =========================     ctl/cpyrcs1I.dat/idx & ctl/cobfilesI.dat/.idx
                                   required by $UV/pf/IBM/mvsfiles53

See the ctl/datactl53I, cpyrcs1I,& cobfilesI files listed on page '1L6' & '1L7'.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1L2. JCL/COBOL test/demo Cross-References

mvsfiles5A - Determine Essential INPUT files


 #1. Login userxx --> /home/userxx  (or whatever Login desired)

 #2. cdl --> $HOME/testlibs (alias cdl='cd $HOME/testlibs)

 #3. mkdir xmvsA    <-- make subdir for temp files & final report
     ===========

 #4. mvsfiles5A jcl2     <-- execute script to create report
     ===============

 #4a. mvsfiles51 jcl2/jgl230.jcl  <-- alternative script for 1 job at a time
      ==========================    - vs all jobs in jcl2/...

 #5. vi xmvsA/mvsfiles3   <-- inspect report of all files from each job
     ==================       (in jobname sequence)

 #6. vi xmvsA/mvsfiles5   <-- inspect report of all files SORTED by FILENAME
     ==================     - more useful to determine INPUT files required
                           - see mvsfiles5 listed on next page after mvsfiles3

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1L3. JCL/COBOL test/demo Cross-References

xmvsA/mvsfiles3 - sorted by job-name

  mvsfiles3 - Add Record-sizes & Packed field indicators - 2013/12/26 10:26:28
  Jobname Step  Program  DDname   MOD Gen Rcsz pb  From To * <-----DSNname------>
  ===============================================================================
 jgl100   001   CGL100   ACCTMAS  S     0 0128 p             GL.ACCOUNT.MASTER_
 jgl100   001   CGL100   ACTLIST  NCD  +1  133               GL.ACCOUNT.ACNTLIST_
 jgl230   001   SORT     SORTIN   O                          GL.ACCOUNT.TRAN1
 jgl230   001   SORT     SORTOUT  NCD  +1   80         002   GL.ACCOUNT.TRANS_
 jgl230   002   CGL200   GLTRANS  O    +1 0080     001       GL.ACCOUNT.TRANS_
 jgl230   002   CGL200   GLMSOLD  O     0 0128 p       003   GL.ACCOUNT.MASTER_
 jgl230   002   CGL200   GLMSNEW  NCD  +1   80         003   GL.ACCOUNT.MASTER_
 jgl230   003   CGL100   ACCTMAS  S     0 0128 p   002       GL.ACCOUNT.MASTER_
 jgl230   003   CGL100   ACTLIST  MKD  +1  133               GL.ACCOUNT.ACNTLIST_

notes re 'mvsfiles3' report above

  1. Script 'mvsfiles5A' (& 'mvsfiles51') run several uvcopy jobs & uvsorts, and creates several files in subdir xmvsA/... 'mvsfiles3' is the 1st report you would be interested in looking at.

  2. 'mvsfiles3' traces the data files thru each step of all jobs showing the 'FROM' origin step & the 'TO' destination step of the data files. This should help you identify the orignal input files required to test the job & distinguish them from the various inter-step files.

  3. 'MOD' codes are the 1st letters from DISP=(New,Keep,Delete), etc. 'Gen' codes are taken from GDG file suffixes (0), (+1), etc, which are replaced by the trailing underscore on the filenames.

  4. 'Rcsz' (record-size) is taken from LRECL on output files or RECSIZE in IEBGENER or IDCAMS. The JCL converter assumes input recsize same as output recsize on SORTs. You need to fix if a SORT changes recsize via OUTREC, etc.

  5. 'mvsfiles5' is the next report & the MOST USEFUL, because it SORTS by FILENAME and '*' flags the essential INPUT files required to execute the JCL/scripts. See notes following the report.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1L4. JCL/COBOL test/demo Cross-References

xmvsA/mvsfiles5 - sorted by file-name

  mvsfiles5 - Insert '*' Flags beside Esential Input files - 2013/12/26 10:26:28
  Jobname Step  Program  DDname   MOD Gen Rcsz pb  From To * <-----DSNname------>
  ===============================================================================
 jgl100   001   CGL100   ACTLIST  NCD- +1  133               GL.ACCOUNT.ACNTLIST_
 jgl230   003   CGL100   ACTLIST  MKD- +1  133               GL.ACCOUNT.ACNTLIST_
 jgl100   001   CGL100   ACCTMAS  S     0 0128 p           * GL.ACCOUNT.MASTER_
 jgl230   002   CGL200   GLMSOLD  O     0 0128 p       003   GL.ACCOUNT.MASTER_
 jgl230   002   CGL200   GLMSNEW  NCD- +1   80         003   GL.ACCOUNT.MASTER_
 jgl230   003   CGL100   ACCTMAS  S     0 0128 p   002       GL.ACCOUNT.MASTER_
 jgl230   001   SORT     SORTIN   O                        * GL.ACCOUNT.TRAN1
 jgl230   001   SORT     SORTOUT  NCD- +1   80         002   GL.ACCOUNT.TRANS_
 jgl230   002   CGL200   GLTRANS  O  - +1 0080     001       GL.ACCOUNT.TRANS_
  Note '*' files are essential Input files, only these '*' files will appear on:
   - next report xmvsA/mvsfiles6 summary of essential files (1 line/file)
   - the script  xmvsA/mvsfiles7 to copy files from $CNVDATA/d2asc to $RUNLIBS/data1

notes re 'mvsfiles5' report (above)

  1. 'mvsfiles5' (listed above) has been sorted by filename (right side), so you can see all jobs using each file. Note the '*' marking essential input files.

  2. Also note other information useful when debugging JCL/scripts - such as RECORD-SIZE & the 'p' flag (Packed fields present indicator).

  3. 'mvsfiles6' (listed later below) is the final report. It drops filename duplicates, temporary &&files, inter-step files,& inter-job files.

  4. Detection of inter-job files might not be perfect, because it assumes that the jobs are run in sequence by jobname. 'mvsfiles5' above shows the files sorted by filename(major) & jobname(minor). If the 1st line of a filename group is a 'N'ew file, then the entire group is '-' flagged for deletion. and the result is 'mvsfiles6' shown below.

  5. The report will be perfect if using the 'mvsfiles51' script (1 job at a time) vs 'mvsfiles5A' (all jobs in jcl2/...). Of course ssome of the '*' files could be coming from prior jobs & not need to FTP'd from the mainframe.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1L5. JCL/COBOL test/demo Cross-References

xmvsA/mvsfiles6 - summary of essential inputs

  mvsfiles6 - data file trace from MVS JCL - 2013/12/25 10:04:44
 jgl100   001   CGL100   ACCTMAS  S     0 0128 p    * GL.ACCOUNT.MASTER_
 jgl230   001   SORT     SORTIN   O                 * GL.ACCOUNT.TRAN1
  1. Our sample report (with our 2 simple jobs) is perfect, you can check back to the listings of the 2 JCLs on pages '2C1' & '2E1'.

  2. If you know that your jobs are run in sequence by jobname, this report should be a big help. You might transfer/convnert these 1st, try running the jobs,& transfer any missing files as required.

  3. The report will be perfect when running 'mvsfiles51' for 1 job at a time (vs mvsfiles5A for all jcl2/*).

  4. The final job 'mvsfiles56' in the scripts (mvsfiles5A/mvsfiles51) creates a script from the xmvsA/mvsfiles6 report above. It is named xmvsA/mvsfiles7, listed below with suggested execution.

 # mvsfiles7 - copy essential files from $CNVDATA/d2asc/... to $RUNDATA/data1/...
 cp $CNVDATA/d2asc/gl.account.master_*                    $RUNDATA/data1
 cp $CNVDATA/d2asc/gl.account.tran1                       $RUNDATA/data1

 #4. ksh xmvsA/mvsfiles7
     ===================
     - execute script, copy essential files from $CNVDATA/d2asc to $RUNDATA/data1

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1L6. JCL/COBOL test/demo Cross-References

pre-requisites for mvsfiles5A/mvsfiles51

Before running mvsfiles5A/51, you must have run 'jcl2ksh51A' & 'cobfiles5A'.


 #1. jcl2ksh51A all   <-- converts all jcl0/* --> jcl1 --> jcl2 --> jcl3
     ==============     - creates ctl/datactl53I.dat/.idx used by mvsfiles53

 #2. cobfiles5A cbls cpys maps <-- creates xref/cobfiles report and also:
     =========================     ctl/cpyrcs1I.dat/idx & ctl/cobfilesI.dat/.idx
                                   required by $UV/pf/IBM/mvsfiles53

ctl/datactl53 - control file for JCL converter

 ar.customer.master    cpy=____________ rcs=00256 typ=RSF  data=p  job=jar100 prg=CAR100
 ar.sales.items        cpy=____________ rcs=00064 typ=LST  data=   job=jar200 prg=SORT
 gl.account.acntlist_  cpy=____________ rcs=00133 typ=LST  data=   job=jgl100 prg=CGL100
 gl.account.master_    cpy=____________ rcs=00128 typ=RSF  data=p  job=jgl230 prg=CGL200
 gl.account.trans_     cpy=____________ rcs=00080 typ=LST  data=   job=jgl230 prg=SORT
 gl.parms@pgl200s1     cpy=____________ rcs=_____ typ=RSF  data=   job=jgl200 prg=SORT
 py.payroll.master     cpy=____________ rcs=00080 typ=RSF  data=pb job=jpy200 prg=CPY200

Notes re datactl53

  1. datactl53 (& Indexed file datactl53I.dat/.idx) were created by jcl2ksh51A (convert all jcl0/* --> jcl1 --> jcl2 --> jcl3). You have already run 'jcl2ksh51A' (on page '3F3').

  2. It is used by uvcopy job $UV/pf/IBM/mvsfiles53 (part of script $UV/sf/IBM/mvsfiles5A), to supply record-sizes for data-files.

  3. datactl53 will have info for all datafiles in all JCL/scripts (vs just 1 job if you run mvsfiles51).

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1L7. JCL/COBOL test/demo Cross-References

pre-requisites for mvsfiles5A/mvsfiles51


 #2. cobfiles5A cbls cpys maps <-- creates xref/cobfiles report and also:
     =========================     ctl/cpyrcs1I.dat/idx & ctl/cobfilesI.dat/.idx
                                   required by $UV/pf/IBM/mvsfiles53

ctl/cpyrcs1 - rec-ssizes from copybooks

 unixwork1.cpy                 rcs=00491 pbz= bz   01=unixwork1
 stline.cpy                    rcs=00120 pbz=      01=
 sdline.cpy                    rcs=00120 pbz=      01=
 saledtl.cpy                   rcs=00064 pbz=  z   01=
 custmas.cpy                   rcs=00256 pbz=p     01=

ctl/cobfiles - file info from COBOL programs

 cobfil51  ** COBOL Files Report ** Dir=cbls  2013/12/25_12:21:06
 progname.cbl DDname       OAM   open recsz pb   copybook.cpy  FDname Key   lines
 ================================================================================
 car100.cbl   custmas      SS_   I___   256 p    custmas.cpy   custmas
 car100.cbl   nalist       L__   O___   120                    nalist
                                                                              52
 car200.cbl   saledtl      SS_   I___    64      saledtl.cpy   saledtl
 car200.cbl   custmas      IR_   I___   256 p    custmas.cpy   custmas
 car200.cbl          key->                                        cm-cust
 car200.cbl   salelst      L__   O___   120      sdline.cpy    salelst
                                                                              65
 cgl100.cbl   acctmas      SS_   I___   128 p                  acctmas
 cgl100.cbl   actlist      L__   O___   120                    actlist
                                                                              57
 cgl200.cbl   glmsold      SS_   I___   128 p                  glmsold
 cgl200.cbl   glmsnew      SS_   O___   128                    glmsnew
 cgl200.cbl   gltrans      SS_   I___    80                    gltrans
                                                                              65
                      ** Notes re cpyrcs1 & cobfiles1 **
  1. You should run 'cobfiles5A' before you run mvsfiles51 or mvsfiles5A. cobfiles5A creates Indexed files from COBOL copybooks & programs to supply record-sizes & packed field indicators.

  2. Many record-sizes are supplied by ctl/datactl53I (see previous page), but cobfiles5A can supply record sizes for files that do not have an output file definition in the JCL.

  3. cobfiles5A supplies the 'p' codes to identify data files with 'p'acked fields present.

  4. cpyrcs1 & cobfiles reports will have info for all copybooks & all COBOL programs (vs just 1 job if you run mvsfiles51 vs mvsfiles5A).

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1L8. JCL/COBOL test/demo Cross-References

mvsfiles5A - determine essential files

 #!/bin/ksh
 # mvsfiles5A - Determine essential input files for All JCLs
 #            - examine all data files in all JCL in subdir jcl2/...
 #              (after JPROCs expanded during conversion to ksh scripts)
 #            - by Owen Townsend, Dec10/2011, updt Sep19/2015
 # mvsfiles51 - allternate job to determine essential Inputs for 1 JCL
 #Sep19/2015 - create reports for All JCL in xmvsA vs mvsfiles5B & mvsfiles51
 # - that create reports for each JCL in separate subdirs in xmvs/jobname
 echo "Determine essential Input files needing FTP from mainframe"
 jd="$1";
 if [[ -d "$jd" && -d xmvsA && -d ctl ]]; then :
 else echo "usage: mvsfiles5A jcl2 xmvsA"
      echo "       ====================="
      echo " - arg1 = subdir of MVS JCL (usually jcl2/... procs expanded)"
      echo " - subdir xmvsA must exist & will be cleared for reports"
      echo " - subdir ctl/ must exist (will store Indexed file)"
      exit 9; fi
 #
 echo "must have run cobfiles5A to create ctl/cobfilesI Indexed file info"
 echo "must have run jcl2ksh51A or jcldata5A to create ctl/datactl53I info"
 reply="x"
 until [[ "$reply" == "y" || "$reply" == "n" ]]
   do echo "- OK to clear all files from xmvsA/... for new reports y/n ?"
     read reply
   done
 if [[ "$reply" == "n" ]]; then echo "job cancelled"; exit 99; fi
 if [[ -d xmvsA ]]; then rm -f xmvsA/*; else mkdir xmvsA; fi
 if [[ -d tmp ]]; then rm -f tmp/*; else mkdir tmp; fi
 #
 uvcopy mvsfiles50,fild1=$jd,filo1=xmvsA/mvsfiles0,rop=q0i7r0
 #===========================================================
 #
 uvsort "fili1=xmvsA/mvsfiles0,rcs=128,typ=LSTt,filo1=xmvsA/mvsfiles1,key1=0(8)"
 #==============================================================================
 # sort by jobname only (do not change order of filenames within jobs)
 #
 uvsort "fili1=xmvsA/mvsfiles1,rcs=128,typ=LST,filo1=ctl/mvsfiles2I,rcs=63\
 ,typ=ISF,isk1=0(54d),key1=0(54),clr=0(64),mvc=0(8)0,mvc=9(44)56,mvc=55(3)9"
 #==========================================================================
 # sort xmvsA/mvsfiles1 by jobname+filename & load Indexed file
 #
 uvcopy mvsfiles53,fili1=xmvsA/mvsfiles1,filr2=ctl/mvsfiles2I,filr3=ctl/datactl53I\
 ,filr4=ctl/cobfilesI,filo1=xmvsA/mvsfiles3,rop=q0i7r0
 #=================================================================================
 #
 uvsort "fili1=xmvsA/mvsfiles3,rcs=128,typ=LSTt,filo1=xmvsA/mvsfiles4\
 ,key1b=60(44),del1=0(1):0x20"
 #====================================================================
 # sort by filename, drop input blank lines,
 # & insert blank lines between output filename groups
 echo " " >>xmvsA/mvsfiles4  # need blank line at EOF
 #
 uvcopy mvsfiles54,fili1=xmvsA/mvsfiles4,filo1=xmvsA/mvsfiles5\
 ,filr2=ctl/datactl53I,rop=q0i7r0
 #=============================================================
 # copy file inserting '*' flags on required input files
 # - insert recsize where absent on lines of filename groups
 #
 uvsort "fili1=xmvsA/mvsfiles5,rcs=128,typ=LSTt,filo1=xmvsA/mvsfiles6\
 ,key1u1=60(44),key2=0(8),sel1=0(1)>0x20,sel2=58(1):0x2A\
 ,clro=50(50),mvco=b51(50)a58"
 #====================================================================
 # sort by filename, unique key sort keep only 1st record of dup key set
 # - drop blank lines & '*' flagged lines (required input files)
 # - shift filename over beside copybook name (dropping From To steps)
 #
 uvcopy mvsfiles56,fili1=xmvsA/mvsfiles6,filo1=xmvsA/mvsfiles7,rop=q0i7r0
 #=======================================================================
 # convert final report of essential datafiles
 # into script to copy from $CNVDATA/d2asc/... to $RUNDATA/data1/...
 ls -l xmvsA
 echo "mvsfiles5 - lists all files (with '*' on non-essential files)"
 echo "mvsfiles6 - summary of essential input files"
 echo "mvsfiles7 - script to copy files from \$CNVDATA/d2asc to \$RUNLIBS/data"
 echo "xmvsA/... report file-names listed above, may display file-contents"
 reply="x"
 until [[ "$reply" == "y" || "$reply" == "n" ]]
   do echo "- reply 'y' to display reports xmvsA/mvsfiles5,6,7 - y/n ?"
     read reply
   done
 if [[ "$reply" == "y" ]]; then more xmvsA/mvsfiles[5-9]; fi
 exit 0
Note
  • see more documentation about mvsfiles5A/mvsfiles51 at CBVaids.htm

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_2 Test/Debug Tips & Techniques

Part 2 - Contents


2A1. Introduction/Overview

2B1. Test Environment, RUNLIBS & RUNDATA superdirs defined in your profile
- allows different programmers to have their own set of Libraries & Data
- best to have common set of Libraries, but different Data dirs
- aliases cdl/cdd allow you to switch easily between Libs/Data dirs
2B2.  JCL converted to scripts in Part_3 & executed in Part_4
- this Part_2 presents Tips & Techniques for testing & debugging

2C1. Use these tips for the Demo JCL/scripts or for your own JCL/scripts
See DATA file conversions in https://uvsoftware.ca/datacnv1.htm
to convert your data files & FTP data from Mainframe to Unix/Linux

2C2. Converting Your files - brief review of DATAcnv1.htm.
 Datafile Conversion Directories Required
 FTP data from Mainframe to Unix/Linux
Re-Naming files as required

2D1. run 'testdatainit' before JCL/scripts
- to clear output files & make it easier to see outputs of current test
- use 'joblog' to capture the log from your JCL/script
- subdirs cleared by testdatainit

2E1. Test/Debug for difficult JCL/scripts
- save all data1/* in datasave/ & load data1/ with files for difficult job
2F2.  Iterations of test/investigate/modify as required
- Animator/Debugger for Micro Focus COBOL
2F3.  Check results of test/debug, investigate output files (use uvhd if packed)
Print-outs using 'uvlp' scripts assist test/debug

2G1. Re-Creating the GDG control file - when new JCLs added with new GDG files
- alternate location for gdgctl51I in $APPSADSM/ctl vs default $RUNDATA/ctl

2G2. Editing the GDG control file with no of generations desired (vs default 7)
- modifying text file version & reloading Indexed file used by functions
  in JCL/scripts

2H1. GDG files & step Restart
- VU JCL conversions write new GDGs into a jobtmp subdir & restore to
  the data subdir only if JobEnd=Normal. This allows you to rerun scripts
  without worrying about any new GDGs created in the failing run.
- If there are steps prior to the failing step with updates in place,
  you can use "step Restart" at the failed step to prevent double updates.
2H2.  RERUN after failure without restart
2H3.  RERUN after failure WITH RESTART

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part 2 - Contents continued


2I1. jobstop & job restart
- activate stop at begin each step to examine files from 2nd screen

2J1. Activating 'console-logging' vs 'job-logging' for 1 JCL/script debug
- use console logging for long seesions running multiple jobs
- captures any operator inputs (job-logging does not)

2K1. uvcmp___ - file comparison for files with packed/binary &/or no LineFeeds
- unix 'diff' does not work for these types of files
- UV Software provides the 'uvcmp' utilities
- uvcopy jobs uvcmp1,2,3 & several scripts uvcmpFA1,uvcmpFE1,etc
  to make the uvcopy jobs easier to run.
- uvcmp prints mismatched record pairs in vertical hexadecimal
  flagging differences with '*'s, see sample report comparing
  2 generations of gl.account.master_000001 & _000002

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2A1. JCL/script Test/Debug Tips

Test/Debug Overview

Here are some tips that will help you to test/debug your JCL/scripts & COBOL programs in applications migrated from the mainframe.

Most jobs can be debugged by comparing the output files & sysout reports to the mainframe files & reports. Here are some tips for the more difficult cases. We assume that your profile has RUNDATA=$HOME/testdata (debugging in homedir), so you do not affect other programmers working. Your might also have your own set of libraries (RUNLIBS=$HOME/testlibs), OR RUNLIBS might point to a common set of libraries (JCL/scripts, COBOL programs, parms, quikjobs, etc).

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2B1. JCL/script Test/Debug Tips

Test Environment

For initial testing & self-education, you might setup testlibs, testdata,& cnvdata in your homedir. Assuming your login is userxx, here is the directory showing only the most essential subdirs. See all subdirs at JCLcnv1demo.htm#3B2.

 /home/userxx         <--- your homedir
 :-----testlibs        <-- $RUNLIBS - LIBRARIEs for JCL & COBOL
 :     :-----cbls        - COBOL program source
 :     :-----cblx        - compiled COBOL programs
 :     :-----jcls        - JCL/scripts (Korn shells converted from JCLs)
 :     :
 :-----testdata        <-- $RUNDATA - DATA directories & subdirs
 :     :-----data1       - datafiles
 :     :-----joblog      - programmer debug log files
 :     :-----jobtmp      - temp files for new GDGs, SYSINs, etc
 :     :-----sysout      - SYSOUT printer files (from COBOL DISPLAYs)
 :     :
 :-----cnvdata         <-- $CNVDATA - data CONVERSION superdir
 :     :-----d1ebc       - EBCDIC files from mainframe for conversion
 :     :-----d2asc       - files converted to ASCII
 :     :-----cpys        - COBOL copybooks
 :     :-----maps        - cobmaps (record layouts) generated from copybooks
 :     :-----pfx1        - uvcopy jobs convert EBCDIC to ASCII (gen from cobmaps)

Note that RUNLIBS, RUNDATA,& CNVDATA are defined in your stub_profile, before calling the common_profile (in /home/appsadm/env). See profiles listed at JCLcnv1demo.htm#1C1 & 1C2. The supplied stub_profile defines these variables as follows (for initial testing, will change for production).

      export RUNLIBS=$HOME/testlibs
      export RUNDATA=$HOME/testdata
      export CNVDATA=$HOME/cnvdata

This means that each programmer would have their own set of libraries & data, which is OK for early testing, especially if only 1 programmer on the project. For multi programmer teams, you would probably want to redefine RUNLIBS to point to a common set of libraries (not in homedirs), but you might well leave RUNDATA in $HOME/testdata, so programmers would not conflict each other.

#2. use 'cdl/cdd' easy change to testlibs/testdata

Aliases cdl/cdd/cdc are supplied (in the common_profile), to make it easy for you to quickly switch to testlibs/testdata/cnvdata as desired.

      alias cdl='cd $RUNLIBS'
      alias cdd='cd $RUNDATA'
      alias cdc='cd $CNVDATA'

These are even more useful after you change testlibs/testdata/cnvdata to common locations for the programmer team members - might have longer paths and you would not have to remember where.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2B2. JCL/script Test/Debug Tips

JCL & COBOL already converted

We assume you have already converted all COBOL & JCL as documented at at JCLcnv1demo.htm#3E1 & 3F1 using super scripts that convert all copybooks & COBOL programs, and all JCLs with PROCs & PARMs.


 cnvMF51Acpy all    <-- convert copybooks thru all steps
 =============      - cpy0 --> cpy1 --> cpy2 --> cpys
 cnvMF51A all     <-- convert COBOL programs thru all steps
 ============       - cbl0 --> cbl1 --> cbl2 --> cbls --> cblx

 jcl2ksh51A all   <-- convert ALL JCL thru all steps
 ==============       jcl0 --> jcl1 --> jcl2 --> jcl3

--- OR for AIX, the conversion scripts would be:


 cnvAIXcpyA all  <-- convert copybooks for AIX COBOL
 ===============
 cnvAIXcblA all   <-- convert COBOL programs for AIX COBOL
 ==============

 jcl2ksh53A all   <-- convert JCL for AIX COBOL
 ==============

The converted JCL/scripts are now in jcl3/..., but need to be copied to jcls/... for execution since only jcls is in $PATH (defined in the common_profile).


 cp jcl3/* jcls   <-- copy All scripts to execution subdir jcls/...
 ==============     - we will do this for our test/demos
                    - BUT do NOT copy ALL for your "real conversion"

 cp jcl3/jobnamex.ksh jcls  <-- copy 1 script at a time before test/debug
 =========================    - recommended for your "real conversion"

You should NOT copy all jcl3/* jcls/ for your "real conversion" project. For your real conversion project, you should copy 1 script at a time when you ready to start test/debug on each script. This is an easy way to keep track of your progress (jcls/... are debugged, jcl3/... not yet)

Another reason NOT to copy all jcl3/* jcls/ when JCL/scripts re-converted:

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2B3. Enhancing & Customizing Conversions

re-converting all JCL

 jcl0 ------> jcl1 ---------> jcl2 -----------> jcl3 ------> jcls
     cleanup     PROC expand    convert to ksh   copy 1 at a time for test/debug

 jcl2ksh51A all   <-- script to convert ALL JCL thru all steps
 ==============         jcl0 --> jcl1 --> jcl2 --> jcl3

You only need to rerun the 'jcl2 --> jcl3' step, using 'jclxx51' when the only changes are enhancements to JCL converter or changing options & control files.


 jclxx51 jcl2 jcl3  <-- script to reconvert jcl2/* --> jcl3/...
 =================

convert 1 JCL at a time

To convert 1 JCL at a time, use the 'jcl2ksh51' script (vs 'jcl2ksh51A').


 jcl2ksh51 jcl0/jobxx.jcl  <-- convert 1 JCL from jcl0->jcl1->jcl2->jcl3->jcls
 ========================

'jcl2ksh51' converts thru all stages to jcl3/... & prompts for copy to jcls/... It prompts before overwriting jcls/... in case you need to save any extensive editing you have already done on some steps of the JCL/script.

JCL conversions for AIX

JCL conversion scripts for AIX are coded with '53' vs '51' for Micro Focus.

converting All JCL for AIX


 jcl2ksh53A all   <-- script to convert ALL JCL thru all steps
 ==============         jcl0 --> jcl1 --> jcl2 --> jcl3

 jclxx53 jcl2 jcl3  <-- script to convert just the JCL jcl2/* --> jcl3/...
 =================    - omitting cleanups & proc expansion prior to JCL convert
                      - save time when only changes are enhancements to JCL
                        converter or changing options & control files

convert 1 JCL at a time for AIX


 jcl2ksh53 jcl0/jobxx.jcl  <-- convert 1 JCL from jcl0->jcl1->jcl2->jcl3->jcls
 ========================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2B4. Enhancing & Customizing Conversions

reasons for re-converting JCL

  1. You have transferred additional JCL to jcl0/... If only 1 or a few you can use 'jcl2ksh51' (vs jcl2ksh51A), but if more than a few, might be faster to use jcl2ksh51A.

  2. You have transferred additional PROCs or PARMs which were reported misisng on your prior conversion. Be sure to use jcl2ksh51A (not jcl2ksh51).

  3. You JCL converter has been enhanced (by UV Software), probably because you have reported a problem which is solved in the new version. This can save a lot of manual corrections if the problem is fixed in the early stages of test/debug - when you have only tested a few of your JCLs in jcls/... & the vast majority are yet to be debugged. This works well if you follow our recommendation to copy 1 at a time from jcl3/... (where auto-conversion stops) to jcls/... which is in $PATH.

  4. You have changed the JCL converter options. See example on paage '3G1' modifying option 'f0' to 'f1' to change default file type from 'typ=RSF' to 'typ=LST' for all uvsort/uvcp files.

  5. You have updated the DATA control files (file types, record sizes). See example on page '3H1' copy/update ctl/add/datactl53b changing desired files from 'typ=RSF' to 'typ=LSTt'

  6. You have obtained the LISTCAT reports from the mainframe & converted to control file ctl/datacat52I.dat/.idx for lookup by ctldata53 in jcldata51A.

    run the Cross-Reference reports

It is a good idea to run the cross-references now, because they can help to resolve various problems you may encounter during testing/debugging.


 xrefall cbls jcl3 ksh   <-- create all cross-ref reports
 =====================     - in subdir xref/...

 cobfiles5A cbls cpys maps <-- creates xref/cobfiles report
 =========================

See samples of cross-refs in JCLcnv3aids.htm#Part_1. Here are some of the most useful cross-references:

xkshparm1
  • list all SYSIN/PARM modules in each JCL/ksh/script
xkshparm2
  • shows all JCL/ksh/scripts referencing each SYSIN/PARM
xkshprog1
  • list all programs executed in each JCL/ksh/script
xkshprog2
  • shows all JCL/ksh/scripts executing each program
xcobcopy1
  • list all 'copy'books used in each program
xcobcopy2
  • shows all programs using each 'copy'book
xcobcall1
  • list all 'called'programs used in each program
xcobcall2
  • shows all programs calling each 'called'program
cobfil51
  • COBOL Files Report
  • information extracted from all COBOL programs in directory
  • filenames, Input/Output, Org, Access, record size, copybook
  • programmers should have a copy before starting test/debug

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2C1. JCL/script Test/Debug Tips & Techniques

Demo conversions or Your file conversions ?

For documentation, we are using testfiles supplied with Vancouver Utilities, but for your conversion, you will need to FTP files from the mainframe & convert as illustrated with the demo files in this documentation.

DATA file conversion vs JCL/COBOL conversion

 See comprehensive data file conversions in 'Part_3' of this JCLcnv2real.doc.
 For simpler data file conversions see https://uvsoftware.ca/datacnv1.htm
 - especially following parts:

https://uvsoftware.ca/datacnv1.htm#Part_3

https://uvsoftware.ca/datacnv1.htm#Part_6

FTP data from Mainframe to Unix/Linux

If you had no packed/binary fields you could FTP in TEXT MODE which automatically translates to ASCII, but the CR/LF makes records 2 bytes longer, & you either have to remove to match COBOL programs, or modify program SELECTs to 'ORGANIZATION LINE SEQUENTIAL' which allows for the CR/LF & also allows you to use unix system utilities (vi,more,lp,cat,etc) that required LineFeeds.

But most mainframes have a lot of files with packed/binary fields & it is probably best to FTP all files in BINARY MODE & use the VU conversion methods, which generate data conversion jobs automatically from the COBOL copybooks, to preserve the packed/binary fields (same for unix COBOLs as mainframe COBOLs). Note that unpacked signed fields require zoned sign conversion. Mainframe units byte positives are coded '{ABCDEFGHI' & negatives '}JKLMNOPQR'. Unix COBOL positives are coded just '0123456789' & negatives 'pqrstuvwxy'. The VU methods allow for this & an alternate version could be used with FTP text/ASCII files.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2C2. JCL/script Test/Debug Tips & Techniques

Converting Your files vs VU demo files ?

As explained above, we are using VU test/demo files for documentation, but for your conversion, you will need to FTP files from the mainframe & convert as documented in https://uvsoftware.ca/datacnv1.htm#Part_6. Here (on pages 2C2-2C4) is a brief overview of DATA file conversion, but you must use the detailed procedures in DATAcnv1.htm.

Datafile Conversion Directories Required

 /home/userxx       <-- your homedir
 :-----cnvdata1     <-- cnvdata superdir for DATA conversion
 :     :
 :     :-----cpys   <-- copybooks <--- copy from $UV/mvstest/cnvdata1/cpys/*
 :     :-----ctl      - control files
 :     :-----d0ebc  <-- EBCDIC data files <-- copy $UV/mvstest/cnvdata1/d0ebc/*
 :     :-----d1ebc    - EBCDIC data files renamed to VU conventions
 :     :-----d2asc    - ASCII data files (converted by uvcopy uvdata51)
 :     :-----d3ebc    - convert back to EBCDIC by uvdata31
 :     :-----d4pipe   - pipe delimited to load DB tables (by genpipe1)
 :     :-----maps     - copybooks with record layouts on right side
 :     :-----pfx1     - uvcopy jobs to convert EBCDIC to ASCII (gen by uvdata51)
 :     :-----pfx2     - with data filenames inserted (vs copybook names)
 :     :-----pfx3     - copied here for modify/execute
 :     :-----pfp1     - uvcopy jobs to create pipe delimited (gen by genpipe1)
 :     :-----pfp2     - pipe delim jobs with datafilenames inserted
 :     :-----pfp3     - copy here before editing for multi-record-type files
 :     :-----sqlTC1   - scripts to Create database tables (gen from copybooks)
 :     :-----sqlTL1   - scripts to Load database tables (gen from copybooks)
 :     :-----tmp      - temp files (keep working directory clean)

Generating ALL jobs to convert ALL datafiles EBCDIC to ASCII

 cpys-------->maps---------->pfx1-------------->pfx2----------->pfx3--------->
     cobmap1       uvdata51         uvdata52          copy/edit       execute
             convert copybooks  change copybooknames
               to uvcopy jobs     to datafilenames

Converting ALL data files EBCDIC to ASCII

               Variable-> fixed  BCDIC->ASCII          copy/rename
 MAINFRAME -----> d0ebc ------> d1ebc ----------> d2asc ---------> $RUNDATA/data1
    FTP/BINARY   ASCII to d2asc  pfx1/uvcopy jobs       cp/renameL
                 rename GDG files

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2C3. JCL/script Test/Debug Tips & Techniques

Here (pages 2C1-2C4) is a brief overview of how you would FTP & convert YOUR datafiles (vs demo files). You must use detailed procedures in DATAcnv1.htm.

FTP data from Mainframe - operating on Unix/Linux


 #1. Login --> your homedir
 #2. cdc   --> $HOME/$CNVDATA
 #3. cd d0ebc   <-- change into the data subdir

 #4. ftp xxx.xxx.xxx.xxx   <-- FTP to mainframe IP#
     ===================
 #4a. userid--> .....      <-- enter userid
 #4b. passwd--> .....      <-- enter password
 #4c. ascii                <-- translate EBCDIC to ASCII & insert CR/LF
      ---OR---                 (ascii is usually the default)
 #4c. binary               <-- ensure binary transfer
 #4d. quote site RDW       <-- required if any variable length files
 #4e. cd ...               <-- change to data files directory ??
 #4f. get 'XXX.XXX.XXX'    <-- get desired data files
 #4g. get '...etc....'       - 'single quotes' may be required

 #5. Might need to change permissions on files FTP'd from mainframe

 #5a. chmod 664 *          <-- files should be 664
      ===========

FTP data to Unix/Linux - via JCL on Mainframe

 //* JCL to FTP Variable-Length with RDW to Unix/Linux
 //ACHP01B   EXEC PGM=FTP,PARM='(TRAN TRTAUGE1'
 //OUTPUT    DD  SYSOUT=*
 //SYSPRINT  DD  SYSOUT=*
 //INPUT     DD  *
 30.252.99.11 21 (exit
 userid password
 cd /p2/cnv1/cnvdata/d0ebc/
 pwd
 type i
 LOCSITE RDW
 PUT 'AAAAA.BBBBB.CCC.DDDDD.X' AAAAA.BBBBB.CCC.DDDDD
 QUIT
 /*

See more detail at https://uvsoftware.ca/datacnv2.htm#5C1 - 5C6.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2C4. JCL/script Test/Debug Tips & Techniques

Re-Naming files as required

On some systems FTP'd files may be enclosed in single quotes, may be UPPER case, may have embedded blanks or '$' signs. ON unix we need to remove single quotes, remove any embedded blanks or '$' signs, & lower case is strongly recommended.


 #6. mv  \'XXX.XXX.XXX\'  xxx.xxx.xxx   <-- the hard way
     ================================     - see #7. the easy way

Imagine how awkward manual renaming via 'mv' commands would be if you have hundreds of files to be renamed. Vancouver Utilities has many 'rename' scripts to make mass changes to filenames easy.


 #7a. renameQQ data1  <-- easy way to remove Quotes from All files in subdir
      ==============
 #7b. renameL data1   <-- easy way to translate names to Lower case
      =============
 #7c. renameBU data1  <-- easy way to convert Blanks to '_' Underscores
      ==============
 #7d. renameDU data1  <-- easy way to convert '$' signs to '_' Underscores
      ==============

VU JCL/script filename standards

Renaming GDG files

After converting GDG files, you must rename them using the VU GDG file naming conventions - appending '_000001' for the 1st generation. For example:


 #8. cp $CNVDATA/d2asc/AR.CUSTOMER.MASTER(0) $RUNDATA/data1/ar.customer.master_000001
     ================================================================================
     - the HARD way of renaming mainframe GDG files to VU GDG standards

 #8a. renameGDG d1ebc  <-- script to rename all files in directory for VU GDGs
      ===============      (the EASY way)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2D1. JCL/script Test/Debug Tips

run 'testdatainit' before JCL/scripts

When you are testing, it is a good idea to run 'testdatainit', before you run the JCL/script being debugged. 'testdatainit' clears several subdirs, which makes it much easier to inspect the results.

Here are the testdata subdirs flagged with '0' to indicate subdirs that are cleared with testdatainit & '*' to indicate subdirs with your data files thet you would not want to clear.

 /home/userxx        <--- userid for testing OR your homedir
 :-----testdata          - data files superdir
 :     :--*--data1       - data files
 :     :--*--datasave    - backup, allow clear data1/.. to files for 1 job debug
 :     :--*--dataMF      - datafiles from MainFrame for comparison
 :     :--*--ctl         - GDG control file
 :     :--0--jobctl      - jobstop control files to debug JCL/script
 :     :--0--joblog      - programmer debug log files
 :     :--0--jobmsgs     - status msgs from JCL/scripts (step begin/end)
 :     :--0--jobtimes    - job/step times date stamped history files
 :     :--0--jobtmp      - temporary files for SYSIN data & GDG temp files
 :     :     :----JOBXX - jobtmp subdir created by jobset51 (script line 10)
 :     :     :    :-----GDG
 :     :     :    :     :-----data1 - new GDG files restored to data1/... at EOJ
 :     :--0--sysout      - SYSOUT printer files
 :     :     :----yymmdd - date stamped subdir for print files
 :     :     :    :-----JOBXX_S0010_SYSPRINT  - named by job+step+DDname
 :     :--0--tmp         - tmp subdir for uvsort & misc use
 :     :--0--tape1       - tape files become GDG files on disc

data file debug strategies

dataMF/...
  • datafiles from MainFrame for comparison
datasave/...
  • backup all data1/... files here before debug difficult job
  • then remove all data1/* & copyback only files for the debug job
data1/...
  • could init to only files required for difficult debug jobs

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2D2. JCL/script Test/Debug Tips

using 'testdatainit' & 'joblog'

1st run 'testdatainit' to clear output subdirs & make it easy to check results. Then run your JCL/script using 'joblog' which captures the log into the 'jblog' subdir for review if measages roll off he screen. Here is an example:


 #1. Login userxx --> $HOME (/home/userxx)
     ============

 #2. cdd --> $HOME/testdata
     ======================

 #3. testdatainit         <-- clear subdirs of prior run data
     ============           - also prompts to reload the GDG control file

 #4. joblog jobxx.ksh     <-- use joblog to run script to be tested
     ================       - saves console messages in joblog/...

The subdirs cleared are: jobctl, joblog, jobmsgs, jobtmp, sysout,& tmp. We will discuss in order of importance:

  1. jobtmp/ - jobset51 sets up a subdir within jobtmp jobtmp/JOBNAME/ - named using the JCL/script name in caps. jobtmp/JOBNAME/... - for temp files such as SYSIN data from instream jobtmp/JOBNAME/GDG/... - for new GDGs not restored to data1/ due to AbTerm

Note
  • new generations of GDG files are created in jobtmp/JOBNAME/GDG/...
    and restored to data1/... at Normal job end.
  • allows you to simply rerun an AbTerm job without worrying about GDGs
  • but you will get a prompt on rerun asking to restore or clear
  • running testdatainit will avoid this prompt, which is probably not
    relevant when testing, but would be important in produciton.
  1. sysout/ - jobset51 sets up a subdir within sysout sysout/JOBNAME/ - named using the JCL/script name in caps. sysout/JOBNAME/... - for output files created by COBOL program DISPLAYs sysout/JOBNAME/JOBNAME_STEP#_SYSOUT_yymmdd_HHMMSS sysout/JOBXX/JOBXX <-- example

Note
  • you can appreciate that runing 'testdatainit' before the JCL/script
    being debugged will make it much easier to inspect the files created
    from your job (vs output files from many previous jobs being debugged)
    llr sysout   <-- will show only new files from your latest run
                   - because testdatainit has cleared outputs of prior runs

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2D3. JCL/script Test/Debug Tips

subdirs cleared by testdatainit

  1. joblog/ - captures console messages from your JCL/script - already didscussed in Tip #3 above

  2. jobctl/ - control file for setting start & stop steps in multi-step jobs - coded on the command line following JCL/script name, for rexample:


               jobxx.ksh start=S0030 stop=S0030
               ================================
               - would run only step 3
               - omit the stop=... to run all remaining steps
               - would use to restart production jobs after failing step
  1. jobmsgs/ - captures only the most important console messages such as job start/end, step start/end,& error messages - not vital for testing since you see the console msgs & logs but more important for production job monitoring

  2. jobtimes/ - captures start/end/elapsed times for the job & each step. - not important for test/debug with ssmall files.

  3. tmp/ - used by uvsort for merge files on large sorts - we recommend you use the tmp/ subdir for any temporary files you might create while editing (cutting & pasting, etc).

Note
  • keep the working directory clean (subdirs only, no files)
  • files created by your JCL/scripts should be in subdirs (data,jobtmp,etc)
  • any files created in testdata/... by your jobs being tested indicate
    an error in the JCL/script file definitions (exportfile, exportgen0/1)
  • you will see the logical name (vs the physical filename) lost if
    the exportfile/exportgen command is misisng or incorrect
  • keeping testdata/ clean of files (intended subdirs only) means that
    any invalid files created by JCL/script dsiconnects will standout
    (like a sore thumb)

check new GDG files


 #7. llr jobtmp     <-- list all filenames in jobtmp/ such as GDG files
     ==========

 #8. l jobtmp/JOBXX/GDG/data1/  <-- list any GDG files left in jobtmp
     ========================
     - JobEnd=Normal moves new GDG files back to data1/... subdir
     - JobEnd=AbTerm leaves new GDG files in jobtmp/JOBXX/GDG/data1/...
       so you can fix problem & rerun without removing new GDGs from data1/...

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2E1. JCL/script Test/Debug Tips

test/debug for DIFFICULT jobs

We assume that you have been test/debugging many jobs & then find a difficult job that requires these exceptional procedures. The data1/... subdir could have hundreds of files from prior tests; we can save them to datasave/..., clear all data1/* & copyback only the files required for the difficult job being debugged.


 #1. Login userxx --> /home/userxx  (or whatever Login desired)

 #2. cdd --> $HOME/testdata
     ======================

 #3. mkdir datajobxx            <-- mkdir for files required in jobxx
     ===============                (problem job to be debugged)

 #3a. cp data1/fileA datajobxx/  <-- select files required for jobxx
      =========================
 #3b. cp data1/fileB datajobxx/          ... etc ...
      =========================

 #4. mkdir datasave            <-- make backup subdir for existing datafiles
     ==============                (if not already existing)

 #5. cp data1/* datasave        <-- backup all existing data1/* files
     ===================
  1. Test/Debug ---> see detailed steps on next page --->

  2. Restore all datafiles after test/debug difficult job


 #7a. cp datasave/* data1
      ===================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2E2. JCL/script Test/Debug Tips

test/debug ITERATION for problem job

This assumes that you have completed the preparations suggested on the previous pages, especially the backup of data1/* files to datasave/..., clear all data1/*, & copyback only the files required for the difficult job being debugged. You could now repeat the following steps for each cycle of test/debug that might be required for a difficult job.


 #1. cdd --> ensure we are in $RUNDATA

 #2. rm -f data1/*         <-- remove all files from data1/...
     =============

 #3. cp datajobxx/* data1  <-- store data1/ files for jobxx only
     ====================

 #4. testdatainit          <-- clear files from all temporary subdirs
     ============              (jobctl,joblog,jobmsgs,jobtmp,sysout,tmp,wrk)

activate ANIMATION (if desired)


 #5. export ANIM="+A"      <-- activate Micro Focus COBOL animator (debugger)
     ================

 #6. joblog jobxx.ksh      <-- test/debug jobxx
     ================

 #7. reply to animation prompts
     ==========================

The Micro Focus Animator is a marvelous debugging tool for COBOL programs. It stops at 1st instruciton & displays a menu of comamnds. Here are a few:

      --> s   <-- Step (execute current high-lighted statement)
                - for Perform: go into paragraph or section
      --> ps  <-- Perform Statement
                - do not animate paragraph/section (appears to step over it)
      --> qc  <-- Query the fieldname under the Cursor
                - move cursor (with arrow keys) to the desired fieldname
      --> lc  <-- Locate definition of fieldname under the Cursor
      --> g   <-- Go, steps continuously (but slowly) thru instructions
      --> 9   <-- (while Go is progressing), enter digit 2-9 to speed up
      --> z   <-- Zoom thru remainder of program (without animating)
      --> Esc <-- Escape to exit program
      --> y   <-- requires 'y' response to actually exit

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2E3. JCL/script Test/Debug Tips

check results


 #8. vi joblog/jobxx.log    <-- view the joblog
     ===================

 #9a. vi data1/...          <-- check data files with 'vi'
      ============            - IF LineFeeds present (no Packed fields)

 #9b. uvhd data1/...        <-- use 'uvhd' if no LF's
      ==============          - &/or if Packed/Binary fields present

 #10a. llr jobtmp               <-- list jobtmp files created
       ==========                 - intermediate files & new GDG's
                                    (new GDGs NOT restored to data1/ if Abterm)

 #10b. vi jobtmp/JOBXX/GDG/...  <-- investigate GDG files
       =======================

 #11a. llr sysout               <-- list sysout files created
       ==========                 - from COBOL 'DISPLAY's

 #11b. vi sysout/JOBXX/...      <-- investigate sysout files as desired
       ===================

 #11c. uvlp18 sysout/JOBXX/...  <-- print sysout files as desired
       =======================    - uvlp18 prints 18 cpi (132 cols on 8" wide)

 #11d. uvlp14L sysout/JOBXX/...   - use uvlp12L (Landscape) 14cpi 150 cols 11"
       ========================

modify JCL/scripts & COBOL programs


 #12. cdl --> $RUNLIBS      <-- switch to $RUNLIBS

 #13a. vi jcls/jobxx.ksh    <-- modify the JCL/script if required
       =================

 #14a. vi cbls/cblxx.ksh    <-- modify COBOL programs if required
       =================

 #14b. mfcbl1 cblxx.cbl     <-- recompile the COBOL program
       ================
 -------------> repeat steps 1 - 14 as required <---------------

 #15. cp datasave/* data1   <-- restore all data1/* files when jobxx debugged
      ===================

 #16. export ANIM=""        <-- de-activate Micro Focus COBOL animator
      ==============

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2E4. JCL/script Test/Debug Tips

Print-outs to help debug difficult jobs

Hard-copy listings can help you debug difficult jobs & document the problems for other team members. Vancouver Utilities provides several scripts (based on the uvlist utility documented at uvlist.htm. These scripts are great for JCL/script & program listings since the page headings capture vital information (filename, date, userid, page#, line# of 1st line on page,etc). Here are a few of the most useful scripts:

uvlp12
  • print portrait at 12 chars/inch
uvlp12D
  • print portrait & Duplex (for longer listings such as COBOL programs)
uvlp14
  • portrait at 14 cpi
uvlp14L
  • print Landscape at 14 cpi
etc

printing that may assist debug


 #1. uvlp14 joblog/jobxx.log  <-- print the joblog for better study
     =======================    - see 'uvlp' scripts in uvlist.htm

 #2. lslp data1     <-- print data1/... to study with jobxx.log
     ==========       - 'lslp' script does ls -l to tmp/ & uvlp12 tmp file

 #3. llr sysout     <-- display printer filenames in sysout/yyyymmdd/...
      ==========

 #4. uvlp14 sysout/yyyymmdd/JOBXX_S0010_DDname
     =========================================
     - print sysout files

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2G1. JCL/script Test/Debug Tips

Re-Creating the gdgctl file

If you have not yet edited gdgctl51 & reloaded gdgctl51I, then you could re-create gdgctl51 from all JCL by running 'uvcopy jclgdgctl51' separately from script 'jcl2ksh51A'. You might do this if you had added a few JCLs via jcl2ksh51 (vs jcl2ksh51A).


 #1. Login userxx --> /home/userxx
 #2. cdl --> /home/userxx/testlibs (alias cdl='cd $RUNLIBS')

 #3. uvcopy jclgdgctl51,fild1=jcl3,filo1=ctl/gdgctl51
     ================================================
     - extract all GDG file-names from converted JCL/scripts
NOTE
  • the control file must be copied to the $GDGCTL directory
  • GDGCTL defined in profile, either $RUNDATA or $APPSADM:
  • GDGCTL=$RUNDATA - for programmers with separate set of testdata
  • GDGCTL=$APPSADM - final testing/production common set of data

 #4. cp ctl/gdgctl51 $GDGCTL/ctl/    <-- copy to GDG control directory
     ============================      - GDGCTL $RUNDATA or $APPSADM

 #5. uvcopy gdgload51,fili1=$GDGCTL/gdgctl51,filo1=$GDGCTL/gdgctl51I
     ===============================================================
 #5a. uvcopy gdgload51  <-- same but easier (files default as shown above)
      ================

re-converting JCL

If you reconvert all JCL (using script 'jcl2ksh51A'), you will NOT overwrite the GDG control file & lose any manual edits to modify the number of generations, because the script tests for existing $GDGCTL/gdgctl51 & will not overwrite.

If you only needed to convert a few additional JCL's you could use script 'jcl2ksh51' which converts 1 at a time & does not recreate the GDG control file. You could then edit the existing gdgctl51 adding any new GDG files & reload the Indexed file by rerunning 'uvcopy gdgload51'.

Alternate location for GDG control file

The location of the GDG control file is determined by the common_profile (stored at /home/appsadm/env/common_profile & called by .profile or .bash_profile). Here are lines 61-62 of $APPSADM/env/common_profile.

 export GDGCTL=$RUNDATA/ctl       #<-- default location
 # export GDGCTL=$APPSADM/ctl     #<-- could change to this ?

If you have multiple RUNDATA directories you might want to activate 'export GDGCTL=$APPSADM/ctl' & #comment out 'export GDGCTL=$RUNDATA/ctl' so you would have only 1 GDG control file vs multiple.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2G2. JCL/COBOL executions using VU demo files

Editing the GDG control file

The GDG control file is 1st created as part of the 'jcl2ksh51A' script that converts all JCL to Korn shell scripts. GDG filenames are identified by suffixes (0),(+1),etc & are extracted from all JCL, sorted dropping duplicates,& loaded into an Indexed file for lookup when the JCL/scripts are executed - to determine latest & next generation,& also to determine how many generations to maintain. These lookups & determinations are performed by functions 'exportgen0' & 'exportgen1'.

Here is the GDG file created from the demo JCL/scripts listed in Part_2. This is the text file that can be edited to modify the number of generations before loading into the Indexed file 'gdgctl51I' (that is looked up by the exportgen0/exportgen1 functions in the JCL/scripts).

$RUNDATA/ctl/gdgctl51

 gl.account.acntlist_                    gdg=07
 gl.account.master_                      gdg=07
 gl.account.trans2_                      gdg=07
 gl.account.trans_                       gdg=07

modifying generations to maintain

As shown above the number of generations defaults to 7, You ca edit the file to set the generations as desired for each file. Note that the location of the GDG control file defaults to $RUNDATA/ctl/gdgctl51 but is configurable via "export GDGCTL=..." (discussed later).


 #1. Login userxx --> /home/userxx

 #2. cdd --> /home/userxx/testdata (alias='cd $RUNDATA)
     ===

 #3. vi ctl/gdgctl51   <-- edit to modify no of generations
     ===============

 #4. uvcopy gdgload51
     ================
     - reload Indexed file for use by exportgen1, exportgen0,& jobend51

I/O files are defaulted within the 'gdgload51' uvcopy job as follows:


 #4a. uvcopy gdgload51,fili1=$GDGCTL/gdgctl51,filo1=$GDGCTL/gdgctl51I
      ===============================================================

Since the common_profile defines "export GDGCTL=$RUNDATA/ctl", the result is:


 #4b. uvcopy gdgload51,fili1=$RUNDATA/ctl/gdgctl51,filo1=$RUNDATA/ctl/gdgctl51I
      =========================================================================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2H1. JCL/script Test/Debug Tips

GDG files & step Restart

The VU JCL conversion writes new GDGs into $RUNDATA/jobtmp/$JOBID2/GDG/data1/... and restores the new GDGs to $RUNDATA/data1/... at S9000 JobEnd=Normal. If any step fails, the job ends at S9900 JobEnd=AbTerm & any new GDG files in jobtmp/... are not restored to data1/...

The logic above means that you can simply rerun JCL/scripts with GDG files and not worry about any new GDGs created in the failing run. However you do need to know if there are any non-GDG files that are updated in place (vs updated by copying to a newfile). If there are updates in place prior to the failing step, then you can use "step Restart" to prevent double updates. This assumes it was not the failing step doing an update in place, in which case you would need to restore that file from a backup made before the job started.

We will demo new GDG file creation & step restart using JCL/script 'jgl200.ksh' (listed at JCLcnv1demo.htm#2D1 & see normal joblog at JCLcnv1demo.htm#4F4 Step1 sorts a transaction file to a GDG file which updates a GDG master file on the step2 COBOL program. We can cause 'AbTerm' by moving the COBOL executable from $RUNLIBS/cblx to tmp/... & return after failed run to demo recovery.


 #1. Login userxx --> /home/userxx  (or whatever Login desired)

 #2. cdl --> $HOME/testlibs (alias cdl='cd $HOME/testlibs)

 #3. mv cblx/cgl200.int tmp   <-- move step2 program out to tmp/...
     ======================     - to cause failure on step2

 #4. cdd --> $HOME/testdata

 #5. l data1    <-- list existing files in $RUNDATA/testdata/data1
     =======       - will show only files for jgl200.ksh
                   - only 1 generation of each file existing
 -rw-rw-r-- 1 userxx apps 13952 Oct 18 15:11 gl.account.master_000001
 -rw-rw-r-- 1 userxx apps  1600 Oct 18 15:11 gl.account.trans_000001

 #6. joblog jgl200.ksh     <-- execute the job (step2 will fail)
     =================       - joblog listed below (just step2 & AbTerm msgs)
 ******** Begin Step S0020 cgl200 (#2) ********
 gen+1 GLTRANS=jobtmp/JGL200/GDG/data1/gl.account.trans_000002 gens=07
 gen0 GLMSOLD=data1/gl.account.master_000001 insize=4.0K
 gen+1 GLMSNEW=jobtmp/JGL200/GDG/data1/gl.account.master_000002 gens=07
 Executing--> cobrun -F /home/userxx/testlibs/cblx/cgl200
 Load error : file '/home/userxx/testlibs/cblx/cgl200'
 error code: 173, pc=0, call=1, seg=0
 173     Called program file not found in drive/directory
 ERR: step#S0020 cgl200 abterm 255
 JobEnd=AbTerm, JCC=255,StepsX/L=2/S0020
 GDG files NOT moved from jobtmp/subdirs to /home/userxx/testdata/subdirs

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2H2. JCL/script Test/Debug Tips

RERUN after failure WITHOUT RESTART


 #7. cdl  --> $HOME/testlibs

 #8. mv tmp/cgl200.int cblx   <-- move step2 program back to execution library
     ======================

 #9. cdd --> $HOME/testdata

 #10. joblog jgl200.ksh   <-- re-execute the job, will get messages
      =================       about GDG files in jobtmp from prior failure
 WARN: files in jobtmp/GDG subdirs (from prior AbTerm ERR?)
 ---- files in jobtmp/JGL200/GDG/*/* listed below:
 jobtmp/JGL200/GDG/data1/gl.account.trans_000002
 If NO restart by step#, GDG files in jobtmp/... will be cleared
    - allows rerun from begin job with no worry about GDGs
 If RESTARTing by step#, example--> jobname.ksh start=S0050
    - GDG files in jobtmp/... will NOT be cleared
    - will be available to steps after restart step#
    - will be restored to data1/... subdir at JobEnd=Normal
 -->   <-- null entry to continue
         - - - lines removed to JobEnd=Normal - - -
 JobEnd=Normal, StepsExecuted=1, LastStep=S0020

 #11. l data1    <-- list existing files in $RUNDATA/testdata/data1
      =======       - will show only files for jgl200.ksh
                    - now 2 generations each
 -rw-rw-r-- 1 userxx apps 13952 Oct 18 15:11 gl.account.master_000001
 -rw-rw-r-- 1 userxx apps 13952 Nov 26 12:15 gl.account.master_000002
 -rw-rw-r-- 1 userxx apps  1600 Oct 18 15:11 gl.account.trans_000001
 -rw-rw-r-- 1 userxx apps  1600 Nov 26 12:15 gl.account.trans_000002

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2H3. JCL/script Test/Debug Tips

RERUN after failure WITH RESTART


 #10. joblog jgl200.ksh start=S0020  <-- re-execute job, with RESTART at step2
      ==================***********
      - will get same msgs re GDG files
      - BUT, notice the 'WARN: **START**' msg now appears
 WARN: files in jobtmp/GDG subdirs (from prior AbTerm ERR?)
 ---- files in jobtmp/JGL200/GDG/*/* listed below:
 jobtmp/JGL200/GDG/data1/gl.account.trans_000002
 If NO restart by step#, GDG files in jobtmp/... will be cleared
    - allows rerun from begin job with no worry about GDGs
 If RESTARTing by step#, example--> jobname.ksh start=S0050
    - GDG files in jobtmp/... will NOT be cleared
    - will be available to steps after restart step#
    - will be restored to data1/... subdir at JobEnd=Normal
 -->   <-- null entry to continue
 WARN: **START** at start=S0020
         - - - lines removed to JobEnd=Normal - - -
 JobEnd=Normal, StepsExecuted=1, LastStep=S0020

 #11. l data1     <-- list existing files in $RUNDATA/testdata/data1
      =======       - will show only files for jgl200.ksh
                    - still 2 generations each
 -rw-rw-r-- 1 userxx apps 13952 Oct 18 15:11 gl.account.master_000001
 -rw-rw-r-- 1 userxx apps 13952 Nov 26 12:15 gl.account.master_000002
 -rw-rw-r-- 1 userxx apps  1600 Oct 18 15:11 gl.account.trans_000001
 -rw-rw-r-- 1 userxx apps  1600 Nov 26 12:15 gl.account.trans_000002

note re rerun with or without restart

  1. Results of the 2 reruns (without restart on page '2H2' & with restart on this page '2H3') are the SAME.

  2. Without restart, GDG files from prior run (in jobtmp/...) are cleared, step#1 is rerun recreating the trans file, to update master file on step#2.

  3. With restart, GDG files from prior run are NOT cleared, step#1 is skipped but trans file from prior run is used to update master file on step#2.

  4. So results same in this case, but you would need to use restart if steps prior to failing step have updated any files in place (vs copy to new output file).

See complete documentation re GDG files at JCLcnv4gdg.htm.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2I1. JCL/script Test/Debug Tips

jobstop - pause at each step

The JCL converter inserts a call to function 'stepctl51' st the begining of each step. This function normally does nothing, but can be activated by 'jobstop' to stop at the begining of each step (or a specified step) & wait for the operator to enter 'go' or 'clear'.


 stepctl51  # test oprtr jobstop/jobclear
 #========  # inserted at begining of each step by JCL converter option 'b2'

Pausing at the begining of each step allows you to investigate the files in data1/... & jobtmp/JOBXX/GDG/data1/... that might be deleted at the end of the step. Also see page '5B1' which suggested you might use the JCL converter option to #comment out 'rm's (converted from original 'DELETE's) & then later uncomment with mass change scripts.

execute job with jobstop activated


 #1. Login --> /home/userxx
 #2. cdd --> $HOME/testdata

---> instructions omitted here (see page '2H2')


 #6a. jobstop jobxx.ksh   <-- activate stop at begin each step
      =================     - stores control record in jobctl/jobxx.ctl

 #6b. joblog jobxx.ksh    <-- execute the job
      ================      - will get following display at begin each step
   --> jobxx.ksh paused by job control file: jobctl/jobxx.ctl
       - job control record: jobxx.ksh S0000 111125_124512
       - waiting until reply 'go' or 'clear'

 #7. Login on a 2nd screen --> $HOME
 #8. cdd --> $HOME/testdata

 #9a. l data1     <-- investigate data files on screen#2
      =======       - while jobxx paused on screen#1

 #9b. l jobtmp/JOBXX/GDG/data1/  <-- investigate GDG files in jobtmp
      =========================

option to specify stop step

The 'jobstop' script may also specify the 1st step# to stop (art end of step)


 #6a. jobstop jobxx.ksh stop=S0030 <-- store jobctl file with stop step#
      ============================

 #6b. joblog jobxx.ksh    <-- execute the job
      ================       - will stop at end of step S0030
                               & subsequent steps if you reply 'cont'inue

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2I2. JCL/script Test/Debug Tips

job restart & job stop

As of Feb 2012, the 'job restart' facility has been enhanced to include the 'job stop' facility. This means you no longer have to precede your job execute (with or without joblog) by the 'jobstop' script to store the jobctl/file. You can restart a job at a specified step# & stop it at the END of a step#, Here are some examples:


 #1. jgl310.ksh         <-- runs all 6 steps of this demo job
     ==========

 #1a. joblog jgl310.ksh  <-- same with joblogging
      =================    - will omit 'joblog' from following examples
                           - BUT you should use joblog for test/debug

 #2. jgl310.ksh start=S0030  <-- rerun job starting at step 3
     ======================    - will run to end of job with no stops

 #3. jgl310.ksh start=S0030 stop=S0040
     =================================
     - rerun job starting at step 3 & stopping at END of step 4
     - displays following choices:
 120210:155447:JGL310: jgl310.ksh paused by job control file: jobctl/jgl310.ctl
 120210:155447:JGL310: - job control record: jgl310.ksh S0030 120210_155447
 120210:155447:JGL310: - waiting for reply: cont(inue), clear, endok, endab

cont - continue (execute) to next step, stop again,& reprompt as above clear - clear the jobctl/jgl310.ctl file, will execute to end job with no stops endok - goto Normal end of job, will restore any new GDG's endab - goto AbNormal end of job, will NOT restore any new GDG's

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2J1. JCL/script Test/Debug Tips

Activating Console-Logging

If desired you could activate console-logging & rerun the demo jobs.

Console-Logging will capture the entire login session from login to logout vs the 'joblog' script intended for programmer test/debug to capture the log for 1 job at a time.

Note that console-logging captures all screen I/O including any reponses to prompts & commands the operator might do between running jobs.

See complete details of console logging at ADMjobs.htm#Part_6 but here is the short version of activating. ADMjobs.htm#6D1 describes the subdirs required in /home/appsadm where logs are collected. We will assume that you are 'userxx'.


 #1. login --> your homedir

 #2a. mkdir $APPSADM/log1/userxx  <-- subdir current log being created
 #2b. mkdir $APPSADM/log2/userxx  <-- subdir current months processed logs
 #2c. mkdir $APPSADM/log3/userxx  <-- subdir last months logs

 #3. vi .profile     <-- edit your profile
     ===========
 #3a. uncomment the 9 '##' lines near the end of your profile
      - see profile listing on page JCLcnv1demo.htm#1C1
 #3b. :wq

 #4. Logout & Log back in to start console logging

 #5. Run some jobs (jar100.ksh for example)
 #5a. cdd          - change to $RUNDATA directory
 #5b. l            - list subdirs in $RUNDATA
 #5c. jar100.ksh   - run 1st job (COBOL program car100.cbl)
 #5d. l data1      - list data subdir

 #6. Logout/Login to process the log
     - copies from $APPSADM/log1/userxx/date_time to $APPSADM/log2/userxx/.
       removing screen control codes that would make log unreadable

 #7. logview    <-- script lists your available log filenames
     =======      - prompts for file# to view (#1 is latest)
 #7a. --> 1 <-- enter '1' to see latest log
 #7b. --> 0 <-- enter '0' to quit

 #8. l $APPSADM/log2/userxx/    <-- can list your log files directly
     =======================        vs logview script

 #8a. vi $APPSADM/log2/userxx/yymmdd_HHMMSS <-- inspect latest log file
      =====================================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2J2. JCL/script Test/Debug Tips

sample console log

'job-logging' is good for test/debug (captures log for 1 job at a time), but 'console-logging' is better for production, because it captures everything, inputs as well as outputs, what commands the operator did between jobs. Here is a sample - I have numbered the operator's commands on right side:


 #1. cdd        <-- change to $RUNDATA
 #2. l          <-- list files in $RUNDATA
 #3. l data1    <-- list files in data1/ (before running job)
 #4. jar100.ksh <-- run JCL/script
 #5. l data1    <-- list files after running job
 <@uvsoft4:userxx:/home/userxx> cdd                             <-- #1.
 <@uvsoft4:userxx:/home/userxx/testdata> l                      <-- #2.
 drwxrwxr-x 2 userxx apps 4096 Oct 17 10:49 ctl
 drwxrwxr-x 2 userxx apps 4096 Oct 17 09:48 data1   <-- data files subdir
 drwxrwxr-x 2 userxx apps 4096 Oct 17 09:27 joblog
 drwxrwxr-x 2 userxx apps 4096 Oct 17 09:27 jobmsgs
 drwxrwxr-x 2 userxx apps 4096 Oct 17 09:27 jobtmp
 drwxrwxr-x 2 userxx apps 4096 Oct 17 09:27 sysout
 drwxrwxr-x 2 userxx apps 4096 Oct 17 09:27 tmp
 drwxrwxr-x 2 userxx apps 4096 Oct 17 09:27 wrk
 <@uvsoft4:userxx:/home/userxx/testdata> l data1                <-- #3.
 -rw-rw-r-- 1 userxx apps  8192 Oct 17 09:48 ar.customer.master
 -rw-rw-r-- 1 userxx apps  1280 Oct 17 09:48 ar.sales.items
 -rw-rw-r-- 1 userxx apps 13952 Oct 17 09:48 gl.account.master_000001
 -rw-rw-r-- 1 userxx apps  1600 Oct 17 09:48 gl.account.trans_000001
 <@uvsoft4:userxx:/home/userxx/testdata> jar100.ksh             <-- #4.
                                         ==========
 111017:105233:JAR100: Begin Job=JAR100
 111017:105233:JAR100: /home/userxx/testlibs/jcls/jar100.ksh
 111017:105233:JAR100: RUNLIBS=/home/userxx/testlibs
 111017:105233:JAR100: RUNDATA=/home/userxx/testdata
 111017:105233:JAR100: JTMP=jobtmp/JAR100 SYOT=sysout/JAR100
 111017:105233:JAR100: RUNDATE=20111017
 111017:105233:JAR100: ******** Begin Step S0010 car100 (#1) ********
 111017:105233:JAR100: file: CUSTMAS=data1/ar.customer.master fsize=8.0K
 111017:105233:JAR100: file: NALIST=data1/ar.customer.nameadrs.list100 fsize=
 111017:105233:JAR100: file: SYSOUT=sysout/JAR100/S0010_SYSOUT fsize=
 111017:105233:JAR100: Executing--> cobrun -F /home/userxx/testlibs/cblx/car100
 111017:105233:JAR100: Job Times: Begun=10:52:33 End=10:52:33Elapsed=00:00:00
 111017:105233:JAR100: EOF filr01 rds=3 size=6144: /home/userxx/testdata/ctl/gdgctl51I
 111017:105233:JAR100: JobEnd=Normal, StepsExecuted=1, LastStep=S0010
 <@uvsoft4:userxx:/home/userxx/testdata> l data1                <-- #5.
 -rw-rw-r-- 1 userxx apps  8192 Oct 17 09:48 ar.customer.master
 -rw-rw-r-- 1 userxx apps  2858 Oct 17 10:52 ar.customer.nameadrs.list100
 -rw-rw-r-- 1 userxx apps  1280 Oct 17 09:48 ar.sales.items
 -rw-rw-r-- 1 userxx apps 13952 Oct 17 09:48 gl.account.master_000001
 -rw-rw-r-- 1 userxx apps  1600 Oct 17 09:48 gl.account.trans_000001

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2K1. JCL/script Test/Debug Tips

uvcmp... data file comparison

'diff' is a marvelous utility, but does not work for files without lineFeeds & with packed/binary fields present. For these files UV Software provides the 'uvcmp' utilities - uvcopy jobs uvcmp1,2,3 & several scripts uvcmpFA1, uvcmpFE1, etc to make the uvcopy jobs easier to run.

Here is an example using JCL/script 'jgl200', listed at JCLcnv1demo.htm#2D1 & executed on page 4F4). jgl200 updates the account.master file with a transaction file while copying to a new output file. Since the account.master file is a GDG file, it is easy to compare the old & new master files without having to save the input file.


 #1. Login mvstest1 --> /home/mvstest1
 #1. or Login yourself (userxx) --> your homedir (/home/userxx)

 #2. cdd --> $HOME/testdata (alias cdd='cd $HOME/testdata)

 #3. joblog jgl200.ksh     <-- execute account.master update
     =================       - see joblog at JCLcnv1demo.htm#4F4

 #4. l data1        <-- list data1/... subdir after execution
     =======
      -rw-rw-r-- 1 userxx apps 13952 Oct 18 15:11 gl.account.master_000001
      -rw-rw-r-- 1 userxx apps 13952 Dec 13 11:31 gl.account.master_000002
      -rw-rw-r-- 1 userxx apps  1600 Oct 18 15:11 gl.account.tran1
      -rw-rw-r-- 1 userxx apps  1600 Oct 18 15:11 gl.account.trans_000001

 #5. mkdir rptcmp    <-- make subdir for comparison report
     ============        (if not already present)

 #6. uvcmpFA1 data1/gl.account.master_000001 data1/gl.account.master_000002 r128
     ===========================================================================
     - compare the before & after update files, 'r128' specifies the record-size
     - may omit option 'r' on the command line & specify when prompted
     - writes difference report to rptcmp/gl.account.master_000001
       (ie - named same as 1st file compared)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2K2. Mainframe Conversion Test/Debug Aids

demo uvcmpFA1 - file compare


 #7. vi rptcmp/gl.account.master_000001   <-- inspect difference report
     ==================================     - 1st 2 record pairs listed below:
 uvcmp1 - compare 2 files, print mismatched records, '*' flag diffs
 2011/12/13_11:32:34 uop=q1p30r256s6t500000u3x2y0q1r128
 recsize  reccount   file-size  typ Report=rptcmp/gl.account.master_000001
 1:  128       109      13,952  RSF  File1=data1/gl.account.master_000001
 2:  128       109      13,952  RSF  File2=data1/gl.account.master_000002
                       1         2         3         4         5         6
 f#record#byte# 0123456789012345678901234567890123456789012345678901234567890123
 ===============================================================================
 1       1    0  11100       11100Royal Bank Lynn Valley        ....b.20090131
                2333332222222333335676624666247662566667222222220010683333333322
                0111000000000111002F91C021EB0C9EE061CC590000000000972C2009013100
                                                                        ******
 2       1       11100       11100Royal Bank Lynn Valley        ....b.20111213
                2333332222222333335676624666247662566667222222220010683333333322
                0111000000000111002F91C021EB0C9EE061CC590000000000972C2011121300
                                                                        ******
                           - - - 5 record pairs omitted - - -
 ==================== EOF or StopPrint/StopRead count reached  ==============
 F1Count=109, F2Count=109, StopPrint=6, StopRead=500000
 F1Reads=109, MisMatches=109, MisMatsPrinted=6, Recsize=128

Notes re file compare

  1. 'uvcmpFA1' compares 2 ASCII files. See 'uvcmpFE1' to compare EBCDIC files. These assume record pairs & do not allow for inserted or deleted records. See uvcmpFAI2 & uvcmpFEI2 to allow for inserted & deleted records. See complete 'uvcmp' documentation at CMPjobs.htm.

  2. Mismatched record pairs are shown in vertical hexadecimal & differences highlight with '*'s. As you can see from the EOF statistics/counts all 109 records mismatch because the run date is inserted in all records.

  3. The StopPrint option default is 6, but I have removed the other 5 pairs to save space.

  4. The record size is 128, but only the 1st 64 bytes are shown because there was no difference in the last 64 byte segment. This feature dramatically reduces the size of difference reports when you have large records.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_3 Updating control-files to improve JCL conversions

Part_3 - Contents


3A1. Control-Files for JCL conversion to supply data-file-info
- that may be missing in JCL (record-sizes,file-types,etc)
- required for script conversions of SORTs,IDCAMs,IEBGNENERs,etc

3B1. 1st conversion of COBOL & JCL (before updating control files)
- operating instrucitons (review, 1st presented at '3D1', 3E1, 3F1)

3B2. control-files before 1st conversion
- summary listing

3B3. capturing COBOL info for JCL conversion (record-sizes, file-types)

3B4. control files created by 1st conversion (in subdir ctl/...)
- datajcl51, datajcl52 extracted from the JCL

3B5. JCL converter script 'jcl2ksh51A' does everything
- calls 'jcldata51A' to create datajcl51 & datajcl52
  & load into Indexed file datactl53I.dat/.idx for JCL converter
- calls 'jclxx51' for JCL conversion to Korn shell scripts
- updates datactl53I with datafile info from COBOL conversions

3B6. datactl53I control file before & after update by 1st JCL conversion

3C1. using LISTCAT mainframe reports to improve JCL conversion
- transfer, translate, store multiple reports in subdir cat0/...

3C2. LISTCAT report sample (mock-up for VU demo files)

3C3. control files extracted from LISTCAT reports
- processed to subdirs cat1/..., cat2/...
- summarized to 1 report in ctl/datacat51
- loaded into Indexed file ctl/datacat52I.dat/.idx
- merged into ctl/datactl53I.dat/.idx by jcldata51A

3D1. Problem - record sizes missing from LISTCAT report
Solution - an extra LRECL report from mainframe to be converted
 to keyword format & combined with other control files

3E1. manual coded control file for missing record-sizes,file-types,etc

3E2. converting Excel spreadsheets of datafile info to control files

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_3 - Contents continued


3F1. Re-Converting JCL with LISTCAT, datamisc,& dataxls
- using 'alldiff2' script to see differences made by re-conversion
- confirm chages as expected & no unintended changes

3F2. dropping unwanted lines on alldiff2 report with vi & diffdrop2
3F3. - alldiff2 report before & after vi & diffdrop2

3F4. re-convert alternative to save time if hundreds of JCLs,PROCs,PARMs
- use jcldata51A & jclxx51 (vs do everything jcl2ksh51A)

3F5. JCL converter control file ctl/datactl53I
- before & after 2nd conversion

3G1. Modifying JCL converter 'options'
- example to change fixed length data files to Line Sequential
- if you have very few files with packed/binary fields

3H1. Modifying JCL converter 'data info control files' & reconverting
- example to correct or supply missing record-sizes & file-types

3I1. JCL converter control file summary

3J1. setup alternate sets of conversion directories
- selected sets of JCL/COBOL for testing, education, etc

3K1. Determine Essential INPUT files for any 1 JCL/script
 script 'mvsfiles51' selects essential input files
- ignoring output files & intermediate created/used within JCL

3L1. Inserting commands to print files
- using uvlist & 'uvlp' scripts to insert PCL5 for laser printers
- uvlp13LD to get mainframe 132 cols 66 lines on 8 1/2 x 11
  (Landscape & Duplex)
 JCLcnv2real.htm#4J1 JCL converter 'options' control file (ctl/jclunixop51)
      - copied from $UV/ctl/... to $RUNLIBS/ctl/...  by 'copymvsctls'
        after initial setup of conversion directories by 'mvslibsdirs'

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3A1. Updating control-files to improve JCL conversions

Directories & Control-Files

Here are the more relevant subdirs we will use to illustrate customizing JCL conversions by modifying control files & re-converting. See the jclunixop51 JCL converter options control file at JCLcnv2real.htm#4J1.

 /home/uvadm/mvstest/testlibs  <-- test/demo libraries supplied in /home/uvadm/...
 /home/userxx          <-- can copy to your homedir
 :-----testlibs
 :     :------cat0        - LISTCAT reports from mainframe (see '3C1')
 :     :------cat1        - LISTCAT reports converted to unix control files
 :     :------cat2        - LISTCAT reports translated to lower case
 :     :------cbl0        - mainframe COBOL programs
 :     :------cbl1/2      - intermediate subdirs used in conversion
 :     :------cbls        - converted COBOL programs
 :     :------cblx        - compiled COBOL programs
 :     :------cpy0        - mainframe COBOL copybooks
 :     :------cpy1/2      - intermediate subdirs used in conversion
 :     :------cpys        - copy here (standard copybook library)
 :     :------ctl         - conversion control files
 :     :      :---add/...        <-- subdir for ADDITIONAL data control files
 :     :      :-----add/datactl53a <-- saved by JCL converter
 :     :      :-----add/datactl53b <-- may copy a to b & update file types
 :     :      :                       rec-sizes, etc for subsequent conversions
 :     :      :-----add/datamisc51 <-- add info missing on 1st convert
 :     :      :-----add/dataxls51  <-- add info from Excel spreadsheet
 :     :      :---datacat51      <-- DATA file info from LISTCAT files
 :     :      :---datactl53I     <-- all DATA file info merged to Indexed file
 :     :      :---gdgctl51       <-- GDG datafile generations to maintain
 :     :      :---jclunixop51    <-- control file/options for JCL converter
 :     :------jcl0        - mainframe JCL
 :     :------jcl1        - intermediate conversion 73-80 cleared
 :     :------jcl2        - PROCs expanded from procs
 :     :------jcl3        - JCLs converted to Korn shell scripts
 :     :------jcls        - copy here manually 1 by 1 during test/debug
 :     :------parm0       - control cards & includes (SORT FIELDS, etc)
 :     :------parms       - control cards with 73-80 cleared
 :     :------proc0       - test/demo PROCs supplied
 :     :------procs       - will be merged with jcl1, output to jcl2
 :     :------tmp         - tmp subdir used by uvsort & various conversions
 :     :------xref        - cross-references

You can modify various options in ctl/jclunixop51 & re-convert. The complete conversion of all JCL components can be pictured as follows:

 jcl0 ------> jcl1 ---------> jcl2 -----------> jcl3 ------> jcls
     cleanup     PROC expand    convert to ksh   copy 1 at a time for test/debug

 jcl2ksh51A all   <-- script to convert ALL JCL thru all steps
 ==============         jcl0 --> jcl1 --> jcl2 --> jcl3

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3A2. Updating control-files to improve JCL conversions

Control-Files

Here are the control files after 'copymvsctls' (see '3D1') has copied initial control files from $UV/ctl/... before the 1st COBOL & JCL conversion. See copymvsctls listed at JCLcnv2real.htm#4C0

 /home/userxx/testlibs/
 :-----ctl/
 :     :----add/
 :     :    :-----dummy_readme
 :     :    :-----add/datamisc51 <-- add info missing on 1st convert
 :     :    :-----add/dataxls51  <-- add info from Excel spreadsheet
 :     :-----cnvcob5.tbl         <-- COBOL conversion search/replace table
 :     :-----cobdirectives       <-- Micro Focus COBOL compile Directives
 :  0  :-----cobfil55bI.dat      <-- COBOL datafile info (initially empty)
 :     :-----cobfil55bI.idx
 :  0  :-----datacat52I.dat      <-- LISTCAT datafile info (initially empty)
 :     :-----datacat52I.idx
 :     :-----datamisc51          <-- you can edit with misisng record sizes
 :     :-----dataxls51.psv       <-- file of record sizes from Excel spread sheet
 :     :-----extfh.cfg           <-- Micro Focus COBOL Extended File Handler config
 :     :-----jclunixop51         <-- JCL converter control-file
 :     :-----utilities

Here are the control files after the 1st conversion - with comments on right only for files changed by 1st conversion.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3A2. Updating control-files to improve JCL conversions

 /home/userxx/testlibs/
 :-----ctl/
 :     :-----add/
 :     :     :-----datactl53a    <-- copied/renamed from ../datactl53I.dat
 :     :     :-----datactl53b    <-- can copy/rename/modify recsizes & filetypes
 :     :     :-----dummy_readme
 :     :-----cnvcob5.tbl
 :     :-----cobdirectives
 :     :-----cobfil55a           <-- COBOL datafile info stored by cnvMF51A
 :     :-----cobfil55aI.dat        - to Identify ORG Line Seqntl (print) files
 :     :-----cobfil55aI.idx
 :     :-----cobfil55b           <-- COBOL datafile info stored by cnvMF51A
 :     :-----cobfil55bI.dat        - to supply record-sizes to JCL converter
 :     :-----cobfil55bI.idx
 :     :-----dataadd51           <-- datafile info from all ctlfiles in add/...
 :     :-----dataadd52I.dat        - added to datactl53I used by JCL converter
 :     :-----dataadd52I.idx
 :     :-----datacat51           <-- datafile info from LISTCAT cat2/...
 :     :-----datacat52I.dat        - added to datactl53I used by JCL converter
 :     :-----datacat52I.idx
 :     :-----datactl53           <-- datafile info for JCL converter (sequential)
 :     :-----datactl53I.dat        - Indexed file version used by JCL converter
 :     :-----datactl53I.idx
 :     :-----datamisc51          <-- you can edit with misisng record sizes
 :     :-----dataxls51.psv       <-- file of record sizes from Excel spread sheet
 :     :-----datajcl51           <-- datafile info extracted from all jcl2/...
 :     :-----datajcl52             - 1st input to create datactl53 for JCL converter
 :     :-----extfh.cfg
 :     :-----gdgctl51            <-- initial GDG control file created by jcl2ksh51A
 :     :                        - copy to $RUNDATA/ctl, modify no of generations
 :     :                          & load Indexed file for JCL/script executions
 :     :-----jclunixop51
 :     :-----utilities

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3B1. Updating control-files to improve JCL conversions

1st conversion of COBOL & JCL

We have already performed the 1st conversion as documented in JCLcnv1demo.htm Here is a brief review of the setup & conversion scripts (showing scripts for Micro Focus COBOL & AIX equivalents)


 #1. login yourself --> /home/userxx

 #2. mkdir testlibs   <-- make superdir for JCL/COBOL subdirs

 #3. cdl     <-- alias cdl='cd $RUNLIBS' --> cd testlibs

 #4. mvslibsdirs  <-- setup 30 subdirs for JCL & COBOL conversions
     ===========

 #5. copymvsctls  <-- script to copy control files from /home/uvadm/ctl
      ===========     to $RUNLIBS/ctl/...
                    - see copymvsctls listed at JCLcnv2real.htm#4C0
Note
  • see initial control files listed on page '3A2'
  • may modify depending on your site & conversion desires

 #6. cnvMF51Acpy all    <-- convert copybooks thru all steps
     =============      - cpy0 --> cpy1 --> cpy2 --> cpys

 #7. cnvMF51A all     <-- convert COBOL programs thru all steps
     ============       - cbl0 --> cbl1 --> cbl2 --> cbls --> cblx

 #8. jcl2ksh51A all   <-- convert ALL JCL thru all steps
     ==============       proc0 --> procs, parm0 --> parms
                          jcl0 --> jcl1 --> jcl2 --> jcl3

 #8a. jcldata51A      <-- create ctl/datactl53 control file
      ==========        - to convert JCL to scripts, jcl2 --> jcl3
Note
  • no need to run 'jcldata51A' separately for 1st converion
  • because it is called by jcl2ksh51A after procs & parms cleanup
    & before the JCL to script conversion jcl2 --> jcl3
  • we might run separately for re-conversions.

 #9. cp jcl3/* jcls   <-- copy converted JCL/scripts to jcls (in $PATH)
     ==============     - only for our demos, not for your conversions
                          (will copy 1 at a time as you test/debug)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3B2. Updating control-files to improve JCL conversions

ctl/files after mvscopyctls & BEFORE 1st conversion

 /home/userxx/testlibs
 :---ctl/
 :   :---add/
 :      245  dummy_readme
 :---ctl/
 : 4612  cnvcob5.tbl       <-- COBOL conversion search/replace table
 : 2238  cobdirectives     <-- Micro Focus COBOL compile Directives
 :    0  cobfil55bI.dat    <-- COBOL datafile info (initially empty)
 : 3072  cobfil55bI.idx
 :    0  datacat52I.dat    <-- LISTCAT datafile info (initially empty)
 : 3072  datacat52I.idx
 :  917  datamisc51          <-- you can edit with misisng record sizes
 : 1334  dataxls51.psv       <-- file of record sizes from Excel spread sheet
 : 2586  extfh.cfg         <-- Micro Focus COBOL Extended File Handler config
 :20055  jclunixop51       <-- JCL converter control-file
 :  662  utilities

control-files AFTER 1st conversion

 /home/userxx/testlibs
 :---ctl/
 :   :---add/
 :      245  dummy_readme
 :     1141  datamisc51     <-- add info missing on 1st convert
 :     1598  dataxls51      <-- add info from Excel spreadsheet
 :     2496  datactl53a     <-- copied/renamed from ../datactl53I.dat
 :---ctl/
 : 4612  cnvcob5.tbl
 : 2238  cobdirectives
 : 1292  cobfil55a         <-- COBOL datafile info stored by cnvMF51A
 : 1280  cobfil55aI.dat      - to Identify ORG Line Seqntl (print) files
 : 3072  cobfil55aI.idx
 : 1292  cobfil55b         <-- COBOL datafile info stored by cnvMF51A
 : 1280  cobfil55bI.dat      - to supply record-sizes to JCL converter
 : 3072  cobfil55bI.idx
 :    0  dataadd51         <-- datafile info from all ctlfiles in add/...
 :    0  dataadd52I.dat      - info added to datactl53I (initially empty)
 : 3072  dataadd52I.idx
 :    0  datacat52         <-- datafile info from LISTCAT cat2/...
 :    0  datacat52I.dat      - info added to datactl53I (initially empty)
 : 3072  datacat52I.idx
 : 1846  datactl53         <-- datafile info for JCL converter (sequential)
 : 2496  datactl53I.dat      - Indexed file version used by JCL converter
 : 3072  datactl53I.idx
 : 2074  datajcl51         <-- datafile info extracted from all jcl2/...
 : 2074  datajcl52           - 1st input to create datactl53 for JCL converter
 : 2586  extfh.cfg
 :  207  gdgctl51          <-- initial GDG control file created by jcl2ksh51A
                             - copy to $RUNDATA/ctl, modify no of generations
                               & load Indexed file for JCL/script executions
 :20055  jclunixop51
 :  662  utilities

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3B3. Updating control-files to improve JCL conversions

capturing COBOL info for JCL conversion

You must perform COBOL conversion prior to JCL conversion, because it stores control file ctl/cobfil55b with record size info for the JCL converter. The COBOL conversion was performed by a do everything script 'cnvMF51A' which was 1st documented on page '3E1' & most recently on page '3B1'.


 cnvMF51Acpy all     <-- convert COBOL copybooks thru all steps
 =============       - cpy0 --> cpy1 --> cpy2 --> cpys

 cnvMF51A all     <-- convert COBOL programs thru all steps
 ============       - cbl0 --> cbl1 --> cbl2 --> cbls --> cblx
 'cnvMF51A'  - includes following 6 steps:
             - see listing at JCLcnv2real.htm#4B2
  1. cleanup cbl0 --> cbl1
  2. cobfil55 - extract data file info (record-sizes,file-types,etc)
  3. uvsort - sort & load Indexed file cobfil55aI for JCL converter
  4. uvsort - sort & load Indexed file cobfil55bI for JCL converter - cobfil55bI key is program-name & DD-name so JCL converter can lookup to get datafile info JCL converter also updates datactl53I with this info
  5. cnvMF5 - actual COBOL conversion cbl1 --> cbl2 - prompts to copy cbl2 --> cbls for compile
  6. mfcblA - compile all programs cbls --> cblx & cblst

Here is asample of cobfile55b, created from 4 of the COBOL programs used for the test/demo conversions in this documentation. You will later see some of these record-sizes used to update ctl/datactl53I.

ctl/cobfil55b

 *cobfil55  COBOL File Info  Dir=cbl1  2015/03/08_17:48:23
 *progname.cbl DDname                        OAL open recsz copybook.cpy FDname  Writes Advances
 *         1         2         3         4         5         6         7
 *123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
 *==========================================================================================
 car200.cbl    saledtl                       SS  I       64 saledtl.cpy  saledtl
 car200.cbl    custmas                       IR  I      256 custmas.cpy  custmas
 car200.cbl    salelst                       L L O      120 sdline.cpy   salelst     2    2
 cgl100.cbl    acctmas                       SS  I      128              acctmas
 cgl100.cbl    actlist                       L L O      120              actlist     1    1
 cgl200.cbl    glmsold                       RS  I      128              glmsold
 cgl200.cbl    glmsnew                       RS  O      128              glmsnew     1      *
 cgl200.cbl    gltrans                       RS  I       80              gltrans
 car100.cbl    custmas                       SS  I      256 custmas.cpy  custmas
 car100.cbl    nalist                        S L O      120              nalist      2    2

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3B4. Updating control-files to improve JCL conversions

JCL conversion & control-files


 #3. jcl2ksh51A all      <-- convert ALL JCL components thru all steps
     ==============        - see listing at JCLcnv2real.htm#4A1
                           - includes folowing functions:

 #3a. cleanup                proc0 --> procs, parm0 --> parms, jcl0 --> jcl1
 #3b. proc expand            jcl1 --> jcl2
 #3c. jcldata51A         <-- create ctl/datactl53 control file
      ==========             combining info from JCL, COBOL, LISTCAT, etc
 #3d. jclxx51 jcl2 jcl3  <-- convert JCL to Korn shell scripts
      =================      jcl2 --> jcl3

The do everything script 'jcl2ksh51A' calls script 'jcldata51A' to create control file ctl/datactl53I.dat/.idx for use by 'jclxx51' (JCL to script).

 'jcldata51A' - see listing at JCLcnv2real.htm#4C1
              - includes following 6 steps:
  1. jcldata51 - extracts all DSNs from all JCL, sorts & reduces to 1 line/DSN
  2. jcldata52 - lowercases & converts mainframe GDG suffixes to unix convention (0), (+1) to '_' (followed by 6 digits on actual datafiles
  3. catcat51 - combines any additional file info you might code in ctl/add/...
  4. uvcp - convert combined dataadd51 to Indexed file dataadd51I for #5
  5. ctldata53 - copies datajcl2 (from #2) to datactl53 (info for JCL converter) - combining any info from matching filenames in dataadd51I & from datacat51I (info from mainframe LISTCAT reports)
  6. uvcp - converts sequential datactl53 to Indexed datactl53I for JCL converter

    ctl/datajcl51

 AR.CUSTOMER.MASTER            rca=_____ rcs=_____ typ=RSF key=___,___ job=jar100 prg=CAR100
 AR.CUSTOMER.MASTER.INDEXED    rca=_____ rcs=_____ typ=RSF key=___,___ job=jar200 prg=CAR200
 AR.CUSTOMER.NAMEADRS.LIST100  rca=00133 rcs=00133 typ=RSF key=___,___ job=jar100 prg=CAR100
 AR.SALES.ITEMS                rca=_____ rcs=_____ typ=RSF key=___,___ job=jar200 prg=SORT
 AR.SALES.LIST                 rca=00133 rcs=00133 typ=RSF key=___,___ job=jar200 prg=CAR200
 GL.ACCOUNT.ACNTLIST(+1)       rca=00133 rcs=00133 typ=RSF key=___,___ job=jgl100 prg=CGL100
 GL.ACCOUNT.MASTER(+1)         rca=00128 rcs=00128 typ=RSF key=___,___ job=jgl320 prg=IDCAMS
 GL.ACCOUNT.MASTER(0)          rca=_____ rcs=_____ typ=RSF key=___,___ job=jgl200 prg=CGL200
 GL.ACCOUNT.TRAN1              rca=_____ rcs=_____ typ=RSF key=___,___ job=jgl200 prg=SORT
 GL.ACCOUNT.TRANS(+1)          rca=00080 rcs=00080 typ=RSF key=___,___ job=jgl230 prg=SORT

ctl/datajcl52

 ar.customer.master            rca=_____ rcs=_____ typ=RSF key=___,___ job=jar100 prg=CAR100
 ar.customer.master.indexed    rca=_____ rcs=_____ typ=RSF key=___,___ job=jar200 prg=CAR200
 ar.customer.nameadrs.list100  rca=00133 rcs=00133 typ=RSF key=___,___ job=jar100 prg=CAR100
 ar.sales.items                rca=_____ rcs=_____ typ=RSF key=___,___ job=jar200 prg=SORT
 ar.sales.list                 rca=00133 rcs=00133 typ=RSF key=___,___ job=jar200 prg=CAR200
 gl.account.acntlist_          rca=00133 rcs=00133 typ=RSF key=___,___ job=jgl100 prg=CGL100
 gl.account.master_            rca=00128 rcs=00128 typ=RSF key=___,___ job=jgl320 prg=IDCAMS
 gl.account.master_            rca=_____ rcs=_____ typ=RSF key=___,___ job=jgl200 prg=CGL200
 gl.account.tran1              rca=_____ rcs=_____ typ=RSF key=___,___ job=jgl200 prg=SORT
 gl.account.trans_             rca=00080 rcs=00080 typ=RSF key=___,___ job=jgl230 prg=SORT

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3B5. Updating control-files to improve JCL conversions

JCL converter reads AND UPDATES ctl/datactl53I

The JCL conversion was 1st documented on page '3F1' & recently on page '3B1'.


 #3. jcl2ksh51A all     <-- convert ALL JCL thru multiple steps
     ==============         proc0 --> procs, parm0 --> parms, jcl0 --> jcl1 --> jcl2
Note
  • 'jcl2ksh51A' performs all steps of JCL,PROCs,PARMs cleanups & conversions
  • see the jcl2ksh51A script listed at JCLcnv2real.htm#4A1
  • jcl2ksh51A calls 'jclxx51' to perform the JCL conversion jcl2 --> jcl3

 #3d. jclxx51 jcl2 jcl3   <-- convert all jcl2/* to Korn shell scripts in jcl3/...
      =================     - no need to jclxx51 if you just ran jcl2ksh51a
                            - might run jclxx51 to reconvert only without repeating
                              cleanup procs,parms,jcls,& proc expansion

'jcl2ksh51A' calls 'jclxx51' which in turn calls the actual JCL converter C program 'jcluni51' as follows:


 jclunix51 jcl2/xxx.jcl jcl3/xxx.ksh  ctl/datactl53I.dat/.idx ctl/cobfil55b.dat/.idx
 ===================================================================================

jclunix51 reads both ctl/datactl53I & cobfil55bI to get record-sizes & file-types (if mising in JCL SORTs,IDCAMS,IEBGENERs,etc). It also updates ctl/datactl53I if it finds info in cobfil55bI that was not in datactl53I.

Here is the BEFORE & AFTER listing of ctl/datactl53I showing record-size info transferred from cobfil55bI (listed on previous page) & used to update ctl/datactl53I.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3B6. Updating control-files to improve JCL conversions

Here are listings of the 2 versions of the JCL converter control file (before & after running the 1st JCL conversion).

ctl/datactl53 - input to 1st JCL conversion

ctl/datactl53
  • sequential version of the JCL converter control file
  • after creation by jcldata51A (part of jcl2ksh51A)
  • will be loaded into the Indexed file for use by the JCL converter
 ar.customer.master           rca=_____ rcs=_____ gdg=    typ=RSF   key=___,___ job=jar100 prg=CAR100
 ar.customer.master.indexed   rca=_____ rcs=_____ gdg=    typ=RSF   key=___,___ job=jar200 prg=CAR200
 ar.customer.nameadrs.list100 rca=00133 rcs=00133 gdg=    typ=RSF   key=___,___ job=jar100 prg=CAR100
 ar.sales.items               rca=_____ rcs=_____ gdg=    typ=RSF   key=___,___ job=jar200 prg=SORT
 ar.sales.list                rca=00133 rcs=00133 gdg=    typ=RSF   key=___,___ job=jar200 prg=CAR200
 gl.account.acntlist_         rca=00133 rcs=00133 gdg=    typ=RSF   key=___,___ job=jgl100 prg=CGL100
 gl.account.master_           rca=00128 rcs=00128 gdg=    typ=RSF   key=___,___ job=jgl320 prg=CGL200
 gl.account.tran1             rca=_____ rcs=_____ gdg=    typ=RSF   key=___,___ job=jgl200 prg=SORT
 gl.account.trans_            rca=00080 rcs=00080 gdg=    typ=RSF   key=___,___ job=jgl230 prg=SORT

ctl/datactl53I.dat after updates by 1st JCL conversion

ctl/datactl53I.dat
  • data partition of JCL converter Indexed control file
  • AFTER being updated by the JCL converter
 ar.customer.master           rca=00256 rcs=00256 gdg=    typ=RSF   key=___,___ job=jar100 prg=CAR100
 ar.customer.master.indexed   rca=00256 rcs=00256 gdg=    typ=RSF   key=___,___ job=jar200 prg=CAR200
 ar.customer.nameadrs.list100 rca=00120 rcs=00120 gdg=    typ=LSTt  key=___,___ job=jar100 prg=CAR100
 ar.sales.items               rca=00064 rcs=00064 gdg=    typ=RSF   key=___,___ job=jar200 prg=SORT
 ar.sales.list                rca=00120 rcs=00120 gdg=    typ=LSTt  key=___,___ job=jar200 prg=CAR200
 gl.account.acntlist_         rca=00120 rcs=00120 gdg=    typ=LSTt  key=___,___ job=jgl100 prg=CGL100
 gl.account.master_           rca=00128 rcs=00128 gdg=    typ=RSF   key=___,___ job=jgl320 prg=CGL200
 gl.account.tran1             rca=00080 rcs=00080 gdg=    typ=RSF   key=___,___ job=jgl200 prg=SORT
 gl.account.trans_            rca=00080 rcs=00080 gdg=    typ=RSF   key=___,___ job=jgl230 prg=SORT

Note JCL control file updates

  1. The JCL converter uses data file info from ctl/cobfil55bI.dat/.idx to update the ctl/datactl53I.dat/.idx as shown above (changes to some rca=,rcs=,& typ=)

  2. Using the COBOL file info will improve JCL conversion for the files used by JCLs executing COBOL programs, but not necessarily for all JCLs using that file & depending on the sequence of JCLs being converted.

  3. Updating the control file on the 1st JCL conversion will improve the conversion for ALL JCLs when the JCL is reconverted.

  4. Note that jcl2ksh51A copies/renames the updated ctl/datactl53I.dat to ctl/add/datactl53a which will be combined with other files on next conversion.

  5. You might rename as ctl/add/datactl53b, so you can update it manually with vi & be sure it will not get overwritten.

  6. Or you might create a separate file such as ctl/add/datamisc51 illustrated later on page '3E1'.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3C1. Updating control-files to improve JCL conversions

using LISTCAT info to improve JCL conversion

If available, the mainframe LISTCAT reports can be transferred to unix & the data file info (record-sizes, file-types, Indexed keys) can be extracted & merged into the JCL converter control file to greatly improve JCL conversions.


 #1. login yourself --> /home/userxx

 #2. cdl  --> /home/userxx/testlibs  (alias cdl='cd $RUNLIBS')

 #3. mkdir cat0 cat1 cat2   <-- make subdirs for LISTCAT files processing

 #4. cd cat0

 #5. FTP LISTCAT reports from mainframe to unix
     - may be multiple files, names do not matter, but must be valid for unix

 #6. cdl        <-- change back to working directory (above cat0,cbl0,jcl0,etc)

 #7.                  <-- change names as necessary for unix
 #7a. renameL   cat0    - rename to Lower case for unix
 #7b. renameB2_ cat0    - change any embedded blanks to underscores
 #7c. renameD2_ cat0    - change any '$' dollar signs to '_' underscores
 #7d. rename-QQ cat0    - remove any 'Quotes'
 #7e. rename-PP cat0    - remove any (Parenthesis)

 #8. catdata50 all    <-- extract datafile info from LISTCAT reports
     =============
   cat0 -------> cat1 ---------> cat2 --------> ctl/datacat51 --> ctl/datacat52I
       catdata51      catdata52       catcat51                uvcp
Note
  • 'catdata50' is a script that calls 4 uvcopy jobs as shown above
    (catdata51, catdata52, catcat51,& uvcp to load Indexed file)
  • see script 'catdata50' listed at JCLcnv2real.htm#4C6
  • see scripts in $UV/pf/IBM/... & uvcopy jobs in $UV/pf/IBM/...
  • the resulting Indexed file will be looked up by script jcldata51A to merge
    data-file info into the Indexed file used by the JCL converter datactl53I

lrecl0cat2 added to catdata50 to solve problem

We had a problem at 1 site - the record sizes were misisng from the LISTCAT report for NON-VSAM files - but the mainframe person was able to create an additional report with LRECLs which we converted to the keyword format & combined with the other files - see details on page '3D1'.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3C2. Updating control-files to improve JCL conversions

LISTCAT mainframe report

 # listcat001 - test file for Vancouver Utilities JCL & DATA conversion
 #            - see www.uvsoftware.ca/datacnv1.htm#Part_6
 #              or www.uvsoftware.ca/jclcnv2real.htm#3C2
 # 'CLUSTER's for a few files extracted from mainframe LISTCAT report
 #  - omitting many items not relevant to desired information
 #  - selecting: KEYLEN, RKP, AVGLRECL, MAXLRECL, RECTOTAL, LIMIT (for GDGs)
 0CLUSTER ------- AR.CUSTOMER.MASTER
       KEYLEN-----------------0  AVGLRECL-----------256  BUFSPACE---------14366
       RKP--------------------0  MAXLRECL-----------256  EXCPEXIT--------(NULL)
       REC-TOTAL-------------32  SPLITS-CI------------0  EXCPS--------------495
 0CLUSTER ------- AR.CUSTOMER.MASTER.INDEXED
     ASSOCIATIONS
       DATA-----AR.CUSTOMER.MASTER.INDEXED.DATA
       INDEX----AR.CUSTOMER.MASTER.INDEXED.INDEX
       KEYLEN-----------------6  AVGLRECL-----------256  BUFSPACE---------14366
       RKP--------------------0  MAXLRECL-----------256  EXCPEXIT--------(NULL)
       REC-TOTAL-------------32  SPLITS-CI------------0  EXCPS--------------495
 0CLUSTER ------- AR.SALES.ITEMS
       KEYLEN-------0  AVGLRECL--------64  BUFSPACE------4096    CISIZE--------2048
       RKP----------0  MAXLRECL--------64  EXCPEXIT------(NULL)  CI/CA----------270
       REC-TOTAL---20  SPLITS-CI------0    EXCPS----------8
 0GDG BASE ------ GL.ACCOUNT.MASTER
       KEYLEN-------0  AVGLRECL-------128  BUFSPACE------4096    CISIZE--------2048
       RKP----------0  MAXLRECL-------128  EXCPEXIT------(NULL)  CI/CA----------270
       LIMIT------------------5            SCRATCH               NOEMPTY
     ASSOCIATIONS
       NONVSAM--GL.ACCOUNT.MASTER.G0005V00
       NONVSAM--GL.ACCOUNT.MASTER.G0006V00
       NONVSAM--GL.ACCOUNT.MASTER.G0007V00
 0CLUSTER ------- GL.ACCOUNT.TRANS
       KEYLEN-------0  AVGLRECL--------80  BUFSPACE------4096    CISIZE--------2048
       RKP----------0  MAXLRECL--------80  EXCPEXIT------(NULL)  CI/CA----------270
       LIMIT------------------6            SCRATCH               NOEMPTY
     ASSOCIATIONS
       NONVSAM--GL.ACCOUNT.TRANS.G0001V00

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3C2. Updating control-files to improve JCL conversions

cat1/listcat001 extracted from LISTCAT reports

 AR.CUSTOMER.MASTER           rca=00256 rcs=00256 typ=RSF    cnt=00000032
 AR.CUSTOMER.MASTER.INDEXED   rca=00256 rcs=00256 typ=IDXf1  cnt=00000032 key=000,006
 AR.SALES.ITEMS               rca=00064 rcs=00064 typ=RSF    cnt=00000020
 GL.ACCOUNT.MASTER_           rca=00000 rcs=00000 typ=RSF    gdg=003
 GL.ACCOUNT.TRANS             rca=00080 rcs=00080 typ=RSF

cat2/listcat001 copied/lowercased from cat1/...

 ar.customer.master           rca=00256 rcs=00256 typ=RSF    cnt=00000032
 ar.customer.master.indexed   rca=00256 rcs=00256 typ=IDXf1  cnt=00000032 key=000,006
 ar.sales.items               rca=00064 rcs=00064 typ=RSF    cnt=00000020
 gl.account.master_           rca=00000 rcs=00000 typ=RSF    gdg=003
 gl.account.trans             rca=00080 rcs=00080 typ=RSF

ctl/datacat51 - all cat2/... files combined

 ar.customer.master           rca=00256 rcs=00256 gdg=    typ=RSF
 ar.customer.master.indexed   rca=00256 rcs=00256 gdg=    typ=IDXf1   key=000,006
 ar.sales.items               rca=00064 rcs=00064 gdg=    typ=RSF
 gl.account.master_           rca=_____ rcs=_____ gdg=003 typ=RSF
 gl.account.trans             rca=00080 rcs=00080 gdg=    typ=RSF

Notes re LISTCAT files

  1. We are showing only 1 LISTCAT report in cat0/... But there could be many for different catalogs, discs, mainframes

  2. All the LISTCAT files in cat0, cat1, cat2 are combined into 1 file in ctl/datacat51 - reducing to 1 record per unique filename Info re recsize, gdgs,& keys could come from multiple records.

  3. ctl/datacat51 is loaded to Indexed file ctl/datacat52I, which is looked up by uvcopy ctldata53 to add any keyword info onto the DSNs extracted from JCLs.

  4. Note that there could be thousands of LISTCAT records, because many of those entries do not represent files actually present in the JCL converted.

  5. ctl/datactl53I then has only the filenames actually present in the JCL being converted.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3D1. Updating control-files to improve JCL conversions

problem - record size missing from LISTCAT report

We had a problem at 1 site - the record sizes were misisng from the LISTCAT report for NON-VSAM files - but the mainframe person was able to create an additional report which showed LRECL's as follows:

sample mainframe LRECL listing

    DATA SET NAME                          VOLUME   ORG  RECFM    LRECL BLKSZ
 PGAPO.ABE.EKSTADA.VLK                     AGS102   PO    FB        80   6160
 PGAPO.DEHUF.ABE.ABEBUCH.ONLINE            AGS039+  PS    FB       350   5950
 PGAPO.DEHJW.GAPOMYEB.MGYART               AGS073+  PS    FB      4000   8000
 PGAPO.EDSKW.GAPO41F2.KAGS416.LISTE2.RE    AGS043+  PS    FBA      133   6118
 PGAPO.EDSKW.GAPO41F2.KAGS420.AUSGABE.RE   AGS102+  PS    FB     14150  14150
 PGAPO.DEHBU.GAPOAYK2.FTP.KRCOPY.G0398V00  AGS065+  PS    FB        80   1600

I then wrote a uvcopy job to convert the LRECL listing into the keyword format used by the Vancouver Utilities JCL & DATA conversion jobs.


 uvcopy lrecl0cat2,fili1=ctl/lrecl0,filo1=cat2/lrecl2
 =====================================================
 - convert mainframe LRECL listing to VU keyword format

sample output in keyword format

 pgapo.abe.ekstada.vlk                        rca=00080 rcs=00080 typ=RSF
 pgapo.dehuf.abe.abebuch.online               rca=00350 rcs=00350 typ=RSF
 pgapo.dehjw.gapomyeb.mgyart                  rca=04000 rcs=04000 typ=RSF
 pgapo.dehpd.gapoaff5.kags416.liste2          rca=00133 rcs=00133 typ=RSF
 pgapo.edskw.gapoayf2.kags420.ausgabe.re      rca=14150 rcs=14150 typ=RSF
 pgapo.dehbu.gapoayk2.ftp.krcopy_             rca=00080 rcs=00080 typ=RSF  gdg=___
 Note - GDG files recognized by last node '.G####V00'
     - replaced by trailing '_' ID for UV GDG file & gdg=___ appended

where this job executed & where output file used

This job is part of script 'catdata50' which calls following jobs: catdata51, catdata52, lrecl0cat2 (this job), cp, catcat51,& uvcp

  catdata51  - convert mainframe LISTCAT report to keyword format
  catdata52  - translate filenames to lower case & convert GDG files ot VU format
 *lrecl0cat2 - converts LRECL mainframe report to keyword format cat2/lrecl2
  cp         - copies cat2/lrecl2 to ctl/add for JCL converter control files
  catcat51   - combines all files in cat2/... into ctl/datacat51
  uvcp       - loads ctl/datacat52I, to assist JCL conversion
             - ctl/datacat52I also input to DATA conversion job 'ctlcat12'

JCL converter script jcl2ksh51A calls script jcldata51, which calls uvcopy job ctldata53, which loads ctl/datactl53I (for the JCL converter) from various inputs including ctl/datacat52I

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3E1. Updating control-files to improve JCL conversions

manual coding to improve JCL conversions

If after 1st 2 conversions, you notice missing record-sizes or incorrect file-types for some files, you can add your own-coded control file in the ctl/add/ subdir, knowing that the keyword info from all files in ctl/add/* are merged into the JCL converter control file on the next conversion. Here is the sample/demo supplied in $UV/ctl/datamisc51.

datamisc51 - sample manual ctlfile

 # datamisc51 - miscellaneous data file info for JCL conversion
 #            - demo file for www.uvsoftware.ca/jclcnv2real.doc#Part_2
 #
 # Must be stored in $RUNLIBS/ctl/add/...
 # But could have any name desired BECAUSE:
 # - uvcopy catcat51 (part of jcldata51A script, called by jcl2ksh51A)
 # - will combine all files found in ctl/add/... into ctl/dataadd51
 #   which is loaded into Indexed file ctl/dataadd51I.dat/.idx
 #   which is looked up by uvcopy ctldata53 to merge any keyword data found
 #   into ctl/datactl53, loaded into ctl/datactl53I.dat/.idx for JCL converter
 # These comment lines ('#' in column 1) will be dropped by catcat51
 # Here are a few lines keyword info for the demo files
 #
 ar.sales.items                rca=00064 rcs=00064 typ=RST
 gl.account.acntlist_          rca=00132 rcs=00132 typ=LSTt2 gdg=003
 gl.account.trans_             rca=00080 rcs=00080 typ=RST   gdg=007

copy demo file to ctl/add/...

For our JCL conversion #2, we will copy the demo file into ctl/add/... where it will be combined with other control files on the next JCL conversion.


 #1. login yourself --> /home/userxx

 #2. cdl  --> /home/userxx/testlibs  (alias cdl='cd $RUNLIBS')

 #3. cp ctl/datamisc51 ctl/add/
     ==========================
     - copy demo manually prepared control file into ctl/add/...
       to make it effective on next JCL conversion

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3E2. Updating control-files to improve JCL conversions

Excel spreadsheet to improve JCL conversions

Some users might prefer to create am Excel spreadsheet to code missing record sizes, file types, Indexed keys, gdgs, etc. OR they may already have Excel spreadsheets that were used for the mainframe.

Here is a sample Excel spreadsheet supplied in $UV/ctl/dataxls51.psv. This is the pipe|delimited version after being exported from the Excel spreadsheet.

demo file supplied in $UV/ctl/dataxls51.psv

 DSN|Volume|Organization|Reclen|Block|RecFormat|Allocated|USED|Extents|created|Referenced
 ar.customer.master|VOL001|PS|256|25600|FB|800|50|1|9/30/2014|3/14/2015|
 ar.sales.items|VOL002|PS|64|6400|FB|750|25|1|8/25/2014|2/17/2015|
 ar.customer.nameadrs.list100|VOL003|PS|132|1320|FB|750|25|1|8/25/2014|2/17/2015|

For this spreadsheet, the only item applicable to unix was the record-size. We made a uvcopy job 'xlsdata51' to extract the data into our required format For example:

 ar.customer.master                            rca=00256 rcs=00256 data=ps__fb_
 ar.sales.items                                rca=00064 rcs=00064 data=ps__fb_
 ar.customer.nameadrs.list100                  rca=00132 rcs=00132 data=ps__fb_

The Output file will be in $RUNLIBS/ctl/add/ where it will be combined with any other control files on the next JCL conversion

See the uvcopy job 'xlsdata51' listed at JCLcnv2real.htm#4C7 You might modify it depending on the format of your Excel spreadsheet. You can run as follows:


 #1. login yourself --> /home/userxx

 #2. cdl  --> /home/userxx/testlibs  (alias cdl='cd $RUNLIBS')

 #3. uvcopy xlsdata51,fili1=ctl/dataxls51.psv,filo1=ctl/add/xlsdata51
     ================================================================

 #3a. uvcopy xlsdata51   <-- same as above, files default as shown
      ================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3F1. Updating control-files to improve JCL conversions

re-convert with LISTCAT, datamisc,& dataxls


 #1. login yourself --> /home/userxx

 #2. cdl  --> /home/userxx/testlibs  (alias cdl='cd $RUNLIBS')

 #3a. mv jcl3 jcl3.old   <-- save prior conversions by changing dirname
      ================     - so we can use alldiff2 below to see differences

 #3b. mkdir jcl3         <-- make new subdir for re-conversions
      ==========

 #4. jcl2ksh51A all      <-- convert ALL JCL thru all steps
     ==============          proc0 --> procs, parm0 --> parms
                             jcl0 --> jcl1 --> jcl2 --> jcl3

optional alldiff2 to see differences

I highly recommend this procedure to see the differences that re-converting makes - to ensure you changed what you intended & did not cause any unintended changes.


 #5a. alldiff2 jcl3.old jcl3  <-- create diff report in tmp/jcl3.dif
      ======================

 #5b. vi tmp/jcl3.dif   <-- investigate diff report
      ===============
      --> :g/ver:/d     <-- drop diffs caused by JCL converter version diff
      --> :g/logmsg/d   <-- drop diffs caused by logmsg lines
Note
  • see these reports listed on the next 2 pages (before & after drops).

Dropping the version, logmsg, & other non-relevant diffs lets you concentrate on the important differences, but can cause many empty difference files (where only diff was the version). I suggest you use the 'uvcopy diffdrop2' job to eliminate these lines:


 #6. uvcopy diffdrop2,fili1=tmp/jcl3.dif,filo1=tmp/jcl3.dif2
     =======================================================

 #6a. uvcopy diffdrop2   <-- same as above (files default as shown)
      ================

 #7. vi tmp/jcl3.dif2    <-- inestigate shorter report
     ================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3F2. Updating control-files to improve JCL conversions

dropping unwanted lines on alldiff2 report

The previous page gave the instructions to run script 'alldiff2' (after reconverting the JCL) to create a differences report to prove that the reconversions changed what was intended with no unintended consequences. Here we will illustrate 3 versions of the alldiff2 report:

  1. original report before any drops (71 lines)
  2. after using vi to drop unwnated lines ver: & logmsg (51 lines)
  3. after diffdrop2 to drop empty diffs & files with no diffs (20 lines)

To save space, we will show results for only 4 of the 9 demo JCL/scripts.

1. jcl3.dif - before any drops

 39c39
 < exit 0 #ver:20150313 a2b0c0d1e2f0g1h1i15j0k15l1m4n3o8p0q0r0s2t1u1v0w0x0y1z1
 ---
 > exit 0 #ver:20150314 a2b0c0d1e2f0g1h1i15j0k15l1m4n3o8p0q0r0s2t1u1v0w0x0y1z1
 diff file# 2 - jcl3.old/... vs jcl3/jar100.ksh
 31c31
 < logmsg2 "Executing--> uvsort \"fili1=$SORTIN,typ=RSF,rcs=64,filo1=$SORTO..."
 ---
 > logmsg2 "Executing--> uvsort \"fili1=$SORTIN,typ=RST,rcs=64,filo1=$SORTO..."
 33c33
 < uvsort "fili1=$SORTIN,typ=RSF,rcs=64,filo1=$SORTOUT,typ=RSF,rcs=64\
 ---
 > uvsort "fili1=$SORTIN,typ=RST,rcs=64,filo1=$SORTOUT,typ=RSF,rcs=64\
 70c70
 < exit 0 #ver:20150313 a2b0c0d1e2f0g1h1i15j0k15l1m4n3o8p0q0r0s2t1u1v0w0x0y1z1
 ---
 > exit 0 #ver:20150314 a2b0c0d1e2f0g1h1i15j0k15l1m4n3o8p0q0r0s2t1u1v0w0x0y1z1
 diff file# 3 - jcl3.old/... vs jcl3/jar200.ksh
 47c47
 < exit 0 #ver:20150313 a2b0c0d1e2f0g1h1i15j0k15l1m4n3o8p0q0r0s2t1u1v0w0x0y1z1
 ---
 > exit 0 #ver:20150314 a2b0c0d1e2f0g1h1i15j0k15l1m4n3o8p0q0r0s2t1u1v0w0x0y1z1
 diff file# 4 - jcl3.old/... vs jcl3/jgl100.ksh
 38c38
 < uvsort "fili1=$SORTIN,typ=RSF,rcs=80,filo1=$SORTOUT,typ=RSF,rcs=80\
 ---
 > uvsort "fili1=$SORTIN,typ=RSF,rcs=80,filo1=$SORTOUT,typ=RST,rcs=80\
 76c76
 < exit 0 #ver:20150313 a2b0c0d1e2f0g1h1i15j0k15l1m4n3o8p0q0r0s2t1u1v0w0x0y1z1
 ---
 > exit 0 #ver:20150314 a2b0c0d1e2f0g1h1i15j0k15l1m4n3o8p0q0r0s2t1u1v0w0x0y1z1
 diff file# 5 - jcl3.old/... vs jcl3/jgl200.ksh
 9 different of 9 files compared jcl3.old to jcl3

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3F3. Updating control-files to improve JCL conversions

2. jcl3.dif - after vi drop ver: & logmsg


 #5b. vi tmp/jcl3.dif   <-- investigate diff report
      ===============
      --> :g/ver:/d     <-- drop diffs caused by JCL converter version diff
      --> :g/logmsg/d   <-- drop diffs caused by logmsg lines
 39c39
 ---
 diff file# 2 - jcl3.old/... vs jcl3/jar100.ksh
 31c31
 ---
 33c33
 < uvsort "fili1=$SORTIN,typ=RSF,rcs=64,filo1=$SORTOUT,typ=RSF,rcs=64\
 ---
 > uvsort "fili1=$SORTIN,typ=RST,rcs=64,filo1=$SORTOUT,typ=RSF,rcs=64\
 70c70
 ---
 diff file# 3 - jcl3.old/... vs jcl3/jar200.ksh
 47c47
 ---
 diff file# 4 - jcl3.old/... vs jcl3/jgl100.ksh
 38c38
 < uvsort "fili1=$SORTIN,typ=RSF,rcs=80,filo1=$SORTOUT,typ=RSF,rcs=80\
 ---
 > uvsort "fili1=$SORTIN,typ=RSF,rcs=80,filo1=$SORTOUT,typ=RST,rcs=80\
 76c76
 ---
 diff file# 5 - jcl3.old/... vs jcl3/jgl200.ksh
 9 different of 9 files compared jcl3.old to jcl3

j3. cl3.dif2 - after diffdrop2 empty difs

 33c33
 < uvsort "fili1=$SORTIN,typ=RSF,rcs=64,filo1=$SORTOUT,typ=RSF,rcs=64\
 ---
 > uvsort "fili1=$SORTIN,typ=RST,rcs=64,filo1=$SORTOUT,typ=RSF,rcs=64\
 diff file# 3 - jcl3.old/... vs jcl3/jar200.ksh
 38c38
 < uvsort "fili1=$SORTIN,typ=RSF,rcs=80,filo1=$SORTOUT,typ=RSF,rcs=80\
 ---
 > uvsort "fili1=$SORTIN,typ=RSF,rcs=80,filo1=$SORTOUT,typ=RST,rcs=80\
 diff file# 5 - jcl3.old/... vs jcl3/jgl200.ksh
 33c33
 < uvsort "fili1=$SORTIN,typ=RSF,rcs=80,filo1=$SORTOUT,typ=RSF,rcs=80\
 ---
 > uvsort "fili1=$SORTIN,typ=RSF,rcs=80,filo1=$SORTOUT,typ=RST,rcs=80\
 diff file# 6 - jcl3.old/... vs jcl3/jgl230.ksh
 9 different of 9 files compared jcl3.old to jcl3

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3F4. Updating control-files to improve JCL conversions

re-convert alternative to save time

Page '3F1' re-converted using 'jcl2ksh51A' (do everything script).


 #4. jcl2ksh51A all      <-- convert ALL JCL thru all steps
     ==============          proc0 --> procs, parm0 --> parms
                             jcl0 --> jcl1 --> jcl2 --> jcl3

But jcl2ksh51A repeats cleanups & proc expands that are unnecessary if we are only changing control files. If you have hundreds of JCLs, PROCs,& PARMs, you can save time by replacing #4 above as shown below with steps #4a, #4b,& #4c.


 #1. login yourself --> /home/userxx
 #2. cdl  --> /home/userxx/testlibs  (alias cdl='cd $RUNLIBS')

 #4a. jcldata51A         <-- re-create ctl/datactl53 control file
      ==========             including LISTCAT info

 #4b. jclxx51 jcl2 jcl3  <-- re-convert JCL to Korn shell scripts
      =================

 #4c. cp ctl/datactl53I.dat ctl/add/datactl53a
      ========================================
      - save updated JCL converter control file for future re-conversions

note re #4c above

'jcl2ksh51A' (reconvert everything script) includes #4c. Since we are replacing jcl2ksh51A with jcldata51A & jclxx51, we also do this manually.

We do this because the converter updates the Indexed file datactl53I.dat/.idx with any datafile info supplied by other sources such as COBOL, LISTCAT, dataxls (spreadsheets), datamisc (misc edited info). We copy the file into ctl/add/... because all files there are combined for following re-conversions.

The jcl2ksh51A script renames as shown above on #4c, but there might be some advantage in renaming it as follows (using suffix 'b' vs 'a').


 #4c2. cp ctl/datactl53I.dat ctl/add/datactl53b
       =======================================*

This allows you to make manual updates to ctl/add/datactl53b & not lose them on reconversion - because ctl/add/datactl53a is overwritten by reconversion, but any added info in ctl/add/datactl53a will be merged into new control files & especially 'dataclt53I.dat/.idx' used by the JCL converter.


 #4d. vi ctl/add/datactl53b      <-- manual update
      =====================
      - add record-sizes, change file-types, etc as desired

 #5. cp jcl3/* jcls   <-- copy converted JCL/scripts to jcls (in $PATH)
     ==============     - only for our demos, not for your conversions
                          (will copy 1 at a time as you test/debug)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3F5. Updating control-files to improve JCL conversions

ctl/datactl53 - before 2nd conversion

 ar.customer.master           rca=00256 rcs=00256 gdg=___ typ=RSF   key=___,___ job=jar100 prg=CAR100
 ar.customer.master.indexed   rca=00256 rcs=00256 gdg=___ typ=RSF   key=___,___ job=jar200 prg=CAR200
 ar.customer.nameadrs.list100 rca=00120 rcs=00120 gdg=___ typ=LSTt  key=___,___ job=jar100 prg=CAR100
 ar.sales.items               rca=00064 rcs=00064 gdg=___ typ=RSF   key=___,___ job=jar200 prg=SORT
 ar.sales.list                rca=00120 rcs=00120 gdg=___ typ=LSTt  key=___,___ job=jar200 prg=CAR200
 gl.account.acntlist_         rca=00120 rcs=00120 gdg=___ typ=LSTt  key=___,___ job=jgl100 prg=CGL100
 gl.account.master_           rca=00128 rcs=00128 gdg=___ typ=RSF   key=___,___ job=jgl320 prg=CGL200
 gl.account.tran1             rca=00080 rcs=00080 gdg=___ typ=RSF   key=___,___ job=jgl200 prg=SORT
 gl.account.trans_            rca=00080 rcs=00080 gdg=___ typ=RSF   key=___,___ job=jgl230 prg=SORT

ctl/datactl53I.dat after updates by 2nd JCL conversion

 ar.customer.master           rca=00256 rcs=00256 gdg=___ typ=RSF   key=___,___ job=jar100 prg=CAR100
 ar.customer.master.indexed   rca=00256 rcs=00256 gdg=___ typ=IDXf1 key=000,006 job=jar200 prg=CAR200
 ar.customer.nameadrs.list100 rca=00120 rcs=00120 gdg=___ typ=LSTt  key=___,___ job=jar100 prg=CAR100
 ar.sales.items               rca=00064 rcs=00064 gdg=___ typ=RST   key=___,___ job=jar200 prg=SORT
 ar.sales.list                rca=00120 rcs=00120 gdg=___ typ=LSTt  key=___,___ job=jar200 prg=CAR200
 gl.account.acntlist_         rca=00120 rcs=00120 gdg=003 typ=LSTt  key=___,___ job=jgl100 prg=CGL100
 gl.account.master_           rca=00128 rcs=00128 gdg=005 typ=RSF   key=___,___ job=jgl320 prg=CGL200
 gl.account.tran1             rca=00080 rcs=00080 gdg=___ typ=RSF   key=___,___ job=jgl200 prg=SORT
 gl.account.trans_            rca=00080 rcs=00080 gdg=007 typ=RST   key=___,___ job=jgl230 prg=SORT

ctl/datactl53I BEFORE/AFTER 2nd JCL conversion

  1. Note changes in the JCL converter control file between 1st & 2nd conversions:

 1a. ar.customer.master.indexed now coded corectly as typ=IDXf1 & key=000,006
     - picked up from the LISTCAT reports (used on 2nd convert but not 1st.

1b. 3 of the GDG files now have the number of generations coded - picked up from the LISTCAT reports.

 1c. ar.sales.items & gl.account.trans changed 'typ=RSF' to 'typ=RST'.
     - picked up from ctl/datamisc51 (used on 2nd convert but not 1st).
  1. There are only a few changes because our demo JCL already had most record size info. In a real conversion there would be a lot more file info picked up from the LISTCAT reports that was not in the JCL.

  2. The typ=IDXf1 & key=000,006 were not coded on the 1st conversion, because that was based only on the information found in the JCL, and the JCL (jar200.ksh on page '2B1' line j#016) references AR.CUSTOMER.MASTER.INDEXED but does not define it (with an IDCAMS).

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3G1. Enhancing & Customizing Conversions

modifying JCL converter options ctl/jclunixop51

We will illustrate changing option 'f0/f1' in ctl/jclunixop51 to change default file typ=RSF to typ=LST for sort & copy utiltiies. We can also change individual file types & record-sizes in ctl/datactl51 & reloading Indexed file ctl/datactl51I & reconverting.

The JCL converter control file is listed at JCLcnv2real.htm#4C1 Here is the options (line 8) before & after changing option 'f0' to 'f1'


 jclunixop51:a2b0c0d1e2f0g1h1i15j0k15l1m4n3o8p0q0r0s2t1u1v0w0x0y1z1
             ==========**==========================================
 jclunixop51:a2b0c0d1e2f1g1h1i15j0k15l1m4n3o8p0q0r0s2t1u1v0w0x0y1z1

Here are descriptions for option 'f' from lines 47-49 of ctl/jclunixop51

 #         f0            - default file typ=RSF (allows packed/binary)
 #         f1            - default uvsort/uvcp file typ=LST
 #         f2            - default uvsort/uvcp file typ=LSTt

You could change to 'f1' if you have very few files with packed/binary. Conversions of SORT, IEBGENER,etc to uvsort & uvcp would then code file types as 'typ=LST' vs 'typ=RSF'. The advantage is that you could then use unix utilities that equire LineFeeds (vi,more,cat,lp) on your datafiles. You could manually change some uvsort/uvcp 'typ=LST' back to 'typ=RSF' for a few files with packed/binary.

modifying options & re-convert Instructions


 #1. Login userxx --> /home/userxx  (or whatever Login desired)

 #2. cdl     <-- alias cdl='cd $RUNLIBS' --> cd testlibs

 #3. vi $RUNLIBS/ctl/jclunixop51  <-- change option 'f0' to 'f1'
     ===========================      (as shown above)

 #4a. mv jcl3 jcl3.old   <-- save existing conversions old/new diff check
      ================
 #4b. mkdir jcl3         <-- make new subdir to reveive re-conversions
      ==========

 #5. jclxx51 jcl2 jcl3       <-- reconvert all JCL jcl2/* --> jcl3/...
     =================           (after option changes)

 #6. alldiff2 jcl3.old jcl3  <-- run 'diff' on all files in old/new subdirs
     ======================    - writes report to tmp/jcl3.dif

 #6a. vi tmp/jcl3.dif        <-- check differences as expected
      ===============            & no unintended changes
                               - see sample report on next page below

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3G2. Enhancing & Customizing Conversions

diff report after reconvert with option 'f1'

 33c33
 < uvsort "fili1=$SORTIN,typ=RSF,rcs=64,filo1=$SORTOUT,typ=RSF,rcs=64\
 ---
 > uvsort "fili1=$SORTIN,typ=RSF,rcs=64,filo1=$SORTOUT,typ=LST,rcs=64\
 diff file# 3 - jcl3.old/... vs jcl3/jar200.ksh
 30c30
 < uvcp "fili1=$SYSUT1,typ=RSF,rcs=128,filo1=$SYSUT2,typ=RSF,rcs=128"
 ---
 > uvcp "fili1=$SYSUT1,typ=RSF,rcs=128,filo1=$SYSUT2,typ=LST,rcs=128"
 59c59
 < uvcp "fili1=$TEMPGLM,typ=RSF,rcs=80,filo1=$ACNTMAS,typ=RSF,rcs=128";
 ---
 > uvcp "fili1=$TEMPGLM,typ=LST,rcs=80,filo1=$ACNTMAS,typ=RSF,rcs=128";
 diff file# 7 - jcl3.old/... vs jcl3/jgl320.ksh

install new jcl/scripts & test


 #7. cp jcl3/* jcls   <-- copy re-converted jcl/scripts for executions
     ==============     - 'jcls' is in the PATH defined in our profile
Note
  • we copy all jcl3/* to jcls for our test/demos, but you should NOT
  • we recommend you copy 1 at a time when ready to test/debug each JCL/script
  • gives you a measure of progress & allows you to reconvert the
    untested jobs if enhancements are made to the control files
    or to the JCL converter iteslf.

 #7a. cp jcl3/jobxx.ksh jcls
      ======================
     - copy 1 at a time when you are ready to debug each JCL/script

 #8. joblog jar200.ksh   <-- might rerun jar200 to check changes
     =================       in file type output by uvsort typ=LST

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3H1. Enhancing & Customizing Conversions

modifying Data Control files & re-converting

These 3 pages 3G1-3G3 are older & perhaps redundant to page '3D1' coding ctl/add/datamisc51 with changes to file info for selected files & to page '3E4' which 1st suggested copying ctl/datactl53I.dat to ctl/add/datactl53b and making changes there for future conversions.

The JCL converter script jcl2ksh51A extracts datafile info (types,rec-sizes,etc) from all JCLs & creates an Indexed control file ctl/datactl53I that is used by the JCL converter jclunix51.c to supply datafile info to JCL jobs that may not define datafile types & record-sizes required (by uvsort,uvcp,etc). Note that the mainframe could get file info from the VTOC & omit from JCL, but file info must be defined on uvsort & uvcp steps.

sample ctl/datactl53I created by 1st JCL conversion

 ar.customer.master           cpy=   rcs=_____ gdg=   typ=RSF  data=   job=jar100   prg=CAR100
 ar.customer.master.indexed   cpy=   rcs=_____ gdg=   typ=RSF  data=   job=jar200   prg=CAR200
 ar.customer.nameadrs.list100 cpy=   rcs=00133 gdg=   typ=RSF  data=   job=jar100   prg=CAR100
 ar.parmslib@parm200s         cpy=   rcs=_____ gdg=   typ=RSF  data=   job=jar200   prg=SORT
 ar.sales.items               cpy=   rcs=00064 gdg=   typ=RSF  data=   job=jar200   prg=SORT
 ar.sales.list                cpy=   rcs=00133 gdg=   typ=RSF  data=   job=jar200   prg=CAR200
 gl.account.acntlist_         cpy=   rcs=00133 gdg=   typ=RSF  data=   job=jgl100   prg=CGL100
 gl.account.master_           cpy=   rcs=00128 gdg=   typ=RSF  data=   job=jgl320   prg=CGL200
 gl.account.tran1             cpy=   rcs=00080 gdg=   typ=RSF  data=   job=jgl200   prg=SORT
 gl.account.trans_            cpy=   rcs=00080 gdg=   typ=RSF  data=   job=jgl230   prg=SORT
 gl.parms@pgl200s1            cpy=   rcs=_____ gdg=   typ=RSF  data=   job=jgl200   prg=SORT

The JCL conversion script jcl2ksh51A saves a text version of ctl/datactl53I in ctl/add/datactl53a. You could copy this to ctl/add/datactl53b & update as desired. The next run of jcl2ksh51A will include any new info supplied in the recreated ctl/datactl53I.

sample ctl/add/datactl53b updated with typs & rec-sizes

 ar.customer.master           cpy=   rcs=00256 gdg=   typ=RSF  data=p  job=jar100   prg=CAR100
 ar.customer.master.indexed   cpy=   rcs=00256 gdg=   typ=RSF  data=p  job=jar200   prg=CAR200
 ar.customer.nameadrs.list100 cpy=   rcs=00133 gdg=   typ=LSTt data=   job=jar100   prg=CAR100
 ar.parmslib@parm200s         cpy=   rcs=_____ gdg=   typ=RSF  data=   job=jar200   prg=SORT
 ar.sales.items               cpy=   rcs=00064 gdg=   typ=LST  data=   job=jar200   prg=SORT
 ar.sales.list                cpy=   rcs=00133 gdg=   typ=LSTt data=   job=jar200   prg=CAR200
 gl.account.acntlist_         cpy=   rcs=00133 gdg=05 typ=LSTt data=   job=jgl100   prg=CGL100
 gl.account.master_           cpy=   rcs=00128 gdg=05 typ=RSF  data=p  job=jgl320   prg=CGL200
 gl.account.tran1             cpy=   rcs=00080 gdg=   typ=LST  data=   job=jgl200   prg=SORT
 gl.account.trans_            cpy=   rcs=00080 gdg=05 typ=LST  data=   job=jgl230   prg=SORT
 gl.parms@pgl200s1            cpy=   rcs=_____ gdg=   typ=RSF  data=   job=jgl200   prg=SORT

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3H2. Enhancing & Customizing Conversions

modify Data Control files & re-convert


 jcl2ksh51A all    <-- script to convert ALL JCL thru all steps
 ==============       jcl0 --> jcl1 --> jcl2 --> jcl3

This assumes that we have already converted all JCL as shown above (see instructions & console log beginning on page '3F1'). After some initial testing, we may wish to modify the data control files & re-convert.


 #1. Login userxx --> /home/userxx  (or whatever Login desired)

 #2. cdl     <-- alias cdl='cd $RUNLIBS' --> cd testlibs

 #3. cp ctl/add/datactl53a ctl/add/datactl53b
     ========================================
     - copy data ccontrol file saved by 1st conversion
       for modify & inclusion in subsequent conversions

 #4. vi ctl/add/datactl53b
     =====================
     - see sample modifications on page above

 #5. rm ctl/add/datactl53a
     =====================
     - OK to remove the '53a' file created by 1st conversion
     - it will be recreated by reconversion script 'jcl2ksh51A' below
       (containing the info from the '53b' file you have edited)

 #6a. mv jcl3 jcl3.old   <-- save existing conversions old/new diff check
      ================
 #6b. mkdir jcl3         <-- make new subdir to reveive re-conversions
      ==========

 #7. jcl2ksh51A all   <-- re-convert all JCL thru all steps
     ==============       jcl0 --> jcl1 --> jcl2 --> jcl3
                        - to include new data control file info

 #8. alldiff2 jcl3.old jcl3  <-- run 'diff' on all files in old/new subdirs
     ======================    - writes report to tmp/jcl3.dif

 #8a. vi tmp/jcl3.dif        <-- check differences as expected
      ===============            & no unintended changes
                               - see sample report on next page below

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3H3. Enhancing & Customizing Conversions

differences after control file updates & re-convert


 #8. alldiff2 jcl3.old jcl3  <-- run 'diff' on all files in old/new subdirs
     ======================    - writes report to tmp/jcl3.dif

 #8a. vi tmp/jcl3.dif        <-- check differences as expected
      ===============            & no unintended changes
                               - sample report below
 33c33
 < uvsort "fili1=$SORTIN,typ=RSF,rcs=64,filo1=$SORTOUT,typ=RSF,rcs=64\
 ---
 > uvsort "fili1=$SORTIN,typ=LST,rcs=64,filo1=$SORTOUT,typ=RSF,rcs=64\
 diff file# 3 - jcl3.old/... vs jcl3/jar200.ksh
 37c37
 < uvsort "fili1=$SORTIN,typ=RSF,rcs=80,filo1=$SORTOUT,typ=RSF,rcs=80\
 ---
 > uvsort "fili1=$SORTIN,typ=LST,rcs=80,filo1=$SORTOUT,typ=LST,rcs=80\
 diff file# 5 - jcl3.old/... vs jcl3/jgl200.ksh
 33c33
 < uvsort "fili1=$SORTIN,typ=RSF,rcs=80,filo1=$SORTOUT,typ=RSF,rcs=80\
 ---
 > uvsort "fili1=$SORTIN,typ=LST,rcs=80,filo1=$SORTOUT,typ=LST,rcs=80\
 diff file# 6 - jcl3.old/... vs jcl3/jgl230.ksh

3 different of 9 files compared jcl3.old to jcl3

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3I1. Enhancing & Customizing Conversions

JCL converter DATA control file summary

Here is a summary of the data control files used by the JCL converter super- script 'jcl2ksh51A'. This should help you understand the process.

jcl2ksh51A
  • the super-script that performs all steps of JCL conversion
  • the JCL converter (jclunix51.c) is just 1 step in super-script
jcldata51A
  • sub-script called by jcl2ksh51A to create ctl/datactl53I
    an Indexed file used by JCL converter step to supply file info

uvcopy jobs & datafiles used by jcldata51A

  1. uvcopy jcldata51 extracts DSNs from all JCLs in jcl2/... & writes control file ctl/datajcl51

  2. uvcopy jcldata52 copies ctl/datajcl51 to ctl/datajcl52, converting the filenames from mainframe to unix/linux script & VU standards

  3. uvcopy ctldata53 copies ctl/datajcl52 to ctl/ctldata53, updating keyword info by lookup Indexed files ctl/datacat52I & ctl/dataadd52I Following files must/may exist before this script is run

 3a. ctl/datacat52I - Indexed file created by script catdata50
                      from multiple mainframe LISTCAT reports
                    - empty file created by copymvsctls (in case no LISTCAT)
 3b. ctl/add/... - may be multiple sequential files in ctl/add/... subdir
                 - will be combined by uvcopy catcat51 & loaded into
                   Indexed file ctl/dataadd52I
 3c. ctl/dataadd52I - Indexed file (interim) created by this script
                      from all additional info files in ctl/add/...
 3d. $UV/ctl/add/dummy_readme supplied if no additional info
              - copied to ctl/add by copymvsctls (in case no other files)

3e. additional files that you may create, name as you like, for example: ctl/datacpy52 - datafilenames & copybooknames ctl/dataedt52 - make with editor to supply missing info as desired ctl/datagdg52 - make with editor to supply missing GDG gnerations

 3f. ctl/add/datactl53a - input from any prior jcl2ksh51A, converts all JCL
                          jcl0 --> jcl1 --> jcl2 --> jcl3
                        - saves updates to datactl53I made by JCL converter
                          for reconversions by jcl2ksh51A
  1. ctl/datactl53I - Indexed file (final result) created by this script to supply datafile info to JCL converter (record sizes, Indexed keylocs/keylens, file types, etc)

  2. ctl/add/datactl53a - output by JCL converter for reconversions by jcl2ksh51A

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3J1. Enhancing & Customizing Conversions

alternate sets of testlibs/testdata/cnvdata

You might want to setup an alternate set of testlibs/testdata/cnvdata. You can name them whatever you want, as long as they are defined in your profile by RUNLIBS/RUNDATA/VNCDATA. You use an alternate set of libraries & data to concentrate on a small system of JCL, COBOL,& DATA files (and make it easier to test/debug). For example, here is how you might setup alternate set testlibs1/testdata1/cnvdata1 & modify profile to match:


 #1. Login userxx --> /home/userxx  (or whatever Login desired)

 #2. mkdir testlibs1 testdata1 cnvdata1  <-- setup alternate superdirs
     ==================================

 #3. vi .bash_profile  <-- edit profile to redefine RUNLIBS/RUNDATA/CNVDATA
 #3a. vi .profile          (.profile for ksh shell vs bash shell)
     ================    - see profile listed on page '1C1'
      export RUNLIBS=$HOME/testlibs    <-- existing definitions
      export RUNDATA=$HOME/testdata
      export CNVDATA=$HOME/cnvdata
      export RUNLIBS=$HOME/testlibs1   <-- update to new definitions
      export RUNDATA=$HOME/testdata1
      export CNVDATA=$HOME/cnvdata1

 #4. Logoff & backon to activate new definitions

 #5a. cdl --> /home/testlibs1   (alias cdl='cd $RUNLIBS)
      =======================
 #5b. mvslibsdirs    <-- script sets up 30 subdirs (see page '3B2')
      ===========

 #6a. cdd --> /home/testdata1   (alias cdd='cd $RUNDATA)
      =======================
 #6b. mvsdatadirs    <-- script sets up 15 subdirs (see page '3B3')
      ===========

 #7a. cdc --> /home/cnvdata1    (alias cdl='cd $CNVDATA)
      =======================
 #7b. cnvdatadirs    <-- script sets up 25 subdirs (see page '3B3')
      ===========

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3J2. alternate sets of testlibs/testdata/cnvdata

Above we have setup alternate set of conversion directories testlibs1/testdata1. We might now copy over a subset of JCL/COBOL & convert/test separately. We might do this for a simpler learning experience.


 #8a. cp ../teslibs/jcl0/... jcl0/    <-- select desired subset of JCL
      ============================
 #8b. cp ../teslibs/proc0/... proc0/  <-- select desired subset of procs
      ==============================
 #8c. cp ../teslibs/parm0/... parm0/  <-- select desired subset of parms
      ==============================

 #9a. cp ../teslibs/cbl0/... cbl0/    <-- select desired subset of COBOL
      ============================
 #9b. cp ../teslibs/cpy0/... cpy0/    <-- select desired subset of copybooks
      ============================

 #10a. cnvMF51Acpy all <-- convert copybooks thru all steps
       =============   - see prompt/reply responses at '3E2'

 #10b. cnvMF51A all  <-- convert COBOL programs thru all steps
       =============   - see prompt/reply responses at '3E3'

 #11. jcl2ksh51A all    <-- convert ALL JCL thru all steps
      ==============        jcl0 --> jcl1 --> jcl2 --> jcl3

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3K1. Enhancing & Customizing Conversions

mvsfiles51 - Determine Essential INPUT files

Script 'mvsfiles51' creates the 'essential files' report for 1 job at a time (vs 'mvsfiles5A' for all jobs in jcl2/...). Run mvsfiles51 before you attempt to debug each JCL/script to determine if you have all INPUT files required for a particular job. See complete documentation on pages '1L1' - '1L8', but here is a demo for JCL jgl230.jcl (3 steps: SORT + COBOL cgl100.cbl & cgl200.cbl).


 #1. Login userxx --> /home/userxx  (or whatever Login desired)

 #2. cdl --> $HOME/testlibs (/home/userxx/testlibs)

 #3. mkdir xmvs    <-- make subdir for temp files & final report
     ==========

 #4. mvsfiles51 jcl2/jgl230.jcl  <-- script for 1 job at a time
     ==========================

 #5. vi xmvs/mvsfiles5   <-- inspect report of all files SORTED by FILENAME
     =================     - more useful to determine INPUT files required

xmvs/mvsfiles5 - sorted by file-name

  mvsfiles5 - Insert '*' Flags beside Esential Input files - 2013/12/31 08:57:22
  Jobname Step  Program  DDname   MOD Gen Rcsz pb  From To * <-----DSNname------>
  ===============================================================================
 jgl230   003   CGL100   ACTLIST  MKD- +1  133               GL.ACCOUNT.ACNTLIST_
 jgl230   002   CGL200   GLMSOLD  O     0 0128 p       003 * GL.ACCOUNT.MASTER_
 jgl230   002   CGL200   GLMSNEW  NCD- +1  128 p       003   GL.ACCOUNT.MASTER_
 jgl230   003   CGL100   ACCTMAS  S     0 0128 p   002       GL.ACCOUNT.MASTER_
 jgl230   001   SORT     SORTIN   O                        * GL.ACCOUNT.TRAN1
 jgl230   001   SORT     SORTOUT  NCD- +1   80         002   GL.ACCOUNT.TRANS_
 jgl230   002   CGL200   GLTRANS  O  - +1 0080     001       GL.ACCOUNT.TRANS_
  1. 'mvsfiles5' (listed above) has been sorted by filename (right side), so you can see all jobs using each file. Note the '*' marking essential input files.

  2. Also note other information useful when debugging JCL/scripts - such as RECORD-SIZE & the 'p' flag (Packed fields present indicator).

  3. Another report 'mvsfiles6' (not shown here) selects just the '*' files dropping duplicates & temporary files inter-step files.

  4. mvsfiles51 also generates a script 'mvsfiles7' to copy '*' essential files from $CNVDATA/d2asc to $RUNDATA/data. For this report the script would be:

 # mvsfiles7 - copy essential files from $CNVDATA/d2asc/... to $RUNDATA/data/...
 cp $CNVDATA/d2asc/gl.account.master_*                    $RUNDATA/data1
 cp $CNVDATA/d2asc/gl.account.tran1                       $RUNDATA/data1

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3L1. Enhancing & Customizing Conversions

Inserting commands to print files

Printer files are not automatically sent to a printer as they might have been on the mainframe, but you can insert print commands into the JCL/scripts. Here is an example based on 'jar100.ksh'. See complete listing on page '2A1', and here is the last 20 lines defining the files & executing COBOL program 'car100.cbl' which creates print file data1/ar.customer.nameadrs.list100.

 -------- 1st 22 lines omitted, see complete listing on page '2A1' --------
 exportfile  CUSTMAS data1/ar.customer.master
 exportfile  NALIST data1/ar.customer.nameadrs.list100    #<--NOTE print file
 exportfile SYSOUT $SYOT/${jobid2}_${JSTEP}_SYSOUT_$(date +%y%m%d_%H%M%S)
 #3----------------------------------------------------------------------
 cobrun $ANIM $RLX/car100
 #4----------------------------------------------------------------------
 LCC=$?; S0010C=$LCC; ((SCC|=LCC)); ((JCC|=LCC)); S0010R=1; alias goto="";
 if ((S0010C != 0))
    then logmsg2 "ERR: step#$JSTEP car100 abterm $SCC"
    alias goto="<<S9900=\A"; fi
 goto
 uvlp13LD $NALIST    #<-- send print file to laser printer  <-- NOTE
 #8======================================================================
 S9000=A   #<--Normal End of Job - see details on page '2A1'
 S9900=A   #<--Abnormal End of Job

NOTE 'uvlp13LD $NALIST' line inserted to send the print file to a laser printer. See the 'uvlp13LD' script listed at https://uvsoftware.ca/uvlist.htm#E3. This script (for the uvlist utility) contains laser printer codes to print at 13 cpi & 8 lpi, Landscape & Duplex. This allows you to print a mainframe report (132 columns wide & 66 lines deep) on 8 1/2 x 11 (Landscape & Duplex).

There are many other 'uvlp...' scripts for a variety of cpi, lpi, Landscape/Portrait, Duplex/Simplex, margins, etc. See uvlist.htm.

SYSOUT reports

COBOL programs often create print reports via 'DISPLAY upon SYSOUT' and the JCL converter allows for this by inserting the following line on COBOL steps.


 exportfile SYSOUT $SYOT/${jobid2}_${JSTEP}_SYSOUT_$(date +%y%m%d_%H%M%S)
 ========================================================================

SYSOUT was not used by our demo program (car100.cbl), but if it had been, you would print the report as follows:


 uvlp13LD $SYSOUT   #<-- send print file (SYSOUT vs NALIST) to laser printer
 #8==============

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_4 Checking results, Searching for potential problems

You will often need to search the JCL/scripts to check conversion results. You can search for patterns that might confirm desired conversions or reveal problems that need to be corrected. 'grep' is the 1st choice if you only need to search for 1 pattern. The Vancouver Utilities has powerful tools to search for multiple patterns on 1 line or multiple lines in a file.

For example 'selectf1' will search entire directories for up to 5 patterns anywhere in each file for specified occurrence counts =, >, or < specified values. Default among multiple conditions is 'AND', but may specify 'OR'. See example to search for SORT's with 'INCLUDE COND' or 'OMIT COND'

Part 4 - Contents


4A1. 'grep' example to search directory of JCLs for 'SORT's & converted
scripts for 'uvsort's.

4B1. 'selectf1' example to search JCLs for SORT's with 'INCLUDE COND' or
'OMIT COND'. Could then search converted scripts for selects or deletes,
but the JCL search gives you the filenames to check in the converted
scripts directory.
Note
  • selectf1 writes a file of filenames containing the matching patterns
  • Vancouver Utilities has many other tools to select the data lines
    matching specified patterns. See complete details in
    https://uvsoftware.ca/scanjobs.htm, but here are a few of the most
    useful search tools.

4C1. scan1d - uvcopy job to search all files in a directory for data lines with
- matches to 1 pattern optionally qualified by 2 other patterns.
- might use prior to 'rep1d' (documented in Part_5).

4D1. scan2d - uvcopy job to search all files in a directory for data lines with
- matches to any 1 of multiple patterns in a pre-edited table file.
- might use prior to 'rep2d' (documented in Part_5).
  that uses a table file of search/replace/qualify patterns.

4E1. scan3d - select lines from a starting pattern to an ending pattern,
  optionally qualified by a 3rd pattern present anywhere between start & end.
- see example selecting IKJEFT01 steps to see various programs being run

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4A1. Checking results, Searching for potential problems

using 'grep' to check conversion results

'grep' is the 1st choice when searching directories for specified patterns, but you can only search for 1 pattern at a time. On the following pages, we will demonstrate 'selectf1' which will search for up to 5 patterns.

Page '4B3' will demonstrate searching the mvstest demo JCLs for SORTs, that have 'INCLUDE COND=...' or OMIT COND=...', but first we will show you the 'grep' method of searching for SORT's in JCLs. We can use grep to search for 'PGM=SORT' in the unconverted jcl2/* or for 'uvsort in the converted jcl3/* We will use the demo JCL's listed in JCLcnv1demo.htm#Part_2 & that you may have copied to your homedir/testlibs as instructed JCLcnv1demo.htm#Part_3.

 /home/userxx          <-- setup testlibs in your homedir
 :-----testlibs        <-- $RUNLIBS libraries for JCL & COBOL conversions
 :     :-----jcl0        - test/demo JCLs supplied
 :     :      :---jar100.jcl,jar200.jcl,jgl100.jcl,jgl200.jcl,jgl230.jcl,jgl320.jcl
 :     :-----jcl1        - intermediate conversion 73-80 cleared
 :     :-----jcl2        - JCLs with PROCs expanded from procs/...
 :     :-----jcl3        - JCLs converted to Korn shell scripts

 #1. Login userxx --> /home/userxx  (or whatever Login desired)

 #2. cdl     <-- alias cdl='cd $RUNLIBS' --> cd testlibs

 #3. grep 'EXEC PGM=SORT' jcl2/*
     ===========================
 jcl2/jar200.jcl:3://STEP010  EXEC PGM=SORT,REGION=2048K
 jcl2/jgl200.jcl:10://STEP010  EXEC PGM=SORT,REGION=2048K,PN=STEPA
 jcl2/jgl230.jcl:5://STEP010  EXEC PGM=SORT

So now we know which JCLs have SORTs, but we would then have to check each of them to see if they have 'INCLUDE COND=...' or 'OMIT COND=...'. Not a big problem when only 3 JCLs to check, but there could be hundreds of JCLs with SORTs, but only a few with the INCLUDEs/OMITs desired.


 #4a. vi jcl2/jar200.jcl     <-- see listing on page '2B1'
      ==================
 #4b. vi jcl2/jgl200.jcl     <-- see listing on page '2D1'
      ==================
 #4c. vi jcl2/jgl230.jcl     <-- see listing on page '2E1'
      ==================

See the page '4B3' for a better way using 'selectf1'.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4B1. Checking results, Searching for potential problems

selectf1 - select files depending on search pattern occurrences

'selectf1' will select filenames from a directory of files depending on search pattern occurrences = > < specified values. You may specify up to 3 search patterns with occurrence counts = or > or < than specified values. The default condition (among multiple search patterns) is 'AND', but you may specify 'OR'.

Here are a few examples (before presenting the command FORMAT & RULES below).


 uvcopy "selectf1,fild1=cbl0,arg1u=EXEC_CICS:>0,arg2u=EXEC_SQL:>0"
 =================================================================
 - select COBOL programs with 'EXEC CICS's AND 'EXEC SQL's

 uvcopy "selectf1,fild1=cbl0,arg1u=EXEC_CICS:>0,arg2u=EXEC_SQL:=0"
 =================================================================
 - select COBOL programs with 'EXEC CICS's AND WITHOUT (=0) 'EXEC SQL's

 uvcopy "selectf1,fild1=jcl0,arg1=PGM=SORT:>1"
 =============================================
 - select JCLs with more than 1 SORT

 uvcopy "selectf1,fild1=jcl0,arg1=PGM=SORT:>0,arg2u=INCLUDE_COND:>0"
 ===================================================================
 - select JCLs with 1 or more SORTs 'AND' that have at least 1 INCLUDE CONDition

 uvcopy "selectf1,fild1=jcl0,arg1=PGM=SORT:>0,arg2u=INCLUDE_COND:>0,arg3u=or:OMIT_COND:>0"
 =========================================================================================
 - select JCLs with SORTs 'AND' with INCLUDE CONDitions 'OR' OMIT CONDitions

 uvcopy "selectf1,fild1=jcl0,arg1=PGM=SORT:>0,arg2u=INCLUDE_COND:>0,arg3u=EXEC_PGM:<5"
 =====================================================================================
 - select JCLs with 'SORT' AND 'INCLUDE COND' and with less than 5 steps

Notes

  1. When checking JCL conversion results, I often want to find JCL's that contain features to be checked (such as SORT CONDitions), but that are not too long, so that is why I added arg3 < 5 'EXEC PGM's (in last example above).

  2. The '_' underscores shown in patterns (such as 'arg2u=INCLUDE_COND') represent spaces. Option 'u' of 'arg2u=...' converts the underscores back to spaces when stored from comamnd line.

  3. If you enter just 'uvcopy selectf1,fild1=jcl0' (without patterns on command line), then you will be prompted to enter arg1,arg2,arg3,arg4,arg5 interactively. Then you would key spaces (no need for underscores). Enter null for arguments not required.

  4. Always enter just 1 space (or underscore) between search words, because each multiple spaces are squeezed to just 1 space between words before comparisons.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4B2. selectf1 - select files depending on search pattern occurrences

Format & Rules

 uvcopy "selectf1,fild1=indir,fild2=outdir,filo1=tmp/filenames,filo2=tmp/copyscript\
        ,arg1=pattern1:=><99,arg2=pattern2:=><99,arg3=or:pattern3:=<>99,arg4=...,arg5=..."
 =========================================================================================
 - command FORMAT

 uvcopy selectf1,fild1=indir
 ===========================
 - minimum command line suggested
 - will be prompted for arg1,arg2,arg3,arg4,arg5
 - enter null for arguments not required
NOTE
  • must code uvcopy arguments in "quotes" if conditions > or < used
  • coding fild1,fild2,arg1,2,3,4,5 on command line optional, may wait for prompts
  • suggest coding at least fild1=...
fild1=...
  • specifies input directory of files to be searched
fild2=...
  • specifies output directory for the script of 'cp' comamnds
  • optional, might not require script, might only want to know filenames
  • defaults to tmp1/ - can change script with vi
 filo1=tmp/searchpattern1_list
           - file of filenames selected
           - defaults to searchpatter1 with '_list' appended
           - will be prompted for command to display filenames at EOJ
           - could enter vi, more, cat, etc
 filo2=tmp/searchpattern1_copy
           - script file of 'cp' commands to copy selected files to outdir
           - defaults to searchpatter1 with '_copy' appended
           - often ignored, when only interested in filenames
 arg1=pattern1:=><99     - must specify pattern1
 arg2=pattern1:=><99     - pattern2 optional
 arg2=or:pattern1:=><99  - may specify 'or:' on patterns 2,3,4,5
   --- up to 5 search args ---
 arg5=pattern1:=><99
 arg5=or:pattern1:=><99

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4B3. selectf1 - select files depending on search pattern occurrences

selectf1 - demo

 Here we will present the same demo as for 'grep' on page '4A1', but using
 'selectf1' which can search for up to 5 desired patterns with 1 command
 - here searching for 'SORT's  with 'INCLUDE COND' OR 'OMIT COND'.

 #1. Login userxx --> /home/userxx  (or whatever Login desired)

 #2. cdl     <-- alias cdl='cd $RUNLIBS' --> cd testlibs

 #3. rm -f tmp/* tmp1/*   <-- clear tmp/ dirs (used for output files)

 #4. uvcopy "selectf1,fild1=jcl0,arg1=PGM=SORT:>0,arg2u=INCLUDE_COND:>0,arg3u=or:OMIT_COND:>0"
     =========================================================================================

console log

 150330:201538:selectf1: EOF fili01 rds=21 size=929: jcl2/jar200.jcl
 150330:201538:selectf1: EOF fili01 rds=16 size=674: jcl2/jgl100.jcl
 150330:201538:selectf1: EOF fili01 rds=25 size=1123: jcl2/jgl200.jcl
 150330:201538:selectf1: EOF fili01 rds=30 size=1301: jcl2/jgl230.jcl
 150330:201538:selectf1: EOF fili01 rds=21 size=844: jcl2/jgl320.jcl
                 - - - 5 lines omitted - - -
 150330:201538:selectf1: EOF fili02 rds=10 size=115: tmp/selectf1files
 150330:201538:selectf1: EOF filo01 wrts=1 size=11: tmp/PGM_SORT_list
 150330:201538:selectf1: EOF filo02 wrts=1 size=55: tmp/PGM_SORT_copy

 #5. more tmp/PGM_SORT_list  <-- display filenames selected
     ======================    - shown on 2 lines to save space
      jar200.jcl    <-- only 1 JCL has EXEC 'PGM=SORT' & 'OMIT COND=...'

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4B4. selectf1 - select files depending on search pattern occurrences

repeat demo with 88 JCLs in mvstest


 #1. Login uvadm --> /home/uvadm

 #2. cd mvstest/testlibs

 #3. uvcopy "selectf1,fild1=jcl0,arg1=PGM=SORT:>0,arg2u=INCLUDE_COND:>0,arg3u=or:OMIT_COND:>0"
     =========================================================================================

console log

 150330:202746:selectf1: EOF fili01 rds=24 size=807: jcl2/adrdssu1.jcl
 150330:202746:selectf1: EOF fili01 rds=8 size=354: jcl2/eztlist.jcl
 150330:202746:selectf1: EOF fili01 rds=13 size=309: jcl2/ftpdemo1.jcl
 150330:202746:selectf1: EOF fili01 rds=9 size=408: jcl2/jar100.jcl
 150330:202746:selectf1: EOF fili01 rds=21 size=929: jcl2/jar200.jcl
                 - - - 83 lines omitted - - -
 150330:202746:selectf1: EOF fili02 rds=88 size=1027: tmp/selectf1files
 150330:202746:selectf1: EOF filo01 wrts=9 size=99: tmp/PGM_SORT_list
 150330:202746:selectf1: EOF filo02 wrts=9 size=495: tmp/PGM_SORT_copy

 #4. vi tmp/PGM_SORT_list  <-- see filenames selected
     ====================    - 9 JCL's selected of 88 total
      # selectf2 - select filenames from jcl2 - 2015/04/01_11:41:11
      # EXEC_PGM=SORT:>0 INCLUDE_COND:>0 or:OMIT_COND:>0
      jar200.jcl                    #   21
      jar205.jcl                    #   21
      jar410.jcl                    #   11
      jar420.jcl                    #   12
      jar430.jcl                    #   22
      jar440.jcl                    #   11
      jar450.jcl                    #   14
      jgl210.jcl                    #   24
      jgl215.jcl                    #   20

 #5. vi tmp/PGM_SORT_list  <-- see script to copy selected files to tmp1/...
     ====================    - ignore if all you need is list of filenames
      cp jcl2/jar200.jcl            tmp1
      cp jcl2/jar205.jcl            tmp1
        --- 5 lines omitted ---     tmp1
      cp jcl2/jgl210.jcl            tmp1
      cp jcl2/jgl215.jcl            tmp1

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4C1. Checking results, Searching for potential problems

scan/replace jobs more powerful than grep & sed

Vancouver Utilities provides several general purpose 'scan/replace' utilities that can be more effective than 'grep' & 'sed' in complex situations because:

  1. The search pattern can be qualified with 1 or 2 other patterns that must or must not be present on the same line.

  2. Table driven scan jobs provide unlimited no of search & qualifier patterns

  3. These scan jobs create a much more informative report, with page headings to document the search & qualifier patterns, directory scanned, date, etc - matching lines reported with line# - End-Of-File total hits & total lines for each file with at least 1 hit - End-Of-Directory total hits & total lines

  4. Some of these scan jobs allow for patterns on 1 line qualified by patterns on other lines, preceding or following.

  5. Most of these jobs show matched data lines with their line numbers, which are useful for subsequent edit sessions.

  6. The scan reports are handy for assigning work to other programmers. Scan for patterns & qualifiers that represent a problem & the report can be used as a check list during editing & to make notes about problem situations.

    sample problem for scan1d

Suppose we wanted to scan a directory of mainframe JCL for all 'EXEC's with CONDition codes. Here are a few lines (of thousands):

 //JOBA     JOB
 .......... most lines not shown ..........
 //STEP020  EXEC PGM=IDCAMS,COND=(4,LT,STEP010)
 //JOBB     JOB
 .......... most lines not shown ..........
 //STEP050  EXEC PGM=CGL100,COND=(4,LT),REGION=1024K,PARM=$PARM1

Operating Instructions


 uvcopy scan1d,fild1=jcl0,arg1=EXEC,arg2=PGM,arg3=COND,uop=p2
 ============================================================
 - could enter all info on the command line as above

 uvcopy scan1d,fild1=jcl0,uop=p2
 ===============================
 - OR you will be prompted for info not on the command line
 - see next page for console display prompts & entries

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4C2. Checking results, Searching for potential problems


 uvcopy scan1d,fild1=jcl0,uop=p2
 ===============================
 uop=q1a1b99999c0d256e0p1 - option defaults
       a1                 - begin scan at line #1 (default) of input file
       a0                 - causes prompt for search pattern to BEGIN scan
         b99999           - end scan at line #99999 (dflt) of input file
         b0               - causes prompt for search pattern to END scan
               c0         - begin scan at col #1 (zero rel) of input lines
                 d256     - length of scan area default 256 bytes
                     e1   - inhibit errmsgs when LF not found within rcsz
                       p1 - 1 present qual(31-60) & 1 absent qual(61-80)
                       p2 - 2 present quals(31-60+61-80) & NO absent quals
                       p4 - 2 absent quals(31-60+61-80) & NO present quals
 User OPtion (uop) defaults  = q1a1b99999c0d256e0p1
  null to accept or re-specify (1 or more) --> p2   <-- enter p2 2 present Quals
 enter search pattern (max 64) --------------> EXEC <-- enter search pattern
 enter 1st qualifier           --------------> PGM  <-- enter 1st qual
 enter 2nd qualifier           --------------> COND <-- entire 2nd qual
 enter options: i,p,q1/q2/q3,n,or null ----->
 080109:205346:scan1d: EOF filo01 wrts=47 size=2172: tmp/jcl0_EXEC hits=14
 EOJ, Output File written to: tmp/jcl0_EXEC
 enter command: vi,cat,more,lp,uvlp12,null -->

sample report

 Job: scan1d  Dir: jcl0  Search: EXEC  Qual1: PGM  Qual2: COND
  SearchBgn:    SearchEnd:    MatchOps:   UserOps: q1a1b99999c0d256e0p1p2
 ====================================================== 2008/01/09_20:53:46
 00005 //STEP020  EXEC PGM=CGL200,COND=(4,LT,STEP010),PARM=&YEAREND
 00010                   1 hits @EOF: jcl0/JGL220.jcl
 00011 //STEP020  EXEC PGM=IDCAMS,COND=(4,LT,STEP010)
 00018                   1 hits @EOF: jcl0/JGL320.jcl
 00012 //STEP020  EXEC PGM=CAR200,COND=(4,LT,STEP010),PARM=2006
 00017                   1 hits @EOF: jcl0/JAR200.jcl
 00006 //STEP020  EXEC PGM=IDCAMS,COND=(4,LT,STEP010)
 00011                   1 hits @EOF: jcl0/JPY402.jcl
 ====================================================== 2008/01/09_20:53:46
  EOD: 00014 hits in 0014 files of 0046 total (00576 lines)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4D1. Checking results, Searching for potential problems

scan2d - scan using table with multiple patterns

Suppose we wanted to find the EXEC's with COND= (similar to above scan1d), BUT only for certain utilties. We can edit a table of patterns for scan2d as follows. The table allows # comments.


 #1. vi ctl/utilcond.tbl    <-- prepare table of patterns
     ===================
 # utilcond.tbl - table of search patterns for scan2d
 #              - to find EXEC's with CONDs for a list of utilities
 # 01-30 - search pattern (ended by 1st tilde)
 # 31-60 - presence qualifier, must ALSO be present, disabled by tildes
 # 61-80 - absence qualifier, or 2nd presence qualifier if uop=p2
 IDCAMS~~~~~~~~~~~~~~~~~~~~~~~~EXEC~~~~~~~~~~~~~~~~~~~~~~~~~~COND=~~~~~~~~~~~~~~~
 IEBGENER~~~~~~~~~~~~~~~~~~~~~~EXEC~~~~~~~~~~~~~~~~~~~~~~~~~~COND=~~~~~~~~~~~~~~~
 IKJEFT~~~~~~~~~~~~~~~~~~~~~~~~EXEC~~~~~~~~~~~~~~~~~~~~~~~~~~COND=~~~~~~~~~~~~~~~
 SORT~~~~~~~~~~~~~~~~~~~~~~~~~~EXEC~~~~~~~~~~~~~~~~~~~~~~~~~~COND=~~~~~~~~~~~~~~~
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

 #2. uvcopy scan2d,fild1=jcl2,fili2=ctl/utilcond.tbl,uop=p2
     ======================================================
      filo1=tmp/jcl0_utilcond.tbl  <-- output file defaults to this
      uop=p2    <--option p2 for 2 presence qualifiers (vs 1 present & 1 absent)

 #3. vi tmp/jcl2_utilcond.tbl    <-- investigate the outpout report
     ========================
 Job: scan2d  Dir: jcl2  MatchOps: ndg2  UserOps: q1a1b99999c0d256p1t1p2
 TableFile: ctl/utilcond.tbl  SearchBgn:    SearchEnd:
 ====================================================== 2015/11/03_16:05:25
 IDCAMS~~~~~~~~~~~~~~~~~~~~~~~~EXEC~~~~~~~~~~~~~~~~~~~~~~~~~~COND=~~~~~~~~~~~~~
 IEBGENER~~~~~~~~~~~~~~~~~~~~~~EXEC~~~~~~~~~~~~~~~~~~~~~~~~~~COND=~~~~~~~~~~~~~
 IKJEFT~~~~~~~~~~~~~~~~~~~~~~~~EXEC~~~~~~~~~~~~~~~~~~~~~~~~~~COND=~~~~~~~~~~~~~
 SORT~~~~~~~~~~~~~~~~~~~~~~~~~~EXEC~~~~~~~~~~~~~~~~~~~~~~~~~~COND=~~~~~~~~~~~~~
 ====================================================== 2015/11/03_16:05:25
 00010 //STEP020  EXEC PGM=IDCAMS,COND=(4,LT,STEP010)
 00016                   1 hits @EOF: jcl2/jgl300.jcl
 00013 //STEP020  EXEC PGM=IEBGENER,COND=(4,LT,STEP010)
 00021                   1 hits @EOF: jcl2/jgl320.jcl
 00013 //STEP020  EXEC PGM=IKJEFT01,COND=(4,LT,STEP010)
 00027                   1 hits @EOF: jcl2/jgl330.jcl
 00013 //STEP020  EXEC PGM=IDCAMS,COND=(4,LT,STEP010)
 00027                   1 hits @EOF: jcl2/jgl331.jcl
 ====================================================== 2015/11/03_16:05:32
  EOD: 00009 hits in 0009 files of 0090 total (01576 lines)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4E1. Checking results, Searching for potential problems

scan3d - select lines between start & end patterns

'scan3d' will select lines from a starting pattern to an ending pattern, optionally qualified by a 3rd pattern present anywhere between start & end.

I have found this job very useful during JCL conversions, for example I wanted to investigate the various programs that were being run under IKJEFT01.

I will run scan3d on the converted scripts in jcl3/... (vs JCLs in jcl2/...) because I want to see the SYSIN & SYSTSIN parms that are inserted into the converted sripts.

IKJEFT01 steps will be something like:

 ##STEP0010 EXEC PGM=IKJEFT01                  <-- start pattern PGM=IKJEFT
   ---- lines omitted ----
 exportfile  SYSTSIN $RUNLIBS/parms/ikjdemo1
   ---- lines omitted ----
 #3----------------------------------------------------------------------
 cobrun $ANIM $CBLX/car100
 #4----------------------------------------------------------------------
                                               <-- end pattern '#4---'

 #1. uvcopy scan3d,fild1=jcl3,arg1=PGM=IKJEFT,arg2=#4---
     ===================================================
     ,filo1=tmp/$fild1_$arg1   <-- output file defaults to directory_pattern

 #2. vi tmp/jcl3_PGMIKJEFT     <-- inspect output report
     =====================
 Job=scan3d   scan files in Dir: jcl3
 Selecting From: "PGM=IKJEFT"  To: "#4---"  Qualified By: "~"
 ====================================================== 2015/11/03_16:27:13
 00019 ##STEP0010 EXEC PGM=IKJEFT01,DYNAMNBR=20
 00020 export PROGID=car100
 00022 exportfile  SYSTSIN $RUNLIBS/parms/ikjdemo1
 00023 #2-------------------------- parms/ikjdemo1 ---------------------------
 00024 # DSN SYSTEM(DB2P)
 00025 # RUN PROGRAM(CAR100) PLAN(IKJPROG1) LIB('MEPP.WXS91.LOADLIB')
 00026 # PARMS('parmsdata added on IKJEFT01 demo Aug28/07')
 00027 # END
 00030 #3----------------------------------------------------------------------
 00031 cobrun $ANIM $CBLX/car100
 00032 #4----------------------------------------------------------------------
 00052                      14 selected from    52 @EOF: jcl3/ikjeft01.ksh
 ====================================================== 2015/11/03_16:27:13
  EOD: 1 selects, 14 lines in 1 files of 90 total (5906 lines)

The output report will includes all matches from all JCL/scripts in the directory and there could be multiple matches in some jobs.

If you want to output separate files for each match, you can use alternate job 'scan3dd' (vs 'scan3d') & specify ',fild2=outputdirectory'

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_5 Mass changes to all files in directories of JCL & COBOL

Part 5 - Contents


5A1. Mass changing all JCL/scripts - summary of tools available

5B1. mass change example using 'sed' & script 'runsed1'
- uncomment all '#rm' file deletes after test/debug
- converter #comments out to assist test/debug
- could remove #comment after test/debug (or could leave as is)
- example using 'runsed1' to prompt for sed 's' substitute commands
  & apply while copying all files in directory to a 2nd directory

5C1. mass change example using sed, runsed1,& runsed2
 change all 'ksh' to 'ksh93', for clients that do not want
 to upgrade their current ksh (ksh88) to ksh93 (best for JCL/scripts)
- uses 'runsed1' for the 1s of 4 directories to be processed
- uses 'runsed2' to process other 3 using sed commands saved by runsed1

5D1. mass change using uvcopy job 'rep1d'
rep1d - copies all files from 1 dir to a 2nd dir
- prompts for 1 pattern to be replaced by a 2nd pattern
- optionally qualified by 2 other patterns (present or absent)

5E1. mass change using uvcopy job 'rep2d'
rep2d - copies all files from 1 dir to a 2nd dir
- searching/replacing depending on a pre-edited table
  of search/replace/qualify patterns

5F1. rep3d - replace specified blocks of code in all files in a directory
- blocks identified by a start pattern, an end pattern,&
  an optional qualifier pattern between start & end patterns
- replacement block must be pre-edited into a text file

5G1. uvfix1 - easy way to fix various problems in datafiles & scripts.
- allows you to apply the powerful uvcopy instructions
- translates, convert data field type (packed/binary/zoned)
- check output in tmp/... if OK - copy back to input directory
- example given to correct EBCDIC zoned signs to unix standards

5H1. uvfixA - same as uvfix1, but processes all files in a directory
  while copying to the tmp1/ directory
- check output & if OK - copy back to input directory

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5A1. Mass changes to all files in directories of JCL & COBOL

mass change tools summary

sed
  • unix utility, processes 1 file at a time
  • used by scripts 'runsed1' & 'runsed2' to process all files in directory
runsed1
  • VU script calling 'sed' to copy all files from 1 dir to 2nd dir
  • prompts for & applies sed commands such as 's' substitute
  • example given to change all '#rm' to 'rm' (uncomment)
runsed2
  • to rerun 'sed' commands saved by a prior 'runsed1'
rep1d
  • uvcopy job to copy all files from 1 dir to a 2nd dir
  • prompts for 1 pattern to be replaced by a 2nd pattern
  • optionally qualified by 2 other patterns (present or absent)
rep2d
  • uvcopy job to copy all files from 1 dir to a 2nd dir
  • must edit a table file of search/replace/qualify patterns
scan1d
  • uvcopy job to search all files in a directory
  • prompts for 1 pattern optionally qualified by 2 other patterns
  • might use prior to rep1d ?
scan2d
  • uvcopy job to search all files in a directory
  • must edit a table file of search/replace/qualify patterns
  • might use prior to rep2d ?
uvfix1
  • script solicits uvcopy commands applied to an input file
  • copies output to subdir tmp/... with same filename
  • would check output & if OK - copy back over input file
uvfixA
  • script solicits uvcopy commands applied to all files in directory
  • copies output files to subdir tmp1/... with same filenames
  • would check output & if OK - copy back to input directory

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5B1. Mass changes to all files in directories

runsed1 example - uncomment all '#rms' in all JCL/scripts

The JCL converter options control file may be coded with option 'v3', which causes all DELETEs to be #commnented when converted to unix 'rm's (#rm). This will help you debug jobs since the intermediate files often give you clues to the problem. Note that unix does not have a problem overwriting files as did the mainframe.

After test/debug complete you could run a sed script to mass change all '#rm's to 'rm's. I suggest using 'runsed1' (script in $UV/sf/util).


 #1. Login userxx --> /home/userxx
 #2. cdl  --> /home/userxx/testlibs (alias cdl='cd $RUNLIBS')

 #3. mv jcls jcls.old        <-- save (change name) of existing jcls subdir
     ================

 #4. mkdir jcls              <-- remake new dir
     ==========

 #5. runsed1 jcls.old jcls   <-- run script (will copy files to new jcls)
     =====================
     --> s/^#rm/rm/          <-- enter sed command at prompt
         ==========              '^' to match at begining of line
     --> control-D           <-- control-D to end command entry

 #6. diff jcls.old/jobxx.ksh jcls/jobxx.ksh  <-- spot check results ?
     ======================================
      -- OR use the 'alldiff2' script to check the entire directory

 #5. alldiff2 jcls.old jcls  <-- creates alldiff report in tmp/jcls.dif
     ======================

 #5a. vi tmp/jcls.dif        <-- examine alldiff report
      ===============

'alldiff2' is great after mass changes, to ensure yu changed what you intended and with no unintended changes.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5C1. Mass changes to all files in directories

mass change example ksh --> ksh93

Here is an example of a mass change that may be required at some sites that are not running a current version of the Korn shell (required for Vancouver Utilities & JCL conversions to Korn shell scripts).

Download (free) current version of ksh (1993) from www.kornshell.com Look for ksh standalone AT&T executable for your machine (Linux,AIX,SUN,HP). You do not need the source version, just download & copy into /bin because the 1st line of all VU scripts (& converted JCL/scripts) is '#!/bin/ksh'.

But what do you do if the customer does not want to update to the latest version & does not want you to modify any system directories such as /bin ?

We can download the lastest version (1993) of ksh, change the name to ksh93, and store it in /home/uvadm/bin/ksh93 (vs /bin/ksh). We then need to change line 1 of all VU scripts & modify ctl/jclunixop51 to convert JCLs to scripts with line 1 coded as '#!/home/uvadm/bin/ksh93' (vs '#!/bin/ksh').

download ksh & store in /home/uvadm/bin


 #1. Internet browser to www.kornshell.com

 #2. Look for ksh standalone AT&T executable for your machine (Linux,AIX,SUN,HP).

 #3. Download into /home/uvadm/bin/ksh

 #4. mv /home/uvadm/bin/ksh /home/uvadm/bin/ksh93  <-- rename as 'ksh93'
     ============================================

alllternative if you have root permissions

If you have root permissions, you can simplify things by storing ksh93 in /bin/ksh93 vs /home/uvadm/bin/ksh93.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5C2. Mass changes to all files in directories

mass change to all VU scripts

Vancouver Utility scripts are split into 4 subdirs as shown below. We will save current subdirs as adm.old, demo.old, IBM.old, util.old, make new subdirs & run mass change script 'runsed1' to modify.

 /home/uvadm
 :-----sf
 :     :-----adm
 :     :-----demo
 :     :-----IBM
 :     :-----util

 #1. Login uvadm --> /home/uvadm

 #2. cd sf

 #3. mv adm adm.old; mv demo demo.old; mv IBM IBM.old; mv util util.old
     ==================================================================
     - save existing subdirs by renaming to ---.old

 #4. mkdir adm demo IBM util tmp
     ===========================
     - make new subdirs to receive modified scripts

 #5. util.old/runsed1 adm.old adm
     ============================
     - runsed1 prompts for sed command, enter as follows:

 #5a. --> s:#!/bin/ksh:#!/home/uvadm/bin/ksh93:  <-- enter sed substitute command
          =====================================
 #5b. --> ^D   <-- enter control-D to end command entry
Note
  • runsed1 saves the command in tmp/runsed1
  • can now use 'runsed2' to avoid re-entering the command for other subdirs

 #6a. util.old/runsed2 demo.old demo tmp/runsed1
      ==========================================
 #6b. util.old/runsed2 IBM.old  IBM  tmp/runsed1
      ==========================================
 #6c. util.old/runsed2 util.old util tmp/runsed1
      ==========================================

 #7a. chmod 775 adm/*; chmod 775 demo/*; chmod 775 IBM/*; chmod 775 util/*
      ====================================================================
      - change permissions back to 775 (adding execute bits)

 #8. Optional - could run alldiff2 to verify changes as intended

 #8a. alldiff2 adm.old adm   <-- create tmp/adm.dif (check with vi)
      ====================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5C3. Mass changes to all files in directories

modify ctl/jclunixop51 to ksh93

The JCL converter generates 1st line of converted JCL/scripts as '#!/bin/ksh'. We must now change the control file to generate as '#!/bin/ksh93'. or '#!/home/uvadm/bin/ksh93' if not allowed to modify /bin or /usr/bin. Here are the 1st 2 lines & lines 86-101.


 jclunixop51:a2b3c0d1e2f0g1i15j0k15l50m4n3o0p0r0s0t1u1v3w0x0y1z1 #<-- demo options
 #           ===================================================
                   --- lines 3-85 omitted ---
 # Following lines will be inserted in scripts until :ENDINSERT
 # - except for '# ' comment (col1 '#', col2 ' ' blank)
 # '#!/.......' is 1st line output - usually '#!/bin/ksh' or '#!/bin/ksh93'
 #!/bin/ksh
 export JOBID2=JOBIDX jobid2=jobidx; scriptpath="$0"; args="$*"
 if [[ -z "$JOBID1" ]]; then export JOBID1=$JOBID2; fi
 for arg in $args; do if [[ "$arg" == *=* ]]; then export $arg; fi; done
 integer JCC=0 SCC=0 LCC=0  # init step status return codes
 autoload jobset51 gdgclear51 jobend51 jobabend51 logmsg1 logmsg2 stepctl51
 autoload exportfile exportgen0 exportgen1 exportgen2 exportgenall exportgenx
 . $APPSADM/env/stub.ini  #<-- for control-M (see notes in env/stub.ini)
 jobset51    # call function for JCL/script initialization
 #gdgclear51 $PUB_... #<-- insert gdgclear51's for exportgen1's to $PUBs
 goto
 S0000=A
 :ENDINSERT: - above non #comment lines inserted at begin output scripts

 #1. Login uvadm -->

 #2. vi ctl/jclunixop51   <-- edit JCL converter control file
     ==================

 #2a. --> #!/home/uvadm/bin/ksh93 <-- change line 89 (from #!/bin/ksh)
          =======================
  1. The JCL converter control file is copied from /home/uvadm/ctl/jclunixop51 to $RUNLIBS/ctl/... by script 'copymvsctls' on page '3D1'. If you have already run copymvsctls, you could now copy just jclunixop51 as follows:


 #3a. cp /ctl/jclunixop51 /home/userxx/testlibs/ctl
      =============================================
  1. But if you already modified some other options in this file, you would edit directly as in #2 above.


 #4a. Login userxx --> /home/userxx  (or whatever Login desired)

 #4b. cdl --> alias cdl='cd $RUNLIBS' --> /home/userxx/testlibs

 #4c. vi ctl/jclunixop51
      ==================
 #4d. --> #!/home/uvadm/bin/ksh93 <-- change line 89 to '#!/home/uvadm/bin/ksh'
          =======================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5D1. Mass changes to all files in directories

rep1d - mass change demo

'rep1d' is a Vancouver Utility pre-programmed uvcopy job to copy all files from 1 directory to another, while changing any 1 specified pattern to an alternate. 'rep1d' provides qualifiers & various options.

Suppose we wanted to insert 'acme.' prior to all filenames that begin with 'ar.' in the JCL/scripts. Here is an example of the BEFORE & AFTER desired.


 exportfile CUSTMAS data1/ar.customer.master       <-- BEFORE change
 ================================================
 exportfile CUSTMAS data1/acme.ar.customer.master  <-- AFTER change

We recommend you first change the name of the existing 'jcls' to 'jcls.old', make a new empty 'jcls' subdir, run the mass change while copying files from jcls.old to jcls,& then verify the results using the 'alldiff' script.


 #1. mv jcls jcls.old          - rename existing subdir (for backup)
     ================

 #2. mkdir jcls                - make new empty subdir (to receive changes)
     ==========

 #3. uvcopy rep1d,fild1=jcls.old,fild2=jcls
     ======================================
 uop=q1a1b99999c0d256e0p1 - option defaults
       a1                 - begin scan at line #1 (default) of input file
       a0                 - causes prompt for search pattern to BEGIN scan
         b99999           - end scan at line #99999 (dflt) of input file
         b0               - causes prompt for search pattern to END scan
               c0         - begin scan at col #1 (zero rel) of input lines
                 d256     - length of scan area default 256 bytes
                     e0   - do not erase any output files
                     e1   - erase output files with no replacements
                       p1 - 1 present qual(31-60) & 1 absence qual(61-80)
                       p2 - 2 present quals(31-60+61-80) & NO absent quals
 User OPtion (uop) defaults  = q1a1b99999c0d256e0p1
  null to accept or re-specify (1 or more) ----->          <-- defaults OK
 did you create tmp outdir or remove files -----> y        <--
 enter your search pattern (max 64 bytes) ------> ar.      <-- SEARCH pattern
 enter replacement pattern (max 64 bytes) ------> acme.ar. <-- REPLACE pattern
 enter qualifier present (null to disable) ----->          <-- n/r
 enter absent qual (2nd present if optn p2) ---->          <-- n/r
 match options: p(@,#,etc), s(in replace)
 quote inhibits: q1=singles,q2=doubles,q3=both
 enter match options: p,s,q1/q2/q3,n,or null --->          <-- n/r
 EOJ, OutFile written to: tmp/jcls.old_tgl0
 enter: vi,cat,more,lp,uvlp12,etc (or null) ----> more     <-- display audit file

Please see the 'rep1d' audit file on the next page --->

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5D2. Mass changes to all files in directories

'rep1d' creates an audit file of the 'after changes' images of any lines modified by the search/replace patterns. By default the audit file is created in the 'tmp' subdir & the filename is a concatenation of the input directory & the search pattern. For example: tmp/jcls_ar.

sample rep1d audit file

 Job: rep1d InDir: jcls.old OutDir: jcls Search: ar. Replace:acme.ar.
 QualPresent: export  Absent:   SearchBgn:    SearchEnd:
 MatchOptions:   UserOptions: q1a1b99999c0d256e0p1
 ====================================================== 2015/01/25_15:48:08
 00023 exportfile  CUSTMAS data1/acme.ar.customer.master
 00024 exportfile  NALIST data1/acme.ar.customer.nameadrs.list100
 00044                   2 reps @EOF: jcls/jar100.ksh
 00023 exportfile  SORTIN data1/acme.ar.sales.items
 00055 exportfile  CUSTMAS data1/acme.ar.customer.master.indexed
 00056 exportfile  SALELST data1/acme.ar.sales.list
 00075                   3 reps @EOF: jcls/jar200.ksh
 ====================================================== 2015/01/25_15:48:08
 EOD: 5 reps in 2 files of 9 total (582 lines)

verify results with 'alldiff'


 #4. alldiff jcls.old jcls
     =====================
 23,24c23,24
 < exportfile  CUSTMAS data1/ar.customer.master
 < exportfile  NALIST data1/ar.customer.nameadrs.list100
 ---
 > exportfile  CUSTMAS data1/acme.ar.customer.master
 > exportfile  NALIST data1/acme.ar.customer.nameadrs.list100
 diff file# 2 - jcls.old/... vs jcls/jar100.ksh
 23c23
 < exportfile  SORTIN data1/ar.sales.items
 ---
 > exportfile  SORTIN data1/acme.ar.sales.items
 55,56c55,56
 < exportfile  CUSTMAS data1/ar.customer.master.indexed
 < exportfile  SALELST data1/ar.sales.list
 ---
 > exportfile  CUSTMAS data1/acme.ar.customer.master.indexed
 > exportfile  SALELST data1/acme.ar.sales.list
 diff file# 3 - jcls.old/... vs jcls/jar200.ksh

2 different of 9 files compared jcls.old to jcls

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5E1. Mass changes to all files in directories

rep2d - table driven search/replace

'rep2d' copies all files from 1 dir to a 2nd, while performing mass change via a pre-edited table of search/replace patterns. 'rep1d' is handy when you only have 1 search/replace pair (because it solicits the patterns), but 'rep2d' is more powerful (because it allows an unlimited number of search/replace patterns in the pre-edited table file).


 #1. vi ctl/rep2d.tbl     <-- create table file of search/replace patterns
     ================         (in this case, inspect supplied demo table)
 # rep2d.tbl - search/replace table to demo uvcopy rep2d
 #           - insert acme. prefix on filenames ar... & gl...
 # 01-30 search                31-60 replace                 61-80 qualify
 ar.~~~~~~~~~~~~~~~~~~~~~~~~~~~acme.ar.~~~~~~~~~~~~~~~~~~~~~~export~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 gl.~~~~~~~~~~~~~~~~~~~~~~~~~~~acme.gl.~~~~~~~~~~~~~~~~~~~~~~export~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

 #2a. mv jcls jcls.old    <-- change name of existing jcls (to backup file)
      ================
 #2b. mkdir jcls          <-- create new subdir to receive updated files
      ==========

 #3.  uvcopy rep2d,fild1=jcls.old,fild2=jcls,fili3=ctl/rep2d.tbl
      ==========================================================
             - copy jcls.old to jcls with search/replace
             - also creates an audit file (tmp/jcls.old_rep2d.tbl)
 uop=q1a1b99999c0d256e0p1 - option defaults
       a1                 - begin scan at line #1 (default) of input file
       a0                 - causes prompt for search pattern to BEGIN scan
         b99999           - end scan at line #99999 (dflt) of input file
         b0               - causes prompt for search pattern to END scan
               c0         - begin scan at col #1 (zero rel) of input lines
                 d256     - length of scan area default 256 bytes
                     e0   - do not erase any output files
                     e1   - erase output files with no replacements
                       p1 - 1 present qual(31-60) & 1 absent qual(61-80)
                       p2 - 2 present quals(31-60+61-80) & NO absent quals
 User OPtion (uop) defaults  = q1a1b99999c0d256e0p1
  null to accept or re-specify (1 or more) ------>      <-- defaults OK
 did you create outdir or remove all files ------> y    <--
 match options: p=matchchars(@,#), s=also replace
 quote inhibits: q1=singles,q2=doubles,q3=both
 enter match options: p,s,q1/q2/q3,n,or null ---->      <-- n/r
 EOJ, OutFile written to: tmp/jcls.old_rep2d.demo
 enter command: vi,cat,more,uvlp12,etc(or null)--> more <-- display audit rpt

Please see audit file report on the next page --->

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5E2. Mass changes to all files in directories

sample rep2d Audit report

 Job: rep2d  InDir: jcls.old  OutDir: jcls  Table: ctl/rep2d.tbl
 SearchBgn:    SearchEnd:    MatchOptns: df2g2  UserOptns: q1a1b99999c0d256e0p1
 ====================================================== 2015/01/25_15:49:27
 ar.acme.ar.export
 gl.acme.gl.export
 ====================================================== 2015/01/25_15:49:27
 00023 exportfile  CUSTMAS data1/acme.ar.customer.master
 00024 exportfile  NALIST data1/acme.ar.customer.nameadrs.list100
 00044                   2 reps @EOF: jcls/jar100.ksh
 00023 exportfile  SORTIN data1/acme.ar.sales.items
 00055 exportfile  CUSTMAS data1/acme.ar.customer.master.indexed
 00056 exportfile  SALELST data1/acme.ar.sales.list
 00075                   3 reps @EOF: jcls/jar200.ksh
 00028 exportgen0 0 ACCTMAS data1/acme.gl.account.master_
 00029 exportgen1 +1 ACTLIST data1/acme.gl.account.acntlist_
 00051                   2 reps @EOF: jcls/jgl100.ksh
 00029 exportfile  SORTIN data1/acme.gl.account.tran1
 00030 exportgen1 +1 SORTOUT data1/acme.gl.account.trans_
 00057 exportgen1 +1 GLTRANS data1/acme.gl.account.trans_
 00059 exportgen0 0 GLMSOLD data1/acme.gl.account.master_
 00061 exportgen1 +1 GLMSNEW data1/acme.gl.account.master_
 00080                   5 reps @EOF: jcls/jgl200.ksh
 00024 exportfile  SORTIN data1/acme.gl.account.tran1
 00025 exportgen1 +1 SORTOUT data1/acme.gl.account.trans_
 00051 exportgen1 +1 GLTRANS data1/acme.gl.account.trans_
 00053 exportgen0 0 GLMSOLD data1/acme.gl.account.master_
 00054 exportgen1 +1 GLMSNEW data1/acme.gl.account.master_
 00074 exportgen0 0 ACCTMAS data1/acme.gl.account.master_
 00075 exportgen1 +1 ACTLIST data1/acme.gl.account.acntlist_ #(MOD,KEEP,DELETE)
 00097                   7 reps @EOF: jcls/jgl230.ksh
 00026 exportgen0 0 SYSUT1 data1/acme.gl.account.master_
 00051 exportgen1 +1 ACNTMAS data1/acme.gl.account.master_
 00077                   2 reps @EOF: jcls/jgl320.ksh
 ====================================================== 2015/01/25_15:49:31
  EOD: 00024 reps in 0008 files of 0009 total (00582 lines)

verify changes with alldiff script


 #4. alldiff jcls.old jcls >tmp/jcls.dif   <-- create alldiff report
     ===================================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5F1. Mass changes to all files in directories

rep3d - replace blocks of code in all files in directory

'rep3d' will replace specified blocks of code in all files in a directory. Blocks of code are identified by a start pattern, an end pattern,& an optional qualifier pattern between start & end patterns. The replacement block must be pre-edited into a text file.

For example, let's assume we want to replace the FTP commands as follows:

      open 214.69.21.9         <-- existing IP# (& start pattern)
      mvstest mvstest11
      cd ftpdata1
      binary
      open $IP_214_69_21_9     <-- replace with $variable
      mvstest mvstest22        <-- change password
      pwd                      <-- insert pwd for console log
      cd ftpdata2              <-- change directories
      binary                   <-- end pattern for block of code

 #0a. login
      =====
 #0b. cdl (alias cd $RUNLIBS) --> /home/userxx/testlibs/

 #1. vi ctl/ftp2           <-- edit the replacement block as shown above
     ===========

 #2. mkdir tmp tmp1        <-- make output subdirs
     ==============
 #2a. rm -f tmp/*  tmp1/*    - or remove all files if tmp1 already exists

 #3. uvcopy rep3d,fild1=jcl3,fild2=tmp1,fili3=ctl/ftp2,filo4=tmp/rep3d.audit
     =======================================================================
      prompt begin pattern--> open 214.69.21.9
      prompt end pattern ---> binary
      prompt qualifier -----> =mvstest        '=' present or '!' not present
      prompt match options-->   <-- null reply (or 'i' case insensetive)

 #4. vi tmp/rep2d.audit    <-- check audit report replacements made
     ==================
 #5. vi tmp1/*             <-- check output JCL/scripts
     =========

 #6. mv jcl3 jcl3.old      <-- save original
     ================

 #7. mv tmp1 jcl3          <-- rename tmp1 to jcl3 if AOK
     ============

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5G1. Mass changes to all files in directories

uvfix1 - Vancouver Utility file fixups

'uvfix1' is a script that makes it easy to use the powerful repertoire of uvcopy instructions to fix various data file problems. See all uvcopy instructions documented at https://uvsoftware.ca/uvcopy3.htm.

The 'uvfix1' script saves having to write the framework of file I/O and prompts you for the instructions to fix your data problem. For example, suppose you have FTP'd a GL transaction file in text mode which translates to ASCII, but any EBCDIC negative zone signs will appear as shown below in our gltranE demo file. Negatives '0123456789' apear as '}JKLMNOPQRS'.

          1         2         3         4         5
 12345678901234567890123456789012345678901234567890
 00011100T BANK FEE                      000000100}
 00061500T BACKUP TAPE DRIVE             000011508J
 00061600T SONY BACKUP TAPES             000000805K
 00062200T STAPLES OFFICE SUPPLIES       000001152L
 00075300T TELEPHONE                     000000525M
 00083700T HYDRO                         000000034N
 00083800T WATER                         000000109O
 00073700T FEDEX                         000000109P
 00075100T OFFICE RENT                   000003000Q
 00063300T DOLLAR RENT A CAR             000001417R

For unix, negative signs for -0123456789 must be translated to 'pqrstuvwxy', because the bit pattern for these letters is the same as numerics 0123456789 with a 4 bit added, ie 0=x'30' & p=x'70', 1=x'31' & q=x'71', etc.

uvcopy provides the 'trt' instruction & translate table '$trtsea' to convert EBCDIC the sign convention to the unix sign convention used by Micro Focus & AIX COBOL & Vancouver Utilities.


   trt b49(1),$trtsea   <-- uvcopy instruction to convert EBCDIC signs to unix
   ==================

Normally you need to write a complete 'uvcopy job' with a framework of I/O file definitions & get/put instructions - as follows:

 # fixsigns - uvcopy job to convert EBCIC sign convention to unix
 #          - for files FTP'd in text/ASCII mode
 fili1=data1/gltranE,rcs=80,typ=LST
 filo1=tmp/gltranA,rcs=80,typ=LSTt
 @run
        opn    all              open files
 loop   get    fili1,a0(80)     get next record into area 'a'
        skp>   eof              (cc set > at EOF)
        mvc    b0(80),a0        move inarea 'a' to outarea 'b'
        trt    b49(1),$trtsea   translate Sign in col 50 (49 zero relative)
        put    filo1,b0(80)     write record to output fiel
        skp    loop             return to get next record
 eof    cls    all              EOF - close files
        eoj                         - end job

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5G2. Mass changes to all files in directories

using 'uvfix1' to make file fixups easy


 #1. Login userxx --> /home/userxx  (or whatever Login desired)
 #2. cdd   --> /home/userxx/testdata (alias cdd='cd $RUNDATA')

 #3. uvfix1 data1/gltranE   <-- uvfix1 prompts as follows:
     ====================
 uvfix1 - copy a text file & modify via uvcopy instruction keyins
        - input rec in area a has already been copied to outputrec in b
 uop=q1n99r8192 - user option defaults
       n99      - output record count limit (high# copies to EOF)
          r8192 - recsize ONLY for Fixed-Length OUTPUT records if desired
                - for Fixed-Length, reply RSF/RST at prompt (default text typ=LSTt)
                - for Text files: accept default recsize & default typ=LSTt
 User OPtion (uop) defaults  = q1r8192n999999999
 -->null to accept or enter/override    -->   <-- null entry for text files
 LSTt = typ default - accept or new typ -->   <-- null entry accept LSTt
 enter a uvcopy instruction (period "." terminates entries)

 entry --> trt b49(1),$trtsea  <-- entry
           ==================
 entry --> .                   <-- enter '.' to terminate
 150125:111159:uvfix1: EOF fili01 rds=10 size=510: data1/gltranE
 150125:111159:uvfix1: EOF filo01 wrts=10 size=510: tmp/gltranE

 cat tmp/gltranE   <-- inspect output to confirm desired results
 ===============
 00011100T BANK FEE                      000000100p
 00061500T BACKUP TAPE DRIVE             000011508q
 00061600T SONY BACKUP TAPES             000000805r
 00062200T STAPLES OFFICE SUPPLIES       000001152s
 00075300T TELEPHONE                     000000525t
 00083700T HYDRO                         000000034u
 00083800T WATER                         000000109v
 00073700T FEDEX                         000000109w
 00075100T OFFICE RENT                   000003000x
 00063300T DOLLAR RENT A CAR             000001417y

 cp tmp/gltranE data1/gltranA
 ============================
 - copy back to data1/... changing name to preserve original for this demo
 - normally you would overwrite same name

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_6 Optional Conversion Enhancements

Part 6 - Contents


6A1. add data file info onto JCL/script file definitions (as #comments)
- appends Record-Sizes & Packed/Binary
- examples of before/after adding datafileinfo as #comments

6A2. ctl/datactl53I - control file info for JCL conversion

6B1. add data file info onto JCL/script file definitions (as #comments)
- step by step instructions

6C1. cblcode1 - determine COBOL program type
- reads a directory of COBOL programs & writes a file of filenames
  with extensions to identify the program type.
- filenames wwith suffix .cbl, .cdb, .cic depending presence of
  EXEC SQL, EXEC CICS, or neither.
- also writes a 2nd file script to copy the directory to a 2nd
  directory appending the .cbl/.cdb/.cic extensions

Reminders to me

Might add 'IP# replace' to part 6 here in JCLcnv3aids.doc from JCL2cnvreal.doc#1E5 - 1E7 & 2A1

Might add jclunixop51 listing somewhere in this doc ?

Might add chkrcs1, chkrcs2, chkrcs2A


 uvcopy chkrcs1,fild1=jcl3,filo2=rpts/chkrcs1.rpt
 ================================================
 - check record sizes on uvsort/uvcp in JCL/scripts
 - see MVSJCL.doc or www.uvsoftware.ca/mvsjcl.htm

 uvcopy chkrcs2,fild1=jcl3,filo1=rpts/chkrcs2_fj,filo2=rpts/chkrcs2_jf
 =====================================================================
 - check record sizes on uvsort/uvcp in JCL/scripts
 - report datafilenames for rcs=99999 from saved SORTIN exports

 chkrcs2A jcl3     <-- execute uvcopy job chkrcs2 + sorts
 =============

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6A1. Optional Conversion Enhancements

jcldatafileinfo - append file info on datafiles as #cmts

The converted JCL/scripts are in jcl3/..., but need to be copied to jcls/... for execution since only jcls is in $PATH (defined in the common_profile).

In November 2013, we added another step in the JCL conversion - running script 'jcldatafileinfo' which copies jcl3/* to jcl4/... appending COBOL I/O & record-size info (as #comments) on the right side of data file definitions, for example, here are the BEFORE & AFTER data files for step2 of jgl230.jcl. See complete listing beginning on page '2E1'.

Before & After jcldatafileinfo

      exportgen1 +1 GLTRANS data1/gl.account.trans_
      exportgen0 0 GLMSOLD data1/gl.account.master_
      exportgen1 +1 GLMSNEW data1/gl.account.master_
      exportgen1 +1 GLTRANS data1/gl.account.trans_      #cblIO=I/SS,rsz=80
      exportgen0 0 GLMSOLD data1/gl.account.master_      #cblIO=I/SS,rsz=128p
      exportgen1 +1 GLMSNEW data1/gl.account.master_     #cblIO=O/SS,rsz=128p

The appended #comments provide INVALUABLE info for JCL/script test/debug:

Note that 'cobfiles5A' is a Pre-Requisite for jcldatafileinfo. It creates Indexed file ctl/cobfilesI.dat/.idx with file info extracted from the COBOL programs & copybooks.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6A2. jcldatafileinfo - append file info on datafiles as #cmts

ctl/datactl53I - control file info for JCL conversion

 ar.customer.master    cpy=____________ rcs=00256 typ=RSF  data=p  job=jar100 prg=CAR100
 ar.sales.items        cpy=____________ rcs=00064 typ=LST  data=   job=jar200 prg=SORT
 gl.account.acntlist_  cpy=____________ rcs=00133 typ=LST  data=   job=jgl100 prg=CGL100
 gl.account.master_    cpy=____________ rcs=00128 typ=RSF  data=p  job=jgl230 prg=CGL200
 gl.account.trans_     cpy=____________ rcs=00080 typ=LST  data=   job=jgl230 prg=SORT
 gl.parms@pgl200s1     cpy=____________ rcs=_____ typ=RSF  data=   job=jgl200 prg=SORT
 py.payroll.master     cpy=____________ rcs=00080 typ=RSF  data=pb job=jpy200 prg=CPY200

'jcl2ksh51A' (run on page '3F1') created ctl/datactl53I, an Indexed control file with data file info (record-sizes, file-types,etc). This file is updated by cobfiles5A, mvsfiles5A,& jcldatafileinfo - so you can improve the conversions by running these jobs & re-converting with 'jclxx51' to get more record-sizes & file types coded on the JCL/scripts.

Note
  • 'data=p' or data=pb' identifies files with Packed/Binary fields
  • used by JCL converter to code uvsort/uvcp steps with proper file type
  • 'typ=RSF' (Record Sequential Fixed) for files with Packed/Binary fields
  • 'typ=LST' (Line Sequential Terminated) could be used for files without P/B
  • also see option 'f0/f1' in ctl/jclunixop51 (listed on previous page)
    and option 'f0/f1' prompt when running jcldatafileinfo which copies
    jcl3/* to jcl4/... encoding file-types & record-sizes

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6B1. Optional Conversion Enhancements

Append Record-Sizes & Packed/Binary #info on data-files


 #1. Login userxx --> /home/userxx  (or whatever Login desired)

 #2. cdl     <-- alias cdl='cd $RUNLIBS' --> cd testlibs

 #3. cobfiles5A cbls cpys maps
     =========================
     - create Indexed file info for jcldatafileinfo

 #3a. uvlp12 ctl/cobfiles  <-- good idea to print
      ===================    - useful report to assist test/debug

 #4. mvsfiles5A jcl2   <-- updates ctl/datactl53I with rec-sizes & packed indicators
     ===============

 #4a. uvlp12D xmvs/mvsfiles5 <-- could print file info report,
      ======================   - but too big for all JCL, better to print
                                 for 1 JCL (after mvsfiles51 vs mvsfiles5A)
Note
  • see complete doc for mvsfiles5A on pages '1L1' - '1L8'
  • note that 'mvsfiles51' (vs mvsfiles5A) is great for creating the
    data files report for 1 job at a time, run it before test/debug

 #5. jcldatafileinfo jcl3 jcl4
     =========================
     - copy jcl3/* to jcl4, appending #cmts of COBOL I/O, recsize,Packed info
     - invaluable during test/debug
     - also updates ctl/datactl53I to improve conversion when re-converted
     - prompts for option f1 ('typ=LST' on uvsort/uvcp files w/o packed/Binary)
       ---> f0 <--- reply f0 if you have many files with Packed/Binary
       ---> f1 <--- reply f1 if you have very few files with P/B
Note
  • could copy jcl4 to jcls (directory in $PATH) & run JCL/scripts
  • but best to reconvert to improve encoding of record-sizes & file types
  • see re-conversion on the next page ---->

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6B2. Append Record-Sizes & Packed/Binary #info on data-files

Re-convert JCL to improve encoding record-sizes & file-types


 #6. jclxx51 jcl2 jcl3    <-- reconvert all JCL jcl2/* --> jcl3/...
     =================      - after cobfiles5A, mvsfiles5A,& jcldatafileinfo
                            - update ctl/datactl53I record-sizes & packed indics

 #7. jcldatafileinfo jcl3 jcl4  <-- rerun to #encode COBOL I/O & rec-sizes & p indics
     =========================

 #8. cp jcl4/* jcls    <-- recopy to jcls/... (directory in $PATH)
     ==============
     - would not copy all for real conversion if you have edited a lot of scripts
     - OK here for test/demos

 #8a. cp jcl4/jobnamex.ksh jcls  <-- copy 1 script at a time before test/debug
      =========================    - recommended for your "real conversion"

You should NOT copy all jcl4/* jcls/ for your "real conversion" project. For your real conversion project, you should copy 1 script at a time when you ready to start test/debug on each script. This is an easy way to keep track of your progress (jcls/... are debugged, jcl4/... not yet)

Another reason NOT to copy all jcl4/* jcls/ when JCL/scripts re-converted:

alldiff2 - confirm changes in options

If you are re-converting after changing options, it is a good idea to 1st save the current conversion results in jcl4/ as jcl4.old, reconvert,& then run script 'alldiff2' which creates a report (tmp/jcl4.dif) showing the differences in the scripts (caused by option changes).


 #1. mv jcl4 jcl4.old        <-- save current results as .old
 #2. mkdir jcl4              <-- make new dir for re-conversions
 #3. vi ctl/jclunixop51      <-- change options as desired

 #6. jclxx51 jcl2 jcl3          <-- reconvert (same as above)
 #7. jcldatafileinfo jcl3 jcl4  <-- rerun #encode COBOL I/O, rec-sizes & p/b indics

 #8. alldiff2 jcl4.old jcl4  <-- create differences report
     ======================
 #8a. vi tmp/jcl4.dif        <-- investigate differences
      ===============

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6B3. Append Record-Sizes & Packed/Binary #info on data-files

updating JCL data control file & reconvert

'jcl2ksh51A' (run on page '3F1') created ctl/datactl53I, an Indexed control file with data file info (record-size, etc). This file is updated by jcldatafileinfo and it might help to re-convert with jclxx51 to get more record-sizes coded on the JCL/scripts. If you wanted to update manually, you should unload the Indexed file to sequential file ctl/datactl53, edit,& reload the Indexed file as follows:


 #1. uvcp "fili1=ctl/datactl53I,rcs=191,typ=ISF,filo1=ctl/datactl53,typ=LSTt"
     ========================================================================
     - convert Indexed file back to sequential text file for 'vi' editor changes

 #2. vi ctl/datactl53    <-- edit data control file
     ================      - update record-sizes, file-types

 #3. uvcp "fili1=ctl/datactl53,rcs=191,typ=LST,key1=0(44),filo1=ctl/datactl53I\
                              ,typ=ISF,isk1=0(44),trl=0(44)"
     ==========================================================================
     - reload Indexed file for JCL converter
      --- OR (easier to key) ---

 #3a. uvcopy gdgload51    <-- reload Indexed file (alternate/easier method)
      ================

 #4. jclxx51 jcl2 jcl3          <-- reconvert all JCL jcl2/* --> jcl3/...
     =================            - after option changes

 #5. jcldatafileinfo jcl3 jcl4  <-- rerun to #encode COBOL I/O & rec-sizes
     =========================

 #6. cp jcl4/* jcls    <-- recopy to jcls/... (directory in $PATH)
     ==============
     - would not copy all for real conversion if you have edited a lot of scripts
     - OK here for test/demos

mvsfiles51 - Determine Essential INPUT files

Also note scripts 'mvsfiles51/mvsfiles5A' to determine the essential INPUT files required to run a job (ignoring intermediate files (outputs for later inputs). The most useful report is 'xmvs/mvsfiles5' which '*' flags the INPUT files actually needed to run any 1 job or all jobs. This report eliminates duplicates & intermediate files & identifies the essential files required to be be transfered from the mainframe. See details on pages '1L1' - '1L8'.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6C1. Optional Conversion Enhancements

cblcode1 - determine COBOL program type

'cblcode1' is a uvcopy job to code COBOL source programs to 3 types by appending suffix .cbl, .cdb, .cic depending presence of EXEC SQL, EXEC CICS, or neither. It reads all programs in the directory, determines the type and writes 2 files:


 #1. A file of filenames coded with .cbl/.cdb/.cic

 #2. A script to copy the input directory of COBOL programs to a 2nd directory
     appending .cbl/.cdb/.cic extensions, depending on presence of 'EXEC CICS'
     or 'EXEC SQL' or neither
  1. .cbl - if neither EXEC CICS or EXEC SQL found
  2. .cdb - if EXEC SQL found but not EXEC CICS
  3. .cic - if EXEC CICS found (EXEC SQL may or may not be present)

    operating instructions


 uvcopy cblcode1,fild1=cbl0,filo1=ctl/cbl0coded,filo2=sf/cbl0copycode
 =====================================================================
 - here is a sample of the 2 output files:

# cblcode1 - 1st output ctl/cbl0coded filenames with ext .cbl/.cdb/.cic a04621a.cbl dehp0405.cdb ytrans1.cbl

# cblcode1 - 2nd output sf/cbl0copycode script to copy to cbl0c/filename.cbl/.cdb..cic cp cbl0/a04621a.cbl cbl0c/a04621a.cbl cp cbl0/dehp0405.cbl cbl0c/dehp0405.cdb cp cbl0/ytrans1.cbl cbl0c/ytrans1.cbl

'cblcode1' is executed by 'xrefall' & output file of filenames coded with .cbl .cdb .cic used to flag program types on crossrefs no flag on .cbl, '+' flag on EXEC SQL, '@' flag on EXEC CICS

The 2nd output file (script to copy to 2nd directory appending extensions) is not used by xrefall, but you might have occasion to use it. If you wanted to split the 3 types of COBOL programs to 3 separate directories you could run the ctl/cbl0coded script & then simply move extensions .cdb & .cic to seprate subdirs


 #1. ctl/cbl0coded   - script copies cbl0/... to cbl0c/... coding .cbl/.cdb/.cic
     =============

 #2. mv cbl0c/*.cdb cbl0cdb   <-- move .cdb programs to separate directory
     ======================

 #3. mv cbl0c/*.cdb cbl0cic   <-- move .cic programs to separate directory
     ======================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_7 Test/Verify Critical Features in Converted JCL/scripts

Part 7 - Contents


7A1. Review of job-logging & console-logging

7B1. Test 'joblog2' run JCL/scripts serially - prove it works using jar100.ksh
- insert prompt/reply, run job from 2 screens, note delay
- answer prompt on 1st job to complete & allow 2nd job to run

7C1. gdgreset1 - test automatic GDG generation# re-sequence at 900000
'gdgrest1' resets gdg#s for 'ONE' file automatically, whenever gdg#
900000 exceeded. 'gdgreset1' is called by function 'jobend51' which was
inserted by the JCL converter at the end of all JCL/scripts.

7D1. gdgreset2 - test batch job to re-sequence gdg#s > specified threshold
'gdgreset2' will reset GDG#s for 'ALL' files whose GDG#s exceed a
specified threshold (suggested default 500000). You would run this
batch job when no JCL/scripts are being executed.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7A1. Test/Verify Critical Features in Converted JCL/scripts

Job-Logging & Console-Logging

Console-Logging captures the entire login session from login to logout (vs the 'joblog' script which captures the log only for 1 job at a time).

'console-logging' is better for batch shifts where an operator is manually running the JCL/scripts, 'better' because it captures the entire login session, including any commands between jobs and any responses to prompts.

'job-logging' is better for programmers who are running 1 job at a time and may want to review the log for jobs that roll off the screen. See 3 joblog scripts reviewed on the next page.

Console Logging

  1. Console-Logging will capture the entire login session from login to logout (vs the 'joblog' script which captures the log only for 1 job at a time).

  2. Console-logging captures ALL screen I/O including any reponses to prompts, and any unix commands the operator might do between running jobs.

  3. Console-logging requires subdirs matching your login name ($LOGNAME) to be setup in $APPSADM/log1,log2,log3.

  4. The log is collected in $APPSADM/log1/$LOGNAME and processed to $APPSADM/log2/$LOGNAME (to remove screen control characters), either by a nightly scheduled cron job, or by logging off & back on.

  5. If operators need to see the console log, they need to logoff & back on They can then use the 'logview' script to see the recent logs.

  6. To activate console-logging you must uncomment the 9 '##' lines near the end of your .profile.

  7. You must also have copied the supplied $APPSADM/env/kshrc to your .kshrc (or $APPSADM/env/bashrc to .bashrc if you use the bash shell vs Korn shell) .kshrc/.bashrc set up the PS1 prompt expected by the logfix processing.

  8. .kshrc/.bashrc also contain the same aliases & umask as the common_profile. These would otherwise be lost by the 'script' command (at end of profile) because 'script' invokes another level of the shell.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7A2. Test/Verify Critical Features in Converted JCL/scripts

Job-Logging

Three versions of the joblog script are provided:

joblog
  • intended for programmer test/debug
  • does NOT date/time stamp joblog/filename
  • prompts for logfile disposition (vi, uvlp12, date/time stamp, etc)
joblog1
  • date/time stamps joblog/filename_date_time
  • does NOT prompt for logfile disposition
  • could use for production if running jobs serially
    either manually or by a scheduler that submits jobs serially
joblog2
  • date/time stamps joblog/filename_date_time
  • waits for any existing same-name job to complete
    before starting a new job of same-name
  • recommended for production by multiple operators
    that may run the same job at the same time.

joblog sample


 joblog jar100.ksh     <-- run jar100.ksh using 'joblog' to capture log
 =================         into $RUNDATA/joblog/jar100.log

 vi joblog/jar100.log  <-- review joblog (good for long jobs)
 ====================

 uvlp12 joblog/jar100.log  <-- can print to document problems
 ========================      for resolution by reponsible programmer
 141210:205215:jar100: Begin Job=jar100
 141210:205215:jar100: /home/mvstest1/testlibs/jcls/jar100.ksh
 141210:205215:jar100: Arguments: jar100.ksh
 141210:205215:jar100: RUNDATE=20141210
 141210:205215:jar100: RUNLIBS=/home/mvstest1/testlibs
 141210:205215:jar100: RUNDATA=/home/mvstest1/testdata
 141210:205215:jar100: JTMP=jobtmp/jar100
 141210:205215:jar100: SYOT=sysout/JAR100
 141210:205215:jar100: LOGNAME=mvstest1 TESTPROD=T000
 141210:205215:jar100: HOSTNAME=uvsoft4
 141210:205215:jar100: ******** Begin Step S0010 car100 (#1) ********
 141210:205215:jar100: file: CUSTMAS=data1/ar.customer.master fsize=8K
 141210:205215:jar100: file: NALIST=data1/ar.customer.nameadrs.list100 fsize=K
 141210:205215:jar100: file: SYSOUT=sysout/JAR100/jar100_S0010_SYSOUT_141210_205215 fsize=K
 141210:205215:jar100: Executing--> cobrun -F /home/mvstest1/testlibs/cblx/car100
 141210:205215:JAR100: StepTimes: S0010 car100 Begun=20:52:15End=20:52:15 Elapsed=00:00:00
 141210:205215:JAR100: Job Times: Begun=20:52:15 NormalEnd=20:52:15 Elapsed=00:00:00
 141210:205215:jar100: JobEnd=Normal, StepsExecuted=1, LastStep=S0010

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7B1. Test/Verify Critical Features in Converted JCL/scripts

joblog2 - script to wait for same-name job completion

 # joblog2 - run JCL/script with delay until any prior same script completes
 #         - creates logfile in subdir joblog/... with JCL/script name
 #         - appends _date_time stamp & return code
 #           (which changes name & allows waiting same jobnames to run)
 #         - writes the log file to joblog/jobname_yymmdd_hhmmss_RC
 #
 # Versions of joblog:
 # joblog  - names log file same as JCL/script with .log replacing .ksh
 #         - intended for test/debug
 # joblog1 - appends date/time stamp to log file
 # joblog2 - delays execution if same jobname already running
 #         - date/time stamps at end of job (allows waiting job to run)
 #
 jclksh="$1"          # capture script filename with extension (jclname.ksh)
 jcl=${jclksh%\.*}    # drop extension .ksh from JCL/script filename
 if [[ -f $RUNLIBS/jcls/$jclksh || -f $RUNLIBS/ksh/$jclksh ]]; then :
   else echo "usage: joblog2 jclname.ksh [args]"
        echo "       =========================="
        echo " - arg1 must be a script in the PATH"
        exit; fi
 #
 # setup joblog directory pathname & make joblog subdir if not existing
 #Jan08/2014 - ensure write to $RUNDATA (testdata) no matter where run
 jld=$RUNDATA/joblog    # setup joblog directory pathname
 if [[ ! -d $jld ]]; then mkdir $jld; fi
 #
 # delay until any other same jobname script completes
 stime=2; stotal=0
 until [[ ! -f $jld/$jcl ]]
   do  echo "waiting $stime seconds for same-name job to complete"
       sleep $stime
       ((stotal+=stime))
       if [[ $stotal -gt 4 ]]; then stime=10; fi
       if [[ $stotal -gt 60 ]]; then stime=60; fi
   done
 #
 # create logfilename by appending date/time stamp
 jcllog=${jcl}_$(date +%y%m%d_%H%M%S)
 #
 $jclksh $* 2>&1 | tee $jld/$jcl
 #==============================
 # "$*" will include all arguments spcfd after jclname.ksh
 JCC=$?  # capture return code to append on logfile name
 # mv $jld/$jcl $jld/${jcllog}_$JCC
 # - does not work because we get status of tee (vs $jclksh)
 #
 # change name of logfile by appending _date_time on JCL/script name
 # - which will allow any waiting same-name job to run
 mv $jld/$jcl $jld/$jcllog
 #========================
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7B2. Test/Verify Critical Features in Converted JCL/scripts

joblog2 - demo preparation

We will modify demo job jar100.ksh, inserting a pause so we can start a 2nd copy before 1st copy completes.


 #1. Login --> /home/userxx
 #2. cdl   --> /home/userxx/testlibs
 #3. vi jcls/jar100.ksh  <-- edit jar100.ksh to insert following at line 34
     ==================

 --> echo "enter to continue"; read reply  #<-- wait for 2nd job to be run  <--**
     ====================================     - see line 34 in script below
 #!/bin/ksh
 ##JAR100   JOB  'TEST MVS2UNIX CONVERSION',DATE=20080323
 export jobid2=jar100 JOBID2=JAR100; scriptpath="$0"; args="$*"
 if [[ -z "$jobid1" ]]; then export jobid1=$jobid2; fi
 for arg in $args; do if [[ "$arg" == *=* ]]; then export $arg; fi; done
 integer JCC=0 SCC=0 LCC=0  # init step status return codes
 autoload jobset51 jobend51 jobabend51 logmsg1 logmsg2 stepctl51
 autoload exportfile exportgen0 exportgen1 exportgen2 exportgenall exportgenx
 . $APPSADM/env/stub.ini  #<-- for control-M (see notes in env/stub.ini)
 jobset51    # call function for JCL/script initialization
 goto
 S0000=A
 # * CUSTOMER NAME & ADDRESS LIST (WITH DATE & RPT HDNGS)
 #1======================= begin step#S0010 CAR100 ========================
 S0010=A
 stepctl51;  #calc StepTimes & check jobstop/goto end
 goto
 export JSTEP=S0010; ((XSTEP+=1)); SCC=0; LCC=0; alias goto="";
 logmsg2 "******** Begin Step $JSTEP car100 (#$XSTEP) ********"
 ##STEP010  EXEC PGM=CAR100,PARM=2008
 export PROGID=car100
 export PARM="2008"
 exportfile  CUSTMAS data1/ar.customer.master
 exportfile  NALIST data1/ar.customer.nameadrs.list100
 #3----------------------------------------------------------------------
 cobrun $ANIM $RLX/car100
 #4----------------------------------------------------------------------
 LCC=$?; S0010C=$LCC; ((SCC|=LCC)); ((JCC|=LCC)); S0010R=1; alias goto="";
 if ((S0010C != 0))
    then logmsg2 "ERR: step#$JSTEP car100 abterm $SCC"
    alias goto="<<S9900=\A"; fi
 goto
 #8======================================================================
 echo "enter to continue"; read reply     #<-- wait for 2nd job to be run   <--**
 #8======================================================================
 #Note - End of job Normal & Abnormal code omitted to save space

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7B3. Test/Verify Critical Features in Converted JCL/scripts

joblog2 - demo execution

This assumes you have performed the 'demo preparation' above (inserting a prompt to stop job 1, allow running job 2, then reply to job 1)


 #1. Login --> /home/userxx

 #2. joblog2 jar100.ksh          <-- run 1st copy of jar100.ksh
     ==================
 --> prompts "enter to continue" <-- waits for operator reply

 #3. Switch to another screen
     ========================
     - via ctl/alt/function 1-8 if on unix console
     - or different putty session if access via Windows putty terminal emulator
     - or could Login as a different user with same $RUNLIBS & $RUNDATA

 #4. joblog2 jar100.ksh          <-- run 2nd copy of jar100.ksh
     ==================            - note following "waiting" displays
     waiting 2 seconds for same-name job to complete
                 --- repeat wait 2 seconds 3 times ---
     waiting 10 seconds for same-name job to complete
                 --- repeat wait 10 seconds 6 times ---
     waiting 60 seconds for same-name job to complete
                 --- repeat wait 60 seconds until reply to 1st job ---

 #5. Switch back to 1st screen
     =========================

 #6. Reply to 1st "enter to continue" prompt
     =======================================
     - 1st job completes & 2nd job starts

 #7. Switch to 2nd screen
     ====================

 #8. Reply to 2nd "enter to continue" prompt
     =======================================
     - 2nd job completes

 #9. l joblog   <-- list log files in joblog/... (date_time stamped)
     ========

 #10. vi joblog/jar100*  <-- inspect logfiles
      =================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7C1. Test/Verify Critical Features in Converted JCL/scripts

Test/verify GDG file generation re-sequence

The JCL converter converts mainframe GDG files to Vancouver Utility function equivalents exportgen0 & exportgen1. 'exportgen0' determines the most recent generation for reading & 'exportgen1' determines the next generation for writing. For example, here are the file conversions from step 2 of jgl230.jcl page '2E1'.

     //GLTRANS  DD DSN=GL.ACCOUNT.TRANS(+1),DISP=OLD
     //GLMSOLD  DD DSN=GL.ACCOUNT.MASTER(0),DISP=OLD
     //GLMSNEW  DD DSN=GL.ACCOUNT.MASTER(+1),DISP=(,CATLG,DELETE),...

exportgen1 +1 GLTRANS data1/gl.account.trans_ exportgen0 0 GLMSOLD data1/gl.account.master_ exportgen1 +1 GLMSNEW data1/gl.account.master_

Vancouver Utility GDG files are identified in JCL/scripts by a '_' trailing underscore. The actual datafiles (in $RUNDATA/data1/...) have 6 digits following the '_'. The number of generations maintained is controlled by gdg=... in a control file $GDGCTL/gdgctl51I.dat/.idx (Indexed file loaded from $GDGCTL/gdgctl51 text file)

 JCL/scripts call function 'exportgen1' to determine next generation# to create.
 exportgen1 deletes oldest gdg#s depending on gdg=___ in $GDGCTL/gdgctl51I.dat/.idx
 - for example if gdg=3, adding a 4th file will delete the 1st file
 gl.account.master_000001
 gl.account.master_000002       <-- sample GDG file with 3 generations
 gl.account.master_000003
 gl.account.master_999999       <-- max gen#, need to reset before reached
 - would take 3000 years to reach max gen# if 1 new generation per day
 - would take 3 years if you create 1000 new generations/day
 - generation#s will be automatically reset at 900000 by jobend51/gdgreset1

New generations are created in jobtmp/GDG/... & moved back to data1/... at successful End-Of-Job by function 'jobend51' 'jobend51' also tests for GDG#s > 900000 & calls script 'gdgreset1' to re-sequence gdg#s > 900000 back to 100001, 100002, etc

 gl.account.master_900001       <-- gdg file with 3 gens > 900000
 gl.account.master_900002
 gl.account.master_900003
 gl.account.master_100001       <-- gdg#s re-sequenced from 100001
 gl.account.master_100002
 gl.account.master_100003

'gdgreset1' called by 'jobend51' at EOJ will reset gdg#s for all members of 1 file whenever a new member is created with gdg# > 900000.

Also see 'gdgreset2' on page '7D1' to reset gdg#s for 'ALL' files in directory with GDG# > a specified threshold. You would run gdgreset2 in an offline batch mode when no jobs are running (to reset ALL files with gdg#s >threshold vs gdgreset1 which resets only 1 file when a new member created > 900000).

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7C2. Test/Verify Critical Features in Converted JCL/scripts

Test/verify GDG file generation re-sequence

If you wish to verify that the GDG# reset works as advetised, you could do so as shown below. We assume that you have already run the JCL conversions in JCLcnv1demo.htm#Part_3 & executions in Part 4, particularly jgl230.ksh executed on page JCLcnv1demo.htm#4F5 (see console log displayed).

Here are 'only' the datafiles relevant to our test (step 2 of jgl230.ksh).

 /home/uvadm/mvstest/testdata  <-- test/demo demo data files supplied
 /home/userxx                  <-- can copy to your homedir
 :-----testdata                <-- data dirs (defined as $RUNDATA in JCL/scripts)
 :     :------ctl                   <-- control files for JCL convert & GDG control
 :     :      :-----gdgctl51        <-- GDG control file text version
 :     :      :-----gdgctl51I.dat   <-- Indexed file to supply gdg=... limit
 :     :      :-----gdgctl51I.idx
 :     :------data1       - datafiles supplied for test/demos
 :     :      :-----gl.account.master_900001
 :     :      :-----gl.account.master_900002   <-- gen# changed to 900001,2,3
 :     :      :-----gl.account.master_900003
 :     :      :-----gl.account.tran1
 :     :      :-----gl.account.trans_000001
 :     :      :-----gl.account.trans_000002
 :     :      :-----gl.account.trans_000003
 :     :------joblog                <-- log files
 :     :------jobtmp/GDG/...        <-- new generations created here
 :     :                              - moved back to data1/ at successful EOJ

 #1. Login userxx --> $HOME (/home/userxx)
 #2. cdd --> $HOME/testdata

 #3. change generation#s on gl.account.master_ to 900001,900002,900003
 #3a. mv  data1/gl.account.master_000001  data1/gl.account.master_900001
 #3b. mv  data1/gl.account.master_000002  data1/gl.account.master_900002
 #3c. mv  data1/gl.account.master_000003  data1/gl.account.master_900003

 #4. testdatainit         <-- clear joblog & jobtmp subdirs of prior runs
     ============           - also prompts to reload the GDG control file

 #5. joblog jgl230.ksh    <-- execute jgl230.ksh, with joblog to save
     =================        console messages in joblog/jgl230.log
Note
  • the console log is listed below, showing only step2 (gdg file update)
  • also showing EndOfJob GDG file move back from jobtmp to data1
    & resequencing of gl.account.master_90001,2,3,4 gdg#s back to 100001,2,3,4

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7C3. Test/Verify Critical Features in Converted JCL/scripts

console log showing gdg# reset

 150422:103638:jgl230: Begin Job=jgl230
 150422:103638:jgl230:            ... Step S0010 uvsort ... (omitted)
 150422:103638:jgl230: ******** Begin Step S0020 cgl200 (#2) ********
 150422:103638:jgl230: gen+1 GLTRANS=/home/mvstest/testdata/data1/gl.account.trans_ gens=7
 150422:103638:jgl230: gen0 GLMSOLD=data1/gl.account.master_900003 insize=16K
 150422:103638:jgl230: gen+1 GLMSNEW=/home/mvstest/testdata/data1/gl.account.master_ gens=7
 150422:103638:jgl230: Executing--> cobrun -F /home/mvstest/testlibs1/cblx/cgl200
 150422:103638:JGL230: StepTimes: S0020 cgl200 Begun=10:36:38 End=10:36:38 Elapsed=00:00:00
 150422:103638:jgl230:            ... Step S0030 cgl100 ... (omitted)
 150422:103638:jgl230: moving jobtmp/jgl230/GDG/files back to intended directories
 /home/mvstest/testdata/data1/gl.account.master_900004
 /home/mvstest/testdata/data1/gl.account.trans_000004
 gdgreset1 - script to resequence gdg#s for 1 file if gdg# > threshold
 gdgreset1 data1/gl.account.master_900001 ---> gl.account.master_100001
 gdgreset1 data1/gl.account.master_900002 ---> gl.account.master_100002
 gdgreset1 data1/gl.account.master_900003 ---> gl.account.master_100003
 gdgreset1 data1/gl.account.master_900004 ---> gl.account.master_100004
 gdgreset1 - 4 GDG files resequenced
 150422:103638:JGL230: StepTimes: S0030 cgl100 Begun=10:36:38 End=10:36:38 Elapsed=00:00:00
 150422:103638:JGL230: Job Times: Begun=10:36:38 NormalEnd=10:36:38 Elapsed=00:00:00
 150422:103638:jgl230: JobEnd=Normal, StepsExecuted=3, LastStep=S0030

 #6. ls -l data1          <-- display results
     ===========            - showing only files relevant to step 2 of jgl230.ksh
 :     :      :-----gl.account.master_100001
 :     :      :-----gl.account.master_100002   <-- gen#s reset to 100001,2,3,4
 :     :      :-----gl.account.master_100003
 :     :      :-----gl.account.master_100004   <-- new gen created by updt as90004
 :     :      ;                                    re-sequenced to 100004 by jobend/gdgreset1
 :     :      :-----gl.account.tran1
 :     :      :-----gl.account.trans_000001
 :     :      :-----gl.account.trans_000002
 :     :      :-----gl.account.trans_000003
 :     :      :-----gl.account.trans_000004
Note
  • compare these GDG files to the previous page listings before update
  • new member created by step2 update as gl.account.master_900004
    but then resequenced to gl.account.master_100001 by EOJ jobend/gdgreset1
  • new menber created by step1 (log not shown) as gl.account.trans_000004
    but NOT resequenced at EOJ by jobend/gdgreset1 BECAUSE gdg#s < 900000

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7D1. Test/Verify Critical Features in Converted JCL/scripts

gdgreset2 - reset gdg#s with Batch job

'gdgrest1' (above) resets gdg#s for 'ONE' file automatically, whenever gdg# 900000 is exceeded. 'gdgreset1' is called by function 'jobend51' which was inserted by the JCL converter at the end of all JCL/scripts.

'gdgreset2' (documented here) will reset GDG#s for 'ALL' files whose GDG#s exceed a specified threshold (suggested default 500000). You would run this batch job when no JCL/scripts are being executed.

We will demonstrate gdgreset2 using the same files as used above for gdgreset1. This is easy to do, but of course will show only the 1 file being reset (vs many that may be reset in a real situation).

'gdgreset2' creates a 'script' to copy/rename files with gdg#s > specified threshold. The script will copy all files to a 2nd directory (renaming if gdg# was > threshold). You can inspect before renaming the input as data1.old & the output dir back to data1.

suggested Operating Instructions

  1. Login appsadm
  2. cd $RUNDATA

 #3. change generation#s on gl.account.master_ to 500001,500002,500003

 #3a. mv  data1/gl.account.master_000001  data1/gl.account.master_500001
 #3b. mv  data1/gl.account.master_000002  data1/gl.account.master_500002
 #3c. mv  data1/gl.account.master_000003  data1/gl.account.master_500003

 #4a. mkdir data1_reset          <-- make dir for output
 #4b. mkdir sf                   <-- make dir to receive output script

 #5. uvcopy gdgreset2,fild1=indir,fild2=outdir,filo1=script-to-copy/resequence-gdgs
                     ,uop=m______(Threshold),n______(ResequenceFrom)
     ==============================================================================
     - command format above
     - see actual command on the next page

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7D2. Test/Verify Critical Features in Converted JCL/scripts


 #5a. uvcopy gdgreset2,fild1=data1,fild2=data1_reset,filo1=sf/data1_reset.ksh
                     ,uop=m500000,n100001
      =======================================================================

 #5b. uvcopy gdgreset2    <-- same as above, files default as shown above
      ================      - prompts allow change, as shown below
      uop=m500000n100001  - option defaults
          m500000         - default gdg# threshold for reset
                 n100001  - default gdg# to resequece from
      User OPtion (uop) defaults  = q1m500000n100001
      null to accept or enter/override -->    <-- null to accept defaults
      did you create empty outdir (or remove files) ------->   <-- null OK
      data1 = default fild01 - null accept or reenter ?  -->   <-- null OK
      data1_reset = default fild02 - accept or reenter ? -->   <-- null OK
      sf/data1.reset.ksh = default filo01 - accept     ? -->   <-- null OK

 #6. vi sf/data1_reset.ksh      <-- inspect script before execution
     =====================
      # gdgreset2 - reset generation numbers for Vancouver Utility GDG files
      # 2015/04/21_16:52:46  InDir: data1  OutDir: data1_reset
      # if gdg#s > 500000  - resequence from 100001
      cp data1/gl.account.master_500001   data1_reset/gl.account.master_100001
      cp data1/gl.account.master_500002   data1_reset/gl.account.master_100002
      cp data1/gl.account.master_500003   data1_reset/gl.account.master_100003
      cp data1/gl.account.trans_000001    data1_reset/
      cp data1/gl.account.trans_000002    data1_reset/
      cp data1/gl.account.trans_000003    data1_reset/

 #7. sf/data1_reset.ksh         <-- execute script to copy/rename files
     ==================             from data1/... --> data1_reset/...

 #8. ls -l data1_reset         <-- inspect output results
     =================
      -rw-r--r--. 1 mvstest1 apps 13952 Apr 21 16:47 gl.account.master_100001
      -rw-r--r--. 1 mvstest1 apps 13952 Apr 21 16:47 gl.account.master_100002
      -rw-r--r--. 1 mvstest1 apps 13952 Apr 21 16:47 gl.account.master_100003
      -rw-r--r--. 1 mvstest1 apps  1600 Apr 21 16:47 gl.account.trans_000001
      -rw-r--r--. 1 mvstest1 apps  1600 Apr 21 16:47 gl.account.trans_000002
      -rw-r--r--. 1 mvstest1 apps  1600 Apr 21 16:47 gl.account.trans_000003

 #9a. mv data1  data1_old       <-- rename current as old
      ===================
 #9b. mv data1_reset data1      <-- rename newdir as current data1 dir
      ====================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Visitor Counters for ThisYear and LastYear