scripts1.doc - Contents by Part#


Part_0 - Introduction & Overview of Vancouver Utility scripts
- scripts1.doc (this book) vs CNVaids.htm vs ADMjobs.htm
- where scripts are stored & how they are accessed (PATH)
- do not modify VU scripts in /home/uvadm/sf/...
- setup appsadm account & copy/modify scripts in /home/appsadm/sf/...
- search PATH priority is $HOME/sf/, /home/appsadm/sf/, /home/uvadm/sf/

Part_1 - file 'rename' scripts (all files in directory)
- over 20 rename scripts, useful for conversions & general purpose
- rename all scripts in a directory matching specified criteria
- add/remove/change suffixes, prefixes, or patterns within filenames
- translate to UPPER, or to lower, convert '$' to '_', blank to '_', etc
- Here are just 1st 3 of 20 listed in Part_1
  renameL - rename files to Lower-case
  renameU - rename files to UPPER-case
  renameX - change any specified extension to an alternate
  renameCC - change all instances of 1 character to an alternate
--etc--

Part_2 - aliases, save a lot of keystrokes, here are a few of 12 listed in Part_2
  alias l='ls -l' - saves keystrokes for this often repeated command
  alias rm='rm -i' - confirm removes, option -i prompts for y/n ?
  alias cdl='cd $RUNLIBS' - quick way to change to libraries (might be long path)
  alias cdl='cd $RUNDATA' - quick way to change to DATA super-directory
  alias grep='grep -n' - ensure line# reported on grep matches
--- etc ---

Part_3 - listing file & directory names (not data contents)
- scripts to list directories (llm, llc, lla, llr, llt, lls, llu)
llm - list long & pipe to more
llc - same, but with file sequence#s & line counts for each file
llr - list recursively, current directory & all subdirectories
--etc--
- lslp list files & print immediately
- spreadA list filenames 4-up across the page
- longest1 find longest line in a text file

Part_4 - scripts to process ALL files in a directory
  alldiff2 - repeat unix 'diff' for all files in 2 directories
  allcmp - repeat unix 'cmp' for all files in 2 directories
  all2lower - translate to lower-case, copying all files to 2nd subdir
  all2upper - translate to UPPER-case, copying all files to 2nd subdir
  allcancel - cancel all printer lp requests for current user
  rmzf - remove zero-length (empty) files in directory
  rmold1 - remove files older than specified number of days
  lsold1 - list files older than ## days (check before remove ?)
  mvold1 - move to alternate subdir (vs remove)
--etc-- - see several other 'process all files' scripts in Part_4

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

scripts1.doc - Contents (continued)


Part_5 - scripts to count Lines & Files in directories & sub-directories
v12 - help screen of scripts with sample outputs for following:
llc - similar to 'ls -l | more', but with file seq# & line counts/file
cfd - Count Files in a Directory (listing each file)
cfdt - Count Files in a Directory, Total only (1 line answer)
cfdd - Count Files in Directories within a parent Directory
cfdp - Count Files in a Directory with a Pattern on any line in any file
clf - Count Lines in 1 File
clfd - Count Lines & Files in a Directory
clfp - Count Lines in a File with a specified Pattern
clfdp - Count Lines in all Files of a Dirctory with a Pattern

Part_6 - scripts to manipulate text & data files
- removeCR - remove Carriage Returns from text files (Windows --> Unix/Linux)
- insertCR - insert Carriage Returns in text files (Unix/Linux --> Windows)
- removeCRall - remove CRs from all files in directory, copying to 2nd dir
- insertCRall - insert CRs into all files in directory, copying to 2nd dir
- runsed1 - apply sed script to process all files from 1 dir to a 2nd dir
- runsed2 - follow-on script to use sed command saved by runsed1 in tmp/runsed1
- uvfix1 - copy 1 Text file to tmp/... applying uvcopy instructions
- may then inspect (with vi tmp/...) & copy back if desired
- uvfixA - copy ALL Text files from 1 subdir to a 2nd subdir
  while applying uvcopy instructions
- uvfix2 - copy 1 Fixed-Length file to tmp/... applying uvcopy instructions
- may then inspect (with uvhd tmp/...) & copy back if desired
- uvfixB - copy ALL Fixed-Length files from 1 subdir to a 2nd subdir
  while applying uvcopy instructions
- dropbl0 - drop ALL blank lines, copy to tmp/ & back, using 'grep'
- dropbl1 - drop ALL blank lines, copy to tmp/ & back, using 'uvcp' optn b2
- dropbl2 - reduce multi blank lines to 1, using 'uvcp' option b1
- dropbl3 - drop blank lines & COBOL cmts with only '*'s & blanks ('uvcopy')
- dropbl0,1,2,3 demo
- printable1 - ensure printable characters in a text file
  - copies to tmp/... converting unprintables to '.' periods
- parmnullfix1 - copy all files from 1 directory to a 2nd directory
  - converting any nulls to blanks, using 'sed'
- parmnullfix2 - copy all files from 1 directory to a 2nd directory
  - blanking nulls & clearing cols 73-80, using 'uvcp'

Part_7 - copying & sorting all files from 1 directory to a 2nd directory
- copycpio1, copycpio2, copydate, cpnew1, cpnew2
- sortcpio, sortdirs, sortall

Part_8 - system/application administration scripts
- some require 'root' permissions
- killuser1, killuser2, chmod1, chmod2, chmod3, ftrigger51, ftrigger52
- we also list 'crontab_root' used to schedule 'killuser2' at 3 AM
- see several other 'unix administration' type scripts in ADMjobs.htm
- scripts for backup/restore, crontabs to run nightly scripts, etc

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_0 Vancouver Utility scripts for Conversion & General use

scripts calling unix/linux vs Vancouver Utilities

This scripts1.doc is intended to document 'general purpose' scripts that could be useful to any unix/linux user. Most of these scripts call standard unix/linux utilities (ls,mv,cp,cpio,diff,cmp,kill,etc) vs Vancouver Utilities (uvcopy,uvlist,uvsort,uvcp,uvhd,etc).

scripts1.doc lists about 40 of the most useful of the more than 500 scripts distributed in the Vancouver Utilities package. You can save this document, cut out the scripts that interest you, & transfer them to your unix/linux.

If you are a Vancouver Utilities customer, you have access to all scripts in the scripts directory as shown on the next page.

scripts1.doc vs CNVaids.doc

If you are more interested in scripts that could be useful during mainframe conversions (of JCL, COBOL,& EBCDIC DATA files), please see CNVaids.htm. Some of the scripts in CNVaids.htm call only unix/linux commands, but many of them do call Vancouver Utilities.

scripts1.doc vs ADMjobs.doc

Also see ADMjobs.htm for administrative type scripts such as 'backups' & scripts run by 'cron'. Again the intention of this scripts1.doc is to present the more general purpose scripts that could be useful to any unix/linux user.

Pre-Requisites to running test/demo scripts

  1. Your site administrator has installed the Vancouver Utilities. Please see the directory diagrams on the next page, showing only the subdirs relevant to running these test/demo scripts.

  2. Your site administrator has setup a login/homedir for you AND has setup the profiles to give you access to the Vancouver Utility programs, scripts, and uvcopy parameter files.

  3. If you are the site administrator, please see ADMjobs.htm#Part_1 for listings of the supplied profiles & how best to set them up.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

0A2. Vancouver Utility scripts for Conversion & General use

VU scripts - where stored, how accessed (PATH)

 /home/uvadm        <-- Vancouver Utilities homedir
 :     ---              --- showing relevant subdirs only ---
 :-----bin              - binaries (uvcopy,uvsort,etc)
 :-----doc              - documentation
 :-----env              - profiles distributed with VU
 :     :
 :-----sf              <-- Script Files (sub-directoried for uvadm only)
 :     :-----adm         - administrative scripts
 :     :-----demo        - demo scripts
 :     :-----IBM         - IBM mainframe conversion scripts
 :     :-----util        - utility scripts
 :     :
 /home/appsadm    <-- appsadm homedir
 :     ---              --- showing relevant subdirs only ---
 :-----bin              - binaries for site developed/modified programs
 :-----env              - profiles modified for your site
 :     :
 :-----sf             <-- shell scripts developed/modified by appsadm
 :     :
 /home/userxx     <-- user homedirs
 :     ---              --- showing relevant subdirs only ---
 :     :
 :-----sf             <-- shell scripts developed/modified by appsadm
 :     :

relevant lines from common_profile

 export UV=/home/uvadm                # <-- UV homedir symbol used below
 export APPSADM=/home/appsadm         # <-- site applications administrator
 export PATH=$PATH:$HOME/bin:$HOME/sf:$APPSADM/bin:$APPSADM/sf:$RUNLIBS/sf
 export PATH=$PATH:$UV/bin:$UV/sf/adm:$UV/sf/demo:$UV/sf/util:$UV/sf/IBM

The uvadm scripts directory is sub-directoried (sf/adm,sf/demo,sf/IBM,sf/util).

Note that the recommended profile (see ADMjobs.htm#1C3) searches the PATH of appsadm before uvadm, so any scripts that appsadm modifies will be found before any of the original scripts/jobs. Also note that the PATH searches $HOME/sf before /home/appsadm, so users could copy/modify a script in their homedir & it will be found before either appsadm/sf or uvadm/sf/...

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

0A3. Vancouver Utility scripts for Conversion & General use

uvadm, appsadm,& user homedirs

We recommend you setup a login/userid 'appsadm' to serve as the applications administrator for the unix/linux site. The appsadm home directory would hold various scripts, crontabs, log files, etc used in application administration.

One important purpose of appsadm is to hold the modified versions of any Vancouver Utility profiles, scripts & uvcopy jobs that you need to customize at your site.

Scripts that are useful to all programmers & operators should be stored in /home/appsadm/sf/... Scripts that are useful only to individual programmers should be stored in their homedirs /home/userxx/sf/...

This protects you from losing your customized versions when you install a future new version of Vancouver Utilities, which would overwrite your modified versions if stored in /home/uvadm/...

setup to run uvcopy test/demo jobs


 #1. Login to your homedir

 #2a. mkdir sf        <-- make subdir for your own scripts
      ========          - or to copy/modify scripts from /home/uvadm/sf/...

 #2b. mkdir dat1 tf   <-- make subdirs for the test/demo files
      =============

 #2c. mkdir tmp       <-- make subdir for test/demo outputs
      =========

 #3a. cp /home/uvadm/dat1/* dat1  <-- copy test/demo files to your homedir
      ==========================

 #3b. cp /home/uvadm/tf/* tf       <-- copy test/demo files to your homedir
      ======================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_1 Vancouver Utility scripts for Conversion & General use

file 'rename' scripts - Contents

These scripts rename all files (or files matching a pattern) in a directory. Renaming dozens or hundreds of files is often required when you import files from outside sources & these scripts can save you hours of keyboard drudgery.


1A1. Example using rename scripts to save hours of drudgery
- renaming hundreds of mainframe files FTP'd to unix/linux

1B1. renameL - translate filenames to lower case
1B2. renameU - translate filenames to UPPER case
1B3. renameLsd - translate filenames to lower case in all subdirs of a superdir

1C1. renameP - replace existing prefix with alternate
1C2. rename-P - remove any specified prefix
1C3. rename-P. - remove specified prefix up to 1st '.'
1C4. rename+P - add any specified prefix

1D1. renameX - change any specified extension to any specified alternate
1D2. rename.X - change specified .extension to any specified .alternate
1D3. rename-X - remove any specified extension
1D4. rename-.X - remove any specified extension back to last '.' (default)
- may specify an alternate extension (vs '.' default)
1D5. rename+X - add any specified extension

1E1. rename-AA - remove Any pattern Anywhere in filename
1E2. renameAA - replace Any pattern with any Alternate

1F1. renameB2U - change any embedded blanks to underscores
1F2. renameD2U - change any '$' dollar signs to '_' underscores

1G1. rename-QQ - remove any 'Quotes'
1G2. rename-PP - remove any (Parenthesis)
1G3. rename2PP - rename to whatever is in (Parenthesis)

1H1. rename-TS - remove any timestamp _HHMMSS ('_' + 6 digits)
- preserves anything following _###### (.dat, .txt, etc)

1I1. renameGDG - change mainframe GDG suffixes to Vancouver Utility gdg suffixes
- change '.G####V00' to '_00####'
1I2. renameGDG1 - change '.G####V00' to '_000001' (keep only latest as 000001)

1J1. renameCC - convert all occurrences of 1 character to a 2nd character
- example - convert all '.'s in filenames to '_'s

You can see a help screen for each script, by entering the script-name only, without its required arguments (omit the directory).

See usage example on the next page & all scripts listed on the following pages.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1A1. Vancouver Utility scripts for Conversion & General use

example - renaming Mainframe Data Files

Here is a great example of the extreme usefulness of these 'rename' scripts. When mainframe files are transferred to unix/linux, we like to remove any quotes, translate to lower case & remove any extraneous extensions. We will also convert any '$' Dollar-signs to '_' Underscores since '$' is not allowed in unix filenames.

Assume the files have been FTP'd to /p4/cnvdata/dat1/... & we wish to copy/ rename the files into /p4/cnvdata/dat2/... as shown below:

 /p4/cnvdata
 :---dat1         <-- directory of files FTP'd from Mainframe
 :   :
 :   :---'E2121653.M$TAXE.PISCINES.CLUS.VSEPRD' <-- filenames from MAINFRAME
 :   :---'E2121656.I$TAXE.FACTURAT.CLUS.VSEPRD'
 :   :---'E2122683.I$TAXE.DEGREVEM.CLUS.VSEPRD'
 :---dat2         <-- directory to rename files for unix/linux
 :   :
 :   :---e2121653.m_taxe.piscines          <-- filenames desired on Unix/Linux
 :   :---e2121656.i_taxe.facturat
 :   :---e2122683.i_taxe.degrevem

the HARD way to rename files


 #1. cp dat1/* dat2  <-- copy to 2nd directory (to preserve original filenames?)
 #2. cd dat2/
 #3. mv 'E2121653.M$TAXE.PISCINES.CLUS.VSEPRD' e2121653.m_taxe.piscines
 #4. mv 'E2121656.I$TAXE.FACTURAT.CLUS.VSEPRD' e2121656.i_taxe.facturat
 #5. mv 'E2122683.I$TAXE.DEGREVEM.CLUS.VSEPRD' e2122683.i_taxe.degrevem
     mv          --- etc, could be 100's or 1000's of files ---

the EASY way to rename files


 #1. cp dat1/* dat2  <-- copy files to dat2/... before renaming (if desired)
     ==============    - no need to copy, when you have satisfied yourself
                         that these scripts work (as advertised)

 #2. rename-QQ dat2  <-- remove the quotes (see 'rename-QQ' on page '1G1')
     ==============
 #3. renameD2U dat2  <-- change '$'s to '_'s (see 'renameD2U on page '1F2')
     ==============                     ('$' not allowed in unix filenames)
 #4. rename-X dat2 .CLUS.VSEPRD  <-- remove unwanted extensions
     ==========================    - see 'rename-X' on page '1D3')
 #5. renameL dat2    <-- translate to lower case (see 'renameL' on page '1B1')
     ============

Without these scripts, it would be a very long, boring, error-prone job to rename the scripts manually using 'mv' commands, OR you would have to develope similar scripts yourself.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1B1. Vancouver Utility scripts to Rename all files in directory

renameL - translate filenames to Lower case

 #!/bin/ksh
 # renameL - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # renameL - rename an entire directory of filenames to lower case
 #
 echo "rename all filenames in subdir to lower case"
 if [ -d "$1" ]; then :
    else echo "usage: renameL directory [y]"
         echo "       ====================="
         echo " - arg1 must be a directory"
         echo " - arg2 may be 'y' to bypass confirmation prompt"
         exit 1; fi
 #
 reply="$2"
 until [ "$reply" = "y" ]
 do echo "will rename all files in $1 to lower case OK ? y/n"
    read reply
 done
 #
 x=0; y=0
 for i in $1/*
 do let x=x+1
    f=${i##*/}
    typeset -l g=$f
    if [[ $g != $f ]]; then
       mv -i $1/$f $1/$g
       #================
       let y=y+1
       echo "file# $y (of $x) $1/$f - renamed to: $1/$g"
    fi
 done
 echo "total $y files in ${1}, $x renamed to lower case"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1B2. Vancouver Utility scripts to Rename all files in directory

renameU - translate filenames to UPPER case

 #!/bin/ksh
 # renameU - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # renameU - rename an entire directory of filenames to UPPER case
 #
 echo "rename all filenames in subdir - to UPPER case"
 if [ -d "$1" ]; then :
    else echo "usage: renameU directory [y]"
         echo "       ====================="
         echo " - arg1 must be a directory"
         echo " - arg2 may be 'y' to bypass confirmation prompt"
         exit 1; fi
 #
 reply="n"
 until [ "$reply" = "y" ]
 do echo "will rename all files in $1 to UPPER case OK ? y/n"
    read reply
 done
 #
 x=0; y=0
 for i in $1/*
 do let y=y+1
    f=${i##*/}
    typeset -u g=$f
    if [[ $g != $f ]]; then
       mv -i $1/$f $1/$g
       #================
       let x=x+1
       echo "file# $x (of $y) $1/$f  - renamed to:  $1/$g"
    fi
 done
 echo "total $y files in ${1}, $x renamed to UPPER case"
 exit 0

renameU same as renameL except for:

'renameU' is the same as 'renameL' except line 21 is changed:


      typeset -l g=$f   <-- from this
      ===============
      typeset -u g=$f   <-- to this
      ===============

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1B3. Vancouver Utility scripts to Rename all files in directory

renameLsd - Lower case all files in all subdirs of superdir

 #!/bin/ksh
 # renameLsd - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # renameLsd - lower case rename all files in all subdirs of a superdir
 #           - by Owen Townsend, UV Software, May 2015
 #
 echo "lower case rename all files in all subdirs of a superdir"
 d1="$1"
 if [ -d "$d1" ]; then :
    else echo "usage: renameLsd directory [y]"
         echo "       ======================="
         echo " - arg1 must be a directory"
         echo " - arg2 may be 'y' to bypass confirmation prompt"
         exit 1; fi
 #
 reply="$2"
 until [ "$reply" = "y" ]
 do echo "will rename all files in all subdirs of $d1 to lower case OK ? y/n"
    read reply
 done
 #
 ds=0; fs=0
 for d1d2 in $d1/*
   { d2=$(basename $d1d2)
     if [[ -d $d1/$d2 ]]; then
        ((ds+=1))
        for d1d2f in $d1/$d2/*
          { f1=$(basename $d1d2f)
            typeset -l f2=$f1
            if [[ $f2 != $f1 ]]; then
               mv -i $d1/$d2/$f1 $d1/$d2/$f2
               #============================
               ((fs+=1))
               echo "file# $fs $d1/$d2/$f1 - renamed to: $d1/$d2/$f2"
            fi
          }
       fi
   }
 echo "total $fs files in $ds subdirs renamed to lower case"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1C1. Vancouver Utility scripts to Rename all files in directory

renameP - replace Prefix with alternate

 #!/bin/ksh
 # renameP - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # renameP - rename an entire directory of filenames
 #         - replacing an existing prefix with a new prefix
 #         - may use pattern characters in prefix
 #
 echo "rename all filenames in subdir - replace prefix with alternate"
 if [[ -d "$1" && -n "$2" && -n "$3" ]]; then :
    else echo "usage: renameP directory oldprefix newprefix"
         echo "       ====================================="
         exit 1; fi
 reply="n"
 until [[ "$reply" = "y" ]]
 do echo "do you have backup ? y/n"
    read reply
 done
 x=0; y=0
 for i in $1/*
 do f=${i##*/}                  # remove any preceding directories
    g=${f#$2}                   # remove smallest matching old prefix
    h=${3}${g}                  # insert new prefix
    let x=x+1
    if [[ ($f == *${2}*) && ($h != $f) ]]; then
       mv -i $1/$f $1/$h
       #================
       let y=y+1
       echo "file# $x - $1/$f  - renamed to:  $1/$h"
    fi
 done
 echo "total $x files, $y renamed (replacing prefix $2 with $3)"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1C2. Vancouver Utility scripts to Rename all files in directory

rename-P - remove any specified prefix

 #!/bin/ksh
 # rename-P - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # rename-P - rename an entire directory of filenames
 #          - removing a specified prefix
 #          - may use pattern characters in prefix
 #
 echo "rename all filenames in subdir - removing a specified prefix"
 if [ -d "$1" -a -n "$2" ]; then :
    else echo "usage: rename-P directory prefix"
         echo "       ========================="
         exit 1; fi
 reply="n"
 until [ "$reply" = "y" ]
 do echo "do you have backup ? y/n"
    read reply
 done
 x=0; y=0
 for i in $1/*
 do f=${i##*/}                  # remove any preceding directories
    g=${f#$2}                   # remove smallest matching prefix
    let x=x+1
    if [[ $g != $f ]]; then
       mv -i $1/$f $1/$g
       #================
       let y=y+1
       echo "file# $x - $1/$f  - renamed to:  $1/$g"
    fi
 done
 echo "total $x files, $y renamed (removing $2 prefix)"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1C3. Vancouver Utility scripts to Rename all files in directory

rename-P. - remove prefix up to 1st '.'

 #!/bin/ksh
 # rename-P. - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # rename-P. - rename an entire directory of filenames
 #           - removing Prefix until &including 1st '.' Dot
 #
 echo "rename all filenames in subdir - removing prefix until 1st '.'"
 if [ -d "$1" ]; then :
    else echo "usage: rename-P. directory "
         echo "       ===================="
         exit 1; fi
 reply="n"
 until [ "$reply" = "y" ]
 do echo "do you have backup ? y/n"
    read reply
 done
 x=0; y=0
 for i in $1/*
 do f=${i##*/}                  # remove any preceding directories
    g=${f#*.}                   # remove smallest match to 1st Dot
    let x=x+1
    if [[ $g != $f ]]; then
       mv -i $1/$f $1/$g
       #================
       let y=y+1
       echo "file# $x - $1/$f  - renamed to:  $1/$g"
    fi
 done
 echo "total $x files, $y renamed, removing prefix to 1st '.'"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1C4. Vancouver Utility scripts to Rename all files in directory

rename+P - add any specified prefix

 #!/bin/ksh
 # rename+P - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # rename+P - rename an entire directory of filenames
 #          - adding a specified prefix
 #          - inhibited if a specified pattern already present
 #
 echo "rename all filenames in subdir - adding a specified prefix"
 if [[ -d "$1" && -n "$2" && -n "$3" ]]; then :
    else echo "USAGE: rename+P directory prefix inhibitor/all"
         echo "       ======================================="
         echo "example: rename+P datadir raw. raw.  #inhibit if already present"
         echo "         =========================="
    exit 1; fi
 q=$3; if [[ "$3" = "all" ]]; then q="allALL"; fi
 x=0;y=0
 for i in $1/*
 do f=${i##*/}
    let x=x+1
    if [[ "$f" = *"$q"* ]]; then  # if filename contains inhibitor
         echo "file# $x - $1/$f"
    else g=${2}${f}               # insert prefix
         mv -i $1/$f $1/$g        # change fielname
         #================
         let y=y+1
         echo "file# $x - $1/$f  - renamed to:  $1/$g"
    fi
 done
 echo "EOD - $y files renamed of $x total in $1"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1D1. Vancouver Utility scripts to Rename all files in directory

renameX - change any extension to any alternate

 #!/bin/ksh
 # renameX - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # renameX - change a specified extension to a specified alternative
 #           for all files within a directory
 #
 echo "rename all filenames in subdir - replace extension with alternate"
 if [[ -d "$1" && -n "$2" && -n "$3" ]]; then :
    else echo "USAGE:   renameX directory ext1 ext2"
         echo "         ==========================="
         echo "example: renameX cobs .cbl .bat"
         echo "         ======================"
    exit 1; fi
 #
 reply="n"
 until [[ "$reply" = "y" ]]
 do echo "will rename all files in $1 changing ext $2 too $3 OK ? y/n"
    read reply
 done
 #
 x=0;y=0
 for i in $1/*
 do f=${i##*/}
    let x=x+1
    if [[ "$f" == *"$2" ]]; then    # if ext1 present ?
       let y=y+1
       g=${f%$2}                  # remove ext1
       h=$g$3                     # add ext2
       mv -i  $1/$f $1/$h         # rename
       #=================
       echo "file# $x - $1/$f  renamed to:  $1/$h"
    fi
 done
 echo "EOD - $y files renamed of $x total in $1"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1D2. Vancouver Utility scripts to Rename all files in directory

rename.X - change any .extension to any .alternate

 #!/bin/ksh
 # rename.X - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # rename.X - change a specified .extension to a specified .alternative
 #            for all files within a directory
 #          - '.' must be present (unlike 'renameX')
 # renameX  - does not insist on '.' in filename
 #
 echo "rename all filenames in subdir - replace .extension with .alternate"
 if [[ -d "$1" && -n "$2" && -n "$3" ]]; then :
    else echo "USAGE:   rename.X directory ext1 ext2"
         echo "         ============================"
         echo "example: rename.X cobprogs .cob .cbl  <--change .cob to .cbl"
         echo "         ============================"
    exit 1; fi
 #
 reply="n"
 until [[ "$reply" = "y" ]]
 do echo "will rename all files in $1 changing ext $2 too $3 OK ? y/n"
    read reply
 done
 #
 x=0;y=0
 for i in $1/*
 do f=${i##*/}
    let x=x+1
    if [[ ($f == *.*) && ($f == *$2) ]]; then # if .ext1 present ?
       let y=y+1
       g=${f%$2}                # remove ext1
       h=$g$3                   # add ext2
       mv -i  $1/$f $1/$h       # rename
       #=================
       echo "file# $x - $1/$f  renamed to:  $1/$h"
    fi
 done
 echo "EOD - $y files renamed of $x total in $1"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1D3. Vancouver Utility scripts to Rename all files in directory

rename-X - remove any extension

 #!/bin/ksh
 # rename-X - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # rename-X - rename an entire directory of filenames
 #          - removing the specified extension (no '.' required)
 #          - may use pattern characters in extension
 #
 echo "rename all files in subdir - removing specified extension (no '.' reqd)"
 if [ -d "$1" -a -n "$2" ]; then :
    else echo "usage:  rename-X directory extension  <-- arg1 subdir, arg2 ext"
         echo "        ============================"
         echo "example: rename-X jcldir .jcl  <--remove .jcl from all jcldir"
         echo "         ===================="
    exit 1; fi
 #
 reply="n"
 until [ "$reply" = "y" ]
 do echo "do you have backup ? y/n"
    read reply
 done
 x=0; y=0
 for i in $1/*
 do f=${i##*/}
    g=${f%$2}
    let x=x+1
    if [[ $g != $f ]]; then
       mv -i $1/$f $1/$g
       #================
       let y=y+1
       echo "file# $x - $1/$f  - renamed to:  $1/$g"
    fi
 done
 echo "total $x files, $y renamed (removing $2)"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1D4. Vancouver Utility scripts to Rename all files in directory

rename-.X - remove any .extension

 #!/bin/ksh
 # rename-.X - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # rename-.X - rename an entire directory of filenames
 #           - removing extension back to specified character (default '.')
 #
 dir="$1"; xchar="$2";
 echo "rename all files in subdir - removing extension back to spcfd character (default '.')"
 if [ -d "$dir" ]; then :
    else echo "usage:  rename-.X directory [xchar]"
         echo "        ==========================="
         echo " - arg1 must be a directory"
         echo " - arg2 may specify a separator '.' default"
         echo " "
         echo "example1: rename-.X scripts   <-- remove any .extension present"
         echo "          =================     - migt be .jcl, .ksh, etc"
         echo "example2: rename-.X parms _   <-- remove any _extension present"
         echo "          ==================    - specifying '_' vs default '.'"
    exit 1; fi
 #
 if [[ -z "$xchar" ]]; then xchar='.'; fi
 #
 reply="n"
 until [ "$reply" = "y" ]
 do echo "do you have backup ? y/n"
    read reply
 done
 x=0; y=0
 for df1 in $dir/*
 do f1=$(basename $df1)
    f2=${f1%${xchar}*}
    let x=x+1
    if [[ $f2 != $f1 ]]; then
       mv -i $dir/$f1 $dir/$f2
       #======================
       let y=y+1
       echo "file# $x - $dir/$f1 - renamed to: $dir/$f2"
    fi
 done
 echo "total $x files, $y renamed (removing ext back to "$xchar")"
 exit 0
 #

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1D5. Vancouver Utility scripts to Rename all files in directory

rename+X - add any specified extension

 #!/bin/ksh
 # rename+X - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # rename+X - rename an entire directory of filenames
 #          - adding any specified extension
 #          - inhibited if extension already present
 #
 echo "rename all filenames in subdir - adding a specified extension"
 if [[ -d "$1" && -n "$2" ]]; then :
    else echo "USAGE:   rename+X directory extension"
         echo "         ============================"
         echo "example: rename+X jcls .jcl"
         echo "         =================="
         echo " - add extension '.jcl' (if not already present)"
    exit 1; fi
 x=0;y=0
 for i in $1/*
 do f=${i##*/}
    let x=x+1
    if [[ "$f" = *$2 ]]; then  # if extnsn allready present ?
         echo "file# $x - $1/$f"
    else g=${f}$2
         mv -i $1/$f $1/$g
         #================
         let y=y+1
         echo "file# $x - $1/$f  - renamed to:  $1/$g"
    fi
 done
 echo "EOD - $y files renamed of $x total in $1"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1E1. Vancouver Utility scripts to Rename all files in directory

rename-AA - remove Any pattern Anywhere in filename

 #!/bin/ksh
 # rename-AA - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # rename-AA - rename an entire directory of filenames
 #           - removing Any pattern occuring Anywhere in filename
 #
 echo "rename files in subdir - removing a pattern that can occur Anywhere"
 if [ -d "$1" -a -n "$2" ]; then :
    else echo "usage: rename-AA directory pattern"
         echo "       ==========================="
         exit 1; fi
 #
 reply="n"
 until [ "$reply" = "y" ]
 do echo "do you have backup ? y/n"
    read reply
 done
 x=0; y=0
 for dfn in $1/*
 do fn=${dfn##*/}           # remove any preceding directories
    fn1=${fn%${2}*}         # remove smallest matching suffix
    fn2=${fn#*${2}}         # remove smallest matching prefix
    fn3=${fn1}${fn2}        # concatenate the left & right remainders
    let x=x+1
    if [[ $fn3 != $fn ]]; then
       mv -i $1/$fn $1/$fn3
       #==================
       let y=y+1
       echo "file# $x $1/$fn - renamed to: $1/$fn3"
    fi
 done
 echo "total $x files, $y renamed (removing $2 pattern)"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1E2. Vancouver Utility scripts to Rename all files in directory

renameAA - replace Any pattern with any Alternate

 #!/bin/ksh
 # renameAA - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # renameAA - rename an entire directory of filenames
 #          - replacing Any pattern Anywhere in filename with an Alternative
 #
 echo "rename files in subdir - replacing a pattern that can occur Anywhere"
 if [[ (-d "$1") && (-n "$2") && (-n "$3") ]]; then :
    else echo "usage: renameAA directory pattern altpattern"
         echo "       ====================================="
         exit 1; fi
 #
 reply="n"
 until [ "$reply" = "y" ]
 do echo "do you have backup ? y/n"
    read reply
 done
 x=0; y=0
 for dfn in $1/*
 do fn=${dfn##*/}           # remove any preceding directories
    if [[ $fn != *${2}* ]] then continue; fi
    fn1=${fn%${2}*}         # remove smallest matching suffix
    fn2=${fn#*${2}}         # remove smallest matching prefix
    fn3=${fn1}${3}${fn2}    # concatenate left-part, altpattern,& right-part
    let x=x+1
    if [[ $fn3 != $fn ]]; then
       mv -i $1/$fn $1/$fn3
       #===================
       let y=y+1
       echo "file# $x $1/$fn - renamed to: $1/$fn3"
    fi
 done
 echo "total $x files, $y renamed (replacing $2 with $3)"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1F1. Vancouver Utility scripts to Rename all files in directory

renameB2U - change any embedded blanks to underscores

 #!/bin/ksh
 # renameB2U - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # renameB2U - rename all files in a directory, changing any Blanks to _underscores
 #
 echo "rename all files in directory, changing any Blanks to _underscores"
 if [ -d "$1" ]; then :
    else echo "usage: renameB2U directory [y]"
         echo "       ======================="
         echo " - arg1 must be a directory"
         echo " - arg2 may be 'y' to bypass confirmation prompt"
         exit 1; fi
 #
 reply="$2"
 until [ "$reply" = "y" ]
 do echo "enter 'y' to rename files in $1 changing any Blanks to _underscores"
    read reply
 done
 #
 x=0; y=0
 for i in $1/*
 do let x=x+1
    f=${i##*/}
 ## typeset -l g=$f
    g=$(echo $f | tr " " "_")
    if [[ $g != $f ]]; then
       mv -i "$1/$f" $1/$g
       #==================
       let y=y+1
       echo "file# $y (of $x) $1/$f - renamed to: $1/$g"
    fi
 done
 echo "total $y files in ${1}, $x renamed changing Blanks to _underscores"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1F2. Vancouver Utility scripts to Rename all files in directory

renameD2U - change any '$' dollar signs to '_' underscores

 #!/bin/ksh
 # renameD2U - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # renameD2U - rename changing Dollar signs to _underscores
 #
 echo "rename changing Dollar signs to _underscores."
 if [ -d "$1" ]; then :
    else echo "usage: renameD2U directory  <-- arg1 must be a directory"
         echo "       ================="
         exit 1; fi
 #
 reply="n"
 until [ "$reply" = "y" ]
 do echo "rename changing Dollar signs to _underscores? y/n"
    read reply
 done
 #
 x=0; y=0
 for i in $1/*
 do let y=y+1
    f=${i##*/}
    g=$(echo $f | tr '$' '_')
    if [[ $g != $f ]]; then
       mv -i $1/$f $1/$g
       #================
       let x=x+1
       echo "file# $x (of $y) $1/$f  - renamed to:  $1/$g"
    fi
 done
 echo "total $y files in ${1}, $x renamed changing Dollar signs to _underscores."
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1G1. Vancouver Utility scripts to Rename all files in directory

rename-QQ - remove any 'Quotes'

 #!/bin/ksh
 # rename-QQ - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # rename-QQ - rename an entire directory of filenames - removing 'Quotes'
 #
 echo "rename all filenames in subdir - removing 'Quotes'"
 if [ ! -d "$1" ]
    then echo "usage: rename-QQ directory  <-- arg1 must be a directory"
         echo "       ==================="
         exit 1; fi
 reply="n"
 until [ "$reply" = "y" ]
 do echo "do you have backup ? y/n"
    read reply
 done
 x=0; y=0
 for i in $1/*
 do
    f=$(basename $i)
    echo "f=$f"
    if [[ $f == \'* && $f == *\' ]]; then
       g=${f#\'}              # remove left side 'quote
       h=${g%\'}              # remove right side quote'
       let x=x+1
       mv $1/$f $1/$h
       #=============
       echo "file# $x - $f renamed to: $h"
    fi
 done
 echo "total $x files in $1 renamed (removed quotes)"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1G2. Vancouver Utility scripts to Rename all files in directory

rename-PP - remove any (Parenthesis)

 #!/bin/ksh
 # rename-PP - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # rename-PP - remove any (parenthesis) from all filenames in a subdir
 #           - also see renamexPP to replace the parens with anotehr character
 #             (default underscore '_')
 #
 #    sample input-names    output-names
 #  ------------------------------------------
 #  wcs41.parm(ed06srt1)    wcs.parmed06srt1
 #  wcs41.parm(lc28srt2)    wcs.parmlc28srt2
 #
 echo "rename all filenames in subdir - removing any '(' & ')' parens"
 if [ ! -d "$1" ]
    then echo "usage: rename-PP directory  <-- arg1 must be a directory"
         echo "       ==================="
         exit 1; fi
 reply="n"
 until [ "$reply" = "y" ]
 do echo "do you have backup ? y/n"
    read reply
 done
 x=0; y=0
 for df in $1/*
 do
    f=$(basename $df)
    echo "f=$f"
    if [[ $f == *\(* && $f == *\)* ]]; then
       g=$(echo $f | tr -d "()")
       let x=x+1
       mv $1/$f $1/$g
       #=============
       echo "file# $x - $1/$f - renamed to:  $1/$g"
    fi
 done
 echo "total $x files in $1 renamed removing any '(' & ')' parens"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1G3. Vancouver Utility scripts to Rename all files in directory

rename2PP - rename to whatever is in (Parenthesis)

 #!/bin/ksh
 # rename2PP - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # rename2PP - rename an entire directory of filenames to the name in parenthesis
 #          - used in mainframe JCL conversion of control card member libraries
 #
 #       sample input-names      output-names
 #  ------------------------------------------
 #  wcs41.parmlib(ed06srt1).rtf    ed06srt1
 #  wcs41.parmlib(lc28srt2).rtf    lc28srt2
 #
 echo "rename all filenames in subdir - to the name in (parenthesis)"
 if [ ! -d "$1" ]
    then echo "usage: rename2PP directory  <-- arg1 must be a directory"
         echo "       ==================="
         exit 1; fi
 reply="n"
 until [ "$reply" = "y" ]
 do echo "do you have backup ? y/n"
    read reply
 done
 x=0; y=0
 for df in $1/*
 do
    f=$(basename $df)
    echo "f=$f"
    if [[ $f == *\(* && $f == *\)* ]]; then
       g=${f#*\(}                  # remove left side to (
       h=${g%\)*}                  # remove right side to (
       let x=x+1
       mv $1/$f $1/$h
       #=============
       echo "file# $x - $1/$f - renamed to:  $1/$h"
    fi
 done
 echo "total $x files in $1 renamed to (name) in parens"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1H1. Vancouver Utility scripts to Rename all files in directory

rename-TS - remove timestamp _HHMMSS

 #!/bin/ksh
 # rename-TS - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # rename-TS - rename an entire directory of filenames
 #           - removing a timestamp format '_HHMMSS' underscore + 6 digits
 # filenamex_175500.txt <-- sample input
 # filenamex.txt        <-- sample output
 #
 echo "rename files in subdir - removing timestamp '_HHMMSS' ('_' + 6 digits)"
 if [[ -d "$1" ]]; then :
    else echo "usage: rename-TS directory"
         echo "       ==================="
         exit 1; fi
 #
 reply="n"
 until [ "$reply" = "y" ]
 do echo "do you have backup ? y/n"
    read reply
 done
 x=0; y=0
 for dfn in $1/*
 do fn=${dfn##*/}           # remove any preceding directories
    fn1=${fn%_[0-9][0-9][0-9][0-9][0-9][0-9]*} # remove timestamp & suffix
    fn2=${fn#*_[0-9][0-9][0-9][0-9][0-9][0-9]} # remove prefix & timestamp
    fn3=${fn1}${fn2}        # concatenate the left & right remainders
    let x=x+1
    if [[ $fn3 != $fn ]]; then
       mv -i $1/$fn $1/$fn3
       #===================
       let y=y+1
       echo "file# $x $1/$fn - renamed to: $1/$fn3"
    fi
 done
 echo "total $x files, $y renamed (removing _timestamp)"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1I1. Vancouver Utility scripts to Rename all files in directory

renameGDG - change mainframe GDG suffix .G####V00 to _000001

 #!/bin/ksh
 # renameGDG - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # renameGDG - change file suffix .G####VXX to _00#### for all files in directory
 #           - for mainframe GDG file conversions to unix/linux/windows
 #           - for exportgen functions called by JCL/scripts converted by VU
 #           - see www.uvsoftware.ca/mvsjcl.htm#Part_5
 #
 d="$1";
 if [[ ! -d "$1" ]]; then
    echo "usage:   renameGDG directory (arg1 must be a directory)"
    echo "         ================="
    echo "example: renameGDG d2asc"
    echo "         ==============="
    exit 1;
 fi
 echo "change all suffixes in $d from G####VXX to _00#### "
 echo " - enter to proceed (or kill)"; read reply
 x=0; y=0;
 for df in $d/*
   do let x=x+1
      if [[ $df == *G[0-9][0-9][0-9][0-9]V[0-9][0-9] ]]; then
         f1=$(basename $df)
         f2=${f1%V[0-9][0-9]}
         f3=${f2%.G[0-9][0-9][0-9][0-9]}
         f4=${f2##*G}
         f5=${f3}_00$f4
         mv -f $d/$f1 $d/$f5
         #==================
         echo "file# $x - $d/$f1 renamed to: $d/$f5"
         let y=y+1
      fi
   done
 echo "EOD - $y files renamed of $x total in $d"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1I2. Vancouver Utility scripts to Rename all files in directory

renameGDG1 - change '.G####V00' to '_000001' (keep latest)

 #!/bin/ksh
 # renameGDG1 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # renameGDG1 - change file suffix G####VXX to _000001 for all files in a directory
 #         - for mainframe GDG file conversions to unix/linux/windows
 #         - for exportgen functions called by JCL/scripts converted by VU
 #         - see www.uvsoftware.ca/mvsjcl.htm#Part_5
 #
 d="$1";
 if [[ ! -d "$1" ]]; then
    echo "usage:   renameGDG1 directory (arg1 must be a directory)"
    echo "         ===================="
    echo "example: renameGDG1 mstr"
    echo "         ==============="
    exit 1;
 fi
 echo "change all suffixes in $d from G####VXX to _00#### "
 echo " - enter to proceed (or kill)"; read reply
 x=0; y=0;
 for df in $d/*
   do let x=x+1
      if [[ $df == *G[0-9][0-9][0-9][0-9]V[0-9][0-9] ]]; then
         f1=$(basename $df)
         f2=${f1%.G[0-9][0-9][0-9][0-9]V[0-9][0-9]}
         f3=${f2}_000001
         mv -i $d/$f1 $d/$f3
         #==================
         echo "file# $x - $d/$f1 renamed to: $d/$f3"
         let y=y+1
      fi
   done
 echo "EOD - $y files renamed of $x total in $d"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1J1. Vancouver Utility scripts to Rename all files in directory

renameCC - convert any 1 character to a 2nd character

 #!/bin/ksh
 # renameCC - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # renameCC - rename an entire directory of filenames
 #          - replacing a specified character in filenames with an alternate
 #          - such as replacing all '.'s with '_'s
 #
 echo "rename files in subdir - replacing a pattern that can occur Anywhere"
 if [[ (-d "$1") && (-n "$2") && (-n "$3") ]]; then :
    else echo "usage: renameCC directory 'character' 'altcharacter'"
         echo "       ============================================="
         echo " - must enclose characters in single quotes"
         exit 1; fi
 #
 reply="n"
 until [ "$reply" = "y" ]
 do echo "do you have backup ? y/n"
    read reply
 done
 x=0; y=0
 for dfn in $1/*
 do fn1=${dfn##*/}           # remove any preceding directories
    fn2=$(echo $fn1 | tr $2 $3)
    let x=x+1
    if [[ $fn1 == $fn2 ]] then continue; fi
    mv -i $1/$fn1 $1/$fn2
    #====================
    let y=y+1
    echo "file# $x $1/$fn1 - renamed to: $1/$fn2"
 done
 echo "total $x files, $y renamed (replacing $2 with $3)"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_2 scripts1.doc - Vancouver Utility scripts for Conversion & General use

2A1. Vancouver Utility 'aliases'

'aliases save keystrokes for often repeated commands (alias l='ls -l' is a good example).

Here are the aliases defined in the profiles provided with Vancouver Utilities. See the common_profile listed at ADMjobs.htm#1C3 starting about line 200.

 alias rm='rm -i'              # confirm removes
 alias mv='mv -i'              # confirm renames
 alias cp='cp -i'              # confirm copy overwrites
 alias rmf='rm -f'             # force removes (no prompts)
 alias mvf='mv -f'             # force renames (no prompts)
 alias cpf='cp -f'             # force copies (no prompts)
 alias l='ls -l'               # save keystrokes
 alias more='less'             # less is way better than more
 alias grep='grep -n'          # ensure -n option used on grep
 alias cdd='cd $RUNDATA'       # quick access to data superdir
 alias cdl='cd $RUNLIBS'       # quick access to libs superdir
 alias cdc='cd $CNVDATA'       # quick access to data conversion superdir
 alias cdk='cd $CMPDATA'       # quick access to data conversion superdir
 alias cdb='cd $BACKUP'        # quick access to backup superdir
 alias cdr='cd $RESTORE'       # quick access to restore superdir

notes

We recommend the aliases to prompt for overwrite, such as alias rm='rm -i' Use option '-f' when you have many files (rm -f tmp/*, etc)

Aliases such as cdd='cd $RUNDATA' depend on the prior definitions (in the profiles) of RUNDATA, RUNLIBS, etc in the profiles. This is described in ADMjobs.htm#Part_1.

The result is that programmers will be changed to their testdata,testlibs,etc. But operators would be changed to their proddata, prodlibs, etc. For example the definitions of RUNDATA & RUNLIBS for a programmer might be:

 export RUNDATA=/home/userxx/testdata     <-- private testdata/testlibs
 export RUNLIBS=/home/userxx/testlibs
 export RUNDATA=/p1/apps/testdata         <-- common testdata/testlibs
 export RUNLIBS=/p1/apps/testlibs           - shared by all programmers

Note that some sites may define RUNDATA & RUNLIBS with rather long pathnames & the aliases can save more keystrokes than indicated here.

A major advantage is that administrators can change the locations of data & libraries. The use of environmental variables RUNDATA & RUNLIBS means that programmers & operators do not have to change their usual commands.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_3 Vancouver Utility scripts for Conversion & General use

listing file & directory names


3A1. llc - list files with File & Line Counts

3A2. llc - sample output from llc /home/uvadm/sf/util
(where Vancouver Utility scripts are stored)

3B1. llm - list long & pipe to more
- similar to alias llm='ls -l | more' (but a script vs an alias)
- saves keystrokes for this often repeated command

3B2. lla - same as llm, but lists All files including hidden .files

3C1. llt - list files, with the latest First

3C2. lls - list files in Size order

3C3. llu - list files Unsorted

3D1. lld - list directories only (omit files)

3E1. llr - list files & subdirs Recursively (all the way down)

3F1. lslp - list files & print immediately
- prints using 'uvlp12' - VU laser printing utility
- see uvlp12 documented in uvlist.htm

3G1. spreadA - list filenames 4-up across the page
- great for JCL/scripts, COBOL programs, copybooks, etc
  (directories with 100's or 1000's of modules)

3H1. longest1 - find longest line in a text file

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3A1. Vancouver Utility scripts for Conversion & General use

llc - list directory with File & Line counts

 #!/bin/ksh
 # llc - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # llc - list filenames in directory with file & line counts
 echo "llc - list filenames in directory with file & line counts"
 if  [[ -z "$1" ]] ; then d="."; else d=$1; fi
 if [[ ! -d $d ]]; then
    echo "usage: llc directory"
    echo "       ============="
    echo "- arg1 may be omitted to assume current directory"
    exit 1; fi
 typeset -R5 cntf=00000 cnte=00000
 typeset -R7 cntl=0000000
 integer page=0 cntfi=0 cntd=0 cntt=0
 test -d tmp || mkdir tmp
 d1=$(echo $d | tr '/' '_')
 echo "File#   Lines" >tmp/llc_$d1
 for df in $d/*
   { ((cnte+=1))
     if [[ ! -f $df ]]; then
        ((cntd+=1))
        cntl=0
     else
     ((cntf+=1))
        lsl=$(ls -l $df)
        cntx=$(wc -l $df)
        cntl=${cntx%% $df}    # drop right hand filename
        ((cntt+=cntl))
     fi
     echo "$cnte $cntl $df" >>tmp/llc_$d1
     ((cntfi+=1));
   }
 echo "$cntf files, $cntd directories, $cntt total lines in directory $d" >>tmp/llc_$d1
 more tmp/llc_$d1
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3A2. Vancouver Utility scripts for Conversion & General use

sample output for 'llc'


 llc /home/uvadm/sf/util   <-- list Vancouver Utility script filenames
 =======================     - with File & Line counts (25 files/screen)
 llc - list filenames in directory with file & line counts
 File#   Lines
     1      43 sf/util/acum1
     2      22 sf/util/all2lower
     3      22 sf/util/all2upper
     --- 133 lines omitted ---
   137      23 sf/util/lla
   138      34 sf/util/llc
   141      17 sf/util/lld
   145      30 sf/util/llf
   146      21 sf/util/llm
   147      28 sf/util/llnot
   148      23 sf/util/llr
   149      18 sf/util/llra
   150      23 sf/util/llrp
   151      23 sf/util/lls
   152      23 sf/util/llsr
   153      23 sf/util/llt
   154      23 sf/util/llta
   155      23 sf/util/lltm
   156      23 sf/util/lltr
   157      22 sf/util/llu
     --- 318 lines omitted ---
   476      65 sf/util/xvseprog2
   477      57 sf/util/xvsesli1
   478      59 sf/util/xvsesli2
   478 files, 0 directories, 18191 total lines in directory sf/util

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3B1. Vancouver Utility scripts for Conversion & General use

llm - list long & pipe to more

 #!/bin/ksh
 # llm - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # llm  - script to list any directory with long option & pipe into more
 echo "llm - list directory with long option & pipe to more"
 if (($# > 1));then
    echo "usage: llm directory"
    echo "       ============="
    echo "example: llm 'jcls/map*'"
    echo "         ==============="
    echo "- enclose any *wildcard patterns in single quotes"
    exit 1; fi
 #
 if  [[ -z "$1" ]] ; then d="."; else d=$1; fi
 ls -l $d | more
 #==============
 cntf=$(ls $d | wc -l)
 echo "$cntf files in $d"
 echo "use script 'llc' for file & line counts within each file"
 echo "lls - biggest files 1st, llt - newest files 1st, lltr - oldest files 1st"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3B2. Vancouver Utility scripts for Conversion & General use

lla - lists All files including hidden .files

 #!/bin/ksh
 # lla - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # lla - script to list directory with ALL & LONG option & pipe into more
 #     - same as llm, but with '-a' option to list hidden .files
 #
 #usage with '*' wildcard patterns
 # lla dir/abc*      <-- does not work (will match only 1st occurrence)
 # lla 'dir/abc*'    <-- does work (enclose patterns in single quotes)
 #
 echo "lla - list All files (including .hidden) & pipe to more"
 if (($# > 1)); then
    echo "usage: lla directory"
    echo "       ============="
    echo "example: lla 'jcls/map*'"
    echo "         ==============="
    echo "- enclose any *wildcard patterns in single quotes"
    exit 1; fi
 #
 if  [[ -z "$1" ]] ; then d="."; else d=$1; fi
 ls -al $d | more
 #===============
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3C1. Vancouver Utility scripts for Conversion & General use

llt - list files, latest First

 #!/bin/ksh
 # llt - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # llt  - list directory long in date sequence with Newest first
 #      - pipe to more (show 1 screen & wait for null entry)
 #
 #usage with '*' wildcard patterns
 # llt dir/abc*      <-- does not work (will match only 1st occurrence)
 # llt 'dir/abc*'    <-- does work (enclose patterns in single quotes)
 #
 echo "llt - list directory in date sequence with Newest files 1ST"
 if (($# > 1));then
    echo "usage: llt directory"
    echo "       ============="
    echo "example: llt 'jcls/map*'"
    echo "         ==============="
    echo "- enclose any *wildcard patterns in single quotes"
    exit 1; fi
 #
 if [[ -z "$1" ]] ; then  d="."; else d=$1; fi
 ls -lt  $d | more
 #================
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3C2. Vancouver Utility scripts for Conversion & General use

lls - list files in Size order

 #!/bin/ksh
 # lls - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # lls - List directory with options Long sorted by file Size & pipe to more
 #     - sorts biggest files first
 #
 #usage with '*' wildcard patterns
 # lls dir/abc*      <-- does not work (will match only 1st occurrence)
 # lls 'dir/abc*'    <-- does work (enclose patterns in single quotes)
 #
 echo "lls - list directory sorted by file-size (biggest files 1st)"
 if (($# > 1));then
    echo "usage: lls directory"
    echo "       ============="
    echo "example: lls 'cbls/car*'"
    echo "         ==============="
    echo "- enclose any *wildcard patterns in single quotes"
    exit 1; fi
 #
 if  [[ -z "$1" ]] ; then d="."; else d=$1; fi
 ls -lS $d | more
 #===============
 exit 0
 #

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3C3. Vancouver Utility scripts for Conversion & General use

llu - list files Unsorted

 #!/bin/ksh
 # llu - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # llu - list directory with options Long & Unsorted & pipe into More
 #
 #usage with '*' wildcard patterns
 # llu dir/abc*      <-- does not work (will match only 1st occurrence)
 # llu 'dir/abc*'    <-- does work (enclose patterns in single quotes)
 #
 echo "llu - list directory with options Long & Unsorted & pipe to More"
 if (($# > 1));then
    echo "usage: llu directory"
    echo "       ============="
    echo "example: llu 'jcls/car*'"
    echo "         ==============="
    echo "- enclose any *wildcard patterns in single quotes"
    exit 1; fi
 #
 if  [[ -z "$1" ]] ; then d="."; else d=$1; fi
 ls -lU $d | more
 #===============
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3D1. Vancouver Utility scripts for Conversion & General use

lld - list directories only (omit files)

 #!/bin/ksh
 # lld - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # lld - list directories ONLY with long option
 #
 echo "lld - list directories only"
 if (($# != 1)); then
    echo "usage: lld 'pattern'"
    echo "       ============="
    echo "example1: lld '/usr/*'    <-- list all subdirs in /usr"
    echo "          ============"
    echo "example2: lld '/usr/lib*' <-- list  /usr subdirs with prefix lib"
    echo "          ==============="
    exit 1; fi
 #
 ls -ld $1 | more
 #===============
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3E1. Vancouver Utility scripts for Conversion & General use

llr - list files & subdirs Recursively

 #!/bin/ksh
 # llr - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # llr  - list a directory with long & Recursive option & pipe into more
 #      - Recursive option shows contents of all sub-directories
 #
 #usage with '*' wildcard patterns
 # llr dir/abc*      <-- does not work (will match only 1st occurrence)
 # llr 'dir/abc*'    <-- does work (enclose patterns in single quotes)
 #
 echo "llr - list directory with long & Recursive options & pipe to more"
 if (($# > 1));then
    echo "usage: llr directory"
    echo "       ============="
    echo "example: llr 'jcls/map*'"
    echo "         ==============="
    echo "- enclose any *wildcard patterns in single quotes"
    exit 1; fi
 #
 if [[ -z "$1" ]] ; then d="."; else d=$1; fi
 ls -lR $d | more
 #===============
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3F1. Vancouver Utility scripts for Conversion & General use

lslp - list filenames in directory & print immediately

 #!/bin/ksh
 # lslp - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # lslp - list directory (long) & print via uvlp12
 #
 echo "lslp - list directory (long) & print via uvlp12"
 d1="$1";
 if [[ "$d1" == "." ]]; then d1=$(pwd); fi
 if [[ ! -d "$d1" ]]; then
    echo "usage: lslp directory"
    echo "       =============="
    exit 9; fi
 #
 if [[ ! -d tmp ]]; then mkdir tmp; fi
 dir=$(basename $d1)
 ls -l $1 > tmp/$dir
 files=$(ls $1 | wc -l)
 echo "directory: $dir, files = $files" >>tmp/$dir
 #================================================
 uvlp12 tmp/$dir
 #==============
 exit 0

Note re 'uvlp12'

This is 1 of the few scripts in this section (scripts1.doc) that call a Vancouver Utility vs a unix/linux command. You could change 'uvlp12' to 'lp'.

'uvlp12' is a script that calls the 'uvlist' utility, which is a Vancouver Utility C program that inserts page headings with filename, userid, page#, etc.

These scripts also insert PCL5 commands for laser printing with margins for 3 hole punching & for various cpi (in this case 12 cpi to get 90 chars across).

You can see many more 'uvlp__' scripts at uvlist.htm.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3G1. Vancouver Utility scripts for Conversion & General use

spreadA - list filenames 4-up across the page

 #!/bin/ksh
 # spreadA - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # spreadA - list directory in multi-columns with options for columns & width
 #
 # - spreadA uses 'ls' to generate a tmp file, input to uvcopy job spread1
 # - uvcopy/spread1 prompts with options for no of columns & column-width
 # - report is created in 5up subdir with headings for dirname & date/time
 # - prompt for report disposition: vi, more, cat, lp, uvlp, etc (or null)
 # - report is left in subdir 5up named the same as the specified directory
 #
 echo "spreadA - list directory in multi-columns, options for columns & width"
 echo "        - will leave report in subdir 5up/dirname.5up"
 dir="$1"
 if [[ ! -d "$dir" ]]; then
    echo "usage: spreadA directory"
    echo "       ================="
    exit 99; fi
 #
 if [[ ! -d tmp ]]; then mkdir tmp; fi
 if [[ ! -d 5up ]]; then mkdir 5up; fi
 ls $dir >tmp/$dir         # create file of filenames in specified directory
 #
 # create full path name of directory, to show on report (if not already)
 if [[ $dir = /* ]]; then DIR=$dir; else DIR=${PWD}/$dir; fi
 export DIR=$DIR           # export full path dirname for uvcopy job hdngs
 export JOBID=spreadA      # export JOBID (spreadA) for uvcopy job hdngs
 # use basename of directory as uvcopy job output filename in 5up
 dirfile=$(basename $dir)
 #
 uvcopy spread1,fili1=tmp/$dir,filo1=5up/${dirfile}.5up
 #=====================================================
 exit 0
Note
  • this is 1 of the few scripts in this scripts1.doc that call a
    Vancouver Utility vs a unix/linux command.
  • see notes about 'uvcopy' on the next page.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3F2. Vancouver Utility scripts for Conversion & General use

script spreadA calls 'uvcopy job' spread1

'uvcopy' is Vancouver Utility C program for general purpose data manipulation. It is an interpreter for the instructions in a separate instruction file. 'spread1' is found via 'PFPATH', defined in the profile ('ADMjobs.doc#1C3').

The 'spreadA' script (above) uses 'ls' to capture all filenames in a tmp/file, and calls uvcopy job 'spread1' to format the filenames across the page.

Note the options 'uop=c18n4' at the end of the uvcopy command line above. 'c18' sets Column width to 18 & 'n4' sets Number of columns to 4.

See the uvcopy job 'spread1' listed at UVjobs1.htm#U6.

Also see sample operating instructions & a sample report at UVjobs1.htm#L1.

uvcopy can perform complex data manipulations with very few instructions. It is an essential part of the mainframe conversions because it can do many things not possible with the standard unix/linux utilities (such as processing packed decimal fields & handling Indexed files compatible with Micro Focus COBOL). See more about uvcopy at uvcopy1.htm.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3H1. Vancouver Utility scripts for Conversion & General use

longest1 - find longest line in a text file

 #!/bin/ksh
 # longest1 - find longest line in a file
 #
 echo "longest1 - find longest line in a file"
 file="$1";
 if [[ ! -f "$file" ]]; then
    echo "usage: longest1 filename"
    echo "       ================="
    echo " - arg1 must be a file"
    echo " "
    echo "example: longest1 sf/util/longest1"
    echo "         ========================="
    exit 1; fi
 #
 integer lines=0;        # init line counter
 integer longest=0;      # init longest line char count
 longestline="";         # init longest line data
 exec 3< $file           # open file for reading
 while read -u3 line     # read next filename
   do ((lines+=1))
      l1c=${#line}
      l1cn=${l1c%% *}
      if ((l1cn > longest)); then
         longest=l1cn; longestline=$line; fi
   done
 exec 3<&-               # close file of filenames
 echo "longest line: $longest characters in $file, contents:"
 echo "$longestline"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_4 Vancouver Utility scripts for Conversion & General use

scripts to process all files in directory


4A1. alldiff2 - script to repeat 'diff' for all file pairs in 2 directories
- 'diff' is a marvelous unix/linux utility for text files
- alldiff is great for verifying changes in conversion techniques
- confirm desired changes & no unintended changes

4A2. alldiff2sub - compare all files in all sub-dirs in a pair of super-dirs

4B1. allcmp - script to repeat 'cmp' for all file pairs in 2 directories
- 'cmp' compares any type of file (packed/binary fields, no LFs)
- but very poor at indicating differences (1 byte per line)
- follow with 'uvcmp' utilities to show record pairs with * diffs
  (see CMPjobs.htm)

4C1. all2lower - translate to lower-case, while copying all files to 2nd subdir

4C2. all2upper - translate to UPPER-case, while copying all files to 2nd subdir

4D1. rmzf - remove all zero-length (empty) files from a directory
- use after mass processing that may produce empty files
  example - converting copybooks to record layouts (cobmaps)
  - no output for procedure copybooks

4E1. allrm - remove all files in a directory
- use when number of files exceed unix/linux limits
  (was required for earlier unix versions with lower limits)
- also see allcp, allmv, etc

4F1. allcancel - cancel all printer requests (spooled by 'lp') for current user
- does an lpstat to get request-ids & cancels them

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part 4 Contents continued


4G1. Summary pf scripts to list/copy/move/remove old/new file
These scripts are recursive, descending all levels of a directory tree,
looking for files older/newer than a specified no of days.

4G2. demo lsold1 & rmold1 to remove files older than 730 days (2 years)

4H1. lsold1 - list files older than a specified number of days
lsold2 - list files created/modified within 2 specified days old

4H2. lsnew1 - list files newer than a specified number of days
lsnew2 - list files created/modified within 2 specified days

4H3. rmold1 - remove files older than the sspecified number of days
- recursive, descends directory tree searching for all old files
- can 1st list,move,or copy old files with lsold1,rmold1,mvold1
- alrternates rmold2,rmold3 to prompt before removal

4H4. rmold2 - lists all old files, then prompts for each old file to be removed

4H5. rmold3 - lists all old files, then prompts once to remove all old files
Note
  • the following scripts are not listed, but may be useful to you
  • if you have Vancouver Utilities installed, see them in sf/util/...
mvold1
  • move files older than the x days to a specified directory
cpold1
  • copy files older than the x days to a specified directory
lsold1p
  • list files older than x days & filenames with a specified pattern
lsold2p
  • list files created between 2 dates with a specified pattern
lsnew1p
  • list files newer than x days & filenames with a specified pattern
cpnew1p
  • copy files newer than the x days to a specified directory

4I1. editmfp1 - edit multiple files in directory in Size order
- arg1 = directory, arg2 = no of files (default 5)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4A1. scripts to process all files in directory

alldiff - compare all text files in 2 directories

 #!/bin/ksh
 # alldiff2 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # alldiff2 - script to compare all text files in 1 directory to a 2nd directory
 #          - redirects output to tmp/dir2.dif & prompts to view/print/etc
 # alldiff  - original version, dsiplays output on screen only
 #          - alldiff2 better, allows subsequent --> vi tmp/dir2.dif <--
 # alldiff2sub - compare all files in all subdirs in a pair of superdirs
 #
 d1="$1"; d2="$2";
 echo "alldiff2 - compare all text files in 2 subdirs & write dif report to tmp"
 if [[ -d "$1" && -d $2 ]]; then :
 else echo "usage: alldiff dir1 dir2"
      echo "       ================="
      echo "example: alldiff cbls.old cbls"
      echo "         ====================="
      exit 1; fi
 #
 d2b=$(basename $d2)   # get basename of dir2 (drop any preceding /path/...)
 log=tmp/$d2b.dif      # make name for output log file
 >$log                 #init logfile in tmp subdir w same name as dir2 + .dif
 x=0; y=0;
 for i in $d1/*
 do
   let x=x+1
   typ=$(file $i)
   if [[  $typ == *text* || $typ == *script* || $typ == *data* ]]
      then f=${i##*/}
           diff -a -bBwZ $d1/$f $d2/$f >>$log
           #=================================
           if [[ $? -gt 0 ]]; then
              echo "diff file# $x - $d1/... vs $d2/$f" >>$log
              echo " " >>$log
              let y=y+1
           fi
      else echo " file# $x $typ - NOT text/script file" >>$log
   fi
 done
 lines=$(wc -l $log)      # capture line count
 echo "$y different of $x files compared $d1 to $d2" >>$log
 echo "$y diff of $x files in $d1 & $d2, report is: $lines"
 echo "--> use uvlp12,uvlp14,uvlp16 to laser print at 12,14,16 cpi"
 echo "--> enter command (vi,cat,more,uvlp12,etc, or null)"
 read ans
 if [[ ! "$ans" = "" ]]; then $ans $log; fi
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4A2. scripts to process all files in directory

alldiff2sub - compare all files in all subdirs in 2 superdirs

 #!/bin/ksh
 # alldiff2sub - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # alldiff2sub - run alldiff2 for all subdirs in a pair of superdirs
 #             - by Owen Townsend, UV Software, Oct22/2013
 #
 # Writes diff reports to tmp/... named as subdir1.dif, subdir2.dif, etc
 #
 # Calls 'alldiff2' for each pair of subdirs
 # - alldiff2 prompts for (vi,more,etc) at end each pair)
 # - so best to enter null until end all pairs, then vi tmp/*
 #
 echo "alldiff2sub - run alldiff2 for all subdirs in a pair of superdirs"
 d1="$1"; d2="$2";
 if [[ -d "$d1" && -d "$d2" ]]; then :
 else echo "usage: alldiff2sub superdir1 superdir2"
      echo "       ==============================="
      echo "- arg1 & arg2 must be directories with multiple subdirs"
      exit 1; fi
 x=0;
 for d1sd in $d1/*
  { sd=$(basename $d1sd)
    if [[ -d $d1/$sd ]]; then
       let x=x+1
       echo "$x alldiff2 $d1/$sd to $d2/$sd"
       alldiff2 $d1/$sd $d2/$sd
       #=======================
    fi
  }
 echo "$x alldiff2 reports created in tmp/... for superdirs $d1 & $d2"
 exit

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4B1. scripts to process all files in directory

allcmp - compare all packed/binary files in 2 directories

 #!/bin/ksh
 # allcmp - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # allcmp - script to execute the UNIX 'cmp' command to compare
 #          corresponding files in 2 directories
 #
 #usage:  allcmp dir1 dir2 ['pattern']     (pattern defaults to *)
 #        ============================
 #
 #example#1: allcmp tmp1 tmp2         - compare all files in tmp1 & tmp2
 #           ================
 #
 #example#1: allcmp tmp1 tmp2 'a*'    - compare all files in tmp1 & tmp2
 #           =====================      whose names begin with 'a'
 #
 echo "allcmp - compare files in 2 directories, using unix 'cmp'"
 echo "- for files with packed/binary & no LineFeeds (use diff for text files)"
 #
 if [[ -d "$1" && -d "$2" ]]; then :
    else echo "usage: allcmp dir1 dir2"
         echo "       ================"
         echo "example1: allcmp gl.old gl"
         echo "          ================"
         echo "example2: allcmp mstr.old mstr 'gl*'"
         echo "          =========================="
         echo " - arg3 optional prefix* must be in single quotes"
         exit 99; fi
 #
 pat='*'            # default pattern to '*'
 if [[ -n "$3" ]]; then pat="$3"; fi
 x=0;y=0;
 for i in $1/$pat
   { let x=x+1
     f=${i##*/}
     cmp $1/$f $2/$f
     #==============
     if (($? != 0)); then
        echo "file# $x cmp $1/$f $2/$f"
        echo " ";       # generate blank line to highlight diffs
        let y=y+1
     fi
   }
 echo "$y mismatches of $x total files compared in $1 & $2"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4C1. scripts to process all files in directory

all2lower - translate to lower copying all files to 2nd dir

 #!/bin/ksh
 # all2lower - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # all2lower - script to translate all files to lower case
 #             while copying from 1 directory to a 2nd directory
 #
 echo "all2lower - translate all files in directory to lower case in 2nd dir"
 d1="$1"; d2="$2";
 if [[ -d "$d1" && -d "$d2" ]]; then :
 else echo "usage: all2lower dir1 dir2"
      echo "       ==================="
      echo " - arg1 & arg2 must be directories"
 exit 1; fi
 integer fn=0
 for df in $d1/*
  {
   f=$(basename $df)
   ((fn+=1))
   echo "file# $fn - $d1/$f"
    tr [:upper:] [:lower:] <$d1/$f >$d2/$f
  }
 echo "$fn total files translated to lower case from $d1 to $d2"
 exit

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4C2. scripts to process all files in directory

all2upper - translate to UPPER copying all files to 2nd dir

 #!/bin/ksh
 # all2upper - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # all2upper - script to translate all files to UPPER case
 #             while copying from 1 directory to a 2nd directory
 #
 echo "all2upper - translate all files in directory to UPPER case in 2nd dir"
 d1="$1"; d2="$2";
 if [[ -d "$d1" && -d "$d2" ]]; then :
 else echo "usage: all2upper dir1 dir2"
      echo "       ==================="
      echo " - arg1 & arg2 must be directories"
 exit 1; fi
 integer fn=0
 for df in $d1/*
  {
   f=$(basename $df)
   ((fn+=1))
   echo "file# $fn - $d1/$f"
    tr [:lower:] [:upper:] <$d1/$f >$d2/$f
  }
 echo "$fn total files translated to UPPER case from $d1 to $d2"
 exit

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4D1. scripts to process all files in directory

rmzf - remove zero-length (empty) files from a directory

 #!/bin/ksh
 # rmzf - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # rmzf - script to remove zero-length files from a directory
 #
 # use after jobs that process 100's of files
 # but produce output for only a small no of the input files
 # this will remove the zero-length files from the directory
 # many other methods wont work because you get arg list too long
 #
 echo "rmzf - remove all zero-length (empty) files in directory"
 if [[ ! -d "$1" ]]; then
    echo "usage: rmzf directory"
    echo "       =============="
    exit 99; fi
 #
 x=0; y=0
 for i in $1/*
 do
   if [[ ! -s $i ]]; then
      rm -f $i
      #=======
      let x=x+1
   fi
   let y=y+1
 done
 echo "removed $x files of $y files in directory $1"
 exit

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4E1. scripts to process all files in directory

allrm - remove all files in a directory

 #!/bin/ksh
 # allrm - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # allrm - script to remove all files from a directory
 #       - use if no of files greater than allowed by UNIX shell (512,1024,?)
 #         (when you get errmsg "arg list too long")
 #       - also provides audit trail with file-counts
 #
 #usage: allrm directory
 #       ===============
 #
 reply="n"
 until [ "$reply" = "y" ]
   do echo "Are you sure you want to remove files from $1 ? y/n"
      read reply
   done
 #
 if [ -d "$1" ]; then :
 else echo "USAGE: allrm dir1 "
 exit 1; fi
 x=0;
 for i in $1/*
   { let x=x+1
     echo "removing file# $x - $i "
     rm -f $i
     #=======
   }
 echo "total $x files removed from $1"
 exit

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4F1. scripts to process all files in directory

allcancel - cancel all printer requests for current user

 #!/bin/ksh
 # allcancel - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # allcancel - cancel all printer requests for the current user
 #           - no arguments required
 #           - very handy when you need to cancel all your print requests
 #             (because cancel does not provide an option for all requests)
 #
 if [[ ! -d tmp ]]; then mkdir tmp; fi  # if tmpdir not present, make it
 lpstat >tmp/allcan1                    # request-id is 1st word on each line
 cut -f1 -d' ' tmp/allcan1 >tmp/allcan2 # cut 1st word to separate file
 # now open the file & read line contents into variable $reqid
 exec 3< tmp/allcan2                    # open file for reading
 while read -u3 reqid                   # read current request id
   do
     cancel $reqid                      # cancel request id
     echo "$reqid cancelled"
     #======================
   done
 exec 3<&-                              # close the file
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4G1. CNVaids: Unix/Linux Aids

scripts to list/copy/move/remove old/new files

These scripts are recursive, they process an entire directory tree, descending all levels, looking for files older/newer than a specified no of days.


4H1. lsold1 - list files older than a specified number of days
lsold2 - list files created/modified within 2 specified days old

4H2. lsnew1 - list files newer than a specified number of days
lsnew2 - list files created/modified within 2 specified days

4H3. rmold1 - remove files older than the sspecified number of days
- recursive, descends directory tree searching for all old files
- can 1st list,move,or copy old files with lsold1,rmold1,mvold1
- alrternates rmold2,rmold3 to prompt before removal

4H4. rmold2 - lists all old files, then prompts for each old file to be removed

4H5. rmold3 - lists all old files, then prompts once to remove all old files
Note
  • see above scripts listed on the following pages
Note
  • the following scripts are not listed, but may be useful to you
  • if you have Vancouver Utilities installed, see them in sf/util/...
mvold1
  • move files older than the x days to a specified directory
cpold1
  • copy files older than the x days to a specified directory
lsold1p
  • list files older than x days & filenames with a specified pattern
lsold2p
  • list files created between 2 dates with a specified pattern
lsnew1p
  • list files newer than x days & filenames with a specified pattern
cpnew1p
  • copy files newer than the x days to a specified directory

help screens

You can see a help screen for each script, by entering the script-name only, without its required arguments (omit the directory & no of days).

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4G2. CNVaids: Unix/Linux Aids

demo lsold1 & rmold1

Given
  • data directory with file creations dates 2010,2011,2012,& 2013
Required
  • Remove any files created more than 2 years ago (730 days)

 #1 l data    <-- list data directory to see file creation dates
 =========
 -rw-rw-r-- 1 berks apps 186208 Mar 22  2013 abilitydeletesfixa
 -rw-rw-r-- 1 berks apps   2081 Dec 26  2012 berkscodes
 -rw-rw-r-- 1 berks apps    980 Sep  8  2010 empnomagmed100_100908
 -rw-rw-r-- 1 berks apps    193 Sep 14  2010 empnotice_091310
 -rw-rw-r-- 1 berks apps    205 Sep 26  2011 networkgetin
 -rw-rw-r-- 1 berks apps    180 Nov 11 09:53 othperscomptime
 -rw-r--r-- 1 berks apps  26584 Nov 24  2010 securedsite.PDF
 -rw-rw-r-- 1 berks apps    139 Sep 26  2011 sharepointnotes
 -rw-rw-r-- 1 berks apps     84 Jun 19  2013 todo

 #2. lsold1 data 730   <-- see which files will be removed
     ===============
 -rw-rw-r-- 1 berks apps 193 Sep 14  2010 data/empnotice_091310
 -rw-rw-r-- 1 berks apps 139 Sep 26  2011 data/sharepointnotes
 -rw-r--r-- 1 berks apps 26584 Nov 24  2010 data/securedsite.PDF
 -rw-rw-r-- 1 berks apps 980 Sep  8  2010 data/empnomagmed100_100908
 -rw-rw-r-- 1 berks apps 205 Sep 26  2011 data/networkgetin

 #3. rmold1 data +730 -f  <-- remove files older than 730 days (2 years)
     ===================      '-f' option Forces removal with no prompt
                              '-i' option would prompt to remove each file

 #4. l data      <-- list to see files remaining
     ======
 -rw-rw-r-- 1 berks apps 186208 Mar 22  2013 abilitydeletesfixa
 -rw-rw-r-- 1 berks apps   2081 Dec 26  2012 berkscodes
 -rw-rw-r-- 1 berks apps    180 Nov 11 09:53 othperscomptime
 -rw-rw-r-- 1 berks apps     84 Jun 19  2013 todo

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4H1. CNVaids: Unix/Linux Aids

lsold1 - list files older than x days

 #!/bin/ksh
 # lsold1 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # lsold1 - list files older than a specified number of days
 #
 if [[ -d "$1" && -n "$2" ]]; then :
    else echo "usage: lsold1 directory days"
         echo "       ====================="
         echo "example: lsold1 /u2/apps/data 30"
         echo "         ======================="
         echo "- list files in directory tree older than 30 days"
         echo " "
         echo "example2: lsold1 . 30  (use '.' if in directory)"
         echo "          ==========="
         echo "- Also see 'lsold1p' that allows a filename matching pattern"
         exit 1; fi
 days="+$2"
 #
 find $1 -mtime $days -type f -exec ls -l {} \;
 #=============================================
 # - list files older than a specified number of days
 exit 0
 #

lsold2 - list files created/modified between 2 days

 #!/bin/ksh
 # lsold2 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # lsold2 - list files modified/created between 2 days
 #
 if [[ -d "$1" && -n "$2" && -n "$3" ]]; then :
    else echo "usage: lsold2 directory days1 days2"
         echo "       ============================"
         echo "example: lsold2 /u2/apps/data 30 60"
         echo "         =========================="
         echo "- list files in directory older than 30 days & newer than 60 days"
         echo " "
         echo "example2: lsold2 . 30 60 "
         echo "         ======================"
         echo "- list files in current dir 30 to 60 days old"
         echo "Note - also see 'lsold2p' allows filename matching pattern"
         exit 1; fi
 #
 from=+$2; to=-$3
 #
 find $1 -mtime $from -mtime $to -exec ls -l {} \;
 #================================================
 exit 0
 #

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4H2. CNVaids: Unix/Linux Aids

lsnew1 - list files newer than x days

 #!/bin/ksh
 # lsnew1 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # lsnew1 - list files newer than or modified within specified number of days
 #
 if [[ -d "$1" && -n "$2" ]]; then :
    else echo "usage: lsnew1 directory days"
         echo "       ====================="
         echo "example: lsnew1 /u2/apps/data 30"
         echo "         ======================="
         echo "- list files in directory tree modified in last 30 days"
         echo " "
         echo "example2: lsnew1 . 30  (use '.' if in directory)"
         echo "          ==========="
         echo "- Also see 'lsnew1p' that allows a filename matching pattern"
         exit 1; fi
 days="-$2"
 #
 find $1 -mtime $days -type f -exec ls -l {} \;
 #=============================================
 # - list files newer than or modified within specified number of days
 exit 0
 #

lsnew2 - list files created/modified between 2 days

 #!/bin/ksh
 # lsnew2 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # lsnew2 - list files in a subdir newer than ctl/lsnew2.ctl
 #        - first create a control file with current time
 #        - then lsnew2 lists files in a subdir newer than the control file
 #
 if [[ ! -d "$1" ]]; then
    echo "prep:  touch ctl/lsnew2.ctl  <-- set current time on control file"
    echo "       ===================="
    echo "prep:  touch -d 10:00:00 ctl/lsnew2.ctl <-- set desired time"
    echo "       ================================     10AM for example"
    echo "usage: lsnew2 directory      <-- list files newer than ctl file"
    echo "       ================"
    echo "example: lsnew2 data"
    echo "         ==========="
    echo " - list files in data subdir newer than ctl/lsnew2.ctl"
    exit 1; fi
 #
 find $1 -newer ctl/lsnew2.ctl -type f -exec ls -l {} \;
 #======================================================
 # - list files in arg1 subdir newer than control file ctl/lsnew2.ctl
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4H3. CNVaids: Unix/Linux Aids

rmold1 - remove files older than x days

 #!/bin/ksh
 # rmold1 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # rmold1 - remove files from a specified directory (& all subdirs),
 #          that are older(+) than a specified number of days
 #        - option to prompt y/n for each file to be removed
 #
 # 3 variations of this script as follows:
 #*rmold1 (this) - remove old files, option to prompt for each file to be removed
 # rmold2 - lists all old files, then prompts for each old file to be removed
 # rmold3 - lists all old files, then prompts once to remove all old files
 #
 # Also see:
 # lsold1 - list files older than specified number of days
 #        - might run this script before 'rmold1' (removes files)
 #          to prove files to be removed are as intended
 # mvold1 - might run this script instead of 'rmold1' (removes files)
 #          so you could recover files from the move to dir if required
 #
 if [[ -d "$1" && "$2" == +* && ("$3" == "-i" || "$3" == "-f") ]]; then :
    else echo "usage: rmold1 directory +days option"
         echo "       ============================="
         echo "example rmold1 history +730 -i"
         echo "        ======================"
         echo " - remove files over 2 yrs old, prompting for confirmation"
         echo " - arg1 must be a directory"
         echo " - arg2 must start with + (older than)"
         echo " - arg3 must be '-i' for confirmation, '-f' for none"
         exit 1; fi
 #
 find $1 -mtime $2 -exec rm $3 {} \;
 #==================================
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4H4. CNVaids: Unix/Linux Aids

rmold2 - lists old files, then prompt to remove EACH

 #!/bin/ksh
 # rmold2 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # rmold2 - remove files from a specified directory (& all subdirs),
 #          that are older(+) than a specified number of days
 #        - lists all old files & prompts y/n for each file to be removed
 #        - prompts y/n for each file to be removed
 #        - 3 variations of this script as follows:
 #
 # rmold1 - prompts y/n for each old file to be removed
 #*rmold2 (this script) - lists all old files & prompts for each file to remove
 # rmold3 - lists all old files & prompts once to remove all old files
 #
 if [[ -d "$1" && "$2" == +* ]]; then :
    else echo "usage: rmold2 directory +days"
         echo "       ======================"
         echo "example: rmold2 history +730  <-- remove files over 2 yrs old"
         echo "         ==================="
         echo " - arg1 must be a directory"
         echo " - arg2 must start with + (older than)"
         exit 1; fi
 #
 find $1 -mtime $2 -exec ls -l {} \;  #<-- list all old files
 #==================================
 echo "files older than $2 days listed above, enter to remove (prompt for each)"
 read reply
 #
 find $1 -mtime $2 -exec rm -i {} \;  #<-- remove old files with prompt for each
 #==================================
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4H5. CNVaids: Unix/Linux Aids

rmold3 - lists old files, then prompt to remove ALL

 #!/bin/ksh
 # rmold3 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # rmold3 - remove files from a specified directory (& all subdirs),
 #          that are older(+) than a specified number of days
 #        - does not prompt y/n for each file to be removed (rmold1 for prompt)
 #          but lists files to be removed & prompts once for all to be removed
 #        - 3 variations of this script as follows:
 #
 # rmold1 - prompts y/n for each old file to be removed
 # rmold2 - lists all old files & prompts for each file to remove
 #*rmold3 (this script) - lists all old files & prompts once to remove all old
 #
 if [[ -d "$1" && "$2" == +* ]]; then :
    else echo "usage: rmold3 directory +days"
         echo "       ======================"
         echo "example: rmold3 history +730  <-- remove files over 2 yrs old"
         echo "         ==================="
         echo " - arg1 must be a directory"
         echo " - arg2 must start with + (older than)"
         exit 1; fi
 #
 # list files to be removed, prompt once y/n, then execute remove all
 #
 find $1 -mtime $2 -exec ls -l {} \;  #<-- list all old files
 #==================================
 #
 reply="x"
 until [[ "$reply" = "y" || "$reply" = "n" ]]
   do echo "remove files (listed above) from $1 ? y/n"
      read reply
   done
 #
 if [[ "$reply" = "y" ]]; then
    find $1 -mtime $2 -exec rm -f {} \;  #<-- remove all old files (no prompts)
    #==================================
 fi
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4I1. scripts to process all files in directory

editmfp1 - edit multi files in directory in Size order

 #!/bin/ksh
 # editmfp1 - edit Multi-files containing a specified Pattern
 #          - by Owen Townsend, UV Software, feb03/2013
 #
 echo "editmfp1 - edit Multi-files containing a specified Pattern"
 d="$1"; p="$2";
 if [[ -d "$d" && -n "$p" ]]; then :
    else echo "usage: editmfp1 directory pattern"
         echo "       =========================="
         echo "examfple: editmfp1 in1fix 'E#'"
         echo "          ===================="
         echo "- arg1 must be a directory, arg2 must be specified"
         echo "Enter :q for next file (not :n)"
         echo "- prompt every 5th file to allow quit before all files edited"
         exit 99; fi
 integer tf=0 ef=0 ef1=0;
 #
 for df in $d/*
   { ((tf++))            # count Total Files
     grep -q $p $df      # grep -q no output, set status 0 if pattern found
     if [[ $? -eq 0 ]];
        then vi $df      # edit files with match to pattern
             ((ef++)); ef1=$((ef%5))
             if [[ $ef1 == 0 ]]; then
                print "\nenter null to edit file# $ef OR 'Q' to Quit\n"
                read reply
                if [[ "$reply" == "Q" ]]; then exit 9; fi
             fi
     fi
   }
 echo "End editmfp1 - $ef files selected of $tf total files in directory"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_5 Vancouver Utility scripts for Conversion & General use

Counting Lines & Files in directories


5A1. v12 - help screen of scripts
- to count Lines, Files,& KiloBytes in directories
- optionally qualified by specified patterns
- sample outputs for each script

5B1. clf - Count Lines in 1 file (1 line result)

5C1. llc - list filenames in directory with line counts

5C2. llc - sample output from llc /home/uvadm/sf/util
(where Vancouver Utility scripts are stored)

5D1. cfdt - Count Files in 1 directory - Total only
- 1 line total count, does not list file-names (as do following)

5D2. cfdd - Count Files in all sub-Dirs of a super-Dir

5D3. cfdp - Count files with a pattern on any line in any file in directory

count Lines (& files) vs count Files above


5E1. clfp - Count Lines with a pattern in 1 file

5E2. clfdp - Count Lines with a pattern in ALL files in a directory

5F1. clfd - Count Lines & Files in a directory, optionally matching a pattern

5F2. clfd2 - Count Lines,Fiels,& KB in a directory, optionally matching a pattern

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5A1. Counting files in directories

v12 - help menu for counting scripts

'help' screens are stored at $UV/help/... & this is in the profile PATH, so you can display the help/menu screen anytime anywhere, for example:


 uvhelp   <-- display main menu of help screens
 ======     - determine desired sub-menu & enter its name, for example:

 v12      <-- display sub-menu of 'counting lines & files' (shown below)
 ====
Note

v12 - counting Lines & Files in directories

wc
  • unix Word Count, examples
llc
  • similar to 'ls -l | more', but with file seq# & line counts/file
cfd
  • Count Files in a Directory (listing each file)
cfdt
  • Count Files in a Directory, Total only (1 line answer)
cfdd
  • Count Files in Directories within a parent Directory
cfdp
  • Count Files in a Directory with a Pattern on any line in any file
clf
  • Count Lines in 1 File
clfd
  • Count Lines & Files in a Directory
clfp
  • Count Lines in a File with a specified Pattern
clfdp
  • Count Lines in all Files of a Dirctory with a Pattern

prep for demos using help files


 #0a. Login yourself --> /home/userxx
 #0b. cp -r $UV/help .  <-- copy help/ directory to your homedir

demo wc - Word Count


 #1. wc help/uvhelp            <-- wc 1 file at a time
     ==============
     88  505 3707 help/uvhelp  <-- sample output
  Lines Words Chars Filename   <-- field headings not shown by wc

 #2. wc help/uvhelp/*          <-- wc all files in directory
     ================
     88    505   3707 help/uvhelp
     58    360   2352 help/v11
     80    444   2953 help/v12
       --- 21 lines omitted ---
     79    389   3240 help/v93
   2417  13798 104767 total

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

demo clf - Count Lines in 1 File

Note
  • following (clf,llc,cfd,cfdp,cfdd,clfp,clfdp,clfd,clfd2)
    are scripts based on 'wc'

 clf help/uvhelp       <-- count Lines in $UV/help/uvhelp (main menu)
 ===============
      110 help/uvhelp  <-- response

demo llc - list all files in subdir with line counts


 llc help     <-- list files in help/... with Line counts
 ========
 File#   Lines
     1     110 -rwxrwxr-x 1 uvadm apps 4555 Apr  5 10:55 help/uvhelp
     2      58 -rwxrwxr-x 1 uvadm apps 2348 Apr  5 10:57 help/v11
     3     113 -rwxrwxr-x 1 uvadm apps 3953 Apr  5 10:57 help/v12
     --- 25 lines omitted ---
    29     148 -rwxrwxr-x 1 uvadm apps 5678 Apr  5 10:47 help/v92
    29 files,      2948 total lines in directory help

demo cfdt - Count Files in 1 directory - Total only


 cfdt help     <-- count files in help/... (1 line response)
 =========
       29 files in directory: help   <-- response

demo clf - Count Files & Lines in subdir

 clfd - Count Lines&Files in Directory
 File#    Lines File-Name
 #0001      198 help/uvhelp
 #0002       58 help/v11
 #0003      159 help/v12
 #0004      105 help/v13
       --- 24 lines omitted ---
 #0029      148 help/v85
 #0030      112 help/v91
 #0031      102 help/v92
 *****     3609 Total in directory help

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

demo clfd2 - Count Files,Lines,& KB in subdir


 clfd2 help
 =========
 File#       KB     Lines File-Name
 #0001        8       120 help/uvhelp
 #0002        4        57 help/v11
 #0003        8       155 help/v12
       --- 25 lines omitted ---
 #0029        8       101 help/v92
  0029      196     3280 Total files & lines in directory help

2nd demo clfd2 - Counts with optional pattern match


 clfd2 $UV/dat1 'vendor*.dat' <-- count files in dat1/* matching 'vendor*.dat'
 ===========================
 File#       KB     Lines File-Name
 #0001        4         8 /home/uvadm/dat1/vendormas4.dat
 #0002        4         0 /home/uvadm/dat1/vendormas5.dat
 #0003        4         0 /home/uvadm/dat1/vendormas6.dat
 #0004        4         6 /home/uvadm/dat1/vendorpay4.dat
  0004       16        14 Total files & lines in directory /home/uvadm/dat1
                          with matching pattern 'vendor*dat'

demo cfdd - count files in all sub-dirs of a super-dir


 cfdd sf
 =======
 cfdd - Count Files in all sub-Directories of a super-Directory
 00156 files in subdir #00001 sf/adm
 00079 files in subdir #00002 sf/demo
 00247 files in subdir #00003 sf/IBM
 00478 files in subdir #00004 sf/util
 00960 Total files in 00004 subdirs of sf as of 20160907:0953

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

demo clfp - Count Lines in a File with a Pattern


 clfp help/v21 'grep'
 ===================
 3 lines of 0000044 contain "grep" in file: help/v21

demo clfdp - Count Lines in all Files of a Directory with a Pattern


 clfdp help 'grep'
 ===============
 clfdp - count lines with a pattern in all files in a directory
       - alternate cflp - count lines with a pattern in 1 file
 0002 lines of 0000198 contain "grep" in file #0001 - help/uvhelp
 0014 lines of 0000158 contain "grep" in file #0003 - help/v12
 0003 lines of 0000044 contain "grep" in file #0007 - help/v21
 0013 lines of 0000051 contain "grep" in file #0008 - help/v22
 0001 lines of 0000116 contain "grep" in file #0011 - help/v25
 0033 Total lines contain "grep" in 0005 of 0031 files in directory: help

clfdp - 2nd demo illustrating 'not'


 clfdp help 'uvsoftware' not
 ==========================
 #0002: env/bashrc does not contain "uvsoftware"
 #0003: env/common_defines does not contain "uvsoftware"
 #0010: env/kshrc does not contain "uvsoftware"
 #0012: env/ReleaseNotes does not contain "uvsoftware"
 #0015: env/stub.ini does not contain "uvsoftware"
  0005 files of 0023 do not contain "uvsoftware" in directory: env

more info on VU scripts

You can see a help screen for each script, by entering the script-name only, without its required arguments (omit the directory). See these scripts listed on following pages http://uvsoftware.ca/scripts1.htm#5B1 - 5E1

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5B1. Counting files in directories

clf - Count Lines in a File

 #!/bin/ksh
 # clf - Vancouver Utility script stored at /home/uvadm/sf/util/clf
 # clf - Count Lines in 1 File
 #     - by Owen Townsend, UV Software, April 12/2012
 #
 echo "clf    - count lines in 1 file"
 if [[ ! -f "$1" ]]; then
    echo "usage: clf file"
    echo "       ========"
    echo " - arg1 must be a file"
    exit 9; fi
 #
 wc -l $1 2>/dev/null
 #===================
 echo "                ** Alternate scripts **"
 echo "    clfd  - Count Lines & Files in a Directory"
 echo "    clfp  - Count Lines with a Pattern in 1 file"
 echo "    clfdp - Count Lines with a Pattern in all files in a Directory"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5C1. Counting files in directories

llc - list filenames in directory with line counts

 #!/bin/ksh
 # llc - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # llc - list filenames in directory with file & line counts
 echo "llc - list filenames in directory with file & line counts"
 if  [[ -z "$1" ]] ; then d="."; else d=$1; fi
 if [[ ! -d $d ]]; then
    echo "usage: llc directory"
    echo "       ============="
    echo "- arg1 may be omitted to assume current directory"
    exit 1; fi
 typeset -R5 cntf=00000 cnte=00000
 typeset -R7 cntl=0000000
 integer page=0 cntfi=0 cntd=0 cntt=0
 test -d tmp || mkdir tmp
 d1=$(echo $d | tr '/' '_')
 echo "File#   Lines" >tmp/llc_$d1
 for df in $d/*
   { ((cnte+=1))
     if [[ ! -f $df ]]; then
        ((cntd+=1))
        cntl=0
     else
     ((cntf+=1))
        lsl=$(ls -l $df)
        cntx=$(wc -l $df)
        cntl=${cntx%% $df}    # drop right hand filename
        ((cntt+=cntl))
     fi
     echo "$cnte $cntl $df" >>tmp/llc_$d1
     ((cntfi+=1));
   }
 echo "$cntf files, $cntd directories, $cntt total lines in directory $d" >>tmp/llc_$d1
 more tmp/llc_$d1
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5C2. Vancouver Utility scripts for Conversion & General use

sample output for 'llc'


 llc /home/uvadm/sf/util   <-- list Vancouver Utility script filenames
 =======================     - with File & Line counts (25 files/screen)
 File#   Lines
     1      43 -rwxrwxr-x 1 uvadm apps 1929 Jan 20 17:29 sf/util/acum1
     2      21 -rwxrwxr-x 1 uvadm apps 900 Jan 20 17:29 sf/util/allcancel
     3      29 -rwxrwxr-x 1 uvadm apps 828 Jan 20 17:29 sf/util/allchmod
                        --- 109 lines omitted ---
   112      23 -rwxrwxr-x 1 uvadm apps 775 Jan 20 17:29 sf/util/lla
   113      42 -rwxrwxr-x 1 uvadm apps 1336 Mar 22 10:36 sf/util/llc
   116      22 -rwxrwxr-x 1 uvadm apps 704 Jan 20 17:29 sf/util/llm
   118      23 -rwxrwxr-x 1 uvadm apps 782 Jan 20 17:29 sf/util/llr
   121      23 -rwxrwxr-x 1 uvadm apps 752 Jan 20 17:29 sf/util/lls
   122      23 -rwxrwxr-x 1 uvadm apps 797 Jan 20 17:29 sf/util/llsr
   123      23 -rwxrwxr-x 1 uvadm apps 767 Jan 20 17:29 sf/util/llt
   124      23 -rwxrwxr-x 1 uvadm apps 763 Jan 20 17:29 sf/util/lltr
   125      22 -rwxrwxr-x 1 uvadm apps 715 Jan 20 17:29 sf/util/llu
                        --- 289 lines omitted ---
   414      65 -rwxrwxr-x 1 uvadm apps 2621 Jan 20 17:29 sf/util/xvseprog2
   415      57 -rwxrwxr-x 1 uvadm apps 2294 Jan 20 17:29 sf/util/xvsesli1
   416      59 -rwxrwxr-x 1 uvadm apps 2729 Jan 20 17:29 sf/util/xvsesli2
   416 files,     15878 total lines in directory sf/util

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5D1. Counting files in directories

cfdt - Count Files in Directory, Total only

 #!/bin/ksh
 # cfdt - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # cfdt - Count Files in 1 Directory (or more with wildcard) Total per Directory
 #      - can count Files in multiple directories if wildcards in quotes
 #      - by Owen Townsend, UV Software, April 12/2012
 echo "cfdt - Count Files in a directory - Total only (1 line per Directory)"
 echo "       (can count Files in multiple directories if wildcards in quotes)"
 d="$1";
 if [ -z "$d" ]; then
    echo "usage: cfdt directory"
    echo "       ============="
    echo "ex#1: cfdt help    <-- count files in help/ subdir"
    echo "      ========"
    echo "ex#2: cfdt 'sf/*'  <-- count files in all subdirs of sf/..."
    echo "      =========="
    echo "- must enclose in quotes if wildcards present"
    exit 1; fi
 #
 integer dn=0
 integer tf=0
 typeset -RZ5 d1f
 typeset -RZ5 tf
 for d1 in $d
   do if [[ -d $d1 ]]; then
         d1f=$(ls $d1 | wc -l)
         ((dn+=1))
         echo "$d1f Files in Directory: #$dn $d1"
         ((tf+=d1f))
      fi
   done
 if ((tf>d1f)); then
    echo "$tf Total Files in $dn Directories"
 fi
 echo "             ** Alternate Scripts **"
 echo "cfd   - count files in a directory Listing Filenames"
 echo "cfdp  - Count Files in Directory with a Pattern"
 echo "clf   - Count Lines in 1 File, clfd - count Lines&Files in a Directory"
 echo "clfdp - Count Lines&Files in Directory with a pattern"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5D2. Counting files in directories

cfdd - Count Files in all sub-Dirs of a super-Dir

 #!/bin/ksh
 # cfdd - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # cfdd - Count Files in a all sub-Directories within a Directory
 #
 # This assumes the parent directory contains only sub-directories
 # - any files directly beneath the parent directory are ignored
 # This script is intended to monitor counts of critical files
 # - in directories of data, programs, scripts, etc
 # - could run this script with your daily backup & maintain a few weeks
 #   of the file count reports, which could be reviewed when required
 #
 echo "cfdd - Count Files in all sub-Directories of a super-Directory"
 if [[ ! -d "$1" ]]; then
    echo "usage: cfdd parentdirectory"
    echo "       ===================="
    echo "ex#1:  cfdd /home/uvadm/   <-- count files in uvadm/subdirs"
    echo "       ================="
    echo "ex#1:  cfdd data >control/data_$(date +%Y%m%d)"
    echo "       ======================================"
    echo "       - save today's data file counts in a control file"
    exit 99; fi
 #
 typeset -RZ5 cnt=0 fcnt=0 dcnt=0
 #
 for dir in $1/*
 do if [[ ! -d $dir ]]; then continue; fi
    ((dcnt+=1))
    cnt=$(ls $dir | wc -w)
    #=====================
    ((fcnt+=cnt))
    echo "$cnt files in subdir #$dcnt $dir"
 done
 echo "$fcnt Total files in $dcnt subdirs of $1 $(date +%Y%d%m:%H%M)"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5D3. Counting files in directories

cfdp - Count Files in a Directory with a Pattern any line any file

 #!/bin/ksh
 # cfdp  - Vancouver Utility script stored at /home/uvadm/sf/util/cfdp
 # cfdp  - Count Files in a Directory with a Pattern on any line in any file
 #       - by Owen Townsend, UV Software, April 2014
 # clfdp - also see clfdp Count Lines in all Files of a Direcctory with a Pattern
 #
 echo "cfdp - count files with a pattern on any line in any file in directory"
 d="$1"; p="$2"; not="$3"; v=""; n="";
 if [[ -d "$d" && -n "$p" ]]; then :
    else echo "usage: cfdp directory pattern [not]"
         echo "       ==========================="
    echo "ex#1: cfdp help 'grep'      <-- count files with 'grep' in directory help/"
    echo "      ==============="
    echo "ex#2: cfdp help 'grep' not  <-- count files that do NOT contain 'grep'"
    echo "      ===================="
    echo " - arg1 is a directory, arg2 is the pattern, arg3 optional 'not' for non-matches"
    exit 9; fi
 #
 if [[ "$not" == "not" || "$not" == "NOT" ]]; then v="-v"; n='!'; fi
 typeset -RZ4 cnt=0 cntt=0 files=0
 for df in $d/*
   { ((files+=1))            # count files in directory
     cnt=$(grep -c $p $df 2>/dev/null)   # count matches in current file
     #================================
     if ((${n}$cnt)); then   # if pattern found in current file
        onxlines=""; if ((cnt)); then onxlines="on $cnt lines"; fi
        echo "file#$files: $df does $not contain \"$p\" $onxlines"
        ((cntt+=1))          # acum matching files in directory
     fi
   }
 echo "     $cntt files of $files do $not contain \"$p\" in directory: $d"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5E1. Counting files in directories

clfp - Count Lines in a File with a Pattern

 #!/bin/ksh
 # clfp  - Vancouver Utility script stored at /home/uvadm/sf/util/clfp
 # clfp  - Count Lines in a File with a specified Pattern
 #      - by Owen Townsend, UV Software, April 2014
 # clfdp - also see clfdp count lines with a pattern in all files of a direcctory
 #
 echo "clfp  - count lines with a specified pattern (in 1 file)"
 echo "     - alternate clfdp - count lines with pattern in all files of directory"
 f="$1"; p="$2";
 if [[ -f "$f" && -n "$p" ]]; then :
    else echo "usage: clfp file pattern"
         echo "       ================"
    echo "ex#1: clfp help/v21 'grep'  <-- count lines with 'grep' in help/v21"
    echo "      ==================="
    echo "ex#2: clfp help/v44 '@'     <-- count lines '@' (email address ?)"
    echo "      ==================="
    echo " - arg1 is a file, arg2 is the pattern"
    exit 9; fi
 #
 wclines=$(wc -l $f)         # get no of lines in file
 flines=${wclines% *}        # drop filename
 typeset -RZ7 plines=$flines # Rt justify left Zero fill to 7 digits
 #
 mlines=$(grep -c $p $f)
 #===================
 echo "$mlines lines of $plines contain \"$p\" in file: $f"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5E2. Counting files in directories

clfdp - Count Lines in all Files of a Directory with a Pattern

 #!/bin/ksh
 # clfdp - Vancouver Utility script stored at /home/uvadm/sf/util/clfdp
 # clfdp - Count Lines in all Files of a Dirctory with a Pattern
 #       - by Owen Townsend, UV Software, April 2014
 # clfp  - also see clfp - Count Lines with a specified pattern (in 1 file)
 #
 echo "clfdp - count lines with a pattern in all files in a directory"
 echo "     - alternate cflp - count lines with a pattern in 1 file"
 d="$1"; p="$2";
 if [[ -d "$d" && -n "$p" ]]; then :
    else echo "usage: clfdp directory pattern"
         echo "       ======================="
    echo "ex#1: clfdp help 'grep'  <-- count lines with 'grep' in all files of help/"
    echo "      ================"
    echo "ex#2: clfdp help '@'     <-- count lines '@' (email addresses ?)"
    echo "      ================"
    echo " - arg1 is a directory, arg2 is the pattern"
    exit 9; fi
 #
 typeset -RZ4 cnt=0 cntt=0 files=0 mfiles=0
 typeset -RZ7 flines=0
 for df in $d/*
   { ((files+=1))            # count files in directory
     wclines=$(wc -l $df 2>/dev/null)  # get no of lines in file
     flines=${wclines% *}    # drop
     cnt=$(grep -c $p $df 2>/dev/null) # count matches in current file
     #================================
     if (($cnt)); then       # if pattern found in current file
        echo "$cnt lines of $flines contain \"$p\" in file #$files - $df"
        ((cntt+=cnt))        # acum total matching lines in all files
        ((mfiles+=1))        # count files with matches
     fi
   }
 echo "$cntt Total lines contain \"$p\" in $mfiles of $files files in directory: $d"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5F1. Counting files in directories

clfd - Count Files & Lines, optionally matching a pattern

 #!/bin/ksh
 # clfd - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # clfd - Count Lines & Files in a Directory, optionally matching a pattern
 #     - if text file, show no of lines in file, else '---'
 #
 echo "clfd - Count Lines&Files in directory [optional pattern match]"
 echo "    - alternates: clfd2 counts Lines,Files,& KiloBytes in a directory"
 echo "      clf/cfd - Count Lines in 1 file /or Files in 1 directory"
 echo "      clfp/clfdp - Count Lines with a pattern in 1 file or all files"
 d="$1"; p="$2"; db=$(basename $d); tmpf=tmp/$db.clfd; >$tmpf
 if [[ -d "$d" && $# -lt 3 ]]; then :
    else echo "usage: clfd directory ['file-match-pattern']"
         echo "       ===================================="
         echo " - arg1 directory, arg2 optional 'file-match-pattern' in single quotes"
         echo " - arg2 optional 'file-match-pattern' in single quotes (default '*')"
         echo " "
         echo "ex#1: clfd dat1 '*.dat'     <-- count files in dat1/ with '.dat' suffix"
         echo "      ================"
         echo "ex#2: clfd dat1 '*vendor*'  <-- count files with 'vendor' anywhere in name"
         echo "      ==================="
    exit 1; fi
 #
 echo "File#    Lines File-Name" >>$tmpf
 integer fs=0 fsp=0 lnt=0 ln2=0
 if [[ -z "$p" ]]; then p='*'; fi
 for df in $d/$p
 do
   ((fs+=1))
   f=${df##*/}
   ((fsp+=1))
   ln1=$(wc -l $df 2>/dev/null); ln2=${ln1% *}; ln3=${ln2##* };
   printf "#%04d " fsp >>$tmpf
   if [[ $ln3 -eq 0 && -s $df ]]
      then printf "     --- " >>$tmpf
      else printf "%8d " ln3  >>$tmpf
   fi
   printf "$df \n"     >>$tmpf
   ((lnt+=ln3))       # accumulate total lines
 done
 if [[ -n "$2" ]]; then
    printf "***** %8d Totals with matching pattern $p\n" lnt >>$tmpf
 else
    printf "***** %8d Total in directory $d\n" lnt >>$tmpf
 fi
 more $tmpf
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5F2. Counting files in directories

clfd2 - Count Files, Lines,& KB, optionally matching a pattern

 #!/bin/ksh
 # clfd2 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # clfd2 - Count Lines,Files,& KB in a directory, optionally matching a pattern
 #      - if text file, show no of lines in file, else '---'
 #
 echo "clfd2 - Count Files&Lines in directory [optional pattern match]"
 echo "     - alternates: clf counts Lines & Files (but not KiloBytes)"
 echo "       clf/clfd - counts Lines in 1 file /or Files in 1 directory"
 echo "       clfp/clfdp - Count Lines with pattern in 1 file or all in directory"
 d="$1"; p="$2"; db=$(basename $d); tmpf=tmp/$db.clfd2; >$tmpf
 if [[ -d "$d" && $# -lt 3 ]]; then :
    else echo "usage: clfd2 directory ['file-match-pattern']"
         echo "       ======================================"
         echo " - arg1 directory, arg2 optional 'file-match-pattern' in single quotes"
         echo " - arg2 optional 'file-match-pattern' in single quotes (default '*')"
         echo " "
         echo "ex#1: clfd2 dat1 '*.dat'     <-- count files in dat1/ with '.dat' suffix"
         echo "      ================="
         echo "ex#2: clfd2 dat1 '*vendor*'  <-- count files with 'vendor' anywhere in name"
         echo "      ===================="
    exit 1; fi
 #
 echo "File#       KB     Lines File-Name" >>$tmpf
 integer fs=0 fsp=0 lnt=0 ln2=0
 if [[ -z "$p" ]]; then p='*'; fi
 for df in $d/$p
 do
   ((fs+=1))
   ((fsp+=1))
   f=${df##*/}
   fdus=$(\du -s $df | tr '\t' ' '); fdu=${fdus%% *};
   ln1=$(wc -l $df); ln2=${ln1% *}; ln3=${ln2##* };
   printf "#%04d " fsp >>$tmpf
   printf "%8d " $fdu  >>$tmpf
   if [[ $ln3 -eq 0 && -s $df ]]
      then printf "     --- " >>$tmpf
      else printf "%8d " ln3  >>$tmpf
   fi
   printf "$df \n"     >>$tmpf
   ((lnt+=ln3))       # accumulate total lines
   ((fdut+=fdu))
 done
 if [[ -n "$2" ]]; then
    printf "***** %8d  %8d Totals with matching pattern $p in directory $d\n" fdut lnt >>$tmpf;
 else
    printf "***** %8d  %8d Totals in directory $d\n" fdut lnt >>$tmpf;
 fi
 more $tmpf
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_6 - Vancouver Utility scripts for Conversion & General use - Contents

scripts to manipulate text&data files - contents


6A1. removeCR - remove Carriage Returns from text files (Windows --> Unix/Linux)
- copies to tmp/... removing CRs & prompts to copy back

6A2. insertCR - insert Carriage Returns in text files (Unix/Linux --> Windows)
- copies to tmp/... removing CRs & prompts to copy back

6A3. demo removeCR using uvhd to see CR's & LF's (can't be seen with editor)
6A4. demo insertCR

6A5. removeCRall - remove CRs from all files in directory, copying to 2nd dir
6A6. insertCRall - insert CRs into all files in directory, copying to 2nd dir

6B1. runsed1 - apply sed script to process all files from 1 dir to a 2nd dir
- great for mass changes to scripts & programs

6B2. runsed2 - follow-on script to use sed command saved by runsed1 in tmp/runsed1
- in case you wish to rerun same script on other dirs

6C0. uvfix1,A,2,B - scripts make it easy to apply powerful uvcopy instructions
to perform whatever data file manipulation required without
having to write the uvcopy job framework (file I/O, etc)

6D1. uvfix1 - copy 1 Text file to tmp/... applying uvcopy instructions
- may then inspect (with vi tmp/...) & copy back if desired

6E1. uvfixA - copy ALL Text files from 1 subdir to a 2nd subdir
  while applying uvcopy instructions
- may then inspect (with vi subdir2/*) & change name to subdir1

6F1. uvfix2 - copy 1 Fixed-Length file to tmp/... applying uvcopy instructions
- may then inspect (with uvhd tmp/...) & copy back if desired

6G1. uvfixB - copy ALL Fixed-Length files from 1 subdir to a 2nd subdir
  while applying uvcopy instructions
- may then inspect (with uvhd subdir2/...) & change name to subdir1

6H0. dropbl0 - drop all blank lines, using 'grep', copies to tmp/...
- for inspection & copy back
6H1. dropbl1 - drop all blank lines, using 'uvcp' with option b2
6H2. dropbl2 - reduce multiple blank lines to 1, using uvcp option b1
6H3. dropbl3 - drop blank lines & COBOL cmts (only '*'s & blanks)
6H4.  dropbl3.uv - uvcopy job called by dropbl3 script
6H5.  dropbl0,1,2,3 demo

6I1. printable1 - ensure printable characters in a text file
- copies to tmp/... converting any unprintables to '.' periods

6J1. parmnullfix1 - copy all files from 1 directory to a 2nd directory
- converting any nulls to blanks, using 'sed'
6J2. parmnullfix2 - copy all files from 1 directory to a 2nd directory
- blanking nulls & clearing cols 73-80, using 'uvcp'

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6A1. scripts to manipulate text files

removeCR - remove Carriage Returns from text files

 #!/bin/ksh
 # removeCR - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # removeCR - remove CR's Carriage Returns x'0D'
 #          - by copy to tmp/... & prompt to copy back
 echo "removeCR - copy a file to tmp/... removing CRs, prompt to copy back"
 echo "insertCR - alternate script to insert Carriage Returns"
 df="$1";
 if [[ -f "$df" && -d tmp ]]; then :
 else echo "usage: removeCR subdir/file"
      echo "       ===================="
      echo " - arg1 must be a text file (with LFs & CRs)"
      echo " - current working directory must also contain a tmp/ subdir"
      echo " - will copy to tmp/ removing CRs & prompt to copy back"
      echo " - also see insertCR to insert CR's (create CR/LF)"
      exit 1; fi
 f=$(basename $df)
 tr -d '\r' <$df >tmp/$f
 #======================
 linesin=$(wc -l $df); sizein=$(du -bs $df);
 linesout=$(wc -l tmp/$f); sizeout=$(du -b tmp/$f);
 echo "input  lines & bytes:    $linesin $sizein"
 echo "output lines & bytes:    $linesout $sizeout"
 until [[ "$reply" = "y" || "$reply" = "n" ]]
    do echo "copy back overwriting original y/n ?"; read reply; done
 if [[ "$reply" = "y" ]]; then
    cp tmp/$f $df; echo "copied back to $df"; ls -l $df
 else echo "copy back aborted"; fi
 exit 0

See the test/demo of removeCR on page '6A3' & insertCR on page '6A4'

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6A2. scripts to manipulate text files

insertCR - insert Carriage Returns in text files

 #!/bin/ksh
 # insertCR - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # insertCR - insert CR's Carriage Returns x'0D' in a text file
 #          - by copy to tmp/... inserting CRs & prompt to copy back
 echo "insertCR - copy a file to tmp/ inserting CRs & prompt to copy back"
 echo "removeCR - alternate script to remove Carriage Returns"
 df="$1";
 if [[ -f "$df" && -d tmp ]]; then :
 else echo "usage: insertCR subdir/file"
      echo "       ===================="
      echo " - arg1 must be a text file (with LFs)"
      echo " - current working directory must also contain a tmp/ subdir"
      echo " - will copy to tmp/... inserting CRs & prompt to copy back"
      echo " - also see removeCR to remove CRs"
      exit 1; fi
 f=$(basename $df)
 sed 's/$/\r/' <$df >tmp/$f
 #=========================
 linesin=$(wc -l $df); sizein=$(du -bs $df);
 linesout=$(wc -l tmp/$f); sizeout=$(du -b tmp/$f);
 echo "input  lines & bytes:    $linesin $sizein"
 echo "output lines & bytes:    $linesout $sizeout"
 until [[ "$reply" = "y" || "$reply" = "n" ]]
    do echo "copy back overwriting original y/n ?"; read reply; done
 if [[ "$reply" = "y" ]]; then
    cp tmp/$f $df; echo "copied back to $df"; ls -l $df
 else echo "copy back aborted"; fi
 exit 0

See the test/demo of removeCR on page '6A3' & insertCR on page '6A4'

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6A3. scripts to manipulate text files

removeCR - remove CR's from Windows text files

Text files from Windows often contain Carriage-Returns x'0D' as well as Line -Feeds x'0A'. Unix text files should be terminated only with a Line-Feed x'0A'. CR's can cause unix scripts to fail (especially if on the 1st line !/bin/ksh..) and you may be mystified because you cannot see them in the unix editors (use uvhd to see them). Here are scripts to remove CR's or insert CR's which might be required if you need to send unix files back to Windows.

Here is a demo using removeCR, 1st we will use 'uvhd' to confirm the presence of CR's (since you cannot see them with unix editors).


 #1. Login yourself --> /home/$HOME

 #2. mkdir tmp tmp1 tmp2  <-- make subdirs for test
     ===================

 #3. cp dat1/prodmas1 tmp1  <-- copy demo file to tmp1
     =====================

 #4. uvhd tmp1/prodmas1 t   <-- display demo file with CR & LF terminators
     ====================     - option 't' for Text files (ended by LFs)
                              - shows 1st record as follows:
                  10        20        30        40        50        60
 r#    1 0123456789012345678901234567890123456789012345678901234567890123
       0 BBQ010 BAR-B-Q         0019500..
         44533324452425222222222333333300
         2210100212D2D10000000000019500DA
 rec#=1 rsize=32 fptr=0 fsize=256 rcount=8
Note
  • Carriage Return x'0D' in byte 30 (column 31)
  • see Line Feed x'0A' in byte 31 (column 32)

 #5. removeCR tmp1/prodmas1   <-- run script to remove CR's
     ====================
     --> copy back overwriting original y/n ?
     --> y <-- reply y to copy back

 #6. uvhd tmp1/prodmas1 t   <-- confirm CR's removed
     ====================
                  10        20        30        40        50        60
 r#    1 0123456789012345678901234567890123456789012345678901234567890123
       0 BBQ010 BAR-B-Q         0019500.
         4453332445242522222222233333330
         2210100212D2D10000000000019500A
 rec#=1 rsize=31 fptr=0 fsize=248 rcount=8
Note
  • CR missing & LF x'0A' shifted back to byte 30 (column 31)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6A4. scripts to manipulate text files

insertCR - insert CR's in unix text files

Now we can demo insertCR, using tmp/prodmas1 (output of from removeCR above).


 #7. insertCR tmp1/prodmas1   <-- run script to insert CR's
     ======================
     --> copy back overwriting original y/n ?
     --> y <-- reply y to copy back

 #8. uvhd tmp1/prodmas1 t   <-- confirm CR's removed
     ====================
                  10        20        30        40        50        60
 r#    1 0123456789012345678901234567890123456789012345678901234567890123
       0 BBQ010 BAR-B-Q         0019500..
         44533324452425222222222333333300
         2210100212D2D10000000000019500DA
 rec#=1 rsize=32 fptr=0 fsize=256 rcount=8
Note
  • CR re-inserted & LF x'0A' shifted forward to byte 31 (column 32)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6A5. scripts to manipulate text files

removeCRall - remove CRs from all files in directory

 #!/bin/ksh
 # removeCRall - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # removeCRall - remove CR's Carriage Returns x'0D'
 #               while copying from 1 directory to a 2nd directory
 #             - by Owen Townsend, UV Software, Feb 2014
 #
 echo "removeCRall - copy all files from 1 dir to 2nd dir, removing CRs"
 echo "insertCRall - alternate script to insert Carriage Returns"
 d1="$1"; d2="$2";
 if [[ -d "$d1" && -d "$d2" ]]; then :
 else echo "usage: removeCRall indir outdir"
      echo "       ========================"
      echo " - arg1 & arg2 must be directories"
      echo " - also see insertCRall to insert CR's (create CR/LF)"
      exit 1; fi
 #
 integer fn=0
 for df in $d1/*
   { f=$(basename $df)
     ((fn+=1))
     echo "file# $fn - $d1/$f --> $d2 inserting CR's"
     tr -d '\r' <$d1/$f >$d2/$f
     #=========================
   }
 echo "$fn files copied from $d1 to $d2, with CR's removed"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6A6. scripts to manipulate text files

insertCRall - insert CRs into all files in directory

 #!/bin/ksh
 # insertCRall - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # insertCRall - insert CR's Carriage Returns x'0D'
 #               while copying from 1 directory to a 2nd directory
 #             - by Owen Townsend, UV Software, Feb 2014
 #
 echo "insertCRall - copy all files from indir to outdir, inserting CRs"
 echo "removeCRall - alternate script to remove Carriage Returns"
 d1="$1"; d2="$2";
 if [[ -d "$d1" && -d "$d2" ]]; then :
 else echo "usage: insertCRall indir outdir"
      echo "       ========================"
      echo " - arg1 & arg2 must be directories"
      echo " - also see removeCRall to remove CR's "
      exit 1; fi
 #
 integer fn=0
 for df in $d1/*
   { f=$(basename $df)
     ((fn+=1))
     echo "file# $fn - $d1/$f --> $d2 removing CR's"
     sed 's/$/\r/' <$d1/$f >$d2/$f
     #============================
   }
 echo "$fn files copied from $d1 to $d2, with CR's inserted"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6B1. scripts to manipulate text files

runsed1 - apply sed script to all files in directory

 #!/bin/ksh
 # runsed1 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # runsed1 - run sed script to process all files from 1 dir to a 2nd dir
 #         - will prompt for 1 or more sed script lines
 #
 #usage: runsed1 dir1 dir2
 #       =================
 #
 #example: change all 'lp' to 'llp' in all scripts in the 'sf' directory
 # 1. mv sf sf.old            - change name of existing dir (save)
 # 2. mkdir sf                - remake new dir
 # 3. runsed1 sf.old sf       - run script (will copy files to new sf)
 #    =================
 #    --> s/ lp / llp /g      - enter sed command at prompt
 #        ==============
 #    --> ^D                  - control-D to end command entry
 # 4. diff sf.old/xxx sf/xxx  - spot check results (or alldiff2 all files)
 #
 # runsed2 - follow-on script to use sed command saved by runsed1 in tmp/runsed1
 #         - to rerun same script on other dirs
 #           or could modify script & rerun on same dirs
 #
 # 1. vi tmp/runsed1                 - edit script saved as tmp/runsed1
 # 2. runsed2 dir1 dir2 tmp/runsed1  - 'runsed2' to execute saved script
 #
 # Also see uvcopy jobs rep1d & rep2d in REPjobs.htm
 # - sed is good when you have UNIQUE patterns & dont need qualifiers
 # - use rep1d or rep2d when you need to specify other patterns that
 #   must be present or not present on the same line
 # - rep1d & rep2d also create an audit report showing modified lines
 #
 d1=$1; d2=$2; sedscript=tmp/runsed1
 if [[ -d $d1 && -d $d2 ]]; then :
    else echo "usage: runsed1 dir1 dir2"
         echo "       ================="
         echo "- arg1 & arg2 must be input & output directories"
    exit 9; fi
 if [[ ! -d tmp ]]; then mkdir tmp; fi
 # prompt for sed script entry, using cat ended by control-d
 echo "enter sed script commands, end via control-D"
 echo "--> s/ lp / llp /g <-- sample sed to change ' lp ' to ' llp '"
 echo "--> ^D             <-- enter control-D to end sed lines entry"
 cat >$sedscript         # enter lines, end via ctl-D
 for i in $d1/*
 do f=$(basename $i)
    let x=x+1
    echo "file# $x - $d1/$f"
    sed -f $sedscript $d1/$f >$d2/$f
 done
 echo "total $x files processed from $d1 to $d2 by $sedscript"
 echo "sed script saved in $sedscript, rerun via runsed2 (vs runsed1)"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6B2. scripts to manipulate text files

runsed2 - run sed script to process all files from 1 dir to 2nd dir

 #!/bin/ksh
 # runsed2 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # runsed2 - run sed script to process all files from 1 dir to a 2nd dir
 #         - 'driver' script to run previously written 'sed' scripts,
 #           on all files in a directory, while copying to a 2nd directory
 #
 #usage: runsed2 dir1 dir2 sedscript
 #       ===========================
 #
 # Also note alternative 'runsed1' (vs this 'runsed2')
 # - runsed1 will prompt you to enter the sed script command lines
 # - runsed1 saves the script as: tmp/runsed1
 # - you may then use this runsed2 to re-execute the sed script
 #   (& you may use vi to modify the script before reruns)
 #
 #example:
 #
 # 1. mv sf sf.old                  - change name of existing dir (save)
 # 2. mkdir sf                      - remake new dir
 # 3. runsed2 sf.old sf tmp/runsed1 - run script (copies files to new sf)
 #    =============================
 # 4a. diff sf.old/xxx sf/xxx       - spot check results ?
 # 4b. alldiff2 sf.old sf           - or check all files with 'alldiff2'
 #
 d1=$1; d2=$2; sedfile=$3;
 if [[ -d $d1 && -d $d2 && -f $sedfile ]]; then :
    else echo "usage: runsed2 dir1 dir2 sedscript"
         echo "       ==========================="
         echo "- arg1 & arg2 must be directories"
         echo "- arg3 must be a file"
    exit 9; fi
 #
 for i in $d1/*
 do
    f=$(basename $i)
    let x=x+1
    echo "file# $x - $d1/$f"
    sed -f $sedfile $d1/$f >$d2/$f
 done
 echo "total $x files processed from $d1 to $d2 by $sedfile"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6C0. CNVaids: Vancouver Utility Aids

uvfix scripts & uvcopy jobs

If you can't perform your desired data file manipulation using unix 'sed', you can do anything you can imagine with the 'uvcopy' utlity.

'uvcopy' can do many things not possible with the standard unix/linux utilities such as 'sed' - processing 'packed decimal fields', handling 'Indexed files', & 'field addressing by column#' (common in mainframe utilities, but not in unix utilities).

The interpretive nature of uvcopy makes it easy to learn & to use. If you have a mainframe backgound, you may already be familiar with the uvcopy instruction formats since they are patterned after the IBM 360 assembler, but interpretive - no compile required, just edit & execute with the uvcopy interpreter.

uvcopy has over 90 instructions and about half of these are pwerful subfunctions. For example 'fix' will convert delimited files with any specified delimiter (tab, comma, pipe, etc) to fixed formats. 'fix' is very useful to convert the tab or csv delimited files from Excel spread- sheets to fixed format files required by COBOL applications. The 'var' instruction (opposite of 'fix') will convert fixed-formats to delimited formats for importing into spread-sheets or database tables.

uvfix scripts - easy way to use uvcopy

The 'uvfix' scripts make it easy to employ the powerful uvcopy instructions without having to program the uvcopy framework (file I/O, etc). You only need to enter the uvcopy instructions desired when prompted. See http://www.uvsoftware.ca/uvcopy3.htm for the 90 instructions available.

There are 4 'uvfix_' scripts (in /home/uvadm/sf/util/...) that call 4 'uvfix_' uvcopy jobs (from /home/uvadm/pf/util/...). The scripts make the command line as simple as possible - specify only the input file-name & record-size if fixed. The output files are written to the tmp/... subdir (in your current dir) where you can inspect & then copy back overwriting the input file if desired.

uvfix1
  • copy 1 Text file to tmp/... applying uvcopy instructions
  • may then inspect (with vi tmp/...) & copy back if desired
uvfixA
  • copy ALL Text files from 1 subdir to a 2nd subdir
    while applying uvcopy instructions
  • may then inspect (with vi subdir2/*) & change name to subdir1
uvfix2
  • copy 1 Fixed-Length file to tmp/... applying uvcopy instructions
  • may then inspect (with uvhd tmp/...) & copy back if desired
uvfixB
  • copy ALL Fixed-Length files from 1 subdir to a 2nd subdir
    while applying uvcopy instructions
  • may then inspect (with uvhd subdir2/...) & change name to subdir1

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6D1. uvfix scripts & uvcopy jobs

demo uvfix1,A,2,B

Here are some demos used to illustrate the 4 uvfix_ scripts available. You can run these demos from /home/uvadm/ using the demo files provided in /home/uvadm/dat1/...


 #0. Login uvadm --> /home/uvadm
 #0a. Or Login yourself if you are in same group as uvadm
 #0b. cd /home/uvadm

uvfix1 ex#1 - modify 1 text file with 'mvc'

Given
  • text file /home/uvadm/dat1/names1
  • multiple sets of 5 line Names & Addresses
  • 1st 5 records shown below

 #1. more dat1/names1      <-- investigate Input file
     ================
      Owen Townsend
      UV Software
      4667 Hoskins Rd
      North Vancouver BC Canada
      V7K2R3
Required
  • replicate the records for 3 up label printing
  • output should be as follows:

 #2. uvfix1 dat1/names1  <-- run uvfix1 & enter instructions
     ==================      at the prompts as follows:
     uop=q1n99r1024 - user option defaults
           n99      - output record count limit (high# copies to EOF)
              r1024 - recsize ONLY for Fixed-Length OUTPUT records if desired
                    - for Fixed-Length, reply RSF/RST at prompt (default text typ=LSTt)
                    - for Text files: accept default recsize & default typ=LSTt
     User OPtion (uop) defaults  = q1r1024n999999

 #2a. -->null accept or override -->  <-- null entry to accept defaults
 #2b. -->LST = default file type -->  <-- null accept typ=LST

 #3. Enter Instructions or '.' to process
     ===================================
 #3a. mvc b27(25),a0    <-- move(copy) cols 1-25 to 28-53"
 #3b. mvc b56(25),a0    <-- move(copy) cols 1-25 to 56-71"
 #3c. .                 <-- enter '.' to end instrns & execute job"

 #4. more tmp/names1    <-- inspect output file
     ===============      - 1st 5 records as follows:
     Owen Townsend              Owen Townsend                Owen Townsend
     UV Software                UV Software                  UV Software
     4667 Hoskins Rd            4667 Hoskins Rd              4667 Hoskins Rd
     North Vancouver BC Canada  North Vancouver BC Canada    North Vancouver BC Canada
     V7K2R3                     V7K2R3                       V7K2R3

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6D2. uvfix scripts & uvcopy jobs

uvfix1 ex#2 - convert Tab-delimited file to Fixed-format

Given
  • tab delimited file of Employer-Codes, User-Names,& Email Addresses
  • from Excel spreadsheet, saved as 'Tab_Delimited',& transferred to Unix
  • see uvhd display of 1st 4 records (field header & 1st 3 data records)
  • note tabs show as '.'s & x'09's in uvhd vertical hexadecimal display
required
  • convert to the Fixed-Format shown below

 uvhd /home/owen/dat1/payemails t
 ================================
 rec#=1 rsize=37 fptr=0 fsize=121 rcount=4
                   10        20        30        40        50        60
 r#     1 0123456789012345678901234567890123456789012345678901234567890123
        0 PayrollService.Username.EmailAddress.
          5677666567766605767666604666646676770
          0192FCC352693595352E1D595D19C1442533A
 rec#=2 rsize=32 fptr=37 fsize=121 rcount=4
                   10        20        30        40        50        60
 r#     2 0123456789012345678901234567890123456789012345678901234567890123
       37 acupay.Sheila.Sheila@acupay.com.   <-- '.'s are Tabs or LineFeeds
          66776705666660566666466776726660   <-- Tabs x'09' (vertical hex)
          13501993859C193859C10135019E3FDA   <-- LF's x'0A' (vertical hex)
 rec#=3 rsize=28 fptr=69 fsize=121 rcount=4
                      10        20        30        40        50        60
 r#     3 0123456789012345678901234567890123456789012345678901234567890123
       69 adcomp.Jack.Jack@adcomp.com.
          6666670466604666466666726660
          143FD09A13B9A13B0143FD0E3FDA
 rec#=4 rsize=24 fptr=97 fsize=121 rcount=4
                      10        20        30        40        50        60
 r#     4 0123456789012345678901234567890123456789012345678901234567890123
       97 adp.Linda.Linda@adp.com.
          667046666046666466726660
          1409C9E419C9E410140E3FDA

Fixed-Format created by uvfix1 from tab-delimited

 payrollservice      username            emailaddress
 acupay              sheila              sheila@acupay.com
 adcomp              jack                jack@adcomp.com
 adp                 linda               linda@adp.com

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

uvfix1 ex#2 (continued) Tab-delimited to Fixed-format


 #0.  Login uvadm --> /home/uvadm
 #0a. Or Login yourself if you are in same group as uvadm
 #0b. cd /home/uvadm

 #1. uvhd dat1/payemails t    <-- use uvhd to investigate file
     =====================      - option 't' (Text file, with tab delimiters)

 #2. uvfix1 dat1/payemails    <-- run uvfix, specify input file only
     =====================      - output file will be tmp/payemails
 #2a. -->      <-- prompt for options, reply null
 #2b. --> LSTt <-- prompt for output file type, reply null accept LSTt default
      --> reply to instruction prompts as follows:

 #3.      reply to instruction prompts as follows:
          ========================================
 #3a. --> fix c100(100),a0(80),3,x'09' <-- convert to fixed fields 100 apart
                                - copies from area 'a' to c100,c200,c300
 #3b. --> mvc b00(20),c100    <-- store Employer-Code in cols 1-20
 #3c. --> mvc b20(20),c200    <-- store User-Name in cols 21-40
 #3d. --> mvc b40(60),c300    <-- store Email-SAddress in cols 41-100
 #3e. --> trl b0(100)         <-- ensure all lower-case
 #3f. --> .                   <-- '.' ends input instrns, executes,
          ===========              & writes output to tmp/payemails

 #5. vi tmp/payemails         <-- inspect output
     ================

 #6. cp tmp/payemails dat1/   <-- copyback overwriting original
     ======================     - or change name as desired

Notes

  1. See all uvcopy instructions documented at www.uvsoftware.ca/uvcopy3.htm

  2. uvfix1 prompts for instructions until you ennter '.' to end input.

  3. uvfix1 executes as follows:

3a. Reads each record into area 'a' & copies to area 'b' for possible output (In example above, we fix to area 'c' & then move to area 'b' for output).

3b. Executes stored instructions on each record.

3c. Writes record from area 'b', would write to output file unchanged if you entered no instructions (just entered '.' to end instruction input).

  1. If you wanted to inhibit output, you would enter 'ret<' instruction, which returns with condition code <, which inhibits the ussual write. You would precede with some compare for something & skip 1 (over the ret<) if you want to output the record (ret= is automatic at end your instructions).

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6E1. uvfix scripts & uvcopy jobs

uvfixA - modify ALL Text files in subdir

Given
  • directory of COBOL programs /home/uvadm/cobol/cbls
Required
  • copy/modify all cobol/cbls/* to tmp2/...
  • clear cols 1-6 & sequence# in cols 1-4

 #0. Login uvadm --> /home/uvadm
 #0a. Or Login yourself if you are in same group as uvadm
 #0b. cd /home/uvadm

 #1. mkdir tmp2          <-- make outpur subdir
 #1a. rm -f tmp2/*         - or clear all files if tmp2 already exists

 #2. uvfixA cobol/cbls tmp2  <-- run uvfixA specifying I/O subdirs
     ======================    - reply to prompts as follows:
      uop=q1r512n999999 - user option defaults
            r512        - output reccsize for fixed length records typ=RSF/RST
                        - for text files: accept default recsize & default typ=LSTt
                n999999 - output record count limit (high# copies to EOF)
      User OPtion (uop) defaults  = q1r512n999999
 #2a. -->null to accept or enter/override -->   <-- null reply, no options required
 #2b. -->did you create outdir (dflt tmp) --> y <-- reply 'y' to continue

 #3.      reply to instruction prompts as follows:
          ========================================
 #3a. --> clr  b0(6),' '   <-- enter instrn to clear cols 1-6
 #3b. --> add  $ca1,1      <-- increment line counter
 #3c. --> mvn  b0(4),$ca1  <-- insert seuw=ence# in cols 1-4
 #3d. --> .                <-- enter '.' to end keyins & execute

sample output - 1st 8 lines of 1st program

 0001   identification division.
 0002  * car200 - test mainframe conversion to micro focus cobol
 0003  *        - vancouver utilities from www.uvsoftware.ca
 0004  *        - list sales details with customer name from custmas
 0005   program-id. car200.
 0006   environment division.
 0007   input-output section.
 0008   file-control.

^^^^ <-- sequence# gnerated by #3b & #3c above

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6F1. uvfix scripts & uvcopy jobs

uvfix2 - modify 1 Fixed-length file

Given
  • dat1/custmas1 with 12 mths sales 5 bytes packed for thisyear & lastyear
  • thisyear sales (12*5=60 bytes) in bytes 120-179 & lastyear in 180-239
Required
  • move this year sales to last year & clear thisyear

 #1. uvhd dat1/custmas1    <-- investigate Input file
     ================== note - thisyr sales 12 * 5 bytes packed start at 120
                             - lastyr sales 12 * 5 bytes packed start at 180
 rec#=1 rsize=256 fptr=0 fsize=8192 rcount=32
                   10        20        30        40        50        60
 r#     1 0123456789012345678901234567890123456789012345678901234567890123
        0 130140    EVERGREEN MOTORS LTD.    1815 BOWEN ROAD          NANA
          3333332222454545444244545524542222233332445442544422222222224444
          130140000056527255E0DF4F230C44E0000181502F75E02F140000000000E1E1
       64 IMO          BC V9S1H1    250-754-5531 LARRY WRENCH     ..4V|...
          4442222222222442535343222233323332333324455525544442222201357000
          9DF00000000002306931810000250D754D55310C12290725E38000000246C000
      128 .........W0....`........)X}..f3.....\.................4V}.......
          0000000005300016000000002570063100095000000000000000013570000000
          0C0000C0270D0540C0000C0098D0263C0444C0000C0000C0000C0246D0000C00
      192 .E|...V}.......................f.....<........f.C 19950531
          0470005700000000880000000018000680001300000000694233333333222222
          35C0046D0000C0023C0000C0083C0056D0012C0000C0016D3019950531000000

 #2. uvfix2 dat1/custmas1 r256  <-- run script, specify input file,& option
     =========================      'r256' for input record-size
                                 - prompts for commands, reply as follows
      uop=q1n99r256s0    - user option defaults
            n99          - output record limit (n99 means all records)
               r256      - Input Record size & output recsize default (if s0)
               r8192     - max record-size 8192
                    s0   - Output Recsize defaulted from Input recsize if s0
                    s80  - may specify value to override the default from insize
      User OPtion (uop) defaults  = q1n99r256s0r256
 #2a. -->null accept or override -->  <-- null reply (options on command line)
 #2b. -->RSF = default file type -->  <-- null accept typ=RSF

 #3.      reply to instruction prompts as follows:
          ========================================
 #3a. --> mvc    b180(60),a120      <-- move thisyear sales to lastyear sales
 #3b. --> mvnx12 b120(5p),0         <-- clear 12 thisyear sales to 5 bytes packed 0
 #3c. --> .                         <-- enter '.' end instructions & execute

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

uvfix2 - move thisyear to last & clear thisyear


 #4. Check output file with uvhd

 #4a. uvhd tmp/custmas1  <-- investigate Output file
      =================    - thisyr sales 12*5 bytes packed 120-179 now all 0
                           - lastyr sales 12*5 bytes packed 180-239
                             now moved from 120-179 (same as 120-179 above Input)
 rec#=1 rsize=256 fptr=0 fsize=8192 rcount=32
                   10        20        30        40        50        60
 r#     1 0123456789012345678901234567890123456789012345678901234567890123
        0 130140    EVERGREEN MOTORS LTD.    1815 BOWEN ROAD          NANA
          3333332222454545444244545524542222233332445442544422222222224444
          130140000056527255E0DF4F230C44E0000181502F75E02F140000000000E1E1
       64 IMO          BC V9S1H1    250-754-5531 LARRY WRENCH     ........
          4442222222222442535343222233323332333324455525544442222200000000
          9DF00000000002306931810000250D754D55310C12290725E38000000000C000
      128 ......................................................4V|.......
          0000000000000000000000000000000000000000000000000000013570000000
          0C0000C0000C0000C0000C0000C0000C0000C0000C0000C0000C0246C0000C00
      192 .....W0....`........)X}..f3.....\...............C 19950531
          0000053000160000000025700631000950000000000000004233333333222222
          00C0270D0540C0000C0098D0263C0444C0000C0000C0000C3019950531000000

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6G1. uvfix scripts & uvcopy jobs

uvfixB - modify ALL Fixed-Length files in subdir

Given
  • directory of sales files with 6 digit dates yymmdd in cols 14-15
  • Y2K demo convert 6 digit dates to 8 digits
Required
  • convert to 8 digit date ccyymmdd, shifting cols 14-62 to 16-64
  • insert cc as '20' if yy < 50 (1950) or '19' if yy > 50

 #1. mkdir tmp1 tmp2          <-- make subdirs for demo
 #1a. rm -f tmp1/* tmp2/*       - or clear if already existing

 #2. cp dat1/[123] tmp1/      <-- copy sales1,2,3 from dat1/... to tmp1/...

 #3. uvhd dat1/tmp1 r64       <-- investigate Input, must use uvhd (no LineFeeds)
     ==================         - see sample Input records for uvfix2 above
                                - note 6 digit date in bytes 13-18 (cols 14-19)
 rec#=1 rsize=64 fptr=0 fsize=1280 rcount=20
                   10        20        30        40        50        60
 r#     1 0123456789012345678901234567890123456789012345678901234567890123
        0 130140    21 940802 IN111001  HAM001  00002{ 0000001 00000002{
          3333332222332333333244333333224443332233333723333333233333333722
          130140000021094080209E1110010081D0010000002B00000001000000002B00
 rec#=20 rsize=64 fptr=1216 fsize=1280 rcount=20
                   10        20        30        40        50        60
 r#    20 0123456789012345678901234567890123456789012345678901234567890123
     1216 406082    65 020816 IN441     VAR200  00001R 0009001 00017101R
          3333332222332333333244333222225453332233333523333333233333333522
          406082000065002081609E441000006122000000001200009001000017101200

 #4. uvfixB tmp1 tmp2 r64s66  <-- run script, specify I/O subdirs,& options
     =======================      for in recsize & out recsize
                                - prompts & replies as follows:
      uop=q1n99r256s0    - user option defaults
            n99          - output record limit (n99 means all records)
               r256      - Input Record size & output recsize default (if s0)
               r8192     - max record-size 8192
                    s0   - Output Recsize defaulted from Input recsize if s0
                    s80  - may specify value to override the default from insize
      User OPtion (uop) defaults  = q1r8192n999999r64s66
 #4a. -->null accept or override -->  <-- null reply (options on command line)
 #4b. -->RSF = default file type -->  <-- null accept typ=RSF
 #4c. -->did you create outdir   --> y <-- reply 'y' to continue

 #5.      reply to instruction prompts as follows:
          ========================================
 #5a. --> ins  b13(51),'20'    <-- assume 2000+
 #5b. --> cmc  a13(2),'50'     <-- 1951-1999 ?
 #5c. --> skp< 1
 #5d. --> mvc  b13(2),'19'     <-- yes - change '20' to '19'
 #5e. --> .                    <-- enter '.' to end keyins & execute

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

uvfixB (cont) - Y2K demo check 1st output file


 #6. uvhd dat1/sales1 r64l66  <-- investigate Ouput, must use uvhd (no LineFeeds)
     =======================    - showing only 1st & last records below:
 rec#=1 rsize=66 fptr=0 fsize=1320 rcount=20
                   10        20        30        40        50        60
 r#     1 012345678901234567890123456789012345678901234567890123456789012345
        0 130140    21 19940802 IN111001  HAM001  00002{ 0000001 00000002{
          333333222233233333333244333333224443332233333723333333233333333722
          13014000002101994080209E1110010081D0010000002B00000001000000002B00
 rec#=20 rsize=66 fptr=1254 fsize=1320 rcount=20
                   10        20        30        40        50        60
 r#    20 012345678901234567890123456789012345678901234567890123456789012345
     1254 406082    65 20020816 IN441     VAR200  00001R 0009001 00017101R
          333333222233233333333244333222225453332233333523333333233333333522
          40608200006502002081609E441000006122000000001200009001000017101200
                       ^^ - Note century inserted 19/20 depending ><1950

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6H0. scripts to manipulate text files

dropbl0 - drop blank lines (using 'grep')

 #!/bin/ksh
 # dropbl0 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # dropbl0 - drop blank lines, by copy to tmp/.., inspect & copy back
 #         - by Owen Townsend, UV Software, Dec 2013
 #
 echo "dropbl0 * drop blank lines with grep (copy to tmp/, inspect & copy back)"
 echo "dropbl1 - alternate, same using uvcp with option b2 (gives line counts)"
 echo "dropbl2 - alternate, reduce multi blank lines to 1 (uvcp option b1)"
 echo "dropbl3 - alternate, drop blank lines & COBOL blank '*' lines (uvcopy)"
 df="$1";
 test -d tmp || mkdir tmp
 if [[ -f "$df" && -d tmp ]]; then :
 else echo "usage: dropbl0 subdir/file"
      echo "       ==================="
      echo " - arg1 must be a text file (with linefeeds & max recsize 512)"
      echo " - current working directory must also contain a tmp/ subdir"
      echo " - will copy file to tmp/..., for inspection & copy back"
      exit 1; fi
 #
 f=$(basename $df)
 d=${df%/*}
 grep -v "^ *$" $df >tmp/$f
 #=========================
 echo "file copied to tmp/$f, may inspect & copy back"
 echo "vi tmp/$f      <-- inspect"
 echo "cp tmp/$f $d   <-- copy back"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6H1. scripts to manipulate text files

dropbl1 - drop blank lines (using 'uvcp')

 #!/bin/ksh
 # dropbl1 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # dropbl1 - drop blank lines, by copy to tmp/.., inspect & copy back
 #         - by Owen Townsend, UV Software, Dec 2013
 #
 echo "dropbl0 - drop blank lines with grep (copy to tmp/, inspect & copy back)"
 echo "dropbl1 * drop blnak lines using uvcp with option b2 (gives line counts)"
 echo "dropbl2 - reduce multi blank lines to 1 (uvcp option b1)"
 echo "dropbl3 - drop blank lines & COBOL blank '*' lines (uvcopy)"
 df="$1";
 test -d tmp || mkdir tmp
 if [[ -f "$df" && -d tmp ]]; then :
 else echo "usage: dropbl1 subdir/file"
      echo "       ==================="
      echo " - arg1 must be a text file (with linefeeds & max recsize 512)"
      echo " - current working directory must also contain a tmp/ subdir"
      echo " - will copy file to tmp/..., for inspection & copy back"
      exit 1; fi
 #
 f=$(basename $df)
 d=${df%/*}
 uvcp "fili1=$df,rcs=512,typ=LSTtb2,filo1=tmp/$f"
 #===============================================
 echo "file copied to tmp/$f, may inspect & copy back"
 echo "vi tmp/$f      <-- inspect"
 echo "cp tmp/$f $d   <-- copy back"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6H2. scripts to manipulate text files

dropbl2 - reduce multiple blank lines to 1

 #!/bin/ksh
 # dropbl2 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # dropbl2 - reduce multiple contiguous blank lines to 1
 #         - by copy to tmp/.., for inspection & copy back
 #         - by Owen Townsend, UV Software, Dec 2013
 #
 echo "dropbl0 - drop blank lines wwith grep (copy to tmp/, inspect & copy back)"
 echo "dropbl1 - drop blnak lines using uvcp with option b2 (gives line counts)"
 echo "dropbl2 * reduce multi blank lines to 1 (uvcp option b1)"
 echo "dropbl3 - drop blank lines & COBOL blank '*' lines (uvcopy)"
 df="$1";
 test -d tmp || mkdir tmp
 if [[ -f "$df" && -d tmp ]]; then :
 else echo "usage: dropbl2 subdir/file"
      echo "       ==================="
      echo " - arg1 must be a text file (with linefeeds & max recsize 512)"
      echo " - current working directory must also contain a tmp/ subdir"
      echo " - will copy file to tmp/..., for inspection & copy back"
      exit 1; fi
 #
 f=$(basename $df)
 d=${df%/*}
 uvcp "fili1=$df,rcs=512,typ=LSTtb1,filo1=tmp/$f"
 #===============================================
 echo "file copied to tmp/$f, may inspect & copy back"
 echo "vi tmp/$f      <-- inspect"
 echo "cp tmp/$f $d   <-- copy back"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6H3. scripts to manipulate text files

dropbl3 - drop blank lines & *comment lines

 #!/bin/ksh
 # dropbl3 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # dropbl3 - drop blank lines, by copy to tmp/.., for inspection & copy back
 #         - option to drop blank COBOL comment lines '*' col 7 & all blank
 #         - by Owen Townsend, UV Software, Dec 2013
 #
 echo "dropbl0 - drop blank lines with grep  (copy to tmp/, inspect & copy back)"
 echo "dropbl1 - drop blnak lines using uvcp with option b2 (gives line counts)"
 echo "dropbl2 - reduce multi blank lines to 1 (uvcp option b1)"
 echo "dropbl3 * drop blank lines & COBOL blank '*' lines (uvcopy)"
 df="$1"; optn="$2"
 test -d tmp || mkdir tmp
 if [[ -f "$df" && -d tmp ]]; then :
 else echo "usage: dropbl3 subdir/file c0/c1"
      echo "       ========================="
      echo " - arg1 must be a text file (with linefeeds & max recsize 512)"
      echo " - arg2 c0/c1 'c1' drop '*' blank comments (as well as blank lines)"
      echo " - current working directory must also contain a tmp/ subdir"
      echo " - will copy file to tmp/..., for inspection & copy back"
      echo " - option c1 to drop lines with only '*'s (as well as blanks)"
      exit 1; fi
 #
 f=$(basename $df)
 d=${df%/*}
 uvcopy dropbl3,fili1=$df,filo1=tmp/$f,uop=$optn
 #==============================================
 echo "file copied to tmp/$f, may inspect & copy back"
 echo "vi tmp/$f      <-- inspect"
 echo "cp tmp/$f $d   <-- copy back"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6H4. scripts to manipulate text files

dropbl3 - uvcopy job to drop blank lines & *comment lines

 # dropbl3 - uvcopy Parameter File stored in: /home/uvadm/pf/util/
 # dropbl3 - drop blank lines with option to drop cobol *cmts
 #Dec14/14 - modify options c1,c2,c4 to drop COBOL *cmts if -,=,blanks
 #
 # uvcopy dropbl3,fili1=indir/infile,filo1=outdir/outfile,uop=c0/1/2/4
 # ===================================================================
 # - copies file to tmp/.., for inspection & copy back
 # - option to drop cobol *cmt lines with only '-'s, '='s, ' 's
 #
 # dropbl3 dir/file c0/c1/c2/c4  <-- script to call uvcopy job (as above)
 # ============================
 #
 # dropbl1 - also see script to drop ALL blank lines
 # dropbl2 - also see script to reduce multi blank lines to 1
 #
 opr='$jobname - drop blank lines with option to drop all "*" lines'
 opr='uop=q1c0j0s0t0 - option defaults'
 opr='    q0           - inhibit prompt to allow user to change options'
 opr='      c0       - do not drop *cmt lines with only "*"s'
 opr='      c1       - drop ALL COBOL *cmts "*" column 7'
 opr='      c2       - drop ALL COBOL *cmts "*" column 1 (cobmaps)'
 opr='      c4       - drop COBOL *cmts ONLY if no text descriptions'
 opr='                 IE - only "-"s, "="s,& blanks'
 opr='                 use c4 in combination with c1 &/or c2, for example:'
 opr='      c5       - drop COBOL *cmts "*" col 7 & only "-"s,"="s,or " "s'
 opr='        j1     - drop //* in cols 1-3 (comments in JCL)'
 opr='        j2     - drop # in col 1 (comments in JCL/scripts)'
 opr='          s1   - Squeeze left to column 1 & only 1 blank between words'
 opr='            t1 - Translate any unprintable characters to Blanks'
 uop=q1c0s0t0    # user option defaults
 fili1=?subdir/filexx,typ=LST,rcs=512
 filo1=?tmp/$fili1,typ=LSTt,rcs=512
 @run
        opn    all                    open files
 #
 # begin loop to get & put records until EOF
 man20  get    fili1,a0               get record into area 'a'
        skp>   man90                  (condition code set > at EOF)
        add    $ca1,1                 count input lines
        mvc    b0(512),a0             copy to output area
 #
 # drop all blank lines
 man24  cmc    a0(80),' '             line all blank ?
        skp=   man20                  yes - return to get next
 #

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

 # test option to drop COBOL *cmt lines '*' col 7 or 1
 man30  tsb    o3(1),x'01'            drop if '*' col 7 ?
        skp!   man35
        cmc    a6(1),'*'              '*' col 7 COBOL comment ?
        skp=   man20
 #
 # test option to drop COBOL *cmt lines '*' col 7 or 1
 man35  tsb    o3(1),x'02'            drop if '*' col 1 ?
        skp!   man40
        cmc    a0(1),'*'              '*' col 1 cobmap comment ?
        skp=   man20
 #
 # qualify drop */1 or */7 if option c4 only other chars '-' '=' ' '
 man40  tsb    o3(1),x'04'            qualify only '-' '=' ' ' ?
        skp!   man50                  no - drop */7 or */1
        mvc    c0(80),a0              copy to work area
        repm   c0(80),'*-=','   '     replace */-/= with blanks ?
        cmc    c0(80),' '             now all blank ?
        skp=   man20                  yes - bypass, return to get next
 #
 # test option to drop //* comments in JCL
 man50  tsb    o10(1),x'01'           drop if '//*' cols 1-3 ?
        skp!   man55
        cmc    a0(3),'//*'            '//*' cols 1-3 ?
        skp=   man20
 #
 # test option to drop //* comments in JCL
 man55  tsb    o10(1),x'02'           drop if '#' col 1 ?
        skp!   man60
        cmc    a0(1),'#'              '#' col 1 ?
        skp=   man20
 #
 # test option to translate any unprintables to blanks
 man60  tsb    o20(1),x'01'           translate unprintables to blanks ?
        skp!   man65
        trt    b0(512),$trtchr
 #
 # test option to squeeze multiblanks to 1 & left adjust
 man65  tsb    o19(1),x'01'           squeeze to col 1 ?
        skp!   man80
        sqzc1  b0(512),' '
 #
 # common point to output current line
 man80  put    filo1,b0               write record to output file
        add    $ca2,1                 count output lines
        skp    man20
 #
 man90  cls    all                    close files
        msgv1  'original file saved in $fili1, in=$ca1, out=$ca2    .'
        eoj                           end job

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6H5. scripts to manipulate text files

dropbl0,1,2,3 demo

Here we will illustrate 4 scripts to drop blank lines in text files. You can see the scripts listed at the link given at bottom of this page.

dropbl0
  • drop ALL blank lines, copy to tmp/ & back, using 'grep'
dropbl1
  • drop ALL blank lines, copy to tmp/ & back, using 'uvcp' optn b2
dropbl2
  • reduce multi blank lines to 1, using 'uvcp' option b1
dropbl3
  • drop blank lines & COBOL cmts with only '*'s & blanks ('uvcopy')
Note
  • these scripts copy to tmp/.., for inspection & copy back

 #1. Login yourself --> /home/userxx
 #2. mkdir dat1 tmp     <-- make subdirs in your homedir
 #2a. rm tmp/*          <-- or remove tmp/* files if subdirs already existing

 #3. copy demo files from /home/uvadm/dat1 to your $HOME/dat1
     - renaming so we can test the 4 dropbl scripts with no conflicts

 #3a. cp /home/uvadm/dat1/testdropbl dat1/testdropbl0
      ===============================================
      - could copy as above, but better to use $UV as below
        (in case Vancouver Utilities not installed in usual /home/uvadm)

 #3a. cp $UV/dat1/testdropbl dat1/testdropbl0  # to test dropbl0
 #3b. cp $UV/dat1/testdropbl dat1/testdropbl1  # to test dropbl1
 #3c. cp $UV/dat1/testdropbl dat1/testdropbl2  # to test dropbl2
 #3d. cp $UV/dat1/testdropbl dat1/testdropbl3  # to test dropbl3

 #4a. dropbl0 dat1/testdropbl0  <-- drop all blank lines (sed)
      ========================
 #4b. dropbl1 dat1/testdropbl1  <-- drop all blank lines (uvcp option b2)
      ========================
 #4c. dropbl2 dat1/testdropbl2  <-- reduce multi-blank lines to 1 uvcp option b1
      ========================
 #4d. dropbl3 dat1/testdropbl3 c1 <-- drop COBOL blank '*' comments (uvcopy)
      ===========================     as well as blank lines

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

dat1/testdropbl demo INPUT file

 # testdropbl - test scripts to drop blank lines
 #            - see scripts dropbl0, dropbl1, dropbl2, dropbl3

# dropbl0 - uses 'sed' to drop all blank lines

# dropbl1 - uses 'uvcp' option 'b2' to drop all blank lines

# dropbl2 - uses 'uvcp' option 'b1' to reduce multi blank lines to 1

# dropbl3 - uses 'uvcopy' to drop all blank lines & all '*' COBOL comments

# begin test drop COBOL '*' comment blank lines (uvcopy) *

OUTPUT file tmp/testdropbl0 & 1 (same)

 # testdropbl - test scripts to drop blank lines
 #            - see scripts dropbl0, dropbl1, dropbl2, dropbl3
 # dropbl0 - uses 'sed' to drop all blank lines
 # dropbl1 - uses 'uvcp' option 'b2' to drop all blank lines
 # dropbl2 - uses 'uvcp' option 'b1' to reduce multi blank lines to 1
 # dropbl3 - uses 'uvcopy' to drop all blank lines & all '*' COBOL comments
 # begin test drop COBOL '*' comment blank lines (uvcopy)
       *

OUTPUT file tmp/testdropbl2

 # testdropbl - test scripts to drop blank lines
 #            - see scripts dropbl0, dropbl1, dropbl2, dropbl3

# dropbl0 - uses 'sed' to drop all blank lines

# dropbl1 - uses 'uvcp' option 'b2' to drop all blank lines

# dropbl2 - uses 'uvcp' option 'b1' to reduce multi blank lines to 1

# dropbl3 - uses 'uvcopy' to drop all blank lines & all '*' COBOL comments

# begin test drop COBOL '*' comment blank lines (uvcopy) *

OUTPUT file tmp/testdropbl3

 # testdropbl - test scripts to drop blank lines
 #            - see scripts dropbl0, dropbl1, dropbl2, dropbl3
 # dropbl0 - uses 'sed' to drop all blank lines
 # dropbl1 - uses 'uvcp' option 'b2' to drop all blank lines
 # dropbl2 - uses 'uvcp' option 'b1' to reduce multi blank lines to 1
 # dropbl3 - uses 'uvcopy' to drop all blank lines & all '*' COBOL comments
 # begin test drop COBOL '*' comment blank lines (uvcopy)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6I1. scripts to manipulate text files

printable1 - ensure printable characters

 #!/bin/ksh
 # printable1 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # printable1 - script to ensure printable characters in a text file
 #            - copies to tmp/... converting any unprintables to '.' periods
 #
 # Also see uvcopy jobs printable1
 #            - same name, but in $UV/pf/util/... vs $UV/sf/util/...
 # printable1 - same function using a uvcopy job (vs shell script 'tr')
 # printable2 - alternate for fixed record length files w/o LineFeeds
 #            - not possible to process fixed length records with tr script
 #
 echo "printable1 - ensure printable characters in a text file"
 echo "           - copies to tmp/... converting any unprintables to '.' periods"
 echo "printable2 - alternate for fixed record length files w/o LineFeeds"
 df="$1"
 if [[ -f "$df" && -d tmp ]]; then :
    else echo "usage: printable1 filename"
         echo "       ==================="
         echo "example: printable1 dat1/citytax1a"
         echo "         ========================="
         echo " - arg1 must be a file"
         echo " - must be a tmp/ subdir in current working dir"
         exit 1; fi
 #
 f=$(basename $df)
 tr -c '[:print:]\012' '.' <$df >tmp/$f
 #=====================================
 echo "$df copied to tmp/... converting unprintables to '.' periods"
 echo "more tmp/$f  <-- enter to display file with more"; read reply
 more tmp/$f
 #==========
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6J1. scripts to manipulate text files

parmnullfix1 - copy all files, nulls to blanks, using 'sed'

 #!/bin/ksh
 # parmnullfix1 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # parmnullfix1 - copy all files from 1 directory to a 2nd directory
 #              - converting any nulls to blanks, using sed
 # parmnullfix2 - alternate script calling 'uvcp' (vs sed)
 #              - also clears any sequence#s in cols 73-80
 d1="$1"; d2="$2";
 if [[ -d "$d1" && -d "$d2" ]]; then :
    else echo "usage: parmnullfix1 dir1 dir2"
         echo "       ======================"
         echo "- arg1 & arg2 must be input & output directories"
    exit 9; fi
 if [[ ! -d tmp ]]; then mkdir tmp; fi
 ls $d2 >tmp/$d2.emptytest
 if [[ -s tmp/$d2.emptytest ]]; then
    echo "the output directory must be empty $d2";exit 93; fi
 integer fcount=0 lcount=0 tlcount=0 nlcount=0 nfcount=0
 for d1f in $d1/*
 do
    f=$(basename $d1f)
    ((fcount+=1)); nlcount=0;
    nlcount=$(grep -Pac '\x00' $d1/$f)
    if ((nlcount)); then ((nfcount+=1)); fi
    wcount=$(wc -l $d1/$f 2>/dev/null); lcount=${wcount% *}; ((tlcount+=lcount));
    echo "file#=$fcount lines=$lcount nulls=$nlcount file=$d1/$f"
    sed 's/\x00/ /g' $d1/$f >$d2/$f
    #==============================
 done
 echo "total $fcount files, $nfcount with nulls converted to blanks"
 echo "total $fcount files, $tlcount lines, copied from $d1 to $d2"
 echo "- also see parmnullfix2 also clears 73-80 (using uvcp vs sed)"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6J2. scripts to manipulate text files

parmnullfix2 - nulls to blanks & clear 73-80, using 'uvcp'

 #!/bin/ksh
 # parmnullfix2 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # parmnullfix2 - copy all files from 1 directory to a 2nd directory
 #              - using 'uvcp' to convert nulls to blanks & clear cols 73-80
 # parmnullfix1 - alternate script using 'sed' (does not clear 73-80)
 d1="$1"; d2="$2";
 if [[ -d "$d1" && -d "$d2" ]]; then :
    else echo "usage: parmnullfix2 dir1 dir2"
         echo "       ======================"
         echo "- arg1 & arg2 must be input & output directories"
    exit 9; fi
 if [[ ! -d tmp ]]; then mkdir tmp; fi
 ls $d2 >tmp/$d2.emptytest
 if [[ -s tmp/$d2.emptytest ]]; then
    echo "the output directory must be empty $d2";exit 93; fi
 integer fcount=0 lcount=0 tlcount=0 nlcount=0 nfcount=0
 for d1f in $d1/*
 do
    f=$(basename $d1f)
    ((fcount+=1)); ncount=0;
    nlcount=$(grep -Pac '\x00' $d1/$f)
    if ((nlcount)); then ((nfcount+=1)); fi
    wcount=$(wc -l $d1/$f 2>/dev/null); lcount=${wcount% *}; ((tlcount+=lcount));
    if ((ncount)); then ((nfcount+=1)); fi
    echo "file#=$fcount nulls=$ncount file=$d1/$f"
    uvcp "fili1=$d1/$f,rcs=256,typ=LSTt,filo1=$d2/$f,clr=72(28),rop=i15"
    #==================================================================
 done
 echo "total $fcount files, $nfcount with nulls converted to blanks"
 echo "total $fcount files, $tlcount lines, copied from $d1 to $d2"
 echo "- using uvcp to clear cols 73-80 (vs parmnullfix1/sed blank nulls only)"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_7 Vancouver Utility scripts for Conversion & General use

copying & sorting files from 1 dir to a 2nd dir


7A1. copycpio1 - copy all files & subdirs (recursively) from dir to dir
- foolproof, ensures you are in input dir & outdir empty

7B1. copycpio2 - copy all files & subdirs (recursively) from dir to dir
- foolproof, ensures you are in output dir & it is empty

7C1. copydate - copy a file to a directory, appending a date-stamp on the file

7D1. cpnew1 - copy files from 1 directory to a 2nd directory
that are newer than a specified number of days

7D2. cpnew2 - copy files from 1 directory to a 2nd directory whose last
modification times are within a specified date range
specify dates as for the 'touch' command--> YYMMDDhhmm

7E1. sortcpio - copy all files & subdirs (recursively) from dir to dir
- sorting on file & directory names
- so filenames will be alphabetic on all utilities
  (not just for 'ls' commands which sort filenames internally)

7F1. sortall - sort all text files in a directory outputting to a 2nd directory

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7A1. copying & sorting files from 1 dir to a 2nd dir

copycpio1 - copy subdirs/files from indir to outdir

 #!/bin/ksh
 # copycpio1 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # copycpio1 - copy a directory tree to a 2nd directory
 #           - input may have sub-directories to any level
 #           - output will have exactly the same tree structure
 #           - you must be in input directory
 #           - output directory must be empty
 #           - you must specify full pathname of indir & outdir
 #           - fail-safe, paranoid script
 #           - also see copycpio2, similar to this *copycpio1, except:
 #*copycpio1 - requires you to be in input directory
 # copycpio2 - requires you to be in output directory
 #
 # Example - copy /p2/proddata/... to /p3/backup/proddata
 # 1. cd /p2/proddata                  <-- change to input directory
 # 2. ls -l /p3/backup/proddata | more <-- ensure outdir OK to erase
 # 3. rm -rf /pr/backup/proddata/*     <-- clear output directory
 # 4. copycpio1 /p2/proddata /p3/backup/proddata <-- do it
 #    ==========================================
 #
 d1="$1"; d2="$2";
 if [[ -d "$d1" && -d "$d2" ]]; then :
    else echo "usage: copycpio1 indir outdir"
         echo "       ======================"
         echo " - arg1 & arg2 must be directories (input & output)"
         exit 91; fi
 #
 dpath=$(pwd)                      # capture current directory path
 if [[ $dpath != $d1 ]]
    then echo "you must be in the input directory $d1";
          exit 92; fi
 #
 ls $2 >/tmp/copycpio_emptytest
 if [[ -s /tmp/copycpio_emptytest ]]
    then echo "the output directory must be empty $d2";exit 93; fi
 #
 echo "copy $d1 to $d2 OK ? (or kill)"; read reply
 cd $d1                          # <-- change to indir
 find . -print | cpio -pdmv $d2  # <-- copy all files to outdir
 #=============================
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7B1. copying & sorting files from 1 dir to a 2nd dir

copycpio2 - copy subdirs/files to outdir from indir

 #!/bin/ksh
 # copycpio2 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # copycpio2 - copy a directory tree to a 2nd directory
 #           - input may have sub-directories to any level
 #           - output will have exactly the same tree structure
 #           - you must be in output directory & it must be empty
 #           - you must specify full pathname of outdir & it must match .
 #           - fail-safe, paranoid script
 #           - also see copycpio1, similar to this *copycpio2, except:
 # copycpio1 - requires you to be in input directory
 #*copycpio2 - requires you to be in output directory
 #
 # Example - copy /p2/proddata/... to /p3/backup/proddata
 # 1. cd /p3/backup/proddata         <-- change to output directory
 # 2. ls -l | more                   <-- ensure outdir OK to erase
 # 3. rm -rf *                       <-- clear output directory
 # 3a. rm -rf /p3/backup/proddata/*  <-- safer (avoid unqualified '*')
 # 4. copycpio2 /p2/proddata /p3/backup/proddata <-- do it
 #    ==========================================
 #
 d1="$1"; d2="$2";
 if [[ -d "$d1" && -d "$d2" ]]; then :
    else echo "usage: copycpio2 indir outdir"
         echo "       ======================"
         echo " - arg1 & arg2 must be directories (input & output)"
         exit 91; fi
 #
 dpath=$(pwd)                      # capture current directory path
 if [[ $dpath != $d2 ]]
    then echo "you must be in output directory (& it must be empty) $d2";
          exit 92; fi
 #
 ls . >/tmp/copycpio_emptytest
 if [[ -s /tmp/copycpio_emptytest ]]
    then echo "the current directory (output) must be empty $d2";exit 93; fi
 #
 echo "copy $d1 to current dir $d2 OK ? (or kill job)"; read reply
 cd $d1                          # <-- change to indir
 find . -print | cpio -pdmv $d2  # <-- copy all files to outdir
 #=============================
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7C1. copying & sorting files from 1 dir to a 2nd dir

copydate - copy file to a directory, appending date-stamp

 # copydate - copy a file, appending a date stamp
 #          - by Owen Townsend, UV Software, Jan30/06
 # copydatetime - alternative script if you want time as well as date
 #
 #usage: copydate inputfile outputdirectory
 #       ==================================
 #       - arg1 must be a file, arg2 must be a directory
 #
 #example: copydate data/test1 tmp
 #         =======================
 #         - output would be: tmp/test1_yymmdd
 #
 if [[ -f "$1" && -d "$2" ]]; then :
 else echo "usage: copydate inputfile outputdirectory"
      echo "       =================================="
      echo "       - arg1 must be a file, arg2 must be a directory"
      exit 1; fi
 #
 fil="$1"; dir="$2";          # capture input-file-name & output-directory-name
 fbn=$(basename $fil)         # extract file basename, dropping any directories
 fbndt=${fbn}_$(date +%y%m%d) # append date stamp
 #
 cp $fil $dir/$fbndt          # copy inputfile to outdir appending date stamp
 #===================
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7D1. copying & sorting files from 1 dir to a 2nd dir

cpnew1 - copy files to a 2nd subdir newer than x days

 #!/bin/ksh
 # cpnew1 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # cpnew1 - copy files from 1 directory to a 2nd directory
 #          that are newer than a specified number of days
 #
 #usage:  cpnew1 indir outdir days
 #        ========================
 #
 #note - this script illustrates reading text lines from a file
 #       (a powerful feature of the KORN shell)
 #       'exec 3<' to open, 'read -u3 field' to read, 'exec 3<&-' to close
 #
 if [ -d "$1" -a -d "$2" -a -n "$3" ]; then :
 else echo "USAGE: cpnew1 indir outdir days"; exit 1; fi
 #
 # create tmp dir in current dir if not already present
 if [[ ! -d tmp ]]; then mkdir tmp; fi
 #
 # create list of filenames newer than specifed no of days
 find $1/* -mtime -$3 -print >tmp/cpnew_files
 #===========================================
 #
 x=0;
 exec 3< tmp/cpnew_files    # open namesfile (using file descriptor #3)
 while read -u3 fp
 do
   let x=x+1
   fn=${fp##*/}                   # strip off left path to filename only
   echo "copy file# $x - $1/$fn "
   cp -p $1/$fn $2/$fn
   #==================
 done
 exec 3<&-           # close file descriptor #3
 echo "$x files (newer than $3 days) copied from $1 to $2"
 exit

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7D2. copying & sorting files from 1 dir to a 2nd dir

cpnew2 - copy files within a modification date range

 #!/bin/ksh
 # cpnew2 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # cpnew2 - copy files from 1 directory to a 2nd directory whose last
 #          modification times are within a specified date range
 #          specify dates as for the 'touch' command--> YYMMDDhhmm
 #
 #usage:  cpnew2 indir outdir YYMMDDhhmm1 YYMMDDhhmm2
 #        ===========================================
 #
 #example: cpnew2 dir1 dir2 9804010000 9804302359   copy April 98 files
 #         ======================================
 #
 if [[ (-d "$1") && (-d "$2") && (-n "$3") && (-n "$4") ]]; then :
 else echo "USAGE: cpnew2 indir outdir YYMMDDhhmm1 YYMMDDhhmm2 "; exit 1; fi
 #
 if (( (${#3} == 10) && (${#4} == 10) )); then :
 else echo "arg3 & arg4 must be 10 digits (YYMMDDhhmm)"; exit 2; fi
 #
 # create tmp dir in current dir if not already present
 if [[ ! -d tmp ]]; then mkdir tmp; fi
 #
 # create 2 files & use touch to set dates to the specified dates
 >tmp/cpnew2date1; >tmp/cpnew2date2
 touch -am -t $3 tmp/cpnew2date1
 touch -am -t $4 tmp/cpnew2date2
 #
 x=0; y=0;
 for f in $1/*
 do
   let x=x+1
   if [[ (-f $f) && ($f -nt tmp/cpnew2date1) && ($f -ot tmp/cpnew2date2) ]]
   then let y=y+1
        fn=${f##*/}                     # strip left to filename only
        echo "copy file# $x - $1/$fn "
        cp -p $1/$fn $2/$fn
        #==================
   fi
 done
 echo "$y files of $x total (dated $3 to $4) copied from $1 to $2"
 exit

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7E1. copying & sorting files from 1 dir to a 2nd dir

sortcpio - copy/sort all subdirs/files from dir1 to dir2

 #!/bin/ksh
 # sortcpio - Korn shell script from UVSI stored in: /home/uvadm/sf/adm/
 # sortcpio  - sort filenames & copy a directory to a 2nd directory
 #           - so filenames will be in sequence for backups, etc
 #             (ls sorts filenames, but not tar, cpio, du, etc)
 #           - input may have sub-directories to any level
 #           - output will have exactly the same tree structure
 #             except directory names & filenames will be sorted
 #
 if [[ -d $1 && -d $2 ]]; then :
    else echo "usage: sortcpio indir outdir"
         echo "       ====================="
         echo "- you must be outside of dirs & outdir must be empty"
         exit 1; fi
 #
 echo "current directory must be outside of indir & have tmp subdir - OK ?";read
 echo "outdir can be anywhere outside of indir & must be empty - OK ?";read
 #
 cwd=$(pwd)
 cd $1; d1=$(pwd); cd $cwd
 cd $2; d2=$(pwd); cd $cwd
 if [[ ! -d tmp ]];then mkdir tmp; fi
 cd $d1
 #
 find . -print >$cwd/tmp/files1
 #=============================
 sort -o$cwd/tmp/files2 $cwd/tmp/files1
 #=====================================
 cat $cwd/tmp/files2 | cpio -pdmv $d2
 #===================================
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7F1. copying & sorting files from 1 dir to a 2nd dir

sortall - sort all text files from dir1 to dir2

 #!/bin/ksh
 # sortall - sort all text files from 1 subdir to a 2nd subdir
 #         - maintaining same filenames in output directory
 #         - using unix/linux 'sort' utility vs 'uvsort' (sortallu1/u2)
 #
 if [[ -d "$1" && -d "$2" ]]; then :
    else echo "arg1 & arg2 must be subdirs"
         echo "usage: sortall indir outdir"
         echo "       ===================="
    exit 1; fi
 #
 d1=$1; d2=$2
 x=0
 for df in $d1/*
   { f=${df##*/}
     sort -o $d2/$f $d1/$f
     let x=x+1
   }
 echo "$x files sorted from $d1 to $d2"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_8 Vancouver Utility scripts for Conversion & General use

system administration scripts


8A1. Notes about scripts requiring 'root' permissions

8B1. killuser1 - kill user by username (vs process-ID)
- saves having to search for process-ID via ps

8B2. killuser2 - kill all users who forgot to logoff
- run by cron at 3AM before processing console logs

8B3. crontab_root - used to schedule 'killuser2' at 3 AM

8C1. chmod1 - change permissions on directories & files
- using 'find' to process all levels of directories & files
- need to manually add 'x' perms on any bin/* & script/* dirs

8C2. chmod2 - change permissions on directories & files
- using 'recursion' to process all levels of subdirs
- need to manually add 'x' perms on any bin/* & script/* dirs

8C3. chmod3 - change permissions on directories, files,& executable files
- using 'recursion' to process all levels of subdirs
- adds 'x' perms on any bin/* & script/* dirs, by testing
  for known names of bin & script subdirs

8D1. ftrigger51 - script to test for a file presence & run a 2nd script
- use to run a JCL/script when a file is FTP'd to server
- simple version with hard-coded filename & jobname

8D2. ftrigger52 - general purpose script test file presence & run 2nd script
- accepts arguments for filename & script to be run

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

8A1. Notes about system/application administration scripts

  1. These scripts are also documented in ADMjobs.htm, along with other system administration scripts such as backup scripts & console log processing scripts.

  2. We should use 'root' only when absolutely necessary (as in killuser).

  3. The backup scripts (& most scripts ADMjobs.doc) do not need root, because they backup the application data & libraries (not the unix/linux system).

  4. The backup scripts are run by cron, but the crontab is not owned by root. The nightly 'crontabs' are owned by 'appsadm' which is in the same group as the programmers & operators, and has access to the data & libraries.

  5. Note that scripts run by 'cron' do not get the owner's profile executed automatically, so these scripts 1st '.' dot execute the profile (/home/appsadm/env/stub_profile). Then these scripts have all the usual environmental variables & PATHs setup to run the same as for console operation.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

8B1. system/application administration scripts

killuser1 - kill user by username (vs process#)

 #!/bin/ksh
 # killuser1 - Korn shell script from UVSI stored in: /home/uvadm/sf/adm/
 # killuser1 - kill any 1 user (must be run by root)
 #           - by Owen Townsend, www.uvsoftware.ca May 2002
 #           - also see killuser2 to kill all users running bash
 #             (killuser2 intended for cron at 3AM or whatever)
 #
 # verify arg1 specified
 user="$1"
 if [ -z "$user" ]; then
    echo "arg1 userid must be specifeid"
    echo "usage: killuser1 userid"
    echo "       ================"
    exit 1; fi
 #
 # redirect ps output to a tmp file
 # - use '-o' output option for 3 fields only (COMMAND, PID,& RUSER)
 ps -u $user -ocomm -opid -oruser >/tmp/kill$user
 #===============================================
 #    ** sample output lines **
 # COMMAND            PID RUSER
 # init                 1 root
 # bash              9022 root
 # bash              9077 uvadm
 # ps                9226 uvadm
 #
 # - open the file & read back into variables for easier manipulation
 exec 3< /tmp/kill$user        # open file #3
 #
 x=0; y=0
 while read -u3 comm pid ruser
   do if [[ ("$ruser" = "$user") && ("$comm" = "bash") ]]
         then let x=x+1
              kill -9 $pid
              #===========
              if [[ $? = 0 ]]; then kok=OK; let y=y+1; else kok=NAK; fi
              echo "#$x kill $comm $pid $ruser - $kok"
      fi
   done
 exec 3<&-                    # close file #3
 echo "$x kills attempted, $y killed OK $(date)"

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

8B2. system/application administration scripts

killuser2 - kill all users who forgot to logoff

 #!/bin/ksh
 # killuser2 - Korn shell script from UVSI stored in: /home/uvadm/sf/adm/
 # killuser2 - kill users who did not logout
 #           - by Owen Townsend, www.uvsoftware.ca, March 2002
 #
 # - intended to be run by crontab_root at 11:45 PM or whenever
 # - killuser2 is run before logfixN which processes user console logs
 # - this kills user 'script' & closes the script output file
 # - also see 'killuser1' script for interactive use to kill any 1 user
 #
 if [ "$1" != "all" ]; then
    echo "usage: killuser2 all"
    echo "       ============="
    echo " - arg1 must be 'all'"
    exit 1; fi
 #
 # redirect ps -f output to a tmp file
 # - use '-o' output option for 3 fields only (COMMAND, PID,& RUSER)
 ps -e -ocomm -opid -oruser >/tmp/psef
 #====================================
 #    ** sample output lines **
 # COMMAND            PID RUSER
 # init                 1 root
 # bash              9022 root
 # bash              9077 uvadm
 # ps                9226 uvadm
 #
 # - open the file & read back into variables for easier manipulation
 exec 3< /tmp/psef            # open file #3
 #
 x=0; y=0
 while read -u3 comm pid ruser
 do if [[ ("$comm" = ksh || "$comm" = bash) && ("$ruser" != root) ]]
       then let x=x+1
            kill -9 $pid
            #===========
            if [[ $? = 0 ]]; then kok=OK; let y=y+1; else kok=NAK; fi
            echo "#$x kill $comm $pid $ruser - $kok"
    fi
 done
 exec 3<&-                    # close file #3
 echo "$x kills attempted, $y killed OK $(date)"

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

8B3. system/application administration scripts

crontab_root - to schedule 'killuser2' at 3 AM

 # crontab_root - crontab file to run under root
 #              - by Owen Townsend, UV Software, May 2008
 # - supplied in /home/uvadm/sf/adm/crontab_root
 # - copy to /home/appsadm/sf/crontab_root before customization & activation
 # - see details at www.uvsoftware.ca/admjobs.htm#Part_5
 #
 # Only for backups, cleanups, etc that require 'root'
 # - see crontab_apps1 for backups & applications (daily,weekly,monthly,etc)
 # - 'crontab_apps1' runs under 'appsadm' (not root) & much safer
 #
 # Actual crontab for root depends on the unix/linux OS
 # - you need to find it & add desired lines
 # OR, you could use the following procedure so you could maintain
 # all crontabs in 1 place /home/appsadm/sf/...
 # - after you have retrieved & combined root crontabs with this file
 # - for RHEL 5.1 there is no root crontab, so just copy this
 #
 # suggested procedures for updating root crontab:
 # 1. logon as root & cd to /home/appsadm
 # 2. crontab -l (if 1st setup)   - list to see if any crontab exists for root
 # 3. crontab -l >sf/crontab_root - ifso, redirect to sf/appsadm
 # 4. vi sf/crontab_root          - edit this file as required
 # 4a. :r /home/uvadm/sf/adm/crontab_root   <-- append this supplied file
 # 5. crontab -r                  - remove old crontab file for root
 # 6. crontab sf/crontab_root     - activate new crontab for root
 # 7. crontab -l                  - list crontab file to confirm installation
 #
 # minute hour day-of-mth mth-of-yr day-of-week <----command---->
 #
 15 00 * * * /home/appsadm/sf/killuser2 all  # kill users at 12:15AM every night
 #=========================================
 # - killuser2 kills all 'ksh' & 'bash' users who did not log off
 # - also see killuser1 to interactively kill any 1 specified userid
 #
 30 00 * * 2-6 /home/appsadm/sf/setperms1 all # set perms at 12:30AM Tues-Sat
 #===========================================
 # At 12:30 AM Tues-Sat, run setperms1 to set permissions on data & libraries
 # - ensure directories 775, data-files 664, script-files 775
 # - ensure owner:group appsadm:apps (see details in setperms1 script)
 #
 00 01 * * 0 /sbin/shutdown -g0 -y -i6  # reboot at 01:00AM Sunday
 #====================================    (above command for traditional unix)
 # reboot Sunday 1 AM -g0(no wait) -y(auto reply y to prompt) -i6(reboot)
 # 00 01 * * 0 /sbin/shutdown -r now    #<-- can use this for Linux
 # =================================
 #
 #Note: Add commands here to bring up software packages, such as:
 # - Micro Focus COBOL license manager
 # - Online systems (MTO, unikix, CICS6000, etc)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

8C1. system/application administration scripts

chmod1 - set perms on all levels - using 'find'

 #!/bin/ksh
 # chmod1 - change permissions on subdirs & files under a specified superdir
 #        - using 'find' to process all levels of directories & files
 #        - need to manually add 'x' perms on any bin/* & script/* dirs
 #        - by Owen Townsend, April 26/2006
 #
 # chmod1 directory dir-perms file-perms   <-- command format
 # ======================================
 # chmod1 directory 775 664                <-- recommended permissions
 # =========================
 #
 # - also see alternatives chmod2 & chmod3 (to this *chmod1)
 #*chmod1 - change perms on dirs & files, using 'find'
 # chmod2 - change perms on dirs & files, using 'recursion'
 # chmod3 - change perms on dirs & files, using 'recursion'
 #        - sets 'x' perm by testing for known names of bin/ & script/ subdirs
 #
 # After running chmod1 or chmod2, you must manually fix permissions on
 # executable programs & scripts via -->chmod 775 bin/*; chmod 775 scripts/*
 #
 # capture arguments & force perms integers
 dir="$1";
 typeset i dperm="$2"; typeset i fperm="$3";
 #
 # ensure arg1 is directory & length of perms are 3 digits
 dpl=${#dperm}; fpl=${#fperm};
 #
 if [[ -d "$dir" ]] && ((dpl==3 && fpl==3)); then :
    else echo "usage: chmod1 directory dir-perms file-perms"
         echo "       ====================================="
         echo "example: chmod1 dirxx 775 664"
         echo "         ===================="
         echo " - arg1 must be dir, args 2 & 3 must be 3 digits"
         exit 90; fi
 #
 echo -n "chmod1: set perms $dperm/$fperm from: $dir - enter to continue"
 read reply
 #
 find $dir -type d -exec chmod $dperm {} \;
 #=========================================
 #
 find $dir -type f -exec chmod $fperm {} \;
 #=========================================
 #
 echo "chmod1: perms set $dperm/$fperm for all subdirs & files within: $dir"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

8C2. system/application administration scripts

chmod2 - set perms on all levels - using 'recursion'

 #!/bin/ksh
 # chmod2 - change permissions on subdirs & files under a specified superdir
 #         - using 'recursion' to process all levels of subdirs
 #         - need to manually add 'x' perms on any bin/* & script/* dirs
 #
 # chmod2 directory dir-perms file-perms    <-- command format
 # =====================================
 # chmod2 directory 775 664     <-- recommended permissions
 # ========================
 # - also see alternatives chmod1 & chmod3 (to this *chmod2)
 # chmod1 - change perms on dirs & files, using 'find'
 #*chmod2 - change perms on dirs & files, using 'recursion'
 # chmod3 - change perms on dirs & files, using 'recursion'
 #        - sets 'x' perm by testing for known names of bin/ & script/ subdirs
 #
 # After running chmod2 or chmod3, you must manually fix permissions on
 # executable programs & scripts via -->chmod 775 bin/*; chmod 775 scripts/*
 #
 dir="$1";
 typeset i dperm="$2"; typeset i fperm="$3";
 # ensure arg1 is directory & length of perms are 3 digits
 dpl=${#dperm}; fpl=${#fperm};
 if [[ -d "$dir" ]] && ((dpl==3 && fpl==3)); then :
    else echo "usage: chmod2 directory dir-perms file-perms"
         echo "       ====================================="
         echo "example: chmod2 dirxx 775 664"
         echo "         ===================="
         echo " - arg1 must be dir, args 2,3 must be 3 digits"
         exit 90; fi
 echo -n "chmod2: Begin directory $dir - enter to continue"
 read reply
 #note - remove the pause (read reply) above if desired
 integer nd=0 nf=0
 for df in $dir/*
   { if [[ -d $df ]]
        then chmod $dperm $df; ((nd+=1))
        else chmod $fperm $df; ((nf+=1))
     fi
     # if current entry in current dir is a subdir
     # - call this script again to process its files & subdirs
     # - using 'recursion' to process all existing levels of subdirs
     # - but do not call if directory is empty
     if [[ -d $df ]]; then
        ls $df >/tmp/chmod2
        if [[ -s /tmp/chmod2 ]]; then
           chmod2 $df $dperm $fperm
           #=======================
        fi
     fi
   }
 echo "chmod2: End dir $dir, subdirs=$nd, files=$nf"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

8C3. system/application administration scripts

chmod3 - set perms using recursion & add x to bin/script

 #!/bin/ksh
 # chmod3 - change permissions on subdirs & files under a specified superdir
 #        - using 'recursion' to process all levels of subdirs
 #        - adds 'x' perms on any bin/* & script/* dirs, by testing
 #          for known names of bin & script subdirs
 #
 # chmod3 directory dir-perms file-perms xfile-perms   <-- command format
 # ==================================================
 # chmod3 directory 775 664 775    <-- recommended permissions
 # =============================
 # - also see alternatives chmod1 & chmod2 (to this *chmod3)
 # chmod1 - change perms on dirs & files, using 'find'
 # chmod2 - change perms on dirs & files, using 'recursion'
 #*chmod3 - change perms on dirs & files, using 'recursion'
 #        - sets 'x' perm by testing for known names of bin/ & script/ subdirs
 # After running this script, you must manually fix permissions on executable
 # programs & scripts via -->chmod 775 bin/*; chmod 775 scripts/*
 # - if you can't modify the script to recognize all your bin & script dirs
 dir="$1";
 typeset i dperm="$2"; typeset i fperm="$3"; typeset i xperm="$4";
 # ensure arg1 is directory & length of perms are 3 digits
 dpl=${#dperm}; fpl=${#fperm}; xpl=${#xperm};
 if [[ -d "$dir" ]] && ((dpl==3 && fpl==3 && xpl==3)); then :
    else echo "usage: chmod3 directory dir-perms file-perms xfile-perms"
         echo "       ================================================="
         echo "example: chmod3 dirxx 775 664 775"
         echo "         ========================"
         echo " - arg1 must be dir, args 2,3,4 must be 3 digits"
         exit 90; fi
 echo -n "chmod3: Begin directory $dir - enter to continue "
 read reply       #note - remove the pause (read reply) if desired
 integer nd=0 nf=0 nxf=0
 for df in $dir/*
   { if [[ -d $df ]] then chmod $dperm $df; ((nd+=1))
        elif [[ $df == *script* || $df == *sf* || $df == *bin*
                                 || $df == *program* || $df == jcls ]]
             then chmod $xperm $df; ((nxf+=1))
        else chmod $fperm $df; ((nf+=1))
     fi
     # if current entry in current dir is a subdir
     # - call this script again to process its files & subdirs
     # - using 'recursion' to process all existing levels of subdirs
     # - but do not call if directory is empty
     if [[ -d $df ]]; then
        ls $df >/tmp/chmod3
        if [[ -s /tmp/chmod3 ]]; then
           chmod3 $df $dperm $fperm $xperm
           #==============================
        fi
     fi
   }
 echo "chmod3: End dir $dir, subdirs=$nd, files=$nf, xfiles=$nxf"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

8D1. system/application administration scripts

ftrigger51 - test file presence & run a script

 # ftrigger51 - script to test for a file presence & run a 2nd script
 #            - could use to run a JCL/script when a file is FTP'd to server
 #            - by Owen Townsend, March 30/2005
 #            - 2 versions of this script (ftrigger51 & ftrigger52)
 #*ftrigger51 - simple version with hard-coded filename & jobname
 #            - need to copy/rename/modify for other filenames & jobnames
 # ftrigger52 - general purpose version with 2 args for filename & jobname
 #
 jobset51;  # call setup function common to scripts converted from mainframe JCL
 #            - remove 'jobset51' for scripts unrelated to mainframe conversions
 # 'jobset51' - a Vancouver Utility function to setup the job environment
 #            - changes directory to $RUNDATA (with subdirs ftp, etc)
 #            - see MVSJCL.htm#1C3 explanation & listing at MVSJCL.htm#3C1
 # assign variables for hard-coded file/job names up front in 1 place only
 # - to make script easier to copy/rename/modify for other filenames/jobnames
 file1="ftp/customer.master"     # hard-code filename here (1 place only)
 script1="jar160.ksh"            # hard-code jobname here (1 place only)
 logmsg "$0 waiting for $file1 to run $script1"
 # sleep 2 seconds repeatedly, until the arg1 file is detected
 until [[ -f $file1 ]]
    do sleep 2
    done
 #
 # file presence detected - run arg2 JCL/script & rename the file
 file2=${file1}_$(date +%y%m%d:%H%M%S)  # add _date:time stamp on filename
 $script1        # run the JCL/script with no arguments (vs ftrigger52)
 #=======
 cc=$?           # capture return status
 if (($cc != 0))
    then logmsg "$script1 $file1 FAILED (return code $cc)"; exit $cc; fi
 mv $file1 $file2   # rename the input file (so we can rerun this script)
 #===============
 logmsg "$script1 ran OK & $file1 renamed to $file2"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

8D1. system/application administration scripts

ftrigger52 - G.P. test file presence & run a script

 # ftrigger52 - script to test for a file presence & run a 2nd script
 #            - could use to run a JCL/script when a file is FTP'd to server
 #            - by Owen Townsend, March 30/2005
 #            - 2 versions of this script (ftrigger51 & ftrigger52)
 # ftrigger51 - simple version with hard-coded filename & jobname
 #            - would need to copy/rename/modify for other filenames & jobnames
 #*ftrigger52 - general purpose version with 2 args for filename/jobname
 #
 jobset51;  # call setup function common to scripts converted from mainframe JCL
 #            - remove 'jobset51' for scripts unrelated to mainframe conversions
 # 'jobset51' - a Vancouver Utility function to setup the job environment
 #            - changes directory to $RUNDATA (with subdirs ftp, etc)
 #            - see MVSJCL.htm#1C3 explanation & listing at MVSJCL.htm#3C1
 #
 if [[ -n "$1" && -n "$2" ]]; then :
 else echo "usage: ftrigger52 filename JCL/script  <-- 2 arguments must be coded"
      echo "       ============================="
      echo "example: ftrigger52 ftp/customer.master jar160.ksh"
      echo "         ========================================"
      exit 1; fi
 #
 file1=$1; script1=$2;   # name the variables
 logmsg "$0 waiting for $file1 to run $script1"
 #
 # sleep 2 seconds repeatedly, until the arg1 file is detected
 until [[ -f $file1 ]]
    do sleep 2
    done
 #
 # file presence detected - run arg2 JCL/script & rename the file
 file2=${file1}_$(date +%y%m%d:%H%M%S)  # add _date:time stamp on filename
 #
 $script1 $file1    # run the JCL/script with filename as arg1
 #==============
 cc=$?              # capture return status
 if (($cc != 0))
    then logmsg "$script1 $file1 FAILED (return code $cc)"; exit $cc; fi
 #
 mv $file1 $file2   # rename the input file (so we can rerun this script)
 #
 logmsg "$script1 ran OK & $file1 renamed to $file2"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Permuted Index of Keywords from ** Headings **

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

6H3 *comment: dropbl3 - drop blank lines & *comment lines
6H4 *comment: dropbl3 - uvcopy job to drop blank lines & *comment lines

1D2 .alternate: rename.X - change any .extension to any .alternate
1D2 .extension: rename.X - change any .extension to any .alternate
1D4 .extension: rename-.X - remove any .extension
3B2 .files: lla - lists All files including hidden .files

0A2 Accessed: VU scripts - where stored, how accessed (PATH)
3G1 Across: spreadA - list filenames 4-up across the page
8C3 Ad: chmod3 - set perms using recursion & add x to bin/script
1C4 Add: rename+P - add any specified prefix
1D5 Add: rename+X - add any specified extension
4C1 All2lower: - translate to lower copying all files to 2nd dir
4C2 All2upper: - translate to UPPER copying all files to 2nd dir
4F1 Allcancel: - cancel all printer requests for current user
4B1 Allcmp: - compare all packed/binary files in 2 directories
4A1 Alldiff: - compare all text files in 2 directories
4A2 Alldiff2sub: - compare all files in all subdirs in 2 superdirs
4E1 Allrm: - remove all files in a directory
8B3 Am: crontab_root - to schedule 'killuser2' at 3 AM
1E1 Anywhere: rename-AA - remove Any pattern Anywhere in filename
7C1 Appending: copydate - copy file to a directory, appending date-stamp
6B1 Apply: runsed1 - apply sed script to all files in directory
0A3 Appsadm: uvadm, appsadm,& user homedirs
8B3 At: crontab_root - to schedule 'killuser2' at 3 AM

6D1 B: demo uvfix1,A,2,B
8C3 Bin: chmod3 - set perms using recursion & add x to bin/script
4B1 Binary: allcmp - compare all packed/binary files in 2 directories
6H0 Blank: dropbl0 - drop blank lines (using 'grep')
6H1 Blank: dropbl1 - drop blank lines (using 'uvcp')
6H2 Blank: dropbl2 - reduce multiple blank lines to 1
6H3 Blank: dropbl3 - drop blank lines & *comment lines
6H4 Blank: dropbl3 - uvcopy job to drop blank lines & *comment lines
1F1 Blanks: renameB2U - change any embedded blanks to underscores
6J1 Blanks: parmnullfix1 - copy all files, nulls to blanks, using 'sed'
6J2 Blanks: parmnullfix2 - nulls to blanks & clear 73-80, using 'uvcp'

3F2 Calls: script spreadA calls 'uvcopy job' spread1
4F1 Cancel: allcancel - cancel all printer requests for current user
6A1 Carriage: removeCR - remove Carriage Returns from text files
6A2 Carriage: insertCR - insert Carriage Returns in text files
1B1 Case: renameL - translate filenames to Lower case
1B2 Case: renameU - translate filenames to UPPER case
1B3 Case: renameLsd - Lower case all files in all subdirs of superdir
5A1 Cfdd: demo cfdd - count files in all sub-dirs of a super-dir
5D2 Cfdd: - Count Files in all sub-Dirs of a super-Dir
5D3 Cfdp: - Count Files in a Directory with a Pattern any line any file
5A1 Cfdt: demo cfdt - Count Files in 1 directory - Total only
5D1 Cfdt: - Count Files in Directory, Total only
1J1 Character: renameCC - convert any 1 character to a 2nd character
1J1 Character: renameCC - convert any 1 character to a 2nd character
6I1 Characters: printable1 - ensure printable characters
8C1 Chmod1: - set perms on all levels - using 'find'
8C2 Chmod2: - set perms on all levels - using 'recursion'
8C3 Chmod3: - set perms using recursion & add x to bin/script
6J2 Clear: parmnullfix2 - nulls to blanks & clear 73-80, using 'uvcp'
5A1 Clf: demo clf - Count Files & Lines in subdir
5B1 Clf: - Count Lines in a File
5F1 Clfd: - Count Files & Lines, optionally matching a pattern
5A1 Clfd2: 2nd demo clfd2 - Counts with optional pattern match
5F2 Clfd2: - Count Files, Lines,& KB, optionally matching a pattern
5A1 Clfdp: - 2nd demo illustrating 'not'
5A1 Clfdp: demo clfdp - Count Lines in all Files of a Directory with a Pattern
5E2 Clfdp: - Count Lines in all Files of a Directory with a Pattern
5E1 Clfp: - Count Lines in a File with a Pattern
0A2 Common_profile: relevant lines from common_profile
4A1 Compare: alldiff - compare all text files in 2 directories
4A2 Compare: alldiff2sub - compare all files in all subdirs in 2 superdirs
4B1 Compare: allcmp - compare all packed/binary files in 2 directories
4G1 Copy: scripts to list/copy/move/remove old/new files
6J1 Copy: parmnullfix1 - copy all files, nulls to blanks, using 'sed'
7A1 Copy: copycpio1 - copy subdirs/files from indir to outdir
7B1 Copy: copycpio2 - copy subdirs/files to outdir from indir
7C1 Copy: copydate - copy file to a directory, appending date-stamp
7D1 Copy: cpnew1 - copy files to a 2nd subdir newer than x days
7D2 Copy: cpnew2 - copy files within a modification date range
7E1 Copy: sortcpio - copy/sort all subdirs/files from dir1 to dir2
7A1 Copycpio1: - copy subdirs/files from indir to outdir
7B1 Copycpio2: - copy subdirs/files to outdir from indir
7C1 Copydate: - copy file to a directory, appending date-stamp
4C1 Copying: all2lower - translate to lower copying all files to 2nd dir
4C2 Copying: all2upper - translate to UPPER copying all files to 2nd dir
5A1 Count: demo cfdd - count files in all sub-dirs of a super-dir
5A1 Count: demo cfdt - Count Files in 1 directory - Total only
5A1 Count: demo clf - Count Files & Lines in subdir
5A1 Count: demo clfdp - Count Lines in all Files of a Directory with a Pattern
5A1 Count: demo wc - Word Count
5B1 Count: clf - Count Lines in a File
5D1 Count: cfdt - Count Files in Directory, Total only
5D2 Count: cfdd - Count Files in all sub-Dirs of a super-Dir
5D3 Count: cfdp - Count Files in a Directory with a Pattern any line any file
5E1 Count: clfp - Count Lines in a File with a Pattern
5E2 Count: clfdp - Count Lines in all Files of a Directory with a Pattern
5F1 Count: clfd - Count Files & Lines, optionally matching a pattern
5F2 Count: clfd2 - Count Files, Lines,& KB, optionally matching a pattern
5A1 Counting: v12 - counting Lines & Files in directories
5A1 Counting: v12 - help menu for counting scripts
3A1 Counts: llc - list directory with File & Line counts
5A1 Counts: 2nd demo clfd2 - Counts with optional pattern match
5A1 Counts: demo llc - list all files in subdir with line counts
5C1 Counts: llc - list filenames in directory with line counts
7D1 Cpnew1: - copy files to a 2nd subdir newer than x days
7D2 Cpnew2: - copy files within a modification date range
8B3 Crontab_root: - to schedule 'killuser2' at 3 AM
6A5 Crs: removeCRall - remove CRs from all files in directory
6A6 Crs: insertCRall - insert CRs into all files in directory
4F1 Current: allcancel - cancel all printer requests for current user

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

1A1 Data: example - renaming Mainframe Data Files
7D2 Date: cpnew2 - copy files within a modification date range
7C1 Date-stamp: copydate - copy file to a directory, appending date-stamp
4H1 Days: lsold1 - list files older than x days
4H1 Days: lsold2 - list files created/modified between 2 days
4H2 Days: lsnew1 - list files newer than x days
4H2 Days: lsnew2 - list files created/modified between 2 days
4H3 Days: rmold1 - remove files older than x days
7D1 Days: cpnew1 - copy files to a 2nd subdir newer than x days
0A3 Demo: setup to run uvcopy test/demo jobs
4G2 Demo: lsold1 & rmold1
5A1 Demo: 2nd demo clfd2 - Counts with optional pattern match
5A1 Demo: clfdp - 2nd demo illustrating 'not'
5A1 Demo: cfdd - count files in all sub-dirs of a super-dir
5A1 Demo: cfdt - Count Files in 1 directory - Total only
5A1 Demo: clf - Count Files & Lines in subdir
5A1 Demo: clfdp - Count Lines in all Files of a Directory with a Pattern
5A1 Demo: llc - list all files in subdir with line counts
5A1 Demo: wc - Word Count
6D1 Demo: uvfix1,A,2,B
6H5 Demo: dropbl0,1,2,3 demo
5A1 Demos: prep for demos using help files
6B2 Di: runsed2 - run sed script to process all files from 1 dir to 2nd dir
4C1 Dir: all2lower - translate to lower copying all files to 2nd dir
4C2 Dir: all2upper - translate to UPPER copying all files to 2nd dir
6B2 Dir: runsed2 - run sed script to process all files from 1 dir to 2nd dir
7E1 Dir1: sortcpio - copy/sort all subdirs/files from dir1 to dir2
7F1 Dir1: sortall - sort all text files from dir1 to dir2
7E1 Dir2: sortcpio - copy/sort all subdirs/files from dir1 to dir2
7F1 Dir2: sortall - sort all text files from dir1 to dir2
4I1 Director: editmfp1 - edit multi files in directory in Size order
3D1 Directories: lld - list directories only (omit files)
4A1 Directories: alldiff - compare all text files in 2 directories
4B1 Directories: allcmp - compare all packed/binary files in 2 directories
5A1 Directories: v12 - counting Lines & Files in directories
3A1 Directory: llc - list directory with File & Line counts
3F1 Directory: lslp - list filenames in directory & print immediately
4D1 Directory: rmzf - remove zero-length (empty) files from a directory
4E1 Directory: allrm - remove all files in a directory
5A1 Directory: demo cfdt - Count Files in 1 directory - Total only
5A1 Directory: demo clfdp - Count Lines in all Files of a Directory with a Pattern
5C1 Directory: llc - list filenames in directory with line counts
5D1 Directory: cfdt - Count Files in Directory, Total only
5D3 Directory: cfdp - Count Files in a Directory with a Pattern any line any file
5E2 Directory: clfdp - Count Lines in all Files of a Directory with a Pattern
6A5 Directory: removeCRall - remove CRs from all files in directory
6A6 Directory: insertCRall - insert CRs into all files in directory
6B1 Directory: runsed1 - apply sed script to all files in directory
7C1 Directory: copydate - copy file to a directory, appending date-stamp
1F2 Dollar: renameD2U - change any '$' dollar signs to '_' underscores
6H0 Drop: dropbl0 - drop blank lines (using 'grep')
6H1 Drop: dropbl1 - drop blank lines (using 'uvcp')
6H3 Drop: dropbl3 - drop blank lines & *comment lines
6H4 Drop: dropbl3 - uvcopy job to drop blank lines & *comment lines
6H0 Dropbl0: - drop blank lines (using 'grep')
6H5 Dropbl0: dropbl0,1,2,3 demo
6H1 Dropbl1: - drop blank lines (using 'uvcp')
6H2 Dropbl2: - reduce multiple blank lines to 1
6H3 Dropbl3: - drop blank lines & *comment lines
6H4 Dropbl3: - uvcopy job to drop blank lines & *comment lines

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

1A1 Easy: the EASY way to rename files
6C0 Easy: uvfix scripts - easy way to use uvcopy
4I1 Edit: editmfp1 - edit multi files in directory in Size order
4I1 Editmfp1: - edit multi files in directory in Size order
1F1 Embedded: renameB2U - change any embedded blanks to underscores
4D1 Empty: rmzf - remove zero-length (empty) files from a directory
6I1 Ensure: printable1 - ensure printable characters
1A1 Example: - renaming Mainframe Data Files
1B2 Except: renameU same as renameL except for
1D1 Extension: renameX - change any extension to any alternate
1D3 Extension: rename-X - remove any extension
1D5 Extension: rename+X - add any specified extension

6D2 Fil: uvfix1 ex#2 - convert Tab-delimited file to Fixed-format
1E1 Filename: rename-AA - remove Any pattern Anywhere in filename
1B1 Filenames: renameL - translate filenames to Lower case
1B2 Filenames: renameU - translate filenames to UPPER case
3F1 Filenames: lslp - list filenames in directory & print immediately
3G1 Filenames: spreadA - list filenames 4-up across the page
5C1 Filenames: llc - list filenames in directory with line counts
3H1 Find: longest1 - find longest line in a text file
8C1 Find: chmod1 - set perms on all levels - using 'find'
3C1 First: llt - list files, latest First
6D2 Fixed-format: Fixed-Format created by uvfix1 from tab-delimited
6D2 Fixed-format: uvfix1 ex#2 - convert Tab-delimited file to Fixed-format
6F1 Fixed-length: uvfix2 - modify 1 Fixed-length file
6G1 Fixed-length: uvfixB - modify ALL Fixed-Length files in subdir
8B2 Forgot: killuser2 - kill all users who forgot to logoff
8D1 Ftrigger51: - test file presence & run a script
8D1 Ftrigger52: - G.P. test file presence & run a script

8D1 G.p.: ftrigger52 - G.P. test file presence & run a script
1I1 Gdg: renameGDG - change mainframe GDG suffix .G####V00 to _000001
6H0 Grep: dropbl0 - drop blank lines (using 'grep')

1A1 Hard: the HARD way to rename files
4G1 Help: screens
5A1 Help: prep for demos using help files
5A1 Help: v12 - help menu for counting scripts
3B2 Hidden: lla - lists All files including hidden .files
0A3 Homedirs: uvadm, appsadm,& user homedirs
0A2 How: VU scripts - where stored, how accessed (PATH)

5A1 Illustrating: clfdp - 2nd demo illustrating 'not'
3F1 Immediately: lslp - list filenames in directory & print immediately
1G3 In: rename2PP - rename to whatever is in (Parenthesis)
3B2 Including: lla - lists All files including hidden .files
7A1 Indir: copycpio1 - copy subdirs/files from indir to outdir
7B1 Indir: copycpio2 - copy subdirs/files to outdir from indir
5A1 Info: more info on VU scripts
6A2 Insert: insertCR - insert Carriage Returns in text files
6A4 Insert: insertCR - insert CR's in unix text files
6A6 Insert: insertCRall - insert CRs into all files in directory
6A2 Insertcr: insertCR - insert Carriage Returns in text files
6A4 Insertcr: insertCR - insert CR's in unix text files
6A6 Insertcrall: insertCRall - insert CRs into all files in directory
6A6 Into: insertCRall - insert CRs into all files in directory

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

1I2 Keep: renameGDG1 - change '.G####V00' to '_000001' (keep latest)
8B1 Kill: killuser1 - kill user by username (vs process#)
8B2 Kill: killuser2 - kill all users who forgot to logoff
8B1 Killuser1: - kill user by username (vs process#)
8B2 Killuser2: - kill all users who forgot to logoff
8B3 Killuser2: crontab_root - to schedule 'killuser2' at 3 AM

8C1 Levels: chmod1 - set perms on all levels - using 'find'
8C2 Levels: chmod2 - set perms on all levels - using 'recursion'
3A1 Line: llc - list directory with File & Line counts
3H1 Line: longest1 - find longest line in a text file
5A1 Line: demo llc - list all files in subdir with line counts
5C1 Line: llc - list filenames in directory with line counts
5D3 Line: cfdp - Count Files in a Directory with a Pattern any line any file
3A1 List: llc - list directory with File & Line counts
3B1 List: llm - list long & pipe to more
3C1 List: llt - list files, latest First
3C2 List: lls - list files in Size order
3C3 List: llu - list files Unsorted
3D1 List: lld - list directories only (omit files)
3E1 List: llr - list files & subdirs Recursively
3F1 List: lslp - list filenames in directory & print immediately
3G1 List: spreadA - list filenames 4-up across the page
4G1 List: scripts to list/copy/move/remove old/new files
4H1 List: lsold1 - list files older than x days
4H1 List: lsold2 - list files created/modified between 2 days
4H2 List: lsnew1 - list files newer than x days
4H2 List: lsnew2 - list files created/modified between 2 days
5A1 List: demo llc - list all files in subdir with line counts
5C1 List: llc - list filenames in directory with line counts
3B2 Lists: lla - lists All files including hidden .files
4H4 Lists: rmold2 - lists old files, then prompt to remove EACH
4H5 Lists: rmold3 - lists old files, then prompt to remove ALL
3B2 Lla: - lists All files including hidden .files
3A1 Llc: - list directory with File & Line counts
3A2 Llc: sample output for 'llc'
5A1 Llc: demo llc - list all files in subdir with line counts
5C1 Llc: - list filenames in directory with line counts
5C2 Llc: sample output for 'llc'
3D1 Lld: - list directories only (omit files)
3B1 Llm: - list long & pipe to more
3E1 Llr: - list files & subdirs Recursively
3C2 Lls: - list files in Size order
3C1 Llt: - list files, latest First
3C3 Llu: - list files Unsorted
8B2 Logoff: killuser2 - kill all users who forgot to logoff
3B1 Long: llm - list long & pipe to more
3H1 Longest: longest1 - find longest line in a text file
3H1 Longest1: - find longest line in a text file
1B1 Lower: renameL - translate filenames to Lower case
1B3 Lower: renameLsd - Lower case all files in all subdirs of superdir
4C1 Lower: all2lower - translate to lower copying all files to 2nd dir
3F1 Lslp: - list filenames in directory & print immediately
4H2 Lsnew1: - list files newer than x days
4H2 Lsnew2: - list files created/modified between 2 days
4G2 Lsold1: demo lsold1 & rmold1
4H1 Lsold1: - list files older than x days
4H1 Lsold2: - list files created/modified between 2 days

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

1A1 Mainframe: example - renaming Mainframe Data Files
1I1 Mainframe: renameGDG - change mainframe GDG suffix .G####V00 to _000001
5A1 Match: 2nd demo clfd2 - Counts with optional pattern match
5F1 Matchin: clfd - Count Files & Lines, optionally matching a pattern
5F2 Matchin: clfd2 - Count Files, Lines,& KB, optionally matching a pattern
5A1 Menu: v12 - help menu for counting scripts
7D2 Modification: cpnew2 - copy files within a modification date range
4H1 Modified: lsold2 - list files created/modified between 2 days
4H2 Modified: lsnew2 - list files created/modified between 2 days
6D1 Modify: uvfix1 ex#1 - modify 1 text file with 'mvc'
6E1 Modify: uvfixA - modify ALL Text files in subdir
6F1 Modify: uvfix2 - modify 1 Fixed-length file
6G1 Modify: uvfixB - modify ALL Fixed-Length files in subdir
4G1 Move: scripts to list/copy/move/remove old/new files
4I1 Multi: editmfp1 - edit multi files in directory in Size order
6D1 Mvc: uvfix1 ex#1 - modify 1 text file with 'mvc'

4G1 New: scripts to list/copy/move/remove old/new files
4H2 Newer: lsnew1 - list files newer than x days
7D1 Newer: cpnew1 - copy files to a 2nd subdir newer than x days
6J1 Nulls: parmnullfix1 - copy all files, nulls to blanks, using 'sed'
6J2 Nulls: parmnullfix2 - nulls to blanks & clear 73-80, using 'uvcp'

4G1 Old: scripts to list/copy/move/remove old/new files
4H4 Old: rmold2 - lists old files, then prompt to remove EACH
4H5 Old: rmold3 - lists old files, then prompt to remove ALL
4H1 Older: lsold1 - list files older than x days
4H3 Older: rmold1 - remove files older than x days
3D1 Omit: lld - list directories only (omit files)
5A1 Optional: 2nd demo clfd2 - Counts with optional pattern match
5F1 Optionally: clfd - Count Files & Lines, optionally matching a pattern
5F2 Optionally: clfd2 - Count Files, Lines,& KB, optionally matching a pattern
3C2 Order: lls - list files in Size order
4I1 Order: editmfp1 - edit multi files in directory in Size order
7A1 Outdir: copycpio1 - copy subdirs/files from indir to outdir
7B1 Outdir: copycpio2 - copy subdirs/files to outdir from indir

4B1 Packed: allcmp - compare all packed/binary files in 2 directories
3G1 Page: spreadA - list filenames 4-up across the page
1G2 Parenthesis: rename-PP - remove any (Parenthesis)
1G3 Parenthesis: rename2PP - rename to whatever is in (Parenthesis)
6J1 Parmnullfix1: - copy all files, nulls to blanks, using 'sed'
6J2 Parmnullfix2: - nulls to blanks & clear 73-80, using 'uvcp'
0A2 Path: VU scripts - where stored, how accessed (PATH)
1E1 Pattern: rename-AA - remove Any pattern Anywhere in filename
1E2 Pattern: renameAA - replace Any pattern with any Alternate
5A1 Pattern: 2nd demo clfd2 - Counts with optional pattern match
5A1 Pattern: demo clfdp - Count Lines in all Files of a Directory with a Pattern
5D3 Pattern: cfdp - Count Files in a Directory with a Pattern any line any file
5E1 Pattern: clfp - Count Lines in a File with a Pattern
5E2 Pattern: clfdp - Count Lines in all Files of a Directory with a Pattern
5F1 Pattern: clfd - Count Files & Lines, optionally matching a pattern
5F2 Pattern: clfd2 - Count Files, Lines,& KB, optionally matching a pattern
8C1 Perms: chmod1 - set perms on all levels - using 'find'
8C2 Perms: chmod2 - set perms on all levels - using 'recursion'
8C3 Perms: chmod3 - set perms using recursion & add x to bin/script
3B1 Pipe: llm - list long & pipe to more
1C1 Prefix: renameP - replace Prefix with alternate
1C2 Prefix: rename-P - remove any specified prefix
1C3 Prefix: rename-P. - remove prefix up to 1st '.'
1C4 Prefix: rename+P - add any specified prefix
5A1 Prep: for demos using help files
8D1 Presence: ftrigger51 - test file presence & run a script
8D1 Presence: ftrigger52 - G.P. test file presence & run a script
3F1 Print: lslp - list filenames in directory & print immediately
6I1 Printable: printable1 - ensure printable characters
6I1 Printable1: - ensure printable characters
4F1 Printer: allcancel - cancel all printer requests for current user
6B2 Process: runsed2 - run sed script to process all files from 1 dir to 2nd dir
8B1 Process: killuser1 - kill user by username (vs process#)
6E1 Program: sample output - 1st 8 lines of 1st program
4H4 Prompt: rmold2 - lists old files, then prompt to remove EACH
4H5 Prompt: rmold3 - lists old files, then prompt to remove ALL

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

1G1 Quotes: rename-QQ - remove any 'Quotes'

7D2 Range: cpnew2 - copy files within a modification date range
8C2 Recursion: chmod2 - set perms on all levels - using 'recursion'
8C3 Recursion: chmod3 - set perms using recursion & add x to bin/script
3E1 Recursively: llr - list files & subdirs Recursively
6H2 Reduce: dropbl2 - reduce multiple blank lines to 1
0A2 Relevant: lines from common_profile
1C2 Remove: rename-P - remove any specified prefix
1C3 Remove: rename-P. - remove prefix up to 1st '.'
1D3 Remove: rename-X - remove any extension
1D4 Remove: rename-.X - remove any .extension
1E1 Remove: rename-AA - remove Any pattern Anywhere in filename
1G1 Remove: rename-QQ - remove any 'Quotes'
1G2 Remove: rename-PP - remove any (Parenthesis)
1H1 Remove: rename-TS - remove timestamp _HHMMSS
4D1 Remove: rmzf - remove zero-length (empty) files from a directory
4E1 Remove: allrm - remove all files in a directory
4G1 Remove: scripts to list/copy/move/remove old/new files
4H3 Remove: rmold1 - remove files older than x days
4H4 Remove: rmold2 - lists old files, then prompt to remove EACH
4H5 Remove: rmold3 - lists old files, then prompt to remove ALL
6A1 Remove: removeCR - remove Carriage Returns from text files
6A3 Remove: removeCR - remove CR's from Windows text files
6A5 Remove: removeCRall - remove CRs from all files in directory
6A1 Removecr: removeCR - remove Carriage Returns from text files
6A3 Removecr: removeCR - remove CR's from Windows text files
6A5 Removecrall: removeCRall - remove CRs from all files in directory
1A1 Rename: the EASY way to rename files
1A1 Rename: the HARD way to rename files
1G3 Rename: rename2PP - rename to whatever is in (Parenthesis)
1C4 Rename+p: rename+P - add any specified prefix
1D5 Rename+x: rename+X - add any specified extension
1D4 Rename-.x: rename-.X - remove any .extension
1E1 Rename-aa: rename-AA - remove Any pattern Anywhere in filename
1C2 Rename-p: rename-P - remove any specified prefix
1C3 Rename-p.: rename-P. - remove prefix up to 1st '.'
1G2 Rename-pp: rename-PP - remove any (Parenthesis)
1G1 Rename-qq: rename-QQ - remove any 'Quotes'
1H1 Rename-ts: rename-TS - remove timestamp _HHMMSS
1D3 Rename-x: rename-X - remove any extension
1D2 Rename.x: rename.X - change any .extension to any .alternate
1G3 Rename2pp: rename2PP - rename to whatever is in (Parenthesis)
1E2 Renameaa: renameAA - replace Any pattern with any Alternate
1F1 Renameb2u: renameB2U - change any embedded blanks to underscores
1J1 Renamecc: renameCC - convert any 1 character to a 2nd character
1F2 Renamed2u: renameD2U - change any '$' dollar signs to '_' underscores
1I1 Renamegdg: renameGDG - change mainframe GDG suffix .G####V00 to _000001
1I2 Renamegdg1: renameGDG1 - change '.G####V00' to '_000001' (keep latest)
1B1 Renamel: renameL - translate filenames to Lower case
1B2 Renamel: renameU same as renameL except for
1B3 Renamelsd: renameLsd - Lower case all files in all subdirs of superdir
1C1 Renamep: renameP - replace Prefix with alternate
1B2 Renameu: renameU - translate filenames to UPPER case
1B2 Renameu: renameU same as renameL except for
1D1 Renamex: renameX - change any extension to any alternate
1A1 Renaming: example - renaming Mainframe Data Files
1C1 Replace: renameP - replace Prefix with alternate
1E2 Replace: renameAA - replace Any pattern with any Alternate
4F1 Requests: allcancel - cancel all printer requests for current user
6A1 Returns: removeCR - remove Carriage Returns from text files
6A2 Returns: insertCR - insert Carriage Returns in text files
4G2 Rmold1: demo lsold1 & rmold1
4H3 Rmold1: - remove files older than x days
4H4 Rmold2: - lists old files, then prompt to remove EACH
4H5 Rmold3: - lists old files, then prompt to remove ALL
4D1 Rmzf: - remove zero-length (empty) files from a directory
8D1 Ru: ftrigger51 - test file presence & run a script
8D1 Ru: ftrigger52 - G.P. test file presence & run a script
0A3 Run: setup to run uvcopy test/demo jobs
6B2 Run: runsed2 - run sed script to process all files from 1 dir to 2nd dir
6B1 Runsed1: - apply sed script to all files in directory
6B2 Runsed2: - run sed script to process all files from 1 dir to 2nd dir

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

1B2 Same: renameU same as renameL except for
6H5 Same: OUTPUT file tmp/testdropbl0 & 1 (same)
3A2 Sample: output for 'llc'
5C2 Sample: output for 'llc'
6E1 Sample: output - 1st 8 lines of 1st program
8B3 Schedule: crontab_root - to schedule 'killuser2' at 3 AM
4G1 Screens: help screens
6B1 Sed: runsed1 - apply sed script to all files in directory
6B2 Sed: runsed2 - run sed script to process all files from 1 dir to 2nd dir
6J1 Sed: parmnullfix1 - copy all files, nulls to blanks, using 'sed'
8C1 Set: chmod1 - set perms on all levels - using 'find'
8C2 Set: chmod2 - set perms on all levels - using 'recursion'
8C3 Set: chmod3 - set perms using recursion & add x to bin/script
0A3 Setup: to run uvcopy test/demo jobs
1F2 Signs: renameD2U - change any '$' dollar signs to '_' underscores
3C2 Size: lls - list files in Size order
4I1 Size: editmfp1 - edit multi files in directory in Size order
7E1 Sort: sortcpio - copy/sort all subdirs/files from dir1 to dir2
7F1 Sort: sortall - sort all text files from dir1 to dir2
7F1 Sortall: - sort all text files from dir1 to dir2
7E1 Sortcpio: - copy/sort all subdirs/files from dir1 to dir2
1C2 Specified: rename-P - remove any specified prefix
1C4 Specified: rename+P - add any specified prefix
1D5 Specified: rename+X - add any specified extension
3F2 Spread1: script spreadA calls 'uvcopy job' spread1
3F2 Spreada: script spreadA calls 'uvcopy job' spread1
3G1 Spreada: spreadA - list filenames 4-up across the page
0A2 Stored: VU scripts - where stored, how accessed (PATH)
5A1 Sub-dir: demo cfdd - count files in all sub-dirs of a super-dir
5D2 Sub-dir: cfdd - Count Files in all sub-Dirs of a super-Dir
1B3 Subdir: renameLsd - Lower case all files in all subdirs of superdir
4A2 Subdir: alldiff2sub - compare all files in all subdirs in 2 superdirs
5A1 Subdir: demo clf - Count Files & Lines in subdir
5A1 Subdir: demo llc - list all files in subdir with line counts
6E1 Subdir: uvfixA - modify ALL Text files in subdir
6G1 Subdir: uvfixB - modify ALL Fixed-Length files in subdir
7D1 Subdir: cpnew1 - copy files to a 2nd subdir newer than x days
3E1 Subdirs: llr - list files & subdirs Recursively
7A1 Subdirs: copycpio1 - copy subdirs/files from indir to outdir
7B1 Subdirs: copycpio2 - copy subdirs/files to outdir from indir
7E1 Subdirs: sortcpio - copy/sort all subdirs/files from dir1 to dir2
1I1 Suffix: renameGDG - change mainframe GDG suffix .G####V00 to _000001
5A1 Super-dir: demo cfdd - count files in all sub-dirs of a super-dir
5D2 Super-dir: cfdd - Count Files in all sub-Dirs of a super-Dir
1B3 Superdir: renameLsd - Lower case all files in all subdirs of superdir
4A2 Superdirs: alldiff2sub - compare all files in all subdirs in 2 superdirs

6D2 Tab-delimited: Fixed-Format created by uvfix1 from tab-delimited
6D2 Tab-delimited: uvfix1 ex#2 - convert Tab-delimited file to Fixed-format
0A3 Test: setup to run uvcopy test/demo jobs
8D1 Test: ftrigger51 - test file presence & run a script
8D1 Test: ftrigger52 - G.P. test file presence & run a script
6H5 Testdropbl0: OUTPUT file tmp/testdropbl0 & 1 (same)
6H5 Testdropbl2: OUTPUT file tmp/testdropbl2
6H5 Testdropbl3: OUTPUT file tmp/testdropbl3
3H1 Text: longest1 - find longest line in a text file
4A1 Text: alldiff - compare all text files in 2 directories
6A1 Text: removeCR - remove Carriage Returns from text files
6A2 Text: insertCR - insert Carriage Returns in text files
6A3 Text: removeCR - remove CR's from Windows text files
6A4 Text: insertCR - insert CR's in unix text files
6D1 Text: uvfix1 ex#1 - modify 1 text file with 'mvc'
6E1 Text: uvfixA - modify ALL Text files in subdir
7F1 Text: sortall - sort all text files from dir1 to dir2
7D1 Tha: cpnew1 - copy files to a 2nd subdir newer than x days
1H1 Timestamp: rename-TS - remove timestamp _HHMMSS
6H5 Tmp: OUTPUT file tmp/testdropbl0 & 1 (same)
6H5 Tmp: OUTPUT file tmp/testdropbl2
6H5 Tmp: OUTPUT file tmp/testdropbl3
1C3 To: rename-P. - remove prefix up to 1st '.'
1I2 To: renameGDG1 - change '.G####V00' to '_000001' (keep latest)
5A1 Total: demo cfdt - Count Files in 1 directory - Total only
5D1 Total: cfdt - Count Files in Directory, Total only
1B1 Translate: renameL - translate filenames to Lower case
1B2 Translate: renameU - translate filenames to UPPER case
4C1 Translate: all2lower - translate to lower copying all files to 2nd dir
4C2 Translate: all2upper - translate to UPPER copying all files to 2nd dir

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

1F1 Underscores: renameB2U - change any embedded blanks to underscores
1F2 Underscores: renameD2U - change any '$' dollar signs to '_' underscores
6A4 Unix: insertCR - insert CR's in unix text files
3C3 Unsorted: llu - list files Unsorted
1B2 Upper: renameU - translate filenames to UPPER case
4C2 Upper: all2upper - translate to UPPER copying all files to 2nd dir
0A3 User: uvadm, appsadm,& user homedirs
4F1 User: allcancel - cancel all printer requests for current user
8B1 User: killuser1 - kill user by username (vs process#)
8B1 Username: killuser1 - kill user by username (vs process#)
8B2 Users: killuser2 - kill all users who forgot to logoff
0A3 Uvadm: uvadm, appsadm,& user homedirs
0A3 Uvcopy: setup to run uvcopy test/demo jobs
3F2 Uvcopy: script spreadA calls 'uvcopy job' spread1
6C0 Uvcopy: uvfix scripts & uvcopy jobs
6C0 Uvcopy: uvfix scripts - easy way to use uvcopy
6H4 Uvcopy: dropbl3 - uvcopy job to drop blank lines & *comment lines
6H1 Uvcp: dropbl1 - drop blank lines (using 'uvcp')
6J2 Uvcp: parmnullfix2 - nulls to blanks & clear 73-80, using 'uvcp'
6C0 Uvfix: scripts & uvcopy jobs
6C0 Uvfix: scripts - easy way to use uvcopy
6D1 Uvfix1: demo uvfix1,A,2,B
6D1 Uvfix1: ex#1 - modify 1 text file with 'mvc'
6D2 Uvfix1: Fixed-Format created by uvfix1 from tab-delimited
6D2 Uvfix1: ex#2 - convert Tab-delimited file to Fixed-format
6F1 Uvfix2: - modify 1 Fixed-length file
6E1 Uvfixa: uvfixA - modify ALL Text files in subdir
6G1 Uvfixb: uvfixB - modify ALL Fixed-Length files in subdir
3F1 Uvlp12: Note re 'uvlp12'

1I1 V0: renameGDG - change mainframe GDG suffix .G####V00 to _000001
1I2 V00: renameGDG1 - change '.G####V00' to '_000001' (keep latest)
5A1 V12: - counting Lines & Files in directories
5A1 V12: - help menu for counting scripts
8B1 Vs: killuser1 - kill user by username (vs process#)
5A1 Vu: more info on VU scripts

1A1 Way: the EASY way to rename files
1A1 Way: the HARD way to rename files
6C0 Way: uvfix scripts - easy way to use uvcopy
1G3 Whateve: rename2PP - rename to whatever is in (Parenthesis)
0A2 Where: VU scripts - where stored, how accessed (PATH)
8B2 Who: killuser2 - kill all users who forgot to logoff
6A3 Windows: removeCR - remove CR's from Windows text files
5A1 Wit: demo clfdp - Count Lines in all Files of a Directory with a Pattern
5D3 Wit: cfdp - Count Files in a Directory with a Pattern any line any file
5E1 Wit: clfp - Count Lines in a File with a Pattern
5E2 Wit: clfdp - Count Lines in all Files of a Directory with a Pattern
7D2 Within: cpnew2 - copy files within a modification date range
5A1 Word: demo wc - Word Count

4D1 Zero-length: rmzf - remove zero-length (empty) files from a directory

1F2 _: renameD2U - change any '$' dollar signs to '_' underscores
1I1 _000001: renameGDG - change mainframe GDG suffix .G####V00 to _000001
1I2 _000001: renameGDG1 - change '.G####V00' to '_000001' (keep latest)
1H1 _hhmmss: rename-TS - remove timestamp _HHMMSS

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Visitor Counters for ThisYear and LastYear

ThisYear=000953   (J=135,F=188,M=208,A=232,M=190,J=0,J=0,A=0,S=0,O=0,N=0,D=0)
LastYear=001594   (J=165,F=142,M=112,A=121,M=146,J=119,J=132,A=84,S=84,O=116,N=182,D=191)