Category Archives: Linux

Frequently used OS (Linux/Solaris/AIX) Command for DBA

As a DBA you need to use frequent OS command or alt least how to query the OS and its hardware. Usually we do it before fresh install upgrade, migrate of database/operating system. Here is some of the useful frequently used day to day OS command for DBA.
To find and delete files older than N number of days:
find . -name ‘*.*’ -mtime +[N in days] -exec rm {} ;
Example : find . -mtime +5 -exec rm {} ;
The above command is specially useful to delete log, trace, tmp file
To list files modified in last N days:
find . -mtime – -exec ls -lt {} ;
Example: find . -mtime +3 -exec ls -lt {} ;1
The above command will find files modified in last 3 days
To sort files based on Size of file:
ls -l | sort -nk 5 | more
useful to find large files in log directory to delete in case disk is full
To find files changed in last N days :
find -mtime -N –print
Example: find -mtime -2 -print
To find CPU & Memory detail of linux:
cat /proc/cpuinfo (CPU)
cat /proc/meminfo (Memory)
Linux: cat /proc/cpuinfo|grep processor|wc -l
HP: ioscan -fkn -C processor|tail +3|wc -l
Solaris: psrinfo -v|grep “Status of processor”|wc –l
psrinfo -v|grep “Status of processor”|wc –l
lscfg -vs|grep proc | wc -l
To find if Operating system in 32 bit or 64 bit:
ON Linux: uname -m
On 64-bit platform, you will get: x86_64 and on 32-bit patform , you will get:i686
On HP: getconf KERNEL_BITS
On Solaris: /usr/bin/isainfo –kv
On 64-bit patform, you will get: 64-bit sparcv9 kernel modules and on 32-bit, you will get: 32-bit sparc kernel modules. For solaris you can use directly: isainfo -v
If you see out put like: “32-bit sparc applications” that means your O.S. is only 32 bit but if you see output like “64-bit sparcv9 applications” that means youe OS is 64 bit & can support both 32 & 64 bit applications.
To find if any service is listening on particular port or not:
netstat -an | grep {port no}
Example: netstat -an | grep 1523
To find Process ID (PID) associated with any port:
This command is useful if any service is running on a particular port (389, 1521..) and that is run away process which you wish to terminate using kill command
lsof | grep {port no.} (lsof should be installed and in path)
How to kill all similar processes with single command:
ps -ef | grep opmn |grep -v grep | awk ‘{print $2}’ |xargs -i kill -9 {}
Locating Files under a particular directory:
find . -print |grep -i test.sql
To remove a specific column of output from a UNIX command:
For example to determine the UNIX process Ids for all Oracle processes on server (second column)
ps -ef |grep -i oracle |awk ‘{ print $2 }’
Changing the standard prompt for Oracle Users:
Edit the .profile for the oracle user
PS1=”`hostname`*$ORACLE_SID:$PWD>”
Display top 10 CPU consumers using the ps command:
/usr/ucb/ps auxgw | head -11
Show number of active Oracle dedicated connection users for a particular ORACLE_SID
ps -ef | grep $ORACLE_SID|grep -v grep|grep -v ora_|wc -l
Display the number of CPU’s in Solaris:
psrinfo -v | grep “Status of processor”|wc -l
Display the number of CPU’s in AIX:
lsdev -C | grep Process|wc -l
Display RAM Memory size on Solaris:
prtconf |grep -i mem
Display RAM memory size on AIX:
First determine name of memory device: lsdev -C |grep mem
then assuming the name of the memory device is ‘mem0’ then the command is: lsattr -El mem0
Swap space allocation and usage:
Solaris : swap -s or swap -l
Aix : lsps -a
Total number of semaphores held by all instances on server:
ipcs -as | awk ‘{sum += $9} END {print sum}’
View allocated RAM memory segments:
ipcs -pmb
Manually deallocate shared memeory segments:
ipcrm -m ”
Show mount points for a disk in AIX:
lspv -l hdisk13
Display occupied space (in KB) for a file or collection of files in a directory or sub-directory:
du -ks * | sort -n| tail
Display total file space in a directory:
du -ks .
Cleanup any unwanted trace files more than seven days old:
find . *.trc -mtime +7 -exec rm {} ;
Locate Oracle files that contain certain strings:
find . -print | xargs grep rollback
Locate recently created UNIX files:
find . -mtime -1 -print
Finding large files on the server:
find . -size +102400 -print
Crontab Use:
To submit a task every Tuesday (day 2) at 2:45PM
45 14 2 * * /opt/oracle/scripts/tr_listener.sh > /dev/null 2>&1
To submit a task to run every 15 minutes on weekdays (days 1-5)
15,30,45 * 1-5 * * /opt/oracle/scripts/tr_listener.sh > /dev/null 2>&1
To submit a task to run every hour at 15 minutes past the hour on weekends (days 6 and 0)
15 * 0,6 * * opt/oracle/scripts/tr_listener.sh > /dev/null 2>&1
For more related Linux/Solaris Basic command click on the link: Basic Linux/Solaris Command

Basic Linux Command for Beginners

The purpose of this document is to understand the Basic Linux command in a fast or simple way. It assumes that we have very little exposure of Linux command.

If we are familiar with DOS command then it is easy to understand Linux command with the following differences:

  1. In Linux/UNIX, commands and filenames are case sensitive, meaning that typing “EXIT” instead of the proper “exit” is a mistake.
  2. In DOS, the forward-slash “/” is the command argument delimiter, while the backslash “” is a directory separator. In Linux/UNIX, the “/” is the directory separator, and the “” is an escape character.
  3. The DOS world uses the “eight dot three” filename convention, meaning that all files followed a format that allowed up to 8 characters in the filename, followed by a period (“dot”), followed by an option extension, up to 3 characters long (e.g. FILENAME.TXT). In UNIX/Linux, there is no such thing as a file extension. Periods can be placed at any part of the filename, and “extensions” may be interpreted differently by all programs, or not at all.

cat

Display the contents of a text file on the screen. For example: cat contact.txt would display the file we created in the previous section.

tail -f

Display the last few lines of a text file, and then output appended data as the file grows (very useful for following log files!). Example: tail -f /var/log/messages

top

Displays CPU processes in a full-screen GUI. A great way to see the activity on your computer in real-time. Type “Q” to quit.

echo

Display text on the screen. Mostly useful when writing shell scripts. For example: echo “Hello World”

find

It can be used to search for files matching certain patterns, as well as many other types of searches. example: find . -name *mp3. This example starts searching in the current directory “.” and all subdirectories, looking for files with “mp3” at the end of their names.

locate

A quick way to search for files anywhere on the file system. For example, you can find all files and directories that contain the name “oracle” by typing: locate oracle

which

Shows the full path of shell commands found in your path. For example, if you want to know exactly where the “grep” command is located on the filesystem, you can type “which grep”. The output should be something like: /bin/grep

cp, mv

First one Copies a file from one location to another. Example: cp mp3files.txt /tmp (copies the mp3files.txt file to the /tmp directory) and second one moves a file to a new location, or renames it. Example: mv mp3files.txt /tmp (copy the file to /tmp, and delete it from the original location)

rm

Delete a file. Example: rm /tmp/mp3files.txt

mkdir, rmdir

First one Make Directory. Example: mkdir /tmp/myfiles/ and second one Remove Directory. Example: rmdir /tmp/myfiles/

ls

List all files in the current directory, in column format For example: ls /usr/bin/d* This command list all files whose names begin with the letter “d” in the /usr/bin directory.

cat /proc/cpuinfo, cat /proc/meminfo

Display information about CPU and Memory usage

uname -a

Prints system information to the screen (kernel version, machine type, etc.)

ps

Lists currently running process (programs).

more, less

First command Display a file, or program output one page at a time and second one is the improved replacement of first one allows you to move backward as well as forward. Example: ls -la | more

grep

Search for a pattern in a file or program output. For example, to find out which TCP network port is used by the “nfs” service, you can do this: grep .nfs. /etc/services. This looks for any line that contains the string “nfs” in the file “/etc/services” and displays only those lines.

su

It allows you to switch to another user account temporarily. Example: su Shahid, su – (Switch to root, and log in with root’s environment)

“history”

It shows your complete command history.

chmod

To change file access permission.

Note: Press (CTRL-R) and then type any portion of a recent command. It will search the commands for you, and once you find the command you want, just press ENTER.

Help: When you are struck and needing help with a Linux command just press “-h” or “help”

Example:  grep  — help (built in help)

Example: man ls (get the help on ls) or man man (a manual about how to use the manual)

Example: info df (load information page)

For more frequently used OS Command for DBA click on the link: Frequently used OS (Linux/Solaris/AIX) Command for DBA

Do not invoke SQL*Plus with a password On UNIX and Linux platforms.

Most of us sometimes start SQL * Plus with a password on UNIX and Linux platforms without knowing security threat.

For example, an application user connects SQL * Plus by passing username and password on Unix/Linux Server.

$ sqlplus apps/apps@proddb

Here the sqlplus command parameters are very much available for viewing by all operating system users on the same host computer; as a result, password entered on the command line could be exposed to other users, as below.

$ ps -efgrep sqlplus
oracle 14490 2190 0 16:31:53 pts/5 0:00 sqlplus apps/apps@proddb
oracle 14493 14491 0 16:32:01 pts/5 0:00 grep sqlplus

So, there might be a chance for an intruder to know the user id and password, and can connect to the database using that credentials.

Then, following is the secure and best way of connecting SQL * Plus where the password is not exposed on the command line.

$ sqlplus apps@proddb
Enter password: ****

Or, even not to expose the username and connecting string.

$ sqlplus
Enter user-name: apps@proddb
Enter password: ****

Or

$ sqlplus /nolog
SQL> connect apps@proddb
Enter password: ****

And also, do not use the password while invoking Export/Import Utility using exp/imp command line, and for any other command line utilities which you think the password will be exposed to others.

On Microsoft Windows, the command recall feature (the Up arrow) remembers user input across command invocations.

For example, if you use the CONNECT APPS/password notation in SQL*Plus, exit, and then press the Up arrow to repeat the CONNECT command, the command recall feature discloses the connect string and shows the password. So, it is advice *NOT* to pass the password while connecting to SQL * Plus on windows as well.

UNIX Useful Commands

1) Find whether OS is 64/32 Bit Kernel in UNIX.

uname -a

2) Find free physical memory in UNIX.

free -m

3) Find CPU details in UNIX.

cat /proc/cpuinfo

4) Find files modified within specific time.

find . -mtime -3 (modified less than 3days ago)

5) command used to alter file permissions.

chmod 777 abc.txt

6) Command used to reset the Ownership.

chown oracle:dba abc.txt

7) command used to set, or reset, the users login password.

Passwd username

8) Kill specific process in UNIX.

Kill -9 processid

9) Command used for display last given lines of a file.

tail -n alert_PROD.log

10) Command used for intall a rpm package.

rpm -ivh packagename.rpm

11) Command used to querry about any rpm package

rpm -q packagename

12) Command to Check the server up time

uptime

13) Command to check the file versions

strings -a <filename> |grep ‘$Header’

14) Command will keep ‘n’ number of days files and remove rest of file.

find . -mtime +n -exec rm  {} ; &

15) Basic commands for  vi editor

i   :- insert before cursor.

l   : insert begining of the line.

a  :- append after the cursor.

A  :- Append at the end of the line.

o :- insert a blank line below the cursor.

O :- insert a blank line above the cursor position.

h :- from current position one char towards left .

I :- from current position one char towards right.

j :- from current position one line towards down.

k :- from current position one line towards up.

Shift+g  :- go to end of the file.

Shift+:1 :- go to top of the file.

dd –> delete the ful line.

:q! —> closing the file without saving any changes.

:wq! –> save the changes and force close.

:w –> to save the changes without closing the file.

Make operating system user non expiry

Many times proper functioning of application demands that the account/password of the OS (Linux/UNIX) user which owns the application should not expire.

chage is the OS command by which we can control the number of days between password changes and the date of the last password change.

The chage command is restricted to the root user, except for the -l option, which may be used by an unprivileged user to determine when the password or account is due to expire.

Let us make password of ‘oracle’ OS user account as non-expiry.

1) CHECK THE CURRENT STATUS OF USER

2) CHANGE AND MAKE IT NON-EXPIRY

# chage -m -1 -M -1 -I -1 -E -1 oracle

Where:

[-m mindays]
[-M maxdays]
[-d lastday]
[-I inactive]
[-E expiredate]
[-W warndays]

3) VERIFY

[root@rclnperp20 ~]# chage -l oracle

UNIX Crontab Basics

A crontab file has five fields for specifying day , date and time followed  by the command to be run at that interval.
#——————————————————————————————
# Minute     Hour    Month Day      Month       Weekday                      Command
#  0-59       0-23      1-31                 1-12           0-6 (0=Sunday)
#——————————————————————————————
* * * * * command to be executed
– – – – –
| | | | |
| | | | +—– day of week (1 – 7) (monday = 1)
| | | +——- month (1 – 12)
| | +——— day of month (1 – 31)
| +———– hour (0 – 23)
+————- min (0 – 59)
crontab -e Edit your crontab file, or create one if it doesn’t already  exist.
crontab -l Display your crontab file.
crontab -r Remove your crontab file.
crontab -v Display the last time you edited your crontab file.
Example:
To run the calendar command at 6:30 a.m. every Monday, Wednesday, and Friday, enter:

To run the calendar command every day of the year at 6:30, enter the following:

To run a script called maintenance every day at midnight in August, enter the following:

You can execute crontab if your name appears in the file /usr/lib/cron/cron.allow. If that file does not exist, you can use  crontab if your name does not appear in the file  /usr/lib/cron/cron.deny.  If only cron.deny exists and is empty, all users can use crontab. If neither  file exists, only the root user can use crontab. The allow/deny files  consist of one user name per line.

Useful UNIX/LINUX commands for DBAs

Below are some of the useful UNIX/LINUX commands which we use frequently:

–LIST YOUR FILES

ls — lists your files
ls -l — lists your files in ‘long format’, which contains lots of useful information, e.g. the  exact size of the file, who owns the file and who has the right to look at it, and when it was last modified.
ls -a — lists all files, including the ones whose filenames begin in a dot, which you do  not always want to see.

–COMPRESSING THE FILE

gzip filename — compresses files, so that they take up much less space. There are other tools for this  purpose, too (e.g. compress), but gzip usually gives the highest compression rate. Gzip produces files with the ending ‘.gz’ appended to the original filename.
gunzip filename — uncompresses files compressed by gzip.

–PRINTING RELATED COMMANDS 

lpr filename — print to default printer
lpr -P<printer_name> filename — Use the -P option to specify the printer name if you want to  use a printer other than your default printer.
lpq — check out the default printer queue.
lpq <printer_name>— check out specific printer queue.
lprm jobnumber — remove specific job queue from the printer queue. Y

–DIRECTORY RELATED COMMANDS

mkdir dirname — make a new directory
cd dirname — change directory.  ‘cd ..‘ will get you one level  up from your current position.
pwd — tells you where you currently are.

–MOVING, RENAMING, AND COPYING FILES

cp file1 file2 —copy a file
mv file1 newname — move or rename a file
rm file1 [file2 ...] —remove or delete a file
rm -r dir1 [dir2...] —recursivly remove a directory and its contents

–VIEWING AND EDITING FILES

cat filename —Dump a file to the screen in ascii.
more filename— Progressively dump a file to the screen: ENTER = one line  down SPACEBAR = page down q=quit
less filename —Like more, but you can use Page-Up too. Not available on all systems.
vi filename —Edit a file using the vi editor. All UNIX systems will have vi in some form.
head filename —Show the first few lines of a file.
head -n filename —Show the first n lines of a file.tail filename —Show the last few lines of a file.
tail -n filename —Show the last n lines of a file

–FIND COMMAND

find search_path -name filename — Find command format
find . -name aaa.txt —Finds all the files named aaa.txt in the current  directory or  any subdirectory tree.
find / -name vimrc —Find all the files named ‘vimrc’ anywhere on the  system.
find /usr/local/games -name “*xpilot*” —Find all files whose names contain the string  ‘xpilot’ which  exist within the ‘/usr/local/games’ directory tree.

–VI EDITOR – VERY IMPORTANT TO LEARN

Start with vi <filename>

Edit modes: These keys enter editing modes and type in the text of your document.

i —Insert before current cursor position

I —Insert at beginning of current line

a —Insert (append) after current cursor position

A —Append to end of line

r —Replace 1 character

R —Replace mode

<ESC> —Terminate insertion or overwrite mode

Deletion of text

x —Delete single character

dd —Delete current line and put in buffer

ndd —Delete n lines (n is a number) and put them in buffer

J —Attaches the next line to the end of the current line (deletes carriage return).

Undo

u —Undo last command

Cut and Paste

yy —Yank current line into buffer

nyy —Yank n lines into buffer

p —Put the contents of the buffer after the current line

P —Put the contents of the buffer before the current line

Cursor Positioning

ctrl-d —Page down

ctrl-u —Page up

:n —Position cursor at line n

:$ —Position cursor at end of file

^g —Display current line number

h,j,k,l —Left,Down,Up, and Right respectivly. Your arrow keys should also work

String Substitution

:n1,n2:s/string1/string2/[g] —Substitute string2 for string1 on lines n1 to n2. If g is included (meaning global),
all instances of string1 on each line are substituted. If g is not included, only the first instance per matching line is substituted.

^ —matches start of line

. —matches any single character

$ —matches end of line

These and other “special characters” (like the forward slash) can be “escaped” with

i.e to match the string “/usr/STRIM100/SOFT” say “/usr/STRIM100/SOFT”

Examples:

:1,$:s/dog/cat/g —Substitute ‘cat’ for ‘dog’, every instance

for the entire file – lines 1 to $ (end of file)
:23,25:/frog/bird/ —Substitute ‘bird’ for ‘frog’ on lines 23 through 25. Only the first instance on each line is substituted.

Saving and quitting commands
These commands are all prefixed by pressing colon (:) and then entered in the lower left corner of the window.

Press <ESC> to exit from an editing mode.

:w —Write the current file.

:w new.file —Write the file to the name ‘new.file’.

:w! existing.file —Overwrite an existing file with the file currently being edited

:wq —Write the file and quit.

:q —Quit.

:q! —Quit with no changes.

:e filename —Open the file ‘filename’ for editing.

:set number —Turns on line numbering

:set nonumber —Turns off line numbering

–KILLING ALL LOCAL=NO PROCESS BY SINGLE COMMAND  

kill ps -ef | grep <INSTNAME> |grep LOCAL=NO| egrep -v grep | awk '{print $2}'

or

ps -ef| grep -i <INSTNAME> | grep LOCAL=NO| awk {‘print $2′}| xargs kill -9

-CUSTOMIZE UNIX COMMAND PROMPT

Put below value in .profile of user

HOSTNAME=$(uname -n)
PS1='(${?})${LOGNAME}@${HOSTNAME}:${ORACLE_SID:-“NO_SID”}: ${PWD}
$ ‘
PS2=’> ‘

–MOUNT COMMAND

mount -t nfs -o <options> server-name-or-ip-address:/<file_system_to_mount> /<file_system_to_mount>

–To find the number of physical CPUs:

$ cat /proc/cpuinfo | grep "^physical id" | sort | uniq | wc -l
 2

–To find the number of cores per CPU:

$ cat /proc/cpuinfo | grep "^cpu cores" | uniq
cpu cores : 4

The total number of processors available is the number of physical CPUs multiplied by the number of cores per CPU.

–To find the total number of processors:

$ cat /proc/cpuinfo | grep "^processor" | wc -l
 16

How to determine OS block size for windows or unix

In many times you probably have heard about set the oracle db block size as a multiple of the OS block size. But whenever you ask how I can determine or find OS block size for windows or Unix then many one get stopped. In fact this question is OS related. In this post I will show how I can get OS block.

1)UNIX or Linux System.
On my Solaris Machine,
SQL> !perl -e ‘$a=(stat “.”)[11]; print $a’
8192
On my debian Linux,
$ perl -e ‘$a=(stat “.”)[11]; print $a’
4096

On my Solaris Machine another way,
SQL> !df -g /oracle
/oracle (/dev/dsk/c1d0s3 ): 8192 block size 1024 frag size
Where /oracle is disk partition.

Another example,
SQL> !df -g | grep ‘block size’
/ (/dev/dsk/c1d0s0 ): 8192 block size 1024 frag size
/devices (/devices ): 512 block size 512 frag size
/system/contract (ctfs ): 512 block size 512 frag size
/proc (proc ): 512 block size 512 frag size
/etc/mnttab (mnttab ): 512 block size 512 frag size
/etc/svc/volatile (swap ): 4096 block size 4096 frag size
/system/object (objfs ): 512 block size 512 frag size
/lib/libc.so.1 (/usr/lib/libc/libc_hwcap1.so.1): 8192 block size 1024 frag size
/dev/fd (fd ): 1024 block size 1024 frag size
/tmp (swap ): 4096 block size 4096 frag size
/var/run (swap ): 4096 block size 4096 frag size
/oradata1 (/dev/dsk/c1d0s4 ): 8192 block size 1024 frag size
/oracle (/dev/dsk/c1d0s3 ): 8192 block size 1024 frag size
/oradata2 (/dev/dsk/c1d0s5 ): 8192 block size 1024 frag size
/export/home (/dev/dsk/c1d0s7 ): 8192 block size 1024 frag size

2)Windows Machine.
If you use ntfs file system you can use
fsutil fsinfo ntfsinfo drivename: to get information of block size.

c:\>fsutil fsinfo ntfsinfo f:
NTFS Volume Serial Number : 0x1e345b64345b3e49
Version : 3.1
Number Sectors : 0x0000000004a852c0
Total Clusters : 0x0000000000950a58
Free Clusters : 0x00000000003ae433
Total Reserved : 0x0000000000000000
Bytes Per Sector : 512
Bytes Per Cluster : 4096 — This is OS block size.
Bytes Per FileRecord Segment : 1024
Clusters Per FileRecord Segment : 0
Mft Valid Data Length : 0x000000000d72a000
Mft Start Lcn : 0x00000000000c0000
Mft2 Start Lcn : 0x00000000004a812c
Mft Zone Start : 0x0000000000800720
Mft Zone End : 0x000000000083b920

To get All possible size on windows machine use,
c:\ format /?
In the /A:size you will get the all possible size for both FAT and NTFS file system. Whenever you format a drive you can select default allocation unit as block size.

3)On HPUX issue like “vgdisplay -v /dev/vg01/lvol”

Linux Regular Commands

Linux
Check number of CPU’s

$cat /proc/cpuinfo

Check memory information

cat /proc/meminfo

Check swap

$swapon -s

Kernel parameter

#/sbin/sysctl -a
cat /etc/sysctl.conf

System error log

/var/log/messages

Nic configuration

ifconfig -a

Directory Commands – mkdir

  • To create a new directory below current directory:

$mkdir new-dir-name

  • To create multiple versions of directory:

$mkdir -p dir1/dir2/dir3

  • Creates dir1 first. Creates dir2 under dir1. Creates dir3 under dir2
  • To create a directory with given permissions:

$mkdir -m 754 new-dir

  • new-dir will be created with 754 permissions

Directory Commands – rmdir

  • To remove a directory:

$rmdir dir-name

  • rmdir can remove only empty directories
  • To remove directory along parent directory:

$rmdir -p dir1/dir2/dir3

  • Deletes all the directories

Directory Commands – cd

  • Stands for change directory
  • Used to navigate among directories

$cd dir1

  • Takes to dir1 directory

$cd

  • Takes to the home directory
  • Can give full-path-name or relative-path-name as argument

UNIX commands – banner

  • Prints the given string in big letters

$banner <string>

UNIX commands – touch

  • Used to change the last access time of a file

$touch -a file1

  • Changes the access time of file1 with current time

$touch file1

  • Changes access time as well as modification time will be changed to current time

UNIX commands – file

  • Used to know the file type like shell script, executable

$file file-name

english text

UNIX file commands – wc

  • Used to know the number of characters, number of files and number of lines in a file

$wc file1

$wc -l file1

  • Displays the number of lines in file1
  • Can take the input from terminal directly

UNIX file commands – sort

  • Used to sort a file

$sort file1

  • Sorts file1 and displays the sorted file on terminal
  • Instead of displaying the sorted file on terminal, can be written in to a file by:

$sort –o file2 file1

  • Writes the sorted file1 in file2

UNIX file commands – cut

  • Cut is a filter.
  • Picks up given number of bytes from a given file by:

$cut -f 5,10 file1

  • Displays 5th field to 10th field in file1. Each field is assumed to be separated by TAB character

UNIX file commands – grep

  • This is the good example of unfriendly UNIX command
  • Acronym for ‘Globally search a Regular Expression and Print it’
  • Searches for a pattern in file(s)

$grep abc file1

  • Searches for the string ‘abc’ in file1

UNIX file commands – head

  • Displays only first few lines of a file as per the argument

$head -15 file1

  • Displays only first 15 lines of file1

UNIX file commands – tail

  • Displays only last few lines of a file as per the argument

$tail -15 file1

  • Displays the last 15 lines of file1

UNIX file commands – pg/more

  • Both commands work almost like the same with few differences

$pg +5 -20 -p -s file1

  • Starts displaying file1 20 lines per page, starting from 5th line, with prompt for each page of display

UNIX file commands – lp

  • Sends the user print job to the print queue

$lp file1 file2

  • request id is lp-23(2 files)
  • Many flags available for different ways of printing
  • Can take the input from terminal

UNIX file commands – lpstat

  • Gives the printer status like which is currently printing, how many jobs are still in queue etc

$lpstat -t

scheduler is running

system default destination : lp

device for lp: /dev/lp0

lp accepting requests since <date>

  • Queue information

UNIX file commands – cancel

  • Used to cancel the given print job

$cancel lp-15

request “lp-15,226 cancelled

UNIX file commands – compress

  • Two commands provided for file compression namely compress and pack
  • They behave the same way

$compress -v file1

file1: Compression: 90.1%

  • Compresses the file file1 and creates a file by name file1.z

UNIX file commands – uncompress

  • Used to unpack the compressed file by:

$uncompress file1

  • Pack and compress will untouch the file if there is no effective compression

UNIX file commands – man

  • UNIX provides on-line manuals by:

$man <command>

  • Displays all the information about <command>

I/O Redirection & Piping

  • I/O can be redirected using ‘<‘ & ‘>’
  • ‘<‘ symbol implies – take the input from a file instead of keyboard
  • ‘>’ symbol implies – write the output in the file instead of display unit
  • ‘2>’ symbols implies – write the error in a file rather than display unit

e.g.:abc<in-file>out-file 2>err-file

  • A ‘>>’ indicates the appending of data to a file if it exists, instead of over-writing
  • 1>&2 indicates – redirect Standard Output to Standard Error
  • <Command-1> | <Command-2> means – the standard output of Command-1 is going to be the standard input of Command-2
  • UNIX supports joining of commands through ‘|’ piping character

$ls | wc -l

  • The output of ls is the input for wc

$ls | wc -l > outfile

  • Output of wc will be written in outfile
  • Output of first process will be written to pipe and the pipe then hands it over to the next process.
  • tee can be used to capture the pipe file

$who | tee file1 | sort

  • Output of who will be written into file1 and the file will be sorted and the sorted file will be written to standard output

UNIX Command – ps

  • To see the processes running currently under UNIX can be had by:

$ps

PID   TTY  TIME  COMMAND

2266  tty2 0:06  sh

2269  tty2 0.02  ps

  • Process Status command is shortly known as ps

UNIX Background Process

  • Time consuming tasks may be run in background
  • Place ‘&’ at the end of the command, the process will go to background
  • Displays the PID after submission
  • Success or failure of the background process will not be reported
  • Better to redirect the background process output to some file. Otherwise, it will disturb the foreground process messages on the screen
  • Too many background processes will degrade the performance
  • If logged out, all the background processes will be terminated

UNIX Command – Sleep

This command is used to delay a process for a period

of time

For Example :

To run a command at regular intervals, enter:

while true

do

date

sleep 60

done

This shell procedure displays the date and time once a minute.

To stop it, press the Interrupt key sequence.

Killing a Process

  • To terminate a process

$kill PID

  • An user can kill only his processes

UNIX Communication – write

  • Any user can ‘write’ a message on other user’s terminal.
  • Other user should give permissions to write message

$write <user-name>

hello

^d (Ctrl+d)

  • Hello will be written on user-name’s terminal
  • The <User-name> user should be logged-in to receive the message
  • The recipient should give permissions for message
  • Message writing can be denied by:

$mesg -n

  • Super user can write to any terminal without permissions

Shell Programming

  • A shell program is a series of UNIX commands
  • Instead of typing the commands one after the other, the commands may be written in a file and execute the file.
  • Shell programming offers much more versatility than mere UNIX commands
  • Almost a system can be developed by using shell script with UNIX commands
  • UNIX shell is the interface between the operating system & user
  • It is an interpreter, not an executable
  • A shell script can be executed by giving execute permissions ($chmod +x shell) or by shell command ($sh shell)
  • Shell programming is so powerful, even system shutdown like tasks can be handled
  • Three shells available under UNIX – Bourne, C & Korn shell
  • The following shell scripts were written in Bourne shell
  • Bourne shell script will work with Korn shell and the reverse is not true
  • Bourne is the default shell in most of the installations
  • Shell scripts are used to accomplish variety of tasks like:
  • Customizing the working environment
  • Automating the daily tasks like backups etc
  • Automating repetitive tasks
  • Executing important system procedures like shutdown
  • When the user logins into the system, the UNIX starts a shell (sh) on that terminal which accepts commands from ‘$’ prompt
  • When a shell-script is given for execution, the shell executes the commands from the shell-script

Interactive Shell Programming

  • Two basic words in shell are read & echo
  • read accepts input
  • echo writes the output

echo Enter your name \?

read name

echo Good morning $name

  • Write the above 3 lines in a file name sh01

$sh01

Enter your name ?

Vijay

Good morning Vijay

  • The ‘\’ symbols makes the shell to understand ‘?’ as an ordinary character rather than special character
  • Double quotes (“) may be used to display a string
  • name is a shell variable receives data from keyboard
  • $name will display the contents of the variable

Interactive Shell Programming

echo Enter three values X Y Z

read x y z

echo $x $y $z

  • If 1 2 3 entered, x will have 1, 2 to y & 3 to z
  • If 1 2 3 4 are entered, then z will have 3 4
  • If only 1, 2 are entered, then z will be assigned null value
  • If more arguments entered, last variable will be assigned rest. If less, null will be assigned
  • echo This is new line \nAnd this is second line
  • The above will be displayed as

This is new line

And this is second line

  • The escape sequence was brought from C language
  • \b is blank, \t is tab char etc.
  • To make the cursor at the end of the echoed line

echo “Enter your Name  :\c”

Enter your Name  :_ (Cursor waits here)

  • echo “07” gives bell

Shell Variables

  • They are an integral part of shell programming
  • Provide ability to store and manipulate data
  • Fully under the control of the shell
  • Can create and destroy the variables as the user wants

Shell Variables – Guidelines

  • Any combination of alphabet, numbers and underscore( _ )
  • No editing characters like commas, blanks
  • The first character of the name should be alpha or underscore
  • May be of any reasonable length
  • Case sensitive ie abc & Abc are different

Shell – Assigning values

  • Values can be assigned to variable using ‘=‘ equal sign

$name=Vijay

$echo $name

displays Vijay

Shell Variables

  • There are 2 types of variables
  • 1. UNIX defined or system defined variables
  • 2. User defined variables

System Variables

  • UNIX defines certain variables for its own usage

e.g.: PS1 stands for prompt string1

$PS1=ABC

  • From this time onwards, UNIX displays ABC as prompt
  • PS2 is prompt string 2, default value is ‘>’
  • PATH defines the directories to searched by shell to execute any command or file
  • HOME defines the default working directory
  • LOGNAME stores the log name of the user
  • MAIL stores the file name of the mailbox along path
  • MAILCHECK duration of the mail checking
  • SHELL keeps the default shell name
  • TERM gives the name of the terminal the user is using
  • All the above variable values can be known by set command

$set

  • Displays all the values

Tips & Traps

  • All the variables are string variables
  • $a=20. 20 is stored as a string not as number. No arithmetic can be carried on this
  • Use double quotes if the value contains more than one word

$name=“Vijay Suri”

  • Can assign more than one variable in a line

$name=“Vijay” age=20

  • Can display more than one variable in a line

$echo Name is $name and age is $age

  • All variables in a shell script are automatic variable. i.e. they will be created as soon as the shell script execution starts and dies as soon as the execution is over
  • A null variable can be created by

$a1=“” or a1=‘’ or a1=

  • Shell ignore if any shell variable is having null value

Unchanging Variable – readonly

  • User can make a variable unchanged during execution by:

$age=20

$readonly age

  • The the shell does not allow us to change the value

Wiping out Variable – unset

  • A variable can be removed from the shell by using unset command

$unset age

  • Unset can not be used for system variables
  • $unset PS1 is not allowed

Positional Parameters

  • Many occasions, a program expects the variables in a certain fashion. This is achieved through positional parameters from $0 through $9.
  • $0 is the program itself. Thus

$abc par1 par2 par3 par4 assigns

abc to $0, par1 to $1 … par4 to $4

Passing command line arguments

  • To write a shell by name sh02 which copies one file to another

echo Copying $1 to $2

cp $1 $2

  • By executing sh02

$sh02 file1 file2

Copying file1 to file2.

  • Positional parameters can not assigned values like $1=100

Shell Variables

  • To know the number of variables given for the shell through $#

$abc file1 file

echo $#

  • The above shell displays 3

Shell Variables – shift

  • Shell can handle only 9 variables at a time. To access more than 9, the shift command is used
  • $set You have the capacity to learn the shell programming in a very easy way
  • $echo $10 displays You0 as shell interprets $10 as $1 with 0.
  • $shift 5 : makes the 5th argument as $1 argument. Thus
  • $echo $1 displays learn
  • $* handles all positional parameters

Positional Parameters

Positional Parameters

If the number of parameters are greater than nine

The shift command can be used to shift the parameters

For Example :

$ set a b c d e f g h i j k l m n o p

$ echo $1 $2 $3 $4 $5 $6 $7 $8 $9

a b c d e f g h i

$ shift 7

$ echo $1 $2 $3 $4 $5 $6 $7 $8 $9

h I j k l m n o p

Arithmetic in Shell

  • Write the following in a file and execute:

a=10 b=4

echo ‘expr $a + $b’

echo ‘expr $a – $b’

echo ‘expr $a \* $b’

echo ‘expr $a / $b’

echo ‘expr $a % $b’    #modulus

  • On execution, the output is:

14

6

40

2

2

  • Anything after ‘#’ sign will be treated as comment
  • expr is the key word for doing arithmetic
  • A multiplication symbol (*) should be preceded by ‘\’
  • Terms in expr should be separated by space
  • Parenthesis may be used for clarity of expression
  • Arithmetic in shell scripts
    • The expr can be used only for integers
    • For Floating point the command  bc is used

# example of floating point arithmetic program

a=15.5 b=7.8

c=‘echo $a + $b | bc‘   (returns 23.3)

d=‘echo $a – $b | bc‘   (returns 7.7)

e=‘echo $a \* $b | bc‘  (returns 120.9)

f=‘echo $a / $b | bc‘    (returns 1)

UNIX decision loops

  • There are 4 decision making loops

if then fi

if then else fi

if then elif else fi

case – esac

UNIX – if then fi

  • Key word if and the delimiter is ‘fi’

if <command>

then

statements

fi

e.g.: if cp $1 $2

then echo “Copied successfully”

fi

UNIX – if then else fi

  • The structure of this construct is:

if <Condition>

then

statements

else

statements

fi

e.g.:

if cp $1 $2

then

echo File copied successfully

else

echo File copy failed

fi

Example “if – else – fi

# Example of numeric test

echo Enter basic salary

read bs

if [ $bs -lt 2500 ]

then

hra=‘echo $bs \* 10 / 100 | bc‘

da=‘echo $bs \* 90 / 100 bc‘

else

hra=500

da=‘echo $bs \* 98 / 100 | bc‘

fi

gs=‘echo $bs + $hra + $da | bc‘

echo Gross salary = Rs. $gs

UNIX – if then elif else fi

  • Used for multilevel decision making.

if <condition> then

statements

elif <condition>

statements

else

statements

fi

UNIX – test

  • if depends upon the exit status of the command given
  • test verbs translates the result into success or failure
  • There are three tests namely
  • Numerical test
  • String test
  • File test

UNIX – test – numerical

  • Used to compare numerical

-gt = greater than

-lt  = less than

-ge = greater than or equal to

-le  = less than or equal to

-ne = not equal

-eq = equal

if [ $1 -lt 5 ]

then

echo the value is < 5

elif [ $1 -le 7 ]

echo the value is <= or equal 7

else

echo the value is > 7

fi

UNIX – test

  • Use square braces to avoid writing test
  • Provide a space after ‘[‘
  • Provide a space before ‘]’

UNIX – test – file

  • The following are the file related flags
  • -s returns True if the file exists and size > 0
  • -f returns True if the file exists and not directory
  • -d return True if the file exists and is a directory

e.g.:

if [ -f $1 ] then

echo File exists

fi

UNIX – test – String

  • s1 = s2 returns true if both are same
  • -n returns true if string length > 0

if [ $1 = $2 ] then

echo Both strings are same

else

echo $?

fi

  • $? Contains the value of the last command
  • 0 = true, 1 = false

UNIX logical conditions

  • -a stands for AND condition
  • -o stands for OR condition
  • -! Is negation
  • if <Condition-1> -a <Condition-2>
  • Returns true if both the conditions are true

if [ $1 -gt 60 ] -a [ $2 -lt 50 ] then

statements…

UNIX case

  • To handle multiple choices
  • case value in
  • choice 1) statements;;
  • choice 2) statements;;
  • *) statements;;
  • esac
  • esac is the delimiter of case
  • Used for menus
  • *) is the default choice
  • All choice statements should be terminated by double semicolon(;;)

UNIX case example

case $option in

1) echo Financial accounting;;

2) echo Materiel accounting;;

*) echo Invalid Opt – Try;;

esac

UNIX case

  • The choices many be in any order
  • case statement may be a shell variable or shell argument or output of a command
  • Need not be numbers – may be strings too

e.g.:

banana) statements;;

orange) statements;;

  • Multiple options can be grouped

e.g.:

case $1 in

banana | orange) echo Fruit;;

dog | pig) echo Animal;;

lion) echo Wild animal;;

esac

  • Can use shell’s pattern matching

e.g.:

case $1 in

[a-z]) echo Small alpha;;

[A-Z]) echo Capital;;

esac

UNIX Loop Controls

  • Provided 3 loop constructs namely:
  • while loop
  • for loop
  • until loop

UNIX – while loop

while <condition>

do

statements

done

  • done is the delimiter of do

UNIX – while example

count = 1

while [ $count -le 3 ]

do

echo Loop value $count

count = `expr $count + 1`

done

Examples of “while” loop

# calculation of simple interest

count=1

while [ $count -le 3 ]

do

echo “\nEnter p,n,r\c”

read p n r

si=‘echo $p \* $n \* $r /100 | bc‘

echo Simple interest = Rs.$si

count=‘expr $count + 1‘

done

# printing numbers using while

i=1

while [ $i -le 10 ]

do

echo $i

i=‘expr $i + 1‘

done

UNIX – while

  • The loop will continue so long as the condition is TRUE
  • When the condition is false, the next command after done will be executed
  • The condition can be any valid UNIX command
  • The while condition can be simple condition or complex condition
  • The condition should have an exit status. Otherwise, it may go into infinite loop

UNIX – until loop

until <condition>

do

statements

done

  • until continues its loop so long as the condition is false
  • except this, while & until are identical

Example – until loop

# printing numbers using until

i=1

until [ $i -gt 10 ]

do

echo $1

i=‘expr $i + 1‘

done

UNIX – for loop

  • Most frequently used loop

for control-var in value1 value2…

do

statements

done

  • for takes a list of variables

UNIX – for example

for word in $*

do

echo $word

done

Example – for loop

Example

# To print names of all sub-directories

for entry in *

do

if [ -d $entry ]

then

echo $entry

fi

done

UNIX – break statement

  • Used to break the current loop and comes out of the loop
  • Usually associated with if

if [ $1 -eq 5 ]

then

I = 2

break

fi

UNIX – continue statement

  • To take the control to the beginning of the loop bypassing the statements

I=1

while [ $I -le 5 ] then

do

I = `expr $I + 1`

continue

done

UNIX – Metacharacters

  • Called as regular expressions.
  • Classified as follows:

File name              : ? * […] [!…]

I/O redirection     : < > >> << m> m>&n

Process execution: ; ( ) & && ||

Positional paras   : $1..$9

Spl characters      : $0 $* $@ $# $! $$ $-

UNIX – File name

  • ls ?? – lists all files with 2 chars long
  • ls a* – lists all file names begin with a
  • ls [a-c]* -file names begin with a,b &c
  • ls [!a-c] – file names not starting with a,b,c

UNIX – I/O Redirection

  • < – take input from
  • > write output to
  • >> append output
  • << abc – takes the input till ‘abc’ encountered
  • m> filename – Makes filename as output of m
  • m<&n – Merges stdin, stdout and stderr

UNIX – Process Execution

  • ; – multiple commands separated by ‘;’
  • & – puts to background process
  • && means AND – Comd1 && Comd2 : Executes Comd2 if and only if Comd1 is successful
  • || means OR – Comd1 || Comd2 – Executes Comd2 if Comd1 fails

UNIX – Special Characters

  • $$ – Gives the Process id of the command
  • $? – Exit status of the last executed command
  • $! – PID of last background process
  • $# – Total number of positional parameters
  • $0 – Command name
  • $* – list all shell arguments

Tips & Traps

  • Try to use the absolute paths in the shell scripts. This saves the search time of the system for the command
  • Remove write permissions to files that are important. This even prevents the accidental update of the files
  • Compress the print/report files. These files occupies lot of disk space. Better pack them and keep
  • The compressed print files can be printed directly without uncompress them by using the pcat command
  • Untouched files in the directory may be found using find command.
  • $find . -mtime 10 -print
  • displays the files that are untouched for the last 10 days
  • Clean up of the directory should be done periodically. Otherwise, any amount disk space will be inadequate.

$pcat textfile.txt.z > /dev/lp0

  • The above command will send the print file directly to printer

UNIX Useful Commands

UNIX Useful Commands

1) Find whether OS is 64/32 Bit Kernel in UNIX.

uname -a

2) Find free physical memory in UNIX.

free -m

3) Find CPU details in UNIX.

cat /proc/cpuinfo

4) Find files modified within specific time.

find . -mtime -3 (modified less than 3days ago)

5) command used to alter file permissions.

chmod 777 abc.txt

6) Command used to reset the Ownership.

chown oracle:dba abc.txt

7) command used to set, or reset, the users login password.

Passwd username

8) Kill specific process in UNIX.

Kill -9 processid

9) Command used for display last given lines of a file.

tail -n alert_PROD.log

10) Command used for intall a rpm package.

rpm -ivh packagename.rpm

11) Command used to querry about any rpm package

rpm -q packagename

12) Command to Check the server up time

uptime

13) Command to check the file versions

strings -a <filename> |grep ‘$Header’

14) Command will keep ‘n’ number of days files and remove rest of file.

find . -mtime +n -exec rm  {} \; &

15) Basic commands for  vi editor

i   :- insert before cursor.

l   : insert begining of the line.

a  :- append after the cursor.

A  :- Append at the end of the line.

o :- insert a blank line below the cursor.

O :- insert a blank line above the cursor position.

h :- from current position one char towards left .

I :- from current position one char towards right.

j :- from current position one line towards down.

k :- from current position one line towards up.

Shift+g  :- go to end of the file.

Shift+:1 :- go to top of the file.

dd –> delete the ful line.

:q! —> closing the file without saving any changes.

:wq! –> save the changes and force close.

:w –> to save the changes without closing the file.