What are Zombie, Orphan and Daemon processes in Unix system?

Are you looking for the information about process management in Unix System? If already know about foreground and background process and would like to know more about Zombie, Orphan and Daemon processes in Unix system? If yes, then refer this article to understand more interesting things about process management in Unix. If you have not gone through the previous article about Process Management in Unix System then I would recommend going through it first.

What are Zombie processes?

Zombie processes are those processes which are killed or completed execution but still shows an entry in the process table.

Important points-

• The zombie process shows the process with a Z state.
• The zombie process is also known as the defunct process.
• Such kind of processes are dead and not being used for any business or system processing.
• The orphan processes and the zombie processes are different.
• A child process always first becomes a zombie before being removed 
• To remove zombies we can use 'kill' command as the SIGCHLD signal can be sent to the parent manually.

Reasons for the zombie processes-

1. The issue in the parent program: If the Zombies that exist for more time then it indicates an issue in the parent program
2. The issue with child program: The Unix system uncommon decision to not reap children.
3. The operating system issue: The bug in the operating system may cause the parent program is no longer running.

Impact of the zombie process-

1. No out of memory issue as the zombie process does not consume memory
2. The issue with running out of process table entries

 What are Orphan processes?

A child process which remains running itself even after its parent process is completed or terminated is called as an orphan process.

Important points

1. The orphan process is get created unknowingly and unintentionally due to process crash.
2. Normally, Unix system tries to protect the orphan process with help of user's shell. The Unix system tries to terminate the child processes by SIGHUP process signal.
3. Re-parenting automatically happens in Unix environment. i.e. assigning the parent to the child process. However, it is internal to Unix system
4. An orphan process is a computer process whose parent process has finished or terminated, though it remains running itself. Normally special init system process will be the parent for such orphan process. The process is still considered an orphan process as its original parent who created this process is no more exists.
5. In some cases, the orphan process is created intentionally. This is to detach the current user's session and let it run in the background, especially when the process is long running. Sometimes such orphan processes are also called a daemon process.

What is the daemon process?

The system related processes which run in the background and often run with the permissions of the root are called daemon processes. In many cases, these daemon processes service requests from other processes.

1. A daemon has no controlling terminal. 
2. The daemon process cannot open /dev/tty. 
3. e.g If we execute the command "ps -ef" and all daemon processes will have ? for the tty field in the output.
4. If we have a program which runs for a long time, it is good idea to make it a daemon and run it in the background.
5. Normally, such processes start when the system is bootstrapped 
6. The daemon processes terminate only when the system is shut down. 

Example of daemon process is real world is - printer daemon waiting for print commands.

Process Management in Unix System

Are you looking for an article on how does process management happen in Unix? Would you like to foreground and background processes in the Unix? Are you also would you like to know about commands related to the process management? If yes, then you reached the right place. This article provides detailed information about process management in Unix system.

Overview

The new process will be created and started whenever we issue a command in the Unix system. In order to execute a program or a command, a special environment is created. e.g. If we execute the command 'grep' or 'ls', it will start the process internally.

The process ID (aka pid) is five-digit id used by Unix operating system to track process. It is unique for each process in given executing Unix environment.
The pid values repeat because all the possible numbers are used up. However, no two processes with the same pid exist in the Unix system because it is used to track each process.

Types of processes

There are two types of processes-

1. Foreground processes
2. Background processes

1. Foreground Processes

The process which takes input from the keyboard and sends its output to the screen is a foreground process. 

If any foreground process is running we cannot execute any other command or start any other process as prompt will not be available until the existing process is finished.

e.g. When we execute 'ls' command output is returned to the screen. It is an example of the foreground process.

2. Background Processes

The process which runs without being connected to your keyboard is called the background process.

The background process goes in wait mode if it requires any keyboard input. We can execute other commands while the existing process is running.  In order to execute the command in background mode, add an ampersand (&) at the end of the command. 

e.g. When we execute the command 'ls * &', it runs as the background process.

Commands

a) Use below command to list currently running processes

$ps

The result of this command will be

PID       TTY      TIME        CMD

18008     ttyp3    00:00:00    abc.sh

18311     ttyp3    00:03:31    test

19789     ttyp3    00:00:00    ps

b) In some cases, -f (i.e. full) option is used to get more information
$ps -f

The result of this command will be
UID      PID  PPID C STIME    TTY   TIME CMD
abcuid   1138 3062 0 01:23:03 pts/6 0:00 abc
abcuid   2239 3602 0 02:22:54 pts/6 0:00 pqer
abcuid   3362 3157 0 03:10:53 pts/6 0:00 xyz

Here,
UID: It is the user ID that the process belongs to
PID: Process ID
PPID: Parent process ID
C: CPU utilization of process
STIME: Process start time
TTY: Terminal type
TIME: CPU time is taken by the process
CMD: The command that started this process

Child and Parent Processes

• Two ID numbers will be assigned to each process.
• These two ID numbers are the parent process ID (PPID) and the process ID (PID) or child id.
• In the Unix system, each user process has a parent process and hence PPID is assigned to each user process.
• The shell will act as a parent whenever we execute any command.
• If we see the output of the command 'ps -f', we can notice that the process ID and the parent process ID is listed.

The video below provides more information about processes in Unix system.

How to use 'grep' commands in Unix?

Are you looking for  details about 'grep' commands in the Unix environment? Are you also looking for what are structures and samples for various 'grep' commands in the Unix system? If so, then this article provides detailed information about 'grep' command with its usage. 

What is the 'grep' command?

The word grep stands for globally search a regular expression and print. The command 'grep' is a command-line utility. It is used for searching plain-text data. The search is performed by using a regular expression.

Commands:

1. Use the command below to search a specific string in the specified file
grep "Techno Guru" test_file

2. Use the command below to search a specific string in all the files
grep "Techno Guru" *

3. Use the command below to search a specific string in only in the .log  files. We can also use regular expressions such as abc*.log, *test*.*, abc*.log. Search will be performed against files which matches this file patterns.
grep "Techno Guru" *.log

4. Use the command below to perform case in-sensitive search. The command below will matches all the words such as "TECHNO GURU", "Techno Guru", "tEchno Guru", "techno guru" etc.
grep -i "Techno Guru" test_file

5. To print the matched line, along with 5 lines after it.
grep  -A 5 -i "Techno Guru" test_file

6. To perform recursive search
grep -r "Techno Guru" test_file

7. To perform recursive search in all the files. Below command searches "Techno Guru" word in all the files under the current directory and its sub directory
grep -r "Techno Guru" *

8. Use the command below to search using regular expression in search string. The command below searches for starting string "Techno" and ends with "Guru". You can use other different search pattern as well.
grep "Techno*Guru" test_file

9. What are the regular patterns can be used?
The below mentioned regular patterns can be used while working with grep command.
? : Matched at most once.
* : Matched zero or more times.
+  : Matched one or more times.
{n} : Matched exactly n times.
{n,} : Matched n or more times.
{,m}  : Matched at most m times.
{n,m} : Matched at least n times, but not more than m times.

10. Use 'grep -w' command to search for full words and not for sub-strings. In the below command exact "Techno" or "techno" or "TECHNO" will be matched in the file. However if file contains "Technoworld" then it will not be identified as match.
grep -iw "Techno" test_file

11. Displaying lines before/after/around the match using grep -A, -B and -C
If you are performing file analysis in real time project, it will be useful to see some lines after or before the match.

a) To display 5 lines after match
grep -A 5 -i "Techno Guru" test_file

b) To display 5 lines before match
grep -B 5 "Techno Guru" test_file

c) To display 4 lines around match. The option -C used for the match to be appeared with the lines from both the side.
grep -C 4 "Techno Guru" test_file

12. To count the number of matches use the command below
grep -c "Technology World"  Test_file

13. To count the number lines which does NOT found match, use the command below
grep -v -c "Technology World"  Test_file

14. To highlight the search. In order to see which part matches the line, we can highlight it as (Use can use any color to highlight)
export GREP_OPTIONS='--color=auto' GREP_COLOR='99;8'

15. To determine the name of files in which match string found.
grep -l "Techno" test_*

16. Use the command below to show only matched string as by default grep command shows the line which matches the given string.
grep -o "Techno Guru" Test_file

Learn more about Unix in the video below:

How to use 'tar' commands in Unix?

Are you looking for various commands in the Unix environment? Are you also looking for what are structures and samples for each command in the Unix system? If so, then this article provides detailed information about command details with its usage. In this article we will focus on the 'tar' command.

'tar' command:

What is the 'tar' command and why is it used? In Unix, 'tar' is the abbreviation for Tape ARchive.  This command is used to store entire file systems onto the magnetic tape. The command 'tar' is also commonly used to combine the multiple files into a single file for easy storage and distribution.

Below are commonly used 'tar' commands -

A.Commands dealing with .tar files

1. The command 'tar cvf' is used to create a new tar archive file. With the 'tar' command, the files are not compressed. The files are only combined and grouped into a single archive file.
$ tar cvf archive_abc.tar dirabc/

Here, 
c - Create a new archive
v – Verbosely list files
f – Following is the archive file name

2. The command 'tar xvf' is used to extract an existing archive file.
$ tar xvf archive_abc.tar

Here,
x – Extract files from the archive file

3. The command 'tar xvf' is used to extract a single file 'file_abc' from the existing archive file.
$ tar xvf archive_abc.tar /location/to/file_abc

4. The command 'tar xvf' is used to extract a single directory 'abcdir' from the existing archive file. The subdirectories and files are extracted with this command.
$ tar xvf archive_abc.tar /location/to/abcdir/

To extract multiple directories use the command below 
$ tar xvf archive_abc.tar /location/to/abcdir1/ /location/to/abcdir1/

5. The command 'tar xvf' is used to extract multiple files from the .tar file. We can use Regular Expression to achieve it. The command below will list all files with extension 'txt'
$ tar xvf archive_abc.tar --wildcards '*.txt'

6. The command 'tar tvf' is used to view the archive file. This will list the files without extracting files.
$ tar tvf archive_abc.tar

7. The command 'tar rvf' is used to add a new file in an existing .tar file. With the command below we can add 'file_pqr' file to archive_abc.tar file.
$ tar rvf archive_abc.tar file_pqr

We can use the command below to add the directory to the existing archive (.tar) file 
$ tar rvf archive_abc.tar dir_xyz/

Note: We cannot add files or directories to compressed files such as .gz or .bz2 file. If we try to add then we will get the 'Cannot update compressed archives' error message.

8. The command 'tar tvfW' is used to verify the archive (.tar) files. Normally we use this command before removing any file from the archive file.
$ tar tvfW file_xyz.tar

Note:
a. If an output line starts with Verify with no differs line then the file/directory is Ok. If that is not the case, then we need to fix the issue.
b. We cannot verify files in a compressed the archive file. (e.g. .gz, .bz2)

9. The command 'tar -cf' is used to determine the size of the .tar file. The size returned is in KB.
$ tar -cf - /location/to/archive/ | wc -c
20660

B.Commands dealing with .gz files

1. The command 'tar cvzf' is used create the gzipped archive file. With the help of this command the files are compressed and grouped into a single file. The file extension .tar.gz and .tgz are both the same.
$ tar cvzf archive_abc.tar.gz dirabc/
Here,
z - Archive through gzip

2. The command 'tar xvfz' is used to extract the existing gzipped file.
$ tar xvfz archive_abc.tar.gz

3. The command 'tar xvfz' is used to extract a single file 'file_abc' from the existing gzipped file.
$ tar xvfz archive_abc.tar.gz /location/to/file_abc

4. The command 'tar xvfz' is used to extract a single directory 'abcdir' from the existing gzipped file. The subdirectories and files are extracted with this command.
$ tar xvfz archive_abc.tar.gz /location/to/abcdir/

To extract multiple directories use the command below 
$ tar xvfz archive_abc.tar.gz /location/to/abcdir1/ /location/to/abcdir1/

5. The command 'tar xvfz' is used to extract multiple files from the .gz file. The command below will list all files with the extension 'txt' with the help of RegExp.
$ tar xvfz archive_abc.tar.gz --wildcards '*.txt'

6. The command 'tar tvfz' is used to view the .gz archive file. This will list the files without extracting files.
$ tar tvfz archive_abc.tar.gz

7. The command 'tar -czf' is used to determine the size of .gz file. The size returned is in KB.
$ tar -czf - /location/to/archive/ | wc -c
200

C.Commands dealing with .bz2 files

1. The command 'tar cvfj' is used to create a bzipped tar archive file. The size of bzip2 is lesser than gzip. Normally the creation of the bzip2 file takes more time to compress and decompress than gzip.
$ tar cvfj archive_abc.tar.bz2 dirabc/

Here, 
j – Filter the archive through bzip2
Bz2 - It is the abbreviation for Burrows-Wheeler compression 

2. The command 'tar xvfj' is used to extract the existing bzipped file.
$ tar xvfj archive_abc.tar.bz2

3. The command 'tar xvfj' is used to extract a single file 'file_abc' from the existing bzipped file.
$ tar xvfj archive_abc.tar.bz2 /location/to/file_abc

4. The command 'tar xvfj' is used to extract a single directory 'abcdir' from the existing bzipped file. The subdirectories and files are extracted with this command.
$ tar xvfj archive_abc.tar.bz2 /location/to/abcdir/

To extract multiple directories use the command below 
$ tar xvfj archive_abc.tar.bz2 /location/to/abcdir1/ /location/to/abcdir1/

5. The command 'tar xvfj' is used to extract multiple files from .bz2 file. The command below will list all files with the extension 'txt' with the help of RegExp.
$ tar xvfj archive_abc.tar.bz2 --wildcards '*.txt'

6. The command 'tar tvfj' is used to view the .bz2 archive file. This will list the files without extracting files.
$ tar tvfj archive_abc.tar.bz2

7. The command 'tar -cjf' is used to determine the size of .bz2 file. The size returned is in KB.
$ tar -cjf - /location/to/archive/ | wc -c
200 

The video below provides details about commands in the Unix environment:

Variables in the Unix system

Are you looking for information about what variables are available in the Unix Operating system? Would you be interested in knowing how to work with variables in the Unix environment? This article provides details about variables used in Unix programming with sample examples.

Introduction

A variable in the Unix environment is used to store values temporarily and use it during program execution. 

A variable can contain only letters (a to z or A to Z), numbers ( 0 to 9) or the underscore character ( _). The standard practice is to use the variable name in UPPERCASE but it is not mandatory. We can write it in lowercase too.

For example: testVar, test1, var1 , 2var, _var 

What are types of variables?

• Local Variables: These variables present within the current instance of the shell. These are not available to programs that are started by the shell.  They are set at the command prompt.
• Environment Variables: These variables are available to any child process of the shell. 
• Shell Variables : A shell variable is a special variable that is set by the shell and is required by the shell in order to function correctly.  These can include environment variables and local variables.

Defining and Initiating Variables 

The syntax for creating a variable is as below

variable_name = variable_value

Here variable 'country' is a scalar variable. A scalar variable can hold only one value at a time.

e.g. country=“India”

Unix shell enables us to store any value - like storing the integer value in the country_cd field

e.g. country_cd=100

How to access value from a variable?

To access value from a variable, we need to use the $ character before the variable name. 

For example,

#!/bin/sh

NAME="Techno Guru"

echo $NAME

How to create the read only variable?

To mark variables as read only by using the readonly command.

The value of  the readonly variable cannot be changed.

e.g.

#!/bin/sh

VAR="Techno Guru"

readonly VAR

VAR="Training"

How to unset variables?

• Unsetting or deleting a variable directs the shell to remove the variable from the list of variables that it tracks. 
• Once we unset a variable, we cannot access the stored value in the variable.
• We cannot use the unset command to unset variables that are marked readonly

e.g.

#!/bin/sh

VAR="Techno Guru"

unset VAR

echo $VAR

The video below provides in-depth knowledge about how to use variables in the Unix environment with the demo.

Important File and Directory permissions in Unix

Are you looking for how permissions works in the Unix Operating system? Would you be interested in knowing what types of permissions are available in the Unix environment? The details about permissions in the Unix are explained in this article. This article also provides highlights on various characteristics about File and Directory permissions.

What are the types of file permissions?

The file permissions categories are as follows:

• Owner permissions − It determines what actions the owner of the file can perform on the file.
• Group permissions − It determines what actions a user, who is a member of the group to which a file belongs, can perform on the file.
• Other (world) permissions − It indicates what action all other users can perform on the file. 

How to display Permissions?

•  To display permissions on screen use ‘ls –l’ command -> read (r), write (w), execute (x)          e.g.

          ls –l /usr/tmp

    -rwxr-xr-- 1 testuser users 1017 Jan 2 00:10 myfile

    drwxr-xr-- 1 testuer users 1017 Jan 2 00:10 mydir

• Here, the first column represents different access modes, i.e., the permission associated with a file or a directory. The first character ‘-‘ stands for the file and the character ‘d’ stands for the directory.
• The first three characters (2-4) represent the permissions for the file's owner. For example, -rwxr-xr-- represents that the owner has read (r), write (w) and execute (x) permission.
• The second group of three characters (5-7) consists of the permissions for the group to which the file belongs. For example, -rwxr-xr-- represents that the group has read (r) and execute (x) permission, but no write permission.
• The last group of three characters (8-10) represents the permissions for everyone else. For example, -rwxr-xr-- represents that there is read (r) only permission. 

Understanding File access modes

There are three types of file access modes: Read, Write and Execute. Mentioned below are the details about each mode:

• Read : Grants the capability to read, i.e., view the contents of the file.
• Write: Grants the capability to modify or remove the content of the file.
• Execute: User with execute permissions can run a file as a program. 

Understanding Directory access mode

There are three types of directory access modes: Read, Write and Execute. Mentioned below are the details about each mode:

• Read: Access to a directory means that the user can read the contents. The user can look at the filenames inside the directory.
• Write: Access means that the user can add or delete files from the directory.
• Execute: Executing a directory doesn't really make sense, so think of this as a traverse permission. A user must have execute access to the bin directory in order to execute the ls or the cd command. 

How to change permissions?

Use the chmod (change mode) command to change permissions.

There are two ways to use chmod:

1. The symbolic mode
2. The absolute mode 

Symbolic mode

With symbolic permissions we can add, delete, or specify the permission set we want by using the operators

 + : Adds the designated permission(s) to a file or directory

  - : Removes the designated permission(s) from a file or directory

  = : Sets the designated permission(s)

a) Change permission for other users

                  chmod o+wx test1file

b) Change permission for owner user

                  chmod u-x testfile

c) Change permission for group

                 chmod g=rx testfile

d) Change permission for users and groups

                 chmod o+wx,u-x,g=rx testfile

Absolute Mode

Use a number to specify each set of permissions for the file

Number	Description	Detail
0	No permission	---
1	Execute permission	--x
2	Write permission	-w-
3	Execute and write permission: 1 (execute) + 2 (write) = 3	-wx
4	Read permission	r--
5	Read and execute permission: 4 (read) + 1 (execute) = 5	r-x
6	Read and write permission: 4 (read) + 2 (write) = 6	rw-
7	All permissions: 4 (read) + 2 (write) + 1 (execute) = 7	rwx

Examples

a) chmod 755 testfile (all, read-write, read-write)

b) chmod 743 testfile (all, read,write-execute)

c) chmod 043 testfile (no permission, read, write-execute)

More details about the file and directory are explained with examples in the video below:

Important and Useful Unix commands

Are you looking for an article which provides a list of important commands used for daily unix activities? This article provides a consolidated list of unix commands.

Introduction

In this article we have listed unix commands for : File and Directories, Compressed Files and Manipulating Data

File and Directory

Command	Details
cat	Displays File Contents
cd	Changes Directory to another directory
chgrp	Changes file group
chmod	Changes permissions
cp	Copies source file into destination
file	Determines file type
find	Finds files
grep	Searches files for regular expressions
head	Displays first few lines of a file
ln	Creates softlink on oldname
ls	Displays information about file type
mkdir	Creates a new directory dirname
more	Displays data in paginated form
mv	Moves (Renames) an oldname to newname
pwd	Prints current working directory
rm	Removes (Deletes) filename
rmdir	Deletes an existing directory provided it is empty
tail	Prints last few lines in a file
touch	Updates access and modification time of a file
vi	To view file content

Compressed Files

Command	Details
compress	Compresses files
gunzip	Helps uncompress gzipped files
gzip	GNU alternative compression method
uncompress	Helps uncompress files
unzip	List, test and extract compressed files in a ZIP archive
zcat	Cat a compressed file
zcmp	Compares compressed files
zdiff	Compares compressed files
zmore	File perusal filter for crt viewing of compressed text
apropos	Locates commands by keyword lookup
info	Displays command information pages online
man	Displays manual pages online
whatis	Searches the whatis database for complete words
yelp	GNOME help viewer

Manipulating Data

Command	Details
awk	Pattern scanning and processing language
cmp	Compares the contents of two files
comm	Compares sorted data
cut	Cuts out selected fields of each line of a file
diff	Differential file comparator
expand	Expands tabs to spaces
join	Joins files on some common field
perl	Data manipulation language
sed	Stream text editor
sort	Sorts file data
split	Splits file into smaller files
tr	Translates characters
uniq	Reports repeated lines in a file
wc	Counts words, lines, and characters
vi	Opens vi text editor
vim	Opens vim text editor
fmt	Simple text formatter
spell	Checks text for spelling error
ispell	Checks text for spelling error
emacs	GNU project Emacs
ex, edit	Line editor

The video below provides a tutorial on Unix topics -