Introduction to Linux And Interview Essentials
- Unix is owned by Bell laboratories while linux is open source
- Bash vs Shell: Shell is a program that takes info from keyboard and inputs in OS. Bash Is Bourne Again Shell, which basically means that it is shell but with more features. Bash is being used in most Linux . Features such as command aliasing, command completion(using tab) and command history.
ex of alias: write
$ alias c=clear
now instead of writing clear, you can just write c and it will clear your screen
Daemon
- Its a program that runs in the background. Daemons have no controlling terminal which means you cannot open a tty connection with it, also if you write below command, the daemons also comes up. Daemons that run usually have name ending with a ‘d’ and their tty is usually marked ‘?’ because they cant be communicated using tty.
$ ps -ef
- In Windows, daemons are called services, and behave exactly how daemons were previously described. Daemons are subset of process. A daemon is just a term for a process that runs continuously and usually is not attached to a terminal.
- Daemons are not a separate class of processes and they have no special privileges or attributes.
- There is a BSD/Linux C function called daemon (man page), but this is just really a simple way to detach your process from its terminal. It is so named because that’s what daemons usually do, not the other way around.
ps stands for process status. It reports a snapshot of current processes. It gets the information being displayed from the virtual files in /proc filesystem.
The output of ps command is as follows
$ ps
PID TTY STAT TIME CMD
5140 pts/4 Ss 00:00:00 bash
61244 pts/4 R+ 00:00:00 ps
PID: Every process is assigned a PID (Process Identifier) which is a unique identifier that is associated with a running process in the system.
TTY: Controlling terminal associated with the process.
STAT: Process State Code
TIME: Total time of CPU Usage
CMD: The command that is executed by the process.
+++++ https://medium.com/100-days-of-linux/understanding-the-output-of-ps-commands-e9e270a418f9
Linux Components
- Kernel: a moderator between hardware and software(OS)
- Shell/GUI(System libraries)
- System utilities
Memory and CPU stats
Memory statics (RAM info) :
$ free -m
this shows memory in megabits, replace with -g for gigabit
$ vmstat -a
this will show the virtual memory information
$ vmstat 2 6
Below command will also give you the info of current Ram usage
$ cat /proc/meminfo
This will keep on giving up the info of virtual memory at interval of 2 secs for 6 times
$ vmstat -t
this outputs in time format
sar = system activity reporter vmstat = virtual memory stat counetriostat = input/output stat counternetstat = network stat counter Sar is basically used for getting the performance stats of a system over a period of time , this gives us a awful amount of data , but once mastered is very good vmstat can be used to gather stats on the cpu , memory like paging , the processes etc iostat is for the stats on disk activity and also nfs mounts netstat is for network stats
Difference in Sar and vmstat
The only reason for both the outputs to be different is that sar reports the output in blocks whereas vmstat continues to report in kilobytes. Since, the sar utility reports its swap in blocks, also we all know that one block is 512 bytes in size and two blocks will be 1024 bytes.
CPU usuage
- below command displays your current CPU usuage, however on mac it needs to be installed/setup first
$ sar -u
sar (system activity reporter)
- below command gives output 3 times at internal of 1 secs, like live monitoring
$ sar -u 1 3
- To get free and used info of CPU utilization:
$ sar -r
For live reports 3 times at internal of 1 sec:
$ sar -r 1 3
Best tool for monitoring this utilization of the current server and all other servers and give alerts to: Nagios
Nagios Core is free and open source monitoring tools which is used to monitor the System resources, applications ,Services, Databases and network devices such asRouters,switches,..etc.
The clients will have NRPE ie Nagios Remote Plugin Exucuter: this will keep on monitoring and send alert
Listening Ports on linux
You can get a list of the listening ports on your system by querying the network stack with commands such as ss, netstat or lsof. Each listening port can be open or closed (filtered) using a firewall.
ex: $ netstat
Proto Recv-Q Send-Q Local Address Foreign Address (state)
tcp4 0 0 freeip.amazon.co.55629 ec2-3-227-250-19.https ESTABLISHED
tcp4 0 0 freeip.amazon.co.55627 ig-in-f188.1e100.https ESTABLISHED
tcp4 53 0 freeip.amazon.co.55626 ec2-3-91-171-132.https ESTABLISHED
tcp4 0 0 freeip.amazon.co.55624 104.19.155.59.https ESTABLISHED
tcp4 0 0 localhost.acme localhost.55620 CLOSE_WAIT
tcp4 0 0 localhost.55620 localhost.acme FIN_WAIT_2
tcp4 53 0 freeip.amazon.co.55613 monitor-api-publ.https CLOSE_WAIT
To check status of a process on linux
$ ps aux
output
USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND
_coreaudiod 253 11.2 0.2 4400504 18148 ?? Ss 12:53PM 6:57.48 /usr/sbin/coreau
mishrjm 1173 8.6 2.4 6571976 201444 ?? S 12:54PM 11:04.11 c
Changing size of Volumes using LVM
LVM partition can be shrinked and edited. Fdisk creates partition of fixed size. LVM is logical volume manager is advanced tool and can modify the size of volumes created by it.
SWAP
A memory used by ram when it does not have enough space to contain the running applications. mostly inactive apps are sent to swap. swap is generally created of double the size of ram.
Linux File System Permissions
There are 3 types of users on Linux System:
- Users: A user is the owner of the file. By default, the person who created a file becomes its owner. Hence, a user is also sometimes called an owner.
- Group : A user- group can contain multiple users. All users belonging to a group will have the same Linux group permissions access to the file. Suppose you have a project where a number of people require access to a file. Instead of manually assigning permissions to each user, you could add all users to a group, and assign group permission to file such that only this group members and no one else can read or modify the files.
- Others: Any other user who has access to a file. This person has neither created the file, nor he belongs to a usergroup who could own the file. Practically, it means everybody else. Hence, when you set the permission for others, it is also referred as set permissions for the world.
Linux divides the file permissions into read, write and execute denoted by r,w, and x
| Number | Permission Type | Symbol |
|---|---|---|
| 0 | No Permission | — |
| 1 (2^0) | Execute | –x |
| 2 (2^1) | Write | -w- |
| 3 | Execute + Write | -wx |
| 4 (2^2) | Read | r– |
| 5 | Read + Execute | r-x |
| 6 | Read +Write | rw- |
| 7 | Read + Write +Execute | rwx |
Default format of permissions:
d rwx rwx rwx
d denotes file/directory. D means directory.
Different permissions commands:
- chmod: to change permissions of files/directories
- chown: to change the owner of files/directories
- chgrp: change the group ownership of the directory
In simple term chown is used to change the ownership of a file while chmod is for changing the file mode bits.
- chown defines who owns the file.
- chmod defines who can do what.
When you make someone the owner of a file, (s)he can do almost wherever (s)he want to that file, for example (s)he can use chmod to changes its mods (say permissions) to define who can do what.
$ ls -l file
-rwxrwxr-x 2 ravexina admins 26 May 9 12:49 file
At the above line we can see that ravexina is the owner of the file and admins is the group. I can use: sudo chown dave:sudo file to change the owner of the file to dave and the group to sudo; Now the file belongs to “dave” and everyone in “sudo” group.
However with chmod we define who can do what? who has the right to read a file, write to a file or execute it. e.g:
chmod 777 file
gives the rights of read, write and execute to everyone including owner, group and everyones else.
The chgrp command can change the group that the file is associated from one group to another and thus give or remove access to a specific group of users.
The chown command does the same thing for the owner. It is a little more robust in that it is able to change not only the owner but the group as well making it a more versatile command.
If I want to change the permissions of a file then i will have to send the new permissions in this format and use – for the permissions which are not been given and this includes permissions for users, groups and others. Format: drwxrwxr-x
Also from the above table we can infer that the permissions read+write+execute = 7 (4+2+1). This would be for one of the users, hence to set of 3 user types of the files, we send 3 values.
So
–rwxrw-r–
above first – is for directory/file. then rest are just permissions of each user type, the Absolute(Numeric) value will be
rwx : 7 (user)
rw- : 6 (group)
r– : 4 (others)
so to change permissions, we can try
$ chmod 764 sample.txt
or
$ chmod –rwxrw-r– sample.txt
good reference:
use below command to show all the files currently active permissions
$Ls -l
Disk Listing on linux
$ fdisk -l
Serial Ports list
Also called communication ports on windows
$ ls -l /dev/ttys*
crw-rw-rw- 1 root wheel 4, 48 Mar 15 12:53 /dev/ttys0
crw–w—- 1 mishrjm tty 16, 0 Mar 15 13:29 /dev/ttys000
crw-rw-rw- 1 root wheel 4, 49 Mar 15 12:53 /dev/ttys1
Maximum file length on Linux : 255 characters and path 4096 characters
The files starting from . are ie they are prefixed with . (dot) are hidden file / directory
In the case of Windows, Docker uses Hyper-V which is in-built virtualisation technology provided by Windows. Docker uses Hypervisor framework in the case of MacOs for virtualization
Creating Files and Folders in Linux
To create a directory :
$ mkdir my_directory_name
To create a file:
vi cat
vim
gedit
nano
To read contents of a file:
cat
vi
vim
gedit
pico
nano
What are environment variables:
Environment variables are used to pass information into the processes that are spawned from the shell. Type $ env to get all the environment variables or write $ path to get all the paths
Redirection In linux ‘>’
‘>’ is the symbol used to for redirection on linux. So one command output can be sent to another command input.
ex: w > my_log
‘w’ will show current sessions on your operating system. The above will send this info in my_log file and its content would be:
USER TTY FROM LOGIN@ IDLE WHAT
mishrjm console – 12:54 56 –
mishrjm s000 – 13:24 – w
Now since ‘>’ will directly write to a file, if you want to append, then use ‘>>’ instead to append an output to end of a file
ex: $ cat>> file3
Pipes in Linux
Pipes are used to send the output of one program to another program while redirection is used to send the output of one program to some file/stream.
Pipe is used to pass output to another program or utility.
Redirect is used to pass output to either a file or stream.
Example: thing1 > thing2 vs thing1 | thing2
thing1 > thing2
- Your shell will run the program named thing1
- Everything that thing1 outputs will be placed in a file called thing2. (Note – if thing2 exists, it will be overwritten)
If you want to pass the output from program thing1 to a program called thing2, you could do the following:
thing1 > temp_file && thing2 < temp_file
which would
- run program named thing1
- save the output into a file named temp_file
- run program named thing2, pretending that the person at the keyboard typed the contents of temp_file as the input.
However, that’s clunky, so they made pipes as a simpler way to do that. thing1 | thing2 does the same thing as thing1 > temp_file && thing2 < temp_file
EDIT to provide more details to question in comment:
If > tried to be both “pass to program” and “write to file”, it could cause problems in both directions.
First example: You are trying to write to a file. There already exists a file with that name that you wish to overwrite. However, the file is executable. Presumably, it would try to execute this file, passing the input. You’d have to do something like write the output to a new filename, then rename the file.
Second example: As Florian Diesch pointed out, what if there’s another command elsewhere in the system with the same name (that is in the execute path). If you intended to make a file with that name in your current folder, you’d be stuck.
Thirdly: if you mis-type a command, it wouldn’t warn you that the command doesn’t exist. Right now, if you type ls | gerp log.txt it will tell you bash: gerp: command not found. If > meant both, it would simply create a new file for you (then warn it doesn’t know what to do with log.txt).
GREP
Used to search a particular word or string in a file. Also supports pattern based searching.
Ex. let file trying.rtf have following content
______________________________________
1
123
a
a
a
bb
c
trying this instead
of normal tries
Asc
Ascac
Xc
and this should work
______________________________________
- Now to check for a particular string/ word . lets try to search of ‘this’
$ grep this trying.rtf
output:
trying this instead\
and this should work\
** had some geberish before second line (\cf2 \outl0\strokewidth0 and this should work\)
- To check at what number line those string were found
$ grep this trying.rtf
output:
18:trying this instead\
24:\cf2 \outl0\strokewidth0 and this should work\
- To find the total number of occurrences of a particular word/string
$ grep -c this trying.rtf
output:
2
- To find the occurrence of a particular string in all the .txt file
XARGS (extra arguments feature)
the above (4. ) scenario can also be achieved using ‘find’ and ‘xargs’
ex:
—————
find / -name .txt | xargs grep -i abcd
find / -name .txt : this part looks for all the files with .txt extension on the root directory ‘/’
xargs grep -i abcd: this will then accept the returned files names and then use ‘xargs’ to find the word abcd in those files.
See here how pipe is being used since the output from one process is being sent as input to another directly. if we would have used a director here ‘>’, in that that we would first have to send the output of first command to a file and then use redirector again to use that file as inout to the second command to find the string.
$ find . -name ‘*.c’ | grep ‘stdlib.h’
This pipes the output (stdout)* from find to (stdin of)* grep ‘stdlib.h’ as text (ie the filenames are treated as text). grep does its usual thing and finds the matching lines in this text (any file names which themselves contain the pattern). The contents of the files are never read.
$ find . -name ‘*.c’ | xargs grep ‘stdlib.h’
This constructs a command grep ‘stdlib.h’ to which each result from find is an argument – so this will look for matches inside each file found by find (xargs can be thought of as turning its stdin into arguments to the given commands)*
Use -type f in your find command, or you will get errors from grep for matching directories. Also, if the filenames have spaces, xargs will screw up badly, so use the null separator by adding -print0 and xargs -0 for more reliable results:
find . -type f -name ‘*.c’ -print0 | xargs -0 grep ‘stdlib.h’
$ grep abcd *.txt
this will find the world “abcd” in all the files that has .txt extension
We can then use -c to count to the total number of occurrence of the word abcd in all the .txt files
ex: $ grep -c abcd *.txt
output:
config.txt:22
file1.txt:12
file2.txt:13
Gives file names and the total number of times that particular word was present
To terminate a process on linux
Kill command is used along with the process name or process ID (PID) to kill a command and used $ kill 0 to kill all command currently running.
To insert comment on command prompt
Use # to inset comment
Command Chaining: Use multiple command in a single command
Use ; (semicolor) between commands and write them in a single line with semicolons between the commands
ex:
$ mkdir new_folder ; cd new_folder ; touch file1 file2 file3 ; ls
output:
file1 file2 file3
To check status of a process on linux
$ ps aux
output
USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND
_coreaudiod 253 11.2 0.2 4400504 18148 ?? Ss 12:53PM 6:57.48 /usr/sbin/coreau
mishrjm 1173 8.6 2.4 6571976 201444 ?? S 12:54PM 11:04.11 c
To Login as root on linux
$ su –
or
$ sudo su –
then enter the root user password to login as enter user. After this the $ sign changes to # because for all other users, the sign used is $ however for root # is used.
So now commands will not be $ ls but rather # ls
To run a program in the backgroup
use NOHUP command for this to place the process in background
Which demon is used to track events on linux
syslogd daemon is used to track events on linux
what is Inode
while the contents of a file are stored on the “data block” the metadata of that file is stored in “Inode”
To increase the priority of a program on linux
Use command Nice or Renice . lesser the nice value, higher the priority and lesser the time taken to complete the job
highest = -20
least prior: 19
check the highest is negative, so lesser the nice value, greater the priority
While Nice will launch a new process with user defined priority, Renice will reset the priority of an ongoing process
To change nice value, use
$ renice
ex: normal command $ cat > file.txt
Here the nice value will be the default ie 0
with nice:
nice –10 cat>file.txt
two – – because one is hyphen and the other is negative value
here nice value will be -10
check the nice value using
$ ps -fl
or ps -fl cat
To renice:
$ renice -n -15 -p processname/processIDhere
this will change the nice value of process “processname/processIDhere” to +15
Shadow Password
All user passwords are in /etc/passwd however using shadow password will encrypt the all the stored password in the path
Listening Ports on linux
You can get a list of the listening ports on your system by querying the network stack with commands such as ss, netstat or lsof. Each listening port can be open or closed (filtered) using a firewall.
ex: $ netstat
Proto Recv-Q Send-Q Local Address Foreign Address (state)
tcp4 0 0 freeip.amazon.co.55629 ec2-3-227-250-19.https ESTABLISHED
tcp4 0 0 freeip.amazon.co.55627 ig-in-f188.1e100.https ESTABLISHED
tcp4 53 0 freeip.amazon.co.55626 ec2-3-91-171-132.https ESTABLISHED
tcp4 0 0 freeip.amazon.co.55624 104.19.155.59.https ESTABLISHED
tcp4 0 0 localhost.acme localhost.55620 CLOSE_WAIT
tcp4 0 0 localhost.55620 localhost.acme FIN_WAIT_2
tcp4 53 0 freeip.amazon.co.55613 monitor-api-publ.https CLOSE_WAIT