2-3: Basic Linux Shell Commands

The best way to follow this discussion is to sit in front of a Linux Terminal and try the examples while you read. Turn on your machine, fire up a shell and put your hands on the keyboard, NOW ;-).

If you have read the previous section of this book, you are now familiar with at least three Linux shell commands, “pwd”, “cd” and “ls”. You have learned that commands can, sometimes optionally, take an argument (ls /etc) and also some options (ls -la /etc). The options are usually preceded by a minus “-” symbol (-la).

While some commands are extremely simple and linear to use, the addition of options and arguments can turn them into powerful and flexible tools.

An in-depth, nice and clean guide of the main linux commands is available online thanks to the Linux Information Project Linux Commands pages.

We will list here the main Linux commands in what we consider a logical order, rather than alphabetically.

man

Linux commands are generally documented in detail. If you want to know everything about the arguments and the options for a particular command you can use the “man” command, that takes as an argument the name of the command on which you wish to have information. Let’s use it to get information on the “ls” list command:

andrea@ubuntu:~$ man ls

The man page for the "ls" command
Figure 2-3-1: The man page for the “ls” command

On man pages, you can press the space bar to read more, if all the contents do not fit into your current view of the Terminal window. Press “q” to exit the man page and go back to the prompt.

touch

andrea@ubuntu:~$ touch my_sequence.fasta

You can use the “touch” command to create one or more files on the fly. In the example above you create a file called “my_sequence.fasta” in the current directory. If the current directory contains a subdirectory called “sequences” you could create the file inside this subdirectory, by providing the relative path of the file to be created as “touch” argument:

andrea@ubuntu:~$ touch sequences/my_sequence.fasta

You can create several files at once:
andrea@ubuntu:~$ touch my_sequence1.fasta my_sequence2.fasta my_sequence3.fasta …

You can also use touch to manipulate timestamps of existing files or new files, but this is beyond the scope of this course. Learn more: “touch” on Linux Info

mkdir

if the “sequences” directory does not exist, you can create it with the “mkdir” Make Directory shell command.

andrea@ubuntu:~$ mkdir sequences

You can create several directories at once:

andrea@ubuntu:~$ mkdir sequences1 sequences2 …

With the “m” option you can assign a specific permission at creation time

andrea@ubuntu:~$ mkdir -m 777 sequences

You can create a directory inside other directories that still do not exist, thereby creating a full directory hierarchy with a single command, with the “p” option:

andrea@ubuntu:~$ mkdir -p bioinfo/sequences/processed/dna

rm and rmdir

The “rm” remove shell command can be used to delete files. It can also delete directories (with the -r option) and be used as an alternative to the more specific “rmdir” remove directory  command for deleting directories. “rm” deletes the files without asking for a confirmation.

andrea@ubuntu:~$ touch mysequence.fasta # creating mysequence.fasta

andrea@ubuntu:~$ rm mysequence.fasta # deleting mysequence.fasta

In order to delete a directory, even if not empty, you can use “rm” with the “rf” options:

andrea@ubuntu:~$ mkdir sequences

andrea@ubuntu:~$ touch sequences/a-file.txt

andrea@ubuntu:~$ rmdir sequences # Using the “rmdir” command on a directory which is not empty will return an error. We can use “rm -rf” instead, which will silently remove the directory and all it’s contents, with no complains. See figure 2-3-2 for an example.

andrea@ubuntu:~$ rm -rf sequences

Exhaustive information on the use of the “rm” and “rmdir” commands can be found here:

“rm” on Linux Info
“rmdir” on Linux Info

cp

The Linux “cp” shell command copy can make a copy of a file.

It accepts 2 arguments. The first is the path of file to be copied. The second can be either a file name/path or the name of an existing directory. Depending on the second argument, the result is different. Let’s see a couple of examples.

andrea@ubuntu:~$ copy file1.txt file2.txt # the second argument is a file name

will make a copy of “file1.txt” and save it with the name “file2.txt”.

andrea@ubuntu:~$ copy file1.txt .. # the second argument is a directory name

will make a copy of “file1.txt”, with the same exact name, to the parent (..) of the present working directory

mv

The “mv” move shell command can either rename a file (by file we always mean any kind of file, including directories) or move a file to a different directory, depending on whether the second argument is a file name or an existing directory. The first argument is the path of the file to be renamed or moved. Let us go through a couple of examples (see also figure 2-3-2).

andrea@ubuntu:~$ mv file1.txt file2.txt # the second argument is a file name

this will rename file1.txt to file2.txt

andrea@ubuntu:~$ mv file1.txt sequences/dna # the second argument is a directory

this will move “file1.txt” from the current directory to the sequences/dna directory

andrea@ubuntu:~$ mv * .. # the second argument is a directory

moves all files of the present working directory up one level, to the parent directory. Indeed, the wildcard character * will match all filenames.

Using Linux shell commands
Figure 2-3-2: Using Linux shell commands

sudo

The “sudo” (super user do)shell command is required to temporarily acquire root privileges on a Linux machine. The root user have the power of performing any action, the normal permission restrictions do not apply to the root user.

For example, the modification of any Linux system file, including configuration files in the /etc folder require root privileges. Root privileges are also require to stop, start, restart system processes and services.

Restarting the apache web server requires root access, which can be acquired by prefixing the “sudo” command to the command to be issued as root.

andrea@ubuntu:~$ service apache2 restart

will give an authorization error (you need Apache installed before you try this). Also trying to create a file in the root folder will give an error, as normal users do not have the permissions for this. Try:

andrea@ubuntu:~$ touch /test.txt

You instead need to use sudo for these operations. Not all users on a machine are “sudoers”. Only the system administrator have these privileges, or other users that were selected and enabled as sudoers by the system administrator. We’ll see how to grant a user sudo privileges to a user in a moment. By default, when you install Ubuntu you will create a default user who indeed has administrator privileges.

When you issue a sudo command for the first time during a session, you will be asked to type your password again, for added security. Subsequent sudo commands will not require a password for a few minutes, after which your magical powers vanish and you have to enter the password again. The default sudo timeout in Ubuntu is 15 minutes.

Back to the examples above, the correct commands are:

andrea@ubuntu:~$ sudo service apache2 restart

and

andrea@ubuntu:~$ sudo touch /test.txt

A relevant shell session is shown in figure 2-3-3

Using sudo to acquire root privileges
Figure 2-3-3: Using sudo to acquire root privileges

adduser and userdel

You can add users to your machine with the “adduser” shell command. This requires the powers of the system administrator and therefore, sudo:

andrea@ubuntu:~$ sudo adduser carl

In the process you will be required to type a password for the user (“Enter new UNIX password:”) and then to confirm it (“Retype new UNIX password:”). You will also be prompted to enter some standard user information (full name, room number …), that you can leave blank if you wish (see session in figure 2-3-4).

When the user is created, an home folder for the user will also be created within the /home folder at /home/newusername.

If you want to grant carl administration privileges (that is if you want to add carl to the sudoers list) you can then type, after the user was already created (otherwise you will get an error message complaining that the user does not exist):

andrea@ubuntu:~$ sudo adduser carl admin

This will add carl to the sudoers (administrators) group.

You can delete a user with the “userdel” shell command. This will not delete the user home folder though. You can then delete the user home folder, if you wish, with a “rm -rf” command.

andrea@ubuntu:~$ sudo userdel carl

andrea@ubuntu:~$ sudo rm -rf /home/carl

Creating and deleting a user with adduser and userdel
Figure 2-3-4: Creating and deleting a user with adduser and userdel

In another sub-chapter we will see how to configure apache and the userdir apache module so that each user on your machine can have it’s own web space to be used for study and applications/web development purposes.

chmod

The “chmod” shell command allows to change the permissions for a file.

andrea@ubuntu:~$ chmod 666 textfile.txt

chmod can be used with a different, non numerical syntax aswell. For example, to add write permission for all (owner, group, others) you can type:

andrea@ubuntu:~$ chmod +w textfile.txt

to add write and execute permission for all:

andrea@ubuntu:~$ chmod +wx textfile.txt

Granting execution privileges to all files ending in .cgi contained in the present working directory:

andrea@ubuntu:~$ chmod +x *.cgi

Granting write privileges, in the present working directory, to all files and directories, and the directory contents, by using the recursive option “R” (capital R):

andrea@ubuntu:~$ chmod -R +w *

You can type:

andrea@ubuntu:~$ man chmod

to access the chmod man page and all the information on the available options for the command.

chown

With the “chown” shell command you can change the owner of a file:

andrea@ubuntu:~$ sudo chown carl textfile.txt

You can also use chown to change the group information, but this is beyond the scope of this course.

Additional resources:

chown examples
chown page on Linux Info

apt-get and apt: installing and updating software

For security and system stability reasons, it is a wise idea, once you have installed Linux on a machine, to keep the OS up to date. On Debian derived systems, such as Ubuntu, this can be done by using the “apt-get” shell command.

In Ubuntu 16.04, Canonical, the company that develops Ubuntu, introduced an alternative more user friendly command to apt-get simply called apt. Rather that a shortcut to apt-get, apt is an entirely new command with a different underlying code, although the usage is largely the same as apt-get. In the examples below, you can replace apt-get with apt. Still, there are some subtle differences in the use of apt vs apt-get. You can read more in this article and also here and here.

A typical usage of apt-get for applying available software updates to an Ubuntu system is as follows:

andrea@ubuntu:~$ sudo apt-get update

Updates the list of the software repositories, from which updates are downloaded.

Followed by:

andrea@ubuntu:~$ sudo apt-get upgrade

Updates the system to the last software releases, including security updates that are essential to keep your system safe. Don’t do this regularly and it will only be a matter of time before your system is compromised/hacked.

You can use apt-get to upgrade to the latest Linux distribution.

andrea@ubuntu:~$ sudo apt-get dist-upgrade

apt-get can be used to install new software with the “install” option:

andrea@ubuntu:~$ sudo apt-get install openssh-server

will install the open SSH Server software in your machine. This is something we will need to do to turn our machine in a web server, the procedure will be described in the next sub-chapter.

You can also remove installed software with the “remove” option:

andrea@ubuntu:~$ sudo apt-get remove openssh-server

nano: a shell text editor

It is quite an essential task for a system administrator to be able to create and edit files right from a shell. This is essential to manage a system remotely, when a GUI (Graphical User Interface) is not available. More on remote shell connections in the next section of this chapter. If working locally, on a graphical environment, there is a wide choice for good code editors. While working on a shell, there are two editors that are mostly used: the nano text editor and the vi text editor.

While individual preference for nano or vi is a religious matter, with people advocating for one or the other divided in antagonizing factions, each with strong and indisputable arguments, it is not a bad idea to be familiar with both. The reason being that while vi is practically omnipresent in every Linux system, nano may sometimes not be available (note that it is available on Ubuntu by default). Therefore even if you routinely use nano, the ability to use vi as well provides you with a greater flexibility and reach on different systems as a developer or system administrator.

You may want to know that nano is basically a clone of another shell text editor called pico, with the difference that nano is licensed under GNU and it is therefore open to modifications and redistribution. For this reason it is also known as GNU nano.

We belong to the nano faction, that has a somewhat smoother learning curve than vi. So nano is the editor we will use during this course.

Typical usage of nano is with the “w” option (no wrap). Failing to use this option can seriously damage your code pages, any code edit should always be edited with the “w” option. The argument is a filepath. if the file already exists it will be opened for editing, otherwise the file will be created.

Let’s open a new file called myseq.fasta:

andrea@ubuntu:~$ nano -w myseq.fasta

Nano will open. We can then type some text and save by hitting Ctrl + o, and then hitting return to confirm the filename for the file. We can then exit from nano and get back to the shell prompt by hitting Ctrl + x. The most frequent keyboard shortcuts are listed in the bottom part of the nano window itself, for quick reference. In figure 2-3-5 you can see how the nano interface looks like. Concerning the shortcuts at the bottom of the screen, be aware that when you see ^C, it really means: Ctrl + c.

Useful shortcuts within nano are:

Ctrl + k

cuts out a line. You could then move the cursor elsewhere and paste back the line with

Ctrl + u

Searching for text inside the opened document:

Ctrl + w

Position the cursor at the beginning of a line:

Ctrl + a

At the end of a line:

Ctrl + e

Editing files with the nano shell text editor
Figure 2-3-5: Editing files with the nano shell text editor

Hello, World!: creating and executing a script

Now that we can create and edit files, let’s write a simple “Hello World!” script in python and run it.

Let’s first check if python is installed, and where, with the which command:

andrea@ubuntu:~$ which python

Python is installed by default on Ubuntu. You should get back something like:

/usr/bin/python

Let’s create the script file and open it for editing with nano, all in one command:

andrea@ubuntu:~$ nano -w helloworld.py

The very same command is valid for editing the script file, if it exists already.

We will then put a first line in the script, called shebang line, to point Linux to the correct interpreter for the code that follows. Since we will write in python, we must include the python path returned previously by the “which python” command:

#!/usr/bin/python

We can then enter python code on the second line:

print “Hello, World!”

The helloworld.py python script

<code>
#!/usr/bin/python
print “Hello, World!”
</code>

That’s it, we are done. We can save with Ctrl + o, press return to confirm the file name and then exit nano with Ctrl + x.

If we try to then run the script like this:

andrea@ubuntu:~$ ./helloworld.py

This will not work, because the file does not have execute permissions. Try it.

Let’s grant execute permission:

andrea@ubuntu:~$ chmod +x helloworld.py

Then we can execute it:

andrea@ubuntu:~$ ./helloworld.py

will print “Hello, World!” to the screen. Please note that in order for Linux to understand that we want to run the script file, rather than executing a command with the same name as the script, we have to use the ./ notation for the relative path, or use an absolute path. So we have to type:

andrea@ubuntu:~$ ./helloworld.py

or

andrea@ubuntu:~$ /home/andrea/helloworld.py

rather then:

andrea@ubuntu:~$ helloworld.py

that will raise an error about the command not existing.

Congratulations, you have just started your journey as an expert programmer, stick with us to build up your web development for bioinformatics programming skills!

It’s now time to move ahead and turn our newly installed Linux distro in a full fledged web server. We will start by installing an SSH Server and learning to connect remotely to our machine. Read on

Chapter Sections

[pagelist include=”63″]

[siblings]

One thought on “2-3: Basic Linux Shell Commands”

Leave a Reply

Your email address will not be published. Required fields are marked *