Here are some command you can use on your system terminal
pwd: print working directory
pwd
This prints the absolute path of your current working directory,which is where the shell runs commands and looks for files by default.
How else can I identify files and directories?
ls lists the contents of your current directory (the one displayed by pwd)
ls /home/repl
Shows all the files and folder in this directory.
###How can I move to another directory? cd change directory
cd seasonal
changes the current directory to seasonal. To confirm this change, you can type pwd
How can I move up a directory?
If you would like to go to the root directory from home/repl/seasonal The command ~/../. takes you to the /home directory.
How can I copy files?
cp seasonal/summer.csv backup/summer.bck This is used to copy file from one directory to another.
cp seasonal/summer.csv backup/summer.bck
You can also copy multiple file using;
cp seasonal/spring.csv seasonal/summer.csv backup
How can I move a file?
mv is used to move file from one directory to another just as cp copies file.
mv seasonal/spring.csv seasonal/summer.csv backup/
spring.csv and summer.csv were moved from seasonal folder to backup folder.
How can I rename files?
mv can also be used to rename files in the same directory.
mv winter.csv winter.csv.bck
###How can I delete files? rm is used to delete unwanted files
rm autumn.csv
deletes autumn.csv from this directory.
How can I create and delete directories?
rmdir is used to remove an empty directory while mkdir creates a new directory
rmdir people
removes people directory
mkdir year
creates a directory named year.
How can I view a file's contents?
cat course.txt
Puts course.txt in read mode.
How can I view a file's contents piece by piece?
As much as you can use cat to view the content of a file. It is only readable if there are few lines. To read a file in fewer lines , use less;
less seasonal/spring.csv seasonal/summer.csv
This allows you to view those two files in that order. Press spacebar to page down, :n to go to the second file, and :q to quit.
How can I look at the start of a file?
To check the first few lines of a file;
head seasonal/summer.csv
How can I control what commands do?
To display the number of lines displayed;
head -n 5 seasonal/winter.csv
How can I list everything below a directory?
In order to see everything underneath a directory, no matter how deeply nested it is, you can give ls the flag -R (which means "recursive").
ls -R -F ~
The above command list out all the directories in the home directory.
How can I get help for a command?
man head
#or
man tail
How can I select columns from a file?
cut -d , -f 1 seasonal/spring.csv
cut -d, -f1 seasonal/spring.csv
Both of these will display the values in the first column of the dataset.
How can I repeat commands?
Here is my first command
head summer.csv
which displayed an error because I was not in the right directory. So, I changed my directory using cd seasonal
cd seasonal
To repeat the command I used formally;
!head
This will execute the head summer.csv To check out the list of commands you have executed. Run history
history
This will display the list of commands you have executed in their order. You can choose to repeat anyone of it by;
!head 1
Run !head and the command number you want re-executed.
How can I select lines containing specific values?
head and tail select rows, cut selects columns, and grep selects lines according to what they contain. In its simplest form, grep takes a piece of text followed by one or more filenames and prints all of the lines in those files that contain that text. For example, grep bicuspid seasonal/winter.csv prints lines from winter.csv that contain "bicuspid".
grep can search for patterns as well; we will explore those in the next course. What's more important right now is some of grep's more common flags:
-c: print a count of matching lines rather than the lines themselves
-h: do not print the names of files when searching multiple files
-i: ignore case (e.g., treat "Regression" and "regression" as matches)
-l: print the names of files that contain matches, not the matches
-n: print line numbers for matching lines
-v: invert the match, i.e., only show lines that don't match
grep -n -v molar seasonal/spring.csv
The above command shows numbered rows in spring.csv that are not molar
How can I store a command's output in a file?
tail -n 5 seasonal/winter.csv > last.csv
tail -n 5 displays the last 5 rows in winter.csv > last.csv copies the displayed rows into the filename last.csv
What's a better way to combine commands?
Using redirection to combine commands has two drawbacks:
It leaves a lot of intermediate files lying around (like top.csv).
The commands to produce your final result are scattered across several lines of history.
The shell provides another tool that solves both of these problems at once called a pipe. Once again, start by running head:
head -n 5 seasonal/summer.csv
Instead of sending head's output to a file, add a vertical bar and the tail command without a filename:
head -n 5 seasonal/summer.csv | tail -n 3
The pipe symbol tells the shell to use the output of the command on the left as the input to the command on the right. Use cut to select all of the tooth names from column 2 of the comma delimited file seasonal/summer.csv, then pipe the result to grep, with an inverted match, to exclude the header line containing the word "Tooth"
cut -d , -f 2 seasonal/summer.csv | grep -v Tooth
How can I combine many commands?
You can chain any number of commands together. For example, this command:
cut -d , -f 1 seasonal/spring.csv | grep -v Date | head -n 10
will:
- select the first column from the spring data;
- remove the header line containing the word "Date"; and
- select the first 10 lines of actual data.
How can I count the records in a file?
The command wc (short for "word count") prints the number of characters, words, and lines in a file. You can make it print only one of these using -c, -w, or -l respectively. Use case; Count how many records in seasonal/spring.csv have dates in July 2017 (2017-07).
grep 2017-07 seasonal/spring.csv | wc -l
grep -c 2017-07 seasonal/spring.csv
How can I specify many files at once?
For example, you can get the first column from all of the seasonal data files at once like this;
cut -d , -f 1 seasonal/winter.csv seasonal/spring.csv seasonal/summer.csv seasonal/autumn.csv
#or
cut -d , -f 1 seasonal/*
#or
cut -d , -f 1 seasonal/*.csv
Write a single command using head to get the first three lines from both seasonal/spring.csv and seasonal/summer.csv, a total of six lines of data, but not from the autumn or winter data files.
head -n 3 seasonal/s*
This becomes even more important if your directory contains hundreds or thousands of files.
What other wildcards can I use?
The shell has other wildcards as well, though they are less commonly used:
- ? matches a single character, so 201?.txt will match 2017.txt or 2018.txt, but not 2017-01.txt.
- [...] matches any one of the characters inside the square brackets, so 201[78].txt matches 2017.txt or 2018.txt, but not 2016.txt.
- {...} matches any of the comma-separated patterns inside the curly brackets, so {.txt, .csv} matches any file whose name ends with .txt or .csv, but not files whose names end with .pdf.
How can I sort lines of text?
As its name suggests, sort puts data in order. By default it does this in ascending alphabetical order, but the flags -n and -r can be used to sort numerically and reverse the order of its output, while -b tells it to ignore leading blanks and -f tells it to fold case (i.e., be case-insensitive). Pipelines often use grep to get rid of unwanted records and then sort to put the remaining records in order.
cut -d , -f 2 seasonal/winter.csv | grep -v Tooth |sort -r
The combination of cut and grep to select all the tooth names from column 2 of seasonal/winter.csv and lastly sorts the order in descending order.
How can I remove duplicate lines?
Another command that is often used with sort is uniq, whose job is to remove duplicated lines. More specifically, it removes adjacent duplicated lines. If a file contains:
2017-07-03
2017-07-03
2017-08-03
2017-08-03
then uniq will produce:
2017-07-03
2017-08-03
but if it contains:
2017-07-03
2017-08-03
2017-07-03
2017-08-03
then uniq will print all four lines. The reason is that uniq is built to work with very large files. In order to remove non-adjacent lines from a file, it would have to keep the whole file in memory (or at least, all the unique lines seen so far). By only removing adjacent duplicates, it only has to keep the most recent unique line in memory.
cut -d , -f 2 seasonal/winter.csv | grep -v Tooth | sort |uniq -c
The above command does the following;
- get the second column from seasonal/winter.csv,
- removes the word "Tooth" from the output so that only tooth names are displayed,
- sort the output so that all occurrences of a particular tooth name are adjacent; and
- display each tooth name once along with a count of how often it occurs.
How can I save the output of a pipe?
cut -d , -f 2 seasonal/*.csv | grep -v Tooth > teeth-only.txt
How can I stop a running program?
Ctrl + C
To build a pipeline to find out how many records are in the shortest of the seasonal data files.
wc -l seasonal/* | grep -v total | sort -n |head -n 1
wc -l seasonal/* displays list the number of lines in all of the seasonal data files. grep -v total remove the line containing the word "total". sort -n |head -n 1 sort -n and head -n 1 to find the file containing the fewest lines.
How does the shell store information?
To determine how many old commands are stored in your command history.
set |grep HISTFILESIZE
How can I print a variable's value?
A simpler way to find a variable's value is to use a command called echo, which prints its arguments.
echo hello World!
#returns hello World!
If you try to use it to print a variable's value like this:
echo $USER
#returns username
How else does the shell store information?
The other kind of variable is called a shell variable, which is like a local variable in a programming language. To create a shell variable, you simply assign a value to a name:
sample=seasonal/abc.csv
head -n 1 $sample
#will return the first line in the variable sample
How can I repeat a command many times?
for filetype in docx odt pdf; do echo $filetype; done
loops over docx out pdf with the iterator name #filetype.
for filename in people/*;do echo $filename; done
#return people/agarwal.txt which is the file that resides in people directory
How can I record the names of a set of files?
files=seasonal/*.csv
for f in $files; do echo $f; done
#returns:
seasonal/autumn.csv
seasonal/spring.csv
seasonal/summer.csv
seasonal/winter.csv
How can I run many commands in a single loop?
Printing filenames is useful for debugging, but the real purpose of loops is to do things with multiple files. This loop prints the second line of each data file:
for file in seasonal/*.csv; do head -n 2 $file | tail -n 1; done
To write a loop that prints the last entry from July 2017 (2017-07) in every seasonal file.
for file in seasonal/*.csv; do grep 2017-07 $file |tail -n 1; done
How can I edit a file?
- Ctrl + K: delete a line.
- Ctrl + U: un-delete a line.
- Ctrl + O: save the file ('O' stands for 'output'). You will also need to press Enter to confirm the filename!
- Ctrl + X: exit the editor.
nano names.txt
is used to create a new file.
How can I record what I just did?
history > tail -n 3 > steps.txt
The above command pipes the last 3 commands into the filename steps.txt.
How can I save commands to re-run later?
You can store commands in files for the shell to run over and over again.
nano dates.sh #creates the file in edit mode
cut -d , -f 1 seasonal/*.csv #the command you want to save in the file for execution
ctr + o #to save the file
ctr + x #to exit edit mode
bash dates.sh #to execute the file you just created.
How can I re-use pipes?
cut -d , -f 1 seasonal/*.csv | grep -v Date | sort | uniq
then
bash all-dates.sh > dates.out
The formal, stores the command in the all-dates.sh ,while the latter pipes the result into dates.out. To display the result of dates.out;
cat dates.out
How can I pass filenames to scripts?
A script that processes specific files is useful as a record of what you did, but one that allows you to process any files you want is more useful. To support this, you can use the special expression $@ (dollar sign immediately followed by at-sign) to mean "all of the command-line parameters given to the script".
For example, if unique-lines.sh contains sort $@ | uniq, when you run:
bash unique-lines.sh seasonal/summer.csv
the shell replaces $@ with seasonal/summer.csv and processes one file. If you run this:
bash unique-lines.sh seasonal/summer.csv seasonal/autumn.csv
it processes two data files, and so on.