The operating system

Author

Thomas H. Simm

import os
import sys

name of operating system

print('os.name',',',os.name,'\n')

os.name , nt

get current dir

print('os.getcwd()',',',os.getcwd(),'\n')

os.getcwd() , C:\Users\44781\pyproj\_misc

list files in dir

print('os.listdir()',',',os.listdir(),'\n')

os.listdir() , ['.ipynb_checkpoints', 'adapic.jpg', 'awarhol.jpg', 'awarhol.webp', 'dog.jpg', 'draw.png', 'ebsd2.tif', 'ebsdmap.png', 'ebsdmap.tif', 'fastAI_C1_notes.ipynb', 'image_1000.jpg', 'image_5000(1).jpg', 'IndeedExtract-Copy1.ipynb', 'IndeedExtract-Copy2.ipynb', 'No_61_Mark_Rothko-thumbnail_webp-9999x9999.webp', 'os.ipynb', 'output', 'PF_500C R.png', 'PythonBook.ipynb', 're.ipynb', 'styleTransfer.ipynb', 'test2', 'Untitled.ipynb', 'vangogh.jpg', 'vgg19_weights_tf_dim_ordering_tf_kernels_notop.h5', 'water.webp']

make a directory

this_dir=os.getcwd()
directory='test'
#this adds either / or \ depending on os
path = os.path.join(this_dir, directory)

try:
    os.mkdir(path)
except:
    pass

#or 
try:
    os.mkdir('test2')
except:
    pass

list files in dir

print('os.listdir()',',',os.listdir(),'\n')

os.listdir() , ['.ipynb_checkpoints', 'adapic.jpg', 'awarhol.jpg', 'awarhol.webp', 'dog.jpg', 'draw.png', 'ebsd2.tif', 'ebsdmap.png', 'ebsdmap.tif', 'fastAI_C1_notes.ipynb', 'image_1000.jpg', 'image_5000(1).jpg', 'IndeedExtract-Copy1.ipynb', 'IndeedExtract-Copy2.ipynb', 'No_61_Mark_Rothko-thumbnail_webp-9999x9999.webp', 'os.ipynb', 'output', 'PF_500C R.png', 'PythonBook.ipynb', 're.ipynb', 'styleTransfer.ipynb', 'test', 'test2', 'Untitled.ipynb', 'vangogh.jpg', 'vgg19_weights_tf_dim_ordering_tf_kernels_notop.h5', 'water.webp']

Check file/dir exists

print(os.path.exists("test2"),os.path.exists("test"))

True True

change directory

os.chdir(directory)
print('os.getcwd()',',',os.getcwd(),'\n')

os.getcwd() , C:\Users\44781\pyproj\_misc\test

Create and Rename a file

# create a file 'a' is append
open('Old.txt','a').close()

# check file exists 
print('old=',os.path.exists("Old.txt"),'. new=',os.path.exists("New.txt"))

# rename a file
fd = "Old.txt"
os.rename(fd,'New.txt')

# check file exists 
print('old=',os.path.exists("Old.txt"),'. new=',os.path.exists("New.txt"))

old= True . new= False
old= False . new= True

Copy a file-

can be done in os but easier in shutil

https://stackabuse.com/how-to-copy-a-file-in-python/

import shutil
shutil.copyfile('New.txt', 'Old.txt')

# check file exists 
print('old=',os.path.exists("Old.txt"),'. new=',os.path.exists("New.txt"))

old= True . new= True

Remove a file

# remove a file
os.remove("New.txt")
# os.remove("Old.txt")

# check file exists 
print('old=',os.path.exists("Old.txt"),'. new=',os.path.exists("New.txt"))

old= True . new= False

Go back up in directory


os.path.dirname(os.path.dirname(   ))

'C:\\Users\\44781\\pyproj'

#remove directory
os.rmdir(path)
#list files in dir
print('os.listdir()',',',os.listdir(),'\n')

os.listdir() , ['.ipynb_checkpoints', 'adapic.jpg', 'awarhol.jpg', 'awarhol.webp', 'dog.jpg', 'draw.png', 'ebsd2.tif', 'ebsdmap.png', 'ebsdmap.tif', 'fastAI_C1_notes.ipynb', 'image_1000.jpg', 'image_5000(1).jpg', 'IndeedExtract-Copy1.ipynb', 'IndeedExtract-Copy2.ipynb', 'No_61_Mark_Rothko-thumbnail_webp-9999x9999.webp', 'os.ipynb', 'output', 'PF_500C R.png', 'PythonBook.ipynb', 're.ipynb', 'styleTransfer.ipynb', 'test2', 'Untitled.ipynb', 'vangogh.jpg', 'vgg19_weights_tf_dim_ordering_tf_kernels_notop.h5', 'water.webp']

Create new files

#create new file and edit
nano file.txt
#create file
touch file.txt

!cd

C:\Users\44781\pyproj\_misc\test

Subprocess module


import subprocess

# subprocess.run("date",shell=True)
print(subprocess.run(['cmd', '/c', 'date']))

CompletedProcess(args=['cmd', '/c', 'date'], returncode=1)

import sys

Object `mv` not found.

Pointers for Getting Your Environment Setup

Learning more about operating systems

We’ve talked briefly about what an operating system is and what we’ll need to know about operating systems for this course. If you want to learn some additional operating system concepts, check out the videos on this subject in the Technical Support Fundamentals course (https://www.coursera.org/lecture/technical-support-fundamentals/module-introduction-I3n9l). If you want to dive deeper onto how to manage Windows and Linux, check out the Operating Systems and You: Becoming a Power User course (https://www.coursera.org/learn/os-power-user).

If you want to discover more about the history of Unix, you can read all the details on the Unix Wikipedia page https://en.wikipedia.org/wiki/History_of_Unix.

Installing Python and additional modules

If you don’t have Python installed yet, we recommend that you visit the official Python website (http://www.python.org/) and download the installer that corresponds to your operating system.

There’s a bunch of guides out there for installing Python and they all follow a similar process to the one we described in the videos. This guide from Real Python (https://realpython.com/installing-python/) includes instructions on how to install python on a range of different operating systems and distributions.

Once you have Python installed on your operating system, it’s a good idea to familiarize yourself with pip and the associated tools. You can find more info about these here https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/.

Using package management systems

Package management systems help you better manage the software installed on your machine. These management systems vary a lot from operating system to operating system. So, you need to pick the one that works for the OS you’re using. Check out these guides for help with this:

Installing Python 3 on Windows 10 with Chocolatey https://www.digitalocean.com/community/tutorials/how-to-install-python-3-and-set-up-a-local-programming-environment-on-windows-10
Installing Python 3 on MacOS with Homebrew http://www.pyladies.com/blog/Get-Your-Mac-Ready-for-Python-Programming/
Package management basics on Linux https://www.digitalocean.com/community/tutorials/package-management-basics-apt-yum-dnf-pkg

Other information

Python in the Microsoft Store for Windows 10 https://devblogs.microsoft.com/python/python-in-the-windows-10-may-2019-update/

Setting up Your Environment

After you’ve installed Python and checked that it works, the next step to set up your developer environment is to choose your main code editor.

These are some of the common editors for Python, available for all platforms:

Atom https://atom.io/
Eclipse https://www.eclipse.org/
PyCharm https://www.jetbrains.com/pycharm/
Sublime Text http://www.sublimetext.com/
Visual Studio Code https://code.visualstudio.com/

You can read more about these editors, and others, in these overview comparatives:

Python IDEs and Code Editors (Guide) https://realpython.com/python-ides-code-editors-guide/#pycharm
Best Python IDEs and Code Editors https://www.softwaretestinghelp.com/python-ide-code-editors/
Top 5 Python IDEs for Data Science https://www.datacamp.com/community/tutorials/data-science-python-ide

We encourage you to try out these editors and pick your favorite. Then, install it on your computer and experiment with writing and executing Python scripts locally.

Reading and Writing Files Cheat-Sheet

Check out the following link for more information:

https://docs.python.org/3/library/functions.html#open

Files and Directories Cheat-Sheet

Check out the following links for more information:

[https://docs.python.org/3/library/os.html]https://docs.python.org/3/library/os.html)
https://docs.python.org/3/library/os.path.html
https://en.wikipedia.org/wiki/Unix_time

CSV Files Cheat Sheet

Check out the following links for more information:

Shell Commands

`echo`

echo HELLO print HELLO to screen
echo $string1 print variable string1

maths operations

echo $(( 10 + 5 )) add two numbers

`cat`

cat [file] command allows us to create single or multiple files, view the contents of a file, concatenate files, and redirect output in terminal or other files.

`grep`

Grep command, which stands for “global regular expression print”, processes text line-by-line and prints any lines that match a specified pattern.

grep [pattern] [file-directory/location]
e.g. grep "jane" list.txt find the occurances of “jane” in list.txt

Here, [file-directory] is the path to the directory/folder where you want to perform a search operation. The grep command is also used to search text and match a string or pattern within a file.

`cut`

cut [options] [file] The cut command extracts a given number of characters or columns from a file. A delimiter is a character or set of characters that separate text strings.
For delimiter separated fields, the - d option is used. The -f option specifies the field, a set of fields, or a range of fields to be extracted. cut -d [delimiter] -f [field number]

`cat`

cat > [file]
Each stream uses redirection commands. A single greater than sign (>) or a double greater than sign (>>) can be used to redirect standard output. If the target file doesn’t exist, a new file with the same name will be created.
cat >> [file]
Commands with a double greater than sign (>>) do not overwrite the existing file content, but it will append to it.
> can be used to create a file
e.g. > test.txt
creates the file test.txt
echo "I am appending text to this test file" >> test.txt
to append to the same file

`test`

tests an assertion can be replaced with []

`nano`

Edit a file / create a file and edit it

for a shell script - #!/bin/bash

And for a python script - #!/usr/bin/env python3

`chmod`

chmod +x findJane.sh

`for` statement

for i in 1 2 3; do echo $i; done
the key elements to note are ; do and done

Examples

cat list.txt

001 jane /data/jane_profile_07272018.doc
002 kwood /data/kwood_profile_04022017.doc
003 pchow /data/pchow_profile_05152019.doc
004 janez /data/janez_profile_11042019.doc
005 jane /data/jane_pic_07282018.jpg
006 kwood /data/kwood_pic_04032017.jpg
007 pchow /data/pchow_pic_05162019.jpg
008 jane /data/jane_contact_07292018.csv
009 kwood /data/kwood_contact_04042017.csv
010 pchow /data/pchow_contact_05172019.csv

cat list.txt | cut -d ' ' -f 2

jane
kwood
pchow
janez
jane
kwood
pchow
jane
kwood
pchow

here | is a pipe to connect commands
each line is cut by delimiter space ’ ’ then the 2nd term is outputted. IF we put 1 instead we get 001, 002 etc
or f 1,3 would get 1st and 3rd parts after split by space

if test -e ~/data/jane_profile_07272018.doc; then echo "File exists"; else echo "File doesn't exist"; fi

the use of e tests if a file exists

# For statement print numbers 1 to 99 in steps of 1
for i in {1..99..1}
do 
  echo $i
done

# For and If statements with math operation
for i in {1..99..1}
do 
  if [ $(( i % 2 )) == 1 ]
  then
    echo $i
  fi
done

#!/bin/bash
> oldFiles.txt

files="$(grep " jane " ../data/list.txt | cut -d " " -f 3)"

for file in $files;
  do
   if test -e ".."$file;
   then echo $file >> oldFiles.txt;
   else echo "no" $file;
   fi

done

Bash Scripting Resources

Check out the following links for more information:

name of operating system

get current dir

list files in dir

make a directory

list files in dir

Check file/dir exists

change directory

Create and Rename a file

Copy a file-

can be done in os but easier in shutil

Remove a file

Go back up in directory

Create new files

Subprocess module

Pointers for Getting Your Environment Setup

Learning more about operating systems

Installing Python and additional modules

Using package management systems

Setting up Your Environment

Reading and Writing Files Cheat-Sheet

Files and Directories Cheat-Sheet

CSV Files Cheat Sheet

Shell Commands

echo

maths operations

cat

grep

cut

cat

test

nano

chmod

for statement

`echo`

`cat`

`grep`

`cut`

`cat`

`test`

`nano`

`chmod`

`for` statement