Python3 Scripting (Part 1?)

Discussion in 'Shell / Command Line' started by DevynCJohnson, Nov 3, 2013.

  1. DevynCJohnson

    DevynCJohnson Well-Known Member Staff Member Staff Writer

    Messages:
    1,336
    Likes Received:
    1,073
    Trophy Points:
    113
    Part 2 - http://www.linux.org/threads/python3-scripting-2.4987/

    Aloha! Many Linux users know at least some shell scripting due to its importance. However, many users are not aware of the importance of Python scripting in Linux. True, shell scripting is more important than Python scripting, but Python is a very useful scripting tool in Linux. This is a general tutorial for Python3. If I see that this tutorial gets at least 5,000 views in one week from posting, I receive many emails/posts requesting that I write a whole series, and/or you click "Contact Us" on the very bottom of this page and comment on how much you love this or all of my articles (remember to mention my name) and want more tutorials about Python3, then I will write a complete (or near complete) Python scripting guide on this website.

    NOTE: This tutorial (or series) will discuss Python3 not Python version 2. Version 2 and 3 are very similar. However, they differ in some areas such as available modules and syntax structure. I will solely focus on Python3. The terms “Python3” and “Python” will be used interchangeably.

    Python3 is a very powerful scripting language. It is almost as simple as shell scripting, but yet as powerful as C/C++ (or nearly so). Python can be compiled to bytecode or converted to C/C++ and then compiled. Python is very popular especially among Open Source Software fans. The plain text or compiled (bytecode) scripts are read by the Python interpreter and then executed. The script must have the executable bit set (set the permissions to allow the script to execute).

    To start writing a script, make a file with a name and end the filename with ".py", if you wish. It is not required that you do so. However, if you are making a script that you place in the system's PATH and wish to use the script in Unix shells like a command, do not add the extension. Otherwise, to use the command, the user would be typing "COMMAND.py" instead of "COMMAND".

    Open the empty file in your preferred text editor or IDE. On the first line, type

    Code:
    #!/usr/bin/env python3
    This is called a hashpling. It allows the system to identify the file as a Python script and tells the system what interpreter to use for execution. Specifically, the hashpling tells the system to use "env" to find the Python3 interpreter. Some developers use this hashpling (incorrect; do not use) -

    Code:
    #!/usr/bin/python3
    However, if the interpreter is not installed here, then the script will not work. Also, do not specify specific Python3 versions as seen below unless you have a valid reason for doing so.

    Code:
    #!/usr/bin/env python3.4
    On the second line, type

    Code:
    # -*- coding: utf-8 -*-
    This is not required, but is recommended by the PEP standards. This line declares the encoding for the script to be UTF-8. In your text editor or IDE, the options or document settings should allow you to set the file encoding to UTF-8. Adding this line will not set the encoding for your file unless you are using a very fancy IDE.


    Figure 1: Example of an IDE. This particular IDE is Geany. The code seen is the core source code of Nova (one of the four Betabots).
    IDE.png

    On the following lines, many developers include general information about the script like what it does, who made it, and such. The most important line among these comments is the license information. When making these comments (characters that the interpreter/compiler ignore), place a "#" at the beginning of the line like below -

    Code:
    #Awesome script that does awesome things
    #Made by Dr. Prof. Awesome
    ##Most important set of comment below##
    #GPLv3 (Dr. Prof. Awesome)©
    #This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as
    #published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
    #   This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
    #   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
    #You should have received a copy of the GNU General Public License along with this program. If not, see <http://www.gnu.org/licenses/>.
    
    If your script is licensed under the GPL license or some other license (like Apache or BSD), you must include the required license statement required by the license of your choice. By the way, if you are wondering how to license your work under GPLv2 or GPLv3, you just paste the appropriate license statement in your script that you legitimately own or have the right to license. Also, read about the license you wish to use. Their are special licenses for library files, .so/.dll files, applications, documentation, etc.

    NOTE TO FANS: If you would like to read an article about the different license and why they would be chosen and other various information, post/email me stating an interest in such an article. If I feel there are enough people wanting such an article, then I will write it and then post it on Linux.org. It may be a 2-5 weeks before you would see it though.

    Comments have other uses in programming. They can be used to give a message to future/current developers, make notes, temporarily disable certain lines of code, or explain the use of certain code.

    Code:
    COMMAND()
    #BROKEN_COMMAND() #Dr. Prof. Awesome will fix this bug
    CREATE = VARIABLE #This variable is needed for function X
    def SOME_BIG_FUNCTION(): #Note to graphics team: only the core developers can make changes to this function. Send requests to Dr. Prof. Awesome
      CODE
      CODE #Bug 1234 is suspected to be caused here
      CODE #This command will do Y
    
    Once you are ready for the actual code writing, you may start by writing a function. A function is a set of code that performs a particular task. Functions can be used as commands. When a function is defined, the name of the function can be used to execute that set of commands. In Python3, start the line with "def" meaning "define". Then include the function's name followed by two parenthesis "()" and a colon ":" which states the function's code follows.

    Code:
    def cat(openfile): #Emulates cat#
       with open(openfile) as file:
         lines = file.readlines()
         return ''.join(lines)
    
    NOTE: Be careful what you name a function. For example, Python has a "print()" command, so do not name a function "print()".

    Yes, the code may look advanced, but I hope to soon explain everything clearly in possible future tutorials. Notice the comment that provides a note. This lets you know the function is used like the "cat" command in Unix shells. In the script, the function will be called by typing "cat(FILE_NAME)" and all of the commands listed in the function's definition will be executed as if you typed all of that later in the script. Generally, functions are used for repetitive tasks and/or to organize the code. Functions save time since now to open a file, you only need to type "cat(OPEN_FILE)" instead of the complete function. In the function, you may be wondering about the "openfile" in parenthesis. This is a variable that equals the text placed in parenthesis when the function is called - "cat('./file')". So, here, the file "./file" will be opened and the contents will be used (not removed). In the function's definition, the variable "openfile" is used in the function since it contains the name of the file to open. I will discuss functions more in possible future tutorials.

    Variables store data. For example, when a file is opened with our function, we will save the file's contents to a variable. Notice below that the file's name is the function's parameter. The single quotes are used to declare that this is a string. If the name of a file and its path are stored in a variable and the variable is used as a parameter, then do not put quotes around the variable's name. Otherwise, Python's interpreter will think the parameter is the string "SOMEFILE" rather than the variable's contents "./other/file.txt".

    Code:
    MYFILE = cat('./file')
    SOMEFILE = './other/file.txt'
    OTHERFILE = cat(SOMEFILE)
    
    In many Python manuals and tutorials, objects are mentioned. In Python, an object may be a variable, function, class, or some other group of data with its own memory address. For example, the cat() function I made above is an object, the variable created in the function (lines), and the variable created in the parameter (openfile) are all three objects.

    In Python, commands have a few different structures as seen and explained below. Notice that all commands have parenthesis even if no parameters exist for the command.

    print() - The print command is very simple. Place a variable or a string (in single quotes) within the parenthesis.

    ''.join() - The join command takes an string or a variable and joins all of the separate units together separate by the character or object in the single quotes. The example below demonstrates what I am saying.

    Code:
    TEXT = 'Python' #On this line I am saving the string "Python" to the variable TEXT
    NEWDATA = '.'.join(TEXT) #I could also type these three lines as one for performance. print('.'.join('Python'))
    print(NEWDATA) #Result = P.y.t.h.o.n
    
    math.pi() - This command "pi()" is part of the "math" library. To use "math.pi()", you must import the math library (discussed next). "math.pi()" acts like a variable rather then a command. This command does not perform an action. Rather, it supplies the value of pi.

    To use commands from a library, the library must be imported. Many of you may wonder why all of the libraries are not already imported. This is for performance and memory usage reasons. Some libraries are very large. Importing many large libraries can consume a lot of memory and reduce start-up time. The easiest way to import a library is by typing "import LIB_NAME", so to import the math module, we would type "import math". If there is only one command you want from a library, then you can import the single command this way

    Code:
    from math import cos
    NUM = cos(37)
    
    Otherwise, using the regular "import math" would make such code look like this

    Code:
    import math
    NUM = math.cos(37)
    
    You could also import like this

    Code:
    import math as m
    NUM = m.cos(37)
    
    Combining these ideas works as seen below

    Code:
    from math import cos as c
    NUM = c(37)
    
    All of the above import examples work to provide the "cos()" command. However, when using the "from" command, the other math commands cannot be used, but memory is saved. Be careful about how you import. By this, I mean, do not try "from math import cos as print" because that will cause conflicts with Python's native "print()" command. You may be wondering how a developer would know of such an accidentally conflict. Over time, developers know Python well. As for newbies, your IDE or debugging tools may help. Also, testing your code often can help. If all else fails, look it up on the Internet.

    Python is case-sensitive, so the variable "DATA", "data", and "Data" are all three different valid variables. Generally, developers write variables in one case and function names in another, but this is personal preference. It is best to choose what case-form to use to variables. For instance, when writing a large program, a developer will not want to have a constant problem of wondering whether the variable is “DATA” or “Data”. Make a standard for your project especially if it is a team project.

    A class can best be explained as a function of functions. Classes can be used to sort functions into groups. So, all functions that pertain to the graphics of a program could be placed in a "graphics" class or what ever you wish to name it. So, the function that defines the help window would be called by typing "graphics.helpwin()".

    Code:
    class graphics:
    
       def help():
         VARIOUS-GRAPHICS-COMMANDS
    
       def open():
         MORE-GRAPHICS
    
    
    Notice that in the function and class examples that the code is indented. This is required in Python3. All code belonging to a class, function, loops, and other constructs are indented. Python does not have statements that indicate where a function, class, or loop end. As a result, the indentation is used. After a function is defined, the next line will be indented less by one tab or space. If a void-line/newline/empty-line is after a construct, do not use any indents on the line, but still remember to use less indentation on the following lines with text. Whether you use spaces or tabs for indentation, that is your choice. However, you can only choose one and that must be used throughout the script. Space and tab indentations cannot be mixed and matched within a script or on the same line. The PEP standards recommend spaces, but some developers prefer using tabs. Again, it is your choice. However, some debugging tools may inform/warn you about the tabs although there is no valid error.

    Here is a sample script used in Novabot (an SI project lead by this author). This is not the complete script. The point of this sample is to help you understand the indentation concept. The ellipses show that code was cut out at that point. Notice that the indentation does not apply to comments since the interpreter ignores them.

    Code:
    #!/usr/bin/env python3
    # -*- coding: utf-8 -*-
    #This script (ProgramPY-SH/Pysh), Xaiml, BetaCode, Betabots (and derivatives), Nova, Neo, and other Pyshbots were made by and owned by Devyn Collier Johnson, NCLA, Linux+, LPIC-1, DCTS#
    #ProgramPY-SH (Pysh) is made using the Geany IDE#
    #GPLv3 # © Copyrighted by Devyn Collier Johnson, NCLA, Linux+, LPIC-1, DCTS <DevynCJohnson@Gmail.com> on March 06, 2013#
    
    #   GPLv3 - 2012-2013. (Devyn Collier Johnson, NCLA, Linux+, LPIC-1, DCTS)©
    #This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as
    #published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
    #   This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
    #   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the   GNU General Public License for more details.
    #You should have received a copy of the GNU General Public License along with this program. If not, see <http://www.gnu.org/licenses/>.
    
    import time, os, datetime, subprocess, getpass
    def write2file(openfile, WRITE):
       with open(openfile, 'w') as file:
         file.write(str(WRITE))
    ......
    ###SETUP###
    global SENSORYSYSTEMPID, BOTINIT, BOTINITCHECK, USERNAMESH, BOTENGINE, BOTENGINECHECK, LAPTOP, CPUTEMPSTATUS, LIDSTATE, CPUTEMPERATURE, LID, LID2, SENSELOG
    ......
    SENSORYSYSTEMPID = os.getpid()
    ......
    write2file('./mem/SENSORYSYSTEMPID', SENSORYSYSTEMPID)
    write2file(SENSELOG, '')
    BOTINIT = ''.join(cat('./mem/BOTINIT_PID'))
    time.sleep(5)
    def CPUTHERM():
       return str(subprocess.getoutput('./lib/CPUTEMP'))
    def LIDCHECK():
       return str(subprocess.getoutput('cat /proc/acpi/button/lid/LID/state | awk \'{ print $2 }\''))
    def ENGINERUNNING():
       global BOTINITCHECK, USERNAMESH, BOTENGINE, BOTENGINECHECK
       BOTINITCHECK = str(subprocess.getoutput('kill -0 ' + str(BOTINIT)))
       USERNAMESH = getpass.getuser()
       BOTENGINE = subprocess.getoutput('pgrep -o -u ' + USERNAMESH + ' beta_engine')
       BOTENGINECHECK = str(subprocess.getoutput('kill -0 ' + str(BOTENGINE)))
    
    ###MAIN LOOP###
    ENGINERUNNING()
    try:
       TEST = str(CPUTHERM())
    except:
       INTERNALTHERM_ENABLE = 0
       add2file(SENSELOG, 'There is an error in CPUTHERM()\n')
    while BOTINITCHECK == '0' or BOTENGINECHECK == '0' or BOTINITCHECK == '' or BOTENGINECHECK == '':
       time.sleep(10)
    
    ###INTERNAL CPU TEMPERATURE###
    ......
       if INTERNALTHERM_ENABLE == 1:
         CPUTEMPERATURE = float(CPUTHERM())
         if CPUTEMPERATURE < float(96.0):
           if CPUTEMPSTATUS.lower() == 'hot'.lower():
             CPUTEMPSTATUS = 'fine'
             write2file('./mem/SenTemp.var', 'ixB06x')
             write2file('./mem/feels.var', 'fine')
           else:
             CPUTEMPSTATUS = 'fine'
         elif CPUTEMPERATURE == float(96.0):
           CPUTEMPSTATUS = 'hot'
           CPUTEMP = str(CPUTEMPERATURE)
           write2file('./mem/feels.var', 'hot')
           write2file('./mem/SenTemp.var', 'ixB00x')
           subprocess.getoutput('notify-send "My CPU is getting ' + CPUTEMPSTATUS + '. The CPU is temperature is ' + CPUTEMP + ' degrees Celsius."')
    ......
         else:
           CPUTEMPSTATUS = 'unknown'
           CPUTEMP = str(CPUTEMPERATURE)
           write2file('./mem/feels.var', 'unknown')
           write2file('./mem/SenTemp.var', 'unknown')
           subprocess.getoutput('notify-send "My CPU sensor is having issues. The CPU is temperature is ' + CPUTEMP + ' degrees Celsius (You should see a valid number in this message)."')
       else:
         if ONELOOP == 0:
           subprocess.getoutput('notify-send "The CPU temperature cannot be read due to some error I cannot detect."')
           add2file(SENSELOG, 'The CPU temperature cannot be read due to some error I cannot detect.\n')
           ONELOOP = 1
           INTERNALTHERM_ENABLE = 0
         else:
           pass
    
    ......
    
    This is a basic first-step into Python3. If many people enjoy this tutorial, then I will continue into a series. Mahalo!

    Attached Files:

    Last edited: May 25, 2014
  2. James Rico

    James Rico New Member

    Messages:
    2
    Likes Received:
    3
    Trophy Points:
    3
    Good job I enjoyed this tutorial and hope more to follow. The length was good as well as examples provided.
    cm1967 and DevynCJohnson like this.
  3. ryanvade

    ryanvade Administrator Staff Member Staff Writer

    Messages:
    1,268
    Likes Received:
    431
    Trophy Points:
    83
    I like to use Numba with my python. Parallel computing with python...
  4. Harikrishnan R

    Harikrishnan R Member

    Messages:
    30
    Likes Received:
    22
    Trophy Points:
    8
    Seems very similar to Python 2.
    Tried this out. Thanks for the article.
    DevynCJohnson likes this.
  5. DevynCJohnson

    DevynCJohnson Well-Known Member Staff Member Staff Writer

    Messages:
    1,336
    Likes Received:
    1,073
    Trophy Points:
    113
    True, Python2 has a LOT in common with Python3, but I do not want to point out all of the differences.
    Last edited: Nov 21, 2013
    Harikrishnan R likes this.
  6. DevynCJohnson

    DevynCJohnson Well-Known Member Staff Member Staff Writer

    Messages:
    1,336
    Likes Received:
    1,073
    Trophy Points:
    113
    Well, I got some likes, emails, posts, G+, etc. I was hoping for more, but oh well. I will make a whole series. It will be 2-3 weeks, but a series should then be posted on a regular basis. Thanks and enjoy!
    cm1967 likes this.
  7. Harikrishnan R

    Harikrishnan R Member

    Messages:
    30
    Likes Received:
    22
    Trophy Points:
    8
    Well ... Okay I guess ...:D

    And keep up the good work.
    DevynCJohnson likes this.
  8. DevynCJohnson

    DevynCJohnson Well-Known Member Staff Member Staff Writer

    Messages:
    1,336
    Likes Received:
    1,073
    Trophy Points:
    113
    Whoops, thanks for catching that error.

Share This Page