Getting Started With Python

Development Environments, Workflows and Interpreters

Python is an interpreted programming language. Therefore, two central elements of any system for programming in Python will involve support for writing and editing Python code and for running that code in an interpreter. (And once code is developed and tested, integrating the results of simulations and analysis is an important part of the research process.) There are three major types of systems / workflows that support such functionality: (1) Editor + Interpreter/Notebook, (2) Integrate Development Environment (IDE), and (3) Notebook Only. Each such system has its advantages and disadvantages, and it is somewhat a matter of personal taste - as well as the specifics of each use case - as to what each person prefers. In this course, you will be asked to use an IPython notebook to summarize the results of your computations, but you can explore different development environments to see which you prefer. The IPython notebook itself is an interpreter, so a useful workflow you will be encouraged to try out is to write code in an editor, run that code within an IPython notebook, and use the notebook for testing, debugging and analyzing the results of simulations. In any case, the basic process of working with course modules will be the same regardless of which environment you choose: you will download a Python Hints file, make a copy of it, and then populate the code to implement the desired functionality, using the exercise instructions and code comments as your guide.

Suggested Workflow for the Course: Editor + Notebook

Editors are good for writing and editing code, and the IPython notebook is a useful and productive environment for both testing code that you are developing, as well as presenting and documenting the outputs of that code.

The general process by which you will work on course exercises is as follows:

From the course home page, following the navigator link to Computer Exercises.
Once you have chosen an exercise to work on, you will download a Python Hints file.
Start a terminal window (look under Applications if you cannot find a way to launch it from your desktop).
If you have a favorite text editor,

Below we discuss each of these different workflows and tools to support them, but first discuss the IPython interpreter that we will use.

IPython

The core Python interpreter that comes with every installation is called python. It is useful for running standalone programs on the command line (e.g., python my_program.py will run my_program.py until completion), but is not as useful for interactive work. IPython is an enhanced interpreter built on top of the core python interpreter that facilitates interactive programming, development and data exploration. If you have IPython installed and in your shell path, you can start it up via:

typing ipython on the command line to start up IPython in your current terminal
typing ipython qtconsole on the command line to start up IPython in a dedicated console window, which includes additional functionality in an associated menu bar
typing ipython notebook on the command line to start up the iPython notebook interface in your web browser

IPython provides support for inspecting objects and getting help within the interpreter. For example,

Appending a question mark to an object or function name (e.g., print?) will print out any help text provided for that object (such as sequence of call paramters for a function).
Hitting the TAB key will turn on name completion, printing a list of available objects or functions whose name starts with whatever letters you have already typed.

In addition, IPython defines a powerful set of "magic" functions that augment the core Python functionality. These magic functions are all preceded by a percent sign (%) to distinguish them from core Python functions, e.g.,

typing %run filename.py in the IPython interpreter will load and execute the code in filename.py
typing %who in the IPython interpreter will print out what variables and functions have been defined in the current session
typing %hist in the IPython interpreter will print out a list of previous commands executed
typing %lsmagic in the IPython interpreter will print out a list of all available magic commands

General IPython documentation can be found here, and specific information on the magic functions is included in the chapter on Built-in magic commands. See below for further information on the use of the IPython interpreter.

Editor + Interpreter Workflow

In this mode, you will edit Python code in a text editor, and run that code in the IPython interpreter (either in a notebook a terminal window, or a qtconsole). If there is a general text editor that you like (e.g., emacs, vim, gedit, etc.), you can edit Python code in the editor, and then load that code (either through the IPython %run magic function, or the core Python import statement) into the interpreter. Since correct Python syntax relies on indentation to separate code blocks, it is imperative that your code editor have support for Python language programs. Fortunately, most editors do and will auto-detect Python code with source filenames that end in the suffix .py .

The advantages of this workflow are:

you get to use your favorite text editor
text editors provide excellent support for manipulating multiple source files and moving/reconfiguring large blocks of code
you can edit code in a separate standalone program without having to start up an interpreter for running it

The disadvantages of this workflow are:

you have to start up two separate programs (although the IPython notebook continues to roll out new functionality for editing text files and starting up terminal windows)
there is no separate support for variable inspection other than that provided by the IPython interpreter
plots are only integrated into the rest of the interpreter session if you use the notebook interface

Integrated Development Environment Workflow

Integrated Development Environments (IDEs) are programs that aim to integrate different aspects of programming and running code. There are many such environments, some targeted to specific programming languages and others that support a broader sweep of languages. Included in the Anaconda Scientific Python distribution that we are using for this course is Spyder, an IDE that supports Python code development. The Spyder IDE brings together three main tools in one integrated environment: code editor, IPython interpreter, and variable inspector.

The advantages of this workflow are:

editor and interpreter are running under the same program, with convenient support for running code within the interpreter
additional tools provide integrated support for debugging, object inspection and variable exploration

The disadvantages of this workflow are:

you might not be able to configure the editor to your liking (if you have other favorite text editors)
plots are not integrated into the rest of the interpreter session (unless you can figure out a way to get the IDE to use the notebook interface rather than the terminal interface)

IPython Notebook Workflow

As noted, the IPython interpreter can also be run in notebook mode. If you have IPython installed and in your shell path, you can start it up via:

typing ipython notebook on the command line to start up the iPython notebook interface in your web browser

The IPython notebook (similar to the notebook interface that you might have used in Mathematica) integrates code development and output within a single graphical interface. Notebooks are organized around cells, smaller bits of code that can be edited and run individually or collectively. The IPython notebook also is well integrated with the Matplotlib (pylab) plotting library, which enables plots to be displayed within the notebook page itself, rather than stored in a separate file.

The advantages of this workflow are:

excellent support for interactive querying of small bits of code and/or generating plots
integration of plots and other code outputs, as well as markup language support, in the notebook interface, allowing for the creation of living documents showing code and results

The disadvantages of this workflow are:

support for editing large and complex sets of interrelated code is not good if using only the notebook for editing
editing support is generally limited to cut-and-paste functionality (although the IPython notebook continues to roll out new functionality for editing text files and starting up terminal windows)
there is a separation (and hence the possibility for drift) between .py source files and .ipynb notebook files

Using the ipython notebook interface
Using the ipython terminal or qtconsole interface

Starting up ipython -- terminal or qtconsole interface

An alternative way to use ipython is either in a terminal window or in the qtconsole window, which provides the same basic input interface as the terminal but which includes additional functionality via graphical menu items.

Start a terminal window (Applications -> Accessories -> Terminal).
Type ipython --matplotlib to launch the interactive Python shell within the terminal, or ipython qtconsole --pylab for a separate qtconsole window. (The --pylab option loads the pylab package for plotting.)
Type in mathematical expressions, and see how they are evaulated. Try expressions using addition (+), subtraction (-), multiplication (*), division (/), exponentiation (**), and modulo (%). Try division for both integers and floats, e.g., 5/2, 5.0/2, 5/2.0, and 5.0/2.0. (The type-dependent behavior of division in Python is in a state of transition; for now, however, the default behavior of an integer division such as 5/2 is different than you might expect.)
Explore how the double equal sign (==) generates expressions that evaluate to a Boolean variable (True or False), as in 2+2==4, or 2+2==5.
Setting a=5 (a single equal sign denotes assignment), write an expression that tests if a is odd using the modulo operator. (It should evaluate to False.)
Define a function isEven(n) to test if n is even (returning True or False). The syntax for defining a function in Python introduces four new features [def, the colon (:), indents, and return], most easily explained by example:
```
def isEven(n):
    blah = XXX       # for some python code XXX
    return blah
```
or even more simply:
```
def isEven(n):
    return XXX
```
Note the importance of indentation. Unlike other languages, where indenting is used only to make programs readable, Python uses indenting and new lines to delimit the ends of expressions, functions, and code blocks. The ipython shell automatically indents based on context. To end a function, use a blank line. When writing Python code in emacs, the Python editing mode will be automatically loaded, and indentation should be handled properly.
With the terminal interface it is a little more work to edit your code when you make mistakes. Try pushing the up arrow and see what happens.

Starting up ipython -- notebook interface

Start a terminal window from set of available applications.
Type ipython notebook --pylab inline to launch the notebook interface to the interactive Python shell. (The --pylab option loads the pylab package for plotting.)
Your web browser should open.
Click on "New Notebook".
In the box after the In []: prompt type 2+3 then hold down the shift key and press enter.
Type in mathematical expressions, and see how they are evaulated. Try expressions using addition (+), subtraction (-), multiplication (*), division (/), exponentiation (**), and modulo (%). Try division for both integers and floats, e.g., 5/2, 5.0/2, 5/2.0, and 5.0/2.0. (The type-dependent behavior of division in Python is in a state of transition; for now, however, the default behavior of an integer division such as 5/2 is different than you might expect.)
To evaluate an expression you always need to use "shift-enter". Hitting enter alone just moves you to a new line within the notebook cell.
Explore how the double equal sign (==) generates expressions that evaluate to a Boolean variable (True or False), as in 2+2==4, or 2+2==5.
Setting a=5 (a single equal sign denotes assignment), write an expression that tests if a is odd using the modulo operator. (It should evaluate to False.)
Define a function isEven(n) to test if n is even (returning True or False). The syntax for defining a function in Python introduces four new features [def, the colon (:), indents, and return], most easily explained by example:
```
def isEven(n):
    blah = XXX       # for some python code XXX
    return blah
```
or even more simply:
```
def isEven(n):
    return XXX
```
Note the importance of indentation. Unlike other languages, where indenting is used only to make programs readable, Python uses indenting and new lines to delimit the ends of expressions, functions, and code blocks. The ipython shell automatically indents based on context. To end a function, use a blank line. When writing Python code in emacs, the Python editing mode will be automatically loaded, and indentation should be handled properly.
With the notebook interface you can edit cells. Hit "shift-enter" to re-evaluate.

Library importing and plotting

Python is a general-purpose programming language, but does not include in its core "special" topics like mathematical functions, graphics, and plotting. External libraries must be imported for these functions.

Import the package scipy using import scipy, and then try using the function scipy.sin and scipy.pi to evaluate sin(pi/2) as scipy.sin(scipy.pi/2)
Notice that one uses functions from inside packages by preceding them with the package name. This is good programming practice, avoiding name collisions (where two different packages might use the same function name).
Use scipy.linspace to define an array x of 20 points starting at zero and ending at 2 pi, as in x = scipy.linspace(0, 2.*scipy.pi, 20). Applying scipy.sin to the entire array x: y = scipy.sin(x).
Import the plotting package pylab. Plot sin(y) versus y, as in pylab.plot(x, y), followed by pylab.show(). If you want to save the figure you made to an image file, click the icon at the bottom of the the figure window that looks like a floppy disk. (Does anybody still know what a floppy disk is?) Close the figure window to regain control within ipython.
Further information about scipy and pylab is available. Typing help(scipy) will provide some documentation on that package. Next try typing scipy. followed by a Tab. (The ipython shell provides "name completion", listing all possible variable or function names that are valid completions of what you've already typed.)
If you have started ipython with the "--pylab" flag, then pylab is already imported, and you don't need to use the "show" command.
If you use the "inline" flag, then the plots will appear in the same screen as the commands. Try starting ipython with/without this flag and see the difference.

Notebooks, editors, Hints files, %run, etc.

Make a directory called "Primes", using the command mkdir Primes or using the graphical tools.
Start up a Web browser, and find the exercise page on prime numbers www.physics.cornell.edu/~myers/teaching/ComputationalMethods/ComputerExercises/Primes/Primes.html
Save the hints notebook file (PrimesNotebookHints.ipynb) into your Primes directory and rename it "PrimesNotebook.ipynb"
Notebook Interface
- If you are using the notebook interface drag "Primes.py" into the "dashboard" window. If you hit the right spot (the grey bar which lists the directory) then it will prompt you to upload the file.
- You can then click on the file to interact with it.
- "Save" often
- Click the "download" button to make a copy of the file -- you can save it as a bare ".py" file, or a more sophisticated ".ipynb" file that includes markup info.
Terminal Interface
- In the terminal window, type
```
	     
  cd Primes  	[this changes directory]
  ls		[should show Primes.py]
  ipython       [this starts ipython]
  In[1]: %run Primes    [%run is an ipython "magic" command, not a Python language command]
```
- In a second terminal window, go to the same directory and open the Primes.py file in a text editor. On the lab computers, good editors include emacs and gedit. Start them with emacs Primes.py or gedit Primes.py. It is important that you are in the correct directory. In the text editor, you will make changes to Primes.py, save the file, and once again execute %run Primes in ipython. Changes that have are made in Primes.py should then be reflected in the ipython session.
- You may want to explore the various editors and pick out which one you like best. All of the editors offer a "Python" editing mode which is capable of bracket completion, auto indentation, and syntax highlighting. Some of these features need to be manually turned on. For example, in emacs do the following:
  - In the emacs Options menu, select the option for "Syntax Highlighting" and then click on "Save options" at the bottom of the Options menu. This will turn on text coloring so that different Python features (keywords, strings, etc.) are colored differently. You should only need to set the option this one time.
The first step in Primes.py is to implement the isEven function that you already defined interactively. You will see the following code in the Primes.py file:
```
def isEven(n):
    """
    Function that will "return True" if n is even, False otherwise.
    Uses the modulo operator, which returns the remainder of the
    division of n by m:  e.g., n%m is 0 if n is divisible by m.
    """
    pass
```
The word pass is a Python keyword that does nothing. It is included here since every function in Python needs a valid body, even if that body does nothing (pass). Even before redefining the function body, you can ask for help about the isEven function (via help(isEven)). You will see the documentation string (docstring) echoed in the ipython window. A good thing to keep in mind is that if you want your function to return a value, you must use the return command.
After implementing isEven, continue to implement other functions that need to be defined. (Some function bodies in this exercise have been left intact, although you could look them over to see what they are computing.) Once you have implemented all the functions in the file, compare the set of generated prime numbers up to 10000 (or whatever limit you choose) with the theoretical density of primes by running the following commands:
```
primes = primeList(10000)
PlotPrimeDensityTheory(primes)
```
Once you've finished with Primes, tackle the Random Text generation module.

Physics 7682

Instructor: Chris Myers

Development Environments, Workflows and Interpreters

Suggested Workflow for the Course: Editor + Notebook

IPython

Editor + Interpreter Workflow

Integrated Development Environment Workflow

IPython Notebook Workflow

Starting up ipython -- terminal or qtconsole interface

Starting up ipython -- notebook interface

Library importing and plotting

Notebooks, editors, Hints files, %run, etc.