Table of Contents
- Python installations: in the computer lab or on your laptop
- Development Environments, Workflows and Interpreters
- Suggested Workflow for the Course
- Some Alternative Workflows for the Course
- Additional information on IPython
Python is a programming language, augmented by a rich set of third-party libraries that build upon the core functionality defined in the language. Since we will be using a number of such libraries, it is helpful to leverage existing Python package bundles tailored specifically for scientific computing. For this course, we will be using the Anaconda Scientific Python distribution, provided by Continuum Analytics. Different versions of this distribution are available; we will be using version 3.4 (latest minor version 3.4.3). If you choose to use your own laptop for the course, you should download Anaconda Python version 3.4 by following the Download links at the Anaconda Python site. Make sure you get select version 3.4, since the version 2.7 installer is brought up by default. If you already have a Python installation on your own machine that you actively maintain (e.g., using a package management system), you might need to install some additional packages, such as SciPy, matplotlib, and IPython.
2. Development Environments, Workflows and Interpreters
Python is an interpreted programming language. Therefore, two central elements of any system for programming in Python will involve support for writing and editing Python code and for running that code in an interpreter. (And once code is developed and tested, integrating the results of simulations and analysis is an important part of the research process.) There are three major types of systems / workflows that support such functionality: (1) Editor + Interpreter/Notebook, (2) Integrate Development Environment (IDE), and (3) Notebook Only. Each such system has its advantages and disadvantages, and it is somewhat a matter of personal taste - as well as the specifics of each use case - as to what each person prefers. In this course, you will be asked to use an IPython notebook to summarize the results of your computations, but you can explore different development environments to see which you prefer. The IPython notebook itself is an interpreter, so a suggested workflow is to write code in an editor, run that code within an IPython notebook, and use the notebook for testing, debugging and analyzing the results of simulations. The basic process of working with course modules will be the same regardless of which environment you choose: you will download a Python Hints file, make a copy of it, and then populate the code to implement the desired functionality, using the exercise instructions and code comments as your guide.
3. Suggested Workflow for the Course: Editor + Notebook
Editors are good for writing and editing code, and the IPython notebook is a useful and productive environment for both testing code that you are developing, as well as presenting and documenting the outputs of that code.
The general process by which you will work on course exercises is as follows:
- From the course home page, following the navigator link to Computer Exercises.
- Once you have chosen an exercise to work on, first read any background material that is provided (e.g., journal article) as well as the specific instructions for the exercise.
- Start a terminal window (look under Applications if you cannot find a way to launch it from your desktop).
- Once you understand the basics of the exercise, download the Python hints file from the navigation toolbar for that page. Your goal is to populate the skeletal code that is contained in the "Hints" file, so the first thing you want to do is make a copy of the Hints file with a new name. For example, you could type cp PrimesHints.py Primes.py to create a new file that you will add code to.
- If you have a favorite text editor, open your new answers file in that editor. If you don't have a favorite text editor, or would like to experiment with editing within the ipython web browser console, proceed to the next step and then revisit the issue of editing the file a few steps below.
- Also from the terminal window, start up the ipython notebook: type ipython notebook on the command line.
- From the ipython notebook console window, from the "New" drop-down menu on the right-hand side, create a new Python3 notebook.
- You can rename the new untitled notebook that you've created to indicate what project you're working on; for example, if you're editing the Primes.py source file, you could rename this PrimesNotebook. (And since it is an IPython notebook, you might later notice that it is created with an .ipynb filename suffix.)
- If you haven't already opened the Python answer file in a text editor, you can do so now within the IPython web console. Open another browser tab or window from the "Jupyter" icon (e.g., by right-clicking to get more options), and then click on the name of the file that you want to edit, which should open up the file within a browser-based editor window. (You can configure options to make the editor look a bit like other editors, and make sure that the source is recognized as Python code if that hasn't been autodetected.)
- Now that you have both an editor open with your Python file loaded (e.g., Primes.py) and a notebook open, you're ready to start working. In the first cell of the notebook, type %run Primes, assuming that you're working on the Primes.py source python file, and hit Shift+Enter. This will load all the contents of the Primes source file into the notebook and execute the code there. If you type %whos into a new cell, followed by Shift+Enter, this should list all of the functions and variables that have been defined in the Primes.py file.
- Edit the Primes.py source file to fill in the function bodies as instructed, and when you want to try those out in the notebook, you can rerun everything by clicking on the Cell->Run all selection. Alternatively you can run individual cells either through the Cell menu entries or by retyping Shift+Enter in a cell of interest.
- An excellent feature of the IPython notebook is that it can integrate plots and other graphics into the notebook. Most of the plotting we will be doing will use the matplotlib / pylab package. If you type %matplotlib inline, this will instruct IPython to display all matplotlib plots into the notebook. There is also a way to configure your IPython installation to specify %matplotlib inline as a default behavior.
- Another excellent feature of the IPython notebook is that you can add headers, titles, and other documentation (including latex / mathjax) into the notebook. The default type for any cell is "Code", but you can change that by selecting a different type, most notably "Markdown". Markdown is a markup language with its own lightweight syntax that allows you to easily embed these different types of text into a notebook. Along the way, you can add titles and documentation to explain what you're doing, which will help you keep track of what you've done and explain the work better to others.
- You can also clean up the notebook as you go along. Sometimes it is useful to use the notebook interactively to try out different commands to figure out what works best, and then transfer the resulting solution into the Python source file (via copy and paste). But you can then delete or merge cells in the notebook to trim it down to a useful report.
4. Alternative Workflows for the Course
For all of these other workflows, follow the basic outline described above: downloading hints files, filling in skeletal code fragments, and then testing and producing output within an interpreter.
IDEs: Integrated Development Environments (IDEs) are programs that aim to integrate different aspects of programming and running code. There are many such environments, some targeted to specific programming languages and others that support a broader sweep of languages. Included in the Anaconda Scientific Python distribution that we are using for this course is Spyder, an IDE that supports Python code development. The Spyder IDE brings together several tools in one integrated environment, most importantly a code editor, IPython interpreter, and variable inspector. Like the IPython notebook, the interpreter window in Spyder enables the incorporation of plots inline with code, and interpreter sessions can be saved to HTML pages. These sessions do not appear to allow the sort of rich text markup that the notebook does, but they will suffice for integrating project results. In addition, the terminal-based IPython interpreter in Spyder does provide much better command-line recall than does the notebook (e.g., by scrolling back and editing previous commands), and there are benefits to the integration of a number of different tools.
Editor + IPython terminal/console: Instead of using an IPython notebook, it is sometimes preferable to simply use IPython in a terminal or a qtconsole. This environment is, for some tasks, easier to navigate, providing better command-line recall than does the notebook (e.g., by scrolling back and editing previous commands), and less pointing-and-clicking.
IPython notebook only: Code writing and editing can be done completely within a notebook, although the editing facilities are not extensive, and it is difficult to develop large, multi-file codes within this environment. The benefits of the notebook interface described above, however, make this quite useful for certain applications.
5. Additional information on IPython
The core Python interpreter that comes with every installation is called python. It is useful for running standalone programs on the command line (e.g., python my_program.py will run my_program.py until completion), but is not as useful for interactive work. IPython is an enhanced interpreter built on top of the core python interpreter that facilitates interactive programming, development and data exploration. If you have IPython installed and in your shell path, you can start it up via:
- typing ipython on the command line to start up IPython in your current terminal
- typing ipython qtconsole on the command line to start up IPython in a dedicated console window, which includes additional functionality in an associated menu bar
- typing ipython notebook on the command line to start up the iPython notebook interface in your web browser
IPython provides support for inspecting objects and getting help within the interpreter. For example,
- Appending a question mark to an object or function name (e.g., print?) will print out any help text provided for that object (such as sequence of call parameters for a function).
- Hitting the TAB key will turn on name completion, printing a list of available objects or functions whose name starts with whatever letters you have already typed.
- typing %run filename.py in the IPython interpreter will load and execute the code in filename.py
- typing %who in the IPython interpreter will print out what variables and functions have been defined in the current session
- typing %hist in the IPython interpreter will print out a list of previous commands executed
- typing %lsmagic in the IPython interpreter will print out a list of all available magic commands
General IPython documentation can be found here, and specific information on the magic functions is included in the chapter on Built-in magic commands. See below for further information on the use of the IPython interpreter.