Open and Reproducible Neuroscience

A shell is a program whose primary purpose is to read commands and run other programs.

The shell’s main advantages are its high action-to-keystroke ratio, its support for automating repetitive tasks, and that it can be used to access networked machines.

The shell’s main disadvantages are its primarily textual nature and how cryptic its commands and operation can be.

We will work on a remote instance of the linux operating system

Although there are many ways to work remotely, we will use a graphical interface using NoMachine.

We will use an IPython console to interact with the Python interpretter

We will learn basic commands for working with the computer file system: pwd, ls, mkdir and cd.

Understand the difference between alias, system commands, and IPython magics

Show how cd behaves differently because each system call is a subprocess

Use variables to store values.

Use print to display values.

Variables must be created before they are used.

Variables can be used in calculations.

Use an index to get a single character from a string.

Use a slice to get a substring.

Use the built-in function len to find the length of a string.

Python is case-sensitive.

Use meaningful variable names.

Variables in the current environment can be displayed or deleted using the %whos and %reset IPython magics.

Every value has a type.

Use the built-in function type to find the type of a value.

Types control what operations can be done on values.

Strings can be added and multiplied.

Strings have a length (but numbers don’t).

Must convert numbers to strings or vice versa when operating on them.

Can mix integers and floats freely in operations.

Variables only change value when something is assigned to them.

Use comments to add documentation to programs.

A function may take zero or more arguments.

Commonly-used built-in functions include max, min, and print.

Functions may only work for certain (combinations of) arguments.

Functions may have default values for some arguments.

Use the built-in function help to get help for a function in python or ‘?’ after the function in IPython.

Every function returns something.

Python reports a syntax error when it can’t understand the source of a program.

Python reports a runtime error when something goes wrong while a program is executing.

Fix syntax errors by reading the source code, and runtime errors by tracing the program’s execution.

A list stores many values in a single structure.

Use an item’s index to fetch it from a list.

Lists’ values can be replaced by assigning to them.

Appending items to a list lengthens it.

Use del to remove items from a list entirely.

The empty list contains no values.

Lists may contain values of different types.

Character strings can be indexed like lists.

Character strings are immutable.

Indexing beyond the end of the collection is an error.

The process of iteration executes commands once for each value in a collection.

Python uses comprehensions to iterate. List comprehensions are especially common.

Loop variables can be called anything (but it is strongly advised to have a meaningful name to the looping variable).

Use range to iterate over a sequence of numbers.

Generator and dictionary comprehensions are similar to list comprehensions but their output is slightly different.

A for loop is made up of a loop variable, a collection, and a body.

The first line of the for loop must end with a colon, and the body must be indented.

Indentation is always meaningful in Python.

The body of a for loop can contain many statements.

The Accumulator pattern turns many values into one.

OpenNeuroLab, among many other data sharing resources, has made many thousands of brain volumes available for community use.

We can explore our input history and save our code to text files. For python files we use ‘.py’.

We can easily open these files in an editor.

Atom is open source

A GUI is convenient. Using the keyboard is faster.

Start learning commands using the command palette.

Settings can be changed in two different ways.

Additional functionality can be added to Atom.

Most of the power of a programming language is in its libraries.

A program must import a package in order to use it.

Use help to find out more about a package’s contents.

Import specific items from a package to shorten programs.

Create an alias for a package when importing it to shorten programs.

Version control is like an unlimited ‘undo’.

Version control also allows many people to work in parallel.

All changes tracked by git are stored in the hidden repository ‘.git’.

Use git config to configure a user name, email address, editor, and other preferences once per machine.

git init initializes a repository.

git status shows the status of a repository.

Files can be stored in a project’s working directory (which users see), the staging area (where the next commit is being built up) and the local repository (where commits are permanently recorded).

git add puts files in the staging area.

git commit saves the staged content as a new commit in the local repository.

Always write a log message when committing changes.

View previous commits using the git log command.

Boolean values, either True or False, represent the truth of a statement in Python

As well as functions that specifically return values of type bool, statements about truth use what are called comparative operators.

Use if statements to control whether or not a block of code is executed.

Conditionals are often used with iteration.

Use else to execute a block of code when an if condition is not true.

Use elif to specify additional tests.

Conditions are tested once, in order.

Create a table showing variables’ values to trace a program’s execution.

Break programs down into functions to make them easier to understand.

Define a function using def with a name, parameters, and a block of code.

Defining a function does not run it.

Arguments in call are matched to parameters in definition.

Functions may return a result to their caller using return.

git diff displays differences between commits.

git checkout recovers old versions of files.

A local Git repository can be connected to one or more remote repositories.

Use the HTTPS protocol to connect to remote repositories until you have learned how to set up SSH.

git push copies changes from a local repository to a remote repository.

git pull copies changes from a remote repository to a local repository.

git clone copies a remote repository to create a local repository with a remote called origin automatically set up.

Conflicts occur when two or more people change the same file(s) at the same time.

The version control system does not allow people to overwrite each other’s changes blindly, but highlights conflicts so that they can be resolved.

Containers allow applications to run in a self-contained, reproducible environment.

Containers can be copied and shared allowing you to replicate your results on a new machine, or allowing colleagues to replicate your results in a different environment.

Individuals and organizations distribute software in containers on Docker Hub and Singularity Hub

You can learn more about containers at Docker Home, Singularity Home, and Singularity at the NIH HPC.