Welcome to the Using Tabular data in Python Course

This course is aimed at people who have completed the Enviromental Scientist’s Introduction to Python Course. If You have not completed this course, we recommend at least reviewing the course to make sure you have comparable background in basic python. The goal of this course is to introduce tabular data manipulation in python and to get you to the point that you no longer need to use excel for your data analysis.

Using tabular data in python (need to have)

• Introduction to Numpy and Pandas
  • What in the world is a Numpy and Why are Pandas relevant
  • The Numpy Array
  • The Pandas Series and Dataframe
  • Vectorised operations (or why pandas and numpy are cool)
  • Datatypes
  • Vectorised boolean operations
  • Missing data in pandas and Numpy
• Reading and Writing Data with Pandas
  • Reading text files
  • Reading from excel
  • Writing text files
  • Other supported formats
  • Dataframe cleanup
• Working with Tabular Data
  • Using a Dataframe Like an Excel Sheet
  • Merging Manipulating and Mapping data
• Summarising Data in Pandas
  • Dataframe and Series summaries
  • Using Groupby to Aggregate Data

Using tabular data in python (nice to have)

• Time Series Analysis with Pandas
• Complex Data Re-arrangement and the Multi-index
  • The Pandas Multi-index
  • Data Structure Magic: Melt and Pivot
• Using Pandas for Quick Data Visualisation

Python build for this course

If you are not familar with using virtual python enviroments, we reccomend you review our lesson on installing python. For This course we recommend installing python as follows:

This installation includes basic python, the packages numpy and pandas, and the IDE spyder. It also installs matplotlib, which is a pandas dependency if you are using the plotting functionality.

1. Install miniconda (if you haven’t already)

   1. go to https://conda.io/miniconda.html and download the appropriate python 3.6 installer and accept all of the defaults

2. Create a virtual environment for this course (tdip) for Tabular Data In Python

   1. open anaconda prompt and enter:

   conda create -n tdip python=3.6 spyder numpy pandas matplotlib
   

2. When conda asks you to proceed, type y:

   proceed ([y]/n)?
   

3. That’s it python and spyder for this course should now be installed. To use python with spyder, in the start menu (under anaconda) you should see spyder (tdip). Open that up and get cracking!

Practice Exercises

We have developed a set of practice exercises to give you a taste of doing your own scripting. These exercises are facillitated through Github Classroom. If you want to get on to the exercises, this link will create a new repository with a copy of the exercises for you to begin working.

exercise	prerequisites / associated lessons
exercise 1	Introduction to Numpy and Pandas
exercise 2	Reading and Writing Data with Pandas
exercise 3	Working with Tabular Data
exercise 4	Summarising Data in Pandas
exercise 5	Time Series Analysis with Pandas
exercise 6	Complex Data Re-arrangement and the Multi-index