Python directly read in USGS flow by given site number

Python packages are very powerful. Here is an example for hydrological data analysis. This small piece of code reads USGS flow by a given site number and store it as pandas DataFrame. You need to install “ulmo” first.

Original post is from here

import pandas as pd
import numpy as np
import ulmo

def importusgssite(siteno):
    sitename = {}
    sitename = ulmo.usgs.nwis.get_site_data(siteno, service="daily", period="all")
    sitename = pd.DataFrame(sitename['00060:00003']['values'])
    sitename['dates'] = pd.to_datetime(pd.Series(sitename['datetime']))
    sitename.set_index(['dates'],inplace=True)
    sitename[siteno] = sitename['value'].astype(float)
    sitename[str(siteno)+'qual'] = sitename['qualifiers']
    sitename = sitename.drop(['datetime','qualifiers','value'],axis=1)
    sitename = sitename.replace('-999999',np.NAN)
    sitename = sitename.dropna()
    #sitename['mon']=sitename.index.month
    return sitename

d = importusgssite('12472800');
d.plot(style='r', linewidth=1.0)

flowseries.png

Advertisements

Plotting in Python

Finally start to use python for plotting.

  • Plotting in notebook

ipython notebook is good and plots can be lined up with scripts. But if I need to run the same script on cluster without Xming, it will crash. Simple way is to add

import matplotlib as mpl
mpl.use('Agg')

when importing libraries.

  • Seaborn

Seaborn is a great package for color scheme picking. Color schemes are called “palettes” in seaborn. There are many pre-set palettes for you to choose. If you want to use “colorblind”, just simply add

import seaborn as sns
with sns.color_palette("colorblind"):

before the plotting script.

Customizing the palette is easy too:

colors = ["windows blue", "amber", "greyish", "faded green", "dusty purple"]
with sns.xkcd_palette(colors):

or

flatui = ["#9b59b6", "#3498db", "#95a5a6", "#e74c3c", "#34495e", "#2ecc71"]
sns.set_palette(flatui)
with sns.color_palette():

 

Python Bootcamp

1. clear all the variables in Sypder

 >>> import sys
 >>> sys.modules[__name__].__dict__.clear()

2. Some tips (python notebook)

                  #shift + enter to run lines
                  #triple quotes for multi-line string
                  #underscore denotes result from last operation
!ls               #see what's in dir
range?            #check info about "range"
something.        #press Tab to see options
x,y = y,x         #easy swap, multipal assign
bool(-1)          #see Boolean data type
str(13)           #Matlab num2str(13) equivalent
print(type(True)) #see the data type, will return bool
x=range(4)        #return [0,1,2,3]

Continue reading

Python in ArcGIS 1

1. Python editor IDLE will be installed with ArcGIS. Right click .py file can find the IDLE option in the menu.

2. print

strName = "Tian"
print strName
print (strName)
print 10*strName
print 'Tian'
print "Tian"

3. in Python Shell, type into variable name gives the value of the variable. (kinda like matlab)

4. type(strName) gives the type of variable.

5. vars() gives all the variables exist.

6. list contains more variables

lstMonths=['May','Jun','July']
lstMonths[0]   #shows the 1st element of the list "May"
len(lstMonths) #length of the list, like matlab
lstMonths[1:]  #all the elements except the 1st one
lstMonths[:-1] #all the element except the last one
lstYears = range (2000,2014,2) #doesn't contain the last one, 2 is the increment
lstYears.index(2008) #shows the index of this element
lstYears.count(2008) #counts how many 2008 in the list

7. Loops

lstMonths=['May','Jun','July']
for i in lstMonths:
    print i

# or
for i in range(0, len(lstMonths)-1):
    print(lstMonths[i])

8. Modules
First, create a module, save it as “testmodule.py”

#test module

#function do the square
def valueSquared(myVar):
 fltSquared = myVar*myVar
 return (fltSquared)

Second, use this module:

import testmodule
dir (testmodule) #see what function in this module
help (testmodule.valueSquared) #show the comment above this function
testmodule.valueSquared(2.0) #will return 4.0

Another way to import the module:

from testmodule import*
valueSquared(2.0) #then you don't need to put the module name (prefix) all the time

9. OS module
OS module operates files:

import os
os.getcwd() #returns the current working dir
os.chdir('C:/temp') #changes dir
os.listdir('C:/temp') #list files

10. glob module

import glob
glob.glob('C:/temp/*.py') #glob can find the files using wildcard

ipython plotting

1. very important websites, containing large number of examples:
http://matplotlib.org/gallery.html
http://nbviewer.ipython.org/

2. to open the notebook:

> ipython notebook

to remotely open the notebook:
http://www.hydro.washington.edu/~jhamman/hydro-logic/blog/2013/10/04/pybook-remote/

3. keep the plots and the code in one page without open and new x-window:
put

> %matplotlib inline

at the top of the code.

4. workflow of creating a public web page for your code and plots:
> save the code as .ipynp
> paste the content to GitHub Gist: https://gist.github.com/
> click “get public gist” and get a number
> input this number into http://nbviewer.ipython.org/ and click “go”
example: http://nbviewer.ipython.org/gist/hydro-tian/8461570