Python magic / advanced numpy indexing

Yeah, indexing python arrays should really be easy.

Stefan van der Walt's page


In [85]: bi = (f.bolo_indices[np.newaxis,:] + zeros([7751,1])).astype('int')

In [86]: whc = (whscan[:,np.newaxis] + zeros([1,107])).astype('int')

In [87]: array2d[whc,bi] = temp2d


Python: one-line arrays

Ahhh, refreshing:

whscan = asarray([arange(scanlen)+i for i in scans_info[:,0]]).ravel()


Not like IDL, which takes at least 4 lines b/c of the variable declaration. There's probably a better way to do that too.

Port vs Fink

Port failed to install:

adam-macbook ~$ sudo port install py25-matplotlib
---> Building py25-hashlib with target build
---> Staging py25-hashlib into destroot
---> Installing py25-hashlib 2.5.2_0
---> Activating py25-hashlib 2.5.2_0
---> Cleaning py25-hashlib
---> Fetching py25-zlib
---> Verifying checksum(s) for py25-zlib
---> Extracting py25-zlib
---> Configuring py25-zlib
---> Building py25-zlib with target build
---> Staging py25-zlib into destroot
---> Installing py25-zlib 2.5.2_0
---> Activating py25-zlib 2.5.2_0
---> Cleaning py25-zlib
---> Fetching py25-setuptools
---> Attempting to fetch setuptools-0.6c8.tar.gz from http://cheeseshop.python.org/packages/source/s/setuptools/
---> Verifying checksum(s) for py25-setuptools
---> Extracting py25-setuptools
---> Applying patches to py25-setuptools
---> Configuring py25-setuptools
---> Building py25-setuptools with target build
---> Staging py25-setuptools into destroot
---> Installing py25-setuptools 0.6c8_2
---> Activating py25-setuptools 0.6c8_2
---> Cleaning py25-setuptools
---> Fetching py25-tz
---> Attempting to fetch pytz-2007k.tar.bz2 from http://cheeseshop.python.org/packages/source/p/pytz/
---> Verifying checksum(s) for py25-tz
---> Extracting py25-tz
---> Configuring py25-tz
---> Building py25-tz with target build
---> Staging py25-tz into destroot
---> Installing py25-tz 2007k_0
---> Activating py25-tz 2007k_0
---> Cleaning py25-tz
---> Fetching py25-dateutil
---> Attempting to fetch python-dateutil-1.4.1.tar.gz from http://labix.org/download/python-dateutil
---> Verifying checksum(s) for py25-dateutil
---> Extracting py25-dateutil
---> Configuring py25-dateutil
---> Building py25-dateutil with target build
---> Staging py25-dateutil into destroot
---> Installing py25-dateutil 1.4.1_0
---> Activating py25-dateutil 1.4.1_0
---> Cleaning py25-dateutil
---> Fetching fftw-3
---> Attempting to fetch fftw-3.1.3.tar.gz from http://www.fftw.org/
---> Verifying checksum(s) for fftw-3
---> Extracting fftw-3
---> Configuring fftw-3
---> Building fftw-3 with target all
---> Staging fftw-3 into destroot
---> Installing fftw-3 3.1.3_0
---> Activating fftw-3 3.1.3_0
---> Cleaning fftw-3
---> Fetching py25-nose
---> Attempting to fetch nose-0.10.4.tar.gz from http://somethingaboutorange.com/mrl/projects/nose/
---> Verifying checksum(s) for py25-nose
---> Extracting py25-nose
---> Configuring py25-nose
---> Building py25-nose with target build
---> Staging py25-nose into destroot
---> Installing py25-nose 0.10.4_0
---> Activating py25-nose 0.10.4_0
---> Cleaning py25-nose
---> Fetching py25-numpy
---> Attempting to fetch numpy-1.2.0.tar.gz from http://downloads.sourceforge.net/numpy
---> Verifying checksum(s) for py25-numpy
---> Extracting py25-numpy
---> Applying patches to py25-numpy
---> Configuring py25-numpy
---> Building py25-numpy with target build
Error: Target org.macports.build returned: shell command " cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_python_py25-numpy/work/numpy-1.2.0" && /opt/local/bin/python2.5 setup.py build " returned error 1
Command output: setup_package()
File "setup.py", line 89, in setup_package
configuration=configuration )
File "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_python_py25-numpy/work/numpy-1.2.0/numpy/distutils/core.py", line 184, in setup
return old_setup(**new_attr)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/core.py", line 151, in setup
dist.run_commands()
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", line 974, in run_commands
self.run_command(cmd)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", line 994, in run_command
cmd_obj.run()
File "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_python_py25-numpy/work/numpy-1.2.0/numpy/distutils/command/build.py", line 37, in run
old_build.run(self)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/command/build.py", line 112, in run
self.run_command(cmd_name)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/cmd.py", line 333, in run_command
self.distribution.run_command(command)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/distutils/dist.py", line 994, in run_command
cmd_obj.run()
File "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_python_py25-numpy/work/numpy-1.2.0/numpy/distutils/command/build_src.py", line 130, in run
self.build_sources()
File "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_python_py25-numpy/work/numpy-1.2.0/numpy/distutils/command/build_src.py", line 147, in build_sources
self.build_extension_sources(ext)
File "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_python_py25-numpy/work/numpy-1.2.0/numpy/distutils/command/build_src.py", line 250, in build_extension_sources
sources = self.generate_sources(sources, ext)
File "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_python_py25-numpy/work/numpy-1.2.0/numpy/distutils/command/build_src.py", line 307, in generate_sources
source = func(extension, build_dir)
File "numpy/core/setup.py", line 87, in generate_config_h
raise SystemError,"Failed to test configuration. "\
SystemError: Failed to test configuration. See previous error messages for more information.

Error: The following dependencies failed to build: py25-numpy py25-tkinter tcl tk
Error: Status 1 encountered during processing.


I don't want to deal with it so I'm just using Fink.

Reinstalling a system

Python died tonight when I foolishly tried to update numpy or matplotlib. One was not compatible with the other in upgraded form, and the update for matplotlib wouldn't install because of GTK issues that were totally opaque.

So, I reinstalled EVERYTHING - fink AND macports - from scratch. If I don't get things running by morning I'm going to have to do a restore, which is ugly as death.

Things I need to do post-install:

sudo port install python_select
sudo python_select python25


test ipython
test matplotlib
test numpy

Python: setting matplotlib defaults

Setting matplotlib defaults is a lot more difficult than it should be.


matplotlib.defaultParams['image.origin']='lower'
matplotlib.defaultParams['image.interpolation']='nearest'


Observing 10/20

I don't have a better place to post this one, so here it is:
http://sites.google.com/site/iras05358/figure-discussion/figures/iras05358_triple_17.29.png?attredirects=0


My automated fitter (Gaussfitting Cube Collapser) has come a long way. I now adaptively choose to fit 1, 2, or 3 Gaussian components to output to a data cube.

The purpose of that code is primarily to find a two-dimensional way to display information about the 3D structure, specifically about the presence/absence of outflows. Outflows will inevitably be confused with multiple velocity components, but they are also likely to be convolved with them.

Screen, nohup, ssh, scp

I learned a lot about the above in the past day, but I didn't keep track of the links.

First, screen is very useful: it allows you to run any task, detach the screen, and let it run in the background. You can resume it later.

Example:

screen
ipython run_fitter.py
<ctrl-a> d
screen -r


Second, it's a huge pain to type a password every time I use scp and ssh. The solution is to make a key on your computer and put it in the authorized_keys file.


ssh-keygen -t rsa
scp ~/.ssh/id_rsa ginsbura@milkyway.colorado.edu:.
ssh ginsbura@milkyway.colorado.edu
cat id_rsa >> ~/.ssh/authorized_keys
ssh -v ginsbura@milkyway.colorado.edu


Use ssh -v to figure out why it fails if it still asks you for a password. In one case, this happened because the computer I was using expected the id to be in ~/.ssh/identity instead of ~/.ssh/id_rsa. There may also be permissions issues (i.e. you want restrictive permissions on your ssh keys).

With the latter, you can still use nohup, which is helpful if you want to pipe your output to a log file.

Listing variables (e.g IDL help) in Python

Again, IDL has the simple 'help' command to tell you all variables in your namespace. Python has the same thing, but the namespace tends to be cluttered with imported functions. The commands who, who_ls, and whos are meant for interactive use. They are a hell of a lot more useful than var, locals, globals, and dir.

examples:

whos float
whos ndarray
who module
float_vars = %who_ls float
grep('x',float_vars)


I'm afraid I don't know how to make the last two lines into a one-liner, as would be desirable.

SAVE / RESTORE in Python

Save/Restore is probably the single best feature of IDL that, sadly, is very poorly replicated in Python. For 1 or 2 dimensional variables, you can use Pylab's save/load, but I never use such piddling tiny arrays. For higher dimensional objects, either using FITS files (a pain because of header definitions) or pickling ought to work.

e.g.:

import numpy
import pickle
x=ones([10,10,10,10],dtype='float64')
pickle.dump(x,open('x.pysav','w'))
X = pickle.load(open('x.pysav','r'))


Last month's observations in L111

Mapped sources:
SH 2-156 [no high resolution observations; covered by 1.2m CO 1-0 survey]
SH 2-157 Yancy CS 5-4 , CO, 13CO 1-0, CS 2-1 , HCN & SCUBA too....
IRAS 23151 (two pointings)

Unmapped:
BFS 18 = IRAS 23042+6000 = IRCO 859
Wu et al mapped this in CO 2-1 and detected no outflows. They used the NRAO 12m with a heterodyne array with very high T_sys ~700-1100K.

How do I print a multi-page postscript?

I want to print a multi-page postscript duplexed and, if possible, two per page. pstopng, pstopdf, and the variants failed me horribly by making postscripts that don't fit the bounding box at all. Any ideas?

So far the best I've come up with is switching gnuplot's output to pdf, which is a workaround rather than a solution, but is useful nonetheless:
set terminal pdf enhanced dashed

Mathematica & Series expansions

Mathematica does something stupid and incomprehensible (to me) with their series expansions. For no clear reason, it makes the O[x]ⁿ uneditable, and adds a bunch of formatting things that prevent you from copying & pasting the expression and then solving for a variable. Argh. While this is repairable using the "show expression" tool (Shift-Apple-E), it's a pain and "show expression" is not meant to be human readable.

teaching

this is the only right way to teach

gardening

I realized I don't have a gardening category and I should.

I have eaten 2 garden tomatoes. There have been a total of 3 that ripened. There is a fourth on the vine. There are ~40-50 big green ones being big and green.

I ate basil in my pasta sauce yesterday. I need to buy some pine nuts. But I'm not pining for nuts.

Next year, start growing in March!

sage casjobs

annoying feature of sage casjobs: no spaces are allowed, no matter what, even if quoted. so when importing must remove all spaces. gurrh.

mathematica

//. and /. use 'rules' of the form {x->y} for substitutions

how do you make the layout of a page look nice? How do you change line spacing?