Print Story I made myself a deal.
Boredom
By ana (Thu Sep 01, 2011 at 01:57:05 PM EST) Writing, work (all tags)
Accomplish this, or at least significant progress toward it, and take next week off. 


 And, I'm happy to say, I just got python to fit a linear regression to a scatterplot with no obvious trend (it looks flat, give or take, to my eye, with a value of about one) with 1.5 million points in it. Result: slope is 3.3 parts per million, offset 0.99677. Voila. Also no trend (for some values of "no") as a function of photon energy.

Yee, comma, haw. I've been groping around in python land for months looking for a way to do this.

So. Next week. Dunno what I'll do with myself. But things seem more hopeful some of the time, so that's good. I need to develop some options of stuff to do, things to pick among when I don't feel up to thinking up something new. I have a book half-read (that's how long the airplane ride was).

There are anniversaries coming up. One year. Ten years. I don't want to compound the angst by fretting about work I'm not getting done. And the last time off I took wasn't a vacation.

So far: dental appointment on Tuesday, to see if the deep cleanings from earlier in the summer are enough. And to see if this dentist actually exists. I got handed off to him when the previous one left the country, and the appointment has been postponed twice. And tomorrow (which is not next week, but), a follow-up with my normal doc about blood pressure issues (I'm thinking of going in before my first cuppa coffay in the morning), and then a massage. The muscles in my back are knotting up again. Rolling a hard rubber ball around between my back and a wall helps a lot, comma, but.

So I think I'll spend some time in the city, some time in the new coffee shop in my town. We'll see. The Cowley Fathers' monastery near my office has recently reopened after a year or two of renovations. Maybe I'll spend some time there.

And somebody at Toasted Cheese (toasted-cheese.com) suggested as a warm-up for nanowrimo, that we write a story at one-tenth the daily quota, which is 167 words a day, for a total of 5,000 in a month. That's something I can do while I'm doing other stuff.
< Reflections on Day 1 | Stuff doesn't work right. >
I made myself a deal. | 13 comments (13 topical, 0 hidden) | Trackback
While there's nothing wrong with by ni (2.00 / 0) #1 Thu Sep 01, 2011 at 02:52:49 PM EST
python, and I've written at least 80,000 lines in it, is there a reason you opted to do it this way? Presented with this problem I would have started an R session in under 3 seconds.


"These days it seems like sometimes dreams of Italian hyper-gonadism are all a man's got to keep him going." -- CRwM
WTF is R? by ana (2.00 / 0) #2 Thu Sep 01, 2011 at 03:05:18 PM EST
The other major candidate is an antique language known as IDL (interactive data language; see also GnuDL) . I needed to be able to read FITS-format files, which are widely used in astronomy and not anywhere else to my knowledge; they have ASCII headers and binary contents. Also, the locals were having a series of tutorials on python, numpy, etc. this spring, so I sat in because, hey, it's better than working, and I decided to use this project as a test case to get me to actually learn stuff. Python also has the advantage that a major fitting engine is written in it, and while that's overkill for what I'm doing here, it might be useful to be able to do this kind of thing for more conventional datasets.

I now know what the noise that is usually spelled "lolwhut" sounds like. --Kellnerin

[ Parent ]
R is by ni (2.00 / 0) #3 Thu Sep 01, 2011 at 03:22:56 PM EST
quoth wikipedia, "a de facto standard among statisticians for developing statistical software." It's free, it's easy, and it Just Works (although it does have some oddities). It supports FITS. While I'm sure that complications arise that make it more difficult than it sounds, loading data and fitting a linear regression to it is usually a 2-5 line affair in R -- it's the sort of thing it does really, really well.

Would you like to know more?

Google "R".


"These days it seems like sometimes dreams of Italian hyper-gonadism are all a man's got to keep him going." -- CRwM

[ Parent ]
Oddly enough... by ana (2.00 / 0) #5 Thu Sep 01, 2011 at 03:32:37 PM EST
googling R actually does turn this up. After the value of the ideal gas constant, of course. :-) 

I now know what the noise that is usually spelled "lolwhut" sounds like. --Kellnerin

[ Parent ]
Curiouser still, by ni (2.00 / 0) #6 Thu Sep 01, 2011 at 03:37:14 PM EST
I suspect you know the author of the R FITS library, being as you share a web host with him.


"These days it seems like sometimes dreams of Italian hyper-gonadism are all a man's got to keep him going." -- CRwM
[ Parent ]
been a long time since I dabbled with APL by clover kicker (2.00 / 0) #11 Fri Sep 02, 2011 at 10:44:49 AM EST
But it looks like a lot of the same ideas and power in a more legible form, and of course unreadability was the biggest knock against APL.

[ Parent ]
obligatory... by ana (4.00 / 1) #12 Fri Sep 02, 2011 at 12:39:25 PM EST
#$(!#$%&^!@$*NO CARRIER

I now know what the noise that is usually spelled "lolwhut" sounds like. --Kellnerin

[ Parent ]
If you're curious, by ni (2.00 / 0) #4 Thu Sep 01, 2011 at 03:29:49 PM EST
feel free to send me a subset of the data (or if you want, the Whole Damn Thing) and I'll send you the R incantation to generate the answer. It's a very useful tool for statistical problems, or anything that can generously be interpreted to be the second cousin of a statistical problem.


"These days it seems like sometimes dreams of Italian hyper-gonadism are all a man's got to keep him going." -- CRwM
[ Parent ]
The manual... by ana (2.00 / 0) #7 Thu Sep 01, 2011 at 03:40:22 PM EST
utterly fails to mention FITS files. 

I now know what the noise that is usually spelled "lolwhut" sounds like. --Kellnerin

[ Parent ]
Yes, it's by ni (4.00 / 1) #8 Thu Sep 01, 2011 at 03:47:34 PM EST
a module.


"These days it seems like sometimes dreams of Italian hyper-gonadism are all a man's got to keep him going." -- CRwM
[ Parent ]
Python and R by miker2 (2.00 / 0) #9 Thu Sep 01, 2011 at 09:22:57 PM EST
You can use R functions from Python.  RPy lets you have the expressiveness of python and the data-crunching of R.  There was a talk about this at the local PUG and the stuff the guy was doing with python and R was amazing. 

Ah, sociopathy. How warm, how comforting, thy sweet embrace. - MNS
[ Parent ]
Fitting engine by ana (2.00 / 0) #10 Thu Sep 01, 2011 at 09:30:53 PM EST
I should probably add that the locally-developed fitting engine is called sherpa, and it's implemented in python. It can be installed stand-alone (without all the other data-reduction software, etc.) in a python API environment.

I now know what the noise that is usually spelled "lolwhut" sounds like. --Kellnerin

obviously YMMV by clover kicker (4.00 / 2) #13 Fri Sep 02, 2011 at 01:07:18 PM EST
but I find this is a fantastic time of year to get outside. It's not too hot, the worst of the bugs are gone, and there aren't as many people out.

In fact I'm going camping/fishing this weekend.

Now obviously I'm a bigger redneck then most but there's still something about a vigorous hike and then sitting on a boulder for a couple of hours and watching the [tide come in|waterfall|lake|sunset].

I made myself a deal. | 13 comments (13 topical, 0 hidden) | Trackback