Max's Maddnes

it's here

2005-04-29T03:08:00.000-07:00

The steaks just got raised. Apple, your move.

instant fan, thanks to file sharing

2005-04-21T23:30:00.000-07:00

I occasionally indulge in some file sharing. My primary goal is usually to get electornic copies of books I own, which makes them much easier to reference. But occasionally I download movies on a whim or if I want to see something early. My moral rule is that if I really enjoy the show I end up buying or renting it when it's available, depending on how much I liked it. Right now I have an all-you-can eat two-discs-at-a-time account with Blockbuster, so I am not quite sure if my rental money gets distributed to the movies I rent in proportion, but I am sure some relation is there.
Anyways, here is a somewhat typical experience that whoever thinks about movie downloads should consider. But primarily it is a review for Battlestar Galactica miniseries DVD.
I heard of the show from NPR interview. I didn't get much of a hook from that, as I only got the tail end of the interview, but the name and the fact of a very strong fan base stuck. So while I was looking for something else on isohunt, I saw the torrent Galactica and clicked it. Now here is a key point: I thought this was a movie (and was even wondering when it was in theaters). Ok, I finally got to start it on the train this morning. With the minor glitch that the beginning of the 'movie' had a few racy scenes (no human gets naked. ok, no robot gets naked either, sorry :) which could be embarassing on a crowded LIRR train, the experience was TOTALLY CAPTIVATING. I finished the movie tonight. As I do not want to spoil this for anyone (hint: click the link above), here is a one sentece summary:

Deep, fascinating exploration human character facing its own possibly mistaken creation, which is complemented, not overpowered by the awesome special effects.

It's a run-on, but I tried my best :).
One of the big bonuses of watching the show as a movie (I did not realize this is a miniseries untill I looked it up to buy) was that I thought the transitions between scenes were extremely suspenseful and poignant. Also the number of subplots was staggering. Once I realized this is a mini I was almost a bit disappointed, as these things that made the movie so special seemed just a detail of doing a mini. But on the second thought I realized that the show was shot as a movie with mini in mind, not like some other shows that are shot as episodes and then pasted together. Goodbye, Star Trek. Orphaned fans, I think you just found a new home.

Garbage-Men

2005-04-12T04:13:00.000-07:00

I think the part of the job garbagemen hate the most is that it is so demeaning. After all, you are being paid to touch, smell, think about people's stinky garbage. Have you ever heard of a superhero with a 'garbagemen' secret identity? Photographers, reporters, retired .com-ers in bat costumes, yes. Garbagemen - NO! This explains why they like so much to make a mess on your lawn, 'accidentally' dropping garbage all over the place, throwing your cans/lids onto the street, forgetting to pick something up on time etc. Now they got you touching you smelly garbage, haha!

another blog

2005-02-27T22:17:00.000-08:00

Sorry to this to you people, but I split my blog. My current 'technical' hobbies, graph theory/networks and python will be blogged @ http://pythonzweb.blogspot.com/. The reason is that I wanted to add my blog to some of the composite python feeds such as planetpython.org, but at my current rate of 10% python posts if would not be acceptable. The new blog also has a feedburner link at the bottom of every post, please use that one!
thanks,
max

to throw or or not to throw

2005-02-24T21:28:00.000-08:00

Ok, I mean 'to raise', this IS a Python post.
I have recently gotten into graph programming, and been readin' the BGL book &amp;lt;br&gt;.
Not Python, but a great book on writing software libraries in general (with many c++ gems), and has some ideas about graph algorithms that would transfer into Python quite nicely. My favorite (so far) has been the concept of 'algorithm visitor'. As I cannot explain this any better than the BGL authors, I'll just quote:
""" The visitor concepts plays the same role in BGL as functors play in the STL. Functors provide a mechanism for extending an algorithm; for customizing what is done at each step of the algorithm. Visitors allow the user to insert their own operations at various steps within a graph algorithm. Unlike the STL algorithms, graph algorithms typically have multiple event points where one may want to insert a call-back via a functor. Therefore visitors do not have a single operator() method like a functor, but instead have several methods that correspond to the various event points""" More here...
So I hacked up an initial bfs implementation using the visitor concept:


def bfs(graph, start, bfs_visitor=default_bfs_visitor()):
    done = {start: True}
    queue = [(start, 0)]
    while queue:
    node, depth = queue.pop(0)
    bfs_visitor.discover_vertex(node, depth)
        for v in graph.adjacent_vertices(node):
        bfs_visitor.discover_edge((node, v))
        if(not done.has_key(v)):
        done[v] = True
        queue.append((v, depth+1))

default_bfs_visitor just implements dummy methods that bfs calls, (discover_vertex and discover_edge). User-defined visitors can selectively override the methods they are interested in.
My next thought was the termination condition. While by default the bfs algorithm would traverse the entire graph there are occasions to terminate early, for example if the node of interest was found or certain depth has been reached. Pygraphlib actually uses an 'end' parameter as a termination condition, which to me seems the 'node of interest' termination condition (their rationale is not well documented). I myself was interested in limiting bfs to a certain depth. First thought was to just add a max_depth paramter to bfs(). But looking at pygraphlib and my case I realized that pygraphlib picking the end vertex as termination and me picking a certain depth is kind of arbitrary. Of course I could provide two termination condition parameters, but this would both complicate and slow down the algorithm. So then I realized that the visitor can already be used for such termination. All I need is to define a special visitor that will, for example, make the bfs function exit when discover_vertex() is calle d with a depth greater than whatever value I initialize the visitor with. But how do I 'exit' bfs? One possibility was to have every visitor funtion return a false if exit is desired at this point. This did not sit right with me, though. I am more inclined to raise an exception in the termination case, which would effectively exit the function. The only problem that I see with this is that the function will now have to be called inside a try-catch block even thoug the exception is pretty much expected in this case. So this also seems wrong. What I finally came up with is the following: a standard traversal_termination_exception should be used by all the visitors that implement a termination condition. This way bfs() can just have a single try-catch for this particular exception, which keeps the code pretty clean. If a special exception really needs to be thrown instead of my traversal_termination_exception I think a Python 2.4 decorator could be useful (I will leave this as a homework exercise due to the hour ;).

unittesting

2005-02-24T19:26:00.000-08:00

Why is this guy the last one to come out of the LIRR train car, right before they they close the door and send the train to the depot? What's he been smoking? Unittesting!
That's right. I have been playing with a small python project that I might eventually release to the open source, if anything comes out of it. I decided to do unittesting from scratch and I was using Dive Into Python for reference - the book has a very nice section on unittesting. So I was sitting till the last minute on the train trying to get the lovely OK to come up, as I broke my first test during refactoring ;(.

what's not working?

2005-02-24T08:51:00.000-08:00

My wife just called me up saying 'google is not working' (she meant the Internet - I have periodic need to reset the router). Talk about brand recognition, those lucky bastards! Can you imagine someone saying 'Macy's is not working' 'cause the car won't start?

the other Ant

2005-02-23T21:09:00.000-08:00

Unofficial Apple Weblog featured ANT under FreewareFebruary . I downloaded it, it rocks! The videoblog space is pretty sparse as of yet (I am going to checkout some feeds tomorrow), but the possibilities!
P.S. I have been following the FreewareMonthX for a while, and Mac really has some unique and cool freeware. In certain categories, such as media/video and maybe some others it kicks other platforms' butts. In general I think the kind of freeware written for a platform indicates somthing about the platform, but for now I will refrain from further generalization.

check out the macros on this guy!

2005-02-23T20:47:00.000-08:00

Eric Niebler explains how his FOR_EACH macro proposal for boost works. This is a good example of code that is
- very powerful
- very complicated
- very dangerous
(very brilliant too, IMO)
Economically such code belongs in a good library repository such as boost. The fact that it is peer-reviewed by other very smart and experienced people (assuming FOR_EACH will get accepted) gives me good confidence that it is correct and useful, even though I do not have the skills to develop such code myself. (By 'economically' I mean here the 'economy of scale' and 'labor specialization' that organization like boost exhibits)

kids and programming

2005-02-21T15:30:00.000-08:00

Don Box wants to teach his kids programming. I recommended VPython.

your security health-check

2005-02-17T17:34:00.000-08:00

This is really cool, especially since it runs on this!

yum!

2005-02-17T17:15:00.000-08:00

I joined the rest of the crew in del.i.cious coolness!

sit a little, blog a little

2005-02-17T17:06:00.000-08:00

Waiting for the Apprentice, a funny thought comes to mind: Donald Trump Chiapet!

coolest python syntax trick

2005-02-16T21:44:00.000-08:00

a lotta pythonistas say this is the coolest

Konfabulator

2005-02-16T21:21:00.000-08:00

Nick Codignotto showed me Konfabulator a couple of month ago. I forgot about it for a while untill I realized that I have trouble dressing for the weather and it would really pay to see the forecast before I dash of to work. I hate using tv for this and my kid usually watches some show on my desktop before I leave, and I realized that K's weather widget was the ticket. After installing it it did a little more research and found a couple of interesting links:
A nice intro to making your own widgets
A good historical take on the Konfabulator vs. Dashboard contraversy
A Linux alternative (ouch K, this one is python-powered!)

google maps

2005-02-15T19:11:00.000-08:00

I finally came around to making a comment. IMO the feature and its association with local search is kickass and will earn google quite a few bucks. They are earning a lot of mindshare by staying on the bleeding edge with top-notch GUI (for the web).
In the recent google-map frenzy I found a couple of worthwhile links:
- a nice reverse-engineering of the map tech
- if you ever read Paul Graham (highly recommended, a lot of smart stuff and even more hubris), especially his LISP-related articles, you will laugh your ass of at this parody.

scary freakin' pizza

2005-02-14T20:49:00.000-08:00

Check out this demo. The only think its missing is POWERED BY
:)

why dmoz?

2005-02-06T09:12:00.000-08:00

I have been perusing parts of the . The book is very good, although I am very stale on some of the math. (Side note: to judge the book by its color, this is one of the best books I held. The color, paper type, weight, size and font are VERY nice. That is why I posted the full-color link.) It got me to think about a few web-related thoughts.
First, the best explaination of Google's PageRank algorithm I heard to date. Basically if you have a random traveler on the graph structure of the web (choosing a random link available from the current location) the page's rank will be proportional to the number of times this random traveler will stop on this page. Cool. For a somewhat different explaination (and a REALLY COOL animation see Doug Gregor's (of boost.org fame) project.
Second, the book mentioned the obvious limitations of human-edited directories such as dmoz.org. The real here killer is coverage - there is no way to get enough volunteer-power to cover a significant chunk of the Web to make the directory approach useful. So why is there still some push behind these? I think one of the big benefectors here is the search engines. Why? As the book shows from some studies even the search engines' coverage is not complete (for technical reasons). So let's put the pieces together: search engine graph traversal is not complete, but their ranking depends of the graph traversal. What to do? Why not seed the search with some nice, non-spammed results of volunteer work? That's what I think, anyway, would love to hear other opinions.
Follow-up: I wrote this up without googling it first. Interesing corraboration from jdMorgan here, and a great thread here.

a new windows ad

2005-02-03T01:30:00.000-08:00

here

DRM & ebooks

2005-01-30T10:48:00.000-08:00

I have been following DRM doscussion on Manning publisher's blog, and dropped my own comment in
"
A note on Safari: it is not a competition to your current method of distribution at all (even if they promote it as such). Their books are TOO annoying to reas AS books. What Safari is occasionally useful for is looking things up, sort of MSDN for open source technologies. Whether that justifies the pricetag I will leave to my employer, but they are not ebooks.
Another note on piracy. People who stuff their HDs with pirated ebooks are not lost customers (which is not the case with movies) - the effort to actually read a book is much greater. So while I am not saying that piracy is a big problem, I am suggesting that it cannot be judged by the traffic of pirated material on the networks.
Lastly, piracy's role as advertising should not be ignored. Software companies have long known this fact and incorporated it into the way they persecute violators. I occasionally use the networks as the "ultimate try and by" mechanizm - I downloaded a physics lecture from ripped from a teach12.com CD and that led to over a thousand $ of business for them.
"

Dude, where is my car?

2005-01-29T17:43:00.000-08:00

Last tuesday this guy decided to park his freakin' car right in front of my driveway. I had no choice but to dig around his car to get out. But what should I do with all the extra snow...

more stupid religion stuff

2005-01-25T18:34:00.000-08:00

This is a another great candidate for Darwin awards. They should have learned from these guys. It's 21st century, but stupidity marches on!

goolge this, BG!

2005-01-24T19:07:00.000-08:00

Well, whatever you think of the google browser rumors, I think it is good for those who do not want MS to take over the entire world to join forces. I only wish Apple got more on board and put itself behind Firefox, the way Google is doing. Not to say Safari ain't cool; I just think they should add their improvements via Firefox codebase. This will keep MS busy while Apple is biting (pun intended) into the PC market share.
I venture to say that if there is emotional weakness at MS it is about market share. Where other companies would occasionally find it smart to yield to a competitor in a specific area and consolidate their strengths I think MS would feel hurt both in terms of public perception and because of their leveraging strategies, where everything is *also* a means for something else. So yeah, hitting them hard in the browser area should do some damage.

wanna stone the devil?

2005-01-23T18:27:00.000-08:00

Well, this will should get the darwin awards in the religious nonsense category.

Guido @amazon devcon

2005-01-23T10:34:00.000-08:00

These are interesting summaries of what Python was/is
http://aws.typepad.com/aws/2005/01/amazon_devcon_g_4.html
http://aws.typepad.com/aws/2005/01/amazon_devcon_g_5.html

what happens if...

2005-01-22T22:06:00.000-08:00

You put a bunch of thugs, thievs, idiots and smirfs in the same building and give some of the most inept vetor power? You got it, the UN! Read about some of their accomplishments here.

calculus Flash-back

2005-01-20T20:56:00.000-08:00

In case you forgot your calculus you can get it back, in a Flash (animation!)

apprentice redux

2005-01-20T18:27:00.000-08:00

So, the Don is back. This time , there is a twist: "booksmarts" (hereafter BS) are competing against "streetsmarts" (SM). BSs lost the first round by a small margin, probably statistically insignificant. Don blurped out somthing very predictible about how he has respect for education, but maybe streetsmarts are better. So let me enter a few comments. First, these guys are not all that nerdy. Graduating college qualifies you as a BS for this contest. People who dropped out of college to pursue an opportunity are on the average smarter than someone who goes though college. So maybe you will tell me these BSs are super-nerdy. My impression, except for a couple of them, is that they aren't. And this is coming from a qualified nerd with the score of 81 :). So what do I think? I think there is a lot to be said for "streetsmarts", but I think there is a glass ceiling most of them, and only partially because of preception of others. This is because most businesses these days REQUIRE a pretty deep intellectual understanding of things that someone with just streetsmarts will not have. E.g. a lot of competitive differentiation in industries is technological. I also think that it was not always so. I think in the "old days" people with streetsmarts had a much better chance to ride on some success mixed with luck untill they learn their business well. But not these days. Just look at the CEOs of Fortune 500 companies, not many SMs there.

I wish I could be this sexy...

2005-01-18T20:41:00.000-08:00

Can I get this in a poster form?

your XML options in Python

2005-01-17T17:12:00.000-08:00

Here is a really good coverage. Not to sound corny or anything on this Martin Luther King day, but I will make the observation that 2 of the best-known python/XML experts, Uche Ogbuji and Paul Prescod, are in fact obviously black. I am sure the redneck idiots can have a theory for this one.

google and common words

2005-01-13T17:49:00.000-08:00

This is kinda interesting: I was curious to know how many PDFs google indexes. One simple way to find out is to look for a common word, so I tried "the filetype:pdf". The result - nada! Of course this has an obvious explaination, according to Shannon: the information content of a symbol is inversely proportional to the probability of occurence. And since "the" is the most common word in the English language, it is THE most meaningless. After a little googling I saw them say so themselves in the Automatic Exclusion of Common Words section. Being a curious monkey I decided not to take their word for it, and got some interesting results. Yes, they do not let you search on "the" in PDF, but allow it in HTML search despite their own disclaimer. Possible explaination is that they are changing their policy (so that people can find this) and have not updated their PDF index yet. But I dug deeper and realized that they have allowed "the" for quite a while, plenty of time to update their PDF index. Then it came to me: google index treats each HTML page as a single document AND every PDF FILE as a single document. Since PDF files are on average significantly longer than an HTML page, the probability of "the" in the document is greatly increased, making the "the" in PDF that much more meaningless than "the" in HTML. So how do I know how many PDFs google indexed? I just keep going down the list:

the nada
of nada
to nada
and nada
a nope
in ;(
s -
it -
you bingo!
about 22M files.
I think this is very close to the total number of PDFs in their index, certainly within an order of magnitude.

2005-01-13T17:48:00.000-08:00

Well, Bill Gates claims that strong IP is responsible for the success of American capitalism. I cannot argue that he is partially correct. But he has a lot of stake in saying what he is saying, so let's take a closer look. Many current objections to IP are about where IP is going, not necessarily where it was (which is the time period when many business successes Bill is talking about emerged). Another issue is the cost of IP.

still diverting

2005-01-12T22:50:00.000-08:00

I am slowly munging through my diversion, about 190 pages into it. In case you want a quick summary of some of the main points, you can get it from Joel, who recommended the book in the first place.

HP is still doing cool research

2005-01-12T20:12:00.000-08:00

As someone said, future is here, it's just not evenly distributed (yet). I am talking about media search. Previous attempts to do media search relied either on metadata, which is very poor or on SAP for TV feeds, which is much better that metadata, but still somewhat poor in relation to the content it represents. If you have just a little more resources than the rest, you can take a very resource-intensive technology such as speech recognition and and build a search engine on top of it. And since it's on the internet... does it mean it IS evenly distributed?

directory iteration

2005-01-06T10:29:00.000-08:00

Ever heard of a cool way to do directory iteration in Windoze? You can do it in Python, too. Your pick.......

blogmail???

2004-12-29T19:43:00.000-08:00

I recently posted a somewhat flaming comment on blogmaverick, to which someone else replyed. I took it offline and replyed to their top blog entry for lack of email, and he did the same to mine. Is this a strange new form of communication???

(I think not, but if was a funny experience and probably typical of the interesting accidental ways Internet brings people together)

Asia donation link

2004-12-28T22:09:00.000-08:00

Our (Liquidnet) CTO posted a donation link for the tsunami/earthquake victims. He got it from Joel Spolsky's mailing list, and I tend to think of Joel as a pretty thorough guy in terms of recommending things.

the rumors of c++ death are premature

2004-12-28T22:02:00.000-08:00

This year I noticed progressive degradation of the CUJ site. With some people's whining about the doom of C++ I (sadly) thought I had CUJ was going to join the C++ report in the programming language heaven. I was apparently wrong.

Google Scholar: good or evil?

2004-12-28T19:29:00.000-08:00

Evil:
All those links to the $10/doc papers locked in ACM. I guess if you are already a member this is good ;).
Good:
Links to find the book in your local library! Really good!

about 'third world' economies

2004-12-28T19:22:00.000-08:00

a friend pointed to this. IDL looks very cool, and there is an interview (text+audio) with De Soto here.

american values

2004-12-26T20:58:00.000-08:00

Just as a proof of AMK's sentiment about the way Americans value lives of people in other parts of the world, the reaction to today's '9' earthquake and the insuing tsunami drowning at least 12,000 people (12 going on 40 in places where communication is so poor, IMO) in Asia this morning was underwhelming, at least in the press... Nothing to spoil those afterchristmas sales spirit, huh?

good food for language junkies

2004-12-23T09:27:00.000-08:00

the 'Panel' is very interesting, and occasionally very funny.

the tale of two ebooks

2004-12-22T21:42:00.000-08:00

I recently made two ebook purchases: I got a speed-reading book from Amazon and a programming book from Manning. The experiences were very different. Amazon's ebook delivery took a nice while (while they were contacting the 3d party DRM system, I assume), right away eliminating one of the benefits of ebook buying: immediacy. What I got then is a special 'ancor' file sent to my email, which upon opening in Acrobat (strike 2: does not even try to work in Preview) required me to sign into MS-Passport and then eventually failed with a nondescript error. So much for Mac. I was able to get it to open on my PC (only a single PC - viewing on a second PC gave me a DRM error that I already downloaded the doc). After the file opened it was pretty useless, as any kind of printing or copying was disabled. In summary: don't do it, 's a freaking waste of money!
In contrast when I got a book from Manning there was an immediate download available, a beautifully typeset book, no DRM insanity. The only protection they implemented is putting my name & email address in every page, basically relying on my integrity. Excellent service, I wish there were more like them.

oreily hacks

2004-12-21T17:49:00.000-08:00

I own some books from O'Reily Hacks series, and like the format quite a bit. I gave a bit of thought as to why I like this very free-format style. This reminded me of a statistical method called Monte Carlo. The method was actually invented by mathematician Stan Ulam, and was instrumental in the development of the H-Bomb. The basic idea of the method is pretty simple, and the best practical explaination I know of is like this: say you have a very irregular shape that you need to measure the area of. Because the shape is so irregular it would be hard to use regular area-measurment algorithms on it. But we can do a pretty easy experiment which Stan proved will give us better and better approximations. First, we draw an easily measurable shape, like a rectangle around the irregular shape. Then we throw a dart randomly inside the rectangle. We record whether it ended up inside the irregular shape. Repeat many times. Tally up how many times the dart ended up inside the shape of interest. The area can be approximated by Area-Of-rectangle*(Dart-Hits-Inside-Weird-Shape/Total-Dart-Throws). This guess will get more and more accurate with the number of tries.
How does this relate to the O'Reily books? I think the disorganized format gives you a good idea of the overall area without covering it in an organized manner. This is similar to the dart throw in the Monte Carlo method. By having multiple somewhat random articles about the subject this format gives you a broad and accurate overview of an area which in some sense is more compact and broad than a well-organized book on the subject.

I am sick...

2004-12-20T20:44:00.000-08:00

I have NADD++

a diversion

2004-12-20T19:39:00.000-08:00

Finally I cracked and decided to blog this. Following Joel Spolsky's recommendation and my habit of occasionally reading business books I got the classic

It's got to be the book on the subject because I did not die of boredom and
actually have enough interest to keep on reading, despite the semi-technical flavor of the book. The secret is that the technicalities are first explained conceptually, then mathematically and then reinforced by examples, which make the whole thing digestible. Another reason I like the book is that it explains things about pricing that you see in the real world, which the consumer (me) better ways to think about values of things and the mental mechanizms businesses use to occasionally get you. The chapter I was reading just right now is on 'reference point' that people use making pricing decisions. The basic idea is simple: we use the price of competing products as a reference point for eveluating other products. Their real-life examples are great, here is a couple:
"Vasiline Intensive Therapy" lip balm is 1400% (no mistake in zeros)
more expencive than a regular tub of vaseline, which is the same
stuff. What allow this markup is competition with "Chapstick", which
sets the reference price.
Another good example is Jelly Belly candy. You will never see them sold in your grocery store next to other jellybeans - the huge markup against the generic reference point would not allow this. Instead they rely on being sold in gourmet candy stores where the reference point is going to be, for example, some expensive chocolates. Another thing not openly mentioned in the book (so far) is that
reference point is really a relative term.

not-so-iron python for .NET

2004-12-20T19:28:00.000-08:00

This certainly deserves a mention. This implementation of python, while not compiling into CLR has pretty much full access to the .NET platform, plus .NET events can be handled and delegates can be written in python.

sync my stuff?

2004-12-20T19:21:00.000-08:00

A while ago I decided that rsync is The Tool for backing up my stuff, like my large collection of interesting PDF files or my programming adventures. Of course I have not so far taken the time to learn the tool and relied on manual copying and prayer for backups. Now since there is a Python implementation
maybe I will get somewhere with this!

pythnon scraping follow-up

2004-12-20T09:21:00.000-08:00

So c.l.python was helpful as usual. What I was missing was the default namespace prefix in my xpath expression - I have to do some reading up on xpath/xml namespaces interaction. In the meantime I asked them back what would happen if I simply yank the namespace from the file for now to make the xpath-ing easier. Their concern was about potential multiple namespaces, which is not a problem for me as Tidy does not produce multiple namespaces anyway. So I got xpath to play along after I removed the default namespace.
On to the next step, finding the right xpath expression to get what I need. BTW the library I have been using is xmllib2, which currently seems to have a lot of momentum. The next problem is finding the right xpath for the parts of the document you want. I used a couple of editors to look at the document's tree, and the nesting was pretty horrendous. If I was an xpath expert perhaps I would be able to crack this easily, but I felt a need for some kind of automation. I thought it would be cool to find an editor with 'xpath reverse-engineering', which would allow you to point to a node graphicaly and have an xpath suggested to you. Of course the theoretical problem with this is possible multiple xpaths that could point to a node, but I had a feeling that some 'reasonable' solution is possible here. I did some googling around and came up with 0/google results. Admittedly I have not tried XmlSpy, which I am told is The Tool for XML. Then I somehow remembered about something I read about a tool that allows you to navigate XML like a file system from the command-line. That somehow sounded neat, even though it did not solve my problem directly. I quickly 'remembered' what the tool was (using google) and fired it up - 'xmllint --shell input.xhtml'. This gives you a prompt and a lod of the standard shell navigation works like you expect: ls lists the current node's subentities, cd allows you to jump to another node, pwd gives you the current path. I wanted to see the rest of the commands, asked the shell for help, and (drum roll...) here it was - 'grep' command! I immediately realised that this is what I was looking for - you could 'grep' on a node's CDATA and get the xpath to it. The answer to the multiple potential xpaths was also right there - there is a 'simple' xpath to a node which can be obtained by navigation from the root without any wildcards or attribute matching stuff. So now I am almost there. What remains is finding a slightly more abstract xpath that would select ALL the nodes that I am interested in, so that I can process the whole list of them in a loop. To be continued...

my latest stuckness

2004-12-15T23:24:00.000-08:00

I just posted the following to c.l.python. Hope they can help...

"I am trying to do some xpath on
http://fluidobjects.com/doc.xhtml
but cannot get past 'point A' (that is, I am totally stuck):

>> import libxml2
>> mydoc = libxml2.parseDoc(text)
>> mydoc.xpathEval('/html')
>> []"

The purpose of this is to improve my www-scraping skill (I'd like to scrape the audio from HOPE 5 grabbing mp3, author info and description and importing directly into my iTunes). I pretty much settled on the idea that the ideal route for scraping HTML is HTML | Tidy | XPath. Tidying in python took a surprising while to get right (it has a lot of keyword optoins which are not reflected into the python wrapper ergo the wrapper is not self-documenting, which is a strike), so now I am stuck on the XPath thingy. If I do not get unstuck it's back to plan B.

blockbust

2004-12-15T03:23:00.000-08:00

For one reason or another I get a bit nostalgic when I witness a great big business in a downward spiral. The one I am thinking of right now is Blockbuster. Their main business model is obviously at the end of the rope - who wants to shlep to the video store and only to find that what they want to rent is not available and if it is available it is scratched up and if you even get to watch it you are pretty likely to be hit with an unexpected late fee which is almost impossible to fight if they are wrong. But when there were no other options we were pretty happy going to BB, the store was clean and well taken care of and there was always an exitement of getting a movie. Or maybe that was just what being a kid feels like.
Anyways, why do I think BB is dead? Aren't they copying the successful model of Netflix?
The brilliance of Netflix was not the idea, but the timing. The idea of delivering movied via mail must have seemed crazy in the time when everything is going towards digital delivery. But Netflix saw the opportunity to gain market share in a small window created by the upheaval of physical-to-digital distribution switchover. This was not their ultimate goal, I venture, and I think the proof is in their recent teaming up with Tivo to do digital delivery. So while in short run BB can suck some market share off them (mostly opportunity market share, as I doubt current netflix customers will be wooed by the BB offering) they are fighting yesterday's battle. The company's creative spirit appears stagnant and while there always room for turnaround, the things are not looking good...
But in the end the decline of the companies is overall a good thing. Without it the world would be dominated by powerful and monopolies and invention and attendant economic improvement would stop. So, goodbye, BB (maybe)!

Swear this wasn't photoshopped!

Channukah

2004-12-13T17:15:00.000-08:00

Here is a Happy Channuka from CLISP (albeit it is one day off, at least if you light at sunset):

And if you forgot your siddur (prayer book) and cannot for some reason remember the Rosh Chodesh Musaf - there is some help from Amazon

most funnest LISP tutorial EVER (with Macros!)

2004-12-11T23:16:00.000-08:00

Go check it out!

python maddness

2004-12-11T18:38:00.000-08:00

I recently send around a bug I found (in my own code!) to some of my collegues as a puzzle and hid the answer in the end in the following obfuscatedpython bit:
"You will get the answer when you execute this from command-line:

python -c "import new, sys;print (lambda s: s and s[-1]+new.function(sys._getframe().f_code, globals())(s[:-1]) or s)('.srotarepo noisrevnoc fo eraweB :nosseL \n.tcejbo TNOFH gniylrednu eht llik lliw rotcurtsed s\'tnoFC lanigiro eht dna ,ton si tI \n.tcerroc yllacitnames si tnemngissa tnoFC taht kniht uoy sekam rotcurtsnoc gnitpecca-TNOFH eht htiw noitanibmoc ni rotarepo noisrevnoc TNOFH\n')"

The only really cool (and hopefully useless :) thing here is the list/string reverse function written as a recursive lambda. Of course there is no normal way to call a lambda in python recursively since it doesn't have a name, so you have to poke around the current frame object to get what you want. So you get this beauty:

(lambda s: s and s[-1]+new.function(sys._getframe().f_code, globals())(s[:-1]) or s)

I think I'll go take a shower now :)

The sounds of thunder...

2004-12-08T06:15:00.000-08:00

Installed Thunderbird 1.0. It seems NOTICABLY faster than the previous version (particularly on windoze), just like Firefox did when you switched to 1.0. Coincidence? Whatever it is, can's stop the open source! Yippie-ki-yay , Mister Falcon!

See KILL BILL (2)

2004-12-06T22:48:00.000-08:00

(note: this was written a few month ago, and is even more poorely edited than my other stuff. I am sorry you are reading it)

This one was worth the wait. (Side note: apparently I was not the only one waiting for it. I went to Blockbuster 3 times to pick it up and there were no copies left. I assume Blockbuster anticipated the initial spike in demand, because the movie was not "guaranteed in stock" as many other new releases are. They probably figured that the due to the "spiky" nature of demand here they will be stuck with a bunch of unrentable copies in the end. This makes sense except for the fact that this movie is bound to become a cult classic and people will want to buy copies to own. But I will let their actuaries do their job. Anyway, the point I was getting at was that I HAD to download the movie ILLEGALLY just to watch in when it came out. (I did rent a copy the following week, which I usually do since I want studios to make more good ones. Besides the quality was much better for the second viewing.). This is a hint to the studios (anyone listening?) about electronic distribution solving at least the demand vs. physical copy mismatch. Also I actually signed up for CinemaNow a while ago, but it appears that CinemaNow and Movielink are getting the shrift here, because the movie is not available electronically when the demand is the highest. It is available on CinemaNow sometime in September, but if you haven't watched it by then you are not really KILL BILL material.). Anyway, back to the movie. If you saw KILL BILL part 1 you only saw half the movie, and probably a the wrong half. Steven King suggested multiple times in EW that this is not 'great cinema', making comparisons to "Mystic River" (which he thought was great. I did too). I seriously wonder if he regrets saying that after seeing part 2. This movie goes back and puts an incredible amount of emotions (plural, as there is much more than revenge here) into the storyline. (don't worry, the action sequences are still awesome). And the cinematography and acting is incredible.
(for those who don't mind me spoiling it: the scene-before-last can qualify as one of the most powerful movie scenes of all time IMO. And here is the (un)real QT twist: the fight scene that ends with BILL being killed is actually a very romantic scene - and we owe that to the "Pai Mei five point exploding heart technique". This technique, when used on an opponent allegedly guarantees a sure death from a "hear explosion" once the opponent takes a few steps (5 or 8 IIRC). This incredible Kung-Fu move (by QT) allows us to see the relationship between Bea and BILL in it's pure raw form: you actually see how much they love each other because there is no other considerations in play - BILL will is effectively dead (but still alive and talking untill he takes the few fateful steps), which frees both him an and Bea his twisted path and allows a sort of greatness to come out. And the name: "exploding heart" technique - can this really be a coincidence? Wow, totally ingenious. The next scene of Bea just crying her heart out is a good reinforcement.)
See Kill Bill 2. QED.

post of the day

2004-12-06T14:41:00.000-08:00

This really made happy - my feelings on Python vs Java are strongly reinforced now:
http://dirtsimple.org/2004/12/python-is-not-java.html
Of course I do not really know Java. But this explains why :)

a simple problem

2004-12-02T20:42:00.000-08:00

My first blog post. Wonder if anyone will read.
I live on Long Island and work in Manhattan (for Liquidnet, a pretty kick-ass company). As this winter has been pretty warm New York felt quite a bit like Seattle (I lived there for 4 years): wet and overcast. All this wetness created the following problem: umbrellas. Well, umbrellas themselves are not the problem, but having one always available is. Of course, some people solved this by having a super-tiny umbrella that they can always carry around. But the recent rains have been accompanied by quite a bit or wind, so I need my Giant Umbrella. That said, the question is where shold I keep it. If I keep it at home and it rains on the way back from work I am screwed and vice versa. What about keeping 2 umbrellas: one at home and one at work? That should work ( and working at Liquidnet one can afford at least two umbrellas :). But this scheme would require always insuring that both locations are umbrella-equipped, which may require me to shuttle my Giant Umbrella next time I go back to the location after the rain forced me to take the umbrella from there. Since this shuttling may be occuring on an obviously sunny day it will shock my co-passengers on Long Island Rail Road, an effect I try to avoid having on people. Solution? More umbrellas! What if I have, say, a 100 umbrellas at each location? Assuming random distribution of rain activity what should happen is that for every umbrella I take from location A to B due to rain I should eventually take one from B back to A. Actually as I thought about this I realized that this is another reincarnation of the Random Walk problem. This final realization is really the reason I am posting this garbage: the moral of the story is that complicated mathematics is all around us, if we only think a little harder.