Sunday, April 28, 2013
Today I drew.
To follow up on last Thursday's post, today I drew an electronic version of the picture I'd been drawing on whiteboards around the office at the end of the week. I suspect this will appear in a talk about MICs some time soon.
Thursday, April 25, 2013
Today I Learned.
And, I actually prefer it that way.
I relearned the lesson of Amdahl's Law, and I learned a new word, "bathetic". No, I didn't spell that wrong. Bathetic, with a 'B'. Which, in a similar but not exactly the same vein as 'pathetic', draws on the Greek word 'bathos' meaning depth. But what it really means is an abrupt, absurd fall into mediocrity or chaos.
The basic lesson of Amdahl's Law is that parallel computing is hard. The slightly more than basic lesson is that adding additional simultaneous processing to a problem only helps in so far as there is simultaneity to be had. Which is to say that if half of your work is completely serial and half is completely parallelizable, you can only expect to halve your run time with all the additional processors in the universe.
Today's lesson from Amdahl takes a bit of a jag. If the gods give you a magic computer that does the parallel portion of your calculation infinitely fast, effectively reducing its run time to zero, but it runs the serial portions of your code slower, then you should think carefully before making promises about what that gift can do. Even if this computer instantly returns the result of the parallel part, if it also doubles the run time of the serial part of the code, then the gift is a wash if half the work is serial and half the work is parallel. Fortunately, in the real world, the best scientists build codes where more than 99.999% of the work is parallelizable.
Unfortunately, this is, in some ways, the problem with GPUs, Xeon Phis, FPGAs, and other computing accelerators. They do some things extremely well, and while they don't reduce the parallel run times to zero, they may cut them by significant factors. But to do this, they have to sacrifice performance on serial execution. If you don't watch out for this, you can get killed by it, bathetically.
I relearned the lesson of Amdahl's Law, and I learned a new word, "bathetic". No, I didn't spell that wrong. Bathetic, with a 'B'. Which, in a similar but not exactly the same vein as 'pathetic', draws on the Greek word 'bathos' meaning depth. But what it really means is an abrupt, absurd fall into mediocrity or chaos.
The basic lesson of Amdahl's Law is that parallel computing is hard. The slightly more than basic lesson is that adding additional simultaneous processing to a problem only helps in so far as there is simultaneity to be had. Which is to say that if half of your work is completely serial and half is completely parallelizable, you can only expect to halve your run time with all the additional processors in the universe.
Today's lesson from Amdahl takes a bit of a jag. If the gods give you a magic computer that does the parallel portion of your calculation infinitely fast, effectively reducing its run time to zero, but it runs the serial portions of your code slower, then you should think carefully before making promises about what that gift can do. Even if this computer instantly returns the result of the parallel part, if it also doubles the run time of the serial part of the code, then the gift is a wash if half the work is serial and half the work is parallel. Fortunately, in the real world, the best scientists build codes where more than 99.999% of the work is parallelizable.
Unfortunately, this is, in some ways, the problem with GPUs, Xeon Phis, FPGAs, and other computing accelerators. They do some things extremely well, and while they don't reduce the parallel run times to zero, they may cut them by significant factors. But to do this, they have to sacrifice performance on serial execution. If you don't watch out for this, you can get killed by it, bathetically.
Wednesday, April 24, 2013
Tuesday, April 23, 2013
Sometimes I'm heartened to see a few of the people I read come together for good. Lawrence Lessig and Orin Kerr wrote a great piece for the The Atlantic yesterday that's worth highlighting.
They had a pretty strong disagreement over whether Aaron Swartz violated the CFAA or not, so seeing a former federal prosecutor and an activist for social change come together on this issue made me smile.
They had a pretty strong disagreement over whether Aaron Swartz violated the CFAA or not, so seeing a former federal prosecutor and an activist for social change come together on this issue made me smile.
Monday, April 22, 2013
Self-Scooping
Based on this post over at the Computational Science StackExchange, I had left myself a note here to write a post on how easy it is to make divergence-free fields for use as manufactured solutions for testing incompressible Navier-Stokes solvers, but since the original poster went ahead and asked the question directly, I answered it.
Some background might be in order. The incompressible Navier-Stokes equations are often written:
$$
\begin{align}
\frac{\partial\boldsymbol{u}}{\partial t} + \boldsymbol{u}\cdot\nabla\boldsymbol{u} &= -\frac{1}{\rho}\nabla p + \nu\nabla^2\boldsymbol{u} + \boldsymbol{f} \\
\nabla \cdot \boldsymbol{u} &= 0
\end{align}
$$
These are the classical equations of fluid mechanics for fluids that don't compress very much (like water). This condition is represented by the second equation above and is the force behind making them somewhat tricky to solve (analytically and numerically). The other thing that makes these equations tricky to solve, is the non-linear term in the first equation: \( \boldsymbol{u}\cdot\nabla\boldsymbol{u} \).
When we computer jocks want test the validity of our programs for approximating these equations, we would really like to have some known correct answers to compare our numerical solutions to. Unfortunately, the non-linearity and incompressibility condition conspire to make it hard to come up with a good set of exact solutions to compare to. It is, in fact, an open problem with a substantial prize behind it to determine whether such solutions even exist, or if they do, under what conditions.
The Method of Manufactured Solutions (PDF) is a technique for generating exact solutions to compare your numerical method results to. With MMS, you pick (virtually) any function you like \(\boldsymbol{u}\), substitute it into the governing equations, and solve for the \(\boldsymbol{f}\) that makes that \(\boldsymbol{u}\) a solution. Since \(\boldsymbol{f}\) was a tunable knob in your code that allowed you to solve different kinds of problems, it's a place for making adjustments for testing it, too.
This method works great when every equation you want to solve has a convenient forcing term like \(\boldsymbol{f}\), but as we can see above, not every equation does. Sometimes you get lucky though, and you can create a solution that satisfies the homogenous equation exactly and can be plugged into the other equation to find the right forcing term.
In this case, the choice is to either pick your favorite (sufficiently differentiable) vector field, and take its curl, or pick your two favorite scalar functions and take the cross product of their gradients. That is, either take
$$
\boldsymbol{u} = \nabla \times \boldsymbol{A}
$$
or$$
\boldsymbol{u} = \nabla g \times \nabla h
$$
and crank it all through.
I highly recommend a symbolic manipulation program for this, though. There's lots of room for mistakes.
Some background might be in order. The incompressible Navier-Stokes equations are often written:
$$
\begin{align}
\frac{\partial\boldsymbol{u}}{\partial t} + \boldsymbol{u}\cdot\nabla\boldsymbol{u} &= -\frac{1}{\rho}\nabla p + \nu\nabla^2\boldsymbol{u} + \boldsymbol{f} \\
\nabla \cdot \boldsymbol{u} &= 0
\end{align}
$$
These are the classical equations of fluid mechanics for fluids that don't compress very much (like water). This condition is represented by the second equation above and is the force behind making them somewhat tricky to solve (analytically and numerically). The other thing that makes these equations tricky to solve, is the non-linear term in the first equation: \( \boldsymbol{u}\cdot\nabla\boldsymbol{u} \).
When we computer jocks want test the validity of our programs for approximating these equations, we would really like to have some known correct answers to compare our numerical solutions to. Unfortunately, the non-linearity and incompressibility condition conspire to make it hard to come up with a good set of exact solutions to compare to. It is, in fact, an open problem with a substantial prize behind it to determine whether such solutions even exist, or if they do, under what conditions.
The Method of Manufactured Solutions (PDF) is a technique for generating exact solutions to compare your numerical method results to. With MMS, you pick (virtually) any function you like \(\boldsymbol{u}\), substitute it into the governing equations, and solve for the \(\boldsymbol{f}\) that makes that \(\boldsymbol{u}\) a solution. Since \(\boldsymbol{f}\) was a tunable knob in your code that allowed you to solve different kinds of problems, it's a place for making adjustments for testing it, too.
This method works great when every equation you want to solve has a convenient forcing term like \(\boldsymbol{f}\), but as we can see above, not every equation does. Sometimes you get lucky though, and you can create a solution that satisfies the homogenous equation exactly and can be plugged into the other equation to find the right forcing term.
In this case, the choice is to either pick your favorite (sufficiently differentiable) vector field, and take its curl, or pick your two favorite scalar functions and take the cross product of their gradients. That is, either take
$$
\boldsymbol{u} = \nabla \times \boldsymbol{A}
$$
or$$
\boldsymbol{u} = \nabla g \times \nabla h
$$
and crank it all through.
I highly recommend a symbolic manipulation program for this, though. There's lots of room for mistakes.
Sunday, April 21, 2013
This may be the best talk with the worst ending I've ever seen. After more than thirty-one minutes of inspiring talk on copyright problems (yeah, I know, doesn't really sound inspiring), it ends not with a call to action but a resignation that assumes that kids raised doing remixes will have to change the world. Lame.
If you came of age during the Summer of Love, you'd be in your mid-sixties now. These people have been legislators, judges, and otherwise in power for decades with virtually no changes to our drug laws. It seems to me that the force behind the recent changes in some states aren't the Boomers and Hippies who lived through that decadent time, but the Gen Xers who witnessed the hypocrisy of their failure to legalize their past.
I hope that legalizing remix doesn't follow the same path, with this generation moving from hip, rebellious remixers to stodgy, paranoid primary content producers who are embarrassed by their past and determined to prevent others from building on their present. It would be better for our current crop of legislators to see the change that must happen now rather that waiting for two generations of societal evolution to catch up.
HT: (BoingBoing)
Subscribe to:
Posts (Atom)