I was at the SIAM Annual Meeting in Boston last week, where I had a surprisingly great time. I also learned a few things, so I thought I’d write about it. (We’ll be back to The Origin series next week.)


The conference

The annual meeting was a big conference with a lot of people and many interesting talks, organized into plenary talks and mini workshops. I gave my talk in the Advances in Large-Scale Optimization workshop organized by Suvrit Sra.

The annual meeting was held jointly with the SIAM Conference on Life Sciences, so I also got to hear some very interesting talks on biology. I liked this plenary talk on the scale-invariance of biological systems and how to understand them using Lie algebra. But my favorite was this talk on the dynamics of hearing, where I was amazed to learn that healthy ears are actively emitting sound! Moreover, these emissions are detected across species and occur at some characteristic frequencies, that I think act as amplifier for hearing. I was also interested to learn about the mechanics of the hair cells and how they are designed to implement two distinct functionalities—hearing and controlling balance—by transitioning between different regimes in the phase diagram of the dynamics.

The plenary talks at the annual meeting were also excellent. Don Knuth gave the John von Neumann lecture on Satisfiability and Combinatorics, where he also spoke about his latest fasiscle on satisfiability in The Art of Computer Programming series. He also gave the following constraint-satisfaction puzzle, which he said could be solved without look-ahead, i.e., just using a pencil without an eraser. (See my solution below.)

Marc Teboulle gave an excellent talk on first-order methods, where he also went through the FISTA algorithm so I feel like I’ve understood it better now. He also emphasized the importance of the Kurdyka-Łojasiewicz property (some sort of “sharpness” condition), which is interesting because I’ve seen it around but I still don’t know what it means.

And I also liked this cool talk on storytelling with visual analytics, because I think the intersection of technology and art will be a very exciting space.


Back to MIT

After my talk on Monday, I went running(!) from my hotel to MIT. It was literally my first time running in Boston, but I’m glad I did it because I got to go through Boston Commons, the Esplanade, and across the Harvard Bridge, which brought back some good memories. It definitely felt good when I finally saw the Great Dome at the Killian Court again. It was my first time back at MIT after graduating in 2010.

There were many new buildings on campus, but the spirit still feels the same. Building 2 (the math department) has been renovated and it is actually very nice, with plenty of sun-filled rooms with blackboards to doodle on. I have many fond memories of late-night psets and study groups in the classrooms, especially with my very good freshman group of friends—Ale, Daniela, and Tat.

It was nice to have a chance to meet up with some dear friends again. It was also nice that my favorite food truck at MIT was still there, and the lady there still remembers me after all these years! (Which probably says something about how often I ate there…)

I also met up with my undergraduate math advisor, Haynes Miller, whose class 18.905 (Algebraic Topology) is still by far my favorite class to this day, because that was where I first learned about the beautiful “abstract nonsense” of category theory. That class was also the first time I had an oral exam, which at the moment was absolutely nerveracking but in hindsight was really fun—and I remember being proud that I actually understood something after taking the class.

I was also excited to speak at the MIT ML seminar, thanks to the invitation of Stefanie Jegelka. The seminar was held at the top floor of the Stata Center, which is the trippy Frank Gehry building below. (In undergrad I used to take classes in the first-floor auditoriums, so now I feel like I’ve made it :^)

This is 32-123, the auditorium where I took 6.001 (Structure and Interpretation of Computer Programs). (The board says “The best things in life aren’t things” and “Tomorrow is always fresh, with no mistakes in it yet”.)

6.001 was my first computer science class, and it changed my life. I really like the beautiful simplicity and recursive structure of Scheme (even the parentheses). I remember one of the assignments was to implement Scheme within Scheme itself. I also remember the object-oriented project where I extended the Harry Potter-themed world into a complete interactive text-based RPG game (and I even won a Godiva chocolate for it! :). But above all, 6.001 made me realize the beautiful abstract structure of computation that underlies programming and computer science. That’s why I decided to double major in computer science, and here I am now ten years later!


Don Knuth

Last week, while everyone else was trying to catch Pokémon, I was busy trying to catch a rare glimpse of Don Knuth.

I knew that Don would give a talk on Tuesday, but I didn’t know whether I would be able to see him. I had only met him briefly once during Stanford visit days in 2010 (I think during lunch he told stories of the $2.56 reward check), but I remember being impressed by him.

On the opening night of the conference, during the reception, I was diligently working on my food when all of a sudden came strolling in, who else, none other than the great Don Knuth himself! My inner fanboy was silently freaking out as he walked nonchalantly right in front of me, so close I could almost feel his brainwave, but I was too shy to bother him.

Then on Thursday, as I was walking to the Stata Center to give my seminar, lo and behold who is that I see in front of the elevator, looking pensively at a pinball-like machine? Why no, it’s Don Knuth again! This time I mustered up my courage and decided to say hi to him.

I introduced myself quickly and mentioned I had talked to him at Stanford before. He was very gracious and friendly (and didn’t mind the fact that I had gone to Berkeley instead of Stanford), but said he didn’t expect his cover to be blown (oops) because he just wanted to walk around in the Stata building, as that was his first time there.

I was running a bit late so I told him I had to go give a talk. He asked what it was about, and I said it’s about accelerated methods for optimization in continuous time. He said it sounds interesting, but perhaps he’d skip it because he’s not planning to write a book on it at the moment. However, he thought it’d be a good idea to go to the 8$^\text{th}$ floor and walk his way down the building. So we went up the elevator together and said our goodbye, and then I went on to give my seminar, which I think went pretty well.

But a little bit into my talk, I saw Don’s head peeking into the seminar room where I was speaking. He left shortly (5 seconds) after, but I still count that as a success for making him come to my talk! :)


The "Toy Models" Lecture

Perhaps the highlight of the entire trip was “Toy Models”, a very entertaining I.E. Block community lecture by Tadashi Tokieda. He very charmingly demonstrated various singularities phenomena in everyday “toys”, thereby illustrating the concepts in a very clear and intuitive way.

The first phenomenon was the four-fold symmetry of the sound that it makes when you hit a spoon on the rim of a teacup, because the circular shape of the cup constrains the vibrations to be symmetric. But when you shift the position a little bit, you still get a four-fold symmetry but now at a higher pitch, which is because of the mass of the cup handle that reduces the oscillations.

Then he showed a “phase transition” phenomenon by rolling some balls in a bowl. Initially when there are only a few balls, they roll in the same direction as the bowl (because the balls act as independent particles). But as we add more balls, they begin to touch and interact with each other, and at seven balls we see a phase transition where the balls roll in the opposite direction (because they now behave collectively as a fluid)! He said this is the simplest known phase transition with the smallest degrees of freedom.

Then he showed two seemingly identical heptagons, but one rolls smoothly while the other stops after only one or two steps. It turns out the difference is that one heptagon has its edges so slightly rounded. But it makes all the difference, because now the “transfer function” is continuous and the heptagon can roll smoothly, whereas with the straight edges, the transfer function is discontinuous so in each step there is a lot of energy being dissipated.

He explained that it was an example of a singular perturbation, in which the limit of the behavior of a model is not the same as the behavior of the limiting model. This is cool, because I’ve heard of singular perturbation before but didn’t understand what it was about. It also feels similar to what I’ve been thinking about in relation to the gradient vs. Lagrangian flows, or deterministic vs. stochastic dynamics, or discrete vs. continuous time, etc., and it’s always exciting to learn there’s a whole field I didn’t know existed.

There were many other fascinating demonstrations, and I highly recommend watching the talk once the video becomes available.

But my favorite was when he rolled some tubes down an inclined plane. He first showed the base cases when the tube is either empty or full of grains, it rolled down at the same speed.

Then he took out a tube that is $2/3$-full, and asked what would happen if he rolled it. Some people guessed it would roll at the same speed, or faster. But to our surprise, it actually rolled more slowly—and in a jagged fashion—because the grains inside the tube form a cycle of avalanches that slow down the motion.

Then he took out another tube that is $1/3$-full, and asked again what would happen if he rolled it. Some people thought it would roll faster, or perhaps at the same speed as the $2/3$-tube due to symmetry (since $0 = 1$ and $1/2 = 1/2$, although I think this he meant this as a joke). But when he finally rolled it, there was an audible gasp from the audience (including myself) because the tube didn’t move at all!

Instead, he showed how it actually moved up very slightly then stayed in an equilibrium position, because of how the center of mass is positioned relative to the direction of motion. He also explained how this only works when the angle of repose of the grain is less (or more?) than the angle of inclination of the plane, which is why this only works when there is little grain in the tube.

So, as he emphasized in this talk, we actually have many concrete physical systems (toys) that exhibit interesting behavior, sometimes multiple regimes that are mathematically incompatible with each other. He is currently working on designing toys to bring the different regimes together, in which case, he said, he was sure mother Nature would have something interesting to show us.

And during the Q&A session, he implored everyone (especially scientists) to always keep being curious like a child, to always keep our mind open to the wonders around us, rather than only to what has been preconceived as interesting by society. The toys that he showed were real examples of the wonders of everyday life that we are still beginning to understand, and he said there is nothing like actually playing with the toys ourselves to understand how they work.

And here is my solution to the puzzle above:

Or here it is in the intended orientation:


A singular perturbation

I definitely feel my life has just gone through a singular perturbation, particularly graduating from UC Berkeley.

Grad school at Berkeley has been a great transformative experience, especially working on the acceleration project with my dear friend Ashia and our great advisor Mike Jordan. (Ashia and I had a really fun summer 2014 watching Susskind + reading Baez, also knitting + watching Cosmos.)

But now I’m very excited to start as a postdoc with the amazing Po-Ling Loh at UW-Madison! :-)

I’m not sure yet what I will do exactly. Eventually it will be nice to understand the Bregman Lagrangian/Hamiltonian. But along the way, I’m happy to learn and explore my ideas.

As a good friend said to me in Boston, there is some sort of “cascade” effect where if something good happens, it might influence your action positively so as to increase the recurrence of the good thing itself. So here’s to hoping that this singular perturbation will ultimately lead to a good thing in life.

And here is a beautiful poem by Feynman that I read on the plane back (from his book “What Do You Care What Other People Think?”):

“There are the rushing waves
mountains of molecules
each stupidly minding its own business
trillions apart
yet forming white surf in unison.

Ages on ages
before any eyes could see
year after year
thunderously pounding the shore as now.
For whom, for what?
On a dead planet
with no life to entertain.

Never at rest
tortured by energy
wasted prodigiously by the sun
poured into space.
A mite makes the sea roar.

Deep in the sea
all molecules repeat
the patterns of one another
till complex new ones are formed.
They make others like themselves
and a new dance starts.

Growing in size and complexity
living things
masses of atoms
DNA, protein
dancing a pattern ever more intricate.

Out of the cradle
onto dry land
here it is
standing:
atoms with consciousness;
matters with curiosity.

Stands at the sea,
wonders at wondering: I
a universe of atoms
an atom in the universe.”

So here we are, dancing waves of thoughts reflecting on itself. Or as Feynman said, atoms with consciousness, matters with curiosity, wondering at wondering!