I'd just like to thank the Foundation for their hospitality and all of you for being here today.
I'm a tool builder by instinct and inclination.
I got my start as a tool builder as a graduate student at Cornell when my advisers were working on an idea
to basically shine light through a hole that's much smaller
than a wavelength of light and use that as a little
nano-flashlight to drive across the sample point by point
and beat that diffraction limit that you heard about from Stefan and get a super-resolution image.
I worked on that technology for six years in graduate school
and was fortunate enough to kind of get it sort of working
to the point where I got my foot in the door at Bell Labs and then had my own lab at Bell Labs,
and proceeded to work on it for another six years.
During that time, we did a number of applications eventually as the technology slowly improved.
We could use polarization contrast to look at magneto-optic materials
or use fluorescence to be able to look at phase transitions and Langmuir-Blodgett monolayers,
do super-resolution lithography, histology,
and low temperature cryogenic spectroscopy of quantum wells.
So the technique started to take off, but the reason I got into near field was two-fold.
I wanted to get into science not to do incremental things,
but to do really impactful big things, and it seemed like
near field was a possible way of doing that kind of thing.
But the other thing I really liked about it was, at the time we started we didn't know anybody doing this.
And I really liked the idea of going in a field from its birth and not having a bunch of elbows
to bang up against while I'm trying to do my work.
It turned out there were a couple of other groups working
on it, and the idea goes all the way back to the twenties,
but still it was pretty wide open territory at that time, and that felt a lot of fun.
As it continued to develop, though, and as we started to have successes the field got more and more crowded.
Particularly with two applications that we demonstrated.
In '92, we were able to basically have the world record of data storage density by writing bits
with near field as small as 60 nanometres.
This started a whole big area with people jumping into this.
I was under a little gentle pressure to start a commercialization programme based on this,
but I didn't really want to go and do that.
I didn't want to lead a big group.
I wanted to still just explore where near field might go.
Fortunately, my bosses at Bell were conducive to that and so I was able to do that.
And it paid off because very quickly we were able to do what
I dreamed of doing which was biology, which was, in this case, to be able to look at the actin cytoskeleton
at about 50 nanometre resolution back in 1993.
The problem is this was a fixed cell, and the dream really was to be able to have an optical microscope
that could look at living cells with the resolution
of an electron microscope, and we still weren't there.
There was one thing about this experiment, which is that these single actin filaments had high enough signal-to-noise
that it suggested that we could even see single molecules.
And so, this was pretty much in the air in this era
because a few years earlier W. E. Moerner had seen at cryogenic temperatures the spectral signature
of single molecules. And you'll hear about that in a few days.
And it turned out to be actually very easy to see single molecules with near field
because the focus is so small that you reduce the background
to near zero, and then it becomes very easy to see these single molecules, but there were surprises
like their shapes were different and so forth.
It turned out this was due to the orientations of the molecules, so we could measure the orientations
and even determine their positions down to about a 40th of the wavelength of light.
So at this time, this takes us up to '94,
many people jumped into the field because of the spectroscopy applications,
the data storage applications, the potential biology and single molecule applications, and I was basically
really, really sick of doing near field at this time, and part of it was because of the fundamental
limitations of the technology.
The worst of them is that the light that comes out
of that sub-wavelength hole spreads extremely rapidly.
And if you're even about ten molecules away, you get a significant loss of resolution.
Well, if the dream is to look at living cells, a cell is a lot rougher than that.
But it had many other limitations, but at the same time so many people were jumping in the field
and saying that, you know, it would cure cancer or that the moon was made of green cheese, or whatever.
And what I basically learned in a career in science is that
science has all of these fads, where there are these booms and busts, and so forth.
And near field became one of these fads.
And technologies basically go like this, and this is where I like to get in.
This is where it's really fun, where it starts to pay off on the investment, and this is the time I found
where you want to bail because this is where basically, you know, people go crazy, you know, with these things.
This is where you'd like to believe it will go, but every new technology to me is like a new-born baby,
and you think that it's going to become president
or that it's going to cure cancer, win a Nobel prize, but usually you're happy if it just stays out
of jail in the end. (laughing)
So this is basically where most technologies end up, with some level that's above zero
so it was a net gain, but not so much.
And with near field, I felt like every good paper I was doing was just providing the justification
for a hundred pieces of crap that followed.
And I really felt like, what I was doing was a net negative to society because it was a waste of time
and a waste of taxpayer's money.
So basically I up and quit, and I quit not just Bell, I quit science.
And so, I didn't have any idea what I wanted to do so I became a house husband.
And a few months later while I was pushing my daughter around in a stroller, I realized that there was a way
of taking some of the experiments I had done previously to come up with a different way
of beating the diffraction limit.
So the idea is if we care about fluorescence, which Stefan
told you, why fluorescence is so important, the problem is that each of the molecules makes
a little fuzzy ball, and those fuzzy balls overlap, and you got nothing but a mush.
But if those molecules were different from one another in some way, say that they glowed in different colours,
then I could look at them by their different colours in a higher dimensional space
of XY and wavelength in that axis.
Once they were isolated from one another, then I can just find the centroid, find the centre
of those fuzzy blobs to much better precision than the diameter of the fuzzy blob and hence get
with near molecular precision the coordinate of every molecule in the sample.
So that was the basic idea.
I was really excited about that for a month, but then I realized the catch is that
even with the best focus you can create with a conventional
microscope, there might be hundreds or thousands of molecules in that diffraction limited spot.
And so you need ridiculously good discrimination
be it wavelength or whatever in that third dimension
to be able to turn on and see one molecule on top of hundreds or thousands of others.
So I didn't really know how to do that, so I just published that while unemployed and left it at that.
And so in the end, I fell back on my backup plan which was: my dad had started a machine tool company in '85
which by '94 had grown to about 50 million in sales.
What they did is, they would make these very customized very large machines to make in very high volume a single
part, customized for that part, for the auto industry.
And so, this was a successful technology, but I had so much ego and still do, and so much naivety
and still do, that I believe that if a single physicist
when to the Rust Belt he could save it from the Japanese.
So I went there and tried to see if I could do something to improve upon this technology.
So I developed this machine which used an old technology
called hydraulics, married it to energy storage principles you have in hybrid cars
and also nonlinear control algorithms, and was able to make a machine that would take something
the size of the width of this auditorium and collapse it to something the size of a compact car.
And it would move four tons at eight g's of acceleration and position it to five micron precision.
It was an amazing thing that I was very proud of.
I spent four years developing it, three years trying to sell it, and in that time I sold two of them.
And so, I learned that while I may not be an academic scientist, I'm a really horrible business man.
And so, after blowing through over a million dollars of my dad's money, I went to him and I apologized,
and I said, "I'm sorry, I gave this everything I had, but I just can't sell this thing."
And so, I quit again.
So then, it's the blackest time in my life because not only
had I pissed away my academic career, I had also thrown away my backup plan of working for my family.
And so, I needed to find something, and so I reconnected with my best friend at Bell, Harald Hess,
who had also left Bell as Bell continued to shrink in the 90s and went to work in industry.
And he was suffering the same sort of midlife crisis I was.
He was more successful in business than I was, but still unsatisfied.
That we really wanted to go back and do curiosity driven research and be able to work with our own hands
and not be managers or anything like that, which typically is what starts to happen to you when you're in your 40s.
We wanted to live like graduate students basically, okay.
So we started to meet at various national parks and start to plan out what could we do
in order to get back into a lab.
And so, I started reading the scientific literature for the first time in a decade,
and the first thing I ran across that just knocked me out
was green florescent protein.
The idea that you could snip some DNA out of a glowing jellyfish, and get that any organism
you want to express any protein you want in a live organism, and have that fluorescence specific, 100% specificity,
to the protein, was magic to me because it was so difficult
in that actin experiment I showed, to get the labelling in there well enough to be able to see the actin
without seeing a whole bunch of nonspecific crap besides.
And so, I said, "Damn it. I guess I got to be a microscopist again."
But I had a problem in that in ten years of not doing any physics, all my physics knowledge
had flown straight out of my head.
I couldn't remember anything.
Well, I would take the kids to school.
I would then go down to the cottage that we had nearby and sit out on the lake, and just start relearning physics
from freshman textbooks onward, but within three months, it pretty much all fell back into place.
I hadn't really forgotten it.
It was just kind of blocked.
And I started trying to figure out how I could use light in order to make a better microscope.
Not a super-resolution microscope at this time
but a better microscope that would exploit the advantages of live cell imaging of GFP.
So I started thinking in fundamental terms from initially two wave vectors creating a standing wave.
I added more wave vectors, got these weird periodic patterns, looked in the literature
and heard about these things called optical lattices, which are used in quantum optics to trap and cool atoms.
And I eventually came up with new classes of optical lattices that would actually be really good
as a 3D multifocal excitation field to do massively parallel imaging at high speed of live cells.
So I called that optical lattice microscopy, and that was the idea I wanted to pitch to get back into science.
So Harald was helping me in this task of getting back in, and one of the places we went to visit to sell this idea was
Florida State University where we met Mike Davidson.
And Mike told us about not just fluorescent proteins
but there was a very new type of fluorescent protein
on the scene, which initially if you shine blue light on it, nothing happens.
But if you shine purple light on it first,
you activate the molecules, and then blue light causes it to turn green or to emit green light.
And so, basically, it's a fluorescent molecule that you can turn on and off.
So Harald and I were sitting in the airport in Tallahassee,
and it struck us that this the missing link to make that idea that I had pitched in my first round
of unemployment ten years earlier to work.
So the idea is you turn the violet light down so low that only a few molecules come on at a time.
And statistically they're likely to be separated by more than the diffraction limit
so then you can find the centres of those fuzzy balls.
Those molecules either turn off or bleach.
You turn on the violet again to turn on another subset and round and round and round you go
until you bleed out every molecule from the sample.
So it was the idea I had in '95 except with time as the discriminating dimension,
and that violet light is the knob by which we could get higher and higher resolution across that time axis.
So obviously we had another problem in that I had convinced Harald on the basis of the lattice microscope,
although he wasn't convinced to do it with me, to quit his job so now you got to unemployed guys.
So how are we going to implement this idea because we think it's too ridiculously simple and a lot
of people have to be thinking of it, and we were right.
There were a lot of people just right on our heels.
The good news is that Harald is a lot smarter than I am.
So when I left Bell, I told them to go to hell, but when Harald left Bell, he was able to take all
of his equipment with him.
So we pulled that out of the storage shed, and then
normally you do this type of work in the garage, but we were able to do it in his living room
because he wasn't married, and it was a lot more comfortable there.
In a couple of months, we had the scope built, and then I was scheduled to give a talk at NIH about
about my lattice microscope, and I begged Jennifer Lippincott-Schwartz and George Patterson,
who invented this photoactivated protein to come to my talk. Took them to lunch, swore them to secrecy,
told them the idea, and that's the last missing link that needed to be filled,
because Harald and I knew zip about biology.
But Jennifer is one of the best cell biologist in the world and so she said just bring it here.
So we packed up the scope, went to Jennifer's dark room,
and within another couple months after that, we went from this sort of...
This is looking at a slice through a cell at two lysosomes, little protein degradation bags,
and then this is the diffraction limit and that's
the photoactivated localization microscope or PALM image, and the resolution went from this to this.
So two things, one Harald and I went from the concept
of this idea to having the data in our science paper in six months, and we were able to do that
because everybody left us alone.
We were able to work just by ourselves.
The second thing is that this type of work really requires that sort of solitude that you get in that sort of situation.
That ability to focus 100%, and that's again and again
I have found in my career critical, is to not be in academia for me has been key.
Because whether I was at Bell, working for my father or working there, or working where I work now,
it's the idea of just being left alone and focused
with 100% attention on a problem is what allows me to get things done.
So that's all the good news about super-resolution,
but every new stage of my career has been influenced
by the limitations of the thing I was doing before.
And super-resolution has a crap-load of limitations.
The first is that again we're looking at fluorescence, and if you don't decorate your fluorescence molecules
densely enough, then you can completely miss a feature if you're at less than half a period in the spacing.
And so, this is called the Nyquist criterion.
You can see here that if you want to sample this image it doesn't matter what the intrinsic resolution
of the microscope is, you don't have enough information.
The bad news is that if you relate this to super-resolution
if you don't have on the order of 500 molecules to get 20 nanometre resolution, you're done.
It turns out that's a very high standard of labelling density
compared to what most biologist had done previously for doing diffraction limited work.
And what it means is that the ownness in making these technologies work is not on the tool developer
it's, unfortunately, all on the biologist to figure out sample prep procedures that will work properly for this.
And so now, you can also ask what is the big deal about super-resolution.
I mean we've had EM for a long time, and its' pretty damn good.
Well, there's two main reasons which Stefan hit upon.
First is to do protein specific contrast, usually that's done by structural imaging on fixed cells.
But the problem is that 95% of what we think we learned by super-resolution has been from chemically fixed cells,
and their purpose is to cross-link proteins so they screw up the ultrastructure,
and so it means we have to put an asterisk next to almost everything we learn when we look at fixed cells.
The other more exciting application would be to do live cell imaging, but the problem is you can think
if I want even just a factor of two resolution improvement
in each of three dimensions, it means my voxels are eight times smaller.
It means I have one eighth as many molecules in each voxel.
If I want to get the same number of photons to get the same signal and noise as in a diffraction limited experiment,
I either need to bang my cell with eight times as much light
which it's not going to like, or else I'm going to have to wait eight times as long for those photons to come out.
And the whole damn image will get smeared by motion artefacts.
So there's claims in the literature of resolution
for live cell imaging that require just by this back of the envelope calculation of a thousand
to over ten thousand times as many photons from the cell as in a diffraction limited experiment.
This is almost fantasy again.
So I'm basically sometimes, with super-resolution, I felt like I was living the same bad nightmare
all over again that I was in the near field era.
By 2008, there were all sorts of crazy claims in terms of what could be done with super-resolution.
And I knew just from my experience that this was not possible.
And so, other problems are the intensities that are required
for STED or for PALM are anywhere from kilowatts to gigawatts per square centimetre.
But life evolved at a tenth of a watt per square centimetre.
So we have to ask ourselves if we're doing live cell imaging,
are we looking at a live cell or are we looking at a dead cell or a dying cell from the application of all this light.
And these methods take a long time to create an image, and the cell is moving,
and so you end up getting motion artefacts.
So the moral of the story is I don't care what method you want to use,
if you want higher spacial resolution you need more pixels.
More pixels means more measurements.
That takes more time.
It means throwing more potentially damaging light at the specimen, and so if you're going to be honest
your always playing with trade-offs between spatial
resolution toxicity, temporal resolution, and imaging depth.
The guy who really understood these trade-offs better than
all of us, earlier than all of us, was Swedish native Mats Gustafsson who developed a third method
of far field super-resolution, called structured illumination microscopy.
And this technique you use a standing wave of excitation
rather than uniform illumination of the sample which creates these Moiré fringes against the information
in the sample to create lower spatial frequencies that the microscope can detect.
One of the arguments in the Nobel committee's report
as to why this did not share the Nobel prize, is it's limited
to a factor of two in resolution gain, whereas STED's localization is diffraction unlimited.
But in my opinion, this weakness of SIM is actually its strength.
Because, first off, you need much lower labelling densities
to achieve that which is much more compatible with current technology.
And second, it requires orders of magnitude less light, and is orders of magnitude faster than the other methods.
So here are a couple of examples.
Here you're looking at the endoplasmic reticulum in a live cell, and yes the resolution is only
100 nanometres, but you can see the wealth of information
you have because you're able to look at this cell for 1800 rounds of imaging at subsecond intervals.
You get this dynamic aspect that you can't get with the other methods.
Here is another example of showing the dynamics of actin in a T-cell after it's provoked the immunological synapse.
So the good news is we were able to recruit Mats
from UCSF to Janelia in 2008, but he was diagnosed with...
Oops, that was interesting.
So he was diagnosed with a brain tumour in 2009, and he ended up dying in 2011.
So I ended up inheriting much of his group and his technology, and we've been trying to extend
his legacy with this tool, too.
Key to that is, if the argument is that SIM doesn't deserve to be spoken of in the same breath with STED and PALM
because it doesn't have the same resolution, how can we get past that 100 nanometre barrier?
Well, the simple and stupid way is to just increase the numerical aperture of your lens.
So recently a 1.7 NA lens came onto the market which allows you with the GFP channel to push
to 80 nanometre resolution.
And then you can study for dozens or hundreds of time points in multiple colours
because it doesn't require specialised labels.
Dynamics in this case looking at clathrin-mediated endocytosis, and its interaction
with the cortical actin, and how that helps to pull in these pits to bring cargoes inside of the cell.
We've also developed recently another technique,
a nonlinear structured illumination technique, that uses again this photo switching principle to put
on a standing wave to generate the activation light in a spatially patterned way and read out the detection
the same way to go from twice beyond the diffraction limit to three times beyond the diffraction limit.
And then, that has gotten us to 60 nanometre resolution.
Not as many time points and not quite as fast, but still allows us to look at live cells.
So if, again, the knock has been that it doesn't have the resolution of other methods:
Here we've compared this nonlinear SIM to RESOLFT and to localization microscopy,
and basically we have resolution every bit as good, if not better with SIM now than we have
with these other tools, but with hundreds of times less light and hundreds of times faster than these other methods.
So I really think for live imaging between 60 and 200 nanometres SIM is going to be the go-to tool,
and for structural imaging below 60 nanometres, PALM will be the tool.
So the moral of the story is that while, again, everybody is crowding up against this goal
of getting the highest spatial resolution,
Mats wanted a space for himself working where others aren't
and by pulling a little bit back, he had this whole play area where he could develop this tool.
So it begs the question of, what if we go back all the way to the diffraction limit.
Is there something we can do as physicists to make microscopes better for biologist and particularly,
is there a way that we can increase
the speed and noninvasiveness of imaging and live cell imaging to improve it by that same order
of magnitude that PALM does in spatial aspect?
And so, why do you want to do that?
Well, the hallmark of life is that it's animate.
And every living thing is a complex thermodynamic pocket of reduced entropy through which matter
and energy is flowing continuously.
So while structural imaging like super-resolution or electron microscopy will always be important,
the only way we're going to understand how inanimate
molecules self-assemble to create the animate cell is by studying it across all four dimensions
of space time at the same time.
So when I got as sick of super-resolution in 2008 as I was of near field in 1994, that's what I set out as my goal.
The good news is that there was a new tool on the market that we could exploit which is the light sheet microscope.
So what it does is it shoots a plane of illumination through a 3D sample to only illuminate one plane at once,
so there's no damage to the regions above and below, and you can take a very quick picture cause the whole plane
is illuminated then go plane by plane through the specimen.
It's been transformative for understanding embryogenesis
at single cell resolution, but it has a limitation in that the light sheets are pretty darn thick on the single cell level.
We were able, basically, to then bring some physics tricks called nondiffracting beams.
First these were Bessel beams, and eventually I was able to dust off my old ten-year-old lattice theory
because two dimensional optical lattices are nondiffracting beams and create a light sheet
that's much thinner, to then be able to look at the dynamics inside of single cells.
So here are a bunch of examples of this.
We've had about 50 different groups come to our lab
to use this microscope and almost everyone has left with big smiles and ten terabytes of data.
We never hear from them after that because they don't know what to do with ten terabytes of data.
But in any case, I'm very proud of what we've done with PALM, but I'm morally convinced at this point
that this will be the high water technique of my career.
So one of the other things that that light sheet turned up, the lattice light sheet as we called it, turned out
to be good for is that single molecule techniques in general have been limited to very thin samples because out of focus
molecules obscure the signatures of the ones in focus.
But the lattice light sheet is so thin that only in focus molecules are illuminated.
And so that allowed us to apply another method of localization microscopy,
developed by Robin Hochstrasser's group in the same year
we developed PALM, where instead of prelabelling your cell and having a fixed number of molecules.
Remember I said the basic problem is getting enough molecules in there to get the resolution you want.
Well with PAINT, instead of prelabelling, the whole media around the cell is labelled with fluorophores.
They're whizzing usually too fast to see, but when they stick they glow as a spot
and then you localize those.
So the advantage to this is there's an infinite army
of molecules that can keep coming and coming and coming and decorate your cell.
The disadvantage is because the media is glowing the SNR normally is too poor to do much single molecule imaging.
So lattice light sheet with this PAINT technology is a marriage made in heaven.
So here you're seeing 3D imaging over large volumes by using this method, and we're able to basically see.
Now, basically, take localization microscopy to much thicker specimens at orders of magnitude higher density,
and basically make something so we don't need to use EM to do sort of global contrast to combine
with a local contrast we get from PALM.
So the last thing Astrid, and then I'll be then off.
I got two minutes and 22 seconds it says.
You're early.
Okay, is that we still have to deal with taking cells away from the cover slip.
So much of what we've learned have been from immortalized cells, and all of these methods,
if you take them from single cells and try to put them in organisms, they're incredibly sensitive
to the refractive index variations that scramble the light as you go in and out with the fluorescent photons.
So with STED, you have to have that perfect node in that doughnut.
That's very sensitive to aberrations.
With SIM, that standing wave gets corrupted.
With confocal, your focus gets corrupted.
With localization, your PSF, you have to have knowledge of the shape of that spot in order to find its centroid properly.
And all of those things get screwed up by aberrations.
Light sheet this is not showing, but as you go into an embryo.
Right now, light sheet is in that same bullshit phase that other techniques are in, where people are saying:
"we can do in total development of entire organisms or see neural activity of an entire brain
by using light sheet", but it just isn't so because the aberrations make the resolution so poor,
you can't resolve single somas in certain areas of the brain.
So what we've also been working on, is stealing from astronomers and using the tools of adaptive optics to
allow us then to be able to look deep into live organisms.
So this is an example here of looking in the brain of a developing zebrafish.
In this case we had to develop a very fast AO technique because as you move from place to place the aberration
is different so this represents 20,000 different adaptive
optic corrections to cover this large volume.
And as we go deep into the midbrain of the zebrafish,
about 200 microns deep, we'll turn the adaptive optics off.
That's what you would see with a state of the art microscope today, and then you turn the AO on
and you get back to recovering the diffraction limit and recovering the signal.
So for my group, what I really believe at this point is that we are on the cusp of a revolution in cell biology,
because we still have not studied cells as cells really are.
We need to study the cell on its own terms, and there's three parts of that puzzle.
While GFP is great, it's largely been uncontrolled in its expression.
So in the last couple years, these genome editing techniques
have come on the scene to allow these proteins to be expressed at endogenous levels.
These end up to be typically low levels so now confocal microscopy and these other techniques
are far too perturbative to study dynamics, and you need things like lattice light sheet to be able to do that.
But again, you can't look at cells in isolation and really see the whole story.
You need to see all the cell-cell interactions, all the signalling that happens inside,
where they actually evolved, and that's where the adaptive optics plays a part.
So the future of my group is to develop adaptive optics to get that to work, to combine it
with the lattice light sheet so we have a fast and noninvasive tool to look at dynamics.
And then scroll in the super-resolution methods like SIM and PALM to add the high resolution on top of that.
And with that, I thank you for your time.